locked
urgent help needed for building a scalable storage solution for a web application RRS feed

  • Question

  • User-1696994173 posted

    Hi,


    We currently are running a website where users can upload HD music and images. Originally i started off with 2TB of storage per drive and had a system where i could just add drives, but soon realised that at this rate I would like to have networked drives and would like to access the volume from different machines (Load Balanced). so for storage instead of having drives i switched to networked access and i can scale by just indicating which is the active volume and by volume i have network drives so for example let us say currently i have two volumes

    \\system1\d\

    \\system2\d\

    So I will use which ever is currently active and this helped me scale for different machines also. But after switching from local drives to network drives. I continously run it an issue with "The process cannot access the file xxx because it is being used by another process" exception this is even though i call close and dispose and release all the handles(I am pretty sure i release the handles). I have a few questions

    1) Most important is why is this file locking happenign and for the immediate is there an immediate fix that can be done.(URGENT PRODUCTION ISSUE!!!!!)

    2) is this mode of access slow ? is there a better way to design drive scalability for over 10 to 20 terrabytes of data?


    I am running IIS6, .net 2.0, VS 2008

    Tuesday, December 22, 2009 1:49 PM

Answers

  • User-16411453 posted

    I ran into that on Load Balanced Servers. a user would make a web request, and the request is split among all the servers, so you had to have the same data available to all the servers at the same time. If you wrote to the drive, it only wrote to one drive on one server. You can change the config to one server at a time.

    As far as the error goes, I'm not sure, I can't see the error now

    There was a way to solve it, but it required a clustered configuration, using Windows Server Enterprise Edition. I never bothered checking it out.

    Better to keep it simple.

    With a NAS box, you get into permission issues. With a SAN box, you need a big backbone and at least 3 ports. I do know that with SATA drives in a raid 5 or 6 config, it's really slow, and the drive rebuild flag seems to trigger alot compared to a huge raid 1.I learned that SCSI drives are much more reliable, but considering that it's just images, SATA is OK.

    I'm not sure how youtube does it, perhaps you can try the FreeForAllForum. I have freinds that work for DLink, but they really have no practical experience, just white paper stuff.






    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, December 23, 2009 7:31 PM

All replies

  • User-1696994173 posted

    Any updates on this?

    Wednesday, December 23, 2009 1:49 PM
  • User-16411453 posted

    I ran into that on Load Balanced Servers. a user would make a web request, and the request is split among all the servers, so you had to have the same data available to all the servers at the same time. If you wrote to the drive, it only wrote to one drive on one server. You can change the config to one server at a time.

    As far as the error goes, I'm not sure, I can't see the error now

    There was a way to solve it, but it required a clustered configuration, using Windows Server Enterprise Edition. I never bothered checking it out.

    Better to keep it simple.

    With a NAS box, you get into permission issues. With a SAN box, you need a big backbone and at least 3 ports. I do know that with SATA drives in a raid 5 or 6 config, it's really slow, and the drive rebuild flag seems to trigger alot compared to a huge raid 1.I learned that SCSI drives are much more reliable, but considering that it's just images, SATA is OK.

    I'm not sure how youtube does it, perhaps you can try the FreeForAllForum. I have freinds that work for DLink, but they really have no practical experience, just white paper stuff.






    • Marked as answer by Anonymous Thursday, October 7, 2021 12:00 AM
    Wednesday, December 23, 2009 7:31 PM