Number of Files in File System - Performance

Question

User1401982410 posted

Hi,

In my ASP.NET application, I have customer profile photos in a sub-folder.  The customer photo is displayed on their web profile using a simple html IMG tag.  With regard to performance, how many files can I store in a folder before performance starts to degrade due to seek time - the web server accessing the file system to find the specified file?

Thanks!

Answer 1

User2039959148 posted

minimum 500 or 1000 images  per folder is better

Answer 2

User1414178881 posted

That's a difficult question because it depends on many things;the capabilities of the server (CPU speed, hard-drive (speed,size and file system;NTFS,..etc),memory,cache,...etc) and on the operating system too (i.e. Windows server 2008 can access/seek files faster than windows server 2003).

Anyway,I think that you will have to do a test on the server and see how many files/sub directories can a folder show within i.e. 1second.

Answer 3

User187056398 posted

The inner search structure is a binary tree.  That means the relationship between the number of files and the search time is logarithmic (reverse exponential).

Illustration:

If the number of files increases from 2 to 4, the search time increases 1 nanosecond 

If the number of files increases from 2g to 4g, the search time increases 1 nanosecond.

http://ixbtlabs.com/articles/ntfs/

 

Answer 4

User1401982410 posted

Thank you all!

I should mention that the .NET app is hosted on a Windows 2008/IIS 7 shared hosting server by CrystalTech. I guess I'm just looking for a rough estimate of how many I image files I should limit the folder to before creating a new folder...

Answer 5

User-385324712 posted

My general rule for this is: create a new folder for year and month. Its easy to implement and it works great.

/photos/2010/5/image.jpg 

Answer 6

User1401982410 posted

that's a good idea.  i still hope to find out how many files is the limit for good performance.. :(

UPDATE:  I found this on the web:  "Although this isn't an answer to your question, keep in mind that with many file systems performance will start to degrade if a directory has more than X files.  I think it around 30,000."

from    http://serverfault.com/questions/18692/what-is-the-maximum-number-of-files-or-folders-that-can-be-stored-in-a-single-f