locked
Performance to get the full name of a specific file from a directory of millons a row. RRS feed

  • Question

  • User-1350042179 posted

    Hi

    I have a file: 'test'

    I would like to know the full name such as: "c:\sss○2dwdw\dwdw\text.xls"

    My code to find the full name is: 

      var file3 = Directory.EnumerateFiles(rutaFileIntegracionEquiposAmSapAura, "*.*", SearchOption.TopDirectoryOnly)
                            .Where(f => extensions.IndexOf(Path.GetExtension(f)) >= 0 && f.Contains(nombreArchivo));
                        rutaArchivo = file3.FirstOrDefault();

    Is it a good option..?

    Saturday, June 15, 2019 12:10 AM

All replies

  • User303363814 posted

    Is it a good option..?
    Depends... does it create a problem for you?  What is the problem?

    If the code works then I suggest that your time is best spent by moving on to the next part of the application.  No one will thank you for spending time just so that random people on the internet give you a tick of approval.

    Sunday, June 16, 2019 7:16 AM
  • User753101303 posted

    Hi,

    How do you know the first one is really the one you are looking for or even that it is an xls file? It's hard to comment about solving a problem you don't describe but when storing files (this is what you are doing ?) it seems better to keep enough info to known directly where it was stored rather than trying to guess.

    Edit: if you just keep "test.xls" rather than "test" do you still have to search which file it is ? what if you have both "test.xls", "test.xlsx" and "test.docx" ?

    Edit: you likely won't keep millions of files into a single directory. A common approach is to spread files in multiple directories as you go. You are trying to build some kind of document management system ?

    Sunday, June 16, 2019 6:50 PM
  • User665608656 posted

    Hi neoaguil,

    According to your requirements and the code you provided, I'm not sure what some of the parameters in your code represent.

    Is the path of your test file a known parameter?

    If you just want to get the full name of the file through the file name and file path, I recommend that you refer to the following code:

                string partialName = "test";
                DirectoryInfo hdDirectoryInWhichToSearch = new DirectoryInfo(@"c:\sss○2dwdw\dwdw");
                FileInfo[] filesInDir = hdDirectoryInWhichToSearch.GetFiles("*" + partialName + "*.*");
    
                foreach (FileInfo foundFile in filesInDir)
                {
                    string fullName = foundFile.FullName;
                    Console.WriteLine(fullName);
                }

    Best Regards,

    YongQing.

    Monday, June 17, 2019 5:58 AM
  • User-943250815 posted

    If you have a millions of files in a directory, for sure you in trouble.
    Have in mind, before anything you under FileSystem, Disk Read Performance, I/O, Memory and Time Spent by Operanting System to collect all data, so IMHO  there not too much to do on coding side.
    As Patrice told, better split files on directories with a lower number of files, by lower number of files you have to try and find what is your best balance between Number of Files x Performance.

    Thursday, June 20, 2019 1:36 PM