Need help with looping through folders to find documents

Question

Hi,

I have a requirement to upload word documents to sharepoint. These documents belong to different members (users). The documents to be uploaded are present in different folders. The members are present in a SQL Server Database Table, each with a unique memberid. I need to find all the documents of each user and upload them. My problem is to find the documents of a member. There are approximately 800 folders and about a million users. Below is the piece of code that i used for this.

DirectoryInfo di = new DirectoryInfo(dir);

searchpattern="*"+memberid+"*";

string[] fileNames = Directory.GetFiles(dir, searchpattern, SearchOption.AllDirectories);

foreach (string fi in fileNames)

{//Code to upload the document

}

dir: is the root folder, which has all the subfolders.

This approach takes approximately 7 secs to retrieve all the filenames(full path) of a memberid. As the number of members is huge, is there a better/faster approach to accomplish this? Any help is much appreciated.

---

MSBI Developer

Answer 1

Using the windows file system api directly might shave a couple of seconds off your search, but the best way would be to index and maintain the directory. Then, you just query your index (which could be a hash table or some other kind of similar searchable structure in memory) to find find the files you want.

I wonder if you can tie into the windows file search index service somehow? See if this might help http://www.codeproject.com/Articles/21142/How-to-Use-Windows-Vista-Search-API-from-a-WPF-App

The beauty of using the windows search is that you wouldn't have to rebuild the index every time new files are added. The service handles that for you.