locked
Windows Desktop Search Gathering Manager Registry Values RRS feed

  • Question

  • Does Microsoft have any documentation on the registry settings for the Windows Desktop Search 4.0 Gathering Manager?  Specifically a registry key that would help reduce the load on a network file server.   We've found that when indexing of network home folder is enabled for only 250 of our users, that the CPU utilization on our EMC Celerra (designed to support 15,000+ users) goes to 99% utilization.  This would be both related to the indexing activity as well as the CIFS Notify activity on the network file system server. 

    Monday, November 30, 2009 2:27 PM

All replies

  • Indexing all those files from the clients is inherently unscalable. A better solution would be to index all the files on the server side, and use remote queries from the clients. Alternatively, using client side caching can also help.
    Monday, November 30, 2009 6:41 PM

  • While the initial indexing activity reading the user files on the network server does put a load on the server, this can be managed by incremental deployment of the search engine to the PCs and by stratifying the group policy so that all don't conduct an initial index simultaneously.

    However the real problem is that even after ALL the user files are indexed, the Celerra maintains for an unspecified amount of time, for an unspecified reason a CIFS Notify process for each user in response to a perception of an SMB file session.  This is maintained even after the service has been disabled on all the Windows Desktop Search clients.  Client side caching is a consideration but in practice it doesn't help - as when using client side caching a read event is still needed for all the data on the server to synchronize to the client.  This opens the SMB session and the notification scenario.  Possibly there is a setting that would modify or tune this behavior on the Celerra, but it is not exposed to the consumer.

    Yet in order to provide professional technical support a product as well as fully test and diagnose it, detailed documentaion is needed.  Microsoft's apparent lack of documentation of all the registry values set under the registry key HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows Search\Gathering Manager lends credible doubt as the sincerity of their stance behind this product - possible reasons could be any or all of: trade secrets, unwillingness to take the effort, cost of documentation, it's really beta software and subject to change without notice, or, as a free product it is use at your own risk.

    A server side solution is the remaining option, however confidence in using the Windows Search product as a companion product would be increased if better-detailed  technical documentation was available that would help in fine tuning it.

    Tuesday, December 1, 2009 10:36 AM
  • Chris,

    If your users are running Vista, the network traffic caused by indexing Client Side Cache content could be tremendously reduced by applying these two hotfixes: http://support.microsoft.com/default.aspx/kb/959488 and http://support.microsoft.com/kb/958705. The later requires Windwos Desktop Search 4. Or you could upgrade to Windows 7 as it contains these improvements and more. Please give them a try, they addressed this issue for our other customers. If the improvements are still insufficient, Client Side Cache can be configured to work in offline mode and synchronized manually to reduce netwrok traffic further.

    Regarding the registry keys, they are supposed to be modified by calling our publically documented APIs. There's nothing in there to help with netwrok traffic, but I'd be happy to assist you if you have any other issues, including pointing you to the right APIs if necessary.

    Regarding indexing network locations, as Ari pointed out we deliberatly decided against supporting indexing SMB shares out of the box because of multiple technical issues that can't be addressed to our customers' satisfaction. No, they can't be worked around with staged deployment. Please give Client Side Cache another try, or consider using Sharepoint Server.

    Thanks,
    Ilia

    • Proposed as answer by Andrei Aron MS Friday, December 4, 2009 6:30 PM
    Wednesday, December 2, 2009 11:38 PM
  • Our Environment is running Windows XP - Vista and Windows 7 are not an option here.  And yes, it does look like much of the WDS 4.0 was lifted from the Windows Vista 7 product.   However, the issue would appear to be mainly due to the internal operating parameters of the windows desktop search 4.0 gathering engine as opposed to API extensibility or configuration of additional filters.  The APIs documented speak mainly to the Crawl Scope Manager and related higher level functions.  However there is no documentation of basic, low-level parameters, such as what is the maximum frequency that the search engine will rescan a folder resource and whether this can be changed.

    Now, I know that the Windows XP lanmanworkstation service will, by default,  keep a connection open for 10 minutes see http://technet.microsoft.com/en-us/library/cc960471.aspx for documentation of the KeepConn parameter.  Yet if the Desktop Search is scanning the folders every 9.9 minutes, any open connections will never close.   I don't see any API call that relates to this for the WDS.

    Yes, we could implement offline files for all of our desktop users' 'My Documents'.  Yet that will increase user logon and logoff times due to the synchronization cycles, and on any Virtualized systems will increase the disk space requirement. 

    Redirecting Home Folder to a SharePoint server is an interesting concept.  Sounds like a very expensive file server solution.

    On another point, if indexing SMB shares out of the box was not supported, then why is the default configured setting to allow any user to add any UNC path in the Control Pannel 'Indexing Options' applet?

    If Microsoft would provide detailed documentation on tuning and optimizing the gathering engine for basic LAN operations it may provide a way ahead.  I would only hope that this documentaion would be forthcoming soon. 
    Friday, December 18, 2009 9:57 AM