Excluding a list from search results immediately

Question

Hi all,

We want to exclude a list with sensitive data from the search results. We have already requested some search scope changes for this, but these require a full crawl of the environment which is not scheduled for a while (a full crawl takes several days in this environment due to its size, which is why the number of full crawls is limited as much as possible). An incremental crawl is running continuously by the way.

So for the meantime, we went to List > Advanced Settings and unchecked the box for 'Allow items from this list to appear in search results'.
However this change does not seem to take effect immediately. Why is that? I would assume this setting to have instant effect, because the search results are cross-checked against this setting?
And so if that doesn't work, what is the proper to exclude a specific list from the search results immediately (short of denying everyone access to the List)?

thanks
Pascal

Answer 1

Create a crawl rule and provide the path as http://server/lists/<mylist>

and choose "Exclude all items in this path " in the crawl configuration section.

 

 

Answer 2

The crawl rule mentioned will ensure your content does not end up in the search index again in the future. However, to remove the list immediately from search, in your Shared Services Provider, go to Search Administration and choose Search result removal.

http://technet.microsoft.com/en-us/library/cc280336%28office.12%29.aspx

Crawl rules

http://technet.microsoft.com/en-us/library/cc262934%28office.12%29.aspx

---

JD Wade, MCITP

SharePoint Consultant, Horizons Consulting, Inc.
Blog: http://wadingthrough.wordpress.com
Twitter: http://twitter.com/jdwade

Answer 3

Thanks JD Wade and Red316, both remarks were very helpful.

We did some testing and searching based on this, and the issue is now solved. Some interesting results though:

• You can only do Search Result removal for a specific URL, so no wildcards for all items in a List
• As a results, we had to enter an URL for search removal for each item in our list, and we have > 1400 items
• It was fairly easy to create the lines with the correct URL's with some Excel work, but:
• For each search removal URL entered, a separate crawl rule is entered, to prevent the result from appearing again after the next crawl.
• So we ended up with... more than 1400 crawl rules
• It seems you can only delete a crawl rule one at a time. So this meant deleting > 1400 crawl rules manually, one by one  :-(

As for the crawl rules:
Those take effect after the next incremental crawl. However in our environment even the incremental crawl takes more than 10 hours.
So in case of emergency, you definitely need the Search Removal function.

thanks
Pascal

Answer 4

As for the crawl rules:
Those take effect after the next incremental crawl. However in our environment even the incremental crawl takes more than 10 hours.
So in case of emergency, you definitely need the Search Removal function.

thanks
Pascal

New or modified crawl rules will always require a full crawl. :)