What is the best practice way to enhance the Search crawling in SharePoint server 2016 or 2019 RRS feed

  • Question

  • I'm building a public website on top of SP 16 On-Premise which contains some custom listing and details pages that shows dynamic texts retrieved from SP Lists (News list as exp.).

    Currently, I'm implementing the SP Search on the website and need to find the best practice way to manage the search crawler to get search results that reference to the custom forms instead of reference to the SP Lists where the text is originally exist.

    here is a sample scenario:

     - SP List named 'News' has an item with title as "Top 5 programming languages"
     - on the Website, there is a page named NewsDetails which contains an SPFx webpart to render the News Details based on ID passed in a query string.
     - when trying to search for a text as "Top 5" using the SP Search Text Box, the SP Search should return a result that pointing to the News Details page with the right correspondent item ID.

    I have worked before with a Custom Connector for a big project but this was complicated and could not find many resources for it.

    is there other ways to handle this case specially for a small website? 

    • Edited by Samer Fayad Wednesday, August 7, 2019 8:42 AM update question title
    Tuesday, August 6, 2019 10:29 AM

All replies

  • Hi sfayad, 

    You can exclude the content from search results by creating crawl rule.

    Go to search service application, crawling section, click crawl rules to create a crawl rule to exclude the content. 

    For more information, please refer to the following article about excluding AllItems.aspx from search results in SharePoint 2013 (which is also work for SharePoint 2016):

    Best Regards,

    Lisa Chen 

    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact

    SharePoint Server 2019 has been released, you can click here to download it.
    Click here to learn new features. Visit the dedicated forum to share, explore and talk to experts about SharePoint Server 2019.

    Wednesday, August 7, 2019 6:33 AM
  • Thank you Lisa for your reply, yes i believe we can exclude the "allitems.aspx", but my case is different, as the page is a custom page which have custom JSOM code to render the items based on an Id in the query string in the URL (exp: http://siteUrl/Pages/NewsDetails.aspx?itemId=3 )

    Therefore, SP Search will not catch the different items loaded in this custom page (itemId =1 , 2 , 3 or 4.. ).

    As previously mentioned, i have solved this last time by implementing a custom connector for search as showing below, but this was required a hard development time and not fully worked as expected because of missing to some important settings in the XML Model that are responsible to show Popups next to the results in the SP Result page in addition to other advanced search features...

    SP Search Custom Connector solution

    I tried to search and couldn't find a modern solution that it's not depend on Full Trust code solution specially at this time when Microsoft is advising to stay away of full trust code and follow the modern solutions.
    • Edited by Samer Fayad Wednesday, August 7, 2019 8:57 AM update answer text
    Wednesday, August 7, 2019 8:40 AM