none
full crawls never end RRS feed

  • Question

  • We've tried to perform full crawls however they won't end. More than 300 hours and counting. Where can I look for the source of the issue?

    Two virtual servers hold the crawl service, each with 24GB RAM and 8 vCPUs.

    Wednesday, October 16, 2019 10:27 AM

All replies

  • Hi JRS ISOIN.

    Is this the first time you experience such an issue? Did you already make an assessment of the CPU, memory and disk load for all the servers in the farm?
    What about your Search Service application topology and content sources? Are you crawling and indexing network shares which contain a huge amount of big files?
    Open the Search Administration page for the Search Service application, click Crawl Log on the Quick Launch and then Click Error Breakdown on the top of the page to have a comprehensive overview of the errors occurred during the crawling process.

    Bye.


    Luigi Bruno
    MCP, MOS, MTA, MCTS, MCSA, MCSE

    Wednesday, October 16, 2019 2:33 PM
  • Hi Luigi,

    This is an inherited environment and I am still in the process of learning how it's configured.

    Regarding the hardware assessment, what I've learnt so far is that we experience medium disk latency when noderunner is at peak use.

    In the crawl log, I've noticed is that we tend to have more than 2, 3 or 4 millions of errors saying "the object was not found", which is a huge number and most likely a source of an issue!

    • Edited by JRS ISOIN Thursday, October 17, 2019 11:20 AM
    Thursday, October 17, 2019 11:10 AM
  • Hi Luigi,

    This is an inherited environment and I am still in the process of learning how it's configured.

    Regarding the hardware assessment, what I've learnt so far is that we experience medium disk latency when noderunner is at peak use.

    In the crawl log, I've noticed is that we tend to have more than 2, 3 or 4 millions of errors saying "the object was not found", which is a huge number and most likely a source of an issue!

    Check the content sources that are being crawled: verify if there is root site collection for the web application; if not, please create the root site collection and make sure the content access account has permissions on the root site collection.

    Bye.


    Luigi Bruno
    MCP, MOS, MTA, MCTS, MCSA, MCSE

    Thursday, October 17, 2019 3:23 PM
  • Hi,

    How are things going? Is there any update on your issue?

    Please remeber to mark the reply as an answer if it helps you.

    Thanks for your understanding.

    Best Regards,

    Michael Han

    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    SharePoint Server 2019 has been released, you can click here to download it.
    Click here to learn new features. Visit the dedicated forum to share, explore and talk to experts about SharePoint Server 2019.

    Tuesday, October 29, 2019 9:45 AM
  • Hi Michael, sorry for not updating here; I've been dealing with lots of issues :)

    Turns out that it was the proxy (not the one Internet Explorer, but the one set up with netsh)  that was preventing the crawling process from succeeding.

    "netsh winhttp reset proxy" and the crawl started working flawlessly!

    thank you all


    • Edited by JRS ISOIN Wednesday, November 27, 2019 4:14 PM typo
    • Proposed as answer by Michael Han6 Thursday, November 28, 2019 12:55 AM
    Wednesday, November 27, 2019 4:13 PM
  • Hi, JRS ISOIN,

    Glad that you solved this issue. Thanks for your sharing.

    You could mark your reply as answer. It will be helpful to others who will have similar problems in the future.

    Best Regards,

    Michael Han


    Please remember to mark the replies as answers if they helped. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.

    SharePoint Server 2019 has been released, you can click here to download it.
    Click here to learn new features. Visit the dedicated forum to share, explore and talk to experts about SharePoint Server 2019.

    Thursday, November 28, 2019 12:57 AM
  • Hi Michael, sorry for not updating here; I've been dealing with lots of issues :)

    Turns out that it was the proxy (not the one Internet Explorer, but the one set up with netsh)  that was preventing the crawling process from succeeding.

    "netsh winhttp reset proxy" and the crawl started working flawlessly!

    thank you all


    Good.

    Bye.


    Luigi Bruno
    MCP, MOS, MTA, MCTS, MCSA, MCSE

    Thursday, November 28, 2019 8:52 AM