locked
SharePoint 2013 request hangs in IIS Worker Process - Huge Memory Consumption RRS feed

  • Question

  • Hi all, I'm looking for some assistance on a memory problem that I'm having with our Project Server/ SharePoint environment. After a recent change with a new dedicated web application for mysites, new metadata service application, new business data connection and a new search service we have been experiencing huge consumption of CPU and memory on one of our WFE.

    From what i'm seeing, the system runs fine until a request hits the SharePoint - 80 Application Pool. The request, /_vti_bin/publishingservice.asmx piles up and hangs in the IIS Worker Process. It does not complete or disappears. 

    Some related log lines:

    Process Product Category Level Message
    w3wp.exe (0x24B0) SharePoint Foundation Logging Correlation Data Medium Name=Request (POST:http://WFEServer1:80/_vti_bin/publishingservice.asmx:GetChangedFriendlyUrlBasedWebs")
    w3wp.exe (0x24B0) Web Content Management Publishing Medium GetSecurityChangesFromSharePoint : Site Id = 'a2e1bb0c-01f9-4be3-8749-c64b3cff0b03'. db id = '8e67f439-f001-4052-80bc-57b67096491c'. Group membership/security policy changes detected. All furls in the in the site collection will be crawled.

    The "GetSecurityChangesFromSharePoint" piles up in the log endlessly. 

    Recycling helps reduce the memory consumption on the Virtual Machine but will build back up to 100%. CPU is highly correlated to memory as well.

    Version: 15.0.4719.1000

    WFE: 64-bit, four-core (2 core 2 CPUs), 2.00 GHz per core, 20 GB RAM

    Anyone have any insight or references I can review to get a better idea on how to resolve this?


    Paul Lor

    Thursday, November 10, 2016 6:23 PM

All replies

  • Well, how much RAM is it? And you're seeing this due to ACLs being modified either at the Web App scope directly or via SharePoint Security Groups.

    https://blogs.msdn.microsoft.com/kaevans/2013/05/06/clarifying-guidance-on-sharepoint-security-groups-versus-active-directory-domain-services-groups/


    Trevor Seward

    Office Servers and Services MVP



    Author, Deploying SharePoint 2016

    This post is my own opinion and does not necessarily reflect the opinion or view of Microsoft, its employees, or other MVPs.

    Thursday, November 10, 2016 7:59 PM
  • WFE: 64-bit, four-core (2 core 2 CPUs), 2.00 GHz per core, 20 GB RAM

    The Application Pool w3wp creeps up to about 13-14 GB of rams. Which at that point is about 100% memory utilization. 

    I believe I'm seeing this when a incremental crawl starts.


    Paul Lor

    Thursday, November 10, 2016 8:29 PM
  • Is this a single server farm? How many indexed items are there?

    Trevor Seward

    Office Servers and Services MVP



    Author, Deploying SharePoint 2016

    This post is my own opinion and does not necessarily reflect the opinion or view of Microsoft, its employees, or other MVPs.

    Thursday, November 10, 2016 8:40 PM
  • 2 WFE 1 App 1 SQL Box, high memory only occurs on one of the WFE.

    Searchable items 99,500
    Crawl rate (dps) 0.2
    Total items 11,333
    Modified items 8,385
    Not modified items 83
    Security items 657
    Deleted items 12
    Retries 179
    Errors 2,017

    Not sure how to find total indexed items.

    Paul Lor

    Thursday, November 10, 2016 8:57 PM
  • Hi Paul Lor,

    You can get the number of the indexed item in search by REST. For more detailed information, you could refer to the article below.

    How to: Return all items in the Search Index in SharePoint 2013.

    http://www.dotnetmafia.com/blogs/dotnettipoftheday/archive/2013/10/11/how-to-return-all-items-in-the-search-index-in-sharepoint-2013.aspx

    To troubleshooting the SharePoint performance issue, you could check things below:

    1. Clear SharePoint configuration cache. For more detailed information, you could refer to the article below.

    Clearing the Configuration Cache for SharePoint.

    http://www.sharepointpitstop.com/2013/09/clearing-configuration-cache-for.html

    2. Use IIS overlapped recycling

    3. Use 64-bit hardware.

    4. IIS reset.

    5. Use fiddler for debugging. For more detailed information, you could refer to the articles below.

    Configuring SharePoint 2013 for Fiddler.

    https://www.techarex.net/blog/microsoft-sharepoint/configuring-sharepoint-2013-for-fiddler/

    How to debug your application (http protocol) using Fiddler.

    http://www.infragistics.com/community/blogs/anton_staykov/archive/2010/08/24/how-to-debug-your-application-http-protocol-using-fiddler.aspx

    More references:

    Troubleshooting IIS and SharePoint When a Worker Process Goes Haywire.

    http://www.collabshow.com/2011/07/12/troubleshooting-iis-and-sharepoint-when-a-worker-process-goes-haywire/

    Troubleshoot hanging requests on IIS in 3 steps

    http://mvolo.com/troubleshoot-iis-hanging-requests/

    And you also could follow the steps in the article below to improve the performance in SharePoint.

    Improving SharePoint Performance.

    https://technet.microsoft.com/en-us/library/2009.08.insidesharepoint.aspx

    Best regards,

    Sara Fan


    Please remember to mark the replies as answers if they help.
    If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com

    • Proposed as answer by Victoria Xia Thursday, November 24, 2016 1:13 AM
    Friday, November 11, 2016 9:13 AM
  • 2 WFE 1 App 1 SQL Box, high memory only occurs on one of the WFE.

    Searchable items 99,500
    Crawl rate (dps) 0.2
    Total items 11,333
    Modified items 8,385
    Not modified items 83
    Security items 657
    Deleted items 12
    Retries 179
    Errors 2,017

    Not sure how to find total indexed items.

    Paul Lor

    That shows you have just under 100k indexed items. What I would suggest you do is target the crawler at one of your App servers. You can do this by making sure the Web App is provisioned on your App servers and then edit the hosts file on your Search Crawler(s) server(s) to point the Web App hostname/FQDN at the local machine. This way the FE will not be crawled for search content.

    Trevor Seward

    Office Servers and Services MVP



    Author, Deploying SharePoint 2016

    This post is my own opinion and does not necessarily reflect the opinion or view of Microsoft, its employees, or other MVPs.

    • Proposed as answer by Victoria Xia Thursday, November 24, 2016 1:13 AM
    Friday, November 11, 2016 6:38 PM
  • Adding on Trevor recommendation:

    1. Move whole search components to App Server. Since, you've less than 100K searchable items dedicated Search server not needed that much.

    2. Reset Index.

    3. Check and verify your content sources for correct URL's

    4. Check with network team if proxy is not an issue during crawling.

    5. Schedule crawling after peak hours.

    Hope this helps

    Friday, November 11, 2016 7:22 PM
  • One should be using Continuous Crawl for SharePoint resources (outside of special circumstances) with *very rarely* a full crawl. If you're going to use full crawls, schedule them every few months. Often times I'll see people schedule these daily or weekly, and it just isn't required.

    Trevor Seward

    Office Servers and Services MVP



    Author, Deploying SharePoint 2016

    This post is my own opinion and does not necessarily reflect the opinion or view of Microsoft, its employees, or other MVPs.

    • Proposed as answer by Victoria Xia Thursday, November 24, 2016 1:13 AM
    Friday, November 11, 2016 7:23 PM