none
WFE Server CPU at 100% for W3WP3.exe

    Question

  • Hi,

    One of my WFE server's CPU is constanly running at 100%.  It is the W3WP.exe process.  It is pointing to the main web aplication app pool.  It use IISPeek to see what's going on and here's the main cause it reported back:

    =========
    Collaboration AppPool
    pid (7108)
    REQUEST: 14771806791734011480     VERB: POST     URL: /rd/_vti_bin/sitedata.asmx
    ELAPSED: 1     PIPE-STATE: Processing     TIME-IN-STATE: 0
    CLIENT-IP: 10.12.30.250     PORT: 80
    ================

    Based on the above, it is point to the RD site collection that is causing the CPU spike.  The "_vti_bin/sitedata.asmx" is the Sharepoint Web Services.  I look at the site, it does not have anything other than the default Announcements, Links, etc.  There are, however, a bunch of subsite but this should not cause it as it is point to the RD (top level site) and each subsites would have its own "_vti_bin/sitedata.asmx".

    I can stop and restart the process... but it would spike up again after some time.  So, how can I figure out what in that site is causing this?

    Thanks in advance.
    Ultra
    Wednesday, October 14, 2009 5:44 PM

Answers

All replies

  • is making a reference to metadata List webservices, have you custom code to call webservices?? how many user are trying to connect to that webservices you have knowledge? this have impact and off course impact on your w3w.exe process, many requests, the more IIS have to process...
    André Lage Microsoft SharePoint and CRM Consultant
    Blog:http://aaclage.blogspot.com
    Codeplex:http://spupload.codeplex.com/http://simplecamlsearch.codeplex.com/
    Wednesday, October 14, 2009 5:55 PM
  • Thanks for the comment.

    This is the top lovel site.  There's nothing on that site I see have been customized.  The webparts are the default webparts.  This is crazy.
    Wednesday, October 14, 2009 9:23 PM
  • Is this a 32 bit deployment? If yes, can you check boot.ini (usually you can tell notepad to open C:\boot.ini to see it) to see if you have the /3GB switch? If yes, take it out and reboot the server. 32bit SharePoint and /3GB switch don't play nice. See KB933560.

    Thursday, October 15, 2009 7:49 PM
  • Also are you running in a virtualized environment?  If so make sure the clock is synced with the other servers.  Another thing to check is the loopback.  Are you able to access the site directly from the server? 

    Aseem Nayar - MSFT
    This posting is provided "AS IS" with no warranties, and confers no rights
    Thursday, October 15, 2009 8:23 PM
  • What are your other WFEs doing when this one is spiking?

    I have seen this when load balancing gets out of synch and directs all traffic to a single node.  I would also suggest that you temporarily add another worker process to the web app - and see if it still spikes at that capacity.

    Regards

    John Timney
    Thursday, October 15, 2009 9:05 PM
  • I think André Lage recommended you to find out how many times the sitedata.asmx was called, that is, the workload of the web service. You may find it out in IIS log, it will record every call to the web services in sitedata.asmx. Just count the number of sitedata.asmx records during a period of time will reveal the workload. And from the c-ip field (http://www.microsoft.com/technet/prodtechnol/WindowsServer2003/Library/IIS/ffdd7079-47be-4277-921f-7a3a6e610dcb.mspx?mfr=true ) in IISlog, you can also figure out where these web service calling were from.

     

    And the article follows also shows how to analyze the performance of web services with the IISLog time-taken field: http://blogs.msdn.com/alikl/archive/2007/11/02/identify-asp-net-web-services-and-wcf-performance-issues-by-examining-iis-logs.aspx .

    Friday, October 16, 2009 4:12 AM
  • Thanks all for the comments.  I have opened a ticket with MS Support.  As always, they want you to run a bunch of diags etc... still working on it.  To answer some of the questions:

    - All my servers are physical
    - Running 64 bit W2k3 R2 Enterprise
    - 6GB RAM

    The spike seems to alternate amonth the two WFE servers.  Meaning, it doesn't seem like it is specific to one server.  However, I do notice in each case, it is telling me that the spike is coming from  http://mydomain/rd/_vti_bin/sitedata.asmx.  This is why it is leading me to feel is a specific site related.

    Wednesday, October 21, 2009 6:39 AM
  • Update... MS Tech tooks some IIS and performance dump.  The results incates a large xml file (4MB) was pushing thru the IIS when the CPU spiked.  Again, I don't see much in the site that would do something like this.  I am waiting to chat with the Tech and see if we can zero it down further.

    BTW.  I am running W2K3 R3 w/ SP2 x64.  Have 12GB RAM on the server w/ Quad Core... all physical servers.  When this server peaks, the other WFE is unaffected.
    Tuesday, November 03, 2009 4:09 PM
  • Did you find a solution for this? I was experiencing the same issue, and like you discovered IISPeek > sitedata.asmx... I soon after tracked it down to my incremental searches.
    Tuesday, July 20, 2010 3:41 AM