WCF service hosted on IIS with custom authentication provider, some requests take 6-7 minutes to respond RRS feed

  • Question

  • We have a WCF service hosted on IIS 7.5 which has custom username authentication using a custom membership provider(we authenticate users from a database with some custom logic). The service uses wshttpbinding with transportwithmessagecredentials security mode.

    It works well with heavy loads but sometimes for about 6-9 minutes it keeps the requests waiting for 6-9 minutes before responding. I added debug logs to the custom authentication module and the service operation that was being called and noticed that for these instances, there was this lag of 6-9 minutes between the end of execution of the authentication module and start of execution of the service method.

    The requests responds after these 6-9 minutes correctly. It looks like they got stuck somewhere between the authentication and actual service method call, but I have got no clue as to what is causing it. This is becoming a problem as the service clients are getting timed out during these periods.

    EDIT- I enabled the service trace and observed that for the requests that were getting delayed, there was this delay between the "Received a message over a channel." and the "Understood message header." events. The thread ID also changes between these two events. Can that mean that there are no free threads? But are there no free threads for 7 minutes?



    • Edited by Ubaid Tahir Monday, March 10, 2014 6:59 PM More info
    Monday, March 10, 2014 5:54 PM

All replies

  • Hi,

    Sounds like that the application pool is recycling causing a recompile.  Are you seeing anything in the system event log or application log unexpected errors.  Can you reproduce on another machine stress testing? And is there sql server involved or some other remote data source such as web services? Also please try to attach a debugger to debug the service.

    Also please try to use the fiddler to help troubleshooting:
    #Fiddler: .

    Best Regards,
    Amy Peng

    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, March 11, 2014 9:54 AM
  • I have checked the event logs and could not find any error regarding this. I am able to reproduce this on  a local machine with stress testing. We do make SQL server calls from within the code but the problem seems to be within the WCF pipeline. I can see in the WCF trace the step where it gets stuck.

    I am even more surprised that it gives response after waiting for 5-7 minutes. I ran process monitor on the server and found out that during the 5-7 minutes interval the w3wp threads are free.

    The count of IO threads however does not exceed 30 in spite of the load that I put on the service. 

    Tuesday, March 11, 2014 1:53 PM