none
WCF Web Service Suddenly Stops During Processing RRS feed

  • Question

  • I am banging my head on this one. Source System sends request to IBM ESB which passes the request to a Netscaler Load Balanced URL which calls a service to process a request. On 2/6 @ ~ 5:03AM, 13 requests were sent nearly at the same time and 12 suddenly stopped processing and 1 was successful. There is no single point to the stoppage to investigate as it could happen in the middle 1/3, middle, or end of the work flow. The framework has significant error handling and the service is wrapped in a try/catch for unhandled exception. No errors were record. Nothing in the machine's EventViewer. No response provided to the client making the call. It just stops. IIS logs indicate a "500" error but nothing else. Why wouldn't the WCF service captures the exception? Any ideas?

    Shawn (Mail@ShawnAugust.Com)



    • Edited by SAgosto Monday, February 8, 2016 4:47 PM
    Monday, February 8, 2016 3:51 PM

Answers

  • Thank you for your help. The root cause was the handling of the service's unhandledexception event which automatically terminates the thread. To resolve, instead of using this catch/all approach, I wrapped the various method calls with try/catch and called a common error handler. I guess you learn something everyday.

    Shawn (Mail@ShawnAugust.Com)

    • Marked as answer by SAgosto Thursday, February 25, 2016 2:17 AM
    Thursday, February 25, 2016 2:17 AM

All replies

  • I am banging my head on this one. Source System sends request to IBM ESB which passes the request to a Netscaler Load Balanced URL which calls a service to process a request. On 2/6 @ ~ 5:03AM, 13 requests were sent nearly at the same time and 12 suddenly stopped processing and 1 was successful. There is no single point to the stoppage to investigate as it could happen in the middle 1/3, middle, or end of the work flow. The framework has significant error handling and the service is wrapped in a try/catch for unhandled exception. No errors were record. Nothing in the machine's EventViewer. No response provided to the client making the call. It just stops. IIS logs indicate a "500" error but nothing else. Why wouldn't the WCF service captures the exception? Any ideas?

    Shawn (Mail@ShawnAugust.Com)


    TTT!


    Shawn (Mail@ShawnAugust.Com)

    Tuesday, February 9, 2016 1:32 PM
  • Hello,

    >>IIS logs indicate a "500" error but nothing else. Why wouldn't the WCF service captures the exception? Any ideas?

    The 500 error message is a general error,  I would suggest you log the messages when you catch the exception and some important operation. After that you could analyze the issue from the log files. Besides, please try to enable the WCF trace to get some useful information.

    The following configuration taken from MSDN can be applied to enable tracing on your WCF service.

    <configuration>
      <system.diagnostics>
        <sources>
          <source name="System.ServiceModel"
                  switchValue="Information, ActivityTracing"
                  propagateActivity="true" >
            <listeners>
                 <add name="xml"/>
            </listeners>
          </source>
          <source name="System.ServiceModel.MessageLogging">
            <listeners>
                <add name="xml"/>
            </listeners>
          </source>
          <source name="myUserTraceSource"
                  switchValue="Information, ActivityTracing">
            <listeners>
                <add name="xml"/>
            </listeners>
          </source>
        </sources>
        <sharedListeners>
            <add name="xml"
                 type="System.Diagnostics.XmlWriterTraceListener"
                 initializeData="Error.svclog" />
        </sharedListeners>
      </system.diagnostics>
    </configuration>

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.



    Tuesday, February 9, 2016 3:25 PM
    Moderator
  • Hello,

    >>IIS logs indicate a "500" error but nothing else. Why wouldn't the WCF service captures the exception? Any ideas?

    The 500 error message is a general error,  I would suggest you log the messages when you catch the exception and some important operation. After that you could analyze the issue from the log files. Besides, please try to enable the WCF trace to get some useful information.

    The following configuration taken from MSDN can be applied to enable tracing on your WCF service.

    <configuration>
      <system.diagnostics>
        <sources>
          <source name="System.ServiceModel"
                  switchValue="Information, ActivityTracing"
                  propagateActivity="true" >
            <listeners>
                 <add name="xml"/>
            </listeners>
          </source>
          <source name="System.ServiceModel.MessageLogging">
            <listeners>
                <add name="xml"/>
            </listeners>
          </source>
          <source name="myUserTraceSource"
                  switchValue="Information, ActivityTracing">
            <listeners>
                <add name="xml"/>
            </listeners>
          </source>
        </sources>
        <sharedListeners>
            <add name="xml"
                 type="System.Diagnostics.XmlWriterTraceListener"
                 initializeData="Error.svclog" />
        </sharedListeners>
      </system.diagnostics>
    </configuration>

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.



    The web method is wrapped in a try/catch which writes to the Event Viewer via MS Enterprise Library. Also, a response object is also returned to the client. It appears that the thread simply stops. I looked at more logging and it appears that there was excessive waits in the response from an external API call. I'd expect a timeout or some other type of error and not a thread stoppage.

    Shawn


    Shawn (Mail@ShawnAugust.Com)

    Tuesday, February 9, 2016 4:21 PM
  • Hi SAgosto,

    By default, WCF do not log any message logs, we need to enable the trace as I described in my first post, then please try to turn on IncludeExceptionDetailInFaults to enable exception information to flow to clients for debugging purposes:

    <system.serviceModel>
        <behaviors>
          <serviceBehaviors>
            <behavior>
              <serviceDebug includeExceptionDetailInFaults="true" />
            </behavior>
          </serviceBehaviors>
        </behaviors>
        ...
      </system.serviceModel>

    Since I do not have your wcf service, it is hard to reproduce this issue in my side, as you said, if 13 requests were sent nearly at the same time and 12 suddenly stopped processing and 1 was successful, please try to a test to do not send your request at the same time to see if it works. Besides, I do not know if this issue is due to the timeout value, after reaching the limit of the timeout value, then the thread stops. Anyway please try to increase the following timeout value in the both client and sevice side:

     <binding name="BindingConfig"
                   openTimeout="00:50:00"
                   closeTimeout="00:50:00"
                   sendTimeout="00:50:00"
                   receiveTimeout="00:50:00">
            <reliableSession enabled="true"
                             inactivityTimeout="00:50:00" />


    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.



    Monday, February 22, 2016 6:08 AM
    Moderator
  • Thank you for your help. The root cause was the handling of the service's unhandledexception event which automatically terminates the thread. To resolve, instead of using this catch/all approach, I wrapped the various method calls with try/catch and called a common error handler. I guess you learn something everyday.

    Shawn (Mail@ShawnAugust.Com)

    • Marked as answer by SAgosto Thursday, February 25, 2016 2:17 AM
    Thursday, February 25, 2016 2:17 AM
  • Hi SAgosto,

    Well Done!
    It is very kind of you to share the solution to us. :)

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Thursday, February 25, 2016 2:46 AM
    Moderator
  • Hi SAgosto,

    Well Done!
    It is very kind of you to share the solution to us. :)

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    I am guilty of forgetting to follow up when i figure it out on my own. But, i am grateful for the help.

    Shawn (Mail@ShawnAugust.Com)

    Thursday, February 25, 2016 2:49 AM