none
WCF services with random AccessViolation exceptions in ntdll. Performance Counter native error? RRS feed

  • Question

  • I have 2 .NET 4.0 WCF self hosted windows services running that randomly crashes 1 or 2 times a day. These have constant traffic on their services and is run on multiple Windows Server 2008 R2 servers.

    They crash equally often on all the servers. I keep them alive with a helper/restarter service.

    There is no advanced native code in these services. Just simple netTcpBinding with callback. Updating to .NET 4.6.1 did not change the issue.

    I have used DebugDiag on the dumps which identifies this error:

    In MyService1.exe.11212.dmp the assembly instruction at ntdll!memcpy+130 in C:\Windows\SysWOW64\ntdll.dll from Microsoft Corporation has caused an access violation exception (0xC0000005) when trying to read from memory location 0x04325a9c on thread

    but was not able to show the stack of the error. The eventlog shows this:

    Faulting application name: MyService1.exe, version: 1.0.60201.1, time stamp: 0x56af133d
    Faulting module name: ntdll.dll, version: 6.1.7601.23418, time stamp: 0x5708a73e
    Exception code: 0xc0000005
    Fault offset: 0x00022470
    Faulting process id: 0x1484
    Faulting application start time: 0x01d1b1ae4e8ae06f
    Faulting application path: C:\Program Files (x86)\MyService\MyService1.exe
    Faulting module path: C:\Windows\SysWOW64\ntdll.dll
    Report Id: ...


    Using WinDbg i was able to get the stack:
    0:013> kv
    *** Stack trace for last set context - .thread/.cxr resets it
    # ChildEBP RetAddr Args to Child
    00 0713f9c0 6fc12f12 008c6020 04325a9c 00000004 ntdll!memcpy+0x130
    01 0713f9e0 6fc13eb7 04325a9c 04325a9c 00000004 pcwum!StmWrite+0x5a (FPO: [Non-Fpo])
    02 0713fa00 6fc13f7f 0713fa6c 06c965c0 008e4720 pcwum!PerfpAddCounterToStream+0x74 (FPO: [Non-Fpo])

    03 0713fa20 6fc14148 0713fa6c 06c965c0 00000000 pcwum!PerfpAddInstanceToStream+0xac (FPO: [Non-Fpo])
    04 0713fa50 6fc150a9 00000000 0085b148 00002000 pcwum!PerfpAddQueryItemToStream+0x111 (FPO: [Non-Fpo])
    05 0713fa9c 6fc1535b 008e4690 0713fabc 00000000 pcwum!PerfpNotifyAndCollect+0x154 (FPO: [Non-Fpo])
    06 0713fac4 7790574e 0713fb64 008e4690 00000000 pcwum!PerfpNotificationCallback+0xd2 (FPO: [Non-Fpo])
    07 0713fb20 778e7176 0713fb64 03a93f40 00000000 ntdll!TppIopExecuteCallback+0x1c5 (FPO: [Non-Fpo])
    08 0713fc78 7561338a 0086dfc0 0713fcc4 778c9902 ntdll!TppWorkerThread+0x594 (FPO: [Non-Fpo])
    09 0713fc84 778c9902 0086dfc0 6e900de2 00000000 kernel32!BaseThreadInitThunk+0xe (FPO: [Non-Fpo])
    0a 0713fcc4 778c98d5 778e7ad8 0086dfc0 ffffffff ntdll!__RtlUserThreadStart+0x70 (FPO: [Non-Fpo])
    0b 0713fcdc 00000000 778e7ad8 0086dfc0 00000000 ntdll!_RtlUserThreadStart+0x1b (FPO: [Non-Fpo])

    What I see is the function "PerfpAddCounterToStream", but google has never seen that word before. "pcwum" and the names of the functions in the stack make me think that this has to be some Performance Counter issue. Have anyone seen this kind of error before in context of this stack and got any idea on how to fix it?

    Friday, May 20, 2016 11:50 AM

All replies

  • Hello,

    Since this issue occurs randomly, then could you please try to post a simple reproduced project in here? We want to have a test in our site to see what happen.

    >>the names of the functions in the stack make me think that this has to be some Performance Counter issue.

    Have you used the performance counters to diagnose performance in your WCF application?

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, May 23, 2016 2:59 PM
    Moderator
  • I have not used any performance counters for diagnostics.

    I also tried to add "Disable Performance Counters=1" on the services in the registry (in hope that these low level performance counter loggings would not occur) but the same error still occurred.

    I will try to create a sample version of these 2 services. I'll come back with the sample projects. I will have to create them and let them run about 24-48 hours to verify they fail before posting.

    Tuesday, May 24, 2016 7:24 AM
  • During 48 hours both my original projects and test projects did not fail running under Visual Studio (on av Windows 7 VM).

    So I am going to install them as services on a server and wait again.

    Extra info. This error does not occur on 1 of the servers in our development environment at all; Windows Server 2008. It only happens on the Windows Servers 2008 R2's.

    Thursday, May 26, 2016 7:33 AM
  • Hello,

    >>So I am going to install them as services on a server and wait again.

    Any update information? Does you meet the error information again?

    Best Regards,
    Amy Peng


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, May 30, 2016 4:48 AM
    Moderator
  • Sorry for replying to such an old thread, but did you manage to solve this problem? We have a problem now wich sounds just like the one you describe here....
    Wednesday, January 23, 2019 9:23 AM