none
Application crash : named pipes + mscorwks.dll = access violation RRS feed

  • Question

  • Hi,

    I'm experiencing trouble with named pipes in a C# module. Basically, this module is meant to allow interop communication between our program (which is hosting the faulting module) and third-party applications (Delphi-based for instance) through Named Pipes.

    Environment:
    Windows XP SP3
    DotNet 3.5 SP1

    Problem:
    Our program crashes after a few seconds/minutes/hours (depending on CPU/RAM resources). WinDbg report shows a "Access violation - code c0000005 (first chance)" problem. Running "!analyze -v" and "!VerifyHeap" leads to memory corruption during GC (FAULTING_IP: mscorwks!WKS::gc_heap::revisit_written_page+7f && object 01718790: does not have valid MT, see "ANALYSE #1" below for details).

    Resolution attempt #1:
    C# code is based on P/Invoke ([DllImport("kernel32.dll", SetLastError = true)], ConnectNamedPipe, WaitForMultipleObjects, ReadFile, etc). I noticed plain IntPtr was used for handles, which appears to be a problem and to cause the kind of crash we're having. So, I replaced IntPtr by SafeHandles (SafeFileHandle).

    This move has done nothing but change the error report : now I have a "Invalid handle - code c0000008 (first chance)", "Thread tried to close a handle that was invalid or illegal to close", see "ANALYSE #2" below for details.

    I tried to guard calls to native code with "!handle.IsInvalid" but the error breaks in native method code (mostly WaitForMultipleObjects(), sometime ReadFile()).

    Any advice would be greatly appreciated !
    Thanks
    Fabrice

    //////////////////////////// ANALYSE #1 //////////////////////////////////////

    0:024> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Exception Analysis                                   *
    *                                                                             *
    *******************************************************************************

    *************************************************************************
    ***                                                                   ***
    ***                                                                   ***
    ***    Your debugger is not using the correct symbols                 ***
    ***                                                                   ***
    ***    In order for this command to work properly, your symbol path   ***
    ***    must point to .pdb files that have full type information.      ***
    ***                                                                   ***
    ***    Certain .pdb files (such as the public OS symbols) do not      ***
    ***    contain the required information.  Contact the group that      ***
    ***    provided you with these symbols if you need this command to    ***
    ***    work.                                                          ***
    ***                                                                   ***
    ***    Type referenced: kernel32!pNlsUserInfo                         ***
    ***                                                                   ***
    *************************************************************************

    FAULTING_IP:
    mscorwks!WKS::gc_heap::revisit_written_page+7f
    79f46adb f70000000080    test    dword ptr [eax],80000000h

    EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
    ExceptionAddress: 79f46adb (mscorwks!WKS::gc_heap::revisit_written_page+0x0000007f)
       ExceptionCode: c0000005 (Access violation)
      ExceptionFlags: 00000000
    NumberParameters: 2
       Parameter[0]: 00000000
       Parameter[1]: 00000000
    Attempt to read from address 00000000

    FAULTING_THREAD:  00001270

    DEFAULT_BUCKET_ID:  NULL_POINTER_READ

    PROCESS_NAME:  Dcns.Sam.Shell.exe

    ERROR_CODE: (NTSTATUS) 0xc0000005 - L'instruction   "0x%08lx" emploie l'adresse m moire "0x%08lx". La m moire ne peut pas  tre "%s".

    EXCEPTION_CODE: (NTSTATUS) 0xc0000005 - L'instruction   "0x%08lx" emploie l'adresse m moire "0x%08lx". La m moire ne peut pas  tre "%s".

    EXCEPTION_PARAMETER1:  00000000

    EXCEPTION_PARAMETER2:  00000000

    READ_ADDRESS:  00000000

    FOLLOWUP_IP:
    mscorwks!WKS::gc_heap::revisit_written_page+7f
    79f46adb f70000000080    test    dword ptr [eax],80000000h

    NTGLOBALFLAG:  0

    APPLICATION_VERIFIER_FLAGS:  0

    MANAGED_STACK: !dumpstack -EE
    OS Thread Id: 0x1270 (24)
    Current frame:
    ChildEBP RetAddr  Caller,Callee

    PRIMARY_PROBLEM_CLASS:  NULL_POINTER_READ

    BUGCHECK_STR:  APPLICATION_FAULT_NULL_POINTER_READ

    LAST_CONTROL_TRANSFER:  from 79f46c72 to 79f46adb

    STACK_TEXT: 
    1065ff04 79f46c72 01718000 0195c000 00000000 mscorwks!WKS::gc_heap::revisit_written_page+0x7f
    1065ff48 79f46d68 00000001 00000002 00000000 mscorwks!WKS::gc_heap::revisit_written_pages+0xe7
    1065ff78 79f47753 7a3b9010 7a3b9028 1065ffac mscorwks!WKS::gc_heap::c_mark_phase+0xb1
    1065ff94 79f823d5 00000000 0dede710 00000000 mscorwks!WKS::gc_heap::gc1+0x59
    1065ffac 79f82427 00000000 7c80b729 00000000 mscorwks!WKS::gc_heap::gc_thread_function+0x9f
    1065ffb4 7c80b729 00000000 00000000 0dede710 mscorwks!WKS::gc_heap::gc_thread_stub+0x73
    1065ffec 00000000 79f823f8 00000000 00000000 KERNEL32!BaseThreadStart+0x37


    SYMBOL_STACK_INDEX:  0

    SYMBOL_NAME:  mscorwks!WKS::gc_heap::revisit_written_page+7f

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: mscorwks

    IMAGE_NAME:  mscorwks.dll

    DEBUG_FLR_IMAGE_TIMESTAMP:  4a7cd88e

    STACK_COMMAND:  ~24s ; kb

    FAILURE_BUCKET_ID:  NULL_POINTER_READ_c0000005_mscorwks.dll!WKS::gc_heap::revisit_written_page

    BUCKET_ID:  APPLICATION_FAULT_NULL_POINTER_READ_mscorwks!WKS::gc_heap::revisit_written_page+7f

    Followup: MachineOwner
    ---------

    0:024> ~
       0  Id: 868.d20 Suspend: 1 Teb: 7ffdf000 Unfrozen
       1  Id: 868.9d0 Suspend: 1 Teb: 7ffde000 Unfrozen
       2  Id: 868.4e4 Suspend: 1 Teb: 7ffdd000 Unfrozen
       3  Id: 868.16b8 Suspend: 1 Teb: 7ffdc000 Unfrozen
       4  Id: 868.d30 Suspend: 1 Teb: 7ffdb000 Unfrozen
       5  Id: 868.fb0 Suspend: 1 Teb: 7ffda000 Unfrozen
       6  Id: 868.f34 Suspend: 1 Teb: 7ffd9000 Unfrozen
       7  Id: 868.a64 Suspend: 1 Teb: 7ffd7000 Unfrozen
       8  Id: 868.9a4 Suspend: 1 Teb: 7ffd4000 Unfrozen
       9  Id: 868.4c4 Suspend: 1 Teb: 7ff4e000 Unfrozen
      10  Id: 868.724 Suspend: 1 Teb: 7ff4d000 Unfrozen
      11  Id: 868.9f8 Suspend: 1 Teb: 7ffd5000 Unfrozen
      12  Id: 868.280 Suspend: 1 Teb: 7ff4f000 Unfrozen
      13  Id: 868.a40 Suspend: 1 Teb: 7ff4c000 Unfrozen
      14  Id: 868.a4c Suspend: 1 Teb: 7ff4b000 Unfrozen
      15  Id: 868.870 Suspend: 1 Teb: 7ff4a000 Unfrozen
      16  Id: 868.f98 Suspend: 1 Teb: 7ff48000 Unfrozen
      17  Id: 868.e74 Suspend: 1 Teb: 7ff47000 Unfrozen
      18  Id: 868.a9c Suspend: 1 Teb: 7ff44000 Unfrozen
      19  Id: 868.fe8 Suspend: 1 Teb: 7ff45000 Unfrozen
      20  Id: 868.1380 Suspend: 1 Teb: 7ff46000 Unfrozen
      21  Id: 868.a04 Suspend: 1 Teb: 7ffd6000 Unfrozen
      22  Id: 868.f20 Suspend: 1 Teb: 7ff49000 Unfrozen
    . 24  Id: 868.1270 Suspend: 1 Teb: 7ff42000 Unfrozen
    0:024> !DumpStack
    OS Thread Id: 0x1270 (24)
    Current frame: mscorwks!WKS::gc_heap::revisit_written_page+0x7f
    ChildEBP RetAddr  Caller,Callee
    1065ff04 79f46c72 mscorwks!WKS::gc_heap::revisit_written_pages+0xe7, calling mscorwks!WKS::gc_heap::revisit_written_page
    1065ff48 79f46d68 mscorwks!WKS::gc_heap::c_mark_phase+0xb1, calling mscorwks!WKS::gc_heap::revisit_written_pages
    1065ff78 79f47753 mscorwks!WKS::gc_heap::gc1+0x59, calling mscorwks!WKS::gc_heap::c_mark_phase
    1065ff88 79f823b9 mscorwks!WKS::gc_heap::gc_thread_function+0x38, calling mscorwks!CLREvent::Wait
    1065ff94 79f823d5 mscorwks!WKS::gc_heap::gc_thread_function+0x9f, calling mscorwks!WKS::gc_heap::gc1
    1065ffac 79f82427 mscorwks!WKS::gc_heap::gc_thread_stub+0x73, calling mscorwks!WKS::gc_heap::gc_thread_function
    1065ffb4 7c80b729 KERNEL32!BaseThreadStart+0x37
    0:024> !VerifyHeap
    -verify will only produce output if there are errors in the heap
    object 01718790: does not have valid MT
    curr_object : 01718790
    Last good object: 01718744
    ----------------
    0:024> !do 01718790
    <Note: this object has an invalid CLASS field>
    Invalid object

    //////////////////////////// ANALYSE #2 //////////////////////////////////////
    (1410.95c): Invalid handle - code c0000008 (first chance)
    First chance exceptions are reported before any exception handling.
    This exception may be expected and handled.
    eax=c0000008 ebx=00170010 ecx=00c4f6dc edx=7c91e4c8 esi=00c4f720 edi=00c4f950
    eip=7c91e4ff esp=00c4f688 ebp=00c4f6d8 iopl=0         nv up ei pl nz na pe nc
    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
    ntdll!KiRaiseUserExceptionDispatcher+0x37:
    7c91e4ff 8b0424          mov     eax,dword ptr [esp]  ss:0023:00c4f688=c0000008
    0:002> !analyze -v
    *******************************************************************************
    *                                                                             *
    *                        Exception Analysis                                   *
    *                                                                             *
    *******************************************************************************

    *************************************************************************
    ***                                                                   ***
    ***                                                                   ***
    ***    Your debugger is not using the correct symbols                 ***
    ***                                                                   ***
    ***    In order for this command to work properly, your symbol path   ***
    ***    must point to .pdb files that have full type information.      ***
    ***                                                                   ***
    ***    Certain .pdb files (such as the public OS symbols) do not      ***
    ***    contain the required information.  Contact the group that      ***
    ***    provided you with these symbols if you need this command to    ***
    ***    work.                                                          ***
    ***                                                                   ***
    ***    Type referenced: kernel32!pNlsUserInfo                         ***
    ***                                                                   ***
    *************************************************************************

    FAULTING_IP:
    ntdll!KiRaiseUserExceptionDispatcher+37
    7c91e4ff 8b0424          mov     eax,dword ptr [esp]

    EXCEPTION_RECORD:  ffffffff -- (.exr 0xffffffffffffffff)
    ExceptionAddress: 7c91e4ff (ntdll!KiRaiseUserExceptionDispatcher+0x00000037)
       ExceptionCode: c0000008 (Invalid handle)
      ExceptionFlags: 00000000
    NumberParameters: 0
    Thread tried to close a handle that was invalid or illegal to close

    FAULTING_THREAD:  0000095c

    DEFAULT_BUCKET_ID:  WRONG_SYMBOLS

    PROCESS_NAME:  Dcns.Sam.Shell.exe

    ERROR_CODE: (NTSTATUS) 0xc0000008 - Un handle non valide a  t  sp cifi .

    EXCEPTION_CODE: (NTSTATUS) 0xc0000008 - Un handle non valide a  t  sp cifi .

    NTGLOBALFLAG:  0

    APPLICATION_VERIFIER_FLAGS:  0

    MANAGED_STACK:
    (TransitionMU)
    00C4F730 7927987D mscorlib_ni!Microsoft.Win32.SafeHandles.SafeFileHandle.ReleaseHandle()+0xd
    (TransitionUM)
    (TransitionMU)
    00C4FC0C 792E5E4F mscorlib_ni!System.Runtime.InteropServices.SafeHandle.Dispose(Boolean)+0xf
    00C4FC14 792E5D6B mscorlib_ni!System.Runtime.InteropServices.SafeHandle.Finalize()+0x1b

    LAST_CONTROL_TRANSFER:  from 7c91e513 to 7c91e4ff

    PRIMARY_PROBLEM_CLASS:  WRONG_SYMBOLS

    BUGCHECK_STR:  APPLICATION_FAULT_WRONG_SYMBOLS

    STACK_TEXT: 
    7927987d mscorlib_ni!Microsoft.Win32.SafeHandles.SafeFileHandle.ReleaseHandle+0xd


    FOLLOWUP_IP:
    mscorlib_ni+1b987d
    7927987d 5d              pop     ebp

    SYMBOL_STACK_INDEX:  0

    SYMBOL_NAME:  mscorlib_ni!Microsoft.Win32.SafeHandles.SafeFileHandle.ReleaseHandle+1b987d

    FOLLOWUP_NAME:  MachineOwner

    MODULE_NAME: mscorlib_ni

    IMAGE_NAME:  mscorlib.ni.dll

    DEBUG_FLR_IMAGE_TIMESTAMP:  4a7cd8f7

    STACK_COMMAND:  ** Pseudo Context ** ; kb

    FAILURE_BUCKET_ID:  WRONG_SYMBOLS_c0000008_mscorlib.ni.dll!Microsoft.Win32.SafeHandles.SafeFileHandle.ReleaseHandle

    BUCKET_ID:  APPLICATION_FAULT_WRONG_SYMBOLS_mscorlib_ni!Microsoft.Win32.SafeHandles.SafeFileHandle.ReleaseHandle+1b987d

    Followup: MachineOwner
    ---------

    Friday, April 9, 2010 10:01 AM

Answers

  • Using SafeHandles seems reasonable approach to find the root cause. I guess that either some component is closing the handle twice, or something (e.g. wrong PInvoke signature) writes garbage into the handle. I would try to add handle value logging around the PInvokes to see if the value changed before it was released.

    I don't quite understand your statement "I tried to guard calls to native code with "!handle.IsInvalid" but the error breaks in native method code (mostly WaitForMultipleObjects(), sometime ReadFile()).", what did you mean by 'the error breaks in native method call'?

    -Karel

    • Marked as answer by SamAgain Friday, April 16, 2010 9:57 AM
    Friday, April 9, 2010 4:08 PM
    Moderator

All replies

  • My question is there a specific reason you are not using the NamedPipeServerStream which comes out of the box in .NET 3.5?
    Thanks Naveen http://naveensrinivasan.com
    Friday, April 9, 2010 1:14 PM
  • Using SafeHandles seems reasonable approach to find the root cause. I guess that either some component is closing the handle twice, or something (e.g. wrong PInvoke signature) writes garbage into the handle. I would try to add handle value logging around the PInvokes to see if the value changed before it was released.

    I don't quite understand your statement "I tried to guard calls to native code with "!handle.IsInvalid" but the error breaks in native method code (mostly WaitForMultipleObjects(), sometime ReadFile()).", what did you mean by 'the error breaks in native method call'?

    -Karel

    • Marked as answer by SamAgain Friday, April 16, 2010 9:57 AM
    Friday, April 9, 2010 4:08 PM
    Moderator