none
.Net Profiler - AppCrash worker process - ways to debug RRS feed

  • Question

  • I have implemented CORProfiler callback functions for instrumenting .Net web applications. 

    My profiler dll loads another dll at the initialization part(LogManager.dll) of profiler . This dll takes profiled data from the profiler dll and process it in a set frequency. But , After few hours of processing the w3wp.exe gets crashed with triggering error in event logs. The error is attached below



    This LogManager dll was loaded by my profiler dll that gets attached to w3wp.exe . Now how can i find out the crash.?

    I tried building debug profiler dll and logmanager dll and attaching to the visual studio.. but gives some crash immediately after attachment(before getting attached a warning displays that "the operation might harm your computer". Is there any other way to find out the crash without a debug version of the dlls? Sometime i get StackHash error in ntdll and not in logmanager dll..!

    I am also not sure why after some hours this crash occurs. Not able to sort this crash. Any idea will be more helpful. 

    Friday, June 16, 2017 5:45 AM

Answers

  • My guess is that the access violation happened while trying to dereference childList. Maybe childList was null, uninitialized or freed ... or maybe logObj was pointing to an address in a committed page but was not actually a valid object, in which case its 'childList' field would contain random garbage.
    • Marked as answer by Selva VS Monday, June 19, 2017 4:41 AM
    Sunday, June 18, 2017 4:02 AM
  • It's usually better to copy the text rather than using screenshots.

    I believe the "fault offset" (0x10C32) is the location where the crash happened relative to the module base address of the faulting module (LogManager.dll). Raymond Chan described a technique for restoring a stacktrace that is missing the source mappings, the same process could be used to convert this offset to source locations (assuming you have the pdb).

    ntsd.exe -z LogManager.dll


    0:000> .lines
    Line number information will be loaded
    0:000> u LogManager+0x10c32
    LogManager!OffendingMethodName+0x0123 [c:\path\to\source\file.cpp @ 42]:

    https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/

    The error code 0xC0000005 indicates an access violation, maybe you dereferenced an uninitialized pointer, maybe you accessed a piece of memory after freeing it, but since it happens after a few hours, I would guess at a buffer overflow.


    • Edited by Brian Reichle Friday, June 16, 2017 2:27 PM
    • Marked as answer by Selva VS Monday, June 19, 2017 4:41 AM
    Friday, June 16, 2017 12:28 PM

All replies

  • It's usually better to copy the text rather than using screenshots.

    I believe the "fault offset" (0x10C32) is the location where the crash happened relative to the module base address of the faulting module (LogManager.dll). Raymond Chan described a technique for restoring a stacktrace that is missing the source mappings, the same process could be used to convert this offset to source locations (assuming you have the pdb).

    ntsd.exe -z LogManager.dll


    0:000> .lines
    Line number information will be loaded
    0:000> u LogManager+0x10c32
    LogManager!OffendingMethodName+0x0123 [c:\path\to\source\file.cpp @ 42]:

    https://docs.microsoft.com/en-us/windows-hardware/drivers/debugger/

    The error code 0xC0000005 indicates an access violation, maybe you dereferenced an uninitialized pointer, maybe you accessed a piece of memory after freeing it, but since it happens after a few hours, I would guess at a buffer overflow.


    • Edited by Brian Reichle Friday, June 16, 2017 2:27 PM
    • Marked as answer by Selva VS Monday, June 19, 2017 4:41 AM
    Friday, June 16, 2017 12:28 PM
  • Yeah.. you are rite, there was a stack overflow issue due to a recursion function. I replaced that logic to something else. But still access violation exception occurs. 

    and i found in ntsd as u said.. I got the line number.. Here s the screen shot..

                    while(true)
    		{
    			bool si = logObj->childList->empty(); // Line Number 1250
    			if(si == true)
    			{
    				logObj = logObj->parent;
    				if(logObj == NULL)
    				{
    					break;
    				}
    				continue;
    			}
    
    			if(logObj != NULL)
    			{
    				(logObj->curChild)++;
    				logObj = logObj->childList->at(logObj->curChild-1);
    				if(logObj->pCount == 1)
    				{
    					continue;
    				}
    				
    				paddingSize++;
    				if(paddingSize <= 199)
    				{
    					fString += this->GetCallTree(paddingSize,logObj,pInfo,&tempConfObj,ExternalDetails,fastDBInfoMap,index);
    				}
    				if(logObj->curChild == logObj->childList->size())
    				{
    					paddingSize--;
    				}
    			}
    			else if(logObj == NULL)
    			{
    				break;
    			}
    		}

    In the code block i have specified the line number 1250 in comment. But this access violation exception happens after looping some times(not at the initial loop).  The logObj is swapped object from profiler dll.. This code block is in another dll. I am deleting the logObj only after this function exists. 

    Any more info from the ntsd screen shot attached??

    Kindly dont change this topic to any other forums , Brian might miss this thread.


    • Edited by Selva VS Saturday, June 17, 2017 5:39 PM clarity
    Saturday, June 17, 2017 5:24 PM
  • My guess is that the access violation happened while trying to dereference childList. Maybe childList was null, uninitialized or freed ... or maybe logObj was pointing to an address in a committed page but was not actually a valid object, in which case its 'childList' field would contain random garbage.
    • Marked as answer by Selva VS Monday, June 19, 2017 4:41 AM
    Sunday, June 18, 2017 4:02 AM
  • Hi Brian,

    I fixed that access violation crash.! Thanks for ur suggestions. 

    But again when i am handling with too much of requests my w3wp.exe crashes. Now the module name from event viewer is helpless i guess.. 

    Faulting application name: w3wp.exe, version: 8.5.9600.16384, time stamp: 0x5215df96
    Faulting module name: ntdll.dll, version: 6.3.9600.17031, time stamp: 0x530895af
    Exception code: 0xc0000374
    Fault offset: 0x00000000000f8c9c
    Faulting process id: 0xff88
    Faulting application start time: 0x01d2e8c0c57b9b6c
    Faulting application path: c:\windows\system32\inetsrv\w3wp.exe
    Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
    Report Id: b68fd4f4-54b5-11e7-826e-b083feb425f0
    Faulting package full name: 
    Faulting package-relative application ID: 

    Any clue??? 

    Monday, June 19, 2017 6:44 AM
  • 0xc0000374 indicates heap corruption. So something, somewhere, probably allocated a piece of memory, wrote too much and overwrote some heap book-keeping information. The actual act of corrupting the heap may have happened long before you got the exception.

    https://blogs.msdn.microsoft.com/webdav_101/2010/06/22/detecting-heap-corruption-using-gflags-and-dumps/

    Monday, June 19, 2017 10:40 AM