none
Software crash: faulting module mscorwks.dll, version 1.1.4322.2379

    Question

  • Hi,

    I am working on software which crashes in 5-10mins after starting it. The software is built with .Net framework 1.1 as target. .Net framework 2.0, 3.0, 3.5 is also installed on the machine. I am logging exceptions in the Windows event viewer in the catch block at every possible location in the code. I have subscribed to UnhandledException also. But, when the software crashes, no exception is caught.

    The software crashes only on Windows Server 2003 and works fine on Windows XP.

    Sometimes, error event is logged by .Net Runtime as follow:

    ----------------------------------------------
    Event Type:    Information
    Event Source:    Application Error
    Event Category:    (100)
    Event ID:    1004
    Date:        11/3/2009
    Time:        11:23:53 AM
    User:        N/A
    Computer:    <Machine Name>
    Description:
    Faulting application <Software Name>, version <Software Version>, faulting module mscorwks.dll, version 1.1.4322.2379, fault address 0x00007f2c.
    ----------------------------------------------


    It was looking like there is some .Net Runtime error/bug which is causing the problem. So, I rebuilt the software with .Net framework 2.0 as target. Still, software crashes and Runtime logs the same kind of exception.

    ----------------------------------------------
    Event Type:    Error
    Event Source:    .NET Runtime
    Event Category:    None
    Event ID:    1023
    Date:        11/4/2009
    Time:        10:12:45 PM
    User:        N/A
    Computer:    <Machine Name>
    Description:
    .NET Runtime version 2.0.50727.3053 - Fatal Execution Engine Error (7A097706) (80131506)


    Event Type:    Error
    Event Source:    .NET Runtime 2.0 Error Reporting
    Event Category:    None
    Event ID:    1000
    Date:        11/4/2009
    Time:        10:13:12 PM
    User:        N/A
    Computer:    <Machine Name>
    Description:
    Faulting application <Software Name>, version <Software Version>, stamp 4af1822b, faulting module mscorwks.dll, version 2.0.50727.3053, stamp 4889dc18, debug? 0, fault address 0x0000c214
    ----------------------------------------------

    I also tried to get the dumps using WinDbg and DebugDiag, but both WinDbg and DebugDiag do not get a chance to dump when software crashes. DebugDiag logged some First Chance Exceptions... some common stack traces are as follows:

    1)
     
    Function     Arg 1     Arg 2     Arg 3   Source
    mscorwks!gc_heap::update_brick_table+86     020600c4     000018c0     b98dcd28   
    mscorwks!gc_heap::plan_phase+3cb     00000000     00000000     00000000   
    mscorwks!gc_heap::gc1+9c     00000000     00000000     00000000   
    mscorwks!gc_heap::garbage_collect+1bf     00000000     00000000     00000000   
    mscorwks!GCHeap::GarbageCollectGeneration+11b     00000000     00000000     00000028   
    mscorwks!gc_heap::allocate_more_space+13a     0015dcc0     00000028     00000000   
    mscorwks!GCHeap::Alloc+5f     0015dcc0     00000028     00000002   
    mscorwks!Alloc+3a     00000028     00000000     00040000   
    mscorwks!AllocateArrayEx+161     00f91022     0012eec4     00000001   
    mscorwks!JIT_NewArr1+bb     021394f0     00000002     038e58bc   
    0x038e5963     014a0190     0000000b     01492f98   
    0x0563761e     0012f238     056ab7a8     0012f3c8   
    0x056370c3     0012f3c8     014dcb64     01ce62a4   
    0x056aaf61     01ce62a4     014dcb64     01ce62a4   
    system_windows_forms_7b810000+726b1     00000001     00100000     00000000   
    system_windows_forms_7b810000+134b0     7b822a5b     7b822a3c     00000000   
    system_windows_forms_7b810000+13036     000a02f6     00000202     00000000   
    user32!InternalCallWinProc+28     0582997e     000a02f6     00000202   
    user32!UserCallWinProcCheckWow+151     00000000     0582997e     000a02f6   
    user32!DispatchMessageWorker+327     0012f5b0     00000000     0012f5e4   
    user32!DispatchMessageW+f     0012f5b0     000a02f6     0012f5b0   
    0x038604ca     00000000     ffffffff     016078d0   
    system_windows_forms_7b810000+1dbea     0143efc8     015cc644     01607890   
    system_windows_forms_7b810000+1d905     0143efc8     00000000     015cc644   
    system_windows_forms_7b810000+418e4     0012f7b4     7923e0bc     0012f6f0   
    mscorwks!CallDescrWorker+30     0012f6f0     00000000     0012f6c8   
    mscorwks!MethodDesc::CallDescr+1b8     00cc5a3b     00163e90     0012f7e0   
    mscorwks!MethodDesc::CallDescr+4f     00cc5a3b     00163e90     00705a42   
    mscorwks!MethodDesc::Call+97     0012f928     00000000     0015dc88   
    mscorwks!ClassLoader::CanAccess+207     00cc5a40     00000001     00000000   
    mscorwks!ClassLoader::ExecuteMainMethod+49d     00163e90     00000000     003a1394   
    mscorwks!Assembly::ExecuteMainMethod+21     00000000     0012fd70     00000000   
    mscorwks!SystemDomain::ExecuteMainMethod+421     00000000     00000001     0012ffe0   
    mscorwks!ExecuteEXE+1ce     8013141b     00000000     00000000   
    mscorwks!_CorExeMain+59     00000000     791b0000     0012fff0   
    mscoree+11b5f     00000000     00000000     7ffda000   
    kernel32!BaseProcessStart+23     79011b2b     00000000     78746341  


    2)


    Entry point   mscorwks!Thread::intermediateThreadProc
    Create time   11/3/2006 6:10:13 PM
    Time spent in user mode   0 Days 0:0:0.421
    Time spent in kernel mode   0 Days 0:0:0.62


    Function     Arg 1     Arg 2     Arg 3   Source
    mscorwks!Object::GetAppDomain+2     77e670b2     00000000     00000000   
    mscorwks!CallFinalizer+225     01c23fac     00000000     00000000   
    mscorwks!GCHeap::FinalizerThreadStart+c2     00000000     80911708     7ffdd000   
    mscorwks!Thread::intermediateThreadProc+44     001626c0     00000000     00000000   
    kernel32!BaseThreadStart+34     791ced9f     001626c0     00000000   



    Thanks in Adavance.

    Regards,
    Bajare

    Wednesday, November 11, 2009 9:27 AM

Answers

  • Hello Bajare

    According to the first chance stack trace, the problem seems related to a GC heap corruption. Please run !verifyheap command of sos.dll , it will walk through the GC heap to verify if any object on GC heap is corrupt. Once you are sure that GC heap is corrupt, our next action would be to enable GCStress by running the following commands and see if we can catch the culprit.

    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v HeapVerify  /t REG_DWORD  /d 1
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v StressLog  /t REG_DWORD  /d 1
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v GCStress  /t REG_DWORD  /d 3
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v FastGcStress  /t REG_DWORD  /d 2

    Also please enable full page heap using:
    gflags -p /enable <application name>/full

    Please capture full memory dump of first chance exception by using adplus with -fullonfirst option. I'd like to help you dig into the dump. After you get the dump, please let me know your email address by sending a mail to jialge@microsoft.com. Then I will create a file transfer workspace where you can upload your dump file. The dump will be kept confidential.

    I also wonder whether you .NET application replies on any native module like a COM dll?

    Do you use handles? If yes, are you using SafeHandle? http://blogs.msdn.com/bclteam/archive/2005/03/16/396900.aspx
    If the GC does run while in the MethodThatSpansGCs code, then Foo will be reported as not live, and will be finalized.  The finalizer thread (of which there is currently one, but we may use multiple in the future) will then run the finalizers for all finalizable, dead objects.  It could run Foo’s Finalize method, which closes the stream.  After this happens, MethodThatSpansGCs may not work right at all, since it has just been called on a closed stream.  Notice that to observe this problem, you must have the GC collect objects at a certain point, and you must have the finalizer thread run your finalizer within this window as well.  This is rare in practice, but will show up with sufficient stress testing.  (Internally we’ve added something called GCStress to the CLR to help find some of these types of problems.  While primarily aimed at finding problems in our “manually managed” C++ code within the CLR itself, it can be useful to help trigger GC’s in unexpected places.)


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Thursday, November 12, 2009 7:37 AM

All replies

  • Hello Bajare

    According to the first chance stack trace, the problem seems related to a GC heap corruption. Please run !verifyheap command of sos.dll , it will walk through the GC heap to verify if any object on GC heap is corrupt. Once you are sure that GC heap is corrupt, our next action would be to enable GCStress by running the following commands and see if we can catch the culprit.

    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v HeapVerify  /t REG_DWORD  /d 1
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v StressLog  /t REG_DWORD  /d 1
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v GCStress  /t REG_DWORD  /d 3
    reg.exe add "HKLM\SOFTWARE\Microsoft\.NETFramework" /f  /v FastGcStress  /t REG_DWORD  /d 2

    Also please enable full page heap using:
    gflags -p /enable <application name>/full

    Please capture full memory dump of first chance exception by using adplus with -fullonfirst option. I'd like to help you dig into the dump. After you get the dump, please let me know your email address by sending a mail to jialge@microsoft.com. Then I will create a file transfer workspace where you can upload your dump file. The dump will be kept confidential.

    I also wonder whether you .NET application replies on any native module like a COM dll?

    Do you use handles? If yes, are you using SafeHandle? http://blogs.msdn.com/bclteam/archive/2005/03/16/396900.aspx
    If the GC does run while in the MethodThatSpansGCs code, then Foo will be reported as not live, and will be finalized.  The finalizer thread (of which there is currently one, but we may use multiple in the future) will then run the finalizers for all finalizable, dead objects.  It could run Foo’s Finalize method, which closes the stream.  After this happens, MethodThatSpansGCs may not work right at all, since it has just been called on a closed stream.  Notice that to observe this problem, you must have the GC collect objects at a certain point, and you must have the finalizer thread run your finalizer within this window as well.  This is rare in practice, but will show up with sufficient stress testing.  (Internally we’ve added something called GCStress to the CLR to help find some of these types of problems.  While primarily aimed at finding problems in our “manually managed” C++ code within the CLR itself, it can be useful to help trigger GC’s in unexpected places.)


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Thursday, November 12, 2009 7:37 AM
  • Hello Bajare

    How are you? Could you please check out my last reply? The requested information is very important to dig into the problem.

    I look forward to your updates.


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Monday, November 16, 2009 4:57 AM
  • Hi Jialiang,

    THANKS for your reply!!!

    Your reply has helped me to shorten the scope to find the culprit code. But, I don't have permission to share the dump files. I am using some unmanaged code [IntPtr] in the managed code. I have a "varbinary" column in a table in the sql 2k5 and I am reading values stored in it through a stored procedure.

    Its seems to be a heap corruption issue.

    Before entering the RegKeys mentioned in your reply:

    !VERIFYHEAP:
    ----------------
    0:000> !verifyheap
    Loaded Son of Strike data table version 5 from "C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
    VerifyHeap will only produce output if there are errors in the heap
    Bad MethodTable for Obj at 0x1b73598
    Last good object: 0x1b7358c


    CLR Stack:
    -------------
    0:000> !clrstack -p
    Thread 0
    ESP       EIP    
    0012ee68  7c82ed54 [FRAME: HelperMethodFrame]
    0012eea0  040f5aab [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.PrepareRecord(I4)
    0012eeac  040f5a04 [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.GetValue(I4)
    0012eeb8  040f56d5 [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.get_Item(String)
    0012eec0  05558e0c [DEFAULT] [hasThis] ValueClass System.TimeSpan TrendPlot.RetrieveHistoricalData(Class TrendPlotData)
      at [+0x46c] [+0x34c]
        PARAM: int32 obVariableData: 21798572
    0012ef5c  05557ad1 [DEFAULT] [hasThis] Void TrendPlot.GetSampleData()
      at [+0x1d1] [+0xce]


    Call Stack:
    ------------
    0:000> kv
    ChildEBP RetAddr  Args to Child             
    0012e998 7c822034 77e5300a ffffffff e0004743 ntdll!KiFastSystemCallRet (FPO: [0,0,0])
    0012e99c 77e5300a ffffffff e0004743 00000000 ntdll!NtTerminateProcess+0xc (FPO: [2,0,0])
    0012ea90 77e5304d e0004743 77e8f3b0 ffffffff KERNEL32!_ExitProcess+0x63 (FPO: [SEH])
    0012eaa4 7925c810 e0004743 00000000 00000000 KERNEL32!ExitProcess+0x14
    0012eafc 791e6f8f 00000000 00000000 00000000 mscorwks!gc_heap::gc1+0x214 (FPO: [SEH])
    0012eb3c 791e7009 00000000 00000000 00000000 mscorwks!gc_heap::garbage_collect+0x1bf (FPO: [Non-Fpo])
    0012eb5c 791ea296 00000000 00000000 00000028 mscorwks!GCHeap::GarbageCollectGeneration+0x11b (FPO: [Non-Fpo])
    0012eb8c 791b3266 0014a6b0 00000028 00000000 mscorwks!gc_heap::allocate_more_space+0x13a (FPO: [Non-Fpo])
    0012edb0 791b3203 0014a6b0 00000028 00000002 mscorwks!GCHeap::Alloc+0x5f (FPO: [Non-Fpo])
    0012edc4 791bd8b0 00000028 00000000 00040000 mscorwks!Alloc+0x3a (FPO: [Non-Fpo])
    0012ee14 7922fd4d 00f91022 0012ee94 00000001 mscorwks!AllocateArrayEx+0x161 (FPO: [Non-Fpo])
    0012ee98 040f5aab 0226d23c 00000002 040f5a04 mscorwks!JIT_NewArr1+0xbb (FPO: [Non-Fpo])
    WARNING: Frame IP not in any known module. Following frames may be wrong.
    0012ef50 05557ad1 014c9eac 0000000b 014b9384 0x40f5aab
    0012f080 791b19b8 00000000 01d0e888 0012f2b8 0x5557ad1
    0012f088 01d0e888 0012f2b8 0616e111 00000000 mscorwks!JIT_Stelem_Ref+0x49 (FPO: [Non-Fpo])
    00000000 00000000 00000000 00000000 00000000 0x1d0e888


    After I entered regkeys mentioned in the above reply, the software becomes unresponsive when I opened the Trend plot. [Software exe launched through WinDbg]. I breaked into the debugger after some time.

    Verify Heap:

    0:008> !verifyheap
    VerifyHeap will only produce output if there are errors in the heap
    Bad MethodTable for Obj at 0x1401000
    object 2401810: bad member 166d7ec at 02401b78
    curr_object : 0x2401810 size = 0
    Last good object: 0x2401000
    SyncBlock 74 corrupted, points to invalid object 01720b30
    SyncBlock 79 corrupted, points to invalid object 0164e6b4
    SyncBlock 80 corrupted, points to invalid object 01720c5c

    CLR Stack:

    0:008> !clrstack -p
    Thread 8
    Not a managed thread.


    If you can help[/guide] me with this little information, it will be highly appreciated.

    Sorry for replying late.

    Regards,
    Bajare
    Wednesday, November 18, 2009 7:55 AM
  • Hello Bajare

    I'm glad to see that we are in the right direction.

    Please dump the call stack of all your threads:

    ~*kvn
    ~*e!clrstack

    From your posted stack traces, I do not see any code about .NET - native interop. How do you use unmanaged code [IntPtr] in the managed code? Could you please check or share the corresponding code?


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Wednesday, November 18, 2009 11:35 AM
  • Hi Jialiang,

    The commands you mentioned in the above reply don't work when sos.dll is loaded. Following are my observations during last 2 days. [System (OS) becomes unresponsive when I entered reg keys that you mentioned in your first reply. So, currently, I have deleted them from registry.]

    I get some access violation exceptions when I call some COM methods using interface. So, while starting software I use "sxd" command for "c0000005" [Access violation exception]. Generally the second chance exception looks like:

    --------------------------------------------------------------------------------------------------------
    (8ac.974): Access violation - code c0000005 (first chance)
    (8ac.db4): Access violation - code c0000005 (first chance)
    eax=7903dff8 ebx=00000000 ecx=7903ddf8 edx=00000001 esi=7c822028 edi=e0004743
    eip=7c82ed54 esp=0012e99c ebp=0012ea90 iopl=0         nv up ei pl zr na pe nc
    cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
    ntdll!KiFastSystemCallRet:
    7c82ed54 c3               ret
    --------------------------------------------------------------------------------------------------------

    CLR stack: (Almost all times, the return address[bold ] is same for command "r" and in the first line of "!clrstack" output.)

    --------------------------------------------------------------------------------------------------------
    0:000> !clrstack
    Thread 0
    ESP       EIP    
    0012ee68  7c82ed54 [FRAME: HelperMethodFrame]
    0012eea0  0430588b [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.PrepareRecord(I4)
    0012eeac  043057e4 [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.GetValue(I4)
    0012eeb8  043054b5 [DEFAULT] [hasThis] Object System.Data.SqlClient.SqlDataReader.get_Item(String)
    0012eec0  036f8981 [DEFAULT] [hasThis] ValueClass System.TimeSpan TrendPlot.CTrendPlotView.RetrieveHistoricalData(Class TrendPlot.CTrendPlotVariableData)
      at [+0x481] [+0x358]
    0012ef5c  036f7631 [DEFAULT] [hasThis] Void TrendPlot.GetSampleData()
      at [+0x1d1] [+0xce]
    0012f06c  036f70cb [DEFAULT] [hasThis] Void TrendPlot.TrendPlotView.GetSampleData()
      at [+0x13] [+0xc]
    0012f074  0370cd64 [DEFAULT] [hasThis] Void TrendPlot.TrendPlotView.UpdatePlotView(Object,ValueClass PlotsCommon.stPlotViewUpdateInfo)
      at [+0x3c] [+0x46]
    0012f0a0  0370b9b9 [DEFAULT] [hasThis] Void TrendPlot.TrendPlotView.MouseUpEvent(Class System.Windows.Forms.MouseEventArgs)
      at [+0x641] [+0x3fd]
    0012f2c0  0370b191 [DEFAULT] [hasThis] Void PlotsCommon.PlotContainer.PlotPanel_MouseUp(Object,Class System.Windows.Forms.MouseEventArgs)
      at [+0x39] [+0x18]
    0012f2f0  7b8826b1 [DEFAULT] [hasThis] Void System.Windows.Forms.Control.OnMouseUp(Class System.Windows.Forms.MouseEventArgs)
    0012f2fc  7b8850c3 [DEFAULT] [hasThis] Void System.Windows.Forms.Control.WmMouseUp(ByRef ValueClass System.Windows.Forms.Message,ValueClass

    System.Windows.Forms.MouseButtons,I4)
    0012f340  7b8234b0 [DEFAULT] [hasThis] Void System.Windows.Forms.Control.WndProc(ByRef ValueClass System.Windows.Forms.Message)
    0012f354  7b823036 [FRAME: InlinedCallFrame]
    0012f3a0  7b823036 [DEFAULT] [hasThis] Void System.Windows.Forms.ScrollableControl.WndProc(ByRef ValueClass System.Windows.Forms.Message)
    0012f3a4  7b822a5b [DEFAULT] [hasThis] Void System.Windows.Forms.Control/ControlNativeWindow.OnMessage(ByRef ValueClass System.Windows.Forms.Message)
    0012f3a8  7b822a3c [DEFAULT] [hasThis] Void System.Windows.Forms.Control/ControlNativeWindow.WndProc(ByRef ValueClass System.Windows.Forms.Message)
    0012f3b8  7b8228e0 [DEFAULT] [hasThis] I System.Windows.Forms.NativeWindow.Callback(I,I4,I,I)
    0012f54c  00c2573a [FRAME: NDirectMethodFrameStandalone] [DEFAULT] I System.Windows.Forms.UnsafeNativeMethods.DispatchMessageW(ByRef ValueClass MSG)
    0012f55c  7b82e07a [DEFAULT] [hasThis] Boolean

    System.Windows.Forms.Application/ComponentManager.System.Windows.Forms.UnsafeNativeMethods+IMsoComponentManager.FPushMessageLoop(I4,I4,I4)
    0012f5f4  7b82dbea [DEFAULT] [hasThis] Void System.Windows.Forms.Application/ThreadContext.RunMessageLoopInner(I4,Class

    System.Windows.Forms.ApplicationContext)
    0012f630  7b82d905 [DEFAULT] [hasThis] Void System.Windows.Forms.Application/ThreadContext.RunMessageLoop(I4,Class System.Windows.Forms.ApplicationContext)
    0012f65c  7b8518e4 [DEFAULT] Void System.Windows.Forms.Application.Run(Class System.Windows.Forms.Form)
    0012f66c  0123068d [DEFAULT] Void MainFrm.MainFrm.Main()
      at [+0x10d] [+0x79] c:\documents and settings\administrator\desktop\simulator\mainfrm\mainfrm.cs:147
    0012f9b0  7923c069 [FRAME: GCFrame]
    0012fa94  7923c069 [FRAME: GCFrame]
    --------------------------------------------------------------------------------------------------------


    Verify Heap:
    --------------------------------------------------------------------------------------------------------
    0:000> !verifyheap
    Loaded Son of Strike data table version 5 from "C:\WINDOWS\Microsoft.NET\Framework\v1.1.4322\mscorwks.dll"
    VerifyHeap will only produce output if there are errors in the heap
    object 145a8e4: bad member f34a8 at 0145a8fc
    curr_object : 0x145a8e4 size = 0
    Last good object: 0x145a824
    object 2402000: bad member f3438 at 02402368
    curr_object : 0x2402000 size = 0
    Last good object: 0x2401810
    --------------------------------------------------------------------------------------------------------

    Object at 0x145a824: is a string

    Object at 145a8e4: is System.Data.SqlClient.SqlConnection


    Object at 0x2401810: is Free Object. Size 2032(0x7f0) bytes

    Object at 0x2402000: is
    -----------
    Name: System.Object[]
    MethodTable 0x00ce209c
    EEClass 0x00ce2018
    Size 4096(0x1000) bytes
    GC Generation: 3
    Array: Rank 1, Type CLASS
    Element Type: System.Object
    Content: 1,020 items
    -----------

    [Generally, if there are 2 corruptions, then the second "Last Good Object" is Free object.]


    Finalize Queue: [Note that a lot of objects are Gen 2 objects]
    --------------------------------------------------------------------------------------------------------
    0:000> !finalizequeue
    SyncBlock to be cleaned up: 0
    ----------------------------------
    MTA interfaces to be released: 0
    Total STA interfaces to be released: 0
    ----------------------------------
    generation 0 has 24 finalizable objects (04246fb0->04247010)
    generation 1 has 280 finalizable objects (04246b50->04246fb0)
    generation 2 has 1752 finalizable objects (04244ff0->04246b50)
    Ready for finalization 24 objects (04247010->04247070)
    Statistics:
          MT    Count TotalSize Class Name
    7ba0e550        2       416 System.Windows.Forms.LinkLabel
    7ba1f7e4        1        16 System.Windows.Forms.ApplicationContext

                     <Deleted some output>

    7b9ed51c       25      2600 System.Windows.Forms.Label
    7ba17c44       71      4260 System.Windows.Forms.MenuItem
    7b9f3f8c      147      7644 System.Windows.Forms.Control/ControlNativeWindow
    7b5a52dc     1217     24340 System.Drawing.Bitmap
    Total 2078 objects

    --------------------------------------------------------------------------------------------------------

    Heap:
    --------------------------------------------------------------------------------------------------------
    0:000> !eeheap -gc
    generation 0 starts at 0x01b7b060
    generation 1 starts at 0x015f2034
    generation 2 starts at 0x01401000
     segment    begin allocated     size
    01400000 01401000  0217b03c 00d7a03c(14131260)
    Large object heap starts at 0x02401000
     segment    begin allocated     size
    02400000 02401000  02566948 0x00165948(1464648)
    Total Size  0xedf984(15595908)
    ------------------------------
    GC Heap Size  0xedf984(15595908)
    --------------------------------------------------------------------------------------------------------



    I am creating few large SortedList or ArrayList containing more than 4096 members. Generally, the last good object is a member of a large SortedList. or the corrupted heap resides between the 2 member objects of a large SortedList. Can you please, tell me how to find out who is corrupting the heap? I don't have much knowledge about what the register values signify?


    Thanks in Advance,
    Bajare
    • Edited by Bajare Friday, November 20, 2009 4:39 PM Updated
    Friday, November 20, 2009 4:18 PM
  • Hello Bajare

    > The commands you mentioned in the above reply don't work when sos.dll is loaded.

    Do you see any errors outputted in windbg.
    ~*kvn
    ~*e!clrstack

    Because your application uses .net v1.1.4322. I hope that you loaded the right sos verison.
    Without loading sos, can you print out the result of ~*kvn?

    The call-stack posted in your last reply shows that the thread calls into TrendPlot.CTrendPlotView.RetrieveHistoricalData which further calls into SqlDataReader to retrieve the data, and then you got the error. Are there any COM modules in this stack trace? I still do not see any code about .NET - native interop. How do you use unmanaged code [IntPtr] in the managed code? Could you please check or share the corresponding code?

    Regarding the bad object, please inspect these thinngs:
    1. What's wrong with the bad object?
    2. Are its member in the GC heap?
    3. Does it look like it was overwritten with junk (run "dd <addr>")
    4. Does it have a valid MethodTable? Class? Size?

    Additionally, please inspect the previous object:
    1. Could the previous object have overwritten the next one?
    2. Does the previous object's data extend beyond previous_obj_addr + previous_obj_size?
    3. Does the next object start before previous_obj_addr + previous_obj_size?

    Third, please inspect their roots:
    1. Look at roots for both corrupters and corruptees (run “!gcroot”)
    2. Are the objects still rooted?
    3. Who owns the roots?
    4. What functions have roots for these objects in their stack space?
    5. Do functions with roots do Pinvokes or interop?


    Here is more information about .NET managed heap corruption:
    It's usually caused by unmanaged code while filling the managed data structure passed to the unmanaged function. For example, Managed object is int16, unmanaged code fills it will int32. Managed code passes a byte array of size 30. Unmanaged code fills it with 35 bytes. They usually turns out to be improper use of P/Invokes or COM interop calls. They are much more difficult to track down than NT heap corruption because objects in the GC heap are constantly moving.

    Symptoms of GC heap corruption without GCStress:
    AV in GC code.

    Here is more information about GCStress:
    GCStress causes a gc to occur very often to shake out the bug. It's very slow - some tests may not run properly. Special logging allows us to determine the cause.


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Monday, November 23, 2009 7:36 AM
  • Hi Jialiang,

    I had to format machine due to some irreversible reasons, now I am trying to reproduce the problem again.

    I will reply as soon as I get something.

    Thanks,
    Bajare
    Friday, November 27, 2009 10:32 AM
  • Hello Bajare

    How are you. How is the issue going?


    Regards,
    Jialiang Ge
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Monday, December 07, 2009 3:45 AM