none
How to access variable values in Crash Dump with Heap Information? RRS feed

  • Question

  • Hello,

    I'm investigating WPF/winforms app crash. User was able to save dump file via Task Manager.

    I can load dump file in VS2013, click Debug with Mixed and it does point to the line of code where crash happens ( I assume that's why it points there).

    On Minidump File Summary page it says "Heap Information Present".

    The msdn site says that "If the dump was saved with heap information... you can examine the stacks, threads, and variable values of the app at the time the dump occurred."

    https://msdn.microsoft.com/en-us/library/d5zhxt22.aspx

    But when I go to Immediate Window and type a variable name that's in scope on that line, I get a message:

    "Cannot obtain value of local or argument 'currentPage' as it is not available at this instruction pointer, possibly because it has been optimized away." 

    I understand it could've been 'optimized away' but I can't see any of the variables that should be in scope including "this".

    How can I see what was the value of the variable from Minidump file with Heap info?

    Thank you!


    Isolda

    Monday, March 2, 2015 8:40 PM

Answers

  • As Steve mentioned you will need the dso DumpStackObjects) command as you are trying to located the value of an object that was passed to a method. However, you will first need to find and switch to the correct thread before a DSO command will do anything as it only pulls up stack objects for the current thread's stack. Alternatively you can call this "~*e!clrstack" which shows all managed stacks across all threads OR "~*e!dso" which dumps the stack objects across all threads. This can produce a lot of output depending on how busy your app was at the time of the error, more threads and deeper call stacks = more output.

    Once you get the interested thread and call stack location you should be able to see the variable of interest if it is a primitive type (like string or int). Otherwise you will need to get the address and do a !do <address> (do=dumpobject), and you can continue the !do command on nested objects until you get what you need.

    Here is another link with help on SOS commands.


    -Igor

    • Marked as answer by Isolda Thursday, March 12, 2015 7:10 PM
    Thursday, March 12, 2015 6:21 PM

All replies

  • I have never used Visual Studio before to analyze a crash dump so I can't speak to that. I generally use WinDbg (win debug) to analyze crash dumps and pull out variables from the stack or heap and see what the state was at the time of the crash. It is not too bad to try to do using WinDbg, here is a handy cheat sheet link if you want to give it a try.

    If you do not have it installed already you can download it as a part of the Debugging tools for windows.

    Good luck!


    -Igor

    Monday, March 2, 2015 10:34 PM
  • Thanks for reply, but not getting anywhere with WinDbg... probably some path config problem but it wouldn't display my locals. Perhaps you have an advice.

    So, opened dump file, set symbol path to the folder with matching .pdbs.

    Ran !itoldyouso <myExe> and got MATCH back.

    Ran kn 2000 to get stack trace with frame numbers.

    In call stack found a line that resembles the name of my code where exception happens.

    Ran .frame <frame# from above line) to load that frame.

    Ran dv to get local variables and got an error back:

    "Unable to enumerate locals, Win32 error 0n318
    Private symbols (symbols.pri) are required for locals."

    Any idea what's missing? Might be the same reason VS does not display the values. May be app had to be built with some specific build parameter that creates private symbols?


    Isolda


    • Edited by Isolda Friday, March 6, 2015 7:01 PM
    Friday, March 6, 2015 5:40 PM
  • Hello Isolda,

    For debug a dumb file, the Windbg should be proper, however, according to the blog, we need to load the symbol firstly:

    http://blogs.msdn.com/b/rihamselim/archive/2012/02/25/windbg-debugging-2.aspx

    Here is a discussion taking about it:

    https://social.technet.microsoft.com/Forums/en-US/368b18d0-eb93-4b33-92b6-3a58786f0add/how-to-use-pdb-files-and-source-code-to-inspect-code-and-variables-while-using-windbg?forum=windbg

    Regards.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, March 10, 2015 8:50 AM
    Moderator
  • Regarding VS, if you've loaded the correct symbols and locals aren't resolving then it's probably correct that they've been optimized away.

    Windbg is a native debugger, so it's built-in commands are native only ('dv').

    All managed commands will be via the SOS extension. To get a local (stack) variable, use !sos.dso (DumpStackObjects)

    For a contrived sample:

            private void Button_Click(object sender, RoutedEventArgs e)
            {
                int x = 10;
                int y = 100;
                MessageBox.Show((x * y).ToString());
            }

    0:000> .frame 0b
    0b 006fe614 6403cc4a WpfApplication2!WpfApplication2.MainWindow.Button_Click(System.Object, System.Windows.RoutedEventArgs)+0x27

    0:000> !loadby sos clr

    0:000> !dso
    OS Thread Id: 0x2451c (0)
    ESP/REG  Object   Name
    006FE598 02451228 System.String   
    006FE59C 02562320 System.String    1000
    006FE5DC 02562320 System.String    1000
    006FE5E0 0249a734 System.Windows.Controls.Button
    006FE5E8 02562308 System.Windows.RoutedEventArgs
    006FE604 02451228 System.String   
    006FE61C 02562308 System.Windows.RoutedEventArgs

    ...

    No x, or y. And they don't appear to have been cached into registers

    0:000> .frame /c /r 0b
    0b 006fe614 6403cc4a WpfApplication2!WpfApplication2.MainWindow.Button_Click(System.Object, System.Windows.RoutedEventArgs)+0x27
    eax=00000001 ebx=0249a734 ecx=00000000 edx=00000000 esi=006fe674 edi=02562308
    eip=02318a57 esp=006fe610 ebp=006fe614 iopl=0         nv up ei pl nz na pe nc
    cs=0023  ss=002b  ds=002b  es=002b  fs=0053  gs=002b             efl=00000206
    WpfApplication2!WpfApplication2.MainWindow.Button_Click(System.Object, System.Windows.RoutedEventArgs)+0x27:
    02318a57 8be5            mov     esp,ebp

    You may well have to resort to interpreting disassembly -- in this we just hard-coded in the 1000

    02318a39 c745fce8030000  mov     dword ptr [ebp-4],3E8h
    02318a40 e8db1ef570      call    mscorlib_ni!System.Globalization.NumberFormatInfo.get_CurrentInfo() (7326a920)
    02318a45 50              push    eax
    02318a46 8b4dfc          mov     ecx,dword ptr [ebp-4]
    02318a49 33d2            xor     edx,edx
    02318a4b e878cada71      call    clr!COMNumber::FormatInt32 (740c54c8)
    02318a50 8bc8            mov     ecx,eax
    02318a52 e8e52a3561      call    PresentationFramework_ni!System.Windows.MessageBox.Show(System.String) (6366b53c)
    02318a57 8be5            mov     esp,ebp

    So there were indeed, no x and y local.

    And that was the JITTER, not the C# compiler, verified by looking at the IL which contains locals

    0:000> !ip2md @eip
    MethodDesc:   008282d4
    Method Name:  WpfApplication2.MainWindow.Button_Click(System.Object, System.Windows.RoutedEventArgs)
    Class:        02320260
    MethodTable:  00828344
    mdToken:      06000005
    Module:       00822edc
    IsJitted:     yes
    CodeAddr:     02318a30
    Transparency: Critical

    0:000> !sos.dumpil 008282d4
    ilAddr = 0057209c
    IL_0000: ldc.i4.s 10
    IL_0002: stloc.0
    IL_0003: ldc.i4.s 100
    IL_0005: stloc.1
    IL_0006: ldloc.0
    IL_0007: ldloc.1
    IL_0008: mul
    IL_0009: stloc.2
    IL_000a: ldloca.s VAR OR ARG 2
    IL_000c: call System.Int32::ToString
    IL_0011: call System.Windows.MessageBox::Show
    IL_0016: pop
    IL_0017: ret

    Tuesday, March 10, 2015 5:25 PM
  • Ok, I give up. Tried SOS and got more errors, at this point I don't think it's worth the effort. But thank you for your help.

    0:000> .load C:\Windows\Microsoft.NET\Framework64\v4.0.30319\sos.dll
    0:000> !dso
    The version of SOS does not match the version of CLR you are debugging.  Please
    load the matching version of SOS for the version of CLR you are debugging.
    CLR Version: 4.0.30319.1026
    SOS Version: 4.0.30319.18444
    Failed to load data access DLL, 0x80004005
    Verify that 1) you have a recent build of the debugger (6.2.14 or newer)
                2) the file mscordacwks.dll that matches your version of clr.dll is
                    in the version directory or on the symbol path
                3) or, if you are debugging a dump file, verify that the file
                    mscordacwks_<arch>_<arch>_<version>.dll is on your symbol path.
                4) you are debugging on supported cross platform architecture as
                    the dump file. For example, an ARM dump file must be debugged
                    on an X86 or an ARM machine; an AMD64 dump file must be
                    debugged on an AMD64 machine.

    You can also run the debugger command .cordll to control the debugger's
    load of mscordacwks.dll.  .cordll -ve -u -l will do a verbose reload.
    If that succeeds, the SOS command should work on retry.

    If you are debugging a minidump, you need to make sure that your executable
    path is pointing to clr.dll as well.
    0:000> .cordll -ve -u -l
    CLRDLL: C:\Windows\Microsoft.NET\Framework64\v4.0.30319\mscordacwks.dll:4.0.30319.18444 f:8
    doesn't match desired version 4.0.30319.1026 f:8
    CLRDLL: Unable to find '' on the path
    Cannot Automatically load SOS
    CLRDLL: Loaded DLL c:\websymbols\mscordacwks_AMD64_AMD64_4.0.30319.1026.dll\53B4FB32965000\mscordacwks_AMD64_AMD64_4.0.30319.1026.dll
    CLR DLL status: Loaded DLL c:\websymbols\mscordacwks_AMD64_AMD64_4.0.30319.1026.dll\53B4FB32965000\mscordacwks_AMD64_AMD64_4.0.30319.1026.dll


    Isolda

    Wednesday, March 11, 2015 8:11 PM
  • That's a 4.0 dump - 'CLR Version: 4.0.30319.1026'

    And a 4.5 SOS you loaded from your system. You'll need the SOS from the crash dump system. Or you could try PSSCOR4:

    http://www.microsoft.com/en-us/download/details.aspx?id=21255

    It should work on a 4.0 dump (there is no PSSCOR45...)

    You need to match the SOS version to the CLR version used in the dump...


    Update: If you're getting sos.dll from the target system, grab the corresponding mscordacwks.dll just in case - http://blogs.msdn.com/b/dougste/archive/2009/02/18/failed-to-load-data-access-dll-0x80004005-or-what-is-mscordacwks-dll.aspx
    Wednesday, March 11, 2015 11:30 PM
  • I probably should point out that the best way to determine 4.0 or 4.5, beyond memorizing the minor build versions, is to use lmvm and check the build branch (it's been in the version resources for quite a while now). E.g:

    0:000> lmvm clr
    start    end        module name
    700e0000 7077b000   clr        (deferred)            
        Image path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
        Image name: clr.dll
        Timestamp:        Wed Sep 11 17:30:37 2013 (52310B2D)
        CheckSum:         0069DB68
        ImageSize:        0069B000
        File version:     4.0.30319.18408
        Product version:  4.0.30319.18408
        File flags:       8 (Mask 3F) Private
        File OS:          4 Unknown Win32
        File type:        2.0 Dll
        File date:        00000000.00000000
        Translations:     0409.04b0
        CompanyName:      Microsoft Corporation
        ProductName:      Microsoft® .NET Framework
        InternalName:     clr.dll
        OriginalFilename: clr.dll
        ProductVersion:   4.0.30319.18408
        FileVersion:      4.0.30319.18408 built by: FX451RTMGREL
        PrivateBuild:     DDBLD128
        FileDescription:  Microsoft .NET Runtime Common Language Runtime - WorkStation
        LegalCopyright:   © Microsoft Corporation.  All rights reserved.
        Comments:         Flavor=Retail

    So, your SOS is actually a 4.51 branch since it's a little bit later than this example.

    Thursday, March 12, 2015 12:03 AM
  • To load the proper SOS extension that matches your CLR version you can do this:

    .loadby sos clr

    As was pointed out by Steve Horne you could try to do this on the machine where the crash occurred so the correct .net framework and matching sos dll are available. Otherwise ensure that the machine where you are analyzing the dump from has the same .net framework version installed as the machine where the crash occurred. If not you will have to copy the correct sos.dll and point to it in your .load command.


    To analyze a crash dump you can then use this command:

    !analyze -v;r;kv;lmtn;.logclose;q

    Finally to see all of your thread's call stacks so you can figure out what variables are of interest you can do this:

    ~*e!clrstack

    Hope this gets you a bit further.


    -Igor


    • Edited by IWolbers Thursday, March 12, 2015 12:22 AM updated answer based on target machine
    Thursday, March 12, 2015 12:14 AM
  • Ok, got sos.dll from user's machine and it worked, but I think I got more than I bargained for... how do I read this??? Can I get the value of "currentPage" and count of "ItemSearchResultPanelViewModelList"?

    0:000> !sos.dumpil 000007fee5b53fb8
    ilAddr = 000007fee5bda054
    .try
    {
      IL_0000: ldarg.0
      IL_0001: ldarg.1
      IL_0002: isinst <unknown token type 1b000000>
      IL_0007: stfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemDataArray
      IL_000c: ldarg.0
      IL_000d: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemDataArray
      IL_0012: brfalse.s IL_001e
      IL_0014: ldarg.0
      IL_0015: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemDataArray
      IL_001a: ldlen
      IL_001b: conv.i4
      IL_001c: br.s IL_001f
      IL_001e: ldc.i4.0
      IL_001f: stloc.0
      IL_0020: ldarg.0
      IL_0021: call MyApp.MyNamespace.WPF.ViewModels.ItemSearch::setCountTextDataLabel
      IL_0026: ldloc.0
      IL_0027: brtrue.s IL_002e
      IL_0029: leave IL_0142
      IL_002e: ldc.i4.0
      IL_002f: stloc.1
      IL_0030: ldarg.0
      IL_0031: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemDataArray
      IL_0036: stloc.s VAR OR ARG 7
      IL_0038: ldc.i4.0
      IL_0039: stloc.s VAR OR ARG 8
      IL_003b: br.s IL_00a6
      IL_003d: ldnull
      IL_003e: stloc.3
      IL_003f: newobj <>c__DisplayClass7::.ctor
      IL_0044: stloc.s VAR OR ARG 4
      IL_0046: ldloc.s VAR OR ARG 4
      IL_0048: ldloc.s VAR OR ARG 7
      IL_004a: ldloc.s VAR OR ARG 8
      IL_004c: ldelem.ref
      IL_004d: stfld <>c__DisplayClass7::ItemData
      IL_0052: ldarg.0
      IL_0053: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemSearchResultPanelViewModelList
      IL_0058: ldloc.3
      IL_0059: brtrue.s IL_0069
      IL_005b: ldloc.s VAR OR ARG 4
      IL_005d: ldftn <>c__DisplayClass7::<LoadData>b__3
      IL_0063: newobj class [mscorlib]System.Func`2<class MyApp::.ctor
      IL_0068: stloc.3
      IL_0069: ldloc.3
      IL_006a: call <unknown token type 2b000000>
      IL_006f: call <unknown token type 2b000000>
      IL_0074: stloc.2
      IL_0075: ldloc.2
      IL_0076: brtrue.s IL_0093
      IL_0078: ldarg.0
      IL_0079: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemSearchResultPanelViewModelList
      IL_007e: ldloc.s VAR OR ARG 4
      IL_0080: ldfld <>c__DisplayClass7::ItemData
      IL_0085: newobj MyApp.MyNamespace.WPF.ViewModels.ItemSearch::.ctor
      IL_008a: callvirt class [mscorlib]System.Collections.ObjectModel.Col::Add
      IL_008f: ldc.i4.1
      IL_0090: stloc.1
      IL_0091: br.s IL_00a0
      IL_0093: ldloc.2
      IL_0094: ldloc.s VAR OR ARG 4
      IL_0096: ldfld <>c__DisplayClass7::ItemData
      IL_009b: callvirt MyApp.MyNamespace.WPF.ViewModels.ItemSearch::set_ItemDataInfo
      IL_00a0: ldloc.s VAR OR ARG 8
      IL_00a2: ldc.i4.1
      IL_00a3: add
      IL_00a4: stloc.s VAR OR ARG 8
      IL_00a6: ldloc.s VAR OR ARG 8
      IL_00a8: ldloc.s VAR OR ARG 7
      IL_00aa: ldlen
      IL_00ab: conv.i4
      IL_00ac: blt.s IL_013d
      IL_00ae: ldarg.0
      IL_00af: ldloca.s VAR OR ARG 1
      IL_00b1: call MyApp.MyNamespace.WPF.ViewModels.ItemSearch::removeItemSearchPanelViewModel
      IL_00b6: ldloc.1
      IL_00b7: brtrue.s IL_00e9
      IL_00b9: ldarg.0
      IL_00ba: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_ItemSearchResultPanelViewModelList
      IL_00bf: ldsfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::CS$<>9__CachedAnonymousMethodDelegate5
      IL_00c4: brtrue.s IL_00d7
      IL_00c6: ldnull
      IL_00c7: ldftn MyApp.MyNamespace.WPF.ViewModels.ItemSearch::<LoadData>b__4
      IL_00cd: newobj class [mscorlib]System.Func`2<class MyApp::.ctor
      IL_00d2: stsfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::CS$<>9__CachedAnonymousMethodDelegate5
      IL_00d7: ldsfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::CS$<>9__CachedAnonymousMethodDelegate5
      IL_00dc: call <unknown token type 2b000000>
      IL_00e1: call <unknown token type 2b000000>
      IL_00e6: ldc.i4.0
      IL_00e7: ble.s IL_0133
      IL_00e9: ldarg.0
      IL_00ea: ldfld MyApp.MyNamespace.WPF.ViewModels.ItemSearch::_currentPage
      IL_00ef: stloc.s VAR OR ARG 5
      IL_00f1: ldarg.0
      IL_00f2: ldarg.0
      IL_00f3: box MyApp.MyNamespace.WPF.ViewModels.ItemSearch
      IL_00f8: ldtoken MyApp.MyNamespace.WPF.ViewModels.ItemSearch
      IL_00fd: call System.Type::GetTypeFromHandle
      IL_0102: call System.Linq.Expressions.Expression::Constant
      IL_0107: ldtoken MyApp.MyNamespace.WPF.ViewModels.ItemSearch::get_ItemSearchResultPanelViewModelList
      IL_010c: call System.Reflection.MethodBase::GetMethodFromHandle
      IL_0111: castclass System.Reflection.MethodInfo
      IL_0116: call System.Linq.Expressions.Expression::Property
      IL_011b: ldc.i4.0
      IL_011c: newarr System.Linq.Expressions.ParameterExpression
      IL_0121: call <unknown token type 2b000000>
      IL_0126: call <unknown token type 2b000000>
      IL_012b: ldarg.0
      IL_012c: ldloc.s VAR OR ARG 5
      IL_012e: call MyApp.MyNamespace.WPF.ViewModels.ItemSearch::set_CurrentPage
      IL_0133: leave.s IL_0142
    } // end .try
    .catch
    {
      IL_0135: stloc.s VAR OR ARG 6
      IL_0137: ldloc.s VAR OR ARG 6
      IL_0139: ldc.i4.2
      IL_013a: call MyApp.MyNamespace.Utilities.ModalErrorDialog::ShowExceptionModal
      IL_013f: pop
      IL_0140: leave.s IL_0142
    } // end .catch
    IL_0142: ret

     

    Isolda


    • Edited by Isolda Thursday, March 12, 2015 3:41 PM
    Thursday, March 12, 2015 3:38 PM
  • My dump of IL was to demonstrate it was the JIT compiler, not the C# compiler optimizing away locals.

    !sos.dso, then disassembly inspection if required to find the value of stack variables.

    Check the registers too, particularly since this was a 64 bit dump per your IL address

    Thursday, March 12, 2015 5:38 PM
  • As Steve mentioned you will need the dso DumpStackObjects) command as you are trying to located the value of an object that was passed to a method. However, you will first need to find and switch to the correct thread before a DSO command will do anything as it only pulls up stack objects for the current thread's stack. Alternatively you can call this "~*e!clrstack" which shows all managed stacks across all threads OR "~*e!dso" which dumps the stack objects across all threads. This can produce a lot of output depending on how busy your app was at the time of the error, more threads and deeper call stacks = more output.

    Once you get the interested thread and call stack location you should be able to see the variable of interest if it is a primitive type (like string or int). Otherwise you will need to get the address and do a !do <address> (do=dumpobject), and you can continue the !do command on nested objects until you get what you need.

    Here is another link with help on SOS commands.


    -Igor

    • Marked as answer by Isolda Thursday, March 12, 2015 7:10 PM
    Thursday, March 12, 2015 6:21 PM
  • OMG! I got it. Ran ~*e!dso, found objects I need, did !do on them and got the values.

    Fields:
                  MT    Field   Offset                 Type VT     Attr            Value Name

    000007fee7f0d610  400064d       84       System.Boolean  1 instance                0 _showDataFilter
    000007fee7f0c7e0  400064e       54         System.Int32  1 instance               10 _itemsPerPage
    000007fee7f0c7e0  400064f       80         System.Int32  1 instance                0 _currentPage

    Thought I'd never  get there. Seriously.

    Thank u very much.


    Isolda

    Thursday, March 12, 2015 7:09 PM