StackWalk64 vs. c++/clr

Question

Hi !

I wrote an AllocHook function to log all my allocations and get a stacktrace of the place where the allocations occur. I use StackWalk64() for the tracing the stack, SymInitialize(), SymCleanup(), SymGetLineFromAddrW64(), SymFromAddrW() for resolving the symbols. This works for native c++.

With c++/clr however, SymFromAddrW() fails for mixed compiled c++/clr code. Is there any way to make it work? Do I have to laod some other symbols manually?

System::Diagnostics::Stacktrace works, but it's managed code. When I try to use that in my allochook, I get OS Loader Lock Problems (because every allocation suddenly executes managed code and therefore all allocations in dllmain get illegal!).

So basically, my question is, how to get method/functionname, line number and file name of the information I get from StackWalk64 for frames that are not native?

I had a look at "Shared Source Common Language Infrastructure 2.0 Release" http://www.microsoft.com/downloads/details.aspx?FamilyId=8C09FD61-3F26-4555-AE17-3121B4F51D4D&displaylang=en but wasn't able to figure out how it works there. Only thing I saw were references to StackWalk64, so I assume calling gcnew StackTrace() actually calls StackWalk64() somewhere as well. But I didn't find where nor how that information is used to get method/functionnames, line numbers and file names.

Maybe sos.dll is related to the solution? But there doesn't seem to be a corresponding .lib?

Best regards
    Marc

Answer 1

Hi, Marc.

If I'm reading correctly, it sounds like you want to have a single solution for tracking memory allocations, regardless of whether they're native or managed.  I wonder if it might be more useful for that data to be separate, since one has to deal with native leaks very differently than managed leaks.  And your diagnostic code would somehow have to deal with managed objects shifting around on the GC heap.

In other words, I'm wondering if you're better off keeping your custom solution for native-only, and then using a managed profiler, like the free CLRProfiler (http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=a362781c-3870-43be-8926-862b40aa0cd0), to view the managed heap and track its allocations and movements that way.

Thanks,
Dave

Answer 2

Hi Dave and others, :)

No it's not quite like that. What I want to do is find memory leaks on the native heap only. Therefore, I use _CrtSetAllocHook() to hook allocations - and then do a Stacktrace and keep it for later and see how the allocation occured. This all works in native only code.

When I create a c++/clr project though, the success depends on how the .cpp file that allocates the memory gets compiled. When I compile it as native, without clr cupport, the method from above works as well even in this mixed mode. However, if I compile it with clr support, it fails to lookup the address on the stack from the symbols. This is all just related to the native heap - only new, not gcnew is used.  The real problem occurs when there is a .cpp file that actually uses some .net framework classes and therefore can't just be compiled without clr support. For leaks that occur at such places, I get a stacktrace with no filenames and linenumbers of my own code. But, as I posted before, using the managed .net Stacktrace function, it works - but as I also said that creates problems with the os loader lock.

So, is there a way to resolve the addresses on the stack for the cpp code compiled with clr support to the actuall symbol? When I enumerate the genereted .pdb file, the symbols are in there, but the addresses are wrong. I assume there is some .net/clr memory shifting going on somehow and therefore SymFromAddrW() fails to find the symbols.

I also found out that with each run, those methodes compiled without clr support all have exactly the same addresses on the stack, while the ones compiled with clr support are at different places every time. I found that out by creating a few classes with methods compiled with and without clr support calling each other and looking at the stacktraces.

Best regards
    Marc

Answer 3

Hi, Marc, I think I get it now.

I'm not an expert on Managed C++, but in general, a managed PDB will connect the source to the IL, and give you no information about the actual executable code generated by the JIT or by NGEN.  Also, note that if your app is getting JITted, then the JITted code will appear in memory dynamically allocated by the CLR's JIT.

If you're a debugger or profiler of managed code, then you get to see the missing link (IL-to-compiled machine code).  You also get access to the metadata, which is another way to resolve down to names.

To use the CLR debugging API, you need to be an out-of-process debugger.  To use the CLR profiling API, you run in-process as a DLL.  In both cases, you're a consumer of a native COM interface (i.e., the debugging & profiling APIs).  Furthermore, we never recommend you code your app in such a way that it expects a profiler is present (e.g., your AllocHook expecting it can call into a profiler DLL), since your code can't guarantee the profiler will always be loaded, and since it precludes the ability to load a different profiler you might need in the future (e.g., sampling CPU performance).  Similarly, if your AllocHook expects your special debugger to be attached, then what if it's not?  (Or what if you need to use a regular debugger?)

A couple ideas.

First, I kind of like your idea of reverse P/Invoking to managed to get a StackTrace.  You might try to detect if you're called while DllMain is on the stack, and just skip getting managed frames for that allocation.  You might also try avoiding as much allocation as you can while inside DllMain for the very reason you encountered--the loader lock is a big pain, and doing anything interesting (including heap allocations that require the heap lock) is probably best avoided if it can be deferred until later.

Second, consider formally separating your app from your diagnostics.  You could code up a proper profiler DLL which dynamically hooks your allocations when the profiler is present (leaving your allocations unhooked when it's not).  The profiler could then do a stack walk with access to metadata for resolving managed IPs.  Though I believe you may still have issues trying to do a walk if the loader lock is held, at least on 64 bit OS's.  To get an idea of what's involved, take a look at this: http://msdn.microsoft.com/en-us/library/bb264782.aspx.


Thanks,
Dave

Answer 4

Interesting what you say. You mentioned some details I didn't know. I'll read this article you linked.

So it isn't possible for the process itself to investigate the managed portions of the stack from unmanaged code - while seperate debuggers or profilers can do it?

Answer 5

Technically, a profiler is actually the same process (it's a DLL), whereas a debugger is out-of-process.  But logically speaking, the profiler is separate from the application it's profiling, in that the application code does not directly communicate with the profiler DLL or expect it to be present.  In any case, yes, for any code to make sense of the full stack, it will need some help from the CLR.  So the managed StackTrace can do this, and the CLR profiling/debugging interfaces can as well.