none
BUG? P/Invoke DLL refcount not decremented before destroying managed application? RRS feed

  • Question

  • I'm working with Visual Studio C# 2008 Express Edition with a target of .net Framework 2.0. I have the latest Win32 SDK (v3.1) installed and configured for use in Visual Studio.

     

    The application is a simple Form that presents a few controls that display values retrieved from calls to an unmanged DLL using Platform Invoke.

     

    I was experiencing a consistent problem that (whether running Debug or Release) Windows Vista would report an Invalid Access (0xc0000005) Exception in an unmanaged DLL when the application was closed. No JIT debugging was available.

     

    After hours of searching and reading up on P/Invoke I came across a fragment of a blog posting that talked about manually forcing a library to unload by using P/Invoke to call LoadLibrary() and FreeLibrary().

     

    In order to force the library to be unloaded it is necessary to call FreeLibrary() twice - to account for the DLLImport refcount as well as the manual LoadLibrary().

     

    After implementing this simple code the application I am working on no longer suffers the Invalid Access exceptions.

     

    This seems like the managed code memory is being disposed of before the unmanaged DLL has its refcount decremented, and therefore any access by the DLL to memory is going to cause the exception.

     

    I imagine this isn't seen much for popular system DLLs since their refcount is probably going to keep them around most of the time. For DLLs that are rarely used, however, this looks to be a particular issue.

     

    Example code to solve issue:

     

    Code Snippet

    public partial MyClass : Form

    {

      // workaround for Platform Invoke issue

      [DllImport("kernel32.dll")]

      public static extern IntPtr LoadLibrary(string lpFileName);

      [DllImport("kernel32.dll")]

      public static extern IntPtr FreeLibrary(IntPtr library);

      private String DllPath = "C:\\Program Files\\Common Files\\Sony Shared\\Sony Utilities\\SnyUtils.dll";

      private IntPtr dll;

     

      // DLL required by application

      [DllImport("C:\\Program Files\\Common Files\\Sony Shared\\Sony Utilities\\SnyUtils.dll")]

      public static extern IntPtr SuOpen([In, Out] ref int arg1, [In, Out] ref int arg2);

      [DllImport("C:\\Program Files\\Common Files\\Sony Shared\\Sony Utilities\\SnyUtils.dll")]

      public static extern int SuClose(IntPtr handle);

     

      private int result = -1;

      private IntPtr handle;

     

      public MyClass()

      {

        // load DLL explicitly (increases refcount)

        dll = LoadLibrary(DllPath);

     

        // use the DLL

        handle = SuOpen(ref arg1,ref arg2);

     

        result = SuClose(handle);

        handle = IntPtr.Zero;

     

        // unload DLL explicitly twice (refcount was incremented by DllImport)

        handle = FreeLibrary(dll);

        handle = FreeLibrary(dll);

      }

    }

     

     

    Removing that 2nd FreeLibrary() call will provoke the Invalid Access exception.

     

    I'm not sure if this is specific to the DLL I'm working with or a more general issue with P/Invoke.

    Monday, April 28, 2008 11:16 PM

All replies

  • I think maybe clr does LoadLibray for you.  You don't need the loadlibrary call in your code.  You just need the SuOpen and SuClosed.  If you want to use LoadLibrary you need to change your code so its like this.

    [DllImport("kernel32.dll")]
    internal static extern IntPtr GetProcAddress(IntPtr module, string procName);

    public delegate IntPtr SuOpenHandler( ref int arg1, ref int arg2);
    public delegate
    int SuCloseHandler(IntPtr handle);

      public void MyClass()

      {

        // load DLL explicitly (increases refcount)
        IntPtr dll = LoadLibrary(DllPath);

        // Get the proc handle
        IntPtr procHandle = GetProcAddress(dll,"SuOpen");

        // Get wrapper
        SuOpenHandler handler =(SuOpenHandler) Marshal.GetDelegateForFunctionPointer(procHandle, typeof(SuOpenHandler));

        // use the DLL
        IntPtr handle = handler(ref arg1,ref arg2);

        // Get the address of the close.
        procHandle = GetProcAddress(dll, "SuClose");

    // Get wrapper for close.
        SuCloseHandler closeHandler =(SuCloseHandler) Marshal.GetDelegateForFunctionPointer(procHandle, typeof(SuCloseHandler));

    // Call close.
        result = closeHandler(handle);

    // Free
        handle = FreeLibrary(dll);

      }


    marshal.getdelegateforfunctionpointer is a new method added to .net 2.0 that allows for dynamic pinvoke.
    http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.marshal.getdelegateforfunctionpointer(vs.80).aspx
    Sunday, May 25, 2008 6:59 AM
  • Program shut-down is always a tricky time for unmanaged DLLs.  That's when it releases memory and other kernel resources.  Any memory allocation or usage errors made previously now tend to become noticeable.  By unloading the DLL explicitly, you certainly found a workaround for this problem.  Most likely because you established a different memory release pattern.  It may bite you again though, especially when the DLL destroyed memory belonging to the CLR.  ExecutionEngineException is one sign of really bad news.  Keep your fingers crossed.
    Sunday, May 25, 2008 11:42 AM
    Moderator

  • I give up! The Forum code is somehow dropping the original content of my reply.  When I spotted the content was missing I edited the post with an explanation of why there was an apparently empty reply but *that* was also lost!

    Sunday, May 25, 2008 12:27 PM
  • What?
    Sunday, May 25, 2008 12:45 PM
    Moderator
  • The Clr calls loadLibrary for you the first time you invoke the SuOpen or SuClose.  The dll will stay loaded for the lifetime of the appdomain.  If you want control over this yourself please use the example I provided using the GetProcAddress function and the Marshal.GetDelegateForFunctionPointer method.  I found some more info on this on the internet.

    PInvoke Library Load/Unload Behavior – Freeing Unmanaged Libraries
    http://blogs.msdn.com/robgruen/archive/2004/11/12/256199.aspx

    About the forum.  The forums here are normally super stable considering the millions of posts and tons of traffic it gets each day.  Sometimes little hiccups happen though.  The people at Microsoft are working on a new forums application to replace this one in the next couple months.  It has got to be one of the most slickest forum applications I have ever seen.  Check it out if you have the chance. 

    http://forums.msdn.microsoft.com/en-US/forums

    Sunday, May 25, 2008 7:49 PM
  • Second attempt to post this reply

    I shall try one final time to reply.

    This forum software is consistently inconsistent in my experience, and it is the only one I've ever seen these kind of problems with! The raw HTML isn't standards compliant (lacks closing tags, etc.)
    I "Sign In", click a "Reply" button and in the sequence of responses and redirects it proceeds to log me out! When I do manage to get to the 'Reply' page, more often than not when I "Post" a reply it proceeds to log me out or the resulting message has no content! Is this some kind of anti-Firefox non-standards-compliant thing?

    Whatever the reason for the Access Exception, using the technique I described works successfully whereas expecting the CLR to handle it always fails. Whether that is a symptom of a bug in the Sony DLL, the CLR, or a combination of both I don't know. The library isn't loaded by any other process prior to my application needing it.

    The call to LoadLibrary() is necessary otherwise there is no valid handle to later call FreeLibrary(). Without the LoadLibrary() call there would be no way to manually force the issue. Because the DLL refcount is incremented by the CLR as well as LoadLibrary() the two FreeLibrary() calls are necessary to correctly adjust the refcount as it was affected by this application.

    Is there a reliability justification for using Marshal.GetDelegateForFunctionPointer() or is it a case of a 'cleaner' way of calling unmanaged code functions? I think I used the 'old' .net 1.x approach which I copied from some older code of mine that followed the guidelines at the time it was coded.

    "By unloading the DLL explicitly, you certainly found a workaround for this problem. Most likely because you established a different memory release pattern. It may bite you again though, especially when the DLL destroyed memory belonging to the CLR."

    What memory might the DLL destroy that belongs to the CLR? Is that a theory, a guess, or is there some objective way (tool, method?) to detect such issues?

    In the defined case there is definitely an issue of the CLR destroying memory 'belonging to' or expected by the DLL. With the work-around there are no detectable issues: the DLL unloads, the process ends, and the AppDomain is destroyed.

    I had read the blog article "PInvoke Library Load/Unload Behavior – Freeing Unmanaged Libraries" as part of my extensive investigation into what was going on and how to solve it. As it says:

    "The module will stay loaded in memory until the AppDomain shuts down. Typically this means that the dll will be loaded into your applications memory until the process goes away"

    What it nor any other reliable resource I could find explains is the precise mechanism used, or how to successfully debug the destruction sequence. When debugging the application the Access Exception occurs immediately after the debugging session ends. The resulting dialog offers the option to JIT debug, but if accepted that reports it can't find an appropriate JIT debugger (I forget the exact message now) and then the session ends.

    It wouldn't surprise me if the Sony DLL is doing something 'silly' but as the purpose of the utility is simply a one-time grab of the 'capabilities' string from an unknown Sony Vaio of its Sony Notebook Control (SNC) so that support for the model can be added to the open source Linux SNC driver, I'm not too worried about other after-effects.

    Monday, May 26, 2008 12:36 PM
  • You've not been the only one with that post problem this weekend.  Yet another bug, although this is a new and recent one.  We're not going to worry about it, the forums are moving to an entirely new platform this week.  Check the announcement at the top of this forum.

    Trying to debug this problem is going to be awfully hard, you'll be groveling in assembly code without any debug info, chasing a corruption that happened millions of cycles ago.  If you are reading a string, you are P/Invoking an export you haven't mentioned before. 
    Monday, May 26, 2008 12:59 PM
    Moderator
  • No, not actually fetching a 'string' but the net effect of the code-that-actually-does-something-useful is to repeatedly call the same SNC function and build the 'string' from the returned ints.


    Code Snippet
    public static extern int SuGetMachineInfo(IntPtr handle, int index, [In,Out] ref int capability);

    ...

     for (int i = 1; i <= 38; i++)
                     SuGetMachineInfo(handle, i, ref caps[i - 1]);

    Monday, May 26, 2008 2:31 PM
  • Like I said by using Marshal.GetDelegateForFunctionPointer() and GetProcAddress function you get around the runtime loading the dll for you.   Normally when using PInvoke use are not supposed to call LoadLibrary yourself.  The CLR does it for you and like the blog says the clr will freelibrary dll when the appdomain is unloaded. 


    Here is some more info
    http://blogs.msdn.com/jonathanswift/archive/2006/10/03/Dynamically-calling-an-unmanaged-dll-from-.NET-_2800_C_23002900_.aspx
    http://blogs.msdn.com/junfeng/archive/2004/07/14/181932.aspx
    Monday, May 26, 2008 11:29 PM
  • You are missing the point Ziad, calling FreeLibrary early was the solution.
    Tuesday, May 27, 2008 12:00 AM
    Moderator