none
Unmanaged to Managed callback stops working. RRS feed

  • Question

  • I'm using a 3rd party API written in C via P/Invoke in C#.  All seems to work well, including the one function that takes a function pointer and calls back to my managed code.  The problem I have is that after some time, the callback code stops working.  That is, the 3rd party API simply stops calling my code.  The 3rd party logs indicate an error of some sort.

    Obviously you cannot troubleshoot this 3rd part API, but I would like to confirm I'm not doing anything wrong as far as interop is concerned before contacting the vendor.  I've read several forums in which it has been hinted that the Garbage Collector can clean up (or relocate) the delegate used as the callback function.

    Here's what I've got...

    The 3rd party C based API declares the following in a header file....

    //Native function to be called.
    DllLinkage int __cdecl stream (int id, S1 *s1, S2 *s2, int (*callback) (int, char *, UINT), USHORT mode);
    
    //Protototype of the callback function to be passed as the 4th parameter to the native function
    int stream_callback (int id, char *buffer, uint length)
    

    I use the following P/Invoke declarations in C# so I can access the C code....

    public static extern int stream(int id, IntPtr s1, S2 s2, stream_callback callback, ushort mode);
    
    [UnmanagedFunctionPointer(System.Runtime.InteropServices.CallingConvention.Cdecl)]
    public delegate int stream_callback(int id, IntPtr buffer, uint length);

    Finally, I have a class the uses the code as follows....

    public class Device
    {
    
      private stream_callback _ecStreamCallback = null;
    
    
      public void Stream(S2 s2)
      {
       ushort mode = 1;
       if (_ecStreamCallback == null)
       {
         //It's possible that the GC will clean up our delegate before the native code is done with it.
         //To prevent this, we keep a reference to it.
         _ecStreamCallback = new stream_callback(Callback_Stream_DataAvailable);
       }
       int rv = stream(1, IntPtr.Zero, s2, _ecStreamCallback, mode);
          
      }
    
      private int Callback_Stream_DataAvailable(int id, IntPtr pBuffer, uint bufferSize)
      {
       //Do my thing here      
        
      }
    

    }

    This all works hundreads of times with hundreds of instances of the device class as the process runs.  However, eventually I will see an instance in which stream() is called and Callback_Stream_DataAvailable is not called, with a corresponding error in the 3rd party logs.  Again, I'm not ruling out that this could be a problem with the 3rd party API, but would like to confirm I'm not doing anything wrong or the Garbage Collector is not coming along and messing up my delegate in some manner.

    Specifically, do I need to use Marshal.GetFunctionPointerForDelegate() or some other code to make sure a "proper" function pointer is created?

    Thanks.

     

    Monday, January 17, 2011 4:58 PM

Answers

  • Hello mmcgrellis,

     

    1. The code in your OP (esp the Device class) should continue to work fine because the reference to the delegate object "_ecStreamCallback" is maintained properly. Hence the problem of the callback not being called after some time is not likely due to the delegate being garbage collected.

     

    2. To test this, after a successful call to Device.Stream(), make a call to GC.Collect() and then call Device.Stream() again.

    2.1 If the "_ecStreamCallback" had been garbage collected, you will get the "When passing delegates to unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called." message box during the execution of the stream() function.

    2.2 For a thorough test, do a loop in which Device::Stream() is called many times, with GC.Collect() called at the end of each Device::Stream() call, e.g. :

                Device device = new Device();

                for (int i = 0; i < 1000000; i++)
                {
                    S2 s2 = new S2();
                    s2.wFileFormat = 1;       // file format
                    s2.wDataFormat = 2;       // data encoding
                    s2.nSamplesPerSec = 3;      // sampling rate
                    s2.wBitsPerSample = 4;      // bits per sample

                    device.Stream(s2);

                    GC.Collect();
                }

    If the reference to "_ecStreamCallback" had been maintained, the loop will run successfully even though GC.Collect() is called upon every loop.

    2.3 If the Device::Stream() method was defined as follows :

            public void Stream(S2 s2)
            {
                ushort mode = 1;

                stream_callback _ecStreamCallback = new stream_callback(Callback_Stream_DataAvailable);

                int rv = stream(1, IntPtr.Zero, s2, _ecStreamCallback, mode);
            }

    That is, with "_ecStreamCallback" declared and instantiated only within the Stream() method, there is a risk that even with no call to GC.Collect(), the "When passing delegates to unmanaged code..." message box may eventually appear after a certain (albeit large) number of loops. That is, "_ecStreamCallback" may get garbage collected before the stream() function gets called.

     

    3. >> Again, do I need to use Marshal.GetFunctionPointerForDelegate(), or is that unnecessary due to the UnmanagedFunctionPointer atttribute?

    3.1 There is no need to use Marshal.GetFunctionPointerForDelegate() and in fact, its return value (of type IntPtr) could not be used as the "callback" delegate parameter to the stream() method.

    3.2 The stream() function would have to be defined as :

    public static extern int stream(int id, IntPtr s1, S2 s2, IntPtr callback, ushort mode);

    3.3 The Device::Stream() method would have to be written as :

            public void Stream(S2 s2)
            {
                ushort mode = 1;

                if (_ecStreamCallback == null)
                {
                    //It's possible that the GC will clean up our delegate before the native code is done with it.
                    //To prevent this, we keep a reference to it.
                    _ecStreamCallback = new stream_callback(Callback_Stream_DataAvailable);
                }

                IntPtr pcallback = Marshal.GetFunctionPointerForDelegate(_ecStreamCallback);

                int rv = stream(1, IntPtr.Zero, s2, pcallback, mode);
            }

     

    4. >> Also, seeing as I keep the delegate around for a long period of time, is it possible that the Gargbage Collector relocates it and somehow renders the delegate inaccessible from unmanaged code?  In other words, does the function pointer that actually gets passed to the unmanaged code somehow lose it's reference to the delegate?

    4.1 No this will not happen. The unmanaged code will obtain a reference to the delegate object and not a fixed memory function pointer. Hence the actual memory location of the delegate may still be moved around and not affect the caller in unmanaged code (as long as it is maintained alive in managed code).

    4.2 BTW, an alternative way to declare the stream() function is as follows :

    public static extern int stream(int id, IntPtr s1, S2 s2, [MarshalAs(UnmanagedType.FunctionPtr)] stream_callback callback, ushort mode);

     

    - Bio.

     

    • Marked as answer by mmcgrellis Wednesday, January 19, 2011 9:35 PM
    Tuesday, January 18, 2011 4:57 PM

All replies

  • The code looks good AFAICT, but it would be interesting to see your DllImport and type declarations on the managed side too.
    Mattias, C# MVP
    Tuesday, January 18, 2011 9:10 AM
    Moderator
  • I included the declarations in the original post, but forgot to include the DllImport attribute.  Here they are again....

    [DllImport("libstream.dll", EntryPoint = "stream", BestFitMapping = false, CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi, ExactSpelling = true, PreserveSig = true, SetLastError = true, ThrowOnUnmappableChar = true)]
    public static extern int stream(int id, IntPtr s1, S2 s2, stream_callback callback, ushort mode);
    
    [UnmanagedFunctionPointer(System.Runtime.InteropServices.CallingConvention.Cdecl)]
    public delegate int stream_callback(int id, IntPtr buffer, uint length);

    I've also included the definition of S2...

    [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi, Pack=1)]
    internal class S2
    {
      public ushort wFileFormat;       // file format
      public ushort wDataFormat;       // data encoding
      public uint  nSamplesPerSec;      // sampling rate
      public ushort wBitsPerSample;      // bits per sample
    }
    

    Again, do I need to use Marshal.GetFunctionPointerForDelegate(), or is that unnecessary due to the UnmanagedFunctionPointer atttribute?

    Also, seeing as I keep the delegate around for a long period of time, is it possible that the Gargbage Collector relocates it and somehow renders the delegate inaccessible from unmanaged code?  In other words, does the function pointer that actually gets passed to the unmanaged code somehow lose it's reference to the delegate?

    Thanks again.

    Tuesday, January 18, 2011 2:12 PM
  • Hello mmcgrellis,

     

    1. The code in your OP (esp the Device class) should continue to work fine because the reference to the delegate object "_ecStreamCallback" is maintained properly. Hence the problem of the callback not being called after some time is not likely due to the delegate being garbage collected.

     

    2. To test this, after a successful call to Device.Stream(), make a call to GC.Collect() and then call Device.Stream() again.

    2.1 If the "_ecStreamCallback" had been garbage collected, you will get the "When passing delegates to unmanaged code, they must be kept alive by the managed application until it is guaranteed that they will never be called." message box during the execution of the stream() function.

    2.2 For a thorough test, do a loop in which Device::Stream() is called many times, with GC.Collect() called at the end of each Device::Stream() call, e.g. :

                Device device = new Device();

                for (int i = 0; i < 1000000; i++)
                {
                    S2 s2 = new S2();
                    s2.wFileFormat = 1;       // file format
                    s2.wDataFormat = 2;       // data encoding
                    s2.nSamplesPerSec = 3;      // sampling rate
                    s2.wBitsPerSample = 4;      // bits per sample

                    device.Stream(s2);

                    GC.Collect();
                }

    If the reference to "_ecStreamCallback" had been maintained, the loop will run successfully even though GC.Collect() is called upon every loop.

    2.3 If the Device::Stream() method was defined as follows :

            public void Stream(S2 s2)
            {
                ushort mode = 1;

                stream_callback _ecStreamCallback = new stream_callback(Callback_Stream_DataAvailable);

                int rv = stream(1, IntPtr.Zero, s2, _ecStreamCallback, mode);
            }

    That is, with "_ecStreamCallback" declared and instantiated only within the Stream() method, there is a risk that even with no call to GC.Collect(), the "When passing delegates to unmanaged code..." message box may eventually appear after a certain (albeit large) number of loops. That is, "_ecStreamCallback" may get garbage collected before the stream() function gets called.

     

    3. >> Again, do I need to use Marshal.GetFunctionPointerForDelegate(), or is that unnecessary due to the UnmanagedFunctionPointer atttribute?

    3.1 There is no need to use Marshal.GetFunctionPointerForDelegate() and in fact, its return value (of type IntPtr) could not be used as the "callback" delegate parameter to the stream() method.

    3.2 The stream() function would have to be defined as :

    public static extern int stream(int id, IntPtr s1, S2 s2, IntPtr callback, ushort mode);

    3.3 The Device::Stream() method would have to be written as :

            public void Stream(S2 s2)
            {
                ushort mode = 1;

                if (_ecStreamCallback == null)
                {
                    //It's possible that the GC will clean up our delegate before the native code is done with it.
                    //To prevent this, we keep a reference to it.
                    _ecStreamCallback = new stream_callback(Callback_Stream_DataAvailable);
                }

                IntPtr pcallback = Marshal.GetFunctionPointerForDelegate(_ecStreamCallback);

                int rv = stream(1, IntPtr.Zero, s2, pcallback, mode);
            }

     

    4. >> Also, seeing as I keep the delegate around for a long period of time, is it possible that the Gargbage Collector relocates it and somehow renders the delegate inaccessible from unmanaged code?  In other words, does the function pointer that actually gets passed to the unmanaged code somehow lose it's reference to the delegate?

    4.1 No this will not happen. The unmanaged code will obtain a reference to the delegate object and not a fixed memory function pointer. Hence the actual memory location of the delegate may still be moved around and not affect the caller in unmanaged code (as long as it is maintained alive in managed code).

    4.2 BTW, an alternative way to declare the stream() function is as follows :

    public static extern int stream(int id, IntPtr s1, S2 s2, [MarshalAs(UnmanagedType.FunctionPtr)] stream_callback callback, ushort mode);

     

    - Bio.

     

    • Marked as answer by mmcgrellis Wednesday, January 19, 2011 9:35 PM
    Tuesday, January 18, 2011 4:57 PM
  • Lim,

    Thank you.  Based on your feedback, it looks like my code and methodology are sound.  As you suggested, I will run some tests with an without the GC.Collect() call.  Assuming my results are the same either way, I'll mark this thread as answered.

    Thanks again,

    Mick

     

    Wednesday, January 19, 2011 1:28 PM
  • Lim,

    My testing did not show the issue to occur any more frequently after introducing calls to GC.Collect().  Therefore I'm going to include that there is nothing wrong with my code and the GC is not doing anything to cause this issue.  I will pursue the issue with the vendor of the API.

    Thanks for your help,

    Mick

     

    Wednesday, January 19, 2011 9:35 PM
  • Most welcome Mick. All the best to you.

    - Bio.

     

    Friday, January 21, 2011 1:14 AM