locked
Marshalling again..Some elusive bug RRS feed

  • General discussion

  • Hi,

    Here is discussion around the problem I am facing with Marshalling. I am using C# /.Net 1.1 (VS2003). I have a C function I am P/Invoking.

     

    Unmanaged C function signature.

     

    //bufferIn is the buffer that is read in the function and based on this buffer,

    //msgstructInstanceOut is filled out. No memory is allocated in this C function. The C function basically takes bufferIn and formats it correctly and copies it to the members of MSG_Struct represented by msgstructInstanceOut.

    ---------------------------------------------------------------------------------------

    int FillMessageStruct_C(void* msgstructInstanceOut, void* bufferIn)

     {

    MSG_Struct *msgstruct=(MSG_Struct *)buff;

    char* buffer=(char*)bufferIn;

    ...........................................

    //some logic

    ..........................................

    }

     

    Managed Side in C#:

     

    [StructLayout(LayoutKind.Sequential, CharSet=CharSet.Ansi)]

    public class MSG_Struct

    {

    [MarshalAs(UnmanagedType.ByValArray,SizeConst=5)]

    public ushort[] msg_offset=new ushort[5];

     

    [MarshalAs(UnmanagedType.ByValArray,SizeConst=1014)]

    public byte[] aprint_buff=new byte[1014];

     

    //Private Default Constructor

    MSG_Struct() {}

     

    public MSG_Struct(byte[] ipcRawBuffer)

    {

    MSG_Struct temp=new MSG_Struct();

     

    //P/Invoke func-call

    int result=MSG_Struct.FillMessageStruct_C(temp,ipcRawBuffer);

     

    if(result==0)

    {

    if(temp!=null)

    {

    for(int x=0;x<this.msg_offset.Length;x++)

    { this.msg_offset[x]=temp.msg_offset[x]; }

     

    Buffer.BlockCopy(temp.aprint_buff,0,this.aprint_buff,0,this.aprint_buff.Length);

    }

    }

    }//End of MSG_Struct C'tor

     

    [DllImport("MyCDll",CharSet=CharSet.Ansi,CallingConvention=CallingConvention.Cdecl)]

    public static extern int FillMessageStruct_C([In,Out]MSG_Struct messages,[In] byte[] ipcRawBuffer);

    }

     

     

    And Here is what I see-

     

    This worked fine for last 1 year or so. But recently testing group has put the app under some heavy load test and after about 8 hrs or so of running without any problem, the P/Invoke function call (the line in red) throws Exception with message "Object Reference not set to an instance of an object ". This problem does not occur for low loads. Load basically means the function being called more times, meaning more packets being processed per sec (upto about 350 times per second is heavy load).

     

    Now I have these questions-

    1) Is GC kicking in (under heavy load) and cleaning up any parameters (for example ipcRawBuffer) even before the P/Invoke call returns.  

    2) I know that by book I should pass IntPtr as parameters to C-function because the unmanaged function with CallingConvention.Cdecl is expecting void* pointers. But as I have stated, this only fails under heavy load and as per my observation, even the C function ultimately casts them form void*.

     

    Hope I have not put you guys to sleep Smile

     

    Thanks

    Raja 

    Thursday, February 14, 2008 4:01 PM

All replies

  • GC will not cleaning up because it will know that the ipcRawBuffer will be used later. But it might move the location of a object on the managed heap. You can use GCHandle to prevent a object's memory being moved by the GC when it's doing the collecting job. For more details, please visit Object Lifetime and Pinning.
    Monday, February 18, 2008 6:10 AM
  • Feng

    Thanks for the answer. But I have more questions on this.

     

    Three things that I want to know-

    1) You said- "...GC will know that the ipcRawBuffer will be used later...

    My question to you is- How will it know that the ipcRawBuffer will be used later? because if you look at the constructor method MSG_Struct you would notice that byte[] ipcRawBuffer is not being accessed anywhere in the constructor method after the call to MSG_Struct.FillMessageStruct_C(..)

     

    Also in the P/Invoke call, it is decorated specifically with [In], so it is also not an Out parameter-

    [DllImport("MyCDll",CharSet=CharSet.Ansi,CallingConvention=CallingConvention.Cdecl)]

    public static extern int FillMessageStruct_C([In,Out]MSG_Struct messages,[In] byte[] ipcRawBuffer); //Managed

     

    //C-equivalent function for which the above P/Invoke was used

    int FillMessageStruct_C(void* msgstructInstanceOut, void* bufferIn)

    I read the artcile from Jason Clark again- quote (2nd para under Object Lifetime and Pinning), it states that collector will neither move the object in memory nor remove the object from memory.

    From your explnation, how can it move the object then ? Thanks for your time.

     

    2) Now I did some more work on this. If I use the following, the program does not crash (the problem disappears). In other words, with the following signature being used, there is no error. As i understand from your answer, the same error should happen with the above signture too. I could not provide myself an explanation for the difference between the signature in point 1) and the following signature.-

    [DllImport("MyCDll",CharSet=CharSet.Ansi,CallingConvention=CallingConvention.Cdecl)]

    public static extern int FillMessageStruct_C(

    [ MarshalAs( UnmanagedType.AsAny ),In,Out] Object messages,

    [ MarshalAs( UnmanagedType.AsAny ),In]       Object ipcRawBuffer        );

     

     

     

    3) Apart from that I also tried GCHandle approach and got into errors (non-blittable data). Do you know how to handle this scenario as it looks like a simple case of a class having 2 arrays of blittable types.

     

    I used the following code-

    public MSG_Struct(byte[] ipcRawBuffer)//Constructor

    {

    GCHandle handleForTemp=new GCHandle();

    GCHandle handleForipcRawBuffer=new GCHandle();

    try

    {

    MSG_Struct temp=new MSG_Struct();

     

    handleForTemp=GCHandle.Alloc(temp,GCHandleType.Pinned);

    IntPtr ptrForTemp=handleForTemp.AddrOfPinnedObject();

    handleForipcRawBuffer=GCHandle.Alloc(ipcRawBuffer,GCHandleType.Pinned);

    IntPtr ptrForipcRawBuffer=handleForipcRawBuffer.AddrOfPinnedObject();

     

    int result=MSG_Struct.FillMessageStruct_C(ptrForTemp,ptrForipcRawBuffer);

    if(result==0) //success

    {

    //accessing temp that should be filled now.

    }

    }

     

    finally

    {

    if(handleForTemp.IsAllocated)

    {

    handleForTemp.Free();

    }

     

    if(handleForipcRawBuffer.IsAllocated)

    {

    handleForipcRawBuffer.Free();

    }

    }

    }

     

    Errors I see are-

    .............................Message: Object contains non-primitive or non-blittable data.

    .............................StackTrace: at System.Runtime.InteropServices.GCHandle.InternalAlloc(Object value, GCHandleType type)

    at System.Runtime.InteropServices.GCHandle..ctor(Object value, GCHandleType type)

    at System.Runtime.InteropServices.GCHandle.Alloc(Object value, GCHandleType type)

     

     

     

    Thanks for your time.

     

    Raja

    Tuesday, February 19, 2008 12:00 AM
  • Could it be that your input array ipcRawBuffer was null? That would cause perhaps the marshaller to hickup at this call.

    Yours,
       Alois Kraus



    Tuesday, February 19, 2008 10:50 PM
  • --How will it know that the ipcRawBuffer will be used later?
    The GC can obtain this information from the JIT which pre-compiled your code before it is executed.

    --From your explnation, how can it move the object then ? Thanks for your time.
    Yes, it will not be moved only if it's pinned, automatically or manually.

    And
    to troubleshoot this issue, we really need the necessary source code and the detailed repro steps to reproduce thet problem, so that we can investigate the issue in house. It is not necessary that you send out the complete source of your project. We just need a simplest sample to reproduce the problem. You can remove any confidential information or business logic from it.





    Wednesday, February 20, 2008 7:52 AM
  • We are changing the issue type to “Comment” because you have not followed up with the necessary information. If you have more time to look at the issue and provide more information, please feel free to change the issue type back to “Question” by editing your initial post and changing the radio button at the top of the post editor window. If the issue is resolved, we will appreciate it if you can share the solution so that the answer can be found and used by other community members having similar questions. 

    Thank you!  
    Monday, February 25, 2008 4:23 AM
  • So the following worked for me:

    //C-equivalent function for which the P/Invoke was used

    int FillMessageStruct_C(void* msgstructInstanceOut, void* bufferIn)

     

    [DllImport("MyCDll",CharSet=CharSet.Ansi,CallingConvention=CallingConvention.Cdecl)]

    public static extern int FillMessageStruct_C(

    [ MarshalAs( UnmanagedType.AsAny ),Out] Object messages,

    [ MarshalAs( UnmanagedType.AsAny ),In]       Object ipcRawBuffer        );

     

    Thanks to Feng Chen and others who gave useful info.

    Please close the issue.

    Monday, March 3, 2008 3:22 PM