none
Passing structure to backend code failing RRS feed

  • Question

  • Hi,

    I am using an unmanaged API which is excepting an structure.

    I have defined all the structures in a class (Say Class A) and all the API in  another class (Say Class B). 

    I am calling the API in another class(Class C) and creating instance of the structure (declared in Class B) and passing to the API.

    When i try to debug from the unmanaged code,i noticed that junk values were received in the structure that was passed from Managed code.

    I suspected that the Garbage Collector was freeing the structure instance before it was passed to the unmanged code.

    My Question is how can i Pin the strcuture to prevent it from getting cleaned by Garbage collector.

    Also i tried declaring the API,the structure in same class and tried calling API from the same class.With this approach ,when i tried to debug from managed code proper values were received.

    Please provide your suggestion on this.

    Regards

    Vinutha

     

    Monday, May 30, 2011 10:56 AM

Answers

  • Hello vinutha kempanna,

     

    1. Blittable and Non-Blittable Types and Reference and Value Types.

    1.1 To understand the nature of the "Object contains non-primitive or non-blittable data." problem, we need to understand the concepts of blittable and non-blittable types as well as reference and value types.

    1.2 By "blittable type", we mean types that have a common representation in both managed and unmanaged memory and do not require conversion when they are passed between managed and unmanaged memory.

    1.3 In short, their data can be copied directly from managed memory to unmanaged memory and vice versa without any special processing by the interop marshaler.

    1.4 An example of a blittable type is the C# int type. Assuming 32-bit applications, an instance of a C# int takes up 4 bytes, same as a C++ int type which also takes up 4 bytes. Furthermore, their binary representation in memory are the same. Hence a C# int variable in managed memory can be directly copied to a C++ int variable in unmanaged memory. And vice versa.

    1.5 Non-blittable-types are just the opposite of blittable-types : i.e. types that do not have a common representation in both managed and unmanaged memory and may require conversion when they are passed between managed and unmanaged memory (this is where the MarshalAsAttribute comes in).

    1.6 An examples of a non-blittable type is the string. A .NET string is intrinsically unicode based and does not have a direct unmanaged equivalent. An unmanaged string may be C-style null-terminated, or it may be a BSTR (COM string). Furthermorem an unmanaged string may be unicode-based or multi-byte character-based.

    1.7 You might be surprised to note that a .NET char type is actually non-blittable unless its unmanaged counterpart is also a unicode character (wchar_t in C++). This is because a .NET char is unicode-based. The same is true of a .NET Boolean which can be represented in unmanaged code in various formats.

    1.8 The overriding thing about non-blittable types is that they do not have a single common unmanaged equivalent. Such types, when required to be marshaled over to unmanaged code, will need specifications in order for the interop marshaler to be able to successfully perform the marshaling. These specifications come in the form of MarshalAsAttributes.

    1.9 By "reference type", we mean types the actual data of which are allocated in the managed heap. They are reference counted internally and are de-allocated by the Garbage Collector. Each programing language variable that is assigned to a value of such a type merely holds a reference to that value. The variable does not get assigned another copy of the value.

    1.10 The opposite of a reference type is the value type. They reside on the stack. They are not managed by the Garbage Collector. They are destroyed when they go out of scope. Each programming variable that is assigned a value-type value gets its own copy of that value.

    1.11 Now, how are the concepts of blittability and reference and value types relevant to the "Object contains non-primitive or non-blittable data" error ? The following is a summary :

    1.11.1 When we want to share a managed class/struct with unmanaged code, a good optimization would be to directly pass the class/struct to the unmanaged code.

    1.11.2 This can only be done if the member data of the managed class/struct are all contained within the memory block of the class/struct itself. That is, the class/struct self-contains all member data just like a C/C++ class/struct.

    1.11.3 Now this is only possible if all members are blittable types. If any member is non-blittable, the entire class/struct is non-blittable.

    1.11.4 The advantage of a completely blittable class/struct is that such a class/struct may be directly passed to unmanaged code. Now in order to pass it to unmanaged code, it needs to be pinned in memory, otherwise, not only can it be moved around by the GC, it may even be completely de-allocated by the GC.

    1.11.5 The unmanaged code may then reference the class/struct directly and may even make modifications to it if this is expected. When the unmanaged code returns, the class/struct may be unpinned from memory.

    1.11.6 Now, if a class/struct is non-blittable, it will require the help of the interop marshaler to create in memory a complete copy of the class/struct.

    1.11.7 This copy must contain the actual data of all the members of the class/struct. It serves as a an unmanaged representation of the class/struct. The copy is made with the help of the various MarshalAsAttributes.

    1.11.8 It is this unmanaged representation that gets passed to the unmanaged code. Now, if the unmanaged code is expected to make modifications to this representation, then when the unmanaged code returns, the (now modified) data from the representation must be copied back to the original managed class/struct.

    1.11.9 Eventually, the unmanaged representation must be freed.

     

    2. The Members of the GatewayConnection Class.

    2.1 Now, let's examine the GatewayConnection class.

    This class contains 2 int members :

    public int NetworkID;
    public int OfflineEdit;

    and 2 managed array members :

    public ushort[] IpAddress;
    public ushort[] Password;

    2.2 In C#, the int type is a value type and is blittable. When used as members of a class/struct, it can directly reside as part of the memory block of the class/struct itself. Hence NetworkID and OfflineEdit form part of the body of the GatewayConnection class. They are destroyed when an instance of the GatewayConnection class is destroyed.

    2.3 Therefore, when we have code like the following :

         GatewayConnection objGWConn = new GatewayConnection();
         objGWConn.NetworkID = 0;
         objGWConn.OfflineEdit = 0;

    The values of members "NetworkID" and "OfflineEdit" (i.e. zero) reside directly as part of the memory block for "objGWConn". The situation is exactly the same as that for the members of a C/C++ class/struct.

    2.4 Now a managed array, when used as a member of a class or struct, renders the containing class/struct non-blittable. This is because a managed array is a reference type. That is, its actual array data exists in the managed heap and is merely referenced by the class/struct member.

    2.5 Therefore, when we have code like the following :

    objGWConn.IpAddress = new ushort[16];

    The array of 16 ushort values resides in the managed heap somewhere and is referenced by "objGWConn.IpAddress". The actual array data of 16 ushorts is not part of the memory block of the objGWConn object. This is totally unlike what we may be familiar with in C/C++.

    The situation is similarly so for member "Password".

    2.6 This is the reason why we cannot pin down in memory a class/struct that contains an array :

    GCHandle handle = GCHandle.Alloc(objGWConn, GCHandleType.Pinned); // Gives rise to error.

    In fact, we cannot pin a class/struct that contains members which are reference types including the string type.

    To pin such a class/struct would also require that the referenced object be pinned. This might seem acceptable initially until you consider code like the following :

    GatewayConnection objGWConn = new GatewayConnection();
    ushort [] ushort_array = new ushort[16];

    objGWConn.IpAddress = ushort_array; //new ushort[16];

    That is, "objGWConn.IpAddress" does not reference a newly allocated ushort array. Instead, it references an existing one. Pinning "objGWConn.IpAddress" would also logically require that "ushort_array" be pinned down. This may not be a desireable thing.

     

    3. How Do We Marshal the GatewayConnection Class to an Unmanaged API ?

    The following is an example of how this can be done. Note that there are simpler ways to accomplish the same objective but this example is meant to show some semblance of how the interop marshaler works under the covers :

    3.1 Allocate memory that provides an unmanaged representation of an instance of the GatewayConnection class (i.e. a GatewayConnection object).

    3.2 This unmanaged memory should be modeled after the equivalent of the GatewayConnection class in unmanaged memory. For example, the GatewayConnection class may be represented in C/C++ as follows :

    #pragma pack(1)
    struct GatewayConnection
    {
      unsigned short IpAddress[16];
      unsigned short Password[9];
      int NetworkID;
      int OfflineEdit;
    };

    Notice that all members of the struct are all packed together end-to-end in such an unmanaged representation of the GatewayConnection class. This is, of course, typical in C/C++ but is not the case for the managed GatewayConnection class. 

    3.3 Transfer the actual data of the members of the GatewayConnection object to this unmanaged memory.

    3.4 Call the unmanaged API passing along a pointer to the unmanaged memory.

    3.5 After the API call, if the API is to modify the unmanaged GatewayConnection struct, the returned data in the unmanaged GatewayConnection struct must be copied back to the managed GatewayConnection class.

    3.6 Finally, the unmanaged GatewayConnection struct must be freed.

    3.7 Why do we have to take such elaborate steps to pass a GatewayConnection object to the API ? Becuase the array members of the managed GatewayConnection object cannot be accessed by unmanaged code if you had simply passed a pointer to the object to the API.

    3.8 An example code, written in C#, is given in section 4 below. The following is a synopsis :

    3.8.1 The example code uses a DLL with a simple exported API named TestAPI().

    3.8.2 The C# code creates an instance of GatewayConnection.

    3.8.3 The code then creates an unmanaged representation of this object and copies all member data from the object to the unmanaged representation.

    3.8.4 A pointer to this unmanaged representation is passed to TestAPI() (the code for TestAPI() is also listed).

    3.8.5 As far as TestAPI() is concerned, the input unmanaged representation of the GatewayConnection object is a GatewayConnection struct which was listed in point 3.2 above.

    3.8.4 TestAPI() then modifies the struct and returns.

    3.8.5 When control returns to the C# code, we copy the now modified contents of the unmanaged struct back to the managed GatewayConnection object. The changes will be clearly seen.

    3.8.6 The unmanaged memory of the GatewayConnection struct which ws passed to TestAPI() is then destroyed.

     

    4. Example Code.

    4.1 To begin, note that you need to decorate the GatewayConnection class with the StructLayoutAttribute :

        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

    This is a necessary attribute in order that an unmanaged representation of the GatewayConnection class be creatable and the size of the unmanaged representation be calculatable. In particular, this attribute is necessary in order that a cdall to  Marshal.SizeOf() will succeed. More on this later.

     

    4.2 Now, in order to allocate memory that provides an unmanaged representation of an instance of the GatewayConnection class, we must perform the following in code :

    int iSize = Marshal.SizeOf(objGWConn);
    IntPtr p = IntPtr.Zero;
    p = Marshal.AllocHGlobal(iSize);
    Marshal.StructureToPtr(objGWConn, p, false);

    4.2.1 We first calculate the size of an unmanaged representation of objGWConn (an instance of GatewayConnection). This is done using Marshal.SizeOf().

    4.2.2 We then declare an IntPtr "p" which will point to this unmanaged representation of objGWConn.

    4.2.3 We then allocate in the unmanaged heap, a memory block the size of the unmanaged representation of objGWConn. This is done using Marshal.AllocHGlobal(). Note that "p" is used to point to this unmanaged memory.

    4.2.4 Finally, we copy the data of objGWConn (including the actual data of the array members) to the unmanaged representation. This is done by Marshal.StructureToPtr() with the help of the MarshalAsAttributes set for the "IpAddress" and "Password" members.

     

    4.3 After this, the unmanaged API may be invoked. The test API, written in C++ for this purpose, is listed below :

    void __stdcall TestAPI(/*[in, out]*/ GatewayConnection* pGatewayConnection)
    {
      for (int i = 0; i < 16; i++)
      {
        (pGatewayConnection -> IpAddress)[i] = i;
      }
     
      for (int i = 0; i < 9; i++)
      {
        (pGatewayConnection -> Password)[i] = i;
      }
     
      pGatewayConnection -> NetworkID = 101;
      pGatewayConnection -> OfflineEdit = 202;
    }

    I have intended that TestAPI() modify the fields of the GatewayConnection struct.

    The C# DLLImport declaration for TestAPI() is :

    [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
    private static extern void TestAPI(IntPtr pGatewayConnection);

     

    4.4 The call to the TestAPI() in C# is very simple :

    TestAPI(p);

     

    4.5 Now TestAPI() will permanently modify the member fields and so after the API call, we copy the now modified contents of the unmanaged representation of objGWConn, pointed to by "p", back to the managed "objGWConn" :

    objGWConn = (GatewayConnection)(Marshal.PtrToStructure(p, typeof(GatewayConnection)));

    This is done by using Marshal.PtrToStructure().

     

    4.6 After that we must free the unmanaged representation of objGWConn :

    Marshal.FreeHGlobal(p);

     

    4.7 Note that I have taken into account your concerns for ensuring that "objGWConn" remains untouched by the Garbage Collector. For this, note that you need not use the version of GCHandle.Alloc() that takes a GCHandleType parameter :

    GCHandle handle = GCHandle.Alloc(objGWConn, GCHandleType.Pinned);

    You may also use the single parameter version :

    GCHandle handle = GCHandle.Alloc(objGWConn);

    This also fixes objGWConn in the managed heap and disallows the GC from collecting it.

     

    4.8 A full source code listing is provided below for your reference :

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Runtime.InteropServices;
    using System.Windows.Forms;

    namespace CSConsoleApp01
    {
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

        class Program
        {
            [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
            private static extern void TestAPI(IntPtr pGatewayConnection);

            static void Main(string[] args)
            {
                GatewayConnection objGWConn = new GatewayConnection();

                objGWConn.IpAddress = new ushort[16];
                objGWConn.IpAddress[0] = 10;
                objGWConn.Password = new ushort[9];
                objGWConn.Password[0] = 0;

                objGWConn.NetworkID = 0;
                objGWConn.OfflineEdit = 0;

                try
                {
                    // Pin down the byte array
                    GCHandle handle = GCHandle.Alloc(objGWConn);

                    // Calculate the size of the unmanaged representation of objGWConn.
                    int iSize = Marshal.SizeOf(objGWConn);
                    IntPtr p = IntPtr.Zero;
                    // Allocate memory for the unmanaged representation of objGWConn
                    // in the unmanaged global heap.
                    p = Marshal.AllocHGlobal(iSize);
                    // Copy the full contents of the members of the managed objGWConn
                    // to the unmanaged representation of objGWConn.
                    Marshal.StructureToPtr(objGWConn, p, false);

                    // Call the API.
                    TestAPI(p);

                    // Copy the modified contents of the unmanaged representation
                    // of objGWConn back to the members of the managed objGWConn.
                    objGWConn = (GatewayConnection)(Marshal.PtrToStructure(p, typeof(GatewayConnection)));
                    // Free the unmanaged representation of objGWConn.
                    Marshal.FreeHGlobal(p);

                    // Free the GCHandle.
                    handle.Free();
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            }
        }
    }

     

    I hope you will benefit from this rather long post.

    - Bio. 

     



    • Proposed as answer by Paul Zhou Wednesday, June 1, 2011 2:17 AM
    • Marked as answer by Paul Zhou Tuesday, June 7, 2011 8:04 AM
    Tuesday, May 31, 2011 3:39 PM

All replies

  • You need to create a GCHandle, and keep this handle alive until your unmanaged method returns. That would keep your objects pinned, and the garbage collector wouldnt collect this memory during this scope.

    You could refer to http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle%28v=vs.71%29.aspx for details.

    Monday, May 30, 2011 11:47 AM
  • Thanks for your reply.

    I tried creating a GCHandle,but got an exception "Object contains non-primitive or non-blittable data."

    The definition of my structure is

     

    public class

    GatewayConnection

    {

    [

    MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]

    public ushort[] IpAddress;

    [MarshalAs(UnmanagedType

    .ByValArray, SizeConst = 9)]

     

    public ushort[] Password;

     public int NetworkID;

     public int OfflineEdit;

    }

    Please let me know what is the problem with this.

    Regards

    Vinutha

    Tuesday, May 31, 2011 7:03 AM
  • From the code above i cant see where you have tried to create a GCHandle.

    There is an example of creating and passing the pinned object into an unmanaged method at the link http://msdn.microsoft.com/en-us/library/system.runtime.interopservices.gchandle%28v=vs.71%29.aspx

    Tuesday, May 31, 2011 10:38 AM
  • Hi,

    This is how i try to create GCHandle.

    GatewayConnection objGWConn =

     

    new GatewayConnection

    ();

    objGWConn.IpAddress =

     

    new byte

    [16];

    objGWConn.IpAddress[0] = 0;

    objGWConn.Password =

     

    new byte

    [9];

    objGWConn.Password[0] = 0;

    objGWConn.NetworkID = 0;

    objGWConn.OfflineEdit = 0;

    try

    {

     

    // Pin down the byte array

     

    GCHandle handle = GCHandle.Alloc(objGWConn, GCHandleType

    .Pinned);

     

    IntPtr

    address = handle.AddrOfPinnedObject();

    handle.Free();

    }

     

    catch (Exception

    ex)

    {

     

    MessageBox

    .Show(ex.Message);

    }

    

    When i try to create handle,the code is failing

    Tuesday, May 31, 2011 11:38 AM
  • Check out http://social.msdn.microsoft.com/forums/en-US/csharplanguage/thread/12943a69-3ed0-4806-809d-8ca15e744f52
    Tuesday, May 31, 2011 12:07 PM
  • Hello vinutha kempanna,

     

    1. Blittable and Non-Blittable Types and Reference and Value Types.

    1.1 To understand the nature of the "Object contains non-primitive or non-blittable data." problem, we need to understand the concepts of blittable and non-blittable types as well as reference and value types.

    1.2 By "blittable type", we mean types that have a common representation in both managed and unmanaged memory and do not require conversion when they are passed between managed and unmanaged memory.

    1.3 In short, their data can be copied directly from managed memory to unmanaged memory and vice versa without any special processing by the interop marshaler.

    1.4 An example of a blittable type is the C# int type. Assuming 32-bit applications, an instance of a C# int takes up 4 bytes, same as a C++ int type which also takes up 4 bytes. Furthermore, their binary representation in memory are the same. Hence a C# int variable in managed memory can be directly copied to a C++ int variable in unmanaged memory. And vice versa.

    1.5 Non-blittable-types are just the opposite of blittable-types : i.e. types that do not have a common representation in both managed and unmanaged memory and may require conversion when they are passed between managed and unmanaged memory (this is where the MarshalAsAttribute comes in).

    1.6 An examples of a non-blittable type is the string. A .NET string is intrinsically unicode based and does not have a direct unmanaged equivalent. An unmanaged string may be C-style null-terminated, or it may be a BSTR (COM string). Furthermorem an unmanaged string may be unicode-based or multi-byte character-based.

    1.7 You might be surprised to note that a .NET char type is actually non-blittable unless its unmanaged counterpart is also a unicode character (wchar_t in C++). This is because a .NET char is unicode-based. The same is true of a .NET Boolean which can be represented in unmanaged code in various formats.

    1.8 The overriding thing about non-blittable types is that they do not have a single common unmanaged equivalent. Such types, when required to be marshaled over to unmanaged code, will need specifications in order for the interop marshaler to be able to successfully perform the marshaling. These specifications come in the form of MarshalAsAttributes.

    1.9 By "reference type", we mean types the actual data of which are allocated in the managed heap. They are reference counted internally and are de-allocated by the Garbage Collector. Each programing language variable that is assigned to a value of such a type merely holds a reference to that value. The variable does not get assigned another copy of the value.

    1.10 The opposite of a reference type is the value type. They reside on the stack. They are not managed by the Garbage Collector. They are destroyed when they go out of scope. Each programming variable that is assigned a value-type value gets its own copy of that value.

    1.11 Now, how are the concepts of blittability and reference and value types relevant to the "Object contains non-primitive or non-blittable data" error ? The following is a summary :

    1.11.1 When we want to share a managed class/struct with unmanaged code, a good optimization would be to directly pass the class/struct to the unmanaged code.

    1.11.2 This can only be done if the member data of the managed class/struct are all contained within the memory block of the class/struct itself. That is, the class/struct self-contains all member data just like a C/C++ class/struct.

    1.11.3 Now this is only possible if all members are blittable types. If any member is non-blittable, the entire class/struct is non-blittable.

    1.11.4 The advantage of a completely blittable class/struct is that such a class/struct may be directly passed to unmanaged code. Now in order to pass it to unmanaged code, it needs to be pinned in memory, otherwise, not only can it be moved around by the GC, it may even be completely de-allocated by the GC.

    1.11.5 The unmanaged code may then reference the class/struct directly and may even make modifications to it if this is expected. When the unmanaged code returns, the class/struct may be unpinned from memory.

    1.11.6 Now, if a class/struct is non-blittable, it will require the help of the interop marshaler to create in memory a complete copy of the class/struct.

    1.11.7 This copy must contain the actual data of all the members of the class/struct. It serves as a an unmanaged representation of the class/struct. The copy is made with the help of the various MarshalAsAttributes.

    1.11.8 It is this unmanaged representation that gets passed to the unmanaged code. Now, if the unmanaged code is expected to make modifications to this representation, then when the unmanaged code returns, the (now modified) data from the representation must be copied back to the original managed class/struct.

    1.11.9 Eventually, the unmanaged representation must be freed.

     

    2. The Members of the GatewayConnection Class.

    2.1 Now, let's examine the GatewayConnection class.

    This class contains 2 int members :

    public int NetworkID;
    public int OfflineEdit;

    and 2 managed array members :

    public ushort[] IpAddress;
    public ushort[] Password;

    2.2 In C#, the int type is a value type and is blittable. When used as members of a class/struct, it can directly reside as part of the memory block of the class/struct itself. Hence NetworkID and OfflineEdit form part of the body of the GatewayConnection class. They are destroyed when an instance of the GatewayConnection class is destroyed.

    2.3 Therefore, when we have code like the following :

         GatewayConnection objGWConn = new GatewayConnection();
         objGWConn.NetworkID = 0;
         objGWConn.OfflineEdit = 0;

    The values of members "NetworkID" and "OfflineEdit" (i.e. zero) reside directly as part of the memory block for "objGWConn". The situation is exactly the same as that for the members of a C/C++ class/struct.

    2.4 Now a managed array, when used as a member of a class or struct, renders the containing class/struct non-blittable. This is because a managed array is a reference type. That is, its actual array data exists in the managed heap and is merely referenced by the class/struct member.

    2.5 Therefore, when we have code like the following :

    objGWConn.IpAddress = new ushort[16];

    The array of 16 ushort values resides in the managed heap somewhere and is referenced by "objGWConn.IpAddress". The actual array data of 16 ushorts is not part of the memory block of the objGWConn object. This is totally unlike what we may be familiar with in C/C++.

    The situation is similarly so for member "Password".

    2.6 This is the reason why we cannot pin down in memory a class/struct that contains an array :

    GCHandle handle = GCHandle.Alloc(objGWConn, GCHandleType.Pinned); // Gives rise to error.

    In fact, we cannot pin a class/struct that contains members which are reference types including the string type.

    To pin such a class/struct would also require that the referenced object be pinned. This might seem acceptable initially until you consider code like the following :

    GatewayConnection objGWConn = new GatewayConnection();
    ushort [] ushort_array = new ushort[16];

    objGWConn.IpAddress = ushort_array; //new ushort[16];

    That is, "objGWConn.IpAddress" does not reference a newly allocated ushort array. Instead, it references an existing one. Pinning "objGWConn.IpAddress" would also logically require that "ushort_array" be pinned down. This may not be a desireable thing.

     

    3. How Do We Marshal the GatewayConnection Class to an Unmanaged API ?

    The following is an example of how this can be done. Note that there are simpler ways to accomplish the same objective but this example is meant to show some semblance of how the interop marshaler works under the covers :

    3.1 Allocate memory that provides an unmanaged representation of an instance of the GatewayConnection class (i.e. a GatewayConnection object).

    3.2 This unmanaged memory should be modeled after the equivalent of the GatewayConnection class in unmanaged memory. For example, the GatewayConnection class may be represented in C/C++ as follows :

    #pragma pack(1)
    struct GatewayConnection
    {
      unsigned short IpAddress[16];
      unsigned short Password[9];
      int NetworkID;
      int OfflineEdit;
    };

    Notice that all members of the struct are all packed together end-to-end in such an unmanaged representation of the GatewayConnection class. This is, of course, typical in C/C++ but is not the case for the managed GatewayConnection class. 

    3.3 Transfer the actual data of the members of the GatewayConnection object to this unmanaged memory.

    3.4 Call the unmanaged API passing along a pointer to the unmanaged memory.

    3.5 After the API call, if the API is to modify the unmanaged GatewayConnection struct, the returned data in the unmanaged GatewayConnection struct must be copied back to the managed GatewayConnection class.

    3.6 Finally, the unmanaged GatewayConnection struct must be freed.

    3.7 Why do we have to take such elaborate steps to pass a GatewayConnection object to the API ? Becuase the array members of the managed GatewayConnection object cannot be accessed by unmanaged code if you had simply passed a pointer to the object to the API.

    3.8 An example code, written in C#, is given in section 4 below. The following is a synopsis :

    3.8.1 The example code uses a DLL with a simple exported API named TestAPI().

    3.8.2 The C# code creates an instance of GatewayConnection.

    3.8.3 The code then creates an unmanaged representation of this object and copies all member data from the object to the unmanaged representation.

    3.8.4 A pointer to this unmanaged representation is passed to TestAPI() (the code for TestAPI() is also listed).

    3.8.5 As far as TestAPI() is concerned, the input unmanaged representation of the GatewayConnection object is a GatewayConnection struct which was listed in point 3.2 above.

    3.8.4 TestAPI() then modifies the struct and returns.

    3.8.5 When control returns to the C# code, we copy the now modified contents of the unmanaged struct back to the managed GatewayConnection object. The changes will be clearly seen.

    3.8.6 The unmanaged memory of the GatewayConnection struct which ws passed to TestAPI() is then destroyed.

     

    4. Example Code.

    4.1 To begin, note that you need to decorate the GatewayConnection class with the StructLayoutAttribute :

        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

    This is a necessary attribute in order that an unmanaged representation of the GatewayConnection class be creatable and the size of the unmanaged representation be calculatable. In particular, this attribute is necessary in order that a cdall to  Marshal.SizeOf() will succeed. More on this later.

     

    4.2 Now, in order to allocate memory that provides an unmanaged representation of an instance of the GatewayConnection class, we must perform the following in code :

    int iSize = Marshal.SizeOf(objGWConn);
    IntPtr p = IntPtr.Zero;
    p = Marshal.AllocHGlobal(iSize);
    Marshal.StructureToPtr(objGWConn, p, false);

    4.2.1 We first calculate the size of an unmanaged representation of objGWConn (an instance of GatewayConnection). This is done using Marshal.SizeOf().

    4.2.2 We then declare an IntPtr "p" which will point to this unmanaged representation of objGWConn.

    4.2.3 We then allocate in the unmanaged heap, a memory block the size of the unmanaged representation of objGWConn. This is done using Marshal.AllocHGlobal(). Note that "p" is used to point to this unmanaged memory.

    4.2.4 Finally, we copy the data of objGWConn (including the actual data of the array members) to the unmanaged representation. This is done by Marshal.StructureToPtr() with the help of the MarshalAsAttributes set for the "IpAddress" and "Password" members.

     

    4.3 After this, the unmanaged API may be invoked. The test API, written in C++ for this purpose, is listed below :

    void __stdcall TestAPI(/*[in, out]*/ GatewayConnection* pGatewayConnection)
    {
      for (int i = 0; i < 16; i++)
      {
        (pGatewayConnection -> IpAddress)[i] = i;
      }
     
      for (int i = 0; i < 9; i++)
      {
        (pGatewayConnection -> Password)[i] = i;
      }
     
      pGatewayConnection -> NetworkID = 101;
      pGatewayConnection -> OfflineEdit = 202;
    }

    I have intended that TestAPI() modify the fields of the GatewayConnection struct.

    The C# DLLImport declaration for TestAPI() is :

    [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
    private static extern void TestAPI(IntPtr pGatewayConnection);

     

    4.4 The call to the TestAPI() in C# is very simple :

    TestAPI(p);

     

    4.5 Now TestAPI() will permanently modify the member fields and so after the API call, we copy the now modified contents of the unmanaged representation of objGWConn, pointed to by "p", back to the managed "objGWConn" :

    objGWConn = (GatewayConnection)(Marshal.PtrToStructure(p, typeof(GatewayConnection)));

    This is done by using Marshal.PtrToStructure().

     

    4.6 After that we must free the unmanaged representation of objGWConn :

    Marshal.FreeHGlobal(p);

     

    4.7 Note that I have taken into account your concerns for ensuring that "objGWConn" remains untouched by the Garbage Collector. For this, note that you need not use the version of GCHandle.Alloc() that takes a GCHandleType parameter :

    GCHandle handle = GCHandle.Alloc(objGWConn, GCHandleType.Pinned);

    You may also use the single parameter version :

    GCHandle handle = GCHandle.Alloc(objGWConn);

    This also fixes objGWConn in the managed heap and disallows the GC from collecting it.

     

    4.8 A full source code listing is provided below for your reference :

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using System.Runtime.InteropServices;
    using System.Windows.Forms;

    namespace CSConsoleApp01
    {
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

        class Program
        {
            [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
            private static extern void TestAPI(IntPtr pGatewayConnection);

            static void Main(string[] args)
            {
                GatewayConnection objGWConn = new GatewayConnection();

                objGWConn.IpAddress = new ushort[16];
                objGWConn.IpAddress[0] = 10;
                objGWConn.Password = new ushort[9];
                objGWConn.Password[0] = 0;

                objGWConn.NetworkID = 0;
                objGWConn.OfflineEdit = 0;

                try
                {
                    // Pin down the byte array
                    GCHandle handle = GCHandle.Alloc(objGWConn);

                    // Calculate the size of the unmanaged representation of objGWConn.
                    int iSize = Marshal.SizeOf(objGWConn);
                    IntPtr p = IntPtr.Zero;
                    // Allocate memory for the unmanaged representation of objGWConn
                    // in the unmanaged global heap.
                    p = Marshal.AllocHGlobal(iSize);
                    // Copy the full contents of the members of the managed objGWConn
                    // to the unmanaged representation of objGWConn.
                    Marshal.StructureToPtr(objGWConn, p, false);

                    // Call the API.
                    TestAPI(p);

                    // Copy the modified contents of the unmanaged representation
                    // of objGWConn back to the members of the managed objGWConn.
                    objGWConn = (GatewayConnection)(Marshal.PtrToStructure(p, typeof(GatewayConnection)));
                    // Free the unmanaged representation of objGWConn.
                    Marshal.FreeHGlobal(p);

                    // Free the GCHandle.
                    handle.Free();
                }
                catch (Exception ex)
                {
                    MessageBox.Show(ex.Message);
                }
            }
        }
    }

     

    I hope you will benefit from this rather long post.

    - Bio. 

     



    • Proposed as answer by Paul Zhou Wednesday, June 1, 2011 2:17 AM
    • Marked as answer by Paul Zhou Tuesday, June 7, 2011 8:04 AM
    Tuesday, May 31, 2011 3:39 PM
  • Hi Bio,

    Thanks a lot for taking time and giving such a detailed explanation.It was very helpful ,Now i got understading of how to deal with reference types and Non - Blittable types.

    But i still have a question.

    I had the "GatewayConnection" structure declared like this,

    namespace CSConsoleApp01
    {

       [StructLayout(LayoutKind.Sequential)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

    I was creating the instance of  "GatewayConnection" class and passing this instance to the Unmanaged API(declared in Class Main) ,like this

     Public Class Main

    {

          /*Unmanaged API ,which is expecting instance of gateway object by referece

        [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
         private static extern void TestAPI(ref GatewayConnection gwConn);

    //Create instance of structure

      GatewayConnection objGWConn = new GatewayConnection();

      objGWConn.IpAddress = new ushort[16];
      objGWConn.IpAddress[0] = 10;
      objGWConn.Password = new ushort[9];
      objGWConn.Password[0] = 0; 

      objGWConn.NetworkID = 0;
      objGWConn.OfflineEdit = 0;

      // Call the API.
      TestAPI(ref objGWConn);

    }

    }

    In this case the call to unmanged Api was failing( which is intended ) i.e when i debug from backend i was getting junk values in the structure..

    =================================================================================

    However when i moved the "GatewayConnection" class definition to the class where the call to unmanged API was made,as shown below.

     

    Class Main

    {

        [StructLayout(LayoutKind.Sequential)]
        public class GatewayConnection
        {
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
            public ushort[] IpAddress;
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
            public ushort[] Password;
            public int NetworkID;
            public int OfflineEdit;
        }

       /*Unmanaged API ,which is expecting instance of gateway object by referece

        [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
        private static extern void TestAPI(ref GatewayConnection gwConn);

     //Create instance.

      GatewayConnection objGWConn = new GatewayConnection();

      objGWConn.IpAddress = new ushort[16];
      objGWConn.IpAddress[0] = 10;
      objGWConn.Password = new ushort[9];
      objGWConn.Password[0] = 0; 

      objGWConn.NetworkID = 0;
      objGWConn.OfflineEdit = 0;

      // Call the API.
      TestAPI(objGWConn);

    };

    The call to api was sucessful and i got the intended results(without even pinning the structure object).

    Want to understand how declaring "GatewayConnection" class in the same class where call to API is made makes difference.

      ==================================================================================

     

    2.The unmanaged API i am using is expecting an instance of the GatewayConnection class passed as

    reference.

    How can i pass the GatewayConnection class instance as reference after pinning the instance.

     Please provide your inputs on this.

    Once again Thank You

    Regards

    Vinutha K

     

     

    • Marked as answer by Paul Zhou Tuesday, June 7, 2011 8:04 AM
    • Unmarked as answer by Paul Zhou Tuesday, June 7, 2011 8:04 AM
    Thursday, June 2, 2011 7:14 AM
  • Hello vinutha kempanna,

     

    1. Concerning Appearance of Junk Values.

    1.1 There are 2 reasons for the appearance of junk values :

    1.1.1 Struct member alignment.

    1.1.2 Declaration of the "gwConn" parameter for TestAPI().

    In my opinion, though I could be wrong, the location where the GatewayConnection class is declared, whether inside the class Main or outside it, makes no difference. Any difference you observed is probably coincidental.

     

    1.2 The struct member alignment for the GatewayConnection class must match its unmanaged counterpart.

    Hence, in the C# code, GatewayConnection must be declared with the "Pack" argument for the StructLayoutAttribute as follows :

    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    public class GatewayConnection
    {
      [MarshalAs(UnmanagedType.ByValArray, SizeConst = 16)]
      public ushort[] IpAddress;
      [MarshalAs(UnmanagedType.ByValArray, SizeConst = 9)]
      public ushort[] Password;
      public int NetworkID;
      public int OfflineEdit;
    }

    And the counterpart C/C++ GatewayConnection struct must be declared using #pragma pack() or the project settings must be set for appropriate struct member alignment :

    #pragma pack(1)
    struct GatewayConnection
    {
      unsigned short IpAddress[16];
      unsigned short Password[9];
      int NetworkID;
      int OfflineEdit;
    };

    I have used the value 1 for the struct member alignment. This is done for simplicity. I suggest that you set it to 1 for your project until the problems are resolved. Thereafter you may look into optimizing the struct member alignment.

     

    1.3 Now you have declared TestAPI() in the C# code as :

    [DllImport("KempannaDLL.dll", CallingConvention = CallingConvention.StdCall)]
    private static extern void TestAPI(ref GatewayConnection gwConn);

    The parameter gwConn has been declared as a "ref" parameter. Because of this, the counterpart parameter for "gwConn" in the C/C++ definition for TestAPI() must be a double pointer to the GatewayConnection struct, e.g. :

    void __stdcall TestAPI(GatewayConnection** pGatewayConnection)
    {
        ...
    }

     

    2. Concerning Pinning.

    2.1 Note that there is no need to pin the C# GatewayConnection class. In fact, as I have mentioned in my last post dated May 31st, it is not possible to memory-pin an instance of the GatewayConnection class because it contains 2 arrays (arrays are reference types). Please refer to section 2 of my May 31st post.

    However, to ensure that the instance of the GatewayConnection class remain in memory, all that is required is to use the GCHandle class to ensure that the instance is not collected by the Garbage Collector when TestAPI() is called , e.g. :

    GatewayConnection objGWConn = new GatewayConnection();
    // Allocate a GCHandle for objGWConn so that objGWConn
    // does not get Garbage Collected when TestAPI() is called.
    GCHandle gch = GCHandle.Alloc(objGWConn);
     
    objGWConn.IpAddress = new ushort[16];
    objGWConn.IpAddress[0] = 10;
    objGWConn.Password = new ushort[9];
    objGWConn.Password[0] = 20; 
    objGWConn.NetworkID = 30;
    objGWConn.OfflineEdit = 40;
               
    // Call the API.
    TestAPI(ref objGWConn);

    gch.Free();

     

    2.2 In actual fact, because the managed GatewayConnection class has been adequately attributed with the StructLayoutAttribute and the MarshalAsAttribute's, the interop marshaler will know how to marshal the class across to unmanaged code and back.

     

    2.3 Note that under the covers, when TestAPI() is called, the serialization of the managed GatewayConnection class object into its unmanaged representation and its deserialization from the unmanaged representation back into the managed GatewayConnection class object are performed by the interop assembler.

     

    2.4 This process is similar to how I have described it in section 3 and section 4 of my post to you dated Tuesday May 31st.

     

    - Bio.

     

    Thursday, June 2, 2011 5:38 PM
  •  

    Hi Vinutha,

     

    Has your issue been resolved? Would you mind letting us know the result of the suggestions?

     

    Now I will mark an answer, you can mark others that you think to be so useful to your issue.

    You could “Unmark As Answer” if it doesn’t make sense.

     

    Have a nice day!


    Paul Zhou [MSFT]
    MSDN Community Support | Feedback to us
    Get or Request Code Sample from Microsoft
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Tuesday, June 7, 2011 8:04 AM