none
Using CustomMarshaler RRS feed

  • Question

  • Hello,
    I can't figure out how to read user defined struct from binary stream. I prepared class which implements required conversion. But my example doesn't work. The code without CustomMarshaler class works well. My example throws the following exception. Could you please help me?


    Cannot marshal field 'Stamp' of type 'Rsj.Cat2.Adapters.Icap.Adapter.Sructures.StampInfo': Invalid managed/unmanaged type combination (the DateTime class must be paired with Struct).


    [StructLayout(LayoutKind.Sequential, Pack = 1)]
    struct StampInfo
    {
    	public byte Position;
    
    	[MarshalAs(UnmanagedType.ByValTStr, SizeConst = 20)]
    	public string Id;
    
    	// this is real native data type
    	//public int Seconds;
    
    	// this is .NET wanted data type
    	[MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(ExpirationDateMarshaler))]
    	public DateTime Stamp;
    }
    
    class ExpirationDateMarshaler : ICustomMarshaler
    {
    	private const int NativeDataSize = 4;
    	private static ICustomMarshaler marshaler = new ExpirationDateMarshaler();
    
    	public static ICustomMarshaler GetInstance(string cookie)
    	{
    		return marshaler;
    	}
    
    	public void CleanUpManagedData(object ManagedObj)
    	{
    	}
    
    	public void CleanUpNativeData(IntPtr pNativeData)
    	{
    		Marshal.Release(pNativeData);
    	}
    
    	public int GetNativeDataSize()
    	{
    		return NativeDataSize;
    	}
    
    	public IntPtr MarshalManagedToNative(object ManagedObj)
    	{
    		throw new NotImplementedException();
    	}
    
    	public object MarshalNativeToManaged(IntPtr pNativeData)
    	{
    		int seconds = (int) Marshal.ReadInt32(pNativeData);
    		return SecToDateTime(seconds);
    	}
    
    	private DateTime SecToDateTime(int seconds)
    	{
    		return new DateTime(2000, 1, 1).AddSeconds(seconds);
    	}
    }
    
    class Example
    {
    	public void Run()
    	{
    		IntPtr buffer = Marshal.AllocHGlobal(100);
    		Test(buffer);
    		Marshal.FreeHGlobal(buffer);
    	}
    
    	public void Test(IntPtr ptr)
    	{			
    		StampInfo stampInfo = Read<StampInfo>(ptr);
    	}
    
    	private T Read<T>(IntPtr ptr)
    	{
    		return (T) Marshal.PtrToStructure(ptr, typeof(T));
    	}
    }
    
    






    Wednesday, January 25, 2012 8:34 AM

Answers

  • Hello Martin,

    1. Thanks for the confirmation.

    2. In this case, Martin, the use of a custom marshaler (i.e. ExpirationDateMarshaler) on the StampInfo.Stamp field will not do. The interop marshaler does not allow the use of custom marshaling for fields of a structure, unfortunately.

    3. The following is an alternative approach which I recommend for you :

    • Create an unmanaged exported API named something like ConvertStreamToStampInfo() the purpose of which is to create an unmanaged StampInfo structure from the bytes of a stream.
    • The byte stream is to be an input parameter.
    • The newly created unmanaged StampInfo structure will be returned out of this API (as an "out" parameter).
    • An example for the ConvertStreamToStampInfo() API is provided in section 4 below.
    • DllImport this API in C# and declare that the "out" unmanaged StampInfo structure will be marshaled using a custom marshaler.
    • The custom marshaler will not be ExpirationDateMarshaler.
    • Rather, it should be a marshaler that converts an unmanaged StampInfo structure to its managed counterpart called something like StampInfoMarshaler (an example is given in section 5 below). 

    4. An Example ConvertStreamToStampInfo() API Implementation.

    4.1 The following is a sample implementation for ConvertStreamToStampInfo() in C++ :

    extern "C" __declspec(dllexport) void __stdcall ConvertStreamToStampInfo
    (
      /*[in]*/ LPBYTE lpbyStream, 
      /*[out]*/ StampInfo** ppstamp_info_receiver
    )
    {
    	// Allocate a StampInfo structure using GlobalAlloc().
    	// The counterpart method in managed code to free this
    	// memory later on is Marshal.FreeHGlobal().
    	//
    	// Another function you can use is CoTaskMemAlloc().
    	// In this case, you will use Marshal.FreeCoTaskMem().
    	//
    	// DO NOT USE "new" or malloc(). There is no counterpart
    	// managed function to free memory allocated using "new"
    	// or malloc(). This is because "new" is C++ compiler 
    	// dependent and malloc() is C-library dependent.
    	//
    	// GlobalAlloc() and CoTaskMemAlloc() on the other hand,
    	// are standard APIs with conjugate freeing APIs.
    	*ppstamp_info_receiver = (StampInfo*)GlobalAlloc(GMEM_FIXED, sizeof(StampInfo));
    	
    	// You should use the data inside the byte stream "lpbyStream"
    	// to fill in the values for the StampInfo structure.
    	// I hardcoded the values for the structure as an example.
    	(*ppstamp_info_receiver) -> byPosition = 100;
    	strcpy((*ppstamp_info_receiver) -> szId, "Hello World.");
    	(*ppstamp_info_receiver) -> iSeconds = 600;
    }
    
    

    Please read carefully the comments on memory allocation.

    5. An Example Custom Marshaler StampInfoMarshaler.

    5.1 The following is a sample implementation for StampInfoMarshaler, a custom marshaler that converts an unmanaged StampInfo structure to its managed counterpart :

    class StampInfoMarshaler : ICustomMarshaler
    {
        // Define an unmanaged version of the StampInfo
        // structure. Note that it contains the "Seconds"
        // field which is an integer.
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        private struct NativeStampInfo
        {
            public byte Position;
    
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 20)]
            public string Id;
    
            // this is real native data type
            public int Seconds;
        }
    
        private static ICustomMarshaler marshaler = new StampInfoMarshaler();
    
        public static ICustomMarshaler GetInstance(string cookie)
        {
            return marshaler;
        }
    
        public void CleanUpManagedData(object ManagedObj)
        {
        }
    
        public void CleanUpNativeData(IntPtr pNativeData)
        {
            // Since we know that the ConvertStreamToStampInfo()
            // API used GlobalAlloc() to allocate the memory 
            // for the unmanaged StampInfo structure, we must
            // use Marshal.FreeHGlobal() to free the memory.
            Marshal.FreeHGlobal(pNativeData);
        }
    
        public int GetNativeDataSize()
        {
            return Marshal.SizeOf(typeof(NativeStampInfo));
        }
    
        public IntPtr MarshalManagedToNative(object ManagedObj)
        {
            throw new NotImplementedException();
        }
    
        public object MarshalNativeToManaged(IntPtr pNativeData)
        {
            // We first convert the native data (which already
            // contains the unmanaged StampInfo structure) to
            // the NativeStampInfo structure.
            NativeStampInfo native_stamp_info = (NativeStampInfo)Marshal.PtrToStructure(pNativeData, typeof(NativeStampInfo));
    
            // We then create a managed StampInfo structure.
            StampInfo stamp_info_ret = new StampInfo();
            // And fill it will field values from the native_stamp_info
            // stucture.
            stamp_info_ret.Id = native_stamp_info.Id;
            stamp_info_ret.Position = native_stamp_info.Position;
            // We use the SecToDateTime() helper function to
            // convert the "Seconds" field to a DateTime
            // object.
            stamp_info_ret.Stamp = SecToDateTime(native_stamp_info.Seconds);
            return stamp_info_ret;
        }
    
        private DateTime SecToDateTime(int seconds)
        {
            return new DateTime(2000, 1, 1).AddSeconds(seconds);
        }
    }
    
    

    6. Example Usage of the ConvertStreamToStampInfo() API.

    6.1 The following is a DllIMport declaration for ConvertStreamToStampInfo() :

    [DllImport("TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
    public static extern void ConvertStreamToStampInfo
        (
          [In] IntPtr pStream, 
          [Out][MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(StampInfoMarshaler))] 
          out StampInfo stampInfo
        );
    
    

    6.2 And a sample call :

    public void Run2()
    {
        IntPtr buffer = Marshal.AllocHGlobal(100);
        StampInfo stampInfo;
        ConvertStreamToStampInfo(buffer, out stampInfo);
        Marshal.FreeHGlobal(buffer);
    }
    
    

    The conversion from the byte stream to the managed StampInfo structure is done all within the call to ConvertStreamToStampInfo(). The unmanaged StampInfo structure is only visible and used within the StampInfoMarshaler class.

    - Bio.

     


    Please visit my blog : http://limbioliong.wordpress.com/
    Tuesday, January 31, 2012 9:25 AM

All replies

  • Well, I can't see how the Test() method can succeed if you give it only 10 bytes.  The struct looks like it is 25 bytes, so I find it strange that you test with 10 bytes only.

    But I don't think that's the problem here because of the error message.  I looked up the ICustomMarshaler interface and for the looks of it, it can only be used to marshal COM interfaces.  So basically it seems that you cannot use a custom marshaler the way you are trying to use it.


    Jose R. MCP
    Thursday, January 26, 2012 1:15 AM
  • I corrected the buffer size.
    Thursday, January 26, 2012 8:49 AM
  • Hello Martin,

    1. After examining the code comments :

    // this is real native data type
    //public int Seconds;
    
    // this is .NET wanted data type
    ...
    
    

    contained in the StampInfo structure, and reading the code for the ExpirationDateMarshaler custom marshaler, my impression is that you are actually interacting with unmanaged code in which a counterpart StampInfo structure is defined.

    2. It is also my impression that in the unmanaged StampInfo structure, there is a field named "Seconds" of type int. The managed counterpart of this field is "Stamp" of type DateTime.

    3. I also believe that the managed code receives a stream from the unmanaged side and this stream contains the StampInfo structure in unmanaged format (i.e. with the "Seconds" int field).

    4. Your target is to convert the unmanaged structure to its managed counterpart, converting the "Seconds" field to "Stamp" along the way.

    5. Are the theories above correct ?

     

    - Bio.

     


    Please visit my blog : http://limbioliong.wordpress.com/
    Tuesday, January 31, 2012 7:25 AM
  • Yes, you are absolutely correct.
    Tuesday, January 31, 2012 8:46 AM
  • Hello Martin,

    1. Thanks for the confirmation.

    2. In this case, Martin, the use of a custom marshaler (i.e. ExpirationDateMarshaler) on the StampInfo.Stamp field will not do. The interop marshaler does not allow the use of custom marshaling for fields of a structure, unfortunately.

    3. The following is an alternative approach which I recommend for you :

    • Create an unmanaged exported API named something like ConvertStreamToStampInfo() the purpose of which is to create an unmanaged StampInfo structure from the bytes of a stream.
    • The byte stream is to be an input parameter.
    • The newly created unmanaged StampInfo structure will be returned out of this API (as an "out" parameter).
    • An example for the ConvertStreamToStampInfo() API is provided in section 4 below.
    • DllImport this API in C# and declare that the "out" unmanaged StampInfo structure will be marshaled using a custom marshaler.
    • The custom marshaler will not be ExpirationDateMarshaler.
    • Rather, it should be a marshaler that converts an unmanaged StampInfo structure to its managed counterpart called something like StampInfoMarshaler (an example is given in section 5 below). 

    4. An Example ConvertStreamToStampInfo() API Implementation.

    4.1 The following is a sample implementation for ConvertStreamToStampInfo() in C++ :

    extern "C" __declspec(dllexport) void __stdcall ConvertStreamToStampInfo
    (
      /*[in]*/ LPBYTE lpbyStream, 
      /*[out]*/ StampInfo** ppstamp_info_receiver
    )
    {
    	// Allocate a StampInfo structure using GlobalAlloc().
    	// The counterpart method in managed code to free this
    	// memory later on is Marshal.FreeHGlobal().
    	//
    	// Another function you can use is CoTaskMemAlloc().
    	// In this case, you will use Marshal.FreeCoTaskMem().
    	//
    	// DO NOT USE "new" or malloc(). There is no counterpart
    	// managed function to free memory allocated using "new"
    	// or malloc(). This is because "new" is C++ compiler 
    	// dependent and malloc() is C-library dependent.
    	//
    	// GlobalAlloc() and CoTaskMemAlloc() on the other hand,
    	// are standard APIs with conjugate freeing APIs.
    	*ppstamp_info_receiver = (StampInfo*)GlobalAlloc(GMEM_FIXED, sizeof(StampInfo));
    	
    	// You should use the data inside the byte stream "lpbyStream"
    	// to fill in the values for the StampInfo structure.
    	// I hardcoded the values for the structure as an example.
    	(*ppstamp_info_receiver) -> byPosition = 100;
    	strcpy((*ppstamp_info_receiver) -> szId, "Hello World.");
    	(*ppstamp_info_receiver) -> iSeconds = 600;
    }
    
    

    Please read carefully the comments on memory allocation.

    5. An Example Custom Marshaler StampInfoMarshaler.

    5.1 The following is a sample implementation for StampInfoMarshaler, a custom marshaler that converts an unmanaged StampInfo structure to its managed counterpart :

    class StampInfoMarshaler : ICustomMarshaler
    {
        // Define an unmanaged version of the StampInfo
        // structure. Note that it contains the "Seconds"
        // field which is an integer.
        [StructLayout(LayoutKind.Sequential, Pack = 1)]
        private struct NativeStampInfo
        {
            public byte Position;
    
            [MarshalAs(UnmanagedType.ByValTStr, SizeConst = 20)]
            public string Id;
    
            // this is real native data type
            public int Seconds;
        }
    
        private static ICustomMarshaler marshaler = new StampInfoMarshaler();
    
        public static ICustomMarshaler GetInstance(string cookie)
        {
            return marshaler;
        }
    
        public void CleanUpManagedData(object ManagedObj)
        {
        }
    
        public void CleanUpNativeData(IntPtr pNativeData)
        {
            // Since we know that the ConvertStreamToStampInfo()
            // API used GlobalAlloc() to allocate the memory 
            // for the unmanaged StampInfo structure, we must
            // use Marshal.FreeHGlobal() to free the memory.
            Marshal.FreeHGlobal(pNativeData);
        }
    
        public int GetNativeDataSize()
        {
            return Marshal.SizeOf(typeof(NativeStampInfo));
        }
    
        public IntPtr MarshalManagedToNative(object ManagedObj)
        {
            throw new NotImplementedException();
        }
    
        public object MarshalNativeToManaged(IntPtr pNativeData)
        {
            // We first convert the native data (which already
            // contains the unmanaged StampInfo structure) to
            // the NativeStampInfo structure.
            NativeStampInfo native_stamp_info = (NativeStampInfo)Marshal.PtrToStructure(pNativeData, typeof(NativeStampInfo));
    
            // We then create a managed StampInfo structure.
            StampInfo stamp_info_ret = new StampInfo();
            // And fill it will field values from the native_stamp_info
            // stucture.
            stamp_info_ret.Id = native_stamp_info.Id;
            stamp_info_ret.Position = native_stamp_info.Position;
            // We use the SecToDateTime() helper function to
            // convert the "Seconds" field to a DateTime
            // object.
            stamp_info_ret.Stamp = SecToDateTime(native_stamp_info.Seconds);
            return stamp_info_ret;
        }
    
        private DateTime SecToDateTime(int seconds)
        {
            return new DateTime(2000, 1, 1).AddSeconds(seconds);
        }
    }
    
    

    6. Example Usage of the ConvertStreamToStampInfo() API.

    6.1 The following is a DllIMport declaration for ConvertStreamToStampInfo() :

    [DllImport("TestDLL.dll", CallingConvention = CallingConvention.StdCall)]
    public static extern void ConvertStreamToStampInfo
        (
          [In] IntPtr pStream, 
          [Out][MarshalAs(UnmanagedType.CustomMarshaler, MarshalTypeRef = typeof(StampInfoMarshaler))] 
          out StampInfo stampInfo
        );
    
    

    6.2 And a sample call :

    public void Run2()
    {
        IntPtr buffer = Marshal.AllocHGlobal(100);
        StampInfo stampInfo;
        ConvertStreamToStampInfo(buffer, out stampInfo);
        Marshal.FreeHGlobal(buffer);
    }
    
    

    The conversion from the byte stream to the managed StampInfo structure is done all within the call to ConvertStreamToStampInfo(). The unmanaged StampInfo structure is only visible and used within the StampInfoMarshaler class.

    - Bio.

     


    Please visit my blog : http://limbioliong.wordpress.com/
    Tuesday, January 31, 2012 9:25 AM
  • Thank You for very detailed explanation. For me is most important information, that  I can't use User Marshaler for fields of a structure.

    I have some simplier solution for my particular problem.

    Tuesday, January 31, 2012 10:08 AM