none
P/Invoke with struct and WCHAR RRS feed

  • Question

  • Hi there.

     

     I run into strange problem with P/Invoke and hope someone could explain it to me. I have unmanaged DLL written in C++ (I have full control on source code and can recompile it and do modification). I'm trying to Marshall array of structures from C++ DLL into C#. At some point development I re-write C# declaration of struct as Explicit and add FieldOffset attributes - after this everything gone bad :( Here is declaration of C++ struct:

     

     

    #pragma pack(push)
    #pragma pack(2)
    
    typedef
     struct
     _MY_STRUCT 
    {
    	int
     integer1;
    	int
     integer2;
    	int
     integer3;
    	WCHAR someString1 [13];
    	int
     integer4;
    	int
     integer5;
    	int
     integer6;
    	int
     integer7;	
    	WCHAR someString2 [29];
    	int
     integer8;
    	WCHAR someString3 [16];
    	int
     integer9;	
    }MY_STRUCT , *PMY_STRUCT ;
    
    #pragma pack(pop)
    
    

     

    and this is C# defenition:

     

            [StructLayoutAttribute(LayoutKind.Explicit, CharSet = CharSet.Auto, Pack = 2)]
            public
     struct
     MY_STRUCT
            {
                [FieldOffset(0), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer1;
    
                [FieldOffset(4), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer2;
    
                [FieldOffset(8), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer3;
    
    
                [FieldOffset(12), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 13)]
                public
     string
     SomeString1;
    
                [FieldOffset(38), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer4;
    
                [FieldOffset(42), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer5;
    
                [FieldOffset(46), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer6;
    
                [FieldOffset(50), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer7;
    
                [FieldOffset(54), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 29)]
                public
     string
     SomeString2;
    
                [FieldOffset(114), MarshalAs(UnmanagedType.U4)]
                public
     int
     integer8;
    
                [FieldOffset(116), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 16)]
                public
     string
     SomeString3;
    
         <pre lang=x-cpp>      [FieldOffset(148), MarshalAs(UnmanagedType.U4)]
    
    public int integer9; }

     

    Actual API call:

    C++: 

    __declspec( dllexport ) GET_DATA(PMY_STRUCT structInfoList, int* sizeOfStructInfoList);

     

    C#:

     

    [DllImportAttribute(FULL_DLL_FILE_NAME, EntryPoint = "GET_DATA"
    , SetLastError = true
    , CharSet = CharSet.None, CallingConvention = CallingConvention.Cdecl)]
    public
     static
     extern
     int
     GET_DATA([In, Out]MY_STRUCT [] CardInfo, ref
     int
     Number);
    
    

    For some reason C# trow exception of TypeLoad with description that 'SomeString2' not correctly alligned at offset 54. 

    Workaround that I found is change offset for SomeString2 from 54 to 56 but I still can not understand why ? and if I do so - I cut off the first character in unmanaged SomeStirng2. I can change defenition from 'string' to 'int'  (public int SomeString2; and Marshall attribute) and yes - I can see correct value been passing and receving as 'int' value'. After long time I fugred that C# just wont accept anything 'text' in place of SomeString2.  char array instead SomeString2 - wont' go, 'string' - wont'g go, but numeric types is welcomed ! SomeString2 could be replaced by 'integer' but not byte []'...

     

    So - could someone explain where I did wrong ?

     

    P.S. This is strange but with similar conditions I have an error again (this time with BOOL value) and again - right in the middle of stuct. Strange... If you take unmanaged struct, calculate it size, devide in half = offset from the beggining to the middle of struct - you can not put anything in this offset but integer.... Strange...

     


    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]
    • Edited by Jasper22 Tuesday, March 23, 2010 2:42 PM New information
    Tuesday, March 23, 2010 1:00 PM

Answers

  • Nope... sorry - it doesn't even compile - error is:

    Pointers and fixed size buffers may only be used in an unsafe context (at line where    'public fixed char SomeString1[13]' )

    and even if I try to replace it to:

     

    [FieldOffset(54), MarshalAs(UnmanagedType.ByValArray, SizeConst = 29, ArraySubType=UnmanagedType.I2)]
    
    public char [] SomeString2;
    
    

     

    it still throws exception TypeLoader and only at Offset(54). The first 'SomeString1' can be passed as whatever fits, as array - no problem, as string - no problem, as int, char, bool, anything - everything ok. Only at offset 54 happens some strange moon-phase error.....

    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]


    Remove all attributes from the struct defintion (and do the same to the C or C++ version, let everything default)

    Make sure you use the "usnafe" term in the struct declaration.

    Ensure the project setting (build) is set to allow unsafe code.

    Your problem is solved, I've done this a very great deal for several years, this is the best solution, if you can't get it to work, get back to us here, it will be something minor.

    There is absolutely no need for any attributes with this approach, the struct is physically identical on both sides of the call, so debugging is MUCH easier. Your current TypeLoader error might be due to some fields overlapping, this is a risk you run when you specify your own field offsets.

    All those attributes are used solely by Marshal.StructureToPtr and Marshal.PtrToStructure. It uses them to dynamically define another (hidden) struct that you never see. It then copies everything from the input struct to this temp one and then it passes a ptr to this temp struct into your C/C++ code.

    You may not be aware that those attributes have no impact at all on how C# lays out the fields in the managed struct, they are totally ignored insofar as managed declarations go. In your case, the struct is perfectly suitable as blittable and in this case; when you do it this way it is very fast too, because the C# code can just pass a pointer to the managed struct (it will pin the struct in this case to stop it being moved) and your called function is then able to directly modify the fields inside the actual managed struct.

    None of this happens if you add those attributes and it gets slow, fiddly and error prone.

    You don't need any attributes for this, believe me.

    Cap'n

     

    • Edited by Captain Kernel Wednesday, March 24, 2010 12:22 PM expand
    • Marked as answer by Jasper22 Wednesday, March 24, 2010 12:42 PM
    Wednesday, March 24, 2010 12:07 PM

All replies

  • Hi there.

     

     I run into strange problem with P/Invoke and hope someone could explain it to me. I have unmanaged DLL written in C++ (I have full control on source code and can recompile it and do modification). I'm trying to Marshall array of structures from C++ DLL into C#. At some point development I re-write C# declaration of struct as Explicit and add FieldOffset attributes - after this everything gone bad :( Here is declaration of C++ struct:

     

     

    #pragma pack(push)
    
    #pragma pack(2)
    
    
    
    typedef
    
     struct
    
     _MY_STRUCT 
    
    {
    
    	int
    
     integer1;
    
    	int
    
     integer2;
    
    	int
    
     integer3;
    
    	WCHAR someString1 [13];
    
    	int
    
     integer4;
    
    	int
    
     integer5;
    
    	int
    
     integer6;
    
    	int
    
     integer7;	
    
    	WCHAR someString2 [29];
    
    	int
    
     integer8;
    
    	WCHAR someString3 [16];
    
    	int
    
     integer9;	
    
    }MY_STRUCT , *PMY_STRUCT ;
    
    
    
    #pragma pack(pop)
    
    
    
    

     

    and this is C# defenition:

     

            [StructLayoutAttribute(LayoutKind.Explicit, CharSet = CharSet.Auto, Pack = 2)]
    
            public
    
     struct
    
     MY_STRUCT
    
            {
    
                [FieldOffset(0), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer1;
    
    
    
                [FieldOffset(4), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer2;
    
    
    
                [FieldOffset(8), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer3;
    
    
    
    
    
                [FieldOffset(12), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 13)]
    
                public
    
     string
    
     SomeString1;
    
    
    
                [FieldOffset(38), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer4;
    
    
    
                [FieldOffset(42), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer5;
    
    
    
                [FieldOffset(46), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer6;
    
    
    
                [FieldOffset(50), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer7;
    
    
    
                [FieldOffset(54), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 29)]
    
                public
    
     string
    
     SomeString2;
    
    
    
                [FieldOffset(114), MarshalAs(UnmanagedType.U4)]
    
                public
    
     int
    
     integer8;
    
    
    
                [FieldOffset(116), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 16)]
    
                public
    
     string
    
     SomeString3;
    
    
    
         <pre lang=x-cpp>      [FieldOffset(148), MarshalAs(UnmanagedType.U4)]
    
    
    public int integer9; }

     

    Actual API call:

    C++: 

     

     

    C#:

     

    [DllImportAttribute(FULL_DLL_FILE_NAME, EntryPoint = "GET_DATA"
    
    , SetLastError = true
    
    , CharSet = CharSet.None, CallingConvention = CallingConvention.Cdecl)]
    
    public
    
     static
    
     extern
    
     int
    
     GET_DATA([In, Out]MY_STRUCT [] CardInfo, ref
    
     int
    
     Number);
    
    
    
    

    For some reason C# trow exception of TypeLoad with description that 'SomeString2' not correctly alligned at offset 54. 

    Workaround that I found is change offset for SomeString2 from 54 to 56 but I still can not understand why ? and if I do so - I cut off the first character in unmanaged SomeStirng2. I can change defenition from 'string' to 'int'  (public int SomeString2; and Marshall attribute) and yes - I can see correct value been passing and receving as 'int' value'. After long time I fugred that C# just wont accept anything 'text' in place of SomeString2.  char array instead SomeString2 - wont' go, 'string' - wont'g go, but numeric types is welcomed ! SomeString2 could be replaced by 'integer' but not byte []'...

     

    So - could someone explain where I did wrong ?

     

    P.S. This is strange but with similar conditions I have an error again (this time with BOOL value) and again - right in the middle of stuct. Strange... If you take unmanaged struct, calculate it size, devide in half = offset from the beggining to the middle of struct - you can not put anything in this offset but integer.... Strange...

     


    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]

    __declspec( dllexport ) GET_DATA(PMY_STRUCT structInfoList, int* sizeOfStructInfoList);

    OK I suggest you stop using "String" and the marshaller this way.

    Try the following C# struct:

            [StructLayoutAttribute(LayoutKind.Explicit, CharSet = CharSet.Auto, Pack = 2)]
            public unsafe struct MY_STRUCT
            {
                [FieldOffset(0), MarshalAs(UnmanagedType.U4)]
                public int integer1;
    
                [FieldOffset(4), MarshalAs(UnmanagedType.U4)]
                public int integer2;
    
                [FieldOffset(8), MarshalAs(UnmanagedType.U4)]
                public int integer3;
    
                //[FieldOffset(12), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 13)]
                //public string SomeString1;
                public fixed Char SomeString1[13];
    
                [FieldOffset(38), MarshalAs(UnmanagedType.U4)]
                public int integer4;
                
                [FieldOffset(42), MarshalAs(UnmanagedType.U4)]
                public int integer5;
    
                [FieldOffset(46), MarshalAs(UnmanagedType.U4)]
                public int integer6;
    
                [FieldOffset(50), MarshalAs(UnmanagedType.U4)]
                public int integer7;
    
                //[FieldOffset(54), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 29)]
                //public string SomeString2;
                public fixed Char SomeString2[29];
    
                [FieldOffset(114), MarshalAs(UnmanagedType.U4)]
                public  int integer8;
    
                //[FieldOffset(116), MarshalAs(UnmanagedType.ByValTStr, SizeConst = 16)]
                //public string SomeString3;
                public fixed Char SomeString3[16];
                
                [FieldOffset(148), MarshalAs(UnmanagedType.U4)]
                public int integer9; }     
        
           }
    

    How does that behave?

    You probably don't need all of those marshaller attributes either, often one can create a 'blittable' struct which has identical managed/unmanaged layout (the one I posted is blittable IF you remove ALL marshaller attributes). A blittable struct is much easier to work with, when you pass one by ref you get a pointer to a struct that 'looks' the same to the managed and native sides of the call.

    In addition, if you want you can add some properties to the C# struct that allow a string to be used to get/set the Char buffers, such properties in structs have no impact on marshalling and will not break any code.

    How does that look?

    Cap'n

     

     

     

     

     

    Tuesday, March 23, 2010 4:37 PM
  • Nope... sorry - it doesn't even compile - error is:

    Pointers and fixed size buffers may only be used in an unsafe context (at line where    'public fixed char SomeString1[13]' )

    and even if I try to replace it to:

    [FieldOffset(54), MarshalAs(UnmanagedType.ByValArray, SizeConst = 29, ArraySubType=UnmanagedType.I2)]
    public char [] SomeString2;

    it still throws exception TypeLoader and only at Offset(54). The first 'SomeString1' can be passed as whatever fits, as array - no problem, as string - no problem, as int, char, bool, anything - everything ok. Only at offset 54 happens some strange moon-phase error.....

    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]
    Wednesday, March 24, 2010 7:40 AM
  • Maybe you forgot to add the unsafe modifier to the struct? Or to enable using unsafe code in your project properties?

    And in case you don't want to use unsafe code:

    For a struct with explicit layout, you must manually ensure that object references (such as your string members) end up on a natural boundary (an offset that's a multiple of 4 or 8 bytes depending on platform). So you'd have to adjust the size of your strings accordingly or change the packing of the struct.


    Mattias, C# MVP
    Wednesday, March 24, 2010 10:15 AM
    Moderator
  • Nope... sorry - it doesn't even compile - error is:

    Pointers and fixed size buffers may only be used in an unsafe context (at line where    'public fixed char SomeString1[13]' )

    and even if I try to replace it to:

     

    [FieldOffset(54), MarshalAs(UnmanagedType.ByValArray, SizeConst = 29, ArraySubType=UnmanagedType.I2)]
    
    public char [] SomeString2;
    
    

     

    it still throws exception TypeLoader and only at Offset(54). The first 'SomeString1' can be passed as whatever fits, as array - no problem, as string - no problem, as int, char, bool, anything - everything ok. Only at offset 54 happens some strange moon-phase error.....

    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]


    Remove all attributes from the struct defintion (and do the same to the C or C++ version, let everything default)

    Make sure you use the "usnafe" term in the struct declaration.

    Ensure the project setting (build) is set to allow unsafe code.

    Your problem is solved, I've done this a very great deal for several years, this is the best solution, if you can't get it to work, get back to us here, it will be something minor.

    There is absolutely no need for any attributes with this approach, the struct is physically identical on both sides of the call, so debugging is MUCH easier. Your current TypeLoader error might be due to some fields overlapping, this is a risk you run when you specify your own field offsets.

    All those attributes are used solely by Marshal.StructureToPtr and Marshal.PtrToStructure. It uses them to dynamically define another (hidden) struct that you never see. It then copies everything from the input struct to this temp one and then it passes a ptr to this temp struct into your C/C++ code.

    You may not be aware that those attributes have no impact at all on how C# lays out the fields in the managed struct, they are totally ignored insofar as managed declarations go. In your case, the struct is perfectly suitable as blittable and in this case; when you do it this way it is very fast too, because the C# code can just pass a pointer to the managed struct (it will pin the struct in this case to stop it being moved) and your called function is then able to directly modify the fields inside the actual managed struct.

    None of this happens if you add those attributes and it gets slow, fiddly and error prone.

    You don't need any attributes for this, believe me.

    Cap'n

     

    • Edited by Captain Kernel Wednesday, March 24, 2010 12:22 PM expand
    • Marked as answer by Jasper22 Wednesday, March 24, 2010 12:42 PM
    Wednesday, March 24, 2010 12:07 PM
  • Eee... it managed struct and I don't use unsafe code - it all declared at struct (C#) declaration - even the Pack=2 attribute

    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]
    Wednesday, March 24, 2010 12:41 PM
  • Yes - I already figured that out. If I remove all attributes then everything runs ok... Strange.. But if you say so - that those attributes used only in Marshal.StructureToPtr and .PtrToStructure....

     

    Ok. Thank you for the answer

     

    P.S. Captain Kernel - I see you have inside info on internal .NET code - you sure that this is not a bug in .NET ? It really seems that .NET reserve (?) one byte in the middle of the internal layout (?). And whatever .net put in the middle can not be casted to char - this is my impression - but I may be wrong

     

     


    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]
    • Edited by Jasper22 Wednesday, March 24, 2010 1:05 PM added question
    Wednesday, March 24, 2010 12:42 PM
  • Yes - I already figured that out. If I remove all attributes then everything runs ok... Strange.. But if you say so - that those attributes used only in Marshal.StructureToPtr and .PtrToStructure....

     

    Ok. Thank you for the answer

     

    P.S. Captain Kernel - I see you have inside info on internal .NET code - you sure that this is not a bug in .NET ? It really seems that .NET reserve (?) one byte in the middle of the internal layout (?). And whatever .net put in the middle can not be casted to char - this is my impression - but I may be wrong

     

     


    C makes it easy to shoot yourself in the foot; C++ makes it harder, but when you do it blows your whole leg off (c) — Bjarne Stroustrup [http://www2.research.att.com/~bs/bs_faq.html#really-say-that]


    Hi

    Well I have no "inside info" (sometimes wich I did!) on any of this, but have done a great deal of low level stuff like this over the past few years. I have also built a small library of powerful helper code that lets me do all kinds of stuff with structs, classes, fields etc.

    Read up all you can on "blittable" and that will give you some good insights.

    Cap'n

     

    Wednesday, March 24, 2010 9:59 PM
  • You can also use Pinvoke  Interop Assistant tool to generate Native signature or Managed signature.

     

    http://clrinterop.codeplex.com/releases/view/14120

     

    Thursday, March 25, 2010 3:10 PM