none
using byte* as return value in C# RRS feed

  • Question

  • Hi.
    I have a function in an old fashion MFC dll which returns a byte*
    I want to use this function in C# using dllimport attribute, but I get
    A MarshalDirectiveException Error.
    here is the code:

    C++ function:
    ----------------
    W32EDDLL_API byte* __stdcall EncryptString(char plaintext[])
    {
    ....
    }

    C# Code:
    ----------------
      [DllImport("W32EDDLL.dll", CharSet = CharSet.Ansi, EntryPoint = "EncryptString")]
      [return: MarshalAs(UnmanagedType.LPArray)]
      public static extern byte[] EncryptString([MarshalAs(UnmanagedType.LPArray)] byte[] plainbytetext);
    .....

    EncryptString(System.Text.Encoding.UTF8.GetBytes("A String"));

    ERROR:
    ---------------
     System.Runtime.InteropServices.MarshalDirectiveException was unhandled
     Message="Cannot marshal 'return value': Invalid managed/unmanaged type combination."


    Does any1 know how to solve this problem???

    Thanks.

    • Edited by Supereric Tuesday, June 23, 2009 6:41 AM
    Tuesday, June 23, 2009 6:36 AM

Answers

  • Functions that return pointers are extraordinarily hard to use, even from C++.  You can declare the return type as IntPtr, then marshal the data with Marshal.Copy().  Assuming you have a good guess at how many bytes you'd need to copy.  What you can't do is release the memory.  That's hard to do reliably in C++, in C# you'll just have a memory leak you can't plug.

    The C++ function declaration should be changed to:

      int EncryptString(const char* plaintext, byte* buffer, size_t bufferSize);

    so that the caller can provide the memory and the return value indicates how many bytes were generated.

    Write a wrapper in the C++/CLI language if you can't change the C++ code.  Still hard to do, you'd have to make sure that the wrapper and the original C++ code use the same CRT library.

    Hans Passant.
    Tuesday, June 23, 2009 7:00 AM
    Moderator

All replies

  • Try to get the return value as an IntPtr.

    If that works you can copy the data to a byte array.
    Ganesh Ranganathan
    [Please mark the post as answer if it answers your question]
    Tuesday, June 23, 2009 6:45 AM
  • Functions that return pointers are extraordinarily hard to use, even from C++.  You can declare the return type as IntPtr, then marshal the data with Marshal.Copy().  Assuming you have a good guess at how many bytes you'd need to copy.  What you can't do is release the memory.  That's hard to do reliably in C++, in C# you'll just have a memory leak you can't plug.

    The C++ function declaration should be changed to:

      int EncryptString(const char* plaintext, byte* buffer, size_t bufferSize);

    so that the caller can provide the memory and the return value indicates how many bytes were generated.

    Write a wrapper in the C++/CLI language if you can't change the C++ code.  Still hard to do, you'd have to make sure that the wrapper and the original C++ code use the same CRT library.

    Hans Passant.
    Tuesday, June 23, 2009 7:00 AM
    Moderator
  • Hi,
    let me explain what I've got from your post and please correct any misunderstanding!
    1) I have to provide the function with a plaintext , a pointer to an array with unknown size and a size_t buffersize.
    so what is the usage of buffersize varibale ?

    2) also, in this case the method, fills the buffer with proper values, right?

    3) at last can you please declare the correct format in C# for using the mentioned new function?
    Tuesday, June 23, 2009 7:14 AM
  • The bufferSize argument is required so the function can avoid writing past the end of the buffer.  Buffer overruns are a classic unmanaged code bug and the entry vector for many viruses.

    [DllImport("bla.dll", CharSet=CharSet.Ansi)]
    public static extern int EncryptString(string plaintext, byte[] buffer, int bufferSize);

    Hans Passant.
    Tuesday, June 23, 2009 7:28 AM
    Moderator
  • so how to use it correctly in C#, I mean what is the usage of marshal.copy here?
    sorry, i am a beginner in C#.
    Tuesday, June 23, 2009 8:28 AM
  • You cannot use it correctly from C#.  Contact the C++ programmer that wrote that code.

    Hans Passant.
    Tuesday, June 23, 2009 8:54 AM
    Moderator