locked
Sharing a file handler between managed and unmanaged code? RRS feed

  • Question

  • This is related to Interops, and sharing a file handler between a WPF UI and a native C++ DLL.

    I'm working on an application using MVVM as a guide. I want to use WPF for the UI, however it is a complex scientific application, which requires performance, and I am using C++ along with some GPU based code for all the modeling/processing. This is a Windows based app, no real network comm involved.

    I come across a challenge in terms of storing data. The data is multi-dimensional, and can be projected in many ways, while each projection can be processed/filtered in many ways too. I end up with a tree of data, with the raw data at the root and many branches.

    XML seems like an ideal way to store the information about this dataset, but as far as the raw data, it is not so convenient. I read about base64 encoding and the various binary-XML techniques, but see a major issue: the size of the converted data (150% of the raw) which in my case is huge (raw data is easily 200MBs+, and the multitude of projections can make it several 100's of MBs easily).

    There is the option of saving several files, but then it may become a nightmare for a user to 'transfer' files properly, so a one-file system seems best.

    I basically need to have the XML-like structure accessible to the UI, while the raw data needs to be accessed by the unmanaged code. So I was thinking I could concatenate all that in my own format, with a table of pointers to the data at the beginning of the file.

    The idea is to open the file, extract the XML, then pass the pointers to the unmanaged code, and let it read the data from the file as needed. I'm just not sure it is possible to share a file handler between managed and unmanaged code. I could possibly open the file in managed code, extract the XML, and then close it an open again in the unmanaged code.

    Has anyone come across such scenario? Maybe there is a better solution out there?

    I appreciate any comment or suggestion.

    Thanks

     

    Tuesday, August 24, 2010 12:56 AM

Answers

  • Hi,

    Below is a code sample that demonstrates open a file in manged C# code, pass the file handle to native C++ code, and continue reading from there.

     

    The manged C# code:

     

    using System;
    using System.IO;
    using System.Runtime.InteropServices;
    
    namespace Share_File_Between_Native_Managed_Code
    {
     class Program
     {
     static void Main(string[] args)
     {
      FileStream fs = File.Open(@"test.txt",
      FileMode.OpenOrCreate);
    
      byte[] bytes=new byte[20];
    
      fs.Read(bytes, 0, 12);
      Console.Write("Read from Managed code:");
      foreach (char c in bytes)
      Console.Write(c);
      Console.WriteLine();
    
    
      IntPtr file_handle = fs.SafeFileHandle.DangerousGetHandle();
    
      read_file(file_handle);
      fs.Close();
     }
    
     [DllImport(@"unmanaged.dll",
      EntryPoint = "read_file", CharSet = CharSet.Auto)]
     public static extern void read_file(IntPtr file_handle);
     }
    }
    
    

     

     

    unmanaged.dll

     

    extern "C" __declspec(dllexport)void read_file(HANDLE file_handle)
    {
    	char* pbuffer=new char[64];
    
    	DWORD number;
    	OVERLAPPED ov;
    	cout<<"Read from unmanaged code:";
    	if(::ReadFile(file_handle,pbuffer,5, &number,NULL))
    	{
    
    		for(int i=0;i<5;i++)
    			cout<<pbuffer[i];
    	}
    	else
    		cout<<"error"<<endl;
    	cout<<endl;
    }
    

     

     

    the content of test.txt is:

     

    123456789abcdef;
    

     

    Output:

     

    Read from Managed code:123456789abc
    Read from unmanaged code:def;
    

     


    Please mark the right answer at right time.
    Thanks,
    Sam
    • Edited by SamAgain Wednesday, August 25, 2010 7:23 AM refine
    • Marked as answer by WiizzZZZ Wednesday, August 25, 2010 6:48 PM
    Wednesday, August 25, 2010 7:18 AM

All replies

  • Hi,

    Thanks for your post. If I understood it correctly, you want to stuff your data with the XML data for WPF UI together. And place a table at the head of this new file format with pointers to your data. I really don't think it's a good idea to mix UI and business data together.

    As to file handle sharing, both managed code and unmanaged code have methods to open and read files, is it really necessary to share file handle between them?


    Please mark the right answer at right time.
    Thanks,
    Sam
    • Edited by SamAgain Tuesday, August 24, 2010 3:35 AM refine
    Tuesday, August 24, 2010 3:34 AM
  • Hi SamAgain,

    As i said, I'm using MVVM, so even though the UI is WPF, I still need the ViewModels to link to the data (the Model) in some ways.

    The 'data' consists in raw data, and lots of filtered/modeled version of the raw data. This is not just a bunch of random pieces of data, it is organized, and the 'structure' of the data is much like a tree, so XML seems like an ideal format to parse the data. However putting raw binary in XML is not convenient, and I am looking for a good way to save all the data (data structure, raw data, projected/modeled data) into one file.

    The file itself needs to be accessible from the managed code (ViewModel) as well as unmanaged code. The processing/projection steps are controlled by the UI but executed in unmanaged CPU/GPU code.

    The idea was to open the file in managed code, extract the data structure so as to be able to represent the data properly, then send the file handler and pointers to the pieces of raw data to unmanaged code. The point of the question is to know if I can share the file handler between managed and unmanaged code, or if the process is totally different.

    I'm not sure what kind of issues I may run into if I load part of a file in managed code, then close the handle and open the same file from unmanaged code. With one handler I can be sure there are no concurrent access to the file.

    Or should I open in managed code and read-copy the raw data into unmanaged memory?

     

    Tuesday, August 24, 2010 5:02 PM
  • Hi,

    As to MVVM, I would suggest WPF forum as a better option.

    As to passing a file handle from managed code to unmanaged code, why not let the managed code finish its job with the file and close it. And let the unmanaged code fiddle with the file afterwards? It would be much simpler.


    Please mark the right answer at right time.
    Thanks,
    Sam
    Wednesday, August 25, 2010 2:55 AM
  • Hi,

    Below is a code sample that demonstrates open a file in manged C# code, pass the file handle to native C++ code, and continue reading from there.

     

    The manged C# code:

     

    using System;
    using System.IO;
    using System.Runtime.InteropServices;
    
    namespace Share_File_Between_Native_Managed_Code
    {
     class Program
     {
     static void Main(string[] args)
     {
      FileStream fs = File.Open(@"test.txt",
      FileMode.OpenOrCreate);
    
      byte[] bytes=new byte[20];
    
      fs.Read(bytes, 0, 12);
      Console.Write("Read from Managed code:");
      foreach (char c in bytes)
      Console.Write(c);
      Console.WriteLine();
    
    
      IntPtr file_handle = fs.SafeFileHandle.DangerousGetHandle();
    
      read_file(file_handle);
      fs.Close();
     }
    
     [DllImport(@"unmanaged.dll",
      EntryPoint = "read_file", CharSet = CharSet.Auto)]
     public static extern void read_file(IntPtr file_handle);
     }
    }
    
    

     

     

    unmanaged.dll

     

    extern "C" __declspec(dllexport)void read_file(HANDLE file_handle)
    {
    	char* pbuffer=new char[64];
    
    	DWORD number;
    	OVERLAPPED ov;
    	cout<<"Read from unmanaged code:";
    	if(::ReadFile(file_handle,pbuffer,5, &number,NULL))
    	{
    
    		for(int i=0;i<5;i++)
    			cout<<pbuffer[i];
    	}
    	else
    		cout<<"error"<<endl;
    	cout<<endl;
    }
    

     

     

    the content of test.txt is:

     

    123456789abcdef;
    

     

    Output:

     

    Read from Managed code:123456789abc
    Read from unmanaged code:def;
    

     


    Please mark the right answer at right time.
    Thanks,
    Sam
    • Edited by SamAgain Wednesday, August 25, 2010 7:23 AM refine
    • Marked as answer by WiizzZZZ Wednesday, August 25, 2010 6:48 PM
    Wednesday, August 25, 2010 7:18 AM
  • Hi,

    Some more info. Be careful when using StreamReader in the managed code if you want to share the same file handle between managed and native code. Because StreamReader keeps an internal buffer to speed up the file reading. The minimal buffer size is 128 bytes. And the file location is advanced at the step quantum of the buffer size. So if you use StreamReader before passing the file handle to the native code, it's possible that the content which has already been read into the buffer will not be reachable by the native file access method.

    The following is a demo:

    The managed code:

     

    using System;
    using System.IO;
    using System.Runtime.InteropServices;
    using System.Text;
    
    namespace Share_File_Between_Native_Managed_Code
    {
     class Program
     {
     static void Main(string[] args)
     {
     String filepath = @"unicode.txt";
     FileStream fs = File.Open(filepath, FileMode.OpenOrCreate, FileAccess.Read);
     
    
     Console.Write("Read from Managed code:");
    
     char[] bytes = new char[20]; //char type represents a Unicode character
     StreamReader sr = new StreamReader(fs, Encoding.Unicode, false, 1); //1<128, so default minimum buffer is 128bytes.
     sr.Read(bytes, 0, 2);//Though I only read 2 characters, 128bytes has been read into the buffer actually.
     foreach (char c in bytes)
     Console.Write(c);
     Console.WriteLine();
    
     IntPtr file_handle = fs.SafeFileHandle.DangerousGetHandle();
     read_file(file_handle);
     fs.Close();
     }
    
     [DllImport(@"unmanaged.dll",
     EntryPoint = "read_file")]
     public static extern void read_file(IntPtr file_handle);
     }
    }
    
    

     

    unmanaged.dll :

     

    // NativeDLL.cpp : Defines the exported functions for the DLL application.
    //
    
    #include "stdafx.h"
    
    using namespace std;
    
    
    extern "C" __declspec(dllexport)void read_file(HANDLE file_handle)
    {
    	WCHAR pbuffer[64];
    
    	DWORD number;
    	OVERLAPPED ov;
    	cout<<"Read from unmanaged code:";
    	if(::ReadFile(file_handle,pbuffer,40, &number,NULL))
    	{
    		for(int i=0;i<16;i++) //total is 16 chars
    			//cout<<pbuffer[i];
    			::wprintf_s(L"%c",pbuffer[i]);
    	}
    	else
    		cout<<"error"<<endl;
    	cout<<endl;
    }
    

     

    The content of the unicode.txt (encoded in Unicode):

    123456789abcdef;A23456789abcdef;B23456789abcdef;C23456789abcdef;D23456789abcdef;E23456789abcdef;F23456789abcdef;G23456789abcdef;H23456789abcdef;I23456789abcdef;

     

    The output:

     

    Read from Managed code:12
    Read from unmanaged code:;D23456789abcdef
    

     

    As we can see, the first 64 characters takes up StreamReader's 128 bytes buffer, so the native code keeps on reading from the 129th byte, where the ; before the D begins. (Unicode text file begins with 0xFF 0xFE, which takes up 2 bytes.)



    Please mark the right answer at right time.
    Thanks,
    Sam
    • Edited by SamAgain Wednesday, August 25, 2010 9:50 AM refine
    Wednesday, August 25, 2010 9:22 AM
  • Thanks that is very helpful!
    Wednesday, August 25, 2010 11:54 PM
  • following up on this... i finally got to the code, and if this sounds great on paper, I am having an issue here:

    I get an error:

    A call to PInvoke function '[...]' has unbalanced the stack. This is likely because the managed PInvoke signature does not match the unmanaged target signature. Check that the calling convention and parameters of the PInvoke signature match the target unmanaged signature.

    Any idea how to fix that?

    I run WIn7 x64, compiling for Win32. As far as I can see when tracing, the IntPtr is 4bytes (32bits) and the DLL is also compiled for 32 bits, so there should not be any signautre issue.

    IntPtr casts to a pointer or handle according to the doc, so what is wrong here?

    Thanks for help.

    Regards

    Tuesday, November 9, 2010 8:51 PM
  • fixed this by adding:

     CallingConvention = CallingConvention.Cdecl

    in the  [DllImport(@"unmanaged.dll", EntryPoint = "read_file", CharSet = CharSet.Auto)]
    declaration...

    works, but am I ignoring an important error here by doing this?

    Tuesday, November 9, 2010 9:27 PM