none
How to get a reference/pointer to a fixed memory block in C#

    Question

  • I have a particular problem that I just cannot find an answer to. Tried all day on google and various fora, but to no avail, and I'm beginning to think that it's not possible.

    I have an existing application in C++ that receives a stream of telemetry data from an unmanned helicopter (UAV) and this app provides low level access to the software onboard the helicopter. To make this somewhat more user friendly, I'm currently writing a ground station GUI in C# and I want to interface to the C++ app. Onboard the helicopter all interprocess communication is done via shared memory (it's running soft realtime Linux), and the C++ app on the ground station maintains an almost realtime replica of this shared memory. This shared mem contains data from GPS, IMU, pressure, speed, attitude, Kalman filter parameters, waypoints, and everything else of interest to the UAV. To easily access all of this, the shared mem is actual just one big class CSharedMemory with numerous subclasses such as CGPS, CNAV440, CControl and so on, and in C++ (Linux as well as Win) I can just take the pointer to the shared memory and cast it to CSharedMemory (looks like this, assuming that a shared memory named "Win32ShMem" is already set up)

     

    hMap = OpenFileMapping(FILE_MAP_ALL_ACCESS, FALSE, "Win32ShMem");
    pBuf = (LPTSTR)MapViewOfFile(hMap, FILE_MAP_ALL_ACCESS, 0, 0, sizeof(CSharedMemory));
    CSharedMemory *sharedData = (CSharedMemory *)pBuf; <br/>

     

    and voila, I can access all of shared mem by statements like

     

    sharedData->GPS.UTMn = 1532.636f;
    printf("Bat: %i", sharedData->system.BatteryLevel);

     

    Now, on the ground station I would like to have the same easy access in C#. I have therefore setup the ground station shared memory from C# using memory mapped file, and this works fine and provides the shared mem needed by the C++ app to run, and it also provides an IntPtr to the memory block in C#. I have also managed to replicate the CSharedMemory class (albeit it is 14 kb large with a variety of classes, structs, array etc) in C#, and I have checked that it is aligned perfectly with the same class in C++ (using a series of attributes to arrange the individual variables; and don't worry, we use auto generated code, so this is done semi-automatically). Consequently, I can now in C# write

     

    IntPtr mappedView = [return from function giving me shared mem]
    CSharedMemory sharedData = new CSharedMemory();
    Marshal.PtrToStructure(mappedView, sharedData);
    int Bat = sharedData.system.BatteryLevel; // This will indeed retrieve the battery level<br/>

     

    and my local sharedData instance will be a copy of the shared mem (and I can write to shared mem, too, by using StructureToPtr). And now for the BIG question: Is it possible to access the shared mem in C# in the same easy way as I can in C++, i.e. sharedData.GPS.UTMe = 10? This would be REALLY useful since copying the entire class every time will overwrite changes made by the external C++ app.

    I know that the IntPtr is not gonna change, since it is shared mem. I also know that CSharedMemory class in C# is aligned with the structure of the shared mem. But, by gully, I cannot make C# "attach" an instance of CSharedMemory to the shared mem, i.e. to the IntPtr. Basically, I would like to tell C# "I would like to have a reference/pointer called sharedData of the type CSharedMemory which points to the same memory that mappedView points to, and you can leave all worries about alignment, deallocation etc to me". I'm no rookie to neither C++ nor to C#, but this has me really puzzled!

    Anyone got any ideas?

    Anders


    alc
    Friday, May 28, 2010 12:03 AM

Answers

  • Captain;

    well, yes and no. Your suggestion is indeed sort of what I'm looking for. So, I have now rewritten all of the classes to structs (loosing a bit of flexibility in the constructors of those classes etc, but that's acceptable), but now I'm facing another problem, namely how to have a struct which has a variable which is an array of another struct, and still be able to get a pointer to the original struct. Let me exemplify:

    I have the main struct (which is layouted with Explicit and FieldOffset, but I have left those out)

     

    public struct CSharedMemory
    
    {
    
     public int SampleNumber;
    
     public double SampleTime;
    
     public gps_t GPS;
    
     public planner_t Planner;
    
     // and many more
    
    };
    
    

     

    Now, planner_t is defined as

     

    [StructLayout(LayoutKind.Sequential)]
    
    public struct planner_t
    
    {
    
     [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
    
     public CManeuver[] ManeuverList;
    
    };
    
    

    where CManeuver is itself a struct with a number of floats, ints, and other structs (albeit no arrays of structs). This does produce a struct planner_t with the correct size (100 times the size of CManeuver), but I would guess that this gives me a reference type variable. And then, when I then try to get the pointer like this

     

    CSharedMemory *sharedData = (CSharedMemory *)mappedView.ToPointer();
    
    

     

    I get a "Cannot take the address of, get the size of, or declare a pointer to a managed type ('CSharedMemory')". If I leave out specifically planner_t from CSharedMemory I am indeed able to get the pointer. Also, it seems that ManeuverList in planner_t is now a reference and not value type (it is 'null' if I instantiate CSharedMemory), which in turn leaves me with the problem again of having this ManeuverList in shared mem.

    So, the question, is it possible to create an array of structs in a struct as a value type? The 'fixed' keyword only works for PODs, and I've been unable to find another way than the above to create this array of structs.

    Any clues?


    alc

    Hmm

    This is interesting.

    Without sitting down and coding this in earnest, its hard to say - but here is an idea - out of the box somewhat.

    Dont declare arrays, but instead declare "fixed" byte arrays of the required equivalent size (bearing in mind any padding that might be needed)

    This will give you a final overall struct that is the correct size, this is step 1.

    Make these fixed buffers private.

    Next in any struct that contains a fixed buffer, add a get property that returns and instance of the desired struct.

    In that get property, accept an single int arg and "pretend" it is an array subscript.

    In the property, use that subscript along with struct size info to compute the offset into the byte buffer for any instance of the struct.

    Next, take the address of that byte element, cast it to a pointer of struct type and return the deref'd pointer.

    here is crude pseudo code:

    struct Outer
    {
    public float F;
    public byte B;
    private fixed byte buffer[100];
    
    public Inner InnerArray (int subscript)
    {
    get {
       byte * byte_ptr;
       Inner * inner_ptr;
       byte_ptr = &(buffer);
       byte_ptr += sizeof(Inner) * (subscript);
       inner_ptr = (Inner *)(byte_ptr);
       return(*inner_ptr);
       }
    
    }
    
    }
    
    struct Inner
    {
    public short S;
    public long L;
    }

    This will probably allow you to code:

    short temp = outer_ptr->InnerArray(12).S;

    Does this help?

    That won't compile as is, but serves to get the idea across (assigning to the structs will not be possible with this model)

    Cap'n

     

     

    • Marked as answer by sokolnikoff Saturday, May 29, 2010 8:42 PM
    Saturday, May 29, 2010 6:17 PM
  • Hi Captain

    Thanx for the quick response. You were right, twice. The idea works and it won't compile :)  Making a few changes, it also allows me to assign to the struct. For future reference of this thread, here is what worked for me (using my original variable names):

     

    unsafe public struct CManeuver
    {
     public int Type; 
     public int Agility; 
     public CWaypoint Waypoint; // Another struct
     public CVeltrack Veltrack; // Another struct
    };
    
    unsafe public struct planner_t
    {
     private fixed byte buffer[11200]; // Array of 100 pcs 112 byte structs
    
     public CManeuver* ManeuverList(int subscript)
     {
      byte* bytePtr;
      CManeuver* pM;
    
      fixed (byte* p = buffer)
      {
       bytePtr = p;
       bytePtr += sizeof(CManeuver) * subscript;
       pM = (CManeuver*)bytePtr;
      }
      return pM;
     }
    };
    
    unsafe public struct CSharedMemory
    {
      public int SampleNumber;
      public float SampleTime;
      public gps_t GPS;
      public planner_t Planner;
      // and many more
    };

     

    And I'm able to write (in unsafe code, of course)

     

    CSharedMemory *Shm = (CSharedMemory *)mappedView.ToPointer();
    Shm->Planner.ManeuverList(12)->Agility = 2;
    

     

    and this is then readable in the shared mem from my C++ app. In short, it works!

     

    Thanx a mill, Captain. You saved my day - although it does annoy me a little that after a day of fumbling with this that I didn't come of with this rather neat solution myself :)

     

    Best,

    Anders


    alc
    • Marked as answer by sokolnikoff Saturday, May 29, 2010 8:42 PM
    Saturday, May 29, 2010 8:38 PM

All replies

  • I didn't have time to study your post in detail, but you can allocate and free memory in the .NET languages using the GCHandle class.
    Friday, May 28, 2010 12:17 AM
  • I have a particular problem that I just cannot find an answer to. Tried all day on google and various fora, but to no avail, and I'm beginning to think that it's not possible.

    I have an existing application in C++ that receives a stream of telemetry data from an unmanned helicopter (UAV) and this app provides low level access to the software onboard the helicopter. To make this somewhat more user friendly, I'm currently writing a ground station GUI in C# and I want to interface to the C++ app. Onboard the helicopter all interprocess communication is done via shared memory (it's running soft realtime Linux), and the C++ app on the ground station maintains an almost realtime replica of this shared memory. This shared mem contains data from GPS, IMU, pressure, speed, attitude, Kalman filter parameters, waypoints, and everything else of interest to the UAV. To easily access all of this, the shared mem is actual just one big class CSharedMemory with numerous subclasses such as CGPS, CNAV440, CControl and so on, and in C++ (Linux as well as Win) I can just take the pointer to the shared memory and cast it to CSharedMemory (looks like this, assuming that a shared memory named "Win32ShMem" is already set up)

     

    hMap = OpenFileMapping(FILE_MAP_ALL_ACCESS, FALSE, "Win32ShMem");
    
    pBuf = (LPTSTR)MapViewOfFile(hMap, FILE_MAP_ALL_ACCESS, 0, 0, sizeof(CSharedMemory));
    CSharedMemory *sharedData = (CSharedMemory *)pBuf; <br/>

     

    and voila, I can access all of shared mem by statements like

     

    sharedData->GPS.UTMn = 1532.636f;
    
    printf("Bat: %i", sharedData->system.BatteryLevel);

     

    Now, on the ground station I would like to have the same easy access in C#. I have therefore setup the ground station shared memory from C# using memory mapped file, and this works fine and provides the shared mem needed by the C++ app to run, and it also provides an IntPtr to the memory block in C#. I have also managed to replicate the CSharedMemory class (albeit it is 14 kb large with a variety of classes, structs, array etc) in C#, and I have checked that it is aligned perfectly with the same class in C++ (using a series of attributes to arrange the individual variables; and don't worry, we use auto generated code, so this is done semi-automatically). Consequently, I can now in C# write

     

    IntPtr mappedView = [return from function giving me shared mem]
    
    CSharedMemory sharedData = new CSharedMemory();
    
    Marshal.PtrToStructure(mappedView, sharedData);
    
    int Bat = sharedData.system.BatteryLevel; // This will indeed retrieve the battery level<br/>

     

    and my local sharedData instance will be a copy of the shared mem (and I can write to shared mem, too, by using StructureToPtr). And now for the BIG question: Is it possible to access the shared mem in C# in the same easy way as I can in C++, i.e. sharedData.GPS.UTMe = 10? This would be REALLY useful since copying the entire class every time will overwrite changes made by the external C++ app.

    I know that the IntPtr is not gonna change, since it is shared mem. I also know that CSharedMemory class in C# is aligned with the structure of the shared mem. But, by gully, I cannot make C# "attach" an instance of CSharedMemory to the shared mem, i.e. to the IntPtr. Basically, I would like to tell C# "I would like to have a reference/pointer called sharedData of the type CSharedMemory which points to the same memory that mappedView points to, and you can leave all worries about alignment, deallocation etc to me". I'm no rookie to neither C++ nor to C#, but this has me really puzzled!

    Anyone got any ideas?

    Anders


    alc


    The answer to your question is yes, you can access shared memory in C# as easily as in C++.

    Firstly, note that if your structure (that has all the data in) is a pure value type (contains no ref types) then you can declare C# pointers:

    MyStruct * struct_ptr; // must be done in unsafe code block.

    Now if this pointer is set to the address of the mapped memory, then you can code:

    struct_ptr->GPS.UTMe = 10;

    and so on.

    Does this help?

    Cap'n

     

     

     

     

    Friday, May 28, 2010 2:23 AM
  • Captain;

    well, yes and no. Your suggestion is indeed sort of what I'm looking for. So, I have now rewritten all of the classes to structs (loosing a bit of flexibility in the constructors of those classes etc, but that's acceptable), but now I'm facing another problem, namely how to have a struct which has a variable which is an array of another struct, and still be able to get a pointer to the original struct. Let me exemplify:

    I have the main struct (which is layouted with Explicit and FieldOffset, but I have left those out)

     

    public struct CSharedMemory
    {
     public int SampleNumber;
     public double SampleTime;
     public gps_t GPS;
     public planner_t Planner;
     // and many more
    };

     

    Now, planner_t is defined as

     

    [StructLayout(LayoutKind.Sequential)]
    public struct planner_t
    {
     [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
     public CManeuver[] ManeuverList;
    };

    where CManeuver is itself a struct with a number of floats, ints, and other structs (albeit no arrays of structs). This does produce a struct planner_t with the correct size (100 times the size of CManeuver), but I would guess that this gives me a reference type variable. And then, when I then try to get the pointer like this

     

    CSharedMemory *sharedData = (CSharedMemory *)mappedView.ToPointer();

     

    I get a "Cannot take the address of, get the size of, or declare a pointer to a managed type ('CSharedMemory')". If I leave out specifically planner_t from CSharedMemory I am indeed able to get the pointer. Also, it seems that ManeuverList in planner_t is now a reference and not value type (it is 'null' if I instantiate CSharedMemory), which in turn leaves me with the problem again of having this ManeuverList in shared mem.

    So, the question, is it possible to create an array of structs in a struct as a value type? The 'fixed' keyword only works for PODs, and I've been unable to find another way than the above to create this array of structs.

    Any clues?


    alc
    Saturday, May 29, 2010 3:28 PM
  • Captain;

    well, yes and no. Your suggestion is indeed sort of what I'm looking for. So, I have now rewritten all of the classes to structs (loosing a bit of flexibility in the constructors of those classes etc, but that's acceptable), but now I'm facing another problem, namely how to have a struct which has a variable which is an array of another struct, and still be able to get a pointer to the original struct. Let me exemplify:

    I have the main struct (which is layouted with Explicit and FieldOffset, but I have left those out)

     

    public struct CSharedMemory
    
    {
    
     public int SampleNumber;
    
     public double SampleTime;
    
     public gps_t GPS;
    
     public planner_t Planner;
    
     // and many more
    
    };
    
    

     

    Now, planner_t is defined as

     

    [StructLayout(LayoutKind.Sequential)]
    
    public struct planner_t
    
    {
    
     [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
    
     public CManeuver[] ManeuverList;
    
    };
    
    

    where CManeuver is itself a struct with a number of floats, ints, and other structs (albeit no arrays of structs). This does produce a struct planner_t with the correct size (100 times the size of CManeuver), but I would guess that this gives me a reference type variable. And then, when I then try to get the pointer like this

     

    CSharedMemory *sharedData = (CSharedMemory *)mappedView.ToPointer();
    
    

     

    I get a "Cannot take the address of, get the size of, or declare a pointer to a managed type ('CSharedMemory')". If I leave out specifically planner_t from CSharedMemory I am indeed able to get the pointer. Also, it seems that ManeuverList in planner_t is now a reference and not value type (it is 'null' if I instantiate CSharedMemory), which in turn leaves me with the problem again of having this ManeuverList in shared mem.

    So, the question, is it possible to create an array of structs in a struct as a value type? The 'fixed' keyword only works for PODs, and I've been unable to find another way than the above to create this array of structs.

    Any clues?


    alc

    Hmm

    This is interesting.

    Without sitting down and coding this in earnest, its hard to say - but here is an idea - out of the box somewhat.

    Dont declare arrays, but instead declare "fixed" byte arrays of the required equivalent size (bearing in mind any padding that might be needed)

    This will give you a final overall struct that is the correct size, this is step 1.

    Make these fixed buffers private.

    Next in any struct that contains a fixed buffer, add a get property that returns and instance of the desired struct.

    In that get property, accept an single int arg and "pretend" it is an array subscript.

    In the property, use that subscript along with struct size info to compute the offset into the byte buffer for any instance of the struct.

    Next, take the address of that byte element, cast it to a pointer of struct type and return the deref'd pointer.

    here is crude pseudo code:

    struct Outer
    {
    public float F;
    public byte B;
    private fixed byte buffer[100];
    
    public Inner InnerArray (int subscript)
    {
    get {
       byte * byte_ptr;
       Inner * inner_ptr;
       byte_ptr = &(buffer);
       byte_ptr += sizeof(Inner) * (subscript);
       inner_ptr = (Inner *)(byte_ptr);
       return(*inner_ptr);
       }
    
    }
    
    }
    
    struct Inner
    {
    public short S;
    public long L;
    }

    This will probably allow you to code:

    short temp = outer_ptr->InnerArray(12).S;

    Does this help?

    That won't compile as is, but serves to get the idea across (assigning to the structs will not be possible with this model)

    Cap'n

     

     

    • Marked as answer by sokolnikoff Saturday, May 29, 2010 8:42 PM
    Saturday, May 29, 2010 6:17 PM
  • Hi Captain

    Thanx for the quick response. You were right, twice. The idea works and it won't compile :)  Making a few changes, it also allows me to assign to the struct. For future reference of this thread, here is what worked for me (using my original variable names):

     

    unsafe public struct CManeuver
    {
     public int Type; 
     public int Agility; 
     public CWaypoint Waypoint; // Another struct
     public CVeltrack Veltrack; // Another struct
    };
    
    unsafe public struct planner_t
    {
     private fixed byte buffer[11200]; // Array of 100 pcs 112 byte structs
    
     public CManeuver* ManeuverList(int subscript)
     {
      byte* bytePtr;
      CManeuver* pM;
    
      fixed (byte* p = buffer)
      {
       bytePtr = p;
       bytePtr += sizeof(CManeuver) * subscript;
       pM = (CManeuver*)bytePtr;
      }
      return pM;
     }
    };
    
    unsafe public struct CSharedMemory
    {
      public int SampleNumber;
      public float SampleTime;
      public gps_t GPS;
      public planner_t Planner;
      // and many more
    };

     

    And I'm able to write (in unsafe code, of course)

     

    CSharedMemory *Shm = (CSharedMemory *)mappedView.ToPointer();
    Shm->Planner.ManeuverList(12)->Agility = 2;
    

     

    and this is then readable in the shared mem from my C++ app. In short, it works!

     

    Thanx a mill, Captain. You saved my day - although it does annoy me a little that after a day of fumbling with this that I didn't come of with this rather neat solution myself :)

     

    Best,

    Anders


    alc
    • Marked as answer by sokolnikoff Saturday, May 29, 2010 8:42 PM
    Saturday, May 29, 2010 8:38 PM