none
Best Practices with Shared Memory and Fixed Size Buffers RRS feed

  • Question

  • Hello,

    within our solution we have 2 applications, one is a high performing c++ application, the other is a C# project that is used for designing a user interface. Both applications communicate with each other via shared memory (file mapping object).

    Once this shared memory only contained a few settings, but now it includes substructures and arrays. A problem I now encounter is that fixed size buffers have some limitations. For one, they can only be based on bool, byte, short, etc. and not on other structs (even though these structs only contain integers). However creating only flat arrays will cause me to loose all nicely designed structures, and though these limitations are placed on fixed size buffers, I don't see any technical reason why they are only limited to ints (and so on).

    My question is now, how can I easily work with fixed size buffers? Are there workarounds, or better ways to do this?

    What are the best practices?
    • Changed type Edsger Friday, November 13, 2009 8:16 AM
    Friday, November 6, 2009 9:34 AM

Answers

  • Thanks again for your elaborate answer.

    To answer your question, the problem is fairly simply to reproduce.
    1) Create a windows forms application
    2) define TestStruct2 as in my previous post
    3) create an instance of TestStruct2

    The run-time will throw an exception ('System.TypeLoadException') at the location where the instance is created.

    Furthermore, it seems like my approach with OffsetOf is leading to a dead end. Since I declared the value arrays in my struct, the StructLayout directives on my shared memory struct are not applied anymore. This also causes the values returned by OffsetOf to be invalid (from unmanaged perspective).

    To clarify what my goal is, I want to share data between a native c++ and a c# application. The data is pretty simple, only POD types (booleans, ints, which are sometimes combined in a structure, and fixed size arrays of these types). The C# application must be able to write values to this shared data structure, in other words, the C# application must be able to write values to an unmanaged data structure.

    Currently I can only think of one other approach I can try and that is to look if c++/cli has some alternatives.

    Hi

    OK so the managed app must either use a pointer or call some code to set/get the fields within unmanaged memory.

    You could avoid defining the managed version of the data entirely and just create a managed class class with properties (that call a little API) this could be done but is a little manual and entails some degree of hard coding perhaps, the little API would be the means that managed code uses to get/set fields or elements of arrays etc.

    Our product (Persistore) supports a wealth of features incuding the ability to binary persist rich structs or classes (using rich index attributes to help you manage multiple instances) into either transient or persistent shared memory and then get/set any field by its name, this is packaged in various abstractions and relies on an ability to analyze the binary data stream created by serialization and extract descriptive metadata.

    If you are not seeking very high update rates (e.g. not seeking hundreds of thousands of updates per second from a single thread) then this may be of interest, if you are seeking these high rates though then pointers are the fastest way and you will either have to design an alternative structuring that satifies the C# rules or devise a new way. We also support the ability to create linked unmanaged structures which is something else you may want to ponder doing.

    e.g

    instead of this:

    public struct TestStruct2
        {
            public unsafe struct Element        
            {
                public fixed char buffer[100];
            }
        
    
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
            public Element[] elementCollection;
        }
    


    consider doing this:

    public unsafe struct Outy
    {
       public int count;
       public Inny * elements;
            
       public Outy (int size)
       {
       count = size;
                
       /* Call some (to be defined) infrastructre to allocate a block of memory */
       /* and set the 'elements' pointer to the address of this block.          */
       }
            
    }
        
    public unsafe struct Inny
        {
        public fixed byte buffer[100];
        }
    
    /* Get some instance */
    
    Outy ot = new Outy(100);
    
    /* Example write operation */
    
    ot.elements[4].buffer[12] = 0;
    
    


    This requires your constructor to call into unmanaged code and allocate storage to hold 'n' instances of the 'Iny' type. This will work, but you need a heap in shared memory which is not something readily available - we wrote our own to support our needs. But if your structs are simple you could devise a simple memory pool approach perhaps.

    Of course when you need multiple instances of the 'Outy' type, then you need a way to identify these as well, will these be an array? will these have a key perhaps? if so how/where will you put an index? and what code or algorithm will you use to managed the index (add/search/delete etc)...

    The above sample code is legal C# and compiles incidentally, of course everything is 'public' but in reality would not be.

    Hope this is of some help.

    Hugh








    Hugh Moran - http://www.morantex.com
    • Marked as answer by Edsger Thursday, November 19, 2009 9:53 AM
    Wednesday, November 18, 2009 5:17 PM

All replies

  • Hello,

    within our solution we have 2 applications, one is a high performing c++ application, the other is a C# project that is used for designing a user interface. Both applications communicate with each other via shared memory (file mapping object).

    Once this shared memory only contained a few settings, but now it includes substructures and arrays. A problem I now encounter is that fixed size buffers have some limitations. For one, they can only be based on bool, byte, short, etc. and not on other structs (even though these structs only contain integers). However creating only flat arrays will cause me to loose all nicely designed structures, and though these limitations are placed on fixed size buffers, I don't see any technical reason why they are only limited to ints (and so on).

    My question is now, how can I easily work with fixed size buffers? Are there workarounds, or better ways to do this?

    What are the best practices?

    Fixed size buffers are excellent for shared memory applications.

    One thing that is sometimes overlooked, is that you can store almost any type in a fixed buffer (e.g. ANSI or Unicode strings) by making the buffer fields private and creating public conversion (get/set) properties that look like strings but simple encapsulate fixed byte/char buffers.

    What specifically is causing you problems?

    Hugh



    Hugh Moran - http://www.morantex.com
    Saturday, November 7, 2009 5:01 PM

  • I'd like to have some kind of data structure in my shared memory without having to resort to flat arrays.
    If I leave out the details I have a struct S that contains N elements of struct T, where T contains itself a number of arrays of some basic type (say int).
    So what I would like is that I can simply create a structure containing the contents of my shared memory that looks like

    public struct S
    {
         public struct T
         {
             public fixed int field1[100];
             public fixed int field2[100];
         }

         public fixed T ts[20];
    }

    I don't like to flatten my hierarchy, because the above example is an over simplification; I have multiple T-like structures, in addition to some other basic fields.

    Monday, November 9, 2009 9:53 AM

  • I'd like to have some kind of data structure in my shared memory without having to resort to flat arrays.
    If I leave out the details I have a struct S that contains N elements of struct T, where T contains itself a number of arrays of some basic type (say int).
    So what I would like is that I can simply create a structure containing the contents of my shared memory that looks like

    public struct S
    {
         public struct T
         {
             public fixed int field1[100];
             public fixed int field2[100];
         }

         public fixed T ts[20];
    }

    I don't like to flatten my hierarchy, because the above example is an over simplification; I have multiple T-like structures, in addition to some other basic fields.


    I see.

    Well you can store all kinds of data in shared memory, assuming you have some degree of management over the memory (or you may just have some big chunk).

    If your struct (nested or not) is blittable, then you can write it to memory easily or you can access the memory using managed C# pointers, because a blittable struct has same organization to managed and unmanaged code.

    However, you do not strictly need blittable structures, you can use Marshal.StructureToPtr to "write" any managed data into arbitrary memory, even classes, you can also use serialization to write the data to memory.

    The problem is however that if you "see" shared memory as just some large chunk, with a start address and a size, then doing this flexibly is very hard indeed because you may not be able to comput the size of the marshaled data so you may not know how much memory to use and so on.

    We have succesfully persisted all kinds of data in shared memory, including blittable structs, blittable arrays, classes, non blittable structs and so on, but we use a powerful shared memory management API to do the heavy work for us.

    In your case, I see no major problem with your basic struct, it seems blittable at first glance and so its size can be computed and that size is the same before and after Marshal.StructureToPtr is called so in theory even a limited memory management approach should work reasonably well.

    Does this help?

    Hugh




    Hugh Moran - http://www.morantex.com
    Monday, November 9, 2009 10:35 AM
  • Thanks for your answer Hugh,
    what I understand from your post, is that in C# - given the size and starting address of the shared memory - I cannot directly map the whole data structure onto the memory (as the above data structure does not compile). But I have to make some wrapper code that extracts them from the shared memory?

    There are two reasons why I want I want to map the data structure directly.

    The first reason is that I use reflection to inspect the shared memory. I'm able to inspect all kinds of members and use reflection to create an appropriate view for that member, depending on its type. All this is done with reflection, so I can add all kinds of members to it without having to adjust any of the viewing code.

    The second reason is that this structure is shared with a C++ application. From the C++ application I can easily "convert" the shared memory declaration to a C# representation (by copy & paste, adding public keywords); this approach is far less error prone than creating a complete new way to access the data structure.


    So creating a wrapper around this shared memory eliminates these capabilities.
    Monday, November 9, 2009 11:07 AM
  • Thanks for your answer Hugh,
    what I understand from your post, is that in C# - given the size and starting address of the shared memory - I cannot directly map the whole data structure onto the memory (as the above data structure does not compile). But I have to make some wrapper code that extracts them from the shared memory?

    There are two reasons why I want I want to map the data structure directly.

    The first reason is that I use reflection to inspect the shared memory. I'm able to inspect all kinds of members and use reflection to create an appropriate view for that member, depending on its type. All this is done with reflection, so I can add all kinds of members to it without having to adjust any of the viewing code.

    The second reason is that this structure is shared with a C++ application. From the C++ application I can easily "convert" the shared memory declaration to a C# representation (by copy & paste, adding public keywords); this approach is far less error prone than creating a complete new way to access the data structure.


    So creating a wrapper around this shared memory eliminates these capabilities.

    Hi Edsger

    I missed a vital detail in your post, of course that code does not compile as 'fixed' cannot be used for that type, I missed that part of your post.

    While you can't use 'fixed' in that way, you can do this:

        public unsafe struct Inner
        {
            public fixed int F1[100];
            public fixed int F2[100];
        }
        
        public unsafe struct Outer
        {
            [MarshalAs(UnmanagedType.ByValArray,SizeConst = 10)] 
            public Inner[] F3;
        }
    
    


    This works fine and you can also use Marshal.SizeOf on objects of type 'Outer' (which for this struct is exactly the size one would expect).

    My reference earlier to wrappers was not trying to solve that problem, I mentioned wrapper properties to stress that one can represent many types as fixed buffers and then use properties to disguise this, in other words one can used 'fixed' as an implementation for a field yet expose the data as some more complex type.

    The reason I mentioned that is that structs that contain 'fixed' are blittable and blittable structs can have their address taken, this is sometimes useful in high performance managed code work.

    You can also consider organizing the data as a linked set of structures (as opposed to embedded types) this is also feasible but requires some support code to do some of the donkey work, but a linked structure will be equally accessible from unmanaged code.

    Incidentally, are you working with 32-bit or 64-bit code here?

    Hugh




    Hugh Moran - http://www.morantex.com
    • Marked as answer by eryang Friday, November 13, 2009 7:34 AM
    Monday, November 9, 2009 11:38 AM

  • That post was of great help. Thanks for that.

    Only one "small" problems remains. I use a pointer to the shared memory to access members directly for efficiency reasons. However, with the array syntax used in Outer a compilation error occurs.

    I don't not quite understand why this is an issue and how to solve it. Simply leaving away the [] results in compilable code but the program crashes at run-time.
    Tuesday, November 10, 2009 4:14 PM

  • That post was of great help. Thanks for that.

    Only one "small" problems remains. I use a pointer to the shared memory to access members directly for efficiency reasons. However, with the array syntax used in Outer a compilation error occurs.

    I don't not quite understand why this is an issue and how to solve it. Simply leaving away the [] results in compilable code but the program crashes at run-time.

    Yes you will get a compile error, if you try to take the address of an instance of that type.

    This is because the compiler considers 'Outer' to be a managed structure due to the array type of 'Inner' (this is declared as a reference array type).

    It would be a great help if Microsoft allowed us to create fixed buffers for any pure value type (like certain other structs) rather than only the primitive value types, this is the crux of your problem - a language limitation.

    A similar issue is the current inability to declare pointers to generic arguments, because the langauge provides no way to constrain the generic arguments to a pure value type (we can constrain using 'struct' but a struct is not always a pure value type).

    I use the term pure value type to mean any type for which it is legal to take its address (i.e. at no level of nesting will one find any reference types).

    Microsoft don't really have an explicit term for pure value type, and it would help if they recognized this class of data types and gave them more support.

    It is not well documented but you can use a managed pointer only for pure value types, but unless that type is also blittable, the physical organization may differ from its managed to unmanaged form.

    Thus if you copy (e.g. Marshal.StructureToPtr) a managed value type to unmanaged memory, do not assume that the offsets etc of every field are the same when 'seen' by managed and unmanaged code, this is only ever true if and only if the managed struct is entirely blittable. (Or you have specified various explicit marshaling attributes).

    Furthermore C# nor .Net provides any built in test or property of a Type that tells you if a type is blittable (we have written support code for doing this, but it would be a great help of the language were more supportive).

    What you are trying to do is possible but fraught with hurdles, this is why we setup Morantex.

    Hugh



    Hugh Moran - http://www.morantex.com
    • Proposed as answer by Morantex Thursday, November 12, 2009 1:14 PM
    • Marked as answer by eryang Friday, November 13, 2009 7:33 AM
    Tuesday, November 10, 2009 6:04 PM
  • Indeed it seems that using shared memory structs - or more generally unmanaged memory - has its issues. Thanks for pointing out the issue with managed type members in structs. I was able to solve this issue by using the Marshal.OffsetOf method. Knowing that managed and unmanaged instances can differ in layout really helped clarifying how Marshalling works. So thanks for that!

    I was just trying to find out if everything works, by combining all tips & tricks to see if this finally got me a working solution (and marking your post as answer ;-)). When I thought everything worked fine and the compiler successfully build my project I ran however in another issue. When I ran my project I got a run-time exception while initiating my shared memory type. I was able to  reproduce the problem with the following rather simple structure.

        public struct TestStruct2
        {
            public unsafe struct Element       
            {
                public fixed char buffer[100];
            }
       

            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
            public Element[] elementCollection;
        }

    The System.TypeLoadException message was as follows:

    Could not load type 'DotTuner.TestStruct2' from assembly 'DotTuner, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

    Apparently the combination of an ByValArray and a fixed buffer is a problem???



    Friday, November 13, 2009 7:56 AM
  • Indeed it seems that using shared memory structs - or more generally unmanaged memory - has its issues. Thanks for pointing out the issue with managed type members in structs. I was able to solve this issue by using the Marshal.OffsetOf method. Knowing that managed and unmanaged instances can differ in layout really helped clarifying how Marshalling works. So thanks for that!

    I was just trying to find out if everything works, by combining all tips & tricks to see if this finally got me a working solution (and marking your post as answer ;-)). When I thought everything worked fine and the compiler successfully build my project I ran however in another issue. When I ran my project I got a run-time exception while initiating my shared memory type. I was able to  reproduce the problem with the following rather simple structure.

        public struct TestStruct2
        {
            public unsafe struct Element       
            {
                public fixed char buffer[100];
            }
       

            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
            public Element[] elementCollection;
        }

    The System.TypeLoadException message was as follows:

    Could not load type 'DotTuner.TestStruct2' from assembly 'DotTuner, Version=1.0.0.0, Culture=neutral, PublicKeyToken=null'.

    Apparently the combination of an ByValArray and a fixed buffer is a problem???




    I'd need a little more info about what the code was actually doing and at what point the exception is raised.

    Is it possible that you have a managed application that tries to access the data from/in shared memory, and that application has not loaded the assembly containing the struct defintion?

    We use shared memory here as the core of a fast general purpose managed data persistence engine, our product also stores "data assemblies" as well as data enabling any arbitrary app to access and reflect on persisted data in shared memory.

    This is how serialization works too, for example when you binary serialize some object the resulting stream contains info that identifies the type (and its containing assembly) is all part of the final serialzied output stream, any app that hopes to deserialize must have loaded or be prepared to dynamically load the assembly that contains the serialzied type.

    If you marshal data to/from shared memory between apps (a form of serialization) then there is no automatic record of the type and its assembly, any app that accesses shared memory with a view to accessing fields, reflecting data etc must be able to find the type at runtime.

    Now I'm not saying this is your issue, but without more info I can only guess at what is happening.

    After years of work with shared memory and managed code, we always recommend that the classes/structs used to represent persisted data in shared memory, be defined in their own distinct assemblies, these are then (often) small, can themselves be persisted and are all one needs to dynamically access the data from arbitrary apps that have no idea about the persisted types.

    Does this make any sense?

    Hugh

    Also note that the struct declaration does not in and of itself allocate storage, unfortunately you must ensure that the reference type member 'elementCollection' does indeed refer to an array of 100 items.

    If you add a constructor and perhaps some helpful constant defintions, then you can overcome this:

        public struct SharedConstants
        {
            public const int INNER_SIZE = 50;
            public const int OUTER_SIZE = 100;
        }
        
        public unsafe struct Inner
        {
            public fixed int F1[SharedConstants.INNER_SIZE];
            public fixed int F2[SharedConstants.INNER_SIZE];
        }
        
        public unsafe struct Outer
        {
            public Outer (bool setup)
            {
                if (setup)
    Member = new Inner[SharedConstants.OUTER_SIZE];
    else
    Member = null; } [MarshalAs(UnmanagedType.ByValArray, SizeConst = SharedConstants.OUTER_SIZE)] public Inner[] Member; }
    Thus every time you declare (instantiate) one of these, you create one with the correct storage (although it is NOT contiguous in managed memory). You can then populate it, and write or read copies of it from shared memory. (the 'bool' param is just there because we can't have parameterless constructors in structs, it is pretty much ignored otherwise).

    What you are attempting is puishing the language and .Net to it's limits here though, to get the arbitrary pointer based fast access to fields and structures like this in shared memory where the data is outside any app domain, is very tricky indeed and may end up imposing so many fiddly restrictions that it becomes hard to maintain, debug or test.

    With non-trivial nested structs or classes, you can binary serialize the data (this is not as fast as Marshaling). We have special unmanaged support for this here. We can binary serialize data and then dynamically access and update specific fields directly at runtime in shared memory because we know where and how the binary serializer stores it's output data (we support a thing called a SharedStream() which is a true stream with its data in shared memory, but this relies upon our proprietary shared heap manager end so on).

    To get the reliability, flexibility and speed you require may entail some effort.

    Hugh

    PS: Sorry for all the edits, the HTML/code editor causes me havoc sometimes!




    Hugh Moran - http://www.morantex.com
    • Edited by Morantex Friday, November 13, 2009 11:26 AM finished
    Friday, November 13, 2009 10:51 AM
  • Thanks again for your elaborate answer.

    To answer your question, the problem is fairly simply to reproduce.
    1) Create a windows forms application
    2) define TestStruct2 as in my previous post
    3) create an instance of TestStruct2

    The run-time will throw an exception ('System.TypeLoadException') at the location where the instance is created.

    Furthermore, it seems like my approach with OffsetOf is leading to a dead end. Since I declared the value arrays in my struct, the StructLayout directives on my shared memory struct are not applied anymore. This also causes the values returned by OffsetOf to be invalid (from unmanaged perspective).

    To clarify what my goal is, I want to share data between a native c++ and a c# application. The data is pretty simple, only POD types (booleans, ints, which are sometimes combined in a structure, and fixed size arrays of these types). The C# application must be able to write values to this shared data structure, in other words, the C# application must be able to write values to an unmanaged data structure.

    Currently I can only think of one other approach I can try and that is to look if c++/cli has some alternatives.
    Monday, November 16, 2009 3:01 PM
  • Thanks again for your elaborate answer.

    To answer your question, the problem is fairly simply to reproduce.
    1) Create a windows forms application
    2) define TestStruct2 as in my previous post
    3) create an instance of TestStruct2

    The run-time will throw an exception ('System.TypeLoadException') at the location where the instance is created.

    Furthermore, it seems like my approach with OffsetOf is leading to a dead end. Since I declared the value arrays in my struct, the StructLayout directives on my shared memory struct are not applied anymore. This also causes the values returned by OffsetOf to be invalid (from unmanaged perspective).

    To clarify what my goal is, I want to share data between a native c++ and a c# application. The data is pretty simple, only POD types (booleans, ints, which are sometimes combined in a structure, and fixed size arrays of these types). The C# application must be able to write values to this shared data structure, in other words, the C# application must be able to write values to an unmanaged data structure.

    Currently I can only think of one other approach I can try and that is to look if c++/cli has some alternatives.

    Hi

    OK so the managed app must either use a pointer or call some code to set/get the fields within unmanaged memory.

    You could avoid defining the managed version of the data entirely and just create a managed class class with properties (that call a little API) this could be done but is a little manual and entails some degree of hard coding perhaps, the little API would be the means that managed code uses to get/set fields or elements of arrays etc.

    Our product (Persistore) supports a wealth of features incuding the ability to binary persist rich structs or classes (using rich index attributes to help you manage multiple instances) into either transient or persistent shared memory and then get/set any field by its name, this is packaged in various abstractions and relies on an ability to analyze the binary data stream created by serialization and extract descriptive metadata.

    If you are not seeking very high update rates (e.g. not seeking hundreds of thousands of updates per second from a single thread) then this may be of interest, if you are seeking these high rates though then pointers are the fastest way and you will either have to design an alternative structuring that satifies the C# rules or devise a new way. We also support the ability to create linked unmanaged structures which is something else you may want to ponder doing.

    e.g

    instead of this:

    public struct TestStruct2
        {
            public unsafe struct Element        
            {
                public fixed char buffer[100];
            }
        
    
            [MarshalAs(UnmanagedType.ByValArray, SizeConst = 100)]
            public Element[] elementCollection;
        }
    


    consider doing this:

    public unsafe struct Outy
    {
       public int count;
       public Inny * elements;
            
       public Outy (int size)
       {
       count = size;
                
       /* Call some (to be defined) infrastructre to allocate a block of memory */
       /* and set the 'elements' pointer to the address of this block.          */
       }
            
    }
        
    public unsafe struct Inny
        {
        public fixed byte buffer[100];
        }
    
    /* Get some instance */
    
    Outy ot = new Outy(100);
    
    /* Example write operation */
    
    ot.elements[4].buffer[12] = 0;
    
    


    This requires your constructor to call into unmanaged code and allocate storage to hold 'n' instances of the 'Iny' type. This will work, but you need a heap in shared memory which is not something readily available - we wrote our own to support our needs. But if your structs are simple you could devise a simple memory pool approach perhaps.

    Of course when you need multiple instances of the 'Outy' type, then you need a way to identify these as well, will these be an array? will these have a key perhaps? if so how/where will you put an index? and what code or algorithm will you use to managed the index (add/search/delete etc)...

    The above sample code is legal C# and compiles incidentally, of course everything is 'public' but in reality would not be.

    Hope this is of some help.

    Hugh








    Hugh Moran - http://www.morantex.com
    • Marked as answer by Edsger Thursday, November 19, 2009 9:53 AM
    Wednesday, November 18, 2009 5:17 PM
  • Thanks for all your help Hugh,

    what I've learned from this thread is that unmanaged structs have a lot of marshalling options, but are still very limited even when it comes down to embedding the most basic C types. In my application I've currently settled for a data layout that has been as little adjusted as possible and can be directly mapped in c#. The only workaround I use is that I use Marshal.OffsetOf for the inline arrays that contain structs.

    You also provided workarounds with some interesting approaches, but unfortunately one of my requirements was the direct and easy mapping with C++. Still they were very insightful so thanks all of the posts!
    Friday, November 20, 2009 8:44 AM