Custom marshaling, threads, and nesting RRS feed

  • Question

  • In order for my question to make sense, it needs some setup.  Forgive me if you know some of this.

    Mostly custom marshaling is pretty straight forward.  When calling a function with a parameter marked as 'UnmanagedType.CustomMarshaler' in addition to either [In] or [Out]:

    - GetInstance() gets called and returns an instance of the custom marshaler.
    - When .Net needs to marshal from native to managed it calls MarshalManagedToNative, which takes an object as a parameter and returns an IntPtr.
    - When .Net needs to marshal from managed to native it calls MarshalNativeToManaged, which takes an IntPtr as a parameter and returns an object.

    However, things get a little tricky for parameters that are [In, Out].

    - If the marshaling starts by managed calling unmanaged, then MarshalManagedToNative gets called first and it returns an IntPtr (as per normal).
    - When the unmanaged code is complete, the second half of the call comes (MarshalNativeToManaged) to send the value back into managed code.  The IntPtr you returned from MarshalManagedToNative is sent in as the parameter (again, as per normal).
    - But in this case, the return value from MarshalNativeToManaged is ignored.

    In order to complete the marshaling, you have to modify the the object that was sent in MarshalManagedToNative.  Since MarshalNativeToManaged doesn't pass this value again, you have to have saved it during the original MarshalManagedToNative call.

    While it's unfortunate that the docs don't describe this requirement, it doesn't *seem* like it should be too hard to work around.  You just create a member variable on your custom marshaler class to save the object from the first call, and modify it in the second call.



    If your custom marshaler is called from multiple threads, .Net *may* create multiple instances of the marshaler.  But even if it does create multiple instance, it doesn't use them.  .Net picks one, and uses it for every call from every thread.  As you might expect, this results in chaos if you are storing data members in the custom marshaler.

    My first thought on how to solve this was to use ThreadStatic variables.  But is that safe?  The docs I've read for how managed threadids (which I assume are the basis for Threadstatic) get assigned are a little vague.  And given that we are alternating between managed code and unmanaged code, I'm even more concerned.

    Am I guaranteed that both MarshalNativeToManaged and MarshalManagedToNative will always get called such that the ThreadStatic values will match?  It *seems* to, but anecdotal observation is not the same as "the spec guarantees that it will."

    And even if it does, that brings us to the next problem.


    InOut calls come in pairs.  You either have MarshalManagedToNative followed by MarshalNativeToManaged, or MarshalNativeToManaged followed by MarshalManagedToNative.  Which pair you get depends on whether you start in managed or unmanaged.

    But what if you start with managed calling unmanaged (MarshalManagedToNative), but then the unmanaged code makes a call to managed code during its operation?  Perhaps it is forwarding the call.  Or maybe it just needs some information from a managed object to satisfy the first request.  It turns out that this too uses the same (single) instance of the custom marshaler.


    C1: MarshalManagedToNative - 1
    C2: MarshalNativeToManaged - 2 (nested call)
    C3: MarshalManagedToNative - 2
    C4: MarshalNativeToManaged - 1

    So when a custom marshaler starts with a MarshalManagedToNative, which is followed by a MarshalNativeToManaged, there is no way to know whether the MarshalNativeToManaged is the second half of a 'normal' call, or the first half of a nested call.  You can try to guess.  If you see that the IntPtr you received in C2 is the same one you just returned from C1, that might indicate it's the end of a normal call.  But that could also mean that the call is being forwarded to managed to respond.

    If you work the problem long enough, you can find some solutions.

    I've written an implementation using ThreadStatic.  It seems to work, but it depends on ThreadStatic working the way I hope it does, along with some hopes about how custom marshaling works (and will continue to work).  As you might guess, the code is complex, if brief.

    I also have an alternative that was proposed using ConcurrentDictionary.  This solves some problems over ThreadStatic, but generates others.

    These are both rather complex and feel like clumsy work-arounds.  Is there an 'official' or commonly used solution?  It would be great if there were some trick I'm just missing here.

    I have some (ugly) sample code if anyone wants to experiment (tested with 2.0 and 4.0.3).  But I'm really hoping that someone else has some code they can share with me.
    Sunday, July 10, 2016 1:20 AM

All replies

  • Hi LGS,

    Thread local data and thread-safe collections are workarounds which come up in my mind after reading your question. I will give you feedback if I have new ideas.

    Best Regards,
    Li Wang

    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, July 11, 2016 9:52 AM
  • Hello LGS,

    1. This is a long shot, but I hope it can help your situation in some way : see if the MarshalAsAttribute.MarshalCookie Field can help in any way.

    2. Although it is not documented, I have found that the CLR will not instantiate a new custom marshaler if the same cookie has been used previously.

    3. See an example use of the MarshalCookie here.

    4. Hope this helps.

    - Bio.

    Please visit my blog :

    Monday, July 11, 2016 1:03 PM
  • Hey Li, thanks for the response.

    Yep, those are what I'm trying.  But *boy* do they make the code more complex.  It's a lot of work just to ensure that I can safely use data members in an otherwise fairly trivial class.

    How comfortable are you with the assertion that "both MarshalNativeToManaged and MarshalManagedToNative will always get called such that the [ThreadStatic] values will match"?  Given the vagueness with which threadids are defined, I worry.  If this is not safe, using the ConcurrentDictionary approach might be more attractive (although it has its warts too).

    I could post the code if you'd be interested in taking a look.  Whether it would be useful to anyone else is not clear.  There's nothing particularly specific to my application in this code.  But the fact that google isn't finding any discussion about this makes me wonder how common a problem this is.  Or at least it makes me wonder how many people *know* it's a problem...

    IMO, the real solution here is for .Net to not have multiple threads using the same instance.  While you don't want to have dozens of ICustomMarshaler laying around taking up space, having multiple threads all calling the same one at the same time seems like a bad plan too.  Especially since data members are virtually a requirement for [In, Out] marshaling.

    .Net is (apparently) already keeping a list of ICustomMarshalers so it knows whether it needs to create another one.  Perhaps some sort of 'checkout' system?  If someone decides to pursue this, they should check back with me.  There are quirks here they might want to think about.

    Monday, July 11, 2016 9:34 PM
  • Not such a long shot.  You are right, using MarshalCookie does keep one method from interfering with another method.  It even helps if two parameters within the same method use the same marshaler.

    However it doesn't help if two threads are actually calling the same method.  Unfortunately there isn't any way to make the cookie include something like the threadid.

    Monday, July 11, 2016 9:36 PM