none
STL containers across DLL boundaries, has something changed in Visual C++ 9.0?

    Question

  • Hi,

    I've been doing some reading up about when it is safe or unsafe to pass STL containers across DLL boundaries (either as parameters, member variables of exported classes, or as return values), and I've come across a few questions that I haven't been able to find answers for (I'm using Visual C++ 9.0).

    1)  I read the article "How to export an instantiation of a Standard Template Library (STL) class and a class that contains a data member that is an STL object” (http://support.microsoft.com/kb/168958 ).  Which gave me the impression that I had to export the template instantiation declaration, even for something as simple as an stl vector of ints.

    However, I noticed that the article above only specifically applied to Visual C++ versions 5 and 6.  In this discussion:

    “Can STL vector cross DLL boundary?”
    http://social.msdn.microsoft.com/Forums/en-US/vclanguage/thread/83a026d8-09b1-4d57-8c4a-ab2d139624b4

    The responses made it sound like you only need to make sure that the same C run-time library is used across .exe and dlls.  There was no mention of exporting template instantiation declarations.  Is this still required in Visual C++ 9?  What is the recommended method for sharing the same declaration across several DLLs? (I considered created a new DLL whose sole purpose is to contain these exported template declarations, but I figured there must be a better way).

    2) All of the code I am running right now appears to run without a problem, but there are maps and vectors shared across boundaries all over the place.  Am I just lucking out?  Or (if I make sure to use the same CRT) can I just freely pass vectors and maps between dlls?  http://support.microsoft.com/kb/172396 mentioned that "Most classes in the Standard C++ Libraries use static data members directly or indirectly".  Could I just not be using any of the functions that would access these static data members?  Does anyone happen to know how these static data members are used, and what sorts of corruption they could cause?

    Thanks!

    Tuesday, August 18, 2009 11:45 PM

All replies

  • Hmya, what exactly does "exporting template instantiation declarations" mean?  Templates don't get exported.  They don't even work when compiled into an .obj file.  Two or three versions ago, we got _HAS_ITERATOR_DEBUGGING, a great good.  _SECURE_SCL too, that one is a bit more foggy.  These macros conspire to *not* make exported standard C++ classes binary compatible across DLL boundaries.  Along with _DEBUG.

    The decision is usually pretty easy: you're in for a pound or you're in for a penny.  The penny is avoiding exporting pointers or class objects at all cost.  The pound is always making sure your binaries are compiled with the same CRT.  Pound is easier in many ways.  But.  It is up to you.
    Hans Passant.
    Wednesday, August 19, 2009 12:13 AM
  • I tend to be more dogmatic than Hans on this subject. Simply put, passing STL containers or iterators across a DLL boundary causes more grief than convenience, and essentially defeats the purpose of creating a DLL in the first place. IMO.

    Wednesday, August 19, 2009 12:45 AM
  • Hi,

    Thank you for the quick response!

    Hmya, what exactly does "exporting template instantiation declarations" mean?  Templates don't get exported.  They don't even work when compiled into an .obj file.  Two or three versions ago, we got _HAS_ITERATOR_DEBUGGING, a great good.  _SECURE_SCL too, that one is a bit more foggy.  These macros conspire to *not* make exported standard C++ classes binary compatible across DLL boundaries.  Along with _DEBUG.



    I took the term "exporting template instantiation declarations" from kb article 168958 .

    Their example:

        // Provide the storage class specifier (extern for an .exe file, null
        // for DLL) and the __declspec specifier (dllimport for .an .exe file,
        // dllexport for DLL).
        // You must define EXP_STL when compiling the DLL.
        // You can now use this header file in both the .exe file and DLL - a
        // much safer means of using common declarations than two different
        // header files.
        #ifdef EXP_STL
        #    define DECLSPECIFIER __declspec(dllexport)
        #    define EXPIMP_TEMPLATE
        #else
        #    define DECLSPECIFIER __declspec(dllimport)
        #    define EXPIMP_TEMPLATE extern
        #endif
    
        // Instantiate classes vector<int> and vector<char>
        // This does not create an object. It only forces the generation of all
        // of the members of classes vector<int> and vector<char>. It exports
        // them from the DLL and imports them into the .exe file.
        EXPIMP_TEMPLATE template class DECLSPECIFIER std::vector<int>;
        EXPIMP_TEMPLATE template class DECLSPECIFIER std::vector<char>;
    


    The decision is usually pretty easy: you're in for a pound or you're in for a penny.  The penny is avoiding exporting pointers or class objects at all cost.  The pound is always making sure your binaries are compiled with the same CRT.  Pound is easier in many ways.  But.  It is up to you.


    I am able to make sure that all of the binaries will be compiled with the same CRT, but I am not sure if that is enough.  I found this summary in kb 172396 :

    "Since these classes are generated through template instantiation, each executable image (usually with DLL or EXE file name extensions) will contain its own copy of the static data member for a given class. When a method of the class that requires the static data member is executed, it uses the static data member in the executable image in which the method code resides. Since the static data members in the executable images are not in sync, this action could result in an access violation or data may appear to be lost or corrupted. "

    From that description it seems like compiling with the same CRT would not be enough to stay safe (because each DLL would still have its own copy of the static data member), but I'm not sure and the summary is from a Visual C++ a few releases ago.  Any details on if this static data member issue is still around and what these statics are used for (or why they are needed) would be greatly appreciated.

    I am most concerned with figuring out am I okay right now because I am using the same CRT across binaries, and if not, is the template exporting code above still required for vectors.

    Thanks again!


    Wednesday, August 19, 2009 1:28 AM
  • I tend to be more dogmatic than Hans on this subject. Simply put, passing STL containers or iterators across a DLL boundary causes more grief than convenience, and essentially defeats the purpose of creating a DLL in the first place. IMO.



    I agree its something that needs to get cleaned up in my code.  There is a lot of code though, and I'm trying to figure out how big of an issue (if it is an issue) this is right now :)

    I only noticed these issues recently when removing MFC from some commonly used DLLs (used by other DLLs whose CRT I have control of). 

    Once MFC was removed, I started getting a lot of compiler warnings from exporting classes containing STL containers or functions that use STL containers.  It ends up I wasn't seeing them before because MFC disables them in afx.h:

    // warnings specific to _AFXDLL version
    
    #ifdef _AFXDLL
    
    #pragma warning(disable: 4275)  // deriving exported class from non-exported
    
    #pragma warning(disable: 4251)  // using non-exported as public in exported
    
    #endif

    Wednesday, August 19, 2009 1:34 AM
  • I will reiterate what I have said in the past and partially echo what nobugz said: having _HAS_ITERATOR_DEBUGGING set in one binary and not in another worked just fine in VC2005.  The sizes were exactly the same.  It was in VC2008 that the size of the STL class changed depending on whether it is set or not.  So if you had some member of a class you were exporting from a DLL that is an STL class then you run into big trouble under VC2008 if _HAS_ITERATOR_DEBUGGING is set differently in two different binaries. 

    Simple solution: just make sure _HAS_ITERATOR_DEBUGGING is off for ALL binaries you compile.  Easiest way to do this is to add /D_HAS_ITERATOR_DEBUGGING=0 to all your compiles.
    Wednesday, August 19, 2009 2:24 AM
  • This is a terminally bad idea in most scenarios, since it means that both the dll project and its consumer have to be compiled with the same runtime (which has to be the DLL version) and the same compiler flags. Unless you control both the DLL and the consumer, this is a requirement that you have no control over and it will cause nothing but grief.

    EDIT:
    The 2 issues at hand are
    - that different STL version could be used, based on the STL version and compiler flags. If the DLL and consumer have different idea on things like size and layout, there will be corruption issues.
    - that different heap managers will be used by the DLL and the consumer. Eventually, one will try to free memory allocated by the other, and this will result in terminal errors. 
    Wednesday, August 19, 2009 8:27 AM
  • Hi,

    Thank you again for the replies.

    This is a terminally bad idea in most scenarios, since it means that both the dll project and its consumer have to be compiled with the same runtime (which has to be the DLL version) and the same compiler flags. Unless you control both the DLL and the consumer, this is a requirement that you have no control over and it will cause nothing but grief.

    I agree.  The current setup I am dealing with is several consumers that share some common DLLs.  Currently, the common DLLs are internal, and only the consumers that I have control over can use them.  While this means that I can make sure that compiler settings are consistent, I agree that it is bad moving forward because of how many restrictions are placed on the consumers.

    The 2 issues at hand are
    - that different STL version could be used, based on the STL version and compiler flags. If the DLL and consumer have different idea on things like size and layout, there will be corruption issues.
    - that different heap managers will be used by the DLL and the consumer. Eventually, one will try to free memory allocated by the other, and this will result in terminal errors.

    I think that for the time being I maybe okay because I keep all of the compiler settings the same, and all of the consumer and DLLs use the same dynamically-linked CRT.  My understanding is that by using the same CRT the same heap manager should be used across DLLs and consumers.

    If the heap management issue becomes a problem, I could use custom allocators and deallocators as a work around.

    I am still concerned about the two issues that the MS article brought up, namely:

    1) Static data members of STL classes will be different across DLLs
    2) STL vector instantiations must be exported, and it is impossible to export certain STL containers (e.g. map, list, deque, etc.)


    Now on the first issue, I could see it being related to what Ted_ wrote:
    I will reiterate what I have said in the past and partially echo what nobugz said: having _HAS_ITERATOR_DEBUGGING set in one binary and not in another worked just fine in VC2005.  The sizes were exactly the same.  It was in VC2008 that the size of the STL class changed depending on whether it is set or not.  So if you had some member of a class you were exporting from a DLL that is an STL class then you run into big trouble under VC2008 if _HAS_ITERATOR_DEBUGGING is set differently in two different binaries.

    and that perhaps these static data members are dependent on compiler settings (which may not be consistent across DLLs). 

    Does anyone know if these compiler-setting issues are the only thing that could cause a problem with static data members?  Or do certain operations use these static data members for something more (e.g. a counter) that would cause problems even if compiler settings are kept consistent.

    For the second issue, I am not sure how to interpret the MS articles.  I could see reading them as either:

    - Maps, lists, deques etc. can never ever be used between DLLs or DLLs and consumers, even if every single compiler setting is the same.

    -or-

    - Unlike vectors, due to the restrictions of nested template classes there is no way to export these (map, list, etc.) classes in your code.  Because vectors can be exported/imported, you can pass them between DLLs and consumers without being concerned about how the consumers were compiled and what settings were used.  However, because classes like std::map cannot be exported, the only way anything would work would be if all of the DLLs and consumers shared the same compiler settings (which is very difficult to guarantee), and therefore should not be used.


    If the second explanation is more accurate, then I should be okay, but I am not sure if it is.  Any thoughts?

    Thanks again.
    Wednesday, August 19, 2009 1:35 PM