locked
[VC9/x86] sizeof(std::vector): 20/24 bytes? RRS feed

  • Question

  • Hi,

     

    I noticed sizeof(std::vector) is 20 (debug mode, iterator debugging enabled) or 24 bytes (debug mode, iterator debugging disabled). I'm wondering, why does the size increase if I disable iterator debugging?

    Why is the size 24 bytes? 12 bytes are expected (3 pointers), what are the other 8/12 bytes used for? The standard allocator? I thought it didn't store state, so wouldn't use any memory.

    Wednesday, January 28, 2009 10:35 AM

All replies

  • Why do you care? What problem are you trying to solve?
    Wednesday, January 28, 2009 5:01 PM
  • Brian Muth said:

    Why do you care? What problem are you trying to solve?


    Improving performance and reducing memory usage. I've got tons (100000+) of those vectors...

    And just curiosity.


    Wednesday, January 28, 2009 6:00 PM
  • 100k+ vectors at the SAME time? You may have bigger problem to worry about, like stack overflow if they are allocated on stack.


    MSMVP VC++
    Wednesday, January 28, 2009 7:23 PM
  • Yes, at the same time. No, they're not allocated on the stack...
    Thursday, January 29, 2009 11:11 AM
  • Somebody?
    Anybody?
    Thursday, January 21, 2010 4:15 PM
  • My sense is that if you are concerned about sizeof(std::vector) because there are 100,000 instances of them, you may have a fundamental design issue. You suggest that there is a performance issue, but you have provided no indication what exactly that issue is.

    Given the lack of information, I find it hard to comment. I can't say I've given it any more thought over the past year.

    Thursday, January 21, 2010 4:50 PM
  • I tried spelunking in STL code that comes with VS2008 SP1 (a.k.a. VC9).

    Considering debug builds as per your request, I came to the following conclusions:

    In both cases of iterator debugging feature (i.e. _HAS_ITERATOR_DEBUGGING) being enabled or disabled:

    1. vector has 3 data members, all pointers. In 32-bit systems, each pointer is 4 bytes, so they occupy 3*4 = 12 bytes.

    2. vector is derived from _Vector_val class. This class has an _Alval data member, which is 1 byte big.
    However, for alignment reasons, the size cost is rounded to 4 bytes.

    So, you have 12 bytes (from 3 vector's pointers) + 4 bytes (from _Vector_val::_Alval) = 16 bytes subtotal.

    Now, let's go to sub-cases of iterator debugging feature enabled or disabled.

    If iterator debugging feature is enabled:
    3a. _Vector_val derives from _Container_base_secure. _Container_base_secure has a data member _Myfirstiter, which is a pointer, so add 4 bytes.
    The total becomes: 16 + 4 = 20 bytes (as per your finding).

    If iterator debugging feature is disabled:
    3b. _Vector_val derives from _Container_base_aux_alloc_real. _Container_base_aux_alloc_real has one data member. Its size is 1 byte, but for alignment reasons rounds up to 4 bytes.
    4b. _Container_base_aux_alloc_real derives from _Container_base_aux. _Container_base_aux has one data member _Myownedaux, which is a pointer; so: add 4 bytes.
    In this case the total becomes: 16 + 4 + 4 = 24 bytes (as per your finding).

    You may want to try this simple C++ program I wrote to try to better understand the total size of STL vector.

    //////////////////////////////////////////////////////////////////////////
    // vecsize.cpp
    // Test size of std::vector
    //
    
    
    // Enable Iterator debugging feature in debug builds:
    //#define _HAS_ITERATOR_DEBUGGING 1
    
    // Disable Iterator debugging feature in debug builds:
    #define _HAS_ITERATOR_DEBUGGING 0
    
    
    #include <vector>       // STL vector
    #include <stdio.h>      // printf
    
    
    // I use this class to inspect vector data members.
    // This class is derived from vector, so it can access vector's protected 
    // data members and query their size).
    template <typename T>
    class vector_inspector : public std::vector<T>
    {
    public:
    
        // _Vector_val::_Alval
        size_t get_Alval_size() const
        {
            return sizeof(_Alval);
        }
    
    #if _HAS_ITERATOR_DEBUGGING
        
        // _Container_base_secure::_Myfirstiter
        size_t get_Myfirstiter_size() const
        {
            return sizeof(_Myfirstiter);
        }
    
    #else
        
        // _Container_base_aux_alloc_real::_Alaux
        size_t get_Alaux_size() const
        {
            return sizeof(_Alaux);
        }
    
        // _Container_base_aux::_Myownedaux
        size_t get_Myownedaux_size() const
        {
            return sizeof(_Myownedaux);
        }
    
    #endif // _HAS_ITERATOR_DEBUGGING
    
    };
    
    
    int main()
    {
        printf("*** Testing std::vector size ***\n\n");
    
        typedef int Type;
        Type * ptr;
        std::vector<Type> vec;
    
    #if defined( _DEBUG )
        printf("Debug build.\n");
    #else
        #error Must build in _DEBUG mode for _HAS_ITERATOR_DEBUGGING.
    #endif
    
    #if _HAS_ITERATOR_DEBUGGING
        printf("Iterator debugging enabled.\n");
    #else
        printf("Iterator debugging disabled.\n");
    #endif // _HAS_ITERATOR_DEBUGGING
        printf("\n");
    
        vector_inspector<Type> vinspec;
        printf("sizeof(vector) %d\n\n", sizeof(vec));
    
        printf("sizeof(pointer) %d\n", sizeof(ptr));
    
        printf("sizeof(_Vector_val::_Alval) %d\n", vinspec.get_Alval_size() );
        
    #if _HAS_ITERATOR_DEBUGGING
        printf("sizeof(_Container_base_secure::_Myfirstiter) %d\n", vinspec.get_Myfirstiter_size());
    #else
        printf("sizeof(_Container_base_aux_alloc_real::_Alaux) %d\n", vinspec.get_Alaux_size());
        printf("sizeof(_Container_base_aux::_Myownedaux) %d\n", vinspec.get_Myownedaux_size());
    #endif // _HAS_ITERATOR_DEBUGGING
    
        printf("\n");
        return 0;
    }
    
    
    //////////////////////////////////////////////////////////////////////////
    


    You can use this command line (or build from the IDE):

      cl /EHsc /nologo /W4 /MDd /Od /D "_DEBUG" /RTC1 vecsize.cpp
     
    These are the outputs:

    ---[output]---

    *** Testing std::vector size ***

    Debug build.
    Iterator debugging enabled.

    sizeof(vector) 20

    sizeof(pointer) 4
    sizeof(_Vector_val::_Alval) 1
    sizeof(_Container_base_secure::_Myfirstiter) 4




    *** Testing std::vector size ***

    Debug build.
    Iterator debugging disabled.

    sizeof(vector) 24

    sizeof(pointer) 4
    sizeof(_Vector_val::_Alval) 1
    sizeof(_Container_base_aux_alloc_real::_Alaux) 1
    sizeof(_Container_base_aux::_Myownedaux) 4

    ---[end output]---


    HTH,
    Giovanni




    • Proposed as answer by ProframFiles Tuesday, February 25, 2014 9:34 PM
    Thursday, January 21, 2010 7:55 PM
  • My sense is that if you are concerned about sizeof(std::vector) because there are 100,000 instances of them, you may have a fundamental design issue. You suggest that there is a performance issue, but you have provided no indication what exactly that issue is.


    Brian: I agree with you.

    My hypothesis is that the OP may be trying to do some numerical computation, and e.g. each std::vector instance stores a 3D (or 4D) physical vector (I mean: (x,y,z) triple or something).
    So there is a vector field (like a 2D matrix each element of which is a vector) of 100,000 elements.

    In this case, I would suggest a different approach, and not using std::vector to store a single 3D vector in the vector field.

    Probably Blitz++ contains classes better suited for this kind of computations.

    Giovanni

    Thursday, January 21, 2010 8:00 PM
  • My sense is that if you are concerned about sizeof(std::vector) because there are 100,000 instances of them, you may have a fundamental design issue. You suggest that there is a performance issue, but you have provided no indication what exactly that issue is.

    Given the lack of information, I find it hard to comment. I can't say I've given it any more thought over the past year.


    There may indeed be a fundamental design issue. I don't have concrete data to backup that it's indeed a performance problem.
    In any way, I'm curious about the memory consumption of these things. Giovanni did a nice job at explaining it. It's basically due to the allocator. Although I'm wondering what the allocator has to store and why it needs more in release than in debug mode.

    It's indeed used for numerical computation and most of the vectors are pretty small and don't really need the dynamic nature of std::vector.
    Monday, January 25, 2010 1:12 PM
  • If your matrices have a fixed size, use std::tr1::array/std::array/boost::array. These have zero overhead.
    Monday, January 25, 2010 1:23 PM
  • Unfortunately they're not fixed-size.
    Monday, January 25, 2010 3:55 PM
  • Unfortunately they're not fixed-size.
    As already suggested, you may want to consider Blitz++, and in particular its TinyVector class.

      The TinyVector class provides a small, lightweight vector object whose size is known at compile time.


    HTH,
    Giovanni


    Wednesday, January 27, 2010 9:31 AM
  • What's the difference with boost::array?
    TinyVector seems fixed-size, which isn't what I need.
    Wednesday, January 27, 2010 1:05 PM
  • What's the difference with boost::array?
    TinyVector seems fixed-size, which isn't what I need.
    I thought you needed something small and fast, without overhead, just like a pure C array.

    If you want something that is resizable, then you must have some overhead.
    You could try disabling checked iterators in release builds.

    In this case the size of vector seems to be 16 bytes:

    c:\TEMP>type testvec.cpp
    #define _SECURE_SCL 0

    #include <vector>
    #include <iostream>

    using namespace std;

    int main()
    {
        vector<double> v;

        cout << "sizeof(vector) : " << sizeof(v) << endl;

        return 0;
    }

    c:\TEMP>cl /EHsc /nologo testvec.cpp
    testvec.cpp

    c:\TEMP>testvec.exe
    sizeof(vector) : 16


    Thursday, January 28, 2010 5:19 PM
  • Like Olaf I discovered this issue, but I have a huge problem with it. My scenario is that I have an array of objects I pass from a program to a DLL. For some reason the DLL and the EXE disagree on how big vectors are, resulting in some really nasty crashes that takes forever to find. Defining _HAS_ITERATOR_DEBUGGING didn't change anything and setting alignment in the projects also had no effect. I have not found a way to make vectors the same size in my two projects so far...

    Monday, September 13, 2010 3:24 PM
  • don't pass STL objects if your DLL and EXE do not agree on specialization of the template class.

    The following is signature, not part of post
    Please mark the post answered your question as the answer, and mark other helpful posts as helpful, so they will appear differently to other users who are visiting your thread for the same problem.
    Visual C++ MVP
    Monday, September 13, 2010 3:28 PM
  • Yea, thanks. I'll just rewrite this 50k line program from scratch. Or maybe you could actually be helpful and offer some kind of help.

    Seriously, is this a forum where MVPs abuse and insult people or where you're trying to help people who have serious problems with your software? I am just trying to MAKE the DLL and EXE agree on the template instantiations but I can't make it work even though I have the code for both projects and I've checked that they have all the same build and link settings. To me this looks like a huge bug in VC2008 and you're not helping.

    Monday, September 20, 2010 12:50 PM
  • It is better in the long run. there are just many restrictions that makes exporting STL so error-prone that is not worth the benefits.

    It is not a bug in VC2008, it is well documented since VC5. You are better off using the standard C interfaces as data transport objects between your DLL and your DLL's clients, your DLL will become callable from other languages and other versions of Visual C++.



    The following is signature, not part of post
    Please mark the post answered your question as the answer, and mark other helpful posts as helpful, so they will appear differently to other users who are visiting your thread for the same problem.
    Visual C++ MVP
    Monday, September 20, 2010 2:35 PM