none
STL performance in 2012RC is slower?

    Question

  • We are using Visual Studio 2005. We have a large C++ application which uses  STL and Boost.  Performance is critical for our application. 

    We are looking at upgrading to Visual Studio 2012 once it is released. I tried this small program in Visual Studio 2012 RC and was surprised to see it ran more than 2X slower in a release build than the same code build with Visual Studio 2005. In VS2012 I used default release build settings. For me it takes about 20ms in VS2005 and about 50ms in VS2012. Why is it that much slower?

    Note: we did find turning exceptions off in the 2012 compiler sped up the timings a lot. Exceptions were on in the 2005 build.  Turning off exceptions would break a lot of our production code.   It would be a huge amount of work to try and get rid of every single exception.  How would we deal with third party libraries?

    Its this performance hit because of STL, just the deque template, exceptions or 2012 in general?   Will this be different in the production version of VS 2012

    //----------------------------------------------------------------------------------------------------------

    #include <windows.h>
    #include <deque>

    using namespace std;

    deque
    <int> d;

    int main(int argc, char* argv[])
    {
       
    const int COUNT = 5000000;

        timeBeginPeriod
    (1);    

       
    for (int i = 0; i < COUNT; ++i)
       
    {
            d
    .push_back(i);
       
    }

       
    double sum = 0;

        DWORD start
    = timeGetTime();

       
    for (int i = 0; i < COUNT; ++i)
       
    {
            sum
    += d[i];
       
    }

        printf
    ("time=%dms\n", timeGetTime() - start);

        printf
    ("sum=%f\n", sum);

       
    return 0;

    Friday, June 22, 2012 12:31 PM

Answers

  • It looks that’s just the changes in deque implementation. :) It might be better to ask Stephan T. Lavavej for the details.

    --

    Each call to d[i] creates an iterator to the beginning of a sequence and offsets it to i-th position.

    You should get similar timings in VC2005 and VC2012 if you run the test like this:

        int sum = 0;

        auto it = d.begin();

        for (int i = 0; i < COUNT; ++i)

        {

            sum += *it++;

            //sum += d[i];   

        }

    OR use std::accumulate, and VC2012RC should outperform VC2005:

    int sum = std::accumulate(d.begin(), d.end(), 0, plus<int>());

    With all new language improvements in C++, updated C++ libraries, PPL, AMP, Casablanca, RxC, there is no doubt VC2012 is one of the best options we have :)

    And use modern C++, especially if you’re a heavy user of STL and BOOST. You won’t regret it.


    • Edited by Grigoriy Chudnov Friday, June 22, 2012 3:09 PM
    • Proposed as answer by SimonRev Friday, June 22, 2012 4:34 PM
    • Marked as answer by Helen Zhao Monday, July 02, 2012 2:29 AM
    Friday, June 22, 2012 3:00 PM
  • On 22/06/2012 19:01, meissnersd wrote:


    Now with a vector:

         for (int i = 0; i < COUNT; ++i)
         {
             sum += v[i];
         }

    VS2005: 28ms
    VS2012: 8ms

         for (auto it = d.begin(); it != d.end(); ++it)
         {
             sum += *it;
         }

    VS2005: 28ms
    VS2012: 14ms

    So 2012 is a big win with std::vector.

    No, it is just that VS2008 and 2005 have _SECURE_SCL switched on in release build by default, instead VS2010/2012 have those security checking switched off in release build.

    Try your VS2005 code with #define _SECURE_SCL 0 in release builds.

    Giovanni

    • Marked as answer by Helen Zhao Monday, July 02, 2012 2:30 AM
    Friday, June 22, 2012 5:04 PM

All replies

  • It looks that’s just the changes in deque implementation. :) It might be better to ask Stephan T. Lavavej for the details.

    --

    Each call to d[i] creates an iterator to the beginning of a sequence and offsets it to i-th position.

    You should get similar timings in VC2005 and VC2012 if you run the test like this:

        int sum = 0;

        auto it = d.begin();

        for (int i = 0; i < COUNT; ++i)

        {

            sum += *it++;

            //sum += d[i];   

        }

    OR use std::accumulate, and VC2012RC should outperform VC2005:

    int sum = std::accumulate(d.begin(), d.end(), 0, plus<int>());

    With all new language improvements in C++, updated C++ libraries, PPL, AMP, Casablanca, RxC, there is no doubt VC2012 is one of the best options we have :)

    And use modern C++, especially if you’re a heavy user of STL and BOOST. You won’t regret it.


    • Edited by Grigoriy Chudnov Friday, June 22, 2012 3:09 PM
    • Proposed as answer by SimonRev Friday, June 22, 2012 4:34 PM
    • Marked as answer by Helen Zhao Monday, July 02, 2012 2:29 AM
    Friday, June 22, 2012 3:00 PM
  • Here are some tests for our setting comparing VS 2005 to 2012 STL containers:

        for (int i = 0; i < COUNT; ++i)
        {
            sum += d[i];
        }

    VS2005: 21ms
    VS2012: 51ms  (OUCH!)

        auto it = d.begin();

        for (int i = 0; i < COUNT; ++i)
        {
            sum += *it++;
        }

    Weird way to do a loop, but he's right it is equal
    VS2005: 33ms
    VS2012: 32ms

        for (deque<int>::iterator it = d.begin(); it != d.end(); ++it)
        {
            sum += *it;
        }

    VS2005: 33ms
    VS2012: 23ms

    double sum = std::accumulate(d.begin(), d.end(), 0.0, std::plus<double>());

    VS2005: 33ms
    VS2012: 8ms

    The last case is cool but not that relevant.  The whole test was to just see about accessing a deque, the addition was just the force the compile not to elide away the code.

    Now with a vector:

        for (int i = 0; i < COUNT; ++i)
        {
            sum += v[i];
        }

    VS2005: 28ms
    VS2012: 8ms

        for (auto it = d.begin(); it != d.end(); ++it)
        {
            sum += *it;
        }

    VS2005: 28ms
    VS2012: 14ms

    So 2012 is a big win with std::vector.    In terms of the slowdown,  turning off project exceptions skips stack object destructor cleanup according to http://msdn.microsoft.com/en-us/library/1deeycx5%28VS.80%29.aspx.   So I suspect the x2 slowdown on std::deque::operator() in 2012 is due to excess local objects.   May I suggest to Stephen he might want to do a little optimization before the release candidate goes out the door.  It probably solvable with a small local change in the operator.  Operator[] gets used a lot so worth it IMHO.  

    Friday, June 22, 2012 5:01 PM
  • On 22/06/2012 19:01, meissnersd wrote:


    Now with a vector:

         for (int i = 0; i < COUNT; ++i)
         {
             sum += v[i];
         }

    VS2005: 28ms
    VS2012: 8ms

         for (auto it = d.begin(); it != d.end(); ++it)
         {
             sum += *it;
         }

    VS2005: 28ms
    VS2012: 14ms

    So 2012 is a big win with std::vector.

    No, it is just that VS2008 and 2005 have _SECURE_SCL switched on in release build by default, instead VS2010/2012 have those security checking switched off in release build.

    Try your VS2005 code with #define _SECURE_SCL 0 in release builds.

    Giovanni

    • Marked as answer by Helen Zhao Monday, July 02, 2012 2:30 AM
    Friday, June 22, 2012 5:04 PM