locked
CppUnitTestFramework Performance RRS feed

  • Question

  • Our team is moving from VS2010 to 2012 and we are looking for appropriate unit testing framework.

    The showstopper we have is performance! When I modify some test and run it again it take from 4 to 10 second to see if test is failed or not.

    It is impossible to make TDD with such framework. May be there are some hidden features which can improve the speed?

    I've compared C++ tests with C# tests. C# is much faster, but they use the same MSTest framework.

    Wednesday, October 31, 2012 3:56 PM

Answers

  • Hello P Vladimir,

    Thank you for posting in the MSDN forum.

    “When I modify some test and run it again it take from 4 to 10 second to see if test is failed or not.”

    If you do some simple changes for a simple C++ unit test in VS2012, will the issue occur?

    As my understanding, it is possible since something more may need load and check if you modify the unit test, which is related to how you change the test.

    “I've compared C++ tests with C# tests. C# is much faster,”

    If you compare the two types of test, maybe it makes no sense. C++ tests and C# tests are different tests with different language, different develop way and something different on many aspects.

    In addition, the run speed may be related to your PC configuration, performance and resource usage. You can go to the Task Manager to check the CPU usage during test run. Other processes which are running may impact the test running at that time.

    If you want to improve the speed about Cpp unit test framework, we feel that it is the product performance issue, and these questions require a more in-depth level of support. We suggest you visit the following link to see the various paid support options that are available to better meet your needs.http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone, you could also submit this feature request: http://visualstudio.uservoice.com/forums/121579-visual-studio. The Visual Studio product team is listening to user voice there.

    Thank you for your understanding.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Proposed as answer by Amanda Zhu Friday, November 2, 2012 6:23 AM
    • Marked as answer by Amanda Zhu Friday, November 2, 2012 8:25 AM
    Thursday, November 1, 2012 3:49 AM
  • Hello Amanda,

    Thank you very much for reply.

    The only solution I have found with appropriate performance is usage of console test runner for post build step, like: 
    "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"  $(OutDir)$(TargetName)$(TargetExt)

    This solution works very fast, but it does not has IDE integration.

    One of the huge advantage of CppUnitTestFramework is IDE integration, but with existing performance we could not use it. So we will switch to console runner.

    • Marked as answer by Amanda Zhu Monday, November 5, 2012 12:07 AM
    Friday, November 2, 2012 1:42 PM

All replies

  • Hello P Vladimir,

    Thank you for posting in the MSDN forum.

    “When I modify some test and run it again it take from 4 to 10 second to see if test is failed or not.”

    If you do some simple changes for a simple C++ unit test in VS2012, will the issue occur?

    As my understanding, it is possible since something more may need load and check if you modify the unit test, which is related to how you change the test.

    “I've compared C++ tests with C# tests. C# is much faster,”

    If you compare the two types of test, maybe it makes no sense. C++ tests and C# tests are different tests with different language, different develop way and something different on many aspects.

    In addition, the run speed may be related to your PC configuration, performance and resource usage. You can go to the Task Manager to check the CPU usage during test run. Other processes which are running may impact the test running at that time.

    If you want to improve the speed about Cpp unit test framework, we feel that it is the product performance issue, and these questions require a more in-depth level of support. We suggest you visit the following link to see the various paid support options that are available to better meet your needs.http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophone, you could also submit this feature request: http://visualstudio.uservoice.com/forums/121579-visual-studio. The Visual Studio product team is listening to user voice there.

    Thank you for your understanding.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Proposed as answer by Amanda Zhu Friday, November 2, 2012 6:23 AM
    • Marked as answer by Amanda Zhu Friday, November 2, 2012 8:25 AM
    Thursday, November 1, 2012 3:49 AM
  • Hello Amanda,

    Thank you very much for reply.

    The only solution I have found with appropriate performance is usage of console test runner for post build step, like: 
    "C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\CommonExtensions\Microsoft\TestWindow\vstest.console.exe"  $(OutDir)$(TargetName)$(TargetExt)

    This solution works very fast, but it does not has IDE integration.

    One of the huge advantage of CppUnitTestFramework is IDE integration, but with existing performance we could not use it. So we will switch to console runner.

    • Marked as answer by Amanda Zhu Monday, November 5, 2012 12:07 AM
    Friday, November 2, 2012 1:42 PM