locked
Unable to cover all code with unit tests RRS feed

  • Question

  • Consider the following code from a C++ native unit test project:

    struct ME { ME() {} ~ME() {} }; void TestMe() { try
    {
    ME e;
    throw e;
    } //<<< (A) catch (...) {} } namespace CppUnitTest { TEST_CLASS(METests) { public: TEST_METHOD(AAAAAAA) { TestMe(); } }; }

    The code coverage analysis in Visual Studio 2015 Enterprise, Update 3 works really good. However, I cannot figure out how to cover the TestMe() function entirely.

    If a destructor exists and the object is thrown, then line (A) is marked red and the results indicate that 7 blocks of TestMe() are covered and 2 blocks are not covered.

    I have no clue as to what these two blocks of code might be. Any suggestions?


    Friday, June 17, 2016 6:10 PM

Answers

  • Hi Frank,

    Thanks for your friendly response:)

    Like this document here:

    https://msdn.microsoft.com/en-us/library/cc667391(v=vs.100).aspx

    It shared us what the real Code coverage data is and the relationship between Code Blocks, Lines of Code, and Partial Lines.

    So it is the real data how the code coverage tool collects now.

    But if you have good suggestion for the code coverage tool/data results, one idea is that you could submit a feature request: http://visualstudio.uservoice.com/forums/121579-visual-studio. The Visual Studio product team is listening to user voice there. You can send your idea there and people can vote.

    Sincerely,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    • Marked as answer by Frank Heimes Tuesday, June 28, 2016 4:34 PM
    Tuesday, June 28, 2016 3:12 AM

All replies

  • Hi Frank Heimes,

    >>I have no clue as to what these two blocks of code might be. Any suggestions?

    What I know is that the coded coverage is used to analyze the IL language not the real code lines in your Code Editor window. So if you want to know the real reason, maybe we would analyze the IL language using certain tool.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, June 20, 2016 11:24 AM
  • Hi Jack,

    I stated that I'm using a C++ native unit test project to emphasize that it is not a C++/CLI project. As such, to my knowledge, no IL code is generated neither for my classes nor for my test methods.

    The feature of purely native C++ unit tests is new to VS 2013+, AFAIK.
    (New Project > Templates > Visual C++ > Test > Native Unit Test Project | Visual C++)

    Nevertheless, I started ILDasm and loaded the test DLL just to get the unsurprising error message saing that the DLL does not contain a valid CLR header.

    You could easily create a unit test project yourself, paste my code and examine the resulting DLL.
    It compiles and executes without changes.

    Best,
    Frank


    Thursday, June 23, 2016 12:36 PM
  • Hi Frank,

    I test it in my side using the VS2015, I got the same result as yours.

    One important issue is that it is not the "lines", it is the "Blocks".

    Like this thread here:

    http://stackoverflow.com/questions/4958209/the-blocks-in-code-coverage-with-vs2010

    We would know that what the block is.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Friday, June 24, 2016 6:55 AM
  • Hi Jack,

    from the article I learned how code coverage sets up blocks.

    But what are the consequences for my initial problem?
    My current conclusion is that this is an artifact of the way the code coverage works, that it may often introduce bogus blocks that can never be covered and that a code coverage of 100% for a module thus is hardly ever possible.

    But this undermines the whole point of that kind of statistics; i.e. to allow me to skim through all classes (most of which saying 100% covered) and only deal with code fragments that really need additional tests.

    Best regards,
    Frank

    Monday, June 27, 2016 8:12 AM
  • Hi Frank,

    Thanks for your friendly response:)

    Like this document here:

    https://msdn.microsoft.com/en-us/library/cc667391(v=vs.100).aspx

    It shared us what the real Code coverage data is and the relationship between Code Blocks, Lines of Code, and Partial Lines.

    So it is the real data how the code coverage tool collects now.

    But if you have good suggestion for the code coverage tool/data results, one idea is that you could submit a feature request: http://visualstudio.uservoice.com/forums/121579-visual-studio. The Visual Studio product team is listening to user voice there. You can send your idea there and people can vote.

    Sincerely,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    • Marked as answer by Frank Heimes Tuesday, June 28, 2016 4:34 PM
    Tuesday, June 28, 2016 3:12 AM
  • Eventually, I found the oversight in my considerations.

    The block belonging to the closing bracket was never executed because the function always threw.

    So if I rewrite it like this and exercise it for false and true, then all lines become blue, i.e. covered:

    void TestMe(bool doThrow)
    {
      try
      {
        ME e;
        if (doThrow)
          throw e;
      }
      catch (...) {}
    }
    But that doesn't help you if the function always throws, even if it is declared __declspec(noreturn). The compiler apparently always includes some code that is never executed in scopes that always throw.
    • Edited by Frank Heimes Wednesday, June 29, 2016 7:59 AM Added last paragraph
    Wednesday, June 29, 2016 7:45 AM