locked
TFS 2012 Code Coverage - Non solution assemblies getting covered RRS feed

  • Question

  • Hi guys,

    This is a cross-post from Visual Studio Team System - Testing (TFS 2012 Code Coverage - Non solution assemblies getting covered). The moderator there suggested I'd get better support here.

    Here's my problem:

    Solution A:
      - Component A
      - UnitTest A

    Build definition A runs the build for solution A on the TFS 2012 with default coverage settings. I get code coverage results for both projects (CompA and UnitTestA):
    Comp A -> 100%
    UnitTest A -> 100%
    Total -> 100%

    That's fine.

    Solution B:
      - Application B (has a binary reference to CompA.dll, CompA.dll is checked in in the project's source folder)
      - UnitTest B

    When I run build definition B, which is supposed to build AppB, I get code coverage results for AppB, UnitTestB AND CompA:
    App B -> 100%
    UnitTest B -> 100%
    Comp A -> 50% (obviously b/c App B won't execute every code path in Comp A)
    Total -> 70%

    That's wrong Comp A is just a reference. It should not be included in the code coverage, because it wrecks the total.

    I know I can explicitly state which assemblies to cover in a runsettings file, but I'd rather avoid that.

    Thanks!

    Cheers,
    Markus

    Thursday, October 11, 2012 8:36 AM

Answers

  • Here's the accepted answer from the other thread:

    Hi Markus,

        What you are seeing is the expected behavior. When you enable code coverage in TFS 2012 builds, it tries to include all the assemblies that get loaded as part of the test run and for which the symbols are available. In your case when you run tests for Solution B, comp A also gets loaded due to which you are seeing it in the code coverage report. This behavior is different from VS2012 IDE because in TFS builds we do not have a way to detect the solution outputs. If you want to exclude compA from code coverage, you'll have to use a custom run settings.

    Also, if you think that this is something that you feel should be addressed, please post your feedback on Visual Studio User Voice(http://visualstudio.uservoice.com)

    Regards,

    Nitya Kumar Sharma

    • Marked as answer by MIrle Monday, October 15, 2012 9:25 AM
    Monday, October 15, 2012 9:24 AM

All replies

  • That's still the solution I'm aware of that works. I'm not aware of a method to specify say a filespec which lists the assemblies to cover and which not.

    It's a nice feature though. If you can't find it, post a suggestion on uservoice and tweet it to Peter provost to get his teams attention :).


    My blog: blog.jessehouwing.nl

    Thursday, October 11, 2012 12:42 PM
  • Hi Jesse,

    Well, I don't really want to specify the projects to cover, if there's a feature that doesn't require you to. I could use the runspec, but that's maintenance effort and error prone when you add new projects to the solution.

    I doubt that what I'm seeing is the intended behaviour since in my mind it doesn't make any sense. When you enable coverage in a build definition for a solution, only projects in that solution should get covered.

    So either I'm missing something trivial (thus my post), or it's a bug (I'm happy to file :)

    Btw what are the criteria for the test runner to select assemblies for coverage?

    Cheers,
    Markus

    Thursday, October 11, 2012 3:01 PM
  • Hi Markus, 

    Thanks for your reply.

    Please refer to Jack’s reply in your another post: http://social.msdn.microsoft.com/Forums/en-US/vststest/thread/59c153d5-3416-488f-854c-fde8d937cb26.  


    John Qiao [MSFT]
    MSDN Community Support | Feedback to us

    Friday, October 12, 2012 5:41 AM
    Moderator
  • Hi John,

    Thanks. I posted a follow up there.

    Please have a look, thank you!

    Cheers,
    Markus

    Friday, October 12, 2012 11:13 AM
  • Hi Markus, 

    Thanks for your reply.

    I will tracking Jake’s reply.  


    John Qiao [MSFT]
    MSDN Community Support | Feedback to us

    Monday, October 15, 2012 2:20 AM
    Moderator
  • Here's the accepted answer from the other thread:

    Hi Markus,

        What you are seeing is the expected behavior. When you enable code coverage in TFS 2012 builds, it tries to include all the assemblies that get loaded as part of the test run and for which the symbols are available. In your case when you run tests for Solution B, comp A also gets loaded due to which you are seeing it in the code coverage report. This behavior is different from VS2012 IDE because in TFS builds we do not have a way to detect the solution outputs. If you want to exclude compA from code coverage, you'll have to use a custom run settings.

    Also, if you think that this is something that you feel should be addressed, please post your feedback on Visual Studio User Voice(http://visualstudio.uservoice.com)

    Regards,

    Nitya Kumar Sharma

    • Marked as answer by MIrle Monday, October 15, 2012 9:25 AM
    Monday, October 15, 2012 9:24 AM