none
Code coverage for web tests RRS feed

  • Question

  • Hi,

    How can I collect code coverage in TFS2010 CI environent for web tests?

    My build server builds product code (bunch of databases, WCF services and MVC apps), runs unit tests and puts ~30 WebDeploy packages to build drop location.

    Deployment process picks up the build, deploys DBs, services and apps to lab environment, and runs web tests. Standard lab environment is 1 SQL server, 1 web server and 1 client. Web tests are being executed on a client against web server. If necessary, I'm fine with running web tests right on web server.

    However, in MTM Test Settings there is no way to configure code coverage collection. Intellitrace and test impact are there, but not code coverage.

    Is there any way to collect code coverage in lab environment? Not on build agent, but on lab VM where I deploy product and run tests.


    Regards, Dmitry



    Wednesday, September 26, 2012 8:47 PM

Answers

All replies

  • In TFS 2010 you need to point out the assemblies, to be instrument for code coverage, by pointing them out in a testsettings file. By doing so you should be able to collect codecoverage at least if you execute automated webtests.

    Wednesday, September 26, 2012 10:05 PM
  • Thank you for the response, Mattias. As far as I know, testsettings files are not used during lab test runs. Instead, it uses test settings configured in MTM.

    The point is to have code coverage collected for integration tests in a lab environment after deployment, not on a local machine, and not on a build agent.

    Am I missing something?


    Regards, Dmitry

    Wednesday, September 26, 2012 10:43 PM
  • Hi Dmitry,

    Thank you for posting in the MSDN forum.

    Based on your description, this issue is related to MTM, so I will move it toTesting with Visual Studio Test Manager (MTM) forum, there you will get better support with experts. Thanks for your understanding.

    Best Regards,


    Jack Zhai [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, September 27, 2012 8:52 AM
    Moderator
  • Oh, Are you executing your automated test through a MTM test plan and a test agent ?

    Thursday, September 27, 2012 2:59 PM
  • Yes, I'm using MTM test plans and test agents. Full blown ALM.

    Regards, Dmitry

    Thursday, September 27, 2012 5:50 PM
  • Hm, have you concidered moving to MTM/TFS 2012 ? At least there is a CodeCoverage option in MTM2012.

    Is it Coded UI or WebTests  ? Other options would perhaps be to run your automated test through MSTest to use the testsettings file.

    Thursday, September 27, 2012 6:12 PM
  • If you mean upgrading our TFS server to 2012, then it is not an option right now.

    You are confusing me when referring to MTM or testsetting files. We only manage test plans in MTM, and use testsettings for local test execution on dev boxes.

    We never start test runs in testing environments from MTM. It is TFS workflow (LabDefaultTemplate) which manages deployment and integration test execution. Do you suggest using different process template which is aware of testsettings files?

    All tests are webtests at the moment, but we plan to have Coded UI tests for MVC apps added soon.


    Regards, Dmitry


    • Edited by Demchuk Friday, September 28, 2012 1:53 AM
    Friday, September 28, 2012 12:53 AM
  • Anybody?


    Regards, Dmitry

    Tuesday, October 2, 2012 4:14 AM
  • Hi Dmitry,

    I am confusing with your words. Can you help understand the situation by ask following questions?

    1. You just use MTM to manage test plan, not run any test with MTM, right?

    2. You run the web test in local VS, right? 

    3. You want to do code coverage for the web test in local VS, right?

    Tuesday, October 2, 2012 7:02 AM
  • Hi jacson,

    Thank you for trying to help. Here you are.

    1. You just use MTM to manage test plan, not run any test with MTM, right?
    Yes

    2. You run the web test in local VS, right?
    The question is how collect them in virtual environment. See "TFS 2010 Workflow" section here, that's how I run webtests: http://blogs.msdn.com/b/mvpawardprogram/archive/2011/10/17/automated-build-deploy-test-using-tfs-2010.aspx

    3. You want to do code coverage for the web test in local VS, right?
    No. In lab environment where I run web tests. That's the point, collecting it during CI build-test-deploy.

    I know how to collect code coverage on local machine. This is not what I am looking for, but everybody keeps talking only about that. Sometimes I feel like nobody besides my team uses Hyper-V virtual labs with TFS :)


    Regards, Dmitry


    • Edited by Demchuk Tuesday, October 2, 2012 7:58 AM
    Tuesday, October 2, 2012 7:51 AM
  • Hi Demchuk,,

    Thank you for your post.

    I am trying to involve someone to further look at this issue. There might be some time delay. Appreciate your patience.

    Thank you for your understanding and support.

    Regards,


    Lily Wu [MSFT]
    MSDN Community Support | Feedback to us

    Wednesday, October 3, 2012 6:22 AM
    Moderator
  • Hi Lily,

    Thank you for folowing up. Very much appreciate it.


    Regards, Dmitry

    Wednesday, October 3, 2012 7:03 AM
  • You can only use code coverage when you run an automated test, and only from the computer that is running the test. Remote collection is not supported. You cannot use code coverage when you run tests from a test plan using Microsoft Test Manager.

    For more information, please see:

    How to: Configure Code Coverage Using Test Settings for Automated Tests

    Thank you

    Wednesday, October 3, 2012 11:42 PM
    Moderator
  • Right. So, when deployment process (based on LabDefaultTwmplate.xaml) executes webtests on web server against localhost, isn't it automated test execution on local machine?

    Regards, Dmitry

    Thursday, October 4, 2012 12:50 AM
  • You can use code coverage only when you run an automated test from Microsoft Visual Studio 2010 or mstest.exe, and only from the machine that runs the test.

    Please see:

    Setting Up Machines and Collecting Diagnostic Information Using Test Settings

    Thank you


    Thank you Teodora Stanev

    Thursday, October 4, 2012 1:31 PM
    Moderator