locked
How to remote debug unit tests with Visual Studio 2013 Pro? RRS feed

  • Question

  • I have a weird problem when my unit tests are working correctly on my machine but some of them fail during TFS build (on TFS server machine). It's not some configuration/run error - the expected results are off. Naturally, I want to run my unit tests on remote machine (TFS server) and debug it remotely on my machine.

    How do I achieve that with Visual Studio 2013 Pro?


    Tuesday, January 21, 2014 11:30 AM

Answers

  • Hi,

    This is achievable but it is far more work than it should be (other than installing Visual Studio on the remote machine). I have had to do this many times to troubleshoot integration tests running on various machines. I will make an assumption first: that your unit tests are NOT running interactively. If they are, you can use a message box below instead of a delay below.

    As a quick heads up, work out whether the unit tests work on the build system by launching them interactively. Just go the folder where they are built and use one of:

    1. mstest.exe, or

    2. vstest.console.exe

    If you can run the tests successfully (just open up the output .TRX file to confirm in Visual Studio) then it might be that your unit tests have some dependencies on having to run as an interactive user or with certain credentials: I find these can be very subtle, especially with data driven tests where the data source can reside elsewhere.

    Failing that, you are forced to do this:

    1. Install the Visual Studio Remote Debugger on the Build system (if your build system is virtualized: snapshot it first, so you can quickly get it back; if not, consider creating a new Build Agent where the tests will run and then target your build to the new Agent using tags).

    2. Use Visual Studio to attach to one of the processes on your build system - doesn't matter which. If this works, you know the connectivity and authentication is good and you can proceed.

    3. In your unit tests, add this to [AssemblyInitialize] - it will just loop forever until you attach and then manually set bcontinue to true. If your tests run interactively, use System.Windows.MessageBox; otherwise, use this:

    [AssemblyInitialize]
    public void WaitForAttach()
    {
        // Will loop forever on the build when the unit tests run - gives you chance to attach the debugger and then change bcontinue to 'true'. 
        bcontinue = false;
        while(!bcontinue)
        {
            System.Theading.Thread.Sleep(3000);
        }
    }

    4. Now kick off your build. At some point, your unit tests will be run and (depending on how they are configured) they will be running under mstest.exe, qtagent.exe or vstest.executionengine.exe (can't remember exactly which and I might be wrong on all counts). Attach to processes that looks like they are executing your unit tests :-) You should be in the AssemblyInitialize loop above which will loop forever.

    5. Change bcontinue to true on the test you want to execute. Set a breakpoint on the test method you want to test. Press F5 to continue.

    6. Your breakpoint will be hit. Depending on how your build system is configured, you MIGHT need to load the symbols manually to step through the unit tests. You will be able to locate them on the build system (the .PDB files).

    Further to Jack's comment above: There is a free tool I wrote that will let you run remote tests under the debugger providing you are using the Test Controller and Test Agent (I am not sure if these are part of Visual Studio Professional). Instructions on configuring the tool with TC/TA can be found here and a full blog post is here explaining the problems and difficulties. 

    Grey Ham


    Blog: http://www.havecomputerwillcode.com/blog





    • Edited by Grey Ham Thursday, January 23, 2014 4:28 AM
    • Marked as answer by Michael M Logutov Thursday, January 23, 2014 5:45 AM
    Thursday, January 23, 2014 1:36 AM

All replies

  • Hi Michael,

    Actually we could run a unit test remotely with test controller and Test Agent with VS IDE. But if you want to debug all your unit tests, you can do this only while you are running tests locally, that is, on your own computer. You cannot debug while you are running all tests remotely, by using a test controller and test agents.

    Reference:

    http://msdn.microsoft.com/en-us/library/ms182484.aspx

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    Wednesday, January 22, 2014 7:55 AM
  • Hi,

    This is achievable but it is far more work than it should be (other than installing Visual Studio on the remote machine). I have had to do this many times to troubleshoot integration tests running on various machines. I will make an assumption first: that your unit tests are NOT running interactively. If they are, you can use a message box below instead of a delay below.

    As a quick heads up, work out whether the unit tests work on the build system by launching them interactively. Just go the folder where they are built and use one of:

    1. mstest.exe, or

    2. vstest.console.exe

    If you can run the tests successfully (just open up the output .TRX file to confirm in Visual Studio) then it might be that your unit tests have some dependencies on having to run as an interactive user or with certain credentials: I find these can be very subtle, especially with data driven tests where the data source can reside elsewhere.

    Failing that, you are forced to do this:

    1. Install the Visual Studio Remote Debugger on the Build system (if your build system is virtualized: snapshot it first, so you can quickly get it back; if not, consider creating a new Build Agent where the tests will run and then target your build to the new Agent using tags).

    2. Use Visual Studio to attach to one of the processes on your build system - doesn't matter which. If this works, you know the connectivity and authentication is good and you can proceed.

    3. In your unit tests, add this to [AssemblyInitialize] - it will just loop forever until you attach and then manually set bcontinue to true. If your tests run interactively, use System.Windows.MessageBox; otherwise, use this:

    [AssemblyInitialize]
    public void WaitForAttach()
    {
        // Will loop forever on the build when the unit tests run - gives you chance to attach the debugger and then change bcontinue to 'true'. 
        bcontinue = false;
        while(!bcontinue)
        {
            System.Theading.Thread.Sleep(3000);
        }
    }

    4. Now kick off your build. At some point, your unit tests will be run and (depending on how they are configured) they will be running under mstest.exe, qtagent.exe or vstest.executionengine.exe (can't remember exactly which and I might be wrong on all counts). Attach to processes that looks like they are executing your unit tests :-) You should be in the AssemblyInitialize loop above which will loop forever.

    5. Change bcontinue to true on the test you want to execute. Set a breakpoint on the test method you want to test. Press F5 to continue.

    6. Your breakpoint will be hit. Depending on how your build system is configured, you MIGHT need to load the symbols manually to step through the unit tests. You will be able to locate them on the build system (the .PDB files).

    Further to Jack's comment above: There is a free tool I wrote that will let you run remote tests under the debugger providing you are using the Test Controller and Test Agent (I am not sure if these are part of Visual Studio Professional). Instructions on configuring the tool with TC/TA can be found here and a full blog post is here explaining the problems and difficulties. 

    Grey Ham


    Blog: http://www.havecomputerwillcode.com/blog





    • Edited by Grey Ham Thursday, January 23, 2014 4:28 AM
    • Marked as answer by Michael M Logutov Thursday, January 23, 2014 5:45 AM
    Thursday, January 23, 2014 1:36 AM
  • Thanks for such detailed explanation.

    I've actually considered doing that myself but first I wanted to check if there are no native solutions to the problem. It's pity there is none. Hope it will be in the next VS release.

    Thursday, January 23, 2014 5:45 AM