none
Visual Studio Test Runner not an option for TFS Build Definition

    Question

  • I am attempting to run some unit tests using the Fakes capability in VS2012.  Unit tests run fine on my developer box, but when checking in changes, the unit tests which use the fakes are failing the CI build with the following error:

    Test method xxxx threw exception:
    Microsoft.QualityTools.Testing.Fakes.Shims.ShimNotSupportedException: System.IO.File

    I believe I have determined the error to be that the unit tests must use the Visual Studio Test Runner, but I am unable to select that as an option (see image below).

    I have VS2012 installed on the build server.  Is there something I am missing to allow VS Test Runner to be used for running automated tests in our build process?

    Wednesday, October 24, 2012 2:48 PM

Answers

  • Hi rjwhite,

    Thanks for your post!

    From your description, I guess you use VS 2012 to connect to TFS 2010. Only when you connect to TFS 2012 from VS2012,  the Visual Studio Test Runner option will be displayed in Test Runner. If you connect to TFS 2010 from VS 2012 or VS 2010, this option cannot exist which is as the same with you. 

    We have isolated any tests using fakes to a category that is not run during the build. AFAIK, due to changes in the build template, if you want an easy experience you need VS12 and TFS12. 

    If you are using TFS 2010 you are effectively using mstest within your build process to execute your tests. Installing VS2012 does not suffice. These will fail since mstest does not comprehend the new framework.


    The new unit testing framework requires the new unit test executor ie vstest.console.exe. Once you upgrade to TFS2012 you will see an option in your process definition to run your tests using vstest.console.exe and only then will the new native UTs be discovered and executed.

    If  you to run your tests using TFS2010 you would have to effectively create a custom build definition that uses your version of RunTests which invokes the new executor and there too you won’t get all our integration eg. code coverage etc seamlessly.

    For more information, please refer to

    http://social.msdn.microsoft.com/Forums/eu/vstest/thread/3c670db1-71f5-446b-9513-d6b070fc2497

    Hope it helps!

    Best Regards,


    Cathy Kong [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, October 25, 2012 7:04 AM
    Moderator

All replies

  • Hello Rjwhite,

    We have a forum Team Foundation Server-Build Automation for this issue. In order to provide better support, I will move this thread.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, October 25, 2012 5:14 AM
  • Hi rjwhite,

    Thanks for your post!

    From your description, I guess you use VS 2012 to connect to TFS 2010. Only when you connect to TFS 2012 from VS2012,  the Visual Studio Test Runner option will be displayed in Test Runner. If you connect to TFS 2010 from VS 2012 or VS 2010, this option cannot exist which is as the same with you. 

    We have isolated any tests using fakes to a category that is not run during the build. AFAIK, due to changes in the build template, if you want an easy experience you need VS12 and TFS12. 

    If you are using TFS 2010 you are effectively using mstest within your build process to execute your tests. Installing VS2012 does not suffice. These will fail since mstest does not comprehend the new framework.


    The new unit testing framework requires the new unit test executor ie vstest.console.exe. Once you upgrade to TFS2012 you will see an option in your process definition to run your tests using vstest.console.exe and only then will the new native UTs be discovered and executed.

    If  you to run your tests using TFS2010 you would have to effectively create a custom build definition that uses your version of RunTests which invokes the new executor and there too you won’t get all our integration eg. code coverage etc seamlessly.

    For more information, please refer to

    http://social.msdn.microsoft.com/Forums/eu/vstest/thread/3c670db1-71f5-446b-9513-d6b070fc2497

    Hope it helps!

    Best Regards,


    Cathy Kong [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, October 25, 2012 7:04 AM
    Moderator
  • Hi Cathy, we recently upgraded from TFS 2010 to 2012 Update 1, and are using VS 2012.  If I use the DefaultTemplate.11.1.xaml build template then the Visual Studio Test Runner shows up as an option to choose from in the drop down, but if I use our build template that was upgraded from TFS 2010, it does not show up as an option (as shown in the original poster's screenshot).

    I opened up the DefaultTemplate.11.1.xaml and copied the entire "If Not DisableTests" block into my build template, overwriting the "If Not DisableTests" block that was there.  This is the block that contains all of the code/logic to run unit tests.  This new block that I copied over now includes logic to check "If spec is AgileTestPlatformSpec" then "Run Visual Studio Test Runner for Test Sources", so I'm pretty sure my build template would be able to use VS Test Runner now, but it still does not show up as an option to choose from in the Build Definition's "Add/Edit Test Run" dialog's "Test runner" drop down.

    Do you know what else I need to change in my build template in order to get that option to show up?

    Thanks


    - Dan - "Can't never could do anything"





    • Edited by deadlydog Wednesday, March 27, 2013 8:40 PM
    • Proposed as answer by RKPatrick Tuesday, April 30, 2013 8:03 PM
    Wednesday, March 27, 2013 8:34 PM
  • We are having the same problem. Upgraded from 2010 VS and TFS to 2012 and we don't see the VS test runner option. Still waiting for an answer for this issue.
    Wednesday, April 10, 2013 7:15 PM
  • Since this thread is already marked as answered because the original poster was using TFS 2010, I've created a new post for our problem.

    - Dan - "Can't never could do anything"

    Wednesday, April 10, 2013 8:13 PM