locked
Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables ... when unit testing with Fakes on TFS 2012 RRS feed

  • Question

  • Trying to run unit tests on a build server using TFS 2012 but getting the error below:

    Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationException: Failed to resolve profiler path from COR_PROFILER_PATH and COR_PROFILER environment variables.

    The unit tests fail on this line of code: using (ShimsContext.Create())

    Visual Studio 2012 CU3 (with Fakes assemblies) and VisualStudio 2013 are loaded on the build server. Some of the unit tests are also failing when run in Visual Studio, but not all. All of them fail on the build server when run under the build process.

    The build definition specifies to use MSTEST.exe for testing.

    Any clues on what to check to fix this?

    Wanted to add the call stack which shows calls to intellitrace, which is not being used, nor is it specified in the build definition:

    Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.ResolveProfilerPath()
    Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.IntelliTraceInstrumentationProvider.Initialize()
    Microsoft.QualityTools.Testing.Fakes.UnitTestIsolation.UnitTestIsolationRuntime.InitializeUnitTestIsolationInstrumentationProvider()
    Microsoft.QualityTools.Testing.Fakes.Shims.ShimRuntime.CreateContext()
    Microsoft.QualityTools.Testing.Fakes.ShimsContext.Create()


    • Edited by bogatiy Tuesday, December 10, 2013 7:24 PM modified to correct problem description
    Tuesday, December 10, 2013 6:47 PM

Answers

  • Did you specify the Test runner to be Visual Studio Test Runner?  You can find this by editing the build definition > Under Automated Tests, open the Test Source dialog > Under Test runner, select Visual Studio Test Runner.
    • Marked as answer by bogatiy Thursday, December 12, 2013 5:46 PM
    Wednesday, December 11, 2013 5:58 PM
  • Ok, specified the Test Runner to be Visual Studio Test Runner ... and ... those failing tests pass (several others now fail, but maybe for good cause, haven't looked into it yet).

    However, the bigger issue is that once the Visual Studio Test Runner is specified, then Code Coverage no longer works and returns as 0%. As we are using a gated check in based on code coverage, it looks like we can't currently use Visual Studio Test Runner. Now using TFS version 2012, but going to 2013. Perhaps this situation is improved in TFS 2013?

    Update: I was using a .testSettings file to initiate code coverage. However, following the instructions for 'Analyzing Code Coverage in the Build' at http://msdn.microsoft.com/en-us/library/dd537628.aspx made the changes and tried it again. Now the code coverage is working, and the unit tests are passing. And more assemblies are being analyzed.

    So it's working ....

    • Edited by bogatiy Thursday, December 12, 2013 5:45 PM
    • Marked as answer by John QiaoModerator Friday, December 13, 2013 4:22 AM
    Thursday, December 12, 2013 5:12 PM

All replies

  • Is this a VS 2012 or 2013 project you are building?  What version did you install of each on the build server; professional, premium, ultimate?  Also, if you are referencing a .testsettings file, remove it.  It will not work if you are referencing that for running your tests.  It forces the tests to run in legacy mode which prevents the IntelliTrace profiler instrumentation to occur.

    Wednesday, December 11, 2013 1:58 AM
  • Hi Bogatiy,

    Thanks for your post.

    You selected TFS 2012 default build process template in your build definition?

    If you selected default build process template in your build definition, and configured use MSTest.exe to run test in build definition, it will try to invoke VS 2010 MSTest.exe to run your test project by default. Your test project can be executed using VS 2010 MSTest.exe? Please check which version of MSTest.exe be invoked in your detailed build log.

    On your build agent machine, try to run your test project using VS 2012, ensure your test cases can be executed correctly.


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    Wednesday, December 11, 2013 5:22 AM
    Moderator
  • This is a VS 2013 project, though originally a VS 2012 project opened and saved from VS 2013.

    The version of VS installed is Premium. Both 2012 and 2013 are used on development machines as well as on the build server.

    Yes, there is a .testsettings file being referenced in the build Process under Automated tests. This .testsettings is where code coverage settings are specified.

    When the .testsettings file is removed, how does one go about specifying code coverage? How can code coverage be activated without a .testsettings file?

    When the .testsettings file is removed, there is no different behavior in the unit testing. The same error occurs. I'm not sure how to verify which MSTEST.exe is running or where to find this in a log. The logs made available with the build include an 'ActivityLog.AgentScope' file, which has reference to this file : C:\Program Files (x86)\Microsoft Visual Studio 11.0\Common7\IDE\MSTest.exe

    On the build server using Visual Studio 2012 or 2013, the unit tests run fine.

    On one dev box some of the unit tests fail similarly to how they fail during the build process (the intellitrace instrumentation error), and this happens whether in VS 2012 or 2013. What's most interesting about this, is only a subset of the tests fail, meaning some pass. On the build server they all fail all the time when running under the TFS build rather than visual studio.

    Maybe the bottom line here is that Fakes is not compatible with code coverage during TFS build?


    • Edited by bogatiy Wednesday, December 11, 2013 1:59 PM
    Wednesday, December 11, 2013 12:41 PM
  • Did you specify the Test runner to be Visual Studio Test Runner?  You can find this by editing the build definition > Under Automated Tests, open the Test Source dialog > Under Test runner, select Visual Studio Test Runner.
    • Marked as answer by bogatiy Thursday, December 12, 2013 5:46 PM
    Wednesday, December 11, 2013 5:58 PM
  • Ok, specified the Test Runner to be Visual Studio Test Runner ... and ... those failing tests pass (several others now fail, but maybe for good cause, haven't looked into it yet).

    However, the bigger issue is that once the Visual Studio Test Runner is specified, then Code Coverage no longer works and returns as 0%. As we are using a gated check in based on code coverage, it looks like we can't currently use Visual Studio Test Runner. Now using TFS version 2012, but going to 2013. Perhaps this situation is improved in TFS 2013?

    Update: I was using a .testSettings file to initiate code coverage. However, following the instructions for 'Analyzing Code Coverage in the Build' at http://msdn.microsoft.com/en-us/library/dd537628.aspx made the changes and tried it again. Now the code coverage is working, and the unit tests are passing. And more assemblies are being analyzed.

    So it's working ....

    • Edited by bogatiy Thursday, December 12, 2013 5:45 PM
    • Marked as answer by John QiaoModerator Friday, December 13, 2013 4:22 AM
    Thursday, December 12, 2013 5:12 PM
  • I'm seeing the same error as described above, but I'm definitely using the Visual Studio Test Runner.  Are there any other reasons why that error messages would be shown that anybody might know?

    I have confirmed VSPerf is disabled (i.e. environment variables are not defined).  When I run the unit tests through vstest.console.exe they run fine, it is only through the build service that they seem to fail.

    Tuesday, January 21, 2014 10:40 PM
  • I had this issue from a solution imported from VS2010 -> VS2012 -> VS2013

    Disabling IntelliTrace from the testsettings file produced a different error. My end solution was to remove the legacy test settings files, and create a new VS2013 solution and re-add all the projects to it. Then everything worked fine after that.

    Hope that helps someone out!

    Friday, February 20, 2015 9:52 PM