none
running unit tests causing build to fail

    Question

  • so i've looked at this for a while now, and i'm stumped.

    I am trying to run a set of unit tests as a part of my nightly build.  When the build runs, everything works great until it gets to the point of running the tests.  Then it fails, seemingly without explanation.

    The relevant portion of the build log looks as follows.

    Target RunTestWithConfiguration:

    TestToolsTask MetadataFile="c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\TruAssets Solution\TruAssets Solution.vsmdi" TestLists="Unit Tests/Domain Unit Tests" SearchPathRoot="c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\..\Binaries\Mixed Platforms\Debug\" PathToResultsFilesRoot="c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\..\TestResults" PublishServer="http://da-truis-wwb04:8080/" TeamProject="TruAssets" PublishBuild="1.0.060827.03" BuildFlavor="Debug" Platform="Mixed Platforms"

    Loading c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\TruAssets Solution\TruAssets Solution.vsmdi...

    C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets : warning : TestTypes\{37e36796-fb51-4610-8d5c-e00ceaa68b9f} could not be loaded because the TIP could not be instantiated for the following reason(s): Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.LoadTestPackage, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets : warning : TestTypes\{4e7599fa-5ecb-43e9-a887-cd63cf72d207} could not be loaded because the TIP could not be instantiated for the following reason(s): Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.LoadTestPackage, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets : warning : TestTypes\{7a24f8ff-85da-d664-ef44-83ff4743dae6} could not be loaded because the TIP could not be instantiated for the following reason(s): Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Tips.ManualTest.Tip, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets : warning : TestTypes\{982b8c01-1a8a-48f5-b98a-67ee64bc8687} could not be loaded because the TIP could not be instantiated for the following reason(s): Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.Tips.GenericTest.Tip, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    C:\Program Files\MSBuild\Microsoft\VisualStudio\v8.0\TeamBuild\Microsoft.TeamFoundation.Build.targets : warning : TestTypes\{fa4c85ff-1236-4463-b954-3ba2f0eabd95} could not be loaded because the TIP could not be instantiated for the following reason(s): Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.LoadTestPackage, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    Search path(s) for tests:

    c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\..\Binaries\Mixed Platforms\Debug\

    c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\Sources\TruAssets Solution

    Search path(s) for default run configuration:

    c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\Sources\TruAssets Solution

    MSBUILD : warning : c:\TeamBuilds\TruAssets\TruAssets Nightly Build v2\BuildType\..\Sources\TruAssets Solution\TruAssets Solution.vsmdi

    MSBUILD : warning : Failed to instantiate type 'Microsoft.VisualStudio.TestTools.WebStress.WebTestRunConfig, Microsoft.VisualStudio.QualityTools.LoadTest, Version=8.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a': Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.LoadTest' or one of its dependencies. The system cannot find the file specified.

    The previous error was converted to a warning because the task was called with ContinueOnError=true.

    Build continuing because "ContinueOnError" on the task "TestToolsTask" is set to "true".

    Done building target "RunTestWithConfiguration" in project "TFSBuild.proj".

     

    Also, it is worth noting here that my TestResults folder is completely empty.

    Please let me know if you have any ideas what this could be!

    thanks,

    _howard

    Sunday, August 27, 2006 7:07 AM

Answers

  • This is a bug in the Test functionality. When editing test lists, we will serialise all your 'test elements' into the file. This is a full object graph serialization -- not just choice bits. Additionally, when we load it back up, we will load the *whole* object graph back in. As we serialize/deserialize we look at the full type name, and thus try to resolve that type reference.

    What you are seeing here is a manfiestation of this behaviour -- it's trying to load the *whole* object graph back in on the build machine, but the load test assemblies etc are not on that machine because they are Team Test/Team Suite only.

    We're sorry for this -- we are currently planning on addressing this in SP1.

    Friday, September 01, 2006 5:40 PM

All replies

  • Hi Howard,

    What version of Visual Studio 2005 do have installed on your build machine? I suspect that it is missing some of the test SKUs.

    Thanks,

    Jason

    Monday, August 28, 2006 3:14 PM
  • Hi Jason,

    I have VSTS : Software Developers installed on the build machine.  Also, I'm not trying to run any other test type than unit tests.

    thanks,

    _howard

    Monday, August 28, 2006 3:16 PM
  • Hi Howard,

    Sorry it took so long to reply, but I wanted to verify my findings. If you take a look at the following site:

    http://msdn.microsoft.com/vstudio/teamsystem/products/compare/default.aspx

    You should see a comparison of the different Visual Studio Team System skus. Software Developers does not include some of the Test components that your tests seem to expect (like perhaps Load Testing?).

    If the unit tests work on your developer machine, then I am curious what sku you have there. I would have guessed they were the same. Can you verify what sku you have on your developer machine and your build machine by doing the following:

    1. Open Visual Studio
    2. Go to Help -> About Microsoft Visual Studio
    3. Look in the upper left hand corner to see the sku that you have

    (Please forgive me for being so specific, I want to make sure others that run into this problem know exactly what to do)

    Compare the two machines to see what's different. My guess is that you have Team Suite on your developer box, which includes ALL other VSTS skus.

    Let me know what you find.

    Thanks, Jason

     

    Tuesday, August 29, 2006 12:27 PM
  • Ah - no problem.  I'm running Team Suite on my development machine.  However, the test list that I'm trying to automate on the server contains only unit tests.

    _howard

    Tuesday, August 29, 2006 3:00 PM
  • Yeah, I remember you saying that. I am not sure why Visual Studio is trying to load the other assemblies. Let me look a little deeper...
    Tuesday, August 29, 2006 3:06 PM
  • This is a bug in the Test functionality. When editing test lists, we will serialise all your 'test elements' into the file. This is a full object graph serialization -- not just choice bits. Additionally, when we load it back up, we will load the *whole* object graph back in. As we serialize/deserialize we look at the full type name, and thus try to resolve that type reference.

    What you are seeing here is a manfiestation of this behaviour -- it's trying to load the *whole* object graph back in on the build machine, but the load test assemblies etc are not on that machine because they are Team Test/Team Suite only.

    We're sorry for this -- we are currently planning on addressing this in SP1.

    Friday, September 01, 2006 5:40 PM
  • Is there a workaround to this (editting the XML)??

    Until SP1 releases, we can't run our tests in the Nightly Build because of this.

    Thanks in advance,

    Bob Hanson

    Friday, November 03, 2006 7:24 PM
  • We use a build server. Some team projects are executing tests successfully, but the tests on my team projects do not execute. Giving exactly the same warnings as above. Any explanation why some work and others don't?

    Thanks

    Tuesday, November 07, 2006 10:37 AM