none
Failed tests do not get uploaded in TFS 2010

    Question

  • TFS 2008 uploaded all test results - suceeded and failed - during build. TFS 2010 only uploads succeeded test results. Bug? Or am I missing some configuration?
    Thursday, May 06, 2010 3:45 PM

Answers

  • Hi Alex,

    I found out today that we do have a bug like this when you are running version 9.0 test lists or assemblies from a 2010 (or version 10.0) build machine. Is that your case. Are these tests that were created with a 2008 machine? If so, you need to update those tests to 2010. If you can't do that yet, because others are still using 2008, you will have to wait for a QFE or service pack.

    Thanks,

    Jason

    Tuesday, June 01, 2010 8:04 PM

All replies

  • Could you open the build details view and click on the Log View link and copy the MSTest output here? Publishing may have failed. I can't say why it would work for succeeded and fail for failed. They should all be published to the server. There should no longer be any TRX files uploaded to the drop location. Publishing now means that the results are stored in the Test Case Management database.

    Thanks, Jason

    Tuesday, May 11, 2010 2:07 PM
  • 18/19 test(s) Passed, 1 Failed
     
     Summary
     -------
     Test Run Failed.
     Passed 18
     Failed  1
     ----------
     Total  19
     Results file:   C:\Builds\18\...\TestResults\Legacy-Results-20100510-11_05_15_95.trx
     Run Configuration: My Test Run
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5,1337,5): warning : TF270015: 'MSTest.exe' returned an unexpected exit code. Expected '0'; actual '1'. [C:\Builds\18\...\BuildType\TFSBuild.proj]
     The previous error was converted to a warning because the task was called with ContinueOnError=true.
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018: The "TestToolsTask" task failed unexpectedly. [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018: Microsoft.TeamFoundation.Build.Workflow.Activities.UnexpectedExitCodeException: TF270015: 'MSTest.exe' returned an unexpected exit code. Expected '0'; actual '1'. [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at System.Activities.WorkflowApplication.Invoke(Activity activity, IDictionary`2 inputs, WorkflowInstanceExtensionManager extensions, TimeSpan timeout) [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at System.Activities.WorkflowInvoker.Invoke(Activity workflow, IDictionary`2 inputs, TimeSpan timeout, WorkflowInstanceExtensionManager extensions) [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at Microsoft.TeamFoundation.Build.Tasks.WorkflowTask.ExecuteInternal() [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at Microsoft.TeamFoundation.Build.Tasks.Task.Execute() [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\Builds\18\...\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1337,5): warning MSB4018:  at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult) [C:\Builds\18\...\BuildType\TFSBuild.proj]
     The previous error was converted to a warning because the task was called with ContinueOnError=true.
     Build continuing because "ContinueOnError" on the task "TestToolsTask" is set to "true".
     
    Tuesday, May 11, 2010 7:55 PM
  • It does look like MSTest had some kind of failure before tests were published. You may want to try logging into the build machine as the account that runs the build and then trying the full MSTest command line in a command shell. This may give you more information, although it is hard to say.

    The most common reason for a failure to publish test results is permissions to the TCM server.

    Sorry I couldn't help more, Jason

    Wednesday, May 12, 2010 4:06 PM
  • How could it be a permission issue if all succeeded test runs do get uploaded?
    Thursday, May 13, 2010 3:02 AM
  • Unfortunately, I can't tell from the log information what caused MSTest to fail. It doesn't look like it even got to the publishing step. I just wanted to provide some basic information on what might cause a publishing failure. Truthfully, I don't know what would have caused this error.

    Did you try running the MSTest command line on the build machine? Did it provide any additional information?

    Thanks,

    Jason

    Thursday, May 13, 2010 12:43 PM
  • Is it possible that the failing test is actually crashing the test agent process?
    Adam
    Thursday, May 13, 2010 4:05 PM
    Moderator
  • How can I tell?
    Thursday, May 13, 2010 4:49 PM
  • We similar test failures prior to upgrade and they were being uploaded just fine.
    Thursday, May 13, 2010 4:50 PM
  • I'm not sure, short of attaching a debugger like windbg to the QTAgent32.exe process.

    Have you tried running the MSTest command from a command line? What were the results?


    Adam
    Thursday, May 13, 2010 5:18 PM
    Moderator
  • You can easily reproduce it:

    1. Create a solution with two unit test: one that succeeds and another that fails.

    2. Create a build using upgrade tempalte.

    3. Run it.

    Thursday, May 13, 2010 5:55 PM
  • Adam,

    Did you try to reproduce it?

    Friday, May 28, 2010 11:12 PM
  • Hi Alex,

    Adam is not able to respond. Have you tried running the MSTest.exe command line as the build service account on the build machine? This test will determine if the problem is with mstest or something build related. We need to isolate the exact problem as much as possible.

    Thanks,

    Jason

    Tuesday, June 01, 2010 3:24 PM
  • I will try.

    One thing I do not understand is that I gave you steps to reproduce and you did not try it yourself.

    Tuesday, June 01, 2010 3:38 PM
  • Hi Alex,

    I found out today that we do have a bug like this when you are running version 9.0 test lists or assemblies from a 2010 (or version 10.0) build machine. Is that your case. Are these tests that were created with a 2008 machine? If so, you need to update those tests to 2010. If you can't do that yet, because others are still using 2008, you will have to wait for a QFE or service pack.

    Thanks,

    Jason

    Tuesday, June 01, 2010 8:04 PM
  • Yes, these are 9.0 unit tests.

    Is there an ETA for QFE/SP?

    Tuesday, June 01, 2010 8:56 PM
  • I am sorry to say, no. We have no eta for those.

    Jason

    Wednesday, June 02, 2010 12:13 PM
  • Do you have an ETA yet?
    Wednesday, October 27, 2010 10:14 AM
  • See the hot fix rollup here that includes a fix for this bug:

    http://support.microsoft.com/kb/983504/en-us

    Thanks,

    Jason

    Thursday, November 18, 2010 2:01 PM