locked
ALL Test Passed But MSTest.exe fails and cause build to be PartiallySucceeded?

    Pregunta

  • I want to move from scenario 1 to scenario 2 because we always want to run all the testcases and not have the issue of running only a specific Test List. But Scenario 2 is causing this issue!. any thoughts?

    My test cases passes and build partially succeeds in Scenaio 2

     

    Scenario 1 :

    Run Specific TestList containing 350+ test cases enabled in this test list

    there is 500+ test cases that are disabled and not part of the test list and are not executed as part of the build

    Result : All my Test cases passes and build succeeds

    Scenario 2 :

    No Test list provided to the build, (the intention is run all the enabled test cases irrespective of a test list) 

    there is 500+ test cases are disabled and 350+ test cases enabled

    Result : All my Test cases passes and build Partiallysucceeds with the below error

    Error from log:

      .......................................................Publish completed successfully. (TaskId:3181)
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5,1326,5): warning : MSTest.exe returned an exit code of 1 indicating that not all tests passed. [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
      The previous error was converted to a warning because the task was called with ContinueOnError=true. (TaskId:3181)
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018: The "TestToolsTask" task failed unexpectedly. [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018: Microsoft.TeamFoundation.Build.Workflow.Activities.TestFailureException: MSTest.exe returned an exit code of 1 indicating that not all tests passed. [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at System.Activities.WorkflowApplication.Invoke(Activity activity, IDictionary`2 inputs, WorkflowInstanceExtensionManager extensions, TimeSpan timeout) [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at System.Activities.WorkflowInvoker.Invoke(Activity workflow, IDictionary`2 inputs, TimeSpan timeout, WorkflowInstanceExtensionManager extensions) [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at Microsoft.TeamFoundation.Build.Tasks.WorkflowTask.ExecuteInternal() [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at Microsoft.TeamFoundation.Build.Tasks.Task.Execute() [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
    C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1326,5): warning MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult) [d:\Builds\226\RXD_PVP_Platform\Build 100\BuildType\TFSBuild.proj]
      The previous error was converted to a warning because the task was called with ContinueOnError=true. (TaskId:3181)
      Build continuing because "ContinueOnError" on the task "TestToolsTask" is set to "true". (TaskId:3181)
    Done executing task "TestToolsTask" -- FAILED. (TaskId:3181)


    shiv
    lunes, 14 de junio de 2010 20:40

Todas las respuestas

  • Hello,

    You may enable logging to see if you can find anything helpful about this issue. You can refer to the following article about how to enable logging:

    http://blogs.msdn.com/b/yutong/archive/2006/10/03/how-to-enable-vsts-logging_3f00_.aspx


    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Send us any feedback you have about the help from MSFT at fbmsdn@microsoft.com.
    jueves, 17 de junio de 2010 9:28
    Moderador
  • Are you sure that all tests have passed. Do you see N/N tests passed in the log or did you verify in the trx that all tests have passed.

    Thanks,

    Anuj

    martes, 17 de agosto de 2010 5:41
  • Hi

    We are experiencing the same issue. We have more in-depth logging level so here it goes.

    The overall result is that the drop location is empty apart from the build log itself.

    Our scenario is build existing VS2008/VS2005 solutions, and therefore running the build with /ToolsVersion:3.5. The curios bit is that while the tests themselves are run with:

    c:\Program Files (x86)\Microsoft Visual Studio 9.0\Common7\IDE\MSTest.exe

    the test results seems to be published using:

    c:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe

    Now this has to be happening AFICT from within the same invocation of TestToolsTask. Seems the new TestToolsTask only halfway supports the /ToolsVersion:3.5 switch and somehow the ToolPath property is changed in mid-flight.

    This is just a peculiarity though because the actual publish seems to complete ok apart from warnings like this after each publish:

    The file 'C:\dev\TR\Legacy-Results-20101006-13_50_59_12.trx' was created in an earlier version of Visual Studio. This file will be converted, in memory only, to the current format

    and sometimes this:

    The file 'C:\dev\TR\Legacy-Results-20101006-13_53_46_16.trx' was created in an earlier version of Visual Studio. This file will be converted, in memory only, to the current format.
      C:\dev\TR\Legacy-Results-20101006-13_53_46_16.trx: (Line: 70, Position: 556). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 114, Position: 555). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 268, Position: 550). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 356, Position: 555). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 455, Position: 555). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 521, Position: 555). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 587, Position: 556). XSD violation: The 'expectedException' attribute is not declared.

    These are all warnings so they are probably not what causes the build to fail.

    We know that not all of our tests are passing. Some are indeed failing. But that should not lead to the whole build failing like this.

    Regards

    Martin

    viernes, 08 de octubre de 2010 13:17
  • Martin,

    The differing mstest.exe versions you are seeing is correct behavior.  MSTest 9.0 can't talk to a 10.0 TFS server.  That's the whole reason that the publish is done in a separate invocation of mstest in the case of 9.0 tests.  If you are running 10.0 tests, then mstest.exe is only called once and it performs both operations in the same invocation.

    I am having trouble understanding what is the incorrect behavior you are experiencing.  You have said that some of your tests are failing.  If that's the case, then don't you expect your build to be marked partially suceeded?  Can you please clarify what is happening and what is your expectation?

     

    Thanks,

    -JP

    Development lead - TFS Build

    lunes, 18 de octubre de 2010 13:58
    Moderador
  • Hi,

    We are also experiencing the same issue. We just upgrade our TFS from 2008 to 2010. The build in question is using the UpgradeTemplate.yaml while our own scripts are doing the actual build.

    All compilation is successful and the drop location is populated. All 2184 test run and pass successfully, however the build FAILS. The error that I notice is the following from the log: 

    The file 'D:\Tfs2008Builds\1\I009\Release-3.8.0\TestResults\Legacy-Results-20101110-04_13_32_79.trx' was created in an earlier version of Visual Studio. This file will be converted, in memory only, to the current format.
      D:\Tfs2008Builds\1\I009\Release-3.8.0\TestResults\Legacy-Results-20101110-04_13_32_79.trx: (Line: 940, Position: 514). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 960, Position: 513). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 998, Position: 471). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 1036, Position: 524). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 1405, Position: 447). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 1786, Position: 432). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 1896, Position: 461). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 1946, Position: 459). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2123, Position: 466). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2137, Position: 570). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2301, Position: 461). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2379, Position: 470). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2417, Position: 458). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2678, Position: 489). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2702, Position: 482). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2775, Position: 512). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 2946, Position: 400). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 3037, Position: 458). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 3695, Position: 497). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 4379, Position: 497). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 4784, Position: 499). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 4810, Position: 520). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 5119, Position: 445). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 5594, Position: 428). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6379, Position: 430). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6387, Position: 442). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6443, Position: 587). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6556, Position: 438). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6601, Position: 408). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6685, Position: 460). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6737, Position: 446). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 6890, Position: 524). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7087, Position: 522). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7123, Position: 527). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7131, Position: 442). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7167, Position: 504). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7321, Position: 447). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7415, Position: 504). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 7717, Position: 401). XSD violation: The 'expectedException' attribute is not declared.
      (Line: 20181, Position: 103). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0112761' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0112761' is not a valid Int32 value.
      (Line: 20228, Position: 112). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0226853' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0226853' is not a valid Int32 value.
      (Line: 20239, Position: 106). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0160397' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0160397' is not a valid Int32 value.
      (Line: 20278, Position: 105). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0018182' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0018182' is not a valid Int32 value.
      (Line: 20297, Position: 103). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0087361' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0087361' is not a valid Int32 value.
      (Line: 20312, Position: 103). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0216731' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0216731' is not a valid Int32 value.
      (Line: 20375, Position: 106). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0009463' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0009463' is not a valid Int32 value.
      (Line: 20410, Position: 117). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0041633' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0041633' is not a valid Int32 value.

    Please advise what steps I can take to address this issue? Or should I just disable the tests as there is no immediate workaround?

    Thanks

    Chris Mouton

    miércoles, 10 de noviembre de 2010 11:01
  • Chris,

    This is a behavior I have not seen before.  We have numerous customers successfully producing 9.0 TRXes and publishing them to 10.0 servers during a build.  SO, I'm, trying to nail down what is different about your situation.  Can you post the portion of your log where 'D:\Tfs2008Builds\1\I009\Release-3.8.0\TestResults\Legacy-Results-20101110-04_13_32_79.trx' is created? 

    Thanks,

    -JP

    viernes, 12 de noviembre de 2010 15:09
    Moderador
  • Hi,

    We too have the same problem the legacy trx file is created but the test results are not published to the drop folder.

    We get:

     Warning: Test Run deployment issue: Failed to get the file for deployment item 'Assay1_0_0.xml' specified by the test 'Renishaw.SPD.ConfigurationSettingsUnitTests.ConfigFileInstallerTest.IdentifyInstallAssayConfigFilesTest': System.IO.FileNotFoundException: Could not find file 'C:\Builds\23\IST 9861 Assay Reader\Daily build\Binaries\x86\Release\Assay1_0_0.xml'.

      c:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\MSTest.exe /nologo /usestderr /maxpriority:"0" /minpriority:"0" /publish:"http://gbvuap08:8080/tfs/defaultcollection" /publishbuild:"vstfs:///Build/Build/12418" /teamproject:"IST 9861 Assay Reader" /publishresultsfile:"C:\Builds\23\IST 9861 Assay Reader\Daily build\TestResults\Legacy-Results-20111104-03_17_19_17.trx" /platform:"x86" /flavor:"Release" 

      Waiting to publish...

      The file 'C:\Builds\23\IST 9861 Assay Reader\Daily build\TestResults\Legacy-Results-20111104-03_17_19_17.trx' was created in an earlier version of Visual Studio. This file will be converted, in memory only, to the current format.

      C:\Builds\23\IST 9861 Assay Reader\Daily build\TestResults\Legacy-Results-20111104-03_17_19_17.trx: (Line: 305, Position: 426). XSD violation: The 'expectedException' attribute is not declared.

    Followed by a number of: 
     (Line: 345, Position: 428). XSD violation: The 'expectedException' attribute is not declared.
    And then:
    Publishing results of test run TFSService@UKNML1868 2011-11-04 03:17:38 to http://gbvuap08:8080/tfs/defaultcollection...
      .................................................................................Publish completed successfully.
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5,1342,5): warning : MSTest.exe returned an exit code of 0 indicating that not all tests passed. [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
      The previous error was converted to a warning because the task was called with ContinueOnError=true.
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018: The "TestToolsTask" task failed unexpectedly. [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018: Microsoft.TeamFoundation.Build.Workflow.Activities.TestFailureException: MSTest.exe returned an exit code of 0 indicating that not all tests passed. [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at System.Activities.WorkflowApplication.Invoke(Activity activity, IDictionary`2 inputs, WorkflowInstanceExtensionManager extensions, TimeSpan timeout) [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at System.Activities.WorkflowInvoker.Invoke(Activity workflow, IDictionary`2 inputs, TimeSpan timeout, WorkflowInstanceExtensionManager extensions) [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at Microsoft.TeamFoundation.Build.Tasks.WorkflowTask.ExecuteInternal() [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at Microsoft.TeamFoundation.Build.Tasks.Task.Execute() [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute() [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
    C:\Program Files\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets(1342,5): warning MSB4018:    at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask(ITaskExecutionHost taskExecutionHost, TaskLoggingContext taskLoggingContext, TaskHost taskHost, ItemBucket bucket, TaskExecutionMode howToExecuteTask, Boolean& taskResult) [C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj]
      The previous error was converted to a warning because the task was called with ContinueOnError=true.
      Build continuing because "ContinueOnError" on the task "TestToolsTask" is set to "true".
    Done Building Project "C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj" (TestConfiguration target(s)).
    Done Building Project "C:\Builds\23\IST 9861 Assay Reader\Daily build\BuildType\TFSBuild.proj" (RunTest target(s)).


    Tom Mason Senior Software Engineer Renishaw plc
    viernes, 04 de noviembre de 2011 8:56
  • Hi,

    I am also facing same issue in vs 2012

    Waiting to publish...

    D:\Builds\249\1397\TestResults\306700d0-e20a-48f4-a981-635ecce9ac98_UnitTestResults.trx: (Line: 4523, Position: 126). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.3199297' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.3199297' is not a valid Int32 value.

    (Line: 4523, Position: 154). XSD violation: The 'timeZone' attribute is not declared.

    (Line: 4545, Position: 99). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0201109' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0201109' is not a valid Int32 value.

    (Line: 4545, Position: 127). XSD violation: The 'timeZone' attribute is not declared.

    (Line: 4555, Position: 101). XSD violation: The 'duration' attribute is invalid - The value '00:00:00.0015124' is invalid according to its datatype 'http://www.w3.org/2001/XMLSchema:int' - The string '00:00:00.0015124' is not a valid Int32 value.

    (Line: 4555, Position: 129). XSD violation: The 'timeZone' attribute is not declared.

    Invalid cast from 'System.String' to 'System.TimeZoneInfo'.

    Can any provide me resolution other than version changing of MSTest.exe

    martes, 27 de noviembre de 2012 9:31
  • Hi,

    The .NET Framework does not support all versions of every platform. For a list of the supported versions, see .NET Framework System Requirements.


    Himanshu Mehta

    miércoles, 05 de diciembre de 2012 12:30