Build test results in MTM

Question

I have automated test cases defined in MTM that are associated with unite testcases in VS. We are using TFS 2010 for build management. Is it possible to show results from test execution during build in MTM ? 

Thanks

Answer 1

Hello Nicpon,

We have a forum Testing with Visual Studio Test Manager(MTM) for this issue. In order to provide better support, I will move this thread.

Best regards,

---

Amanda Zhu [MSFT]
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 2

Hi Nicpon,

Thank you for your post.

What result you want to see, test result, or build result?

Build result is generated after build complete. You can view build result click the completed build in VS build explorer.

Test result is generated after auto test is complete in MTM, you can click the .trx file in test result attachments. the .trx file will open the auto test run result in VS.

Build and test are different situation. Run auto test via test plan, you first need a complete build, assign the build to test plan, so build is always complete before test. Test result and build result are separate.

If anything is unclear, please free feel to let me know.

Regards,

---

Lily Wu
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 3

I want to see test results from test cases ran during build process in MTM. I have test cases defined in MTM that are run during build process and those are the ones that I'd like to see results for in MTM after the build is done. Otherwise I have to manually correlate the test results from the build with test cases in MTM and this seems to be a waste of time.

Here is my scenario:

I have a automated test case AX that is associated with scripted test in VS. If I decide to run this test case during my build process why the results of test aren't updated in MTM ?

Thanks for moving the question to the right forum.

 

Answer 4

Hi nicpon,

Can you let us know how you run test during build process?

Regards,

---

Lily Wu
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 5

I have a TFS servers with local controller that does all the builds. 

The test cases are run through adding test assemblies in basic category process tab in build definition. 

I queue new builds and they run no problem and I can see the results in VS too. But, since those test cases are associated with test in MTM, what do I need to do see results there ?

Answer 6

Hi nicpon,

Ok, seems you run test within team build, not run from MTM. Now, you want the test result run in build could update to test case in MTM, right?

Please try to run the test in VS, run tcm command to publish test run results from a VS test run file(.trx) to TFS, and associate test result with test case in MTM. For more information, you can refer to http://msdn.microsoft.com/en-us/library/ff942469(v=vs.100).aspx. If this cannot work, please free feel to let us know.

Regards,

---

Lily Wu
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 7

That is a great pointer and it looks like it will do what I need as far as results go.

Whats the best way to integrate tcm command into to the build process, so it is executed after the test run automatically?

Answer 8

So I added the following command 

C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC>tcm run /publish /results
file:"C:\Users\Admin\Documents\Visual Studio 2010\Projects\WebApplication2\Test
Results\Admin_Admin-PC 2012-12-03 14_39_00.trx" /suiteid:8 /configid:2 /result
owner:Admin /collection:"http://Admin-pc:8080/tfs/DefaultCollection" /teamproj
ect:"WebApp2"

and i get this response:

Starting Publish.
Tmi test run 4a983173-a73d-4256-a906-1581530926d3 is already published to this T
CM Server. A tmi test run cannot be published twice.

If the results are published then why can't I see them in MTM ? or this is just saying that they are publish to tfs ? But then still I can't see them in MTM.

Thanks

Answer 9

so the results are automatically published during build to TFS, but there must be not enough parameters because when i publish them manually using tcm they appear in MTM, but via automated build they don't. Any ideas on where to change that behavior ?

Thanks

 

Answer 10

Hi nicpo,

Seems publish test result to TFS is hard coded in team build. You can refer to http://social.msdn.microsoft.com/Forums/eu/tfsbuild/thread/ad991766-149a-4aba-b6d9-3205f7a655ca

May be you can run test locally, then run tcm command to publish test result with test case in MTM. Or you can run test in MTM directly.

Regards,

---

Lily Wu
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.

Answer 11

I don't have that option. I check in build definition and project properties as well and it isn't there. Am I looking in the right place ?

Answer 12

Hi

As I understand it is not possible to disable publishing the test results to TFS when executed as part of build. Also since published results are not associated with any test plan, they can not be viewed with MTM.
As someone already pointed, you could use tcm.exe to publish the trx to TFS server. If the test run associated with trx is already published this would fail. One possible workaround is to delete the already published test run and than republish it using tcm.exe.
To delete the published test run  http://blogs.msdn.com/b/dhruvk/archive/2012/10/24/deleting-test-run-or-test-results-published-from-gated-build-or-mstest-exe.aspx

Thanks

Answer 13

So wouldn't be easier to associate results with a given test plan rather than deleting results and then publishing them again with tcm ?

Can that be done ?

Answer 14

Hi,

I am not sure whether you know or not, but using TCM.exe you can execute the tests locally also similar to mstest.

http://blogs.msdn.com/b/visualstudioalm/archive/2012/12/09/how-to-execute-tests-locally-using-tcm-exe-command-line.aspx

The only difference is that this will run the MTM based unit tests and will update the test plan data in MTM as well.

So you can customize your build definition and use TCM.exe to runs tests instead of the default mstest activity. Please try this out and let us know in case you need any help in this.

Regards

Aseem Bansal

Answer 15

Hi nicpo,

Since we haven't heard from you for a long time, I assume the issue is resolved, I mark useful reply as answer. If you have any concerns, please feel free to reopen it or submit a new question. Thanks for your understanding.

Regards,

---

Lily Wu
MSDN Community Support | Feedback to us
Develop and promote your apps in Windows Store
Please remember to mark the replies as answers if they help and unmark them if they provide no help.