Running automated tests on Visual Studio Team Services / Visual Studio Online using Test Manager RRS feed

  • Question

  • We have an instance of VSTS/TFS/VSO (or whatever you guys like to change it's name to next! :) ) and we have some coded-ui tests we'd like to trigger via Test Manager.

    We've got a vNext build which can successfully run those same test as part of a CI build but our QA guys would like to trigger them "manually" via Test Manager.

    So we have our VSTS instance... (ourcompany.visualstudio.com) and a couple of AWS EC2 boxes, one as the test client/agent and the other as the test controller. Although neither are domain connected, we can configure things so that the controller can "see" both VSTS (i.e. the test suites etc.) and the test client/agent machine.

    The problem we have is that VSTS creates a zip file of "the code" but the controller can't access it using the local Windows user everything runs as. However, we can't seem to give it the credentials of a valid VSTS user such that it can get at the code file.

    Is this supported? This page (https://msdn.microsoft.com/en-gb/library/hh546460.aspx) seems to show such things but I suspect that the part that isn't supported is the getting of the code drop from VSTS. However, we can't find how to tell the test controller to get the code from a file share either.

    Can anyone assist?

    Wednesday, January 4, 2017 3:48 PM

All replies

  • Hi

    Running automated test from MTM works only with XAML builds. It does not support vNext builds. You have to use the Build or Release workflow to run automated tests form vNext Builds. See this blog for more information: https://blogs.msdn.microsoft.com/visualstudioalm/2016/03/31/executing-automated-tests-in-build-vnext-using-test-plan-test-suites/

    We are working on enabling the capability to trigger on-demand test runs from Test hub in Team Web Access. If you are interested in trying out a preview version of the feature, please drop me a mail at mabables at microsoft dot com.

    Visual Studio Testing Tools

    Thursday, January 5, 2017 5:02 AM