none
Triggering test suites from web interface for multiples test projects

    Question

  • My problem is, I have multiple test projects/suites which are build to test different applications. I need to trigger the tests from a web based application by choosing which suite to run - like a one click triggering. I need to know if I can use a test controller or any other method to trigger the test suites/projects from a web application to execute the tests in a remote machine and give me the results.

    I also heard that test agents and controllers can only work for one project collection and it cant share same physical space for another test project collection. Is there anyway to configure them for my problem statement.

    OR

    I have even explored the MSTest.exe and VSTest.Console.exe methods of triggering the test cases from the web page, it works fine in the local machine, but when I publish and host the website on IIS, it say "To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop""

    I am stuck here and need some pointers on how to go about this.Any sort of help be really helpful and appreciable.

    Wednesday, September 11, 2013 8:17 AM

Answers

  • Hello,

    Thank you for your post.

    You can use test controller and test agent to execution tests in remote machines.

    I assume that you create test projects in Visual Studio rather than creating test cases in MTM. After you run test remotely, you will get test results on local machine.

    As far as I know, test controller and test agents can work for one solution rather than one test project in one running. You can select ‘Run All’ in Test Explorer if you use VS2012 to run all coded UI test/unit test in the solution. Also you can add those test methods such as unit test, coded UI test and web test in the solution in one load test and then run test remotely. About load test, please see: Load Test Walkthroughs

    If you want to execute tests remotely, you can follow the steps below:

    1. Install and configure correct version of Test Controller and Test Agent on machines. Test Controller can be installed on local machine having Visual Studio and Test Agents can be installed on remote machines. If you use the VS2012, you have to use the Test Controller 2012 and Test Agent 2012 when you run the coded UI test remotely. So if you just have the Test Controller 2010 and Test Agent 2010, you would use the VS2010 version.

    About installing and configuring test controller and test agent, please do it based on the steps described in this article: Setting Up Test Controllers and Test Agents to Manage Tests with Visual Studio If you want to run coded UI tests, you need to set Test Agent to run as a process. And the computer that is running an agent on which you want to run coded UI tests cannot be locked or have an active screen saver. For more information, please see: How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop

    2. Creating a test setting for specifying test execution method (remote or local), assigning roles to a test controller and test agents and assigning a diagnostic and data adapter. For more information, please see:

    Assigning Roles to a Test Controller and Test Agent for Automated Testing in Visual Studio

    Best regards,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.



    Thursday, September 12, 2013 6:50 AM
  • Hello,

    As for how to triggering the Test Controller from a Web interface like a click of button on the web page should be able to trigger the controller, I think that you can write a batch/script file using MSTest command line options to run tests remotely with test controller and test agent and then write your own function for the button control on a web interface to call the batch/script file.

    Using MSTest command line, you can select the expected test methods in a test project rather than all test methods of a test project to run with /test option.

    Reference: MSTest.exe Command-Line Options

    How to: Run Automated Tests from the Command Line Using MSTest

    How to: Schedule Test Runs

    How to create batch file to Run VSTS test commands?

    As far as I know, there is no way to pass some parameters which stands for running specific tests to Test Controller.

    Best regards,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.

    Friday, September 13, 2013 3:01 AM

All replies

  • My problem is, I have multiple test projects/suites which are build to test different applications. I need to trigger the tests from a web based application by choosing which suite to run - like a one click triggering. I need to know if I can use a test controller or any other method to trigger the test suites/projects from a web application to execute the tests in a remote machine and give me the results.

    I also heard that test agents and controllers can only work for one project collection and it cant share same physical space for another test project collection. Is there anyway to configure them for my problem statement.

    OR

    I have even explored the MSTest.exe and VSTest.Console.exe methods of triggering the test cases from the web page, it works fine in the local machine, but when I publish and host the website on IIS, it say "To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop""

    I am stuck here and need some pointers on how to go about this.Any sort of help be really helpful and appreciable.

    Wednesday, September 11, 2013 8:20 AM
  • Hello,

    Thank you for your post.

    You can use test controller and test agent to execution tests in remote machines.

    I assume that you create test projects in Visual Studio rather than creating test cases in MTM. After you run test remotely, you will get test results on local machine.

    As far as I know, test controller and test agents can work for one solution rather than one test project in one running. You can select ‘Run All’ in Test Explorer if you use VS2012 to run all coded UI test/unit test in the solution. Also you can add those test methods such as unit test, coded UI test and web test in the solution in one load test and then run test remotely. About load test, please see: Load Test Walkthroughs

    If you want to execute tests remotely, you can follow the steps below:

    1. Install and configure correct version of Test Controller and Test Agent on machines. Test Controller can be installed on local machine having Visual Studio and Test Agents can be installed on remote machines. If you use the VS2012, you have to use the Test Controller 2012 and Test Agent 2012 when you run the coded UI test remotely. So if you just have the Test Controller 2010 and Test Agent 2010, you would use the VS2010 version.

    About installing and configuring test controller and test agent, please do it based on the steps described in this article: Setting Up Test Controllers and Test Agents to Manage Tests with Visual Studio If you want to run coded UI tests, you need to set Test Agent to run as a process. And the computer that is running an agent on which you want to run coded UI tests cannot be locked or have an active screen saver. For more information, please see: How to: Set Up Your Test Agent to Run Tests that Interact with the Desktop

    2. Creating a test setting for specifying test execution method (remote or local), assigning roles to a test controller and test agents and assigning a diagnostic and data adapter. For more information, please see:

    Assigning Roles to a Test Controller and Test Agent for Automated Testing in Visual Studio

    Best regards,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.



    Thursday, September 12, 2013 6:50 AM
  • Thanks Amanda for the reply. I have an over head of calling or triggering the Test Controller from a Web interface like a click of button on the web page should be able to trigger the controller. Can you please give me some suggestion on that too. I have another problem, I have multiple solutions(Coded UI projects) say 5 solutions to be run remotely, as per the pointers you mentioned - you want me to put these projects in the same solution and try to run the tests. I can do that but I don't want to run all the tests all the time. Can I pass some parameters or something to the Test Controllers to say that I just want only specific tests to run.

    Thanks in advance.

    Thursday, September 12, 2013 1:25 PM
  • Hello,

    As for how to triggering the Test Controller from a Web interface like a click of button on the web page should be able to trigger the controller, I think that you can write a batch/script file using MSTest command line options to run tests remotely with test controller and test agent and then write your own function for the button control on a web interface to call the batch/script file.

    Using MSTest command line, you can select the expected test methods in a test project rather than all test methods of a test project to run with /test option.

    Reference: MSTest.exe Command-Line Options

    How to: Run Automated Tests from the Command Line Using MSTest

    How to: Schedule Test Runs

    How to create batch file to Run VSTS test commands?

    As far as I know, there is no way to pass some parameters which stands for running specific tests to Test Controller.

    Best regards,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.

    Friday, September 13, 2013 3:01 AM
  • Hi,

    I have already explored the MSTest.exe and VSTest.Console.exe methods of triggering the test cases from the web page, it works fine in the local machine, but when I publish and host the website on IIS, it say "To run tests that interact with the desktop, you must set up the test agent to run as an interactive process. For more information, see "How to: Set Up Your Test Agent to Run Tests That Interact with the Desktop""

    As far as i know, these commands just take the "testproject.dll" and just execute and as you said we can give test options, but can "MSTest command line options run tests remotely with test controller and test agent"? it can only take the dll and just run it. But the MSTest and VSTest.Console  don't have anything to trigger the test controller, does it ?

    Thanks for the reply


    Friday, September 13, 2013 5:51 AM
  • Hi,

    You can run test remotely using test controller and test agent from MSTest command line. You just need to set test to run remotely in test settings file and then run test from command line.

    If you are working with VS2012, you need to add the test setting file for coded UI test project/unit test project and then apply it. You can see ‘How to use a test settings file’ in this article: Specifying Test Settings for Visual Studio Tests

    Also you can use the expected test settings file with /testsettings option in command line.

    As for how to make test to run remotely, you can refer to my first reply.

    Best regards,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.

    Friday, September 13, 2013 6:49 AM
  • Hi,

    So as per my understanding, Test Controller can be triggered by MSTest.exe right ?

    If that is so, i might need to set up a test agent and a test controller in two different machines, write a batch file to invoke test controller which will invoke the test agent to execute the tests and then collect the results. I might need to host my web page in the same system as my test controller to invoke it with the batch file. Please correct me if i am wrong.

    So in this case, every coded UI project should have a testsettings file which will have the controller name and other details, so while executing with the MSTest.exe, i need to give the testsettings as a parameter also. am i right ?

    Saturday, September 14, 2013 5:30 AM
  • Hello.

    Yes, Test controller can be triggered by MSTest.exe.

    If you run coded UI test remotely using test controller and test agent, you need to put the application under test(your web page) on the test agent machines.

    Multiple coded UI test projects can be in the same solution. In this case, you just need one test setting file. If you apply the test setting file to the solution based on this article: Specifying Test Settings for Visual Studio Tests, in my opinion, you don’t need to use /testsettings option in mstest command line.

    Thanks,


    Amanda Zhu <THE CONTENT IS PROVIDED "AS IS" WITHOUT WARRANTY OF ANY KIND, WHETHER EXPRESS OR IMPLIED>
    Thanks
    MSDN Community Support

    Please remember to "Mark as Answer" the responses that resolved your issue. It is a common way to recognize those who have helped you, and makes it easier for other visitors to find the resolution later.

    Monday, September 16, 2013 1:50 AM