none
Running Visual Studio 2015 Coded UI Tests from command line against a Production Environment

    Question

  • Are there any licensing requirements explicitly restricting the use of Coded UI Tests against a Production Environment (eg. website)?

    The Visual Studio 2015 Licensing Whitepaper from November 2016 states: 

    The licensed user can install and use the software on any number of devices... However, the software is otherwise not licensed for use in production environments... A production environment is defined as an environment that is accessed by end users of an application (such as an Internet Web site)

    Obviously developing and debugging against Production are in violation of these terms, but what about running Coded UI Tests against a Production site? Example: VSTest.Console.exe used to run a UI Test which opens a production URL in IE and fills out a form on the website 

    Note that Load Tests appear to be unrestricted:

    monthly subscribers can use the software to execute load tests with any number of virtual users, including load tests that run in a production environment






    • Edited by Jesse FIS Friday, April 21, 2017 10:23 PM
    Friday, April 21, 2017 10:05 PM

All replies

  • Hi Jesse FIS

    Welcome to MSDN forums.

    >>Are there any licensing requirements explicitly restricting the use of Coded UI Tests against a Production Environment (eg. website)?

    Normally, we execute Coded UI Test on deployed application, for which we do not need access to environment servers. We need access to the deployed application. We open the instance and perform actions/verifications on it.

    So my first suggestion is, you could do a deployment that target to your client machine, which like you do the deployment to the production environment. Then you can install the needed software on your client machine if necessary.

    And according your description, you want to run code UI test through the command line, am I right?

    If yes, you have access to the deployed application, then you could use MSTest.exe or VSTest.console.exe to run the test project in your Developer Command Prompt, please refer to below steps:

    1. Open Developer Command Prompt, usual location in C:\ProgramData\Microsoft\Windows\Start Menu\Programs\Visual Studio 2015\Visual Studio Tools
    2. To identify your solution folder, first identify the Visual Studio Projects folder. To do this, choose Options on the Tools menu in Visual Studio, and then choose Projects and Solutions. Under Visual Studio projects location, you see a path such as the following:

    <drive letter>:\Documents and Settings\<user name>\My Documents\Visual Studio\Projects

    1. Run the MSTest.exe or VSTest.console.exe.

    For example:

    Use MSTest.exe:              mstest /testcontainer:UnitTestProject.dll                                                                             

    Use VSTest.console.exe:              Vstest.console UnitTestProject.dll

    Please refer to below links for more information about how to use MSTest.exe or VSTest.console.exe:

    https://msdn.microsoft.com/en-us/library/ms182489(v=vs.120).aspx

    https://msdn.microsoft.com/en-us/library/jj155800(v=vs.120).aspx

    Typically, we avoid executing automated tests on Production environment.

    Hope it helps you!

    Best regards,

    Fletch


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    Monday, April 24, 2017 5:44 AM
  • Thanks for your response Fletch.

    "you could do a deployment that target to your client machine"

    We already have Coded UI Tests running in an environment setup to this effect. With our application deployed on a test server. My question is with regards to running tests specifically against a production deployment.


    "Typically, we avoid executing automated tests on Production environment."

    So is running against production just discouraged, or is it a direct violation of license terms?  I can't seem to find a definitive answer.

    Thanks.

    Monday, April 24, 2017 5:00 PM
  • Hi friend,

    It's just not recommended, and it's not a violation of license term, no worry.

    Best regards,

    Fletch


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    Wednesday, April 26, 2017 5:23 AM