none
How to run coded UI test VS2012 on tfs 2010 test agent

    Question

  • Hi,

    I have a VS2012 coded UI project, targeted framework .Net 4.0 and try to integrate this solution with our TFS and Test Manager. TFS server is 2010 version, I do have configured Test Controller and Test Agent 2010. TFS build was configured to build coded UI project and release was associated with a test suite.
    When test case runs from TestManager, it fails with error:

    System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.CodedUITestFramework, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    I tried to install VS2012 on agent machine, but it didn't help. Coded UI assemblies are not installed into GAC, they are stored in Visual Studio PrivateAssemblies and PublicAssemblies folders.

    Does anybody have this kind of configuration working Coded UI VS2012 with TFS 2010?

    Is there ability to configure TestAgent 2010 to use a new version of Coded UI to run tests?


    Best Regards, Sergey.



    Saturday, January 19, 2013 9:16 PM

Answers

  • I am not sure if this supported scenario.

    As you can use:

     

    VS 2010  

    http://msdn.microsoft.com/en-us/library/dd380742(v=vs.100).aspx

    and  VS 2012

    http://msdn.microsoft.com/en-us/library/dd380742(v=vs.110).aspx

     support different Configurations and Platforms for Coded UI Tests and Action Recordings

     

    Maybe you can try installing .NET Framework 4.5 on TFS 2010 and the Agent.

     

    Please see the following

    http://msdn.microsoft.com/en-us/library/hh506981.aspx

     

    If you create Coded UI Tests using the new UI Testing API in Visual Studio 2012, these projects cannot be opened in Visual Studio 2010.

     

    Based on the above I think it is safe to say this is not supported.

    Thank you

     

     

    Thursday, January 24, 2013 5:59 PM
    Owner

All replies

  • Hello Sergey,

    Thank you for your post.

    Do you want to run coded UI test using TFS build? Or using MTM?

    Hope you can provide us detailed  information so that we can further look at this issue.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, January 21, 2013 1:40 AM
    Moderator
  • Hi Amanda,

    Thanks for a reply, I want to use coded UI tests through MTM and have configured Test Controller with Test Agent 2010. Please let me know what kind of details could help to assist in this issue.


    Best Regards, Sergey.

    Monday, January 21, 2013 1:45 AM
  • Hello Sergey,

    We have a forum Testing with Visual Studio Test Manager (MTM) for this issue. In order to provide better support, I moved this thread.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, January 21, 2013 1:49 AM
    Moderator
  • Hi Sergey,

    Thank you for your post.

    Use MTM run automated test, you need set up test environment, test setting in MTM Lab center. Associate test case with test method in VS, associate test plan with correct team build, test setting.  Test controller specified during create test environment.

    1. About test case with test method in VS, you can refer to How to: Associate an Automated Test with a Test Case.

    2. MTM use test plan to run automated test. Test plan contains test case which associate with test method. Test setting and environment is create for test plan. Correct build is select for test plan. For detail information, you can refer to How to: Run Automated Tests from a Test Plan Using Microsoft Test Manager.

    3. You should set up lab to hold test environment and test settings. This test environment is need to run automated test in MTM. For detail information, you can refer to Using a Virtual Lab for Your Application Lifecycle.

    4. You need use TFS build to build the test application, then choose the correct build for test plan. For detail information, you can refer to Create a Basic Build Definition.

    Regards,


    Lily Wu
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Monday, January 21, 2013 5:53 AM
    Moderator
  • Hi Lily,

    Thank you for reply and clarifications. Probably my first post wasn't too clear. I do have successfully complete all steps you specified.

    1. There is association between test methods and test cases

    2. There is a test plan with associated build

    3. There is defined a test lab with physical environment

    4. There is TFS build to build test cases and it was selected for a test plan

    All these steps were complete. Problem is Test Agent 2010 fails to run Coded UI tests implemented with VS2012. Please, see my first post for an error message.



    Best Regards, Sergey.


    Monday, January 21, 2013 6:04 AM
  • Hi Sergey,

    Please check the test agent is configured run as Interactive process.

    Can you show us the test controller status?

    Regards,


    Lily Wu
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Tuesday, January 22, 2013 3:32 AM
    Moderator
  • Hi Lily,

    Thank you for a reply.

    Test Agent is configured as interactive process. I'm not sure where test controller status could be seen. Here is test agent status: 

    And here is test controller configuration:

    Let me know if you need some additional details.


    Best Regards, Sergey.

    Tuesday, January 22, 2013 2:23 PM
  • Hi Sergey,

    I am trying to involve someone to further look at this issue. There might be some time delay. Appreciate your patience.

    Thank you for your understanding and support.

    Regards,


    Lily Wu
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, January 23, 2013 8:05 AM
    Moderator
  • I am not sure if this supported scenario.

    As you can use:

     

    VS 2010  

    http://msdn.microsoft.com/en-us/library/dd380742(v=vs.100).aspx

    and  VS 2012

    http://msdn.microsoft.com/en-us/library/dd380742(v=vs.110).aspx

     support different Configurations and Platforms for Coded UI Tests and Action Recordings

     

    Maybe you can try installing .NET Framework 4.5 on TFS 2010 and the Agent.

     

    Please see the following

    http://msdn.microsoft.com/en-us/library/hh506981.aspx

     

    If you create Coded UI Tests using the new UI Testing API in Visual Studio 2012, these projects cannot be opened in Visual Studio 2010.

     

    Based on the above I think it is safe to say this is not supported.

    Thank you

     

     

    Thursday, January 24, 2013 5:59 PM
    Owner
  • Hi,

     

    What do you suggest to resolve this issue?

    I also have the same scenario, my coded UI Tests project/solution is created in VS2012, but we use TFS2010 to check in all the codes and connect MTM.

    Now, when I connect MTM2012 to TFS2010, I am not able to create a Lab Environment since it is not supported.

     

    I then tried below things-

    • Install and register Test Controller2010 to TFS team project collection and Test Agent 2010 to test controller.
    • Launch MTM2010 and created a Physical Lab Environment
    • Tried to run automated test (associated TestMethod with test case) using MTM2010 after selecting build
    • Test run starts but gives following error - Unable to load the test container '*test*.dll' or one of its dependencies. Error details: System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.TestTools.UITesting, Version=11.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.

    What should I do?

    Will I have to create my tests from scratch using VS2010, please do not suggest that :(... It would be a big headache...

    Tuesday, June 11, 2013 1:59 PM
  • Hi Prasant,

    In my case, tests solution was extended to be compatible with VS2010, it didn't take too much effort. To run tests from MTM, tests solution is compiled with VS2010 (CodedUI 10). Some new functions from Coded UI 11 were added into solution as extensions to provide code compatibility with old Coded UI framework.



    Best Regards, Sergey.

    Tuesday, June 11, 2013 2:59 PM
  • Hi Sergey,

    Tests solution was extended to be compatible with VS2010 - You mean to say you created your tests in Visual Studio 2010 again from scratch?

    After compiling the VS2010 code, are you successful in running automated tests from MTM?

    Depending on your input I will proceed with migrating my tests to VS2010. Please suggest what were the challenges you faced while doing so and if it was not much of a task.

    Also, it would be great if you can share the approach you followed to extend the compatibility to VS2010

    Thank you, Prasant


    Tuesday, June 11, 2013 4:04 PM
  • Hi Prasant,

    The solution wasn't recreated from scratch, we modified project target .Net framework to 4.0 version and made changes in .sln file (first two lines)

    Microsoft Visual Studio Solution File, Format Version 11.00

    # Visual Studio 2010


    It allows to open solution in VS2012 and VS2010

    The time consuming part was fixing compilation errors when solution was opened in VS2010, because some Coded UI functions are missing in VS2010. To make as much as possible changes in original tests, extension methods were added. For example:

    #if CODED_UI_COMPATIBILITY
        public static class CodedUICompatibility
        {
            public static bool TryFind(this UITestControl control)
            {
                try
                {
                    control.Find();
                    return true;
                }
                catch
                {
                    return false;
                }
            }
            ...
       }
    #endif

     TryFind function was introduced in new version of Coded UI.

    Modification in project files was made, to add CODED_UI_COMPATIBILITY conditional const when solution is opened and compiled in VS 2010.

    <Choose> <When Condition="'$(VisualStudioVersion)' == '10.0' And '$(IsCodedUITest)' == 'True'"> <PropertyGroup> <DefineConstants>CODED_UI_COMPATIBILITY</DefineConstants> </PropertyGroup>

    ...

    After these modifications we got solution that behaves as regular Coded UI VS2010 code and it was running though MTM.


    Best Regards, Sergey.

    Tuesday, June 11, 2013 4:56 PM
  • Hello Sergey,

    Thank you for your input.

    I successfully edited the .sln file and changed the target .NET framework to 4.0 and opened the coded ui test solution in VS2010. In order to avoid any conflicts I have also uninstalled VS2012 from my system. (Since, I had both 2010 and 2012 ultimate)

    I have edited code for dependencies/added references which are not supported in VS2010 and successfully compiled coded UI tests and got the tests in Test View window.

    But, when I try to run any test, my test fails with below error -

    Initialization method Product_Test_Automation.Scripts.SetupScripts.MyTestInitialize threw exception. Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: Microsoft.VisualStudio.TestTools.UITest.Extension.UITestException: Exception was thrown from the UITest module..

    The exact line of code which fails-

    public static void LaunchApplicationUnderTest()
            {
                
                //Kill all open instances of app
    
                CloseApplicationUnderTest();
                ApplicationUnderTest rv= new ApplicationUnderTest();
    
                //Launch App for Test
                try
                {
    
                    rv.Process.StartInfo.FileName = AppConfig.GetValue("ApplicationPath");
                    rv.Process.StartInfo.WindowStyle = ProcessWindowStyle.Maximized;
                    rv.Process.Start();
    
                    //Trace.WriteLine("Launched Application Under Test: " + AppConfig.GetValue("ApplicationProcessName"));
                    //WriteLogFile("Launched Application Under Test: " + AppConfig.GetValue("ApplicationProcessName"));
    
                }
                catch (Exception e)
                {
                    Trace.WriteLine(e.Message);
                    Trace.WriteLine(AppConfig.GetValue("ApplicationProcessName") + " may not be installed");
                    WinApplication.WriteLogFile(AppConfig.GetValue("ApplicationProcessName") + " may not be installed");
                    Assert.Fail(e.Message + ". " + rv.Process.StartInfo.FileName);
                    WriteLogFile(e.Message + ". " + rv.Process.StartInfo.FileName);
                }
    
            }

    While debug, my test fails at

    rv.Process.StartInfo.FileName = AppConfig.GetValue("ApplicationPath");

    and captures Exception in try block.

    Is there any such error you have come across?

    When I try to open the same solution in different computer with VS2010 (VS2012 was never installed in this system) it works successfully and I am able to run the tests and get the results.

    Wednesday, June 12, 2013 9:55 AM
  • Prasant,

    It seems like exception you have related to test deployment and irrelevant to VS2012 and TFS2010 compatibility.

    I would recommend check if all required files were deployed correctly by Test Agent. You can find your test assemblies copied into "C:\Users\<username>\AppData\Local\VSEQT\QTAgent\<id>\<machinename>\Deployment" (in case of Windows 7 agent). It could be situation, when some configuration xml or app.config files are missing and you need to mark dependencies for tests to get it deployed by Test Agent.



    Best Regards, Sergey.

    Wednesday, June 12, 2013 1:19 PM
  • Yes Sergey,

    It figured out to be an issue with VS2010 and I had to upgrade to SP1.

    My coded ui tests works fine now on VS2010 and thanks a lot for your help in migrating from VS2012 to VS2010.

    -Prasant

    Thursday, June 13, 2013 5:46 AM
  • We also have this problem. On TFS 2010 SP1 server we have installed Visual studio 2010 SP1 and Visual Studio 2012 SP4 RC. We created Coded UI test project locally in Visual Studio 2012 SP4 RC, and when we send this project to the TFS 2010 SP1 we got the error as in the first message. What can we do ?
    Monday, November 04, 2013 10:59 AM