locked
Coded UI test: Browser closes after each test RRS feed

  • Question

  • Hi

    Since upgrading to VS2012 I've noticed that when I run coded UI tests, the browser shuts down after every [TestMethod] making it impossible to run tests in sequence using the same browser meaning that the same steps will have to be repeated for every test. We work a lot with workflows meaning that I used to have multiple scripts that could be run either independently or, by switching a flag, in sequence, one test continuing from where the previous left off.

    This is no longer possible since Microsoft decided to switch the BrowserWindow.CloseOnPlaybackCleanup flag to false by default and there does not seem to be any way of switching it back in a way that (a) makes the framework recognize it and (b) let a following script continue using the browser. 

    The only way I was able to stop the browser from being closed was like this:

    [TestInitialize()]
    public void MyTestInitialize()
    {
          BrowserWindow b = BrowserWindow.Launch(new System.Uri("www.google.com"));
          b.CloseOnPlaybackCleanup = false;
    }

    Unfortunately, the next script will fail to find the HTML document contained in the browser and fail.

    I tried flipping this switch in a number of ways on the browser instance created on the UI map but it seems that any browser launched by the following:

    public void LaunchUrl(System.Uri url)
    {
           this.CopyFrom(BrowserWindow.Launch(url));
    }

    Cannot have the CloseOnPlaybackCleanup set to false and actually used by the framework. No matter where i set it (TestInitialize, ClassInitialize, UIMap.cs, UIMap.Designer.cs, ctor, you name it) the browser always closes when I reach [TestCleanup()]

    Is there any way of getting around this? Is there any way of setting CloseOnPlaybackCleanup = false on a browser created by the CopyFrom that actually prevents it from being closed at clean-up?

    It seems that the CopyFrom takes a UITestControl object, not a ApplicationUnderTest object (which inherits from UITestControl) and the UITestControl object does not have this attribute. Is this causing the problem?

    Tuesday, October 9, 2012 7:00 PM

Answers

  • Also, I checked the flag "Keep execution engine running between test runs" in Tools => Options => Web Performance Test Tools => Test Execution and it is checked so the session should be kept alive, yes? Or is this the bug you mentioned?

    Lastly, why does the QTAgent32.exe shut down between [TestMethod] scripts even when I run multiple tests in sequence? Let me explain:

    I have three [TestMethod]'s in one coded UI test class and each method shows up as a test script in the Test Explorer window in VS2012. When I mark all three and run them I'd imagine all three was run within one test run and that the QTAgent32.exe does not shut down between each test case but this seems to be the case.

    I will follow up UnitTest infrastructure team more on "Keep execution engine running between test runs" and will update this thread.

    As you said if you set CloseOnPlaybackCleanUp to False,  BrowserWindow doesn't get closed after the test case is finished, however the BrowserWindow object you created in previous TestCase is not valid anymore, you have to relocate the BrowserWindow in new test method.

    Please refer to the example below.

    BrowserWindow window;
    
            [TestMethod]
            public void Method1()
            {
                window = BrowserWindow.Launch(new Uri("http://www.bing.com"));
                window.CloseOnPlaybackCleanup = false;
            }
    
            [TestMethod]
            public void Method2()
            {
                window = BrowserWindow.Locate("Bing");
                window.CloseOnPlaybackCleanup = false;
            }
    
            [TestMethod]
            public void Method3()
            {
                window = BrowserWindow.Locate("Bing");
            }
    

    Please let me know if it solves your issue.

    Thanks


    Aditya Kumar Agrawal (MSFT)

    • Marked as answer by Peter Harlid Tuesday, October 16, 2012 1:55 PM
    Tuesday, October 16, 2012 4:29 AM

All replies

  • Hello Peter,

    Thank you for your post.

    I have try to record a coded UI test and have two recorded methods. I put the two recorded into two test methods. The end browser in the recorded method1 is the same with the first browser in the recorded method2.

    When I run the two test methods in sequence, the same browser was not closed when the recorded method1 finished.

    And for the recorded method2, the playback will start in the same browser.

    So I suggest that you can create a simple coded UI test on another machine with the same VS2012 to run to check if tests can be run in sequence using the same browser.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Wednesday, October 10, 2012 7:42 AM
    Moderator
  • Hello Peter,

    What about your issue now? Have you resolved it?

    Would you mind letting us know the result of the suggestion?

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Friday, October 12, 2012 3:13 AM
    Moderator
  • Hi Peter,

    public void LaunchUrl(System.Uri url)
    {
           this.CopyFrom(BrowserWindow.Launch(url));
    }

    Did you try changing the above code to:

    public void LaunchUrl(System.Uri url)
    {
            BrowserWindow bw = BrowserWindow.Launch(url);
            bw.CloseOnPlaybackCleanup = false;
            this.CopyFrom(bw);
    }

    Thanks,

    Nagaraju

    Friday, October 12, 2012 4:44 AM
  • Hi Peter,

      If you are using CopyFrom(browserWindow) to create another BrowserWindow object, you should get CloseOnPlaybackCleanUp flag also copied from the original object; I am not able to reproduce it on my machine.

    BrowserWindow windopw = BrowserWindow.Launch(new Uri("http://www.Bing.com"));
             Console.WriteLine(windopw.CloseOnPlaybackCleanup);
             windopw.CloseOnPlaybackCleanup = false;
            
             BrowserWindow win2 = new BrowserWindow();
             win2.CopyFrom(windopw);
             Console.WriteLine(win2.CloseOnPlaybackCleanup);
    both are printing same value False

    CloseOnPlaybackCleanUp flag is there to make sure your application doesn't get closed after we call Playback.CleanUp() after each test case. But if the process which was running tests (in your case QTAgent32.exe) gets closed, all the child processes which were launched by it will get closed.

    This is the same behavior in VS 2010 also. The difference which you are observing is because of changes in Test execution process from Visual Studio itself. In VS 2012 QTAgent32 process gets closed after the run is over, so all the children get killed along with it. In VS 2010 this process used to run even after run was over.

    Now coming back to your problem:

       Do you want to use Browser window across multiple tests  or across multiple runs ?

    First you can achieve by putting CloseOnPlaybackCleanUp flag to false for BrowserWindow  object. Also you might want to set "Keep execution engine running between test runs" in Tools => Options => Web Performance Test Tools => Test Execution

    Second is not feasible with current implementation of Test Framework. I will log a bug in our database and will try to address this issue.

    Please let me know if it helps.

    Thanks

     


    Aditya Kumar Agrawal (MSFT)

    Friday, October 12, 2012 4:55 AM
  • In response to Amanda Zhu:

    Hi

    I have not yet tried it, I will and get back to you


    Monday, October 15, 2012 6:35 PM
  • In response to Nagaraju:

    Hi

    I tried it and it didn't work. The first browser was indeed kept alive but the next test method in line could not find it. It may have something to do with what Aditya is talking about in his answer.

    Anyway, this does not work because you cannot edit the Designer class as all changes are overwritten when it's regenerated. Maybe it can be overridden?


    Monday, October 15, 2012 6:38 PM
  • In response to Aditya Kumar Agrawal:

    Hi

    I was unclear about setting the CloseOnPlaybackCleanUp flag, sorry about that. I am able to set it to false and debugging shows that it was indeed set to false but no matter where I do this, the browser closes after a [TestMethod] has been run.

    Now, if I understand you correctly, this is because after the code within a [TestMethod] has finished executing the QTAgent32.exe thread is killed and everything that lives inside it dies meaning that no matter the value of the CloseOnPlaybackCleanUp flag, the browser is closed. Is this correct?

    Also, I checked the flag "Keep execution engine running between test runs" in Tools => Options => Web Performance Test Tools => Test Execution and it is checked so the session should be kept alive, yes? Or is this the bug you mentioned?

    Lastly, why does the QTAgent32.exe shut down between [TestMethod] scripts even when I run multiple tests in sequence? Let me explain:

    I have three [TestMethod]'s in one coded UI test class and each method shows up as a test script in the Test Explorer window in VS2012. When I mark all three and run them I'd imagine all three was run within one test run and that the QTAgent32.exe does not shut down between each test case but this seems to be the case.


    Monday, October 15, 2012 7:17 PM
  • Also, I checked the flag "Keep execution engine running between test runs" in Tools => Options => Web Performance Test Tools => Test Execution and it is checked so the session should be kept alive, yes? Or is this the bug you mentioned?

    Lastly, why does the QTAgent32.exe shut down between [TestMethod] scripts even when I run multiple tests in sequence? Let me explain:

    I have three [TestMethod]'s in one coded UI test class and each method shows up as a test script in the Test Explorer window in VS2012. When I mark all three and run them I'd imagine all three was run within one test run and that the QTAgent32.exe does not shut down between each test case but this seems to be the case.

    I will follow up UnitTest infrastructure team more on "Keep execution engine running between test runs" and will update this thread.

    As you said if you set CloseOnPlaybackCleanUp to False,  BrowserWindow doesn't get closed after the test case is finished, however the BrowserWindow object you created in previous TestCase is not valid anymore, you have to relocate the BrowserWindow in new test method.

    Please refer to the example below.

    BrowserWindow window;
    
            [TestMethod]
            public void Method1()
            {
                window = BrowserWindow.Launch(new Uri("http://www.bing.com"));
                window.CloseOnPlaybackCleanup = false;
            }
    
            [TestMethod]
            public void Method2()
            {
                window = BrowserWindow.Locate("Bing");
                window.CloseOnPlaybackCleanup = false;
            }
    
            [TestMethod]
            public void Method3()
            {
                window = BrowserWindow.Locate("Bing");
            }
    

    Please let me know if it solves your issue.

    Thanks


    Aditya Kumar Agrawal (MSFT)

    • Marked as answer by Peter Harlid Tuesday, October 16, 2012 1:55 PM
    Tuesday, October 16, 2012 4:29 AM
  • In response to Aditya Kumar Agrawal:

    Hi

    I was indeed able to keep a browser window alive between two [TestMethod]'s and use the browser window created in method 1 when running method 2 if I ran both methods in the same test run.

    It's not a particularly pretty solution especially since you have to be mindful of the window title when using BrowserWindow.Locate(string title) because the title of the window potentially changes and it also contains the browser type but it seems to work. This way I'll probably be able to fuse test cases together into workflows by encasing the browser launch mechanism in some kind of switch bases on a config flag read from an external file since the browser window can be passed along.

    Thanks for your help!

    Tuesday, October 16, 2012 1:54 PM
  • BrowserWindow.Locate was one of the example, what I meant was, you need to create BrowserWindow object in each testcase, the object you created in previous test method becomes obsolete in another test case.


    Aditya Kumar Agrawal (MSFT)

    Wednesday, October 17, 2012 3:51 AM
  • In reply to Aditya Kumar Agrawal:

    Understood, and in order to make use of a browser window created in a previous [TestMethod] and preserved the next [TestMethod] needs some way of finding it which Locate(string title) does. This was the missing piece in my code, the next [TestMethod] in line was unable to find the browser window previously opened so it would fail.

    One thing I still do not fully understand is why I cannot keep a browser window open by setting the CloseOnPlaybackCleanup property to false on the browser window created using the UIMap.Designer.cs. Let me explain:

    This is how the browser window is created on the UIMap.Designer.cs:

    [GeneratedCode("Coded UITest Builder", "11.0.50727.1")]
        public class BrowserWindow1 : BrowserWindow
        {
            
            public BrowserWindow1()
            {
                #region Search Criteria
                this.SearchProperties[UITestControl.PropertyNames.Name] = "Bing";
                this.SearchProperties[UITestControl.PropertyNames.ClassName] = "IEFrame";
                this.WindowTitles.Add("Bing");
                #endregion
            }
            
            public void LaunchUrl(System.Uri url)
            {
                this.CopyFrom(BrowserWindow.Launch(url));
            }
            
            #region Properties
            public LoginPage LoginPage
            {
                get
                {
                    if ((this.mLoginPage == null))
                    {
                        this.mLoginPage = new LoginPage(this);
                    }
                    return this.mLoginPage;
                }
            }
            #endregion
            
            #region Fields
            private LoginPage mLoginPage;
            #endregion
        }

    And then in the UIMap.cs class it is modified like so:

    public void OpenBrowser(string url)
            {
                BrowserWindow.LaunchUrl(new System.Uri(url));
            }

    and then finally called in a script like this:

    UiMap.OpenBrowser("http://bing.com");

    But if I set the CloseOnPlaybackCleanup to false in UIMap.cs like this:

    public void OpenBrowser(string url)
            {
                BrowserWindow.LaunchUrl(new System.Uri(url));
                BrowserWindow.CloseOnPlaybackCleanup = false;
            }

    it is still closed after the [TestMethod] using OpenBrowser(string url) finishes. Why is this, do you know? 

    Wednesday, October 17, 2012 12:34 PM
  • This isn't really a work around to the solution and I haven't seen anything that does resolve this issue, but I found a way around the problem.  It has pro's and con's...

    Create your Test Methods as you need them.  Then create a controlling Test Method that you actually run.  In that controlling Test Method, reference the other Test Methods that you want to execute.

        [TestMethod]
        public void CreateAllUsers()
        {
            // Open Browser to site and log in
            RootSteps.SingleUserSignIn("http://www.msn.com", "Joe", "JoesPWD");
            // Navigate to Entity Management
            this.ClickReferencepane();
            this.ClickEntityManagementlink();
            // Now we begin to create the first user
            this.UIMap.CreateUser_ClickCreateLink();
            this.UIMap.CreateUser_ClickUserButton();
            this.UIMap.CreateUser_ClickSaveButton();
            RootSteps.ClickSignOutLink();
            RootSteps.CloseInternetExplorerBrowserFromSignout();
        }
        [TestMethod]
        public void ClickReferencepane()
        {
            this.UIMap.ClickReferencepane();
        }
        [TestMethod]
        public void ClickEntityManagementlink()
        {
            this.UIMap.ClickEntityManagementlink();
        }

    This code shows calls made to local Test Methods, methods associated with the UIMap and Methods in another Project that is utilized in other tests.  This forces excess tests to show in the Test Explorer (the children tests) but you only execute the parent test, which then screws up your Code Coverage, because they do not register as tests that have been run when they are called in the parent test method.  Another problem is that if a test method is associated with a data table, then it will not iterate through the table when it is called as a child test method as above.  In essence the data table associated method will need to be duplicated and designed to stand alone as a child method if you wish to use it in the collective test parent.  I don't like this work around because many tests go incomplete and the boss is seeing the results of what has been run versus what has not, so I have to explain how the tests are being run in the parent and it does not reflect for the children tests.  It also clutters the Test explorer list and then you have 2 choices, name children tests unique, then filter the list accordingly OR comment out the [TestMethod] attribute on the children Test Methods making them just normal methods.  Again by doing this, you simplify the Test Explorer list, but you also lose tracking on what has been covered in your testing.

    LEHayes

    • Proposed as answer by SMSBVT Friday, December 7, 2012 4:57 PM
    Friday, December 7, 2012 4:57 PM
  • Taking a little time to play with the idea...

    If you have a browser entity open prior to running a code set like the following, it will actually run:

            public CodedUITest1()
            {   
            }
            BrowserWindow InitWindow = new BrowserWindow();
            [TestMethod]
            public void ClickAPic()
            {
                BrowserWindow ClickAPicWindow = new BrowserWindow();
                ClickAPicWindow.CopyFrom(InitWindow);
                this.UIMap.ClickAPic();
            }
            [TestMethod]
            public void ClickASecondPic()
            {
                BrowserWindow ClickAPicWindow = new BrowserWindow();
                ClickAPicWindow.CopyFrom(InitWindow);
                this.UIMap.ClickASecondPic();
            }
            [TestMethod]
            public void ClickBothPics()
            {
                this.UIMap.ClickAPic();
                this.UIMap.ClickASecondPic();
            }
            #region Additional test attributes
            [TestInitialize()]
            public void MyTestInitialize()
            {
                InitWindow.NavigateToUrl(new Uri("http://www.msn.com"));
                InitWindow.CloseOnPlaybackCleanup = false;
            }
            //Use TestCleanup to run code after each test has run
            [TestCleanup()]
            public void MyTestCleanup()
            {
            }
            #endregion

    You can test this by running The first two test methods, but you better put some code behind those methods.  Notice that ClickBothPics will do the same thing, but it will not record results for the child test methods.

    LEHayes

    Friday, December 7, 2012 10:34 PM
  • Another answer to Peter's original question might be provided by the "CloseOnPlaybackCleanup" property explained in this blog:

    Coded UI Test: Why does application close after each test in Visual Studio 2012?

    http://blogs.msdn.com/b/visualstudioalm/archive/2012/11/08/using-same-applicationundertest-browserwindow-across-multiple-tests.aspx

    Regards

    Adrian

    Monday, December 10, 2012 9:08 AM