none
coded ui tests dont cleanup after timeout. RRS feed

  • Question

  • Hi, 

    I've a number of tests running on Visual Studio Enterprise 2015 testing a Windows GUI (not browser) application. 

    The tests dynamically alter configurations at startup and clean them up at the end. However, when a test times out for whatever reason the cleanup method is not run, leaving subsequent tests in a mess. 

    [TestMethod, TestCategory("Smoke"), , TestCategory("Fixed")]
    [Timeout(1800000)] 
    public void TestCaseName()
    {
     // Do test stuff
    }

    [TestInitialize()]
    public void MyTestInitialize()
    {
     // Dynamic config setup
    }

    [TestCleanup()]
    public void MyTestCleanup()
    {
     // Cleanup everything.
    }

    I've read a number of threads on this, but the solution seems to be that cleanup should run after timeout. 

    Regards,

    John.

    Wednesday, October 4, 2017 7:26 AM

All replies

  • Hi John,

    If there is an exception thrown in TestInitialize, then TestCleanup won't be called.

    For TestInitialize, you could put a try catch around it and see.

     

    Best regards,

    Fletcher


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.



    • Edited by Fletch Zhou Thursday, October 5, 2017 3:11 AM
    • Proposed as answer by Fletch Zhou Friday, October 6, 2017 8:42 AM
    Thursday, October 5, 2017 3:10 AM
  • Hi John,

    Do you have tried use try catch in your TestInitialize method to check whether there has any exception as Fletch suggested?

    You also could add a try catch in your TestCaseName method and add clean up operation in both catch block in TestCaseName method and TestInitialize method. Then no matter your test get exception in TestCaseName method and TestInitialize method, it will execute the clean up operation in catch block.

    Best Regards,
    Weiwei


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, October 13, 2017 1:38 AM
    Moderator
  • Hi Fletcher,

    I've a try catch around test Initialize and in the general test method. The test Initalize completes successfully and moves into the general part of the test case. There are no exceptions reported other than it might hang and eventually timeout. 

    Say I get the testcase to timeout after 10 minutes and its doing its thing for 10 minutes, at that point it stops abruptly and does not go into cleanup. Some of our test cases dynamically update a config file in Test Initialize and undo these changes in the Cleanup. However if they Timeout and don't attempt to cleanup the next test case and subsequent test cases are messed up.

    Regards,

    John. 

    Friday, November 3, 2017 2:57 PM
  • Hi John,

    >>I get the testcase to timeout after 10 minutes and its doing its thing for 10 minutes, at that point it stops abruptly and does not go into cleanup

    you mean the test run 10 minutes after time's out it stops without running test clean up right?

    Your code you posted above seems fine, i also wrote simple test like yours, which always gave me the right order of execution, and test cleanup run as well.

    Then, would you mind testing a simple test(just output something), without setting any Timeout attribute, to see if the test clean could be executed this time.

    I just want to know if the timeout attribute caused this issue in your side.

     

    Best regards,

    Fletcher  


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    • Edited by Fletch Zhou Monday, November 6, 2017 6:03 AM
    Monday, November 6, 2017 6:03 AM
  • I think just figured it out.

    If I set the time out short than that my test need, I wont run test clean, but if set the time out long enough, it will run the test clean up.

    So, I suggest you set more time please.

    My simple sample:

      [TestMethod]
            [Timeout(10000)]
            public void TestCaseName()
            {
                
                //Console.WriteLine("NoTimeout");
                //Console.WriteLine("NoTimeout");
                int i = 0;
                while(i<1000)
                {
                    i++;
                    Console.WriteLine("NoTimeout");
    
                }
    
    
            }
    
    
       
            [TestInitialize()]
            public void MyTestInitialize()
            {
                Console.WriteLine("Initialize");
                // Dynamic config setup
            }
    
            
            [TestCleanup()]
            public void MyTestCleanup()
            {
                Console.WriteLine("cleanup");
                // Cleanup everything.
            }

    the result is like NOT completed:

    Initialize

    NoTimeout

    NoTimeout

    NoTimeout

    NoTimeout

    ...

    NoTi

    But if change the code while(i<1000) to while(i<100), which means it need less time which is within the time I set in Timeout.

    The result is completed:

    Initialize

    NoTimeout

    NoTimeout

    NoTimeout

    NoTimeout

    NoTimeout

    NoTimeout

    cleanup

    NOTE: the last line is cleanup, which means it was executed successfully.

    Best regards,

    Fletcher


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.


    • Edited by Fletch Zhou Monday, November 6, 2017 6:23 AM
    Monday, November 6, 2017 6:23 AM