locked
Rerun Failed Coded UI Tests In Test Suite RRS feed

  • Question

  •  Hi All,

    We have to execute a test suite (containing more than thousand Coded UI tests) on regular basis. As of now, we are using 'MSTest.exe' and calling it using a batch file at the EOD. The problem we are facing is that many of the test cases (approx 50) are getting failed in the suite. Although most of the failed test cases are passing if we execute them individually.

    The failed test case count can be reduced if we rerun the failed test cases in the test suite result. This can be done manually using VS2010 but our requirement is to rerun them automatically.

    How can we achieve this?

    An early reply will always be appreciated.


    Ankur Saini

    Monday, December 10, 2012 12:36 PM

Answers

  • Hello Saini,

    Thank you for your post.

    According to your description, I think that your test cases are normal because you can execute them successfully individual. You said that there are more than thousand Coded UI tests in the test suite and you will run all of them using a batch file and MSTest.exe. I am afraid that it is a challenge to your computer performance because of running so many tests once. Maybe some performance issues will occur.

    In addition, if you want to run those failed tests automatically instead of doing manually in the test results, I do a research and don’t find any related solutions. As far as I know, VS IDE does not support this feature.

    To help you resolve this issue, you could submit this feature request:

    http://visualstudio.uservoice.com/forums/121579-visual-studio

    The Visual Studio product team is listening to user voice there. You can send your ideas/suggestions there and people can vote.

    Thanks for your understanding.

    If I have misunderstood anything, please feel free to let me know.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.




    Tuesday, December 11, 2012 3:35 AM
    Moderator
  • Hello Ankur,

    perhaps this is a solution for you. Using Reflection from inside TestCleanup

      
      [TestCleanup]
        public override void MyTestCleanup()
        {
          if (TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
          {       
            var type = Type.GetType(TestContext.FullyQualifiedTestClassName);       
            if (type != null)
            {
              var instance = Activator.CreateInstance(type);
              var method = type.GetMethod(TestContext.TestName);
              try
              {
                method.Invoke(instance, null);
              }
              catch
              {
               
              }
            }
          }

    Friday, December 14, 2012 10:36 AM

All replies

  • Hello Saini,

    Thank you for your post.

    According to your description, I think that your test cases are normal because you can execute them successfully individual. You said that there are more than thousand Coded UI tests in the test suite and you will run all of them using a batch file and MSTest.exe. I am afraid that it is a challenge to your computer performance because of running so many tests once. Maybe some performance issues will occur.

    In addition, if you want to run those failed tests automatically instead of doing manually in the test results, I do a research and don’t find any related solutions. As far as I know, VS IDE does not support this feature.

    To help you resolve this issue, you could submit this feature request:

    http://visualstudio.uservoice.com/forums/121579-visual-studio

    The Visual Studio product team is listening to user voice there. You can send your ideas/suggestions there and people can vote.

    Thanks for your understanding.

    If I have misunderstood anything, please feel free to let me know.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.




    Tuesday, December 11, 2012 3:35 AM
    Moderator
  • Hello Ankur,

    I marked my reply as answer. If my suggestion can’t help you, you can come back and unmark it as answer. If you have anything unclear, please feel free to come back and ask me.

    Best reagrds,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Thursday, December 13, 2012 12:20 AM
    Moderator
  •  Thanks Amanda for reply!

    I also think that VS does not support this feature. So I am trying to find some workaround about this. As a first step, I am planning to rerun failed test cases by doing custom coding in batch file.

    I have submitted feature request for the same.


    Ankur Saini

    Thursday, December 13, 2012 12:13 PM
  • Hello Ankur,

    Thank you for your understanding and support. If possible, you can update the workaround here if you have succeeded to rerun failed test cases by doing custom coding in batch file, which will benefit other members with the similar issue.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.


    Friday, December 14, 2012 1:06 AM
    Moderator
  • Hello Ankur,

    perhaps this is a solution for you. Using Reflection from inside TestCleanup

      
      [TestCleanup]
        public override void MyTestCleanup()
        {
          if (TestContext.CurrentTestOutcome == UnitTestOutcome.Failed)
          {       
            var type = Type.GetType(TestContext.FullyQualifiedTestClassName);       
            if (type != null)
            {
              var instance = Activator.CreateInstance(type);
              var method = type.GetMethod(TestContext.TestName);
              try
              {
                method.Invoke(instance, null);
              }
              catch
              {
               
              }
            }
          }

    Friday, December 14, 2012 10:36 AM
  • Hi Markus,

    Thanks for the reply!

    The above code seems good. Let me try this.


    Ankur Saini

    Friday, December 14, 2012 10:42 AM
  • Hello Ankur,

    I am marking Markus' reply as answer. If his suggestion can’t help you, you can come back and unmark it as answer. If you have anything unclear, please feel free to come back and ask me.

    Best reagrds,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Tuesday, December 18, 2012 12:21 AM
    Moderator