locked
test cases' execution order in Test List RRS feed

  • Question

  • Hi all,

      I find it is curious of the execution order in a test list.

      I put some test cases in a test list. And select the test list to run. In the following execution of the bulk test cases in the test list, the run sequence is not based on the alphabet.  Could anybody tell me the sequence of the test run for a test list?

      Thanks a lot!

    Leon

    Thursday, March 30, 2006 9:28 AM

Answers

  • Hi,
    If u need to run tests in specific sequence u can create OrderedTest.

    Regards,
    Tomas

    Thursday, March 30, 2006 11:04 AM
  •  "For an ordered test, there is only one test result for it, even there are several test cases in an ordered test. This will make the test result statistics inaccurate."

    This is not true. If you have two tests in an ordered test and run the ordered test, the Test Result tool bar shows 3/3 passed. Furthermore, if you double click and open the result detail, if shows you the two contained tests. You can further click and open the result detail for each inner test.

    That being said, yes, the execution order in a test list is random.

    Thanks,

    James

    VSTS

    Thursday, March 30, 2006 5:53 PM

All replies

  • Hi,
    If u need to run tests in specific sequence u can create OrderedTest.

    Regards,
    Tomas

    Thursday, March 30, 2006 11:04 AM
  • Tomas,

      For an ordered test, there is only one test result for it, even there are several test cases in an ordered test. This will make the test result statistics inaccurate.

      My thought is: no matter which sequence it is, there should be one sequence for the test cases' execution in a test list. If so, what is it? Or, is it random? Sounds impossible.

      Thanks,

    Leon

    Thursday, March 30, 2006 2:41 PM
  •  "For an ordered test, there is only one test result for it, even there are several test cases in an ordered test. This will make the test result statistics inaccurate."

    This is not true. If you have two tests in an ordered test and run the ordered test, the Test Result tool bar shows 3/3 passed. Furthermore, if you double click and open the result detail, if shows you the two contained tests. You can further click and open the result detail for each inner test.

    That being said, yes, the execution order in a test list is random.

    Thanks,

    James

    VSTS

    Thursday, March 30, 2006 5:53 PM
  • James,

      I go through your tips. It is right, but sounds not what I desire.

      My purpose is to count the test result summary of test cas. To download all the test results for a build to local, from the test result view, i can see how many test case pass, how many fail. If there are orderd test case mixed into it, it becomes hard to count it.

      Or, is there any ways to count the summary of test result for a build easily? just count how many pass, how many fail. I know in test build view there is also such kind of data, but it sounds not straightforward to get the summary data -- users still have to count manually.

      And as the execution order in a test list, I did a quick test. It sounds the order is based on the creation time of the test cases

      Thanks,

      Leon.

    Friday, March 31, 2006 3:27 AM
  • Leon,
    The easiest way to browse the test results is to double click in Team Explorer/Team Project/Team Builds/All Build Types, than double click on interested build number, test result data appers in Result details. But you have to publish test run before.

    regards,
    Tomas

     

    Friday, March 31, 2006 8:00 AM