locked
Coded UI Results numbers - how is this calculated? RRS feed

  • Question

  • When running several test cases at once in Coded UI, the results display as such:

    Results 10/10 passed

    How is the total number calculated?  We frequently run a set of ten test cases.  Each test case has it's own method, with a TestMethod attribute.  Those are the only methods with TestMethod attribute.  But after running all ten test cases, the results get reported as a greater number than that.  For instance, I have a class containing 10 test cases, with one method per test case, but the results are reported as 21/21 passed.

    Other factors to consider - we are data-driving our test methods with XML input parameters, one node per test case.  We sometimes call shared steps as methods (although shared step methods do not get TestMethod attributes).  There is no looping in the code to call any TestMethod more than once.  We typically call the tests from the Test View window, although some classes have OrderedTests, which also report the numbers incorrectly.  We have frequent assertions in the methods, and the results number is much less than the number of assertions.

    Is there a way to know how this number is calculated?  Is there a way to force reporting of one passed per TestMethod?

     

    • Moved by Siddhartha Pandey - MSFTMicrosoft employee Wednesday, November 10, 2010 4:14 AM Moving to Unittesting forum for a better answer to the question. (From:Visual Studio UI Automation Testing (includes CodedUI))
    Tuesday, November 9, 2010 5:47 PM

Answers

  • Hi,

    How many tests do you see in the test results window. I think the ordered tests are the culprit here. If you run an ordered tests which has 3 unit tests in it, the reult will be reported as 4/4 passed and you will only see 1 test in the test results window. If you double click this test result, you will see 3 more tests inside it.

    Thanks,

    Anuj

    Friday, November 12, 2010 10:20 PM