locked
MSTest marks tests as Failed instead of Inconclusive when any exception is thrown RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    Recently I’ve discovered that MSTest can label tests as Inconclusive.  This has been extremely helpful for weeding out tests that didn’t pass because they need maintenance, and not because the actual test case failed.  The issue is that tests only fall into this category if I explicitly call Assert.Inconclusive() in each test method.

    Is there a reason why any test that throws an exception (besides AssertInconclusiveException) is labeled as Failed?  Ideally we would only want tests to show up failed if they throw AssertFailedException. 

    Is there any way to control this without wrapping each test in a try-catch?  Right now, anytime we have UI changes in our application our tests fail.  However, we don’t really know if the test failed because the test couldn’t finish.

    Friday, March 18, 2011 2:35 PM

Answers

  • Question
    Sign in to vote
    0
    Sign in to vote

    The failure of test is not defined only by AssertsFailures, there could be n different reasons where a test execution did not proceed as expected hence the test is marked as failed. Assert.Inconclusive helps to filter out those reasons and log them appropriately reasons could be configuration, build etc.. Hence user has to define that set and user Assert.Inconclusive where ever needed not the unit testing framework.

    When you say, UI changes ideally there should a sync between your tests at the same time if it is not, the test fails and you get to know what was the change. This is the default behavior, if you want test not to fail in this case but to be inconclusive you have to handle the failure and call out appropriately.

    To handle the playback failure in such case you can refer the following thread, where your code will get a call back for any playback failure, filter out the reason and do what you want after that.

    http://social.msdn.microsoft.com/Forums/en/vsautotest/thread/b8b946e5-305b-4952-9172-f7a31846198f

     

    Thanks.

    Friday, March 18, 2011 3:14 PM