Unit tests fail with code coverage

Question

I have a project with about 400 unit tests.  When I run the tests in debug mode, they all pass - even if I don't have any breakpoints.  When  run by clicking Run, 32 of the tests fail.  Since I get a popup about code coverage not running in debug, I finally decided to try turning code coverage off.  With only that one change, now all the tests work whether I run or debug.  The error returned is:

Test method MyNamespace.MyTestMethod threw exception:  System.NullReferenceException: Object reference not set to an instance of an object..

There is nothing in the call stack shown in the test results that helps because the line numbers shown are usually lat the end of a method rather than an actual failure so there's no way to tell what object is null or why.  And I can't debug because it always works when debugging.

Any ideas why tests won't pass with code coverage on?

Thanks,

Dale

Answer 1

Hello,

Based on your description, if you run the 400 tests in debug mode, they all work well, if they run without debugging, 32 of them fail. If you turn off the code coverage, the tests all work well in either the two modes. You may verify the output generated in the detailed results window (double clicking the unit test in the test results window) to see if there are any more detailed information. In addition, does the test work well separately? You may debug this test separately to see if the isssue still exists. As the error message shows MyNamespace.MyTestMethod fails, you may verify the method to see if something may not have been initialized correctly.

We could turn off the popup about coverage like this: Click Tools -> Options, expand the Test Tools node and select Default Dialog Box Action. Set the value of "When starting a remote test run or a run with code coverage under the debugger" to "Don't prompt".

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Send us any feedback you have about the help from MSFT at [e-mail].

Answer 2

For the tests that fail, I can run them individually with code coverage on and they fail.  I can turn off code coverage and they pass.  That's just one test.

*rant*  I know that this thread and forum is about testing, and please be assured that I really just want to solve my test problem, but I can't resist adding that this problem would be a lot easier if Microsoft would fix the lack of information in the NullReferenceException.  There has been enough screaming over the years that one would think that Microsoft would have heard by now.  It would be very helpful to include the type of object and even the instance id of the object in the NullReferenceException message. */rant*

Regards,

Dale

Answer 3

Hello Dale,

Could you please send a few tests that fail with code coverage on and pass with code coverage off to me( v-jinfan(at)microsoft.com ) for further troubleshoot?

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Send us any feedback you have about the help from MSFT at [e-mail].

Answer 4

I will do that next week, Edwer.  Thank you.

Answer 5

We are changing the issue type to “General Discussion” because you have not followed up with the necessary information. If you have more time to look at the issue and provide more information, please feel free to change the issue type back to “Question” by opening the Options list at the top of the post window, and changing the type. If the issue is resolved, we will appreciate it if you can share the solution so that the answer can be found and used by other community members having similar questions.

---

Please remember to mark the replies as answers if they help and unmark them if they provide no help.
Send us any feedback you have about the help from MSFT at [e-mail].