locked
Tests running with resharper but not with ms test RRS feed

  • Question

  • Hi,

    we have a set of unit tests, which all pass using ReSharper to run them. Now we are installing a build server. It already builds our solution well, but if we want to run our unit tests, more than 40% of them fail. I assume, the build server is using ms test. Therefore I today tried to run our test on my dev-machine with ms test instead of resharper but I had to realize that quite a few also don't pass (different %age though).

    What also is odd, ReSharpers UnitTestExplorer finds 129 Tests, while the "Test View" from VS only finds 111 tests.

    I assume, that during the developement, while using ReSharper only for testing, something must have beend done in a way, that it now does not suite ms test any more....

    I also assume, that I will have to get all tests running on my dev machine with ms test again, before I can proceede to have build server do the testing as well.

    I apreciate any hints and help on how to aproach my issue.

    Regards
    Rainer

    Friday, October 26, 2012 9:28 AM

Answers

  • Hello Rainer,

    Glad to receive your reply.

    So, where does the "Source\TestResults\Queck_GH13448_NB 2012-10-29 11_29_42\Out" come from?

    Each test run folder contains a folder named Out. The Out folder is the actual deployment folder. Assemblies and other files or folders that are required for the test run are copied to the <solution>\TestResults\<test run>\Out folder every time that a test run is started if you select Enable Deployment.

    For detailed information, you can refer to this article:

    How to: Configure Test Deployment

    In addition, MSTest runs tests through the test project dll file in the bin\               Debug folder. When the dll is found, all tests in the dll can be run.

    According to your error message, a dll file can’t be found. You can check if the dll file is in the Debug folder. Generally when you build test project or solution in VS, the test project dll file will be in the bin\Debug folder.

    Also I would like to know if all your tests are unit tests. Are there other types of tests such as ordered test in your test project?

    If there are not, not all tests can be display, and some dll files can’t be found even though they are in bin\Debug folder, I think that maybe it has something to do with ReSharper tool. Generally if you create tests in VS, test view can display all tests. So I suggest that you can create unit tests in VS to see if all tests can be found.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Marked as answer by Rainer Queck Thursday, November 1, 2012 9:00 AM
    Tuesday, October 30, 2012 2:45 AM

All replies

  • Hello Rainer,

    Thank you for posting in the MSDN forum.

    What also is odd, ReSharpers UnitTestExplorer finds 129 Tests, while the "Test View" from VS only finds 111 tests.

    I would like to know where you create your unit tests. In ReSharpers? Or in VS?

    If you create unit tests in ReSharpers tool, I suggest that you create them from VS to check if all of them can be recognized by VS.

    but if we want to run our unit tests, more than 40% of them fail. I assume, the build server is using ms test.

    More than 40% of unit tests can fail using MStest, if you run them from VS IDE, will this similar situation occur? If you run them on another machine with the same VS, will it still occur?

    In addition, are there any errors when some unit tests failed? If so, you can provide us detailed error message for resolving this issue.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Monday, October 29, 2012 1:23 AM
  • Hello Amanda,

    thank you for your efforts to help me!

    <------
    I would like to know where you create your unit tests. In ReSharpers? Or in VS?
    >------
    In our team we started out to create the test projects (we have a couple of them in our solution) in VS and later on used a self written template in ReSharper to add test classes to those projects.

    <-----
    If you run them on another machine with the same VS, will it still occur?
    >-----
    Yes.

    <-----
    In addition, are there any errors when some unit tests failed? If so, you can provide us detailed error message for resolving this issue.
    >-----
    To get started with one error...
    -- Error Message ----
    Failed When_FisDataProviderSubprocess_Is_Started_Then_No_Critical_Nor_Error_Nor_Warning_Logs_Occur FisDataProvider.MapApp.Tests.Interation Class Initialization method FisDataProvider.MapApp.Tests.Interation.FisDataProviderTest.GeneralSetup threw exception. System.IO.FileNotFoundException: System.IO.FileNotFoundException: Die Datei "D:\Daten\Projects\ChCoater\3_Entwicklung\Dev\Source\TestResults\Queck_GH13448_NB 2012-10-29 11_29_42\Out\FisDataProvider.MapApp.Tests.Interation.dll" konnte nicht gefunden werden.. 
    -- End Error Message ---

    Actually the test assembly referenced ist located at :
    D:\Daten\Projects\ChCoater\3_Entwicklung\Dev\bin\FisDataProvider.MapApp.Tests.Interation.dll
    D:\Daten\Projects\ChCoater\3_Entwicklung\Dev\Source\TestResults\Queck_GH13448_NB 2012-10-29 11_29_42\Out\FisDataProvider.MapApp.Tests.Interation.dll
    So, where does the "Source\TestResults\Queck_GH13448_NB 2012-10-29 11_29_42\Out" come from? Runing this test from ReSharper causes no problem...


    What are the "mechanics" behind ms test? How does it know about unit tests to do? Ist it possible, that it does not know about some tests, because I added them using my resharper code generation templates?

    Looking forward to further help ;-)

    Regards
    Rainer

    Monday, October 29, 2012 10:39 AM
  • Hello Rainer,

    Glad to receive your reply.

    So, where does the "Source\TestResults\Queck_GH13448_NB 2012-10-29 11_29_42\Out" come from?

    Each test run folder contains a folder named Out. The Out folder is the actual deployment folder. Assemblies and other files or folders that are required for the test run are copied to the <solution>\TestResults\<test run>\Out folder every time that a test run is started if you select Enable Deployment.

    For detailed information, you can refer to this article:

    How to: Configure Test Deployment

    In addition, MSTest runs tests through the test project dll file in the bin\               Debug folder. When the dll is found, all tests in the dll can be run.

    According to your error message, a dll file can’t be found. You can check if the dll file is in the Debug folder. Generally when you build test project or solution in VS, the test project dll file will be in the bin\Debug folder.

    Also I would like to know if all your tests are unit tests. Are there other types of tests such as ordered test in your test project?

    If there are not, not all tests can be display, and some dll files can’t be found even though they are in bin\Debug folder, I think that maybe it has something to do with ReSharper tool. Generally if you create tests in VS, test view can display all tests. So I suggest that you can create unit tests in VS to see if all tests can be found.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    • Marked as answer by Rainer Queck Thursday, November 1, 2012 9:00 AM
    Tuesday, October 30, 2012 2:45 AM
  • Hello Rainer,

    Any update? Could you get useful information from our reply?

    Would you mind letting us know the result of the suggestion?

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us

    Thursday, November 1, 2012 5:47 AM
  • Hello Amanda,

    Sorry I didn't reply till now, but I am quite busy with other asspects of our current project, but I am still into investigating the ms teset odds. We have most of our tests running now. The main problem is, that ms test runs our unit tests in different folders than we assumed (<solution>\TestResults\<test run>\). Since we are using MEF within our project, we required assemblies in our test directory which is the project-build-output path of all our projects within the solution. since ReSharper doesn't move arround the test projects, things work fine. MS Test moves assemblies other locations "...<test run> ...." Which in my eyes is a bit odd.

    I will do some more invetigations on this subject with my team. I am also considering to move to NUnit for the next project instead of using MS Test since it seems to be more strait forward....

    How ever, I thank you very much for your help and hints!!

    Regards
    Rainer

    Thursday, November 1, 2012 8:59 AM