none
Ordered tests with VS2017 Database Unit Test Project

    Question

  • Hi all

    I know using ordered tests for unit testing is not strictly correct. However, I'm using the test project to run system tests and want to run SPs in a particular order. My end goal is to hook this up in an Azure DevOps pipeline.

    I'm using VS2017 on my dev box and have a unit test project for my database project. I've created an ordered test file and can run that from VS fine. I'm trying to run the same ordered list from the command line on my dev box with vstest.console.exe but can't get it to work. I can run tests using the console when specifying the test dll or a particular test method. The error message I have is:

    No test is available in C:\Users\...\TestProject2\Build\createnewsolution.orderedtest. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.

    Additionally, path to test adapters can be specified using /TestAdapterPath command. Example  /TestAdapterPath:<pathToCustomAdapters>.

    As a workaround I did wonder about calling individual tests from the console but they would take ages to run due the console  initialisation with each call. Is there a way to fix this, or achieve the same thing by a different means?

    Thank you for having a look at this, Brian


    Brian Searle

    Friday, May 17, 2019 9:33 AM

Answers

  • Hi Brian Searle,

    Sorry for delay in reply.

    Did the test project use MSTest framework(V1)?

    As far as I know, the ordered test just works on the MSTest framework(V1), but the vstest.console.exe works on MSTest framework(V2). Here is github issue: https://github.com/Microsoft/testfx/issues/25

    BTW, MStest.exe works on the MSTest framework(V1), so I suggest you could use MStest.exe to execute the test, please refer the below sample:

    mstest /testcontainer:"D:\UnitTest\RunSingleTest\UnitTestProject1\bin\Debug\OrderedTest1.orderedtest"

    Note: please refer this doc for more command line options.

    Sorry for this inconvenience.

    Best Regards,

    Dylan


    MSDN Community Support Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com

    • Marked as answer by Brian Searle Tuesday, May 21, 2019 9:13 AM
    Tuesday, May 21, 2019 3:12 AM

All replies

  • Hi Brian Searle,

    Sorry for delay in reply.

    Did the test project use MSTest framework(V1)?

    As far as I know, the ordered test just works on the MSTest framework(V1), but the vstest.console.exe works on MSTest framework(V2). Here is github issue: https://github.com/Microsoft/testfx/issues/25

    BTW, MStest.exe works on the MSTest framework(V1), so I suggest you could use MStest.exe to execute the test, please refer the below sample:

    mstest /testcontainer:"D:\UnitTest\RunSingleTest\UnitTestProject1\bin\Debug\OrderedTest1.orderedtest"

    Note: please refer this doc for more command line options.

    Sorry for this inconvenience.

    Best Regards,

    Dylan


    MSDN Community Support Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com

    • Marked as answer by Brian Searle Tuesday, May 21, 2019 9:13 AM
    Tuesday, May 21, 2019 3:12 AM
  • Hey Dylan, thank you very much for your reply and I thought the reply was quick!

    I have it working, so chuffed.

    The only thing left is to see if I can fix some warnings with missing dependencies to Microsoft.Build, Microsoft.Build.Framework and Microsoft.Data.Tools.Schema.Sql.

    I also downloaded and tested against the VS2019 Test Agent and it works on that so I should be able to automate this on my test server in my pipeline!

    You mentioned "Did the test project use MSTest framework(V1)?" is there a way to define which is used in the project? The reason I ask is it looks like the test runners MSTest and VSTest (console.exe) both work on the same project. Anyway, it's not a big thing, just part of learning the test tools....

    In case this might be useful for someone else, I found the VS Test Agent installation folder structure also includes 2 different versions of vstest.console.exe (assume this is the V2 test framework). The versions seem to be for VS2017 and VS2019!



    Brian Searle

    Tuesday, May 21, 2019 9:38 AM