locked
vstest and continuous integration tools RRS feed

  • Question

  • Hi All.  I was wondering if anyone has been using continuous integration tools with the visual studio unit testing framework (vstest.console.exe)?  If so, what tool(s) are you using and what are your opinions of said tool?

    I have had a look into Jenkins but I'm stomped with it and can't find any information on getting vstest working with it.  Thanks for any help.  Much appreciated


    Tuesday, December 16, 2014 2:58 PM

Answers

  • Hi veli.  Thanks for getting back.  It's frustrating for sure.  I know it can be done but finding any "how-to" guides or some forum posts online has proved to be non-existent - at least for vstest.console.exe and jenkins.

    Although I'm hoping to use Jenkins, in terms of a CI tool, I wont need to use most of it's features.  Our needs are pretty simple.  Run our tests on a schedule and get email notifications including the test results.  We don't actually need to continuously run tests after each build.

    We where running our unit tests through the test explorer window in visual studio until I realized we could create batch files and invoke our DLL's through vstest.sonsole.exe

    This suited our needs and was working fine to a point.  I still had to manually run the batch files, check on the results then upload the trx files onto our network.  I have since scheduled the batch files to run as a windows task throughout the day and night on a remote server running 8.1 so this takes care of having to run the batch files manually.  

    What I'm lacking now are the email alerts with the nice reports that I can just forward to management :)  

    I hope you get sorted.  I'm continuing to look.  Should I find anything interesting, I'll come back and post here 

    Thursday, December 18, 2014 4:54 PM

All replies

  • Hi r.y.a.n,

    Thank you for posting in MSDN forum.

    Based on your issue, to further help solve this issue, please provide us more detailed message for this issue.

    What did you want to do function operation using continuous integration tools for unit test?

    Could you please tell me if you use the TFS to run your unit test framework?

    If yes, I know that there have a way to build the unit test using continuous integration from TFS like the following screen shot.

    Therefore, I suggest you can try to create a build definition and then select the Continuous Integration- Build each check in. It will help you build and run your unit test automatically after you each check in the unit test project.

    Reference:

    http://msdn.microsoft.com/en-us/library/ms181716.aspx

    If I misunderstand your issue, please you for free me.

    Best Regards,


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    • Edited by Tina-Shi Wednesday, December 17, 2014 5:31 AM
    Wednesday, December 17, 2014 5:23 AM
  • Hi r.y.a.n - funny you should ask that, as I am on the exact same quest; getting my Windows Phone 8.1 unit tests integrated into a jenkins pipeline using vstest.console.

    My plan was to setup a jenkins job with a build step which just calls vstest.console.exe against the unit test app package (*.appx) from the command-line (but then this happened https://connect.microsoft.com/VisualStudio/feedback/details/1056305/vstest-console-execution-of-appx-packages-broken).

    I did not look into any jenkins "plug-ins" for vstest.console or other tools for executing unit tests.  AFAIK, vstest.console is the simplest and most basic way to execute VS unit tests from command line - and once you have command-line power, integration is easy.  There are may other ways; Team Foundation Server (TFS), complicated MSBuild jenkins plugins, etc., but all require certain time/knowledge investment.

    Without knowing too much about your environment and end goal, I am confident that vstest.console is the fastest way to unit test CI...you're on the right track.

    /veli


     
    Thursday, December 18, 2014 1:12 AM
  • Hi veli.  Thanks for getting back.  It's frustrating for sure.  I know it can be done but finding any "how-to" guides or some forum posts online has proved to be non-existent - at least for vstest.console.exe and jenkins.

    Although I'm hoping to use Jenkins, in terms of a CI tool, I wont need to use most of it's features.  Our needs are pretty simple.  Run our tests on a schedule and get email notifications including the test results.  We don't actually need to continuously run tests after each build.

    We where running our unit tests through the test explorer window in visual studio until I realized we could create batch files and invoke our DLL's through vstest.sonsole.exe

    This suited our needs and was working fine to a point.  I still had to manually run the batch files, check on the results then upload the trx files onto our network.  I have since scheduled the batch files to run as a windows task throughout the day and night on a remote server running 8.1 so this takes care of having to run the batch files manually.  

    What I'm lacking now are the email alerts with the nice reports that I can just forward to management :)  

    I hope you get sorted.  I'm continuing to look.  Should I find anything interesting, I'll come back and post here 

    Thursday, December 18, 2014 4:54 PM
  • Hi Tina-Shi, Thanks for helping.  I have replied below.  This might provide you with more information.  Many thanks
    Thursday, December 18, 2014 5:07 PM
  • Thanks for veli’s help.

    Hi r.y.a.n,

    Thank you for posting in the MSDN forum.

    >>We where running our unit tests through the test explorer window in visual studio until I realized we could create batch files and invoke our DLL's through vstest.sonsole.exe

    This suited our needs and was working fine to a point.  I still had to manually run the batch files, check on the results then upload the trx files onto our network.  I have since scheduled the batch files to run as a windows task throughout the day and night on a remote server running 8.1 so this takes care of having to run the batch files manually.  

    So now you have resolved one issue about running the test, am I right?

    >> What I'm lacking now are the email alerts with the nice reports that I can just forward to management :)  

    For the new issue, do you mean that you want to send the trx file with emanil or others?

    Actually you know that one thread for one issue, if possible, I suggest you post a new thread for it, so we could discuss the new issue conveniently, and this will make answer searching in the forum easier and be beneficial to other community members as well.

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Friday, December 19, 2014 6:43 AM