locked
Multiple WebDev servers for Integration Testing (VS2010 & MSTest) RRS feed

  • Question

  • I have a set of Integration tests to be run on a Remote Test Agent that require several web services to be running. These services are not being tested directly, the code under test connects to them through a proxy.  From the test's perspective they are simply an environmental dependency.

    Here is what I've tried so far:

    1) AspNetDevelopmentServerAttribute: Because the port number cannot be specified, for my tests this would mean runtime deployment of a configuration file for the Proxy – definitely not something to depend on.  If the AspNetDevelopmentAttribute always used the same port, this might be the preferred solution, minus the fact that every single test needs this decoration.  I could find no documentation that clearly stated if controlling the Port was supported.

    2) Launching WebDev from Code: I created helper classes that launched Webdev.Webserver.exe as a Diagnostics.Process. Although I was able to get this working, this solution is limited to the ClassInitializeAttribute – which forces me to create/destroy the services for each unit test class (of which there are many).  Moreover, I definitely get the sense that I’m reinventing code that already exists and adding unneeded complexity to my test setup/teardown.

    3) Testsettings Deployment Script: Using the Deployment section to copy my published WebService bits to the Test Deployment Directory, I created a Setup Script (.bat) as follows:

    start Webdev.Webserver.exe [args for service]
    
    start Webdev.Webserver.exe [args for service]

     

    When executing the tests, each service is successfully launched, but the Deployment fails saying it timedout waiting for the Setup script to complete. This is quite strange considering the batch file executes fully when invoked manually.  I would really like to know what causes this, as I’m getting the impression that Test Deployment in VS is quite brittle.

     

    Questions:

    What might be causing the script timeout?

    Are there other deployment options that I’m missing? 

    What is recommended for Integration testing with web-services on Remote Agents?

     

    Thanks!

    -John

     




    Tuesday, March 29, 2011 2:45 AM

Answers

  • After opening a ticket with MS Support, my timeout issue was discovered to be a bug. Specifically, launching processes from a batch file on an agent.

    We were given a patched dll and it has resolved the issue. The fix will soon be made generally available according to MS.


    • Marked as answer by J. Zumwalt Thursday, June 30, 2011 9:00 PM
    Thursday, June 30, 2011 9:00 PM

All replies

  • Tuesday, March 29, 2011 7:26 AM
  • AnujChaudhary,

    I tried both tips in that thread:

    - Adding the Timeouts node to the testsettings xml resulted in a warning message, seems that maybe this is no longer a valid node in vs2010...  No matter, I doubt increasing the timeout would fix this problem.

    - Redirecting standard input seemed to have no effect either.  The batch file still hangs and causes a timeout.

     

    In attempting these workarounds, I did notice something.

    If launching the server fails (because it is already running), a dialog is displayed.  If I dismiss this dialog on the Remote machine, the test will continue and complete.  So it seems that it's the actual execution of the Webdeb.WebServer.exe that is holding up the script.

    Tuesday, March 29, 2011 2:28 PM
  • Hi John,

    What kind ot dialog is it, is there a way to dismiss it forever.

    Thanks,

    Anuj

    Wednesday, March 30, 2011 4:54 PM
  • Sorry, I should have explained a bit more...

    It's simply the Usage dialog displayed by WebDev.WebServer.exe when the command-line arguments are incorrect or there was an error hosting the service.

    The dialog could be avoided by checking if the service is already running before launching. 

    At this point, I'm not concerned about the dialog... I mentioned it because the deployment continues, if the Webdev.WebServer.exe does not actually stay open.  Dismissing the dialog closes the WebDev process.


    Wednesday, March 30, 2011 5:21 PM
  • I found this on stackoverflow:

    http://stackoverflow.com/questions/4455334/visual-studio-test-settings-setup-script-doesnt-stop-executing

    Looks to be the exact problem I'm seeing.

     

     

    A deployment script starting multiple WebDev servers should be supported by VS2010, correct?

     

     



    Wednesday, April 6, 2011 2:20 PM
  • Yes, you can write scripts and call them is 'Setup and Cleanup scripts' in the testsettings

    Thanks,

    Anuj

    Wednesday, April 6, 2011 4:18 PM
  • I'm definitely aware that scripts can be called during deployment.  The 3rd point in my original post mentions this fact.

     

    My question is simply...

     

    Is the timeout that I am seeing a bug?  Can you reproduce the same behavior with a similar script?


    Wednesday, April 6, 2011 6:36 PM
  • After opening a ticket with MS Support, my timeout issue was discovered to be a bug. Specifically, launching processes from a batch file on an agent.

    We were given a patched dll and it has resolved the issue. The fix will soon be made generally available according to MS.


    • Marked as answer by J. Zumwalt Thursday, June 30, 2011 9:00 PM
    Thursday, June 30, 2011 9:00 PM