none
MSTest VS2008 and SmartDevice projects unreliable. RRS feed

  • Question

  • I'm trying to automate calling VS2008 unit tests from MSTest.exe onto a smart device. From within the IDE this works quite well.

    Manually I have:

    1. The device has an OS running and has ConManClient2 launched to enable connections.

    2. I have a testRunConfig file that specifies the platform and device.

    3. I launch MSTest with the testContainer and the runConfig settings.

     

    Here's the output from 3 successive calls to MSTest.exe

     

    Run 1...

    Loading ..\src\Investigations\VS2008\SmartDeviceBuildTest\SmartDeviceTestRun.testrunConfig...
    Loading SmartDeviceUnitTests.dll...
    Starting execution...

    Results               Top Level Tests
    -------               ---------------
    Passed                SmartDeviceUnitTests.TestLibraryTest.ExposedMethodTest
    1/1 test(s) Passed

    Summary
    -------
    Test Run Completed.
      Passed  1
      ---------
      Total   1


    Run 2...

    Loading ..\src\Investigations\VS2008\SmartDeviceBuildTest\SmartDeviceTestRun.tes
    trunConfig...
    Loading SmartDeviceUnitTests.dll...
    Starting execution...

    Results               Top Level Tests
    -------               ---------------
    Error                 SmartDeviceUnitTests.TestLibraryTest.ExposedMethodTest

    Final Test Results:
    Results               Top Level Tests
    -------               ---------------
    Error                 SmartDeviceUnitTests.TestLibraryTest.ExposedMethodTest
    0/1 test(s) Passed, 1 Error

    Summary
    -------
    Test Run Error.
      Error  1
      --------
      Total  1

    Run Configuration: Local SmartDevices Test Run

    Run has the following issue(s):
    Error: The remote connection to the device has been lost. Verify the device conn
    ection and restart the test.
    Details: An established connection was aborted by the software in your host mach
    ine.

    The test adapter ('Microsoft.VisualStudio.SmartDevice.TestHostAdapter.DeviceAgen
    t.DeviceHostAdapter, Microsoft.VisualStudio.SmartDevice.TestHostAdapter, Version
    =9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') required to execute
     this test could not be loaded. Check that the test adapter is installed properl
    y. An established connection was aborted by the software in your host machine.


    Run 3...

    Loading ..\src\Investigations\VS2008\SmartDeviceBuildTest\SmartDeviceTestRun.tes
    trunConfig...
    Loading SmartDeviceUnitTests.dll...
    Starting execution...

    Results               Top Level Tests
    -------               ---------------
    Error                 SmartDeviceUnitTests.TestLibraryTest.ExposedMethodTest

    Final Test Results:
    Results               Top Level Tests
    -------               ---------------
    Error                 SmartDeviceUnitTests.TestLibraryTest.ExposedMethodTest
    0/1 test(s) Passed, 1 Error

    Summary
    -------
    Test Run Error.
      Error  1
      --------
      Total  1
    ...

    Run Configuration: Local SmartDevices Test Run

    Run has the following issue(s):
    Error: A test run is currently using an instance of SmartDeviceTestHost.exe. Bec
    ause only one instance is allowed, wait for the current instance to finish, stop
     the current test run, or end the current instance of SmartDeviceTestHost.exe be
    fore starting a new test run.
    The test adapter ('Microsoft.VisualStudio.SmartDevice.TestHostAdapter.DeviceAgen
    t.DeviceHostAdapter, Microsoft.VisualStudio.SmartDevice.TestHostAdapter, Version
    =9.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a') required to execute
     this test could not be loaded. Check that the test adapter is installed properl
    y. Exception of type 'Microsoft.VisualStudio.SmartDevice.TestHostAdapter.DeviceA
    gent.TestAlreadyRunningException' was thrown.

    Is there any reason why subsequent calls fail? Is there some teardown process / steps that need to be performed first?

    Monday, February 11, 2013 12:34 PM

Answers

  • I've managed to get this working - I fixed a couple of issues in the deployment process to do with UDP sockets being left open and increasing a timeout and it made this process more reliable too - seems to be related, but can't put my finger exactly on why it should affect it.

    Thanks

    J

    • Marked as answer by John Dr Monday, February 25, 2013 4:11 PM
    Monday, February 25, 2013 4:11 PM

All replies

  • Hi John,

    I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

    Best regards,


    Amanda Zhu [MSFT]
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Tuesday, February 12, 2013 4:45 AM
    Moderator
  • Hello,

    Please check the Hosts property in the testrunconfig file. Please check the blog below:

    http://blogs.msdn.com/b/prativen/archive/2007/12/13/a-starter-on-smart-device-unit-testing.aspx

    Thanks & Regards,

    Jian-Wei Yu [MSFT]

    Microsoft Online Community Support

    --------------------------------------------------------------------------------

    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    Wednesday, February 13, 2013 1:49 AM
  • The hosts property in the testrunConfig file is correct. This seems to be an intermittent problem - if I repeatedly call MSTest.exe, if it worked the first time - at some point it will fail and I can see that the SmartDeviceTestHost.exe is running on the device. If I kill that process, and run MSTest again, sometimes it works.

    At this stage I'm not sure what why the connection is dropping out. I'm investigating further, but as far as I can see everything on the smart device has been configured correctly.

    Do you know if the running the tests from the Visual Studio uses the same functionality that MSTest does, because it seems a lot more reliable running from within Visual Studio than stand alone.

    Thanks for your time.

    Best Regards

    John

    Wednesday, February 13, 2013 8:26 AM
  • I've managed to get this working - I fixed a couple of issues in the deployment process to do with UDP sockets being left open and increasing a timeout and it made this process more reliable too - seems to be related, but can't put my finger exactly on why it should affect it.

    Thanks

    J

    • Marked as answer by John Dr Monday, February 25, 2013 4:11 PM
    Monday, February 25, 2013 4:11 PM