locked
How to enable windows store app unit test execution mode RRS feed

  • Question

  • I created a Windows Phone 8.1 project using template Installed->Store Apps->Windows Phone Apps->DLL (Windows Phone), a Windows Phone 8.1 Unit Test app using template Installed->Store Apps->Windows Phone Apps->Unit Test App(Windows Phone).

    I built my projects for Release/x86. I could run my unit tests using the Test Manager inside Visual Studio 2013.  But when I ran the unit tests using the following command line I got warnings:

    C:\Release\WindowsPhone_Test>VSTest.Console WindowsPhone_Test.dll 
    Microsoft (R) Test Execution Command Line Tool Version 12.0.30723.0
    Copyright (c) Microsoft Corporation.  All rights reserved.

    Starting test execution, please wait...
    Warning: Test container 'C:\Release\WindowsPhone_Test\WindowsPhone_Test.dll' skipped. The tests in this test container can be executed only in windows store app unit test execution mode

    Warning: No test is available in WindowsPhone_Test.dll. Make sure that installed test discoverers & executors, platform & framework version settings are appropriate and try again.

    Question: How do I enable the windows store app unit test execution mode? And what do I do to make sure that the platform and framework version settings are appropriate?

    Thanks for your help,

    Jenny


    Thursday, October 2, 2014 7:06 PM

Answers

  • Hi Jack,

    I have submitted a connect report as you suggested, but I am not sure I created it with the right category.  I didn't find a category for unit tests.

    Connect Report

    What's the next step for me to take at this point?

    Thanks,

    Jenny



    Monday, November 3, 2014 7:33 PM

All replies

  • Hi Jenny,

    To run the windows phone unit test, we need to provide the name of XAP file.

    Is there the XAP file in the Debug or Release folder?

    If that is the case, please provide the XAP file to run test.

    For example:

    vstest.console D:\Projects\BankAccountTest\Bin\x86\Debug\BankAccountTest_Debug_x86.xap

    More information, please refer to:

    # Additional features for unit testing

    http://msdn.microsoft.com/en-us/library/windows/apps/dn168930(v=vs.105).aspx#BKMK_more

    Best Regards

    Starain Chen


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Friday, October 3, 2014 7:32 AM
  • Thanks for the quick reply.  

    My solution contains three projects, which are all in C++. 

    The first project, MyPhoneApp , is a VC++ dll created using template Installed->Visual C++ ->Store Apps->Windows Phone Apps->DLL (Windows Phone).  This project is converted from a desktop WPF client dll.

    The second project, MyPhoneApp_CPPUnitTest,  is also a VC++ dll created using template Installed->Visual C++ ->Store Apps->Windows Phone Apps->DLL (Windows Phone).  This project contains all of the CPPUnit tests that were converted from CPPUnit tests that were written for a WPF client.

    The third project, MyPhoneAPP_VSUnittest, is created using template Installed->Visual C++->Store Apps->Windows Phone Apps->Unit Test App(Windows Phone).  This test project turned out to be a dll too.  This project contains only one Visual Studio test method, which is to call a function in MyPhoneApp_CPPUnitTest, which runs all the CPPUnit unit tests in MyPhoneApp_CPPUnitTest:

    TEST_METHOD(LaunchAllCPPUnitSuites)
    {
        bool wasSucessful = runAllTestSuites();
        Assert::IsTrue(wasSucessful);
    }

    runAllTestSuites() is defined inside the MyPhoneApp_CPPUnitTest dll.

    It seems that to create a .xap file you need to have a Test App in C#, so I didn't find a way to generate a .xap file for my C++ MyPhoneAPP_VSUnittest project.

    I could run the the Visual Studio unit tests using the Test Explorer from within Visual Studio 2013 fine, but the test results the cppunit library's runner() function spit to the stardard out doesn't get displayed to the Output Window, so I have no way to know which of the CPPUnit unit tests have failed.

    Is there a way to make the VS Test Explorer to display in the Output window what test functions have written to the standard out?

    Is there a way for the VSTest.Console.exe to run the VS tests in MyPhoneAPP_VSUnittest, which is a VC++ Unit Test APP for Window Phone?

    Thanks for your help.



    • Edited by Jenny Duan Monday, November 3, 2014 8:56 PM Corrected typo
    Friday, October 3, 2014 4:44 PM
  • Hi Jenny,

    >> It seems that to create a .xap file you need to have a Test App in C#, so I didn't find a way to generate a .xap file for my C++ MyPhoneAPP_VSUnittest project.

    Actually we also need to create the app package (.appx file) for VC++ unit test library.

    After tests are authored and ready to run from command line create an app package (.appx file) as follows:

    (1)Right click on the project in Solution Explorer.

    (2)Go to Store -> Create App Package...

    (3)Select "Create a package to use locally only" and click Next

    (4)Set desired Package location and Build configuration

    (5)Click Create

    (6)Click OK

    Screen shot:

    Reference:

    Running Unit Tests for Windows Metro style apps from Command Line

    Walkthrough: Creating and Running Unit Tests for Windows Store Apps

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, October 6, 2014 7:10 AM
  • Jack,

    Thanks for your reply.  

    I created an .appx file for my unit test project followng your instructions with the following options:

    Do you want to build packages to upload to the Windows Phone STore? No

    Generate app bundle: Never

    Architecture and Configuration: ARM / Relase (ARM)

    When I ran the tests on the Windows 8.1 laptop, I got the following error:

    vstest.console "C:\Visual Studio Solutions\AppPackages\WindowsPhone_Test\WindowsPhone_Test_1.0.0.1_ARM_Test\WindowsPhone_Test_1.0.0.1_ARM.appx" /InIsolation 

    Starting test execution, please wait...
    Error: App package 'C:\Visual Studio Solutions\AppPackages\WindowsPhone_Test\WindowsPhone_Test_1.0.0.1_ARM_Test\WindowsPhone_Test_1.0.0.1_ARM.appx' does not has test executor entry point. For running unit tests for Windows Store apps, create app package using Windows Store app Unit Test Library project.

    I have two questions now:

     I created my unit test project using template Installed->Visual C++->Store Apps->Windows Phone Apps->Unit Test App(Windows Phone). Is this a Windows Store App Unit Test library project as mentioned in the error message above? What do I need to do to add a test executor entry point to it?

    - Does vstest.console run the unit tests on an emulated Windows Phone or do I have to run the Windows Phone unit tests on a Windows Phone device? 

    Thanks,

    Jenny


    • Edited by Jenny Duan Monday, October 6, 2014 5:44 PM Corrected typo
    Monday, October 6, 2014 5:43 PM
  • Hi Jenny,

    Sorry for that I misunderstood this issue. It seems that it is different from the windows store unit tests.

    Reference:

    (1) Run Windows Phone unit tests in command line:

    http://msdn.microsoft.com/en-us/library/windows/apps/dn168930(v=vs.105).aspx

    http://blogs.msdn.com/b/visualstudioalm/archive/2013/01/31/windows-phone-unit-tests-in-visual-studio-2012-update-2.aspx

    http://developer.nokia.com/community/wiki/How_to_create_unit_tests_for_Windows_Phone_with_the_Visual_Studio_Unit_Testing_Framework

    (2) Run Windows Store unit test in command line.http://blogs.msdn.com/b/vikramagrawal/archive/2012/05/06/running-unit-tests-for-windows-metro-style-apps-from-command-line.aspx

    So it seems that it still has certain differences between the Windows store unit tests and Windows Phone unit tests.

    >>Does vstest.console run the unit tests on an emulated Windows Phone or do I have to run the Windows Phone unit tests on a Windows Phone device? 

    To run our test function deploys the test project to the phone or the emulator and runs it.

    Since my test machines Environment meet an issue now, I just provide the sample links which shared how to run the C# windows phone unit tests using VS IDE or the command line, I will check the VC++ test ptoject and update it later.

    If possible, please run a C# windows phone unit test, check the result, before you run it in command line, please make sure that you could run it in Test Explorer windows.

    Sincerely.

    Jack 


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, October 7, 2014 6:54 AM
  • Hi Jack,

    Thanks for your info.  I'll be on vacation starting tomorrow so I'll be trying what you suggestged in your last reply after I come back from vacation.  

    A colleage of mine might try to pick up from where I left off before I come back though.

    Jenny

    Tuesday, October 7, 2014 9:31 PM
  • Hi Jenny,

    I think I found more information, maybe it could help you even if I didn’t really repro this issue in my VS2013 Environment.

    I found that it is different between VS2012 and VS2013 for windows Phone unit tests.

    In VS2012:

    I could run the test in command line with .xap file like this document:

    http://blogs.msdn.com/b/visualstudioalm/archive/2013/01/31/windows-phone-unit-tests-in-visual-studio-2012-update-2.aspx

    In VS2013, it seems that it doesn’t have the .xap file in the bin\debug folder, so it seems that it would require the app package like the windows store app, and if we create a windows unit tests, we would find that they have the same files or the structures as the windows phone unit tests.

    But I also repro it in my side, it doesn’t have the specific file like the .cer file or others in the Windows Phone project folder, if I run it in my command line, I also get the same issue as yours. But if we run it with VS IDE in Test Explorer windows, it seems that it works well.

    I did some research about this issue, but I also didn’t find the latest official document about the window phone unit test of VS2013 using command line.

    Since I get the same issue as yours, to get the issue confirmed and diagnose by product team, would you please create connect report for it? You will get email notification for update.http://connect.microsoft.com/VisualStudio/feedback/CreateFeedback.aspx

    If you submit it, please share me the link here, I will help you vote it. I think we could get the latest information from the product team experts. Of course, if I get any latest useful information, I will update it here. Thanks for your understanding.

    Sincerely,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.


    Thursday, October 9, 2014 7:03 AM
  • Hi Jack,

    I am  currently on vacation in another country.  I'll do what you suggested above after I come back to work.

    Thanks,

    Jenny

    Wednesday, October 15, 2014 6:27 AM
  • Hi Jack,

    I am  currently on vacation in another country.  I'll do what you suggested above after I come back to work.

    Thanks,

    Jenny

    Hi Jenny,

    If so, we are changing the issue type to General Discussion before you come back from your holiday:) 

    If you have more time to look at the issue and provide more information, please feel free to change the issue type back to Question by opening the Options list at the top of the post editor window, and changing the type. 

    If the issue is resolved, we will appreciate it if you can share the solution so that the answer can be found and used by other community members having similar questions.

    If you submit a feedback, please also share me the connect report link here, I will help you vote it.

    Sincerely,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Wednesday, October 15, 2014 7:18 AM
  • Hi Jack,

    I have submitted a connect report as you suggested, but I am not sure I created it with the right category.  I didn't find a category for unit tests.

    Connect Report

    What's the next step for me to take at this point?

    Thanks,

    Jenny



    Monday, November 3, 2014 7:33 PM
  • @Jenny,

    Thanks for your response. As you already submit one feedback on the Microsoft Connect site, let's focus on the case on that channel, the senior members can provide more professional support and you would get E-mail update if there is any latest information in the connect report.

    I will vote it! 

    Best Regards,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, November 4, 2014 3:20 AM
  • Hi Jack,

    This issue is blocking me and I need to have it addressed as soon as possible.  Nothing is going on with the bug report I filed on Microsoft Connect.  What do you recommend me to do to escalate it?

    Thanks,

    Jenny

    Wednesday, November 12, 2014 5:33 PM
  • Jenny I met the same issue

    you can have a try like this:

    use msbuild to generate a appex file

    then use vstest.console to run against that file

    that works to me

    • Proposed as answer by Uilque Messias Wednesday, December 9, 2015 7:27 PM
    Monday, July 27, 2015 7:59 AM
  • Jenny I met the same issue

    you can have a try like this:

    use msbuild to generate a appex file

    then use vstest.console to run against that file

    that works to me

    I did follow your suggested and it worked for me.

    I even post my test code on GitHub, feel free if someone want to take a look.

    VSTestConsoleSample

    Wednesday, December 9, 2015 7:52 PM