locked
Command Line Tools does not attach to the process which is to be profiled and therefore does not return code coverage results for manual tests RRS feed

  • Question

  • I am using VS2010 Command Line Profiling tools for code coverage for manual tests. Following are the steps i am following for our Desktop application developed using winforms and WPF:-

    1)Open the Visual Studio Command Prompt

    Start ->All Programs ->Microsoft Visual Studio 2010 -> Visual Studio Tools -> Visual Studio Command Prompt(2010)

    2)Change the Directory to :-

    C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools

    3)Instrument the required DLLS :

    C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools > VSInstr.exe /coverage “C:\BinRel_original5021 – Copy\ AccessManagerO+.dll”

    4)Start the profiling of the application for collecting performance or code data :-

    C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools >VSPerfCmd.exe /Start:Coverage /Output:<OutputFilename and Location>

    5)  Carry out the manual tests on your application.

    6)When you are done with your testing cycle ,for the code coverage results to be generated close the application windows and then shutdown the performance monitor

     C:\Program Files (x86)\Microsoft Visual Studio 10.0\Team Tools\Performance Tools >VSPerfCmd.exe /shutdown

    When I run the command - vsperfcmd /status i see ZERO as number of active process  and thats why no results are returned. This behavior is inconsistent and  i am able to get results when the process is recognized.

    Monday, September 7, 2015 2:33 PM

Answers

  • Hi,

    The following steps worked for me but please note these are specific  to my application and steps might differ for some.

    My application is a Windows Desktop application developed in WPF and Winforms.

    CODE COVERAGE FOR MANUAL TESTS USING VISUAL STUDIO    COMMAND LINE TOOLS

    Important Pre Requisites :-

    1)Make sure your application exe location has all the required DLLS and their corresponding PDB’S  OR

    2)As in case of my application I copied all the dlls and pdbs in a different folder on my test machine,then configured my application using these dlls and accessed this configured instance for the code coverage cycle.

    Note:- Debug Files/PDB files for each DLL. The pdb files are necessary to get the coverage data. Code Coverage Report will not be generated if the path for the PDB(Program Debug Database) files is not given correctly or the PDB files do not exist at all

                                            -: Profiling Steps :-

    Note:- If the results are not generated by following the mentioned steps you can login into the test machine with Console login and give it a try.

    Eg- mstsc /v:xx.xx.x.xxx /console

    mstsc / v:xx.xx.x.xxx /admin

    a)The number is the IP address of the machine

    b)Type the whole string in Windows search or Run

    ==================================================================================

    1)Open the Visual Studio Command Prompt

    Start ->All Programs ->Microsoft Visual Studio 2010 -> Visual Studio Tools -> Visual Studio Command Prompt(2010)

    For Visual Studio 2012/13/15 - Start ->All Programs ->Microsoft Visual Studio<Version> -> Visual Studio Tools -> Visual Studio X86 Native tools prompt

    2)Run the following command to activate GLOBALTRACING on the machine

    VsPerfCLREnv  /globaltraceon

    Note- You will have to restart your machine after executing this command. It is mandatory. This is a one time activity and no need to execute the command and restart the machine every time a code coverage cycle is initiated.

    3)Change the Directory to the folder  which contains the PDBS and DLLS copied onto the hard drive of your test machine :-

     Eg – cd C:\DLLFOLDER

     cd:- change directory command in DOS

    4)Instrument the required DLLS :-

    Command for Instrumentation :-

    Eg -C:\ DLLFOLDER >VSINSTR /coverage “<YourDLL name goes here inside the quotes>

    -Make sure the application under test is closed before you commence instrumenting

    -You will see a message  when your file/dll has been successfully instrumented

    5)Start the Coverage Monitor and specify the location and name of the output file which will store the coverage results :

    Eg:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:OutputFilename and Location” /CS /user:”Everyone” OR :NetworkService”

    Eg 2:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:OutputLog.coverage” /CS /user:”Everyone” OR :NetworkService”

    Eg 3:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:“C:\CoverageLogs\OutputLog.coverage” /CS /user:”Everyone” OR :NetworkService”

    Note:- No need to explicitly create a file and then execute the command. Executing the above command will create a file with the mentioned name.

    5)  Carry out the manual tests on your Application.

    a)Open the TESTAPP and navigate to your configured  app . Carry out your tests which touch your dll.

    b)Midway through your cycle you can execute the following command to verify if the profiler is in fact covering your application –

    Vsperfcmd  /status

    c)If the Number of Active processes is 1 or >1 ,This is an positive indication that the VSProfiler has recognized your application process and the code coverage results file will be populated with the expected results.

    6)Stop the Profiling and View the Results:-

    a)When you are done with your testing cycle  close the application windows and then shutdown the performance monitor.

    Command - Vsperfcmd /shutdown

    b)Navigate to the Output File Location. Right click the coverage file and open with Visual Studio

                                 Some Observations /Tips / Things to Analyze :-

    Some of the teams have notified that code coverage result  is not returned for some of the impacted/instrumented DLLS.

    Please note that VS command line tool will list only those DLLS in the result file which are touched upon during the testing cycle.

    The following points can be verified as part of analyzing the NO results scenario :

    1)Whether the code/dll  has really been touched upon by our manual tests?

    2)Whether  the code/dll  can be touched upon through manual tests?

    • Marked as answer by HarshadP Tuesday, December 1, 2015 11:05 AM
    Tuesday, December 1, 2015 11:05 AM

All replies

  • Hi HarshadP,

    This document shared us a Winform sample and detailed steps about how to collect code coverage using the profiler tool:

    http://www.codeproject.com/Articles/590346/CodeplusCoverageplusforplusRegressionplusTestsplus

    If possible, you could test it, so we could know that whether it is related to the command line step or others.

    Thanks,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Wednesday, September 9, 2015 6:41 AM
  • Hi Jack,

    Thanks for the reply.

    I had gone through this link when i was researching on the code coverage. I am doing all the steps mentioned in the article. As i mentioned  the steps which i listed above were working for me .

    Is this something related to our application?

    One observation is i was able to get the results but as soon as we had a build upgrade i was back to square one. No results .

    Let me know if you need any specific information on our application.T

    Thanks

    Harshad

    Friday, September 11, 2015 11:30 AM
  • Hi Harshad,

    >>One observation is i was able to get the results but as soon as we had a build upgrade i was back to square one. No results .

    What does it mean "as we had a build upgrade"?

    To make sure that it is not the project's issue, how about testing a simple Winform app like the sample in my previous reply?

    Please run your command line as the admin, in addition, you don't have to cd the folder of the profiler tool, you could change directories to the directory containing windows application exe, and then run the command lines again.

    https://msdn.microsoft.com/en-us/library/dd299398%28v=vs.90%29.aspx?f=255&MSPPError=-2147217396

    In addition, maybe you could build your app with X86 target, and then test it again in command line.

    Maybe it is related to the VS version you were using if all projects have the same issue.

    https://social.msdn.microsoft.com/Forums/en-US/f27a8345-5fe5-4275-ba13-50a6a78950f1/vs2008-stand-alone-profiler-but-it-does-not-support-intrumenting-64-bit-binaries-with-coverage?forum=vstsprofiler

    Best Regards,

    Jack 


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Monday, September 14, 2015 8:53 AM
  • Revisited a lot of articles and some modifications in the steps which i was following worked. I am now able to get results.

    Thanks.

    -Harshad

    Monday, November 30, 2015 9:53 AM
  • Hi Harshad,

    Glad to know that you have resolved this issue, if so, could you share us the detailed solution here? So it would be helpful for other members who get the same issue as yours.

    If possible, you could share the solution in a new reply and mark a helpful reply as the answer, so other members could research and find the answer directly, and I could close this case.

    Thank you for your support and co-operation. Harshad:)

    Have a nice day,

    Jack


    We are trying to better understand customer views on social support experience, so your participation in this interview project would be greatly appreciated if you have time. Thanks for helping make community forums a great place.
    Click HERE to participate the survey.

    Tuesday, December 1, 2015 2:53 AM
  • Hi,

    The following steps worked for me but please note these are specific  to my application and steps might differ for some.

    My application is a Windows Desktop application developed in WPF and Winforms.

    CODE COVERAGE FOR MANUAL TESTS USING VISUAL STUDIO    COMMAND LINE TOOLS

    Important Pre Requisites :-

    1)Make sure your application exe location has all the required DLLS and their corresponding PDB’S  OR

    2)As in case of my application I copied all the dlls and pdbs in a different folder on my test machine,then configured my application using these dlls and accessed this configured instance for the code coverage cycle.

    Note:- Debug Files/PDB files for each DLL. The pdb files are necessary to get the coverage data. Code Coverage Report will not be generated if the path for the PDB(Program Debug Database) files is not given correctly or the PDB files do not exist at all

                                            -: Profiling Steps :-

    Note:- If the results are not generated by following the mentioned steps you can login into the test machine with Console login and give it a try.

    Eg- mstsc /v:xx.xx.x.xxx /console

    mstsc / v:xx.xx.x.xxx /admin

    a)The number is the IP address of the machine

    b)Type the whole string in Windows search or Run

    ==================================================================================

    1)Open the Visual Studio Command Prompt

    Start ->All Programs ->Microsoft Visual Studio 2010 -> Visual Studio Tools -> Visual Studio Command Prompt(2010)

    For Visual Studio 2012/13/15 - Start ->All Programs ->Microsoft Visual Studio<Version> -> Visual Studio Tools -> Visual Studio X86 Native tools prompt

    2)Run the following command to activate GLOBALTRACING on the machine

    VsPerfCLREnv  /globaltraceon

    Note- You will have to restart your machine after executing this command. It is mandatory. This is a one time activity and no need to execute the command and restart the machine every time a code coverage cycle is initiated.

    3)Change the Directory to the folder  which contains the PDBS and DLLS copied onto the hard drive of your test machine :-

     Eg – cd C:\DLLFOLDER

     cd:- change directory command in DOS

    4)Instrument the required DLLS :-

    Command for Instrumentation :-

    Eg -C:\ DLLFOLDER >VSINSTR /coverage “<YourDLL name goes here inside the quotes>

    -Make sure the application under test is closed before you commence instrumenting

    -You will see a message  when your file/dll has been successfully instrumented

    5)Start the Coverage Monitor and specify the location and name of the output file which will store the coverage results :

    Eg:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:OutputFilename and Location” /CS /user:”Everyone” OR :NetworkService”

    Eg 2:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:OutputLog.coverage” /CS /user:”Everyone” OR :NetworkService”

    Eg 3:-C:\ DLLFOLDER > vsperfcmd /start:coverage /output:“C:\CoverageLogs\OutputLog.coverage” /CS /user:”Everyone” OR :NetworkService”

    Note:- No need to explicitly create a file and then execute the command. Executing the above command will create a file with the mentioned name.

    5)  Carry out the manual tests on your Application.

    a)Open the TESTAPP and navigate to your configured  app . Carry out your tests which touch your dll.

    b)Midway through your cycle you can execute the following command to verify if the profiler is in fact covering your application –

    Vsperfcmd  /status

    c)If the Number of Active processes is 1 or >1 ,This is an positive indication that the VSProfiler has recognized your application process and the code coverage results file will be populated with the expected results.

    6)Stop the Profiling and View the Results:-

    a)When you are done with your testing cycle  close the application windows and then shutdown the performance monitor.

    Command - Vsperfcmd /shutdown

    b)Navigate to the Output File Location. Right click the coverage file and open with Visual Studio

                                 Some Observations /Tips / Things to Analyze :-

    Some of the teams have notified that code coverage result  is not returned for some of the impacted/instrumented DLLS.

    Please note that VS command line tool will list only those DLLS in the result file which are touched upon during the testing cycle.

    The following points can be verified as part of analyzing the NO results scenario :

    1)Whether the code/dll  has really been touched upon by our manual tests?

    2)Whether  the code/dll  can be touched upon through manual tests?

    • Marked as answer by HarshadP Tuesday, December 1, 2015 11:05 AM
    Tuesday, December 1, 2015 11:05 AM