locked
How can I get covered(% block) information from a coverage file without using visual studio screen RRS feed

  • Question

  • I have Visual Studio 2017 Enterprise in another account. When I run the tests from console by running following command in Visual Studio 2017 Enterprise Command Prompt vstest.console.exe Fleck.Tests.dll, I get following message:


        Microsoft (R) Test Execution Command Line Tool Version 15.3.0-preview-20170628-02

        Copyright (c) Microsoft Corporation. All rights reserved.

        ....

        Total tests: 1. Passed: 1. Failed: 0. Skipped: 0.

        Test Run Successful.

        Test execution time: 2.6671 Seconds


    Here is the command I used.

        Using Command : vstest.console.exe Fleck.Tests.dll /logger:trx;LogFileName=coverage.xml

    There is no information about code coverage. How can I get code coverage output on vstest console?




    • Edited by papacamping Friday, January 25, 2019 4:55 AM
    Thursday, January 24, 2019 2:57 AM

All replies

  • Hi friend,

    We can use command like this: vstest.console.exe Fleck.Tests.dll /logger:trx  /Enablecodecoverage to get the code coverage output. (/Enablecodecoverage: Enables CodeCoverage in the test run)

    After that we can find the code coverage file path in the attachments like below:


    In addition: More details about vstest.console.exe options can be found here: VSTest.Console.exe command-line options

    Hope all above can help solve this issue. Looking forward to your reply!

    Best Regards

    Lance


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Thursday, January 24, 2019 5:52 AM
  • Hi Lance

    First of all thank you for your feedback. But we have not solved the problem yet.

    I have created the .Coverage file(file name : Administrator_2019-01-25 11_41_30.coverage) by executing the command as shown below.

     -use command : vstest.console.exe TestModule.dll /logger:trx /Enablecodecoverage

    I can see covered(% block) in the visual studio screen by double clicking the coverage file.
    However, I can not find covered(% block) information when I open a coverage file with a text editor.

    Question1 - How can I get covered(% block) information from a .coverage file without using visual studio screen.

    Question2 - Can I get covered(%block) information by converting the coverage file to another file?

    Best Regards

    papacamping

    Friday, January 25, 2019 4:26 AM
  • Hi papacamping,

    For question1, without visual studio screen, I’m afraid we can’t open the coverage file directly. But good news is we can convert the .coverage file to another file by some tools. And that’s also the answer for question2.

    As I know, we can use code coverage tools to convert it to xml format. Also, some third-party tools can convert it to html or other formats.

    1.For code coverage tools:  As you have the vs2017, we can find the tool in “C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools”.

    Step1: Open cmd, cd the folder where exists this tool. As for vs2017: enter the command(If you haven't changed the path): cd C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\Team Tools\Dynamic Code Coverage Tools

    Step2: Enter the command : codecoverage analyze /output:X\XXX\result.xml xxx\xxx.coverage  ,after that, we can find the result.xml file in the X\XXX folder and we can open it without vs ide.

    Note: Please use absolute path and avoid to output the result.xml in a C: root directory.

    Screenshot:

     2.Also, if you want to convert the coverage file to other formats: html or what. You can see ReportGenerator.

    Hope all above can help, looking forward to your reply!

    Best Regards

    Lance


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, January 25, 2019 5:13 AM
  • I did as you said, but the result does not match as you can see below image.

    command : 

        1. codecoverage analyze /output: xxx\xxx\result.xml xxx\xxx\xxx.coverage

        2. ReportGenerator.exe -reports: xxx\xxx\result.xml -targetdir:xxx\result

        3. index.html <- In the below, right is the result

    Question - Why are the results not the same? Is this a reliable result?

    Best Regards

    papacamping


    • Edited by papacamping Friday, January 25, 2019 6:14 AM
    Friday, January 25, 2019 6:09 AM
  • Hi friend,

    It seems the result from third-tool is not accurate in this situation in large project. , I feel sorry for this not perfect suggestion.

    And what about the result of code coverage tool? Since code coverage tool use the same parser as the IDE, the result from this tool can be more accurate than third-tool. So if the results of result.xml can match the results from IDE, I recommend you use code coverage tool to generate reports.

    And if same issue occurs between result.xml and .coverage, could you share a simple sample which have same issue by one-drive, so that we can check for you.

    Best Regards

    Lance


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Friday, January 25, 2019 6:34 AM
  • Hi Lance,
    There is a still a difference between result.xml and .coverage.

    Due to internal regulations of a company, it cannot disclose source code.
    So I uploaded only two files, result.xml and .coverage to my one-drive.

    my one-drive : https://1drv.ms/f/s!Ap2lynI0A4NUaLAF66EjQEvPiZw

    Question1 - Would you check these two files?

    Question2 - There is no information about "Total covered(% block)" in result.xml. Like below.
    How can I get that information?

    

    Best Regards

    papacamping


    • Edited by papacamping Monday, January 28, 2019 7:02 AM
    Monday, January 28, 2019 6:36 AM
  • Hi papacamping,

    Yes, I reproduce the same result and it seems that the covered blocks by xml are more than that in VS IDE. That’s really puzzled, and I’m not sure if such situation results from optimization in VS or what.

    Also, it seems no valid documents which explain the relationship between codecoverage.exe and VS IDE. I’m sorry for this inconvenience.

    So I’m afraid the codecoverage.exe can’t match your needs though I think VS ide parse the coverage file by calling codecoverage.exe. If possible, could you vote for this post, more comments and scores can make it easier to be watched by product team.

    Thank you for your understanding. And please let us know if there has any update.

    Best regards

    Lance


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Monday, January 28, 2019 11:13 AM
  • Hi Lance

    thank you for your enthusiastic support.

    I can't find the Total Covered (% Blocks) information (Red rectangle in the image file) in the xml file. Is there any other way?

    Does codecoverage.exe not support this information?

    How can I resolve the current issue with visual studio technical support?

    Best Regards

    papacamping


    • Edited by papacamping Tuesday, January 29, 2019 1:14 AM
    Tuesday, January 29, 2019 1:12 AM
  • Hi papacamping,

    Currently, the xml file converted by codecoverage.exe seems no option for the total covered information.

    So I’m afraid we have to read the xml data by coding.

    Here is a simple sample:

    In vs2017, create a new console app(.net framework) project. And copy the code below:

    class Program
        {
            public static double NumBlockCovered=0;
            public static double NumBlockNotCovered=0;
            static void Main(string[] args)
            {
                Console.WriteLine("Please enter the path where xml exists:");
                ReadParseXml(Console.ReadLine());
            }
            public static void ReadParseXml(string path)
            {
                XmlDocument xmlDoc = new XmlDocument();
                xmlDoc.Load(path);
                //xmlDoc.Load(@"D:\Result.xml");
                //search for <results>
                XmlNode root = xmlDoc.SelectSingleNode("results");
                XmlNodeList nodeList = root.ChildNodes;
                //get the <modules> node
                foreach (XmlNode modules in nodeList)
                {
                    XmlNodeList modulesList=modules.ChildNodes;
                    foreach (var module in modulesList)
                    {
                        XmlElement xe=(XmlElement)module;
                        NumBlockCovered+=Convert.ToDouble(xe.GetAttribute("blocks_covered"));
                        NumBlockNotCovered+=Convert.ToDouble(xe.GetAttribute("blocks_not_covered"));
                    }
                }
                Console.WriteLine("block_covered:"+NumBlockCovered);
                Console.WriteLine("block_not_covered:"+NumBlockNotCovered);
                double CoveredRate=NumBlockCovered/(NumBlockCovered+NumBlockNotCovered);
                double NotCoveredRate=NumBlockNotCovered/(NumBlockCovered+NumBlockNotCovered);
                Console.WriteLine("block_covered rate:" + CoveredRate*100+"%");
                Console.WriteLine("block_not_covered rate:"+NotCoveredRate*100+"%");
                Console.ReadLine();
            }
        }

    After that, build the solution and get the .exe file from the debug or release folder.

    Every time if we want to get the total information, open the .exe and enter the xml path, then the information will display.

    Hope it helps solve the issue.  And sorry for this inconvenience.

    Best Regards

    Lance


    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue, and to click "Unmark as Answer" if not. This can be beneficial to other community members reading this thread. If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.

    Tuesday, January 29, 2019 5:37 AM
  • Hey all.

    I had this same problem and I opened a MS service ticket with them to provide an explanation:

    Finally got an answer from Microsoft after much back and forth:

    "We analyzed it further, the difference in block count in VS IDE vs that of XML is because while computing certain attributes for a block like it's start line, start column, etc.. We fail to find this data from the symbols, or the data is simply invalid.

    Note assigning this additional information to block happens after we have computed blocks based on instructions, but since some of these instructions could be compiler generated, they don't have a valid code point. Hence the block information for such blocks is empty, but the block_count value is preserved. 

    When VS reads this .coverage file it ignores such block, whereas when codecoverage.exe parses them, it simply reads the value of block_count, & puts it in xml file.

    That’s why the Code Coverage Report does not match."

    I found another tool that does the conversion correctly and it is very simple.  Have a look here https://github.com/danielpalme/ReportGenerator/wiki/Visual-Studio-Coverage-Tools

    Ref: https://social.msdn.microsoft.com/Forums/vstudio/en-US/6abb60ad-55eb-4be5-865f-78490e6ba18c/code-coverage-report-does-not-match-exported-xml-report?forum=vsunittest#af3d51be-57de-49a4-a88e-add47dbd7eca



    -7

    Friday, May 10, 2019 11:36 PM