Code coverage for C++: thousands of "Global Functions" in coverage results

Question

I'm working on a project in C++ for which we have unit test harnesses written using CppUnit. The unit test harnesses are linked with /profile, and an automated script automatically runs vsinstr on them, starts the profiler, runs the tests, and shuts down the profiler.

 

The .coverage file that results has a line item for each unit test harness named "Global Functions" that includes thousands of functions from the CRT library. Can someone tell me if there any way to exclude these CRT library functions from code coverage results?

 

 

Thanks,

Steve

Answer 1

Moving to code coverage forum

Answer 2

Moved to Profiler.

 

Hi Steve

 

When you view the code coverage report in Visual Studio, you can use Filter to filter some unwanted function. For more information, see Profiler Report View Filter. But I don't see a easy way to filter all CLR functions. 

Answer 3

The filtering mentioned by Bill only works for performance profiles (.vsp).  This does not work for coverage.  Also, the performance filter does not let you filter based on function name.

 

The only way, currently, to filter functions from coverage is to not instrument them in the first place.  Check out vsinstr's /exclude command-line option.  You can /exclude functions that you don't want to show up in your coverage numbers.  * is treated as a wildcard.

 

Regards,

Chris

Answer 4

I have used the /EXCLUDE command line option, but in order to be effective it needs to refer to a namespace. When I ask about excluding global functions, I mean to ask if its possible to exclude the global namespace en-masse or if the CRT functions have some property that that allow them to be excluded as a whole. As far as I can tell, there is no naming pattern common to all CRT functions, so I don't see how wildcard matches will help.

Also, can someone please move this back to the correct forum (where I posted it originally) that deals with code coverage/unit test questions. Thanks.

Answer 5

Unfortunately, the only filtering we currently have is name-based.  You can specify /exclude multiple times, though, so you could specifically exclude each global CRT function.  For example, you could create a response file (call it foo.rsp) with the following contents:

 

/exclude:CRTGlobalFunc1

/exclude:CRTGlobalFunc2

...

/exclude:CRTGlobalFuncN

mybinary.exe

 

Then, you could invoke vsinstr as such:

 

vsinstr @foo.rsp

 

Taking a step back, are you statically linking the CRT?  If you dynamically link it instead, the global CRT functions should be in a separate, non-instrumented DLL and then you shouldn't get coverage numbers for them.  Is this the case?

 

This is actually the correct forum for this issue.  The instrumentation/collection technology behind code coverage shares a lot with the profiler, so we handle support issues for both.

 

Regards,

Chris

Answer 6

Well, I can look into get a listing of all the CRT functions and passing them via a response file. There are about 11,000 of them. Does vsinstr have any kind of limit on the number of functions it can exclude?

As for linking with the CRT, yes, we are statically linking to it. This is how the production binaries will ship. As much as I dislike having the code coverage results polluted with CRT functions, I'm not willing to run all our tests under different conditions simply to address this problem.

Ultimately, the real issue is that VS2008 just doesn't have a rich enough interface for processing code coverage results or for exporting them to an application that could do so (such as Excel).

Answer 7

Command-lines have a limit on the number of characters they can handle, but if you use response files, you can get around that.  vsinstr has no known limit on the number of functions you can exclude.

 

I agree, code coverage filtering from the UI is lacking and needed.  You can export the coverage results to an XML-formatted file for post-processing, however.  To avoid skew in the coverage numbers, though, I would recommend doing the instrumentation-based exclusions.

Answer 8

Steve,

I've marked Chris's response here as the answer so that we can keep better track of open issues on the forum since it looks like it did answer your questions. If this isn't the case feel free to un-mark it as the answer and add the details about what issues you're facing with this. Or if you like you can open a completely new thread and let us know the details of the issue you're facing and we'll do our best to help you out.

 

Thanks,

Tim

 

Answer 9

Is there any way to exclude MFC classes in code coverage?

---

मनोज कुमार जांगिड

Answer 10

Yes we can exclude MFC classes. we have to create a .rsp file and mansion all MFC  those you want to exclude.

like:

/coverage

/exclude (MFC)filename

bijaybinary.exe

Thanks