Code Analysis Policy

Question

I was looking to create a Code Analysis Policy for Checkin but noticed that the docs are still under development. Is there a pointer to a blog anywhere on this?

Answer 1

Specifically what sort of things are you looking for?

It's a reletively simple Checkin Policy. You must be FxCop Clean to successfully checkin.

Yours,
Dominic

Answer 2

Wanted to add a number of rules into Code Analysis, turn them on and then require developers to have tested against them when they check-in. What I want is PREfast to fire and execute the check refusing code and then sending a message to an administrator.

Answer 3

See Jiange Sun's overview of checkin policies for configuring the code analysis policy.
http://blogs.msdn.com/team_foundation/archive/2005/04/15/408700.aspx

Buck

Answer 4

I dont think it is possible to cutomise which Pre-Fast rules run.

As for sending mail to an administrator - I can see where you are coming from, but am unsure of the value - the developer cannot check in unless he passes those requirements; at no point does the code enter the scc until it's passed the policy OR it is over-ridden - for which justification etc needs to be provided.

Answer 5

The reason for the email is to be able to manage inexperienced developers. If you are having problems getting code checked in, you might just be tempted to resort to commenting out blocks of code rather than fixing them there and then.

Alternatively, how long before people simply resort to using #pragma to change the status of the error when they don't understand it or are unable to fix it? It's not a huge leap

Both of these get around the need to over-ride and therefore have to make an explanation and the second has the potential to cause chaos. I've been on site where developers leaving their employers have left nasty stuff in code and these days a bug is harder to prove than a deliberate piece of malware.

Answer 6

Code analysis errors will be responsible for failing the policy. In Beta2, prefast errors will fail the 'Clean Build Policy'. [This will change in RTM, where they will fail the Code analysis policy as well]. So, if you want to enforce prefast errors to fail the build, enforce the Clean build policy in addition to the code analysis policy. [From the Source Control Setting dialog]

Enforcing PREfast as part of the policy
-------------------------------------

For enabling Prefast as part of the policy, check the C/C++ Code analysis Checkbox in the code analysis policy editor dialog. To configure which rules you want to developers to not violate, treat them as errors. You could do something like this:

Include a header file in a shared header file. This describes which prefast warnings to emit, and which to treat as errors.
This header file should have the following:
#include <CodeAnalysis\Warnings.h> // For the definition of //ALL_CODE_ANALYSIS_WARNINGS

#pragma warning(error: ALL_CODE_ANALYSIS_WARNINGS) // all rules will be treated as errors and will fail the policy.

Some other useful pragmas
#pragma warning (disable: 6014)   //disables warning 6014

#pragma warning(default: 6011 6035) // enabled as warning

These are examples of how to enable and disable individual rules, or all prefast rules.

Enforcing FxCop as part of the policy
-----------------------------------
In the code analysis policy editor, click on the 'Warning' icon next to the rule you want enforced. This will change the state of the rule to an 'Error'. FxCop errors in the build will fail the policy.

Answer 7

Ian, you can have it send mail.  I don't remember how, and I'm not connected to my work computer to look into it.  I'll send myself a note to look into it.

Checkin policies are meant to help folks do the right thing.  As you point out, you can work around them by adding pragmas, etc.  For associating work items policy, you could always pick a random one.  Since checkin policies are evaluated on the client, you can always get extreme and just call the checkin web service method directly.  It's not a security-type mechanism, and it wasn't designed to be.  It's a very useful, though, in reminding people to things the right way.

What is secure is that no one can ever check something into the repository without leaving a trail of evidence as to who did it (assuming you don't steal someone's password and use their own machine).  History shows what happened.  The repository records who actually did the checkin.

Buck

Answer 8

To follow up on email for overriding the checkin policy failures, we don't send email for that in particular.  The decision was made that the general checkin email was sufficient because it contains the policy override comment, if there was one.

If you want a special email triggered by overriding a policy, you can write some code to listen for the checkin event on the mid-tier and send email to a particular address if there is a policy override comment present in the checkin.

Buck

Answer 9

Buck,

As far as your comment on the email: "The decision was made that the general checkin email was sufficient because it contains the policy override comment, if there was one."

I don't think that is a valid assumption in most cases.  For instance as the project lead on a large product with many developers, I don't want emails on every checkin... however, I would DEFINITELY want a specific email that told me why a developer was overriding the checking policy.  I believe there should be by default a separate email event that could be fired off for a policy override.

In any case, does the extensibility kit provide example code on listening to the checkin event?

Thanks,
~slee

Answer 10

slee,

Yes, you can indeed sign up for the checkin event using the docs in the extensibility kit.  I don't know for sure, but I would expect there's an example in there.

There are couple of forum posts that show subscribing to events, such as this one.
http://forums.microsoft.com/msdn/ShowPost.aspx?PostID=11700

Buck