locked
Before I start the Code Contracts using... RRS feed

  • General discussion

  • Hi there

    I'm considering the "NUnit -> Code Contracts migration" for my new projects. Upon reading documentation I came to conclusion that, comparatively to Unit Testing, the latter approach tests my code while it runs (or while I develope it) on the client side. The Unit Testing works as a separate project and condacts all checkings/validation on my side, not on the client's. I can run all tests overnight and have it done by the morning with no client side involving.

    Is it the case that I cannot do this scenario with Code Contracts? Do all problem/violations manifest itself primarily when the software is working on the client side? Do all my Code Contracts assertions stay in production code turning my cusomer into "tester"?

    Thanks

    Tuesday, August 19, 2014 12:34 PM

All replies

  • Hello,

    I thought I answered you at the C# forum but I'll just copy/paste my post in case you didn't read it.

    Well, there are few things to note here and I'll break it down into multiple paragraphs.

    What is Unit Testing? you can hear different people say different things but all agree that unit testing is a kind of test that the programmer writes to assert that the assumptions he makes about the behaviour of a unit under test or system under test (SUT) matches her expectations.

    Now, there are two camps here.

    Pessimistic/Purist: Contracts are not part of your test suit.

    Before CodeContracts, Design by Contract (DbC) was enforced by testing the arguments of the functions but assertions on the other hand are never tested because a) they removed on release builds. b) during development you use them in conjunction to Unit Testing so it doesn't make sense to test them. c) they are not really meant to be testable.

    A purist person does not treat contracts as if they were asserts and she will most likely use either one of the following approaches to verify that the arguments are validated correctly.

    a) She will create tests around contracts and probably throw specific exceptions either through legacy exceptions or using Contract.Requires<TException>(...).

    b) She will use Contract.Requires(...) and then verify that the generic exception is thrown or use the Contract.ContractFailed event.

    Either way using CodeContracts like this becomes a great burden both in development time, performance and maintainability.

    I think that if I was a purist like that I wouldn't bother with CodeContracts at all.

    Optimistic: Contracts are part of your test suit.

    An optimistic person will treat contracts as if they were assert statements and won't bother test them at all and she will most likely use Contract.Requires(...).

    Now, a very important thing about Unit Testing is that it can't verify whether you programmed something correctly it can only verify whether the result you get is the same result you expected so if you think about a precondition/postcondition of a function or a class and you explicitly say that something cannot go below zero and then you have the static analyzer that proves this assumption for you why would you bother having a test around it?

    Some people will answer this by saying it prevents someone else from breaking the code by somehow eliminating this condition and maybe have regression but then the static analyzer can tell you that it's violated and trust me it does much better job than having a unit test around your arguments validation.

    Others will answer this by saying that the static analyzer can be disabled but then again you can delete tests and the nice thing about the tooling in CodeContracts is that once it's enabled you can see what was violated and fix it.

    So to summarize, the precondition/postcondition of CodeContracts are like assert statements, the static analyzer uses them to prove/disprove the assumptions so it does exactly the same job as a test runner which is to run tests and then tell you whether they passed but here you don't have to write the actual tests, they are embedded in the code itself.

    To answer your question I tend to treat contracts like asserts so I remove them on release builds but in most cases I'll still keep them on my public APIs for arguments validation, it depends on the library/application.


    Regards, Eyal Shilony


    • Edited by Eyal Solnik Tuesday, August 19, 2014 4:17 PM
    Tuesday, August 19, 2014 3:43 PM
  • >To answer your question I tend to treat contracts like asserts so I remove them on release builds but in most cases I'll still keep them on my public APIs for arguments validation, it depends on the library/application

    Ok, I'll put it down as clear as possible.

    Suggest, I insert a "x should not be null ever" as a pre-condition. Now it is a part of my method scope. Each time I run my software on my development environment it gets checked. Note that I do not run a Test project - I got rid of it!

    I do not see any problems with a metod and I deploy it to a customer. Do my customer becomes a tester then?

    I think Yes. Reasons:

    - software has not been tested in that reliable way as my Unit Test project does it. It spends hours (usually 12 hours overnight) for testing all cases you can even think out.

    - pre-condition code is still in the scope method. Your note "I remove them on release builds" - how to remove all pre-conditions interspersed in code? It can be tons of places, right? This is just appoling.

    Thought?

    Thanks.

    Tuesday, August 19, 2014 7:35 PM
  • I do not see any problems with a metod and I deploy it to a customer. Do my customer becomes a tester then?

    Define a tester? by definition anyone that uses a software is testing it to some extent.

    software has not been tested in that reliable way as my Unit Test project does it. It spends hours (usually 12 hours overnight) for testing all cases you can even think out.

    Are you using the static analyzer of CodeContracts? I'm not quite sure I understand you, what do you mean by "not been tested" so how do you verify the contracts?

    pre-condition code is still in the scope method. Your note "I remove them on release builds" - how to remove all pre-conditions interspersed in code? It can be tons of places, right? This is just appoling.

    I'm not removing it manually, I use the tooling of CodeContracts to do it.

    You can also disable it completely and iirc it will do the same thing (as long as the tooling are installed).

    Regards, Eyal Shilony


    • Edited by Eyal Solnik Tuesday, August 19, 2014 8:18 PM
    Tuesday, August 19, 2014 8:12 PM
  • Hi there

    It is getting clear for me, Eyal. I'm just a Code Contracts beginner being alerted towards it. Now I'm loving it more.

    One more question.

    The build time is critical for me. My solution build takes ~ 3min. Do you think this time will grow noticably if I populate every method/function/action with CodeContract functionality (I heard that Code contracts performs a pre-building additionally to a regular solution build)?

    Thanks.

    Wednesday, August 20, 2014 12:36 PM
  • Well, if you notice in the image when you enable the static analyzer you have the ability to cache results. :)

    Regards, Eyal Shilony

    Wednesday, August 20, 2014 2:01 PM
  • Thanks.

    I'm marking this as an answer, but there is no such link here :-)

    Wednesday, August 20, 2014 2:24 PM