none
cccheck - excessive memory & processor usage?

    Question

  • Hi

    First up, I'd like to congratulate you for you work; I'm excited to be using Code Contracts and have high hopes that it will help increase developer productivity and improve overall code quality.

    Unfortunately, at present the runtime checking process is killing application performance and development experience to the point where it is unusable - perhaps we are not using it correctly?

    First up a little background; we're developing a WPF 4.0 application and using the latest version of Contracts.  I'm running a 32 bit version of Windows 7 and VS2010 Ultimate.

    Contracts have been defined on models to validate incoming property values (pre cons), and also to describe the guarantees for the properties return value (post cons).  I understand this to be the correct approach for ensuring a good static analysis experience, but it also nicely replaces the usual if arg check throw guards we write so warrant being included at runtime also.  Further I have an invariant method (marked with ContractInvariantMethodAttribute) that assess' the overall state of the object as the validity of property1 may be dependent on the value held in property2 .. this is more designed for runtime validation of the objects state.

    The ContractInvariantMethod is checking the invariant state by accessing the public getter methods of the class to retrieve the value of the property backing fields.  I believe this is more appropriate than accessing the backing field direct (in line with examples in help files) as it means I do not need to duplicate contracts on the fields and seem to get better static analysis (please correct me if I should be validating the backing fields and I have misunderstood).  The trade off is that each time the property is read, there are post condition checks 'unnecessarily' running in the getter.

    We are using the exception thrown by the contract framework as a mechanism of notifying WPF bindings that there are validation errors using Contract.Requires<ArgumentXXXException>() in our list of pre-conditions.  Further the contract invariant method exception where thrown is used in the same way.

    The behaviour of the WPF bindings is as desired.

    The build definition is defined as ...

    Assembly Mode - Custom Parameter Validation
    Perform Runtime Contract Checking - Full
    Perform Static Contract Checking - True
    Check in Background - True
    Show Squiggles - True

    When we run the application, cccheck starts to consume RAM (1-1.5GB of 2GB available) and 80/90% of the processor.  The RAM consumption continues to grow and the processor fluctuates.  Even in the visual studio idle state (not running app, not writing code etc) I can sit and watch the memory consumption of cccheck grow quickly and the processor fluctuate between 20/50%.

    Has anyone else experienced such problems or is this due to a misuse of the framework?

    Any advice would be appreciated.

    Richard

     

     

    [Edit: Mistakenly blamed ccrewrite instead of cccheck.  Replaced with correct process]

    • Edited by iProgrammer.co.uk Wednesday, June 23, 2010 4:43 PM Used wrong process name
    Wednesday, June 23, 2010 4:22 PM

All replies

  • Hi Richard,

    thanks for the comments. Yes, we are aware of the memory/time consumption issues on large projects, or projects that reference many dlls. We have made some progress on this on our internal builds and hope to release that soon.

    For static checking large projects, the checker is quite slow and we are working on incremental checking which should also help.

     


    Cheers, -MaF (Manuel Fahndrich)
    Wednesday, June 23, 2010 4:59 PM
  • Hi Richard,

    thanks for the comments. Yes, we are aware of the memory/time consumption issues on large projects, or projects that reference many dlls. We have made some progress on this on our internal builds and hope to release that soon.

    For static checking large projects, the checker is quite slow and we are working on incremental checking which should also help.

     


    Cheers, -MaF (Manuel Fahndrich)


    Hi Manuel

    Thanks for the reply ... I'll keep an eye out for the release.

    I would however mention that this is not a large project (I just started work in it and only have a couple of classes with contracts) and it references nothing other than the standard .net binaries and a couple of Microsoft.Expression references added by Blend.  In total, 20 dlls, all from Microsoft.

    It's also running away with the processor and consuming more and more memory even when I leave VS2010 sat on its own doing nothing (ie not running app and not changing code).

    Richard

    Friday, June 25, 2010 11:20 AM
  • I've also noticed performance problems and on an older machine the static checker is not an option. Just too slow. A shame because the static checker is great.
    Tuesday, June 29, 2010 1:04 PM
  • Okay, in that case, it might be a bug. If you could share a small repro with us, that would be excellent.

    We also released a new version, so give it a spin and see if it happens to solve your problem.

     


    Cheers, -MaF (Manuel Fahndrich)
    Wednesday, June 30, 2010 2:39 PM
  • Hello,

      you want to try the latest versions of the checker, which are faster and in particular have a caching mechanism which really improves the user experience.

    f

    Thursday, February 17, 2011 1:03 AM
  • Nope ... still the same problem.

    I'm also experiencing big problems trying to use it with a Windows Phone 7 project where VS stops (like a break point) every time a contract is checked.

    It's a shame, would really like to use these tools.

    Thanks anyway.

    Richard

    Thursday, February 17, 2011 9:14 AM
  • Sorry to hear that.

    I'd really love if you can send me a (small?) repro.

    A way to get it is by adding the switch "-show progress" in the extra static checker options, and see which method is causing the static checker to spin.

    (We've worked towards improving the memory usage of the checker in the last internal releases)

    Thanks,

    f

    Thursday, February 17, 2011 9:37 PM
  • I'm still having issues with the performance of the static checker in a medium sized solution.

     

    CodeContracts: xxx: Validated: 91.9%

    CodeContracts: xxx: Contract density: 1.70

    CodeContracts: xxx: Total methods analyzed 149

    CodeContracts: xxx: Total time 6:54min. 2781ms/method

    CodeContracts: xxx: Discovered 17 candidate preconditions in 00:00:01.7211721

    CodeContracts: xxx: Retained 2 preconditions after filtering

     

    I will try and see if I can isolate the particular method but it's difficult as VS2010 becomes unresponsive and useless until the build finishes.  This is quite a poor performance though.

    Windows 7 x64,  VS2010 Ultimate, Code Contracts 1.4.40602.0, Dual Core 3.17GHz, 4 GB RAM.

    Monday, June 27, 2011 3:03 PM
  • Hi Stephen,

      I'd love a small repro. One way to get it is to

    1. enable the checking in background (so VS is not unresponsive)
    2. add the -show progress switch (to be added in the "extra static checker options")
    3. note which method spins

     

    As for performance, it should be noticed that the checker perform deep semantic analysis, which can be very expensive. One way of mitigate it is to enable the caching mechanism from the Vs UI.

     

    ciao

    f

    Wednesday, July 13, 2011 12:38 AM
  • Hi Francesco

    Unfortunately small repros are not too easy to come by, as small projects do not exhibit gargantuan memory consumption and epic static check durations.

    I have tried analysing performance of various aspects of the static checker and the two that seem to be the worst are the "arrays analysis" and "validating the explicit assertions"  (also the non-null assertions but I have disabled these to get the static checker to complete).

    Here's an example from a fairly simple method, with a few LINQ enumerable queries:

    == 6: Validating the Explicit assertions
    CodeContracts: XXX: (elapsed: 00:04:09.1102483, mem: 655Mb)

    On a side issue, the memory consumption at this stage was nearly 2gb, not 655mb as stated.

    I have 4gb RAM, Windows 7 x64 and a 3gHz quad core.

    Regards.

    Stephen


    Tuesday, March 06, 2012 6:47 PM
  • Hello,

    Does the latest version contain any improvements on excessive memory usage?

    Right now one of our projects using almost 3gb of memory and I usually end up killing it and turning off the static checker.

    If it continues that this becomes our standard workflow, we're probably going to have take out all our contracts and replace them with standard argument validation which would be a shame, not to mention considerable effort for over 15,000 assertions.

    Can you provide any information whatsoever about anything that is being done to make cccheck a little less intensive?  

    Regards,

    Stephen

    Wednesday, April 04, 2012 8:49 PM
  • In the latest version we had some improvements in the memory usage, but I do not know if this solves your problem.

    Can you test it?

    Do you have any way of seeing which option is causing the excessive memory usage? Can you try for instance by turning off all the inference options? (just to see if it is one those)

    sorry about that, any info that you can provide me to repro the problem is very welcomed.

    f

    Wednesday, April 04, 2012 8:54 PM
  • Hi Francesco,

    Thanks for the quick reply.

    I rewrote a method that was causing cccheck to spend minutes inside it.  I have no idea why, it had a large if statement, which I sanitized a little and it fixed it (still takes a few seconds for that method though).  We're down from well over 10 minutes to 35 seconds.  It uses over 2gb memory still though which is fine on this PC, but in the office we are not so fortunate.

    However, it is only reasonably quick with these set:

    Check in Background (this has to be set or VS is unusable)

    Show Squigglies

    Cache Results

    Warning Level:  low or high, makes no difference.

    If I turn on any of the other options, it grinds to a halt.  Memory usage is not affected by any of these settings it seems.

    We don't use inference anyway, so it wasn't those.

    I have a question (might not be easy to do this, but am curious) - is it better to have many small assemblies, or fewer larger ones?  I'm guessing many small ones.  Currently this assembly I am using for experimentation is 2.4mb when built in release.

    Cheers,

    Steve

    Wednesday, April 04, 2012 10:38 PM
  • Thanks Stephen.

    I guess you hit some bad performance corner of the static checker. If you have a repro, I will be interested into seeing it.

    As for the assemblies, once the cache is turned on, it should make no real difference (except of course if the assembly is *very* big, and then the SQL access will become the bottleneck)

    ciao

    f

    Wednesday, April 04, 2012 10:43 PM
  • Interestingly, in another project, consisting mainly of view models, the static checker is extremely slow (not to mention the 2.5gb memory use).

    A notable feature is here that the view models are all quite deep in an inheritance chain:

    MyViewModel

       SomeViewModelBase<T>

           SomeOtherViewModelBase<T>

                YetAnotherViewModelBase<T>

                    ViewModelBase

                         ComponentViewModelBase

                               ObservableObject

                                     DispatcherObject

    Could this be a cause for poor performance?  It literally takes around a second for most property get/set methods on these view models (even after I commented out all the NotifyPropertyChanged calls which take a Linq expression to identify the property - I thought this might be onerous for the checker).

    Most of these get/set look like this:

             public string QuickSearchField 
            { 
                get 
                {
                    Contract.Ensures(Contract.Result<string>() != null);
                    return _quickSearchField ?? string.Empty;
                }
                set
                {
                    Contract.Requires(value != null);
                    if (_quickSearchField != value)
                    {
                        _quickSearchField = value;
                        //NotifyPropertyChanged(() => this.QuickSearchField);
                    }
                }
            }

    Note that the performance was not improved from commenting out the notification.

    Regards,

    Steve


    Wednesday, April 04, 2012 11:11 PM