none
Blocking garbage collection occurred outside background garbage collection. RRS feed

  • Question

  • I have a large application that occasionally suffers complete stoppage of all threads. We are using gcServer = enabled and gcConncurrent = enabled, but we are still suffering from occasional Gen 2 blocking collections.  I've traced it down using Windows Performance Tools (WPT) and recording the CLR and Kernel events. The stoppage occurs when we have a Gen 2 collection due to a Large Object heap allocation of type 0 (GCState_V1Event : Reason - Blocking garbage collection occurred outside background garbage collection.).  The application regularly stops for 2-4 seconds on a 16 Core machine with 200 gig of memory during these collections, and  occasionally stops of 15 seconds. The total heap size during the stoppages is about 4-6 gig. During type 1 collections the pause is 100's of msecs, which is acceptable.

    The documentation on WPT and EWT (Event Tracing for Windows) are very thin. Any help understanding "GCState_V1Event:type : 0x0 - Blocking garbage collection occurred outside background garbage collection." is appreciated.

    Tuesday, April 30, 2013 6:06 PM

All replies

  • The GC, even with concurrent server GC, still has to occasionally block and stop threads.  The message is exactly that - when it has to stop the threads to cleanup, it notifies you.

    The LOH is probably the main culprit -so trying to reduce or minimize the LOH collections will help.  Also - .NET 4.5 made some huge progress in this respect, and has a much better algorithm for the GC in general, especially the new background server GC (see http://blogs.msdn.com/b/dotnet/archive/2012/07/20/the-net-framework-4-5-includes-new-garbage-collector-enhancements-for-client-and-server-apps.aspx )


    Reed Copsey, Jr. - http://reedcopsey.com
    If a post answers your question, please click "Mark As Answer" on that post and "Mark as Helpful".

    Tuesday, April 30, 2013 6:43 PM
    Moderator
  • We are currently using 4.0.3 with gcServer = enabled and gcConcurrent = enabled, so we should be getting the benefits of .NET 4.5 GC improvements. WPT shows the CRL suspended and resumed during the background GC, but only for short periods of time (type=1). During type=2, the CRL is suspend for extended periods, the entire collection period. I need to understand why the CLR decides to use type=2 occasionally verse type=1 so I can code appropriately. We have already begun optimizing our LOH usage, reducing the frequency of these events an order or magnitude.  Except that we still suffer them occasionally.  

    The LOH does not get compacted, and our allocation rate is fairly constant - so what causes the CRL to select a blocking Gen 2 collection over a background Gen 2 collection. By understanding the rationale I will be able to design something that doesn't trigger these events.

    Something I have not been able to verify is exactly when the LOH is collected. With every Gen 2 collection, or only when an allocation request can't be fulfilled, akin to but separate from the SOH operations. This would make sense, because the events only appear to occur with LOH allocation requests.

    Tuesday, April 30, 2013 7:02 PM
  • Hi Symbot,

    Thanks for your participation.

    I'm trying to involve some senior engineers into this issue and it will take some time. Your patience will be greatly appreciated. Sorry for any inconvenience and have a nice day!

    Best regards,


    Mike Feng
    MSDN Community Support | Feedback to us
    Develop and promote your apps in Windows Store
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.

    Wednesday, May 1, 2013 1:55 PM
    Moderator
  • Hi,

    Have you set the GCSettings.LatencyMode property to GCLatencyMode.SustainedLowLatency?


    Regards,
    Christian HL
    Microsoft Online Community Support


    Please remember to click “Mark as Answer” on the post that helps you, and to click “Unmark as Answer” if a marked post does not actually answer your question. This can be beneficial to other community members reading the thread.

    Thursday, May 2, 2013 3:24 AM
  • No, we haven't tried that setting yet. It requires us to update our target platform to 4.0.3 and right now we are targeted at 4.0.0.  Upgrading to 4.0.3 would be a lot of work - hundreds of projects. LatencyMode is only available for Workstation, and the SustainedLowLatency documentation makes us believe that this should only be used during critical times, not as a full time setting. Please correct if our interpretation  is incorrect.


    Thursday, May 2, 2013 11:56 AM
  • Hi Symbot,

    I'm the tester for the Garbage Collection in CLR. You mention you use .NET 4 with update 4.0.3. There is no Background GC for Server GC in this version.  Background GC for Server was implemented in .NET 4.5.

    Please refer to the MSDN GC documentation:

    http://msdn.microsoft.com/en-us/library/vstudio/ee787088(v=vs.110).aspx

    So for the version you use it is expected that with Server GC, all Gen2 collections are blocking.

    For a quick analysis of GC collections you can use PerfView which you can download from Microsoft Download center: http://www.microsoft.com/en-us/download/details.aspx?id=28567

    Open your etl file in PerfView and look at GCStats. You can also collect ETW traces directly with PerfView.

    Thanks,

    Alina

    • Proposed as answer by Alina Popa Wednesday, May 15, 2013 6:07 PM
    Wednesday, May 15, 2013 5:58 PM
  • To answer to this question >"Something I have not been able to verify is exactly when the LOH is collected"

    LOH is collected with every Gen2. A Gen2 collects everything (all SOH + LOH).

    Wednesday, May 15, 2013 6:11 PM

  • We have installed .NET 4.5 on our machines and even though we target 4.0.3 we are currently enjoying the benefits of the GC improvements in 4.5.  

    Thanks for clarifying the Gen 2 collection question.


    Symbot

    Wednesday, May 15, 2013 6:24 PM
  • Do you still get the original issue (blocking GC when you would expect background) when running on 4.5?

    If yes that means Gen2 (Small Object Heap) is fragmented. Perfview GCStats has a column that shows the percent fragmentation.

    Wednesday, May 15, 2013 6:38 PM
  • Yes, we are still having problems. I've traced it down to a fully blocking gen 2 collection that is somehow induced (someone is calling GC.Collect). Nothing in our code induces a collection. The primary culprit right now is an AddMemoryPressure call from within the clr - windb shows this is the call stack for an induced collection. I'm presently trying to confirm this call results in our system doing this full blocking collection.

    Symbot

    Wednesday, May 15, 2013 6:42 PM
  • AddMemoryPressure induces a background GC. It's up to the Garbage Collector to choose to make it blocking if there are certain conditions in the application (such as high gen2 fragmentation).

    It's easy to see if your blocking GC is induced, the GCStart event should have the Reason 0x1 - Induced. From the original email my understanding was that the reason is 0x4 - Large object heap allocation.

    Wednesday, May 15, 2013 7:19 PM