none
99% Unreachable Memory RRS feed

  • General discussion

  • Hi all,

    I have a .NET 4.5.2 app that is supposed to run continuously for months. The things it does are not particularly memory- or CPU-intensive, but it does create and destroy many objects, some of which may be large. It does not use any unmanaged resources.

    I observe that over time the app's RAM usage, as reported by Task Manager, tends to grow to a very large factor of what it should be. I've used PerfView to get a heap dump, and it tells me that 99% of it is "Unreachable Memory".

    I understand that this is memory that is unreferenced, but that the GC has not reclaimed yet. My question is: why does the GC let 99% of the memory allocated to my application remain unclaimed, even after days or weeks? I've read several articles on the GC, but I still can't figure this out.

    From searching around, I've found two broad categories of answers.

    1) Trust the GC to release that memory if and when the system requires it. This makes me nervous. Even assuming that I can trust the GC to release memory before other things start throwing OOM exceptions, surely there should be a way to tell it that it's OK to run an extremely aggressive GC operation right now, and I don't care if it takes in the order of seconds, just free everything and compact everything until all I have allocated are live objects nicely stacked one after the other? For example, for diagnostic purposes, or because I know that I'm going to need that RAM quickly in a minute or so, or just because I'm a control freak? I have tried GC.Compact with every possible flag, as well as GCSettings.LargeObjectHeapCompactionMode, but nothing I do seems to even put a dent in that 99% Unreachable Memory.

    2) Redesign your application so that it's more GC-friendly; reuse large objects, and so on. Again, this makes me nervous. I get that if you want performance, you need to take control of things. But this app is very much not a performance app. It does not deal with massive objects. It does not deal with millions of objects. It does not require millisecond response times. If the GC can't handle that, surely that's a major failure on its part? I mean, I get that my app is GC-unfriendly, but I would expect this to result in maybe the app taking two or three times as much RAM as it needs, or maybe freezing for a second every now and then. Worst case, maybe have to tell it once per day to do an exceptionally aggressive GC. I wouldn't expect it to expand to fill all available system RAM forever with no way to release it.

    Now, what I hope is, that there is something else I'm doing wrong, and that once I find it and fix it the problem will be gone and the app will go back to using the 50 megs or so it actually needs at any given time. But I'm having a lot of problems investigating this, because nearly every search result I find falls into one of the two categories above.

    Also, I'm really confused by the fact that nothing I do with GC.Collect() seems to have any effect. I would think I have a good old memory leak, except that I don't use unmanaged resources and PerfView tells me that all of the memory is Unreachable. So why is there no way to GC it?

    Any ideas?

    Saturday, May 26, 2018 6:56 PM

All replies

  • We don't know your program. 

    But if your program exist for instance from one method, which runs in a loop without that method ever stops, then the gc will never run. 

    Then probably a simple application.doevents in a timer (whatever time you think is reasonable) can fix this. 


    Success
    Cor

    Sunday, May 27, 2018 12:41 PM
  • Hi Cor,

    It's a WPF program. It has a few background threads that run in a loop (with Thread.Pause or WaitHandle calls in the middle). But the main thread is doing standard WPF stuff and has plenty of idle time.

    Sunday, May 27, 2018 1:15 PM
  • Yea that I was guessing (the Pause and the Wait) but still kept the method running. 

    Be aware that in .Net a thread is not meant as a running process, it is a task, which constantly ends in nanoseconds. 

    You are not the first one making this mistake.  




    Success
    Cor

    Sunday, May 27, 2018 3:13 PM
  • These are not ThreadPool threads, nor Task threads. They are full Threads, created by invoking the Thread constructor, and suitable for long-running tasks. They won't hog the ThreadPool.

    Sunday, May 27, 2018 5:23 PM
  • Reading your replies I more and more get the idea you are using non managed code which is not released. 

    Be aware that therefore only calling the method dispose is not enough. You have to override that method with code to release those unmanaged objects. 


    Success
    Cor

    Monday, May 28, 2018 11:40 AM
  • I'm not using any unmanaged resources directly.

    I am using CLR IDisposables, and I am calling Dispose() on those.

    Also, unless I'm mistaken, it is not true that not calling Dispose() will cause a memory leak, provided that the IDisposable correctly implements the pattern (which I assume all CLR objects do). Once the object has no more managed references, the GC will eventually call its finalizer, and the finalizer will call Dispose(false). This is inefficient, but it should not cause unmanaged resources to stick around literally forever.

    Monday, May 28, 2018 12:13 PM
  • I thought maybe the finalizer thread was blocked by something. No luck; if, while the problem is manifest, I create an object with a finalizer and set the reference to null, the object's finalizer runs during GC. So it's not that either.
    Monday, May 28, 2018 1:11 PM
  • You are aware that you made a discussion. Therefore for those who are sick of all discussions about the GC will probably avoid your discussion.

    Somehow I think you want to know how to solve it. 

    Then make again a question of this and at least show with some code samples what you mean. 

    Words are nice in a discussion, but if it is about code which is used (you wrote it already yourself) are there thousands of ways which people think it is the absolute only one. Therefore show some relevant code.  


    Success
    Cor


    Monday, May 28, 2018 3:22 PM
  • Fair enough. I'll leave it here a while, and then repost as a question.

    Unfortunately, I won't be able to post code, because the application is huge and I have absolutely no idea of how to determine which code is causing the problem. All I have is a vast, apparently undifferentiated blob of Unreachable Memory.

    If there's any way to get PerfView to tell me what are all these unreclaimed objects, that would be marvelous. I can even see the problem in the VS debugger, but I can't find any additional information there either.

    Monday, May 28, 2018 3:41 PM
  • Just in case anyone stumbles on this, eventually I discovered that PerfView was just wrong. It classified as "Unreachable" some stuff that was very much reachable. I switched to the latest version of PerfView and made sure to tick "Save ETL" while taking the snapshot, and that made the dump truthful.

    At that point, I easily found that I had a lot of objects being referenced by some XAML bindings that were supposed to be Mode=OneTime but weren't - a good old memory leak. I fixed it, and the problem went away.

    Saturday, December 14, 2019 5:28 PM