none
Check if an object is referenced before Garbage Collection RRS feed

  • Question

  • Is there a way of finding out if an object is referenced by the application roots before it is being Garbage Collected?

    Here's the full story:

    I need to create image buffers that can be accessed using pointers. At times I need to use the pointer for longer periods. If I pin the buffer in the GC I would prevent the memory from being collected when there are no references to my buffer. Since I'm not really into the C/C++ way of thinking my application does not track when objects are no longer used and can be disposed of. So, the GC semantics suits me just fine.

    My idea is to instead allocate unmanaged memory and implement a Finalizer in my buffer object, so far no problem. I was thinking keeping a list of WeakReferences to my buffers so that when a new buffer is needed I can resuse unmanaged memory that hasn't yet been destroyed. But the WeakReference cannot tell me if the object is used, it can only tell me if the object still hasn't been collected, right !?

    So, back to the original question; Before I try the WeakReference, is there a way of telling whether the object is referenced or not?

                object buffer = new object();  
                WeakReference wr = new WeakReference(buffer);  
     
                // buffer is being used  
                // ...  
     
                // A new buffer need to be created  
                bool referenced = IsObjectReferenced(wr.Target);  
                if (!referenced)  
                {  
                    object oldBuffer = wr.Target;  
                    if (oldBuffer != null)  
                    {  
                        // Use the oldBuffer as a new strong reference for   
                        // the new buffer.  
                    }  
                } 

    I know the above code wouldn't really work since passing the wr.Target to a method would create a strong reference and undermine the purpose of the ckeck, but its just to demonstrate the idea.

    Feel free to comment on this, say I'm a total idiot for trying, or suggest me to take another approach...

    regards
    /Calle
     

    - Still confused, but on a higher level -
    Thursday, August 7, 2008 9:24 AM

Answers

  • Unpinning in the finalizer can't work, it will never run since you always have a reference.  Pinning GC memory for a long time is in general very bad, you effectively reduce the size of the 1st gen heap, the collector will run much more frequently and a lot less effectively.  One workaround for that is to allocate an array that is at least 85 KB.  The allocation will then be made in the Large Object Heap.  Pinning such an array is no issue, the LOH is never compacted.  Allocating unmanaged memory with Marshal.AllocHGlobal() is certainly an option too.  But may be a lot harder to use, I can't tell.  If that's not a consideration, I'd definitely go for unmanaged memory.
    Hans Passant.
    Thursday, August 7, 2008 11:50 AM
    Moderator

All replies

  • I don't get it, a WeakReference is exactly what you need.  Not turning it back into a strong reference cannot work, the buffer would be collected while you are using it.  
    Hans Passant.
    Thursday, August 7, 2008 10:15 AM
    Moderator
  • Yes it would be collected some time, but there's a delay from when an object becomes unreferenced until the GC actually collects it (especially when large memory allocations exist in unmanaged memory). In this time gap I would like to reuse the buffer when the app requests new buffers to be created, sort of like memory pooling.

    When I turn the WeakReference into a strong reference I don't know if the application still has references to the object tracked by the WeakReference. All I get is an object or null if the object has been collected. What I wan't is to be able to ckeck whether the object is not referenced and still alive, only then would I reuse the memory allocated by the buffer.

    /Calle
    - Still confused, but on a higher level -
    Thursday, August 7, 2008 10:26 AM
  • There is no delay or time gap, the garbage collector can kick in any moment due to allocation requests from other threads.  Even if you don't have other threads, the concurrent collector might already be at work while your code is executing.  Although you can turn it off with <gcConcurrent>.  That's an awfully big gun though.  Besides, it is awfully hard to not have other threads, an I/O completion thread will ruin your day.

    Even if it could work, the best you'd get is "cheap memory", reusing a heap block that is no longer in use but still available.  There is however no way that the plumbing you'd have to put in place could actually work faster than just allocating a new buffer with the new operator.  The .NET memory allocator is blindingly fast, many times faster than any unmanaged memory allocator.  It just adjusts a pointer.

    But it can't work, the CLR won't let you find unreferenced memory.  The object tracking handle table is behind a locked door.

    Hans Passant.
    Thursday, August 7, 2008 10:48 AM
    Moderator
  • I hear you Hans, I would need to explore some other options. However, one problem still exist, and that is the use of data that does not move around.

    I your opinion, what do you think is the most efficient way of deeling with large memory:

    Allocate unmanaged memory to be able to keep a steady data pointer and use the GC's AddMemoryPressure/RemoveMemoryPressure to make the GC do more realistic collections.

    or

    Use the GCHandle to pin down managed memory and in my finalizer free the pinned mem (would this mean that the pinned memory would be reclaimed in the next collection? ) I've got different result from this though, where in my first test my object that holds the handle (and the byte array) didn't get finalized !?

    I really value your input on this Hans.
    thanks

    /Calle
    - Still confused, but on a higher level -
    Thursday, August 7, 2008 11:05 AM
  • Unpinning in the finalizer can't work, it will never run since you always have a reference.  Pinning GC memory for a long time is in general very bad, you effectively reduce the size of the 1st gen heap, the collector will run much more frequently and a lot less effectively.  One workaround for that is to allocate an array that is at least 85 KB.  The allocation will then be made in the Large Object Heap.  Pinning such an array is no issue, the LOH is never compacted.  Allocating unmanaged memory with Marshal.AllocHGlobal() is certainly an option too.  But may be a lot harder to use, I can't tell.  If that's not a consideration, I'd definitely go for unmanaged memory.
    Hans Passant.
    Thursday, August 7, 2008 11:50 AM
    Moderator
  • The memory in each buffer will always be larger than 85KB. Try at least 1024KB, and a possible throughput of up to 50 alloc/destroy per second.

    I just ran a simple test comparing the two options. Releasing the GCHandle for a pinned byte array do work in the finalizer, however it will take another GC collection to reclaim that memory. What does not work so well is the freqency of garbage collection.
    Using the code below and the ManagedBuffer class makes the PF usage peek at a very high level before the GC frequency speeds up. Then the PF usage drops and stays steady. I know the PF usage is a bad meter to be using but what I do know is that my computer tends to slow down whenever the PF usage meter goes up, so its really undesirable that it reaches those top levels before stabilizing. 

    Now using the AllocHGlobal (using the same code but swithing to the UnmanagedBuffer class) the PF usage peeks at a much lower level before going steady. 

    In both cases the CPU is practicly zero all the time which is good and when things have stabilized the GC collects about every 10 object or so (every 14-15 MB).

    namespace ConsoleApplication1  
    {  
        class Program  
        {  
            static void Main(string[] args)  
            {  
                Thread t1 = new Thread(new ThreadStart(Test));  
                t1.IsBackground = true;  
                t1.Start();  
                Console.ReadKey();  
            }  
     
            static void Test()  
            {  
                while (true)  
                {  
                    UnmanagedBuffer buffer = new UnmanagedBuffer();  
                    //ManagedBuffer buffer = new ManagedBuffer();  
                    Thread.Sleep(20);  
                }  
            }  
        }  
     
        public class UnmanagedBuffer  
        {  
            private IntPtr handle;  
            static int newCount = 0;  
            static int finalizedCount = 0;  
     
            public UnmanagedBuffer()  
            {  
                handle = Marshal.AllocHGlobal(1445600);  
                GC.AddMemoryPressure(1445600);  
                newCount++;  
            }  
     
            ~UnmanagedBuffer()  
            {  
                finalizedCount++;  
                Console.WriteLine("Finalized: {0}, Created: {1}, Not collected: {2}",   
                    finalizedCount, newCount, newCount - finalizedCount);             
                if (handle!= IntPtr.Zero)  
                {  
                    Marshal.FreeHGlobal(handle);  
                    GC.RemoveMemoryPressure(1445600);  
                }  
            }  
        }  
     
     
        public class ManagedBuffer  
        {  
            public byte[] buf;  
            private GCHandle handle;  
            static int newCount = 0;  
            static int finalizedCount = 0;  
     
            public ManagedBuffer()  
            {  
                buf = new byte[1445600];  
                handle = GCHandle.Alloc(buf, GCHandleType.Pinned);  
                newCount++;  
            }  
     
            ~ManagedBuffer()  
            {  
                finalizedCount++;  
                Console.WriteLine("Finalized: {0}, Created: {1}, Not collected: {2}",   
                    finalizedCount, newCount, newCount - finalizedCount);  
                if (handle != null)  
                {  
                    if (handle.IsAllocated)  
                    {  
                        handle.Free();  
                    }  
                }  
            }  
        }  

    If its not more work involved using unmanaged memory I think that seems most interesting, what do you think?

    /Calle

    - Still confused, but on a higher level -
    Thursday, August 7, 2008 12:18 PM
  • Ah, I see what you mean about unpinning in the finalizer, In my world the Finalizer was not in the pinned object, therefore it works.

    Sorry about the anwer marked/removed, hit the wrong button, for now that is... ;)

    /Calle 


    - Still confused, but on a higher level -
    Thursday, August 7, 2008 12:30 PM
  • I have no good explanation why you'd see page file activity higher when allocating managed memory.  You should be consuming virtual memory space at the same rate in either case since it will be the GC that is triggering the deallocation.  Perhaps the Windows memory manager is more likely to page out pages containing managed code or data when it needs to make room for more buffers in RAM when you allocate managed memory.  Last I read about it, it was a simple last-used algorithm.  But that was a long time ago, I don't doubt it has been tweaked a lot since then.  The .NET memory manager uses a different heap than AllocHGlobal(), maybe the memory manager is more apt to page out pages from the same heap.

    Sounds to me you talked yourself into using AllocHGlobal().

    Hans Passant.
    Thursday, August 7, 2008 12:56 PM
    Moderator
  • You're like a dictionary Hans, or maybe you just got a lot of books on your shelf  ;)

    Thanks anyway, really appreciate it!

    /Calle


    - Still confused, but on a higher level -
    Thursday, August 7, 2008 1:22 PM