none
virtual memory is not released in asp.net RRS feed

  • Question

  • We are getting OutOfMemoryException in our Asp.Net WebService (on a 32 bit machine) and while analyzing the problem we found that when we release some memory in .Net (and call GC just to be sure it is collected), the "Process/Private Bytes" counter of the aspnet_wp goes down, but the "Process/Virtual Bytes" does not.

    Here is a simple test method: It allocates blocks of 10MB until all the memory is filled. Then it releases everything and finally allocates a 100MB block. (The sleep statements are there so you can observe memory usage in performance monitor)

        [WebMethod]
        public void MemFragmentationTest()
        {
          const int allocSize = 10000000;  // 10 MB
          List<byte[]> buffers = new List<byte[]>();
          // allocate
          while (true)
          {
            try
            {
              buffers.Add(new byte[allocSize]);
              Thread.Sleep(100);
            }
            catch (OutOfMemoryException)
            {
              break;
            }
          }

          // release
          buffers.Clear();
          GC.Collect();
          Thread.Sleep(5000);

          // allocate some more
          byte[] buffer2 = new byte[10 * allocSize];
        }


    If I run this code as a standard console application, it works fine. In performance explorer, I can see both private bytes and virtual memory going down when the buffer is cleared. If however I run it as Asp.Net webservice, the virtual memory usage remains high during the 5 second sleep and I get an OutOfMemoryException when trying to allocate buffer2.


    My machine is a 32 bit windows XP, SP3, 2GB ram, .Net 2.0.
    Thanks for your help.
    Monday, August 16, 2010 2:34 PM

Answers

  •  

    It is an expected behavior that the virtual memory doesn't drop down;

    CLR 2.0 introduced a feature called VM Hoarding which doesn't free segments but keep the segments in hand for future use, and we can specify a startup flag called STARTUP_HOARD_GC_VM via the hosting API to enable this feature.

     

    Anyway, I haven't found a method to tell IIS ( or ASP.NET) to turn this feature off when starting CLR.


    Sincerely,
    Eric
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    • Marked as answer by tiktok Wednesday, August 25, 2010 7:48 AM
    Friday, August 20, 2010 12:42 AM

All replies

  • The counters you are looking at are OS counters, but the .NET GC does not often release memory all the way back to the OS.  Rather, it holds on to it for use when you allocate subsequent objects in .NET.  Use the Windows Performance Monitor to examine the .NET specific counters (under .NET CLR memory).  There are three "generations" each with a heap size.  The number of bytes currently allocated in all heaps is also available.

    If you notice that the heaps grow to high, then you might be holding on to objects too long.  Consider acquring and using a .NET memory profiling tool for in-depth troubleshooting of these kinds of issues.

    Note that to help compare the exe and ASP.NET environment, you should configure the .exe for Server GC (instead of the default Workstation GC):

    http://blogs.msdn.com/b/junfeng/archive/2004/07/13/181534.aspx

    But I wouldn't read too much into this--you really need to be looking at the .NET performance numbers when troubleshooting your app.

     

     

    Tuesday, August 17, 2010 12:32 AM
  • Thanks for the pointer to the GC modes, I didn't know about that. So I tried setting the Server GC in the app.config (at runtime I'm checking GCSettings.IsServerGC, just to be sure), but still, the virtual memory behaves different in console application and Asp.Net. The .NET CLR memory counters (Committed, Reserved, Bytes in all Heaps) all follow the Process/private bytes counter, i.e. they go down when I clear the buffers. Just the Process/Virtual Memory remains high in Asp.Net.

    My understanding so far is that memory is reserved from the OS (using VirtualAlloc) to be used for the .Net heaps. Usually this is done in junks of 32 MB (16 MB in workstation mode), unless the application needs more for a single allocation. When the application no longer needs the memory and the GC has collected it, these junks can be given back to the OS (with VirtualFree). Asp.Net however prefers to keep them such that if the application agains needs memory, it can use these already allocated blocks. I assume this is done to save the time.

    Is my understanding more or less correct? And if so, is there any way to force Asp.Net to release memory?

    The problem occurs if you have an application that first needs lots of smaller memory junks. In this phase, the memory gets fragmented into 32 MB blocks. If the application later releases all the small junks and instead wants one big (>32MB) piece, it gets OutOfMemoryException. One could argue that this is a very special case that does not occur in practice and that it would be better to avoid using so much memory in the first place. But still...

     

    Tuesday, August 17, 2010 3:23 PM
  •  

    It is an expected behavior that the virtual memory doesn't drop down;

    CLR 2.0 introduced a feature called VM Hoarding which doesn't free segments but keep the segments in hand for future use, and we can specify a startup flag called STARTUP_HOARD_GC_VM via the hosting API to enable this feature.

     

    Anyway, I haven't found a method to tell IIS ( or ASP.NET) to turn this feature off when starting CLR.


    Sincerely,
    Eric
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    • Marked as answer by tiktok Wednesday, August 25, 2010 7:48 AM
    Friday, August 20, 2010 12:42 AM
  • Hi tiktok,

    Are you there?


    Sincerely,
    Eric
    MSDN Subscriber Support in Forum
    If you have any feedback of our support, please contact msdnmg@microsoft.com.
    Please remember to mark the replies as answers if they help and unmark them if they provide no help.
    Welcome to the All-In-One Code Framework! If you have any feedback, please tell us.
    Wednesday, August 25, 2010 7:37 AM
  • Hi eryang

     

    Thanks for your efforts. Knowing that this behaviour is 'by design' of Asp.Net and not because we made some coding error should be good enough. So I will focus on reducing memory usage such that private bytes never goes too high.

     

    Best Regards,

    Dominic

    Wednesday, August 25, 2010 7:47 AM