locked
ASP.Net / MemoryCache behaviour for w3wp worker process memory usage RRS feed

  • Question

  • User-2143183981 posted

    Hi,

    Actually I had different question / problem asked  here before but I realized there is something specific needs to be clear.

    I am using simple MVC 5 project for test. w3wp process start with 150 MB and (after calling home page with doing nothing 250 MB) when I call an action to populate cache it increase to around 2 GB.

    Every item has a short time out value and all items timed out after 11 minutes.

    I can see them in performance counters. (firs increase in few seconds and all timed out in around 10 minutes).

    While item is time out it calls Removed event with Expired statu as expected.

    And same thing hapining with ObjectCache and asp.net internal cache too. 

    The problem is worker process memory usage stay stay around 2 GB and never drops till recycle.

    Is there something needs to be done extra to remove items expired or only solution is recycle ? 

    Worker process memory usage stay same if I remove items manually with waiting time out.

    My environment  : IIS 8.5 .Net 4.5 

     string rootKey = DateTime.Now.ToString("hh.mm.ss.ffffff");
    
                string smallText = "small text";
                int smallTextSize = 0;
                for (int i = 0; i < 1000000; i++)
                {
                    smallTextSize += smallText.Length;
                    string key = string.Format("smallkey:{0}.{1}", rootKey, i);
                    this.ControllerContext.HttpContext.Cache.Add(key, smallText, null, DateTime.Now.AddSeconds(i / 1000),
                        System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Default, new System.Web.Caching.CacheItemRemovedCallback(CacheItemRemoved));
                    //MemoryCache.Default.AddOrGetExisting(string.Format("smallkey:{0}.{1}", rootKey, i), smallText, DateTime.Now.AddSeconds(i / 1000));
    
                    System.Diagnostics.Debug.WriteLine("added-smallkey:{0}, date:{1}", key, DateTime.Now.ToString("hh.mm.ss.ffffff"));
                }
    
                System.Threading.Thread.Sleep(2000);
    
                string bigText = "2 K string";
                int bigTextSize = 0;
                for (int i = 0; i < 1000000; i++)
                {
                    bigTextSize += bigText.Length;
    
                    string key = string.Format("bigkey:{0}.{1}", rootKey, i);
                    //default cache
                    this.ControllerContext.HttpContext.Cache.Add(key, smallText, null, DateTime.Now.AddSeconds(i / 1000), 
                        System.Web.Caching.Cache.NoSlidingExpiration, System.Web.Caching.CacheItemPriority.Default, null);
                    //Memory cache
                    //MemoryCache.Default.AddOrGetExisting(string.Format("bigkey:{0}.{1}", rootKey, i), bigText, DateTime.Now.AddSeconds(i / 1000));
                    //memory cache with lock
                    //AddOrGetExistingWithLock<string>(rootKey + "." + i, () => bigText, i);    
    
                    System.Diagnostics.Debug.WriteLine("added-bigkey:{0}, date:{1}", key, DateTime.Now.ToString("hh.mm.ss.ffffff"));
                }

    Friday, September 5, 2014 2:25 AM

All replies

  • User-1454326058 posted

    Hi jandark,

    Based on my test, I can produce that scene, the expired data haven’t been recycled unless the application pool has been recycled.

    Best Regards,

    Starain Chen

    Tuesday, September 9, 2014 1:42 AM
  • User1104055534 posted

    Hi Jandark,

    If your server is a 64-bit system, and the w3wp.exe is a 64-bit process, then  its Virtual Memory address is up to 16 TB, while if it's a 32-bit application, it can still up to 4GB in user mode. If the server is a 32-bit system, the process virtual memory address can up to 4GB, which 2GB split to user mode.

    You can refer to  http://msdn.microsoft.com/en-us/library/windows/desktop/aa366912(v=vs.85).aspx

    http://blogs.technet.com/b/markrussinovich/archive/2008/11/17/3155406.aspx

    You should not mix up the Virtual Memory and Physical Memory (RAM). Virtual memory mappings are made to correspond to actual physical addresses. The operating system creates and deals with these mappings - utilizing the page table, among other data structures to maintain the mappings. Virtual memory mappings are always found in the page table or some similar data structure (in case of other implementations of virtual memory, we maybe shouldn't call it the "page table"). The page table is in physical memory as well - often in kernel-reserved spaces that user programs cannot write over.

    http://stackoverflow.com/questions/14347206/what-are-the-differences-between-virtual-memory-and-physical-memory

    On the other hand, let talk about the .Net GC. Since w3wp.exe is a server side process, it more likely running at GC Server Mode. Not like the other mode. It performs more frequently GC in GEN 0 and 1, but less in GC 2.

    http://blogs.msdn.com/b/clyon/archive/2004/09/08/226981.aspx

    In you case, while objects rooted by cache, they were prompted to GEN 2. Even thought it finally got the timeouted, which mean released from GC rooted. They will be ready to be recycled. However, the server mode perform a less GC in GEN 2, it will take longer to recycle objects in GEN 2. You should keep in mind that, even you release the resource, it doesn't mean it will be release immediately from Virtual Memory.

    So you should the In-Memory runtime cache carefully. You refer to the link : http://msdn.microsoft.com/en-us/library/aa478965.aspx

    Friday, September 12, 2014 11:16 AM
  • User-2143183981 posted

    Hi Starain,

    Thank for your reply. I did not get mail for your reply, sorry for delay.

    I asked same question in 3 different forum but everyone advised me to ask it somewhere else...

    You seem working in MS. Is there any right place to ask this question and get proper asnwer ?

    http://social.msdn.microsoft.com/Forums/vstudio/en-US/17a030a3-4e0f-4316-94b5-dd35f270fe5d/aspnet-memorycache-behaviour-for-w3wp-worker-process-memory-usage?forum=netfxbcl

    http://forums.iis.net/t/1216258.aspx?ASP+Net+MemoryCache+behaviour+for+w3wp+worker+process+memory+usage

    Thursday, September 18, 2014 7:07 AM
  • User-2143183981 posted

    Hi gtscdsi

    Thank you for your reply.

    Ok I understood 32 and 64 bit process difference, Virtual Memory and Physical Memory, GEN 0, 1, GC 2 differences too.

    But there is still a problem. I should be careful while using cache but how ?

    I waited lots of hours GEN 2 does not seem doing its job.

    It is happening in "desktop" / "console" application too with ObjectCache (not asp.net in memory cache).

    I want to emphasize on "desktop" and ObjectCache.

    Here you can see interesting dialog as I state lots of times it is happening in desktop application too but got an answer that maybe it is because of IIS worker process.

    http://social.msdn.microsoft.com/Forums/vstudio/en-US/17a030a3-4e0f-4316-94b5-dd35f270fe5d/aspnet-memorycache-behaviour-for-w3wp-worker-process-memory-usage?forum=netfxbcl

    Finally maybe I (and every .net developer) should understand that we can not rely Net framework related cache libraries....

    Thursday, September 18, 2014 7:18 AM