locked
ASP .Net 2.0 IIS6 W2k3 SP2 64Bit Cache Vs Memory Release - Architecture Trade Off RRS feed

  • Question

  • Hi Team,

    Hope this is the right place to discuss the architecture for performance against problems  - trade off.

    We have a Web App (Environment as subj), we use Caching , state server, we store 1MB-8MB of files in cache, we retrieve and show in the web pages.

    I refer this URl for caching: http://social.msdn.microsoft.com/Forums/en-US/architecturegeneral/thread/ef0f623b-55f8-4742-b39b-78880f2dd5df

    Wow the performance is good. But then the real issue is,

    1. The w3wp.exe grows and grows for fewer user base and it never release memory. It cause the APP pool to reach recycle limit (4 times a day :( not acceptable)
        because of reading file(IO) and memory ,
        http://support.microsoft.com/default.aspx/kb/916984
        Does any body this KB fix applicable to ASP .net 2.0 ? Because we are also handling Binary Files and Memory. (related to store/retrieve from cache)
        Caching of bigger files recommended?
       
    2. I also referred the following for similar problems.
        And we tried playing for gcServerEnabled and gcConcurrent tags. still no better
       http://www.smartertools.com/forums/p/16716/44530.aspx#44530
       http://www.clubfarpoint.com/Forums/forums/thread/54117.aspx


    And any body have similar expertise?

    also see this it claims actually the problem is not fixed by Microsoft for ASP .Net

    http://techrepublic.com.com/5208-11184-0.html?forumID=57&threadID=187879&messageID=1925636&tag=content;leftCol


    Any kind of pointers welcome,


    Thanks

    sahridhayan


    Wednesday, November 11, 2009 3:44 AM

All replies

  • You mean to say that when you stored your session on a state server running on a 64 bit machine, the performance increased but the memory consumption and leakage on state server also increased.
    Was this not the case when you were using 32 bit machine for state server.

    I am asking this because you talked about state server which is meant to store session and not cache.

    Cheers
    Thompson
    Wednesday, November 11, 2009 4:14 AM
  • Thanks for writing back,

    1. The performance for 32 Bit machine is still same. (it is worst than this), but still the recycle limit problem does not get solved.

    2.  Yes, State server part and cache part are two different aspects of the application


    Let me know you need any more information,


    Thank you,

    Sahridhayan
    Wednesday, November 11, 2009 4:28 AM
  • Hi,

    So your problem didn't got solved by 64 bit machine (:
    Here are some of the steps which can be taken here:-

    1. Please do code profiling and memory profiling.See for large object size.Reduce them to smaller size.Check if it stays in memory for long.if yes,why and is there any way by which this can be released.comment the faulting method ,check memory usage and refactor code accordingly.
    2. Reduce Session size to below 1 MB.
    3. I hope, you have already implemented /3GB option in boot.ini
    4. Check the size of state and cache which u r trying to store on state server. Is its size chocking the network.what is ur network bandwidth inside server center like 100 Mbps,1Gbps.chech bytes sent received performance counter on webserver and database server.if it is exceeding the network limit, it can create bottleneck and inturn would send cpu and memory of webserver high.
    5. I am sure, you assigning proper memory to app pools in 64 bit machine.

    I am not sure how much it will help you but it should definitely help to a bit till somebody from microsoft helps out here

    Cheers
    Bhaskar Sharma
    --------
    If it helps you in some way, please mark this as answer,though it doesn't restrict you from discussing further
    Wednesday, November 11, 2009 5:24 AM
  • There are a couple of potential problems here - the most obvious one is that you are probably not being very efficient with the way that you are transferring data, are you loading up all the data in the ASP page, and filtering it there - something like that?

    Second, and I don't know how you've implemented your caching - what cache expiry policy are you using?  Is it just expiring over time, and being reloaded, or never expiring?  Sounds like it is never expiring to me, so the process is consuming more memory.  You need to get it to expire, and then decide the best way to reload the solution.  I'm not surprised that its fast if you have the whole database in memory!

    Is then the full cache of data being sent over the wire, and thus causing your problems?

    Please ask more questions, and I'll try to answer,

    Thanks,

    Martin.
    MCSD, MCTS, MCPD. Please mark my post as helpful if you find the information good! http://www.consultantvault.com
    Friday, November 13, 2009 9:48 AM
  • There are a couple of potential problems here - the most obvious one is that you are probably not being very efficient with the way that you are transferring data, are you loading up all the data in the ASP page, and filtering it there - something like that?

    [Raja]
    The aspx page shows only page of records whereever there record listing and the dataset travels back and forth carries only needed records example only modified and added records are traversing to server.

    Paging implementation is discussed here : (real paging!)
    http://aspspider.info/sahridhayan 

    and discussed over here
    http://stackoverflow.com/questions/1061574/custom-paging-or-standard-paging-in-asp-net-which-method-is-efficient


    Second, and I don't know how you've implemented your caching - what cache expiry policy are you using?  Is it just expiring over time, and being reloaded, or never expiring?  Sounds like it is never expiring to me, so the process is consuming more memory.  You need to get it to expire, and then decide the best way to reload the solution.  I'm not surprised that its fast if you have the whole database in memory!

    [Raja] We are using sliding expiration for cache, and what we cache is not related to database data, it is template (related to page,usercontrols rendering) files we load dynamically.

    Is then the full cache of data being sent over the wire, and thus causing your problems?

    [Raja] I am not sure about this, sent over the wire between layers you mean ?


    Please ask more questions, and I'll try to answer,

    Thanks,

    Martin.
    MCSD, MCTS, MCPD. Please mark my post as helpful if you find the information good! http://www.consultantvault.com

    Monday, November 16, 2009 2:33 AM
  • Try to replace dataset with custom entity objects to reduce the size of object been transferred.

    Cheers

    Bhaskar

    Monday, November 16, 2009 5:57 AM
  • Didn't read paging implementation - so long as you're paging there's no reason to believe that that would be the cause.

    How much data is actually being pushed backwards and forward over the wire?  I think you should be able to use some tools like wireshark or fiddler to work these things out easily enough.

    The cache - based on data on the page - okay, so where does the cache live?  I mean what layer is it in?  Is if on a different tier (machine)?

    I am just trying to figure it out - the data is cached due to what is displayed on the page, so the cache would be populated based on historical page hits, in effect?  Are there a lot of pages, and do those pages have any overlap in data?  The reason I ask is, you're really wanting to avoid sending data over the wire, to avoid the overhead, and definitely wanting to avoid database queries, again due to overhead.  What I am then trying to work out is, if you are caching at that level whether you will potentially be caching the same data a number of times, since each page can potentially contain the same data?  I don't know if it would help, but you might want to use a profiler to see whether that is the case or not?

    If you are using a dataset, there is most definitely going to be an overhead of data going back and forth, that could be reduced by using your own objects, and how much depends upon how much changes between pages.  You could also possibly as a way of troubleshooting, try stripping out the dataset on one page, and seeing if that makes a difference at all to anything?  You'd probably have to put it under load to notice any differences though.

    The sliding window for expiration - what is this set at?  Does the window expiring too slowly mean that data is being kept in the cache for too long, and is building up in size?  Potentially if there is also redundant data being stored as mention above, this could potentially make the cache a lot bigger than it needs to be.

    The big thing with caching is really understanding how caching affects your application, what is cached, how long it is cached for, what happens if it is cached and the data is old, and so on.  This shouldn't have any real impact no matter what caching technology you use - I'm assuming you're using MemCached, although the url that pointed to caching didn't appear to say.  I think that you are likely to come to your own answers once you fully understand the implications of caching.

    One question - if you switch the cache off - I assume that you can do, does the problem still occur?
    If you either short-circuit the cache, or set it's expiration to immediate (not as good as the cache is still involved, just not actively) what impact does that have?

    Hope these idea give you some places to look to find out what the problem is,

    Thanks,

    Martin.
    MCSD, MCTS, MCPD. Please mark my post as helpful if you find the information good! http://www.consultantvault.com
    Monday, November 16, 2009 6:24 AM
  • AT the time of page rendering we can either regenerate the page or use information already stored in valid cache. We can validate a cache page through our asp.net code behind. We can decide whether to use a cached version of a page or simply render it as fresh new page. For that we have to define the event handlerHttpCacheValidateHandler delegate in page load and set one of the enumeration value of HttpValidationStatus in corresponding method.

    The three possible values are :

    1.Invalid: Consider cache page as invalid and remove it. Finally generate the page

    2.IgnoreThisRequest: For this page is regenerated without invalidating or overwriting cache page.

    3.Valid: This means cache page is valid. So, without regenerating the page again cache is returned

     


    Cheers, Eliza
    Friday, February 12, 2010 10:41 AM