locked
Managing cache connections RRS feed

  • Question

  • In a dev cycle doing some research I plod along trying this and that.  Not paying much attention I created a few DataCache(s) using new DataCacheFactory().GetDefaultCache().

    Things work fine for plodding.

    Time comes to test using multiple (say 6) executions of the same little program and I am throttled for exceeding connections.  Fine, I did this to myself and knew the time would come that I would have to clean this up.  I pull up fiddler to watch a little bit and I can tell that the factory is at least managing the authentication token.  I read some here and there but I can't tell where the connection boundry is(?) and I could run into a situation where I might wish to manage this.

    Say I have 2 instances of WebRole running 3 sites and they all use DistributedCacheSessionStateStoreProvider, I can imagine a pooling scheme where this is 2 connections or I can imagine that I just exceeded my available count of 5 or perhaps DataCache manages it itself and I don't really need to concern myself for the typical case.

    So, being so far from the actual TCP, what (if anything) do I do to manage connections?

    Yes, I have done the cleanup and don't seem to have any problems, but I do wonder.


    Friday, May 27, 2011 8:34 PM

Answers

  • The concept is that each DataCacheFactory keeps a single instance of a particular DataCache. So if you have a Factory A and you do a GetDefaultCache() it will return you DataCache-A instance. Next time when you do a GetDefaultCache() with this factory, it again returns the same DataCache-A instance.

    In case you have another Factory B, when you do a FactoryB.GetDefaultCache() you get DataCache-B instance. Now each DataCache instance has connections of its own to the server. Only when you dispose the factory would those connections go away.

     

    "This means that if you want to use both the cache session provider (which has its own DataCacheFactory internally somewhere) and to use the cache for general use, that is two connections each website is using." - That is true.


    Connection count is currently not on the portal but we would do a refresh some time to depict those there as well.

    • Marked as answer by IAmJohnF Friday, June 3, 2011 3:27 PM
    Friday, June 3, 2011 11:38 AM

All replies

  • This might be helpful.

    http://appfabriccat.com/2011/04/windows-azure-appfabric-caching-service-released-and-notes-on-quotas/

    "Also understanding the amount of connection is essential. A common misinterpretation of this limit is to assume that this is the same as the amount of transactions your web role can handle. The number of DataCacheFactory instances along with your MaxConnectionsToServer is what will determine the amount of transactions you can accumulate. For instance, in the case of 3 web roles utilizing one datacachefactory each and with  maxconnectiontoserver set to 1, there will be only 3 connection been used and they would have many transactions (get/put) going over them – we will not hit any quota limits (the smallest quota limit is 5)."

    This doesn't seem to completely answer your question though. It seems to imply that you're only running 1 website on 1 webrole. If that were the case then the number of connections is equal to the number of instances. Does disposing of the DataCacheFactory instance close the connection?


    http://www.IActionable.com - Gamification and Analytics Platform
    Friday, May 27, 2011 9:39 PM
  • Hi IActionable,

    Thanks for the reply.  Yes, I had read that and, like you, am not sure if it clears things up for me.  It was the motivation behind the 3 site X 2 instance scenario actually.

    In http://social.msdn.microsoft.com/Forums/en-US/windowsazuredata/thread/216a5159-0186-403c-89d8-99058d51a542, Wenchao Zang implies that DataCache creates the connection but perhaps DataCacheFactory hold them.  In fact, there is a suggestion that if we are running into to many connections then we need to dispose the factories.

    128M is huge for my purposes right now but 5 connections seems really small.  I could host service in the WebRole through netPipe so that the sites share a connection but then I am building a distributed service so that I can get to my distributed resource.  That seems at odds.  If I create/dispose factories all the time then I create a contention problem precisely in the high frequency situations in which I am trying to leverage caching.

    Tuesday, May 31, 2011 2:36 PM
  • We seem to have similar issues. 128mb is large enough for us but we need more than 5 connections.

    We also have 3+ websites on a single web role. It would be really nice to know if those connections are per webrole or website. If no one responds, it might take creating an example project with 6 websites in a single role and seeing if any exceptions are thrown.


    http://www.IActionable.com - Gamification and Analytics Platform
    Tuesday, May 31, 2011 4:49 PM
  • The original post mentioned above makes it sound like every instance of the DataCacheFactory holds open a connection to the cache service.  So in the case were you have 3 websites in one web role, because each of the websites would have its own instance of the DataCacheFactory I would think that this would use 3 connections.

    Rather than creating a new DataCacheFactory every time you want to use the cache, all of the samples seem to create one DataCacheFactory in a static variable and use that to create their cache objects, giving you more control over the number of connections you have.  This means that if you want to use both the cache session provider (which has its own DataCacheFactory internally somewhere) and to use the cache for general use, that is two connections each website is using.

    I would love it if someone from MS could clear this up.  I too have no issues with size, but the limited number of connections does mean that we need to use a bigger cache than we strictly speaking need to.  It would be nice to see the number of currently used connections in the portal as well so we could experiment with this stuff without just waiting for stuff to break.

    Tuesday, May 31, 2011 11:52 PM
  • The concept is that each DataCacheFactory keeps a single instance of a particular DataCache. So if you have a Factory A and you do a GetDefaultCache() it will return you DataCache-A instance. Next time when you do a GetDefaultCache() with this factory, it again returns the same DataCache-A instance.

    In case you have another Factory B, when you do a FactoryB.GetDefaultCache() you get DataCache-B instance. Now each DataCache instance has connections of its own to the server. Only when you dispose the factory would those connections go away.

     

    "This means that if you want to use both the cache session provider (which has its own DataCacheFactory internally somewhere) and to use the cache for general use, that is two connections each website is using." - That is true.


    Connection count is currently not on the portal but we would do a refresh some time to depict those there as well.

    • Marked as answer by IAmJohnF Friday, June 3, 2011 3:27 PM
    Friday, June 3, 2011 11:38 AM
  • Hi John,

    Post here : http://blogs.msdn.com/b/akshar/archive/2011/06/03/appfabric-azure-caching-connection-quota-throttling-errors.aspx might help as to how to manage the connections from the cache layer. We don't directly expose the connections for managing them.

    fiddler won't help since Cache uses TCP sockets to connect and communicate whereas fiddler would only show you the HTTP level traffic.

     

    Thanks,

    Akshat

    Friday, June 3, 2011 12:01 PM
  • Hi Akshat,

    Thanks for the response.  I am going to mark your first as the answer since it states: connections are managed by the DataCacheFactory objects, creating a factory and its DataCache object will open connections and disposing the DataCacheFactory (and I assume DataCache object) will close them.  I also gather from your post that the connections are not opened until the DataCache object is created.

    Also, the DistributedCacheSessionStateStoreProvider internally uses a DataCacheFactoy/DataCache.

    If the 3 sites of the scenario I mention each cache session state and also need to to use the cache generally then I can't run a single test instance.  It appears that our scale is 1 redundant site with no VApps. (that would use 4 connections)

    Yes, I was aware that cache used TCP and fiddler wouldn't help to see that, but it did expose that each new DataCacheFactory (perhaps when the DataCache is created) negotiates with ACS.

    For me and I imagine many MSDN subscribers that are taking advantage of Azure's promotion to MSDN subscribers, upgrading isn't just a $10/month increase.  I am not in a place where cost increases make sense.  It looks to me like, to be scalable, I may have to build a remoting interface to the cache and develop my own cache interface for session state so that I can make connections scale "per instance".  This injects marshalling overhead and may kill using the cache all together.

    If I could make a suggestion it would be to make connections a per instance item and have DataCacheFactory(s) share.  This helps you manage dividing the host TCP space between the VMs serving the cache, and helps us scale out.  Perhaps better use of resources.

    Thanks Akshat.


    [In Edit]

    Just a note: IActionable states that they do have a real implementation that matches my scenario.  That scenario will use 12 connections - the 512MB size.  My current use fits easily in 128M (and will use 4 or 6 connections depending on a couple of design choices), I have to look at the pricing by the connection count and not the size.

    Friday, June 3, 2011 3:27 PM
  • "If I could make a suggestion it would be to make connections a per instance item and have DataCacheFactory(s) share. This helps you manage dividing the host TCP space between the VMs serving the cache, and helps us scale out. Perhaps better use of resources." - Thanks John for your valuable suggestion, We'll consider this for our next cycle.

     

    Thanks,

    Akshat

    Saturday, June 4, 2011 5:57 PM
  • I have to chime in as well and state that the current throttling points do not make any sense to me. If I want to build a truly scalable system I need to know the Cache will be there for me as my app grows. As I increase capacity on my end (adding new Compute instances for example) the Cache should automatically scale with it. I should not have to worry about my cache size or connection usage. The fact that I do have to truly hobbles the Caching service if you ask me. It is not truly an elastic service as it stands.

    Given that I think the Caching service should remove any preset throttling limits and just come up with some reasonable billing axis (charge per concurrent connections or transactions/sec, whatever makes sense). Still allow me to define the max cache size so that the cache can throw old stuff out and keep it truly a 'hot cache'. That way my app can just scale naturally and I just pay for what I use. A true Elastic Cloud Service.

    I suppose one could argue that you insert some gross level thresholds like SQL Azure does (Web vs Business editions) where you must opt-in to get higher usage limits and allow the Caching backend people to do capacity planning. But even that is arguable for true elasticity.

    End result for me: The caching service is cool and I really want it, but it is fatally broken as it stands.

     


    Wednesday, July 20, 2011 8:17 PM
  • I am having the similar problem where I have 5 web sites in one web role and hosted with two instances  therefore I need to upgrade the cache size. I am just using around 2.0MB of the cache in total. Upgrading 128MB to 256MB doesn't sounds reasonable. If we can have one connection per instance would be very helpful.
    Boon Sheng
    Sunday, July 24, 2011 3:40 PM