Azure Cache Preview - Premature absolute expiry

Question

We're running Azure Cache (Preview) in co-located mode and we've been noticing that sometimes items can not be found in the cache or expires prematurely.

To understand my problem better have a look at the following output from our logs

2012-07-26 03:14:15,093 [13] DEBUG XXX.CacheBase - Update: XXX-CacheKey-XXX | Abs Exp: 07/26/2012 04:15:15 +00:00 | Sli Exp: 00:00:00
2012-07-26 03:14:17,452 [22] DEBUG XXX.CacheBase - PessimisticFetch - XXX-CacheKey-XXX
2012-07-26 03:24:32,229 [23] DEBUG XXX.CacheBase - PessimisticFetch - XXX-CacheKey-XXX
2012-07-26 03:24:32,245 [23] DEBUG XXX.SomeClass - Item not found in cache: XXX-CacheKey-XXX 

Explination:

1. Line 1: Insert into cache with absolute expiry, notice the absolute ex
2. Line 2: Minute or so later a fetch - A OK
3. Line 3: 10 Minute's or so later an attempt to fetch
4. Line 4: Item reported not found in cache

In short we use GetAndLock / PutAndLock to store these specific items. These items have an absolute expiry (1 Hour) but anything from 5 -- 10 minutes later the item is not in the cache any more.

What is more disturbing on my CacheItemPolicy I've setup a remove notification, but this never gets triggerd

//---
cacheItemPolicy.RemovedCallback = ItemRemoved;
//---    

static void ItemRemoved(CacheEntryRemovedArguments args)
{
    _log.DebugFormat("Item evicted from cache: {0}. Reason: {1}", args.CacheItem.Key, args.RemovedReason);
}

I also attached screen shot of our cache setup See Image -> http://i.stack.imgur.com/4ORQQ.jpg

I've ran out of ideas what it could be... any leads explanations, WTF's would is appreciated :)

Answer 1

Hello Barend,

Regarding the notifications part, I see that the notifications aren't enabled from the image you shared? You need to enabled notifications for the cache on which you are interested in notifications. And while creating the cache factory did you configure Notifications policies?

Can you share a sample repro application for this issue.

Thanks

Answer 2

Can you please share your code snippet where you insert items in the cache, with an expiry of 1 hour?

Also, it is possible that items are getting removed from the cache because there is not enough memory available. To check this, you can monitor the Performance counter category 'Appfabric Caching: Host' for eviction related counters (Total Eviction Runs, Total Memory Evicted, Total Evicted Objects) and also the 'Total Expired Objects'. You can either do this by logging into the remote VM and opening up perfmon, or by configuring diagnostics to upload these counters to azure storage (Link).

Answer 3

After a long winded battle we figured it out.

Somehow Visual Studio added a specific assembly binding to our web.config this assembly referred to the wrong version.

   <runtime>
     <assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
-      <dependentAssembly>
-        <assemblyIdentity name="Microsoft.ApplicationServer.Caching.Core" publicKeyToken="31bf3856ad364e35" culture="neutral" />
-        <bindingRedirect oldVersion="0.0.0.0-1.0.0.0" newVersion="1.0.0.0" />
-      </dependentAssembly>
     </assemblyBinding>
   </runtime>

Removing this solved a whole heap of issues, from unexplained CPU usage to premature expiry.