none
Azure Redis Cache Timeouts RRS feed

  • Question

  • Since today morning we are seeing several of these timeouts with redis cache (North Central US Datacenter).  How can we troubleshoot further?

    System.TimeoutException: Timeout performing GET <OneofKeys>, inst: 2, queue: 11, qu=0, qs=11, qc=0, wr=0/0, in=65536/0 at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\ConnectionMultiplexer.cs:line 1754 at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor`1 processor, ServerEndPoint server) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\RedisBase.cs:line 79 at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) in c:\TeamCity\buildAgent\work\58bc9a6df18a3782\StackExchange.Redis\StackExchange\Redis\RedisDatabase.cs:line 1346 at

    Tuesday, July 8, 2014 7:22 PM

All replies

  • We are working with Siva offline on this. Will update thread on resolution.

    Program Manager Azure


    Tuesday, July 15, 2014 6:54 PM
  • Saurabh

    I'm having the same problem in Brazil.

    Is there any issue going on?

    Thanks a lot!


    Thursday, July 31, 2014 4:02 PM
  • Hi Saurabh,

    Any good news? we also have the same problem on our production.

    {"Exception":"Timeout performing GET Setting:SettingScopeType=0-SettingScopeValue=-SettingName=Emails.ConfirmationEmail.Url, inst: 1, queue: 2, qu=0, qs=2, qc=0, wr=0/0, in=181/0\r\nStackTrace\r\n at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor`1 processor, ServerEndPoint server)

    Thks very much.
    ZhangPeng Chen

    Wednesday, August 27, 2014 12:02 PM
  • Hi,
    I've got the same problem in North Europe, posted my request last week and nobody from Microsoft was able to provide a solution,

    does any one resolve this issue?

    Please to find out

    Rgds

    Monday, September 1, 2014 7:30 PM
  • Folks can you provide more context on when you are seeing the timeouts?

    Does it only happened under load? Or are you seeing it in low load conditions as well?

    Also are folks using the 250 MB cache or are you seeing this for bigger caches as well?


    Program Manager UI Framework Team

    Wednesday, September 3, 2014 5:26 PM
  • Folks can you provide more context on when you are seeing the timeouts?

    Does it only happened under load? Or are you seeing it in low load conditions as well?

    Also are folks using the 250 MB cache or are you seeing this for bigger caches as well?


    Program Manager UI Framework Team

     Several 1GB caches, happens under load, but not much load, the timeouts make it worse because we keep retrying.

    Saturday, September 6, 2014 3:44 AM
  • Folks can you provide more context on when you are seeing the timeouts?

    Does it only happened under load? Or are you seeing it in low load conditions as well?

    Also are folks using the 250 MB cache or are you seeing this for bigger caches as well?


    Program Manager UI Framework Team

     Several 1GB caches, happens under load, but not much load, the timeouts make it worse because we keep retrying.

    We have tried redis servers in all 4 us datacenters doing the same thing
    Saturday, September 6, 2014 3:47 AM
  • Folks can you provide more context on when you are seeing the timeouts?

    Does it only happened under load? Or are you seeing it in low load conditions as well?

    Also are folks using the 250 MB cache or are you seeing this for bigger caches as well?


    Program Manager UI Framework Team

    You can see that the usage and CPU don't even correlate correctly. 1Gb cache, this one being in westus

    Saturday, September 6, 2014 7:19 PM
  • hey,

    we have exactly the same exception on our self hosted Redis under Azure. 

    Tuesday, September 9, 2014 8:16 AM
  • Can you send me the name of your cache?

    Program Manager Azure


    Tuesday, September 9, 2014 5:10 PM
  • Can you send me the name of your cache?

    Program Manager Azure


    do you have an email? I have 10 1GB redis instances at the moment, all of them with high cpu at some point
    • Edited by EV_Anonymous Tuesday, September 9, 2014 10:09 PM
    Tuesday, September 9, 2014 9:59 PM
  • Hi,

    We have implemented the "Redis Cache" before 1 week ago and since today evening we are facing the Timeout error as below:

    "Timeout performing EVAL, inst: 15, mgr: ExecuteSelect, queue: 1, qu=0, qs=1, qc=0, wr=0/0, in=0/0"

    This was working fine this issue is being created since today evening.

    Could you please suggest for the above?

    Thursday, September 18, 2014 3:24 PM
  • Same problem on 250MB Cache Redis Azure , Timeout performing GET . West . This happens just under load, I mean concurrency. I just Replicated every time 7 threads every 2 seconds.
    Thursday, September 18, 2014 10:01 PM
  • We have identified and fixed two issues reported in this thread.

    First, there was an issue that caused high CPU usage on the replica instance of a Standard Redis cache.  This should not have impacted the performance of your cache, however it did cause the CPU Usage reported in the portal to be high.  This should now be fixed for all caches.

    Second, there was an issue in the StackExchange.Redis client library that would sometimes cause the client to create a very large number of connections to the server.  This could make your cache either unavailable or very slow.  We recommend all customers upgrade to StackExchange.Redis version 1.0.333 or later.


    Program Manager Azure


    Friday, September 19, 2014 11:53 PM
  • Hi Saurabh,

    I have upgrade the DLLs as per your above suggestion but we are still facing the same issue.

    For your reference :

    PM> Install-Package StackExchange.Redis
    Attempting to resolve dependency 'Microsoft.Bcl (= 1.1.9)'.
    Attempting to resolve dependency 'Microsoft.Bcl.Build (= 1.0.14)'.
    Attempting to resolve dependency 'Microsoft.Bcl.Async (= 1.0.168)'.
    'StackExchange.Redis 1.0.333' already installed.
    Ellerfield.Ellercom.Web already has a reference to 'StackExchange.Redis 1.0.333'.

    i.e "Timeout performing EVAL, inst: 15, mgr: ExecuteSelect, queue: 1, qu=0, qs=1, qc=0, wr=0/0, in=0/0"

    Please suggest for the above asap.

    Monday, September 22, 2014 11:19 AM
  • themsteam2013,

    If you are still having problems please contact us at 'azurecache@microsoft.com'.  Send the name of your cache and the time (including time zone) of the most recent exception.

    Thursday, September 25, 2014 12:24 AM
  • I receive same timeout error, when I have a static class e put a static IDatabase... I can set the cache ok, but when I trying receive, timeout. I tried to open a new connection and not use the static IDatabase and works...
    Saturday, September 27, 2014 6:58 PM