Azure Table Storage, How to avoid connection error (I tried retry)

Question

I asked this question at StackOverflow but cannot get proper answer

http://stackoverflow.com/questions/9859286/azure-table-storage-how-to-avoid-connection-error-i-tried-retry

To be clear, I have lots of quires so got lots of errors, in less than 1% but still  annoying

I got lots of errors like No connection could be made because the target machine actively refused it 70.37.127.112:443 (fyi, I tried :80 as well)

So I add Microsoft.Practices.TransientFaultHandling to retry.

But still, lots of errors (even after retry)

I like to know:

1st, since Azure is a distributed service, why am I always reach 70.37.127.112 ( I access storage by simple original DefaultEndpointsProtocol=https;AccountName=storage_account;....., then use tableContext.CreateQuery<datatype>("table_name"); ). I noticed all my tables (even in different Storage account) all hit 70.37.127.112. My question, is there a way to change it?

2nd, how to eliminate the error above totally?

error details

 System . Net . WebException :   Unable  to connect to the remote server  --->   System . Net . Sockets . SocketException :   No  connection could be made because the target machine actively refused it  70.37 . 127.112 : 443 
   at  System . Net . Sockets . Socket . DoConnect ( EndPoint  endPointSnapshot ,   SocketAddress  socketAddress ) 
   at  System . Net . ServicePoint . ConnectSocketInternal ( Boolean  connectFailure ,   Socket  s4 ,   Socket  s6 ,   Socket &  socket ,   IPAddress &  address ,   ConnectSocketState  state ,   IAsyncResult  asyncResult ,   Int32  timeout ,   Exception &  exception ) 
    ---   End  of inner exception stack trace  --- 
   at  System . Net . HttpWebRequest . GetResponse () 
   at  System . Data . Services . Client . QueryResult . Execute () 
   at  System . Data . Services . Client . DataServiceRequest . Execute [ TElement ]( DataServiceContext  context ,   QueryComponents  queryComponents ) 
   at  System . Data . Services . Client . DataServiceQuery ` 1.Execute () 
   at  System . Data . Services . Client . DataServiceQuery ` 1.GetEnumerator () 
   at  System . Linq . Enumerable . ToList [ TSource ]( IEnumerable ` 1  source ) 
   at  Microsoft . Practices . TransientFaultHandling . RetryPolicy .<> c__DisplayClass1 .< ExecuteAction > b__0 () 
   at  Microsoft . Practices . TransientFaultHandling . RetryPolicy . ExecuteAction [ TResult ]( Func ` 1  func ) 

Answer 1

Hi,

I would like to know whether you have waited for a few seconds and then retry when those errors occur. From my experience, in most cases, it will then work fine. If you’re issuing more than 500 requests to the same partition every second, some requests may get throttled. The target is to support 500 requests per second per partition. For each account, we are allowed to issue 5000 requests every second. Please refer to http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx for more information.

As for why you see a single IP address, in Windows Azure, our requests are sent to load balancers, which in turn forward the requests to internal servers whose IP addresses are not known to us.

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 2

default retry is wait 1 second, retry 3 times. Some may become ok but not all, lets say around 10% retry also failed.

It's not even close to 500 requests per partition per seconds or 5000 per account.

Answer 3

From your billing page, I had 209.65 10,000s till today to Azure storage transactions

Thats less than 1 per second, and even speared on several account, plenty of partition.

Answer 4

Hi,

1 second may not be enough. Please try to wait for 3 seconds and then request again. If that still fails, please wait for another 7 seconds and give it another try. You can find the SLA document here: http://www.microsoft.com/download/en/details.aspx?displaylang=en&id=6656.

 

Table Query List Operations

Must complete processing or return a continuation within 10 seconds

SLA states if you wait for 10 seconds, the second request must succeed (if not due to throttling). If it still fails, you can contact our customer support on https://support.microsoft.com/oas/default.aspx?prid=14234&st=1&wfxredirect=1&sd=gn to get further help.

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 5

I changed to 1,3,5 seconds, still lots of errors

please check my code here, and advice

http://stackoverflow.com/questions/9904588/whats-the-correct-code-for-microsoft-practices-transientfaulthandling-retrypolic

Answer 6

Is this a recent problem and are you in North-Central US?

---

Auto-scaling & monitoring service for Windows Azure applications at http://www.paraleap.com

Answer 7

yes, from awhile ago, still lots

App in Azure south central, and table storage should to there as well.

Answer 8

Hi,

If you continue to receive error after several retries, you can capture the “x-ms-request-id” header found in a failed response, and send it to our customer support: https://support.microsoft.com/oas/default.aspx?gprid=14928&st=1&wfxredirect=1&sd=gn. A support guy will use the request ID to trace down the issue.

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework

Answer 9

Hi,

I will mark the reply as an answer. If you find it no help, please feel free to unmark it and follow up.

Thanks.

Best Regards,

Ming Xu.

---

Please mark the replies as answers if they help or unmark if not.
If you have any feedback about my replies, please contact [e-mail].
Microsoft One Code Framework