SDK 1.5 DevStorage deadlocking on parallel uploads

Question

When using the Blob API of the 1.5.20830.1814 SDK (VS 2010 Tools 1.5.40909.1602), we are experiencing frequent deadlock scenarios when uploading blobs in parallel (typically, invoked via Task.Factory.Start wrappers that ultimately call the synchronous UploadFromStream method). Here is an example stack trace from DevStorage:

3/13/2012 11:54:45 PM [Error_debug] Exception raised during processing of request. Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.FatalServerCrashingException: The fatal unexpected exception 'Transaction (Process ID 55) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.' encountered during processing of request. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 55) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
   at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.EndPerformOperation(IAsyncResult ar)
   at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer`1.<ProcessImpl>d__4.MoveNext()
   --- End of inner exception stack trace ---

According to a previous post from back in the 1.2 days [1], this was reportedly to be fixed in "a future version"...any ideas on why we're still able to cause this?

[1]: http://social.msdn.microsoft.com/Forums/en-US/windowsazuredata/thread/7ac51103-3d41-483d-84a1-0d70dfa1177e

Answer 1

Hi,

I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay.
 
Appreciate your patience.

---

Please mark the replies as answers if they help or unmark if not. If you have any feedback about my replies, please contact [e-mail] Microsoft One Code Framework

Answer 2

Thank you - we'll be patiently waiting.

Cheers!

Answer 3

I don't find any open bugs on devStorage parallel uploads causing SQL deadlocks. So I think you might get something different. The latest Azure SDK is 1.6 . Would you upgrade and see if it avoids the deadlock?

If you have to stick to 1.5, or find 1.6 still gets the deadlock, check out http://msdn.microsoft.com/en-us/library/ms178104.aspx which talks about how to detect , trace and end SQL Server deadlocks. You might find something to do on the database server side. 

If all these does not help you out, I suggest you to raise a service ticket from http://www.windowsazure.com/en-us/support/contact/. Provide the code that you use to trigger the issue, and the SQL Server trace files that you've collected following the SQL doc above.  The Azure technical support team will help to troubleshoot.  Once the rootcause and solution is identified, please come back to this thread and share that out. Other people can benifit from this.

-Emma

Answer 4

Thank you for the response. As a first step, we are going to do the following:

1) Isolate the repro-case against the installed 1.5 SDK.

2) Re-try against the 1.6 SDK.

I'll post the results back here to let everyone know what we found, and whether we're moving on to opening a formal support ticket.

Thanks!!!