none
Storage Client 1.1 (Azure SDK 1.3) throws condition failure on 200 OK Response

    Question

  • Request:

    GET http://abdullin.blob.core.windows.net/tc-18fd1edb-f86c-4559-8dad-e3bb16f4b650/c065a4d6-b04b-4e45-b3b3-547368ad0c8d?comp=blocklist&blocklisttype=Committed&timeout=90 HTTP/1.1
    x-ms-version: 2009-09-19
    User-Agent: WA-Storage/6.0.6002.18006
    If-Match: *
    x-ms-date: Thu, 02 Dec 2010 09:23:30 GMT
    Host: abdullin.blob.core.windows.net
    Connection: Keep-Alive
    

    Response:

     

    HTTP/1.1 200 OK
    Transfer-Encoding: chunked
    Content-Type: application/xml
    Last-Modified: Thu, 02 Dec 2010 09:22:35 GMT
    ETag: 0x8CD6019974EDFF3
    Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
    x-ms-request-id: f7911dfc-8418-49d8-b16c-12e220f36bea
    x-ms-version: 2009-09-19
    x-ms-blob-content-length: 16
    Date: Thu, 02 Dec 2010 09:23:26 GMT
    
    AA
    <?xml version="1.0" encoding="utf-8"?><BlockList><CommittedBlocks><Block><Name>MD5/sz0MglDOd3ZjtvURKlFLlw==</Name><Size>16</Size></Block></CommittedBlocks></BlockList>
    0
    

    At this point Storage Client throws this:

    Microsoft.WindowsAzure.StorageClient.StorageClientException was caught
     Message=The conditionals specified for this operation did not match server.
     Source=Microsoft.WindowsAzure.StorageClient
     StackTrace:
     at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.get_Result()
     at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.ExecuteAndWait()
     at Microsoft.WindowsAzure.StorageClient.TaskImplHelper.ExecuteImpl[T](Func`2 impl)
     at Microsoft.WindowsAzure.StorageClient.BlobReadStream.Read(Byte[] buffer, Int32 offset, Int32 count) 
    

    Any clue why exception shows up on a 200 OK response? Am I missing something?

    StorageClient is closed source, so I can't really tell what's going inside there or how to fix it.


    http://abdullin.com
    Thursday, December 2, 2010 9:31 AM

Answers

  • Looks like I did have it right.  If you're curious, AccessCondition.VerifyConditionHolds has a bug where it's doing a direct equality comparison between the If-Match value you passed in ("*") and the ETag that comes back from the server, instead of understanding that the asterisk just means any ETag is valid.  A bug has been filed, so we should see a fix for this next time we rev the library.

    For now, I would just avoid using If-Match: * with OpenRead.  (OpenRead should fail if the blob doesn't exist anyway.)  It looks like OpenRead is the only method that ends up using AccessCondition.VerifyConditionHolds, so I don't think anything else is impacted by this bug.

    Sure, I'll take a look at the other failures you've found with your tests and pass those along too.

    • Marked as answer by Mog Liang Thursday, December 9, 2010 8:51 AM
    Friday, December 3, 2010 6:36 AM

All replies

  • I can't reproduce this... can you share your code?  I tried:

          foreach (var block in account.CreateCloudBlobClient().GetBlockBlobReference("5c6a7dac-df67-4922-a95b-c453dc97f389/FriendsShareStuff.zip").DownloadBlockList(new BlobRequestOptions { AccessCondition = AccessCondition.IfMatch("*") }))
          {
            Console.WriteLine(block.Name);
          }
    
    
    And saw similar HTTP traffic as what you had, but I didn't get any exception.

    Thursday, December 2, 2010 8:07 PM
  • Check out latest version of Lokad.CQRS (either codeplex or google code would work). I've just pushed the latest code there.

    Unit test (you will need to specify Azure connection string in BlobStorage) is:

    Lokad.Cqrs.Tests.Storage.BlobStorage+When_reading_blob_item.When_reading_item_in`1.Valid_item_and_valid_match_wild_return:
    
    Lokad.Cqrs.StorageConditionFailedException : Storage condition 'IfMatch '*'' failed for 'http://abdullin.blob.core.windows.net/tc-94d14ef6-dee9-47ba-8e42-c83cdfb00f2d/c2f0e663-1f90-48c5-9eb0-3ed66b89d52e'
    
     ----> Microsoft.WindowsAzure.StorageClient.StorageClientException : The conditionals specified for this operation did not match server.

    Full Stack trace should look like:

    at Lokad.Cqrs.Storage.BlobStorageItem.ReadInto(ReaderDelegate reader, StorageCondition condition) in BlobStorageItem.cs: line 105
    at Lokad.Cqrs.Tests.Storage.StorageItemFixture`1.ShouldHaveGuid(IStorageItem storageItem, Guid g, StorageCondition condition) in StorageItemFixture.cs: line 99
    at Lokad.Cqrs.Tests.Storage.When_reading_item_in`1.Valid_item_and_valid_match_wild_return() in When_reading_item_in.cs: line 138
    --StorageClientException
    at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.get_Result()
    at Microsoft.WindowsAzure.StorageClient.Tasks.Task`1.ExecuteAndWait()
    at Lokad.Cqrs.Storage.SuppressFlushForStream.Read(Byte[] buffer, Int32 offset, Int32 count) in SuppressFlushForStream.cs: line 139
    at System.Security.Cryptography.CryptoStream.Read(Byte[] buffer, Int32 offset, Int32 count)
    at Lokad.Cqrs.Tests.Storage.StorageItemFixture`1.<>c__DisplayClass6.<ShouldHaveGuid>b__5(StorageItemInfo properties, Stream stream) in StorageItemFixture.cs: line 102
    at Lokad.Cqrs.Storage.BlobStorageUtil.<>c__DisplayClass4.<Read>b__1(Stream s) in BlobStorageUtil.cs: line 88
    at Lokad.Cqrs.Storage.BlobStorageUtil.ReadAndVerifyHash(Stream stream, Action`1 reader, String hash) in BlobStorageUtil.cs: line 51
    at Lokad.Cqrs.Storage.BlobStorageUtil.Read(BlobRequestOptions mapped, CloudBlob blob, ReaderDelegate reader) in BlobStorageUtil.cs: line 88
    at Lokad.Cqrs.Storage.BlobStorageItem.ReadInto(ReaderDelegate reader, StorageCondition condition) in BlobStorageItem.cs: line 87 

    NB: you can ignore Azure Blob unit test failures with conditions in missing container scenarios. That's RFC in-compliances that were discussed earlier (and that I doubt will ever be fixed).

    NB2: Ever considered uploading storage client PDBs to the symbol server? 


    http://abdullin.com
    Thursday, December 2, 2010 8:21 PM
  • I downloaded and built the project, but I can't quickly figure out how to run the tests.  Can you help?
    Thursday, December 2, 2010 10:00 PM
  • Also, I couldn't find any reference to DownloadBlockList in the code.  Are you using some different API?  Could you tell me where in your code you're seeing this exception?
    Thursday, December 2, 2010 10:03 PM
  • The used API is basically the BlobStream class, as in CloudBlob.OpenRead/OpenWrite.

    Thanks for looking into it,
    Chris

    Friday, December 3, 2010 2:21 AM
  • A file name or method name from Lokad.CQRS or just pasting some code here that demonstrates the problem would help me out.
    Friday, December 3, 2010 4:25 AM
  • Method within Lokad.CQRS that throws the exception: Lokad.Cqrs.Storage.BlobStorageItem.ReadInto (more details in the stack trace above).

    Easiest way to reproduce:

    1. Put your Azure Storage credentials as string into D:\Environment\Azure.blob.test
    2. Run this test in Resharper Test Runner, NUnit or whatever fits you most: Lokad.Cqrs.Tests.Storage.BlobStorage+When_reading_blob_item.When_reading_item_in`1.Valid_item_and_valid_match_wild_return
    Here's the screenshot:

    http://abdullin.com/storage/uploads/2010/12/2010-12-03_093351.jpg

    Here's how everything should look like with the debugger attached. Output in Fiddler2 will match to what I posted earlier.

    http://abdullin.com/storage/uploads/2010/12/2010-12-03_093729.jpg

    Does this help?

     


    http://abdullin.com
    Friday, December 3, 2010 4:38 AM
  • Okay, this helped a lot.  I installed NUnit and was able to reproduce your unit test failure.  I then wrote a small repro of my own:

          var blob = account.CreateCloudBlobClient().GetBlobReference("5c6a7dac-df67-4922-a95b-c453dc97f389/FriendsShareStuff.zip");
          using (var stream = blob.OpenRead(new BlobRequestOptions { AccessCondition = AccessCondition.IfMatch("*") }))
          {
            var bytes = new byte[100000000];
            Console.WriteLine(stream.Read(bytes, 0, bytes.Length));
          }
    
    
    If I remove the AccessCondition, this works, but with it, I get an exception at the same place you do (before the actual read has begun).  I'm skimming the StorageClient code in Reflector in case I can spot something, but I'll probably have to send this over to the storage team to understand why this generates an exception.

    Friday, December 3, 2010 5:32 AM
  • I think I've spotted a bug in the StorageClient library... I'm pinging the storage team now, and we'll see if I'm right.  It seems to be a very specific bug about using OpenRead with an If-Match: * condition.
    Friday, December 3, 2010 5:45 AM
  • That's terrific, thank you! Please, keep me updated.

    NB: is there any chance we can also "fix" the other few tests by bringing Azure BLOB behavior on errors with conditions in accordance with the RFC standards?


    http://abdullin.com
    Friday, December 3, 2010 5:53 AM
  • Looks like I did have it right.  If you're curious, AccessCondition.VerifyConditionHolds has a bug where it's doing a direct equality comparison between the If-Match value you passed in ("*") and the ETag that comes back from the server, instead of understanding that the asterisk just means any ETag is valid.  A bug has been filed, so we should see a fix for this next time we rev the library.

    For now, I would just avoid using If-Match: * with OpenRead.  (OpenRead should fail if the blob doesn't exist anyway.)  It looks like OpenRead is the only method that ends up using AccessCondition.VerifyConditionHolds, so I don't think anything else is impacted by this bug.

    Sure, I'll take a look at the other failures you've found with your tests and pass those along too.

    • Marked as answer by Mog Liang Thursday, December 9, 2010 8:51 AM
    Friday, December 3, 2010 6:36 AM
  • And thanks for catching this and pointing it out!
    Friday, December 3, 2010 6:36 AM