New Windows Azure Blob Features - August 2009

Question

Today we are releasing a new set of features for Windows Azure Blob.    Windows Azure Blob enables applications to store and manipulate large objects and files in the cloud.  The blobs (files) can be up to 50GB in size for the CTP.  

 

All changes for this release are versioned changes, using “x-ms-version: 2009-07-17”.  All prior versions of commands executed against the storage system will continue to work, as we extend the capabilities of the existing commands and introduce new commands.

 

The features for this release are:

 

·         Update Blob with PutBlockList  – We have added the ability to update the contents of a blob   The prior version of Azure Blob required you to re-upload the whole blob in order to change a single block of a blob.   With this new version, PutBlockList now allows you to add blocks, remove blocks, replace blocks, shuffle the order or existing blocks, or any combination of these for an existing blob.  The benefit of this new feature is that to perform these changes you only have to upload the blocks you want to add or change.   PutBlockList now allows you to create a new version of the blob by choosing blocks from a combination of (a) the existing set of blocks that comprise the current blob from the committed block list and (b) the newly uploaded blocks from the uncommitted block list.

 

 

·         Root Blob Container – We have provided the ability for all storage accounts to have a single root blob container.  This allows applications to store and reference blobs right off the domain address.    This allows you to store and access blobs using a sample URL as follows:

 

http://myaccount.blob.core.windows.net/profile.jpg

 

With this versioned change, we also changed the way applications perform operations on containers to make it explicit that it is a container operation instead of a blob operation.  For example, to create a container called pictures, this now looks like:

 

PUT http://myaccount.blob.core.windows.net/pictures?restype=container

 

Note, to use the root blob container “x-ms-version: 2009-07-17” must be specified.   Support for anonymous requests that do not require a x-ms-version HTTP header to be specified will be added in the future.

 

·         Shared Access Signatures for Signed URLs – It is now possible to create signatures, with an expiration date, that can be given out to provide access to Azure Blobs without having to give out your private key or make the blob container public.   Applications can now create these Signed Access Signatures and add these to requests to Azure Blobs to create Signed URLs.   The below is an example showing a signature (all of the fields after the “?”) giving read access to all of the blobs in the pictures container up until July 20^th. The URL then shows using this signature to access the blob profile.jpg in the pictures container.

 

GET http://myaccount.blob.core.windows.net/pictures/profile.jpg?se=2009-07-20&sr=c&sp=r&sig=xUXi%2f%2fxnETUHQoV0MGS06OkEiTo%3d

 

You create a signature computing a hash over a canonicalization of the request using your storage account secret key.  The signature can then be used as part of the URL to provide read, write, or delete access for blob requests.   Note, Shared Access Signatures do not require the use of x-ms-version.

 

In addition to the above new features we made the following semantic changes as part of this versioned CTP release:

·         Conditional Headers for Windows Azure Blobs and Queues – Previous versions left it ambiguous as to the meaning of a request that specified multiple conditional headers (e.g., If-Match, If-None-Match, If-Modified-Since and If-Unmodified-Since).   We now only allow a single conditional header to be specified, or one of the two following combinations:  (1) If-Match and If-Unmodified-Since, or (2) If-None-Match and If-Modified-Since.

 

·         Windows Azure Blobs and “/” –  Prior versions of Azure Blobs would trim off slashes from a blob name if multiple slashes occurred in a row and could misinterpret the request if the container name was empty.  We changed this behavior and now use the blob name as is, without trimming off preceding or trailing slashes, and correctly fail a request with an empty container name.

 

·         Table Query and All List Operations – The max timeout is now 30 seconds for listing Blobs, Blob Containers and Queues and for performing Table Queries.

 

 

Note that the above features for this new release are currently only available via the Windows Azure Storage REST interface.  They will be part of a future release of the Windows Azure SDK and storage client library.

 

For more information, details about these new features can be found in the MSDN documentation here later today:

 

[Edited below to provide a better link to the documentation for the new features]:
http://msdn.microsoft.com/en-us/library/dd894041.aspx

In addition, Steve Marx will have some examples on using the above new functionality on his blog soon:

http://blog.smarx.com/

 

As always, we appreciate any feedback you might have.

 

Brad Calder

Windows Azure Storage

Answer 1

@Brad: You may want to pin this to the top for a while :)

---

Pita.O:
http://www.arizentax.com/
http://www.cgi.com

Answer 2

Calling PutBlockList with "2009-07-17" version header now does not seem to accept old-style "Block" XML-elements anymore. It works when I remove the version header or use "Latest" instead of "Block". Is this intended?

Answer 3

Calling PutBlockList with "2009-07-17" version header now does not seem to accept old-style "Block" XML-elements anymore. It works when I remove the version header or use "Latest" instead of "Block". Is this intended?

Yes, the new version doesn't support "Block" anymore. 

If you want the exact same semantics as before, with the new version  2009-07017 you would use "Uncommitted" for all of the blocks you are specifying.  You can also use "Latest", but if the block is not in the uncommitted block list then it will try to find the block in the committed block list to complete the PutBlockList operation.

Brad

Answer 4

When i creating Shared Access Singature

It will give this error

<?xml version="1.0" encoding="utf-8" ?>

- <Error>

  <Code>AuthenticationFailed</Code>

  <Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature. RequestId:0c910ea6-ab55-4518-ae7f-7b2d3ee4230b Time:2009-08-20T12:14:59.5604908Z</Message>

  <AuthenticationErrorDetail>Signature not valid in the specified time frame</AuthenticationErrorDetail>

  </Error>

code is here...

byte[] b = Encoding.UTF8.GetBytes(AzureContainer.SignedIdentifier);

string SignedIdentifier = Convert.ToBase64String(b);           

string StringToSign = string.Format("r\n{0}\n{1}\n/{2}/{3}/{4}\n",
                AzureContainer.StartTime.ToString("yyyy-MM-ddTHH:mm:ssZ"),
                AzureContainer.EndTime.ToString("yyyy-MM-ddTHH:mm:ssZ"),
                AccountInfo.AccountName, AzureContainer.Name, "DPP_0050.JPG"); 

string Signature = Common.Utilities.ComputeSignature(StringToSign, Convert.FromBase64String(AccountInfo.AccountKey));
           
            string requestUrl = string.Format("{0}/{1}/{2}?st={3}&se={4}&sr=b&sp=r&sig={5}",
                EndPoint,
                Uri.EscapeDataString(AzureContainer.Name),
                Uri.EscapeDataString("DPP_0050.JPG"),
                Uri.EscapeDataString(AzureContainer.StartTime.ToString("yyyy-MM-ddTHH:mm:ssZ")),
                Uri.EscapeDataString(AzureContainer.EndTime.ToString("yyyy-MM-ddTHH:mm:ssZ")),
                Uri.EscapeDataString(Signature));

Can you tell where i am doing wrong.....

Thanks
Shrey Chouhan

---

Shrey Chouhan Cerebrata Software http://www.cerebrata.com http://omega.cerebrata.com [e-mail] http://shreychouhan.blogspot.com/

Answer 5

Your code seems to be fine.

The server has rejected the request becuase the current time (at the time of processing the request) was outside the timeinterval specified in the signature. Can you please post the start time (st), expiry time (et). Please note that all time should be specified in GMT.

Thanks,
Karthik

Answer 6

Correction. All time should be specified in UTC.

Thanks,
Karthik