none
Cross-origin resource sharing (CORS) in Azure Storage

    Question

  • Hi,

    Here is my user story for a website allowing users to add files into storage.

    • User clicks 'upload a file'
    • Browser requests an upload Url from the service.
    • Service creates a blob container in the user account
    • Service creates a time-bombed SAS url on the container, say, 1hr.
    • Service passes back the upload Url.
    • User, meanwhile, is choosing a file from disk (fileToUpload).
    • Upon selection of file and with the uploadUrl in hand, the browser does a PUT of the file to the SAS url + filename (putUrl).
    • When the upload is done, the browser notifies the service which revokes the SAS and starts processing the file (say, adding an uploaded jpg to a user-managed gallery).

    Unacceptable architectural work arounds:

    Uploading the file to the web-role or worker using a POST, and then having the service do the write to storage.  Why not: This forces me to scale out my service to manage all the inbound bandwidth, Azure Storage is there for that, I shouldn't design a bottleneck on purpose.  Also, I may not geo-replicate the service, so the user's storage account may be across the globe from any intermediate upload service.

    .

    This can be implemented using the following simple calls:

    fileToUpload = "c:\\temp\\some.jpg"

    putUrl = "http://useraccount.blob.core.windows.net/image-guid/some.jpg?st=2012-10-11T16:32:18Z&se=2012-10-11T17:32:18Z&sr=c&si=e657ca66-b4cc-44d4-a9fe-a6b000000000&sig=K1g2L0n/XYDoa5YJQ9lZzwfc63HGRHC4C400000000="

    ulFile = new FileReader(fileToUpload);
    req = new XMLHttpRequest(); 
    req.open("PUT", putUrl); 
    req.onload = function(event) { alert(event.target.responseText); }
    req.send(ulFile);

    Now here is what I'm getting when that .send is called:

    Outbound:

    OPTIONS http://useraccount.blob.core.windows.net/image-guid/some.jpg?st=2012-10-11T16:32:18Z&se=2013-10-11T16:32:18Z&sr=c&si=e657ca66-b4cc-44d4-a9fe-a6b000000000&sig=K1g2L0n/XYDoa5YJQ9lZzwfc63HGRHC4C4000000000= HTTP/1.1
    Host: useraccount.blob.core.windows.net
    Connection: keep-alive
    Access-Control-Request-Method: PUT
    Origin: chrome://newtab
    User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.4 (KHTML, like Gecko) Chrome/22.0.1229.94 Safari/537.4
    Access-Control-Request-Headers: origin, content-type
    Accept: */*
    Accept-Encoding: gzip,deflate,sdch
    Accept-Language: en-US,en;q=0.8
    Accept-Charset: ISO-8859-1,utf-8;q=0.7,*;q=0.3


    Notice the request for OPTIONS to validate the cross-origin resource sharing, specifically the "Access-Control-Request-Method: PUT" and "Access-Control-Request-Headers" for origin and content type.

    Responce:

    HTTP/1.1 405 The resource doesn't support specified Http Verb.
    Transfer-Encoding: chunked
    Allow: GET,HEAD,PUT,DELETE
    Server: Microsoft-HTTPAPI/2.0
    x-ms-request-id: 3b0c6b92-5bbe-48a9-8ee0-901d393b0359
    Date: Thu, 11 Oct 2012 19:43:31 GMT
    0

    Any here we see that the Azure Storage allowed HTTP verbs does not include OPTIONS.

    In Chrome, this kills the upload right there and then.

    Can you share plans for supporting OPTIONS which implements CORS to support simple PUTs via XMLHttpRequest, which is availabe on all modern browsers?

    Otherwise, what is the recommended method for uploading from the browser directly into storage via PUTs and SAS urls?

    Thanks,

    Nick Drouin

    Refs for CORS and the code snippet:

    http://enable-cors.org/

    http://www.html5rocks.com/en/tutorials/file/xhr2/

    https://developer.mozilla.org/en-US/docs/FileGuide/FileUpDown

    http://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#request-method


    • Edited by Nick Drouin Friday, October 12, 2012 3:46 AM
    Thursday, October 11, 2012 7:58 PM

Answers

  • Great news, CORS has arrived!

    http://blogs.msdn.com/b/windowsazurestorage/archive/2013/11/27/windows-azure-storage-release-introducing-cors-json-minute-metrics-and-more.aspx

    To close off this thread, here are a few snippets to complete the scenario above.

    First you need to set the API version and some CORs rules on your storage account, I used:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using Microsoft.WindowsAzure.Storage;
    using Microsoft.WindowsAzure.Storage.Auth;
    using Microsoft.WindowsAzure.Storage.Blob;
    using Microsoft.WindowsAzure.Storage.Shared.Protocol;
    
    //Ref: http://blogs.msdn.com/b/windowsazurestorage/archive/2013/11/27/windows-azure-storage-release-introducing-cors-json-minute-metrics-and-more.aspx
    //Ref: http://msdn.microsoft.com/en-us/library/windowsazure/dn535601.aspx
    //Ref: https://github.com/WindowsAzure/azure-storage-net/blob/c9d52db3f18f971933111f5ba3f7ce4e79927a73/Test/ClassLibraryCommon/Queue/QueueAnalyticsUnitTests.cs
    
    
    namespace EnableCors
    {
        class Program
        {
            private static void Main(string[] args)
            {
                var _storageAccountName = "youraccount";
                var _storageAccountKey =
                    "yourkey==";
    
                var accountAndKey = new StorageCredentials(_storageAccountName, _storageAccountKey);
                var storageaccount = new CloudStorageAccount(accountAndKey, true);
                var blobClient = storageaccount.CreateCloudBlobClient();
    
                Console.WriteLine("Storage Account: " + storageaccount.BlobEndpoint);
                var newProperties = CurrentProperties(blobClient);
    
                //Set service to new version:
                newProperties.DefaultServiceVersion = "2013-08-15"; //"2012-02-12"; // "2011-08-18"; // null;
                blobClient.SetServiceProperties(newProperties);
    
                var addRule = true;
                if (addRule)
                {
    
                    //Add a wide open rule to allow uploads:
                    var ruleWideOpenWriter = new CorsRule()
                    {
                        AllowedHeaders = { "*" },
                        AllowedOrigins = { "*" },
                        AllowedMethods =
                            CorsHttpMethods.Options |
                            CorsHttpMethods.Post |
                            CorsHttpMethods.Put |
                            CorsHttpMethods.Merge,
                        ExposedHeaders = { "*" },
                        MaxAgeInSeconds = (int)TimeSpan.FromDays(5).TotalSeconds
                    };
                    newProperties.Cors.CorsRules.Add(ruleWideOpenWriter);
                    blobClient.SetServiceProperties(newProperties);
    
                    Console.WriteLine("New Properties:");
                    CurrentProperties(blobClient);
                }
    
                Console.ReadKey();
            }
    
            private static ServiceProperties CurrentProperties(CloudBlobClient blobClient)
            {
                var currentProperties = blobClient.GetServiceProperties();
                if (currentProperties != null)
                {
                    if (currentProperties.Cors != null)
                    {
                        Console.WriteLine("Cors.CorsRules.Count          : " + currentProperties.Cors.CorsRules.Count);
                        for (int index = 0; index < currentProperties.Cors.CorsRules.Count; index++)
                        {
                            var corsRule = currentProperties.Cors.CorsRules[index];
                            Console.WriteLine("corsRule[index]              : " + index);
                            foreach (var allowedHeader in corsRule.AllowedHeaders)
                            {
                                Console.WriteLine("corsRule.AllowedHeaders      : " + allowedHeader);
                            }
                            Console.WriteLine("corsRule.AllowedMethods      : " + corsRule.AllowedMethods);
    
                            foreach (var allowedOrigins in corsRule.AllowedOrigins)
                            {
                                Console.WriteLine("corsRule.AllowedOrigins      : " + allowedOrigins);
                            }
                            foreach (var exposedHeaders in corsRule.ExposedHeaders)
                            {
                                Console.WriteLine("corsRule.ExposedHeaders      : " + exposedHeaders);
                            }
                            Console.WriteLine("corsRule.MaxAgeInSeconds     : " + corsRule.MaxAgeInSeconds);
                        }
                    }
                    Console.WriteLine("DefaultServiceVersion         : " + currentProperties.DefaultServiceVersion);
                    Console.WriteLine("HourMetrics.MetricsLevel      : " + currentProperties.HourMetrics.MetricsLevel);
                    Console.WriteLine("HourMetrics.RetentionDays     : " + currentProperties.HourMetrics.RetentionDays);
                    Console.WriteLine("HourMetrics.Version           : " + currentProperties.HourMetrics.Version);
                    Console.WriteLine("Logging.LoggingOperations     : " + currentProperties.Logging.LoggingOperations);
                    Console.WriteLine("Logging.RetentionDays         : " + currentProperties.Logging.RetentionDays);
                    Console.WriteLine("Logging.Version               : " + currentProperties.Logging.Version);
                    Console.WriteLine("MinuteMetrics.MetricsLevel    : " + currentProperties.MinuteMetrics.MetricsLevel);
                    Console.WriteLine("MinuteMetrics.RetentionDays   : " + currentProperties.MinuteMetrics.RetentionDays);
                    Console.WriteLine("MinuteMetrics.Version         : " + currentProperties.MinuteMetrics.Version);
                }
                return currentProperties;
            }
    
        }
    }
    

    .

    Then you need an HTML5 page to do the upload:

    <!DOCTYPE html>
    <html lang="en">
    <head>
    <title>HTML5 Upload Demo: Using a SAS to upload a file</title>
    </head>
    <body>
    <section id="wrapper">
    
    <header>
    <h1>File API (simple)</h1>
    </header>
    
    <article>
    
    <p><input type=file></p>
    <p>Select media from your machine to upload to your storage account.</p>
    </article>
    
    <script>
    
    //TODO: Have the server side set the storagePath
        var storagePath = "http://youraccount.blob.core.windows.net/asset-5c98bb6c-0f76-41f3-b5d7-3326ae8114cb/";
    
    //TODO: Have the server side get and set the SAS query, use a stort expiry time.
        var sasQueryString = "?sv=2012-02-12&se=2013-12-04T21%3A06%3A25Z&sr=c&si=d59c1971-6017-4843-ba91-21cbd0267f79&sig=XyaX1wwsCAFPD9vY5yCtsXp9MPlVCxNkPHLpH6rNlkg%3D";
    
        var upload = document.getElementsByTagName('input')[0];
        
        upload.onchange = function (e) {
    
            e.preventDefault();
    
            //You can iterate through and upload all the selected files for ideas on how to do that, see:
            //Ref: http://www.html5rocks.com/en/tutorials/file/dndfiles/
            //But here, we simply use the first file "[0]":
            var file = upload.files[0];
    
            putUrl = storagePath + file.name + sasQueryString;
    
            req = new XMLHttpRequest();
            req.open("PUT", putUrl);
            //Need to specify a few headers.  
            //For very large files, you'll need a different strategy (PutBlock/PutBlockList API)
            req.setRequestHeader("x-ms-version", "2013-08-15");
            req.setRequestHeader("x-ms-blob-type", "BlockBlob")
            //You can find code snippets to set content type correctly for you file, here we hardcode it:
            req.setRequestHeader("Content-Type", "video/mp4")
            //Give some sort of user feedback:
            req.onload = function (event) { alert(event.target.responseText); }
            req.send(file);
    
            return false;
        };
    </script>
    </section>
    </body>
    </html>

    Regards,

    -Nick Drouin

    Program Manager

    Windows Azure Media Services

    • Marked as answer by Nick Drouin Tuesday, December 03, 2013 8:34 PM
    Tuesday, December 03, 2013 4:57 PM

All replies

  • To my knowledge, there have been no announced plans to enable CORS for Windows Azure blob storage.

    Browser plugins (Silverlight or Flash) are potential workarounds.

    FYI, I believe your use of "non-goal" is nonstandard. A non-goal is something desirable but out of scope. I was confused about this when I first joined Microsoft, as the term is used frequently there in specs.

    Thursday, October 11, 2012 8:16 PM
  • Thanks, edited for clarity.

    I also saw your posts on this in other threads - at least two other requests similar to this, and much older.  One of which asked for more details, so I added them here. 

    I have very real scenarios that need to push large files to Azure blobs without a middle-tier.  Any specific suggestions for a Flash implementation?

    Prefered would be a client that could make use of the proper Block Blob REST calls which would allow for threaded async uploads of 1Mb blocks using PUTs, and validating that they are all committed prior to declaring upload completion.

    Thanks,

    -Nick

    Friday, October 12, 2012 3:56 AM
  • I once built an implementation in Silverlight, but I don't have any Flash experience. (I assume the code would be similar, though.)

    http://blog.smarx.com/posts/uploading-windows-azure-blobs-from-silverlight-part-3-handling-big-files-by-using-the-block-apis

    Friday, October 12, 2012 4:55 AM
  • Steve,

       Thanks for the response.  Let's assume what you're saying is true. If it is true, then this necessarily implies that the Microsoft's plug-in free strategy for Windows 8 is DOA for use cases like this.  In other words if we want to deliver content to our plug-in free apps to any platform who's browser is plug-in free (including Windows 8), and we need those resources loaded asynchronously, then Windows Azure is not our answer for such a storage and delivery capability.  Window 8 not compatible with Windows Azure Blob Storage for Web Apps???? I find this extremely difficult to believe that CORS could be a "non-goal" and out of scope much longer.  I would hope that good people like yourself at Microsoft would ponder a little deeper on the many implications of this use case.  In the mean time while I wait, I'm unfortunately forced to have at least part of my Windows Azure Web App delivered by Google's cloud storage.  

    Monday, December 10, 2012 5:59 PM
  • Daron,

    I don't work at Microsoft.

    And you can just use a Web Role or Web Site. Just because blob storage doesn't support CORS doesn't mean you can't use Windows Azure.

    Monday, December 10, 2012 7:07 PM
  • Sorry about that Steve, did mean to tag you as an MS employee.  You are also correct you can sub-optimally put square peg in round hole and route everything through a Web Role which unfortunately kills many of the benefits of cloud storage not to mention incurring more costs.  Perhaps the impact isn't as bad as I project, would like to hear your experience with such solution.
    Thursday, December 13, 2012 4:59 PM
  • I have to agree all the way with Nick and Daron on this. CORS not being available in Azure Storage would render quite a few scenarios useless. Sure, we could use Web Roles as proxies / routers, but how about CDN? It would require us to deploy Webroles in all datacenters in order to get Azure Storage content close to our users. Also, with this proxy 'solution' we would lose the great scalability of the Storge / CDN infrastructure....

    Apparently in a presentation in Channel9 someone at Microsoft mentioned that CORS is actually on the radar. It would be very helpful if someone can confirm this is correct and shed some light on planning / availability.

    Thanks!

    Friday, December 21, 2012 10:02 AM
  • easyXDM is a great open source library for solving cross-domain problems like this one. I did a proof of concept with Azure Blob Storage and recently wrote a blog post about it that includes a small demo:

    http://blog.appliedis.com/2013/01/11/cross-origin-resource-sharing-using-azure-storage-with-easyxdm/

    (sorry for the lack of links but I'm a new poster and the forums won't let me embed links yet apparently)

    Thursday, January 17, 2013 1:30 AM
  • Hi Ted,

    I've just read that CORS will be soon ("few weeks") available on Azure, referring to this tweet by Scott Guthrie ;)


    Christophe H.

    Tuesday, June 04, 2013 9:59 AM
  • Great news, CORS has arrived!

    http://blogs.msdn.com/b/windowsazurestorage/archive/2013/11/27/windows-azure-storage-release-introducing-cors-json-minute-metrics-and-more.aspx

    To close off this thread, here are a few snippets to complete the scenario above.

    First you need to set the API version and some CORs rules on your storage account, I used:

    using System;
    using System.Collections.Generic;
    using System.Linq;
    using System.Text;
    using Microsoft.WindowsAzure.Storage;
    using Microsoft.WindowsAzure.Storage.Auth;
    using Microsoft.WindowsAzure.Storage.Blob;
    using Microsoft.WindowsAzure.Storage.Shared.Protocol;
    
    //Ref: http://blogs.msdn.com/b/windowsazurestorage/archive/2013/11/27/windows-azure-storage-release-introducing-cors-json-minute-metrics-and-more.aspx
    //Ref: http://msdn.microsoft.com/en-us/library/windowsazure/dn535601.aspx
    //Ref: https://github.com/WindowsAzure/azure-storage-net/blob/c9d52db3f18f971933111f5ba3f7ce4e79927a73/Test/ClassLibraryCommon/Queue/QueueAnalyticsUnitTests.cs
    
    
    namespace EnableCors
    {
        class Program
        {
            private static void Main(string[] args)
            {
                var _storageAccountName = "youraccount";
                var _storageAccountKey =
                    "yourkey==";
    
                var accountAndKey = new StorageCredentials(_storageAccountName, _storageAccountKey);
                var storageaccount = new CloudStorageAccount(accountAndKey, true);
                var blobClient = storageaccount.CreateCloudBlobClient();
    
                Console.WriteLine("Storage Account: " + storageaccount.BlobEndpoint);
                var newProperties = CurrentProperties(blobClient);
    
                //Set service to new version:
                newProperties.DefaultServiceVersion = "2013-08-15"; //"2012-02-12"; // "2011-08-18"; // null;
                blobClient.SetServiceProperties(newProperties);
    
                var addRule = true;
                if (addRule)
                {
    
                    //Add a wide open rule to allow uploads:
                    var ruleWideOpenWriter = new CorsRule()
                    {
                        AllowedHeaders = { "*" },
                        AllowedOrigins = { "*" },
                        AllowedMethods =
                            CorsHttpMethods.Options |
                            CorsHttpMethods.Post |
                            CorsHttpMethods.Put |
                            CorsHttpMethods.Merge,
                        ExposedHeaders = { "*" },
                        MaxAgeInSeconds = (int)TimeSpan.FromDays(5).TotalSeconds
                    };
                    newProperties.Cors.CorsRules.Add(ruleWideOpenWriter);
                    blobClient.SetServiceProperties(newProperties);
    
                    Console.WriteLine("New Properties:");
                    CurrentProperties(blobClient);
                }
    
                Console.ReadKey();
            }
    
            private static ServiceProperties CurrentProperties(CloudBlobClient blobClient)
            {
                var currentProperties = blobClient.GetServiceProperties();
                if (currentProperties != null)
                {
                    if (currentProperties.Cors != null)
                    {
                        Console.WriteLine("Cors.CorsRules.Count          : " + currentProperties.Cors.CorsRules.Count);
                        for (int index = 0; index < currentProperties.Cors.CorsRules.Count; index++)
                        {
                            var corsRule = currentProperties.Cors.CorsRules[index];
                            Console.WriteLine("corsRule[index]              : " + index);
                            foreach (var allowedHeader in corsRule.AllowedHeaders)
                            {
                                Console.WriteLine("corsRule.AllowedHeaders      : " + allowedHeader);
                            }
                            Console.WriteLine("corsRule.AllowedMethods      : " + corsRule.AllowedMethods);
    
                            foreach (var allowedOrigins in corsRule.AllowedOrigins)
                            {
                                Console.WriteLine("corsRule.AllowedOrigins      : " + allowedOrigins);
                            }
                            foreach (var exposedHeaders in corsRule.ExposedHeaders)
                            {
                                Console.WriteLine("corsRule.ExposedHeaders      : " + exposedHeaders);
                            }
                            Console.WriteLine("corsRule.MaxAgeInSeconds     : " + corsRule.MaxAgeInSeconds);
                        }
                    }
                    Console.WriteLine("DefaultServiceVersion         : " + currentProperties.DefaultServiceVersion);
                    Console.WriteLine("HourMetrics.MetricsLevel      : " + currentProperties.HourMetrics.MetricsLevel);
                    Console.WriteLine("HourMetrics.RetentionDays     : " + currentProperties.HourMetrics.RetentionDays);
                    Console.WriteLine("HourMetrics.Version           : " + currentProperties.HourMetrics.Version);
                    Console.WriteLine("Logging.LoggingOperations     : " + currentProperties.Logging.LoggingOperations);
                    Console.WriteLine("Logging.RetentionDays         : " + currentProperties.Logging.RetentionDays);
                    Console.WriteLine("Logging.Version               : " + currentProperties.Logging.Version);
                    Console.WriteLine("MinuteMetrics.MetricsLevel    : " + currentProperties.MinuteMetrics.MetricsLevel);
                    Console.WriteLine("MinuteMetrics.RetentionDays   : " + currentProperties.MinuteMetrics.RetentionDays);
                    Console.WriteLine("MinuteMetrics.Version         : " + currentProperties.MinuteMetrics.Version);
                }
                return currentProperties;
            }
    
        }
    }
    

    .

    Then you need an HTML5 page to do the upload:

    <!DOCTYPE html>
    <html lang="en">
    <head>
    <title>HTML5 Upload Demo: Using a SAS to upload a file</title>
    </head>
    <body>
    <section id="wrapper">
    
    <header>
    <h1>File API (simple)</h1>
    </header>
    
    <article>
    
    <p><input type=file></p>
    <p>Select media from your machine to upload to your storage account.</p>
    </article>
    
    <script>
    
    //TODO: Have the server side set the storagePath
        var storagePath = "http://youraccount.blob.core.windows.net/asset-5c98bb6c-0f76-41f3-b5d7-3326ae8114cb/";
    
    //TODO: Have the server side get and set the SAS query, use a stort expiry time.
        var sasQueryString = "?sv=2012-02-12&se=2013-12-04T21%3A06%3A25Z&sr=c&si=d59c1971-6017-4843-ba91-21cbd0267f79&sig=XyaX1wwsCAFPD9vY5yCtsXp9MPlVCxNkPHLpH6rNlkg%3D";
    
        var upload = document.getElementsByTagName('input')[0];
        
        upload.onchange = function (e) {
    
            e.preventDefault();
    
            //You can iterate through and upload all the selected files for ideas on how to do that, see:
            //Ref: http://www.html5rocks.com/en/tutorials/file/dndfiles/
            //But here, we simply use the first file "[0]":
            var file = upload.files[0];
    
            putUrl = storagePath + file.name + sasQueryString;
    
            req = new XMLHttpRequest();
            req.open("PUT", putUrl);
            //Need to specify a few headers.  
            //For very large files, you'll need a different strategy (PutBlock/PutBlockList API)
            req.setRequestHeader("x-ms-version", "2013-08-15");
            req.setRequestHeader("x-ms-blob-type", "BlockBlob")
            //You can find code snippets to set content type correctly for you file, here we hardcode it:
            req.setRequestHeader("Content-Type", "video/mp4")
            //Give some sort of user feedback:
            req.onload = function (event) { alert(event.target.responseText); }
            req.send(file);
    
            return false;
        };
    </script>
    </section>
    </body>
    </html>

    Regards,

    -Nick Drouin

    Program Manager

    Windows Azure Media Services

    • Marked as answer by Nick Drouin Tuesday, December 03, 2013 8:34 PM
    Tuesday, December 03, 2013 4:57 PM
  • Is there a simple solution to allow the upload of large blobs in blocks? At the moment the file size is limited to 64Mb only

    Thank you

    Thursday, January 02, 2014 9:52 AM
  • @Nick Drouin 

    Hello, 

    I am using Azure websites to pull static content from blob storage and deliver if via CDN is there a post that beaks this down a bit more and goes through it step by step? 

    Ryan 


    Ryan Parrish

    Thursday, July 17, 2014 6:15 PM