Multi-Tenant Storage Relay Service

Question

Hi all,


I want to integrate Access Service Control with BlobStorage as Authentication Keys alone do not meet my requirements.
Suppose the following scenario:
I have one blob storage account where each tenant would get his own container to download data from. To guarantee that the containers are not visible to the outside and to other tenants, their visibility is set to private. As I'm the only one who has the authentication keys (and thus the tenants cannot access their data through the storage client API, for example) I want to provide them a RESTful service (which has acces to the authentication keys) whose endpoint is registered at the service bus (allowing me to exploit the Access Control Service) that encapsulates access to the blob storage. The service itself is also hosted in the cloud in a web role.
This service is currently implemented similar to this:

public Stream getResource(string uri)
        {
            WebOperationContext.Current.OutgoingResponse.ContentType = "binary/octet-stream";
            WebClient client = new WebClient();
           
            // Add appropriate authentication headers to webclient's request
            // by invoking client.Headers.Add(string key, string value) etc.
            // --> not shown here
           
            return client.OpenRead(uri);
        }


Now, I have the following problem: If a tenant makes a request to download a software package (which may be up to 1GB, for example), then the blob is actually downloaded twice. First from the blob storage into the machine's memory where the RESTful service is hosted and then from there to the client. This introduces huge delays which is not desired, of course.

Do you have any recommendations? Is there a way to pipe the response of the blobstorage directly through to the client? Any example code?
Thank you!

Best Regards
Oliver.

Answer 1

Hello, first of all, please note ACS is not fully supported in Windows Azure yet. Essentially, you cannot use Geneva Framework unless you sacrifice a lot of security features (which contradicts to the reason you want to use ACS). So I would suggest you to use simpler ASP.NET membership service for the period. We're working to see if it is possible to support ACS and Geneva Framework in Windows Azure in the future. But currently I cannot give you any assurance.

Anyway, your main concern seems to be the data is downloaded twice. To address this issue, you need to divide large blobs into blocks in your REST service. That is, you download a block from Azure blob service, which is, say, 1 Mb. Then while you're downloading the next block in your service, you begin to transfer the first block to your client. That way, the total time it takes to download the 1 Gb data is T(1 Gb) + T(1 Mb), which is acceptable. Of course, this is just the time it takes in an ideal environment. But as long as you download from blob service and transfer to client at the same time, the time it takes to download the data twice will only be a little longer than the time it takes to download the data once. Also, the blocks that're already transferred to the client can be garbage collected, which can improve the performance further more.

---

Lante, shanaolanxing This posting is provided "AS IS" with no warranties, and confers no rights.