none
Can we monitor requests to blobs?

    Question

  • Hi agian,

    Can we monitor incomming requests to blobs? As far I understand the concept the answer is no... but maybe I'm wrong.

    Why monitor incomming requests? Well, some plugings allow you for example to manipulate images on the fly (resize, add client cache headers, etc). How can we add such functionality to blob images?

    Expose storage for images via special url (ashx file?) that streams images??? And what about CDN? Are they still possible?

    Thank you!



    • Edited by Arjen Monday, April 25, 2011 9:54 PM Added CDN
    Monday, April 25, 2011 9:49 PM

Answers

  • To add functionality like you describe, you would need to proxy access to your blobs.  In other words, use a web role that accesses the blob storage and serves up the data to your clients modified as you see fit.  There would be a cost for hosting your web role, but the data transfer costs would be about the same if the web role and blob storage is both hosted in the same data center--you pay for the data transfer only when it goes out of the data center, you pay for storage access transaction the same either way, and the amount of storage.

    This also allows you to authenticate requests as you see fit, if shared access signatures or public access are insufficient for your needs. 

    But consider performance implications of this solution too because you now have two legs of the request to serve up the data--from blob storage to your web role (which will be fast within the data center but still not instantaneous), and from your web role to the requester.

    You can still cache the output from your web role at the web role server with output caching (or Appfabric Caching's distributed cache), or via CDN even from the web role now if you like.  But of course, if the data you are serving up is very custom for each request, caching might not be very useful. 

    As an alternative, if you have a small number of variants of the blob to be served (e.g. 5 different sizes of an image), you should probably upload multiple blobs rather than have the custom work dynamically created for the requests.

    There is a new feature of the CDN in CTP to support smooth streaming as well.

     

     


    Jeff G. Young @jeffcodes http://jeffcodes.net
    • Marked as answer by Wenchao Zeng Monday, May 02, 2011 3:10 AM
    Tuesday, April 26, 2011 3:49 AM

All replies

  • You might consider Share Access Signature feature of blob storage to provide time bound access on blobs or containers. You can find more information about this here.

     

    HTH.


    Please mark it as answer by clicking on "Propose As Answer", if it helps. My Blog : http://dotnetizen.blogspot.com
    Monday, April 25, 2011 11:35 PM
  • To add functionality like you describe, you would need to proxy access to your blobs.  In other words, use a web role that accesses the blob storage and serves up the data to your clients modified as you see fit.  There would be a cost for hosting your web role, but the data transfer costs would be about the same if the web role and blob storage is both hosted in the same data center--you pay for the data transfer only when it goes out of the data center, you pay for storage access transaction the same either way, and the amount of storage.

    This also allows you to authenticate requests as you see fit, if shared access signatures or public access are insufficient for your needs. 

    But consider performance implications of this solution too because you now have two legs of the request to serve up the data--from blob storage to your web role (which will be fast within the data center but still not instantaneous), and from your web role to the requester.

    You can still cache the output from your web role at the web role server with output caching (or Appfabric Caching's distributed cache), or via CDN even from the web role now if you like.  But of course, if the data you are serving up is very custom for each request, caching might not be very useful. 

    As an alternative, if you have a small number of variants of the blob to be served (e.g. 5 different sizes of an image), you should probably upload multiple blobs rather than have the custom work dynamically created for the requests.

    There is a new feature of the CDN in CTP to support smooth streaming as well.

     

     


    Jeff G. Young @jeffcodes http://jeffcodes.net
    • Marked as answer by Wenchao Zeng Monday, May 02, 2011 3:10 AM
    Tuesday, April 26, 2011 3:49 AM
  • You can still cache the output from your web role at the web role server with output caching (or Appfabric Caching's distributed cache), or via CDN even from the web role now if you like.  But of course, if the data you are serving up is very custom for each request, caching might not be very useful. 


    Thank you Jeff. Can you tell me more about the CDN? I thought the blob url is the first entry point for the CDN (the CDN intercepts this request and forwards the request to the nearest storage provider). In case of proxy solution we should redirect to a blob url, and not stream the file. Correct?

    Tuesday, April 26, 2011 9:11 AM
  • Hi Arjen,

    I agree with Jeff's solution.

    We can stream (not redirect to a blob url) the request through an ashx handler so that we can do resizing ect before returning the stream. Because we don't want the resizing operation be performed again and again for the same image (this will waste CPU), we can distribute the ashx handler via CDN. So if a request is submitted to a CDN url, the request will be transferred to the ashx handler if it is not already cached in CDN. Then subsequent requests to the same url will not be transferred to the ashx handler because it has been cached in CDN.

    Resource about CDN: http://blogs.msdn.com/b/windowsazure/archive/2009/11/05/introducing-the-windows-azure-content-delivery-network.aspx.

    Thanks


    Wengchao Zeng
    Please mark the replies as answers if they help or unmark if not.
    If you have any feedback about my replies, please contact msdnmg@microsoft.com.
    Microsoft One Code Framework
    Wednesday, April 27, 2011 7:22 AM
  • Hi,

    Yes you can. Write an application to monitor the incoming request to and from the blob storage, then store these information to SQLAzure or any SQL DB.

    Unless you are talking about many parties downloading from the blobs, then you might need to get them to use your application to monitor those activities.

    As for CDN, CDN is content delivery network. They are just leveraging the nearest EDGE Servers near you to transfer the data to you. What is does is, if you want a certain content from another CDN Endpoint, your nearest EDGE server will request from that particular CDN Endpoint, if it already exists there at your EDGE server, it will be sent to you immediately, else they will pull from the other location and send it to the EDGE servers near you and back to you. The files are cache for a default time expire of 72hours.

    Unforunately, monitoring the incoming and outgoing requests to and from the blob storage, you might need to write an application to monitor that. It is definitely possible.

     

    Hope this helps!

    • Proposed as answer by Tan Tzy Siong Thursday, May 05, 2011 10:46 AM
    Sunday, May 01, 2011 1:53 PM