How-To: Container's Size, Details and Performance ?

Question

Hi there,

Please consider the following scenario:

-         Assume that one storage account that has 2 containers: (Container-A and Container B).

-         Container-A : consists of 700 HTML files.

-         Container-B : consists of 500 HTML files.

Q1) How to determine the Container size? Is it possible to return a simple report about how many Containers do I have, and the size of each one?

Q2) How to count number of requests/views of each file inside the Containers?  Is it possible to return a simple report about the files inside the Containers, and the number of requests/views of each file?

Q3) Assume that there are so many visitors who surf/brows the files of (Container-A), and no one surfs/browses the files of (Container-B). Therefore, obviously I would like to improve the performance of Container-A only. How to accomplish that? (Please note that I ONLY want to improve the performance/scalability of Container-A).

 

I would greatly appreciate any constructive input.

Answer 1

Hi Alvaro,

Let me try and answer your questions:

Q1) How to determine the Container size? Is it possible to return a simple report about how many Containers do I have, and the size of each one?

Only way to determine this is by listing blobs in a container (listing does not download the blobs). When you list the blobs in a container, you will get the size of the blobs which you can add up and find the size of the blob container.

Q2) How to count number of requests/views of each file inside the Containers?   Is it possible to return a simple report about the files inside the Containers, and the number of requests/views of each file?

To the best of my knowledge, it is currently not possible to view how many times a blob has been viewed. Through MOCP portal you can find the total bandwidth consumed but unfortunately that is not the best way to achieve what you're trying to accomplish.

Q3) Assume that there are so many visitors who surf/brows the files of (Container-A ), and no one surfs/browses the files of (Container-B ). Therefore, obviously I would like to improve the performance of Container-A only. How to accomplish that? (Please note that I ONLY want to improve the performance/scalability of Container-A ).

You may find this blog post useful: http://blogs.msdn.com/b/windowsazurestorage/archive/2010/05/10/windows-azure-storage-abstractions-and-their-scalability-targets.aspx .

Hope this helps.

Thanks

Gaurav Mantri

Cerebrata Software

http://www.cerebrata.com

Answer 2

Thanks Gaurav Mantri for your input.

1)  As for determining the Container size?

Do you have code snippet for listing blobs? Or any code snippet to find out the size of Container(s) in Storage account ?

 

2)  As for How to count number of requests/views of each file inside the Containers?

I think we both agree that knowing the total bandwidth consumed doesn’t help. I wonder if this could be possible on Container level (I mean: Finding out which container has been access by visitors more than the others?)

 

3)  As for improving the performance and scalability of a Container?

In fact I have read the article that you sent me a while ago and I am aware about the value of proper partitioning. All that have been taken into consideration.

But as I explained in my post that, there will situations where visitors only who surf/brows the files of (Container-A) But they never ever surf/brows the files of (Container-B). Or it might be the another way around.

 

OR there might be a situation that my storage account has 20 Containers, and only the files of 2 Containers of the 20 are getting requests. So how to find out which Containers are getting more hits and how to improve their performance and scalability?   * Do you feel me?

 

4)  Alternatively, in case if I failed to find out a proper approach to improve the performance and scalability for some Containers. Then, How to improve the overall performance and scalability of my service? (Precisely and Concisely: My application is completely rely on many partitioned containers and the files inside these are getting hits by users, and I am worry about that.) I read Steve Marx’s post (http://social.msdn.microsoft.com/Forums/en/windowsazure/thread/e5002407-51b3-4650-9db8-5c87aa71cb2b), and I wonder if it helps my case or not. Because my case is fully depend on multiple containers with few UIs.

 

 

5)   Once again, I am worry about the performance and I might run into IO throttling. So I want to ask about implementing any caching mechanism, Could you please provide me any code samples for that?

Answer 3

Hi Alvaro,

Let me ask you a question: Are the contents of your blob container exposed only through your applications running in Azure? To explain what I mean, consider this scenario: Let's say you have a blob container named "images" which contains some images. Are the links to these image files only exposed through your application or are you distributing the links to these images outside of your application (for example some downloadable content)?

Regarding the performance and caching, may I suggest you look at Azure CDN.

Regarding #1, I don't have a code snippet ready however I do have something that may help you there which I am not ready (just yet) to share something publicly (I'm still a few weeks out to discuss it publicly). Please send me an email at gmantri at cerebrata dot com and I will respond back.

Thanks

Gaurav

Answer 4

Thanks Gaurav for your prompt response.

You’ve asked me an excellent question that will help to clarify my case.

My Application has only 2 aspx pages and multiple Containers. Each Container contains (e.g. 500 Images, 100 HTML page …etc). The files of every Container MUST be public and accessible by all visitors.

So, basically all visitors will browse my application and they will be able to browse/download the files that inside containers.

Therefore, the performance and scalability are crucial issues in my case, and frankly I am not sure how to tackle that with the Containers.

Answer 5

One possible way to achieve this is by running reports on IIS logs and figuring that out. Let's assume that each of your aspx page include two images served from blob container. Now every time this aspx page is served by the application, you can somehow assume that your blob storage is accessed twice (since there are two images). Based on this you can "guess " the blob storage utilization. This is obviously not the best way to do this and is full of loop holes especially if the HTML pages stored in blob storage are accessed directly and not through your aspx pages but I just wanted to throw this idea out and see if it makes any sense in your case.

Hope this helps.

Thanks

Gaurav