none
Best practice for using Blog Storage in a Social Network application

    Question

  • Hi,

    I am working on a social networking application and I was wondering if anybody over here could recommend the best way to use Windows Azure Blob storage for storing images.

    Our application stores user profiles. Users have multiple images. Avatar, picture library etc., and I want to leverage Blob storage in azure rather than storing in the sql azure database.

    Since each user has his/her own images, I want to create a separate folder for each user in an 'images' container.

    Since Containers doesn't support folders, what is the best practice to implement such a scenario. Should I create a separate container for every user? Is that an over kill? The applications has the potential to have millions of users, if successful.

    What are the best practices to store user specific content on Azure storage? Should I use a Table Storage instead?

    Thanks a lot for your help.

     

    Friday, December 23, 2011 6:52 PM

Answers

  • As you have said the application is going to be a social application, it expected to have massive reads or writes or both. Essentially, the mantra is to use what storage component for what task. Social applications have the luxury or facility of using new databases like newSQL or NoSQL. There are several blogs from Steve Marx and many other MS techies having implemented the installation/deployment of mongoDB in azure roles. You may try that, if it works out for your application.

    SQL Azure is great but for its greatness & sophistications to provide relational property it becomes more costly. As it is, for now it is big work-around for storing binary data like images and Azure storage blob comes for the rescue, still better it is really very cheap.

    Regarding the strategy for the blob container, If I architect the application, I will go for each blob container for each user based on username or some unique Id. Use the table storage for the massive data which will be generated like comments, likes,logs, chats  etc. Again you can leverage the partition key & row key for logical separtion and the kind of individual access you had for username based blob container. Queues for tasks. All the essential components you need for social application is available out of the box in Azure.

    Again with all points said above, Azure Storage is not a replacement for SQL Azure or SQL Server but the compliment the application with new possibilities. 

    Saturday, December 24, 2011 4:20 AM
  • hi pchamkura,

    Blobg storage actually supports folder representation. When you insert a blob, you can have the blob name as "someid/avatar" and "someid/picturelibrary" and when retrieving you can use it as Directory. CloudContainer.GetDirectoryReference method will give you CloudBlobDirectory object and you can traverse into it like a file system.

    API information: http://msdn.microsoft.com/en-us/library/windowsazure/microsoft.windowsazure.storageclient.cloudblobcontainer.getdirectoryreference.aspx

    Sample query could be:

    var blobs = container.GetDirectoryReference("avatar").ListBlobs().OfType<CloudBlob>().Where(b => b.Uri.Segments.Last().EndsWith(".png"));

     

    And for your problem, this mentioned approach would be a best suit.

    Thanks,

    Seetha

     

    Monday, December 26, 2011 8:26 AM

All replies

  • As you have said the application is going to be a social application, it expected to have massive reads or writes or both. Essentially, the mantra is to use what storage component for what task. Social applications have the luxury or facility of using new databases like newSQL or NoSQL. There are several blogs from Steve Marx and many other MS techies having implemented the installation/deployment of mongoDB in azure roles. You may try that, if it works out for your application.

    SQL Azure is great but for its greatness & sophistications to provide relational property it becomes more costly. As it is, for now it is big work-around for storing binary data like images and Azure storage blob comes for the rescue, still better it is really very cheap.

    Regarding the strategy for the blob container, If I architect the application, I will go for each blob container for each user based on username or some unique Id. Use the table storage for the massive data which will be generated like comments, likes,logs, chats  etc. Again you can leverage the partition key & row key for logical separtion and the kind of individual access you had for username based blob container. Queues for tasks. All the essential components you need for social application is available out of the box in Azure.

    Again with all points said above, Azure Storage is not a replacement for SQL Azure or SQL Server but the compliment the application with new possibilities. 

    Saturday, December 24, 2011 4:20 AM
  • Hi NavCode,

     

    Thanks a lot for your help... I just wanted to make sure if my logic was right...

    I use SQL Azure for transactional data, Users, Profiles, Messages etc.,

    Blog Storage for storing media files (images, videos etc)

    Queue for communication between worker and web roles...

     

    Also, if I create a container for every single user, will it be a problem if the system grows upto 50 million users? At that point, the Blob storage will have 50million containers...

     

    Any help is appreciated.

    Saturday, December 24, 2011 9:59 AM
  • 50 Million or 500 trillion, no problem with it. The billing of the Azure Storage Blob is for storage(capacity), transactions and bandwidth. Either you create so many containers or not the storage capacity is going to be almost same how ever you architect the application. The number of transactions might grow big as you pointed out, how ever the transaction cost is $0.01 per 10,000 storage transactions

    check out here [https://www.windowsazure.com/en-us/home/tour/storage/]

    Saturday, December 24, 2011 5:35 PM
  • hi pchamkura,

    Blobg storage actually supports folder representation. When you insert a blob, you can have the blob name as "someid/avatar" and "someid/picturelibrary" and when retrieving you can use it as Directory. CloudContainer.GetDirectoryReference method will give you CloudBlobDirectory object and you can traverse into it like a file system.

    API information: http://msdn.microsoft.com/en-us/library/windowsazure/microsoft.windowsazure.storageclient.cloudblobcontainer.getdirectoryreference.aspx

    Sample query could be:

    var blobs = container.GetDirectoryReference("avatar").ListBlobs().OfType<CloudBlob>().Where(b => b.Uri.Segments.Last().EndsWith(".png"));

     

    And for your problem, this mentioned approach would be a best suit.

    Thanks,

    Seetha

     

    Monday, December 26, 2011 8:26 AM
  • hi pchamkura,

    Blobg storage actually supports folder representation. When you insert a blob, you can have the blob name as "someid/avatar" and "someid/picturelibrary" and when retrieving you can use it as Directory. CloudContainer.GetDirectoryReference method will give you CloudBlobDirectory object and you can traverse into it like a file system.

    API information: http://msdn.microsoft.com/en-us/library/windowsazure/microsoft.windowsazure.storageclient.cloudblobcontainer.getdirectoryreference.aspx

    Sample query could be:

    var blobs = container.GetDirectoryReference("avatar").ListBlobs().OfType<CloudBlob>().Where(b => b.Uri.Segments.Last().EndsWith(".png"));

     

    And for your problem, this mentioned approach would be a best suit.

    Thanks,

    Seetha

     


    True,Thats why you can use it for click once application  hosting, or simple html site hosting.
    P Velachoutakos
    Wednesday, December 28, 2011 9:45 AM