none
Azure tables and throughput for highly scalable app

    Question

  • I just don’t see how an app in azure will scale for thousands of simultaneous users.

    Microsoft “sells” Azure so we can “build the next Facebook on it”, but I don’t get it.

    We have a limitation to get up to 500 entities in the same context per second.

    If I have let say reviews of a product, and the PK is Product ID, and the product itself has 800 reviews, and 85 people are trying to access the same product review information at the exact same time, I will suffer from throughput related issued, is this correct?

    Please give me an insight.

    Thanks a lot.

    Monday, November 29, 2010 4:09 PM

Answers

  • Ah sorry, too many years of SQL databases got me to misread this... A lot has been written about choosing the right partition key on this forum, and there is an excellent paper mentioned by steve marx in his reply to one of my posts that contains the detailed information about the performance characteristics that you want.

    That said clever partitioning will get you a long way in Azure Tables. Picking the right partition key is really the most complex technical issue you will face when dealing with Azure tables and you will probably want to play with different scenarios.

    With local cache I mean the non distributed traditional cache on asp.net.

    If I'd be in your shoes I would really think about a clear separation of read, write and search. One feasible way of creating a scalable site on Azure is to

    * Have a couple of sql azure databases to handle primary entity information (product slug,name,image link - although you might want to derive that directly from the slug, some view statistics) for inexpensive access when users browse,

    * an azure table store for all the detail information,

    * and a solr implementation (we use websolr.com) for all search operations.

    Always keep in mind that Azure Table incur transaction costs. That's a huge part of the equation for larger sites.

     


    -- My Azure Sites: www.eblizz.com , www.freshfugu.com Blog: martinatsunset.com Twitter: martin_sunset
    • Marked as answer by Igor33 Monday, December 06, 2010 10:05 PM
    Saturday, December 04, 2010 10:55 AM

All replies

  • If you are going back to the original datasource for ever transaction, you're inherently going to have scalability issues. A key factor in any scalable design is having a local cache available.

    That said, in your example, you've got 85 read requests can each return a single back of up to (if I'm remembering correctly), 100 rows of the 800 available). its not a best practice to return all 800 when the viewer will likely only be viewing a single page at a time anyways. Even if you did want all 800, that's still only 680 calls to get all 800 reviews for all 85 clients. If I put the reviews for a given item in their own table, and then partition it by say the date the review was place, this would help distribute the load for retrieving those rows across multiple Azure Storage Nodes, thus allowing us to rise above the 500/s limit.

    BTW, I'm no Azure Storage expert. So this post is without any implied or explicit guarantee of accuracy. :D

    Monday, November 29, 2010 5:05 PM
    Moderator
  • Hi Brent, thanks for sharing your thoughts.
    Monday, November 29, 2010 7:21 PM
  • Will any expert like to weigh in?

    Thanks.

    Monday, November 29, 2010 8:54 PM
  • The High Scalability blog is always worth reading. It has the following to say about FaceBook:

    Remember that Facebook caches everything. They have 28 terabytes of memcached data on 800 servers. The database is the system of record, but memory is where the action is. So when a problem happens that involves the caching layer, it can and did take down the system.

    Monday, November 29, 2010 10:39 PM
    Answerer
  • A single GET to table storage can retrieve up to 1000 entities.
    Tuesday, November 30, 2010 7:23 AM
  • You need to use distributed caching and a fan out database design and do some minimal denormalization, then all is good.

     


    -- My Azure Sites: www.eblizz.com , www.freshfugu.com Blog: martinatsunset.com Twitter: martin_sunset
    Wednesday, December 01, 2010 6:57 AM
  • Hi Martin,

    Can you please define fan out database design?

    Do you know when AppFaric Cache will be released?
    Is it safe to use the beta version on live deployments now?
    Do you know something about its cost?

    Thanks a lot an thanks to everyone that has contributed so far

    Wednesday, December 01, 2010 5:38 PM
  • fanout means that instead of thinking of sql azure as one big database server you treat it as n little ones. You distribute the load over the whole set by partitioning your data over those n databases. Oversimplified example: You have 1 million customer records, you create say 10 databases and store all whose name begins with a, b, or c in the first database, d,e,f in the second and so on. So except for queries in overlapping ranges you have only 1/10 load per server (assuming you partition the data well). Plinq turns out to be very useful in those scenarios. 

    My guess regarding Cache is 1q2011, and we are using the beta version in deployment. Downside is that it runs in the south central datacenter (we are in north central). We use local cache in case app fabric is down.

    There was an online questionnaire with some proposed pricing (asking questions like how much would you want to pay: x,y,or z). There is a thread in this forum about this shortly after pdc which shows the proposed price points

     

    Martin


    -- My Azure Sites: www.eblizz.com , www.freshfugu.com Blog: martinatsunset.com Twitter: martin_sunset
    Wednesday, December 01, 2010 6:31 PM
  • Martin,

    My solution uses azure tables instead of SQL azure.
    Do the sharding techniques you describe apply to azure tables as well or it’s unnecessary?

    When you say “local cache” you mean a cache maintained let’s say at your data center or in the client (user) machine?

    Thanks.

    Wednesday, December 01, 2010 7:40 PM
  • Ah sorry, too many years of SQL databases got me to misread this... A lot has been written about choosing the right partition key on this forum, and there is an excellent paper mentioned by steve marx in his reply to one of my posts that contains the detailed information about the performance characteristics that you want.

    That said clever partitioning will get you a long way in Azure Tables. Picking the right partition key is really the most complex technical issue you will face when dealing with Azure tables and you will probably want to play with different scenarios.

    With local cache I mean the non distributed traditional cache on asp.net.

    If I'd be in your shoes I would really think about a clear separation of read, write and search. One feasible way of creating a scalable site on Azure is to

    * Have a couple of sql azure databases to handle primary entity information (product slug,name,image link - although you might want to derive that directly from the slug, some view statistics) for inexpensive access when users browse,

    * an azure table store for all the detail information,

    * and a solr implementation (we use websolr.com) for all search operations.

    Always keep in mind that Azure Table incur transaction costs. That's a huge part of the equation for larger sites.

     


    -- My Azure Sites: www.eblizz.com , www.freshfugu.com Blog: martinatsunset.com Twitter: martin_sunset
    • Marked as answer by Igor33 Monday, December 06, 2010 10:05 PM
    Saturday, December 04, 2010 10:55 AM
  • Thanks to Martin and everyone who contributed to this post.
    Tuesday, December 07, 2010 3:37 PM