locked
Azure Blob storage as Database RRS feed

  • Question

  • Hi,

    I am creating an application which will contain data for filling multiple forms, and data objects and relationships are pretty nested.

    No two forms are connected as they will be from different users, the each form :- data can go up to few MBs.

    I was keen to implement no SQL so initially thought of using document DB, How ever It has a limit of 512KB and yearly my data can go to 500GB.

    My solution is deployed on azure cloud...and hence I do not want to take maintenance over head.

    Therefore, I started looking for more option now like CouchDB/Raven/Mongo but all these comes out with their own maintenance and set up.

    So I now thought to store form data in terms of JSON on Azure Blob storage, does it sound like a right solution ?

    Thanks in advance.

    Sunday, April 17, 2016 4:18 AM

Answers

  • Varun,

    Document DB's per-document size limit is 512KB. However, your collection itself can scale practivally infinitely (petabytes). See:https://azure.microsoft.com/en-us/documentation/articles/documentdb-partition-data/

    If you have larger or non-JSON documents, you should think about using DocumentDB as the metadata store and then put the actual large documents in blob store. You would write one or more documents to DocumentDB for every one of the documents you put in blob store. The JSON docs in DocumentDB would contain all the metadata (filename, modified date etc.) that you need to query over, and a link/reference to the actual document in blob store.

    This way, you can leverage DocumentDB for its strength (low latency indexing+query etc.) to find the document you need and then perform one retrieval from blob storage.

    • Marked as answer by Han, MSFT Friday, September 30, 2016 10:55 PM
    Friday, May 20, 2016 8:35 PM
  • Hi Varun,

      A DocumentDB partitioned collection has a default allocation of 250 GB and be increased to support higher data volume upon request.   Without knowing the requirements of your app, I would suggest you model your form data so it is broken down to several documents based on relevancy.  Here's an article on how to model data in DocumentDB that you may find helpful:

    https://azure.microsoft.com/en-us/documentation/articles/documentdb-modeling-data/

        

    • Proposed as answer by Han, MSFT Friday, May 20, 2016 8:37 PM
    • Marked as answer by Han, MSFT Friday, September 30, 2016 10:55 PM
    Friday, May 20, 2016 8:37 PM

All replies

  • Can some one reply to this ?
    Tuesday, April 19, 2016 4:10 AM
  • Varun,

    Document DB's per-document size limit is 512KB. However, your collection itself can scale practivally infinitely (petabytes). See:https://azure.microsoft.com/en-us/documentation/articles/documentdb-partition-data/

    If you have larger or non-JSON documents, you should think about using DocumentDB as the metadata store and then put the actual large documents in blob store. You would write one or more documents to DocumentDB for every one of the documents you put in blob store. The JSON docs in DocumentDB would contain all the metadata (filename, modified date etc.) that you need to query over, and a link/reference to the actual document in blob store.

    This way, you can leverage DocumentDB for its strength (low latency indexing+query etc.) to find the document you need and then perform one retrieval from blob storage.

    • Marked as answer by Han, MSFT Friday, September 30, 2016 10:55 PM
    Friday, May 20, 2016 8:35 PM
  • Hi Varun,

      A DocumentDB partitioned collection has a default allocation of 250 GB and be increased to support higher data volume upon request.   Without knowing the requirements of your app, I would suggest you model your form data so it is broken down to several documents based on relevancy.  Here's an article on how to model data in DocumentDB that you may find helpful:

    https://azure.microsoft.com/en-us/documentation/articles/documentdb-modeling-data/

        

    • Proposed as answer by Han, MSFT Friday, May 20, 2016 8:37 PM
    • Marked as answer by Han, MSFT Friday, September 30, 2016 10:55 PM
    Friday, May 20, 2016 8:37 PM