none
Need help for explaining Transaction Prices for Data Lake Store

    Question

  • Hi everyone,

    I have data which has a total size of 200 GB. I want to load this data in our Data Lake Store. I don't understand the Transaction Prices (10,000 Write operations for €0.043).

    What does Microsoft means with one transaction/operation? Is one transaction/operation one file? For example, can we put our 200 GB amount of data in one transaction or are there limits? This seems not realistic, otherwise everybody would do the same...

    Can someone explain this and/or can someone give us a cost indication (with details please). Thanks in advance!

    With kind regards,

    Ricardo


    Tuesday, June 27, 2017 2:21 PM

Answers

  • We are in the midst of updating the FAQ on our pricing page related to this. In the meanwhile I am pasting the content of that below. Please refer to this and let us know if you have any questions. 

    ---

     

    How are transactions calculated?

    Azure Data Lake Store transactions are incurred any time you read and write data to the service. Every time a user, an application or another Azure service reads or writes data up to 4MB in size, it is billed as one transaction. For example, if one write operation puts 128 KB of data into ADLS,  it is billed as 1 transaction. If the item is larger than 4MB, transactions are billed in increments of up to 4MB. For example, if one read operation gets 9MB of data from ADLS, it is billed as 3 transactions (4MB + 4MB + 1MB).

    Let us see how transactions shows up on a your bill based on read and write operations. For this assume a scenario where your application writes data into ADLS at a rate of 10 items / second, each item being 8MB. Additionally, let’s say you’re ingesting data using Azure IoT Hub into ADLS at a rate of 5 items / second, each item being less than 4MB. Let’s also assume that you have an ADLA job that runs for 4 hours / day, while reading 1,000 items / second when the job is running each item being less than 4MB. In the above scenario, ADLS will charge for write transactions for your application and Azure IoT Hub writing data, and read transactions for ADLA reading data from ADLS. You will be charged the following:

    Item

    Usage Volume per Month

    Rate per Month

    Monthly Cost

    Write transactions  from customer application

    2 * (10 items/second * 3,600 * 744)

    $0.05 per 10,000 transactions

    $267.84

    Write transactions from Azure IoT Hub

    5 items/second * 3,600 * 744

    $0.05 per 10,000 transactions

    $66.96

    Read transactions from Azure IoT Hub

    1,000 items/second * 3600 * 4 * 31

    $0.004 per 10,000 transactions

    $178.56

    Total Transactions Cost

    $513.36

    The read and write operations generated from all applications and services, including Azure Data Lake Analytics, depend on a large variety of factors. There is no easy way to accurately predict the number of operations generated by every application. To estimate your transaction costs, you should run a representative workload for a controlled duration and observe the number of reads and writes, and the resulting transactions that are billed.

    Tuesday, June 27, 2017 11:54 PM

All replies

  • We are in the midst of updating the FAQ on our pricing page related to this. In the meanwhile I am pasting the content of that below. Please refer to this and let us know if you have any questions. 

    ---

     

    How are transactions calculated?

    Azure Data Lake Store transactions are incurred any time you read and write data to the service. Every time a user, an application or another Azure service reads or writes data up to 4MB in size, it is billed as one transaction. For example, if one write operation puts 128 KB of data into ADLS,  it is billed as 1 transaction. If the item is larger than 4MB, transactions are billed in increments of up to 4MB. For example, if one read operation gets 9MB of data from ADLS, it is billed as 3 transactions (4MB + 4MB + 1MB).

    Let us see how transactions shows up on a your bill based on read and write operations. For this assume a scenario where your application writes data into ADLS at a rate of 10 items / second, each item being 8MB. Additionally, let’s say you’re ingesting data using Azure IoT Hub into ADLS at a rate of 5 items / second, each item being less than 4MB. Let’s also assume that you have an ADLA job that runs for 4 hours / day, while reading 1,000 items / second when the job is running each item being less than 4MB. In the above scenario, ADLS will charge for write transactions for your application and Azure IoT Hub writing data, and read transactions for ADLA reading data from ADLS. You will be charged the following:

    Item

    Usage Volume per Month

    Rate per Month

    Monthly Cost

    Write transactions  from customer application

    2 * (10 items/second * 3,600 * 744)

    $0.05 per 10,000 transactions

    $267.84

    Write transactions from Azure IoT Hub

    5 items/second * 3,600 * 744

    $0.05 per 10,000 transactions

    $66.96

    Read transactions from Azure IoT Hub

    1,000 items/second * 3600 * 4 * 31

    $0.004 per 10,000 transactions

    $178.56

    Total Transactions Cost

    $513.36

    The read and write operations generated from all applications and services, including Azure Data Lake Analytics, depend on a large variety of factors. There is no easy way to accurately predict the number of operations generated by every application. To estimate your transaction costs, you should run a representative workload for a controlled duration and observe the number of reads and writes, and the resulting transactions that are billed.

    Tuesday, June 27, 2017 11:54 PM
  • Thank you Amit.

    With this information, I can make a rough estimate for now. I will keep an eye on the FAQ.

    Have a nice day!

    Wednesday, June 28, 2017 6:14 AM