locked
Clarity around ADF costs RRS feed

  • Question

  • I have a client who are interested in near real time data ingestion, but ADF seems like it may simply be too cost prohibitive.

    The requirement is to load data into Azure SQLDB from on premise (mostly SQL Server Databases and Oracle Databases) in near real time. Let’s assume this is once every 5 minutes across 20 databases. I am struggling to cost this from an ADF perspective, and wondered if someone could provide sound advice.

    20 databases x 5 minute intervals into SQLDB.

    This seems to equate to 12 data movement activities for 20 databases each hour, or 12 x 20 x 24 x 31 = 178,560 activities per month (or do I exclude the 20 databases, which makes it 8,928 activities per month). Given the standard Azure calculator (using Australian Dollar) this seems to equate to an eye watering cost:

    • For 178,560 activities - $127,368.73/mo
    • For 8,928 activities - $11,371.59/mo

    Am I misunderstanding the cost? Should I be focusing on data movement hours only? I cannot really find any definitive description as to what constitutes an activity. If its data movement hours ONLY and no activity, then that equates to a much more palatable Dollar value.


    'If this was helpful, please mark and an answer or as being helpful'

    Wednesday, May 31, 2017 5:57 AM

All replies

  • as far as I understand the pricing model, you pay per activity and not by the number of executions of the activity

    also, the lowest time frame you can use would be 15 minutes in ADF

    -gerhard


    Gerhard Brueckl
    blogging @ http://blog.gbrueckl.at
    working @ http://www.pmOne.com

    Wednesday, May 31, 2017 11:32 AM
  • What do you mean by activity vs number of executions of the activity?

    Lets assume I create a data movement task. I.e. load data from an on premise SQL Database into Azure SQLDB. And I do this every 15 minutes. How does the costing work?


    'If this was helpful, please mark and an answer or as being helpful'

    Tuesday, June 6, 2017 5:10 AM
  • this is the official page (which I guess you also used):
    https://azure.microsoft.com/en-us/pricing/details/data-factory/

    there are basically two things that you pay for 

    1) the orchestration of the activities (thats what I was referring to with my first post)
    2) the number of hours the activities are running

    regarding 1)
    for 20 databases you would have 20 activities. As they execute more then once a day you need to take the high-frequency price. further, the data management gateway is involved. 

    so you would calculate 20 x $3.185 (assuming the ADF runs in West Central US) = $63.7/Month

    regarding 2)
    this very much depends on how long your activities actually run. If each of them only runs for 2 minutes then you would calculate it like this: 20 DBs * 12 executions/hour (=5 minutes interval)* 24 hours * 30 days * 2 minutes = 345600 minutes = 5760 hours * $0.128 = $737/Month

    in total this would be about $800/Month

    thats at least my understanding of how the pricing works

    -gerhard


    Gerhard Brueckl
    blogging @ http://blog.gbrueckl.at
    working @ http://www.pmOne.com

    Tuesday, June 6, 2017 7:05 AM