none
DLAU Usage - What Does this Mean in the Context of a USQL job?

    Question

  • So I was running a USQL script in Visual Studio which simply created a user-defined table function for reading in a web log.  I noticed that the entire run time was over 20 seconds and was concerned so I looked at the Diagnostics tab on the job summary page.  It indicated that I used 5 DLAUs while the script was running.  What does DLAU stand for and how does using 5 of them represent a script warning on the diagnostics tab?  Thanks.
    Saturday, August 20, 2016 7:53 PM

Answers

  • When you submit a job, you indicate how many nodes you want to reserve and use as maximal degree of parallelism. These nodes are also referred to as Azure Data Lake Analytics Units (or (A)DLAU in short).

    Now when we run your script, we will take a look at how the data can be partitioned, and what operations you are doing on that data to decide what your actual degree of parallelism is. E.g., if you operate on a small set of data that fits into one partition, we probably will not parallelize your job to your specified max degree. On the other hand, if we think we can parallelize to a larger degree than you specified, we will honor your limit and not parallelize more.

    Since you are currently paying for the DLAUs you specify as max degree of parallelism, we give you the diagnostic warning, if you overprovisioned the job and we did not use all. That way, you can improve your utilization and cost by submitting your job with a lower max.


    Michael Rys

    • Marked as answer by Brian Custer Sunday, August 21, 2016 9:32 PM
    Sunday, August 21, 2016 8:58 PM
    Moderator

All replies

  • Brian can you share a screenshot of the warning and any additional information that is present in Visual Studio about the warning?
    Saturday, August 20, 2016 8:02 PM
    Moderator
  • Okay, I've submitted a screen shot as you requested.Screen Shot of Diagnostics Tab

    Brian L Custer

    Saturday, August 20, 2016 8:07 PM
  • When you submit a job, you indicate how many nodes you want to reserve and use as maximal degree of parallelism. These nodes are also referred to as Azure Data Lake Analytics Units (or (A)DLAU in short).

    Now when we run your script, we will take a look at how the data can be partitioned, and what operations you are doing on that data to decide what your actual degree of parallelism is. E.g., if you operate on a small set of data that fits into one partition, we probably will not parallelize your job to your specified max degree. On the other hand, if we think we can parallelize to a larger degree than you specified, we will honor your limit and not parallelize more.

    Since you are currently paying for the DLAUs you specify as max degree of parallelism, we give you the diagnostic warning, if you overprovisioned the job and we did not use all. That way, you can improve your utilization and cost by submitting your job with a lower max.


    Michael Rys

    • Marked as answer by Brian Custer Sunday, August 21, 2016 9:32 PM
    Sunday, August 21, 2016 8:58 PM
    Moderator