none
Writing u-sql queries over linked azure blob storage account

    Question

  • Hi there,

    We have a lot of data to analyse but this is all stored in the Europe West region. Azure Data Lake analytics is only available in the Europe North region. What are our options? I can think of:

    1. Copy the data on a schedule to the Europe North region Data Lake.
    2. Link the blob storage in the West Europe region.

    I have some questions about option 2

    • Will the data be copied to the Europe north datacenter each time we submit a job? Or is the data processed on the location
    • How will this affect performance? If so, what kind of performance degradation can we expect?

    We are talking about roughly a couple of GB of data stored in one azure blob storage container divided over a couple of hundred json files (each file contains multiple json documents, one per line).

    Thursday, August 17, 2017 8:39 AM

All replies

  • Hi

    If you link the WE blob storage to the NE region ADL account, then everytime you run a job the data will get copied which will incur data data egress charges and will negatively impact your performance.

    Thus copying the data once on schedule is more cost effective. Alternatively, if you are ok with using a preview version of ADL in the WE region, please contact me via email usql at microsoft dot com to see if that works for you.

    Cheers

    Michael


    Michael Rys

    Thursday, August 17, 2017 9:48 PM
    Moderator