none
"Allow all Azure services to access this Data Lake Storage Gen1 account." Mandatory ?

    Question

  • Hi,

    When I create an external table in a SQL Datawarehouse that target a file in Azure DataLake Store Gen1.

    In my Datalake target (Firewall and Virtual Network menu). I have to tick the option "Allow all Azure services to access this Data Lake Storage Gen1 account." otherwise I throw an error :

    Msg 105019, Niveau 16, État 1, Ligne 1
    EXTERNAL TABLE access failed due to internal error: 'Error occurred while accessing HDFS: Java exception raised on call to HdfsBridge_IsDirExist. Java exception message:
    Error getting info for file /test/test_adls.txt
    Operation GETFILESTATUS failed with HTTP403 : null
    Last encountered exception thrown after 1 tries. [HTTP403(null)]
     [ServerRequestId:c7ed437f-226f-46ba-a294-d68c0b8dfeff]'

    My test case was very simple : 1 ADLS, 1 SQL Datawarehouse, one file, everything in the same virtual network and subnet (no NSG). Access is made with an application registration registered in the AD.

    Everything runs fine if I tick the option, but fails if not. 

    As it's an exception that is not recommended by Microsoft, I'd like to know if, in this use case, it's mandatory or if there's a solution to avoid ticking this security exception.

    Thanks for your help.

    Laurent

    Monday, February 11, 2019 1:33 PM

All replies

  • Hi Laurent,

    Yes, at this time, you need to enable this configuration item for the pipeline to function. Can you please make an entry in the Azure Data Factory User Voice, please make product enahancements and feature requests for the desired functionality. There are a few entries that are somewhat related, so please do a quick search to see if there is a related thread already created. If not, create a new entry with details of the functionality you are seeking.

    Regards,

    Mike

    Tuesday, February 12, 2019 12:33 AM
    Moderator