locked
How to set security for ADF on ADLS Gen 2 folders.? RRS feed

  • Question

  • Hi,

    I am trying to come up with a security mechanism for our ADFs on Storage account (ADLS Gen 2 hierarchy enabled).

    I gave it storage blob data reader RBAC role on the storage account , we have multiple folders under top container and we like to set Read\Write\Execute permissions to our ADF based on the folders.

    Since i cannot use Storage Blob Data Contributor role at subfolder level , i am using storage explorer and grant RWX on the folder to ADFs managed identity. But still it fails when ADF tried to write in to that folder with forbidden and AuthorizationPermissionMismatch error.

    any idea on how to set permissions on the subfolders so that ADF can write to them.?

    Regards,

    Sai


    Wednesday, February 26, 2020 6:35 PM

All replies

  • any update from experts on this.??
    Friday, February 28, 2020 9:19 PM
  • Hello Sai , 

    Just wanted to point you to : https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-data-lake-storage , it says 

    If you use Data Factory UI to author and the managed identity is not set with "Storage Blob Data Reader/Contributor" role in IAM, when doing test connection or browsing/navigating folders, choose "Test connection to file path" or "Browse from specified path", and specify a path with Read + Execute permission to continue.

    Let me know how it goes .




    Thanks Himanshu

    Saturday, February 29, 2020 12:09 AM
  • I tried it , but still not able to write to the sink.

    On the Sink i gave storage account Storage Blob Data Reader RBAC , and then on one folder in a container i gave RWX ACL for ADF. its not working.

    Regards,

    Sai

    Monday, March 2, 2020 5:25 PM
  • Hello Sai , 

    This is what I did and it worked for me . 

    1.Created a Service Principal from the portal . 

    2.Went to the Portal->Azure active directory -> Enterprise application -> Copied the ObjectID .

    3.Open the Storage Explorer - > Select the Container -> Set the execute(X) permission for the ObjectID

    4.Open the Storage Explorer  -> Move to the Folder -> Set the Permission to R + W 

    The pipeline should execute fine now . Just in case if you are getting the error , please check if the files have been copied or not . I had the error initially but  the file was getting copied just fine .

    Please beware that we have not used any role based access control here . 

    Let me know how it goes .

     



    Thanks Himanshu

    Tuesday, March 17, 2020 6:03 AM
  • Just wanted to walk you through what you are doing wrong . 
    Below was your response.

    I gave it storage blob data reader RBAC role on the storage account , we have multiple folders under top container and we like to set Read\Write\Execute permissions to our ADF based on the folders.

    [Himanshu] : The RBAC access takes a higher precedence over ACL  and when you set the RBAC , ADF will not even bother to check the ACL's . Please remove all the RBAC access provided the Service principal (SP) .
    Use Storage explorer and provide EXECUTE permission on the container ( use the access access option ) .

    Using the Storage explorer please provide the READ , Write & EXECUTE for both access and default on the folders ( I think you are already doing this ) . As I was able to repro the error and also make it work , hope my answer helps you . 


    Since i cannot use Storage Blob Data Contributor role at subfolder level , i am using storage explorer and grant RWX on the folder to ADFs managed identity. But still it fails when ADF tried to write in to that folder with forbidden and AuthorizationPermissionMismatch error.

    any idea on how to set permissions on the subfolders so that ADF can write to them.?

     

    Thanks Himanshu

    Tuesday, March 17, 2020 8:02 PM