none
Azure Automation bug in submitting ADLA job

    Question

  • Hi,

    I have a problem with Azure Automation in conjunction with Azure Data Lake Analytics. 

    The use case is, I have automation account with RunAsAccount which is the subscription contributor. Even I can via this automation account manage Azure services, list Azure Dala Lake jobs, connected storage accounts, I cannot submit ADLA job. The App user (RunAsAccount) is listed in ADLA Access control (IAM) as contributor. ADLA firewall is disabled. If I submit ADLA job directly in Azure Data Lake Analytics, everything is fine. USQL job perform dara reading from Azure SQL Database and writes data into Azure Blob Storage. But if I submit the same job via Azure Automation then I get an error:

    Submit-AzureRmDataLakeAnalyticsJob : The user is not authorized to perform this operation on storage.
    At line:65 char:1
    + Submit-AzureRmDataLakeAnalyticsJob -Name $JobName -AccountName $AdlaA ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : CloseError: (:) [Submit-AzureRmDataLakeAnalyticsJob], CloudException
        + FullyQualifiedErrorId : Microsoft.Azure.Commands.DataLakeAnalytics.SubmitAzureDataLakeAnalyticsJob
    

    It's critical issue. It's a new feature issue, because I did this solution previously in different subscriptions and it is still working for several my clients. But new automation account does not work with ADLA.

    THe ADLA job is fine

    For example, following works fine without errors:

    Get-AzureRmDataLakeAnalyticsAccount -Name $AdlaAccountName

    Get-AzureRmDataLakeAnalyticsDataSource -Account $AdlaAccountName

    Get-AzureRmDataLakeAnalyticsJob -Account $AdlaAccountName -SubmittedAfter (Get-Date).AddDays(-7)

    But this one is not working in new created Automation Account:

    Submit-AzureRmDataLakeAnalyticsJob -Name $JobName -AccountName $AdlaAccountName -DegreeOfParallelism 3 -Script $UsqlScript

    I got an error in detail (debug mode). In report below I changed request id:

    ============================ HTTP RESPONSE ============================
    Status Code:
    Forbidden
    
    Headers:
    Transfer-Encoding             : chunked
    x-ms-request-id               : xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
    X-Content-Type-Options        : nosniff
    Strict-Transport-Security     : max-age=15724800; includeSubDomains
    Cache-Control                 : no-store, no-cache, max-age=0, private
    Date                          : Thu, 28 Sep 2017 21:01:33 GMT
    
    Body:
    {
      "error": {
        "code": "StorageAccessDenied",
        "message": "The user is not authorized to perform this operation on storage."
      }
    }

    Friday, September 29, 2017 1:26 PM

All replies

  • No help here?
    Monday, October 2, 2017 12:39 PM
  • Hi,

    This error message usually appears when the user performing the operation doesn't have sufficient permissions to metadata in the Data Lake Store filesystem, where the service stores info about running jobs.

    To make sure you have permissions set up correctly, please follow the steps outlined in the "Add a new user" section of this article.

    We'll improve this error message to make it clearer what went wrong and how to fix it.

    I hope this helps!

    Best regards,

    Matthew

    Azure Data Lake team

    Monday, October 2, 2017 4:46 PM
  • Hello Peter,

    Did you get this error message resolved. I'm building CI/CD for ADLA - USQL and having azure powershell scripts for deploying this release using VSTS and facing this below error while deployment - 

    The user is not authorised to perform this operation on storage.

    I assume since I'm deploying using vsts the release will be deployed through a build user and how do i give access to this user and where i can get the user details. Is this my VSTS login id ?? I use same id for vsts and visual studio code. Any help or suggestions will be appreciated.

    The user is not authorized to perform this operation on storage.

    Thursday, April 5, 2018 2:02 PM