locked
ADLS Gen2: Creating a filesystem with REST API from python RRS feed

  • Question

  • I get an error during the call for a filesystem creation:

    b'\xef\xbb\xbf<?xml version="1.0" encoding="utf-8"?>\n<Error><Code>MissingRequiredHeader</Code><Message>An HTTP header that&apos;s mandatory for this request is not specified.\nRequestId:0e8a8ca6-601e-0113-4216-72967a000000\nTime:2019-09-23T13:51:43.9234964Z</Message><HeaderName>x-ms-blob-type</HeaderName></Error>' {'Content-Length': '301', 'Content-Type': 'application/xml', 'Server': 'Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0', 'x-ms-error-code': 'MissingRequiredHeader', 'x-ms-request-id': '0e8a8ca6-601e-0113-4216-72967a000000', 'x-ms-version': '2018-11-09', 'Date': 'Mon, 23 Sep 2019 13:51:42 GMT'}

    I'm using python 3.6.4.

    The header:

    import requests
    headers1 = {
        'Authorization': "Bearer %s" %token,
        'content-length': '0',
        'Content-Type': 'application/json',
        'x-ms-version': '2018-11-09'
    }

    Monday, September 23, 2019 1:56 PM

Answers

  • Okay.  I see a likely cause now.  You are mixing blob and ADLS gen2 endpoints.  They are differentiated in the {dnsSuffix} mentioned above.  Here is example for comparison.

    Gen2 endpoint:
    https://myStorageName.dfs.core.windows.net/
    
    Blob endpoint:
    https://myStorageName.blob.core.windows.net/

    You can get the url in the portal by going to your storage account, and clicking on the properties blade.

    When I get a token for working with storage, I use the header
    'scope': 'https://storage.azure.com/.default'
    I do not use
    'resource': 'https://management.azure.com/'

    • Marked as answer by F.he Thursday, September 26, 2019 1:37 PM
    Tuesday, September 24, 2019 6:26 PM

All replies

  • Hello F.he and thank you for your inquiry.

    Could you please share what your URL looks like?  Similar errors have been known to happen when the

    ?resource=filesystem

    is left off of

    https://{accountName}.{dnsSuffix}/{filesystem}?resource=filesystem

    .  Without it, the meaning of the call is unclear.  Compare:

    Create File/Folder:
    https://{accountName}.{dnsSuffix}/{filesystem}/{path}?resource={resource}
    
    Create Filesystem:
    https://{accountName}.{dnsSuffix}/{filesystem}?resource=filesystem

    If the {path} is empty (top level directory), then the two calls look similar except for the resource piece.

    Monday, September 23, 2019 9:34 PM
  • 1)I get an access token

    import requests
    # 'resource': 'https://management.azure.com/'
    data = {
      'grant_type': 'client_credentials',
      'client_id': 'ee162673-838d-48ad-a730-976b1aba9b38',
      'client_secret': 'XXXXXXXXXXXX',
      'resource': 'https://management.azure.com/'
    }
    response = requests.post('https://login.microsoftonline.com/72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/token', data=data)

    2) I call the filesystem creation

    headers2 = {
        'Authorization': "Bearer %s" %token,
        'content-length': '0',
        'Content-Type': 'application/json',
        'x-ms-version': '2018-11-09',
        'x-ms-blob-type': 'BlockBlob'
    }
    response2 = requests.put("https://storengyacct1.blob.core.windows.net/mydata?resource=filesystem", headers=headers2)

    3) Now by adding the 'x-ms-blob-type': 'BlockBlob' I have a new error message:

    b'\xef\xbb\xbf<?xml version="1.0" encoding="utf-8"?><Error><Code>AuthenticationFailed</Code><Message>Server failed to authenticate the request. Make sure the value of Authorization header is formed correctly including the signature.\nRequestId:73705133-201e-007f-0eb5-7283e0000000\nTime:2019-09-24T08:49:47.3327156Z</Message><AuthenticationErrorDetail>Audience validation failed. Audience did not match.</AuthenticationErrorDetail></Error>' {'Content-Length': '426', 'Content-Type': 'application/xml', 'Server': 'Microsoft-HTTPAPI/2.0', 'x-ms-request-id': '73705133-201e-007f-0eb5-7283e0000000', 'x-ms-error-code': 'AuthenticationFailed', 'Date': 'Tue, 24 Sep 2019 08:49:46 GMT'}


    • Edited by F.he Tuesday, September 24, 2019 8:51 AM
    Tuesday, September 24, 2019 8:50 AM
  • Okay.  I see a likely cause now.  You are mixing blob and ADLS gen2 endpoints.  They are differentiated in the {dnsSuffix} mentioned above.  Here is example for comparison.

    Gen2 endpoint:
    https://myStorageName.dfs.core.windows.net/
    
    Blob endpoint:
    https://myStorageName.blob.core.windows.net/

    You can get the url in the portal by going to your storage account, and clicking on the properties blade.

    When I get a token for working with storage, I use the header
    'scope': 'https://storage.azure.com/.default'
    I do not use
    'resource': 'https://management.azure.com/'

    • Marked as answer by F.he Thursday, September 26, 2019 1:37 PM
    Tuesday, September 24, 2019 6:26 PM
  • Please let me know if this helped, or if you would like more details.
    Thursday, September 26, 2019 12:15 AM