locked
Not able to upload file to adls gen2 RRS feed

  • Question

  • getting MissingRequiredHeader every time

    curl -i -X PUT -L -T 'test.txt' -H 'authorization: Bearer <token>' \
    https://<storagename>.dfs.core.windows.net/<filesystem>/folder/test.txt?operation=CREATE

    HTTP/1.1 400 An HTTP header that's mandatory for this request is not specified.

    Content-Length: 204
    Content-Type: application/json;charset=utf-8
    Server: Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0
    x-ms-error-code: MissingRequiredHeader
    x-ms-version: 2018-03-28
    Date: Tue, 17 Sep 2019 08:35:51 GMT


    Tuesday, September 17, 2019 8:43 AM

All replies

  • Hi Manish,

    You are missing the following header :

    "x-ms-version" : "2018-11-09"

    Please set the same and give it a try.

    Hope this helps.

    Tuesday, September 17, 2019 11:25 AM
  • No still same issue .

    curl -i -X PUT -L -T test.txt \
    -H 'authorization: Bearer <token>' \
    https://<storage>.dfs.core.windows.net/<filesystem>/folder/test.txt?action=CREATE \
    -H 'x-ms-version: 2018-11-09'

    {"error":{"code":"MissingRequiredHeader","message":"An HTTP header that's mandatory for this request is not specified.\nRequestId:4a371be4-7


    Tuesday, September 17, 2019 11:38 AM
  • Hi Manish,

    Can you please try passing another header :

    x-ms-date : <current date>

    Hope this helps.

    Wednesday, September 18, 2019 8:32 AM
  • Still not luck.

    DATE_ISO=$(TZ=GMT date "+%a, %d %h %Y %H:%M:%S %Z")

    curl -i -X PUT -L -T test.txt \
    -H 'authorization: Bearer <Token> \
    -H 'x-ms-date:$DATE_ISO' \
    -H 'x-ms-version: 2018-03-28' \
    https://<storage>.dfs.core.windows.net/<filesystem>/folder/test.txt?action=CREATE

    {"error":{"code":"MissingRequiredHeader","message":"An HTTP header that's mandatory for this request is not specified."}}

    Wednesday, September 18, 2019 9:56 AM
  • Hi Actually if i try with this url.

    https://<storage>.dfs.core.windows.net/<filesystem>/folder/sample_1.txt?resource=file

    Then i am able to create empty file with name sample_1.txt but if i add any file or content to this file  its throwing error

    {"error":{"code":"ContentLengthMustBeZero","message":"The Content-Length request header must be zero.


    Wednesday, September 18, 2019 1:18 PM
  • Hi Manish,

    To update an exisiting file, you can do the following : 

    HTTP VERB:- PATCH

    URL:- https://<storageaccount>.dfs.core.windows.net/datalakefs2/test5.txt?action=append&position=0

    Headers:-

    Authorization:- Bearer <access_token>

    Content-Type:- text/plain

    Content-Length:-9

    Body:- test data

    Once you do the PATCH operation, the text sits in an uncommitted buffer on the server. To endure data to the file system, flush the file using the following API :

    https://<storage account name>.dfs.core.windows.net/<filesystem name>/<folder>/<file>?action=flush&position=<no. of characters>

    Note : No. of characters is the same as the Content-Length in the previous (PATCH) step - 9 in your case.

    Headers :

    Content-Length : 0

    "x-ms-version":"2018-11-09"

    Content-Type : text.plain

    Authorization : Bearer <access_token>

    Hope this helps.



    Monday, September 23, 2019 8:31 AM
  • Thanks Chirag, But i am trying to upload existing file to adls as file content is more i cant give Content-Length i want to push existing file to adls from my local system.
    Monday, September 23, 2019 12:04 PM
  • Content-Length is the size of your file in bytes.  It just happens that ASCII characters are each 1 byte.  Extended unicode characters may be multiple bytes per character.

    Monday, September 23, 2019 9:25 PM