none
Azure Data Lake GEN2 REST API

    Question

  • We are converting from REST API GEN1 to REST API GEN2 and I could use some CURL Examples. I see the examples to create a directory or file but I need an example of the list command to check for a file or directory and the upload of a file to the data lake.
    Thursday, April 4, 2019 4:03 PM

All replies

  • Hi Barney,

    CURL example to list files in a directory:

    When listing files in the file system, you have the option to allow recursion in order to list all files including subdirectories.

    To list all files recursively from the root, run the following command:

    curl -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer $ACCESS_TOKEN" "https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/mydata?resource=filesystem&recursive=true" | jq "."

    To list just the files in a single directory, run this command:

    curl -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer $ACCESS_TOKEN" "https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/mydata?resource=filesystem&directory=data&recursive=false" | jq "."

    Hope this helps.

    Friday, April 5, 2019 7:08 AM
    Moderator
  • Hi Barney,

    Just checking in to see if the above answer helped. If this answers your query, do click “Mark as Answer” and Up-Vote for the same. And, if you have any further query do let us know.

    Tuesday, April 9, 2019 6:04 AM
    Moderator
  • Hi  Thank you for the response.  I am looking to see if the directory exists or not or a specific file. How would I do that ? Also I was testing and it looks like the Authorization Token call for Gen1 is different then Gen2. Is there another curl command to get the access token for an application in Gen2 ?
    Thursday, April 18, 2019 2:53 PM
  • Hi Barney,

    You can acquire an Access token by making a POST request to https://login.microsoftonline.com/<tenant id>/oauth2/v2.0/token

    Headers :

    "Content-Type: application/x-www-form-urlencoded"

    Body : 

    {"client_id": <CLIENT_ID>,

    "client_secret": <CLIENT_SECRET>,

    "scope" : "https://storage.azure.com/.default",

    "grant_type" : "client_credentials"

    }

    Also, for more details, please please have a look at the proposed answer on this thread. I have posted a comprehensive step-by-step guide to use ADLS Gen2 REST APIs.

    Hope this helps.


    MSDN

    Tuesday, April 23, 2019 1:07 PM
    Moderator