none
Unable to use the REST Data Lake Storage Gen API - Need to know where to get the parameters from ?

    Question

  • I am trying to READ information from my Data Lake Storage Gen2 Filesystem into Collibra Connect, but before I can venture that far I tried to use the API provided here : https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/getproperties

    Specifically the GET Property. I want to know where do I find the information to populate accountName, dnsSuffix, fileSystem, and resource.

    Wednesday, March 13, 2019 5:13 PM

All replies

  • Hello shanksab, and thank you for bringing this issue to our attention.  We are in the midst of updating the Storage SDK to support ADLS Gen2.  After that point we expect much of the REST API usage to move to the SDK.  In the interim, let me share with you the parameters to call the rest API.

    An excerpt

    curl -I -H "x-ms-version: 2018-11-09" -H "Authorization: Bearer $ACCESS_TOKEN" "https://$STORAGE_ACCOUNT_NAME.dfs.core.windows.net/mydata/data/file2?action=getAccessControl"


    The suffix in the case of Files storage, would be "dfs.core.windows.net".

    The filesystem is which of the file stores in your storage account you want to look inside.

    The resource is the filename / path.

    The accountName would be the name of the storage account, as in the storage resource you created in the portal.
    

    The rest you can find by exploring your storage account.

    Suppose I have a fileshare in my storage account with the name "ocean", with a file named "fish.txt".
    https://demostorageaccountname.file.core.windows.net/ocean/fish.txt

    You can explore your storage account and find a url for the file you are looking for.  In the portal, clicking on the file opens a pane on the right, with a URL you can copy.

    Monday, March 18, 2019 6:37 PM
    Moderator
  • Hi Martin,

    Can you please give me an example for the resource. Specifically where you have specified 'resource/path'. The documentation says to simply write the term 'filesystem' for resource field.

    My current path is this: https://dlafstorageacct.file.core.windows.net/dlaftest

    And the file is: Master Data.xlsx

    Here are the parameters I entered here -  https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/filesystem/getproperties

    accountName : dlafstorageacct

    dnsSuffix: dfs.core.windows.net

    filesystem: dlaftest

    resource: ?

    Under resource, I tried

    1.  URL of the file from Azure Storage explorer : https://dlafstorageacct.file.core.windows.net/dlaftest

    2. https://dlafstorageacct.file.core.windows.net/dlaftest/Master Data.xlsx

    3. Master Data.xlsx/ https://dlafstorageacct.file.core.windows.net/dlaftest

    4. Master Data.xlsx/dlafstorageacct.file.core.windows.net/dlaftest

    5. The word filesystem

    6. Just the name of the file - Master Data.xlsx

    None of these seem to be working. Help would be appreciated with this!

    Also is there a timeline as to when the Storage SDK will be updated?

    Thanks

    Monday, March 18, 2019 11:03 PM
  • I found that the "Try it" section of the docs is broken.  Can you use another tool to make the call, such as Postman?

    Also, I may be getting confused.  Are you trying to get the properties of an entire fileshare, or just a single file inside it?  There are different calls for each.

    Monday, March 18, 2019 11:40 PM
    Moderator
  • Hi Martin,

    Trying to do both at this point. If you can outline the approach for both, it will be appreciated.

    From the list of entries for 'resources'  above - if I am trying to access properties of a file, what should it be ? 

    And if I am doing it for an entire fileshare, what should it be ?

    Thanks!

    Tuesday, March 19, 2019 3:54 PM
  • The REST API has been very difficult.  Here is a related thread: https://social.msdn.microsoft.com/Forums/en-US/45be0931-379d-4252-9d20-164261cc64c5/error-while-calling-adls-gen-2-rest-api-to-create-file?forum=AzureDataLake
    Monday, March 25, 2019 9:37 PM
    Moderator