none
Unable to Read Image files on DataLakeStore

    Question

  • Using R, I am trying to read image files on Azure DataLakeStore. It is not working.

    however, i can read the text files sitting next to image file, using same code, with no issues.

    also, I can read image files which is openly available on internet.

    1. Code to read JPG on DataLakeStore (Doesnt Work)

    a <- GET("https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/pic.jpg?op=OPEN&read=true",
                   add_headers(Authorization = "Bearer <Auth Token>"))

    a$status_code == 403

    OR 

    a <- GET("https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/pic.jpg",
                   add_headers(Authorization = "Bearer <Auth token>"))

    a$status_code  == 400

    plot(0:1, 0:1, type = "n")
    rasterImage(rlogo, 0, 0, 1, 1) 


    2. Code to read JPG on Open Web (Works fine)

    a <- GET("http://cran.r-project.org/Rlogo.jpg")

    a$status_code == 200

    x <- content(a)

    plot(0:1, 0:1, type = "n")

    rasterImage(x, 0, 0, 1, 1)

    3. Code to read Text (sitting next to image file on DataLakeStore (Works fine)

    a <- GET("https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<myfolder>/testfile3.txt?op=OPEN&read=true",
                           add_headers(Authorization = "Bearer <Auth Token>"))

    a$status_code == 200

    ------------------

    Update:

    Update on issue

    I am able to zero in on the issue.But dont know how to proceed.

    Azure API : https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/file.txt?op=OPEN&read=true
    above code works...
    but below doesnt.
    https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/pic.jpg?op=OPEN&read=true

    I need to find out ... what below command should i replace for image
    for text this works ->  ?op=OPEN&read=true

    • Edited by Ravi.p.rai Friday, July 28, 2017 7:54 PM
    Thursday, July 27, 2017 9:15 PM

All replies

  • Update on issue

    I am able to zero in on the issue.But dont know how to proceed.

    Azure API : https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/file.txt?op=OPEN&read=true
    above code works...
    but below doesnt.
    https://<mydatalakestore>.azuredatalakestore.net/webhdfs/v1/<folder>/pic.jpg?op=OPEN&read=true

    I need to find out ... what below command should i replace for image
    for text this works ->  ?op=OPEN&read=true

    Friday, July 28, 2017 7:53 PM
  • If you Login to your Azure Data Lake Store account and download the file (pic.jpg) does it work ? As a first step, it would be good to eliminate any access issues you may have. 
    Monday, July 31, 2017 6:42 PM
  • Yes. No issues downloading the file.
    using RENAME() i have renamed the file just to be sure.
    Monday, July 31, 2017 7:13 PM
  • Just to make sure that we did not miss a step, have you followed this two step process : 

    ( https://docs.microsoft.com/en-us/azure/data-lake-store/data-lake-store-get-started-rest-api#read-data-from-a-data-lake-store-account)

    Read data from a Data Lake Store account

    This operation is based on the WebHDFS REST API call defined here.

    Reading data from a Data Lake Store account is a two-step process.

    • You first submit a GET request against the endpoint https://<yourstorename>.azuredatalakestore.net/webhdfs/v1/mytempdir/myinputfile.txt?op=OPEN
    • This will return a location to submit the next GET request to.

    • You then submit the GET request against the endpoint https://<yourstorename>.azuredatalakestore.net/webhdfs/v1/mytempdir/myinputfile.txt?op=OPEN&read=true> This will display the contents of the file.


    • Edited by YaGupta Monday, July 31, 2017 8:08 PM
    Monday, July 31, 2017 8:07 PM