none
Hive cannot access Azure Data Lake store

    Question

  • I have CSV file Azure Data Lake store. I have created HDInsight, created data_lake_service_principal AAD identity and gave read/write/execute rights to data lake store folders and children.  However I cannot access the file with Hive. What could help?

    I'm following this tutorial. https://docs.microsoft.com/en-us/azure/machine-learning/machine-learning-data-science-process-data-lake-walkthrough

    CREATE EXTERNAL TABLE nyc_stratified_sample
    (
        medallion string,
        hack_license string,
        vendor_id string,
        rate_code string,
        store_and_fwd_flag string,
        pickup_datetime string,
        dropoff_datetime string,
        passenger_count string,
        trip_time_in_secs string,
        trip_distance string,
        pickup_longitude string,
        pickup_latitude string,
        dropoff_longitude string,
        dropoff_latitude string,
      payment_type string,
      fare_amount string,
      surcharge string,
      mta_tax string,
      tolls_amount string,
      total_amount string,
      tip_amount string,
      tipped string,
      tip_class string,
      rownum string
      )
    ROW FORMAT DELIMITED FIELDS TERMINATED BY ',' lines terminated by '\n'
    LOCATION 'adl://data_lake_storage_name.azuredatalakestore.net:443/nyctaxi_folder/demo_ex_9_stratified_1_1000_copy.csv'; //edited by my path

     java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got exception: org.apache.hadoop.security.AccessControlException GETFILESTATUS failed with error 0x83090aa2 (Forbidden. ACL verification failed. Either the resource does not exist or the user is not authorized to perform the requested operation.). [3d2e047e-78d2-420a-a4ce-29c5cd755395][2017-07-20T03:29:33.7771117-07:00])


    Kenny_I

    Thursday, July 20, 2017 10:34 AM

All replies

  • Got SQL Exception issue with the same tutorial as below; any pointers? 
    <small style="box-sizing:border-box;font-size:11.05px;">java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:adl://anuadls1.azuredatalakestore.net:443/nyctaxi_folder/demo_ex_9_stratified_1_1000_copy.csv is not a directory or unable to create one) </small>
    Tuesday, March 13, 2018 8:32 AM