none
U-SQL Blob Storage local test

    Question

  • Hello I am trying to test reading data from blob storage by using U-SQL Locally.

    @assets =
        EXTRACT [QID] int,
                Title string,
                [Vuln Status] string,
                Type string,
                Severity int
        FROM "wasb://security@<storage-account-name>.blob.core.windows.net/csv-unique.csv"
        USING Extractors.Csv(skipFirstNRows: 1);

    This is the error I am getting when trying to extract data from the file.

    Error E_STORE_USER_FILENOTFOUND: File not found or access denied: wasb://security@<storage account name>.blob.core.windows.net/csv-unique.csv.
    Description:
    Unable to obtain information about this file.
    Resolution:
    Ensure the file with exact casing exists (paths are case-sensitive) and the account has read permission to the file.

    Please let me know what I am doing incorrectly. 


    Wednesday, March 20, 2019 9:28 PM

Answers

  • Hi Nathaniel Moss,

    This is not possible to locally grab the csv in blob storage from the endpoint without an ADL account.

    You need to use Windows Azure Blob Storage accounts need to be registered with the Azure Data Lake Analytics account where you will run your U-SQL script.

    Friday, March 22, 2019 5:54 AM
    Moderator

All replies

  • Hi Nathaniel Moss,

    U-SQL provides access to files that are stored either in Azure Data Lake Storage accounts or in Windows Azure Blob Storage accounts. Azure Data Lake Storage accounts have to be in the same Azure region as the Azure Data Lake Analytics account where you run your U-SQL script. Windows Azure Blob Storage accounts need to be registered with the Azure Data Lake Analytics account where you will run your U-SQL script but can be in any region.

    Note that if the data is in a different region than the Azure Data Lake Analytics account, you will get billed for data egress and data transfer rates will be based on the internet connection between the two regions and considerably lower than accessing data inside the same region. Any input files referenced in a script have to exist at the time the script is compiled.

    For more details, refer “Input Files (U-SQL)”.

    And also, you may refer the MSDN and SO thread which addressing similar issue.

    Hope this helps.

    Thursday, March 21, 2019 3:53 PM
    Moderator
  • I am attempting to run this query locally without an ADL account as a proof of concept for the office. 

    Is it not possible for me to locally grab the csv in blob storage from the endpoint without an ADL account?


    Thursday, March 21, 2019 5:27 PM
  • Hi Nathaniel Moss,

    This is not possible to locally grab the csv in blob storage from the endpoint without an ADL account.

    You need to use Windows Azure Blob Storage accounts need to be registered with the Azure Data Lake Analytics account where you will run your U-SQL script.

    Friday, March 22, 2019 5:54 AM
    Moderator