none
Reading Data from Azure Data Lake

    Question

  • 1) How to check which files are in Azure Data Lake Storage, how to view them, check their size, view/read them in portal?

    2) While copying Data from Azure SQL on Premise to Azure Data Lake Storage  do we have any option in which format to store  data or Azure Data Lake Storage stores data by default in some native format.

    3) I have installed Azure Storage Explorer and can view list of files but how to read them. Do I need to copy data from these files to Azure SQL Database or how to read these files OR depending on the format file is saved in Azure Data Lake Storage we  need to install some other tool to read that type of file.




    Saturday, January 5, 2019 4:31 AM

All replies

  • Hi Dhiraj,

    With reference to Azure Data Lake Storage Gen 1 :

    1. On the portal, you can navigate to your Data Lake Store and go to the Data Explorer to read/write files, check their size, etc.

    2. You can use Azure Data Factory to copy data from Azure SQL to Azure Data Lake Storage and specify the file format under the file format settings. The supported formats are - Text, Avro, JSON, ORC and Parquet. You can also use the detect text format feature to select the suitable format for you.

    3. In the latest version (1.6.1) of Azure Storage Explorer, you can read the files by double clicking on them or by clicking on "Open"

    PS : Azure Data Lake Storage Gen 2 is a specialised Storage account and hence it's contents can be read as you would do for a normal Storage account. From Azure Storage Explorer, you will have to download your blobs locally to view them.

    Let us know if this helps. Else, we can continue to probe in further.


    MSDN

    Monday, January 7, 2019 9:19 AM
    Moderator