How Data lake can save data in source Native format?


    • While going through the concepts about the Data lake, it suggest that data lake can preserve data in source native format only.
    • But while looking into a scenario of copying ‘On Premises SQL Table’ to ‘Azure lake’, we have observed that it converts ‘On Premises SQL Table’ data into ‘Data lake file(while creating Dataset for Azure lake we have to specify output folder path and format)’. Then how can we say that data are in their native format? Click here for reference link.
    • And if We are copying On Premise SQL table to Azure SQL or Blob storage then these 2 (Azure SQL and Blob Storage) becomes external components and not a part of the Azure Lake. Analytic account can access blob Storage, Azure SQL and other source for further process but these becomes external components and not a part of lake hence how can we say that we have moved all the data @ 1 place that is lake?

    Can it be clear how it is saving data of any format?

    Thanks, Manthan Upadhyay

    Wednesday, March 16, 2016 12:27 PM


  • The key concept to know about Azure Data Lake store is that it does not require one to transform data a certain format for copying data to the data lake. Using a tool of your choice or a custom tool, it is certainly possible to copy a SQL  table in the example above as a database file, as long as you have the ability to read the data and process the data in that format when needed.

    Thus, the format is dictated by the tool copying the data, which will further enable processing for the analytics engine of your choice.

    As for your second question, even while you copy the on-premises relational data to Azure SQL, the Azure Data Lake Analytics (the U-SQL analytics engine) supports in-place querying across these disparate sources of data, thus making the whole process seamless from the perspective of data analytics.

    in summary, for the example above, you have the option to copy data at one place and/or analyze it from one place.

    Wednesday, March 16, 2016 12:59 PM