none
How to Implement UPDATE in U-SQL in Data Lake

    Question

  • U-SQL DML has INSERT and SELECT statement . Please suggest a way to Update record using U-SQL
    Sunday, April 16, 2017 5:57 AM

All replies

  • U-SQL does not currently support updating an existing record. In most big data programming models files and tables are considered immutable - once written they are expected to stay unchanged. This means that writing new files/tables and doing appends are the only patterns.

    Common ways to handle updates are to write delta to the system and then either run a query that merges the delta and base into a new file, or create a view that does something similar. In other words, load the base data, load the delta (new versions of some existing records), do selects and joins to create a new merged set, and write the merged set into a new destination while (possibly) dropping the original base set.

    You can see an example of this using u-sql on stackoverflow:

    http://stackoverflow.com/questions/43319388/how-to-merge-base-and-multiple-delta-structured-streams-in-azure-datalake/43328225

    Tuesday, April 18, 2017 8:11 AM