none
Implementing Upsert- U-SQL Table

    Question

  • Hi,

    Is there any limitation of U-SQL table for Upsert operations. My requirement is daily I will get one file which I need to use to update my existing data in U-SQL table. Can I use U-SQL for insert/update operations to manage incremental changes. Please advise.

    Also, the data stored in U-SQL table is it encrypted? Please let me know.

    Thanks,

    Soumitra.

    Monday, July 30, 2018 3:43 PM

Answers

  • Yes, you can. Since you don't have update command, you have to use a similar workaround as above for tables. if you are using the internal U-SQL tables. The tables are stored in data lake store and data lake store is encrypted. You can study more about data lake store encryption here.
    Monday, July 30, 2018 9:39 PM
    Moderator

All replies

  • Hi Soumitra,

        U-SQL does not support updating an existing record. But you can achieve this task by:

    • extracting the existing data and new data separately.
    • use select and join operation to create a merged data set
    • Store the output

    For reference, please take a look at this post

    http://stackoverflow.com/questions/43319388/how-to-merge-base-and-multiple-delta-structured-streams-in-azure-datalake/43328225

    Monday, July 30, 2018 6:59 PM
    Moderator
  • Hi,

    Thank you for your reply. Can I use U-SQL table for upsert? Also, the data stored in U-SQL table is  encrypted?

    Can you please let me know.

    Thanks,

    Soumitra.

    Monday, July 30, 2018 7:16 PM
  • Yes, you can. Since you don't have update command, you have to use a similar workaround as above for tables. if you are using the internal U-SQL tables. The tables are stored in data lake store and data lake store is encrypted. You can study more about data lake store encryption here.
    Monday, July 30, 2018 9:39 PM
    Moderator
  • Thank you!
    Monday, July 30, 2018 9:52 PM