none
Incremental copy from Lake store to Lake store using Data factory

    Question

  • Incremental copy from Lake store to Lake store using Data factory-

    Whenever we copy the lake store files from to backup lake store it make changes to old files as well but the requirement is to just copy the new files as a incremental data or files copy 

    Thursday, April 5, 2018 9:24 AM

All replies

  • You can try getting delta data which are not already inserted in backup lake store. The delta of source and sink store can be extracted in the transform field by removing the data which are already present in the sink store. Then just store the delta data into the sink.

    You can take the inspiration from the below link:

    https://docs.microsoft.com/en-us/azure/data-factory/tutorial-incremental-copy-portal

    Monday, April 9, 2018 8:37 PM
    Moderator