Recurring Blob Copy Pipeline - Is it only copying new files? RRS feed

  • Question

  • I have a daily recurring pipeline that transfers files from a blob source into an Azure Data Lake Store instance. I couldn't find explicitly in the documentation any mention of how it determines what files to copy. Can someone confirm if a copy wizard generate pipeline is only going to transfer new files? I'm concerned its transfering every file every time it runs and there are a lot of files.


    - James

    Blades Don't Need Reloading...

    Monday, January 16, 2017 7:33 PM