Data Factory Copy RRS feed

  • Question

  • I have been investigating the Data Factory Copy activity and have come across some things that don't seem to be inline with a data migration process. I have put together a Data Factory Copy Activity (within a pipeline). The pipeline is scheduled to run for a long period of time. The copy activity works - BUT data is getting copied more that one. This brings up a couple of question:

    1. How do you configure the Dataset OR Activity Or Pipeline to not copy the same data more than once
    2. If data is being copied more than once -- then what s the purpose of a schedule if the same data is constantly being retrieved from the data sources. This could entail a massive processing time when a data source becomes very large. And goes against the concept of only copying data that has not been copied before (which I thought was the purpose of the Copy Activity).
    Saturday, June 3, 2017 11:56 AM


All replies