Azure Data Factory Data Flow Sink Batch Size and Partition Optimization RRS feed

  • Question

  • Looking for some guidance for optimizing our data flow that sinks to Azure SQL database table. By default there is no Sink batch size value in Settings. Under the Sink Optimize the partitioning options is set to - Use Current partitioning. I've put a batch size of 100 and switched the partitioning to round robin and that has reduced the time the data factory runs by 50%.

    Does anyone know how to correctly calculate the batch size for a data flow?

    Are they any guidelines to determine what partitioning strategy one should use?

    Wednesday, March 25, 2020 4:07 PM

All replies