none
pipeline with multiple dataflows activities RRS feed

  • Question

  • I have a requirement where I have to migrate a bunch of tables(for one product) and a different set of tables for another product. Using (per table->per data-flow->per pipeline) will cause creating huge pipelines which will be un-maintainable and costly.

    Can i use multiple data flows in single pipeline(1-Product->1-Pipeline->multiple data-flows for multiple tables) instead for creating new pipeline for each table and how? please suggest...

    I am not sure if for-each Activity will work here as it require something unique in tables to loop through which is not the case here.

    Friday, February 14, 2020 11:10 PM

Answers

  • You can pack this into different areas of ADF.

    Each table source and sink and can be packed into a single data flow.

    You can have many data flow activities in a pipeline, executed in parallel or sequentially.

    You can also for For-Each to loop through tables.

    Saturday, February 15, 2020 12:05 AM

All replies

  • You can pack this into different areas of ADF.

    Each table source and sink and can be packed into a single data flow.

    You can have many data flow activities in a pipeline, executed in parallel or sequentially.

    You can also for For-Each to loop through tables.

    Saturday, February 15, 2020 12:05 AM
  • Do we have any reference link or example?
    Monday, February 17, 2020 5:59 PM