none
How to copy multiple SQL Server tables to CSV format in Azure Data Lake RRS feed

  • Question

  • I was successfully able to copy one table data  from SQL Server On-Premise to Azure data applying following tutorial. I use ADF v2.

    I would like to copy 5 tables to Data Lake. How to do with one data pipeline? Currently I have one SQLServerDataSet. Should I create one for each tables? How to call them in Pipeline? Any advice?

    https://docs.microsoft.com/en-us/azure/data-factory/tutorial-hybrid-copy-powershell


    Kenny_I

    Wednesday, December 13, 2017 2:20 PM

Answers

  • Hi Kenny,

    With ADF V2 you don't need to create one dataset for each table.  You can use Lookup activity, ForEach activity, and parameter passing to construct a small number of artifacts (pipelines, activities, datasets) for loading hundreds of tables.

    Here is a tutorial with step-by-step instructions:

    https://docs.microsoft.com/en-us/azure/data-factory/tutorial-bulk-copy

    • Marked as answer by Kenny_I Thursday, December 14, 2017 2:01 PM
    Thursday, December 14, 2017 6:14 AM
    Moderator