none
Choose ETL tools RRS feed

  • Question

  • When choose an ETL tools for loading data to Azure SQL datawarehouse.

    Are there any comparison between using Azure Data Factory and PolyBase ?

    Thursday, September 6, 2018 10:06 AM

Answers

  • ADF leverages PolyBase when loading data into Azure SQL Data Warehouse:

    https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-sql-data-warehouse#use-polybase-to-load-data-into-azure-sql-data-warehouse

    Through ADF integration, you get the same optimized load speeds plus a few additional benefits:

    - ADF issues SQL queries against SQL DW to create external data source and external data table to set up for Polybase, so the user does not need to perform those operations manually

    - The ingestion can be triggered on demand for historical load and on a schedule for incremental load

    - The SQL DW load can be dependency-chained with previous & subsequent activities (e.g. after running Databricks Notebooks to process data in Azure Data Lake, load processed results into SQL DW)

    Thursday, September 6, 2018 1:11 PM
    Moderator

All replies

  • Hello,

    Loading patterns and strategies to Azure SQL Data Warehouse are compared on the following article:

    https://blogs.msdn.microsoft.com/sqlcat/2017/05/17/azure-sql-data-warehouse-loading-patterns-and-strategies/


    Polybase is considered the preferred and fastest loading method for ingesting data into Azure SQL Data Warehouse


    Hope this helps.


    Regards,

    Alberto Morillo
    SQLCoffee.com


    Thursday, September 6, 2018 12:24 PM
  • ADF leverages PolyBase when loading data into Azure SQL Data Warehouse:

    https://docs.microsoft.com/en-us/azure/data-factory/connector-azure-sql-data-warehouse#use-polybase-to-load-data-into-azure-sql-data-warehouse

    Through ADF integration, you get the same optimized load speeds plus a few additional benefits:

    - ADF issues SQL queries against SQL DW to create external data source and external data table to set up for Polybase, so the user does not need to perform those operations manually

    - The ingestion can be triggered on demand for historical load and on a schedule for incremental load

    - The SQL DW load can be dependency-chained with previous & subsequent activities (e.g. after running Databricks Notebooks to process data in Azure Data Lake, load processed results into SQL DW)

    Thursday, September 6, 2018 1:11 PM
    Moderator