locked
Data Factory Encryption RRS feed

  • Question

  • I'd need explicit statement on how Data Management Gateway -> Data Factory traffic is encrypted, but I couldn't find any good references. The best what I have found are:

    https://docs.microsoft.com/en-us/azure/data-factory/data-factory-data-management-gateway

    ...The gateway copies data from an on-premises store to a cloud storage, or vice versa depending on how the Copy Activity is configured in the data pipeline. For this step, the gateway directly communicates with cloud-based storage services such as Azure Blob Storage over a secure (HTTPS) channel....

    https://docs.microsoft.com/en-us/azure/sql-database/sql-database-security-overview

    ...All connections to Azure SQL Database require encryption (SSL/TLS) at all times while data is "in transit" to and from the database....

    So, what is Data Factory Encryption Policy for external data transfers, and does anybody has any references?

    Thanks!

     

    Monday, February 6, 2017 9:58 AM

Answers

  • The Data Management Gateway communicates with Cloud Services (such as Azure Storage Blob, SQL Data Warehouse, SQL Azure, and Azure Data Lake) via a secure channel (HTTPS or TCP over TLS) to secure data in motion.  There is no added data encryption on top of that since the communication channels are secure to prevent man-in-the-middle-attack.

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Monday, February 6, 2017 2:43 PM
  • If you are communicating with an Azure SQL DB PaaS service, encryption is required (See here)

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Tuesday, February 7, 2017 12:44 AM
  • For AzureSqlDatabase and AzureSqlDW Linked Services: you need to ensure “Encrypt=True” is specified as part of “connectionString” in its JSON definition:

    "connectionString": "Server=tcp:<servername>.database.windows.net,1433;Database=<databasename>;User ID=<username>@<servername>;Password=<password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"

    For AzureStorage Linked Service: set DefaultEndpointsProtocol=https as part of “connectionString” in its JSON definition:

    "connectionString": "DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>"

    For AzureDataLakeStore Linked Service, usage of TLS is implicit in its URI (enabled by default and cannot be turned off):

    "dataLakeStoreUri": "https://<accountname>.azuredatalakestore.net/webhdfs/v1"

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Tuesday, February 7, 2017 2:03 AM

All replies

  • The Data Management Gateway communicates with Cloud Services (such as Azure Storage Blob, SQL Data Warehouse, SQL Azure, and Azure Data Lake) via a secure channel (HTTPS or TCP over TLS) to secure data in motion.  There is no added data encryption on top of that since the communication channels are secure to prevent man-in-the-middle-attack.

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Monday, February 6, 2017 2:43 PM
  • Thanks for this!

    Just to ensure I have understood everything, is TLS default, or should it somehow be configured to be "on"?

    I am asking this because I found that SQLConnectionString has attribute Encryption=True/False

    Monday, February 6, 2017 3:00 PM
  • If you are communicating with an Azure SQL DB PaaS service, encryption is required (See here)

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Tuesday, February 7, 2017 12:44 AM
  • For AzureSqlDatabase and AzureSqlDW Linked Services: you need to ensure “Encrypt=True” is specified as part of “connectionString” in its JSON definition:

    "connectionString": "Server=tcp:<servername>.database.windows.net,1433;Database=<databasename>;User ID=<username>@<servername>;Password=<password>;Trusted_Connection=False;Encrypt=True;Connection Timeout=30"

    For AzureStorage Linked Service: set DefaultEndpointsProtocol=https as part of “connectionString” in its JSON definition:

    "connectionString": "DefaultEndpointsProtocol=https;AccountName=<accountname>;AccountKey=<accountkey>"

    For AzureDataLakeStore Linked Service, usage of TLS is implicit in its URI (enabled by default and cannot be turned off):

    "dataLakeStoreUri": "https://<accountname>.azuredatalakestore.net/webhdfs/v1"

    • Marked as answer by hpahkala Tuesday, February 14, 2017 3:20 PM
    Tuesday, February 7, 2017 2:03 AM