none
Data Factory not working with Key Vault with VNet/Subnet restrictions

    Question

  • Hi everyone.
    I don't know if this is the place to ask this question but hope to have your understanding.

    I setup Azure Key Vault and Azure Data Factory and I can test connection to the KV linked service in ADF successfully.
    I have added the ADF principal in AKV policies to be able to GET secrets.
    My AKV is restricted in terms of networking to two VNETs/Subnets and I have allow trusted Microsoft services to bypass this firewall on.

    The problem is that everytime I try to get a secret I get an exception saying:

    "The error message is: Client address (xxx.xxx.xxx.xxx) is not authorized and caller is not a trusted service"

    Why is this happening? Should not ADF be a trusted service? How to configure this in terms of Firewall/VNet?

    Thank you
    Tuesday, October 9, 2018 5:56 AM

All replies

  • Have you tried setting up a selfhosted IR and whitelist the IP address of your selfhosted IR machine and then use the selfhosted IR to connect your data store?
    Tuesday, October 9, 2018 6:47 AM
  • Hi Fang,
    Yes I did it and it worked. This is not great... since it means that the customer will have to pay for extra VMs to be able to host the self-hosted IR just because they want network isolation on their cloud services. It is something hard to justify in my opinion, plus also considering they would be having to guarantee high availability and performance optimization for the IR.
    Plus, let me add that the following:
    - In Data Factory you will not be able to use a Blob Storage account using VNet/Firewall as your Polybase staging
    - In Data Factory and Databricks I cannot leverage Polybase using Blob Storage on VNet. There is some documentation supporting this problem but I don't have the "Fix" yet. I opened a support case to solve this.

    ////Docs at: https://docs.microsoft.com/en-us/azure/sql-database/sql-database-vnet-service-endpoint-rule-overview

    Azure SQLDW PolyBase
    PolyBase is commonly used to load data into Azure SQLDW from Storage accounts. If the Storage account that you are loading data from limits access only to a set of VNet-subnets, connectivity from PolyBase to the Account will break. There is a mitigation for this, and you may contact Microsoft support for more information.



    Luis Simoes

    Wednesday, October 10, 2018 12:16 PM