none
Blob Storage with Firewall/Virtual Networks not accessible from ADF

    Question

  • If your blob storage is protected to allow access from selected networks only (virtual networks) and Azure trusted services, then Data Factory linked service is not able to connect.
    Is this a normal behavior? Do I have to whitelist ADF with IP Range?
    Where to find this information?

    Thank you

    Tuesday, October 9, 2018 7:50 AM

All replies

  • The recommended way is to setup a selfhosted IR and whitelist the IP address of the machine hosting your selfhosted IR. 

    You could reference this selfhosted IR doc here

    • Proposed as answer by Bhushan Gawale Tuesday, October 9, 2018 11:12 AM
    Tuesday, October 9, 2018 9:13 AM
  • Hi Fang,
    Yes I did it and it worked. This is not great... since it means that the customer will have to pay for extra VMs to be able to host the self-hosted IR just because they want network isolation on their cloud services. It is something hard to justify in my opinion, plus also considering they would be having to guarantee high availability and performance optimization for the IR.
    Plus, let me add that the following:
    - In Data Factory you will not be able to use a Blob Storage account using VNet/Firewall as your Polybase staging
    - In Data Factory and Databricks I cannot leverage Polybase using Blob Storage on VNet. There is some documentation supporting this problem but I don't have the "Fix" yet. I opened a support case to solve this.
    Azure SQLDW PolyBase
    PolyBase is commonly used to load data into Azure SQLDW from Storage accounts. If the Storage account that you are loading data from limits access only to a set of VNet-subnets, connectivity from PolyBase to the Account will break. There is a mitigation for this, and you may contact Microsoft support for more information.

    Luis Simoes

    Thursday, October 11, 2018 5:29 AM
  • Hi Luis.

    I got your concern and I heard that Providing fixed set of IPs for ADF Azure IRs is the item at the backlog. But for now, I don't know the ETA.

    And another thing I want to mention here is if your are using Azure Data Lake Storage (Gen1), it provides an button "Allow access to Azure services". If you checked that, ADF will be whitelisted. But azure blob storage doesn't provide this function.

    Thanks.


    Thursday, October 11, 2018 6:43 AM