none
Multiple nodes in integration runtime RRS feed

  • Question

  • Hi Team,

    We found the following post : 

    Setting Up the Self-Hosted Integration Runtime. The first step is setting up the self-hosted IR in ADF. Then we install a node (or multiple nodes) on a local server and connect the node with the integration runtime

    How can we set up multiple nodes on a local server to connect with integration runtime.

    Wednesday, August 21, 2019 12:09 PM

Answers

  • Hi Martin,

    Thanks for the response.

    Upon further inspection we found that, at the time of gateway setup someone manually provided the node name instead of using the VM name that automatically gets populated as the node name value. Hence two of our Data factories had same node name but their integration runtime was hosted on different VM's.

    • Marked as answer by PriyaJha Thursday, September 19, 2019 11:21 AM
    Monday, September 9, 2019 7:05 AM

All replies

  • Hello PriyaJha, and thank you for your inquiry.  To better assist you, could you please help me understand your situation better?

    • Are you using Azure Data Factory, Version 1 or Version 2?
    • You mention nodes.  Is this an on-premises cluster, or an Azure hosted cluster, or for an Azure service/product which runs on a cluster?
    • You mention you found a 'post'.  This sounds like you got your information from a forum thread or a blog post.  Have you looked at the documentation on the subject?

    Doc links:

    Self-hosted IR Nodes :

    You can associate multiple nodes by installing the self-hosted integration runtime software from the Download Center. Then, register it by using either of the authentication keys obtained from the New-AzDataFactoryV2IntegrationRuntimeKey cmdlet, as described in the tutorial.

    Types of Integration Runtimes
    General self-hosted IR steps

    Wednesday, August 21, 2019 6:30 PM
    Moderator
  • Hi Martin,

    Thanks for your response.

    We are using Azure Data Factory Version 1.

    By nodes i meant Onpremise VM where integration runtime is installed.

    I know that if we have multiple data factories, each data factory requires a gateway to connect to Onpremise DB and that gateway is configured by installing integration runtime on different OnPremise VM's i.e. if i have 2 DF's on different resource groups i would require two Onpremise VM's to setup gateway.

    But we are facing a scenario wherein two of our DF's present on different resource group are using the same Onpremise VM i.e. when we are checking the integration runtime code for both the gateways they have same node name.

    Can you please explain how this scenario is occurring.

    Thursday, August 22, 2019 5:22 AM
  • Hi Priya,

    You can add a new IR to the ADF and you can share it with the another ADF so i doubt you have shared your existing IR with the other ADF environment.


    Cheers,

    Thursday, August 22, 2019 6:04 AM
  • Hi Amit,

    Sharing feature is available only in Azure Data Factory Version V2. This scenario has occurred in Azure Data Factory V1 and gateway sharing is not available in this version.

    Thanks

    Thursday, August 22, 2019 8:57 AM
  • Well, this isn't reccomended, but I think I found one way to do it.  During the manual setup process, there is a step where you register the runtime.  ( See Step 4 of this doc.)

    You can re-register the runtime, switching which factory it is associated with.  This means you cannot run jobs from both factories at the same time.  The runtime only associates with one factory at a time.

    This means you would have to carefully schedule your jobs to not conflict, and give some extra time between jobs to swap which factory the runtime is registered with.

    Thursday, August 22, 2019 10:09 PM
    Moderator
  • Do you mean to say, that you have an existing situation where both factories are pointing to the same VM at the same time?  What happens when both factories try to use the same IR at the same time?  Do they conflict?

    Monday, August 26, 2019 8:29 PM
    Moderator
  • Hi Martin,

    Thanks for the response.

    Upon further inspection we found that, at the time of gateway setup someone manually provided the node name instead of using the VM name that automatically gets populated as the node name value. Hence two of our Data factories had same node name but their integration runtime was hosted on different VM's.

    • Marked as answer by PriyaJha Thursday, September 19, 2019 11:21 AM
    Monday, September 9, 2019 7:05 AM
  • Thank you for following up with the clarification, PriyaJha.
    Tuesday, September 10, 2019 9:34 PM
    Moderator