locked
Hdinsight cluster -----EDGE node IAAS vs PAAS RRS feed

  • Question

  • Hi Team,

    What are the options for adding an edge node to an HDinsight cluster. I understand that once the HDInsight cluster is created we can deploy and attach an edge node to the existing cluster through ARM template. But once the cluster is deleted, my understanding is that the edge node also gets destroyed.

    If i want to keep the edge node static and  doesn't want to delete even after the cluster is deleted, then is there a way to create a container and then install the client services on the node and then add the host through Ambari.

    Please provide me the available options for PAAS and IAAS edge node.

    Thanks-Rahul.!

    +31-617854586

    Monday, May 4, 2020 12:43 PM

Answers

  • Hi Rahul,

    Unfortunately, we don’t support any other methods to create edge nodes on HDInsight clusters.

    Here is an important note: You can add an empty edge node to an existing HDInsight cluster, to a new cluster when you create the cluster. Adding an empty edge node is done using Azure Resource Manager template.


    • Marked as answer by Rahul Akk Wednesday, May 6, 2020 12:56 PM
    Wednesday, May 6, 2020 12:31 PM

All replies

  • Hello,

    What are the options for adding an edge node to an HDInsight cluster? 

    Only option: You use a Resource Manager template to add an edge node to an existing HDInsight cluster. The Resource Manager template can be found in GitHub

    Yes, your understanding is correct, once the HDInsight cluster is deleted the edge node also gets deleted.

    If I want to keep the edge node static and  doesn't want to delete even after the cluster is deleted, then is there a way to create a container and then install the client services on the node and then add the host through Ambari.

    Official statement:

    Unfortunately, Azure HDInsight don’t support attach/detach edge node capability.

    I would suggest you to vote up an idea submitted by another Azure customer.

    https://feedback.azure.com/forums/217335-hdinsight/suggestions/35006689-need-hdinsight-attach-detach-edge-node-capability

    All of the feedback you share in these forums will be monitored and reviewed by the Microsoft engineering teams responsible for building Azure.

    Unofficial statement:

    There is third-party blog post where they have demonstrated how to attach and detach edge nodes/virtual machines from a HDInsight cluster. 

    Please note: there are some limitations and restraints to this solution. Please review these here before proceeding. Disclaimer: This contribution is not subject to official support from Dataiku nor Microsoft and does not guarantee compatibility with future versions of HDI 

    Disclaimer: This response contains a reference to a third-party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

    Hope this helps. Do let us know if you any further queries.

    ----------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Tuesday, May 5, 2020 9:03 AM
  • Hi Pradeep,

    Why cant we create a container and then add it through Ambari interface wherein we have an option of adding hosts??

    Please clarify.

    Thanks-Rahul.!

    Tuesday, May 5, 2020 9:10 AM
  • Hi Rahul,

    Why can’t we create a container and then add it through Ambari interface wherein we have an option of adding hosts??

    In Azure HDInsight cluster, you cannot add hosts through Ambari interface.

    Note: Adding, decommissioning, and recommissioning a host should not be used with HDInsight clusters.

    Hope this helps. Do let us know if you any further queries.

    ----------------------------------------------------------------------------------------

    Do click on "Mark as Answer" and Upvote on the post that helps you, this can be beneficial to other community members.

    Wednesday, May 6, 2020 8:06 AM
  • Hi Pradeep,

    Could you please let me know what configuration changes need to be done on the container to be used as edge node for the HDInsight cluster ??

    Thanks-Rahul.!

    Wednesday, May 6, 2020 9:53 AM
  • Hi Rahul,

    Unfortunately, we don’t support any other methods to create edge nodes on HDInsight clusters.

    Here is an important note: You can add an empty edge node to an existing HDInsight cluster, to a new cluster when you create the cluster. Adding an empty edge node is done using Azure Resource Manager template.


    • Marked as answer by Rahul Akk Wednesday, May 6, 2020 12:56 PM
    Wednesday, May 6, 2020 12:31 PM