none
How to access the Ambari page for an HDInsight cluster created by a pipeline in Data Factory?

    Question

  • Hello everyone!

    I have created a pipeline that runs a Spark script on an HDInsight cluster, following this tutorial: https://docs.microsoft.com/en-us/azure/data-factory/tutorial-transform-data-spark-portal.

    When I create the <g class="gr_ gr_26 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" data-gr-id="26" id="26">LinkedinService</g> for HDInsight I see how to setup username and password for SSH access. I do not see how to setup credentials for logging in into the Ambari view for the cluster.

    Once the pipeline is running, the HDInsight cluster is created. I can access the cluster via SSH, however, if I try to open the Ambari page, I don't know which credentials to use. When I try to insert the SSH credentials I get an error message and access is forbidden.

    How can I access the Ambari dashboard for the HDInsight cluster created by the pipeline in Data Factory?

    Thanks!

    • Edited by Lele Libero Tuesday, April 17, 2018 3:36 PM rendering issues in the post
    Tuesday, April 17, 2018 3:29 PM

All replies

  • Hi Lele,

    You can set the 'clusterUserName' and 'clusterPassword' properties when defining the HDI on-demand linked service:

    https://docs.microsoft.com/en-us/azure/data-factory/compute-linked-services#azure-hdinsight-on-demand-linked-service

    Have you already tried this?  You may have skipped those properties since they are not mandatory when creating the linked service.

    Tuesday, April 17, 2018 9:53 PM
    Moderator
  • Hi Jason,

    thank you for your reply!

    I had a look at the page you linked, however, when I configure the HDInsight Linked Service (via <g class="gr_ gr_275 gr-alert gr_spell gr_inline_cards gr_run_anim ContextualSpelling ins-del multiReplace" data-gr-id="275" id="275">gui</g>) I cannot find these fields.

    Instead, I see "Cluster SSH user name" and "Cluster SSH pasword", nested in "OS type".

    Please have a look at the screenshot.

    Where should I find the cluster username and password that you mention and that appear in the documentation?

    Wednesday, April 18, 2018 12:11 PM
  • Hi Lele,

    Please click the "Advanced" in your picture above, and paste following JSON code:

    {
        "properties": {
          "typeProperties": {
            "clusterUserName": "your cluster admin user",
            "clusterPassword": {
                "value": "Your password",
                "type": "SecureString"
              }
          }
        }
    }

    Then you can use the cluster admin user you set to access Ambari portal.

    Thursday, April 19, 2018 3:05 AM
  • Hey Jason, thank you very much. It worked.

    I didn't know how to use the Advanced node in the edit page.

    Thank you!

    Thursday, April 26, 2018 12:29 PM