none
Installaling HDINSIGHT on Multi Nodes

    Question

  • To install HDInsight as node is there any configuration setting that to be done? I wanted to install in a distributed environment.


    • Edited by Sasidhar A Monday, November 19, 2012 11:10 AM
    Monday, November 19, 2012 11:09 AM

All replies

  • I'd like to know this as well.

    -Dan


    If this answers your question, please Mark as Answer

    Wednesday, November 21, 2012 7:06 PM
  • The HDInsight preview was developed as a 'onebox' stand alone install.  While it's technically possible to install in a multi-node configuration we are going to finish off the necessary security work that's required to install it in a multiple node configuration.  If one was super eager to do and deleted the firewall rule and modified the *-conf.xml Hadoop config files; you'd be open to allowing anyone to run code on your machine and access your filesystem.

    The HDInsight Azure service preview runs in a multi-node configuration and runs in an external port 'lockdown' where we only open up the HiveODBC port at the moment.

    Stay tuned.

    Thanks,

    Brad Sarsfield

     

    Wednesday, November 21, 2012 7:58 PM
  • Thanks!  Understood.  I'll stick to portal for now then, but it is good to know it's coming.  BTW, this is really cool stuff!  Great work.

    If this answers your question, please Mark as Answer

    Monday, November 26, 2012 2:06 PM
  • Yep I was able to this for an 18 datanode cluster installation in our labs. Installed the single node installer on 19 servers and set services to be appropriate for 1 namenode and 18 datanodes with multiple physical drives accordingly.  Lab install so firewall rule was off for simplicity.  Then i had to modify the following five files accordingly to indicate the multi-cluster configuration:

    core-site

    hdfs-site

    mapred-site (to distribute across multiple physical disks)

    masters (1 headnode)

    slaves (18 datanodes)

    The only issue I'm still working through is the Dashboard view doesnt seem to load the cluster, sits and wait while cluster is starting up, even though the cluster is up and functional and i can run MR jobs against it successfully.  If anyone has any thoughts on what might be breaking this I'm all ears. 

    Monday, November 26, 2012 4:13 PM
  • Awesome!! and tt would be appreciated if you can elaborate steps in detail. ;-)
    Monday, November 26, 2012 4:17 PM
  • Thanks for update. Please can you let us know when HDinsight multi-node version will be available for preview.

    Probably a road map will help.

    Thanks,

    Raj

    Friday, December 14, 2012 5:54 AM
  • Raj,

    In terms of roadmap, our last set of public announcements was about a month ago at Strata/Hadoop World, and you can find details for all of those here.

    --matt


    Program Manager -- hadoop -- http://blogs.msdn.com/mwinkle

    Saturday, December 15, 2012 5:31 PM
    Owner
  • Awesome!! and tt would be appreciated if you can elaborate steps in detail。

    I want to know.....

    Monday, December 17, 2012 11:46 AM
  • May be its too late, but I was able to setup HDInsight in multiple nodes. Step by step here: http://binyoga.blogspot.in/2013/07/virtual-lab-multi-node-hadoop-cluster.html
    Thursday, August 01, 2013 7:18 AM