where is the Microsoft HDInsight Dashboard?

Question

this technet article, Installing the Developer Preview of Apache Hadoop-based services on Windows, states that after installing the CTP we should see three shortcuts on the desktop, including one for the "Microsoft HDInsight Dashboard".  i am not seeing that shortcut.  the other shortcuts are for urls like http://localhost:50030/ and http://localhost:50070/.  what is the url for the dashboard?

Answer 1

The dashboard on my machine is http://localhost:8085/.

I ran into a similar thing on my first install with no icons. I uninstalled the HDInsight and Hortonworks packages, reinstalled through web platform installer, and I had the shortcuts.

Hope this helps.

Answer 2

OK, I uninstalled both and reinstalled through WPI, but there is still no shortcut for the dashboard.  I'm not missing all three shortcuts...the shortcuts for "Hadoop MapReduce Status" and "Hadoop Name Node Status" are there.  But no shortcut for "Microsoft HDInsight Dashboard".  And it's not running on http://locahost:8085/.  Looking through the install logs and powershell scripts, i can see where the other two shortcuts were successfully created, but I don't see anything anywhere about the "Microsoft HDInsight Dashboard".  Was it removed from this release?  Or do I need to launch it some other way?

Answer 3

It should be there on the desktop and at that URL.

One of my co-workers mentioned that he had problems with the install also, and it turned out to be that he had beta versions of the .NET 4.0/4.5 frameworks and ASP.NET on the computer. He removed the ASP.NET 4 MVC beta, as well as the 4.5 framework beta, and install the released version of .NET 4.0, then reinstalled HDInsight and that fixed the problems. Can you check to see if you have any beta versions of .NET 4.0/4.5 and/or ASP.NET MVC?

Answer 4

I *just* installed HDInsight on Server 2012.  I also don't have a dashboard shortcut, and localhost:8085 doesn't work either.  Assuming this isn't some kind of packaged web server, I don't see anything in IIS8 other than the default web site on localhost:80.

Now, I don't know, but I didn't see anything specifically mentioning that Server 2012 was or wasn't supported...  I did only see a reference to Windows 7 and Server 2008 R2 here: http://www.microsoft.com/en-us/download/details.aspx?id=35397, but that might not mean anything in this case.

Answer 5

i'm installing the HDInsight Preview on a clean installation of Windows Server 2012, so there's no possibility of .NET beta issues.

I found the install log for the "HadoopDashboard-winpkg.zip" package with this error:
"You cannot service a running 64-bit operating system with a 32-bit version of DISM. 
Please use the version of DISM that corresponds to your computer's architecture"

I was able to get around this by editing the file "Scripts\Modules\iis-prerequisitecommon.psm1" in that package.  I changed line 111 to:

$registerOutput = & "C:\windows\Sysnative\dism.exe" "/online" "/enable-feature" "/featurename:IIS-ASPNET45" "/All"

Interestingly, there's a comment just above that line in the file:

#HACK for windows 8 / Server 2012
#Windows 8 does not support registration aspnet handler 

yet, the hack doesn't seem to succeed on win server 2012...

after editing that line, the package installation succeeds and the shortcut for the Microsoft HDInsight Dashboard is on the desktop.  but now i have another issue.  when i launch that shortcut, i get an error when the web app runs:

"No connection could be made because the target machine actively refused it 127.0.0.1:6001"

not sure where to go with that error...any ideas?

Answer 6

I was able to get around this by editing the file "Scripts\iis-prerequisitecommon.psm1" in that package.  I changed line 111 to:

$registerOutput = & "C:\windows\Sysnative\dism.exe" "/online" "/enable-feature" "/featurename:IIS-ASPNET45" "/All"

(Time for bed, so my VM is shutdown.)

Sorry, what sequence did you follow?  Or is that .psm1 file not part of HDInsight?  Just wondering how you managed to change a file if it is was downloaded and installed by Web PI.

As for the port, maybe there's a Windows Firewall issues?

Answer 7

heh, it was tricky!  i extracted the zip for the "HadoopDashboard-winpkg.zip" package, edited the file in the Scripts\Modules folder in the extracted package folder, rezipped and replaced the original "HadoopDashboard-winpkg.zip" to be exactly as it was, only with my updates to that file.  I copied that zip to a location outside of C:\HadoopFeaturePackSetup, and uninstalled the "Microsoft HDInsight Community Technology Preview" in control panel.

Then I opened the web platform installer, added the HDInsight package, and clicked install.  I watched the C:\HadoopFeaturePackSetup folder in explorer, and when I saw that the installer was creating the C:\HadoopFeaturePackSetup\Packages directory, I ninja-copied my zip file into that folder, overwriting the "C:\HadoopFeaturePackSetup\Packages\HadoopDashboard-winpkg.zip" file that was part of the WPI package.

I just noticed that the "HadoopWebApi-winpkg.zip" package has the same issue in the "Scripts\Modules\iis-prerequisitecommon.psm1" file.  I uninstalled the "Microsoft HDInsight Community Technology Preview" in control panel, and followed the procedure described above except ninja-overwriting both the "HadoopDashboard-winpkg.zip" and "HadoopWebApi-winpkg.zip" files with my versions, each containing the edited iis-prerequisitecommon.psm1 file, and now the dashboard web app loads without any error.  the error about localhost:6001 was happening because the dashboard app connects to the webapi app, which wasn't successfully installing for the same reason.

maybe someone knows an easier / better way to get around this. 

Answer 8

Thanks again for sharing.

I ran full installation on clean windwos server 2012, the shortcut is there and the dashboard is fully functional.

If you are installing on a VM, you should make sure that it's updated with the latest updates and that your VM is not pending restart.

We are continuously testing the webpi installation on all the supported version of Windows, so I wonder what went wrong in your case. Did you make sure that your VM was not pending restart prior to the installation?

---

aviade

Answer 9

Aviad

When I ran into trouble, I decided to install the HDInsight preview on a clean windows server 2012 VM for troubleshooting.  When i downloaded webpi and installed the HDInsight preview, I don't believe there was a restart pending.  But I did not run windows update prior to attempting to install the HDInsight preview.

I'll try again now, making sure that I run Windows Update and restart prior to downloading webpi and installing the HDInsight preview, and I'll report back here.

Answer 10

Much appreciated! Looking forward to your reply.

---

aviade

Answer 11

OK, reporting back now: still the same issue.  Clean win server 2012 VM, windows update until no futher updates available, no restart pending, install webpi and HDInsight preview, and no dashboard icon on the desktop.

Answer 12

OK, reporting back now: still the same issue.  Clean win server 2012 VM, windows update until no futher updates available, no restart pending, install webpi and HDInsight preview, and no dashboard icon on the desktop.

Same here...  Still don't see a dashboard after an uninstall/reinstall.

I haven't tried a Windows update first before trying to load HDInsight.  I'm going to try a 2nd install and boost the RAM from 2GB to 4GB.

Answer 13

Yup, 2nd try, no different...  Still no dashboard.  I made sure to update and install that 170MB update first...

Answer 14

Same as almaplayera reported earlier, the last few lines from this file C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\hadoopDashboard-winpkg.install.log:

...

HadoopWebApi: Extracting service credentials for user ''
HadoopWebApi: Reading credentials from C:\Hadoop\singlenodecreds.xml
HadoopWebApi: Registering ASP  with IIS
HadoopWebApi: Registeration failed with IIS:  Deployment Image Servicing and Management tool Version: 6.2.9200.16384   Error: 11  You cannot service a running 64-bit operating system with a 32-bit version of DISM.  Please use the version of DISM that corresponds to your computer's architecture.  The DISM log file can be found at C:\Windows\Logs\DISM\dism.log
WINPKG: rd /s /q C:\HadoopFeaturePackSetup\Packages\hadoopDashboard-winpkg
WINPKG: Finished processing C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\..\Packages\hadoopDashboard-winpkg.zip
WINPKG: Removing ENV:WINPKG_LOG

Answer 15

Thanks. I will investigate and update this thread.

For now, you can manually install IIS on your machine, make sure to include ASP.NET, BasicAuth and WinAuth.

---

aviade

Answer 16

on all 3 of my win server 2012 instances, IIS was already installed with basicauth, winauth, and ASP.NET 4.5 prior to installing webpi and HDInsight Preview.  it looks like the HDInsight installer is running into trouble when it tries to register asp.net, but i'm not sure why it's trying to do that when asp.net is already installed and registered.

Answer 17

Thanks. I will investigate and update this thread.

For now, you can manually install IIS on your machine, make sure to include ASP.NET, BasicAuth and WinAuth.

---

aviade

Well, IIS is getting loaded properly.  I was able to connect to localhost:80 to confirm...  I didn't check about whether the Auth had been setup.

I'll try with Server 2008 R2 over the weekend.  There's definitely something wrong with the Web PI download.  How this can work for you, and not us, is a mystery.

Answer 18

Hey Marco, 

Here is a workaround that should get you past the install issue. Once the installer has finished executing, please follow the following steps:

1. Open up an Administrator powershell prompt 

1.a. Set remote execution policy of the powershell to accept scripts if you havent done so

PS:>      Set-ExecutionPolicy RemoteSigned

2. Navigate to C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools

cd C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools

3. Install HadoopWebApi

.\winpkg.ps1 ..\Packages\HadoopWebApi-winpkg.zip install -CredentialFilePath c:\Hadoop\Singenodecreds.xml

4. Install dashboard

.\winpkg.ps1 ..\Packages\HadoopDashboard-winpkg.zip install -CredentialFilePath c:\Hadoop\Singenodecreds.xml

Let me know if this fixes the issue. We are actively trying to patch our installer and apologize for any inconvenience caused.

Answer 19

I'll try the workaround maybe tonight.  I'm actually using Datacenter from my TechNet sub, so I might have to try another image...

Answer 20

The workaround Augustine provided has resolved my problem.  And I'm confirming that I have Datacenter installed, yet was still having the issue.

Answer 21

I can confirm that this workaround suggested by Augustine  works on Window 8 Pro /IIS 7 .

Make sure you run commands on Powershell in "Admin Mode"

---

Pavan Keerthi

Answer 22

I can confirm that this workaround suggested by Augustine  works on Window 8 Pro /IIS 7 .

Make sure you run commands on Powershell in "Admin Mode"

---

Pavan Keerthi

Do you mean "Windows 8 Pro/IIS 8"?  Windows 8/Server 2012 come with IIS8.  Just wanted to make sure...

Answer 23

Augustine,

No luck with a Windows Server 2012 Standard running as a guest Windows 8 Hyper-V VM.

First of all, the file reference in both commands should be c:\hadoop\singlenodecreds.xml (you've missed an "l").

Second, running the PS script results in an exception:

HadoopWebApi: ENV:WINPKG_BIN is C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools
HadoopWebApi: Logging to C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\HadoopDashboard-winpkg.install.log
HadoopWebApi: HDP_INSTALL_PATH: C:\HadoopFeaturePackSetup\Packages\HadoopDashboard-winpkg\scripts
HadoopWebApi: HDP_RESOURCES_DIR: C:\HadoopFeaturePackSetup\Packages\HadoopDashboard-winpkg\resources
HadoopWebApi: Setting full control ACLs and owner for 'c:\hadoop\websites' ELITEVM1\hadoop
powershell.exe : C:\HadoopFeaturePackSetup\Packages\HadoopDashboard-winpkg\scripts\install.ps1 : Exception calling
"SetAccessRule" with
At C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\winpkg.ps1:118 char:9
+     $out = powershell.exe -InputFormat none -Command "$command" 2>&1
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\HadoopFeatur...cessRule" with :String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

"1" argument(s): "Some or all identity references could not be translated."
At line:1 char:1
+ C:\HadoopFeaturePackSetup\Packages\HadoopDashboard-winpkg\scripts\install.ps1 -C ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (:) [install.ps1], MethodInvocationException
    + FullyQualifiedErrorId : IdentityNotMappedException,install.ps1

I've read in another tread that there isn't issue with Standard. Is that why it's failing to create the dashboard?

---

Teo Lachev, Prologika LLC
http://www.prologika.com

Answer 24

Teo: You've tried this on a fresh install of Server 2012 Standard?  You're running the script with elevated creds?  You're getting an error on SetAccessRule() which seems to imply you're not using an account with enough privileges to set credentials properly?

Answer 25

Thanks Marco,

Server 2012 Standard is a fresh install but it's configured as a domain controller. Yes, I ran PS as an administrator which is my account.

Did we say before that there was an issue with Windows Server 2012 Standard?

---

Teo Lachev, Prologika LLC
http://www.prologika.com

Answer 26

There's another user having issues with HDInsight on a Server 2008 R2 Domain Controller...  Starting to see a pattern where perhaps DCs might not be "supported"...

Answer 27

When I run the script to Install HadoopWebApi I get following error, any thoughts:

powershell.exe : C:\HadoopFeaturePackSetup\Packages\HadoopWebApi-winpkg\scripts\install.ps1 : Cannot validate argument
on parameter
At C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\winpkg.ps1:118 char:9
+     $out = powershell.exe -InputFormat none -Command "$command" 2>&1
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (C:\HadoopFeatur...t on parameter :String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

'credentialFilePath'. The "Test-Path $_" validation script for the argument with value "c:\Hadoop\Singenodecreds.xml"
did not return true. Determine why the validation script failed and then try the command again.
At line:1 char:96
+ ... entialFilePath c:\Hadoop\Singenodecreds.xml
+                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidData: (:) [install.ps1], ParameterBindingValidationException
    + FullyQualifiedErrorId : ParameterArgumentValidationError,install.ps1

---

Sumit

Answer 28

The script provided by Augustine has a typo, the xml file name is Singlenodecreds.xml, not Singenodecreds.xml

---

aviade

Answer 29

Thanks, now I was able to get through the first hurdle but it is stuck at password.. not sure why it is looking for following:

~~~~~~~~~~~~~~~~~~~

powershell.exe : ConvertTo-SecureString : Key not valid for use in specified state.
At C:\HadoopFeaturePackSetup\HadoopFeaturePackSetupTools\winpkg.ps1:118 char:9
+     $out = powershell.exe -InputFormat none -Command "$command" 2>&1
+            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : NotSpecified: (ConvertTo-Secur...pecified state.:String) [], RemoteException
    + FullyQualifiedErrorId : NativeCommandError

At C:\HadoopFeaturePackSetup\Packages\HadoopWebApi-winpkg\scripts\Modules\Winpkg.Utils.psm1:191 char:40
+         $securePassword = $import.Password | ConvertTo-SecureString
+                                              ~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidArgument: (:) [ConvertTo-SecureString], CryptographicException
    + FullyQualifiedErrorId : ImportSecureString_InvalidArgument_CryptographicError,Microsoft.PowerShell.Commands.Conv
   ertToSecureStringCommand

New-Object : Exception calling ".ctor" with "2" argument(s): "Cannot process argument because the value of argument
"password" is null. Change the value of argument "password" to a non-null value."
At C:\HadoopFeaturePackSetup\Packages\HadoopWebApi-winpkg\scripts\Modules\Winpkg.Utils.psm1:194 char:11
+     $creds = New-Object System.Management.Automation.PSCredential $username, $secur ...
+              ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (:) [New-Object], MethodInvocationException
    + FullyQualifiedErrorId : ConstructorInvokedThrowException,Microsoft.PowerShell.Commands.NewObjectCommand

~~~~~~~~~~~~

Sumit

---

Sumit