locked
Azure Stack TP setup fails in 60.61 RRS feed

  • Question

  • Hi All,

    I've tried various times now, but installation consistently fails in 60.61, without any usable error message:

    VERBOSE: 1> Step: Status of step '60.61 - Phase 3 - ConfigureVMs-Part1' is 'Error'. - 3/8/2017 6:08:50 AM
    Invoke-EceAction : 1> Action: Invocation of step 60.61 failed. Stopping invocation of action plan. - 3/8/2017 6:08:50 AM
    At line:1 char:1
    + Invoke-EceAction -RolePath Cloud -ActionType Deployment -Verbose -Sta ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [Invoke-EceAction], Exception
        + FullyQualifiedErrorId : Unspecified error,CloudEngine.Cmdlets.InvokeCmdlet

    VERBOSE: 1> Action: Status of 'Deployment-Phase2-ConfigureStack' is 'Error'. - 3/8/2017 6:08:50 AM
    COMPLETE: Task Cloud - Deployment-Phase2-ConfigureStack
    VERBOSE: 1> Task: Status of action 'Deployment-Phase2-ConfigureStack' of role 'Cloud' is 'Error'. - 3/8/2017 6:08:50 AM
    VERBOSE: Step: Status of step '60 - Phase 2 - ConfigureVMs' is 'Error'. - 3/8/2017 6:08:50 AM
    Invoke-EceAction : Action: Invocation of step 60 failed. Stopping invocation of action plan. - 3/8/2017 6:08:50 AM
    At line:1 char:1
    + Invoke-EceAction -RolePath Cloud -ActionType Deployment -Verbose -Sta ...
    + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : InvalidOperation: (:) [Invoke-EceAction], Exception
        + FullyQualifiedErrorId : Unspecified error,CloudEngine.Cmdlets.InvokeCmdlet

    One thing I noticed is that the Internal adapter of MAS-Gwy01 has an APIPA address. Is that expected?

    What log file would contain relevant information on step 60.61?

    Friday, March 10, 2017 8:27 AM

Answers

All replies

  • Hello,

    Can you please confirm that the host machine has a D drive?  Typically if you are booting from VHD the VHD becomes the C drive and the underlying drive hosting the VHD becomes D.  The join path error is trying to create a path from where it is executing, not the target.  If the host machine does not have a D drive or that is assigned to an CD/DVD with nothing inserted then this may be causing the issue.  You can try changing drive assignments on the host such that there is a volume mounted to D.  Note that the full path does not have to be there, it is constructing the path and PowerShell simply needs a volume mounted on D.  This is something that will be addressed in later releases.

    If this does not apply, then please try to Redeploy Azure Stack TP3 using the following command:

     

    Import-ModuleC:\CloudDeployment\CloudDeployment.psd1 -Force

    Import-ModuleC:\CloudDeployment\ECEngine\EnterpriseCloudEngine.psd1 -Force

    Invoke-EceAction-RolePath Cloud -ActionType Deployment -Start 60 -Verbose

     

    If the deployment continues to fail at 60.61, we’ll require additional logs file analysis to continue troubleshooting. 

    If you could, please email ascustfeedback@microsoft.com to get a workspace setup to upload your logs.  

     

    Make sure to use a Work, Organizational or Student address when contacting ascustfeedback@microsoft.com and include the thread URL in the subject.

     

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    https://azure.microsoft.com/en-us/blog/hybrid-application-innovation-with-azure-and-azure-stack/

    If you continue experience any issues with TP3 release, feel free to contact us.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

      

    Thanks,


    Gary Gallanes

    Friday, March 10, 2017 7:28 PM
  • Hi Gary,

    I’ve confirmed the host has a D-drive which is the original drive holding the CloudBuilder vhdx:

    PS C:\AzureStackLogs> Get-Volume c,D

    DriveLetter FileSystemLabel FileSystem DriveType HealthStatus OperationalStatus SizeRemaining      Size

    ----------- --------------- ---------- --------- ------------ ----------------- -------------      ----

    C                           NTFS       Fixed     Healthy      OK                    232.71 GB 299.66 GB

    D                           NTFS       Fixed     Healthy      OK                    633.61 GB 999.45 GB

    I have re-run the setup multiple times. It consistently fails in the same location.

    Thanks,

    \Rob

    Monday, March 13, 2017 7:20 AM
  • Hi,

    We will require some additional logs to continue out investigation into error at step 60.61. 

    If you could, please email ascustfeedback@microsoft.com to get a workspace setup to upload your logs.  

     

    Make sure to use a Work, Organizational or Student address when contacting ascustfeedback@microsoft.com and include the thread URL in the subject.

     

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    https://azure.microsoft.com/en-us/blog/hybrid-application-innovation-with-azure-and-azure-stack/

    If you continue experience any issues with TP3 release, feel free to contact us.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,


    Gary Gallanes

    Monday, March 13, 2017 4:57 PM
  • Hi Rob,

    We received your log files and continuing our investigation.   We will reply with next steps as soon as possible.

    Thanks,


    Gary Gallanes

    Wednesday, March 15, 2017 12:02 AM
  • Hi Gary,

    Do you have an update on this issue?

    Thanks,
    \Rob

    Monday, March 27, 2017 5:25 PM
  • Rob,

    We've almost completed on analysis on your logs and we'll provide an update in the next few hours.

    Thanks,


    Gary Gallanes

    Tuesday, March 28, 2017 6:12 PM
  • Rob,

    We found a problem. Could you please connect to MAS-Gwy01, delete the following registry key:

    HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\ServerManager\ServicingStorage

    and -rerun the deployment?

     

    Invoke-command -computer MAS-Gwy01 -ScriptBlock `

    {Remove-Item -Path hklm:\ SOFTWARE\Microsoft\ServerManager\ServicingStorage}

     

    NOTE: Make sure you are running the deployment as under the AzureStack\AzureStackAdmin account.

     

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,


    Gary Gallanes

    Tuesday, March 28, 2017 6:56 PM
  • Hi Gary,

    Could you unmark you reply as Answer, since it did not resolve my issue?

    \Rob

    Tuesday, April 4, 2017 3:12 PM
  • Hi Rob,

    We have setup a Workspace for you to upload your logs & sent email with a link and detailed instructions for gathering and uploading your logs.

     

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    https://azure.microsoft.com/en-us/blog/hybrid-application-innovation-with-azure-and-azure-stack/

    If you continue experience any issues with TP3 release, feel free to contact us.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,


    Gary Gallanes

    Tuesday, April 4, 2017 6:44 PM
  • Hi Rob,

    I've noticed a Time synchronization issue may cause a deployment to fail at 60.61

      

    To fix

    Run the following script from an elevated PowerShell session  on the DVM Host as AzureStackAdmin if deployment fails due to a time sync issue.

    NOTE: During deployment, after the Host reboots, the DC (MAS-DC01) syncs with time.windows.com and then the Host and all other VMs sync to the DC.

     

    ### Create array of VMs

    $ALLVMS = @("MAS-ACS01","MAS-ADFS01","MAS-Sql01","MAS-BGPNAT01","MAS-CA01","MAS-Con01","MAS-Gwy01","MAS-NC01", "MAS-SLB01", "MAS-SUS01", "MAS-WAS01", "MAS-Xrp01")

    # Compare server time between Host and VMs

    Invoke-command -Computername $ALLVMS -ScriptBlock {Get-Date}

    Get-Date

    # Sync MAS-DC01 to time server time.windows.com

    W32tm /resync /Computer:MAS-DC01 /nowait

    # Syncs all VMs with MAS-DC01

    Invoke-Command -Computer $ALLVMS -ScriptBlock { gsv w32time | sasv; w32tm /resync /force }

    # Syncs Host with MAS-DC01

    Invoke-Command -Computer $env:Computername -ScriptBlock { gsv w32time | sasv; w32tm /resync /force }

    # Compare server time between Host and VMs

    Invoke-command -Computername $ALLVMS -ScriptBlock {Get-Date}

    Get-Date

      

    Once the time is synced between all Server VMs redeploy using the -rerun parameter

    NOTE: Make sure you are running the deployment as under the AzureStack\AzureStackAdmin account.

      

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

       

    If that does get you past 60.61 then please confirm that the host machine has a D drive? 

    Typically if you are booting from VHD the VHD becomes the C drive and the underlying drive hosting the VHD becomes D.  The join path error is trying to create a path from where it is executing, not the target.  If the host machine does not have a D drive or that is assigned to an CD/DVD with nothing inserted then this may be causing the issue.  You can try changing drive assignments on the host such that there is a volume mounted to D.  Note that the full path does not have to be there, it is constructing the path and PowerShell simply needs a volume mounted on D.  This is something that will be addressed in later releases.

      

    If this does not apply, then please try to Redeploy Azure Stack TP3 using the following command:

      

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

      

    Also - Azure Stack requires access to the Internet either directly or through a transparent proxy.

    Public IP's are not required by Azure Stack TP3. 2 private NAT IP addresses assigned via DHCP or statically are all that is required.

      

    Both the host IP and the new IP assigned to the MAS-BGPNAT01 (by DHCP or static IP) must be able to access Internet. Ports 80 and 443 are used under the graph.windows.net and login.windows.net domains.

     

    Azure Stack TP3 does not support Web Proxy for Internet access.

    Make sure to disable ALL but one network adapter.

      

    If your environment DOESN'T have DHCP enabled

    You must include the following parameters to you deploy command to assign an IP to the MAS-BGPNAT gateway:

     

    .\InstallAzureStackPOC.ps1 -AdminPassword $adminpass -AADAdminCredential $aadcred

    -NatIPv4Subnet 10.10.10.0/24 -NatIPv4Address 10.10.10.3 -NatIPv4DefaultGateway 10.10.10.1

     

    If DNS resolution seems to be the issue:

    -          Use the optional -EnvironmentDNS setup parameter to add an external DNS server. 

    A DNS server is created as part of the Azure Stack deployment. To allow computers inside the solution to resolve names outside of the stamp, provide your existing infrastructure DNS server or a reliable known public DNS Server IP. The in-stamp DNS server forwards unknown name resolution requests to this server.

     

    Once network settings have been verified please redeploy with the command below:

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

     

    NOTE: Make sure you are running the deployment as under the AzureStack\AzureStackAdmin account.

     

    If you continue to receive error at step 60.61:

    Reboot the Host and MAS-Gwy01 VM.

    Make sure all VM roles are up by checking the roles in 'Failover Cluster Manager'

    Redeploy with the command below:

     

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,


    Gary Gallanes

    Tuesday, April 4, 2017 8:22 PM
  • Hello,

    Our investigation has determined error at 61.60.101 & 60.61.82 to be a known issue with TP3 that has been addressed in the TP3 refresh build released on April 6, 2017.

     

    Please download and redeploy using the latest Azure Stack TP3 Refresh POC deployment package

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 refresh, feel free to contact us.

     

    TP3 refresh Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

      

     Thanks,


    Gary Gallanes

    Thursday, April 6, 2017 7:11 PM