none
AzureStack Deployment Error: 60.61.82 - (FBI) Deploy Azure Stack Fabric Ring Services RRS feed

  • Question

  • Hi 

    I am trying to deploy AzureStack to one of our hosts for testing purposes. I originally managed to deploy the full AzureStack on this host but using 250GB disks, we filled them up with a few VMs and testing ISOs.

    Now deploying AzureStack on the same host with 1TB disks instead and I'm hitting errors on the "60.61.82 - (FBI) Deploy Azure Stack Fabric Ring Services" section. I have tried running it multiple times, checking the times across all servers are correct, rebooted the gwy01 server but still hitting the same error.

    ########

    Invoke-EceAction : Type 'Deployment' of Role 'ACSFabric' raised an exception:
    System.AggregateException: One or more errors occurred. --->
    System.Fabric.FabricValidationException: Cluster Setup cancelled due to validation error(s) found
    by Best Practices Analyzer. Inspect details in DeploymentTraces log folder local to executing
    location.
       at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInter
    nal>d__1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
    task)
       at
    Microsoft.ServiceFabric.DeploymentManager.DeploymentManager.<CreateClusterAsync>d__2.MoveNext()
       --- End of inner exception stack trace ---
       at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)
       at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken
    cancellationToken)
       at Microsoft.ServiceFabric.Powershell.ClusterCmdletBase.NewCluster(String
    clusterConfigurationFilePath, String fabricPackageSourcePath, Boolean noCleanupOnFailure, Boolean
    force, Int32 maxPercentFailedNodes, UInt32 timeoutInSeconds)
    ---> (Inner Exception #0) System.Fabric.FabricValidationException: Cluster Setup cancelled due to
    validation error(s) found by Best Practices Analyzer. Inspect details in DeploymentTraces log
    folder local to executing location.
       at Microsoft.ServiceFabric.DeploymentManager.DeploymentManagerInternal.<CreateClusterAsyncInter
    nal>d__1.MoveNext()
    --- End of stack trace from previous location where exception was thrown ---
       at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
       at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task
    task)
       at Microsoft.ServiceFabric.DeploymentManager.DeploymentManager.<CreateClusterAsync>d__2.MoveNex
    t()<---
    at Install-ServiceFabricCluster, D:\Microsoft.AzureStack.Solution.Deploy.Fabric.1.1905.0.40\conten
    t\AzureStackInstaller\Utils\WinFabricCluster\WinFabCluster.ps1: line 251
    at <ScriptBlock>, <No file>: line 15
    At C:\CloudDeployment\Setup\Common\DeploySingleNodeCommon.ps1:752 char:5
    +     Invoke-EceAction -RolePath $masterRole -ActionType $ActionPlan @d ...
    +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : NotSpecified: (:) [Invoke-EceAction], InterfaceInvocationFailedExce
       ption
        + FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,Install-ServiceFa
       bricCluster,CloudEngine.Cmdlets.InvokeCmdlet

    VERBOSE: 2> Step: Status of step '60.61.83.1 - (ACS) Setup Azure-consistent Storage service fabric
     ring creation.' is 'Error'. - 7/10/2019 11:29:27 AM

    #########

    I haven't yet rebooted the actual host, is there a shutdown process to follow or a simple reboot is sufficient?

    Thanks

    Wednesday, July 10, 2019 10:59 AM

All replies

  • This Customer is having the same issue. I have worked with them offline, and we are still trying to find the cause. 

    Due to the storage changes, I would make sure that your storage configuration is properly set up, and run the Deployment Checker for the Azure Stack Development Kit

    also, as specified in the error message, do you have a folder called "DeploymentTraces" in the executing directory?

    Thursday, July 11, 2019 12:55 AM
    Moderator
  • Hi Travis,

    I ran the deployment checker for AzureStack scripts and everything passes correctly. The storage pool itself and the virtual disk all seem setup correctly too. I can also see all of the Infrastructure VMs on Hyper-V manager running fine.

    I have looked for the DeploymentTraces folder but unable to locate this to check the logs.

    Not sure what else to check?

    Thursday, July 11, 2019 7:38 AM
  • Just to add.. After a reboot of the actual AzureStack Host.

    The majority of the VM's under Hyper-V manager have now disappeared. Only AzS-DC01, AzS-ERCS01 and AzS-SRNG01 remain in the list of running and the rest are no longer on there. AzureStack deployment now fails to missing VMs.

    Thursday, July 11, 2019 8:25 AM
  • In the other instance of this error, the deployment is failing after a minute, and no VMs are being created. 

    If you would like to reboot the host, there is no special process, just reboot as needed. Note that it will take some time for the VMs to come back up. 

    You can also try to re-run the deployment from the last successful point by re-executing the PowerShell script using the -rerun flag

    cd C:\CloudDeployment\Setup
    .\InstallAzureStackPOC.ps1 -Rerun

    If this fails with the same error, your best bet is to gather the logs. Because of the error, we know the deployment is failing at the Storage or Fabric ring, so you can use the Get-AzureStackLog cmdlet with the -FilterByRole parameter filtering for FabricRing, Storage, FabricRingServices. 

    Thursday, July 11, 2019 11:36 PM
    Moderator