locked
Azure Stack TP3 Hang at step 60.140.149 on New-ASCpiCluster command... RRS feed

 > 

  • Question

  • Question
    Sign in to vote
    0
    Sign in to vote

    The install is hanging at Step 60.140.149. The offending command is:

    New-ASCpiCluster -Name S-Cluster -Cluster
    Microsoft.AzureStack.Fabric.Compute.Client.Powershell.Cluster -GatewayUri
    https://ASAPPGATEWAY.AzureStack.Local:4443

    The ASAPPGateway.AzureStack.Local is registered and found:

    #>nslookup asappgateway.azurestack.local
    Server:  UnKnown
    Address:  192.168.200.224

    Name:    asappgateway.azurestack.local
    Address:  192.168.105.6

    Network connection looks good and routable:

    #> tnc -ComputerName asappgateway.azurestack.local -Port 4443

    ComputerName     : asappgateway.azurestack.local
    RemoteAddress    : 192.168.105.6
    RemotePort       : 4443
    InterfaceAlias   : Management
    SourceAddress    : 192.168.200.65
    TcpTestSucceeded : True

    The connection certificates are valid:

    PS #> $socket = New-Object System.Net.Sockets.TcpClient("asappgateway.azurestack.local", 4443)
    PS #> $stream = $socket.GetStream()
    PS #> $sslStream = New-Object System.Net.Security.SslStream $stream,$true
    PS #> $sslstream.AuthenticateAsClient("asappgateway.azurestack.local", $null,"Tls",$true)
    PS #> $sslstream
    TransportContext          : System.Net.SslStreamContext
    IsAuthenticated           : True
    IsMutuallyAuthenticated   : False
    IsEncrypted               : True
    IsSigned                  : True
    IsServer                  : False
    SslProtocol               : Tls
    CheckCertRevocationStatus : True
    LocalCertificate          :
    RemoteCertificate         : System.Security.Cryptography.X509Certificates.X509Certificate
    CipherAlgorithm           : Aes256
    CipherStrength            : 256
    HashAlgorithm             : Sha1
    HashStrength              : 160
    KeyExchangeAlgorithm      : 44550
    KeyExchangeStrength       : 255
    CanSeek                   : False
    CanRead                   : True
    CanTimeout                : True
    CanWrite                  : True
    ReadTimeout               : -1
    WriteTimeout              : -1
    Length                    :
    Position                  :
    LeaveInnerStreamOpen      : True

    PS #> $sslstream.RemoteCertificate

           Handle Issuer                                                       Subject
           ------ ------                                                       -------
    2221524400336 CN=AzureStackCertificationAuthority, DC=AzureStack, DC=Local CN=ASAppGateway

    PS #> $sslstream.close()
    PS #> $socket.Close()

    **Edited to Check server certificate revocation**

    Why does this registration hang?


    • Edited by cbuzzsaw Wednesday, March 29, 2017 8:35 PM added results with certificate revocation check -- which is valid
    Wednesday, March 29, 2017 8:19 PM

Answers

All replies

  • Question
    Sign in to vote
    0
    Sign in to vote

    We are investigating your issue and will reply ASAP.

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

     

    In the interim, if you haven't already done so, try redeploying using the -rerun parameter.

    NOTE:Make sure you are running the deployment as under the AzureStack\AzureStackAdmin account.

     

    cd C:\CloudDeployment\setup

    .\InstallAzureStackPOC.ps1 -rerun

     

    Please see the updated Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    If you experience any issues with TP3 release, feel free to contact us.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    Thanks,

    Gary Gallanes

    Thursday, March 30, 2017 8:12 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    We are Investigating your issue and require some logs in order to continue troubleshooting. 

    If you could, please email ascustfeedback@microsoft.com to get a workspace setup to upload your logs.  

     

    Make sure to use a Work, Organizational or Student address when contacting ascustfeedback@microsoft.comand include the thread URL in the subject.

     

    https://aka.ms/GetAzureStackLogs :)

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    Thanks,

    Gary Gallanes

    Thursday, March 30, 2017 8:13 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Ok I sent the email to get the workspace.

    After a full 8 hours of letting it hang in that spot in hopes it was just running slow, I stopped the process and restarted the installer while monitoring the process. The process doesn't seem to be able to reach the asappgateway.azurestack.local (192.168.105.6) address anymore. It looks like routing changed somehow as I don't see any routes to that subnet from the POC machine and I can no longer access it. Is there something in the 60.140.??? set of steps that change the networking? Which VM would I look at to validate the asappgateway.azurestack.local is running? All VMs are running. Should I reboot all of them and try the installer again?

    Is there a document that lays out what machines have what services (and mapped names)? 

    I checked networking from the POC machine and am providing the details here:

    Windows IP Configuration
    Ethernet adapter Storage1:

       Connection-specific DNS Suffix  . :
       Link-local IPv6 Address . . . . . : X:X:X:X:c4bd%23
       IPv4 Address. . . . . . . . . . . : 192.168.100.4
       Subnet Mask . . . . . . . . . . . : 255.255.255.192
       Default Gateway . . . . . . . . . :

    Ethernet adapter Management:

       Connection-specific DNS Suffix  . :
       Link-local IPv6 Address . . . . . : X:X:X:X:aee5%9
       IPv4 Address. . . . . . . . . . . : 192.168.200.65
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       IPv4 Address. . . . . . . . . . . : 192.168.200.66
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 192.168.200.1

    Tunnel adapter Local Area Connection* 2:

       Media State . . . . . . . . . . . : Media disconnected
       Connection-specific DNS Suffix  . :

    Ethernet adapter Deployment:

       Connection-specific DNS Suffix  . :
       Link-local IPv6 Address . . . . . : X:X:X:X:ff08%6
       IPv4 Address. . . . . . . . . . . : 10.x.x.x
       Subnet Mask . . . . . . . . . . . : 255.255.255.0
       Default Gateway . . . . . . . . . : 10.x.x.x

    With Routes:

    IPv4 Route Table
    ===========================================================================
    Active Routes:
    Network Destination        Netmask          Gateway       Interface  Metric
              0.0.0.0          0.0.0.0      10.x.x.1     10.x.x.x    281
              0.0.0.0          0.0.0.0    192.168.200.1   192.168.200.65  65515
          10.x.x.0    255.255.255.0         On-link      10.x.x.x    281
         10.x.x.x  255.255.255.255         On-link      10.x.x.x    281
        10.x.x.255  255.255.255.255         On-link      10.x.x.x    281
            127.0.0.0        255.0.0.0         On-link         127.0.0.1    331
            127.0.0.1  255.255.255.255         On-link         127.0.0.1    331
      127.255.255.255  255.255.255.255         On-link         127.0.0.1    331
        192.168.100.0  255.255.255.192         On-link     192.168.100.4    271
        192.168.100.4  255.255.255.255         On-link     192.168.100.4    271
       192.168.100.63  255.255.255.255         On-link     192.168.100.4    271
        192.168.102.0    255.255.255.0    192.168.200.1   192.168.200.65    271
        192.168.105.0    255.255.255.0    192.168.200.1   192.168.200.65    271
        192.168.200.0    255.255.255.0         On-link    192.168.200.65    271
       192.168.200.65  255.255.255.255         On-link    192.168.200.65    271
       192.168.200.66  255.255.255.255         On-link    192.168.200.65    271
      192.168.200.255  255.255.255.255         On-link    192.168.200.65    271
            224.0.0.0        240.0.0.0         On-link         127.0.0.1    331
            224.0.0.0        240.0.0.0         On-link      10.x.x.x    281
            224.0.0.0        240.0.0.0         On-link    192.168.200.65    271
            224.0.0.0        240.0.0.0         On-link     192.168.100.4    271
      255.255.255.255  255.255.255.255         On-link         127.0.0.1    331
      255.255.255.255  255.255.255.255         On-link      10.x.x.x    281
      255.255.255.255  255.255.255.255         On-link    192.168.200.65    271
      255.255.255.255  255.255.255.255         On-link     192.168.100.4    271
    ===========================================================================
    Persistent Routes:
      Network Address          Netmask  Gateway Address  Metric
              0.0.0.0          0.0.0.0      10.x.x.1  Default
              0.0.0.0          0.0.0.0    192.168.200.1   65500
              0.0.0.0          0.0.0.0      10.x.x.1  Default
        192.168.102.0    255.255.255.0    192.168.200.1  Default
        192.168.105.0    255.255.255.0    192.168.200.1  Default
    ===========================================================================

    Friday, March 31, 2017 12:23 AM
  • Question
    Sign in to vote
    0
    Sign in to vote

    There are a number of entries in DNS that are in the 192.168.105.0/24 subnet:

    • ASACSSFClient
    • ASACSSFPortal
    • ApiInternal
    • WASPInternal
    • ASFabSFClient
    • ASFabSFPortal
    • Cpi
    • Crp
    • NonPrivilegedAppGateway
    • Nrp
    • SFAppGateway
    • Srp
    • ASSDRSFClient
    • ASSDRSFPortal

    What setup step or AzureStack component controls the Access to the subnet that those components are in (192.168.105.0/24)?

    EDIT:

    Ok after looking at the logs it looks like step 40.53 failed on the first install (Configure VIPs for all Roles). with exception:

     2017-03-28 18:28:33 Verbose  JSON Get [/networkinterfaces]
    2017-03-28 18:28:33 Verbose  Invoke-WebRequestWithRetries: Get Exception: {
      "error": {
        "code": "NotFound",
        "message": "Resource /loadBalancers/34656680-BD84-41B3-9454-9D366618F7D6 not found.",
        "innerError": "Microsoft.WindowsAzure.Networking.Nrp.Frontend.Common.NrpException: Resource /loadBalancers/34656680-BD84-41B3-9454-9D366618F7D6 not found.\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Common.DataAccess.ResourceProvider.ThrowNotFound(String id)\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Common.DataAccess.ResourceProvider.GetResourceInternal(Resource resourceRef, String etag, Func`2 shouldResolveDependency, ITransaction transaction)\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Common.DataAccess.ResourceProvider.c__DisplayClass7_0`1.b__0(ITransaction internalTransaction)\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Common.DataAccess.DataProvider.Execute[TResult](Func`2 readWriteFunc, ITransaction transaction)\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Operations.Csm.GetResourceOperation`1.Execute()\r\n   at Microsoft.WindowsAzure.Networking.Nrp.Frontend.Operations.OperationBase`1.Run()"
      }
    }

    • Edited by cbuzzsaw Friday, March 31, 2017 3:25 AM Adding additional info
    Friday, March 31, 2017 1:29 AM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Chris,

    We’ve replied to your email @ 3:00PM PST 3/30/2017 with a Workspace link to upload your logs.

      

    Please see the updated instructions for gathering logs in Azure Stack here: https://aka.ms/GetAzureStackLogs

      

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

      

    Thanks,

    Gary Gallanes

    Friday, March 31, 2017 5:44 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    I ran a new AzureStack install and getting hung up at the same spot. I collected and uploaded the logs for this installation which is hanging as the same spot (i.e. installer script trying to reach 192.168.105.6). Please review the logs and let me know how to correct this situation.

    Saturday, April 1, 2017 6:35 AM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Chris,

    We have your logs and are currently investigating. We will reply with ‘next steps’ ASAP.

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,

    Gary Gallanes

    Monday, April 3, 2017 5:57 PM
  • Question
    Sign in to vote
    0
    Sign in to vote
    After looking into the actual error... as before it looks like step 60.140.144 failed first. A rerun doesn't help. The article (https://social.msdn.microsoft.com/Forums/azure/en-US/f8936bbc-ee69-4dc9-aeb5-b52dcf12a3be/azure-stack-tp3-setup-hung-on-step-60140144?forum=AzureStack) didn't seem to help in the previous build try, so I'll wait until you respond. I'll keep this installation at a standard install (i.e. only attempted to run with the -rerun parameter up to this point, until you direct me to do additional steps). Thanks.
    Monday, April 3, 2017 6:35 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hello,

    Our investigation has determined this to be a known issue with TP3 that has been addressed in the TP3 refresh build released on April 6, 2017.

     

    Please download and redeploy using the latest Azure Stack TP3 Refresh POC deployment package. 

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 refresh, feel free to contact us.

     

    TP3 refresh Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

      

     Thanks,

    Gary Gallanes

    Thursday, April 6, 2017 7:19 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Well that didn't seem to work. It looks like it is doing the same thing:

    COMPLETE: Step 60.140.147 - (ACS) Configure Storage Accounts information
    VERBOSE: 1> 1> Step: Status of step '60.140.147 - (ACS) Configure Storage Accounts information' is
     'Success'. - 4/7/2017 1:51:16 PM
    VERBOSE: 1> 1> Action: Action plan 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts'
    completed. - 4/7/2017 1:51:16 PM
    COMPLETE: Action 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts'
    VERBOSE: 1> 1> Action: Status of 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' is
    'Success'. - 4/7/2017 1:51:16 PM
    COMPLETE: Task Cloud - Deployment-Phase5-ConfigureWASAndCreateStorageAccounts
    VERBOSE: 1> 1> Task: Status of action 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' of
    role 'Cloud' is 'Success'. - 4/7/2017 1:51:16 PM

    It is now 3:40PM - been at the same step for almost 2 hours with not much of anything happening except short burst of communication to 192.168.105.6 again over and over.

    The install encountered one error:

    2017-04-07 11:20:46 Verbose  Exception ([MAS-SLB01] Connecting to remote server MAS-SLB01 failed with the following error message : WinRM cannot complete the operation. Verify that the specified computer name is valid, that the computer is accessible over the network, and that a firewall exception for the WinRM service is enabled and allows access from this computer. By default, the WinRM firewall exception for public profiles limits access to remote computers within the same local subnet. For more information, see the about_Remote_Troubleshooting Help topic.) when trying to verify connectivity
    2017-04-07 11:20:46 Verbose  MuxID f8c1cc3b-26b9-4714-b6c3-a47717c81e0e DIP Endpoint [192.168.200.64:8560] routes:
    2017-04-07 11:20:46 Verbose  Route:
    2017-04-07 11:20:46 Verbose  Error: Route not up2017-04-07 11:20:46 Warning  Slb Mux is not up and running.

    ...

    2017-04-07 12:58:07 Error    1> 1> Action: Invocation of step 60.140.142 failed. Stopping invocation of action plan.

    ...

    2017-04-07 12:58:52 Verbose  1> 2> Step: Status of step '60.140.149 - (CPI) Migrate Created VMs, Hosts, and Cluster to CPI' is 'Error'.

    After checking the MAS-SLB01, it was indeed not running.

    I started the node via the Failover Cluster Manager and waited until it was up before restarting with the -rerun option. Where I sit now again waiting :(

    There are no failures in the new log and the last few entries are:

    2017-04-07 13:51:16 Verbose  1> 1> Step: Status of step '60.140.147 - (ACS) Configure Storage Accounts information' is 'Success'.
    2017-04-07 13:51:16 Verbose  1> 1> Action: Action plan 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' completed.
    2017-04-07 13:51:16 Verbose  1> 1> Action: Status of 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' is 'Success'.
    2017-04-07 13:51:16 Verbose  1> 1> Task: Status of action 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' of role 'Cloud' is 'Success'.

    This is very similar to what was happening before the TP3 Refresh.

    Friday, April 7, 2017 9:53 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Chris,

    We have created your Workspace and are awaiting your log files.

    We look forward to continuing this investigation and appreciate your interest in Azure Stack.

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    Thanks,

    Gary Gallanes

    Monday, April 10, 2017 4:58 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Logs uploaded earlier today.

    Thanks,

    Monday, April 10, 2017 7:40 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Thanks Chris,

    We are reviewing your Logs and will reach out to you soon with next steps or possibly to request additional Logs.

     Thanks,

    Gary Gallanes

    Tuesday, April 11, 2017 5:40 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    **Nudge** ... any status?

    Monday, May 1, 2017 10:43 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Don't know if this helps at all but the Roles in Cluster Manager still have GUIDs referenced instead of real names. Possibly something failed prior to this issue?

    Cluster Manager Roles

    Tuesday, May 2, 2017 7:52 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Chris,

    The MAS-SLB01 VM event log shows an unexpected shutdown @ 2:46:41 PM on ‎4/‎17/‎2017 which was just before the deployment began to fail and was not running until next day @ 12:44:36 PM 4/18/2017.

     

    We are still investigating why that unexpected shutdown keeps happening.

    Can you check for crash dump ‘MEMORY.DMP’ files on \\MAS-SLB01\C$\Windows\memory.* and \\MAS-SLB01\C$\memory.* to indicate if MAS-SLB01 was down due to System crash?  

    In the interim, if the environment is still available, can you verify MAS-SLB01 and MAS-SQL01 are up and try redeploying with -rerun and monitor both VMs in VM Manager on the DVM Host. Look for reboots and or shutdowns.

     

    If they stay up and running the deployment should complete.  

    Let us know how it goes.

     

    I hope to have more information regarding the MAS-SLB01 shut down soon.

     

     Thanks,

    Gary Gallanes


    Tuesday, May 2, 2017 10:01 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Gary,

    • There were no dump files (c:\Windows\memory.*) found on either system (MAS-SLB01 or MAS-SQL01).
    • Both systems were running and required services were available (SQL, etc.).
    • The installation failed with a new error this time:

    Error validating credentials. AADSTS*****: The refresh token has expired due to inactivity.The token was issued on 2017-04-17T17:20:07.9945190Z and was inactive for 14.00:00:00.

    How do we reset the token or extend the length past 14 (I assume this is in days)?

    Thanks,

    Chris

    Monday, May 8, 2017 3:22 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Chris,

    To update the refresh token, rerun the deployment as:

      

    Import-Module C:\CloudDeployment\Ecengine\EnterpriseCloudEngine.psd1

    Set-EceSecret -ConfigurationName 'Default' -ContainerName 'AADAdmin' -Credential (get-credential)

    C:\CloudDeployment\Setup\InstallAzureStack.ps1 –Rerun

      

    Let us know how it goes.

     

    We apologize for any inconvenience and appreciate your time and interest in Azure Stack.

    If you continue experience any issues with TP3 release, feel free to contact us.

    https://azure.microsoft.com/en-us/blog/hybrid-application-innovation-with-azure-and-azure-stack/

     

    TP3 Azure Stack Docs:

    https://docs.microsoft.com/en-us/azure/azure-stack/

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-troubleshooting

    https://docs.microsoft.com/en-us/azure/azure-stack/azure-stack-diagnostics

    https://aka.ms/GetAzureStackLogs

     

    Thanks,

    Gary Gallanes

    Monday, May 8, 2017 5:47 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    So which Credential should I use here. Is this the Tenant Admin account?

    Thanks,

    Chris

    Wednesday, May 10, 2017 4:40 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Chris,

    You are Correct. You need to use the Tenant Admin account, also known as the Azure AD Global Admin.

     Thanks,

    Gary Gallanes

    Wednesday, May 10, 2017 5:01 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    That doesn't seem to work. I'm still getting this error:

    Invoke-EceAction : 1> 1> Task: Invocation of interface 'Configure' of role 'Cloud\Fabric\WAS' failed:
    Function 'WAS:Configure' in module 'Classes\WAS\WAS.psm1' raised an exception:
    An error occurred while trying to make a graph API call: {"error":"invalid_grant","error_description":"AADSTS70002:
    Error validating credentials. AADSTS70008: The refresh token has expired due to inactivity.The token was issued on
    2017-04-17T17:20:07.9945190Z and was inactive for 14.00:00:00.\r\nTrace ID:
    b1997461-c45e-47d2-aa0d-2ec202080400\r\nCorrelation ID: 4a8dbf1d-bfb9-49f0-bada-59476418faea\r\nTimestamp: 2017-05-10
    16:58:08Z","error_codes":[70002,70008],"timestamp":"2017-05-10
    16:58:08Z","trace_id":"b1997461-c45e-47d2-aa0d-2ec202080400","correlation_id":"4a8dbf1d-bfb9-49f0-bada-59476418faea"}
    Additional details: {
        "Method":  "POST",
        "ResponseUri":  "https://login.windows.net/90051a1b-20fc-4614-aad6-b90e350b8b5e/oauth2/token?api-version=1.6",
        "StatusCode":  400,
        "StatusDescription":  "Bad Request",
        "IsFromCache":  false,
        "LastModified":  "\/Date(1494435487716)\/"
    }
    at Invoke-GraphApi, C:\CloudDeployment\Roles\IdentityProvider\GraphAPI.psm1: line 625
    at Get-GraphToken, C:\CloudDeployment\Roles\IdentityProvider\GraphAPI.psm1: line 485
    at Update-GraphAccessToken, C:\CloudDeployment\Roles\IdentityProvider\GraphAPI.psm1: line 501
    at Initialize-GraphEnvironment, C:\CloudDeployment\Roles\IdentityProvider\GraphAPI.psm1: line 238
    at ConfigureWAS, C:\CloudDeployment\Roles\WAS\WAS.psm1: line 894
    at Configure, C:\CloudDeployment\Classes\WAS\WAS.psm1: line 39
    at <ScriptBlock>, <No file>: line 9 - 5/10/2017 9:58:07 AM
    At C:\CloudDeployment\Setup\DeploySingleNode.ps1:535 char:5
    +     Invoke-EceAction -RolePath $masterRole -ActionType $actionPlan @d ...
    +     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        + CategoryInfo          : OperationStopped: (:) [Invoke-EceAction], Exception
        + FullyQualifiedErrorId : An error occurred while trying to make a graph API call: {"error":"invalid_grant","error
       _description":"AADSTS70002: Error validating credentials. AADSTS70008: The refresh token has expired due to inactivity.The token was issued on 2017-04-17T17:20:07.9945190Z and was inactive for

    Wednesday, May 10, 2017 5:15 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hi Chris,

     

    The Add-AzureRMAccount updates the configuration file with the subscription information for all subscriptions associated with the account.  The authentication tokens are cached allowing Azure PowerShell cmdlets to be invoked in later sessions without the need to re-authenticate.

      

    Please run the following code to:

    -Update and cache tokens

    -Update the ECS secret in the AADAdmin container

    -Rerun deployment

       

    Add-AzureRMAccount

    Import-Module C:\CloudDeployment\Ecengine\EnterpriseCloudEngine.psd1

    Set-EceSecret -ConfigurationName 'Default' -ContainerName 'AADAdmin' -Credential (get-credential)

    C:\CloudDeployment\Setup\InstallAzureStack.ps1 –Rerun 

    Let us know how it goes.

     Thanks,

    Gary Gallanes


    Thursday, May 11, 2017 6:35 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Gary,

    I had to reboot for the changes to take after running the Set-EceSecret function. The above instruction for "Add-AzureRMAAccount" can't be found so don't know what that is, but before I go further it looks like  I'm running into the following error regarding a certificate thumbprint now:

    Invoke-EceAction : 1> 1> Task: Invocation of interface 'Configure' of role 'Cloud\Fabric\WAS' failed:
    Function 'WAS:Configure' in module 'Classes\WAS\WAS.psm1' raised an exception:
    ERROR during storage initialization: An error occurred while trying to make an API call to Microsoft Storage service:
    {"Message":"Microsoft.Azure.ResourceProvider.Common.Exceptions.ResourceProviderException: The request is unauthorized because the client certificate thumbprint '21D1F4D11E1DBFEC1111C9C05DE1F2D6A111CC11' is not authorized.\r\n   at Microsoft.Azure.ResourceProvider.Authorization.CertificateRequestAuthorizationHandler.ValidateRequestIsAuthorized(HttpRequest Message requestMessage)\r\n   at
    Microsoft.AzureStack.Gateway.Service.MessageHandlers.AuthorizationMessageHandler.d__4.MoveNext()"}
    Additional details: {
        "Method":  "PUT",
        "ResponseUri":  "https://asappgateway.azurestack.local:4443/ResourceGUID/xxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxx/subscriptions/xxxxxx-XXXX-XXXX-XXXX-XXXXXXXXXXX?api-version=2.0",
        "StatusCode":  401,
        "StatusDescription":  "Unauthorized",
        "IsFromCache":  false,
        "LastModified":  "\/Date(1494516253039)\/"

    Thursday, May 11, 2017 10:42 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Gary,

    The last install was going off the rails so I started a new install. I'm at the point where I'm hanging at 60.140.149 with no errors up to this stage. All servers are running, and I seem to be in that endless loop again. I'm going to leave it running until I hear back from you.

    Here is where I'm at in the installer:

    ...

    VERBOSE: 1> 1> Interface: Interface Configure completed. - 5/13/2017 11:03:40 AM
    COMPLETE: Task Cloud\Fabric\ACS - Configure
    VERBOSE: 1> 1> Task: Task completed. - 5/13/2017 11:03:40 AM
    COMPLETE: Step 60.140.147 - (ACS) Configure Storage Accounts information
    VERBOSE: 1> 1> Step: Status of step '60.140.147 - (ACS) Configure Storage Accounts information' is
     'Success'. - 5/13/2017 11:03:40 AM
    VERBOSE: 1> 1> Action: Action plan 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts'
    completed. - 5/13/2017 11:03:40 AM
    COMPLETE: Action 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts'
    VERBOSE: 1> 1> Action: Status of 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' is
    'Success'. - 5/13/2017 11:03:40 AM
    COMPLETE: Task Cloud - Deployment-Phase5-ConfigureWASAndCreateStorageAccounts
    VERBOSE: 1> 1> Task: Status of action 'Deployment-Phase5-ConfigureWASAndCreateStorageAccounts' of
    role 'Cloud' is 'Success'. - 5/13/2017 11:03:40 AM

    I've run a process monitor on the installer script and I'm looping with this (from a network perspective last message was at 11:03:40 AM and as you can see it is 11:45:24 AM - looping for ~45 minutes):

    11:45:24.2337284 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 19296
    11:45:24.2340471 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 14584
    11:45:24.2342524 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 6724
    11:45:25.6812645 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 17124, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:29.9499321 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 17728
    11:45:31.4751494 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 17728, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:32.9508934 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 16916
    11:45:34.2683779 AM powershell.exe 17288 TCP Send WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 618, startime: 3050236, endtime: 3050238, seqnum: 0, connid: 0
    11:45:34.2839643 AM powershell.exe 17288 TCP Send WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 442, startime: 3050236, endtime: 3050240, seqnum: 0, connid: 0
    11:45:34.3022827 AM powershell.exe 17288 TCP Receive WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 5, seqnum: 0, connid: 0
    11:45:34.3023294 AM powershell.exe 17288 TCP Receive WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 277, seqnum: 0, connid: 0
    11:45:34.9617915 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 16916, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:39.2346225 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 19296, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:39.2346938 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 14584, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:39.2347851 AM powershell.exe 17288 Thread Exit  SUCCESS Thread ID: 17084, User Time: 0.0000000, Kernel Time: 0.0000000
    11:45:42.9860510 AM powershell.exe 17288 Thread Create  SUCCESS Thread ID: 9452
    11:45:44.3369892 AM powershell.exe 17288 TCP Send WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 618, startime: 3051242, endtime: 3051245, seqnum: 0, connid: 0
    11:45:44.3525900 AM powershell.exe 17288 TCP Send WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 442, startime: 3051242, endtime: 3051246, seqnum: 0, connid: 0
    11:45:44.3696910 AM powershell.exe 17288 TCP Receive WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 5, seqnum: 0, connid: 0
    11:45:44.3697377 AM powershell.exe 17288 TCP Receive WIN-HFS66U9RG2H.AzureStack.Local:52889 -> 192.168.105.6:4443 SUCCESS Length: 277, seqnum: 0, connid: 0

    Any ideas as to why we are looping?

    Thanks.

    Saturday, May 13, 2017 7:01 PM
  • Question
    Sign in to vote
    0
    Sign in to vote

    Hello Chris,

    We appreciate your patients. This issue has been going on for weeks now. I’d like to setup a call with you if possible to discus next steps and possible trying a ADFS deployment.  I’d really like to get root cause of this error at step 60.140.149 as well.

    If you could, please email ascustfeedback@microsoft.comand I’ll reply with my email and contact info.

     

    Thanks,

    Gary Gallanes

    Monday, May 15, 2017 5:11 PM