locked
Ubuntu VM Provisioning failed on ASDK 1811 RRS feed

  • Question

  • Hello, Experts.

    I just installed ASDK 1811 with ADFS mode, modified it to Routing mode, and registered it as Connected model, according to the post "AZURE STACK DEVELOPMENT KIT – ROUTING MODE (REMOVING NAT)".

    It looked good while testing using Test-AzureStack. However, Ubuntu VM provisioning always failed with the following message:

    Provisioning state Provisioning failed. VM 'testvm' failed to provision with timeout.. VmProvisioningTimeout
    Provisioning state error code ProvisioningState/failed/VmProvisioningTimeout
    Guest agent Not Ready. VM agent is not ready
    DISKS
    testvm_OsDisk_1_135a1b08c8004f0682a5582a12a93e31 Provisioning Succeeded.
    



    There were some errors logged in VM Boot Diagnostics log:

    [   24.713391] cloud-init[508]: Cloud-init v. 18.3-9-g2e62cb8a-0ubuntu1~16.04.2 running 'init-local' at Tue, 12 Feb 2019 03:34:26 +0000. Up 24.16 seconds.
    [  OK  ] Started Initial cloud-init job (pre-networking).
    [  OK  ] Reached target Network (Pre).
             Starting Raise network interfaces...
    [  OK  ] Started Raise network interfaces.
    [  OK  ] Reached target Network.
             Starting Initial cloud-init job (metadata service crawler)...
    [  OK  ] Started Azure Linux Agent.
    2019/02/12 03:34:29.029911 INFO Azure Linux Agent Version:2.2.20
    2019/02/12 03:34:29.075747 INFO OS: ubuntu 16.04
    2019/02/12 03:34:29.097596 INFO Python: 3.5.2
    2019/02/12 03:34:29.119110 INFO Run daemon
    2019/02/12 03:34:29.146693 INFO Clean protocol
    2019/02/12 03:34:29.167807 INFO Running CloudInit provisioning handler
    2019/02/12 03:34:29.207763 INFO Detect protocol endpoints
    2019/02/12 03:34:29.235607 INFO Clean protocol
    2019/02/12 03:34:29.257618 INFO WireServer endpoint is not found. Rerun dhcp handler
    2019/02/12 03:34:29.299242 INFO Test for route to 168.63.129.16
    2019/02/12 03:34:29.325752 WARNING No route exists to 168.63.129.16
    2019/02/12 03:34:29.361470 INFO Checking for dhcp lease cache
    2019/02/12 03:34:29.386259 INFO looking for leases in path [/var/lib/dhcp/dhclient.*.leases]
    2019/02/12 03:34:29.448562 INFO cached endpoint not found
    2019/02/12 03:34:29.475903 INFO Cache exists [False]
    2019/02/12 03:34:29.500963 INFO Send dhcp request
    2019/02/12 03:34:29.583514 INFO Examine /proc/net/route for primary interface
    2019/02/12 03:34:29.620743 INFO Primary interface is [eth0]
    2019/02/12 03:34:29.658273 INFO interface [b'lo'] has flags [73], is loopback [True]
    2019/02/12 03:34:29.697695 INFO Interface [b'lo'] skipped
    2019/02/12 03:34:29.734485 INFO interface [b'eth0'] has flags [4163], is loopback [False]
    2019/02/12 03:34:29.766143 INFO Interface [b'eth0'] selected
    2019/02/12 03:34:29.808670 INFO Examine /proc/net/route for primary interface
    2019/02/12 03:34:29.836976 INFO Primary interface is [eth0]
    2019/02/12 03:34:29.863660 INFO interface [b'lo'] has flags [73], is loopback [True]
    2019/02/12 03:34:29.900369 INFO Interface [b'lo'] skipped
    2019/02/12 03:34:30.009755 INFO interface [b'eth0'] has flags [4163], is loopback [False]
    2019/02/12 03:34:30.036043 INFO Interface [b'eth0'] selected
    2019/02/12 03:34:30.154574 INFO Configure routes
    2019/02/12 03:34:30.177206 INFO Gateway:10.1.1.1
    2019/02/12 03:34:30.203236 INFO Routes:None
    2019/02/12 03:34:30.233404 INFO WireServer is not responding. Reset endpoint
    2019/02/12 03:34:30.273980 INFO Protocol endpoint not found: WireProtocol, [ProtocolError] WireProtocol endpoint is None
    2019/02/12 03:36:01.410544 INFO Protocol endpoint not found: MetadataProtocol, [ProtocolError] [HttpError] [HTTP Failed] GET http://169.254.169.254/Microsoft.Compute/identity?api-version=2015-05-01-preview -- IOError timed out -- 6 attempts made
    2019/02/12 03:36:01.455127 INFO Retry detect protocols: retry=0
    2019/02/12 03:36:11.483252 INFO WireServer endpoint is not found. Rerun dhcp handler
    2019/02/12 03:36:11.494313 INFO Test for route to 168.63.129.16
    2019/02/12 03:36:11.503069 WARNING No route exists to 168.63.129.16
    2019/02/12 03:36:11.512283 INFO Send dhcp request
    2019/02/12 03:36:11.523660 INFO Examine /proc/net/route for primary interface
    2019/02/12 03:36:11.533171 INFO Primary interface is [eth0]
    2019/02/12 03:36:11.542514 INFO interface [b'lo'] has flags [73], is loopback [True]
    2019/02/12 03:36:11.565703 INFO Interface [b'lo'] skipped
    2019/02/12 03:36:11.574621 INFO interface [b'eth0'] has flags [4163], is loopback [False]
    2019/02/12 03:36:11.587484 INFO Interface [b'eth0'] selected
    2019/02/12 03:36:11.600452 INFO Examine /proc/net/route for primary interface
    2019/02/12 03:36:11.611244 INFO Primary interface is [eth0]
    2019/02/12 03:36:11.619911 INFO interface [b'lo'] has flags [73], is loopback [True]
    2019/02/12 03:36:11.631631 INFO Interface [b'lo'] skipped
    2019/02/12 03:36:11.641645 INFO interface [b'eth0'] has flags [4163], is loopback [False]
    2019/02/12 03:36:11.652054 INFO Interface [b'eth0'] selected
    2019/02/12 03:36:11.671504 INFO Configure routes
    2019/02/12 03:36:11.680090 INFO Gateway:10.1.1.1
    2019/02/12 03:36:11.698752 INFO Routes:None
    2019/02/12 03:36:11.710308 INFO WireServer is not responding. Reset endpoint
    2019/02/12 03:36:11.724262 INFO Protocol endpoint not found: WireProtocol, [ProtocolError] WireProtocol endpoint is None


    It looked like that the Azure Linux Agent failed to connected to 168.63.129.16. So, how to check and fix the problem on ASDK?

    Thank in advance.

    Best Regards,

    Ryan Bao




    Tuesday, February 12, 2019 5:51 AM

Answers

  • The reference link is: http://blog.azureinfra.com/2017/08/07/azure-stack-development-kit-routing-mode-removing-nat/.

    I'm curious where the address 168.63.129.16 is coming from, is it a virtual address inside Network Controller?  

    Anyway, I've just moved to ASDK 1901, using the default VPN connections, ADFS Mode, and Connected to Azure. Single VM deployment is OK now, but AKS deployment failed again. The link for the AKS issue is:   

    https://social.msdn.microsoft.com/Forums/windows/en-US/43bf2165-5de7-4a5e-8f9b-df625abe2735/aks-deployment-in-asdk?forum=AzureStack

    Thanks for your kindly help.



    Friday, February 15, 2019 12:57 PM

All replies

  • I am unable to find the post you are referring to unfortunately. 

    VMs report their status to the wire server, and for some reason the route seems to be gone, likely due to the routing changes. 

    2019/02/12 03:34:29.257618 INFO WireServer endpoint is not found. Rerun dhcp handler
    2019/02/12 03:34:29.299242 INFO Test for route to 168.63.129.16
    2019/02/12 03:34:29.325752 WARNING No route exists to 168.63.129.16

    the VMs should still function for most things, but the status will show as "failed" in the portal. Can you share a link to the post you followed?

    Friday, February 15, 2019 1:10 AM
  • The reference link is: http://blog.azureinfra.com/2017/08/07/azure-stack-development-kit-routing-mode-removing-nat/.

    I'm curious where the address 168.63.129.16 is coming from, is it a virtual address inside Network Controller?  

    Anyway, I've just moved to ASDK 1901, using the default VPN connections, ADFS Mode, and Connected to Azure. Single VM deployment is OK now, but AKS deployment failed again. The link for the AKS issue is:   

    https://social.msdn.microsoft.com/Forums/windows/en-US/43bf2165-5de7-4a5e-8f9b-df625abe2735/aks-deployment-in-asdk?forum=AzureStack

    Thanks for your kindly help.



    Friday, February 15, 2019 12:57 PM