none
Container running on ACI on a VNET is unable to reach internet addresses (Outbound connections) RRS feed

  • Question

  • We have a container running via ACI on a VNET that recently has stopped being able to reach internet addresses (google.com, Azure SQL Databases, etc). I have brought up several other containers in the VNET as well as in brand new VNETS that are created by the "az container create" command and they all display the same behavior. I have tested with stock ubuntu, and alpine images (in addition to our original custom image), and have even tried Microsoft's Hello World image. In every instance (every vnet and region I have tried them in) ping to any external address fails (but internal addresses don't), but DNS does resolve. We get the following network interfaces (with the eth0 getting a VNET/Subnet local address)

    eth0      Link encap:Ethernet  HWaddr 36:57:E0:61:78:DF
              inet addr:10.202.94.4  Bcast:0.0.0.0  Mask:255.255.255.0
              inet6 addr: fe80::3457:e0ff:fe61:78df/64 Scope:Link
              UP BROADCAST RUNNING  MTU:1500  Metric:1
              RX packets:12 errors:0 dropped:0 overruns:0 frame:0
              TX packets:12 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:936 (936.0 B)  TX bytes:936 (936.0 B)
    eth1      Link encap:Ethernet  HWaddr C2:DB:44:D5:48:5D
              inet addr:169.254.0.4  Bcast:0.0.0.0  Mask:255.255.0.0
              inet6 addr: fe80::c0db:44ff:fed5:485d/64 Scope:Link
              UP BROADCAST RUNNING  MTU:1500  Metric:1
              RX packets:50 errors:0 dropped:0 overruns:0 frame:0
              TX packets:30 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:3835 (3.7 KiB)  TX bytes:2392 (2.3 KiB)
    lo        Link encap:Local Loopback
              inet addr:127.0.0.1  Mask:255.0.0.0
              inet6 addr: ::1/128 Scope:Host
              UP LOOPBACK RUNNING  MTU:65536  Metric:1
              RX packets:0 errors:0 dropped:0 overruns:0 frame:0
              TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
              collisions:0 txqueuelen:1000
              RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

    We get the following route table (again vnet specific on eth0)

    default via 169.254.0.1 dev eth1
    10.0.0.0/8 via 10.202.94.1 dev eth0
    10.202.94.0/24 dev eth0  src 10.202.94.4
    169.254.0.0/16 dev eth1  src 169.254.0.4
    172.16.0.0/12 via 10.202.94.1 dev eth0
    192.168.0.0/16 via 10.202.94.1 dev eth0

    and the result of traceroute is as follows for google and one of our databases

    / # traceroute google.com
    traceroute to google.com (216.58.193.78), 30 hops max, 46 byte packets
     1  169.254.0.1 (169.254.0.1)  0.007 ms  0.008 ms  0.005 ms
     2  *
    
    # traceroute [ourdb].database.windows.net
    traceroute to [ourdb].database.windows.net (xxx.xx.xxx.xxx), 30 hops max, 46 byte packets
     1  169.254.0.1 (169.254.0.1)  0.008 ms  0.008 ms  0.005 ms
     2  *  *  *
     3  *  *  *
     4  *

    An example of what I used to auto create a vnet with an instance on it.

    az container create --resource-group rg-dev-aci --image alpine:latest --vnet vnet-dev-aci-east2  --subnet subnet-aci --subscription [my subscription id] --name test-alpine-networking-autovnet --command-line "tail -f /dev/null" --location "East US 2" --vnet-address-prefix 10.201.0.0/16 --subnet-address-prefix 10.201.94.0/24


    It seems like something has changed with ACI (on VNET?) network configuration or something else with ACI. I know it is a preview feature of ACI, but I would not think that containers running in a VNET would be prevented from connecting to internet addresses.

    Anyone have any ideas on what we could be doing wrong or what we could do to change things (like the default route's gateway). Is anyone else seeing the same issues?

    Wednesday, July 31, 2019 3:10 PM

All replies

  • I would recommend going through the Preview limitations as described in this document and see if any of these apply to your scenario. 

    There is also a limitation with ACI that it cannot have more than 1024 connections just like standalone VM. Please refer to this article for VM connection limits.

    Let me know if this doesn't help, we can dig deeper.

    Wednesday, July 31, 2019 11:27 PM
    Moderator
  • I appreciate the reply, but it does not appear that any of the limitations apply in this case, especially since it was working within the last week. The only one that seems possible is the one about outbound rules on NSG's not applying, but that would seem to say to me that there are no or should be no restrictions on outbound traffic. Also this is occurring with a fresh linux image that has no connections to it other than the one I am using to get to the bash prompt, so I do not think that the connection limits would apply. My test after the initial problem has been to load a fresh image in various vnets/regions, login to bash or sh and try to ping/traceroute to google or our azure sql database. Any help in digging deeper would be appreciated.
    • Edited by Luke Miles Thursday, August 1, 2019 5:57 AM
    Thursday, August 1, 2019 5:56 AM
  • Thanks you for confirming and providing the details. We would need a support engineer to take a look at the backend for the issue you are seeing. Please send an email to AzCommunity@microsoft.com with your subscription id and link to this thread. Mention 'Attn Karishma' in the subject. I will enable one time free support request for you and share the instructions to open a support ticket. Thanks.
    Thursday, August 1, 2019 5:34 PM
    Moderator
  • Is there any update on this issue? Did you get a chance to send and email with subscription id. Let me know.

    Karishma Tiwari - MSFT

    Wednesday, August 14, 2019 10:41 PM
    Moderator