none
Azure Kubernetes Services - Trying to update authorized apiserver ip ranges fails due to service CIDR RRS feed

  • Question

  • Hi

    I have a running AKS where I want to change the authorized apiserver ip ranges. However, running the following command:

    az aks update --resource-group TLP --name tlpkube --api-server-authorized-ip-ranges 217.63.xx.xx/32,77.221.xx.xx/29,2.104.xx.xx/32 (full IP's replaced here with xx)

    It fails with the error "Operation failed with status: 'Bad Request'. Details: The specified service CIDR 10.41.0.0/24 is conflicted with an existing subnet CIDR". But I don't attempt to change the service CIDR - and you cannot according to the documentation. It's a parameter when setting it up initially.

    I have it hooked up to an existing virtual network (that is connected to a site-to-site VPN) which have a subnet setup like below:


    I have attempted to delete the subnet, but it relates to the running cluster, so it's not possible.

    My attempted network setup is like this:


    The network profile from the "az aks show" command is like this:

    Because I'm now connecting from a new IP, I'm unable to connect to the administration API in my cluster.

    What is going on here? How do I resolve it?

    Thanks!

    Wednesday, July 1, 2020 1:43 PM

Answers

All replies

  • Because I'm still pre-production, I ended up deleting the AKS instance.

    But even after re-creating it, the issue persists.

    I use the same virtual network as pictures above with the subnets, however before creating a new AKS instance I had to delete the "KubeClusterSubnet".

    Steps to reproduce:

    1. "az aks create --name timekube --resource-group TLP --generate-ssh-keys --subscription 304b3e38-1c50-4c5c-8dc7-xxxxxx --service-cidr 10.41.0.0/24 --node-count 1 --admin-username adm --vnet-subnet-id /subscriptions/304b3e38-1c50-4c5c-8dc7-xxxxx/resourceGroups/TLP/providers/Microsoft.Network/virtualNetworks/tl-vpn-connected/subnets/KubePodSubnet --workspace-resource-id /subscriptions/304b3e38-1c50-4c5c-8dc7-xxxxxx/resourcegroups/tlp/providers/microsoft.operationalinsights/workspaces/time-aks-log --network-plugin azure --enable-addons monitoring --dns-service-ip 10.41.0.10 --service-principal 82c11079-5f9f-4964-b8a8-xxxxxx--client-secret a70d1413-eab8-46d9-a5e4-xxxxxx

    2. Re-create the "KubeClusterSubnet" with IP range 10.41.0.0/24

    3. "az aks update --resource-group TLP --name timekube --api-server-authorized-ip-ranges "77.221.xxx.xxx/29"

    However, this time around I was able to delete the subnet, and then rerun #3 with success.

    As far as I understand in order to route traffic correctly in the virtual network, I need the subnet to be explicitly available in the subnets page.


    • Edited by Oexenhave Thursday, July 2, 2020 1:39 PM
    Thursday, July 2, 2020 1:27 PM
  • After reaching out for further help on Twitter, Sean McKenna replied back to me with the answer.

    https://twitter.com/seanmckmsft/status/1281298683309375488

    Since the service CIDR is solely a K8s concept internal to the cluster, the Azure SDN doesn't block you from overlapping. You can check with the service CIDR with:

    az aks show -n clusterName -g rgName -o json --query networkProfile.serviceCidr"

    Conclusion being that I should generate a separate subnet for the internal loadbalancer (e.g. 10.45.0.0/24) and specify it in the yaml.

    https://docs.microsoft.com/en-us/azure/aks/internal-lb


    • Marked as answer by Oexenhave Monday, July 13, 2020 11:32 AM
    • Edited by Oexenhave Monday, July 13, 2020 11:32 AM
    Monday, July 13, 2020 11:32 AM