The following forum(s) are migrating to a new home on Microsoft Q&A (Preview): Azure Virtual Machines!

Ask new questions on Microsoft Q&A (Preview).
Interact with existing posts until December 13, 2019, after which content will be closed to all new and existing posts.

Learn More

 none
How to create custom script extension on virtual machine scale set of linux using azure resource manager template? RRS feed

  • Question

  • Here is my template1.json which is used to create cluster with virtual machine scale set. I want to add extensions to it using the same arm template1. I tried including script of extensions in the properties section of managed clusters ( template2.json) but it is throwing error of parent resource vnetName not found.

    I tried adding extensionProfile in the properties section of managed clusters but that also didn't work.

    can anyone help me out how can I set custom script extension using same arm template?

    template1.json

    { "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": { "vnetName": { "type": "string", "metadata": { "description": "VNet name" } }, "location": { "type": "string", "defaultValue": "[resourceGroup().location]", "metadata": { "description": "Azure resource location" } }, "vnetAddressPrefix": { "type": "string", "metadata": { "description": "Virtual network address range" } }, "subnetName": { "type": "string", "metadata": { "description": "Azure subnet Name" } }, "subnetPrefix": { "type": "string", "metadata": { "description": "Azure subnet prefix" } }, "kubernetesVersion": { "type": "string", "metadata": { "description": "kubernetesVersion" } }, "enableRBAC": { "defaultValue": false, "type": "bool", "metadata": { "description": "boolean flag to turn on and off of RBAC" } }, "dnsPrefix": { "type": "string", "metadata": { "description": "Optional DNS prefix to use with hosted Kubernetes API server FQDN." } }, "enableHttpApplicationRouting": { "defaultValue": false, "type": "bool", "metadata": { "description": "boolean flag to turn on and off of http application routing" } }, "osDiskSizeGB": { "defaultValue": 0, "minValue": 0, "maxValue": 1023, "type": "int", "metadata": { "description": "Disk size (in GB) to provision for each of the agent pool nodes. This value ranges from 0 to 1023. Specifying 0 will apply the default disk size for that agentVMSize." } }, "agentCount": { "minValue": 1, "maxValue": 50, "type": "int", "metadata": { "description": "The number of agent nodes for the cluster. Production workloads have a recommended minimum of 3." } }, "agentVMSize": { "type": "string", "metadata": { "description": "The size of the Virtual Machine." } }, "masterCount": { "type": "int", "allowedValues": [ 1 ], "metadata": { "description": "The number of Kubernetes masters for the cluster." } }, "adminUsername": { "type": "string", "defaultValue": "azureuser", "metadata": { "description": "User name for the Linux Virtual Machines." } }, "osType": { "defaultValue": "Linux", "allowedValues": [ "Linux" ], "type": "string", "metadata": { "description": "The type of operating system." } }, "maxPods": { "defaultValue": 30, "type": "int", "metadata": { "description": "Maximum number of pods that can run on a node." } }, "ServicePrincipalClientId": { "type": "string", "metadata": { "description": "ServicePrincipalClientId" } }, "ServicePrincipalClientSecret": { "type": "string", "metadata": { "description": "ServicePrincipalClientId" } }, "ServicePrincipalObjectId": { "type": "string", "metadata": { "description": "ServicePrincipalClientId" } }, "sshRSAPublicKey": { "type": "securestring", "metadata": { "description": "Configure all linux machines with the SSH RSA public key string" } }, "networkPlugin": { "allowedValues": [ "azure", "kubenet" ], "defaultValue": "kubenet", "type": "string", "metadata": { "description": "Network plugin used for building Kubernetes network." } }, "dockerBridgeCidr": { "type": "string", "metadata": { "description": "A CIDR notation IP for Docker bridge." } }, "dnsServiceIP": { "type": "string", "metadata": { "description": "Containers DNS server IP address." } }, "serviceCidr": { "type": "string", "metadata": { "description": "A CIDR notation IP range from which to assign service cluster IPs." } }, "resourceGroup": { "type": "string", "defaultValue": "[resourceGroup().name]", "metadata": { "description": "Azure resource group." } }, "autoscalingEnabled": { "type": "bool", "defaultValue": true, "metadata": { "description": "Flag to enable autoscaling." } }, "nodesMinimum": { "type": "int", "defaultValue": 2, "metadata": { "description": "Minimum number of nodes for autoscaling." } }, "nodesMaximum": { "type": "int", "defaultValue": 4, "metadata": { "description": "Maximum number of nodes for autoscaling." } } }, "variables": { "mastersEndpointDNSNamePrefix":"[concat(parameters('dnsPrefix'),'mgmt')]" }, "resources": [ { "type": "Microsoft.Resources/deployments", "apiVersion": "2019-08-01", "name": "nestedTemplate", "properties": { "mode": "Incremental", "template": { "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "resources": [ { "apiVersion": "2019-06-01", "type": "Microsoft.Network/virtualNetworks", "name": "[parameters('vnetName')]", "location": "[parameters('location')]", "properties": { "addressSpace": { "addressPrefixes": [ "[parameters('vnetAddressPrefix')]" ] }, "subnets": [ { "name": "[parameters('subnetName')]", "properties": { "addressPrefix": "[parameters('subnetPrefix')]" } } ] } }, { "type": "Microsoft.ContainerService/managedClusters", "name":"[concat('Kluster-',resourceGroup().name)]", "apiVersion": "2019-08-01", "location": "[parameters('location')]", "tags": {}, "dependsOn": [ "[concat('Microsoft.Resources/deployments/', 'ClusterSubnetRoleAssignmentDeployment')]" ], "properties": { "kubernetesVersion": "[parameters('kubernetesVersion')]", "enableRBAC": "[parameters('enableRBAC')]", "dnsPrefix": "[parameters('dnsPrefix')]", "addonProfiles": { "httpApplicationRouting": { "enabled": "[parameters('enableHttpApplicationRouting')]" } }, "masterProfile": { "count": "[parameters('masterCount')]", "vmSize": "[parameters('agentVMSize')]", "dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]" }, "agentPoolProfiles": [ { "name": "agentpool", "osDiskSizeGB": "[parameters('osDiskSizeGB')]", "count": "[parameters('agentCount')]", "vmSize": "[parameters('agentVMSize')]", "osType": "[parameters('osType')]", "storageProfile": "ManagedDisks", "vnetSubnetID": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]", "maxPods": "[parameters('maxPods')]", "enableAutoScaling": "[parameters('autoscalingEnabled')]", "minCount": "[parameters('nodesMinimum')]", "maxCount": "[parameters('nodesMaximum')]", "type": "VirtualMachineScaleSets" } ], "linuxProfile": { "adminUsername": "[parameters('adminUsername')]", "ssh": { "publicKeys": [ { "keyData": "[parameters('sshRSAPublicKey')]" } ] } }, "servicePrincipalProfile": { "clientId": "[parameters('ServicePrincipalClientId')]", "secret": "[parameters('ServicePrincipalClientSecret')]" }, "networkProfile": { "networkPlugin": "[parameters('networkPlugin')]", "serviceCidr": "[parameters('serviceCidr')]", "dnsServiceIP": "[parameters('dnsServiceIP')]", "dockerBridgeCidr": "[parameters('dockerBridgeCidr')]" } } }, { "type": "Microsoft.Resources/deployments", "name": "ClusterSubnetRoleAssignmentDeployment", "apiVersion": "2019-08-01", "subscriptionId": "[subscription().subscriptionId]", "resourceGroup": "[parameters('resourceGroup')]", "dependsOn": [ "[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]" ], "properties": { "mode": "Incremental", "template": { "$schema": "https://schema.management.azure.com/schemas/2018-05-01/deploymentTemplate.json#", "contentVersion": "1.0.0.0", "parameters": {}, "variables": {}, "resources": [ { "type": "Microsoft.Network/virtualNetworks/subnets/providers/roleAssignments", "apiVersion": "2019-04-01-preview", "name": "[concat(parameters('vnetName'), '/', parameters('subnetName'), '/Microsoft.Authorization/', guid(resourceGroup().id, deployment().name))]", "properties": { "roleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', '4d97b98b-1d4f-4787-a291-c67834d212e7')]", "principalId": "[parameters('ServicePrincipalObjectId')]", "scope": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]" } } ] } } } ] } } } ] }

    template2.json

                                                                                                     

    {
        "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
        "contentVersion": "1.0.0.0",
        "parameters": {
            "vnetName": {
                "type": "string",
                "metadata": {
                    "description": "VNet name"
                  }
            },
            "location": {
                "type": "string",
                "defaultValue": "[resourceGroup().location]",
                "metadata": {
                    "description": "Azure resource location"
                  }
            },
            "vnetAddressPrefix": {
                "type": "string",
                "metadata": {
                    "description": "Virtual network address range"
                  }
            },
            "subnetName": {
                "type": "string",
                "metadata": {
                    "description": "Azure subnet Name"
                }
            },
            "subnetPrefix": {
                "type": "string",
                "metadata": {
                    "description": "Azure subnet prefix"
                }
            },
            "kubernetesVersion": {
                "type": "string",
                "metadata": {
                    "description": "kubernetesVersion"
                }
            },
            "enableRBAC": {
                "defaultValue": false,
                "type": "bool",
                "metadata": {
                    "description": "boolean flag to turn on and off of RBAC"
                }
            },
            "dnsPrefix": {
                "type": "string",
                "metadata": {
                    "description": "Optional DNS prefix to use with hosted Kubernetes API server FQDN."
                }
            },
            "enableHttpApplicationRouting": {
                "defaultValue": false,
                "type": "bool",
                "metadata": {
                    "description": "boolean flag to turn on and off of http application routing"
                }
            },
            "osDiskSizeGB": {
                "defaultValue": 0,
                "minValue": 0,
                "maxValue": 1023,
                "type": "int",
                "metadata": {
                    "description": "Disk size (in GB) to provision for each of the agent pool nodes. This value ranges from 0 to 1023. Specifying 0 will apply the default disk size for that agentVMSize."
                }
            },
            "agentCount": {
                "minValue": 1,
                "maxValue": 50,
                "type": "int",
                "metadata": {
                    "description": "The number of agent nodes for the cluster. Production workloads have a recommended minimum of 3."
                }
            },
            "agentVMSize": {
                "type": "string",
                "metadata": {
                    "description": "The size of the Virtual Machine."
                }
            },
            "masterCount": {
                "type": "int",
                "allowedValues": [
                  1
                ],
                "metadata": {
                  "description": "The number of Kubernetes masters for the cluster."
                }
            },
            "adminUsername": {
                "type": "string",
                "defaultValue": "azureuser",
                "metadata": {
                   "description": "User name for the Linux Virtual Machines."
                }
            },
            "osType": {
                "defaultValue": "Linux",
                "allowedValues": [
                    "Linux"
                ],
                "type": "string",
                "metadata": {
                    "description": "The type of operating system."
                }
            },
            "maxPods": {
                "defaultValue": 30,
                "type": "int",
                "metadata": {
                    "description": "Maximum number of pods that can run on a node."
                }
            },
            "ServicePrincipalClientId": {
                "type": "string",
                "metadata": {
                    "description": "ServicePrincipalClientId"
                }
            },
            "ServicePrincipalClientSecret": {
                "type": "string",
                "metadata": {
                    "description": "ServicePrincipalClientId"
                }
            },
            "ServicePrincipalObjectId": {
                "type": "string",
                "metadata": {
                    "description": "ServicePrincipalClientId"
                }
            },
            "sshRSAPublicKey": {
                "type": "securestring",
                "metadata": {
                   "description": "Configure all linux machines with the SSH RSA public key string"
                }
            },
            "networkPlugin": {
                "allowedValues": [
                    "azure",
                    "kubenet"
                ],
                "defaultValue": "kubenet",
                "type": "string",
                "metadata": {
                    "description": "Network plugin used for building Kubernetes network."
                }
            },
            "dockerBridgeCidr": {
                "type": "string",
                "metadata": {
                    "description": "A CIDR notation IP for Docker bridge."
                }
            },
            "dnsServiceIP": {
                "type": "string",
                "metadata": {
                    "description": "Containers DNS server IP address."
                }
            },
            "serviceCidr": {
                "type": "string",
                "metadata": {
                    "description": "A CIDR notation IP range from which to assign service cluster IPs."
                }
            },
            "resourceGroup": {
                "type": "string",
                "defaultValue": "[resourceGroup().name]",
                "metadata": {
                    "description": "Azure resource group."
                }
            },
           "autoscalingEnabled": {
                "type": "bool",
                "defaultValue": true,
                "metadata": {
                    "description": "Flag to enable autoscaling."
                }
            },
           "nodesMinimum": {
                "type": "int",
                "defaultValue": 2,
                "metadata": {
                    "description": "Minimum number of nodes for autoscaling."
                }
            },
          "nodesMaximum": {
                "type": "int",
                "defaultValue": 4,
                "metadata": {
                    "description": "Maximum number of nodes for autoscaling."
                }
            }
        },
        "variables": {
    "mastersEndpointDNSNamePrefix":"[concat(parameters('dnsPrefix'),'mgmt')]",
            "vmssName":"aks-agentpool-43678249-vmss"
        },
        "resources": [
               {
                "type": "Microsoft.Resources/deployments",
                "apiVersion": "2019-08-01",
                "name": "nestedTemplate",
                "properties": {
                  "mode": "Incremental",
                  "template": {
                    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentTemplate.json#",
                    "contentVersion": "1.0.0.0",
                    "resources": [
                        {
                            "apiVersion": "2019-06-01",
                            "type": "Microsoft.Network/virtualNetworks",
                            "name": "[parameters('vnetName')]",
                            "location": "[parameters('location')]",
                            "properties": {
                                "addressSpace": {
                                    "addressPrefixes": [
                                        "[parameters('vnetAddressPrefix')]"
                                    ]
                                },
                                "subnets": [
                                    {
                                        "name": "[parameters('subnetName')]",
                                        "properties": {
                                            "addressPrefix": "[parameters('subnetPrefix')]"
                                        }
                                    }
                                ]
                            }
                       },
                       {
                            "type": "Microsoft.ContainerService/managedClusters",
                            "name":"[concat('Kluster-',resourceGroup().name)]",
                            "apiVersion": "2019-08-01",
                            "location": "[parameters('location')]",
                            "tags": {},
                            "dependsOn": [
                                "[concat('Microsoft.Resources/deployments/', 'ClusterSubnetRoleAssignmentDeployment')]"
                            ],
                            "properties": {
                                "kubernetesVersion": "[parameters('kubernetesVersion')]",
                                "enableRBAC": "[parameters('enableRBAC')]",
                                "dnsPrefix": "[parameters('dnsPrefix')]",
                                "addonProfiles": {
                                    "httpApplicationRouting": {
                                        "enabled": "[parameters('enableHttpApplicationRouting')]"
                                    }
                                },
        "masterProfile": {
    "count": "[parameters('masterCount')]",
    "vmSize": "[parameters('agentVMSize')]",
    "dnsPrefix": "[variables('mastersEndpointDNSNamePrefix')]"
                },
                                "agentPoolProfiles": [
                                    {
                                        "name": "agentpool",
                                        "osDiskSizeGB": "[parameters('osDiskSizeGB')]",
                                        "count": "[parameters('agentCount')]",
                                        "vmSize": "[parameters('agentVMSize')]",
                                        "osType": "[parameters('osType')]",
                                        "storageProfile": "ManagedDisks",
                                        "vnetSubnetID": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]",
                                        "maxPods": "[parameters('maxPods')]",
                                        "enableAutoScaling": "[parameters('autoscalingEnabled')]",
                                        "minCount": "[parameters('nodesMinimum')]",
                                        "maxCount": "[parameters('nodesMaximum')]",
                                        "type": "virtualMachineScaleSets"
                                    }
                                ],
        "linuxProfile": {
        "adminUsername": "[parameters('adminUsername')]",
        "ssh": {
                  "publicKeys": [
    {
                      "keyData": "[parameters('sshRSAPublicKey')]"
    }
          ]
                }
        },
                                "servicePrincipalProfile": {
                                    "clientId": "[parameters('ServicePrincipalClientId')]",
                                    "secret": "[parameters('ServicePrincipalClientSecret')]"
                                },
                                "networkProfile": {
                                    "networkPlugin": "[parameters('networkPlugin')]",
                                    "serviceCidr": "[parameters('serviceCidr')]",
                                    "dnsServiceIP": "[parameters('dnsServiceIP')]",
                                    "dockerBridgeCidr": "[parameters('dockerBridgeCidr')]"
                                }
                            },
                            "resources": [
                             {
                               "apiVersion": "2019-03-01",
                               "type": "Microsoft.Compute/virtualMachines/extensions",
                               "name": "[concat('aks-agentpool-43678249-vmss','/', 'Extensions')]",
                               "location": "[parameters('location')]",
                               "dependsOn": [

                                   "[concat('Microsoft.ContainerService/managedClusters/', concat('Kluster-',resourceGroup().name))]"
                                ],
                               "properties": {
                                   "publisher": "Microsoft.Azure.Extensions",
                                   "type": "CustomScript",
                                   "typeHandlerVersion": "2.0",
                                   "autoUpgradeMinorVersion":true,
                                   "settings": {
                                       "skipDos2Unix":false,
                                       "fileUris": ["publicfilelink"],
                                       "commandToExecute": "./filename.sh"
                                }
                              }
                            }
                          ]
                        },
                        {
                            "type": "Microsoft.Resources/deployments",
                            "name": "ClusterSubnetRoleAssignmentDeployment",
                            "apiVersion": "2019-08-01",
                            "subscriptionId": "[subscription().subscriptionId]",
                            "resourceGroup": "[parameters('resourceGroup')]",
                            "dependsOn": [
                                "[concat('Microsoft.Network/virtualNetworks/', parameters('vnetName'))]"
                            ],
                            "properties": {
                                "mode": "Incremental",
                                "template": {
                                    "$schema": "https://schema.management.azure.com/schemas/2018-05-01/deploymentTemplate.json#",
                                    "contentVersion": "1.0.0.0",
                                    "parameters": {},
                                    "variables": {},
                                    "resources": [
                                        {
                                            "type": "Microsoft.Network/virtualNetworks/subnets/providers/roleAssignments",
                                            "apiVersion": "2019-04-01-preview",
                                            "name": "[concat(parameters('vnetName'), '/', parameters('subnetName'), '/Microsoft.Authorization/', guid(resourceGroup().id, deployment().name))]",
                                            "properties": {
                                                "roleDefinitionId": "[concat('/subscriptions/', subscription().subscriptionId, '/providers/Microsoft.Authorization/roleDefinitions/', '4d97b98b-1d4f-4787-a291-c67834d212e7')]",
                                                "principalId": "[parameters('ServicePrincipalObjectId')]",
                                                "scope": "[concat(resourceGroup().id,'/providers/Microsoft.Network/virtualNetworks/',parameters('vnetName'),'/subnets/',parameters('subnetName'))]"
                                            }
                                        }
                                    ]
                                }
                            }
                        }
                    ]
                  }
                }
            }

        ]
    }

    Error thrown:

    • Edited by Soumya.S Tuesday, November 19, 2019 7:43 AM
    Tuesday, November 19, 2019 7:28 AM

All replies

  • Hi Soumya,

    "Microsoft.ContainerService/managedClusters" will create a kubernetes cluster. Thats the second resource in the ARM template which you shared.

    If you are planning to create a vmss, Please use the type "Microsoft.Compute/virtualMachineScaleSets"

    Reference for the VMSS ARM template is here.

    https://github.com/Azure/azure-quickstart-templates/tree/master/201-vmss-custom-script-windows this template will help you to create a VMSS with custom script extension.

    For the benefit of the community, Please click on "mark as answer" for the replies which helped you to solve your issue

    Tuesday, November 19, 2019 10:29 AM
    Moderator
  • Hello,

    Any update on the issue?

    If the suggested response helped you resolve your issue, do click on "Mark as Answer" and "Up-Vote" for the answer that helped you for benefit of the community.

    Thanks.
    Tuesday, November 26, 2019 10:50 AM
    Moderator
  • Hello,

    Any update on the issue?

    If the suggested response helped you resolve your issue, do click on "Mark as Answer" and "Up-Vote" for the answer that helped you for benefit of the community.

    Thanks.
    Friday, November 29, 2019 9:57 AM
    Moderator