none
Linux Encryption failing "Install Succeeded is not a valid versioned Key Vault Secret URL" RRS feed

  • Question

  • Hi,

    I've been struggling with this one for a day.  I've been building a number of Linux systems over the past week, everything is built from PowerShell scripts and up until yesterday everything was going great.  I've got one Ubuntu 14.04LTS VM that just will not encrypt.  As mentioned I am using PowerShell scripts and these scripts have been working perfectly on the other VMs (created from the same templates).  I've even deleted the VMs and rebuilt it, I've tried encrypting before making any changes to the VM i.e. updates.  I am only trying to encrypt the data drive, so not the OS which can be problematic.  I've done some searching on the web and found that the error I am getting could be to do with sequence numbers, however I've changed these and I can see the changes in the settings files within the VM so I know they are taking.

    The error:

    Set-AzureRmVMDiskEncryptionExtension : Install Succeeded is not a valid versioned Key Vault Secret URL. It should be in the format https://<vaultEndpoint>/secrets/<secretName>/<secretVersion>.
    ErrorCode: InvalidParameter
    ErrorMessage: Install Succeeded is not a valid versioned Key Vault Secret URL. It should be in the format https://<vaultEndpoint>/secrets/<secretName>/<secretVersion>.
    StatusCode: 400
    ReasonPhrase: Bad Request

    I've tried creating a new key vault and AAD App and the error still persists, the only thing I've not changed is the subscription.  For info I have 8 other VMs encrypted against the same Key vault and AAD and they are all working (they are Windows VMs).  VMs have access to the internet and DNS is resolving ok.

    Comparing the extension.log files of a VM that has encrypted it looks like the BEK isn't being created but the logs are limited so I can't tell where this is failing exactly.

    Extension.log from failed VM:

    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: Encryption operation: EnableEncryption
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Enabling encryption
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] make sure path exists, executing: /bin/mkdir -p /var/lib/azure_disk_encryption_config/
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Executing: /bin/mkdir -p /var/lib/azure_disk_encryption_config/
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] value of prop_name:BekFileName not found.
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] found one ide with vmbus: vmbus_0_1 and the sdx path is: sda
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] found one ide with vmbus: vmbus_0_2 and the sdx path is: sdb
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] getting blk info for: /dev/sda
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Executing: lvs --noheadings --nameprefixes --unquoted -o lv_name,vg_name,lv_kernel_major,lv_kernel_minor
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Process creation failed: [Errno 2] No such file or directory
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] getting blk info for: /dev/sdb
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Executing: lvs --noheadings --nameprefixes --unquoted -o lv_name,vg_name,lv_kernel_major,lv_kernel_minor
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Info] Process creation failed: [Errno 2] No such file or directory
    2018/03/16 11:14:17 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 2548: [Warning] EncryptionConfig is present, but could not get the BEK file.

    Extension.log from a successful VM:

    17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 95904: [Info] Enabling encryption
    2018/03/15 17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 95904: [Info] make sure path exists, executing: /bin/mkdir -p /var/lib/azure_disk_encryption_config/
    2018/03/15 17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 95904: [Info] Executing: /bin/mkdir -p /var/lib/azure_disk_encryption_config/
    2018/03/15 17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: set most recent sequence number to 0
    2018/03/15 17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 95904: [Info] start creating kek secret
    2018/03/15 17:05:31 [Microsoft.Azure.Security.AzureDiskEncryptionForLinux-1.0]: 95904: [Info] getting the access token.

    Any tips on how to troubleshoot this would be great?

    Rob


    • Edited by RobC_CTL Friday, March 16, 2018 1:01 PM
    Friday, March 16, 2018 1:00 PM

All replies

  • Rob, 

    Thanks for reaching out. The error message could be related to older metadata that's being used rather than the new generated keys. I'd recommend cleaning some of the directories on this machine which are related to the encryption:

    This would remove the ADE and backup encryption
    $ResourceGroupName= 'yourRG';
    $VMName= 'YourVM';
     
    $VirtualMachine= Get-AzureRmVM -ResourceGroupName $ResourceGroupName -VMName $VMName
    $Extensions= $VirtualMachine.Extensions |? { $_.VirtualMachineExtensionType -eq "AzureDiskEncryptionForLinux" -or $_.VirtualMachineExtensionType -eq "VMBackupForLinuxExtension" }
    $Extensions|% { Remove-AzureRmVMExtension -ResourceGroupName $ResourceGroupName -VMName $VMName -Name $_.Name } 
    Remove the following directory: 

    rm -rf /var/lib/azure_disk_encryption_*

    Extension logs:
    rm -rf /var/log/azure/Microsoft.Azure.Security.AzureDiskEncryptionForLinux*


    Once these steps were done, try to re-run the encryption on this VM/Data disk again. Let me know if this works.
    Friday, March 16, 2018 7:40 PM
  • Hi Adam

    Thanks for the reply.

    I've followed your steps, unfortunately the installation fails with the following error:

    Set-AzureRmVMDiskEncryptionExtension : Long running operation failed with status 'Failed'. Additional Info:'VM has reported a failure when processing extension 'AzureDiskEncryptionForLinux'. Error message: "Failed  to enable the extension with error: u'\xc2', stack trace: Traceback (most recent call last):

    File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/handle.py", line 689, in enable_encryption DiskEncryptionKeyFileName=extension_parameter.DiskEncryptionKeyFileName)

    File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/KeyVaultUtil.py", line 74, in create_kek_secret access_token = self.get_access_token(kv_resource_name, authorize_uri, AADClientID, AADClientCertThumbprint, AADClientSecret)

    File

    "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/KeyVaultUtil.py", line 122, in get_access_token request_content = "resource=" + urllib.quote(KeyVaultResourceName) + "&client_id=" + AADClientID + "&client_secret=" + urllib.quote(AADClientSecret) + "&grant_type=client_credentials"

    File "/usr/lib/python2.7/urllib.py", line 1288, in quote return ''.join(map(quoter, s)) KeyError: u'\xc2'
    ".' ErrorCode: VMExtensionProvisioningError ErrorMessage: VM has reported a failure when processing extension 'AzureDiskEncryptionForLinux'. Error message: "Failed to enable the extension with error: u'\xc2', stack trace: Traceback (most recent call last):

    File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/handle.py", line 689, in enable_encryption DiskEncryptionKeyFileName=extension_parameter.DiskEncryptionKeyFileName)

    File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/KeyVaultUtil.py", line 74, in create_kek_secret access_token = self.get_access_token(kv_resource_name, authorize_uri, AADClientID, AADClientCertThumbprint, AADClientSecret)

    File "/var/lib/waagent/Microsoft.Azure.Security.AzureDiskEncryptionForLinux-0.1.0.999309/main/KeyVaultUtil.py", line 122, in get_access_token request_content = "resource=" + urllib.quote(KeyVaultResourceName) + &client_id=" + AADClientID + "&client_secret=" + urllib.quote(AADClientSecret) + "&grant_type=client_credentials"

    File "/usr/lib/python2.7/urllib.py", line 1288, in quote return ''.join(map(quoter, s)) KeyError: u'\xc2'

    Thanks in advance.

    Rob

    Monday, March 19, 2018 10:18 AM
  • Hi Rob, 

    Can you shoot me an email to AzCommunity[at]microsoft.com including your subscription ID, and Attn: Adam Smith in the subject ? I'll enable a support case for this scenario since it needs an in depth troubleshooting. 

    Thanks, 
    Adam
    Tuesday, March 20, 2018 3:59 PM
  • This solution worked for me! thanks
    Thursday, June 7, 2018 7:43 PM
  • glad to hear it worked @tvidi :) 
    Thursday, June 7, 2018 8:27 PM
  • Just wanted to add that this fixed my issue on Ubuntu 18.04 LTS!
    Wednesday, August 28, 2019 3:00 PM
  • Glad it helped :) @Surfer_L
    Wednesday, August 28, 2019 4:52 PM