Nested Virtualization Inside Azure

Question

Hi,

This may be a dumb question but is there a way to run hypervisor or ESXI as an Azure instance.

To my understanding the CPU requires the vmx flag in order to support nesting.

I guess the real question is can you get access to that part of the CPU to enable nesting. I'm assuming you can't due to security.

Thanks in advance.

Answer 1

Hello,

Thank you for using the Microsoft Azure forums!

Normally you could install a VMware ESXi host using tools like Intel-VT but for this you would need to have access to the underlying CPU hardware. Since you do not have control over that in an Azure environment you will not be able to install a VMware ESXi host in a Azure VM.

As for installing a Microsoft hypervisor i.e. Hyper-V on a Azure VM, lets say running a Windows Server will also give you errors since the Hyper-V is designed to be installed on the underlying hardware. When this is tried on a Azure VM which is already running on a Hyper-V, the setup will identify that the underlying hardware is virtualized (i.e. a Hyper-V) and will not install all the modules although you may manage to install it.

There is a unsupported workaround for this though but even I this case you will not be able to utilize the functionality of Hyper-V since the VM will probably not boot up. You may refer to How to Install Hyper-V on a Virtual Machine in Hyper-V for this.

So in summary you do not have control of the Azure (Hosted environment) to make the changes to servers and so you will not be able to configure Nested Hypervisors or to allow Intel-VT passthrough,, and start a VM. So this would apply to any hypervisor as well. 

Hope this helps.

Regards,

Loydon

________________________________________________________________________________________________________________

When you see answers and helpful posts, please click Vote As Helpful, Propose As Answer, and/or Mark As Answer so that other customers can benefit from it.

Answer 2

Loydon,

Thank you for the clarification. 

I did come across a company called Ravello that has nesting capabilities in AWS/Google Cloud.

How is that possible? Could they possibly have full access to the hardware?

Thank you.

Answer 3

Hello,

Thank you for reverting!

As I mentioned earlier, installing an Intel-VT extension on a Azure hypervisor is not supported. That is why if you were to install another Hypervisor on top of an existing VM then the nested hypervisor would not be able to emulate the underlying virtualized hardware that the primary hypervisor is using.

As per the technology used by Ravello, it uses a different approach towards nested virtualization. Ravello is not a Cloud service provider but it provides its product called HVX which enables virtualization extensions such as Intel-VT. As a result a nested virtualization environment can be constructed on top of an existing Cloud based hypervisor.

Please have a go through the documentation for HVX here. It will give you more details. As of now this is supported on AWS and Google Cloud but not on Azure. If required you may get in touch with them to explore the possibilities (if any) with Azure.

In case you want to give a suggestion to our Azure team please feel free to visit https://feedback.azure.com/forums/216843-virtual-machines and post your feedback.

Hope this helps.

Regards,

Loydon

Disclaimer: This response contains a reference to a third party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

________________________________________________________________________________________________________________

When you see answers and helpful posts, please click Vote As Helpful, Propose As Answer, and/or Mark As Answer so that other customers can benefit from it.

Answer 4

Now with nested virtualization comming to Windows server 2016, will this feature be available to Azure VMs in the near future?
You mention that a switch need to be turned on and in Azure you dont have the possibility for this, maybe will there be such a possibility in the near future?

Answer 5

Hello,

Thank you for posting here.

With the launch of Windows Server 2016 Microsoft has integrated Docker containers feature in this version of Windows Server. So as per the open source usage of Docker it should allow you to run any Linux distros or Windows images on top of any underlying infrastructure. It is clear that Microsoft does not support Nested Virtualization in Azure using hyper-V containers. However, you can check http://searchwindowsserver.techtarget.com/tutorial/How-to-deploy-Windows-containers-in-Microsoft-Azure which describes a way to utilize the Docker containers feature introduced in Windows Azure. The Windows Server 2016 is still in TP4 so we do not have a confirmation about how this will behave. So if you are looking to utilize this and give it a try it would be interesting to know your feedback.

Regards,

Loydon

Disclaimer: This response contains a reference to a third party World Wide Web site. Microsoft is providing this information as a convenience to you. Microsoft does not control these sites and has not tested any software or information found on these sites; therefore, Microsoft cannot make any representations regarding the quality, safety, or suitability of any software or information found there. There are inherent dangers in the use of any software found on the Internet, and Microsoft cautions you to make sure that you completely understand the risk before retrieving any software from the Internet.

Answer 6

Hi Loydon,

Is there anyway to run a Windows 2003 32 bit server version on an Azure VM?  The problem that we face with my clients is that their not going to upgrade their legacy ERP anytime soon.  We, therefore, have to continue running Tuxedo 6.5, which does not boot up when running on a Windows 2008 64 bit machine.  Any suggestions?  Thanks.

Ahmed Al-Hujazi

[e-mail]

Answer 7

Hello Alhujazi,

Thank you for posting on the Azure forums.

Since this would be a new topic I have gone ahead and opened a new thread for this. Kindly refer to https://social.msdn.microsoft.com/Forums/en-US/7ce4b933-45c1-4864-bedf-69ee3e47bc1a/is-there-anyway-to-run-a-windows-2003-32-bit-server-version-on-an-azure-vm?forum=WAVirtualMachinesforWindows where we can discuss this.

I have responded in the link mentioned above. Kindly check and post your response or queries in the above thread from hereon.

Best regards,

Loydon

Answer 8

This could be quite frustrating.  I'm setting up a Visual Studio development environment on an Azure Virtual Machine (Windows 10).  This development environment won't be fully featured because the Windows Mobile emulator used within Visual Studio is based on Hyper-V.  It won't be possible for this emulator to function without nested virtualisation.

I'm aware that the following command should activate nesting on Windows 10, if run against a stopped VM:

     Set-VMProcessor -VMName <VMName> -ExposeVirtualizationExtensions $true

Is there a way of running this or achieving the same effect? It would really disappointing to be prevented from running an effective cloud-based development environment using a 100% Microsoft branded toolset :(

Thanks,

Simon

Answer 9

I echo Simons sentiments. I use the cloud for all my infrastructure development work. Not being able to use nested virtualisation within Azure really hampers the Azure IaaS offering for release/dev work, especially when other virtualisation vendors like VMware have supported nested virtualisation for years. I'm hoping this is addressed really soon.

Answer 10

Hello Simon / Shawn,

This has been an ask from our customers for a while now and we are working on getting this addressed. However, we don not have a timeline so I request you to get this through to us on the feedback forums as well. Please visit https://feedback.azure.com/forums/34192--general-feedback for writing your feedback about nested virtualization or any other improvements you would want with Azure. Your opinions and views are much appreciated and valued.

Regards,

Loydon

Answer 11

It has been months since this forum has been updated and Server 2016 is in its full production version. Is nested virtualization in Azure when running Server 2016 now possible?

Answer 12

This is still not possible in the Azure environment. 

-------------------------------------------------------------------------------------------------------------------------
Do click on "Mark as Answer" on the post that helps you, this can be beneficial to other community members.

Answer 13

Here you go, much awaited announcement is here.

Nested Virtualization in Azure

---

Please use Mark as Answer if my post solved your problem and use Vote As Helpful if a post was useful.

Answer 14

Yes, you can with the new Dv3 and Ev3 VM sizes

The Dv3 and Ev3 sizes are also some of the first VM’s to be running on Windows Server 2016 hosts.  Windows 2016 hosts enable Nested Virtualization and Hyper-V Containers for these new VM sizes.  Nested virtualization allows you to run a Hyper-V server on an Azure virtual machine. With nested virtualization you can run a Hyper-V Container in a virtualized container host, set-up a Hyper-V lab in a virtualized environment, or to test multi-machine scenarios

For details: https://azure.microsoft.com/en-us/blog/introducing-the-new-dv3-and-ev3-vm-sizes/

---

http://nicogis.blogspot.com

Answer 15

now you can deploy Dv3 and Ev3 vm with 2016 OS and install hyper-v feature and on that hyper-v you can run esxi.

rg

Answer 16

I dont see a Dv3 size. I see D3 and D2_V3

Is HyperV  available on the D2_V3?