How to enable Nested Virtualization in KVM on CentOS 7 / RHEL 7

Nested virtualization means to configure virtualization environment inside a virtual machine. In other words we can say nested virtualization is a feature in the hypervisor which allows us to install & run a virtual machine inside a virtual server via hardware acceleration from the hypervisor (host).

In this article, we will discuss how to enable nested virtualization in KVM on CentOS 7 / RHEL 7. I am assuming you have already configured KVM hypervisor. In case you have not familiar on how to install and configure KVM hypervisor, then refer the following article

Install KVM Hypervisor on CentOS 7.x and RHEL 7.x

Let’s jump into the hypervisor and verify whether nested virtualization is enabled or not on your KVM host

For Intel based Processors run the command,

[[email protected] ~]# cat /sys/module/kvm_intel/parameters/nested
N
[[email protected] ~]#

For AMD based Processors run the command,

[[email protected] ~]# cat /sys/module/kvm_amd/parameters/nested
N
[[email protected] ~]#

In the above command output ‘N’ indicates that Nested virtualization is disabled. If we get the output as ‘Y’ then it indicates that nested virtualization is enabled on your host.

Now to enable nested virtualization, create a file with the name “/etc/modprobe.d/kvm-nested.conf” with the following content.

[[email protected] ~]# vi /etc/modprobe.d/kvm-nested.conf
options kvm-intel nested=1
options kvm-intel enable_shadow_vmcs=1
options kvm-intel enable_apicv=1
options kvm-intel ept=1

Save & exit the file

Now remove ‘kvm_intel‘ module and then add the same module with modprobe command. Before removing the module, make sure VMs are shutdown otherwise we will get error message like “modprobe: FATAL: Module kvm_intel is in use

[[email protected] ~]# modprobe -r kvm_intel
[[email protected] ~]# modprobe -a kvm_intel
[[email protected] ~]#

Now verify whether nested virtualization feature enabled or not.

[[email protected] ~]# cat /sys/module/kvm_intel/parameters/nested
Y
[[email protected] ~]#

For AMD based systems, run the below commands,

[[email protected] ~]# rmmod kvm-amd
[[email protected] ~]# echo 'options kvm-amd nested=1' >> /etc/modprobe.d/dist.conf
[[email protected] ~]# modprobe kvm-amd

Test  Nested Virtualization

Let’s suppose we have a VM with name “director” on KVM hypervisor on which I have enabled nested virtualization. Before testing, make sure CPU mode for the VM is either as “host-model” or “host-passthrough” , to check cpu mode of a virtual machine use either Virt-Manager GUI or virsh edit command

cpu_mode_vm_kvm

Now login to the director VM and run lscpu and lsmod command

[[email protected] ~]# ssh 192.168.126.1 -l root
[email protected]'s password:
Last login: Sun Dec 10 07:05:59 2017 from 192.168.126.254
[[email protected] ~]# lsmod | grep kvm
kvm_intel             170200  0
kvm                   566604  1 kvm_intel
irqbypass              13503  1 kvm
[[email protected] ~]#
[[email protected] ~]# lscpu

lscpu_command_rhel7_centos7

Let’s try creating a virtual machine either from virtual manager GUI or virt-install inside the director vm, in my case i am using virt-install command

[[email protected] ~]# virt-install  -n Nested-VM  --description "Test Nested VM"  --os-type=Linux  --os-variant=rhel7  --ram=2048  --vcpus=2  --disk path=/var/lib/libvirt/images/nestedvm.img,bus=virtio,size=10  --graphics none  --location /var/lib/libvirt/images/CentOS-7-x86_64-DVD-1511.iso --extra-args console=ttyS0
Starting install...
Retrieving file .treeinfo...                                                   | 1.1 kB  00:00:00
Retrieving file vmlinuz...                                                     | 4.9 MB  00:00:00
Retrieving file initrd.img...                                                  |  37 MB  00:00:00
Allocating 'nestedvm.img'                                                      |  10 GB  00:00:00
Connected to domain Nested-VM
Escape character is ^]
[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Initializing cgroup subsys cpuacct
[    0.000000] Linux version 3.10.0-327.el7.x86_64 ([email protected]) (gcc version 4.8.3 20140911 (Red Hat 4.8.3-9) (GCC) ) #1 SMP Thu Nov 19 22:10:57 UTC 2015
………………………………………………

cli-installer-virt-install-command-kvm

This confirms that nested virtualization has been enabled successfully as we are able to create virtual machine inside a virtual machine.

This Concludes the article, please do share your feedback and comments.

5 Responses

  1. Andrew Carlson says:

    It was an interesting article, but I am a bit dismayed about AMD. You specifically called out AMD in the checking phase, but only showed intel on the setup phase. Would have been nice to have the AMD side too.

  2. K L says:

    Thank you for this post! Didn’t even realize that nested virtualization was a parameter until I found this!

  3. Will Osorio says:

    Hi Pradeep, excellent post !!
    Just a question, does it work on Cloud instances (AWS, GCP) ?

    Thx

  4. Erik says:

    Hi,
    Pretty good article, can you explain why the additional setting for the kvm_intel module have to be set, shouldn’t “nested=1” be enough?
    Thanks,
    Erik

Leave a Reply

Your email address will not be published. Required fields are marked *

Pin It on Pinterest