[Qemu-discuss] Supported hypervisors running VMs in nested VM

[Rain Maker]

Hello list,

I am trying to setup a lab environment for a backup solution. This lab
should be able to demonstrate the interaction the software has with
certain hypervisors, such as VMWare, Hyper-V and KVM, and the VMs
therein. My main focus is currently on Hyper-V.

I've enabled nested virtualization in the kernel module. When booting
a Linux-based LiveCD, I can see the vmx flag being passed to the VM.

However, I am unable to run any virtual machines within the hypervisors.

I've done tests on a couple of versions of Windows / Hyper-V, and
these are the results. Host system is Ubuntu 15.10, KVM version is
2.3.

Windows 2008 R2
-------------------------
Installation of Hyper-V via add / remove features works.
Booting works (which, because HV is a "type 1" hypervisor, is also a
special kind of Virtual Machine). However, I am unable to boot guest
VMs within this HV Instance.

The Event Viewer in Windows shows that 2 drivers are unable to load:
'Virtualization Infrastructure' driver required by the Virtual Machine
Management service is not installed or is disabled. Check your
settings or try reinstalling the Hyper-V role.
'Virtual Machine Bus' driver required by the Virtual Machine
Management service is not installed or is disabled. Check your
settings or try reinstalling the Hyper-V role.

Device Manager shows these 2 devices, with an error state of "Windows
cannot load the device driver for this hardware. The driver may be
corrupted or missing. (Code 39)"

Windows 2012 / 2016 technical preview 3
--------------------------------------------------------
The installation via the "default" method of Add/Remove Features does
not work. Hyper-V displays the error message "A hypervisor is already
running".

This check can be skipped by using a different method of installation
(from PowerShell):
Enable-WindowsOptionalFeature –Online -FeatureName Microsoft-Hyper-V
–All -NoRestart

This results in (again) the server booting up, but being unable to run
any guest VMs. The error message is less clear then that in 2008, just
"The Virtual Machine Management Service failed to start the virtual
machine 'New Virtual Machine' because one of the Hyper-V components is
not running (Virtual machine ID
0C063B29-249A-41C8-8A5B-6D4D2E37EF7C)."
is what I could find.

Other
--------
Just to verify that "nesting" is actually working, I've also installed
a Ubuntu 15.10 VM and installed Qemu on it.
This combination CAN successfully run a VM.

I've also installed VirtualBox on one of the Windows VMs. This
VirtualBox instance is also capable of running virtual machines.
According to the icon in the bottom right, VirtualBox IS using the
hardware virtualization.

Is this a problem specific to Hyper-V? Is there a method to get
Hyper-V working including running guests? I know for a fact that
VMWare Workstation / ESX is able to run Hyper-V fully, so it should
not be completely impossible (but I dislike VMWare for different
reasons).

My Qemu command line (generated by virt-manager). Except for disks and
domain names, all are identical:

qemu-system-x86_64 -enable-kvm -name Windows_2008_R2 -S -machine
pc-i440fx-vivid,accel=kvm,usb=off -cpu
SandyBridge,+invtsc,+osxsave,+pcid,+pdcm,+xtpr,+tm2,+est,+smx,+vmx,+ds_cpl,+monitor,+dtes64,+pbe,+tm,+ht,+ss,+acpi,+ds,+vme
-m 2048 -realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid
54a8f3a3-66c2-45a5-a280-ecf7019a67fa -no-user-config -nodefaults
-chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/Windows_2008_R2.monitor,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=localtime,driftfix=slew -global kvm-pit.lost_tick_policy=discard
-no-hpet -no-shutdown -global PIIX4_PM.disable_s3=1 -global
PIIX4_PM.disable_s4=1 -boot strict=on -device
ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7 -device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive
file=/sub/kvm/Windows_2008_R2.qcow2,if=none,id=drive-ide0-0-0,format=qcow2,cache=unsafe,aio=threads
-device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1
-drive 
file=/sub/ISO/en_windows_server_2008_r2_with_sp1_x64_dvd_617601.iso,if=none,id=drive-ide0-0-1,readonly=on,format=raw
-device ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1
-netdev tap,fd=24,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:7b:d7:d2,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-device usb-tablet,id=input0 -spice
port=5903,addr=127.0.0.1,disable-ticketing,seamless-migration=on
-device 
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device
usb-redir,chardev=charredir0,id=redir0 -chardev
spicevmc,id=charredir1,name=usbredir -device
usb-redir,chardev=charredir1,id=redir1 -chardev
spicevmc,id=charredir2,name=usbredir -device
usb-redir,chardev=charredir2,id=redir2 -chardev
spicevmc,id=charredir3,name=usbredir -device
usb-redir,chardev=charredir3,id=redir3 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on

Thank you in advance for response.

Sincerely,
Roel Brook