X86: Abnormal variation in Freebsd VM launch time w.r.t freebsd guest co

Hi Team,

We are observing abnormal variation in VM launch time w.r.t guest config.

A simple VM(2gb ram no passthrough device) creation takes usually 6sec (Time from execution of 'virsh create guest.xml to get print "Welcome to FreeBSD"' ),but when we add a USB passthrough device launch time increased to 18-19 sec and further increased to 39-44sec when we have increased guest ram to 48Gb.

The increase in time is too high than excepted. Hence this is marked as performance blocker in our test.

Here is the table summarises our experiments .

--------------------------------------------------------------------------------------------|---------------------|--------------------

Sl | Case | time taken to launch vm

--------------------------------------------------------------------------------------------|---------------------|--------------------

1. | Freebsd 11 with 2gb guest ram and no pci pass through | 6-7 sec

2. | Freebsd 11 with 48gb guest ram and no pci pass through | 6-7 sec

3. | Freebsd 11 with 48gb guest ram and usb pci pass-through | 39-44 sec

4. | Freebsd 11 with 2gb guest ram and usb pci pass-through | 18-19 sec

Attaching guest config:

1. vm_2gb_usbpassthrough.xml --- Feebsd with 2gb RAM and USB as pci pass through device

2. vm_48gb_usbpassthrough.xml --- Feebsd with 48gb RAM and USB as pci pass through device

3. vm_48gb_nopci.xml --- Feebsd with 48gb RAM and no pci device passed

Note : 1) This we have seen with legacy pci passthrough device not with vfio. And we have not noticed any perfoance impact other than qemu-init/vm laucn time .

2). We are using "FreeBSD-11.2-RELEASE-amd64.qcow2"(changed to raw format) image downloaded from https://download.freebsd.org/ftp/releases/VM-IMAGES/11.2-RELEASE/amd64/Latest/

-> Then modified the settings to set console to serial refer to the link -> https://www.freebsd.org/doc/handbook/serialconsole-setup.html

-> Changed the /boot/default/loader.conf

add the console="comconsole"

and then change the /etc/ttys

USB hub device details :

------------------------------

root@root-node:/vm/vm# lspci -v -s 00:14.0

00:14.0 USB controller: Intel Corporation C610/X99 series chipset USB xHCI Host Controller (rev 05) (prog-if 30 [XHCI])

Subsystem: Intel Corporation C610/X99 series chipset USB xHCI Host Controller

Flags: medium devsel, IRQ 16, NUMA node 0

Memory at 183fc0000000 (64-bit, non-prefetchable) [size=64K]

Capabilities: [70] Power Management version 2

Kernel driver in use: pci-stub

root@root-node:/vm/vm#

root@root-node:/vm/vm# virsh version

Compiled against library: libvirt 1.3.5

Using library: libvirt 1.3.5

Using API: QEMU 1.3.5

Running hypervisor: QEMU 2.7.0

root@root-node:/vm/vm# uname -a

Linux root-node 4.8.28-rt10-WR9.0.0.24_ovp #1 SMP Wed Feb 19 12:47:32 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

root@root-node:/vm/vm#

Host CPU details :

CPU model : Haswell E

microcode : 0x3d

cpu cores : 6

Have you ever faced similar issues ? is it a qemu bug ?

Any suggestions to improve launch time with legacy passthrough ?

Regards,

Gokul

From:	gokul cg
Subject:	X86: Abnormal variation in Freebsd VM launch time w.r.t freebsd guest config
Date:	Tue, 7 Apr 2020 09:59:59 +0530