[Qemu-discuss] performance penalty of virtio disk in QEMU

[Markus Stockhausen]

Hello,

at the moment I'm evaulating OVirt and see quite a big latency penalty 
between request on the hypervisor and the VM. Maybe someone can give
me a hint if that is normal for QEMU or if I missed something.

A few words to my setup:

Hypervisor host: 
- Fedora 19 - Kernel 3.11.10 - 200.fc19.x86_64
- QEMU: 1.4.2 - 14.fc19
- libvirt: 1.0.5.7-2.fc19
- processor: Intel(R) Core(TM) i7 CPU address@hidden

VM guest:
- Windows 2003 32 bit 
- Virtio drivers 0.1-74

Disk image storage:
- NFS Server (Ubuntu)
- Connected by IPoIB (datagram mode MTU 2044)

When I run a single threaded direct I/O read test on the hypervisor host onto 
a file served from the NFS server I get ~12000 IOPS for 1024 byte blocks. 
Remember that the files are cached in RAM on the NFS server side so we 
measure more or less the network latency. I'm more than satisfied with that. 
Using parallel threads we can get much higher values but I just focus on the
most simple case.

The same test inside a VM cannot achive more that 2200 IOPS. Simple math
determines that IO request times rise from 83 us to 450 us giving a penalty
of 370 us. That would be more than 700000 processor cycles on top of each
read request.

The difference between virtio and virtio-scsi seems to make only a difference
for large reads (beyond 256KB). So what is it all about?

Finally the command line of qemu:

/usr/bin/qemu-system-x86_64 -machine accel=kvm -name winspeed_test -S 
-machine pc-1.0,accel=kvm,usb=off -cpu Nehalem -m 2048 
-smp 2,sockets=2,cores=1,threads=1 -uuid e4d409fc-d8c7-4ee0-a2f7-8771e14914d5 
-smbios type=1,manufacturer=oVirt,product=oVirt Node,version=19-5,
serial=49434D53-0200-48D7-3000-D7483000C49E,
uuid=e4d409fc-d8c7-4ee0-a2f7-8771e14914d5 -no-user-config -nodefaults 
-chardev socket,id=charmonitor,
path=/var/lib/libvirt/qemu/winspeed_test.monitor,server,nowait 
-mon chardev=charmonitor,id=monitor,mode=control 
-rtc base=2013-12-14T17:27:14,driftfix=slew -no-shutdown 
-device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 
-device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x4 
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 
-drive if=none,id=drive-ide0-1-0,readonly=on,format=raw,serial= 
-device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 
-drive file=/rhev/data-center/mnt/10.10.30.252:_var_nas2_OVirtIB/
272ec473-6041-42ee-bd1a-732789dd18d4/images/
fe25b062-96ac-4918-8ae3-2c4375358f49/
a36b083e-2146-46b4-b199-9288633262f7,if=none,
id=drive-scsi0-0-0-0,format=raw,
serial=fe25b062-96ac-4918-8ae3-2c4375358f49,cache=none,
werror=stop,rerror=stop,aio=threads -device scsi-hd,bus=scsi0.0,
channel=0,scsi-id=0,lun=0,drive=drive-scsi0-0-0-0,id=scsi0-0-0-0,
bootindex=1 
-drive file=/rhev/data-center/mnt/10.10.30.252:_var_nas2_OVirtIB/
272ec473-6041-42ee-bd1a-732789dd18d4/images/
490452c7-b26e-48f9-b74c-e3b893f0a7f2/
67223d3e-f6c4-4d34-8d5a-9b4bb9ece6ae,if=none,
id=drive-virtio-disk0,format=raw,
serial=490452c7-b26e-48f9-b74c-e3b893f0a7f2,cache=none,
werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,
bus=pci.0,addr=0x7,drive=drive-virtio-disk0,id=virtio-disk0 
-netdev tap,fd=27,id=hostnet0 -device e1000,netdev=hostnet0,
id=net0,mac=00:0c:29:45:6a:90,bus=pci.0,addr=0x3 
-chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/
e4d409fc-d8c7-4ee0-a2f7-8771e14914d5.com.redhat.rhevm.vdsm,
server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,
chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm 
-chardev socket,id=charchannel1, path=/var/lib/libvirt/qemu/channels/
e4d409fc-d8c7-4ee0-a2f7-8771e14914d5.org.qemu.guest_agent.0,
server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,
chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 
-device usb-tablet,id=input0 -vnc 192.168.10.51:0,password 
-k en-us -vga cirrus

Thanks in advance.

Markus

InterScan_Disclaimer.txt
Description: Text document