[Qemu-discuss] Enable 64-bit BAR

[Greg Santino]

 Hello,
I have a PCIe card in my hypervisor machine and it is passed through to my VM 
(both VM and hypervisor are running CentOS 7.3 x86_64).  I've installed 
corresponding driver for the PCIe card as well as our software that utilizes 
the driver.  Our software requires the PCIe card to be able to access a 12GB 
buffer.  We run our software and it eventually crashes and causes the server to 
reboot.  We can only set the driver to use 2GB buffers.  Running lspci, the 
PCIe device uses a 64-bit BAR, but it seems like the PCIe controller or OS 
cannot properly conduct MMIO for address ranges above 4GB?  Everything works 
using 12GB buffer when on bare-metal.  Is there a setting in QEMU that can 
address the issue?
I am using qemu-rhev 2.10.0-21 (i've tried 2.6 and 2.9 as well)Here is the qemu 
command I use for my VM.  The PCIe device is the parameter: -device 
vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x7
/usr/libexec/qemu-kvm -name guest-myhost-1,debug-threads=on -S -object 
secret,id=masterKey0,format=raw,file=/var/lib/libvirt/qemu/domain-9-myhost-1/master-key.aes
 -machine pc-i440fx-rhel7.0.0,accel=kvm,usb=off,dump-guest-core=off -cpu host 
-m 196608 -realtime mlock=off -smp 22,sockets=22,cores=1,threads=1 -uuid 
1f44c0ca-fc67-487e-90c2-a283979e86e8 -no-user-config -nodefaults -chardev 
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-9-myhost-1/monitor.sock,server,nowait
 -mon chardev=charmonitor,id=monitor,mode=control -rtc base=utc,driftfix=slew 
-global kvm-pit.lost_tick_policy=delay -no-hpet -no-shutdown -global 
PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1 -boot menu=on,strict=on 
-device ich9-usb-uhci2,masterbus-usb.0,firstport=2,bus=pci.0,addr=0x6.0x1 
-device ich9-usb-uhci3,masterbus-usb=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2 
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr-0x5 -drive 
file-/var/lib/libvirt/images/vm1/myhost-1-disk0,format=raw,if-none,id=drive-ide0-0-0
 -device ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 
-netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=32 -device 
virtio-net-pci,netdev=hostnet0,id=net0,mac=54:e4:76:ff:2f:72,bus=pci.0,addr=0x3 
-netdev tap,fd=33,host=hostnet1,vhost=on,vhostfd=34 -device 
virtio-net-pci,netdev=hostnet0,id=net1,mac=54:e4:76:ff:2f:84,bus=pci.0,addr=0x4 
-chardev pty,id=charserial0 -device isa-serial,chardev=charserial0,id=serial0 
-chardev 
socket,id=charchannel0,path=/var/lib/libvirt/qemu/channel/target/domain-9-myhost-1/org.qemu.guest_agent.0,server,nowait
 -device 
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=org.qemu.guest_agent.0
 -device usb-tablet,id=input0,bus=usb.0,port=1 -vnc 127.0.0.1:0 -device 
cirrus-vga,id-video0,bus=pci.0,addr=0x2 -device 
vfio-pci,host=09:00.0,id=hostdev0,bus=pci.0,addr=0x7 -device 
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x8 -msg timestamp=on

Thank you for your time.