[Qemu-discuss] Preloading shared library with libVirt while running QEMU

[Probir Roy]

Hi,

I am trying to preload two shared libraries while running Qemu using
libvirt. To preload the shared libraries I have added an environment
variable in the $domain.xml file with the following command:

```
virt-xml generic --edit --confirm --qemu-commandline
'env=LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so'

```
However, while running the domain, I get the following message of failure
to preload the shared files in the log.

```
2019-07-17 17:35:22.939+0000: starting up libvirt version: 1.3.1, package:
1ubuntu10.27 (Marc Deslauriers <address@hidden> Tue, 02 Jul
2019 09:22:37 -0400), qemu version: 2.5.0 (Debian 1:2.5+dfsg-5ubuntu10.39),
hostname: probir-XPS-13-9343
LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
QEMU_AUDIO_DRV=spice
LD_PRELOAD=/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so:/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so
/usr/bin/kvm-spice -name generic -S -machine
pc-i440fx-xenial,accel=kvm,usb=off -cpu Broadwell-noTSX-IBRS -m 4096
-realtime mlock=off -smp 2,sockets=2,cores=1,threads=1 -uuid
8f755c9c-adf8-4baf-8d6a-947ebdf61ff8 -no-user-config -nodefaults -chardev
socket,id=charmonitor,path=/var/lib/libvirt/qemu/domain-generic/monitor.sock,server,nowait
-mon chardev=charmonitor,id=monitor,mode=control -rtc
base=utc,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet
-no-shutdown -global PIIX4_PM.disable_s3=1 -global PIIX4_PM.disable_s4=1
-boot strict=on -device ich9-usb-ehci1,id=usb,bus=pci.0,addr=0x6.0x7
-device
ich9-usb-uhci1,masterbus=usb.0,firstport=0,bus=pci.0,multifunction=on,addr=0x6
-device ich9-usb-uhci2,masterbus=usb.0,firstport=2,bus=pci.0,addr=0x6.0x1
-device ich9-usb-uhci3,masterbus=usb.0,firstport=4,bus=pci.0,addr=0x6.0x2
-device virtio-serial-pci,id=virtio-serial0,bus=pci.0,addr=0x5 -drive
file=/var/lib/libvirt/images/generic.img,format=qcow2,if=none,id=drive-ide0-0-0
-device
ide-hd,bus=ide.0,unit=0,drive=drive-ide0-0-0,id=ide0-0-0,bootindex=1 -drive
if=none,id=drive-ide0-0-1,readonly=on -device
ide-cd,bus=ide.0,unit=1,drive=drive-ide0-0-1,id=ide0-0-1 -netdev
tap,fd=26,id=hostnet0 -device
rtl8139,netdev=hostnet0,id=net0,mac=52:54:00:82:62:ba,bus=pci.0,addr=0x3
-chardev pty,id=charserial0 -device
isa-serial,chardev=charserial0,id=serial0 -chardev
spicevmc,id=charchannel0,name=vdagent -device
virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.spice.0
-spice
port=5900,addr=127.0.0.1,disable-ticketing,image-compression=off,seamless-migration=on
-device
qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vgamem_mb=16,bus=pci.0,addr=0x2
-device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device
hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev
spicevmc,id=charredir0,name=usbredir -device
usb-redir,chardev=charredir0,id=redir0 -chardev
spicevmc,id=charredir1,name=usbredir -device
usb-redir,chardev=charredir1,id=redir1 -device
virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on
Domain id=9 is tainted: custom-argv
ERROR: ld.so: object
'/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from
LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object
'/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so'
from LD_PRELOAD cannot be preloaded (cannot open shared object file):
ignored.
ERROR: ld.so: object
'/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so' from
LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
ERROR: ld.so: object
'/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so'
from LD_PRELOAD cannot be preloaded (cannot open shared object file):
ignored.
char device redirected to /dev/pts/21 (label charserial0)

```

I further went ahead to verify the shared library with ldd and ls -l.
Following results came out:

```
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l
/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
-rwxrwxr-x 1 probir probir 834240 Jun 13 17:53
/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd
/home/probir/Downloads/kpv_temp/temp/kvmTest/build_kvm/libperform.so
linux-vdso.so.1 =>  (0x00007ffdd97d5000)
libxed.so =>
/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/xed/kits/xed-install-base/lib/libxed.so
(0x00007f706442e000)
libmonitor.so.0 =>
/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so.0
(0x00007f706420a000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
(0x00007f7063fed000)
libbfd-2.26.1-system.so => /usr/lib/x86_64-linux-gnu/libbfd-2.26.1-system.so
(0x00007f7063ca8000)
libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6
(0x00007f7063926000)
libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f7063710000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f7063346000)
/lib64/ld-linux-x86-64.so.2 (0x00007f7064f48000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f7063142000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f7062f28000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f7062c1f000)

```
and
```
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ls -l
/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so
lrwxrwxrwx 1 probir probir 19 Mar 28 13:45
/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so
-> libmonitor.so.0.0.0
probir@probir-XPS-13-9343:~/Downloads/kpv_temp/temp/kvmTest/test/kvm$ ldd
/home/probir/Downloads/kpv_temp/temp/kvmTest/dependencies/libmonitor/installDir/lib/libmonitor.so
linux-vdso.so.1 =>  (0x00007ffc5d12b000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007ff8ae23e000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007ff8ade74000)
/lib64/ld-linux-x86-64.so.2 (0x00007ff8ae666000)
```
I cannot figure out what is causing the failure of preload. Previously, I
manually preloaded the same shared libraries for QEMU and that worked well.

-- 
Probir