Re: [PATCH v2 0/2] s390x: support virtio-mem-pci

[Mario Casquero]

This series has been successfully tested in s390x. Here the detailed test steps:

Boot up a VM already containing a memory object
/home/qemu/build/qemu-system-s390x \
...
-m 4G,maxmem=20G \
-object memory-backend-ram,id=mem0,size=16G,reserve=off \
...

Check the memory devices
(qemu) info memory-devices
(qemu)

Check the PCI devices in the VM
[root@localhost ~]# lspci -v
[root@localhost ~]#

Hotplug a virtio-mem-pci device and check the memory devices
(qemu) device_add virtio-mem-pci,id=vmem0,memdev=mem0,requested-size=1G
(qemu) info memory-devices
Memory device [virtio-mem]: "vmem0"
  memaddr: 0x100000000
  node: 0
  requested-size: 1073741824
  size: 1073741824
  max-size: 17179869184
  block-size: 1048576
  memdev: /objects/mem0

Check the PCI device is correct in the guest
[root@localhost ~]# lspci -v
0001:00:00.0 Unclassified device [00ff]: Red Hat, Inc. virtio-mem (rev 01)
Subsystem: Red Hat, Inc. Device 1100
Physical Slot: 00000000
Flags: bus master, fast devsel, latency 0, IOMMU group 0
Memory at 4000000000000000 (32-bit, non-prefetchable) [virtual] [size=4K]
Memory at 4001000000000000 (64-bit, prefetchable) [virtual] [size=16K]
Capabilities: [98] MSI-X: Enable+ Count=2 Masked-
Capabilities: [84] Vendor Specific Information: VirtIO: <unknown>
Capabilities: [70] Vendor Specific Information: VirtIO: Notify
Capabilities: [60] Vendor Specific Information: VirtIO: DeviceCfg
Capabilities: [50] Vendor Specific Information: VirtIO: ISR
Capabilities: [40] Vendor Specific Information: VirtIO: CommonCfg
Kernel driver in use: virtio-pci

Resize the virtio_mem device and verify the new size
(qemu) qom-set vmem0 requested-size 4G
(qemu) info memory-devices
Memory device [virtio-mem]: "vmem0"
  memaddr: 0x100000000
  node: 0
  requested-size: 4294967296
  size: 4294967296
  max-size: 17179869184
  block-size: 1048576
  memdev: /objects/mem0

Finally, try to perform a failed unplug, resize the device to 0, and
unplug it seamlessly:
(qemu) device_del vmem0
Error: virtio-mem device cannot get unplugged while some of its memory
is still plugged
(qemu) qom-set vmem0 requested-size 0
(qemu) device_del vmem0

Tested-by: Mario Casquero <mcasquer@redhat.com>

On Tue, Jan 28, 2025 at 7:57 PM David Hildenbrand <david@redhat.com> wrote:
>
> This is based-on [1], which adds MSI-X support to virtio-balloon-pci,
> but can be applied independently.
>
> Turns out it is fairly easy to get virtio-mem-pci running on s390x. We
> only have to add MSI-X support to virtio-mem-pci, and wire-up the
> (un)plugging in the machine.
>
> Tried some simple stuff (hotplug/hotunplug/resize/reboot), and all seems
> to be working as expected.
>
> The kernel in the VM needs both, CONFIG_VIRTIO_PCI and CONFIG_VIRTIO_MEM
> for it to work.
>
> [1] 20250115161425.246348-1-arbab@linux.ibm.com">https://lkml.kernel.org/r/20250115161425.246348-1-arbab@linux.ibm.com
>
> v1 -> v2:
> * There are no transitional/non_transitional devices for virtio-mem
> * Spell out removal of "return;" in second patch
>
> Cc: Eduardo Habkost <eduardo@habkost.net>
> Cc: Marcel Apfelbaum <marcel.apfelbaum@gmail.com>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: David Hildenbrand <david@redhat.com>
> Cc: Ilya Leoshkevich <iii@linux.ibm.com>
> Cc: Halil Pasic <pasic@linux.ibm.com>
> Cc: Christian Borntraeger <borntraeger@linux.ibm.com>
> Cc: Eric Farman <farman@linux.ibm.com>
> Cc: Thomas Huth <thuth@redhat.com>
> Cc: "Michael S. Tsirkin" <mst@redhat.com>
> Cc: Cornelia Huck <cohuck@redhat.com>
> Cc: Boris Fiuczynski <fiuczy@linux.ibm.com>
> Cc: Michal Privoznik <mprivozn@redhat.com>
> Cc: Mario Casquero <mcasquer@redhat.com>
>
> David Hildenbrand (2):
>   virtio-mem-pci: Allow setting nvectors, so we can use MSI-X
>   s390x/s390-virtio-ccw: Support plugging PCI-based virtio memory
>     devices
>
>  hw/core/machine.c          |  1 +
>  hw/s390x/s390-virtio-ccw.c | 20 ++++++++++++++------
>  hw/virtio/virtio-mem-pci.c | 12 ++++++++++++
>  3 files changed, 27 insertions(+), 6 deletions(-)
>
> --
> 2.48.1
>