Re: [PATCH v2 0/2] hw/arm/virt: Support for virtio-mem-pci

[Jonathan Cameron]

On Fri,  3 Dec 2021 11:35:20 +0800
Gavin Shan <gshan@redhat.com> wrote:

> This series supports virtio-mem-pci device, by simply following the
> implementation on x86. The exception is the block size is 512MB on
> ARM64 instead of 128MB on x86, compatible with the memory section
> size in linux guest.
> 
> The work was done by David Hildenbrand and then Jonathan Cameron. I'm
> taking the patch and putting more efforts, which is all about testing
> to me at current stage.

Hi Gavin,

Thanks for taking this forwards.  What you have here looks good to me, but
I've not looked at this for a while, so I'll go with whatever David and
others say :)

Jonathan

> 
> Testing
> =======
> The upstream linux kernel (v5.16.rc3) is used on host/guest during
> the testing. The guest kernel includes changes to enable virtio-mem
> driver, which is simply to enable CONFIG_VIRTIO_MEM on ARM64.
> 
> Mutiple combinations like page sizes on host/guest, memory backend
> device etc are covered in the testing. Besides, migration is also
> tested. The following command lines are used for VM or virtio-mem-pci
> device hot-add. It's notable that virtio-mem-pci device hot-remove
> isn't supported, similar to what we have on x86. 
> 
>   host.pgsize  guest.pgsize  backend    hot-add  hot-remove  migration
>   ---------------------------------------------------------------------
>    4KB         4KB           normal     ok       ok          ok
>                              THP        ok       ok          ok
>                              hugeTLB    ok       ok          ok
>    4KB         64KB          normal     ok       ok          ok
>                              THP        ok       ok          ok
>                              hugeTLB    ok       ok          ok
>   64KB         4KB           normal     ok       ok          ok
>                              THP        ok       ok          ok
>                              hugeTLB    ok       ok          ok
>   64KB         64KB          normal     ok       ok          ok
>                              THP        ok       ok          ok
>                              hugeTLB    ok       ok          ok
> 
> The command lines are used for VM. When hugeTLBfs is used, all memory
> backend objects are popuated on /dev/hugepages-2048kB or
> /dev/hugepages-524288kB, depending on the host page sizes.
> 
>   /home/gavin/sandbox/qemu.main/build/qemu-system-aarch64                     
>   \
>   -accel kvm -machine virt,gic-version=host                                   
>   \
>   -cpu host -smp 4,sockets=2,cores=2,threads=1                                
>   \
>   -m 1024M,slots=16,maxmem=64G                                                
>   \
>   -object memory-backend-ram,id=mem0,size=512M                                
>   \
>   -object memory-backend-ram,id=mem1,size=512M                                
>   \
>   -numa node,nodeid=0,cpus=0-1,memdev=mem0                                    
>   \
>   -numa node,nodeid=1,cpus=2-3,memdev=mem1                                    
>   \
>      :
>   -kernel /home/gavin/sandbox/linux.guest/arch/arm64/boot/Image               
>   \
>   -initrd /home/gavin/sandbox/images/rootfs.cpio.xz                           
>   \
>   -append earlycon=pl011,mmio,0x9000000                                       
>   \
>   -device pcie-root-port,bus=pcie.0,chassis=1,id=pcie.1                       
>   \
>   -device pcie-root-port,bus=pcie.0,chassis=2,id=pcie.2                       
>   \
>   -device pcie-root-port,bus=pcie.0,chassis=3,id=pcie.3                       
>   \
>   -object memory-backend-ram,id=vmem0,size=512M                               
>   \
>   -device 
> virtio-mem-pci,id=vm0,bus=pcie.1,memdev=vmem0,node=0,requested-size=0 \
>   -object memory-backend-ram,id=vmem1,size=512M                               
>   \
>   -device 
> virtio-mem-pci,id=vm1,bus=pcie.2,memdev=vmem1,node=1,requested-size=0 
> 
> Command lines used for memory hot-add and hot-remove:
> 
>   (qemu) qom-set vm1 requested-size 512M
>   (qemu) qom-set vm1 requested-size 0
>   (qemu) qom-set vm1 requested-size 512M
> 
> Command lines used for virtio-mem-pci device hot-add:
> 
>   (qemu) object_add memory-backend-ram,id=hp-mem1,size=512M
>   (qemu) device_add virtio-mem-pci,id=hp-vm1,bus=pcie.3,memdev=hp-mem1,node=1
>   (qemu) qom-set hp-vm1 requested-size 512M
>   (qemu) qom-set hp-vm1 requested-size 0
>   (qemu) qom-set hp-vm1 requested-size 512M
> 
> Changelog
> =========
> v2:
>   * Include David/Jonathan as co-developers in the commit log           
> (David)
>   * Decrease VIRTIO_MEM_USABLE_EXTENT to 512MB on ARM64 in PATCH[1/2]   
> (David)
>   * PATCH[2/2] is added to correct the THP sizes on ARM64               
> (David)
> 
> Gavin Shan (2):
>   hw/arm/virt: Support for virtio-mem-pci
>   virtio-mem: Correct default THP size for ARM64
> 
>  hw/arm/Kconfig         |  1 +
>  hw/arm/virt.c          | 68 +++++++++++++++++++++++++++++++++++++++++-
>  hw/virtio/virtio-mem.c | 36 ++++++++++++++--------
>  3 files changed, 91 insertions(+), 14 deletions(-)
>