Re: [RFC v4 PATCH 00/49] Initial support of multi-process qemu - status update

[Dr. David Alan Gilbert]

* John G Johnson (address@hidden) wrote:

<snip>

> DMA
> 
>       This is one place where I might diverge from VFIO.  It uses an
> ioctl to tell the kernel driver what areas of guest memory the device
> can address.  The driver then pins that memory so it can be programmed
> into a HW IOMMU.  We could avoid pinning of guest memory by adopting
> the vhost-user idea of sending the file descriptors used by QEMU to
> create guest memory to the emulation process, and having it mmap() the
> guest itself.  IOMMUs are handled by having the emulation process
> request device DMA to guest PA translations from QEMU.

The interface in vhost-user to pass these memory fd's is a bit hairy;
so it would be great if there was something better for multi-process.

Some things to think about:
  a) vhost-user filters it so that areas of memory not backed by an fd
aren't passed to the client; this filters out some of the device
specific RAM blocks that aren't really normal RAM.

  b) Hugepages are tricky; especially on a PC where the 0-1MB area is
broken up into chunks and you're trying to mmap 2MB chunks into the
client.

  c) Postcopy with vhost-user was pretty tricky as well; there needs
to be some coordination with the qemu to handle pages that are missing.

  d) Some RAM mappings can change; mostly not the ones sent to the
client; but just watch out that these can happen at unexpected times.

Dave


> 
> 
> > If implementations can use the kernel uapi vfio header files then we're
> > on track for compatibility with VFIO.
> > 
> >>> This is just a more elaborate explanation for the "the cat is out of the
> >>> bag" comments that have already been made on licensing.  Does anyone
> >>> still disagree or want to discuss further?
> >>> 
> >>> If there is agreement that a stable API is okay then I think the
> >>> practical way to do this is to first merge a cleaned-up version of
> >>> multi-process QEMU as an unstable experimental API.  Once it's being
> >>> tested and used we can write a protocol specification and publish it as
> >>> a stable interface when the spec has addressed most use cases.
> >>> 
> >>> Does this sound good?
> >> 
> >> In that case, wouldn't it be preferable to revive our proposal from
> >> Edinburgh (KVM Forum 2018)? Our prototypes moved more of the Qemu VFIO
> >> code to "common" and added a "user" backend underneath it, similar to
> >> how vhost-user-scsi moved some of vhost-scsi to vhost-scsi-common and
> >> added vhost-user-scsi. It was centric on PCI, but it doesn't have to
> >> be. The other side can be implemented in libmuser for facilitating things.
> > 
> > That sounds good.
> > 
> 
>        The emulation program API could be based on the current
> libmuser API or the libvfio-user API.  The protocol itself wouldn’t
> care which is chosen.  Our multi-processQEMU project would have to
> change how devices are specified from the QEMU command line to the
> emulation process command line.
> 
>                                                       JJ
> 
--
Dr. David Alan Gilbert / address@hidden / Manchester, UK