Re: [PATCH v4] introduce vfio-user protocol specification

[Felipe Franciosi]

> On Sep 30, 2020, at 3:24 PM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
> 
> On Tue, Sep 29, 2020 at 09:21:54AM -0700, John G Johnson wrote:
>>> On Sep 29, 2020, at 3:37 AM, Stefan Hajnoczi <stefanha@redhat.com> wrote:
>>> 
>>> On Mon, Sep 28, 2020 at 09:58:37AM +0000, Thanos Makatos wrote:
>>>>> It should be accompanied by a test in tests/. PCI-level testing APIS for
>>>>> BARs, configuration space, interrupts, etc are available in
>>>>> tests/qtest/libqos/pci.h. The test case needs to include a vfio-user
>>>>> device backend interact with QEMU's vfio-user-pci implementation.
>>>> 
>>>> We plan to use a libmuser-based backend for testing. This, I suppose, will
>>>> make libmuser a dependency of QEMU (either as a submodule or as a library),
>>>> which for now can be disabled in the default configuration. Is this 
>>>> acceptable?
>>> 
>>> If there are no other dependencies and libmuser supports all host
>>> operating systems that QEMU's -device vfio-user supports, then I think
>>> it's a good idea to use libmuser for at least one in-tree test in QEMU.
>>> 
>>>>> Also please let us know who is working on what so additional people can
>>>>> get involved in areas that need work!
>>>> 
>>>> Swapnil and I will be working on libmuser and the test in QEMU, John and
>>>> the mp-qemu folks will be working on the patches for implementing
>>>> --device vfio-user-pci.
>>> 
>>> Great!
>>> 
>>> John: Will mpqemu use libmuser to implement the remote PCI host
>>> controller?
>>> 
>> 
>> 
>>      The vfio-user-pci plan is to use libmuser on the server side.
> 
> Okay. Using libmuser in tests seems like a good choice in that case.
> 
> We'll need to figure out the details of how to do it because the
> traditional shared library dependency approach is not well-suited to
> in-development code. It would involve shipping libmuser distro packages
> so QEMU's build system can declare a library dependency (with details
> provided in a pkg-config file).
> 
> Here are approaches that are better for in-development libraries:
> 1. Keep the libmuser code in qemu.git.
> 2. A copy of libmuser in qemu.git with changes being sent upstream
>   (allows more flexibility in case QEMU-specific issues require
>   experimentation).
> 3. Git submodules.
> 
> #1 if you're happy to use the QEMU development process for merging
> libmuser code then it's easiest to officially host the code in qemu.git.
> libmuser gets a subdirectory in the qemu.git tree and you (the
> maintainers) send pull requests. A libmuser library build target
> provides installable static and shared libraries so external
> applications can link against libmuser too. The big advantage here is
> that QEMU can instantly use the latest libmuser code changes.

I think there's a couple of limitations here which we should keep in mind.

1. Does putting it in qemu.git precludes it being BSD-3?
There's been evidence of people using (or at least trying out) muser
from where it currently lives. That doesn't mean we can't move it, but
I'm wondering if it means we have to make it GPL.

2. What about other projects that need libmuser code?
What worries me more is projects like SPDK/DPDK wanting to link
against the library and having to clone the entire QEMU repo as a
submodule. That sounds a lot more expensive than option 3 and probably
have further complications if they aren't GPL.

> 
> #2 works best if the library is a small (just a few source files) with
> no fancy build system requirements.

The risk here is that they go out of sync. There's the same (or even
more) maintenance burden as point 3 below, with the added risk that
someone could patch the files and make cherry-picks non-trivial.

> 
> #3 is used in QEMU for several other components. Submodules are a pain
> to sync (requires sending a qemu.git patch to move to a new commit ID),
> so this isn't good for a dependency that moves quickly.

I argue this is no worse than option 2. It's what I think aligns best,
but let's keep weighing pros/cons and come to a conclusion together.
The list of maintainers for muser.git should be extended to include
more QEMU stakeholders and probably other projects that will use it
(as) heavily. The topic has been raised in SPDK's Slack team on
whether the client library should live in a repo of its own (eg.
libvfio-user.git). Given the reference implementation is in libmuser,
I still think muser.git is accurate (but can easily be persuaded
otherwise).

Cheers,
Felipe

> 
> Stefan