Re: [Qemu-devel] [PATCH for-3.2 01/11] vhost-user: define conventions for vhost-user backends

[Michael S. Tsirkin]

On Wed, Dec 19, 2018 at 12:01:59PM +0400, Marc-André Lureau wrote:
> Hi
> 
> On Wed, Dec 19, 2018 at 3:20 AM Michael S. Tsirkin <address@hidden> wrote:
> >
> > On Tue, Dec 18, 2018 at 10:35:05PM +0400, Marc-André Lureau wrote:
> > > Hi
> > >
> > > On Tue, Dec 11, 2018 at 10:56 PM Michael S. Tsirkin <address@hidden> 
> > > wrote:
> > > >
> > > > On Tue, Dec 11, 2018 at 09:29:44AM +0000, Daniel P. Berrangé wrote:
> > > > > On Tue, Dec 11, 2018 at 08:42:41AM +0100, Hoffmann, Gerd wrote:
> > > > > >   Hi,
> > > > > >
> > > > > > > Right. The main issue is that we need to make sure only
> > > > > > > in-tree devices are supported.
> > > > > >
> > > > > > Well, that is under debate right now, see:
> > > > > > https://lists.gnu.org/archive/html/qemu-devel/2018-11/msg04912.html
> > > > >
> > > > > I've previously been against the idea of external plugins for QEMU,
> > > > > however, that was when the plugin was something that would be dlopen'd
> > > > > by QEMU. That would cause our internal ABI to be exposed to 3rd 
> > > > > parties
> > > > > which is highly undesirable, even if they were open source to comply
> > > > > with the license needs.
> > > > >
> > > > > When the plugin is a completely isolated process communicating with a
> > > > > well defined protocol, it is not placing a significant burden on the
> > > > > QEMU developers' ongoing maintainence, nor has problems with license
> > > > > compliance. The main problem would come from debugging the combined
> > > > > system as the external process is essentially a black box from QEMU's
> > > > > POV. Downstream OS vendors are free to place restrictions on which
> > > > > backend processes they'd be willing to support with QEMU, and upstream
> > > > > is under no obligation to debug stuff beyond the QEMU boundary.
> > > > >
> > > > > We have already accepted that tradeoff with networking by supporting
> > > > > vhost-user and have externals impls like DPDK, so I don't see a
> > > > > compelling reason to try to restrict it for other vhost-user backends.
> > > >
> > > > OK seems to be more or less a rough concensus then.
> > > >
> > > > I wonder what's the approach wrt migration though.
> > >
> > > The series doesn't take care of migration.
> > >
> > > >
> > > > Even the compatibility story about vhost-user isn't
> > > > great, I would like to see something solid before
> > > > we allow that.
> > >
> > > To allow migration? vhost-user has partial support for migration
> > > (dirty memory tracking), and there is also "[PATCH v2 for-4.0 0/7]
> > > vhost-user-blk: Add support for backend reconnecting" - allowing the
> > > backend to store some state, if I understand correctly, which could be
> > > leveraged I guess...
> > >
> > > But I don't think we should block this series because migration isn't
> > > tackled here.
> > >
> > > thanks
> > >
> > >
> > > .
> >
> > How about blocking migration for now then?
> 
> The device here (vhost-user-input) blocks migration (unmigratable = 1)

Right. But that device is just an excersize, right?
It bothers me that next device might not remember and
we will get a mess.
Could we make it somehow that if there is no vmsd
then migration is blocked?


> >
> > We need someone to work on a solution for cross version/device
> > compatibility, vhost net/blk without that is bad enough but at least we
> > have a feature bits, for random devices it would be very very bad.
> 
> For now, if migration is somehow supported, it must be handled mostly
> by the qemu device, and the vhost-user backend must track dirty memory
> and be "stateless": *able to reconnect & resume where it left off).
> 
> A backend shouldn't declare VHOST_USER_PROTOCOL_F_LOG_SHMFD if it can't do 
> that.
> 
> When a backend will have a state to migrate, we will have to implement
> a new feature. I assume that's what you are asking.

No, I am talking about a cross version/backend migration.

And it's such a boutique problem backends do not even
understand what it's about. A way to handle it
in the frontend is needed.

We also need a way to make sure incorrect backends do not
connect to incorrect devices, and we need a mechanism
that allows cross-version migration compatibility.

There's been a long discussion about cross-version
migration for vhost-user and I think Maxime had
a plan there. Maxime could you repost the
set of issues and what the plan is maybe?


> >
> >
> > > >
> > > > Are we happy to just block live migration?
> > > > For sure that's already the case with VFIO.
> > > >
> > > >
> > > > > > > vhost-user by design
> > > > > > > is for out of tree users. It needn't be hard,
> > > > > > > maybe it's enough to just make qemu launch these processes
> > > > > > > as opposed to connecting to them on command line.
> > > > > >
> > > > > > Not sure this is a good idea, with security being one of the 
> > > > > > motivating
> > > > > > factors to move device emulation to other processes.  When libvirt
> > > > > > launches the processes it can place them in separate sandboxes ...
> > > > >
> > > > > Yep, libvirt already turns on seccomp policies which forbid QEMU from
> > > > > forking/execing anything, and we have no desire to go backwards here.
> > > > > Any external processes have to be launched by libvirt ahead of time.
> > > > >
> > > > >
> > > > > Regards,
> > > > > Daniel
> > > > > --
> > > > > |: https://berrange.com      -o-    
> > > > > https://www.flickr.com/photos/dberrange :|
> > > > > |: https://libvirt.org         -o-            
> > > > > https://fstop138.berrange.com :|
> > > > > |: https://entangle-photo.org    -o-    
> > > > > https://www.instagram.com/dberrange :|
> > > >
> > >
> > >
> > > --
> > > Marc-André Lureau
> 
> 
> 
> -- 
> Marc-André Lureau