RE: [PATCH v1 1/1] vfio: Make migration support non experimental by default.

[Tian, Kevin]

> From: Alex Williamson <alex.williamson@redhat.com>
> Sent: Tuesday, March 9, 2021 6:51 AM
> 
> [Cc +Intel]
> 
> On Mon, 8 Mar 2021 21:39:49 +0530
> Tarun Gupta <targupta@nvidia.com> wrote:
> 
> > VFIO migration support in QEMU is experimental as of now, which was
> done to
> > provide soak time and resolve concerns regarding bit-stream.
> > But, with the patches discussed in
> > https://www.mail-archive.com/qemu-
> devel@nongnu.org/msg784931.html , we have
> > corrected ordering of saving PCI config space and bit-stream.
> >
> > So, this patch proposes to make vfio migration support in QEMU to be
> enabled
> > by default. Tested by successfully migrating mdev device.
> >
> > Signed-off-by: Tarun Gupta <targupta@nvidia.com>
> > Signed-off-by: Kirti Wankhede <kwankhede@nvidia.com>
> > ---
> >  hw/vfio/pci.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/hw/vfio/pci.c b/hw/vfio/pci.c
> > index f74be78209..15e26f460b 100644
> > --- a/hw/vfio/pci.c
> > +++ b/hw/vfio/pci.c
> > @@ -3199,7 +3199,7 @@ static Property vfio_pci_dev_properties[] = {
> >      DEFINE_PROP_BIT("x-igd-opregion", VFIOPCIDevice, features,
> >                      VFIO_FEATURE_ENABLE_IGD_OPREGION_BIT, false),
> >      DEFINE_PROP_BOOL("x-enable-migration", VFIOPCIDevice,
> > -                     vbasedev.enable_migration, false),
> > +                     vbasedev.enable_migration, true),
> >      DEFINE_PROP_BOOL("x-no-mmap", VFIOPCIDevice, vbasedev.no_mmap,
> false),
> >      DEFINE_PROP_BOOL("x-balloon-allowed", VFIOPCIDevice,
> >                       vbasedev.ram_block_discard_allowed, false),
> 
> Looking back at the commit where this was added:
> 
> commit cf254988a50d4164c86a356c80b8d3ae0ccaa005
> Author: Alex Williamson <alex.williamson@redhat.com>
> Date:   Mon Nov 9 11:56:02 2020 -0700
> 
>     vfio: Make migration support experimental
> 
>     Support for migration of vfio devices is still in flux.  Developers
>     are attempting to add support for new devices and new architectures,
>     but none are yet readily available for validation.  We have concerns
>     whether we're transferring device resources at the right point in the
>     migration, whether we're guaranteeing that updates during pre-copy are
>     migrated, and whether we can provide bit-stream compatibility should
>     any of this change.  Even the question of whether devices should
>     participate in dirty page tracking during pre-copy seems contentious.
>     In short, migration support has not had enough soak time and it feels
>     premature to mark it as supported.
> 
>     Create an experimental option such that we can continue to develop.
> 
>     [Retaining previous acks/reviews for a previously identical code
>      change with different specifics in the commit log.]
> 
>     Reviewed-by: Dr. David Alan Gilbert <dgilbert@redhat.com>
>     Acked-by: Cornelia Huck <cohuck@redhat.com>
>     Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
> 
> 
> What has tangibly changed since then?  I think we have patches on-list
> to address the known issue of PCI config space (MSI) ordering, which
> related to enabling migration on ARM platforms.  Do we have
> significantly more confidence in our ability to make compatible
> enhancement to the migration bitstream?  This was a particularly
> troublesome point for me if we have any hope of calling this
> supportable.  As far as I know, there are still no open source vendor
> drivers supporting migration for community testing.  We're also still
> missing the documentation that was promised previously, as Connie noted.
> 
> Huawei and Intel, what's your confidence level and what can you share
> regarding support for this implementation?  Thanks,
> 

Internally our GVT-g live migration support is still experimental, and due
to resource/priority adjustment the upstreaming plan for this feature is
currently on hold. Timing-wise I'd expect IDXD will be the 1st Intel driver
which formally supports live migration (after its core functionalities - mdev/
vSVA are upstreamed). Alternatively once the vfio-pci-core library work 
is completed I believe many interests will be also arose regarding to VF
live migration (e.g. NIC). But none of the options may come in short term... 

Thanks
Kevin