Re: [PATCH v3 12/17] vfio/migration: Implement VFIO migration protocol v2

[Jason Gunthorpe]

On Mon, Nov 28, 2022 at 01:36:30PM -0700, Alex Williamson wrote:
> On Mon, 28 Nov 2022 15:40:23 -0400
> Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> > On Mon, Nov 28, 2022 at 11:50:03AM -0700, Alex Williamson wrote:
> > 
> > > There's a claim here about added complexity that I'm not really seeing.
> > > It looks like we simply make an ioctl call here and scale our buffer
> > > based on the minimum of the returned device estimate or our upper
> > > bound.  
> > 
> > I'm not keen on this, for something like mlx5 that has a small precopy
> > size and large post-copy size it risks running with an under allocated
> > buffer, which is harmful to performance.
> 
> I'm trying to weed out whether there are device assumptions in the
> implementation, seems like maybe we found one.  

I don't think there are assumptions. Any correct kernel driver should
be able to do this transfer out of the FD byte-at-a-time.

This buffer size is just a random selection for now until we get
multi-fd and can sit down, benchmark and optimize this properly.

The ideal realization of this has no buffer at all.

> MIG_DATA_SIZE specifies that it's an estimated data size for
> stop-copy, so shouldn't that provide the buffer size you're looking
> for? 

Yes, it should, and it should be OK for mlx5

Jason