External email: Use caution opening links or attachments
On Mon, Sep 09, 2024 at 03:52:39PM +0300, Avihai Horon wrote:
On 05/09/2024 21:31, Peter Xu wrote:
External email: Use caution opening links or attachments
On Thu, Sep 05, 2024 at 07:45:43PM +0300, Avihai Horon wrote:
Does it also mean then that the currently reported stop-size - precopy-size
will be very close to the constant non-iterable data size?
It's not constant, while the VM is running it can change.
I wonder how heavy is VFIO_DEVICE_FEATURE_MIG_DATA_SIZE ioctl.
I just gave it a quick shot with a busy VM migrating and estimate() is
invoked only every ~100ms.
VFIO might be different, but I wonder whether we can fetch stop-size in
estimate() somehow, so it's still a pretty fast estimate() meanwhile we
avoid the rest of exact() calls (which are destined to be useless without
VFIO).
IIUC so far the estimate()/exact() was because ram sync is heavy when
exact(). When idle it's 80+ms now for 32G VM with current master (which
has a bug and I'm fixing it up [1]..), even if after the fix it's 3ms (I
think both numbers contain dirty bitmap sync for both vfio and kvm). So in
that case maybe we can still try fetching stop-size only for both
estimate() and exact(), but only sync bitmap in exact().
IIUC, the end goal is to prevent migration thread spinning uselessly in
pre-copy in such scenarios, right?
If eventually we do call get stop-copy-size in estimate(), we will move the
spinning from "exact() -> estimate() -> exact() -> estimate() ..." to
"estimate() -> estimate() -> ...".
If so, what benefit would we get from this? We only move the useless work to
other place.
We can avoid exact() calls invoked for other vmstate handlers, e.g. RAM,
which can be much heavier and can require BQL during the slow process,
which can further block more vcpu operations during migration.
And as mentioned previously, VFIO is, AFAIK, the only handler that provide
different definitions of estimate() and exact(), which can be confusing,
and it's against the "estimate() is the fast-path" logic.
But I agree it's not fundamentally changing much..
Shouldn't we directly go for the non precopy-able vs precopy-able report
that you suggested?
Yep, I just thought the previous one would be much easier to achieve.
And
as you said, VFIO is still pretty special that the user will need manual
involvements anyway to specify e.g. very large downtimes, so this condition
shouldn't be a major case to happen. Said that, if you have a solid idea
on this please feel free to go ahead directly with a complete solution.