[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [PATCH 01/18] vfio/migration: Add VFIO migration pre-copy support
From: |
Jason Gunthorpe |
Subject: |
Re: [PATCH 01/18] vfio/migration: Add VFIO migration pre-copy support |
Date: |
Tue, 31 Jan 2023 19:29:48 -0400 |
On Tue, Jan 31, 2023 at 03:43:01PM -0700, Alex Williamson wrote:
> How does this affect our path towards supported migration? I'm
> thinking about a user experience where QEMU supports migration if
> device A OR device B are attached, but not devices A and B attached to
> the same VM. We might have a device C where QEMU supports migration
> with B AND C, but not A AND C, nor A AND B AND C. This would be the
> case if device B and device C both supported P2P states, but device A
> did not. The user has no observability of this feature, so all of this
> looks effectively random to the user.
I think qemu should just log if it encounters a device without P2P
support.
> Even in the single device case, we need to make an assumption that a
> device that does not support P2P migration states (or when QEMU doesn't
> make use of P2P states) cannot be a DMA target, or otherwise have its
> MMIO space accessed while in a STOP state. Can we guarantee that when
> other devices have not yet transitioned to STOP?
You mean the software devices created by qemu?
> We could disable the direct map MemoryRegions when we move to a STOP
> state, which would give QEMU visibility to those accesses, but besides
> pulling an abort should such an access occur, could we queue them in
> software, add them to the migration stream, and replay them after the
> device moves to the RUNNING state? We'd need to account for the lack of
> RESUMING_P2P states as well, trapping and queue accesses from devices
> already RUNNING to those still in RESUMING (not _P2P).
I think any internal SW devices should just fail all accesses to the
P2P space, all the time.
qemu simply acts like a real system that doesn't support P2P.
IMHO this is generally the way forward to do multi-device as well,
remove the MMIO from all the address maps: VFIO, SW access, all of
them. Nothing can touch MMIO except for the vCPU.
> This all looks complicated. Is it better to start with requiring P2P
> state support? Thanks,
People have built HW without it, so I don't see this as so good..
Jason
[PATCH 03/18] vfio/common: Fix wrong %m usages, Avihai Horon, 2023/01/26
[PATCH 04/18] vfio/common: Abort migration if dirty log start/stop/sync fails, Avihai Horon, 2023/01/26
[PATCH 07/18] util: Extend iova_tree_foreach() to take data argument, Avihai Horon, 2023/01/26
[PATCH 05/18] vfio/common: Add VFIOBitmap and (de)alloc functions, Avihai Horon, 2023/01/26
[PATCH 08/18] vfio/common: Record DMA mapped IOVA ranges, Avihai Horon, 2023/01/26
[PATCH 09/18] vfio/common: Add device dirty page tracking start/stop, Avihai Horon, 2023/01/26
[PATCH 06/18] util: Add iova_tree_nnodes(), Avihai Horon, 2023/01/26