Hi all,
Since the SMMUv3's nested translation stages[1] has been introduced by Eric, we
need to pay attention to the migration of VFIO PCI devices in SMMUv3 nested
stage
mode. At present, it is not yet supported in QEMU. There are two problems in the
existing framework.
First, the current way to get dirty pages is not applicable to nested stage
mode.
Because of the "Caching Mode", VTD can map the RAM through the host single stage
(giova->hpa). "vfio_listener_log_sync" gets dirty pages by transferring "giova"
to the kernel for the RAM block section of mapped MMIO region. In nested stage
mode, we setup the stage 2 (gpa->hpa) and the stage 1(giova->gpa) separately. So
it is inapplicable to get dirty pages by the current way in nested stage mode.
Second, it also need to pass stage 1 configurations to the destination host
after
the migration. In Eric's patch, it passes the stage 1 configuration to the host
on
each STE update for the devices set the PASID PciOps. The configuration will be
applied at physical level. But the data of physical level will not be sent to
the
destination host. So we have to pass stage 1 configurations to the destination
host after the migration.
This Patch set includes patches as below:
Patch 1-2:
- Refactor the vfio_listener_log_sync and added a new function to get dirty
pages
in nested stage mode.
Patch 3:
- Added the post_load function to vmstate_smmuv3 for passing stage 1
configuration
to the destination host after the migration.
@Eric, Could you please add this Patch set to your future version of
"vSMMUv3/pSMMUv3 2 stage VFIO integration", if you think this Patch set makes
sense? :)
Best Regards
Kunkun Jiang
[1] [RFC,v7,00/26] vSMMUv3/pSMMUv3 2 stage VFIO integration
http://patchwork.ozlabs.org/project/qemu-devel/cover/20201116181349.11908-1-eric.auger@redhat.com/
Kunkun Jiang (3):
vfio: Introduce helpers to mark dirty pages of a RAM section
vfio: Add vfio_prereg_listener_log_sync in nested stage
hw/arm/smmuv3: Post-load stage 1 configurations to the host
hw/arm/smmuv3.c | 60 +++++++++++++++++++++++++++++++++++++++++++++
hw/arm/trace-events | 1 +
hw/vfio/common.c | 47 +++++++++++++++++++++++++++++------
3 files changed, 100 insertions(+), 8 deletions(-)