Re: Kernel patch cases qemu live migration failed.

[Andrew Jones]

On Thu, Oct 15, 2020 at 12:06:39PM +0800, 张东旭 wrote:
> I'm so sorry for disturbing you.When I apply this kernel patch：KVM: 
> arm64/sve: System register context switch and access 
> supporthttps://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/arch/arm64/kvm/sys_regs.c?id=73433762fcaeb9d59e84d299021c6b15466c96ddqemu
>  live migration will failed with messages：              ​qemu-kvm: Invalid 
> value 233 expecting positive value <= 232              qemu-kvm: Failed to 
> load cpu:cpreg_vmstate_array_lenNew version kernel  exclude ID_AA64ZFR0_EL1 
> register when host not support SVE feature, so qemu ioctl 
> kvm(KVM_GET_REG_LIST) will not contain ID_AA64ZFR0_EL1 register.
> I'm using CentOS Linux kernel, old kernel version：4.18.0-80.11.el8 (migration 
> source)                                      new kernel 
> version：4.18.0-147.5.el8 (migration destination)
> CentOS linux kernel version 4.18.0-111.el8 applied this patch. So 
> 4.18.0-147.5.el8 also applied this patch.
> Migration source and destination hosts have the same hardware, just kernel 
> version is different, and the hardware on either side of the migration not 
> support SVE.
>  Is there some good suggestions，which can make sure old version kernel live 
> migration to new version kernel with qemu?Thanks a lot.

ARM KVM guests must be run with CPU host passthrough. This means the
host hardware and host kernel versions must be identical in order
to guarantee successful migrations.

That said, upgrading the host kernel without shutting down the guests
is a reasonable thing to try. Unfortunately, in this case, the only
way to do it would be to hack QEMU on the destination to allow this
extra register in KVM_GET_REG_LIST. It should be harmless, as it's
not used. Allowing it is similar to allowing the destination to have
a larger number of registers than the source in KVM_GET_REG_LIST (which
is why the failing test is <=, not ==).

I wouldn't post a real patch to resolve this issue, though, as everything
is working as expected. The failing test is failing because it detected
a risky migration. And, KVM's filtering of registers from KVM_GET_REG_LIST
is also correct, even if previous KVM versions didn't do that. With CPU
models you could request the reg list for a particular model and expect
it to be the same across all kernel versions that support it, but we
don't have those.

Thanks,
drew