Re: [Qemu-arm] [PATCH v7 03/12] target/arm: Swap PMU values before/after migrations

[Peter Maydell]

On 16 November 2018 at 16:09, Aaron Lindsay
<address@hidden> wrote:
> On Nov 16 14:53, Peter Maydell wrote:
>> On 5 November 2018 at 18:51, Aaron Lindsay <address@hidden> wrote:
>> > Because of the PMU's design, many register accesses have side effects
>> > which are inter-related, meaning that the normal method of saving CP
>> > registers can result in inconsistent state. These side-effects are
>> > largely handled in pmu_op_start/finish functions which can be called
>> > before and after the state is saved/restored. By doing this and adding
>> > raw read/write functions for the affected registers, we avoid
>> > migration-related inconsistencies.
>> >
>> > Signed-off-by: Aaron Lindsay <address@hidden>
>>
>> > --- a/target/arm/machine.c
>> > +++ b/target/arm/machine.c
>> > @@ -604,6 +604,8 @@ static int cpu_pre_save(void *opaque)
>> >  {
>> >      ARMCPU *cpu = opaque;
>> >
>> > +    pmu_op_start(&cpu->env);
>> > +
>> >      if (kvm_enabled()) {
>> >          if (!write_kvmstate_to_list(cpu)) {
>> >              /* This should never fail */
>> > @@ -625,6 +627,20 @@ static int cpu_pre_save(void *opaque)
>> >      return 0;
>> >  }
>> >
>> > +static int cpu_post_save(void *opaque)
>> > +{
>> > +    ARMCPU *cpu = opaque;
>> > +    pmu_op_finish(&cpu->env);
>> > +    return 0;
>> > +}
>> > +
>> > +static int cpu_pre_load(void *opaque)
>> > +{
>> > +    ARMCPU *cpu = opaque;
>> > +    pmu_op_start(&cpu->env);
>> > +    return 0;
>> > +}
>> > +
>> >  static int cpu_post_load(void *opaque, int version_id)
>> >  {
>> >      ARMCPU *cpu = opaque;
>> > @@ -672,6 +688,8 @@ static int cpu_post_load(void *opaque, int version_id)
>> >      hw_breakpoint_update_all(cpu);
>> >      hw_watchpoint_update_all(cpu);
>> >
>> > +    pmu_op_finish(&cpu->env);
>> > +
>> >      return 0;
>> >  }
>>
>> This will end up calling pmu_op_start() and pmu_op_finish()
>> even if the guest is running KVM and we're not using the
>> TCG code. Is that what you intended?
>
> The counters are still stored in their 'difference' format for KVM, so I
> think this still makes sense. Or is there something I'm missing about
> how this will interact with KVM? I'm much more familiar with TCG.

For KVM the counter values are stored in the kernel, until the
write_kvmstate_to_list() function (which is performed after your
pmu_op_start() call in cpu_pre_save()) writes them from the
kernel into the cpreg_vmstate array. Similarly on load they
go straight from the vmstate array into the kernel registers.
It's not clear to me what the pmu_op_start()/finish() calls
are intended to do in the KVM case, and they look at fields
in the env->cp15 struct which will not have valid values at
this point.

thanks
-- PMM