[Qemu-devel] Exposing and calculating CPU APIC IDs (was Re: [RFC 1/3] target-i386: moving registers of vmstate from cpu_exec_init() to x86_cpu_realizefn())

[Eduardo Habkost]

On Wed, Jan 15, 2014 at 03:37:04PM +0100, Igor Mammedov wrote:
> On Wed, 15 Jan 2014 20:24:01 +0800
> Chen Fan <address@hidden> wrote:
> > On Tue, 2014-01-14 at 11:40 +0100, Igor Mammedov wrote:
> > > On Tue, 14 Jan 2014 17:27:20 +0800
> > > Chen Fan <address@hidden> wrote:
> > > 
> > > > the intend of this patch is to register cpu vmstates with apic id 
> > > > instead of cpu
> > > > index, due to the property setting of apic_id is behind the cpu 
> > > > initialization. so
> > > > we move the registers of cpu vmstate from cpu_exec_init() to 
> > > > x86_cpu_realizefn() to
> > > > let the set apicid as the parameter.
> > > > 
> > > > Signed-off-by: Chen Fan <address@hidden>
> > > > ---
> > > >  exec.c            | 5 +++++
> > > >  target-i386/cpu.c | 9 +++++++++
> > > >  2 files changed, 14 insertions(+)
> > > > 
> > > > diff --git a/exec.c b/exec.c
> > > > index 7e49e8e..9be5855 100644
> > > > --- a/exec.c
> > > > +++ b/exec.c
> > > > @@ -438,7 +438,9 @@ CPUState *qemu_get_cpu(int index)
> > > >  void cpu_exec_init(CPUArchState *env)
> > > >  {
> > > >      CPUState *cpu = ENV_GET_CPU(env);
> > > > +#if !defined(TARGET_I386)
> > > >      CPUClass *cc = CPU_GET_CLASS(cpu);
> > > > +#endif
> > > >      CPUState *some_cpu;
> > > >      int cpu_index;
> > > >  
> > > > @@ -460,6 +462,8 @@ void cpu_exec_init(CPUArchState *env)
> > > >  #if defined(CONFIG_USER_ONLY)
> > > >      cpu_list_unlock();
> > > >  #endif
> > > > +
> > > > +#if !defined(TARGET_I386)
> > > >      if (qdev_get_vmsd(DEVICE(cpu)) == NULL) {
> > > >          vmstate_register(NULL, cpu_index, &vmstate_cpu_common, cpu);
> > > >      }
> > > > @@ -472,6 +476,7 @@ void cpu_exec_init(CPUArchState *env)
> > > >      if (cc->vmsd != NULL) {
> > > >          vmstate_register(NULL, cpu_index, cc->vmsd, cpu);
> > > >      }
> > > > +#endif /* !TARGET_I386 */
> > > >  }
> > > >  
> > > >  #if defined(TARGET_HAS_ICE)
> > > > diff --git a/target-i386/cpu.c b/target-i386/cpu.c
> > > > index 967529a..dada6f6 100644
> > > > --- a/target-i386/cpu.c
> > > > +++ b/target-i386/cpu.c
> > > > @@ -2552,6 +2552,7 @@ static void x86_cpu_apic_realize(X86CPU *cpu, 
> > > > Error **errp)
> > > >  static void x86_cpu_realizefn(DeviceState *dev, Error **errp)
> > > >  {
> > > >      CPUState *cs = CPU(dev);
> > > > +    CPUClass *cc = CPU_GET_CLASS(cs);
> > > >      X86CPU *cpu = X86_CPU(dev);
> > > >      X86CPUClass *xcc = X86_CPU_GET_CLASS(dev);
> > > >      CPUX86State *env = &cpu->env;
> > > > @@ -2615,6 +2616,14 @@ static void x86_cpu_realizefn(DeviceState *dev, 
> > > > Error **errp)
> > > >      cpu_reset(cs);
> > > >  
> > > >      xcc->parent_realize(dev, &local_err);
> > > > +
> > > > +    if (qdev_get_vmsd(DEVICE(cs)) == NULL) {
> > > > +        vmstate_register(NULL, env->cpuid_apic_id, 
> > > > &vmstate_cpu_common, cs);
> > > > +    }
> > > > +
> > > > +    if (cc->vmsd != NULL) {
> > > > +        vmstate_register(NULL, env->cpuid_apic_id, cc->vmsd, cs);
> > > > +    }
> > > how about doing it in common CPUclass.realize()
> > > you can use get_arch_id() for getting CPU id, it returns cpu_index by 
> > > default
> > > and apic_id for target-i386.
> > 
> > Thanks for your kind suggestion, does this mean we can directly move
> > vmstate_register to cpu_common_realizefn()? 
> yep, that is a gist of it.
> 
> There is more to the issue with discontinuous CPUs, a lot of code still
> use cpu_index and the way it's allocated is not compatible with discontinuous
> CPUs, so this issue should be fixed as well.
> 
> Also you propose to use a raw apic id with CLI/user interface.
> I recall there were objections to it since APIC ID contains topology
> information and it's not trivial for user to get it right.
> The last idea that was discussed to fix it was not expose APIC ID to
> user but rather introduce QOM hierarchy like:
>   /machine/node/N/socket/X/core/Y/thread/Z
> and use it in user interface as a means to specify an arbitrary CPU
> and let QEMU calculate APIC ID based on this path.
> 
> But nobody took on implementing it yet.

We're taking so long to get a decent interface implemented, that part of
me is considering exposing the APIC ID directly like suggested before,
and requiring libvirt to calculate topology-aware APIC IDs[1] to
properly implement CPU hotplug (and possibly for other tasks).

Another part of me is hoping that the libvirt developers ask us to
please not do that, so I can use it as argument against exposing the
APIC IDs directly the next time we discuss this.  :)


[1] See target-i386/topology.h for references

-- 
Eduardo