Re: [RFC PATCH 08/10] ppc/pnv: Invert the design for big-core machine modelling

[Nicholas Piggin]

On Wed May 29, 2024 at 4:57 PM AEST, Cédric Le Goater wrote:
> On 5/26/24 14:26, Nicholas Piggin wrote:
> > POWER9 and POWER10 machines come in two variants, "big-core" and
> > "small-core".
> > 
> > Big core machines are SMT8 from the software point of view, but in the
> > low level platform topology ("xscom registers and pervasive
> > addressing"), these look more like a pair of small cores ganged
> > together.
> > 
> > Presently, the way this is modelled is to create an SMT8 PnvCore and
> > add special cases to xscom and pervasive for big-core mode. This is
> > becoming too complicated to manage as more of the machine is modelled.
> > The better approach looks like the inverse, which is creating 2xPnvCore
> > ganging them together to look like an SMT8 core in TCG. The TCG SMT code
> > is quite simple to do that, and then the xscom and pervasive modelling
> > does not need to differentiate big and small core modes for the most
> > part.
> > 
> > device-tree building does need a special case to only build one
> > CPU node for each big-core because that's what the firmware expects.
> > And so does a special case workaround in the ChipTOD model.
> > 
> > A big-core machine option is added for powernv9 and 10 machines.
>
> That's another patch.

Okay.

> It is difficult to follow all the changes. I think this patch
> needs further splitting.

Sure.

> > Signed-off-by: Nicholas Piggin <npiggin@gmail.com>
> > ---
> >   include/hw/ppc/pnv.h         |   3 +
> >   include/hw/ppc/pnv_core.h    |   8 ++
> >   target/ppc/cpu.h             |   4 +-
> >   hw/ppc/pnv.c                 | 183 ++++++++++++++++++++++++++++-------
> >   hw/ppc/pnv_core.c            |  20 +++-
> >   hw/ppc/spapr_cpu_core.c      |   6 +-
> >   target/ppc/misc_helper.c     |   6 +-
> >   target/ppc/timebase_helper.c |   9 ++
> >   8 files changed, 197 insertions(+), 42 deletions(-)
> > 
> > diff --git a/include/hw/ppc/pnv.h b/include/hw/ppc/pnv.h
> > index 476b136146..93ecb062b4 100644
> > --- a/include/hw/ppc/pnv.h
> > +++ b/include/hw/ppc/pnv.h
> > @@ -100,6 +100,9 @@ struct PnvMachineState {
> >       PnvPnor      *pnor;
> >   
> >       hwaddr       fw_load_addr;
> > +
> > +    bool         big_core;
> > +    bool         big_core_tbst_quirk;
>
> I think the quirk should be introduced in its own patch.

Good idea.

> > @@ -157,6 +157,14 @@ static int pnv_dt_core(PnvChip *chip, PnvCore *pc, 
> > void *fdt)
> >   
> >       pnv_cc->processor_id(chip, pc->hwid, 0, &pir, &tir);
> >   
> > +    /* Only one DT node per (big) core */
> > +    if (tir != 0) {
> > +        g_assert(pc->big_core);
> > +        g_assert(tir == 1);
> > +        g_assert(pc->hwid & 1);
> > +        return -1;
>
> return is -1 but it's not an error. right ?

Not an error just a "no CPU node to insert".

It's a bit ugly. Could return bool for yes/no and take a *offset
maybe?

> > @@ -1088,10 +1119,37 @@ static void pnv_power8_init(MachineState *machine)
> >   
> >   static void pnv_power9_init(MachineState *machine)
> >   {
> > -    if (machine->smp.threads > 8) {
> > -        error_report("Cannot support more than 8 threads/core "
> > -                     "on a powernv9/10 machine");
> > -        exit(1);
> > +    PnvMachineState *pnv = PNV_MACHINE(machine);
> > +
> > +    if (pnv->big_core) {
>
> It would be interesting to have a max_smt machine class attribute too.

Yeah, as we talked about in the other thread. Probably helps
reduce code.

> > +        if (machine->smp.threads > 8) {
> > +            error_report("Cannot support more than 8 threads/core "
> > +                         "on a powernv9/10  machine");
> > +            exit(1);
> > +        }
> > +        if (machine->smp.threads % 2 == 1) {
>
> is_power_of_2()

It does have that check later in pnv_init(), but I wanted
to be careful that we're dividing by 2 below I think it makes
it more obvious (and big-core can't have 1 thread per big core).

> > @@ -1099,6 +1157,8 @@ static void pnv_power9_init(MachineState *machine)
> >   
> >   static void pnv_power10_init(MachineState *machine)
> >   {
> > +    PnvMachineState *pnv = PNV_MACHINE(machine);
> > +    pnv->big_core_tbst_quirk = true;
> >       pnv_power9_init(machine);
> >   }
> >   
> > @@ -1169,9 +1229,15 @@ static void pnv_processor_id_p9(PnvChip *chip,
> >                                   uint32_t core_id, uint32_t thread_id,
> >                                   uint32_t *pir, uint32_t *tir)
> >   {
> > -    if (chip->nr_threads == 8) {
> > -        *pir = (chip->chip_id << 8) | ((thread_id & 1) << 2) | (core_id << 
> > 3) |
> > -               (thread_id >> 1);
> > +    PnvMachineState *pnv = PNV_MACHINE(qdev_get_machine());
>
> arg. We should avoid these qdev_get_machine() calls. Could big_core be a
> chip property instead ?

We could, but per machine probably makes more sense. It's
funny there seems to be no good way to get machine from CPU.
Maybe we can just add a machine pointer in PnvChip?

I'l probably leave that for another series and try to convert
most things.

> > +static bool pnv_machine_get_hb(Object *obj, Error **errp)
> > +{
> > +    PnvMachineState *pnv = PNV_MACHINE(obj);
> > +
> > +    return !!pnv->fw_load_addr;
> > +}
> > +
> > +static void pnv_machine_set_hb(Object *obj, bool value, Error **errp)
> > +{
> > +    PnvMachineState *pnv = PNV_MACHINE(obj);
> > +
> > +    if (value) {
> > +        pnv->fw_load_addr = 0x8000000;
> > +    }
> > +}
>
> we might want to get rid of the hostboot mode oneday. This was really
> experimental stuff.

Okay sure, I don't use it. Although we may want to run the
open source hostboot part of the firmware on QEMU one day,
we can always add back some options for it.

We do have a PowerVM mode too which tweaks a few things, but
that's no use for upstream.

> > diff --git a/hw/ppc/spapr_cpu_core.c b/hw/ppc/spapr_cpu_core.c
> > index 059d372c8a..05195527a5 100644
> > --- a/hw/ppc/spapr_cpu_core.c
> > +++ b/hw/ppc/spapr_cpu_core.c
>
> This change should come in another patch preferably

Yeah this might have got into the wrong patch.

Thanks,
Nick