Re: [PATCH qemu v19] spapr: Implement Open Firmware client interface

[Alexey Kardashevskiy]

On 5/17/21 09:34, BALATON Zoltan wrote:
> On Sat, 15 May 2021, BALATON Zoltan wrote:
> > On Sat, 15 May 2021, BALATON Zoltan wrote:
> > > On Thu, 22 Apr 2021, Alexey Kardashevskiy wrote:

[snip]

> > > > +/* Defined as Big Endian */
> > > > +struct prom_args {
> > > > +    uint32_t service;
> > > > +    uint32_t nargs;
> > > > +    uint32_t nret;
> > > > +    uint32_t args[10];
> > > > +} QEMU_PACKED;
> > >
> > > This #define and struct definition should probably be moved to 
> > > include/hw/ppc/vof.h as I had to copy these when trying to get VOF 
> > > running with pegasos2 and these seem to be VOF specific not spapr.
> > >
> > > I was trying to wire up VOF on pegasos2 as proof of concept but I did 
> > > not get very far as it crashed at the first move due to using mtmsrd 
> > > which does not exist on the 32 bit CPUs (G4 or G3) used by pegasos2:
> > >
> > > vof_claim virt=0x0 size=0xc38 align=0x0 => 0x0
> > > vof_claim virt=0x0 size=0x8000 align=0x8000 => 0x8000
> > > vof_claim virt=0xc00000 size=0x18fd62 align=0x0 => 0xc00000
> > > vof_claimed 0x0..0xc38 size=0xc38
> > > vof_claimed 0x8000..0x10000 size=0x8000
> > > vof_claimed 0xc00000..0xd8fd62 size=0x18fd62
> > > vof_avail 0xc38..0x8000 size=0x73c8
> > > vof_avail 0x10000..0xc00000 size=0xbf0000
> > > vof_avail 0xd8fd62..0x20000000 size=0x1f27029e
> > > via_superio_cfg: unimplemented register 0xf2
> > > via_superio_cfg: unimplemented register 0xf4
> > > via_superio_cfg: unimplemented register 0xf6
> > > via_superio_cfg: unimplemented register 0xf7
> > > invalid/unsupported opcode: 1f - 12 - 05 - 00 (7fe00164) fff00108 0
> > > ----------------
> > > IN:
> > > 0xfff00100:  3fe00000  lis      r31, 0
> > > 0xfff00104:  63ff0000  ori      r31, r31, 0
> > > 0xfff00108:  7fe00164  mtmsrd   r31
> > >
> > > ----------------
> > > IN:
> > > 0xfff00700:  807e8020  lwz      r3, -0x7fe0(r30)
> > > 0xfff00704:  4cc63182  crclr    6
> > > 0xfff00708:  4bfffd1d  bl       0xfff00424
> > >
> > > Invalid access at addr 0xFFFF8020, size 4, region '(null)', reason: 
> > > rejected
> > >
> > > Is this mtmsrd really needed? Do 64-bit Power CPUs start in 64 bit 
> > > mode? I'd expect them to be in compatibility mode unless 64 bit is 
> > > enabled but I did not check the docs. If it's needed maybe a dummy 
> > > handler has to be at 0x700 to ignore this exception if it's running 
> > > on a 32-bit CPU.
> > >
> > > By the way does vof need to be loaded at addr 0 or it could work at 
> > > the default ROM address as well? That would simplify usage if it 
> > > could run position independent.
> >
> > Just for the sake of experimenting I've patched out the mtmsrd for now 
> > from vof.bin and got a bit further to the point that it's trying to do 
> > hypercalls now but there must be something wrong with decoding the 
> > parameters as I'm getting:
> >
> > IN:
> > 0xfff00590:  393f000c  addi     r9, r31, 0xc
> > 0xfff00594:  7d234b78  mr       r3, r9
> > 0xfff00598:  4bfffbad  bl       0xfff00144
> >
> > ----------------
> > IN:
> > 0xfff00144:  7c641b78  mr       r4, r3
> > 0xfff00148:  3c600000  lis      r3, 0
> > 0xfff0014c:  6063f005  ori      r3, r3, 0xf005
> > 0xfff00150:  44000022  sc       1
> >
> > Raise exception at fff00150 => 00000008 (01)
> > hypercall r3=000000000000f005 r4=000000000000fe7c r5=0000000000000001 
> > r6=0000000000000be8 r7=0000000000000000 r8=000000000000fe78 
> > r9=000000000000fe7c r10=0000000000000001 r11=0000000000000000 
> > r12=0000000000000000 nip=fff00150
> > vof_error_unknown_service "" args=1 rets=1
> >
> > (This is the first hypercall vof does, did not check what this should 
> > be.) I've basically blindly copied spapr_h_vof_client() from 
> > spapr_vof.c but I only vagely understand what it tries to do. Since I 
> > did not have ppc64_phys_to_real() on the 32-bit PPC CPU pegasos2 is 
> > using I've just dropped that and using _args[0] as args_real but maybe 
> > that's wrong and it needs some conversion? For other hypercalls later 
> > I get same result with service being empty while args and rets change. 
> > Any idea what I could be missing or how to debug it?
> >
> > One thing I don't get is how it will find the kernel entry point to 
> > start executing? Does it have to be in the device tree somewhere or it 
> > expects a fixed address? (I could probably find out from the source 
> > but it's easier to ask.)
>
> OK, I've found that vof.bin needs to be at address 0 then hypercalls 
> work and it tries to query /chosen/qemu.boot-kernel but I get len = -1 
> for some reason. I'm adding the kernel address and size like this:
>
>     uint64_t cells[2];
>
>     cells[0] = cpu_to_be64(pm->kernel_addr);
>     cells[1] = cpu_to_be64(pm->kernel_size);
>     qemu_fdt_setprop(fdt, "/chosen", "qemu,boot-kernel",
>                      cells, sizeof(cells));
>
> which is very much like what spapr does but when vof tries to query it I 
> get:


spapr_vof_reset() also claims the kernel/initrd/VOF memory and allocates 
the stack,  vof_claim() barfs if there are overlaps.


> Raise exception at 00000150 => 00000008 (01)
> hypercall r3=000000000000f005 r4=000000000000fe6c r5=0000000000000001 
> r6=0000000000000005 r7=0000000000000bf0 r8=000000000000fe68 
> r9=000000000000fe6c r10=0000000000000001 r11=000000000000ff60 
> r12=0000000000000000 nip=00000150
> vof_getprop ph=0x5 "qemu,boot-kernel" => len=-1 []
>
> then it calls exit and the VM stops. Any idea what could be wrong with 
> the above or what to check?

Memory allocation. Another thing I saw was clang/llvm incorrectly 
initializing bss start/end for prom (very early boot) so the prim init 
code in the kernel would memset(0) wrong page and break things. Dunno.




--
Alexey