Re: softmmu 'at' instruction support

[Peter Maydell]

On Thu, 18 Nov 2021 at 14:25, Janne Karhunen <janne.karhunen@gmail.com> wrote:
>
> On Thu, Nov 18, 2021 at 1:53 PM Janne Karhunen <janne.karhunen@gmail.com> 
> wrote:
>
> > > If you want to walk through what QEMU does and why it
> > > returns the fault indication, you can run QEMU under
> > > a debugger and put a breakpoint at ats_write64().
> > > That will do the page table walk (in get_phys_addr())
> > > and you can see where and why we decide that it should
> > > report a fault to the PAR_EL1.
> >
> > Thanks, I'll do this and post the results here. I've been walking
> > around the 'get_phys_addr_lpae' a million times but not via 'at'.
>
> Here we go. The debugger attached to the kernel says:
> (gdb) info all-registers TTBR1_EL1
> TTBR1_EL1      0x41731001          1098059777
>
> That table is populated by the kernel and looks like this when I print it:
> Table 0x41731000
> 000:0x000000ffff9803 256:0x000000fffff803 507:0x00000041fbc803
> 508:0x000000ff9ef803
>
> Note that the first gigabyte is populated. Then we move to the qemu:
> Thread 4 "qemu-system-aar" hit Breakpoint 1, ats_write64
> (env=0x80ec001fb8, ri=0x7ffff5b802d0, value=93825004025122) at
> ../target/arm/helper.c:3691
> ..
> 11610     ttbr = regime_ttbr(env, mmu_idx, param.select);
> (gdb) n
> 11616     if (param.epd) {
> (gdb) p/x ttbr
> $1 = 0x41730000
>
> So it starts looking from the wrong table (off by one page) ... ?

My guess is that your guest code and QEMU disagree about how aligned
the page table needs to be, and that extra 1 in bit 12 is being
cleared out because QEMU is implementing it as RES0.

TCR_EL1 = 0x400035b5503510
so TG1 = 0b01 (16K pages), T1SZ = 0b110101 = 53

Page tables for 16K pages need to be more aligned than those
for 4K pages. See table D5-12 in the v8A Arm ARM: for 4K
page tables TTBR_ELx[47:12] supplies the page table address,
but for 16K page tables only [47:14] do. (This assumes you've
set things up so the initial lookup level is a 3; if you've
set T1SZ so the initial lookup level is 2 then the table
might need to be more aligned.)

-- PMM