Re: [Qemu-discuss] Fwd: Possible Qemu inconsistency when emulating TLB for rpi3 machine

[Peter Maydell]

On Mon, 4 Feb 2019 at 18:19, Mattia Maldini <address@hidden> wrote:
> The problem was that after loading the new table with a different ASID to
> TTBR0 all the entries in the TLB were invalidated (either discarded by Qemu
> or ignored because of the different ASID); before I could launch the process
> at EL0 a new page was loaded from the EL0 table *but* while still at EL1
> -> permission abort.
> This happened because both my kernel and process code are in the same
> memory range. I solved it (now working both on Qemu and Raspberry) by
> changing the loading function address to use TTBR1 instead.
>
> Now, however, I'm questioning why it *did* work on Raspberry Pi... the same
> rules should apply, maybe I'm somehow disabling the TLB?
> Oh well, questions for another day.

The architecture doesn't have the concept of separate page tables
for EL0 and EL1. There is a single translation regime for "EL1&0",
which has only one set of page tables. You can't change TTBR0
to be a set of page tables that work for your user process code
while still expecting it to work for the kernel code.

My guess is that your old code was working by accident on the rpi
because you weren't doing the necessary TLB maintenance operations
for the change of TTBR0. (Changing TTBR0 requires that you then do
an ISB followed by a TLB invalidation of all entries that are
affected by the change.) So in the rpi hw implementation stale
TLB entries happen to be lying around in the TLB and the code
execution hits them and continues OK. This sort of "works on
some implementations but not others" is exactly the sort of bug
you can see if you don't follow the TLB maintenance requirements
set out by the architecture.

Anyway, glad you have managed to get your code working -- making
the kernel use a distinct part of the address space from the
user processes is indeed the best way to approach this. (Compare
Linux, which puts the kernel in the top half of memory and user
processes in the bottom half.)

thanks
-- PMM