[riscv64-softmmu] Clarification on ecall control and memory interactions.

[Klein Joshua Alexander Harrison]

Greetings,

I am teaching myself more about Linux and RISC-V systems using QEMU and came across something in the debug logs that I would like some clarification on.

I am emulating a RV64 virt machine running the latest Linux kernel.  The debug flags I use are exec, in_asm, int, cpu, and mmu.  The part of the logs in question is pasted below, and occurs during the riscv_clock_next_event()/time_init() methods within start_kernel (aka, the kernel boot process).

----------------

IN: 

0xffffffe0004e0f06:  4581              mv              a1,zero

0xffffffe0004e0f08:  953e              add             a0,a0,a5

0xffffffe0004e0f0a:  4601              mv              a2,zero

0xffffffe0004e0f0c:  4681              mv              a3,zero

0xffffffe0004e0f0e:  4881              mv              a7,zero

0xffffffe0004e0f10:  00000073          ecall           

Trace 0: 0x7f284c2c6ac0 [0000000000000000/ffffffe0004e0f06/0x1] 

 pc       ffffffe0004e0f06

 mhartid  0000000000000000

 mstatus  0000000000000100

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f06

 mcause   0000000000000002

 zero 0000000000000000 ra ffffffe0000944cc sp ffffffe000801cc0 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 ffffffe00086f6a0 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000009c40 a1 ffffffe00ba0da40

 a2 0000000000000000 a3 00000000028f5c29 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

riscv_raise_exception: 8

Trace 0: 0x7f284c2912c0 [0000000000000000/0000000080000488/0x3] 

 pc       0000000080000488

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp ffffffe000801cc0 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 ffffffe00086f6a0 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c291400 [0000000000000000/000000008000048c/0x3] 

 pc       000000008000048c

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp ffffffe000801cc0 gp ffffffe00085e2c0

 tp 000000008001ce00 t0 ffffffe00086f6a0 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c2915c0 [0000000000000000/0000000080000494/0x3] 

 pc       0000000080000494

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp ffffffe000801cc0 gp ffffffe00085e2c0

 tp 000000008001ce00 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c291a80 [0000000000000000/00000000800004c2/0x3] 

 pc       00000000800004c2

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp 000000008001cce8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 ffffffe00086f6a0 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c291bc0 [0000000000000000/00000000800004c6/0x3] 

 pc       00000000800004c6

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp 000000008001cce8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 ffffffe0004e0f10 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c291d80 [0000000000000000/00000000800004cc/0x3] 

 pc       00000000800004cc

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp 000000008001cce8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 0000000000747fe2 a1 0000000000000000

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c292980 [0000000000000000/0000000080000514/0x3] 

 pc       0000000080000514

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra ffffffe0000944cc sp 000000008001cce8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 ffffffe000801cd0 s1 ffffffe00ba0da40 a0 000000008001cce8 a1 000000008001ce00

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 00000000003d0900 s3 0000000000000000

 s4 0000000000000000 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Chain 0: 0x7f284c292ac0 [0000000000000000/0000000080002282/0x3] sbi_trap_handler

Chain 0: 0x7f284c004a80 [0000000000000000/00000000800052b2/0x3] sbi_current_hartid

Trace 0: 0x7f284c004c40 [0000000000000000/00000000800052bc/0x3] sbi_current_hartid

 pc       00000000800052bc

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra 000000008000229e sp 000000008001cc98 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 000000008001cca8 s1 ffffffe00ba0da40 a0 0000000000000000 a1 000000008001ce00

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 000000008001cce8 s3 0000000000000000

 s4 000000008001ce00 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Chain 0: 0x7f284c292ec0 [0000000000000000/000000008000229e/0x3] sbi_trap_handler

Trace 0: 0x7f284c293000 [0000000000000000/00000000800022a6/0x3] sbi_trap_handler

 pc       00000000800022a6

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra 000000008000229e sp 000000008001cca8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 000000008001cce8 s1 0000000000000009 a0 0000000000000000 a1 000000008001ce00

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 000000008001cce8 s3 0000000000000000

 s4 000000008001ce00 s5 0000000000000000 s6 ffffffe00085e628 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

Trace 0: 0x7f284c293140 [0000000000000000/00000000800022aa/0x3] sbi_trap_handler

 pc       00000000800022aa

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra 000000008000229e sp 000000008001cca8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 000000008001cce8 s1 0000000000000009 a0 0000000000000000 a1 000000008001ce00

 a2 0000000000000000 a3 0000000000000000 a4 0000000000000020 a5 000000000073e3a2

 a6 00000000fffedb08 a7 0000000000000000 s2 000000008001cce8 s3 0000000000000000

 s4 000000008001ce00 s5 0000000000000000 s6 0000000000000000 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

----------------

IN: sbi_trap_handler

0x0000000080002e86:  86d2              mv              a3,s4

0x0000000080002e88:  864a              mv              a2,s2

0x0000000080002e8a:  85a6              mv              a1,s1

0x0000000080002e8c:  8556              mv              a0,s5

0x0000000080002e8e:  237010ef          jal             ra,6710         # 0x800048c4

Trace 0: 0x7f284c2c6c00 [0000000000000000/0000000080002e86/0x3] sbi_trap_handler

 pc       0000000080002e86

 mhartid  0000000000000000

 mstatus  0000000000000900

 mip      0000000000000080

 mie      000000000000022a

 mideleg  0000000000000222

 medeleg  000000000000b109

 mtvec    0000000080000488

 mepc     ffffffe0004e0f10

 mcause   0000000000000009

 zero 0000000000000000 ra 000000008000229e sp 000000008001cca8 gp ffffffe00085e2c0

 tp ffffffe000807cc0 t0 0000000000000900 t1 000000003b9aca00 t2 0000000000000001

 s0 000000008001cce8 s1 0000000000000009 a0 0000000000000000 a1 000000008001ce00

 a2 000000008000a348 a3 000000008000a36c a4 0000000000000020 a5 0000000080002e86

 a6 00000000fffedb08 a7 0000000000000000 s2 000000008001cce8 s3 0000000000000009

 s4 000000008001ce00 s5 0000000000000000 s6 0000000000000000 s7 0000000000000001

 s8 ffffffe0004e0f3c s9 ffffffe000809e78 s10 0000000000000000 s11 0000000000000000

 t3 000000000001f638 t4 000000000001f638 t5 0000000000000000 t6 ffffffe00086c4f3

riscv_cpu_tlb_fill ad 800048c4 rw 2 mmu_idx 3

riscv_cpu_tlb_fill address=800048c4 ret 0 physical 00000000800048c4 prot 7

----------------

According to the logs, no instructions are being translated and then executed between the ecall in Linux and then the sbi_trap_handler (which is in OpenSBI), but I do see a lot of traces and chains.  That said, my questions are,

1. What are the "Trace 0" and "Chain 0" lines exactly?  It seems to contain the privilege mode number and PC value, but what are the other values?
2. How is the emulated ecall transferring to control to sbi_trap_handler?  Shouldn't the exception call cause the PC to be set to mtvec (0x80000488), and then instructions are executed from there on?  You can see the PC resumes at 0x80002e86 instead. 
3. How does QEMU's TLB know that sbi_trap_handler's address is physical? (the SATP register isn't visible, so is its mode supposed to be set during the ecall?  If true, does that mean its mode is set back to virtual upon a call to mret?)

Thank you and best regards,

Joshua Klein