Re: [Qemu-discuss] Debugging qemu inside chroot

[Paulo Matos]

On 21/03/2019 09:15, Peter Maydell wrote:
>> Then I attempted to run gdb outside of the chroot like:
>> gdb --args qemu-3.1.0-install/bin/qemu-aarch64
>> aarch64-chroot/racket/racket/bin/racket3m ...
> 
> If you want to run x86 gdb on QEMU (to debug/get backtraces
> for QEMU itself) you can do something like:
> 
> gdb --args chroot aarch64-chroot /usr/bin/qemu-aarch64-static
> path/to/racket3m ...
> 
> gdb will of course start out by debugging the host chroot binary,
> but you can use 'break main' which should cause gdb to ask
> "Make breakpoint pending on future shared library load?" -- if you
> say yes, then continue, it will break when chroot execs QEMU
> and we enter QEMU's main program. (Won't work if you have debug
> symbols for chroot for some reason -- if so then use a bp on
> some function that's only in QEMU and not in chroot.)
> 

Thanks for your help.
At this point I get this from debugging qemu-aarch64-static:
process 4399 is executing new program:
/root/aarch64-chroot/usr/bin/qemu-aarch64-static
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff7ff9700 (LWP 4403)]

Thread 1 "mksystem.rkt" received signal SIGSEGV, Segmentation fault.
0x0000000060b6f9a9 in static_code_gen_buffer ()
(gdb) bt
#0  0x0000000060b6f9a9 in static_code_gen_buffer ()
#1  0x000000006004abc4 in cpu_tb_exec (cpu=0x6296ee30,
    itb=0x60b6f8c0 <static_code_gen_buffer+2357056>)
    at /root/qemu-3.1.0/accel/tcg/cpu-exec.c:171
#2  0x000000006004b807 in cpu_loop_exec_tb (cpu=0x6296ee30,
    tb=0x60b6f8c0 <static_code_gen_buffer+2357056>, last_tb=0x7fffffffdbd8,
    tb_exit=0x7fffffffdbd0) at /root/qemu-3.1.0/accel/tcg/cpu-exec.c:615
#3  0x000000006004baa0 in cpu_exec (cpu=0x6296ee30)
    at /root/qemu-3.1.0/accel/tcg/cpu-exec.c:725
#4  0x0000000060084eef in cpu_loop (env=0x629770e0)
    at /root/qemu-3.1.0/linux-user/aarch64/cpu_loop.c:82
#5  0x000000006005a042 in main (argc=10, argv=0x7fffffffe508,
envp=0x7fffffffe560)
    at /root/qemu-3.1.0/linux-user/main.c:819

This smells like something is going on in qemu.
I, however, have no knowledge of qemu internals. Is there some sort of
manual or reference to read about the internals or guide on how to debug
this sort of thing?

I saw in cpu_tb_exec:
#if defined(DEBUG_DISAS)
    if (qemu_loglevel_mask(CPU_LOG_TB_CPU)

so I looked for logging and tried:
chroot aarch64-chroot/ qemu-aarch64-static -d
out_asm,in_asm,cpu,guest_errors /racket/racket/src/build/racket/racket3m
...

which dumps a massive log ending with:

IN:
0x40011309e0:  2a0003e4  mov      w4, w0
0x40011309e4:  d2800040  movz     x0, #0x2
0x40011309e8:  aa0503e1  mov      x1, x5
0x40011309ec:  d2800002  movz     x2, #0
0x40011309f0:  d2800103  movz     x3, #0x8
0x40011309f4:  d28010e8  movz     x8, #0x87
0x40011309f8:  d4000001  svc      #0

OUT: [size=115]
0x61560900:  41 8b 6e e4              movl     -0x1c(%r14), %ebp
0x61560904:  85 ed                    testl    %ebp, %ebp
0x61560906:  0f 8c 5d 00 00 00        jl       0x61560969
0x6156090c:  49 8b 6e 40              movq     0x40(%r14), %rbp
0x61560910:  8b ed                    movl     %ebp, %ebp
0x61560912:  49 89 6e 60              movq     %rbp, 0x60(%r14)
0x61560916:  49 c7 46 40 02 00 00 00  movq     $2, 0x40(%r14)
0x6156091e:  49 8b 6e 68              movq     0x68(%r14), %rbp
0x61560922:  49 89 6e 48              movq     %rbp, 0x48(%r14)
0x61560926:  49 c7 46 50 00 00 00 00  movq     $0, 0x50(%r14)
0x6156092e:  49 c7 46 58 08 00 00 00  movq     $8, 0x58(%r14)
0x61560936:  49 c7 86 80 00 00 00 87  movq     $0x87, 0x80(%r14)
0x6156093e:  00 00 00
0x61560941:  48 bd fc 09 13 01 40 00  movabsq  $0x40011309fc, %rbp
0x61560949:  00 00
0x6156094b:  49 89 ae 40 01 00 00     movq     %rbp, 0x140(%r14)
0x61560952:  49 8b fe                 movq     %r14, %rdi
0x61560955:  be 02 00 00 00           movl     $2, %esi
0x6156095a:  ba 00 00 00 56           movl     $0x56000000, %edx
0x6156095f:  b9 01 00 00 00           movl     $1, %ecx
0x61560964:  e8 bb 48 b6 fe           callq    0x600c5224
0x61560969:  b8 43 08 56 61           movl     $0x61560843, %eax
0x6156096e:  e9 a5 06 3d ff           jmp      0x60931018

 PC=00000040011309e0 X00=0000000000000000 X01=000000000000113a
X02=0000000000000006 X03=000000000000113a X04=0000004001210bd8
X05=00000040012f92a0 X06=ffffffffffffffff X07=ffffffffffffffff
X08=0000000000000083 X09=ffffffffffffffff X10=ffffffffffffffff
X11=ffffffffffffffff X12=ffffffffffffffff X13=ffffffffffffffff
X14=0000000000000000 X15=00000000000001fc X16=0000004001131cd0
X17=0000004000781298 X18=0000000000000000 X19=0000000000000006
X20=000000400124a000 X21=00000040012459d0 X22=0000000000000000
X23=0000000000000000 X24=0000000000000000 X25=0000000000000000
X26=0000000000000000 X27=0000000000000000 X28=0000000000000000
X29=00000040012f9280 X30=0000004001130988  SP=00000040012f9280
PSTATE=00000000 ---- S EL0t
qemu: uncaught target signal 6 (Aborted) - core dumped
Aborted (core dumped)

So, am I correct in thinking that qemu converts guest assembly (IN -
aarch64) into host assembly (OUT - x86_64) and then executes somehow on
the host this OUT assembly?

So above, qemu crashes at PC 0x40011309e0 which corresponds to guest:
mov      w4, w0

At this point I should mention I don't know much about aarch64 but I
didn't think it had wX registers.
Is there a way to determine which host PC caused the crash or even
single step at this point the host instructions? (or am I asking for too
much)

Regards,

-- 
Paulo Matos