Re: [Qemu-discuss] Debugging a qemu hang using gdb

[Mark Wood-Patrick]

When qemu stops responding I'm still able to use the qemu monitor and "info
status" shows the vm as running but if I do "info cpus" the monitor also
hangs

I'm using qemu 2.8.0

Looks like we are hung up in pthread_cond_wait

#0  0x00002aaaac98f68c in pthread_cond_wait@@GLIBC_2.3.2 () from
/lib64/libpthread.so.0
#1  0x0000555555d006da in qemu_cond_wait (cond=0x55555638b060
<qemu_work_cond>, mutex=0x555556384660 <qemu_global_mutex>) at
util/qemu-thread-posix.c:137
#2  0x00005555559754f7 in do_run_on_cpu (cpu=0x55555693b1a0,
func=0x5555557f2c07 <do_kvm_cpu_synchronize_state>, data=...,
mutex=0x555556384660 <qemu_global_mutex>)
    at cpus-common.c:152
#3  0x00005555557d614c in run_on_cpu (cpu=0x55555693b1a0,
func=0x5555557f2c07 <do_kvm_cpu_synchronize_state>, data=...)
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/cpus.c:920
#4  0x00005555557f2cb1 in kvm_cpu_synchronize_state (cpu=0x55555693b1a0)
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/kvm-all.c:1870
#5  0x00005555557d42bf in cpu_synchronize_state (cpu=0x55555693b1a0)
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/include/sysemu/kvm.h:469
#6  0x00005555557d7892 in qmp_query_cpus (errp=0x0) at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/cpus.c:1552
#7  0x000055555596e5b1 in hmp_info_cpus (mon=0x555556904890,
qdict=0x5555570f9300) at hmp.c:341
#8  0x00005555557dfccf in handle_hmp_command (mon=0x555556904890,
cmdline=0x555556938639 "")
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/monitor.c:2949
#9  0x00005555557e2492 in monitor_command_cb (opaque=0x555556904890,
cmdline=0x555556938630 "info cpus", readline_opaque=0x0)
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/monitor.c:3819
#10 0x0000555555d1907a in readline_handle_byte (rs=0x555556938630, ch=0xd)
at util/readline.c:393
#11 0x00005555557e23cc in monitor_read (opaque=0x555556904890,
buf=0x7fffffff83d0 "\r\277\331UUU", size=0x1)
---Type <return> to continue, or q <return> to quit---
    at
/home/scratch.mwoodpatrick_gpu/fsa/fsf_trees/Mark__2GPU_acos/fsf-tree/src/kvm-2.8.0.devel.vpath/qemu-kvm-2.8.0/monitor.c:3802
#12 0x000055555593ed84 in qemu_chr_be_write_impl (s=0x5555568faad0,
buf=0x7fffffff83d0 "\r\277\331UUU", len=0x1) at qemu-char.c:419
#13 0x000055555593ee0c in qemu_chr_be_write (s=0x5555568faad0,
buf=0x7fffffff83d0 "\r\277\331UUU", len=0x1) at qemu-char.c:431
#14 0x000055555594410e in tcp_chr_read (chan=0x555562f2ca00, cond=G_IO_IN,
opaque=0x5555568faad0) at qemu-char.c:3145
#15 0x0000555555ca3ba2 in qio_channel_fd_source_dispatch
(source=0x5555580aa540, callback=0x555555943fb4 <tcp_chr_read>,
user_data=0x5555568faad0) at io/channel-watch.c:84
#16 0x00002aaaaf9f044c in g_main_dispatch (context=<optimized out>) at
gmain.c:2715
#17 g_main_context_dispatch (context=0x5555568ed260) at gmain.c:3219
#18 0x0000555555c081f0 in glib_pollfds_poll () at main-loop.c:215
#19 0x0000555555c08301 in os_host_main_loop_wait (timeout=0x1c6fd23) at
main-loop.c:260
#20 0x0000555555c083c5 in main_loop_wait (nonblocking=0x0) at
main-loop.c:508
#21 0x000055555594f4ad in main_loop () at vl.c:1977
#22 0x0000555555957a12 in main (argc=0x26, argv=0x7fffffff9b18,
envp=0x7fffffff9c50) at vl.c:4836

On Fri, May 24, 2019 at 5:32 AM Mark Wood-Patrick <address@hidden>
wrote:

> I'm running some tests using qemu but once in a while these tests cause
> qemu to hang so my current ssh session into guest os running on qemu does
> not respond & pings are not replied to, I'm not seeing any obvious issues
> in the kernel log. I would appreciate any pointers on the best way to debug
> these hangs.
>


-- 
Mark Wood-Patrick