Re: [PATCH] system/runstate: Fix regression, clarify BQL status of exit notifiers

[David Woodhouse]

On Wed, 2025-01-15 at 20:17 +0100, Phil Dennis-Jordan wrote:
> 
> BQL_LOCK_GUARD expands to a call to bql_auto_lock(), which in turn
> defends against recursive locking by checking bql_locked(). 
> 
> https://gitlab.com/qemu-project/qemu/-/blob/master/include/qemu/main-loop.h#L377
> 
> I think that should make it safe?

Looks like it. I did this to test:

--- a/hw/i386/kvm/xen_evtchn.c
+++ b/hw/i386/kvm/xen_evtchn.c
@@ -451,6 +451,10 @@ void xen_evtchn_set_callback_level(int level)
         if (level && !s->extern_gsi_level) {
             kvm_xen_set_callback_asserted();
         }
+        if (level) {
+            printf("Exiting, BQL held\n");
+            exit(77);
+        }
     }
 }
 
--- a/system/runstate.c
+++ b/system/runstate.c
@@ -851,6 +851,7 @@ void qemu_remove_exit_notifier(Notifier *notify)
 static void qemu_run_exit_notifiers(void)
 {
     BQL_LOCK_GUARD();
+    printf("%s has BQL\n", __func__);
     notifier_list_notify(&exit_notifiers, NULL);
 }
 

So the first time a Xen guest's callback IRQ is asserted, it exited
with the BQL held, and qemu_run_exit_notifiers() didn't get stuck.

[    0.521568] ACPI: \_SB_.GSIF: Enabled at IRQ 21
Exiting, BQL held
qemu_run_exit_notifiers has BQL


The actual cleanup of the XenDevice did then deadlock on the Xen evtchn
port_lock, which had *also* been held when my hack exited in the evtchn
code. But that one is expected.

#0  0x00007fc5b2a7b0c0 in __lll_lock_wait () at /lib64/libc.so.6
#1  0x00007fc5b2a81d81 in pthread_mutex_lock@@GLIBC_2.2.5 ()
    at /lib64/libc.so.6
#2  0x0000558286c07a63 in qemu_mutex_lock_impl
    (mutex=0x558294179998, file=0x558286f9b905 "../hw/i386/kvm/xen_evtchn.c", 
line=2147) at ../util/qemu-thread-posix.c:95
#3  0x00005582868d774f in xen_be_evtchn_unbind (xc=0x5582939b3810, port=2)
    at ../hw/i386/kvm/xen_evtchn.c:2147
#4  0x000055828679e0a9 in qemu_xen_evtchn_unbind
    (xc=<optimized out>, port=<optimized out>)
    at /home/dwmw2/git/qemu/include/hw/xen/xen_backend_ops.h:91
#5  xen_device_unbind_event_channel
    (xendev=<optimized out>, channel=0x5582939b4cb0, errp=0x0)
    at ../hw/xen/xen-bus.c:961
#6  0x00005582865f64b9 in xen_console_disconnect
    (xendev=xendev@entry=0x5582942df4a0, errp=errp@entry=0x0)
    at ../hw/char/xen_console.c:298
#7  0x00005582865f6673 in xen_console_unrealize (xendev=0x5582942df4a0)
    at ../hw/char/xen_console.c:411
#8  0x000055828679e201 in xen_device_unrealize (dev=<optimized out>)
    at ../hw/xen/xen-bus.c:988
#9  0x0000558286c0da5f in notifier_list_notify (list=<optimized out>, data=0x0)
    at ../util/notify.c:39
#10 0x00007fc5b2a2a461 in __run_exit_handlers () at /lib64/libc.so.6
#11 0x00007fc5b2a2a52e in exit () at /lib64/libc.so.6
#12 0x00005582868d86dd in xen_evtchn_set_callback_level (level=1)
    at ../hw/i386/kvm/xen_evtchn.c:456
#13 0x00005582868d7c74 in inject_callback
    (s=0x558294179650, vcpu=<optimized out>) at ../hw/i386/kvm/xen_evtchn.c:548
#14 do_set_port_compat
    (s=<optimized out>, port=<optimized out>, shinfo=<optimized out>, 
vcpu_info=<optimized out>) at ../hw/i386/kvm/xen_evtchn.c:921
#15 set_port_pending (s=s@entry=0x558294179650, port=<optimized out>)
    at ../hw/i386/kvm/xen_evtchn.c:963

smime.p7s
Description: S/MIME cryptographic signature