Re: [Qemu-stable] [Qemu-devel] [PATCH] block: Fix AioContext switch for bs->drv == NULL

[Stefano Garzarella]

On Wed, Apr 17, 2019 at 05:48:50PM +0200, Kevin Wolf wrote:
> Even for block nodes with bs->drv == NULL, we can't just ignore a
> bdrv_set_aio_context() call. Leaving the node in its old context can
> mean that it's still in an iothread context in bdrv_close_all() during
> shutdown, resulting in an attempted unlock of the AioContext lock which
> we don't hold.
> 
> This is an example stack trace of a related crash:
> 
>  #0  0x00007ffff59da57f in raise () at /lib64/libc.so.6
>  #1  0x00007ffff59c4895 in abort () at /lib64/libc.so.6
>  #2  0x0000555555b97b1e in error_exit (err=<optimized out>, address@hidden 
> <__func__.19059> "qemu_mutex_unlock_impl") at util/qemu-thread-posix.c:36
>  #3  0x0000555555b97f7f in qemu_mutex_unlock_impl (address@hidden, 
> address@hidden "util/async.c", address@hidden) at util/qemu-thread-posix.c:97
>  #4  0x0000555555b92f55 in aio_context_release (address@hidden) at 
> util/async.c:507
>  #5  0x0000555555b05cf8 in bdrv_prwv_co (address@hidden, address@hidden, 
> address@hidden, address@hidden, address@hidden)
>          at block/io.c:833
>  #6  0x0000555555b060a9 in bdrv_pwritev (qiov=0x7fffffffd4f0, offset=131072, 
> child=0x7fffc80012f0) at block/io.c:990
>  #7  0x0000555555b060a9 in bdrv_pwrite (child=0x7fffc80012f0, offset=131072, 
> buf=<optimized out>, bytes=<optimized out>) at block/io.c:990
>  #8  0x0000555555ae172b in qcow2_cache_entry_flush (address@hidden, 
> address@hidden, address@hidden) at block/qcow2-cache.c:51
>  #9  0x0000555555ae18dd in qcow2_cache_write (address@hidden, 
> c=0x5555568cc740) at block/qcow2-cache.c:248
>  #10 0x0000555555ae15de in qcow2_cache_flush (bs=0x555556810680, c=<optimized 
> out>) at block/qcow2-cache.c:259
>  #11 0x0000555555ae16b1 in qcow2_cache_flush_dependency (c=0x5555568a1700, 
> c=0x5555568a1700, bs=0x555556810680) at block/qcow2-cache.c:194
>  #12 0x0000555555ae16b1 in qcow2_cache_entry_flush (address@hidden, 
> address@hidden, address@hidden) at block/qcow2-cache.c:194
>  #13 0x0000555555ae18dd in qcow2_cache_write (address@hidden, 
> c=0x5555568a1700) at block/qcow2-cache.c:248
>  #14 0x0000555555ae15de in qcow2_cache_flush (address@hidden, c=<optimized 
> out>) at block/qcow2-cache.c:259
>  #15 0x0000555555ad242c in qcow2_inactivate (address@hidden) at 
> block/qcow2.c:2124
>  #16 0x0000555555ad2590 in qcow2_close (bs=0x555556810680) at 
> block/qcow2.c:2153
>  #17 0x0000555555ab0c62 in bdrv_close (bs=0x555556810680) at block.c:3358
>  #18 0x0000555555ab0c62 in bdrv_delete (bs=0x555556810680) at block.c:3542
>  #19 0x0000555555ab0c62 in bdrv_unref (bs=0x555556810680) at block.c:4598
>  #20 0x0000555555af4d72 in blk_remove_bs (address@hidden) at 
> block/block-backend.c:785
>  #21 0x0000555555af4dbb in blk_remove_all_bs () at block/block-backend.c:483
>  #22 0x0000555555aae02f in bdrv_close_all () at block.c:3412
>  #23 0x00005555557f9796 in main (argc=<optimized out>, argv=<optimized out>, 
> envp=<optimized out>) at vl.c:4776
> 
> The reproducer I used is a qcow2 image on gluster volume, where the
> virtual disk size (4 GB) is larger than the gluster volume size (64M),
> so we can easily trigger an ENOSPC. This backend is assigned to a
> virtio-blk device using an iothread, and then from the guest a
> 'dd if=/dev/zero of=/dev/vda bs=1G count=1' causes the VM to stop
> because of an I/O error. qemu_gluster_co_flush_to_disk() sets
> bs->drv = NULL on error, so when virtio-blk stops the dataplane, the
> block nodes stay in the iothread AioContext. A 'quit' monitor command
> issued from this paused state crashes the process.
> 
> Fixes: https://bugzilla.redhat.com/show_bug.cgi?id=1631227
> Cc: address@hidden
> Signed-off-by: Kevin Wolf <address@hidden>
> ---
>  block.c | 12 ++----------
>  1 file changed, 2 insertions(+), 10 deletions(-)
> 

Reviewed-by: Stefano Garzarella <address@hidden>