[Qemu-commits] [qemu/qemu] beb71c: iotests/059: Fix reference output

[Peter Maydell]

  Branch: refs/heads/master
  Home:   https://github.com/qemu/qemu
  Commit: beb71c1c02ed05a705f5af3e0395a35cfa7bb02b
      
https://github.com/qemu/qemu/commit/beb71c1c02ed05a705f5af3e0395a35cfa7bb02b
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M tests/qemu-iotests/059.out

  Log Message:
  -----------
  iotests/059: Fix reference output

As of the patch to flush qemu-img's "Formatting" message before the
error message, 059 has been broken for vmdk.  Fix it.

Fixes: 4e2f4418784da09cb106264340241856cd2846df
       ("qemu-img: Flush stdout before before potential stderr messages")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200811084150.326377-1-mreitz@redhat.com>
Reviewed-by: Eric blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 985d7f150c5d6ca0266c4c2844b54806311e0a40
      
https://github.com/qemu/qemu/commit/985d7f150c5d6ca0266c4c2844b54806311e0a40
  Author: Max Reitz <mreitz@redhat.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M tests/qemu-iotests/259.out

  Log Message:
  -----------
  iotests/259: Fix reference output

The error message has changed recently, breaking the test.  Fix it.

Fixes: a2b333c01880f56056d50c238834d62e32001e54
       ("block: nbd: Fix convert qcow2 compressed to nbd")
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200811080830.289136-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 1dc4718d849e1a1fe665ce5241ed79048cfa2cfc
      
https://github.com/qemu/qemu/commit/1dc4718d849e1a1fe665ce5241ed79048cfa2cfc
  Author: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M block/nbd.c

  Log Message:
  -----------
  block/nbd: use non-blocking connect: fix vm hang on connect()

This makes nbd's connection_co yield during reconnects, so that
reconnect doesn't block the main thread. This is very important in
case of an unavailable nbd server host: connect() call may take a long
time, blocking the main thread (and due to reconnect, it will hang
again and again with small gaps of working time during pauses between
connection attempts).

Realization notes:

 - We don't want to implement non-blocking connect() over non-blocking
 socket, because getaddrinfo() doesn't have portable non-blocking
 realization anyway, so let's just use a thread for both getaddrinfo()
 and connect().

 - We can't use qio_channel_socket_connect_async (which behaves
 similarly and starts a thread to execute connect() call), as it's relying
 on someone iterating main loop (g_main_loop_run() or something like
 this), which is not always the case.

 - We can't use thread_pool_submit_co API, as thread pool waits for all
 threads to finish (but we don't want to wait for blocking reconnect
 attempt on shutdown.

 So, we just create the thread by hand. Some additional difficulties
 are:

 - We want our connect to avoid blocking drained sections and aio context
 switches. To achieve this, we make it possible to "cancel" synchronous
 wait for the connect (which is a coroutine yield actually), still,
 the thread continues in background, and if successful, its result may be
 reused on next reconnect attempt.

 - We don't want to wait for reconnect on shutdown, so there is
 CONNECT_THREAD_RUNNING_DETACHED thread state, which means that the block
 layer is no longer interested in a result, and thread should close new
 connected socket on finish and free the state.

How to reproduce the bug, fixed with this commit:

1. Create an image on node1:
   qemu-img create -f qcow2 xx 100M

2. Start NBD server on node1:
   qemu-nbd xx

3. Start vm with second nbd disk on node2, like this:

  ./x86_64-softmmu/qemu-system-x86_64 -nodefaults -drive \
    file=/work/images/cent7.qcow2 -drive file=nbd+tcp://192.168.100.2 \
    -vnc :0 -qmp stdio -m 2G -enable-kvm -vga std

4. Access the vm through vnc (or some other way?), and check that NBD
   drive works:

   dd if=/dev/sdb of=/dev/null bs=1M count=10

   - the command should succeed.

5. Now, let's trigger nbd-reconnect loop in Qemu process. For this:

5.1 Kill NBD server on node1

5.2 run "dd if=/dev/sdb of=/dev/null bs=1M count=10" in the guest
    again. The command should fail and a lot of error messages about
    failing disk may appear as well.

    Now NBD client driver in Qemu tries to reconnect.
    Still, VM works well.

6. Make node1 unavailable on NBD port, so connect() from node2 will
   last for a long time:

   On node1 (Note, that 10809 is just a default NBD port):

   sudo iptables -A INPUT -p tcp --dport 10809 -j DROP

   After some time the guest hangs, and you may check in gdb that Qemu
   hangs in connect() call, issued from the main thread. This is the
   BUG.

7. Don't forget to drop iptables rule from your node1:

   sudo iptables -D INPUT -p tcp --dport 10809 -j DROP

Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20200812145237.4396-1-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: minor wording and formatting tweaks]
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 98c5d2e7010a60eddeabd057c9e0cd4e3a08f85f
      
https://github.com/qemu/qemu/commit/98c5d2e7010a60eddeabd057c9e0cd4e3a08f85f
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M qemu-img.c
    M qemu-io.c
    M qemu-nbd.c

  Log Message:
  -----------
  block: add missing socket_init() calls to tools

Any tool that uses sockets needs to call socket_init() in order to work
on the Windows platform.

Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200825103850.119911-2-berrange@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: 6e64dd572aa548aa6664ed02c6901d691f6a10ba
      
https://github.com/qemu/qemu/commit/6e64dd572aa548aa6664ed02c6901d691f6a10ba
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M qemu-nbd.c

  Log Message:
  -----------
  nbd: skip SIGTERM handler if NBD device support is not built

The termsig_handler function is used by the client thread handling the
host NBD device connection to do a graceful shutdown. IOW, if we have
disabled NBD device support at compile time, we don't need the SIGTERM
handler. This fixes a build issue for Windows.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200825103850.119911-3-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: eb705985f43d631438a318f1146eac61ae10d273
      
https://github.com/qemu/qemu/commit/eb705985f43d631438a318f1146eac61ae10d273
  Author: Daniel P. Berrangé <berrange@redhat.com>
  Date:   2020-09-02 (Wed, 02 Sep 2020)

  Changed paths:
    M meson.build
    M qemu-nbd.c

  Log Message:
  -----------
  nbd: disable signals and forking on Windows builds

Disabling these parts are sufficient to get the qemu-nbd program
compiling in a Windows build.

Signed-off-by: Daniel P. Berrangé <berrange@redhat.com>
Message-Id: <20200825103850.119911-4-berrange@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>


  Commit: df8176274a19c3b4502a10b2bffba914f690f1ab
      
https://github.com/qemu/qemu/commit/df8176274a19c3b4502a10b2bffba914f690f1ab
  Author: Peter Maydell <peter.maydell@linaro.org>
  Date:   2020-09-03 (Thu, 03 Sep 2020)

  Changed paths:
    M block/nbd.c
    M meson.build
    M qemu-img.c
    M qemu-io.c
    M qemu-nbd.c
    M tests/qemu-iotests/059.out
    M tests/qemu-iotests/259.out

  Log Message:
  -----------
  Merge remote-tracking branch 'remotes/ericb/tags/pull-nbd-2020-09-02' into 
staging

nbd patches for 2020-09-02

- fix a few iotests affected by earlier nbd changes
- avoid blocking qemu by nbd client in connect()
- build qemu-nbd for mingw

# gpg: Signature made Wed 02 Sep 2020 22:52:31 BST
# gpg:                using RSA key 71C2CC22B1C4602927D2F3AAA7A16B4A2527436A
# gpg: Good signature from "Eric Blake <eblake@redhat.com>" [full]
# gpg:                 aka "Eric Blake (Free Software Programmer) 
<ebb9@byu.net>" [full]
# gpg:                 aka "[jpeg image of size 6874]" [full]
# Primary key fingerprint: 71C2 CC22 B1C4 6029 27D2  F3AA A7A1 6B4A 2527 436A

* remotes/ericb/tags/pull-nbd-2020-09-02:
  nbd: disable signals and forking on Windows builds
  nbd: skip SIGTERM handler if NBD device support is not built
  block: add missing socket_init() calls to tools
  block/nbd: use non-blocking connect: fix vm hang on connect()
  iotests/259: Fix reference output
  iotests/059: Fix reference output

Signed-off-by: Peter Maydell <peter.maydell@linaro.org>


Compare: https://github.com/qemu/qemu/compare/67a7bfe560a1...df8176274a19