Re: [Qemu-discuss] Fwd: qemu VM cannot be killed

[Lukáš Kubín]

OK, thanks Michael! We'll consider disabling it too.

Lukas

On Wed, Nov 15, 2017 at 3:38 PM, Michael S. Tsirkin <address@hidden> wrote:

> Yes - I suspect a packet is stuck somewhere in networking stack.
> This is what vhost is waiting for.
>
> Yes, host reboot is the only way out.
>
> RHEL disables zero copy tx in vhost to avoid these issues.
>
> On Wed, Nov 15, 2017 at 02:55:32PM +0100, Lukáš Kubín wrote:
> > CC-ing Michael and Jason as I was suggested in OFTC:#virt forum. Thanks!
> >
> > ---------- Forwarded message ----------
> > From: Lukáš Kubín <address@hidden>
> > Date: Wed, Nov 15, 2017 at 1:39 PM
> > Subject: qemu VM cannot be killed
> > To: address@hidden
> >
> >
> > Hi, we've experienced an issue with kvm instance which got stuck at
> reboot.
> > It's an OpenStack environment, with OpenContrail networking (vrouter
> agent
> > running on host), Ubuntu 16.04.
> >
> > Machine was first called to reboot from guest OS by user, had issues
> with NFS
> > unmount during that, user sent a hard-reboot call from OpenStack again
> then.
> > Then we (platform operator) got involved, tried to "virsh destroy" it
> with this
> > output:
> >
> >
> >     error: Failed to destroy domain instance-00004243
> >     error: Failed to terminate process 140529 with SIGKILL: Device or
> resource
> >     busy
> >
> >
> > Neither "kill -9" sent to the qemu process helped.
> >
> > Good guys at OFTC:#virt have guided me to collect the following traces
> and ask
> > for help here:
> >
> >
> >     # cat /proc/140529/wchan
> >     vhost_net_ubuf_put_and_wait
> >
> >     # cat /proc/140529/stack
> >     [<ffffffffc03c2f74>] vhost_net_ubuf_put_and_wait+0x54/0xa0
> [vhost_net]
> >     [<ffffffffc03c33b4>] vhost_net_ioctl+0x354/0x8a0 [vhost_net]
> >     [<ffffffffa26478c1>] do_vfs_ioctl+0xa1/0x5f0
> >     [<ffffffffa2647e89>] SyS_ioctl+0x79/0x90
> >     [<ffffffffa2c9b7f6>] entry_SYSCALL_64_fastpath+0x1e/0xa8
> >     [<ffffffffffffffff>] 0xffffffffffffffff
> >
> >
> > The versions we use are:
> >
> >   • kernel 4.8.0-41-generic
> >   • qemu-kvm 1:2.5+dfsg-5ubuntu10.2~xenial0+contrail1
> >   • libvirt-bin 1.3.1-1ubuntu10.1~xenial1+contrail1
> >
> > What can be the cause for this error? What can we do in such a situation
> to
> > destroy the VM - is physical server reboot the only option?
> >
> > Thanks and greetings,
> >
> > Lukáš
> >
>