Freebsd 12 strange disk behaviour on QEMU machine

[Roland Giesler]

We have a FreeBSD machine running an IRIS poller that behaves really strange. 

We had FreeBSD 12.3 at first, then changed to 12.4.  Same result.

We also changed from BIOS to UEFI.  Same result.

The storage is ceph RBD on NVMe SSD drives, so it's fast.

The symptoms are as follows:

After installation of the OS and the script (most perl) that IRIS uses, the service run perflectly.  I then take a snapshot.  If the machine is restarted for whatever else reason everything comes up, but the FreeBSD processes go into a "D state", meaning they are waiting for the disk.

This happens very time.  It's then not possible to fix the machine, although all seems quite normal and not config changes took place that we can detect.

If I roll back the snapshot, things are normal for a while put soon processes go into D state again.

I'm not sure that this is a qemu problem, but we have to start somewhere. 

Finally, I not have the VM running an a local volume (non ceph, standalone SSD) and there's no problem.  This is actually an installation that was done on ceph and then the disk volume was moved to the current location.