Re: Guest Ubuntu 18.04 fails to boot with -serial mon:stdio, cannot find ttyS0.

[David Fernandez]

As I have not hear anything yet, I thought I would summarize the current 
status
for this problem. Please, let me know if any other tests or information are
needed.

I am running qemu-system-x86_64 v5.2.0 (also tried v6.1.0 and top of 
master) on:
    - aarch64 (Jetson AGX Xavier) with Ubuntu 18.04.5 as a host 
(compiled from
      git sources as distro version for it was 2.11, which is too old), 
and on
    - x86_64 (my laptop) with Fedora 34 as a host (here the 
qemu-system-x86_64
      distro version is 5.2.0).

Running Ubuntu 18.04.6 server install cdrom (also tried Ubuntu 20.04.3) 
as the
guest.

The following services fail on the Jetson, but not on the laptop. The 
first one
is the ttyS0 console, which seems the most important thing as it is provided
directly by the virtual emulation (-serial mon:stdio):

[ TIME ] Timed out waiting for device dev-ttyS0.device.
[DEPEND] Dependency failed for Serial Getty on ttyS0.
...
[FAILED] Failed to start Dispatcher daemon for systemd-networkd. <== 
network does start fine though.
See 'systemctl status networkd-dispatcher.service' for details.
...
[FAILED] Failed to start Wait until snapd is fully seeded. <== snapd 
runs fine though.
See 'systemctl status snapd.seeded.service' for details.
...
[FAILED] Failed to start Holds Snappy daemon refresh.
See 'systemctl status snapd.hold.service' for details.
[  OK  ] Started Update UTMP about System Runlevel Changes.
... waits forever ...

Note that the Jetson has 8 cores running at 2.25GHz, and this tests is 
run just
after boot with no other user applications launched.

I wonder if I need something in my build options or if I need to rebuild my
kernel with some added kernel configuration options...

Hopefully, some experts around here can help me with that if it is a 
known thing
(I google around but other than mentioning that 2.11 is too old, could 
not find
any clear reason about this problem).

My build options for the Jetson (I did not do cross-compiling, as I was 
a bit
unsure about pkg-config/glib-2.0 for build and host/target, so I compiled
natively on the Jetson host machine, using a separate build folder):

../configure \
   --target-list=x86_64-softmmu \
   --enable-plugins \
   --enable-attr \
   --enable-auth-pam \
   --enable-cap-ng \
   --enable-curl \
   --enable-gnutls \
   --enable-kvm \ <== not available as an accelerator for Ubuntu host on 
Jetson
   --enable-libnfs \
   --enable-libudev \
   --enable-libusb \
   --enable-libxml2 \
   --enable-linux-aio \
   --enable-nettle \
   --enable-seccomp \
   --enable-snappy \
   --enable-spice \
   --enable-usb-redir \
   --enable-vde \
   --enable-virtfs \
   --enable-virtiofsd \
   --enable-xkbcommon \
   --enable-pie \
   --enable-modules \
   --enable-membarrier \
   --enable-tools \
   --enable-vvfat

Installed as per the default prefix in /usr/local (the distro already 
have that
in the path before the standard distro folders, so all runs as expected):

$ which qemu-system-x86_64
/usr/local/bin/qemu-system-x86_64

Some things like the following could not be used due to current kernel or
ubuntu packages available (perhaps I need to compile fuse from sources?):

    - --enable-libpmem (absent package, couldn't find the right one)
    - --enable-libssh (0.8.0 but >= 0.8.7 for libssh-4-dev)
    - --enable-fuse --enable-fuse-lseek (fuse2 available but fuse3 needed)
    - --enable-netmap (not in current kernel kernel, the required header 
only
      exists for newer kernels)

I run it with the following command line on both Jetson and laptop:

qemu-system-x86_64 \
   -boot order=dc,menu=on \
   -cdrom ubuntu-18.04.6-live-server-amd64.iso \
   -nographic \
   -serial mon:stdio \
   -kernel ufm/vmlinuz \
   -initrd ufm/initrd \
   -append 'boot=casper console=ttyS0 ---' \
   -m 16384 \
   -drive  file=ufm/ufm.fd0,format=raw,if=floppy \ <= empty image to 
avoid ubuntu complaining about fd0.
   -drive  file=ufm/ufm.img,format=raw,if=ide \
   -netdev bridge,br=virbr0,id=net0 \
   -device virtio-net-pci,netdev=net0,id=nic1 \
   -device usb-ehci,id=ehci

The vmlinuz & initrd come from the ubuntu iso in the casper folder, the 
append
uses what the grub configuration had for the normal default kernel in 
the iso.

The virtual bridge works as expected with the right allow line in
/usr/local/etc/qemu/bridge.conf and setting the qemu-bridge-helper u+s 
(plus a
few extra packages).

Anything else is as per the Ubunt u 18.04.5 (LTS) repos used by the host 
(I did
not upgraded the packages, other than the packages needed to get the bridge
working and the dev packages to compile the qemu on the aarch64).

In the Jetson machine running Ubuntu 18.04.5 I get:

$ uname -a
Linux vpm-devkit 4.9.201-tegra #1 SMP PREEMPT Fri Jul 2 15:24:18 BST 
2021 aarch64 aarch64 aarch64 GNU/Linux

$ cat /proc/cpuinfo
processor       : 0
model name      : ARMv8 Processor rev 0 (v8l)
BogoMIPS        : 62.50
Features        : fp asimd evtstrm aes pmull sha1 sha2 crc32 atomics 
fphp asimdhp
CPU implementer : 0x4e
CPU architecture: 8
CPU variant     : 0x0
CPU part        : 0x004
CPU revision    : 0
MTS version     : 51035886

... 8 cpus ...

$ sudo virt-host-validate
   QEMU: Checking if device /dev/kvm 
exists                                   : FAIL (Check that CPU and 
firmware supports virtualization and kvm module is loaded)
   QEMU: Checking if device /dev/vhost-net 
exists                             : WARN (Load the 'vhost_net' module 
to improve performance of virtio networking)
   QEMU: Checking if device /dev/net/tun 
exists                               : PASS
   QEMU: Checking for cgroup 'memory' controller 
support                      : PASS
   QEMU: Checking for cgroup 'memory' controller 
mount-point                  : PASS
   QEMU: Checking for cgroup 'cpu' controller 
support                         : PASS
   QEMU: Checking for cgroup 'cpu' controller 
mount-point                     : PASS
   QEMU: Checking for cgroup 'cpuacct' controller 
support                     : PASS
   QEMU: Checking for cgroup 'cpuacct' controller 
mount-point                 : PASS
   QEMU: Checking for cgroup 'cpuset' controller 
support                      : PASS
   QEMU: Checking for cgroup 'cpuset' controller 
mount-point                  : PASS
   QEMU: Checking for cgroup 'devices' controller 
support                     : PASS
   QEMU: Checking for cgroup 'devices' controller 
mount-point                 : PASS
   QEMU: Checking for cgroup 'blkio' controller 
support                       : PASS
   QEMU: Checking for cgroup 'blkio' controller 
mount-point                   : PASS
WARN (Unknown if this platform has IOMMU support) <= it does have IOMMU, 
I know for sure from playing with v4l2...
    LXC: Checking for Linux >= 
2.6.26                                         : PASS
    LXC: Checking for namespace 
ipc                                           : PASS
    LXC: Checking for namespace 
mnt                                           : PASS
    LXC: Checking for namespace 
pid                                           : PASS
    LXC: Checking for namespace 
uts                                           : PASS
    LXC: Checking for namespace 
net                                           : PASS
    LXC: Checking for namespace 
user                                          : PASS
    LXC: Checking for cgroup 'memory' controller 
support                      : PASS
    LXC: Checking for cgroup 'memory' controller 
mount-point                  : PASS
    LXC: Checking for cgroup 'cpu' controller 
support                         : PASS
    LXC: Checking for cgroup 'cpu' controller 
mount-point                     : PASS
    LXC: Checking for cgroup 'cpuacct' controller 
support                     : PASS
    LXC: Checking for cgroup 'cpuacct' controller 
mount-point                 : PASS
    LXC: Checking for cgroup 'cpuset' controller 
support                      : PASS
    LXC: Checking for cgroup 'cpuset' controller 
mount-point                  : PASS
    LXC: Checking for cgroup 'devices' controller 
support                     : PASS
    LXC: Checking for cgroup 'devices' controller 
mount-point                 : PASS
    LXC: Checking for cgroup 'blkio' controller 
support                       : PASS
    LXC: Checking for cgroup 'blkio' controller 
mount-point                   : PASS
    LXC: Checking if device /sys/fs/fuse/connections 
exists                   : FAIL (Load the 'fuse' module to enable /proc/ 
overrides)