How to bind a DHCP server to a bridge interface inside QEMU?

[Charlie Turner]

Hi,
My goal is to have a virtual networking infrastructure, with one dual-NIC
machine acting as a gateway. One of its NICs is connected to the private
LAN, and provides a DHCP service, the other NIC is connected to the
"upstream", and provides Internet service.

I start by creating a bridge to connect the private NIC of the gateway,
to the virtual clients that will join the network later on,

┌────
│ ip link add name br0 type bridge
│ ip addr add 10.42.0.1/24 dev br0   # this is the network ID of the
=private= interface, inside the host
│ ip link set br0 up
└────

To experiment with what works, I did the following

┌────
│ IFACE=br0
│ dnsmasq --dhcp-match=set:efi-x86_64,option:client-arch,7
--dhcp-boot=tag:efi-x86_64,syslinux.efi --dhcp-boot=lpxelinux.0 \
│    --dhcp-range=10.42.0.10,10.42.0.100 --dhcp-script=/bin/echo \
│    --enable-tftp=$IFACE \
│    --tftp-root=/mnt/tmp/boots/tftp \
│    --log-queries=extra --conf-file=/dev/null --interface=$IFACE
└────

And then spin up a test machine, connected to the `br0'. It uses the
interface `tap15', which is created like so,

┌────
│ ip tuntap add mode tap tap15
│ ip link set tap15 up
│ ip link set tap15 master br0
└────

┌────
│ qemu-system-x86_64 \
│    -machine pc-q35-6.0,accel=kvm \
│    -m 1024 -smp 2,sockets=2,cores=1,threads=1 \
│    -hda dut_disk0.qcow2 \
│    -chardev file,id=logfile,path=dut-log-525400112200-193528-19092021.log \
│    -chardev 
socket,id=foo,path=/run/salad_socks/machine0.socket,server=on,wait=off,logfile=dut-log-525400112200-193528-19092021.log
\
│    -device pci-serial,chardev=foo \
│    -netdev tap,id=net0,ifname=tap15,script=no,downscript=no \
│    -device virtio-net-pci,netdev=net0,bootindex=1,mac=52:54:00:11:22:00 \
│    -vga virtio
└────

This works fine. Now, I'd like to go a step further and run the
`dnsmasq' inside a VM, not on my host. It will be listening to the
`private' interface I described above.

For the host-side of the private NIC interface, I create a tap device,
like before,

┌────
│ ip tuntap add mode tap tap0
│ ip link set tap0 up
│ ip link set tap0 master br0
└────

With this setup out of the way, I start the dual-NIC gateway machine,

┌────
│ qemu-system-x86_64 \
│     -hda $DISK_FILE \
│     -boot c \
│     -device virtio-net-pci,romfile=,netdev=net0 \
│     -device virtio-net-pci,romfile=,netdev=net1 \
│     `# Simulate the plugged in "upstream" cable with user-mode networking` \
│     -netdev 
user,id=net0,hostfwd=tcp::60022-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8081-:8000,hostfwd=tcp::2375-:2375
\
│     `# And now the unplugged one with, with TAP networks` \
│     -netdev tap,id=net1,ifname=tap0,script=no,downscript=no \    #
this is interface dnsmasq will listen on
│     $BOOT_MODE \
│     -m 4G \
│     -display sdl \
│     -enable-kvm \
│     -serial stdio
└────

This VM has a pre-configured "disk file", that will launch a
dnsmasq server, listening on the net1 interface.

I try the above experiment again, this time the dnsmasq instance is
listening from within a VM, not on the `br0' interface, but rather, on
`tap0', which is plugged into the bridge,

┌────
│ qemu-system-x86_64 \
│    -machine pc-q35-6.0,accel=kvm \
│    -m 1024 -smp 2,sockets=2,cores=1,threads=1 \
│    -hda dut_disk0.qcow2 \
│    -chardev file,id=logfile,path=dut-log-525400112200-193528-19092021.log \
│    -chardev 
socket,id=foo,path=/run/salad_socks/machine0.socket,server=on,wait=off,logfile=dut-log-525400112200-193528-19092021.log
\
│    -device pci-serial,chardev=foo \
│    -netdev tap,id=net0,ifname=tap15,script=no,downscript=no \
│    -device virtio-net-pci,netdev=net0,bootindex=1,mac=52:54:00:11:22:00 \
│    -vga virtio
└────

When this machine boots, by monitoring the bridge, I can see the BOOTP
requests on the `br0' interface using `tcpdump',

┌────
│ # tcpdump  -i br0 -nN
│ tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
│ listening on br0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
│ 18:47:53.882429 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from 52:54:00:11:22:01, length 397
│ 18:47:54.924377 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from 52:54:00:11:22:01, length 397
│ 18:47:56.956663 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from 52:54:00:11:22:01, length 397
│ 18:48:01.021086 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP,
Request from 52:54:00:11:22:01, length 397
└────

But, nothing responds to it, and so the boot fails. I was expecting that
the `dnsmasq' listening on `tap0', which is also connected to the `br0',
would see these requests, and send a BOOTP response.

During research, I found that "When you junction interfaces into a
bridge, you no longer use the individual interfaces but the entire
bridge as an interface. You probably need to change your DHCP server to
listen on br0 instead of tap0." <https://superuser.com/a/419612/291744>

Unfortunately, I can't ask QEMU to assign one of the interfaces to be
`br0', rather than `tap0'. Changing the gateway invocation to,

┌────
│ qemu-system-x86_64 \
│     -hda $DISK_FILE \
│     -boot c \
│     -device virtio-net-pci,romfile=,netdev=net0 \
│     -device virtio-net-pci,romfile=,netdev=net1 \
│     `# Simulate the plugged in "upstream" cable with user-mode networking` \
│     -netdev 
user,id=net0,hostfwd=tcp::60022-:22,hostfwd=tcp::8080-:80,hostfwd=tcp::8081-:8000,hostfwd=tcp::2375-:2375
\
│     `# And now the unplugged one with, with TAP networks` \
│     -netdev tap,id=net1,ifname=br0,script=no,downscript=no \
│     $BOOT_MODE \
│     -m 4G \
│     -display sdl \
│     -enable-kvm \
│     -serial stdio
└────

(Note, `net1''s `ifname' is now specified as `br0', not `tap0')

This unfortunately fails with

┌────
│ qemu-system-x86_64: -netdev
tap,id=net1,ifname=br0,script=no,downscript=no: could not configure
/dev/net/tun (br0): Invalid argument
└────

And I'm stuck at this point, how can I have the gateway's private NIC
respond to DHCP requests on the bridge?

Thanks for reading!

B.R.
  Charles.