Re: Issues to set up a tap network backend

Il giorno mar 13 apr 2021 alle ore 02:56 Berto Furth <bertofurth@sent.com> ha scritto:

Hi Daniele,

Thanks for collecting all that data. Very thorough. You certainly know what you're doing!

As you said the ARP request packets are making it from the router to the guest

00:17:33.816326 78:81:02:bc:4f:d0 > 52:54:00:12:34:56, ethertype ARP (0x0806), length 60: Request who-has 192.168.0.30 tell 192.168.0.1, length 46

and as you mentioned the router has an ARP entry for the guest, but it seems that IP packets aren't making it.

Is there any possibility that there's a packet filter / firewall of some kind applied on the host, perhaps on the eth0 or br0 interface? I mean, since you've got a different guest working in virtual-box it seems unlikely, but that's all I can think of. I say this because IP Packet filters _tend_ to always pass ARP traffic but stop IP traffic.

Other than the usual generic advice of "can you try another host" or "are you using the latest and greatest version of QEMU" I can't think of anything else to suggest.

Some random ideas...

* Could you try turning IP Forwarding off? You don't need it for bridging.
* Is IPv6 enabled on your router? Can you do an IPv6 ping from the guest to the router?
* Is there any possibility that you could add another IPv4 subnet to your network? So for example add 192.168.9.x/24 to the subnet as well as the existing 192.168.0.x and reconfigure your host and guest to use that network and see what happens?
* Can you boot your host up with a bare bones basic "live CD" / USB key version of linux/ubuntu, re-install QEMU and try again? Perhaps there's some other component of your current host that's blocking IP traffic?
* It is a hard/cabled Ethernet connection right? There's no wifi involved here right? Wifi and guest bridging doesn't work well in my experience.
* Unlikely that this will work but maybe look at turning your host into a router (so keep IP Forwarding on) and create an entirely new bridge interface and subnet for your guest(s) and then see if the host will pass routed traffic. Naturally you'll have to also reconfigure your router to route traffic for the new subnet via the host. I have no idea if that will work given your weird problem but it's something else to try.
* Did you fast on Good Friday? If not maybe that's why it's not working! ;-)

Best of luck Daniele,

Berto.

On Tue, 13 Apr 2021, at 08:25, Daniele Palmisano wrote:
Hi Berto,
Thanks for your reply. I'll add the info requested inline. Please find them below.

Il giorno lun 12 apr 2021 alle ore 01:22 Berto Furth <bertofurth@sent.com> ha scritto:

Hi Daniele,

Is it possible to get a "brctl showmacs br0" on the host while QEMU is running to confirm that the mac address of the guest (which is distinct from the MAC address of the tap0 interface) is showing up in the bridge table? Please also ensure that the MAC address of your router is in the bridge table but on a different port number.

» brctl showmacs br0
port no mac addr is local? ageing timer
1 00:26:86:00:00:00 no 273.57
1 10:40:f3:eb:d2:34 no 246.64
1 4c:e1:73:4a:a7:80 yes 0.00 ETH0
1 4c:e1:73:4a:a7:80 yes 0.00 ETH0
2 52:54:00:12:34:56 no 0.83 GUEST: ETH1
1 78:81:02:bc:4f:d0 no 0.24 ROUTER
2 fe:18:35:a4:9a:85 yes 0.00 TAP0
2 fe:18:35:a4:9a:85 yes 0.00 TAP0

3: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether de:ca:fe:75:61:f1 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.18/24 brd 192.168.0.255 scope global dynamic br0
valid_lft 75841sec preferred_lft 75841sec
inet6 fe80::dcca:feff:fe75:61f1/64 scope link
valid_lft forever preferred_lft forever
13: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff
20: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 1000
link/ether fe:18:35:a4:9a:85 brd ff:ff:ff:ff:ff:ff
inet6 fe80::fc18:35ff:fea4:9a85/64 scope link
valid_lft forever preferred_lft forever

GUEST:

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.30/24 brd 192.168.0.255 scope global eth1
valid_lft forever preferred_lft forever
pi@raspberrypi:~ $

In addition can you post a copy of "ifconfig" as run on the guest?

pi@raspberrypi:~ $ ifconfig
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.30 netmask 255.255.255.0 broadcast 192.168.0.255
ether 52:54:00:12:34:56 txqueuelen 1000 (Ethernet)
RX packets 407 bytes 31908 (31.1 KiB)
RX errors 0 dropped 7 overruns 0 frame 0
TX packets 169 bytes 16776 (16.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 57 base 0x8000 dma 0xff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 0 bytes 0 (0.0 B)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 0 bytes 0 (0.0 B)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

You could also run the command "brctl showstp br0 " on the host to double check that your bridge is properly forwarding traffic on both your ethernet and tap interfaces (although it must be if you can ping the guest and the router from the host).

» brctl showstp br0
br0
bridge id 8000.decafe7561f1
designated root 8000.decafe7561f1
root port 0 path cost 0
max age 20.00 bridge max age 20.00
hello time 2.00 bridge hello time 2.00
forward delay 15.00 bridge forward delay 15.00
ageing time 300.00
hello timer 0.00 tcn timer 0.00
topology change timer 0.00 gc timer 113.50
flags

eth0 (1)
port id 8001 state forwarding
designated root 8000.decafe7561f1 path cost 4
designated bridge 8000.decafe7561f1 message age timer 0.00
designated port 8001 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags

tap0 (2)
port id 8002 state forwarding
designated root 8000.decafe7561f1 path cost 100
designated bridge 8000.decafe7561f1 message age timer 0.00
designated port 8002 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags

You mentioned that you've done some network sniffing. As you try to ping from the guest to the router and vice versa could you post the output of a command like

tcpdump -e -n -i tap0 ether host c4:aa:bb:11:22:33

where "c4"aa:bb:11:22:33" is replaced by the MAC address of the guest (not the mac-address of the tap0 interface)

While Pinging the router from the guest:

» sudo tcpdump -e -n -i tap0 ether host 52:54:00:12:34:56
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
00:17:33.816326 78:81:02:bc:4f:d0 > 52:54:00:12:34:56, ethertype ARP (0x0806), length 60: Request who-has 192.168.0.30 tell 192.168.0.1, length 46
00:17:33.819474 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype ARP (0x0806), length 64: Reply 192.168.0.30 is-at 52:54:00:12:34:56, length 50
00:17:35.616800 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.58805 > 10.0.2.3.53: 708+ A? 3.debian.pool.ntp.org. (39)
00:17:40.655280 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.36607 > 8.8.8.8.53: 59127+ A? 0.debian.pool.ntp.org. (39)
00:17:44.563366 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 1, length 64
00:17:45.577762 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 2, length 64
00:17:45.692291 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.37885 > 10.0.2.3.53: 59127+ A? 0.debian.pool.ntp.org. (39)
00:17:46.576812 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 3, length 64
00:17:47.575970 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 4, length 64
00:17:48.576785 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 5, length 64
00:17:49.576511 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 6, length 64
00:17:50.577048 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 7, length 64
00:17:50.697446 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.36607 > 8.8.8.8.53: 59127+ A? 0.debian.pool.ntp.org. (39)
00:17:51.576683 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 8, length 64
00:17:52.576848 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 9, length 64
00:17:53.575292 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 10, length 64
00:17:54.575357 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 11, length 64
00:17:55.575405 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 832, seq 12, length 64
00:17:55.707270 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.37885 > 10.0.2.3.53: 59127+ A? 0.debian.pool.ntp.org. (39)
00:17:59.795867 78:81:02:bc:4f:d0 > 52:54:00:12:34:56, ethertype ARP (0x0806), length 60: Request who-has 192.168.0.30 tell 192.168.0.1, length 46
00:17:59.803801 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype ARP (0x0806), length 64: Reply 192.168.0.30 is-at 52:54:00:12:34:56, length 50
^C
21 packets captured
21 packets received by filter
0 packets dropped by kernel

While Pinging the guest from the router:

» sudo tcpdump -e -n -i tap0 ether host 52:54:00:12:34:56
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on tap0, link-type EN10MB (Ethernet), capture size 262144 bytes
00:19:21.000358 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.41595 > 8.8.8.8.53: 44124+ A? 1.debian.pool.ntp.org. (39)
00:19:26.007835 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.39188 > 10.0.2.3.53: 44124+ A? 1.debian.pool.ntp.org. (39)
00:19:27.139780 78:81:02:bc:4f:d0 > 52:54:00:12:34:56, ethertype ARP (0x0806), length 60: Request who-has 192.168.0.30 tell 192.168.0.1, length 46
00:19:27.140766 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype ARP (0x0806), length 64: Reply 192.168.0.30 is-at 52:54:00:12:34:56, length 50
^C
4 packets captured
4 packets received by filter
0 packets dropped by kernel

Can we check whether the router is getting an ARP entry for the guest? Assuming it's a linux / openWRT / FreeBSD based router the command will be something like "arp -na" or a router with a cisco style command interface the command will be "show arp". I ask this because there's a chance that packets might be getting from the guest to the router, but packets are unable to make it back to the guest (or vice versa). That might help us to narrow down what's happening.

The router is getting ARP entries for the guest. This can also be seen in the TCPDUMP added above. It seems that packets from the guests are detected but packets from the LAN are not.

Finally you said that virtual-box is working. Is it setup in bridging mode or is it doing NAT? That is, can you ping the guest from the router when you're running virtual box?

Yes. That's correct. From virtual-box it seems to work (running a different type of VM, not the Raspberry PI, of course). It is setup in bridging mode and I can ping the guest from the router.

Additionally I have also run the same tests using a TAP as a back-end network type with the following command:

» sudo qemu-system-arm \
-kernel ./qemu-rpi-kernel/kernel-qemu-4.4.34-jessie \
-append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
-hda 2021-01-11-raspios-buster-armhf-full.img \
-cpu arm1176 -m 256 \
-M versatilepb \
-no-reboot \
-serial stdio \
-net nic,netdev=mynet0 -netdev tap,ifname=vnet0,id=mynet0,script=no,downscript=no

Please also find the outputs below, but the outcome is exactly the same, I am afraid:

» brctl showmacs br0
port no mac addr is local? ageing timer
1 00:26:86:00:00:00 no 200.99
1 10:40:f3:eb:d2:34 no 0.03
1 4c:e1:73:4a:a7:80 yes 0.00 HOST: ETH0
1 4c:e1:73:4a:a7:80 yes 0.00 HOST: ETH0
2 52:54:00:12:34:56 no 1.37 GUEST: ETH1
1 78:81:02:bc:4f:d0 no 1.03 ROUTER
1 94:65:2d:47:a1:c4 no 195.40
2 9a:06:a3:3e:8a:3a yes 0.00 HOST: VNET0
2 9a:06:a3:3e:8a:3a yes 0.00 HOST: VNET0
1 cc:c7:60:51:a2:89 no 5.93

vnet0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 1000
link/ether 9a:06:a3:3e:8a:3a
eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff

GUEST:

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 52:54:00:12:34:56 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.30/24 brd 192.168.0.255 scope global eth1
valid_lft forever preferred_lft forever
pi@raspberrypi:~ $

pi@raspberrypi:~ $ ifconfig
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 192.168.0.30 netmask 255.255.255.0 broadcast 192.168.0.255
ether 52:54:00:12:34:56 txqueuelen 1000 (Ethernet)
RX packets 1375 bytes 106906 (104.4 KiB)
RX errors 0 dropped 38 overruns 0 frame 0
TX packets 495 bytes 41142 (40.1 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 57 base 0x8000 dma 0xff

lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
loop txqueuelen 1 (Local Loopback)
RX packets 59 bytes 5587 (5.4 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 59 bytes 5587 (5.4 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0

» brctl showstp br0
br0
bridge id 8000.decafe7561f1
designated root 8000.decafe7561f1
root port 0 path cost 0
max age 20.00 bridge max age 20.00
hello time 2.00 bridge hello time 2.00
forward delay 15.00 bridge forward delay 15.00
ageing time 300.00
hello timer 0.00 tcn timer 0.00
topology change timer 0.00 gc timer 281.75
flags

eth0 (1)
port id 8001 state forwarding
designated root 8000.decafe7561f1 path cost 4
designated bridge 8000.decafe7561f1 message age timer 0.00
designated port 8001 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags

vnet0 (2)
port id 8002 state forwarding
designated root 8000.decafe7561f1 path cost 100
designated bridge 8000.decafe7561f1 message age timer 0.00
designated port 8002 forward delay timer 0.00
designated cost 0 hold timer 0.00
flags

While Pinging the router from the guest:

» sudo tcpdump -e -n -i vnet0 ether host 52:54:00:12:34:56
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
23:44:22.233333 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.48163 > 8.8.8.8.53: 10283+ A? 3.debian.pool.ntp.org. (39)
23:44:27.251763 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.38857 > 10.0.2.3.53: 10283+ A? 3.debian.pool.ntp.org. (39)
23:44:32.257718 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.48163 > 8.8.8.8.53: 10283+ A? 3.debian.pool.ntp.org. (39)
23:44:34.480035 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 1, length 64
23:44:34.581989 78:81:02:bc:4f:d0 > 52:54:00:12:34:56, ethertype ARP (0x0806), length 60: Request who-has 192.168.0.30 tell 192.168.0.1, length 46
23:44:34.584371 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype ARP (0x0806), length 64: Reply 192.168.0.30 is-at 52:54:00:12:34:56, length 50
23:44:35.478370 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 2, length 64
23:44:36.496464 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 3, length 64
23:44:37.269043 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.38857 > 10.0.2.3.53: 10283+ A? 3.debian.pool.ntp.org. (39)
23:44:37.494945 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 4, length 64
23:44:38.495756 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 5, length 64
23:44:39.495146 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 6, length 64
23:44:40.496194 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 7, length 64
23:44:41.495744 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 8, length 64
23:44:42.314985 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.40402 > 8.8.8.8.53: 54336+ A? 0.debian.pool.ntp.org. (39)
23:44:42.496955 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 9, length 64
23:44:43.496088 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 10, length 64
23:44:44.496148 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 11, length 64
23:44:45.494478 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 12, length 64
23:44:46.496191 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 13, length 64
23:44:47.325046 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.43347 > 10.0.2.3.53: 54336+ A? 0.debian.pool.ntp.org. (39)
23:44:47.496084 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 98: 192.168.0.30 > 192.168.0.1: ICMP echo request, id 1175, seq 14, length 64
23:44:52.335942 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.40402 > 8.8.8.8.53: 54336+ A? 0.debian.pool.ntp.org. (39)
^C
23 packets captured
23 packets received by filter
0 packets dropped by kernel

While Pinging the guest from the router:

» sudo tcpdump -e -n -i vnet0 ether host 52:54:00:12:34:56
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on vnet0, link-type EN10MB (Ethernet), capture size 262144 bytes
23:46:42.683867 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.42361 > 8.8.8.8.53: 50618+ A? 2.debian.pool.ntp.org. (39)
23:46:47.697433 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.48992 > 10.0.2.3.53: 50618+ A? 2.debian.pool.ntp.org. (39)
23:46:52.706566 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.42361 > 8.8.8.8.53: 50618+ A? 2.debian.pool.ntp.org. (39)
23:46:57.716242 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.48992 > 10.0.2.3.53: 50618+ A? 2.debian.pool.ntp.org. (39)
23:47:02.749588 52:54:00:12:34:56 > 78:81:02:bc:4f:d0, ethertype IPv4 (0x0800), length 82: 192.168.0.30.41696 > 8.8.8.8.53: 53931+ A? 3.debian.pool.ntp.org. (39)
^C
5 packets captured
5 packets received by filter
0 packets dropped by kernel

Good luck!!

Berto.

On Mon, 12 Apr 2021, at 01:37, Daniele Palmisano wrote:
Hi,
I am writing as I am having some networking issues while I emulate a Raspberry PI image using QEMU on my Ubuntu 18.04 host machine. I was wondering whether you could give me a hand to troubleshoot the issue.

I would like to use a bridged adapter as a guest network type. Following the official Qemu documentation https://wiki.qemu.org/Documentation/Networking#Tap and an Archlinux section https://wiki.archlinux.org/index.php/QEMU#Bridged_networking_using_qemu-bridge-helper I ran the Rapberry PI using the following command:

sudo qemu-system-arm \
-kernel ./qemu-rpi-kernel/kernel-qemu-4.4.34-jessie \
-append "root=/dev/sda2 panic=1 rootfstype=ext4 rw" \
-hda 2021-01-11-raspios-buster-armhf-full.img \
-cpu arm1176 -m 256 \
-M versatilepb \
-no-reboot \
-serial stdio \
-net nic,netdev=mynet0 -netdev bridge,br=br0,id=mynet0

Before running the `qemu-system-arm` command, on the host, I created a bridge br0 with an IP address (192.168.0.14/24) removing the address previously assigned to the physical interface.
My `ip address` output will look like:

152: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff

157: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic br0
valid_lft 80984sec preferred_lft 80984sec
inet6 fe80::4ee1:73ff:fe4a:a780/64 scope link
valid_lft forever preferred_lft forever

After running the `qemu-system-arm` command, a `tap0` interface will also be created and added into the bridge br0.

XPS-13-7390 /etc/netplan » sudo brctl show
bridge name bridge id STP enabled interfaces
br0 8000.4ce1734aa780 no eth0
tap0

XPS-13-7390 /etc/netplan » ip ad
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
...
152: eth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff
157: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
link/ether 4c:e1:73:4a:a7:80 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.14/24 brd 192.168.0.255 scope global dynamic br0
valid_lft 80720sec preferred_lft 80720sec
inet6 fe80::4ee1:73ff:fe4a:a780/64 scope link
valid_lft forever preferred_lft forever
158: tap0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel master br0 state UNKNOWN group default qlen 1000
link/ether fe:c1:a1:b7:45:bc brd ff:ff:ff:ff:ff:ff
inet6 fe80::fcc1:a1ff:feb7:45bc/64 scope link
valid_lft forever preferred_lft forever

From the host I have got the Internet connectivity.
From the guest OS, It is not able to get an IP address using the DHCP protocol, therefore I am forced to assign a static address to the guest interface (192.168.0.30/24). Nevertheless it shows that the interface is in UNKNOWN state.

From here ( guest machine) I can ping the host but I cannot reach any other machines inside the host LAN. Not even the gateway. Hence, I don't have Internet connectivity either.

In fact, using TCPDUMP tool I can see ICMP requests on the BR0 interface but I can't on the host physical interface ETH0. On the other hand, if I try to ping the guest machine from any other machine inside the LAN, I can see ICMP request packets on the BR0 but not on the tap interface tap0.

IP forwarding is also enabled:

cat /proc/sys/net/ipv4/ip_forward
1

I am trying to troubleshoot this issue for one month now. I really hope I can hear back from you.

As an additional note, my host physical network interface is not an embedded one. I am using an USB adapter with the ethernet cable. If I try to emulate a linux machine with VirtualBox, instead, using the same host network configuration, I can reach the Internet from the Guest machine.

I look forward to hearing from you.
Many Thanks and regards,
Daniel

From:	Berto Furth
Subject:	Re: Issues to set up a tap network backend
Date:	Wed, 14 Apr 2021 12:35:00 +1000
User-agent:	Cyrus-JMAP/3.5.0-alpha0-273-g8500d2492d-fm-20210323.002-g8500d249