Re: [Qemu-discuss] Port-forwarding to a QEMU/KVM VM

[Jakob Bohm]

On 21/09/2018 08:06, Jaap Winius wrote:
> Hi folks,
>
> For years I've maintained Debian Linux servers that run QEMU/KVM 
> virtual machines along with ppp, bridge-utils (brctl) and iptables. In 
> these cases it is simple to configure the latter to forward packets 
> from the Internet, arriving on interface ppp0, over to VMs on the 
> internal bridged interface, br0. This interface is configured like:
>
>   auto enp35s0
>   iface enp35s0 inet manual
>
>   auto br0
>   iface br0 inet static
>         address 10.1.1.5/24
>         gateway 10.1.1.1
>         bridge_ports enp35s0
>         bridge_stp off
>         bridge_fd 0
>         bridge_ageing 0
>         bridge_maxwait 2
>
> The relevant iptables rules I use to forward HTTPS traffic on to the 
> VM, 10.1.1.10, look like:
>
>   iptables -t nat -A PREROUTING -i ppp0 \
>     -p tcp --dport 443 -j DNAT --to 10.1.1.10:443
>
>   iptables -A FORWARD -i ppp0 \
>     -p tcp -d 10.1.1.10 --dport 443 --syn -m state --state NEW -j ACCEPT
>
> However, this forwarding configuration stopped working after ppp and 
> iptables were moved to a physically separate gateway machine. Now the 
> packets from outside are still forwarded on to the VM (that uses a 
> virtio network interface), which responds, but the replies never make 
> it out of the bridged network segment. Using tcpdump, the reply 
> packets can be detected on the VM and the host server, but not on the 
> gateway.
>
> How can port-forwarding functionality best be restored in this case?
>
> To be honest, this problem seems more like something to do with brctl 
> than with QEMU/KVM, but as brctl appears to be the bridge of choice in 
> these environments, surely someone here has already encountered this 
> problem and found a fix for it. And as I'm rather stumped on this one, 
> I'd be very grateful if someone were to share their solution here.
Check the following:

Your VM MAC addresses are unique network wide and don't have
the multicast bit set (so packets can be routed correctly by
your layer 2 Ethernet switches).

Your IP tables allow ARP requests (for IPv4) and neighbor
discovery ICMPv6 (for IPv6) to travel between your VMs and
the physical network (so other machines, including the
gateway can find the IP-to-MAC mapping for your VMs).

Your physical Ethernet switch is configured to accept that
the cable to your machine behaves like a cable to another
Ethernet switch (this is the default for most switches with
a few annoying exceptions).

If you have a spare machine with two ethernet ports, configure
it to bridge the two ports with brctl and run wireshark on that,
then "splice" it into the connection between your machine and the
physical switch, then again between the physical switch and the
gateway.  Look if the packets are dropped on one side of the
physical switch or the other.  Also look for any Layer 2 and
layer 3 packets that negotiate the routing of IP and MAC
addresses.


Enjoy

Jakob
--
Jakob Bohm, CIO, Partner, WiseMo A/S.  https://www.wisemo.com
Transformervej 29, 2860 Søborg, Denmark.  Direct +45 31 13 16 10
This public discussion message is non-binding and may contain errors.
WiseMo - Remote Service Management for PCs, Phones and Embedded