Re: [Qemu-discuss] Connecting two qemu VMs

[Mike Lovell]

On 11/05/2014 05:15 AM, Dominik G. wrote:
> Is it possible to connect two qemu instances with each other via TCP
> or socket and have both connected with the host network using only
> user networking (SLIRP)?
> Basically like
> http://wiki.qemu.org/Documentation/Networking#User_Networking_.28SLIRP.29
> but with a second qemu VM:
>
> +-----------------+        +-----------------+
> |  qemu Process 1 |        |  qemu Process 2 |
> |  +---------+    |        |  +---------+    |
> |  |         |    |        |  |         |    |
> |  | Guest 1 |    |        |  | Guest 2 |    |
> |  |         |    |        |  |         |    |
> |  +----+----+    |        |  +----+----+    |
> |       |         |        |       |         |
> |       |         |        |       |         |
> | =+====+===+========TCP/==========+         |
> |  |        |     | Socket |                 |
> |  |       DNS    |        +-----------------+
> |  |              |
> +--|--------------+
>     |
> Host network
>
> I am able to do this with a third qemu VM which acts as a router for
> the two VMs and I can get two qemu VMs to talk to each other but I
> cannot get the one VM to use the built-in gateway and DNS of the
> second one.
>
> I know that there are ways other than SLIRP which can achieve this
> but I want to be able to set this up without root access and I don't
> care about performance since this setup is just for testing
> purposes. Right now I'm just wondering if I can simplify my setup
> leaving out the third router vm and just use the gateway/dns/routing
> that qemu has already built in.

i can think of two possible ways to do this. the first way to is use 
multiple -net options with the vlan set for guest 1. something like the 
following options to the qemu processes.

Guest1: '-net nic,vlan=0,macaddr=52:54:00:01:12:23 -net user,vlan=0 -net 
socket,vlan=0,listen=localhost:8765'
Guest2: '-net nic,macaddr=52:54:00:02:12:34 -net 
socket,connect=localhost:8765'

this uses the older 'vlan' functionality in qemu. it isn't 802.1q vlans. 
only an internal construct to qemu to specify which network guest 
devices and host devices get connected together. these act as a ethernet 
hub internally in that traffic from one device goes to all others. this 
behavior allows for more than 2 to be connected and communicate. packets 
from Guest2 will pass through to the qemu process on Guest1 and be 
passed to Guest1 as well as to the slirp stack. this also means you need 
to specify the macaddr option with unique values so that the two guests 
aren't fighting over the same mac address.

there are a couple downsides to this approach. the -net functionality is 
considered deprecated. the other downside is that if the qemu process 
for Guest1 exits then Guest2 will no longer be able to talk out. It will 
work if qemu is running for Guest1 but Guest1 itself is hung, like a 
kernel panic or is booting.

the second way i can think of to do this is to use vde and slirpvde. 
this method requires some additional steps before the qemu processes are 
started. a vde software ethernet switch has to be created first and then 
a slirpvde process connected to it which will provide the IP gateway 
features. After those are started, then the qemu processes can be 
connected to the vde switch. these steps should take care of it.

'vde_switch -d -sock /tmp/sw1 -M /tmp/sw1.mgmt'
'slirpvde -d -s /tmp/sw1 -q'
Guest1: '-net nic,macaddr=52:54:00:01:12:23 -net vde,sock=/tmp/sw1'
Guest2: -'net nic,macaddr=52:54:00:02:12:23 -net vde,sock=/tmp/sw1'

the first two are full commands and the second two are options for your 
qemu-system commands.

there are some downsides to this approach as well. the biggest is that 
it seems the some major distributions either don't have support for vde. 
redhat based distributions don't even provide it and ubuntu stopped 
compiling qemu with support for it since 13.10. it seems debian still 
has support for it (install the vde2 package) and i think arch has 
support. i don't know about suse based distributions but i would guess 
not. this method may require building several components from source but 
has the advantage that it doesn't require qemu for Guest1 to be running 
for Guest2 to talk on the network.

of course, there is also the option of using libvirt to manage your 
guests. this is probably the most flexible and best supported option. i 
will defer to libvirt documentation on the exact method of accomplishing 
this with libvirt but i would recommend going with that method if you can.

mike