I wanted to start hosting VMs on my Hetzner root server. For IPv6 this should be straight-forward since my server has a /64 subnet, for IPv4 I need NAT. If you know what to do, both is extremely simple. But being new to networking and iptables I had to try a few things. Thus, I will use this blog post as personal note-keeping. What I am doing might not be perfect, so feel free to continue your research to find better approaches.

IPv6 Bridge Setup

I like to run my VMs on a bridge network. It makes networking from inside the VM quite easy and should work well with IPv6 in this case. Thus, the first step is to setup a bridge network on the actual physical network. My Hetzner server came with an existing systemd-networkd configuration. In the following I have replaced all real IP addresses with documentation IP addresses (192.0.2.33/24 and 2001:db8::/32).

# enp2s0.network

### Hetzner Online GmbH installimage
[Match]
Name=enp2s0

[Network]
Address=2001:db8::/64
Gateway=192.0.2.33
Gateway=fe80::1

[Address]
Address=192.0.2.1
Peer=192.0.2.33/32

Establishing a bridge device for this connection is actually quite simple. The only important step is to assign the original MAC address of the network card to the bridge device. The existing enp2s0.network will basically become the new br0.network and we have to create two new files br0.netdev and enp2s0.network:

# br0.network

[Match]
Name=br0

[Network]
Address=2001:db8::/64
Gateway=192.0.2.33
Gateway=fe80::1

[Address]
Address=192.0.2.1
Peer=192.0.2.33/32

In the netdev file we assign the MAC address (again real values replaced with documentation values):

# br0.netdev

[NetDev]
Name=br0
Kind=bridge
MACAddress=00:00:5e:00:53:00

And finally we define enp2s0 to be the slave of br0:

[Match]
Name=enp2s0

[Network]
Bridge=br0

With these changes you can reload the network configuration with systemctl restart systemd-networkd.

Guest IPv6 Setup

Next, we need to setup the IPv6 address inside the guest. I opted for static configuration, because during the startup phase I didn’t want to additionally handle dynamic components like DHCP servers or Router Advertisment daemons.

Since I have an Ubuntu server for Jitsi, I had to learn the basics of netplan. If you’re using another network manager, do it your way. The configuration file should be self explanatory. The configured DNS servers are Hetzner’s nameservers.

network:
  version: 2
  ethernets:
    ens3:
      dhcp4: false
      dhcp6: false
      addresses:
        - 2001:db8::1234/64
      gateway6: fe80::1
      nameservers:
        addresses:
          - 2a01:4f8:0:1::add:9898
          - 2a01:4f8:0:1::add:1010
          - 2a01:4f8:0:1::add:9999

This assigns the IP address 2001:db8::1234 to the interface ens3 inside the guest. I tried both fe80::1 (the same gateway as the host) and fe80::d63d:7eff:fee3:18d (the host’s link-local address) as gateways, both worked fine. I have to admit I do not know which is the technically correct selection.

Interestingly even though I have a DROP policy on forwarding in my iptables rules, connection to the outside world works just fine. On the other hand, connecting from the outside world to the VM’s public IP address only works with an ACCEPT rule in the FORWARD table. This is something I will have to further investigate, because I don’t understand it.

IPv4 Setup

Hetzner only gives us a single IPv4 for the root server. This means - if we don’t want to request additional IPv4 addresses - we will have to use NAT. I will only be using this for outgoing connections to web sites that don’t have IPv6 configured. I want to avoid NAT’ing of incoming connections (and thus port related fun) for as long as possible.

According to my research the way to do NAT with QEMU (probably also for other use cases) is to create a second bridge network, which is not connected to the physical interface. We can assign the gateway IP address (e.g. 192.168.0.1) to the bridge interface. Then we attach tuntap slave devices which we can assign to the VMs. Inside the VMs we can then assign static public IPv4 addresses from the 192.168.0.0/24 network.

In systemd-networkd we can create this interface with a br-vm.netdev file:

[NetDev]
Name=br-vm
Kind=bridge

And with a br-vm.network file (without ConfigureWithoutCarrier systemd will not configure the IP address on this carrier-less interface):

[Match]
Name=br-vm

[Network]
Address=192.168.0.1/24
ConfigureWithoutCarrier=yes

If we want to create a Qemu guest with a private IPv4 address, we add a new tuntap device as slave to this bridge and assign the tuntap device to the Qemu VM.

With the bridge and tuntap devices existing we can setup the IPv4 configuration inside the guest:

network:
  version: 2
  ethernets:
    ens4:
      dhcp4: false
      dhcp6: false
      addresses:
        - 192.168.0.2/24
      gateway4: 192.168.0.1
      nameservers:
        addresses:
          - 213.133.98.98
          - 213.133.99.99
          - 213.133.100.100

IPv4 Forwarding

In order for the IPv4 NAT to work we need to enable forwarding on the host. According to my understanding this requires both changes to sysctl as well as to iptables (if you have a standard rejection policy).

To allow forwarding on the Linux machine we can set it with sysctl:

sysctl -w net.ipv4.ip_forward=1

For persistent configuration add it to a file inside /etc/sysctl.d/.

Regarding iptables I have allowed forwarded traffic between br0 and brvm:

# Allow FORWARD for VMs
iptables -A FORWARD -o br0 -i brvm -j ACCEPT
iptables -A FORWARD -o brvm -i br0 -j ACCEPT
iptables -A FORWARD -j REJECT --reject-with icmp-port-unreachable

We also need to set masquerading for packets coming from the VMs, so that the host will replace the VM’s IP address inside these packets with its own IP address:

iptables -t nat -A POSTROUTING -s 192.168.0.0/24 ! -d 192.168.0.0/24 -j MASQUERADE

Open Issues

I don’t particularly like the match on the interface name in netplan. I didn’t check, but I think that ens3 and ens4 might also be in reverse order if defined in reverse order in the Qemu argument list. Matching on the MAC address should work much better and is possible with the macaddress match in netplan:

match:
  macadress: 00:00:5e:00:53:00

Comments

The comments section is a test at the moment. I might decide to remove it again. If you have any questions you can also send an e-mail to blog@stefan-koch.name. The software (Commento) is running on my own server.