I wanted to start hosting VMs on my Hetzner root server. For IPv6 this should be
straight-forward since my server has a
/64 subnet, for IPv4 I need NAT.
If you know what to do, both is extremely simple. But being new to networking and
iptables I had to try a few things. Thus, I will use this blog post as personal
note-keeping. What I am doing might not be perfect, so feel free to continue your
research to find better approaches.
IPv6 Bridge Setup
I like to run my VMs on a bridge network. It makes networking from inside the VM
quite easy and should work well with IPv6 in this case. Thus, the first step is to
setup a bridge network on the actual physical network. My Hetzner server came with
systemd-networkd configuration. In the following I have replaced
all real IP addresses with documentation IP addresses (
# enp2s0.network ### Hetzner Online GmbH installimage [Match] Name=enp2s0 [Network] Address=2001:db8::/64 Gateway=192.0.2.33 Gateway=fe80::1 [Address] Address=192.0.2.1 Peer=192.0.2.33/32
Establishing a bridge device for this connection is actually quite simple.
The only important step is to assign the original MAC address of the network
card to the bridge device. The existing
enp2s0.network will basically become
br0.network and we have to create two new files
# br0.network [Match] Name=br0 [Network] Address=2001:db8::/64 Gateway=192.0.2.33 Gateway=fe80::1 [Address] Address=192.0.2.1 Peer=192.0.2.33/32
In the netdev file we assign the MAC address (again real values replaced with documentation values):
# br0.netdev [NetDev] Name=br0 Kind=bridge MACAddress=00:00:5e:00:53:00
And finally we define
enp2s0 to be the slave of
[Match] Name=enp2s0 [Network] Bridge=br0
With these changes you can reload the network configuration with
systemctl restart systemd-networkd.
Guest IPv6 Setup
Next, we need to setup the IPv6 address inside the guest. I opted for static configuration, because during the startup phase I didn’t want to additionally handle dynamic components like DHCP servers or Router Advertisment daemons.
Since I have an Ubuntu server for Jitsi, I had to learn the basics of netplan. If you’re using another network manager, do it your way. The configuration file should be self explanatory. The configured DNS servers are Hetzner’s nameservers.
network: version: 2 ethernets: ens3: dhcp4: false dhcp6: false addresses: - 2001:db8::1234/64 gateway6: fe80::1 nameservers: addresses: - 2a01:4f8:0:1::add:9898 - 2a01:4f8:0:1::add:1010 - 2a01:4f8:0:1::add:9999
This assigns the IP address
2001:db8::1234 to the interface
the guest. I tried both
fe80::1 (the same gateway as the host) and
fe80::d63d:7eff:fee3:18d (the host’s link-local address) as gateways, both
worked fine. I have to admit I do not know which is the technically correct
Interestingly even though I have a
DROP policy on forwarding in my iptables
rules, connection to the outside world works just fine. On the other hand,
connecting from the outside world to the VM’s public IP address only works
with an ACCEPT rule in the FORWARD table. This is something I will have to
further investigate, because I don’t understand it.
Hetzner only gives us a single IPv4 for the root server. This means - if we don’t want to request additional IPv4 addresses - we will have to use NAT. I will only be using this for outgoing connections to web sites that don’t have IPv6 configured. I want to avoid NAT’ing of incoming connections (and thus port related fun) for as long as possible.
According to my research the way to do NAT with QEMU
(probably also for other use cases) is to create a second bridge network, which
is not connected to the physical interface. We can assign the gateway IP
192.168.0.1) to the bridge interface. Then we attach
slave devices which we can assign to the VMs. Inside the VMs we can then
assign static public IPv4 addresses from the
systemd-networkd we can create this interface with a
[NetDev] Name=br-vm Kind=bridge
And with a
br-vm.network file (without
ConfigureWithoutCarrier systemd will not
configure the IP address
on this carrier-less interface):
[Match] Name=br-vm [Network] Address=192.168.0.1/24 ConfigureWithoutCarrier=yes
If we want to create a Qemu guest with a private IPv4 address, we add a new
tuntap device as slave to this bridge and assign the
tuntap device to
the Qemu VM.
With the bridge and
tuntap devices existing we
can setup the IPv4 configuration inside the guest:
network: version: 2 ethernets: ens4: dhcp4: false dhcp6: false addresses: - 192.168.0.2/24 gateway4: 192.168.0.1 nameservers: addresses: - 18.104.22.168 - 22.214.171.124 - 126.96.36.199
In order for the IPv4 NAT to work we need to enable forwarding on the host.
According to my understanding this requires both changes to
sysctl as well
iptables (if you have a standard rejection policy).
To allow forwarding on the Linux machine we can set it with sysctl:
sysctl -w net.ipv4.ip_forward=1
For persistent configuration add it to a file inside
Regarding iptables I have allowed forwarded traffic between
# Allow FORWARD for VMs iptables -A FORWARD -o br0 -i brvm -j ACCEPT iptables -A FORWARD -o brvm -i br0 -j ACCEPT iptables -A FORWARD -j REJECT --reject-with icmp-port-unreachable
We also need to set masquerading for packets coming from the VMs, so that the host will replace the VM’s IP address inside these packets with its own IP address:
iptables -t nat -A POSTROUTING -s 192.168.0.0/24 ! -d 192.168.0.0/24 -j MASQUERADE
I don’t particularly like the match on the interface name in netplan. I didn’t check,
but I think that
ens4 might also be in reverse order
if defined in reverse order
in the Qemu argument list. Matching on the MAC address should work much better
and is possible with the macaddress match in netplan:
match: macadress: 00:00:5e:00:53:00