CollabNet Guide
This guide will walk you through setting up a Virtual Network for your VMs. This will allow you to route your VM traffic behind a VM (strongly recommended for several reasons), filter web traffic, and prevent access to your local network from the VMs.
Prerequisites
- A few hours of your time
- A host running a Linux distribution
- Basic computer and command line literacy. Nobody is going to hold your hand
Host Preparation
IP Forwarding
First, we're going to enable IP forwarding on your host. This will allow traffic from the VMs to be routed to and from the router. The following command will write this to the sysctl configuration.
echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/router.conf
To apply, either reboot or run the following:
sudo sysctl --system
Bridge configuration
Next, we'll set up a network bridge with multiple TAP interfaces. You can think of a TAP interface as a virtual Ethernet port connected to your VM, and the network bridge as a virtual Ethernet switch connecting them all together. The instructions to do this vary based on your network daemon. For this guide we'll assume (and recommend) you're using systemd-networkd, or netifrc on OpenRC.
We'll also be giving the collabvm group access to the TAPs. If you have yet to create a user for CollabVM, do so now.
Systemd
All network configuration is done in the /etc/systemd/network directory. We'll start by creating the bridge itself. The following simply creates a network bridge named collabnet
/etc/systemd/network/collabnet.netdev
[NetDev]
Name=collabnet
Kind=bridge
Next, we'll create a TAP for our router VM. The following creates a new TAP named ktrouter
/etc/systemd/network/ktrouter.netdev
[NetDev]
Name=ktrouter
Kind=tap
[Tap]
Group=collabvm
Now, you can create a TAP for each VM. The following adds a TAP named ktvm1. To add more, repeat the following, creating files named ktvm2, ktvm3, and so on. Make sure to also change the TAP name inside the file to match this.
/etc/systemd/network/ktvm1.netdev
[NetDev]
Name=ktvm1
Kind=tap
[Tap]
Group=collabvm
Finally, we connect all of our TAPs to the collabnet bridge.
/etc/systemd/network/collabnet.network
[Match]
Name=kt*
[Network]
Bridge=collabnet
Finally, you can either reboot or run the following to reload the network configuration
sudo systemctl restart systemd-networkd
OpenRC
TODO
Setting up the router
Now for the """fun""" part. We're going to set up a router VM, responsible for routing all traffic from the VMs. You'll want to give it two network adapters, one for the WAN (the internet) and another LAN (your VMs, the collabnet bridge).
The router will be running Debian. You can either netboot it from within QEMU (Press Ctrl+B when prompted on boot, enter dhcp && boot http://boot.netboot.xyz) or download an ISO and mount it. Your choice.
Here is an example QEMU start command for the router VM. You'll need to create the disk image and adjust paths.
sudo -u collabvm qemu-system-x86_64 \
-accel kvm \
-cpu host \
-m 2G \
-hda /srv/collabvm/router/router.qcow2 \
-netdev user,id=wan -device virtio-net,netdev=wan \
-netdev tap,id=lan,ifname=ktrouter,script=no,downscript=no -device virtio-net,netdev=lan \
-vnc 127.0.0.1:10
With this command, you can SSH forward and VNC to port 5910. When installing Debian, you can accept defaults, although I recommend not using a desktop environment on your router.
Initial configuration
Once you boot to a command line, the first thing we'll do is remove the builtin ifupdown network daemon and use systemd-networkd, as it's much easier to manage.
sudo apt-get purge -y ifupdown
sudo ip addr flush
sudo systemctl enable --now systemd-networkd
We'll then disable systemd-resolved and set up a static DNS config
sudo systemctl disable --now systemd-resolved
sudo systemctl mask systemd-resolved
sudo rm /etc/resolv.conf # remove the symlink
sudo tee /etc/resolv.conf <<EOF
nameserver 1.1.1.1
nameserver 1.0.0.1
EOF
Note that the internet will die on the router. This is because systemd-networkd is not configured by default. First, let's figure out our interface names using the command ip a. For me, the WAN interface was ens3 and the LAN was ens4. This may vary however if you're using the QEMU command above the WAN interface will appear first in the list. For the rest of the guide I will be assuming the above interface names. Make sure to change them if yours are different.
Let's first configure the WAN interface to use DHCP.
/etc/systemd/network/wan.network
[Match]
Name=ens3
[Network]
DHCP=ipv4
Then, we'll assign the LAN interface the static ip of 192.168.1.1
/etc/systemd/network/lan.network
[Match]
Name=ens4
[Network]
Address=192.168.1.1/24
You can then reload the network configuration:
sudo systemctl restart systemd-networkd
If all went well, you should be able to access the internet (test with ping google.com)
Lastly, set some sysctl values:
sudo tee /etc/sysctl.d/router.conf <<EOF
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
net.ipv4.ip_forward = 1
EOF
nftables
Now, we can set up basic routing. First, install some required packages
sudo apt-get install -y nftables dnsmasq curl
Now, download our nftables config template. This sets up a simple router. We'll expand on it later.
sudo curl 'https://computernewb.com/~elijah/nftables.conf' -o /etc/nftables.conf
If your WAN and LAN IPs differ from the examples above, edit nftables.conf and change them at the top.
dnsmasq
Next, copy our DNSMASQ config. DNSMASQ is responsible for DHCP and DNS.
sudo curl 'https://computernewb.com/~elijah/dnsmasq.conf' -o /etc/dnsmasq.conf
There are a few instances of ens4 you'll need to change if that's not your WAN interface. You can also change the instance of collabnet.local if you want to use a different hostname.
There are also some examples of static leases. You can either use the MAC addresses provided in the examples on your VMs, or change them. You need to add a static lease to dnsmasq.conf for each VM you have or it won't be able to connect to the internet.
Start it up
The following commands should get all the router components up and running
sudo systemctl enable --now dnsmasq nftables
sudo nft -f /etc/nftables.conf
Testing it all out
If all went well, you should now have a basic working router. To test this, start up one of your VMs on its TAP. You can do this by removing any -net, -netdev, or -netdev arguments from the QEMU command, and adding the following:
-netdev tap,id=lan,ifname=ktvm1,script=no,downscript=no -device virtio-net,netdev=lan,mac=c0:11:ab:69:42:01
Note that the ifname should correspond to the TAPs added earlier and should be unique per VM. The MAC address should be unique to each VM and MUST be given a static lease in dnsmasq.conf.
If everything went well, the VM should obtain an IP on boot and be able to access the internet. If not, you can try to troubleshoot or join our discord and create a post in #support, and we can try to help.