CollabNet Guide

This guide will walk you through setting up a Virtual Network for your VMs. This will allow you to route your VM traffic behind a VPN (strongly recommended for several reasons), filter web traffic, and prevent access to your local network from the VMs.

Prerequisites

• An hour or two of your time
• A host running a Linux distribution
• Basic computer and command line literacy. Nobody is going to hold your hand

Host Preparation

IP Forwarding

First, we're going to enable IP forwarding on your host. This will allow traffic from the VMs to be routed to and from the router. The following command will write this to the sysctl configuration.

echo 'net.ipv4.ip_forward = 1' | sudo tee /etc/sysctl.d/router.conf

To apply, either reboot or run the following:

sudo sysctl --system

Bridge configuration

Next, we'll set up a network bridge with multiple TAP interfaces. You can think of a TAP interface as a virtual Ethernet port connected to your VM, and the network bridge as a virtual Ethernet switch connecting them all together. The instructions to do this vary based on your network daemon. For this guide we'll assume (and recommend) you're using systemd-networkd, or netifrc on OpenRC.

We'll also be giving the collabvm group access to the TAPs. If you have yet to create a user for CollabVM, do so now.

Systemd

All network configuration is done in the /etc/systemd/network directory. We'll start by making sure systemd-networkd is enabled and started.

sudo systemctl enable --now systemd-networkd

Next, create the bridge. The following simply creates a network bridge named collabnet

/etc/systemd/network/collabnet.netdev

[NetDev]
Name=collabnet
Kind=bridge

Next, we'll create a TAP for our router VM. The following creates a new TAP named ktrouter

/etc/systemd/network/ktrouter.netdev

[NetDev]
Name=ktrouter
Kind=tap

[Tap]
Group=collabvm

Now, you can create a TAP for each VM. The following adds a TAP named ktvm1. To add more, repeat the following, creating files named ktvm2, ktvm3, and so on. Make sure to also change the TAP name inside the file to match this.

/etc/systemd/network/ktvm1.netdev

[NetDev]
Name=ktvm1
Kind=tap

[Tap]
Group=collabvm

Finally, we connect all of our TAPs to the collabnet bridge.

/etc/systemd/network/collabnet.network

[Match]
Name=kt*

[Network]
Bridge=collabnet

Finally, you can either reboot or run the following to reload the network configuration

sudo systemctl restart systemd-networkd

To make sure the bridge is started properly at system boot, run sudo crontab -e and add the following line to the bottom of the file that opens, then save it:

@reboot sleep 10; ip link set collabnet up

OpenRC

TODO

Setting up the router

Now for the """fun""" part. We're going to set up a router VM, responsible for routing all traffic from the VMs. You'll want to give it two network adapters, one for the WAN (the internet) and another LAN (your VMs, the collabnet bridge).

The router will be running Debian. You can either netboot it from within QEMU (Press Ctrl+B when prompted on boot, enter dhcp && boot http://boot.netboot.xyz) or download an ISO and mount it. Your choice.

Here is an example QEMU start command for the router VM. You'll need to create the disk image and adjust paths.

sudo -u collabvm qemu-system-x86_64 \
-accel kvm \
-cpu host \
-m 2G \
-hda /srv/collabvm/router/router.qcow2 \
-netdev user,id=wan -device virtio-net,netdev=wan \
-netdev tap,id=lan,ifname=ktrouter,script=no,downscript=no -device virtio-net,netdev=lan \
-vnc 127.0.0.1:10

With this command, you can SSH forward and VNC to port 5910. When installing Debian, you can accept defaults, although I recommend not using a desktop environment on your router.

Initial configuration

Once you boot to a command line, the first thing we'll do is remove the builtin ifupdown network daemon and use systemd-networkd, as it's much easier to manage.

sudo apt-get purge -y ifupdown
sudo systemctl enable --now systemd-networkd

Note that the internet will die on the router. This is because systemd-networkd is not configured by default. First, let's figure out our interface names using the command ip a. For me, the WAN interface was ens3 and the LAN was ens4. This may vary however if you're using the QEMU command above the WAN interface will appear first in the list. For the rest of the guide I will be assuming the above interface names. Make sure to change them if yours are different.

Let's first configure the WAN interface to use DHCP.

/etc/systemd/network/wan.network

[Match]
Name=ens3

[Network]
DHCP=ipv4

Then, we'll assign the LAN interface the static ip of 192.168.1.1

/etc/systemd/network/lan.network

[Match]
Name=ens4

[Network]
Address=192.168.1.1/24

You can then reload the network configuration:

sudo systemctl restart systemd-networkd

If all went well, you should be able to access the internet (test with ping google.com)

Lastly, set some sysctl values:

sudo tee /etc/sysctl.d/router.conf <<EOF
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
net.ipv4.ip_forward = 1
EOF

Apply them:

sudo sysctl --system

nftables

Now, we can set up basic routing. First, install some required packages

sudo apt-get install -y nftables dnsmasq curl openresolv

Now, download our nftables config template. This sets up a simple router. We'll expand on it later.

sudo curl 'https://computernewb.com/~elijah/nftables.conf' -o /etc/nftables.conf

If your WAN and LAN IPs differ from the examples above, edit nftables.conf and change them at the top.

dnsmasq

Next, copy our DNSMASQ config. DNSMASQ is responsible for DHCP and DNS.

sudo curl 'https://computernewb.com/~elijah/dnsmasq.conf' -o /etc/dnsmasq.conf

There are a few instances of ens4 you'll need to change if that's not your WAN interface. You can also change the instance of collabnet.local if you want to use a different hostname.

There are also some examples of static leases. You can either use the MAC addresses provided in the examples on your VMs, or change them. You need to add a static lease to dnsmasq.conf for each VM you have or it won't be able to connect to the internet.

Start it up

The following commands should get all the router components up and running

sudo systemctl enable --now dnsmasq nftables
sudo nft -f /etc/nftables.conf

Testing it all out

If all went well, you should now have a basic working router. To test this, start up one of your VMs on its TAP. You can do this by removing any -net, -netdev, or -netdev arguments from the QEMU command, and adding the following:

-netdev tap,id=lan,ifname=ktvm1,script=no,downscript=no -device virtio-net,netdev=lan,mac=c0:11:ab:69:42:01

Note that the ifname should correspond to the TAPs added earlier and should be unique per VM. The MAC address should be unique to each VM and MUST be given a static lease in dnsmasq.conf.

If everything went well, the VM should obtain an IP on boot and be able to access the internet. If not, you can try to troubleshoot or join our discord and create a post in #support, and we can try to help.

Optional router configuration

The following is not strictly required, however is recommended.

VPN

If you don't want users to be able to make traffic from your IP address (you almost definitely do not), you should set up a VPN, for all your users' traffic to be run through. You'll need a VPN that supports wireguard (and additionally you'll want to make sure that this usecase is allowed by their Terms of Service). Here are a few:

• ProtonVPN (Free plan allows WireGuard, premium plan $10/month)
• CloudFlare WARP: You can use the wgcf third party CLI tool to generate a WireGuard profile. Mileage may vary.
• Mullvad ($5 a month)

Once you have your VPN picked out, register an account and generate a WireGuard profile. This varies by VPN but its usually in the Downloads section or its own category. It should give you a wireguard configuration either as a downloadable file or to copy and paste.

Next, install wireguard into your router:

sudo apt-get install wireguard-tools

Once this is installed, paste the contents of your WireGuard configuration into /etc/wireguard/wg.conf. An example file looks like this:

[Interface]
PrivateKey = ThisIsAnExampleDontActuallyUseThis=
Address = 10.65.2.87/32,fc00:bbbb:bbbb:bb01::2:256/128
DNS = 10.64.0.1
[Peer]
PublicKey = gH/ThisIsAnExampleDontActuallyUseThis=
Endpoint = 1.1.1.1:51820
AllowedIPs = 0.0.0.0/0, ::0/0

With that all set, you can now enable the VPN with the following command:

sudo systemctl enable --now wg-quick@wg

If all went well, you should now be connected to the VPN:

$ curl ipinfo.io/what-is-my-ip
{
  "ip": "143.244.47.86",
  "hostname": "unn-143-244-47-86.datapacket.com",
  "city": "Weehawken",
  "region": "New Jersey",
  "country": "US",
  "loc": "40.7696,-74.0204",
  "org": "AS212238 Datacamp Limited",
  "postal": "07086",
  "timezone": "America/New_York",
  "readme": "https://ipinfo.io/missingauth"
}

The last step is to connect the VMs to the VPN. First, get your private IP on the vpn network:

$ ip a
(...)
7: wg: <POINTOPOINT,NOARP,UP,LOWER_UP> mtu 1420 qdisc noqueue state UNKNOWN group default qlen 1000
    link/none
    inet 10.65.2.87/32 <---- THIS RIGHT HERE scope global wg

As we can see, our private IP in this example is 10.65.2.87. All we have to do is open nftables.conf and set WAN to wg, and SNAT to this private IP:

/etc/nftables.conf
(...)
define SNAT = 10.65.2.87
(...)
define WAN = wg

Reload your nftables configuration:

sudo nft -f /etc/nftables.conf

If all went well, your VMs should now be connected to the VPN

E2Guardian

E2Guardian can be used to implement network filtering on your VM. It supports transparent HTTP and HTTPS filtering making it ideal for a setup like CollabVM where the client can't be trusted to always use a proxy or DNS server. Here's how to set it up.

Installation

First is to install e2guardian. Run the following command:

wget https://e2guardian.numsys.eu/v5.5/e2debian_bookworm_V5.5.5r_20240509.deb # IF THERE'S A NEWER VERSION AVAILABLE AT https://e2guardian.numsys.eu/, USE THAT INSTEAD
sudo apt-get install ./e2debian_bookworm_V5.5.5_20240209.deb

Set e2guardian to automatically restart in case something fails:

1. Run sudo systemctl edit e2guardian
2. Add the following at the top of the file where indicated (read the comments!):

   [Service]
   Restart=always
   

3. Save the file

Install some other dependencies:

sudo apt-get install git curl tar sed

Configuration

Now we need to do some configuration. The e2guardian config directory is not readable by a normal user, so shell in as root.

$ sudo -i
# cd /etc/e2guardian

Create a directory for generated certificates, and give e2guardian permissions:

mkdir -p private/generatedcerts
chown e2guardian:e2guardian private/generatedcerts

Generate the required keys and CA certificate for SSL MITM:

openssl genrsa 4096 > private/ca.key
openssl genrsa 4096 > private/cert.key
openssl req -new -x509 -days 3650 -sha256 -key private/ca.key -out private/ca.pem

The last command will ask for some information, you can leave most of it blank and fill what you want.

Next, some configuration. Open e2guardian.conf in an editor, then find and set the following values:

transparenthttpsport = 8443
enablessl = on
sslcertificatepath = '/etc/ssl/certs'
useoriginalip = on
checkip = 192.168.1.1

Now open e2guardianf1.conf in an editor, then find and set the following values:

## COMMENT OUT THESE LINES:
# storyboard = '/etc/e2guardian/examplef1.story'   # comment out for production
# .Define LISTDIR </etc/e2guardian/lists/example.group>  # comment out for production 

sslmitm = on

Copy examplef1.story to group1.story:

cp -p examplef1.story group1.story

Filter lists

Now you can download the collabnet filter lists:

cd /etc/e2guardian/lists
git clone --depth 1 https://git.computernewb.com/collabvm/e2guardian.git group1

Download the ut1 and oisd filter lists:

cd group1
./getall.sh

Updating the filter lists

To enable automatic updates of the filter lists, run sudo crontab -e and add the following to the bottom of the file:

0 * * * * /etc/e2guardian/lists/group1/getall.sh >/dev/null 2>&1

This will check for filter list updates once an hour.

Start it up

You should now be able to start e2guardian without error using the following command:

sudo systemctl enable --now e2guardian

You can confirm that it's working by seeing if it's listening on ports 8080 and 8443:

$ sudo ss -tulnp
(...)
tcp   LISTEN 0      256                                  0.0.0.0:8080       0.0.0.0:*    users:(("e2guardian",pid=168580,fd=5))
tcp   LISTEN 0      256                                  0.0.0.0:8443       0.0.0.0:*    users:(("e2guardian",pid=168580,fd=7))
(...)

Routing the VM traffic

We can now set a firewall rule to redirect all VM traffic through e2guardian. Open /etc/nftables.conf and uncomment the following lines near the end:

iifname $LAN ip daddr != 192.168.1.0/24 tcp dport 80 redirect to :8080
iifname $LAN ip daddr != 192.168.1.0/24 tcp dport 443 redirect to :8443

Reload nftables:

sudo nft -f /etc/nftables.conf

Test it out

First, put /etc/e2guardian/private/ca.pem onto one of your VMs. This is usually done through installing nginx onto the router and copying ca.pem into the webroot. Next, install ca.pem as a Trusted Root Certification Authority (on Windows, you need to change the file extension to .crt for it to be recognized.)

If all went well, you should be able to browse to websites and see that the SSL certificate was signed by your generated CA. Additionally, sites blocked in the filter list should show a block page (try something like anydesk.com)

Change the block page

You can create a custom block page by editing /usr/share/e2guardian/languages/ukenglish/template.html. For assets like images you'll need to install a webserver (usually nginx) on the router and host them that way.

Application Blocking (DNS)

Some applications could still continue to work despite being blocked by E2Guardian, to solve this, DNS blocking can be used.

In this example, the application we will be blocking is AnyDesk, but this can be done to any application that relies on DNS to find what servers to communicate with.

Open /etc/dnsmasq.conf and insert the following line at the end of the file:

address=/anydesk.com/0.0.0.0

After saving the file, reload dnsmasq using sudo systemctl restart dnsmasq

If everything went well, the AnyDesk client should no longer be able to connect successfully (anynet_no_dns_entry).