Scraping up a home network in a pinch
For a couple of reasons, I've been building a new home network in the flat I'm currently staying in. Given current world events, however, I've largely been confined to whatever miscellaneous devices I have lying around, as it's presently neither logistically nor financially feasible to acquire shiny new networking toys.
The interwebs here comes in the door over a VDSL line (which I assume is vaguely BT Openreach flavoured), which was previously met by a Draytek Vigor 130 in bridge mode, to which an Apple Aiport Extreme talked PPPoE. This all worked mostly fine, however for some of the above mentioned reasons, I don't have login credentials for the Airport, and I ideally want the device which is sitting between the LAN and the rest of the internet to be something which I can SSH into and which looks like a vaguely Unixoid system on the inside.
The setup
I have a fistful of Raspberry Pis (of varying models), a bag full of ethernet cables, and one or two old ethernet switches.
Take 1
I have a shiny new Raspberry Pi 4 (one of the ones with four gigabytes of RAM) which I'd never used for much before now, so I selected that device as a router. I plugged the ethernet port into the Draytek modem and set up pppd(8) to speak PPPoE. I then ran hostapd(8) on the WiFi interface and glued everything together with iptables(8).
I then decided that WiFi wasn't quite enough, and that I wanted to have an ethernet connection to my laptop. So, I added a USB ethernet adaptor to the Pi, plugged in a little 5-port switch I had, and then ran a cable back to my desk.
With the benefit of hindsight, having the LAN connected to the internet via a Raspberry Pi's USB bus might not have been a smart idea (at least the Pi 4 has its builtin ethernet interface on a separate bus from the rest of the USB ports, but it still involved USB on one side of the connection). On the bright side, most of this setup broke before I had a chance to run into that limitation.
Take 2
A couple of things in this configuration hit the leading edge of the bathtub curve alarmingly quickly.
The USB ethernet interface I had to hand is quite old and well used at this point, so the physical connector is quite well worn. The Pi 4's USB sockets also don't seem to grip inserted plugs that firmly (as opposed to the Pi 3 which I compared with), so the connection to the USB NIC was quite unstable, and started to drop out once every few minutes or so, which disconnected it from the bridge interface it was attached to on the Pi.
The LAN-side switch I was using also died spontaneously one night, which was an additional inconvenience.
I wasn't sure how to fix this mess, seeing as I was stuck with a single ethernet interface on the Pi and two networks to connect it to. While I do have other Pi's around with more reliable USB, they mostly already had assigned missions in life, and the Pi 4 both has the newest ethernet chipset and is the only Pi I have which doesn't have its ethernet and its USB interfaces on the same bus.
Someone then reminded me that 802.1Q exists, and that such things can be used to multiplex multiple layer 2 networks over a single cable. Conveniently enough, I had a second switch spare (a TP-Link TL-SG108E) which had more ports, was faster, and was capable of managing VLAN tags.
There's something to be said here about computer networks, network engineers, and arbitrary complexity.
Either way, the network now has three VLANs (arbitrarily numbered 20, 21, and 22). VLAN 20 is the local LAN, which is untagged on all the LAN switch ports and tagged on the router switch port. VLAN 21 is the "management" VLAN, which is untagged on the modem port and tagged on the router port. The part which makes this all work nicely is that the modem supports 802.1Q tagging of all traffic going to and coming from upstream, so the modem tags upstream traffic on VLAN 22, which then goes to the router too.
As a result, the router has a mess of VLAN interfaces (eth0.20, eth0.21, etc), which is kind of amusing, and adding multiple WireGuard interfaces doesn't help. For all the added complexity, however, this setup does appear to be stable, and it's been working satisactorily for several weeks now (unlike the first iteration, which lasted for a matter of days).
Future steps
The Pi 4 kind of sucks as a WiFi access point. I'm not sure if this is because the WiFi chipset isn't really intended for serious use in AP mode, or whether the SDIO bus on the SoC isn't up to the job. The ethernet hasn't given me any issues thankfully, as my laptop spends much of its time plugged into to a physically cabled network, but I'm likely to eventually install a dedicated access point just to make downloads on my phone a bit less painful.
The TP-Link switch only cost me 30 pounds several years ago, and it shows. There doesn't appear to be any way to disable access to the management interface on certain ports, which is a bit of a risk. On the other hand, that shouldn't be too easy to exploit from outside the flat, so I can just about live with it otherwise.