Routing tricks with SSH and iptables

Like a lot of the rest of the world right now, I'm now working remotely. Part of my work is reliant on a given GitLab server, however this server isn't accessible over HTTP(S) from offsite (for reasons which are outwith the scope of this post). The administrators of the GitLab have provided us with some pointers and documentation on how to use tools like OpenSSH's SOCKS proxy mode when connecting to the (globally accessible) on-site shell servers in order to get access to the web interface, and I've previously used sshuttle for the same purpose.

The problem with using OpenSSH's SOCKS proxy functionality is that (in Firefox at least) it's not possible to set proxy settings on a per-site basis (short of using a proxy.pac file, which I'm not terribly familiar with), and sshuttle requires uploading and running code on the remote system as well as mangling the local firewall.

Seeing as my site border router (i.e. the box sitting between the LAN and the ISP) is running Debian (or, at least, close enough), I decided to see if I could do something on the router which is a bit more transparent to end clients on the LAN. As OpenSSH has the ability to listen on a local TCP port and then tunnel data received on that port over the SSH connection to a remote server, it's possible to set up an ssh(1) instance or two on my router to connect to the remote site and forward HTTPS traffic to the appropriate remote machine. iptables can then be used to glue everything together.

Server-side setup

I first generated an SSH key for my router for accessing my shell account. I then added the following entry to my account's SSH authorized_keys file on the shell server. It looks a bit like this (lines wrapped for readability):

# this is my normal remote access ssh key
ssh-ed25519 AAAA.... molly@laptop

# this is the router's key
restrict,port-forwarding,command="/bin/false",permitopen="gitlab.example.com:443" 
        ssh-ed25519 AAAA... ssh-proxy@router

This restricts any client connecting with the router's key to only establishing TCP connections to the host gitlab.example.com on port 443 (as this particular GitLab instance does not listen on port 80). If the client tries to request command execution, the command /bin/false will be run instead.

Client-side setup

First, the tunnels need to be started:

$ ssh -N -L localhost:8443:gitlab.example.com:443 molly@shell-server.example.com

NAT is evil, but a necessary one here. Incoming TCP traffic needs to have its destination address and port changed to redirect it so it goes to the local SSH tunnel endpoint instead of to the remote GitLab. Note that my LAN subnet is 10.0.1.0/24, and the LAN interface on the router is br0. The GitLab instance is also only accessible over IPv4, so there's no need to additionally set ip6tables rules.

# iptables -t nat -A PREROUTING -s 10.0.1.0/24 -d gitlab.example.com -p tcp --dport 443 \
> -j DNAT --to-destination 127.0.0.1:8443

And we need a sysctl setting to tell the kernel that it can use loopback addresses (i.e. 127.0.0.0/8) for routing on non-loopback interfaces, on the return path from the router back to my laptop.

# sysctl net.ipv4.conf.br0.route_localnet=1

Curiously, no further configuration for the reverse path (i.e. source address NAT) is necessary, as the kernel sets the remote source address for the GitLab server automatically. I was a little confused about this initially, but after speaking to some friends and reading the iptables manual page, I discovered that the rules in the nat table are only checked when packets which create new connections are encountered (e.g. a TCP SYN segment). When this happens, the kernel connection tracking sets up a state entry and a translation rule, and handles all subsequent packets in the connection using the conntrack entry. While I initially thought that this means you can't do stateful NAT with iptables, multiple friends commented that it is actually possible with the right rules in the right places, but that's an adventure for another time.


Changelog