Sakura VPS¶

VPS Details¶

• Provider: Sakura Internet (さくらのVPS)
• Location: Tokyo, Japan
• OS: Ubuntu 24.04 LTS
• Role: Public reverse proxy (HAProxy) + Tailscale VPN gateway

Monitoring¶

The Sakura VPS sends an hourly Pulse ping to updown.io. If the ping stops arriving, updown.io triggers an SMS alert.

0 * * * * curl -sSo /dev/null -m 10 --retry 5 https://pulse.updown.io/<token>/<token>

SSH Access¶

sshd listens on port 28, IPv6 only (:::28). Two reasons:

• Bot noise reduction: automated scanners almost exclusively hammer port 22 on IPv4. An IPv6-only non-standard port receives virtually no unsolicited connection attempts.
• Tailscale fallback: if Tailscale goes down and the VPN tunnel is unreachable, direct SSH over IPv6 is still available as a recovery path.

HAProxy Configuration¶

HAProxy is the only process listening on public-facing ports. All traffic is forwarded through Tailscale to Incus containers on the home server. Full configuration at Benoit/HAProxy.

Ports¶

Web Backends¶

All HTTP/HTTPS traffic enters through a single frontend_default. Routing is done via hdr(host) ACLs:

Key Features¶

• Country-based ACLs: Per-country IP blocklists loaded from flat files (/etc/haproxy/country/*.txt). Used to restrict sensitive services to JP/FR or to block specific regions. The script to generate these files is at Benoit/Scripts.
• PROXY protocol: TCP listens (mail ports, Forgejo SSH) forward with send-proxy so backends see the real client IP.
• HSTS + security headers: All HTTPS backends set Strict-Transport-Security (1 year), X-Frame-Options, X-Content-Type-Options, and Referrer-Policy.
• HTTP caching + compression: The default frontend caches cacheable responses and compresses text content with deflate/gzip.
• www → non-www redirect: http-request redirect strips the www. prefix with a 301.

TLS Certificates¶

Certificates are managed with Certbot using the standalone HTTP-01 challenge on a non-standard port, so HAProxy can keep listening on port 80.

apt install certbot

certbot certonly \
  --standalone \
  --non-interactive \
  --agree-tos \
  --email certbot@<your-domain> \# (1)!
  --http-01-port=8899 \
  --domains <app>.benoit.jp.net # (2)!

HAProxy expects a single PEM file per domain containing both the certificate chain and the private key. The following script reads all Certbot certificates and assembles them into /etc/haproxy/crt/:

#!/bin/bash

# Directory where HAProxy expects certificate files
HAPROXY_CERT_DIR="/etc/haproxy/crt"

# Ensure the directory exists
mkdir -p "$HAPROXY_CERT_DIR"

# Get certbot certificates output
CERTBOT_OUTPUT=$(certbot certificates)

# Process each certificate block
echo "$CERTBOT_OUTPUT" | awk '
    BEGIN { cert_name=""; fullchain=""; privkey=""; }
    /Certificate Name:/ { cert_name=$3; }
    /Certificate Path:/ { fullchain=$3; }
    /Private Key Path:/ { privkey=$4;
        if (cert_name && fullchain && privkey) {
            print cert_name, fullchain, privkey;
            cert_name=""; fullchain=""; privkey="";
        }
    }
' | while read -r cert_name fullchain privkey; do
    if [[ -n "$cert_name" && -n "$fullchain" && -n "$privkey" ]]; then
        output_cert="$HAPROXY_CERT_DIR/$cert_name.pem"
        cat "$fullchain" "$privkey" > "$output_cert"
        chmod 600 "$output_cert"
        echo "Created $output_cert"
    fi
done

cp haproxy_cert.sh /usr/local/bin/haproxy_cert.sh
chmod 700 /usr/local/bin/haproxy_cert.sh
haproxy_cert.sh
systemctl reload haproxy

Firewall Setup¶

Why Not ufw?¶

Ubuntu ships ufw as its default firewall tool, but it uses legacy iptables under the hood. Tailscale also manages its own chains via iptables-nft, and mixing the two leads to unpredictable behavior. The clean solution is native nft directly.

Comparison of options on Ubuntu:

• ufw: Default, easiest, but conflicts with native nftables rules from Tailscale, Docker, and WireGuard.
• nft directly: What Ubuntu's own docs recommend for granular control. Write rules in /etc/nftables.conf or a custom script.
• firewalld: Uses nftables natively, but is primarily the Red Hat/Fedora world's tool.

Coexisting with Tailscale¶

Tailscale manages its own table ip filter and table ip6 filter chains via iptables-nft. Running nft flush ruleset would wipe those chains until Tailscale restarts.

The correct approach is to add a separate table inet filter at priority filter + 10. Tailscale's chains run at priority 0 (the filter constant), so they accept Tailscale traffic first, then your inet filter acts as the gate for everything else.

#!/bin/sh
# Adds inet filter table without touching Tailscale's ip/ip6 tables

nft flush table inet filter 2>/dev/null
nft add table inet filter 2>/dev/null || true

nft -f - <<'EOF'
table inet filter {

    chain input {
        type filter hook input priority filter + 10; policy drop;

        iif lo accept
        ct state established,related accept

        ip  protocol icmp   accept
        ip6 nexthdr  ipv6-icmp accept

        iif "tailscale0" accept
        udp dport 41641 accept

        tcp dport { 22, 25, 28, 80, 443, 465, 993, 4190 } accept
    }

    chain forward {
        type filter hook forward priority filter + 10; policy drop;

        iif "tailscale0" accept
        oif "tailscale0" accept
    }

    chain output {
        type filter hook output priority filter + 10; policy accept;
    }
}
EOF

echo "Rules loaded."
nft list ruleset

After loading, verify the rule ordering with nft list ruleset. You should see Tailscale's table ip filter chains at priority filter (0) and your table inet filter at priority filter + 10.

Persistent via Systemd Service¶

Rather than saving with nft list ruleset > /etc/nftables.conf (which would also capture Tailscale's chains and cause conflicts on reboot), run the script as a oneshot service that starts after Tailscale:

sudo cp nftables-local.sh /etc/nftables-local.sh
sudo chmod 750 /etc/nftables-local.sh

[Unit]
Description=Local nftables ruleset (inet filter)
After=network-pre.target tailscaled.service
BindsTo=tailscaled.service

[Service]
Type=oneshot
ExecStart=/etc/nftables-local.sh
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable nftables-local.service
sudo systemctl start nftables-local.service
sudo nft list ruleset

systemd-networkd and Tailscale¶

systemd-networkd flushes routing policy rules it considers foreign on every restart. Tailscale installs its own ip rules at priorities 5210 to 5270 for policy routing via table 52, and loses them each time networkd restarts. Tailscale's recovery path logs the flush and re-installs the ip rules, but does not always re-sync the throw routes inside table 52, which can leave advertised subnet routes broken until tailscaled is fully restarted. See Post Mortem: Tailscale Subnet Route Lost After networkd Restart for the full story.

The fix is to tell networkd to stop managing foreign routing policy rules:

[Network]
ManageForeignRoutingPolicyRules=no

systemctl restart systemd-networkd

With this in place, any future networkd restart leaves Tailscale's ip rules and policy routing alone, regardless of what triggered it (package upgrades, interface changes, manual reload).

zram Swap¶

The VPS has 1 GB of RAM. To improve performance under memory pressure, a compressed in-memory swap device is set up via zram-tools. zram creates a block device backed by compressed RAM (using lz4 by default), exposed as a swap partition at higher priority than the regular swapfile.

Current swap layout:

zram is used first (higher priority). The swapfile acts as a safety net for heavy memory pressure that exceeds the zram device capacity.

sudo apt install zram-tools

# Fraction of RAM to use for zram (percent)
PERCENTAGE=50

# Compression algorithm: lz4 is a good balance of speed and ratio
ALGO=lz4

GRUB_CMDLINE_LINUX_DEFAULT="quiet zswap.enabled=0"

sudo update-grub

/swapfile none swap sw,pri=50 0 0

sudo systemctl enable --now zramswap.service

Verify the swap layout after starting:

swapon -s