PVE Infra Docs

topology

8 min read

Network topology

Canonical doc for the host network. Bootstrap script that produces this: network-bootstrap.

Hetzner side

  • Single public IPv4 «PUBLIC_IP» on nic0 (no additional IPs, no KVM console networking — SDN per the CyanLabs guide). The NIC was enp0s31f6 pre-PVE-9 — it’s now pinned to nic0 by MAC via /usr/local/lib/systemd/network/50-pve-nic0.link (generated by pve-network-interface-pinning during the 8→9 upgrade, 2026-10).
  • Proxmox installed via Hetzner’s own server setup, then the PVE post-install script.

Bridges

BridgeRole
vmbr0Linux bridge to nic0 — holds «PUBLIC_IP», the WAN side
vnet0SDN bridge — internal 10.0.1.0/24, gateway 10.0.1.1

Internal subnet 10.0.1.0/24

  • All guests get static 10.0.1.x addresses (see inventory).
  • Outbound: SNAT via vmbr0 to «PUBLIC_IP».
  • Inbound: no NAT/port-forwarding — nothing internal is reachable from the internet. All ingress is over Tailscale (tailscale).

Provider edge — Hetzner Robot firewall

«PUBLIC_IP» sits behind the Robot firewall (status: active). Snapshot: reference/hetzner-firewall.png.

Incoming — default-deny except:

  • ICMP (all)
  • TCP ack to ephemeral ports 32768–65535 (replies to outbound conns)
  • Hetzner Services (provider infra/monitoring)

Everything else inbound is dropped — no new inbound connections can be initiated to «PUBLIC_IP». The “no public exposure” invariant is therefore enforced at the provider edge and by the absence of NAT/port-forwarding on the host.

Stateless firewall — no inbound UDP, ever

The edge is stateless: the tcp esta rule passes inbound TCP replies (ACK flag) to outbound connections, but UDP has no equivalent — all inbound UDP is dropped, including replies to outbound UDP. Live consequence: plain UDP DNS and raw WireGuard handshakes can never work through this edge. Everything on this host is TCP-shaped: host DNS is MagicDNS→DoH (TCP/443) or split-DNS over the tailnet, tailnet traffic itself rides DERP (TCP). Deliberately kept this way — a UDP allow rule would open initiated inbound UDP too, breaking the no-public- exposure invariant.

Outgoing — allow all EXCEPT tcp/25+465 (SMTP, v4+v6 — provider anti-spam block). UDP leaves fine; its replies just never come back.

IPv6 — “Filter IPv6 packets” is off, but «HOSTNAME» has no public IPv6 configured (confirmed), so it’s moot. If v6 is ever added to vmbr0, inbound would arrive unfiltered — these rules are v4-only and host INPUT is unfiltered (below). Enable v6 filtering or add host rules before configuring a public v6.

Host firewall state

pve-firewall is enabled at the datacenter level: cluster.fw [OPTIONS] contains only enable: 1 (default policies apply). IN rules exist for vnet0 (DHCPfwd/DNS accepts) plus an empty management-ipset artifact from before tailnet SSH existed.

Consequence of the vnet0 IN rules: guests cannot ping the gateway (ping 10.0.1.1 fails on every guest) — only DHCP/DNS are accepted to the host-side IP. Guest health checks should use ARP resolution (ip neigh), a DNS query against 10.0.1.1, and egress — never ICMP to the gateway.

Confirmed: /etc/pve/nodes/«HOSTNAME»/host.fw does not exist, and INPUT jumps to ts-input (Tailscale) and PVEFW-INPUT — yet tailnet→host ssh works with no matching IN rule, so PVEFW-INPUT passes unmatched traffic through to the chain policy (ACCEPT). Host INPUT is therefore effectively unfiltered by default.

Because of that, vnet1’s isolation lives entirely in the mangle table (evaluated before the filter table — before PVEFW-INPUT/ PVEFW-FORWARD — so it applies regardless of PVEFW state or policies) rather than cluster.fw or appended filter rules.

vnet0 config

/etc/network/interfaces.d/sdn on the host — fully SDN-generated (#version: header). Proxmox emits the post-up/post-down SNAT and fwbr+ conntrack-zone rules itself when the subnet has SNAT enabled — they are not manual additions. Hand-added lines are the only thing an SDN Apply wipes; never edit this file.

auto vnet0
iface vnet0
  address 10.0.1.1/24
  post-up iptables -t nat -A POSTROUTING -s '10.0.1.0/24' -o vmbr0 -j SNAT --to-source «PUBLIC_IP»
  post-down iptables -t nat -D POSTROUTING -s '10.0.1.0/24' -o vmbr0 -j SNAT --to-source «PUBLIC_IP»
  post-up iptables -t raw -I PREROUTING -i fwbr+ -j CT --zone 1
  post-down iptables -t raw -D PREROUTING -i fwbr+ -j CT --zone 1
  bridge_ports none
  bridge_stp off
  bridge_fd 0
  ip-forward on

Custom vnet rules — /etc/network/if-up.d/vnet-rules

The only rules SDN doesn’t know about are vnet1’s isolation rules. They live in this ifupdown2 hook — runs on every ifup with $IFACE set, idempotent (-C check before -A), and survives SDN regeneration since it’s a separate file. Must be executable (chmod +x). Re-apply without reboot: IFACE=vnet1 /etc/network/if-up.d/vnet-rules.

#!/bin/sh
# vnet1 isolation rules. SDN owns interfaces.d/sdn (Apply regenerates
# it, wiping hand-added lines) — these must not live there.
 
[ "$IFACE" = "vnet1" ] || exit 0
 
add() { iptables -t mangle -C "$@" 2>/dev/null || iptables -t mangle -A "$@"; }
 
# host-local guard: established replies (ssh jump) pass; nothing else
# may initiate to ANY host-local address
add PREROUTING -i vnet1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
add PREROUTING -i vnet1 -m addrtype --dst-type LOCAL -j DROP
 
# transit: replies in; ALL intra-bridge traffic allowed (packets
# bridged to vpngw carry their FINAL dst — e.g. Mullvad DNS 10.64.0.1
# is inside 10/8 — so a /24-scoped accept would wrongly drop them;
# LAN-private enforcement is vpngw's own killswitch + the egress guard);
# nothing new into vnet1 from outside (! -i needed — bridged packets
# show vnet1 as BOTH in/out); no private/tailnet egress; only vpngw
# leaves via vmbr0
add FORWARD -o vnet1 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
add FORWARD -i vnet1 -o vnet1 -j ACCEPT
add FORWARD ! -i vnet1 -o vnet1 -m conntrack ! --ctstate ESTABLISHED,RELATED -j DROP
add FORWARD -i vnet1 -d 10.0.0.0/8     -j DROP
add FORWARD -i vnet1 -d 172.16.0.0/12  -j DROP
add FORWARD -i vnet1 -d 192.168.0.0/16 -j DROP
add FORWARD -i vnet1 -d 100.64.0.0/10  -j DROP
add FORWARD -i vnet1 -d 169.254.0.0/16 -j DROP
add FORWARD -i vnet1 -o vmbr0 ! -s 10.0.2.2 -j DROP

Bringing the vnet down does not remove its rules (no post-down hook — stale rules while a bridge is down are harmless).

Duplicate rules are expected

SDN’s generated post-up/post-down lines are unconditional -A/-I — every Apply, ifreload, or ifup appends another copy, so nat POSTROUTING and raw PREROUTING accumulate duplicates over time (observed: ~8 copies). Harmless — identical rules, first match wins. Apply also only wipes the file: live kernel rules persist, so rules added by earlier hand-edited post-up lines survive as duplicates until manually deleted (iptables -D in a loop) or the host reboots.

Isolated dev subnet 10.0.2.0/24 (deployed — dev-vm)

A second SDN vnet, vnet1, for the agent/dev environment. Two guests:

  • vpngw (LXC 111, 10.0.2.2) — Mullvad VPN gateway; the subnet’s only path to the internet. Runs the mullvad daemon with udp2tcp transport (raw WireGuard can’t handshake — inbound UDP replies are dropped at the provider edge). devvm’s traffic exits as a Mullvad IP, never «PUBLIC_IP».
  • devvm (QEMU 112, 10.0.2.10) — default route via 10.0.2.2.

The host holds 10.0.2.1 purely as the jump/access point — it is not devvm’s gateway. Host-side rules isolate vnet1 (no LAN, no tailnet, no host-local services, no inbound connections) and restrict vnet1’s only permitted egress through vmbr0 to vpngw’s encrypted tunnel traffic.

Create vnet1 in the SDN UI (zone + vnet, subnet 10.0.2.0/24 gateway 10.0.2.1, SNAT enabled — vpngw’s tunnel endpoint traffic needs it), which generates the stanza in interfaces.d/sdn identical in shape to vnet0’s above (including its own post-up SNAT rule — that generated SNAT is exactly what the ! -s 10.0.2.2 killswitch rule guards against, since it would otherwise let devvm egress via «PUBLIC_IP»).

vnet1’s isolation rules are applied by /etc/network/if-up.d/vnet-rules (above) — the mangle PREROUTING host-local guard and the mangle FORWARD transit policy. Do not add post-up lines to the sdn file — SDN Apply wipes them.

Rule-by-rule rationale is in dev-vm. Key points: --dst-type LOCAL covers every host-local address (the PVE UI binds 0.0.0.0, and the host’s tailnet IP isn’t in any private range); 100.64.0.0/10 blocks tailnet CGNAT; and ! -s 10.0.2.2 -o vmbr0 -j DROP prevents any vnet1 guest from bypassing vpngw and SNATing out via «PUBLIC_IP» (the killswitch’s second half — the first half is ! -o wg0-mullvad -j DROP on vpngw itself).

verify at build

iptables -t mangle -L PREROUTING -n and -L FORWARD -n — vnet1 rules present; from devvm, confirm «PUBLIC_IP»:8006 and 10.0.2.1:22 are unreachable. Optionally add IN DROP -i vnet1 to cluster.fw as a second layer (only effective if the host firewall is enabled — see above).

Traffic paths

PathAllowed
internet → «PUBLIC_IP» new connsno (Hetzner edge default-deny; icmp + Hetzner services excepted)
guest → internetSNAT out vmbr0
internet → guestno
tailnet → caddy :443yes (all HTTP services)
tailnet → gitea :22yes (git over SSH)
tailnet → technitium :53yes (split-horizon DNS)
tailnet → «HOSTNAME» (PVE UI, SSH)yes (management)
«HOSTNAME» → devvm (10.0.2.x) :22yes (jump path)
devvm → internal (LAN/tailnet/host)no (isolated)
devvm → internetvia vpngw → Mullvad exit (never «PUBLIC_IP»)
vpngw → internettunnel endpoint TCP only (udp2tcp), SNAT out vmbr0
devvm → internet bypassing vpngwno (! -s 10.0.2.2 drop)

Graph View

Backlinks