miren RPC: QUIC handshake fails over 1280-MTU tunnels (Tailscale/WireGuard) — InitialPacketSize 1280 exceeds path MTU
Summary
Remote miren RPCs to any cluster reached over a 1280-MTU path (Tailscale/WireGuard tunnels) hang on the first real RPC with timeout: no recent network activity. quic-go defaults InitialPacketSize to 1280 bytes; with IPv4+UDP headers that's a 1308-byte datagram sent with the DF bit, which exceeds the 1280 tunnel MTU. The kernel rejects it with EMSGSIZE and nothing reaches the peer. On-box calls and public-IP (1500-MTU) cloud clusters are unaffected.
Repro
- Client (foxtrotbase, single-homed) → cluster
miren01at its Tailscale IP100.91.121.48:8443. miren app list -C miren01→error performing http request: timeout: no recent network activity.- tcpdump on the server during the call: zero inbound packets on udp/8443.
Evidence it's MTU, not the network or RPC layer
- Plain UDP (native v4 and dual-stack v4-mapped) to the same dest/port arrives fine; ICMP and
tailscale pingwork. The path is healthy. - DF-set UDP boundary probe: a 1252-byte payload arrives (1252 + 28 = 1280 = MTU); 1253+ is rejected locally with EMSGSIZE.
- quic-go v0.57.1 hardcodes
const InitialPacketSize = 1280; ourDefaultQUICConfigdoesn't override it.tailscale0MTU is 1280 on both ends. - Symmetric: with only the client patched, the Initial reaches the server and the handshake starts, but the server's cert-bearing flight (also 1280) drops on its own egress → "handshake did not complete in time". Both ends need the fix.
- Verified:
InitialPacketSize: 1200on both client and server →app list -C miren01completes over Tailscale, every datagram ≤ 1280 on the wire.
History (not a recent regression)
- quic-go regressed at v0.44.0, unifying
InitialPacketSizeIPv4=1252/IPv6=1232into a singleInitialPacketSize=1280. Pre-0.44 just fit a 1280-MTU tunnel; 0.44+ doesn't. - miren has carried quic-go ≥ v0.44.0 since the cbor/QUIC RPC stack landed (commit
7bedadae, 2024-12-17, quic-go v0.48.2). Every tagged release (v0.0.x → v0.6.1 → v0.10.0) contains it. The recent v0.49.0→v0.57.1 bump is unrelated (both carry 1280).
Upstream
Known gotcha: tailscale/tailscale#2633 (open since 2021), quic-go#5573 (same symptom, closed as MTU), quic-go#5634 (argues the 1280 default violates RFC 9000 §14.1; closed not-planned). The fix belongs in our app config.
Fix
Set InitialPacketSize: 1200 (RFC 9000 minimum) in pkg/rpc/state.go DefaultQUICConfig. PMTUD grows packets back up after the handshake, so 1500-MTU paths are unaffected. PR incoming.
Relationship to MIR-1264
Distinct. MIR-1264 is dial-side source-interface selection on multi-homed clients (EHOSTUNREACH); this is packet-size vs path-MTU and reproduces from a single-homed client with a working route. Fixing one does not fix the other.
Environment
Server miren01 v0.10.0; client built from HEAD (794dca1c); quic-go v0.57.1.