Submit an issue View all issues Source
MIR-1013

Sandbox metrics lost after miren restart for recovered containers

Open public
phinze phinze Opened Apr 14, 2026 Updated Jul 9, 2026

When miren restarts and reconcileSandboxesOnBoot() recovers surviving containers, it reattaches log consumers and re-reserves IPs, but never re-registers cgroup metrics monitoring via Metrics.Add(). This means CPU and memory metrics silently stop for all recovered sandboxes and never come back.

Verified on Garden: after the most recent restart (2026-04-14 01:22:13 UTC), 13 sandboxes were successfully recovered, but only 3 sandboxes report CPU metrics (all created after the restart). The 12 surviving sandboxes have no metrics at all.

The fix should be straightforward: during reconcileSandboxesOnBoot(), after confirming a sandbox is healthy and reattaching logs, call Metrics.Add() with the sandbox's cgroup paths and log entity attributes, the same way createSandbox() does at controllers/sandbox/sandbox.go:1006.