Submit an issue View all issues Source
MIR-1466

Extract a shared container-teardown primitive (base + sandbox + watchdog + saga + buildkit)

Open public
phinze phinze Opened Jul 23, 2026 Updated Sep 3, 2026

Follow-up from MIR-1463. Fixing the etcd plain→TLS recreate wedge surfaced that the same low-level container-teardown operation is reimplemented in several places with subtly different behavior:

  • components/base (etcd, victoriametrics, victorialogs) — now the cleanest version after MIR-1463: force-delete the task with WithProcessKill, tolerate not-found, delete the container, and explicitly remove the snapshot as a fallback.
  • components/buildkit — its own private copies of stopTask / deleteContainerWithRetry (doesn't embed BaseComponent).
  • controllers/sandbox (sandbox.go, watchdog.go, create_saga.go) — already does the not-found-tolerant force-delete correctly, but each site reimplements the leaf plus its own snapshot handling.

The leaf operation — "kill the task, force-delete-and-remove it tolerating not-found, delete the container, fall back to an explicit snapshot remove" — is identical everywhere. What legitimately differs is the orchestration on top (sandbox batches many containers with graceful timeouts, pause containers, and attach handles; base is single-component lifecycle with retry). Those don't merge, but they can share the leaf.

Proposal: extract a small shared primitive (e.g. pkg/ctrutil with DeleteContainerAndSnapshot + ForceDeleteTask) and converge base, buildkit, sandbox, watchdog, and the create saga onto it. base is the reference impl to lift from.

Note this is cleanup, not a bugfix — sandbox already handles the dead-task case, so nothing here is currently wedged. Scoped out of MIR-1463 deliberately to keep a GA-blocking fix tight and revertable.