Submit an issue View all issues Source
MIR-1755

Forced server shutdown leaves `miren-buildkit` task behind and blocks restart

Done Bug public
phinze phinze Opened Sep 3, 2026 Updated Sep 4, 2026

The Depot CI canary exposed a BuildKit lifecycle race in TestPOP. The test restarts the Miren server to pick up POP registration. Graceful shutdown times out and forces termination:

Stopping server (PID 10036)...
Warning: Graceful shutdown timed out, forcing termination...
Server stopped

The new server finds the existing BuildKit container and tries to create a fresh task, but the previous task still exists:

found existing buildkit container, restarting it
creating new task for existing buildkit container
failed to start runtime dependency graph: starting buildkit:
failed to create new task for existing container:
task miren-buildkit: already exists

The test-blackbox-pop job failed this way on both its initial attempt and retry in mirendev/runtime#1147.

Expected behavior: Miren should recover after a forced shutdown by cleaning up or adopting the existing BuildKit task before trying to create another one.

The same test passed under GitHub Actions earlier that day, so Depot’s timing appears to expose an existing restart race rather than a Depot-specific failure.

Related:

  • MIR-1303 fixed a similar stale BuildKit process problem after restart, but is marked done.
  • MIR-1466 covers consolidating the duplicated container teardown paths.
  • MIR-625 covers the related but different stale containerd-shim state-directory failure.