MIR-1250

Flaky TestRabbitmqAddonDeployWithAppToml: bun app crash-loops post-boot, fails the route check

Done runtime Bug public

phinze Opened Jun 23, 2026 Updated Jun 23, 2026

Fix flaky RabbitMQ addon test broken by an upstream broker tag float

Surfaced as a Release-pipeline failure on the MIR-1245 merge (PR #861), but confirmed not caused by that change (controlled experiment below), so filing separately.

Symptom

TestRabbitmqAddonDeployWithAppToml (blackbox/addon_rabbitmq_test.go:74) times out waiting for the app to respond via route (30s); the ingress returns "The application app/bb-bun-rabbitmq failed to boot." Failed twice in the #861 Release run and reproduces deterministically in a local dev env.

What's actually happening (from server logs)

The RabbitMQ broker (-rmq-) is healthy: its single-writer disk lease binds cleanly, no boot failure.
The bun web app (which has no disk) boots and then crashes repeatedly: repeated proxy error: EOF ... app: bb-bun-rabbitmq as the bun process exits, looping until consecutive crashes: 4 → crash cooldown. So it's a post-boot application crash, not a boot/health-check/lease failure.

Ruled out MIR-1245

Reverted both fixes from #861 (the boot-failure lease release and the recurring orphan sweep), rebuilt, and re-ran in the same env: the test fails identically. The disk-lease code is uninvolved (the failing component holds no lease; the broker's lease behaves correctly).

Environment-sensitive

Passes in the standalone test-blackbox job (#861 PR CI, twice) and on the previous release (#860, 16.9s), but fails in the heavier Release pipeline (#861, twice) and locally. Looks like resource contention / timing tipping the bun app into a crash loop, but root cause not yet isolated.

Next step

Capture the bun app's stderr (miren logs mid-run, or deploy testdata/bun-rabbitmq manually and watch) to see why the process exits, likely an AMQP startup/connection issue under load, but unconfirmed.