Submit an issue View all issues Source
MIR-1777

Report runtime-owned app health alongside entity sync

In Progress public
phinze phinze Opened Sep 4, 2026 Updated Sep 4, 2026

After the MIR-1770 rollout, garden completed entity sync successfully and Cloud projected all 34 app identities. The console nevertheless shows every app as “awaiting report.”

This is not eventual-consistency lag. Cloud renders that label when an app exists in Postgres but has no corresponding sample in Valkey. The existing apps.SampleStore still serves the read path, but no production code calls RecordSample or RecordSamples. Runtime’s five-minute cluster status report carries only a workload count, so waiting for another report will not fill the samples.

Keep health out of generic entity sync

This gap does not mean the app export contract is missing fields. RFD 109 deliberately limits entity sync to facts stored on exported entities and says that derived values such as app health and instance counts may continue to use purpose-built reports.

Runtime derives app health from active app configuration and sandbox-pool state. AppInfo.ListApps already owns the classification used by miren app list, including fixed versus autoscaled services, task-only apps, cooldowns, and ready versus desired instances. Pool readiness already incorporates whether its sandboxes passed their health checks, so v1 does not need raw sandbox entities.

Do not export sandbox pools or sandboxes through generic entity sync for this fix. They are high-churn implementation state, would enlarge the wire and landing store substantially, and would make Cloud reproduce runtime’s health semantics. MIR-1607 can remain the later per-service and instance-drilldown discussion.

Direction

Add a small, versioned app-health capability to the negotiated uplink session. It should be a source-side projection rather than a synthetic persisted entity.

Runtime should factor or reuse the health calculation behind AppInfo.ListApps and send only:

  • app name
  • health
  • ready instances
  • desired instances
  • observed time

The reporter should send a bounded full sample set when the capability is selected on a new session. It should then rederive state on a modest cadence and send only apps whose sample changed. A slower periodic full sample repairs a Valkey wipe or a lost delta. The old reporter’s 15-second sample cadence, 15-minute repair floor, 100-app batches, and fleet-spread reconnect are reasonable starting points rather than new protocol guarantees.

Cloud should bind every sample to the authenticated cluster identity, validate names and health values, clamp implausible timestamps, and write batches through the existing Valkey sample store. Its last-writer-wins comparison by runtime observed_at should remain the ordering rule.

Entity sync remains the sole owner of app existence and deletion. The health feed must not call EnsureApps, maintain snapshot epochs, sweep app rows, or advance the entity-sync cursor. A sample arriving before its app projection is harmless, and an orphaned sample after deletion is unreachable through the app read path and can expire naturally.

This purpose-built feed should not add synthetic health entities to the export contract or land health samples in Postgres.

Done when

  • Garden’s Cloud app list shows the same health and ready/desired counts as miren app list -C garden.
  • A health or instance-count change normally reaches Cloud within 30 seconds.
  • An unchanged cluster sends no health deltas.
  • Reconnecting sends a complete bounded sample set after fleet spreading.
  • Clearing Cloud’s sample keys self-heals on the next full sample without rebuilding app identity.
  • Older runtimes that do not offer the capability continue to show honest “awaiting report” state.
  • Malformed names, unknown health values, clock skew, stale delivery, and batches larger than 100 apps are covered.
  • App deletion remains owned by entity sync and cannot be undone by a late health sample.
  • Garden passes before MIR-1762 resumes promotion to toys and club.