Submit an issue View all issues Source
MIR-856

Entity store: clean up orphaned index entries

Open public
phinze phinze Opened Mar 23, 2026 Updated Apr 2, 2026

Problem

When an entity is deleted from the etcd store but its index entry is not cleaned up, the reconciler logs a WARN + ERROR pair every 30 seconds, forever:

[WARN]  coordinator.etcdstore failed to get primary entity from etcd │ id: lsvd_mount-CXxPKn4o8A125BizS1AJe
[ERROR] coordinator.entityserver entity in index but not in store, skipping │ id: lsvd_mount-CXxPKn4o8A125BizS1AJe

On James's server this has been firing continuously since at least Feb 22 — over 40 days of noise at the time of discovery.

Expected Behavior

When the entity store encounters an index entry pointing to a nonexistent entity, it should:

  1. Remove the stale index entry
  2. Log a single WARN that it cleaned up an orphan
  3. Not retry on subsequent reconciliation loops

Stretch

Add a startup validation sweep that checks all index entries against the store and cleans up any orphans found.