Submit an issue View all issues Source
MIR-1170

Race-free release publishing via OCI-style content-addressed layout

Open public
phinze phinze Opened May 27, 2026 Updated May 27, 2026

The release pipeline overwrites artifacts in place at api.miren.cloud/assets/release/miren/{channel}/{file} for mutable channels (main, latest). Each artifact has a sidecar .sha256, and uploads run through a parallel worker pool (MAX_PARALLEL=4 in .github/workflows/release.yml). The small .sha256 lands before the big .tar.gz, so consumers hitting both during the upload window get a new hash next to an old tarball and fail with a checksum mismatch.

The docker build job was a guaranteed loser of this race because it ran in parallel with the upload job. That specific case is fixed in PR (link TBD). The underlying race still exists for every other consumer.

Why the obvious fixes don't work

  • Reordering the upload queue (tarballs first, then sha files) shifts the race window from "new sha, old tar" to "old sha, new tar." Same problem.
  • Uploading version.json last (already done in Phase 3 of upload-to-miren) doesn't help because no consumer actually treats it as the source of truth for artifact URLs. They all construct URLs from the flat {channel}/{file} pattern.

Today's consumers (none of them resolve URLs through the manifest)

  • cli/commands/download_release.go — blind URL construction, no manifest lookup. This is what broke in CI.
  • pkg/release/downloader.go (used by miren upgrade, server upgrade, runner upgrade) — reads version.json for version metadata only, then constructs artifact URLs via pkg/release/artifact.go::GetDownloadURL().
  • mirendev/actions/setup — reads version.json for the expected sha256, then constructs the artifact URL from the channel pattern. Hash-checks; races against the in-flight tarball.
  • docs/docs/ci-deploy.md — curl snippets that hardcode the URL pattern.

Proposed endgame: OCI-style content-addressed store

Restructure the asset layout to mirror how an OCI registry organizes things, since we already operate one internally and use LLB in stackbuild. Conceptually three namespaces:

  • /blobs/sha256:DIGEST — raw artifact bytes, immutable forever.
  • /manifests/NAME — small JSON pointer, atomically swapped on publish.
  • /refs/NAME/FILENAME — convenience redirect that resolves to the current blob URL for that name and filename.

Manifest schema (sketch):

{
  "version": "main:abc1234",
  "commit": "abc1234...",
  "build_date": "...",
  "artifacts": [
    {"name": "miren-base-linux-amd64.tar.gz", "blob": "sha256:20084964...", "size": 124589332}
  ]
}

Properties that drop out of the design:

  • Storage indexed by actual content, not by commit or filename. Two builds of the same commit naturally produce different blobs; identical builds dedup automatically.
  • The blob URL contains its own integrity check. Sidecar .sha256 files go away entirely. The original race becomes structurally impossible because there are no paired files to get out of sync.
  • Manifests are the only thing that mutates in place. They're tiny JSON, single-object atomic PUTs.
  • GC has a one-sentence invariant: delete any blob not referenced by any manifest. No date cutoffs, no rollback retention math. Pure reference walk.
  • latest is just a manifest whose blob refs happen to match the current stable tag's manifest. No dual-publish gymnastics.
  • The refs namespace keeps one-liner shell downloads working for docs snippets and humans. Redirect is server-side atomic, updated alongside the manifest.

Client work

  • cli/commands/download_release.go — rewrite to fetch manifest, look up blob, download from /blobs/.
  • pkg/release/downloader.go and artifact.go — same. Replace GetDownloadURL() with manifest-driven resolution.
  • mirendev/actions/setup — new tag that uses manifest URLs.
  • Docs snippets — switch to the refs form, or update to fetch the manifest first.

Server work

  • Asset service at api.miren.cloud needs to support the new endpoints. Worth confirming what the backing store supports today and what the server-side lift looks like.
  • LLB/OCI synergy: we already speak this dialect, so there's hope of reusing infrastructure rather than building from scratch.

Backcompat (pre-1.0, narrow scope)

The only must-preserve is that an old miren binary can run miren upgrade and pull down a new binary that knows the new layout. To support that:

  • Keep {channel}/version.json at its existing URL during transition.
  • For one transitional release, keep dual-publishing artifacts to the flat {channel}/{file} URLs so older miren upgrade invocations still work. Older clients race during that window the same as today; we accept it.
  • After the transitional release, retire the flat layout. Pre-1.0; external pins on mirendev/actions/setup and similar that aren't updated will break, and that's fine.

Probably RFD-worthy. The blob/manifest/refs schema, server endpoints, GC daemon, and migration plan deserve more deliberate design than a Linear issue. Capturing the spark here so it's not lost; promote to RFD when picked up.

Related