Submit an issue View all issues Source
MIR-728

Speed up release workflow by parallelizing artifact uploads

Done public
phinze phinze Opened Feb 18, 2026 Updated Mar 9, 2026

Context

The release workflow (release.yml) currently takes ~15 minutes end-to-end. The upload-to-miren job is a major bottleneck at 3m 17s, and it gates build-and-push-docker downstream.

Analysis based on completed run: https://github.com/mirendev/runtime/actions/runs/22119206457

Problem

The upload-to-miren step runs ~28-30 sequential curl calls to the cloud assets API (PUT /assets/release/miren/{path}/{artifact}):

  • 7 artifacts × 2 files each (artifact + sha256) × 2 paths (tag + latest) = 28 calls
  • Plus version.json uploads at the end

Each call takes ~7s, all run sequentially.

Proposed Fix

Split upload_artifact into two phases:

  1. Prepare (sequential, fast) — generate checksums, build artifact manifest with jq
  2. Upload (parallel) — fire off all curl calls concurrently, collect PIDs, fail if any fail

Keep version.json upload sequential at the end, since it should only go out after all artifacts are confirmed.

Cap concurrency at ~4-6 to be conservative — the cloud assets API proxies uploads inline through the server to GCS (no signed URLs / direct upload), so we don't want to overwhelm it. Even 4-6 concurrent uploads should cut the step from ~3m to well under a minute.

Longer-term consideration

The cloud assets API currently proxies every byte through itself to GCS. A future optimization would be to have it hand out GCS signed upload URLs so CI uploads directly to the bucket, eliminating the proxy bottleneck entirely. This is a separate change in the cloud repo.

Timeline from last successful release

T+0:00   init (3s)
T+0:03   test phase (~3m 10s)
T+3:10   package + build-binaries (~5m 43s, gated by darwin-arm64 notarization)
T+8:53   upload-to-miren (3m 53s)  ← THIS IS THE TARGET
T+12:53  docker + homebrew (~2m)
T+14:49  notify