Submit an issue View all issues Source
MIR-441

Build Process Saga

Open public
phinze phinze Opened Oct 2, 2025 Updated Apr 2, 2026

Convert build process to use saga framework, implementing streaming I/O pattern for tar data recovery.

Scope

StreamRegistry Infrastructure:

  • StreamRegistry (application-level, not framework)
  • Maps stream IDs to active readers and staged filesystem paths
  • Enables recovery when original stream is gone
  • Cleanup hooks for saga completion

Extract Build Actions:

  • ReceiveTarAction - stage tar stream to filesystem, handle recovery gracefully
  • LoadConfigAction - load app.toml
  • DetectStackAction - detect/validate build stack
  • ResolveClusterIPAction - resolve cluster.local
  • LaunchBuildkitAction - launch buildkit sandbox
  • GetNextVersionAction - generate version/artifact IDs
  • BuildImageAction - execute buildkit build (long-running)
  • ParseServicesAction - parse Procfile and app config services
  • CreateVersionAction - create app version entity
  • SetActiveVersionAction - set active version on app
  • StopOldSandboxesAction - stop sandboxes running old version

Action Factories:

  • Create factories with Builder dependencies injected
  • Register in SagaRegistry

Definition Registration:

  • Register "build-from-tar" definition
  • Define action dependencies (DAG structure)

Entry Point:

  • Update BuildFromTar() to register tar stream and launch saga
  • Pass app name and stream ID via InitialInputs
  • Wire up progress callbacks for build status updates
  • Cleanup StreamRegistry after saga completes

Testing:

  • Success path with real tar data
  • Failure injection at each build step
  • Recovery scenarios (crash during tar receive, during build, etc.)
  • Stream recovery (original stream gone, fallback to staged path)
  • Multiple concurrent builds
  • Performance benchmarking (saga overhead acceptable)

Performance Tuning:

  • Measure saga logging overhead
  • Optimize entity writes if needed
  • Ensure build times don't regress

Why One Issue

The streaming pattern is unique to builds and requires all pieces working together. Splitting would create incomplete, non-testable code.

Dependencies

Requires: MIR-439 (Saga Framework Foundation)

Reference

RFD 0035: Saga Pattern for Distributed Operations "Example: Build Process Saga with Streaming" section Phase 3 from Migration Strategy