redwoodjs__local-ci/.docs/rust-rewrite-tasks.md

Rust Native Binary Rewrite Tasks

This is the working checklist for moving Agent CI from a Node/TypeScript CLI to a native Rust binary while keeping npm install support and adding direct binary distribution.

Phase 0 — Planning

• RUST-000: Rust rewrite RFC

  • Define the migration strategy.
  • Decide which platforms are supported first.
  • Decide how long the TypeScript implementation remains available as a fallback.
  • Document install targets: npm, GitHub Releases, Homebrew, and optional shell installer.
  • Test: pnpm format:check

• RUST-001: Rust workspace scaffold

  • Add a Cargo workspace.
  • Add the native agent-ci binary crate.
  • Add cargo fmt, cargo check, cargo clippy, and test scripts.
  • Add initial CI checks for the Rust workspace.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 1 — CLI Parity Foundation

• RUST-010: CLI argument parser

  • Implement agent-ci --help.
  • Define commands: run, retry, abort, and clean.
  • Match the existing usage text closely enough for golden tests.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets
  • Test: diff -u <(node packages/cli/src/cli.ts --help) <(cargo run --quiet -p agent-ci -- --help)

• RUST-011: Config/env loading

  • Port .env.agent-ci loading.
  • Support AGENT_CI_* environment variables.
  • Preserve shell environment precedence.
  • Preserve DOCKER_HOST rejection behavior.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets
  • Test: DOCKER_HOST=unix:///bad.sock cargo run --quiet -p agent-ci -- --help exits 1 with the rename error

• RUST-012: Log/state directory handling

  • Port the current log directory layout.
  • Preserve run metadata file formats.
  • Preserve result file formats used by agents and tests.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 2 — Low-Risk Commands

• RUST-020: clean command

  • Port old log pruning behavior.
  • Preserve retention rules.
  • Add parity tests against fixture directories.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-021: retry / abort commands

  • Port paused runner signaling.
  • Keep the signaling protocol compatible with the TypeScript implementation.
  • Support retrying the failed step.
  • Support --from-step <N>.
  • Support --from-start.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 3 — Workflow Parsing

• RUST-030: Workflow file loading

  • Parse GitHub Actions YAML.
  • Preserve unsupported-feature diagnostics.
  • Add fixtures for existing smoke workflows.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-031: run --workflow discovery

  • Load one workflow file.
  • Discover runnable jobs.
  • Preserve current default SHA behavior.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-032: run --all discovery

  • Discover relevant workflows for the current branch/event.
  • Preserve path/filter behavior.
  • Preserve skipped-workflow reporting.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-033: Matrix expansion

  • Port matrix include/exclude behavior.
  • Preserve generated job names.
  • Support --no-matrix.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-034: Expression evaluation

  • Port required GitHub expression behavior.
  • Cover string functions like contains, startsWith, and endsWith.
  • Cover contexts used by existing smoke tests.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-035: Reusable workflows

  • Support local reusable workflows.
  • Support remote reusable workflow fetching.
  • Preserve remote workflow cache behavior.
  • Preserve --github-token behavior.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 4 — DTU Server

• RUST-040: Ephemeral DTU server

  • Start a local mock GitHub Actions API server.
  • Stop it cleanly after runs.
  • Preserve assigned URLs and environment values expected by runners.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-041: Runner registration/job endpoints

  • Port endpoints used by the official GitHub Actions runner.
  • Preserve runner registration flow.
  • Preserve job assignment flow.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-042: Cache API

  • Port local cache restore behavior.
  • Port local cache save behavior.
  • Preserve cache key/version matching behavior.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-043: Artifact API

  • Port artifact upload behavior.
  • Port artifact download behavior.
  • Preserve Azure block blob compatibility used by actions.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 5 — Docker Runner Execution

• RUST-050: Docker socket resolution

  • Port default Docker socket behavior.
  • Port OrbStack handling.
  • Port Docker Desktop handling.
  • Preserve diagnostics and hints.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-051: Container config builder

  • Port environment variable construction.
  • Port bind mounts.
  • Port extra hosts.
  • Port Docker network settings.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-052: Runner image handling

  • Pull the default runner image.
  • Build custom runner images from .github/agent-ci.Dockerfile.
  • Preserve image hash/tag behavior.
  • Preserve missing-tool hints.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-053: Job execution

  • Start runner containers.
  • Stream logs.
  • Collect timeline files.
  • Build job results.
  • Write run summaries.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-054: Service containers

  • Port workflow service container creation.
  • Port service networking.
  • Port service health checks.
  • Port cleanup behavior.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-055: Pause/retry on failure

  • Preserve paused container lifecycle.
  • Preserve paused runner reporting.
  • Retry failed step.
  • Retry from a specific step.
  • Retry from the start.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 6 — macOS VM Support

• RUST-060: macOS host capability detection

  • Detect supported Apple Silicon hosts.
  • Detect tart.
  • Detect sshpass.
  • Preserve skip reasons and install hints.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-061: Tart VM lifecycle

  • Clone/start VMs.
  • Resolve VM IP addresses.
  • Stop/delete VMs after runs.
  • Preserve concurrency limits.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-062: macOS runner binary caching

  • Fetch the macOS Actions runner binary.
  • Cache it on the host.
  • Reuse it across runs.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-063: macOS job execution parity

  • Copy workspace into the VM.
  • Configure runner credentials.
  • Stream logs/results back to the host.
  • Preserve skip/degraded reporting.
  • Test: cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 7 — Packaging and Distribution

• RUST-070: Native release build workflow

  • Build Linux x64 binary.
  • Build Linux arm64 binary.
  • Build macOS x64 binary.
  • Build macOS arm64 binary.
  • Sign/notarize macOS binaries if required.
  • Test: node --input-type=module -e "import fs from 'node:fs'; import YAML from 'yaml'; const doc = YAML.parse(fs.readFileSync('.github/workflows/native-binaries.yml','utf8')); if (!doc.jobs?.build?.strategy?.matrix?.include || doc.jobs.build.strategy.matrix.include.length !== 4) throw new Error('native matrix missing targets')" && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-071: npm native binary packages

  • Keep npm install @redwoodjs/agent-ci working.
  • Add platform-specific optional packages.
  • Add a small JS launcher that resolves the native binary.
  • Preserve npx @redwoodjs/agent-ci behavior.
  • Test: pnpm --filter @redwoodjs/agent-ci test && pnpm --filter @redwoodjs/agent-ci typecheck && pnpm check && pnpm --filter @redwoodjs/agent-ci build && AGENT_CI_FORCE_TYPESCRIPT=1 node packages/cli/dist/native-launcher.js --help && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-072: GitHub Release assets

  • Publish downloadable binaries.
  • Publish checksums.
  • Document direct download usage.
  • Test: node --input-type=module -e "import fs from 'node:fs'; import YAML from 'yaml'; const doc = YAML.parse(fs.readFileSync('.github/workflows/native-binaries.yml','utf8')); if (doc.permissions.contents !== 'write') throw new Error('release upload needs contents write'); const steps = doc.jobs.build.steps.map(s => s.name).filter(Boolean); if (!steps.includes('Publish GitHub Release assets')) throw new Error('missing release upload step')" && pnpm --filter @redwoodjs/agent-ci test && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-073: Homebrew formula

  • Create or update a Homebrew tap.
  • Install the native binary.
  • Add a smoke test to the formula.
  • Test: node scripts/render-homebrew-formula.mjs v0.16.1 <checksums-dir> <out-file> && grep -q 'bin.install "agent-ci"' <out-file> && grep -q 'agent-ci --help' <out-file> && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-074: Shell installer

  • Add optional curl | sh installer.
  • Detect OS and architecture.
  • Verify checksums.
  • Install into a user-selected prefix.
  • Test: AGENT_CI_BASE_URL=file://<tmp>/releases AGENT_CI_OS=linux AGENT_CI_ARCH=x64 ./install.sh --version v0.0.0 --prefix <tmp>/install && <tmp>/install/bin/agent-ci && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

Phase 8 — Validation and Migration

• RUST-080: Golden CLI tests

  • Compare TypeScript and Rust output for --help.
  • Compare common error messages.
  • Compare basic command behavior.
  • Test: pnpm golden:cli && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-081: Smoke workflow parity

  • Run existing smoke workflows through the Rust binary.
  • Track unsupported gaps.
  • Close parity gaps before switching defaults.
  • Test: pnpm rust:smoke:parity && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-082: Performance benchmarks

  • Measure startup time.
  • Measure workflow parse time.
  • Measure job orchestration overhead.
  • Compare against the TypeScript implementation.
  • Test: pnpm rust:perf && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-083: Default binary switch

  • Make the Rust implementation the default agent-ci binary.
  • Keep TypeScript fallback temporarily if needed.
  • Document rollback instructions.
  • Test: pnpm golden:cli && pnpm check && cargo fmt --all && cargo check --workspace --all-targets && cargo clippy --workspace --all-targets -- -D warnings && cargo test --workspace --all-targets

• RUST-084: Remove TypeScript implementation

  • Remove obsolete TypeScript CLI code after parity is proven.
  • Remove fallback launcher code.
  • Remove unused dependencies.
  • Update docs to describe the native implementation.
  • Blocked: top-level Rust run still has the execution gap tracked in .docs/rust-smoke-parity.md; keep the TypeScript fallback until .docs/rust-execution-parity-plan.md is complete.