skills/redwoodjs__local-ci
1
0
Fork 0
mirror of https://github.com/redwoodjs/local-ci.git synced 2026-09-19 07:54:29 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
fix/immutable-node-modules-snapshots
redwoodjs__local-ci/.docs/rfcs/rust-native-binary-rewrite.md
T
Peter Pistorius 7a349fdb43 Add opt-in Rust native execution 
## Problem

Agent CI still depended on the TypeScript runner as the main execution path. That made it harder to ship a small native binary and harder to prove the Rust path matched the TypeScript path. Review also found places where Rust and TypeScript could drift before the Rust path becomes the default.

## Solution

This PR adds the opt-in Rust native `agent-ci` binary and platform npm packages while keeping the TypeScript fallback. It splits the Rust code into `agent-ci-core`, `agent-ci-runtime`, and a thin CLI/orchestration crate. Pure planning, reusable workflow expansion, event contracts, and run-result contracts now live in core; runtime owns Docker, DTU, runner, macOS VM, workspace, execution-plan adapters, and the generic wave worker pool.

The fixture checks now use real TypeScript helpers instead of a shadow planner. Shared fixtures cover plans, events, run results, Docker socket resolution, and default job-limit CPU/memory vectors. The Rust runner honors `--jobs`, probes Docker memory for the default job limit, supports macOS VM concurrency limits, avoids nested runner container name collisions, has `--all` smoke coverage, and stages `bin/agent-ci` in every platform package.

## Validation

- `cargo check --workspace --all-targets`
- `cargo fmt --all -- --check`
- `cargo clippy --workspace --all-targets -- -D warnings`
- `cargo test --workspace --all-targets`
- `pnpm check`
- `pnpm fixtures:check`
- `pnpm golden:cli`
- `pnpm rust:smoke:parity`
- `pnpm agent-ci-dev run --all -q -p --json --github-token`

## Notes

Rust execution is still opt-in. The TypeScript implementation and fallback remain in place until the later default-switch and removal gates pass.
2026-05-24 15:22:20 +02:00

5.3 KiB
Raw Permalink Blame History

RFC: Rust Native Binary Rewrite

Status

Accepted as the working plan for the Rust rewrite checklist in ../rust-rewrite-tasks.md.

Problem

Agent CI is currently distributed as an npm-installed Node.js CLI. That works well for JavaScript users, but it ties every install path to a Node runtime and npm-compatible package installation. We want Agent CI to become a native binary so users can install it through npm, GitHub Releases, Homebrew, and other package managers without changing the CLI contract.

Goals

  • Keep the existing agent-ci command and CLI behavior stable.
  • Keep npm install @redwoodjs/agent-ci and npx @redwoodjs/agent-ci working.
  • Add direct native binary installs for users who do not want npm.
  • Preserve compatibility with the official GitHub Actions runner.
  • Preserve current pause/retry, cache, artifact, Docker, and macOS VM behavior.
  • Migrate safely with tests that compare the Rust implementation against the TypeScript implementation before switching defaults.

Non-goals

  • Do not rewrite the official GitHub Actions runner.
  • Do not remove npm distribution during the migration.
  • Do not switch the default implementation until smoke workflow parity is proven.
  • Do not intentionally change the public CLI interface as part of the rewrite.

Language choice

The native implementation will be written in Rust.

Reasons:

  • Mature ecosystem for HTTP servers/clients, async orchestration, Docker clients, YAML parsing, filesystem work, and cross-platform releases.
  • Good binary distribution story for Linux and macOS on x64 and arm64.
  • Strong type system and memory safety for a long-running orchestration tool.
  • Good test tooling for golden CLI tests, integration tests, and benchmarks.

Migration strategy

Use an incremental side-by-side migration:

  1. Add a Rust workspace and native agent-ci binary crate.
  2. Implement low-risk CLI behavior first: help text, config/env loading, clean, retry, and abort.
  3. Add golden tests that compare Rust output to the current TypeScript CLI.
  4. Port workflow parsing and scheduling.
  5. Port the DTU server.
  6. Port Docker and macOS runner execution.
  7. Run the existing smoke workflows through the Rust binary.
  8. Switch the npm package launcher to prefer the Rust binary after parity is proven.
  9. Keep a temporary TypeScript fallback during the first native release window.
  10. Remove the TypeScript implementation only after the Rust binary has passed the full smoke suite and at least one release cycle.

Supported platforms

Initial native binary targets:

  • aarch64-apple-darwin — macOS Apple Silicon
  • x86_64-apple-darwin — macOS Intel
  • x86_64-unknown-linux-gnu — Linux x64
  • aarch64-unknown-linux-gnu — Linux arm64

Windows is not an initial target because Agent CI does not currently support Windows workflow execution locally. The Rust code should avoid unnecessary Unix-only assumptions outside runner execution paths so Windows support can be evaluated later.

TypeScript fallback policy

The TypeScript implementation remains available until all of these are true:

  • The Rust binary passes golden CLI tests against the TypeScript CLI.
  • The Rust binary passes the existing smoke workflow suite.
  • npm, direct binary, and Homebrew installs all run the same Rust binary.
  • A rollback path is documented.
  • A native release has shipped without blocking regressions.

The fallback can then be removed in a later cleanup task.

Install and distribution targets

npm

The npm package remains the compatibility install path.

Planned shape:

  • @redwoodjs/agent-ci keeps the public package name and agent-ci bin entry.
  • The package ships a small JavaScript launcher.
  • The launcher resolves a platform-specific optional package when available.
  • If no platform package is available during the migration window, the launcher can fall back to the TypeScript implementation.

GitHub Releases

Each release should attach native binaries and checksums for the supported platforms. The assets are the source of truth for direct downloads, Homebrew, and the shell installer.

Homebrew

A Homebrew formula should install the macOS binaries from GitHub Releases and include a smoke test such as agent-ci --help.

Shell installer

A shell installer is optional, but if added it should:

  • Detect OS and architecture.
  • Download the matching release asset.
  • Verify checksums.
  • Install into a user-selected prefix.

Testing gates

Every checklist item should define the smallest meaningful test before being marked complete. Examples:

  • CLI tasks: Rust unit tests and golden output comparisons.
  • Config tasks: fixture-based environment and .env.agent-ci tests.
  • Workflow tasks: fixture workflows and smoke workflow parity.
  • DTU tasks: HTTP route tests and runner protocol fixtures.
  • Docker tasks: container config unit tests plus local smoke workflows.
  • Packaging tasks: archive/package smoke tests for each supported target.

Do not advance a task checkbox unless its specific tests pass.

Rollback plan

Until the default switch is complete, rollback is simple: keep the npm launcher pointed at the TypeScript implementation or restore the TypeScript fallback path. After the Rust binary becomes the only implementation, rollback should use a normal patch release that restores the previous known-good binary or re-enables the fallback launcher.

Reference in New Issue View Git Blame Copy Permalink