## Problem
Agent CI still depended on the TypeScript runner as the main execution path. That made it harder to ship a small native binary and harder to prove the Rust path matched the TypeScript path. Review also found places where Rust and TypeScript could drift before the Rust path becomes the default.
## Solution
This PR adds the opt-in Rust native `agent-ci` binary and platform npm packages while keeping the TypeScript fallback. It splits the Rust code into `agent-ci-core`, `agent-ci-runtime`, and a thin CLI/orchestration crate. Pure planning, reusable workflow expansion, event contracts, and run-result contracts now live in core; runtime owns Docker, DTU, runner, macOS VM, workspace, execution-plan adapters, and the generic wave worker pool.
The fixture checks now use real TypeScript helpers instead of a shadow planner. Shared fixtures cover plans, events, run results, Docker socket resolution, and default job-limit CPU/memory vectors. The Rust runner honors `--jobs`, probes Docker memory for the default job limit, supports macOS VM concurrency limits, avoids nested runner container name collisions, has `--all` smoke coverage, and stages `bin/agent-ci` in every platform package.
## Validation
- `cargo check --workspace --all-targets`
- `cargo fmt --all -- --check`
- `cargo clippy --workspace --all-targets -- -D warnings`
- `cargo test --workspace --all-targets`
- `pnpm check`
- `pnpm fixtures:check`
- `pnpm golden:cli`
- `pnpm rust:smoke:parity`
- `pnpm agent-ci-dev run --all -q -p --json --github-token`
## Notes
Rust execution is still opt-in. The TypeScript implementation and fallback remain in place until the later default-switch and removal gates pass.
* fix(cli): unblock --pause-on-failure in non-TTY pipes (#315)
Spawn the run as a detached worker when stdout isn't a TTY (and we're
not under an LLM harness), then exit 77 the moment the worker emits a
`run.paused` NDJSON event — freeing the caller's pipe while the worker
keeps the container + DTU alive for `agent-ci retry`. Retry tails the
same log so a re-failure surfaces as another exit-77.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cli): NDJSON event stream behind --json (#289)
Extends the existing pause/finish sentinel emitter into a full lifecycle
event stream. New `--json` flag (and `AGENT_CI_JSON=1` env var) emits
`run.start`, `job.start`/`finish`, `step.start`/`finish`, `run.paused`,
`run.finish`, `diagnostic` events on stdout — one JSON object per line —
each carrying an `event` discriminator and a `schemaVersion: 1` on
`run.start`.
`--json` is decoupled from `--quiet` so existing `-q` callers keep silent
stdout; the diff renderer is auto-suppressed under `--json` so ANSI
sequences don't collide with the stream. Detached-worker mode (#315)
keeps emitting only sentinels — the launcher filters non-sentinel events
anyway, so non-JSON callers don't pay the cost.
Renames `run.completed` -> `run.finish` to match the issue's naming.
Closes#289.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(cli): suppress printSummary under --json
The ━━━ FAILURES ━━━ / ━━━ SUMMARY ━━━ blocks write to stdout
unconditionally, which interleaves with the NDJSON event stream and
breaks line-based parsers. Skip them when isJsonMode() — consumers get
the same information from job.finish + run.finish events.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cli): AGENT_CI_DETACHED=1 forces detached launcher on TTY
Overload AGENT_CI_DETACHED so it can serve both roles:
- `=<absolute path>` → set by the launcher on its worker child;
`isDetachedWorker()` returns true.
- `=1` (or any non-absolute value) → caller-side opt-in: force the
launcher path even when stdout is a TTY. Useful for manual
verification of the #315 pause-then-exit-77 flow without having to
construct a non-TTY pipe.
The two are distinguished by whether the value starts with `/`. New
`isForceDetachedRequested()` helper drives a `forceDetached` arg into
`shouldLaunchDetached`, which still respects `alreadyWorker`,
`agentMode`, and the `pauseOnFailure` requirement.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test(ci): smoke for --json NDJSON event stream (#289)
scripts/smoke-289.sh creates a tmp repo with a 2-step passing workflow,
runs `agent-ci run --json`, and asserts:
- run.start carries schemaVersion=1
- step.start + step.finish for each declared step
- job.finish status=passed
- run.finish status=passed
- every JSON-shaped line on stdout has a recognized event field
.github/workflows/smoke-json-events.yml runs the script in CI on every
PR (gated by the same safe-to-run / collaborator predicate as the other
smokes).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix(ci): stop smoke-resource-mismatch from hanging on real GitHub runners
The fixture in .github/workflows/smoke-resource-mismatch.yml declares
`runs-on: ubuntu-latest-999-cores` so agent-ci's resource-fidelity
classifier deterministically tags the job as `degraded` on any host.
With its previous `pull_request_target` trigger, GitHub Actions also
picked it up on every PR and queued a job that no real runner can
satisfy — five queued runs piled up on PR #323 alone, each timing out
after hours.
Switch the workflow to `workflow_dispatch:`-only so GitHub never
auto-dispatches it, and teach `isWorkflowRelevant` to treat
`workflow_dispatch:`-only workflows as relevant for `agent-ci run --all`
so the local smoke keeps running. The opt-in is gated to the sole-event
case to avoid overriding `pull_request` / `push` path/branch filters on
workflows that legitimately combine triggers.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
DOCKER_HOST collides with the shell's Docker CLI expectation and can't
live in .env.agent-ci without leaking to every docker command. Rename
to AGENT_CI_DOCKER_HOST and hard-exit if the legacy DOCKER_HOST is set.
At bootstrap, apply AGENT_CI_*-prefixed keys from .env.agent-ci to
process.env so Docker/network config persists without shell exports.
Closes#308.
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* feat(cli): run macos-* jobs in real macOS VMs via tart
When the host is darwin/arm64 with tart + sshpass installed, jobs
targeting runs-on: macos-* boot an ephemeral cirruslabs macOS VM,
rsync the macOS actions-runner binary in, and connect it to the
in-process DTU via the host bridge.
- New runner/macos-vm/ module (tart wrapper, image mapping, host
capability probe, runner binary fetch/cache, semaphore, job driver).
- cli.ts routes macOS jobs to the VM executor when the host supports
it; otherwise falls through to the existing skip-with-warning path.
- DTU generators.ts accepts runnerWorkDir/runnerOs/runnerArch so the
seed payload reflects where the runner actually materializes _work
(sibling of run.sh, not a caller-chosen path).
- runner-credentials.ts extracted from local-job.ts so the macOS path
can reuse the same deterministic .runner/.credentials generation.
Concurrency capped at 2 VMs by default (AGENT_CI_MACOS_VM_CONCURRENCY).
Image override via AGENT_CI_MACOS_VM_IMAGE.
Closes#258.
* feat(cli): say *why* macOS jobs skip on incapable hosts
Before: the skip warning for macos-* jobs on a host that can't run the
VM just pointed at #258 and parroted the pre-Phase-3 line about only
running in a Linux container — which is no longer accurate now that
macOS jobs *can* run locally when the host supports it.
Now: formatUnsupportedOSWarning accepts the HostCapability result and
prints the specific reason + install hint when available — e.g.
'tart not installed' with a brew command, or 'requires Apple Silicon
(got x64)'.
* refactor(cli): deduplicate runs-on classification + warning body
- cli.ts: extract classifyJob(ej) + canRunMacosHere(kind) so the
skip path and the executor-routing path share one source of truth
for 'should this job run in the macOS VM?' — the two previously
duplicated the parseJobRunsOn + classifyRunsOn + capability check.
- runs-on-compat.ts: flatten formatUnsupportedOSWarning to a single
body array per kind, filter undefined, then uniformly indent —
drops the optional-hint if-branch and the per-push indent literals.
Pure refactor; all 14 runs-on-compat tests still pass, warning output
is byte-identical across all 4 permutations (macOS with/without
capability, with/without hint, windows).
* docs: document macOS VM runner support
- README: add tart + sshpass to 'Optional' prerequisites and a new
'macOS jobs' section (image table, env vars, caveats).
- compatibility.md: unstale the three entries that claimed Linux-only
(jobs.<id>.runs-on, runner.os, runner.arch). runs-on now reads as a
partial (⚠️) that explains linux/macos/windows separately.
Leaves apps/website/CompatibilityMatrix.tsx alone — it has the same
stale rows but is in an unrelated mid-merge state on this branch.
* chore: oxfmt packages/cli/README.md and compatibility.md
* kindling: start task [skip ci]
* Add behavioral specs and tests for vars context support (SPEC-V-001 through SPEC-V-015)
Tests cover loading vars from dotenv file, env-var allowlist fallback, file-over-env
precedence, extracting ${{ vars.X }} refs from workflow YAML (whole-file and job-scoped),
validating that all referenced vars are supplied (error names missing keys and config file
path), and that workflows with no vars refs run without any config present. Tests mirror
the existing secrets test conventions and will pass once the implementation (Track A) is
in place.
* harness: save uncommitted work from Developer+QA (cycle 1) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 2) [skip ci]
* harness: save uncommitted work from Developer (cycle 3) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 4) [skip ci]
* Add vars context loading, extraction, validation, and expression substitution
- loadMachineVars in config.ts reads .env.agent-ci.vars with the same
KEY=VALUE parsing and envFallbackKeys allowlist as loadMachineSecrets
- extractVarRefs in workflow-parser.ts scans workflow YAML for
${{ vars.NAME }} references using the same pattern as extractSecretRefs
- validateVars throws a descriptive error listing missing var names and
the expected file path, mirroring validateSecrets
- resolveExprAtom gains a vars? 7th param and a vars. prefix case after
the secrets. case; all 11 internal evaluateExprValue calls for
fromJSON/toJSON/format/contains/startsWith/endsWith/join updated
- evaluateExprValue gains vars? 7th param threaded through all 6
recursive calls (||, &&, !, comparisons, parens) and the
resolveExprAtom leaf at line 606
- expandExpressions and parseWorkflowSteps each gain vars? as optional
7th param; all internal expandExpressions call sites updated
* Wire vars context into all three CLI call sites
- Import loadMachineVars, extractVarRefs, validateVars from their
respective modules
- Single-job path: extract refs, load vars, validate, thread vars
through expandExpressions (inputsContext expansion) and
parseWorkflowSteps
- buildJob: extract refs, load vars, validate (no parseWorkflowSteps
call here, consistent with how secrets are handled)
- runJob: extract refs, load vars, validate, thread vars through
resolveInputsForJob and parseWorkflowSteps
- resolveInputsForJob: accepts vars? as 4th optional param and threads
it through its expandExpressions call
* Add Vars section to README and minor-bump changeset
README documents .env.agent-ci.vars file and shell env fallback,
explaining that vars are non-sensitive and can be committed.
Changeset bumps both @redwoodjs/agent-ci and dtu-github-actions minor
as they are in the same fixed versioning group.
* Fix loadMachineVars test helper to write .env.agent-ci.vars
The QA-authored loadMachineVars tests used .env.agent-ci as the
fixture file name, but loadMachineVars reads from .env.agent-ci.vars
per the approved RFC. Updated the writeEnvFile helper and test
description in the loadMachineVars block to match the correct file.
* harness: save uncommitted work from Developer (cycle 5) [skip ci]
* Replace file+env vars sourcing with --var CLI flag
Address PR feedback: workflow vars (${{ vars.FOO }}) are now supplied
exclusively via the repeatable --var KEY=VALUE CLI flag. Removes the
.env.agent-ci.vars file and the shell-env fallback so workflow vars
don't overload shell env vars. Missing vars fail fast with an error
listing the --var flags needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Pre-flight missing vars across all workflows before setup
Scan every workflow for ${{ vars.* }} refs at the start of the run
and fail with one combined message listing every missing var across
every file. Happens before Docker prefetch and workspace setup, so
users find out immediately instead of waiting through bootstrap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add smoke test for --var pre-flight
Covers three cases against a temp workflow referencing vars.API_URL
and vars.DEPLOY_ENV:
1. No --var flags → exit 1, error lists both vars with --var syntax
2. Partial fill → error lists only the still-missing var
3. All vars supplied → run proceeds and the job succeeds
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: kindling <kindling@harness>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* docs: promote packages/cli/README as the main README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: symlink root README to packages/cli/README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: reframe Why section — remote CI is the gatekeeper, agent-ci is the pre-flight check
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: note Docker Desktop is also supported on Linux
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip orphan cleanup when running inside a Docker container
When agent-ci runs nested inside a container (e.g. smoke-bun-setup step 8),
it shares the host's Docker socket. killOrphanedContainers() checks
agent-ci.pid labels using process.kill(pid, 0), but host PIDs don't
exist in the container's PID namespace — so every container looks like
an orphan. This caused the nested agent-ci to force-remove its own
parent container (exit 137).
Both pruneOrphanedDockerResources() and killOrphanedContainers() now
bail out when /.dockerenv exists.
Fixes#228
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: ignore inherited AGENT_CI_DTU_HOST inside containers
When agent-ci runs nested inside a container (e.g. smoke-bun-setup),
the outer container's AGENT_CI_DTU_HOST=host.docker.internal is
inherited. The nested agent-ci used this to tell the inner container
where the DTU is, but the nested DTU is inside the outer container,
not on the host. The inner container got "Connection refused" trying
to reach host.docker.internal:PORT.
Now when /.dockerenv exists, AGENT_CI_DTU_HOST is ignored and the
container's own IP (via hostname -I) is used instead, so sibling
containers can reach the DTU.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move /.dockerenv guard to call site so tests still work
The /.dockerenv check inside killOrphanedContainers() broke the
shutdown integration tests — they run inside a Docker container (the
test runner) and need the function to work. Moved the guard to the
session bootstrap in cli.ts where the nested agent-ci problem occurs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add global concurrency limiter across all workflows
In --all mode, each workflow created its own per-wave limiter, allowing
20+ containers to run simultaneously and exhaust available memory (OOM
kills with exit 137). Now a single global limiter (CPU/2) is created in
the orchestrator and shared by all handleWorkflow calls, including
single-job workflows and cold-cache first jobs.
Closes#225
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: factor Docker VM memory into concurrency limit
CPU-based limit alone (cpuCount/2 = 7) still causes OOM kills. Now
reads MemAvailable from /proc/meminfo inside the Docker VM and caps
at floor(available / 3GB per container). Falls back to docker info
with a 4GB reserve if busybox isn't available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard orphan cleanup in the function, not just the call site
The /.dockerenv guard was only in cli.ts, but killOrphanedContainers
is also called directly by the Docker integration tests (via a spawned
tsx subprocess). Inside a container, the test's subprocess would see
the parent container's PID label, misidentify it as an orphan, and
kill the container running the tests.
Put the guard back in both functions AND keep the cli.ts guard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase per-container memory estimate to 4 GB
3 GB per container wasn't enough — tests.yml (vitest + Docker
integration tests) and smoke-binary.yml (full build) both got killed
at 3 concurrent containers. 4 GB gives 2 concurrent containers on a
12 GB Docker VM, with ~5.4 GB each.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: mock fs.existsSync in shutdown unit tests for /.dockerenv guard
The unit tests mock execSync but not fs. Inside a runner container,
the real fs.existsSync("/.dockerenv") returns true, causing
killOrphanedContainers to bail before reaching the mocked execSync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip Docker integration tests for orphan cleanup inside containers
The integration tests spawn a real killOrphanedContainers() subprocess.
Inside a container, the function bails due to /.dockerenv (PID namespace
mismatch), so the tests can't exercise it. Skip them with insideDocker().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add --jobs flag and document concurrency
Adds -j/--jobs <n> to override the auto-detected concurrency limit.
Adds a Concurrency section to the README explaining how the limit is
calculated and how to override it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(website): include install steps in copy agent instructions
The "Copy agent instructions" button and Terminal preview now include
setup steps: installing the package (with package manager detection for
Node.js projects, or npx for non-Node projects) and installing the
agent skill.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: service containers leak on unclean shutdown
Problem: Service containers (e.g. `agent-ci-*-svc-cache-db`) were created
without the `agent-ci.pid` label, making them invisible to the bootstrap
orphan cleanup (`killOrphanedContainers`). When the parent process died
without hitting its signal handler (crash, SIGKILL, Bash tool timeout),
these containers leaked permanently. Additionally, SIGHUP was not handled,
so closing a tmux pane or SSH disconnect also leaked runner containers.
Containers stuck in `created` state were also missed by the prune logic.
Solution: Label service containers with `agent-ci.pid` at creation time.
Update `killOrphanedContainers` to clean up unlabeled containers (handles
pre-fix containers) and derive the runner name from svc container names
(strip `-svc-*` suffix) with deduplication. Add SIGHUP to signal handlers
in both `cli.ts` and `local-job.ts`. Extend `pruneOrphanedDockerResources`
to also prune `status=created` containers.
Fixes#216
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use XDG cache on Linux Docker Desktop so bind mounts work (#215)
`/tmp` is not in Docker Desktop's default shared-folder list on Linux,
so bind mounts from `/tmp` silently fail. Detect Linux Docker Desktop
via `~/.docker/desktop/docker.sock` and fall back to `$XDG_CACHE_HOME`
(or `~/.cache`). Also update the validate command to run agent-ci in
the background with Monitor for proper pause-on-failure support, and
note Docker Desktop as a supported Linux provider in the README.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove stop hook that ran agent-ci synchronously
The stop hook can't support pause-on-failure since it runs after the
conversation ends. Use `/validate` instead, which runs agent-ci in the
background with Monitor support for fixing and retrying failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make GitHub auth opt-in for remote reusable workflow fetching
Remove implicit credential resolution (gh auth token / GITHUB_TOKEN fallback)
that could silently send user credentials. Auth is now opt-in via
--github-token CLI flag or AGENT_CI_GITHUB_TOKEN env var.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* chore: add changeset for opt-in GitHub auth
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: add --github-token flag and AGENT_CI_GITHUB_TOKEN to docs
Update CLI reference tables, environment variables section, and
compatibility matrix to reflect opt-in GitHub auth for remote
reusable workflows.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* fix: make commit status posting opt-in via --github-token
Gate postCommitStatus() on the githubToken parameter. When no token
is provided, print a warning instead of silently using implicit
gh CLI credentials.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: add --commit-status flag to opt in to posting GitHub commit statuses
Commit status posting now requires both --commit-status and
--github-token. Without --commit-status, no warning is printed
and no status is posted.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add tests for 403 error hint and public repo fetch without auth
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Limit Docker API URL rewrites to loopback hosts, keep alias-based defaults unless an explicit bridge gateway override is configured, and update docs/tests to match behavior.
Peter Pistorius