* fix(runner): chmod /home/runner/_diag and _work for Colima/Docker Desktop (#263)
On macOS with Colima or Docker Desktop the bind-mounted _diag and _work
directories surface as root:root 0755 inside the container (host perms
don't translate through the VM mount layer), so the runner user (uid
1001) crashes with UnauthorizedAccessException when writing diag logs
and hits write failures in the workspace. Add MAYBE_SUDO chmod 1777 on
both mount points during container boot, mirroring the existing fix
for /home/runner/.cache (#234). OrbStack and native Linux Docker are
unaffected — the chmod is a no-op there.
* fix(cli): require /var/run/docker.sock, link to per-provider setup doc
Today resolveDockerSocket falls back through a list of well-known
provider sockets (~/.orbstack/..., ~/.docker/..., ~/.colima/<profile>/...)
whenever /var/run/docker.sock isn't usable. On macOS + Colima, those
provider paths are not bind-mountable through the provider's VM, and
Docker rejects the later container launch with a confusing
error while creating mount source path: operation not supported
Stop guessing. Require /var/run/docker.sock to exist (or an explicit
DOCKER_HOST) and fail fast with a short error that links to the new
packages/cli/docs/docker-socket.md — which walks users through the
per-provider setup (Docker Desktop / OrbStack / Colima / native Linux
/ rootless).
The bind-mount invariant comment in docker-socket.ts was already
implicitly assuming this rule; this change just enforces it at the
resolver boundary instead of deferring the failure to container
launch, and gives users a doc URL with the recipe they need.
* feat(cli): run macos-* jobs in real macOS VMs via tart
When the host is darwin/arm64 with tart + sshpass installed, jobs
targeting runs-on: macos-* boot an ephemeral cirruslabs macOS VM,
rsync the macOS actions-runner binary in, and connect it to the
in-process DTU via the host bridge.
- New runner/macos-vm/ module (tart wrapper, image mapping, host
capability probe, runner binary fetch/cache, semaphore, job driver).
- cli.ts routes macOS jobs to the VM executor when the host supports
it; otherwise falls through to the existing skip-with-warning path.
- DTU generators.ts accepts runnerWorkDir/runnerOs/runnerArch so the
seed payload reflects where the runner actually materializes _work
(sibling of run.sh, not a caller-chosen path).
- runner-credentials.ts extracted from local-job.ts so the macOS path
can reuse the same deterministic .runner/.credentials generation.
Concurrency capped at 2 VMs by default (AGENT_CI_MACOS_VM_CONCURRENCY).
Image override via AGENT_CI_MACOS_VM_IMAGE.
Closes#258.
* feat(cli): say *why* macOS jobs skip on incapable hosts
Before: the skip warning for macos-* jobs on a host that can't run the
VM just pointed at #258 and parroted the pre-Phase-3 line about only
running in a Linux container — which is no longer accurate now that
macOS jobs *can* run locally when the host supports it.
Now: formatUnsupportedOSWarning accepts the HostCapability result and
prints the specific reason + install hint when available — e.g.
'tart not installed' with a brew command, or 'requires Apple Silicon
(got x64)'.
* refactor(cli): deduplicate runs-on classification + warning body
- cli.ts: extract classifyJob(ej) + canRunMacosHere(kind) so the
skip path and the executor-routing path share one source of truth
for 'should this job run in the macOS VM?' — the two previously
duplicated the parseJobRunsOn + classifyRunsOn + capability check.
- runs-on-compat.ts: flatten formatUnsupportedOSWarning to a single
body array per kind, filter undefined, then uniformly indent —
drops the optional-hint if-branch and the per-push indent literals.
Pure refactor; all 14 runs-on-compat tests still pass, warning output
is byte-identical across all 4 permutations (macOS with/without
capability, with/without hint, windows).
* docs: document macOS VM runner support
- README: add tart + sshpass to 'Optional' prerequisites and a new
'macOS jobs' section (image table, env vars, caveats).
- compatibility.md: unstale the three entries that claimed Linux-only
(jobs.<id>.runs-on, runner.os, runner.arch). runs-on now reads as a
partial (⚠️) that explains linux/macos/windows separately.
Leaves apps/website/CompatibilityMatrix.tsx alone — it has the same
stale rows but is in an unrelated mid-merge state on this branch.
* chore: oxfmt packages/cli/README.md and compatibility.md
Jobs with runs-on: macos-* or windows-* were silently routed to the
Linux runner container and failed at the first OS-specific step.
Parse runs-on (string, array, or {group,labels} form), classify the
OS, and skip the job with a clear stderr warning when it is not Linux.
Linux jobs are unchanged. macOS support lands later via tart (#258).
* test: smoke reproducer for #257 docker/setup-buildx-action
* chore: retrigger CI
* ci: switch smoke to pull_request so it triggers from the PR branch
* ci(smoke): dump agent-ci step logs + upload as artifact on failure
* fix: use MAYBE_SUDO to chmod bind-mounted docker socket (#257)
On native Linux Docker the socket is owned root:docker 0660. The runner
user isn't in the host's docker group, so the existing plain 'chmod 666
/var/run/docker.sock' silently failed — leaving every subsequent Docker
client call (docker/setup-buildx-action@v4, docker login, compose, buildx)
to fail with 'permission denied while trying to connect to the docker
API'. Escalating via MAYBE_SUDO matches the surrounding entrypoint ops.
* ci(smoke): assert against Verify-buildx.log, not top-level agent-ci stdout
---------
Co-authored-by: Peter Pistorius <peter@redwoodjs.com>
The workflow parser only read env: declared directly on a step.
Workflow-level and job-level env: blocks were silently ignored, so
any workflow that relied on them — including the smoke-vars-preflight
smoke test that uses a job-level env with a vars.X reference — saw
empty values at runtime.
The parser now merges env: from all three levels per the GitHub
Actions spec: workflow → job → step, step-most-specific wins.
Expressions (${{ vars.X }}, ${{ secrets.X }}, etc.) are expanded on
each value with the full context, so var/secret references resolve
identically regardless of which level declares the env.
Implementation:
* New internal buildStepEnv() helper in workflow-parser.ts that
merges the three levels and runs expandExpressions per value.
* Both the run: and uses: step branches now call it, dropping
the duplicated inline Env-construction logic.
* 5 new unit tests covering job-only, workflow-only, three-level
precedence, vars expansion in job env, and the no-env case.
Fixes the Case 3 failure in .github/workflows/smoke-vars-preflight.yml
that #270 uncovered — Case 3 had been silently passing via env leak
from the outer runner process, not via actual var substitution.
* kindling: start task [skip ci]
* harness: save uncommitted work from Analyst (cycle 1) [skip ci]
* Fix misleading 401/403 error hint when GitHub token is already provided
When fetching a remote workflow from a private repo with a token supplied,
a 401 or 403 from GitHub now says the token may be invalid, expired, or
lack the required 'repo' scope — instead of repeating instructions to use
--github-token that the user already followed.
Added two tests covering the with-token 401 and 403 paths, which were
previously uncovered.
Fixes#255
* harness: save uncommitted work from Developer (cycle 2) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 3) [skip ci]
* harness: pre-DONE sweep [skip ci]
* kindling: ready for CI
* harness: save uncommitted work from Analyst (cycle 1) [skip ci]
* harness: save uncommitted work from Developer (cycle 2) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 3) [skip ci]
* harness: save uncommitted work from Developer (cycle 4) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 5) [skip ci]
* harness: pre-DONE sweep [skip ci]
* fix: handle 404 on remote workflow fetch as private-repo auth failure
GitHub's Contents API returns HTTP 404 (not 401/403) when a caller lacks
access to a private repository — this is intentional to avoid leaking repo
existence. The previous error hint only covered 401/403, so users hitting
the most common private-repo failure modes (missing token, classic PAT
without 'repo' scope, fine-grained PAT without 'contents: read', unauthorized
SSO PAT) saw a bare "HTTP 404" with no guidance.
Unify hint generation into buildAuthHint() covering 401/403/404 × with/
without token. Hints now include explicit gh auth login instructions and
spell out the token-insufficient scenarios.
Verified end-to-end against a private repro repo (peterp/agent-ci-private):
no-token → 404 path now emits full auth guidance, invalid-token → 401 path
emits token-insufficient hint, valid-token → resolves.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* test: smoke test for private-repo remote workflow fetch (#261)
Adds .github/workflows/smoke-remote-private-workflow.yml — the end-to-end
verification workflow used to reproduce and prove the #261 fix, now preserved
as a regression-guard smoke test.
It references a reusable workflow in a private repo
(peterp/agent-ci-private/.github/workflows/lint.yml@main). Under agent-ci this
exercises prefetchRemoteWorkflows against the GitHub Contents API with the
supplied token, covering the success path (HTTP 200 from a private repo when
a valid token is attached) that the unit tests in
packages/cli/src/workflow/remote-workflow-fetch.test.ts don't cover.
Verified locally with 'pnpm agent-ci-dev run --workflow … --github-token':
Status: ✓ 2 passed (2 total)
Duration: 9s
Without a token, the same smoke surfaces the improved #261 hint chain verbatim
(HTTP 404 → 'The repository or ref was not found …' → auth instructions).
---------
Co-authored-by: kindling <kindling@harness>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* kindling: start task [skip ci]
* Add behavioral specs and tests for vars context support (SPEC-V-001 through SPEC-V-015)
Tests cover loading vars from dotenv file, env-var allowlist fallback, file-over-env
precedence, extracting ${{ vars.X }} refs from workflow YAML (whole-file and job-scoped),
validating that all referenced vars are supplied (error names missing keys and config file
path), and that workflows with no vars refs run without any config present. Tests mirror
the existing secrets test conventions and will pass once the implementation (Track A) is
in place.
* harness: save uncommitted work from Developer+QA (cycle 1) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 2) [skip ci]
* harness: save uncommitted work from Developer (cycle 3) [skip ci]
* harness: save uncommitted work from Reviewer (cycle 4) [skip ci]
* Add vars context loading, extraction, validation, and expression substitution
- loadMachineVars in config.ts reads .env.agent-ci.vars with the same
KEY=VALUE parsing and envFallbackKeys allowlist as loadMachineSecrets
- extractVarRefs in workflow-parser.ts scans workflow YAML for
${{ vars.NAME }} references using the same pattern as extractSecretRefs
- validateVars throws a descriptive error listing missing var names and
the expected file path, mirroring validateSecrets
- resolveExprAtom gains a vars? 7th param and a vars. prefix case after
the secrets. case; all 11 internal evaluateExprValue calls for
fromJSON/toJSON/format/contains/startsWith/endsWith/join updated
- evaluateExprValue gains vars? 7th param threaded through all 6
recursive calls (||, &&, !, comparisons, parens) and the
resolveExprAtom leaf at line 606
- expandExpressions and parseWorkflowSteps each gain vars? as optional
7th param; all internal expandExpressions call sites updated
* Wire vars context into all three CLI call sites
- Import loadMachineVars, extractVarRefs, validateVars from their
respective modules
- Single-job path: extract refs, load vars, validate, thread vars
through expandExpressions (inputsContext expansion) and
parseWorkflowSteps
- buildJob: extract refs, load vars, validate (no parseWorkflowSteps
call here, consistent with how secrets are handled)
- runJob: extract refs, load vars, validate, thread vars through
resolveInputsForJob and parseWorkflowSteps
- resolveInputsForJob: accepts vars? as 4th optional param and threads
it through its expandExpressions call
* Add Vars section to README and minor-bump changeset
README documents .env.agent-ci.vars file and shell env fallback,
explaining that vars are non-sensitive and can be committed.
Changeset bumps both @redwoodjs/agent-ci and dtu-github-actions minor
as they are in the same fixed versioning group.
* Fix loadMachineVars test helper to write .env.agent-ci.vars
The QA-authored loadMachineVars tests used .env.agent-ci as the
fixture file name, but loadMachineVars reads from .env.agent-ci.vars
per the approved RFC. Updated the writeEnvFile helper and test
description in the loadMachineVars block to match the correct file.
* harness: save uncommitted work from Developer (cycle 5) [skip ci]
* Replace file+env vars sourcing with --var CLI flag
Address PR feedback: workflow vars (${{ vars.FOO }}) are now supplied
exclusively via the repeatable --var KEY=VALUE CLI flag. Removes the
.env.agent-ci.vars file and the shell-env fallback so workflow vars
don't overload shell env vars. Missing vars fail fast with an error
listing the --var flags needed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Pre-flight missing vars across all workflows before setup
Scan every workflow for ${{ vars.* }} refs at the start of the run
and fail with one combined message listing every missing var across
every file. Happens before Docker prefetch and workspace setup, so
users find out immediately instead of waiting through bootstrap.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* Add smoke test for --var pre-flight
Covers three cases against a temp workflow referencing vars.API_URL
and vars.DEPLOY_ENV:
1. No --var flags → exit 1, error lists both vars with --var syntax
2. Partial fill → error lists only the still-missing var
3. All vars supplied → run proceeds and the job succeeds
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: kindling <kindling@harness>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: setup-node "Bad credentials" + stale-toolcache hint (#171, #249)
Route setup-node's versions-manifest fetch through the DTU so it no
longer 401s against real api.github.com, and surface an actionable
hint when a stale /opt/hostedtoolcache bind mount blocks tar extract.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: unit + smoke coverage for setup-node manifest mock
Adds unit tests for rewriteSetupNodeTarball and for the cached tree/blob
route paths, plus a nested-agent-ci smoke workflow that fails if
setup-node logs "Bad credentials" (the symptom of the manifest mock
breaking and falling back to nodejs.org).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: promote packages/cli/README as the main README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: symlink root README to packages/cli/README
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: reframe Why section — remote CI is the gatekeeper, agent-ci is the pre-flight check
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: note Docker Desktop is also supported on Linux
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
First-time users saw a frozen spinner or "No such image" error because
the runner image pull was silent and failures were only debug-logged.
Now the CLI detects a missing image, shows download/extraction progress,
and surfaces pull errors visibly.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Actions like dorny/paths-filter read github.event.repository.default_branch
to determine the base branch. The DTU was already passing this value in
Resources.Repositories but omitting it from the event context.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: always show workflows and jobs in --all mode, fix duplicate matrix jobs
Two problems fixed:
1. Jobs would disappear. When running --all, jobs under a workflow
(like install-a, install-b, install-c) would only show up while
the workflow was running. Before it started or after it finished,
they'd vanish. Now they always stay visible.
2. Matrix jobs showed up twice. When a workflow had matrix shards
(like 3 copies of the same job), the display would show 5 entries
instead of 3. This happened because each job got two different
names — one when we reserved its spot, and a different one when
it actually started running. The second name didn't match the
first, so the system thought they were different jobs. Now both
use the same name.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: reduce spinner flicker in --all mode with line-diff renderer
Replace log-update's full erase-and-rewrite approach with a line-level
diff renderer that only sends ANSI escape codes for changed lines. For a
30-line tree where only spinner characters change, this skips ~90% of
terminal I/O per frame.
Also make job cleanup non-blocking: convert fs.rmSync to fs.promises.rm
and store.save() to async fs operations so completing runners don't
freeze the event loop (and stall all other spinners).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: unblock event loop during workspace prep to keep spinners smooth
Convert initFakeGitRepo (7 sequential execSync calls) and the
git-archive path in prepareWorkspace to use promisified exec. Each
await yields to the event loop between git process spawns, letting
the render timer fire instead of freezing all spinners for ~200ms.
Also convert the chmod -R 777 call to async exec, and wrap diff-renderer
updates in DEC synchronized output markers so terminals that support it
batch the entire frame into a single repaint.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: return Promise from save() and update tests to await it
The async save() was fire-and-forget, so tests checking the file
immediately after save() failed. Return the Promise so callers
(including tests) can await completion when needed.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add smoke test reproducing Cypress cache permission failure (#234)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: make /home/runner/.cache world-writable so Cypress can install (#234)
The Playwright bind mount causes Docker to create /home/runner/.cache as
root, so Cypress postinstall fails with EACCES when it tries to mkdir
/home/runner/.cache/Cypress. chmod 1777 in the entrypoint fixes it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip orphan cleanup when running inside a Docker container
When agent-ci runs nested inside a container (e.g. smoke-bun-setup step 8),
it shares the host's Docker socket. killOrphanedContainers() checks
agent-ci.pid labels using process.kill(pid, 0), but host PIDs don't
exist in the container's PID namespace — so every container looks like
an orphan. This caused the nested agent-ci to force-remove its own
parent container (exit 137).
Both pruneOrphanedDockerResources() and killOrphanedContainers() now
bail out when /.dockerenv exists.
Fixes#228
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: ignore inherited AGENT_CI_DTU_HOST inside containers
When agent-ci runs nested inside a container (e.g. smoke-bun-setup),
the outer container's AGENT_CI_DTU_HOST=host.docker.internal is
inherited. The nested agent-ci used this to tell the inner container
where the DTU is, but the nested DTU is inside the outer container,
not on the host. The inner container got "Connection refused" trying
to reach host.docker.internal:PORT.
Now when /.dockerenv exists, AGENT_CI_DTU_HOST is ignored and the
container's own IP (via hostname -I) is used instead, so sibling
containers can reach the DTU.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move /.dockerenv guard to call site so tests still work
The /.dockerenv check inside killOrphanedContainers() broke the
shutdown integration tests — they run inside a Docker container (the
test runner) and need the function to work. Moved the guard to the
session bootstrap in cli.ts where the nested agent-ci problem occurs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add global concurrency limiter across all workflows
In --all mode, each workflow created its own per-wave limiter, allowing
20+ containers to run simultaneously and exhaust available memory (OOM
kills with exit 137). Now a single global limiter (CPU/2) is created in
the orchestrator and shared by all handleWorkflow calls, including
single-job workflows and cold-cache first jobs.
Closes#225
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: factor Docker VM memory into concurrency limit
CPU-based limit alone (cpuCount/2 = 7) still causes OOM kills. Now
reads MemAvailable from /proc/meminfo inside the Docker VM and caps
at floor(available / 3GB per container). Falls back to docker info
with a 4GB reserve if busybox isn't available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard orphan cleanup in the function, not just the call site
The /.dockerenv guard was only in cli.ts, but killOrphanedContainers
is also called directly by the Docker integration tests (via a spawned
tsx subprocess). Inside a container, the test's subprocess would see
the parent container's PID label, misidentify it as an orphan, and
kill the container running the tests.
Put the guard back in both functions AND keep the cli.ts guard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase per-container memory estimate to 4 GB
3 GB per container wasn't enough — tests.yml (vitest + Docker
integration tests) and smoke-binary.yml (full build) both got killed
at 3 concurrent containers. 4 GB gives 2 concurrent containers on a
12 GB Docker VM, with ~5.4 GB each.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: mock fs.existsSync in shutdown unit tests for /.dockerenv guard
The unit tests mock execSync but not fs. Inside a runner container,
the real fs.existsSync("/.dockerenv") returns true, causing
killOrphanedContainers to bail before reaching the mocked execSync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip Docker integration tests for orphan cleanup inside containers
The integration tests spawn a real killOrphanedContainers() subprocess.
Inside a container, the function bails due to /.dockerenv (PID namespace
mismatch), so the tests can't exercise it. Skip them with insideDocker().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add --jobs flag and document concurrency
Adds -j/--jobs <n> to override the auto-detected concurrency limit.
Adds a Concurrency section to the README explaining how the limit is
calculated and how to override it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: catch any missing tool in runner image hint, add Nix recipe and docs
Closes#229
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Revert "feat: catch any missing tool in runner image hint, add Nix recipe and docs"
This reverts commit ca261639e9.
* feat: generic missing-tool detection with Nix/xz recipe
Replace the hardcoded per-tool pattern list with a single generic
"command not found" regex that catches any missing tool. Add detection
for the Nix installer's "you do not have 'xz' installed" message.
Update hint to point to runner-image.md docs instead of inlining a
Dockerfile snippet. Add Nix recipe to runner-image.md and a runner
image section to the README.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: support boolean operators and built-in functions in workflow expressions (#224)
The workflow expression evaluator only understood simple variable lookups
and hashFiles(). Expressions like `!github.event.pull_request && format(...) || 'plan'`
would silently fail, producing wrong values.
Add support for `&&`, `||`, `!`, parentheses, `format()`, `toJSON()`,
and string literals so these expressions evaluate correctly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* revert: drop MAYBE_SUDO from docker socket chmod
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add contains, startsWith, endsWith, join, comparison operators, and type literals
Expand the expression evaluator to cover the remaining GitHub Actions
expression features: string search functions, comparison operators
(==, !=, <, >, <=, >=), status functions, and literal types
(true, false, null, numbers).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update compatibility table for new expression features
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: fix formatting in compatibility.md
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip orphan cleanup when running inside a Docker container
When agent-ci runs nested inside a container (e.g. smoke-bun-setup step 8),
it shares the host's Docker socket. killOrphanedContainers() checks
agent-ci.pid labels using process.kill(pid, 0), but host PIDs don't
exist in the container's PID namespace — so every container looks like
an orphan. This caused the nested agent-ci to force-remove its own
parent container (exit 137).
Both pruneOrphanedDockerResources() and killOrphanedContainers() now
bail out when /.dockerenv exists.
Fixes#228
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: ignore inherited AGENT_CI_DTU_HOST inside containers
When agent-ci runs nested inside a container (e.g. smoke-bun-setup),
the outer container's AGENT_CI_DTU_HOST=host.docker.internal is
inherited. The nested agent-ci used this to tell the inner container
where the DTU is, but the nested DTU is inside the outer container,
not on the host. The inner container got "Connection refused" trying
to reach host.docker.internal:PORT.
Now when /.dockerenv exists, AGENT_CI_DTU_HOST is ignored and the
container's own IP (via hostname -I) is used instead, so sibling
containers can reach the DTU.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: move /.dockerenv guard to call site so tests still work
The /.dockerenv check inside killOrphanedContainers() broke the
shutdown integration tests — they run inside a Docker container (the
test runner) and need the function to work. Moved the guard to the
session bootstrap in cli.ts where the nested agent-ci problem occurs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add global concurrency limiter across all workflows
In --all mode, each workflow created its own per-wave limiter, allowing
20+ containers to run simultaneously and exhaust available memory (OOM
kills with exit 137). Now a single global limiter (CPU/2) is created in
the orchestrator and shared by all handleWorkflow calls, including
single-job workflows and cold-cache first jobs.
Closes#225
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: factor Docker VM memory into concurrency limit
CPU-based limit alone (cpuCount/2 = 7) still causes OOM kills. Now
reads MemAvailable from /proc/meminfo inside the Docker VM and caps
at floor(available / 3GB per container). Falls back to docker info
with a 4GB reserve if busybox isn't available.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: guard orphan cleanup in the function, not just the call site
The /.dockerenv guard was only in cli.ts, but killOrphanedContainers
is also called directly by the Docker integration tests (via a spawned
tsx subprocess). Inside a container, the test's subprocess would see
the parent container's PID label, misidentify it as an orphan, and
kill the container running the tests.
Put the guard back in both functions AND keep the cli.ts guard.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: increase per-container memory estimate to 4 GB
3 GB per container wasn't enough — tests.yml (vitest + Docker
integration tests) and smoke-binary.yml (full build) both got killed
at 3 concurrent containers. 4 GB gives 2 concurrent containers on a
12 GB Docker VM, with ~5.4 GB each.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: mock fs.existsSync in shutdown unit tests for /.dockerenv guard
The unit tests mock execSync but not fs. Inside a runner container,
the real fs.existsSync("/.dockerenv") returns true, causing
killOrphanedContainers to bail before reaching the mocked execSync.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: skip Docker integration tests for orphan cleanup inside containers
The integration tests spawn a real killOrphanedContainers() subprocess.
Inside a container, the function bails due to /.dockerenv (PID namespace
mismatch), so the tests can't exercise it. Skip them with insideDocker().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: add --jobs flag and document concurrency
Adds -j/--jobs <n> to override the auto-detected concurrency limit.
Adds a Concurrency section to the README explaining how the limit is
calculated and how to override it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat(website): include install steps in copy agent instructions
The "Copy agent instructions" button and Terminal preview now include
setup steps: installing the package (with package manager detection for
Node.js projects, or npx for non-Node projects) and installing the
agent skill.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Test files (*.test.ts) were being compiled into dist/ and included in
the npm package. Add tsconfig exclude to keep them out of the build.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The release workflow failed because changeset-generated CHANGELOGs weren't
formatted. Fix the existing files and run oxfmt after `changeset version`
so future Version Packages PRs include formatted markdown.
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: update compatibility.md to reflect current behavior
- GITHUB_TOKEN: no longer auto-injected, requires --github-token or AGENT_CI_GITHUB_TOKEN
- github.sha: now uses real commit SHA from HEAD, not hardcoded zeros
- secrets: documents env var fallback and --github-token auto-population
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* docs: update github.sha to reflect dirty-tree SHA support
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: service containers leak on unclean shutdown
Problem: Service containers (e.g. `agent-ci-*-svc-cache-db`) were created
without the `agent-ci.pid` label, making them invisible to the bootstrap
orphan cleanup (`killOrphanedContainers`). When the parent process died
without hitting its signal handler (crash, SIGKILL, Bash tool timeout),
these containers leaked permanently. Additionally, SIGHUP was not handled,
so closing a tmux pane or SSH disconnect also leaked runner containers.
Containers stuck in `created` state were also missed by the prune logic.
Solution: Label service containers with `agent-ci.pid` at creation time.
Update `killOrphanedContainers` to clean up unlabeled containers (handles
pre-fix containers) and derive the runner name from svc container names
(strip `-svc-*` suffix) with deduplication. Add SIGHUP to signal handlers
in both `cli.ts` and `local-job.ts`. Extend `pruneOrphanedDockerResources`
to also prune `status=created` containers.
Fixes#216
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add Docker integration smoke tests for orphan cleanup
Spawn real Docker containers (unlabeled, dead-PID-labeled, alive-PID-labeled)
and verify killOrphanedContainers handles each case correctly against the
real Docker daemon. Tests are skipped when Docker is unavailable.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: correct import path in orphan cleanup smoke test
The test spawns a subprocess via `npx tsx -e` which imports
shutdown.ts. The path was relative to the repo root but vitest
runs with cwd=packages/cli/, causing MODULE_NOT_FOUND in CI.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: smoke repro for setup-bun cache collision (#212)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: setup-bun fails in bun projects (#212)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: service containers leak on unclean shutdown
Problem: Service containers (e.g. `agent-ci-*-svc-cache-db`) were created
without the `agent-ci.pid` label, making them invisible to the bootstrap
orphan cleanup (`killOrphanedContainers`). When the parent process died
without hitting its signal handler (crash, SIGKILL, Bash tool timeout),
these containers leaked permanently. Additionally, SIGHUP was not handled,
so closing a tmux pane or SSH disconnect also leaked runner containers.
Containers stuck in `created` state were also missed by the prune logic.
Solution: Label service containers with `agent-ci.pid` at creation time.
Update `killOrphanedContainers` to clean up unlabeled containers (handles
pre-fix containers) and derive the runner name from svc container names
(strip `-svc-*` suffix) with deduplication. Add SIGHUP to signal handlers
in both `cli.ts` and `local-job.ts`. Extend `pruneOrphanedDockerResources`
to also prune `status=created` containers.
Fixes#216
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: use XDG cache on Linux Docker Desktop so bind mounts work (#215)
`/tmp` is not in Docker Desktop's default shared-folder list on Linux,
so bind mounts from `/tmp` silently fail. Detect Linux Docker Desktop
via `~/.docker/desktop/docker.sock` and fall back to `$XDG_CACHE_HOME`
(or `~/.cache`). Also update the validate command to run agent-ci in
the background with Monitor for proper pause-on-failure support, and
note Docker Desktop as a supported Linux provider in the README.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: remove stop hook that ran agent-ci synchronously
The stop hook can't support pause-on-failure since it runs after the
conversation ends. Use `/validate` instead, which runs agent-ci in the
background with Monitor support for fixing and retrying failures.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Fixes#209 — on Linux with Docker Desktop active, agent-ci failed at
runner container creation with "mounts denied" when the user isn't in
the docker group.
Splits socketPath (API client, needs R/W) and bindMountPath (container
mount source, only needs path recognition by Docker's VM) as
independent decisions. Whenever /var/run/docker.sock exists on the
host, use it as bindMountPath regardless of our process's R/W access.
Collapses #197 (macOS Desktop symlink) and #209 (Linux Desktop
non-docker-group) into one rule.
The working-directory half of the original #210 report is tracked
separately in #215.
Closes#209
Co-Authored-By: Bastiaan Steinmeier <bastiaan.steinmeier@broekhuis.nl>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Global Docker/workspace cleanup and runner-image prefetch now run once
per session instead of once per workflow. Also dedup concurrent
ensureImagePulled calls so parallel workflows share a single in-flight
docker pull. Together these eliminate the N× daemon storm that was
serializing `agent-ci run --all`.
Per-job ensureImagePulled calls remain as a safety net and take the
inspect() fast path since images are already warm.
Fixes#211
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* test: add smoke repro for missing cc in runner image
Minimal smoke workflow that calls `command -v cc` and compiles a
trivial C program. Reproduces a cargo/Rust workflow failure where
`linker 'cc' not found` breaks builds under agent-ci-dev because
ghcr.io/actions/actions-runner:latest does not ship build-essential.
CI-only change (no packages/ code modified) — no changeset.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: pluggable runner image via .github/agent-ci.Dockerfile (#208)
agent-ci now discovers a user-provided Dockerfile at
.github/agent-ci.Dockerfile (or .github/agent-ci/Dockerfile for builds
with a COPY context), hashes its contents, builds it locally via
`docker build`, and uses the resulting agent-ci-runner:<hash> tag as
the default runner image. Edits trigger automatic rebuilds; identical
contents reuse the cached image.
This closes the gap where the minimal ghcr.io/actions/actions-runner
image lacks build-essential, python3, and other toolchains that
GitHub's hosted ubuntu-latest VM ships preinstalled. Workflows green
on GitHub but failing locally with `linker 'cc' not found` can now
opt into a richer image by dropping a 5-line Dockerfile into .github/.
Resolution order (highest wins):
1. per-job `container:` (unchanged)
2. AGENT_CI_RUNNER_IMAGE env var
3. .github/agent-ci/Dockerfile (directory form, supports COPY)
4. .github/agent-ci.Dockerfile (simple form, empty context)
5. ghcr.io/actions/actions-runner:latest (default)
Also adds an error-hint heuristic: when a step fails with "command not
found" for cc/gcc/make/python3/pkg-config and the user is still on the
default image, the failure summary includes a paste-ready Dockerfile
pointing at the fix. Docs at packages/cli/runner-image.md.
Dogfoods the feature by shipping .github/agent-ci.Dockerfile for this
repo, which makes smoke-cc-compiler.yml (the issue #208 reproduction)
pass end-to-end.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: announce first-run runner image build
Print a one-time stderr notice before `docker build` fires so users
understand why the first run pauses for ~60s. The message lives in
stderr so log-update can't clobber it and stays in scrollback after.
Docs updated to match.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: clarify that container: runs verbatim, no Dockerfile layering
Spell out explicitly that a job-level container: directive bypasses
.github/agent-ci.Dockerfile entirely — agent-ci does not layer the
Dockerfile on top. This preserves the "same YAML runs locally and on
GitHub" parity promise.
Also document the two recommended ways to combine a custom base
image with agent-ci's extras.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: simplify testing instructions in CLAUDE.md
Use single `npx agent-ci run --all` command instead of separate unit/e2e steps.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: preserve symlink path for Docker socket bind mount on macOS (#197)
When /var/run/docker.sock is a symlink (Docker Desktop), the resolved
real path was used as the container bind-mount source, causing
"error while creating mount source path" in Docker's VM.
Adds `bindMountPath` to `DockerSocket` (the pre-symlink path) for bind
mounts, while `socketPath` (resolved) continues to be used for the
Docker API client. Adds a regression test for the specific failure case.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: suppress passing step output in quiet mode
In --quiet mode, only emit failures and pause messages. Completed
and running step lines (✓/▸) and "Starting runner" headers were
noise on a passing run -- the final printSummary covers everything.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* test: add repro-197 integration test for Docker Desktop socket bind mount
Tests the full chain from filesystem state → socket resolution → container
bind string. Verifies that /var/run/docker.sock (the symlink) is used as
the bind-mount source, never the resolved ~/.docker/run/docker.sock that
causes Docker Desktop's VM to fail with "error while creating mount source path".
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
* docs: simplify testing instructions in CLAUDE.md
Use single `npx agent-ci run --all` command instead of separate unit/e2e steps.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: pull runner image before createContainer to fix first-run failure
Fixes#203
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
When no --sha flag is given and the working tree is dirty, compute an
ephemeral commit SHA via git write-tree / commit-tree that represents
the actual code being executed. No refs are moved and history is
untouched.
Closes#198
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The last output lines and retry/abort hints were only displayed in
single-job mode. Multi-job workflows (e.g. build + test) would show
"Step failed attempt #1" with no error context or instructions.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
When multiple matrix jobs fail with the same error (e.g. missing secrets),
the FAILURES section now groups them and shows the error once instead of
repeating it for each job. Streaming error messages are also deduplicated.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* feat: resolve workflow secrets from shell environment variables
Shell env vars now act as a fallback for .env.agent-ci when resolving
${{ secrets.FOO }} references. Also auto-populates secrets.GITHUB_TOKEN
from --github-token.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* test: add loadMachineSecrets env fallback tests, update README
Add 7 tests covering process.env fallback behavior in loadMachineSecrets:
file precedence, env-only resolution, key scoping, and merge behavior.
Add a Secrets section to the README documenting resolution order.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Checkout v6 added post-fetch validation that the checked-out ref matches
github.sha. The git shim was returning a fake placeholder SHA (aaaa...)
in dirty workspace mode while github.sha used the real HEAD SHA, causing
the validation to fail. Use realHeadSha consistently everywhere and
remove the now-unnecessary computeFakeSha helper.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
The previous approach symlinked workspace/node_modules → /tmp/node_modules,
which broke esbuild path resolution in pnpm monorepos. Tools like rwsdk's
directive scanner resolved paths through the symlink incorrectly, producing
paths like apps/web/tmp/node_modules/.pnpm/... instead of /tmp/node_modules/.pnpm/...
Mount the warm cache as a Docker bind mount directly at the workspace
node_modules path. This makes the filesystem layout identical to production
GitHub Actions. The original EBUSY concern is already mitigated by the git
shim (blocks git clean) and checkout patching (clean: false).
Adds a smoke test exercising esbuild resolution through pnpm workspace
symlinks to prevent regression.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* fix: clean up runner directory on SIGINT/SIGTERM
The signal handler was missing `hostRunnerDir` from its cleanup list,
causing runner directories to leak when the process was interrupted
via Ctrl+C. Moved variable declarations above the handler so it can
reference `hostRunnerDir`, and added it to the cleanup loop.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: detect and kill orphaned containers via parent-PID liveness
Label every container with `agent-ci.pid=<PID>`. On startup, scan
running agent-ci-* containers and check if their parent PID is still
alive. Dead PID → orphan → kill it along with its sidecars and network.
Also wires up the existing but unused `pruneStaleWorkspaces()` to
remove run directories older than 24 hours.
Closes#186
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
github-actions[bot]