Files
redwoodjs__local-ci/packages/cli/runner-image.md
Peter Pistorius ef2ee43835 Catch any missing tool in runner image hint (#231)
* fix: skip orphan cleanup when running inside a Docker container

When agent-ci runs nested inside a container (e.g. smoke-bun-setup step 8),
it shares the host's Docker socket. killOrphanedContainers() checks
agent-ci.pid labels using process.kill(pid, 0), but host PIDs don't
exist in the container's PID namespace — so every container looks like
an orphan. This caused the nested agent-ci to force-remove its own
parent container (exit 137).

Both pruneOrphanedDockerResources() and killOrphanedContainers() now
bail out when /.dockerenv exists.

Fixes #228

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: ignore inherited AGENT_CI_DTU_HOST inside containers

When agent-ci runs nested inside a container (e.g. smoke-bun-setup),
the outer container's AGENT_CI_DTU_HOST=host.docker.internal is
inherited. The nested agent-ci used this to tell the inner container
where the DTU is, but the nested DTU is inside the outer container,
not on the host. The inner container got "Connection refused" trying
to reach host.docker.internal:PORT.

Now when /.dockerenv exists, AGENT_CI_DTU_HOST is ignored and the
container's own IP (via hostname -I) is used instead, so sibling
containers can reach the DTU.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: move /.dockerenv guard to call site so tests still work

The /.dockerenv check inside killOrphanedContainers() broke the
shutdown integration tests — they run inside a Docker container (the
test runner) and need the function to work. Moved the guard to the
session bootstrap in cli.ts where the nested agent-ci problem occurs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add global concurrency limiter across all workflows

In --all mode, each workflow created its own per-wave limiter, allowing
20+ containers to run simultaneously and exhaust available memory (OOM
kills with exit 137). Now a single global limiter (CPU/2) is created in
the orchestrator and shared by all handleWorkflow calls, including
single-job workflows and cold-cache first jobs.

Closes #225

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: factor Docker VM memory into concurrency limit

CPU-based limit alone (cpuCount/2 = 7) still causes OOM kills. Now
reads MemAvailable from /proc/meminfo inside the Docker VM and caps
at floor(available / 3GB per container). Falls back to docker info
with a 4GB reserve if busybox isn't available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: guard orphan cleanup in the function, not just the call site

The /.dockerenv guard was only in cli.ts, but killOrphanedContainers
is also called directly by the Docker integration tests (via a spawned
tsx subprocess). Inside a container, the test's subprocess would see
the parent container's PID label, misidentify it as an orphan, and
kill the container running the tests.

Put the guard back in both functions AND keep the cli.ts guard.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: increase per-container memory estimate to 4 GB

3 GB per container wasn't enough — tests.yml (vitest + Docker
integration tests) and smoke-binary.yml (full build) both got killed
at 3 concurrent containers. 4 GB gives 2 concurrent containers on a
12 GB Docker VM, with ~5.4 GB each.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: mock fs.existsSync in shutdown unit tests for /.dockerenv guard

The unit tests mock execSync but not fs. Inside a runner container,
the real fs.existsSync("/.dockerenv") returns true, causing
killOrphanedContainers to bail before reaching the mocked execSync.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: skip Docker integration tests for orphan cleanup inside containers

The integration tests spawn a real killOrphanedContainers() subprocess.
Inside a container, the function bails due to /.dockerenv (PID namespace
mismatch), so the tests can't exercise it. Skip them with insideDocker().

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: add --jobs flag and document concurrency

Adds -j/--jobs <n> to override the auto-detected concurrency limit.
Adds a Concurrency section to the README explaining how the limit is
calculated and how to override it.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat: catch any missing tool in runner image hint, add Nix recipe and docs

Closes #229

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* Revert "feat: catch any missing tool in runner image hint, add Nix recipe and docs"

This reverts commit ca261639e9.

* feat: generic missing-tool detection with Nix/xz recipe

Replace the hardcoded per-tool pattern list with a single generic
"command not found" regex that catches any missing tool. Add detection
for the Nix installer's "you do not have 'xz' installed" message.
Update hint to point to runner-image.md docs instead of inlining a
Dockerfile snippet. Add Nix recipe to runner-image.md and a runner
image section to the README.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-13 12:48:26 +02:00

6.2 KiB
Raw Permalink Blame History

Runner image

agent-ci runs each job inside a Docker container. By default that container is ghcr.io/actions/actions-runner:latest — the official GitHub self-hosted runner image. It ships the runner agent, Node.js, git, curl, jq, unzip, and not much else. In particular, it does not include build-essential, gcc, cc, python3, clang, or any language toolchain.

This is very different from GitHub's hosted ubuntu-latest runner, which is a ~30GB VM image preloaded with hundreds of tools (see actions/runner-images). That VM is not published as a container image — there is nothing to pull — so agent-ci cannot use it directly.

If a workflow that runs green on GitHub fails locally with linker 'cc' not found, python: command not found, you do not have 'xz' installed, or similar, this gap is why.

Adding tools: .github/agent-ci.Dockerfile

Create a Dockerfile at .github/agent-ci.Dockerfile in your repo. agent-ci picks it up automatically on the next run, builds it once, and caches the result.

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends \
      build-essential \
      pkg-config \
 && sudo rm -rf /var/lib/apt/lists/*

That's it. No config file. No flags.

  • The image is tagged agent-ci-runner:<sha-of-Dockerfile-contents>, so every edit forces a rebuild and identical contents reuse the cached build.
  • The first run is slow — agent-ci has to build the image and cache it (~60–90s for a common toolchain). Every run after that reuses the cached image and takes ~0s. agent-ci prints a one-line notice while the build is running so you can tell why it's pausing.
  • All jobs in your repo use this image unless they set an explicit container: directive (which still works — see Per-job overrides).

Directory form: .github/agent-ci/Dockerfile

If you need to COPY files into the image — corporate CA certs, a pinned binary, a lockfile for apt — use the directory form:

.github/
  agent-ci/
    Dockerfile
    ca-bundle.pem
    ...
FROM ghcr.io/actions/actions-runner:latest
COPY ca-bundle.pem /usr/local/share/ca-certificates/corp.crt
RUN sudo update-ca-certificates \
 && sudo apt-get update && sudo apt-get install -y build-essential

agent-ci builds it with .github/agent-ci/ as the context, so everything next to the Dockerfile is copyable. The hash includes every file in the context, so adding or changing any of them forces a rebuild.

If both forms are present, the directory form wins.

Common recipes

These are copy-paste examples — agent-ci does not ship any of them as a default.

Rust / cargo (cc-rs, native build scripts):

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends build-essential pkg-config \
 && sudo rm -rf /var/lib/apt/lists/*

Node.js native modules (node-gyp, sharp, bcrypt):

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends build-essential python3 \
 && sudo rm -rf /var/lib/apt/lists/*

Go with cgo:

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends build-essential \
 && sudo rm -rf /var/lib/apt/lists/*

Nix (cachix/install-nix-action, DeterminateSystems/nix-installer-action):

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends xz-utils \
 && sudo rm -rf /var/lib/apt/lists/*

Ruby native gems:

FROM ghcr.io/actions/actions-runner:latest
RUN sudo apt-get update \
 && sudo apt-get install -y --no-install-recommends build-essential libffi-dev libyaml-dev \
 && sudo rm -rf /var/lib/apt/lists/*

Per-job overrides

The standard GitHub Actions jobs.<id>.container: directive is still honored — and takes priority over .github/agent-ci.Dockerfile. Use it when a single job needs a different environment than the rest of your repo:

jobs:
  deploy:
    runs-on: ubuntu-latest
    container: my-org/deploy-runner:v2
    steps:
      - run: ./deploy.sh

When a job sets container:, agent-ci runs that image verbatim. It does not layer .github/agent-ci.Dockerfile on top, and it does not build anything. Whatever you put in container: is what runs, exactly as on GitHub-hosted runners. This keeps the "same YAML runs locally and on GitHub" parity promise — locally layering would silently change the environment relative to GitHub.

If you want both your own base image and agent-ci's extras, pick one:

  • Skip container: and put everything in .github/agent-ci.Dockerfile (one image, applies to all jobs).
  • Or build your own image that inherits from your preferred base and adds what you need, then reference it with container:.

CI escape hatch: AGENT_CI_RUNNER_IMAGE

Setting the AGENT_CI_RUNNER_IMAGE environment variable overrides all discovery and uses that image directly:

AGENT_CI_RUNNER_IMAGE=my-org/internal-runner:v3 agent-ci run --all

Highest priority (after per-job container:). Useful for CI pipelines that want to force a specific image without touching files in the repo.

Resolution order

Highest priority wins:

  1. jobs.<id>.container: in workflow YAML (per-job override)
  2. AGENT_CI_RUNNER_IMAGE environment variable
  3. .github/agent-ci/Dockerfile (directory form)
  4. .github/agent-ci.Dockerfile (simple form)
  5. ghcr.io/actions/actions-runner:latest (built-in default)

Build caching

When agent-ci builds from one of your Dockerfiles, it:

  1. Hashes the Dockerfile contents (simple form) or Dockerfile + all context files (directory form), sorted deterministically.
  2. Tags the image agent-ci-runner:<hash-prefix>.
  3. Checks if that tag already exists locally — if so, reuses it.
  4. Otherwise runs docker build, pulling ghcr.io/actions/actions-runner:latest first if needed (most Dockerfiles inherit from it).

Clean up old builds with docker image prune or docker rmi agent-ci-runner:<tag> as normal — they're regular Docker images with no special lifecycle.