Commit Graph

57 Commits

Author SHA1 Message Date
github-actions[bot] 70f548cdcc chore: version packages (#118)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-04-06 20:17:45 +02:00
Peter Pistorius c2fe31b888 feat: action tarball disk cache, step output capture, and demo recording (#117)
* feat: action tarball disk cache, step output capture, and demo recording

- Cache action tarballs on first download and serve from disk on re-runs,
  eliminating ~30s GitHub CDN delays
- Capture step output via tee to signals dir for reliable pause-on-failure
  tail display
- Fix CLI to treat empty results as failure and skip summary print
- Add marketing demo recording setup with VHS tape and test project
- Add `agent-ci` script alias in root package.json

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* test: add tests for action tarball cache, group filtering, and empty-results fix (#119)

Cover the untested changes from the tarball cache PR: action tarball proxy
cache hit/miss, download info URL rewriting with sub-path stripping, step
output group filtering, tee output capture, and CLI empty-results exit logic.

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: prevent concurrent tarball cache-miss race condition

Use an in-flight download map so concurrent requests for the same
tarball coalesce into a single GitHub fetch instead of racing on
the same tmp file, which could cause corruption or rename errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: use relative path in demo.tape for portability

Replace hard-coded absolute path with a relative path so the demo
recording works on any contributor's machine.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct sanitized-refs test to assert 404 for slash-containing refs

The test was passing vacuously (status >= 200 matches 404). Now it
properly asserts that Polka does not bind :ref across slashes, so
slash-containing refs return 404 and do not serve cached content.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-06 20:16:18 +02:00
Peter Pistorius f9f17fd0b9 fix: only mount relevant PM cache for detected package manager (#114)
* fix: only mount relevant PM cache based on detected package manager

Projects using npm/yarn/bun no longer get unnecessary pnpm store bind
mounts (and vice versa). Detects PM from lockfile and conditionally
creates cache dirs, bind mounts, and virtual cache patterns.

Closes #112

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add debug log for detected PM, remove verbose bind mount log

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add PM-scoped caching integration tests

Tests createRunDirectories and buildContainerBinds for each package
manager (npm, pnpm, yarn, bun) plus the no-lockfile fallback. Each
test creates a real git repo fixture with the relevant lockfile and
verifies only the correct PM's cache dir and bind mount are produced.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 18:44:55 +02:00
Peter Pistorius acb750f346 feat: show Docker pull progress during boot (#116)
* feat: show Docker image pull progress during runner boot

Display download progress (bytes downloaded / total with percentage) as
a sub-step under "Starting runner" while pulling the container image.
Aggregates per-layer progress from the Docker API and throttles UI
updates to every 250ms.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: only track Downloading events for pull progress

Extracting events reset the current byte count per layer, making
overall progress appear stuck or go backwards. Filter to only
"Downloading" status events so progress climbs steadily to 100%.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle Download complete events to reach 100%

Docker's last event per layer is "Download complete" with no
progressDetail, so layers were never marked as fully downloaded.
Now explicitly set current = total when a layer completes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: log all docker pull events to diagnose stuck progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: track download and extraction phases separately

The progress was getting stuck because:
1. "Download complete" events fired rapidly and got throttled
2. Extraction phase was invisible — no events passed the filter

Now tracks downloading and extracting as separate phases with
independent progress maps. The UI shows "Downloading: X/Y (Z%)"
then transitions to "Extracting: X/Y (Z%)" once extraction begins.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: force UI update on download→extract phase transition

The throttle was swallowing the first "Extracting" event, leaving
the display frozen on the last download snapshot. Now flushProgress
accepts a force flag and the first extraction event always triggers
an immediate store update regardless of throttle timing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: show reset hint when image pull takes over 60s

After 60s of pulling, shows a dim hint beneath the progress line:
  "Taking a while? Run: docker rmi <image>"

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* debug: log logDir presence in boot node renderer

* fix: remove debug logging, shorten logDir, lower hint threshold to 30s

- Remove debug-renderer and pull flush diagnostic logging
- Shorten logDir display path to agent-ci/cli/runs/... so it stays
  visible alongside the progress line without wrapping
- Lower the "Taking a while?" hint threshold from 60s to 30s since
  Docker stalls immediately when resuming a partially cached layer

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: remove reset hint from pull progress display

The docker rmi hint wasn't helpful since it doesn't fix the stalled
download (the partial blob lives inside Docker's VM, not the layer
store). Removed along with the now-unused image/startedAt fields.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-06 18:43:59 +02:00
Jökull Sólberg 3bb8525421 fix: support large monorepos and sub-path actions
Three fixes discovered while running agent-ci against a ~40K file
pnpm/turbo monorepo:

1. Sub-path actions (e.g. `actions/cache/save@v5`) produce invalid
   tarball URLs. The DTU was using `action.nameWithOwner` directly
   which includes the sub-path, producing
   `api.github.com/repos/actions/cache/save/tarball/v5` (404).
   Fix: strip to `owner/repo` before constructing the URL.

2. `copyWorkspace` uses `execSync("git ls-files ...")` with the
   default 1MB maxBuffer. Repos with 30K+ tracked files overflow
   this, causing ENOBUFS and an empty workspace in the container.
   Fix: increase maxBuffer to 100MB.

3. Post-success cleanup `fs.rmSync(containerWorkDir)` can throw
   ENOTEMPTY when the container hasn't fully released file handles.
   Fix: wrap in try/catch for best-effort cleanup.
2026-03-27 22:58:00 +00:00
github-actions[bot] 987477bae4 chore: version packages 2026-03-27 20:32:23 +00:00
Peter Pistorius 5b52862ed0 fix: update commit status target URL to agent-ci.dev
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 07:27:20 +11:00
Peter Pistorius d273b76c64 fix: show full per-step log in failure summary
Replace the 20-line tail (with "Last output:" label and indentation)
with the full content of the per-step log file already captured at
failedStepLogPath. Falls back to lastOutputLines when no log file
exists (e.g. container crash).

Adds reporter.test.ts to prove the behaviour, and an end-to-end check
in tests.yml that runs the retry-proof workflow and asserts the step
output appears in stdout.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-28 07:15:11 +11:00
Peter Pistorius bdd269ff10 feat: move website to apps/website, improve content and fix SSR
- Move website from website/ to apps/website/ (update pnpm workspace)
- Remove logo icon from navbar, keep AGENT-CI text
- Configure domain routes for agent-ci.dev in wrangler.jsonc
- Replace tabs with stacked tables in compatibility matrix
- Expand all ⚠️ and ❌ notes with reasons and details
- Update compatibility.md to match website content
- Fix useContext SSR error by removing motion/react, using CSS animations
- Add img-src CSP header for pbs.twimg.com (Twitter profile images)
- Remove npx run --all command from hero, rename button to Quick start
- Remove decorative border elements from hero section
- Update footer "Built by" link to rwsdk.com

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 17:27:26 +11:00
Peter Pistorius 60b0105413 Merge pull request #83 from redwoodjs/feat/commit-status
feat: post GitHub commit status via gh CLI
2026-03-26 05:29:29 +13:00
Peter Pistorius 6e7494bc8b fix: remove tickmark from commit status description
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 05:28:33 +13:00
Peter Pistorius 98ccd6fb8f fix: use checkmark instead of emoji in commit status description
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 05:27:13 +13:00
Peter Pistorius ec535e1508 fix: remove 4-byte emoji from commit status description
GitHub's commit status API rejects 4-byte Unicode characters.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 03:40:41 +13:00
Peter Pistorius a9cc7f7fca feat: update success commit status message
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 03:04:52 +13:00
Peter Pistorius 97227c4251 Merge pull request #77 from simoncrypta/fix/docker-host-ip
fix: make Docker host resolution universal
2026-03-26 02:41:29 +13:00
Simon Gagnon 79c7d65f36 refactor: remove host-side DNS probe for DTU host resolution
Simplify resolveDtuHost to deterministic precedence: AGENT_CI_DTU_HOST > AGENT_CI_DOCKER_BRIDGE_GATEWAY (outside Docker) > host.docker.internal. Removes misleading host-side dns.lookup that did not predict container-side reachability.
2026-03-24 19:31:00 -04:00
Simon Gagnon 1d1e4ec68c fix: preserve external DTU URLs and align host fallback docs
Limit Docker API URL rewrites to loopback hosts, keep alias-based defaults unless an explicit bridge gateway override is configured, and update docs/tests to match behavior.
2026-03-24 19:31:00 -04:00
Simon Gagnon 6d76c2d426 fix: fallback DTU host alias to bridge gateway
Try host.docker.internal first, then fallback to bridge gateway when alias resolution fails, with debug logging for fallback paths.
2026-03-24 19:31:00 -04:00
Simon Gagnon a9878186a0 fix: make Docker host resolution universal 2026-03-24 19:31:00 -04:00
Simon Gagnon c7c35248ec fix: use platform-specific Docker host IP for Linux compatibility
On Linux, host.docker.internal doesn't work by default (only available
since Docker 20.10+ and requires explicit configuration). This change:

- Detects macOS vs Linux and returns appropriate host IP
- Uses 172.17.0.1 (docker bridge gateway) on Linux host
- Uses host.docker.internal on macOS
- Adds containerUrl to ephemeral DTU for container-to-host communication
- Adds ExtraHosts mapping for containers to resolve host.docker.internal

Fixes connectivity issues when running agent-ci on Linux hosts.
2026-03-24 19:31:00 -04:00
Peter Pistorius 6e5375322a feat: post GitHub commit status via gh CLI after run completes
Closes #82

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 21:59:39 +13:00
github-actions[bot] c9d76f8d2d chore: version packages 2026-03-23 18:35:52 +00:00
Peter Pistorius 99c794a517 fix: use scoped package name @redwoodjs/agent-ci in npx commands
Ensures the commands work even when the package isn't installed as a
local devDependency.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 07:32:08 +13:00
Peter Pistorius 6971663e04 fix: address PR review comments on SKILL.md and README
- Add "(in Docker)" clarification to SKILL.md
- Explain --all flag matches on triggers like GitHub
- Add --no-matrix tip for collapsing matrix jobs locally
- Use --quiet flag in CLAUDE.md snippet
- Add "CI was green before you started" instruction for agents

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 07:18:53 +13:00
Peter Pistorius 179405b960 feat: add package metadata, SKILL.md, and AI agent discoverability
Improve npm and AI agent discoverability by filling in package.json
keywords/description, creating a SKILL.md for node_modules scanning,
and adding a ready-to-paste CLAUDE.md snippet to the README.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-24 07:07:09 +13:00
Peter Pistorius 54ef949483 Add docs. 2026-03-23 22:45:46 +13:00
github-actions[bot] 66dfd109e1 chore: version packages 2026-03-23 09:40:16 +00:00
Peter Pistorius 0cb10daed9 Add --no-matrix flag to collapse matrix workflows into a single job
Parse --no-matrix as a boolean flag in the run command. Thread noMatrix
through runWorkflows and handleWorkflow. When active, use a new
collapseMatrixToSingle helper (exported from workflow-parser.ts) to
produce one job with the first value of each matrix key, __job_total='1',
and __job_index='0', so strategy.job-total expressions resolve correctly
without parser changes. Document the flag in printUsage.
2026-03-23 06:17:17 +00:00
Peter Pistorius f4e1face21 test: behavioral specs for --no-matrix matrix collapse
Covers: collapseMatrixToSingle produces exactly one job, uses first value
of each matrix key, sets __job_total='1' and __job_index='0', handles
multi-key matrices and edge cases (degenerate single-combo, empty value
list, numeric values). Also guards that expandMatrixCombinations (default
path) is unaffected and produces the full cartesian product without
injecting __job_total or __job_index.
2026-03-23 06:14:57 +00:00
github-actions[bot] e5bb9804ce chore: version packages 2026-03-22 21:46:27 +00:00
Peter Pistorius 6ada7215bf fix: register ESM loader hook for Node 22 JSON import attributes
@actions/workflow-parser imports .json files without `with { type: "json" }`,
which Node 22 rejects. This adds a custom ESM loader hook that transparently
patches the missing attribute at runtime.

Closes #67

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-22 09:30:57 +13:00
Peter Pistorius 17d8a34e37 chore: version packages 0.3.3 2026-03-18 08:22:48 +13:00
Peter Pistorius 6233723198 fix(dtu): replace execa with node:child_process
execa was in devDependencies but imported in production code,
causing ERR_MODULE_NOT_FOUND when running via npx. Replaced with
Node's built-in child_process.exec and removed execa entirely.
2026-03-18 08:14:43 +13:00
Peter Pistorius 7357393c0e fix: add shebang to CLI entry point for npx compatibility
The dist/cli.js was missing #!/usr/bin/env node, causing npx to execute it as a shell script. Version 0.3.2
2026-03-18 07:04:34 +13:00
github-actions[bot] 2e9e8bb6f9 chore: version packages 2026-03-17 08:41:15 +00:00
Simon Gagnon 4bfd515bff refactor: use type guards and attach taskName to error 2026-03-16 23:21:17 -04:00
Simon Gagnon 1dcf0d74ef fix: handle rejected promises in job execution 2026-03-16 23:03:14 -04:00
Peter Pistorius 8fadb39584 Fix docs formatting. 2026-03-17 08:13:29 +13:00
Peter Pistorius 9769aac57b Update docs. 2026-03-17 08:10:35 +13:00
Peter Pistorius e8e2e823ea chore: update README, add engines field, scope release workflow paths 2026-03-16 21:26:58 +13:00
Peter Pistorius ec1b9d12c6 fix: add repository field for npm OIDC provenance verification 2026-03-16 07:31:17 +13:00
github-actions[bot] b67be925f1 chore: version packages 2026-03-15 17:00:23 +00:00
Peter Pistorius 38af50c9b2 fix: prevent generic pool from stealing runner-targeted jobs (#47)
When a runner-specific job was seeded, it was stored in both state.jobs
(generic pool) and state.runnerJobs (runner-targeted). A runner polling
before its own job was seeded could fall back to state.jobs and steal
another runner's job, leaving the original runner spinning indefinitely.

Fix: runner-targeted jobs now go only into state.runnerJobs. The generic
pool is reserved for jobs without runner affinity.

Adds three tests covering the race condition, concurrent stress with
8 runners, and the job-theft scenario.

Closes #47
2026-03-16 05:41:47 +13:00
Peter Pistorius 6c7ebad5b9 Merge pull request #57 from redwoodjs/feat/workflow-compatibility
feat: workflow compatibility — cross-job outputs, job-level if, fail-fast
2026-03-15 20:36:06 +13:00
Peter Pistorius d543138c0c feat: cross-job output capture via synthetic step + DTU parsing
The runner's FinalizeJob deletes _temp/_runner_file_commands/ inside the
container before exit, making  files unreachable from the host.

Solution: inject a synthetic 'Capture outputs' step (runs with if:always())
that reads all set_output_* files and echoes them as ::agent-ci-output::key=value.
The DTU parses these prefixed lines and persists them to outputs.json in the
runner's log directory. On job completion, local-job.ts reads outputs.json and
attaches the outputs to the JobResult.

Pipeline verified end-to-end — all 8 smoke tests pass:
- Cross-job outputs (setup-outputs → consume-outputs) ✅
- Job-level if:always() ✅
- Matrix shards with fail-fast:false ✅
- Artifacts, services, env vars ✅

All 279 unit tests pass.
2026-03-15 20:18:37 +13:00
Peter Pistorius b2c061ba1f fix: capture step outputs via DTU and fix _runner_file_commands path
- Fix extractStepOutputs path: _temp/_runner_file_commands/ (not root)
- DTU outputs endpoint now captures and persists outputs.json
- local-job.ts reads outputs.json from log dir instead of work dir
- Add smoke test jobs for cross-job outputs and if:always()

All 279 tests pass.
2026-03-15 19:22:17 +13:00
Peter Pistorius e6ecd4a0ac feat(#54,#55): wire job-level if and fail-fast into CLI orchestration
#54 - Job-level if conditions:
- shouldSkipJob() evaluates if: conditions before dispatching
- Tracks upstream job results (success/failure/skipped)
- Creates synthetic skipped results for skipped jobs
- Default behavior: skip if any upstream dependency failed

#55 - Respect fail-fast setting:
- Replaces unconditional wave abort with parseFailFast check
- Only aborts if not all jobs in wave have fail-fast: false
- Logs when continuing despite failures

All 279 tests pass.
2026-03-15 08:06:49 +13:00
Peter Pistorius 12806b6af0 feat(#53): wire needsContext through CLI orchestration
- parseWorkflowSteps now accepts needsContext parameter
- All expandExpressions calls in step parsing thread needsContext
- cli.ts wave loop accumulates job outputs and builds needsContext
  for downstream jobs
- Step names now use raw YAML value for expression preservation

2 new tests, all 279 pass.
2026-03-15 08:03:43 +13:00
Peter Pistorius 394d748ac9 feat(#52): add resolveJobOutputs to map step outputs to job outputs
resolveJobOutputs() resolves job output templates (e.g.
${{ steps.check.outputs.skip }}) against the flat step outputs
captured by extractStepOutputs().

6 new tests, all 277 pass.
2026-03-15 07:57:02 +13:00
Peter Pistorius 275cf2bff6 feat: add workflow compatibility features
- fromJSON()/toJSON() expression functions
- Cross-job outputs: needsContext in expandExpressions, parseJobOutputDefs
- Job-level if: parseJobIf, evaluateJobIf with always/success/failure/==/!=
- strategy.fail-fast: parseFailFast
- extractStepOutputs: reads $GITHUB_OUTPUT files from _runner_file_commands
- outputs field added to JobResult
- Updated compatibility.md

39 new tests, all 271 tests pass.
2026-03-15 07:53:17 +13:00