* ci: skip device lanes for root-level docs-only changes (#1781 A9)
Add AGENTS.md, CHANGELOG.md, CONTEXT.md, CONTRIBUTING.md, LICENSE, and
SECURITY.md to the pull_request paths-ignore block in ios.yml,
android.yml, linux.yml, macos.yml, ci.yml, and size.yml. These
root-level docs files were the only gap left after docs/**,
website/**, and README.md — PRs #1568 (SECURITY.md only), #1697
(CONTEXT.md + docs/adr only), and #1722 (AGENTS.md + docs/) each still
triggered a full 9-15 min macOS iOS run despite touching only prose.
Why each file is safe to ignore for every one of these six workflows:
- None of the four device workflows (ios/android/linux/macos) or their
composite actions read any of these six files at runtime; the only
hits from `grep -rln` across scripts/, src/, test/, and
.github/actions/ are prose comments pointing humans at CONTEXT.md or
AGENTS.md sections (e.g. scripts/layering/check.ts,
scripts/wire-compat/run.ts, src/mcp/tool-ref-pins.ts) — never an
`fs.readFileSync`/`readFile` of the file itself.
- The check-affected selector (scripts/check-affected/model.ts)
already classifies all six as pure docs: `isDocs()` matches any
`.md` file plus the literal `LICENSE`, and `docsOwnership()` only
special-cases `website/docs/docs/commands.md` (unrelated). So these
files already select zero checks — they only ever produced
`docsOnlyPaths` entries, never `SelectionReason`s.
- Because they select zero checks, the gate-manifest's path-coverage
category derivation (`scripts/gate/model.ts` `categories()`, which
iterates `plan.reasons`) never records a category for them, so
ci.yml has nothing check-manifest-only that these six files would
need to keep reachable. `pnpm check:gate-manifest` and
`pnpm check:gate-manifest:test` both stay green after the change
(48 checks / 33 lanes, 28/28 gate tests passing).
- size.yml's bundle-size job (scripts/size-report.mjs) measures the
`pnpm build` dist output and startup timing only — no reference to
any of these six files. (npm packs LICENSE/README.md into the
publishable tarball, but that's a `pnpm check:package` node-22.12
concern in ci.yml's packaged-cli job, which is driven by `dist`
contents and `package.json`, not by LICENSE/README prose — already
evidenced by README.md being ignored here since before this change.)
Scope disclosure: `mutation-affected.yml` uses a `paths:` allowlist
(not paths-ignore) so it's structurally unaffected; `test-app-build-cache.yml`
has no path filter at all. Neither was touched.
actionlint and `pnpm check:gate-manifest`/`:test` pass on the changed
workflows.
* test: pin root-doc paths-ignore entries with a regression test
Addresses review feedback on #1791 from thymikee: the docs-only
classifier for AGENTS.md/CHANGELOG.md/CONTEXT.md/CONTRIBUTING.md/
LICENSE/SECURITY.md across ios.yml/android.yml/linux.yml/macos.yml/
ci.yml/size.yml had no regression pin. Neither check:gate-manifest
(only proves a *registered check* is reachable) nor actionlint (only
validates YAML shape) nor generic Markdown coverage would catch a
single dropped entry — e.g. LICENSE reappearing in one workflow's
paths-ignore list but not another's would silently put a full 9-15 min
device run back on prose-only PRs.
test/ci/root-docs-paths-ignore.test.ts parses the six real workflow
files and asserts, using the same matchesGlob the gate-manifest model
uses to decide lane triggering, that each of the six root docs is
ignored by each workflow's pull_request paths-ignore. Registered in
vitest.config.ts's unit-core project next to its sibling
upload-agent-device-artifacts.test.ts (parse-only, no device/subprocess
lane needed).
Verified red on main (all 36 file x doc assertions fail — confirmed via
a throwaway script reading `git show main:.github/workflows/*.yml`)
and green on this branch (6/6). Full unit-core project (873 files /
6641 tests) still passes; check:gate-manifest and
check:gate-manifest:test unchanged (48 checks / 33 lanes, 28/28).
* test(ci): prove every registered gate is owned and reachable (#1429)
A check that silently stops running looks exactly like a green build. Two
suites had already stopped: `check:tmpdir-leaks` (with its model tests) and
`test:fixture-cache` are real package scripts that no workflow ran, reachable
only through the `check:unit` aggregate CI never invokes.
`CHECK_CATALOG` becomes the registry of every check and `pnpm gate <id>` the
only way CI runs one, so finding what a lane runs is a scan for `pnpm gate`
rather than an attempt to interpret shell. `pnpm check:gate-manifest` then
asserts against the real workflows that every registered check is run by some
qualifying lane (per unit, not per script name), that every check the real
selector activates for a path is run by a lane that path would start (#1420's
class), and that every Vitest project and suite script belongs to a check.
The wiring that keeps those honest is asserted too: a gate id must name a
registered check, an `if:` must be ruled on in GATE_CONDITIONS so `if: false`
unowns what it guards, an action declared to run a gate is proven to, and a
job whose steps the loader cannot open fails closed.
It deliberately does not try to prove CI runs project code only through
`pnpm gate`. Whether a shell block executes project code is not decidable from
its text, so shell this model does not recognise earns no ownership credit —
the failure direction is a check reported unowned, never one waved through.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* test(ci): update the two suites that assert on rewired workflow text
`scripts/mutation/workflow.test.ts` and `test/ci/trusted-fixture-artifact.test.mjs`
read the workflow and action files and assert on their command text, so routing
those steps through `pnpm gate <id>` moved what they were matching.
They are the two suites the manifest cannot help with: it proves a gate is still
run, not that a test asserting on how CI spells a command was updated with it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* fix(ci): credit gates by execution shape, and keep every guard
Three ways the manifest could report a gate as owned when it does not run.
1. Crediting was a substring scan over `run:`, which #1429 explicitly rules
out — "do not infer reachability from a command name merely appearing in
workflow text". `false && pnpm gate x`, a gate inside `if false; then … fi`,
one named in a heredoc, and `echo pnpm gate x` all credited it. There is a
live instance: conformance-regenerate.yml's "Fail if regeneration changed
anything" step names `pnpm gate maestro-regenerate` inside an error message
telling a human to run it, and that credited the gate.
A gate now counts only as the first command segment of a line, and a body
carrying shell structure earns nothing. Reachability inside a script is not
decidable, so this does not try: unrecognised shape means no credit and the
check reports unowned. `VAR=$(pnpm gate x …)` is read, since the assignment
form is unambiguous and the gate runs.
2. Job-level `if:` was not modelled at all, though six live jobs carry one, so
a job that cannot run still credited every gate inside it. Two conditions on
the mutation lanes are now declared.
3. A caller's `if:` REPLACED the guard on a nested composite-action step
(`guard[0] ?? step.condition`), so an outer `always()` erased an inner
`if: false`. Steps carry every guard between the lane and the step.
Also corrects two source comments that still claimed project code run outside
the runner fails the manifest. It does not: such a step earns no credit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* ci: add the run-gate action that names a gate structurally
The seam the ownership proof will read instead of shell. A lane says which
gate it runs in `with.gate`, a typed input the manifest reads straight out of
the YAML and validates against CHECK_CATALOG.
Nothing here is wired yet — the ~60 call sites and the model change follow.
Added first so the target of that conversion is reviewable on its own.
`args` cannot select which gate runs; it is appended after the id, so the
worst a wrong value does is fail the gate it already named. There is no
`|| true` and no output capture: the gate's exit code is the step's exit code,
so a gate cannot run without being able to fail its lane.
Part of #1429.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* merge: main (#1770) and route its three new steps through the runner
#1770 landed the orphan-check fix on main, wiring `check:tmpdir-leaks`,
`check:tmpdir-leaks:test` and `test:fixture-cache` into Coverage, Layering
Guard and Integration Tests. This branch had wired the same three through
`pnpm gate`, so the merge produced two steps per check rather than a conflict
— each check ran twice.
Kept main's steps, with the placement and reasoning reviewed on #1770, and
changed only their `run:` line to the canonical runner. Dropped this branch's
duplicates. Net effect on CI is unchanged: the same three checks, in the same
three lanes, once each.
Gate manifest green after the merge: 47 checks wired across 33 lanes.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* fix(ci): address review — suite detection, freerange, glob, vacuous skip-list
Six review findings plus the mutation blocker.
[bug] `registered` was shape-only, so a `test:*` script running
`node src/bin.ts test <dir>` resolved to a `script:` leaf and was invisible.
Four `test:replay:*` scripts were owned only because someone hand-registered
them; `test:replay:android` was neither registered nor reported while the
nightly ran the same six .ad files by inlining them. A `test:*` script is now
a suite by name. `replay-android` is registered, and the nightly runs the
script instead of re-listing its files so the two cannot drift.
The nightly invokes it inside `reactivecircus/android-emulator-runner`'s
`script:` input — shell handed to a third-party action this loader does not
read — so the suite executes but cannot be credited. Recorded in
UNPROVABLE_OWNERS with that exact reason rather than assumed.
The fixed detector also found a second orphan the review did not name:
`test:integration:progress`. That one is a reporter whose `--check` sibling
is the registered gate, so it is declared in REPORTING_SCRIPTS — a
declaration that itself fails when inert.
[bug] `freerange` defaulted to localRunnable, so fail-open ran `fr` (a Bun
binary) on the pre-push path. Now false.
[suggestion] The `--run` skip-list asserted `build:android-snapshot-helper`,
a name `android-helpers` no longer uses, so it could not fail. Derived from
the catalog instead.
[suggestion] `matchesGlob` joined `**` splits with `.*`, making the adjacent
slash mandatory — GitHub's `**` matches zero directories, so
`src/**/*.test.ts` did not match `src/a.test.ts`. Pinned against
`packages/*/src/**/*.test.ts`.
[suggestion] Deleted the unwired `run-gate` action. It had no callers, was
absent from GATE_ACTIONS, and its comment described a system that had not
shipped. It returns with the rewiring, not before.
[suggestion] Collapsed the module headers that narrated discarded designs.
Mutation: `daemon entrypoint publishes HTTP metadata and cleans up on
shutdown` is the only test here that spawns a real daemon process. It takes
~1.1s alone but exceeds Vitest's 5s default inside Stryker's dry run, which
aborts the sweep before a single mutant runs. Given 30s.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* fix(mutation): order sandbox aliases longest-first so subpaths resolve
Every shard of the mutation sweep aborted in Stryker's dry run with:
Cannot find package '@agent-device/selectors/engine' imported from
.tmp/stryker/sandbox-*/src/core/selector-pipeline.ts
The alias was generated correctly; it just never won. Vite matches a STRING
alias by prefix and takes the first hit, and `workspaceSpecifierTargets`
emitted the bare `@agent-device/selectors` ahead of the subpath entries. The
bare entry therefore captured `@agent-device/selectors/engine` and rewrote it
to `…/src/index.ts/engine`, which does not exist; Node fell back to real
package resolution, could not find the subpath inside the sandbox, and the dry
run failed before a single mutant ran — so the shard uploaded an empty
envelope instead of a report and the ratchet failed for want of one.
Sorting longest specifier first makes the most specific alias win:
@agent-device/selectors/engine -> packages/selectors/src/engine.ts
@agent-device/selectors/ast -> packages/selectors/src/ast.ts
@agent-device/selectors -> packages/selectors/src/index.ts
`/ast` never tripped this because nothing in a related test set imported it;
`selector-pipeline.ts` introduced the first subpath import that mattered
(#1744), so the mutation lane has been unable to run since that landed. Any
PR touching `scripts/mutation/**` — which fails open into the full sweep —
would have hit it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SkS4S8XXrfkJ8TD1VBKkvJ
* refactor: derive gate ownership from workflow structure
* fix: run gates without optional arguments
* fix: resolve mutation workspace subpaths exactly
---------
Co-authored-by: Claude <noreply@anthropic.com>
* ci: fold single-grep jobs into steps, call named pnpm scripts
- Merge ios-runner-swift-compat and no-test-di-seams (each just
checkout + one rg assertion) into steps of a new static-checks job,
keeping each step's own failure message. Removes two job-scheduling/
checkout overheads and two PR status-check lines.
- Replace the layering-guard job's inlined copies of check:layering and
depgraph:test with the named pnpm scripts, removing the silent-drift
risk between the workflow and package.json.
- Fix the same drift in conformance-regenerate.yml, which inlined
maestro:conformance:regenerate byte-for-byte.
- Leave affected-selector's inline node invocation as-is: R8's zero-dep
closure check (scripts/layering/zero-dep-jobs.ts) finds a job's entry
scripts by matching literal paths in the run: block, so switching to
`pnpm check:affected:test` would zero out its entries and make R8
fail closed. Documented inline why this one stays inlined.
- Leave publish-mcp-registry.yml's sync-mcp-metadata --check alone: that
job never runs the setup-node-pnpm action, so pnpm isn't provisioned
there at all.
Refs #1462
* ci: teach R8 to resolve pnpm script names, drop affected-selector's inline copy
R8's zero-dep-job entry scan matched literal script paths in a run: block,
so a bare `pnpm <script>` invocation found zero entries and R8 failed
closed — the reason affected-selector kept an inline node command instead
of calling pnpm check:affected:test (#1462). zeroDepJobs now also resolves
a pnpm script name against package.json and scans the resolved command for
entry paths, so affected-selector can call the named script like every
other job.
Also replaced the other workflows' inlined copies of named package.json
scripts (test:replay:*, perf, perf:android, maestro:conformance:differential,
check:mcp-metadata, size) with their pnpm names, keeping each job's
CI-specific trailing flags — found via a repo-wide sweep for any run: block
whose text duplicates a scripts entry.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L67kDSTAJwaoLCwANEJFRM
* fix: revert publish-mcp-registry pnpm regression, recurse R8 alias resolution
publish-mcp-registry.yml's job only provisions Node via actions/setup-node,
never the repo's setup-node-pnpm action, so pnpm is never installed there —
the earlier sweep's `pnpm check:mcp-metadata` would have broken the release
path. Reverted to the direct node invocation with a comment explaining why,
matching the PR's own stated rationale for leaving it alone.
zeroDepJobs' pnpm-alias resolution only expanded one level: a resolved
script that itself invoked another named pnpm script had its entries
silently dropped from R8's closure. resolveRunEntries now recurses through
chained aliases with a per-chain visited set, so a nested alias's entries
are found and a cycle stops re-expanding a repeated name instead of
recursing forever. Added coverage for both the chained and cyclic cases.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01L67kDSTAJwaoLCwANEJFRM
---------
Co-authored-by: Claude <noreply@anthropic.com>
* chore(deps): add Renovate config and enforce packageManager pnpm version in CI
Refs #1422
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore: bump pnpm to 11.17.0 and format the whole repo with oxfmt
format/format:check drop their hand-maintained path list: oxfmt already skips
node_modules and honors .gitignore, so the only exclusion list is
.oxfmtrc.json ignorePatterns.
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* test(mutation): accept either quote style in the affected-lane path filter
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* chore(deps): keep fixture-app runtime deps as individual Renovate PRs
Co-Authored-By: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
---------
Co-authored-by: Michał Pierzchała <thymikee@gmail.com>
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
* refactor: remove JS test wrappers, run .ad replay suites directly on CI
The platform JS test files (ios.test.ts, android.test.ts, macos.test.ts)
were thin wrappers that shelled out to `agent-device test` and asserted
on JSON counts. Since the CLI already exits non-zero on failure, the
wrappers added no value. CI now invokes the replay suites directly.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: add local replay entrypoints and restore physical device test
- Add pnpm scripts for each replay suite (test:replay:ios,
test:replay:ios-device, test:replay:android, test:replay:macos)
so local runs still exercise platform replays
- Restore physical-device iOS replay step in CI workflow
(conditional on IOS_UDID variable)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Michał Pierzchała