Five changes from a measured review of the CLI surface and of a consumer's CI. 1. BEHAVIOUR — a positional target that matches nothing now FAILS. Bare discovery finding nothing still exits 0: an empty repo is a legitimate place to stand, but a path or glob you NAMED that matched nothing is stale, wrong, or never reached. Collapsing the two cost a real repo three days of green CI verifying zero files. Jest and Vitest fail on no-tests-found and make you opt in with --passWithNoTests; pytest exits 5. --min=0 is the opt-out here, so no flag is added. 2. HELP — the four checkers (audit/lint/test/eval) all read as "check my stuff". They are now grouped by READ vs RUN, and each line states its own consequence positively. The success criterion was checkable and is met: "NOT a CI step — use vigiles lint in CI" is deleted, because nothing needs defending. A help line that says what a command ISN'T is a description that failed. Per-verb flag detail moves into the per-verb help; audit's four wrapped lines naming nine flags were most of the felt crowding. Verb count is unchanged at 8 — the smallest of every tool measured (vitest 8 verbs and 164 flags, cargo 48, docker 57, git 166), and none of them thinned their surface by removing verbs. 3. DOCBLOCK — src/cli.ts listed four of the eight commands, having quietly stopped growing. Replaced by a pointer to the single source; a new test asserts every verb appears in the RENDERED help exactly once, because a constant is what drifted. 4. AUDIT — new Tested finding: "N surfaces counted as covered by PLACEMENT only — no run on record ever exercised one". Colocated evidence is a claim about the filesystem and the provenance line already admits it; nobody reads a provenance line. Measured in a consumer: 26 colocated and 0 executed, while its CI skipped every one of those harnesses and reported success. The corpus-wide threshold is a stated limit in the code, not a silence. 5. DOCS + SKILL — the Action reference now explains why there is no test input (the Action never runs npm ci; harnesses import the library), and the test-harness skill gained a six-line CI pointer: an Action exists, init wires it, inputs are documented there. It REFERENCES rather than explains, because copied explanations rot — three measured proofs, one of them inside this product. Gates, in order: vitest, lint, prettier, api:check, build — via the check script, 18 of 18. Co-authored-by: Claude <noreply@anthropic.com>
9.5 KiB
GitHub Action
Run vigiles in CI. The README has the pitch; this is the full Action reference. For the CLI the Action wraps, see the CLI reference.
The Action is a composite action over the published npx vigiles CLI — it
runs the exact artifact you'd run locally, so there's no separate bundle to drift.
Every input maps to a real CLI flag.
Quick start
name: vigiles
on:
pull_request:
push:
branches: [main]
permissions:
contents: read # all the Action needs; it only reads files and emits annotations
jobs:
vigiles:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
- uses: zernie/vigiles@v1 # runs `lint` by default
That's the whole thing — lint verifies that every linter rule, file path,
script, and symbol your CLAUDE.md / AGENTS.md cites is real and enabled, checks
the integrity hashes, and reports coverage. Failures appear inline as GitHub
annotations and fail the job.
Compile specs in CI
- uses: zernie/vigiles@v1
with:
command: compile # spec.ts → markdown; fails if a reference is stale
paths: CLAUDE.md.spec.ts # optional; auto-discovers when omitted
Check eval results aren't stale (no model)
Real-model evals run locally on your subscription (vigiles eval --update, which
commits a lock). This job verifies those committed results against the current
inputs without a model call — failing if you changed a skill but forgot to
re-eval. It's a green no-op until you commit your first lock.
- uses: zernie/vigiles@v1
with:
command: eval-check # → `vigiles eval --check`; no API key, no subscription
Inputs
| Input | Default | Description |
|---|---|---|
command |
lint |
lint (verify references + integrity + coverage), compile (specs → markdown), or eval-check (verify committed eval locks vs current inputs — the staleness gate, no model). |
paths |
(auto) | Comma/space-separated paths — .md for lint, .spec.ts for compile. Auto-discovers. |
version |
latest |
npm version of vigiles to run (1, 1.2.3, latest). local runs a checked-out build. |
max-rules |
(unset) | Cap rules per spec (maps to --max-rules). |
catalog-only |
false |
Only check that linter rules exist; skip config-enabled checks (maps to --catalog-only). |
working-directory |
. |
Directory to run vigiles in. |
comment |
true |
On pull_request events, post/update a sticky PR comment with the result. |
capability-diff |
false |
On pull_request events, diff the agent's capability surface (subagents' tool/effect blast radius) vs the PR base and fold it into the sticky comment (maps to --capability-diff). Needs fetch-depth: 0 on checkout. |
fail-on-widen |
false |
With capability-diff, fail the run when the PR widens the blast radius (maps to --fail-on-widen). |
github-token |
(auto) | Token for the PR comment. Defaults to the workflow token (${{ github.token }}). |
Why there is no command: test
test and eval are missing from that list deliberately, and the omission is worth stating
because the input is a pass-through — nothing validates it, so command: test becomes
vigiles test and then fails in a way that looks like a vigiles bug.
It fails because the Action runs npx vigiles@<version> <cmd> in your working directory and
never runs npm ci. Harness files import the library:
import { runHarnessTest, skip } from "vigiles";
That import needs repo-local node_modules, which the Action does not install. vigiles init
knows this: the workflow it generates wires the Action for the jobs that can use it and writes
a plain npx vigiles test job — with its own npm ci — for the harness tier.
So the split is not an oversight to route around:
| tier | how it runs in CI | why |
|---|---|---|
lint, compile, eval-check |
the Action | reads files, needs no repo dependencies |
test, eval |
a normal job with npm ci |
executes your harness, which imports the library |
Pin version: in any job you hand-write. The latest default deliberately ignores your
package-lock.json, so CI can lint with one major while your harnesses run against another —
a split measured in a real consumer on 2026-08-18 (npm ls vigiles → 15.2.1 invalid: "^16.1.0"), which is what led a new harness to import a subpath v16 had removed.
Output channels
Beyond the valid step output, the Action reports three ways:
- Inline annotations — failures appear on the diff (
::error). - Job summary — a markdown result block on the run page (
$GITHUB_STEP_SUMMARY). - Sticky PR comment — on
pull_requestevents, one comment that is updated in place each run (found by a hidden marker, never duplicated). Requirespull-requests: write; setcomment: falseto disable.
permissions:
contents: read
pull-requests: write # needed for the sticky PR comment
# ...
- id: vigiles
uses: zernie/vigiles@v1
- run: echo "passed=${{ steps.vigiles.outputs.valid }}"
The valid output is 'true' if vigiles passed (exit 0), 'false' otherwise.
Exit codes (also reflected in valid): 0 clean · 1 warnings · 2 hard errors.
On a fork PR (read-only token) the comment step degrades to a warning — the job still passes/fails on the result.
Capability diff in PRs
Set capability-diff: true to answer one question on every PR: did this change widen what your agents can do? The Action materializes the PR base, computes the capability diff (new side-effecting/unknown tools, a loosened purity floor), and folds a short section into the same sticky comment — only when something actually changed. It's informational by default; add fail-on-widen: true to block a PR that grows the blast radius.
- uses: actions/checkout@v4
with:
fetch-depth: 0 # required: the base commit must be present to diff against
- uses: zernie/vigiles@v1
with:
capability-diff: "true"
# fail-on-widen: "true" # optional gate
fetch-depth: 0 is required because a shallow checkout (the default) doesn't carry the PR base commit the diff compares against.
Versioning
The Action tag and the npm version are two separate version lines. The Action
is a thin composite that runs the published npx vigiles@<version> CLI, so:
- Action ref (
uses: zernie/vigiles@v1) — pin the floating major tag@v1for automatic patch/minor updates to the Action wrapper (the release pipeline keepsv1pointed at the latest1.xof the action). Pin a full tag (@v1.2.3) or a commit SHA for byte-for-byte reproducibility.@maintracks unreleasedHEAD. - CLI version (
version:input, defaultlatest) — selects which publishedvigilesnpm release the Action runs (currently3.x). Leave itlatest, or pinversion: '3'/version: '3.0.0'to lock the CLI independently of the Action tag.
So uses: zernie/vigiles@v1 with the default version: latest runs the newest
vigiles CLI (3.x today) through the v1 action wrapper. The @v1 does not
mean "vigiles 1.x". To lock both: uses: zernie/vigiles@v1 + with: { version: '3' }.
- uses: zernie/vigiles@v1
with:
version: "3" # pin the CLI major; @v1 pins the action wrapper
To verify generated types are fresh in CI:
- run: npx vigiles generate types --check
See also
- CLI reference — the commands the Action wraps.
- Verifying instruction files — what
lintchecks.