Claude
1443ace83c
Codex flagged that the guard added in 8f790e5 does not enforce its advertised
invariant: `src/comparison-snapshot.test.ts` reads
`site/src/comparison/validate-overlap.json` off disk and the pattern missed it.
Verified — the file exists, the read is real, and the grep found nothing.
TWO independent reasons it evaded, either enough on its own:
1. `grep` is LINE-based and prettier had split the call, so `resolve(` and
the literal are never on one line:
const SNAPSHOT = resolve(
__dirname,
"..",
"site/src/comparison/validate-overlap.json",
);
2. the pattern encoded ONE SPELLING (`"../../site/`) — the path assembled
from separate segments did not match it.
Both are the same defect: a guard written from the spelling its author had
just deleted rather than from the shape it forbids.
The data moves instead of the dependency being declared. The snapshot is
WRITTEN by a root tool (tools/measure-validate-overlap.mjs) and ASSERTED by a
root test, so it now lives at the root; the site imports it through a new
`@measured/validate-overlap.json` alias. The arrow points site -> root, which
a site-only diff cannot break.
The guard is Node, not grep: it matches the whole CALL across line breaks, so
neither reason above can recur. It keys on a `site/` segment inside a string
literal passed to resolve/join/read*, NOT on the mere mention — measured, not
assumed: forbidding the mention fires on 3 legitimate fixtures today (the
classifier is FED "site/src/App.tsx", it does not read it), and a guard with
false positives gets switched off, which is how the first hole survived.
| | |
| --- | --- |
| root reads under site/ | 1 -> 0 |
| forms the guard sees | one spelling, single-line -> any call, any layout |
| new fixture | the exact multi-line call the old guard missed |
Mutation-proven: restoring the site/ path in comparison-snapshot.test.ts (patch
landed, grep count 1) fails exactly `finds no disk read of site/ anywhere under
src/`, and reverting restores green.
Two of my own errors on the way, both caught by a run rather than by reading:
dropping the `execFileSync` import broke `decide()` six tests above, and the
first `SITE_LITERAL` anchored `^` to the start of the argument STRING instead
of the position after the quote, so it never matched a literal that was not
first in the call — its own fixture caught that one.
Gates: npm run check 18/18 · coverage 3960 passed 0 failed · site build ok ·
site test:browser 51 passed. The 99.94% statement gate fails locally on
src/mock-model.ts (untouched; 25 tests skip locally), green on CI.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
tools/ — human-run maintenance scripts (never in CI)
Occasional scripts a maintainer runs by hand — network I/O (cloning repos),
breadth sweeps, asset generation. They are NOT part of the shipped library: not
src/, not compiled to dist/, not tested, never run in CI. Run them from the
repo root. (The BUILD pipeline — api-extractor.mjs, build-report.mjs — lives in
scripts/, not here; tools/ is only the human-run stuff.)
Live — dogfood-corpus maintenance
refresh-vendor.sh— re-fetch + re-pin the SHA-pinned vendored plugin snapshots undertest/dogfood/. Run when: bumping a vendored plugin to a newer upstream commit. Seeresearch/dogfood-corpus.md.dogfood-sweep.sh— runvigiles auditacross a pinned list of real OSS Claude Code plugins and tally what the detectors find. Run when: before a release or after a detector change, to prove no false-positive regression on real plugins (breadth, vs the few SHA-pinned slices asserted in tests).
Live — harness ground truth
-
measure-hook-matcher-semantics.mjs— measure how the harness ACTUALLY matches a hookmatcher(literal equality vs unanchored regex; which MCP patterns fire on which server naming), by running the realclaudeCLI against the scripted mock model with one hook per row and a marker file as the oracle. No API key, no cost. Run when: a new Claude Code version lands, or before changingsrc/core/hook-matcher.ts— that detector encodes this table, and issue #131 is what happens when it is assumed instead of measured. Usage:node tools/measure-hook-matcher-semantics.mjs [--suite=builtin|mcp] [--server=<name>]. -
measure-hook-startup.mjs— price what a compiled hook costs to START: layer by layer (onerequireper module, against a bare-Node baseline) and end-to-end per hook ROLE (hook-runtime run-programover a real hook, event on stdin). Run when: you are about to add an import anywhere on the hook runtime's path (src/cli.ts,src/hook-runtime.ts,src/core/hook-program.ts,src/core/bash-effects.ts), or after a Node upgrade. A hook runs on EVERY matching tool call, so this cost is per tool call — issue #216 is what happens when it is not measured. The deterministic half is a test (src/hook-runtime-graph.test.tsasserts the module GRAPH, which cannot be flaky); this script owns the NUMBERS, which are machine-specific and must be re-measured rather than quoted. Usage:node tools/measure-hook-startup.mjs [--runs=N]afternpm run build.
Occasional — pre-launch
-
smoke-published.sh— run the PUBLISHED package from the registry against a throwaway Python repo (pyproject.toml + ruff, no package.json) and assert the OUTPUT ofaudit/lint/init/compile, not just their exit codes. Run when: before announcing a release, on the exact version you are about to point people at. Every gate inci.ymlruns the CLI out ofdist/(version: local), so the suite can be green while the artifact users download is broken — a file missing fromfiles[], a subpath not exported, a resolve hook that only works when vigiles is a local dependency. Usage:bash tools/smoke-published.sh [version](defaultlatest); exits 77 and says so whenruffis absent, since the linter check is the point. Measured 2026-09-07 againstvigiles@latest: it FAILED, naming both live launch blockers —lintprinting "No linters detected" on a configured ruff repo, andcompileexiting 1 with a raw Node module-resolution stack. -
fp-sweep.sh— launch-readiness "don't cry wolf" sweep: clone popular plugins/marketplaces, runvigiles audit, surface any high-precision rule flag as a false-positive candidate to triage. Run when: pre-launch FP triage.
Demo assets — for the UNBUILT "polished demo" (roadmap)
-
demo.sh— record an asciinema cast ofvigiles lintcatching stale references (asciinema rec --command "bash tools/demo.sh" vigiles.cast). -
make-demo-gif.py— generate an animated terminal GIF (Pillow).⚠️ Status: both feed the roadmap item "Build the ONE polished front-door demo," which is not built yet. The old
vigiles-demo.gifwas removed as unreferenced, so their output is currently unused. Kept as the tooling for that item — delete if the demo direction is dropped.
Not skills — on purpose
A tools/ script is deterministic automation; a skill is a model procedure with
judgment, so these are not skills. The two with real judgment on their output
(fp-sweep triage, dogfood-sweep regression summary) are a roadmap candidate to
wrap as .claude/skills/ contributor skills at launch — the skill would invoke
the script and reason about the result. refresh-vendor / demo.sh /
make-demo-gif.py stay plain scripts (mechanical or interactive, no judgment).