* docs: sharpen README lint/guard copy - Replace internal "7 catalogs" jargon with "7 linters (ESLint, Ruff, Clippy, and four more)"; Cedar stays in the subdoc - Drop the plugin-audit paragraph from the Lint section (secondary use case, still reachable from More) - Rewrite the Guard intro to lead with the concrete "fail open" failure and name what vigiles emits (exit code, JSON, AST-backed matcher) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: surface the $0/subscription cost story in the first paragraph The "no metered tokens, runs on your Claude subscription" wedge was buried in the last line of the Eval section. Pull a crisp version into the opening paragraph (above the fold); the Eval section keeps the detailed promptfoo/DeepEval contrast as proof. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: surface agent-writes-the-harness flow, reorder More, drop internal links - Quick start now leads with "then your agent writes the harness for you": the three model-invocable skills (test-harness/strengthen/ edit-spec) and the plain-English ask each one answers - Reorder More so CLI sits above the leaderboard (whose link points into the CLI docs); trim the now-redundant Skills bullet to a pointer - Stop linking internal research/ docs from the README (the moat link and the Research index) per public-vs-internal-docs Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(compile): auto-refresh an existing harness.gen.ts compile now keeps a whole-harness registry in sync as a side effect, so the user never hand-runs generate-harness. Gated on the file already existing (compile maintains a registry you opted into, never imposes one) and cheap (parses specs, no linter spawn — unlike generate-types/ generate-schema, which spawn every linter and stay on the config-change guard). A duplicate agent name fails the compile. Docs: note the auto-refresh in the compile section and mark the generate-* artifacts as dev-toolchain output rarely run by hand. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(src): add nested CLAUDE.md with a high-bar-for-new-commands rule Directory-scoped guidance loaded when working in src/: adding a new CLI verb or hook-runtime subcommand must clear a high bar (genuinely new mental action, no overlap with an existing command, siblings collapse to one verb + subcommand, runtime entrypoints stay hidden). Complements the root cohesive-cli-surface rule, placed next to the code that defines the command surface (cli.ts, cli-commands.ts). Spec-managed (require-spec + integrity); dogfoods vigiles compile. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: drop the "(generated)" annotation from the API reference link Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(build): stop src/CLAUDE.md.spec.ts shadowing the root spec + add docs rule The nested src/CLAUDE.md.spec.ts (added last commit) compiles under rootDir:src to dist/CLAUDE.md.spec.js — the exact path loadSpec resolves the root CLAUDE.md.spec.ts to, so recompiling the root spec loaded the src spec and clobbered CLAUDE.md. Exclude src/*.md.spec.ts from tsc so both load via the tsx fallback (as the root always has). Also adds the no-internal-links-in-public-docs guidance rule (public docs never link research/*.md; keep the fact in research, don't link it), reconciles the old "link to research" line in public-vs-internal-docs, and files the deterministic lint-rule version as a P1 roadmap item. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: separate public/internal indexes + surface eval safety - docs/README.md no longer links into internal research/ (public docs don't point at the private record) - research/README.md is explicitly labelled the internal record - README Eval section gains a "Safe to repeat" note (sandbox + ephemeral dir + egress + interceptTools, past-the-sandbox case flagged) linking docs/safety.md Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: stop public docs linking internal research/ + add safety FAQ Strip every research/*.md link from docs/*.md (the no-internal-links-in- public-docs rule) — ~60 links across 16 guides plus the docs/ index; prose kept readable, no orphaned research doc (all still cited from CLAUDE.md keyFiles / research/README.md, orphan-docs green). Add a FAQ to docs/safety.md answering the side-effects-beyond-the-sandbox question honestly: network via egress allowlist, a real DB/redis/browser via a disposable container you stand up (turnkey provisioning is a committed next step), irreversible externals via interceptTools, and the no-Docker deterministic/record-replay tiers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(cli)!: consolidate verbs — measure/explain→scan flags, refs→lint, generate-*→generate <kind> De-sprawl the human verb surface from 13 to 8 (the high-bar-for-new-commands rule, dogfooded). All breaking: - `measure <dir>` → `scan <dir> --trigger` (the model-gated behavioral column folded into scan; trigger-rate + collision matrix unchanged) - `explain <dir> [name]` → `scan <dir> --explain [name]` (the deterministic WHY, a sibling of --fix-plan on the same ScanReport) - `refs <file>` → folded into `lint` (the unmarked-refs check is already a lint rule; the PostToolUse hook keeps its shared collectRefIssues entry) - `generate-types|schema|harness` → `generate <kind>` (one verb + subcommand; the three dev-toolchain generators grouped, not N hyphenated siblings) scan reuses handleMeasure/handleExplain directly behind --trigger/--explain. Docs, help text, the regen guard, tests, and the self-command-refs dogfood all updated together (cohesive-cli-surface); 1588 tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: split the GitHub Action reference out of cli.md into its own doc cli.md was 718 lines (kitchen sink). Extract the ~108-line GitHub Action section into docs/github-action.md (quick start, inputs, output channels, versioning) and leave a short pointer; retitle the cli.md intro. Update the README More bullet (CLI · GitHub Action) and the docs index. cli.md → 619 lines. orphan-docs + self-command-refs green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat(lint): add prefer-compiled-hooks rule — one nudge toward compiled hooks A single repo-level recommendation (one finding regardless of hook count): when a plugin ships hand-written hook commands that aren't compiled vigiles/hook artifacts, nudge toward compiling them — the message links docs/compiled-hooks.md and points at guardrail-check for verifying the ones you have. A discovery nudge, not a defect: the hand-written shell lane stays first-class, so it fires ONCE (never per-hook), defaults to warn (info- level ℹ; informational, doesn't fail CI), and is opt-out. Detector (manualHookCount: a command that isn't a hook-runtime invocation) lives in scan.ts and is shared by scan + lint (one-detector-no-drift). Doc + matrix rows added (rules-docs-in-sync). Honest caveat in the doc: it's form-based, so it stays a recommendation. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(research): record the eval-startups critique mapped onto positioning New research/eval-startups-positioning.md captures Thomas Liao's "why eval startups fail" critique and its implications: vigiles dodges the trap (harness-eval not model-eval, tooling not services, eval already a feature), but it's a yellow flag on the public ecosystem-benchmark as a MOAT (Goodhart-gameable → keep it a thin acquisition flywheel) and it independently validates the attestation/safety direction (capability-diff, guardrail proof) as the only defensible eval-shaped niche. Linked from research/README.md, measurement-authority.md (whose benchmark thesis it challenges), and strategic-synthesis-2026-06.md (whose Tier-2 conformance/attestation play it validates). Internal-only per no-internal-links-in-public-docs; orphan-docs green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: purge internal positioning from public docs (jargon, name-drops, benchmark) Public docs (README + docs/) now state user-facing value, not vigiles's internal strategic vocabulary: - strip "moat" from 14 places → user-facing wording (engine / cost advantage / differentiator / cross-reference) - remove all bare research/*.md name-drops from docs/ (the backtick form the link-strip + deterministic check miss); scope-descriptions reworded to "configured directories" rather than naming research/ - README: drop the "ranks the hyped ecosystem: what works vs hype" benchmark banner from the Eval row + kicker (depth stays in measuring-skills.md) — per eval-startups, the benchmark is a flywheel, not a headline - docs/cli.md: "moat #2 (research/…)" → "the capability-diff check" Codified in CLAUDE.md: public-vs-internal-docs now has a strategic- vocabulary clause (moat/measurement-authority/flywheel stay internal; public docs name the benefit), so this doesn't regress. orphan-docs + self-command-refs green; the public↔private split stays docs/ vs research/. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): apply multi-persona review fixes From a multi-persona README review (power user / staff eng / plugin author / non-TS user / eng manager): - non-TS users: a visible "No TypeScript needed — lint + inline comments work on any CLAUDE.md" line (was invisible; contradicted progressive adoption) - plugin authors: surface "vigiles scan ranks a marketplace, no key" above the fold (was buried in More) - eng-manager cost: hero now says deterministic layers run free in CI; merged kicker+affordability nails "real-model on a dev's own subscription, never a CI token" - staff-eng skepticism: the 2/7→7/7 claim now reads "measured, not asserted"; the typed-pipeline claim is concretized to the real pipe()/needs() mechanism - fixed a broken blockquote and trimmed the safe-to-repeat para to hold length Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): capture the README design direction as an editor-facing comment A top-of-file comment block records the framing the README must hold: lead with benefits, present compiled specs as the easy/auto-enforced next step (skills write them, hooks compile them; JS→TS evolution, not a wall), keep all four instruments first-class incl. Eval, stay scannable + under the length cap, and no internal vocabulary/research links. Guides future edits. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): benefits-forward rewrite + specs-are-easy framing Per the README direction: lead with what the user GETS, and present the typed spec as the frictionless auto-enforced next step, not a TS wall. - hero: stronger, concrete failure modes (stale ref / guard fails open / skill never fires), keeps Agent=Model frame, cost above the fold; dropped the redundant centered pitch (merge) - table cells condensed to punchy benefit one-liners - "You don't hand-write specs — your agent does. TypeScript for your harness: opt-in, gradual, auto-compiled, never a chore" (skills write them; JS→TS evolution) - de-dupe cost (drop "not metered tokens" from the Test cell) - "Not for you if …" honest-scope line - docs/verifying-instruction-files.md: back the claim — "you rarely hand-write one; edit-spec/strengthen write it, a hook recompiles on save" Eval stays a first-class instrument. fmt + self-command-refs + orphan-docs green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(readme): don't lead callouts with "typed spec" — benefit first Casual readers shouldn't be greeted by "a typed spec is a program" / "TypeScript for your harness" in the prominent callouts. Reframe both to lead with the outcome and demote the spec to a soft, opt-in aside: - specs-are-easy block: "You don't hand-write any of this — your agent does … you never touch it by hand" (drop the TypeScript-for-your-harness headline) - Lint blockquote: "Bad states stop compiling … turn it up only when you want it" (drop "Markdown is prose; a typed spec is a program") - the edit-spec bullet says "edits the source", not "the typed spec" "No TypeScript?" reassurance and the optional .spec.ts mention stay. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: refresh HANDOFF for the CLI-consolidation + docs/README overhaul session Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * chore: prettier-format HANDOFF Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(ci): update the stale generate-types ref the verb rename missed The CI "Check generated types compile" step ran `node dist/cli.js generate-types` — renamed to `generate types` in the verb consolidation, so the job failed with "Unknown command". self-command-refs didn't catch it because it didn't scan .github/; add .github/ to its scan scope so a stale `cli.js <cmd>` ref in a workflow fails the dogfood from now on. Also saves the prefer-compiled-hooks-vs-require-hook-spec decision to research/hook-modes-and-testing.md (the nudge, not a requirement; the write-hook authoring skill is the real authoring lever, unbuilt). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(lint): eslint-ignore src/*.md.spec.ts (excluded from tsconfig) The dist-collision fix excluded src/*.md.spec.ts from tsconfig, so eslint's type-aware project service can't resolve src/CLAUDE.md.spec.ts and errors "not found by the project service". It's a tsx-loaded build input (like the root CLAUDE.md.spec.ts, which isn't linted either), so ignore it in eslint to match. Latent since the exclude landed; surfaced now that the earlier generate-types step no longer fails first. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1.9 KiB
coverage
Enforce minimum spec coverage thresholds. Measures how much of the project surface is documented in specs.
Configuration
{
"rules": {
"coverage": ["warn", { "scripts": 50, "linterRules": 5 }]
}
}
Severity
| Value | Behavior |
|---|---|
"error" |
vigiles lint exits non-zero when below threshold |
"warn" |
Prints warning when below threshold, exits 0 |
false (default) |
Skip coverage checks |
Thresholds
| Option | Type | Description |
|---|---|---|
scripts |
number | Min % of npm scripts documented in spec commands |
linterRules |
number | Min % of enabled linter rules with enforce() in specs |
Both thresholds are optional. Omit a threshold to skip that metric.
What it measures
Script coverage
Compares package.json scripts against commands declared in specs. Checks both npm run <script> and npm <script> forms.
Script coverage: 4/5 (80%) (threshold: 50%)
missing: deploy
Linter rule coverage
Compares enabled linter rules (detected via vigiles generate types) against enforce() declarations in specs. A rule is "covered" when the spec explicitly references it.
Linter rule coverage: 3/64 (5%) (threshold: 5%)
Example
{
"rules": {
"coverage": ["error", { "scripts": 80 }]
}
}
This fails lint if more than 20% of npm scripts are undocumented in specs. Linter rule coverage is not checked (no linterRules threshold).
Why
Specs that cover only half the project surface give agents an incomplete picture. Coverage thresholds ensure specs grow alongside the project. Start with low thresholds and raise them as specs mature.