Files
zernie__vigiles/src/cli-flags.ts
T
zernie cf40a0f96c docs: clarify pillar decoupling and surface install in harness section (#32)
* docs: clarify pillar decoupling and surface install in harness section

State the two-pillar decoupling plainly (neither pillar gates the other)
instead of framing it by which file each pillar doesn't need. Add a
concrete install command to the "Test your harness" setup block so it
anchors to `npx skills add zernie/vigiles` / `vigiles/testing` instead of
a vague "Install vigiles" prompt.

* chore: gitignore locally-installed dev skills

Ignore .agents/, .claude/skills/, and skills-lock.json so locally
installed skills (e.g. obra/superpowers) used for internal development
stay out of the repo and don't show as uncommitted changes.

* chore: vendor obra/superpowers skills for internal dev use

Add the superpowers skills under .agents/skills/ (with .claude/skills/
symlinks) and skills-lock.json for internal development. Verified via
`npm pack --dry-run` that none of these paths ship in the npm package or
the Claude Code plugin (.claude-plugin/plugin.json ships only ./skills/),
so they're part of the repo but not part of what vigiles ships.

Reverts the prior .gitignore exclusion of these paths.

* chore: enable superpowers via plugin marketplace instead of vendoring files

Replace the vendored obra/superpowers skills (.agents/, .claude/skills/,
skills-lock.json) with a project-scoped plugin declaration in
.claude/settings.json (extraKnownMarketplaces + enabledPlugins). Claude
Code fetches the plugin into its ephemeral cache on startup, so the repo
carries the enablement config, not 60+ third-party files.

* docs: add production-grade GHA + CLI as a tool requirement

Add the "prod-grade-gha-cli" guidance rule to the spec (compiled into
CLAUDE.md): the CLI and GitHub Action are first-class, production-grade
surfaces — CLI is the single source of truth with every Action input
mapped to a real flag, the Action is a composite over the published CLI
that works as uses: zernie/vigiles@v1, declares outputs via
$GITHUB_OUTPUT, ships a full documented workflow, and is dogfooded in CI.

* feat!: production-grade GitHub Action + CLI flag parity

Make the GitHub Action and CLI first-class, production-grade surfaces
(satisfies the new prod-grade-gha-cli rule).

CLI:
- Add --max-rules=<n> and --catalog-only flags (src/cli-flags.ts,
  applyConfigFlags) so every Action input maps to a real CLI flag instead
  of a config-file-only knob. Pure + unit-tested.

GitHub Action:
- Rewrite action.yml as a composite action over the published `npx
  vigiles` CLI. The old node20 entry pointed at dist/action.js, which is
  gitignored and never committed, so `uses: zernie/vigiles@<ref>` could
  never actually run — only npm did. The composite reuses the same tested,
  published artifact.
- Declare the `valid` output and set it via $GITHUB_OUTPUT (drops the
  deprecated ::set-output). Add `version` (incl. `local` for dogfood) and
  `working-directory` inputs.
- Remove the now-dead src/action.ts entry.

CI / release:
- Dogfood the Action in ci.yml via `uses: ./` with version: local and
  assert the `valid` output.
- release.yml maintains a floating `v1` major tag so `uses:
  zernie/vigiles@v1` tracks the latest release.

Docs/tests:
- Full Action reference in docs/cli.md (copy-paste workflow, permissions,
  inputs, output, @v1-vs-@main versioning) + README pointer.
- Structural test (src/action-yml.test.ts) locks in the composite/outputs/
  no-set-output contract.

BREAKING CHANGE: the GitHub Action is now a composite action and no longer
ships dist/action.js. Inputs/outputs are unchanged in name where they
overlap (command, paths, max-rules, catalog-only) but the action now adds
`version`/`working-directory` inputs and a `valid` output. Pin to
`zernie/vigiles@v1` going forward.

* feat: PR comments + job summary output, JS/TS harness scripts

GitHub Action output:
- Post an automatic STICKY PR comment on pull_request events (found by a
  hidden marker and updated in place, never duplicated) + always write a
  job summary ($GITHUB_STEP_SUMMARY), alongside the existing inline
  annotations. New `comment` / `github-token` inputs; best-effort so a
  fork PR without write access degrades to a warning. Dogfooded in ci.yml
  (check job granted pull-requests: write).

Dual-language harness/eval scripts:
- `vigiles test` / `vigiles eval` now discover and run scripts authored in
  JavaScript (.mjs/.cjs/.js) OR TypeScript (.ts/.mts/.cts). A TS script
  runs through tsx when installed, else Node's native type stripping
  (>=22.6), with an actionable error otherwise (src/adapters/claude-code/
  run-scripts.ts: interpreterArgs/detectNodeCaps/scriptGlob).

Requirements + docs:
- Extend the prod-grade-gha-cli rule (three output channels) and add the
  dual-language-tests rule to the spec (compiled into CLAUDE.md).
- Document the comment behavior + pull-requests: write in docs/cli.md;
  rewrite the JS-vs-TS section of docs/testing-matrix.md.

Our own unit suite remains 100% TypeScript (every src/**/*.test.ts);
examples/harness/*.mjs stay JS on purpose as zero-dep CLI demos.

* docs: slim README to a billboard landing page

Cut the README from ~207 to ~118 lines, moving it out of the "long but
neither pitch nor index" middle into a focused landing page (Knip/esbuild
shape), per a survey of dev-tool READMEs (Ruff/Biome/Vitest/ast-grep/…)
and standard-readme/Diátaxis guidance.

- Pitch once, up top: tagline + one "why" paragraph + the single
  two-pillar table (was restated 4x across header/nav/table/TOC).
- Keep the two concrete "first taste" artifacts: the stale-CLAUDE.md
  snippet and the runHook example.
- Collapse the CLI, Skills, and Related Tools sections + the duplicated
  promptfoo rebuttal into one-line links (the detail already lives in
  docs/). Delete the <details> table of contents.
- Drop the README section anchor from docs/verifying-instruction-files.md
  (now points at the README top).

Verified: `vigiles audit` passes with zero orphan docs (every doc the
README no longer links is still indexed by docs/README.md or CLAUDE.md).

* docs: document how the test tiers run in CI

Capture the CI tier mapping in docs/testing-matrix.md (it previously
lived only in discussion): which tier runs in which ci.yml job, that CI
runs every tier except evals, that evals are manual ($/non-deterministic),
and the cheapest-first best practice + the per-tier test:unit/integration/
e2e scripts that enable the split.

* docs: recast both README pillars as Without/With with real output

Replace the prose-heavy pillar pitches with tight "Without vigiles / With
vigiles" comparisons that show ACTUAL vigiles output (captured from real
runs, not invented):

- Pillar ①: `vigiles audit` flagging a rule that "exists but is disabled
  in eslint config" (the valid-≠-true differentiator), a passing rule, and
  a typo with a did-you-mean suggestion.
- Pillar ②: the runHook assertion + a real `vigiles test` pass summary,
  framed around a silently-broken --no-verify guard.

Shorter and more concrete; shows the payoff instead of describing it.

* feat: refs-hook nudges the agent to mark references on save

Close the authoring-time half of pillar 1: an automatic, in-loop nudge so
the agent expresses references as verifiable vigiles marks instead of
unverifiable prose. Ships in the plugin — end-users author nothing.

- New `unmarked-refs` rule (default "warn"). The PostToolUse refs-hook
  (hooks/refs-nudge.sh → `vigiles refs-hook`) fires on every CLAUDE.md /
  AGENTS.md / SKILL.md edit and, by severity: "warn" → injects a
  non-blocking nudge into the agent's context (mark these refs); "error" →
  blocks the edit (exit 2); false → off. Wired into .claude-plugin/
  plugin.json so it's on by default after `vigiles init`.
- Refactor the detection into core: collectRefIssues (shared by the
  `vigiles refs` CLI and the hook) + the pure refsHookAction severity map,
  unit-tested.
- Honest scope: catches code-shaped backticked refs; plaintext prose is
  undecidable (the floor). Hooks are Claude Code / Codex only; other
  harnesses fall back to instructions + the `vigiles audit` CI backstop.
- Docs: the from-file-save flow in verifying-instruction-files.md, a new
  rules/unmarked-refs.md, and the rule added to the cli.md / docs index.

* test: dogfood the refs-hook through the runHook tier

skills-dogfood.test.ts only proves the hook script exists; this drives the
actual built `vigiles refs-hook` via the repo's own runHook unit tier and
asserts it FIRES correctly — nudges (non-blocking, additionalContext) by
default, blocks (exit 2) under unmarked-refs:"error", stays silent when
off, and no-ops on a clean or non-instruction file. Closes the "test your
harness, don't trust it on vibes" gap for the new hook.

* fix: scope refs-hook nudge to linter-rule names (kill false positives)

Dogfooding the hook on the repo's own SKILL.md files exposed heavy
over-firing: skills/test-harness/SKILL.md got 23 nudges, all on API names
(`runHook`, `runEval`, …) that are documentation prose, not stale
references. The `isCodeShaped` heuristic flagged every backticked
camelCase/Pascal/SCREAMING identifier — far too noisy to ship at
warn-default.

Tighten to high-signal only: flag a slash-scoped name with NO file
extension (`eslint/no-console`, `@typescript-eslint/no-explicit-any`,
`boundaries/dependencies`) and nudge toward `enforce()` / inline
`<!-- vigiles:enforce -->`. Bare identifiers and file paths are no longer
flagged (paths re-introduce the noise — the same SKILL.md files cite
`docs/harness-testing.md`, `package.json`, …; symbol marking stays opt-in
via an explicit `vigiles:symbol`).

Re-dogfooded: test-harness 23→0, strengthen 2→1 (keeps the one real rule
`@typescript-eslint/no-unused-vars`), migrate-to-spec 0. Tests + docs
updated to the narrowed scope.

* test: real-claude dogfood proves the refs-hook fires in a session

Install-and-verify: with the claude CLI present, drive the deterministic
harness tier (real claude + scripted mock model, no API key) so the mock
Writes a CLAUDE.md naming `eslint/no-console`, the PostToolUse refs-hook
fires, and requestContains asserts the non-blocking nudge reached the
model's context. Closes the last gap — previously verified only through
the runHook unit tier. examples/harness/refs-nudge.harness.mjs runs in CI
via `npm run test:harness` (claude is installed there) and skips cleanly
without the binary.

* feat: vigiles init sets up both pillars, interactive-or-not, via the v1 Action

Make onboarding smooth and complete (the smooth-adoption rule):

- BOTH pillars by default. Pillar 1 (spec + types + compile) as before, PLUS
  Pillar 2 — scaffold a starter vigiles.harness.mjs the user adapts to a
  real hook. `--pillars=verify|test|both` scopes it.
- Interactive when a human's at a TTY (prompts: pillars / CI / plugin),
  NON-INTERACTIVE for agents / CI / piped input or `--yes` — so "set up
  vigiles" from a Claude Code / Codex prompt Just Works without hanging.
  Pure plan logic in src/setup-plan.ts (parseSetupArgs/shouldPrompt/
  resolvePlan), unit-tested; IO stays in cli.ts.
- CI wiring now creates a real `.github/workflows/vigiles.yml` using the
  production `zernie/vigiles@v1` Action (verify job + a harness job when
  Pillar 2 is on), instead of appending a raw `npx vigiles audit` line to
  an existing workflow.
- New flags: --yes/-y, --pillars=, --no-plugin. Docs updated (cli.md init
  reference, agent-setup.md agent flow, README quick start).

* test: CI-test the init command's new behaviors end-to-end

Spawn the real CLI in a temp project and assert the new init behaviors,
offline via --no-plugin (no network skills-add): default sets up both
pillars + a zernie/vigiles@v1 workflow (asserts the spec, vigiles.harness.mjs,
and the workflow YAML contents); --pillars=test scaffolds only the harness
(no spec/workflow); --pillars=verify only the spec. Also add --no-plugin to
the existing bare-init spawns so they stay offline and deterministic.

* docs: make the "agent installs + tests skills" flow first-class

Viewed from an agent handed "install vigiles and test my skills", the
README install path was clear but the test-skills path was muddy — the
only pillar-2 example was a hook, measureTriggerRate (the "does the skill
fire?" primitive) was never named, and the model-invocable test-harness
skill was buried as 1-of-8.

- README: name measureTriggerRate (recall + precision) in the tier line,
  and surface that `init` installs the test-harness skill so "test my
  skills" fires it.
- Add the `great-agent-flow` guidance rule: the agent-told-a-short-prompt
  experience must be discoverable from the README front door and
  frictionless (one non-interactive install, the skill fires the test,
  the skill-testing path is high-signal not buried).

* fix: no blanket claude-gate; skips are loud, never a silent green

Two fixes for `vigiles test` / `vigiles eval`, behind a stated principle
(the new no-silent-skips rule):

1. Drop the blanket `claude`-gate in handleRunScripts. Unit-tier runHook
   tests need no model, so they now ALWAYS run — previously the whole
   command bailed if `claude` was absent, so a scaffolded vigiles.harness.mjs
   + `npx vigiles test` looked broken on a machine without claude.

2. Make skips LOUD. The runner classifies each script pass/skip/fail
   (exit 0 / SKIP_EXIT_CODE 77 / else), tallies them separately ("1 passed,
   1 skipped"), shows `⊘ SKIPPED`, and a skip never fails the run (anyFailed).
   Add a `skip(reason)` helper (vigiles/testing) and convert the four
   deterministic examples from `console.log(...); process.exit(0)` (a silent
   fake ✓) to `skip()`. Up front, `vigiles test` notes when claude is absent.

Verified end-to-end with claude off PATH: the unit example runs+passes, the
deterministic example reports ⊘ SKIPPED, summary "1 passed, 1 skipped",
exit 0. Adds the no-silent-skips guidance rule + skip-aware runner tests.

* feat: vigiles test --no-skip (fail on skip) + README agent prompt

Answer "shouldn't a skip fail loudly?" — yes, where you assert the
capability is present. A skip passes by default (tiers need different
capabilities not present everywhere; failing-by-default would make the
command red in any partial env — this repo's harness job legitimately
skips the egress tier), but `vigiles test --no-skip` makes any skip a
failure for the CI job that asserts everything must run: a green-with-skips
is itself untested surface. The skip stays loud + tallied either way.
Extends the no-silent-skips rule; integration-tested.

Also (per the agent-flow ask): add a paste-into-Claude-Code/Codex prompt to
the README quick start — install + verify + write/run a harness test with
good defaults, asking about CI gating / real-model eval / --strict.

* fix: cover skip() under the 100% gate; flush skip/--no-skip docs

The no-silent-skips commit broke CI's coverage gate (step 13): skip() in
harness-assert.ts (a coverage-included file) calls process.exit, which is
unreachable in-process → dropped below 100%. Mark it with the repo's
`v8 ignore` convention (same as the spawn boundary); it's exercised via a
child process in cli.test.ts / run-scripts.test.ts. Verified locally with
`npm run coverage`: 100% stmts/funcs/lines, branches 92.43% (≥90 floor).

Docs flush (before merge): document the loud-skip / unit-runs-without-claude
/ `--no-skip` behavior in docs/testing-matrix.md and the test-harness skill's
"Run it" step, so the agent's authoring guide matches the runner.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-15 03:30:43 +05:00

45 lines
1.3 KiB
TypeScript

/**
* Command-line config overrides.
*
* The GitHub Action is a thin composite over this CLI, so every Action
* input must map to a real CLI flag (no config-file-only knobs the Action
* can't reach — see the `prod-grade-gha-cli` rule). This module turns the
* shared flags into `VigilesConfig` overrides on top of the loaded config.
*
* Pure and exported for unit testing.
*/
import type { VigilesConfig } from "./core/types.js";
const MAX_RULES = "--max-rules=";
/**
* Apply shared command-line overrides to a loaded config.
*
* --max-rules=<n> → config.maxRules (positive integer; ignored otherwise)
* --catalog-only → config.catalogOnly = true
*
* Unknown flags are left untouched (each command parses its own besides
* these). Returns a new object; the input is not mutated.
*/
export function applyConfigFlags(
config: VigilesConfig,
args: readonly string[],
): VigilesConfig {
const next: VigilesConfig = { ...config };
const maxRulesFlag = args.find((a) => a.startsWith(MAX_RULES));
if (maxRulesFlag) {
const n = Number(maxRulesFlag.slice(MAX_RULES.length));
if (Number.isInteger(n) && n > 0) {
next.maxRules = n;
}
}
if (args.includes("--catalog-only")) {
next.catalogOnly = true;
}
return next;
}