mirror of
https://github.com/zernie/vigiles.git
synced 2026-09-14 20:53:57 +08:00
41169e379a
* feat(audit): split Tested into Tested + Evaluated, with a `not measured` state `Tested` collapsed deterministic harnesses and real-model evals into one number, and its finding read "N surfaces with no vigiles test/eval" — one sentence for two prescriptions three orders of magnitude apart in cost. A harness (`*.harness.mjs`, `*.test.*`) is free, runs in milliseconds on every push, and asks whether a gate still catches what it claims. An eval (`*.eval.mjs`) spends real model calls, runs on a schedule, and asks the one question a deterministic read cannot: does this skill fire at all? Folded together, a repo with complete deterministic coverage and no evals scored identically to a repo with neither. - test-coverage: one discovery pass, partitioned by suffix into a harness tier and an eval tier; the union `covered`/`untested` fields are unchanged. Mirrored in the browser-safe twin. - scan: carries `untestedHarness` / `unevaluated` / `evaluable` (optional and additive; a producer predating the split falls back to the union). - audit-score: `Tested` (deterministic) and `Evaluated` (real-model), both advisory. `Evaluated` has THREE states — a number, a measured `0`, and `not measured` when the executing checks were skipped and no eval exists. Scoring 0 there would report the absence of a check as its result, which is the pattern this tool exists to name elsewhere. In that state the ring names the command that would measure it. - cli: threads the existing execute-skip signal into the ring. - report/HTML/JSON: both rings render; the inventory carries both counts. Grade weighting is unchanged — both rings are advisory and neither enters `reportDeductions`. Audited on this repo before and after: A (100/100) both times; only the diagnostic rings move (Tested 67 → Tested 58 + Evaluated 22). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012omt6sirxNMyZN2x5RhK2u * docs(site): the audit grades six categories, not five Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012omt6sirxNMyZN2x5RhK2u * fix(lint): a clean union must not read as 'firing verified' Every surface can have a deterministic harness while nothing has ever measured that a skill fires. The gate is unchanged (still the union), but the ✓ now says so instead of implying the question was asked. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012omt6sirxNMyZN2x5RhK2u * test(site): regenerate the browser-parity fixture for the Evaluated ring The audit now emits a sixth category, so the Node-generated expected report and the in-browser one differ until the fixture is regenerated — which is what the test's own header prescribes for an intentional engine change. Regenerated with `node site/scripts/gen-parity-expected.mjs`. The diff adds the Evaluated ring and its two inventory counts and nothing else: grade and overall are untouched, which is the check that the split did not move the weighting. * fix(coverage): `Tested` measured a STRING, not a test DEFECT. A surface counted as covered when its path or namespace appeared as a SUBSTRING anywhere in any discovered test file (`content-reference`, in src/test-coverage.ts). Two halves, and the second was worse. REPRODUCTION (a real repo: 37 skills, 14 hooks). Appending ONE line to an existing harness file: // probe: skills/argument-arc moved the audit's untested count 33 -> 32. A comment. Not a test. Reverting the line put it back. So `Tested` was trivially gameable and the gaming was indistinguishable from real work. The false NEGATIVE: that repo's `.claude/pipeline/skills.harness.mjs` genuinely asserts over all 21 of its pipeline skills, but builds each path at runtime (`join(root, ".claude", "skills", name, "SKILL.md")`), so it contains ZERO literal `skills/<name>` strings (grep: 0 matches) and covered NOTHING. Generality was PENALISED: 21 trivial files each naming their own subject would have scored better than the one harness that actually tests all 21. This is the exact substitution vigiles names in other people's repos — presence of a surface taken for presence of the property, the way `noExplicitAny` in a config is taken for the rule being enforced. The tool was committing it in its own scoring. FIX, three parts (new src/coverage-evidence.ts — pure and browser-safe, so the disk detector and its in-browser twin cannot drift on what decides a grade): 1. DECLARATION. A test file may carry `vigiles:covers <surface> ...`, mirroring the existing `vigiles:ignore-test` marker convention on the other side. A runtime-path harness can now say what it covers; an accidental mention cannot. Declaration beats inference — the same move as declaring `allowed-tools` instead of inferring capability. 2. PROVENANCE. Every coverage decision now carries HOW it was reached — declared / colocated / mention — and the counts are printed by `formatUntestedReport`, by `formatScanReport`, and carried in `AuditReport.inventory.coverageEvidence` (additive; schema version unchanged). A repo whose coverage rests entirely on substring matches can finally see that about itself; "28 covered" and "28 names that happen to appear in a file" used to print identically. 3. The bare substring detector is KEPT, deliberately, and said so in the code: it is the zero-config path that makes an ordinary `foo.test.ts` naming `skills/foo` count without anyone learning a marker, and dropping it would flip honestly-covered surfaces to "untested" overnight. What it no longer reads is COMMENTS. A path in a comment is prose about a test; a path in executable code is the test reaching for the surface. Prose isn't policy, including ours. It is also now visibly the weakest tier in the report, and when it is the ONLY evidence the report says so in words. Grade weighting is untouched. TESTS prove both halves, each watched failing first: with the fix reverted, "a surface named ONLY in a comment is NOT covered", "a runtime-path harness that DECLARES its surfaces is counted" and "a declaration OUTRANKS a mention" fail (3 failed | 20 passed); with it, 23 passed. Also guarded: a mention in CODE still counts, and a `"https://..."` string literal survives comment-stripping. Docs (untested-skill/-subagent/-hook) updated — untested-skill.md previously recommended "add the skill's path in a comment", which was advice to exploit the defect. * fix(site): key the browser-test prebundle to dist, so a stale cache can't lie DEFECT. `site/src/demo/browser-parity.browser.test.ts` guards node:zlib vs pako divergence by comparing an in-browser AuditReport against one generated in Node. `site/vitest.config.ts` pre-bundles the engine (`optimizeDeps.include` on `@engine/scan-files` / `@engine/audit-report` / `pako`) and Vite caches the result in `site/node_modules/.vite`. That cache SURVIVES `npm run build` — Vite invalidates on config/lockfile changes, not on the mtimes of an aliased path outside node_modules. So the browser side can compute against a PREVIOUS engine while Node computes against the current one. REPRODUCTION (observed 2026-08-07, twice — once by accident, once deliberately). Accidentally: an engine change added a report field; the parity test failed showing the field present on the Node side and absent in the browser. I read that aloud as a genuine engine divergence and started hunting a pako bug that does not exist. `rm -rf site/node_modules/.vite` -> 42/42 pass. Deliberately, to prove the fix: plant an extra `inventory` field in src/audit-report.ts, `npm run build`, regenerate the expected fixture, run the browser suite with a warm cache. - ORIGINAL config: `1 failed | 41 passed`, diff showing `"probeStaleCache": 1` on one side only — a field-level diff that reads exactly like a real divergence and says nothing about the cache. - THIS config: `42 passed`, cache dir rotated `.vite-engine-8baf73f9e373` -> `.vite-engine-a52a27e0137e`. Reverting the plant returns the fixture to its committed bytes. FIX. Key the prebundle cache DIRECTORY on a fingerprint of `dist/` (`site/vite.engine-stamp.ts`; `cacheDir: node_modules/.vite-engine-<stamp>`), sweeping the directories keyed to other fingerprints so they don't pile up. WHY a key and not the alternatives. Comparing the cache's mtime against `dist/` and forcing a re-bundle, or clearing `.vite` in the test script, both work and both leave the bad state REACHABLE — a comparison someone can get wrong, or a script step bypassed by running `vitest` directly. Keying the directory removes the state instead: a different `dist/` is a different path, so "prebundle built from another engine" cannot be reached, and an unchanged `dist/` still gets a warm cache. By construction rather than by check — the argument this project makes everywhere else about removing a capability rather than policing its use. The test's header instruction ("pako and node:zlib disagree — do NOT paper over it") was correct for one of the two states and actively misleading for the other; it now says which state is no longer reachable and why, and the assertion carries the same note in its failure message. * fix(site): make the parity-fixture generator emit prettier-clean JSON DEFECT. `site/scripts/gen-parity-expected.mjs` wrote the fixture with `JSON.stringify(x, null, 2)`, which is NOT what prettier prints for JSON — it keeps short arrays exploded across three lines where prettier collapses them. CI gates on `prettier --check .`, so every regeneration of the fixture committed a file that reddened CI later and far from the cause. It just did: branch headdec140afailed the `test` job on exactly this file after the Evaluated-ring change required a regeneration. REPRODUCTION: `node site/scripts/gen-parity-expected.mjs` then `npx prettier --check site/src/demo/__fixtures__/sample-repo.expected.json` → exit 1, six array literals reflowed. FIX: the generator formats through prettier before writing. Chosen over adding the file to `.prettierignore` because the fixture is committed and reviewed, so its diffs should look like every other file's — and because ignoring it would silence the alarm while leaving the state that trips it. Same shape as the stale-prebundle defect this fixture's own test hit: an artifact whose regeneration leaves a bad state is fixed at the regeneration, not downstream. Also adds `site/checks/` to `.prettierignore` — generated per-rule pages from the site prebuild, gitignored by `site/.gitignore`, but not ignored by prettier, so building the site locally reddened `fmt:check` on files nobody commits. Same class as the `site/test-results/` entry already there. `npm run fmt:check` is clean. * fix(cli): refuse unknown flags; `<verb> --help`; nothing-to-audit ≠ grade F DEFECT, in three parts. Every verb parsed its own flags with `args.includes("--x")` and ignored everything else. 1. AN UNKNOWN FLAG WAS SWALLOWED. Measured: $ npx vigiles audit --this-flag-does-not-exist Detected harness: claude-code Harness audit … EXIT=0 A full, confident audit as if the flag had never been typed. Not cosmetic: audit's flags govern what LEAVES the machine — `--no-html`, `--no-json`, `--out=<dir>`, `--serve`. `--no-htlm` wrote the HTML report while the author believed they had suppressed it. For a tool whose thesis is "prose isn't policy", accepting an argument it did not understand and continuing is the same class it names elsewhere: the argument is PRESENT, therefore the property is assumed enforced. The hole was on EVERY verb, not just audit — checked and closed uniformly (`lint`, `test`, `eval`, `compile`, `eject`, `init`, `generate`). `hook-runtime` is deliberately exempt: its argv is emitted into hooks configs by the harness, never typed by a human, so validating it would break the runtime contract. 2. `vigiles audit --help` RAN AN AUDIT. Help existed only on the bare `vigiles --help`. Asking what a flag is called executed the command. Now each verb answers `--help` with its own entry plus its complete flag list, and `printUsage` renders from the same table, so the banner and the per-verb help cannot drift. 3. "FOUND NOTHING" WAS INDISTINGUISHABLE FROM "FOUND SOMETHING BAD". How it bit: `npx vigiles audit --json /some/path`. `--json` is real but takes no value, so the PATH fell through to the positional scan directory. The audit ran over a directory containing no harness and reported "No loadable harness surface — nothing to grade yet", `skills: 0`, grade F, EXIT 0 — byte-wise indistinguishable from a genuine audit of a repo that scores badly. A confident measurement OF THE WRONG OBJECT, in silence. An operator who did not already know the repo had 37 skills would take it as fact. `score.empty` (no instruction file AND no surface) is now a distinct, loud outcome on stderr that says what was scanned, that nothing was measurable, and that a flag may have swallowed the path. No grade is printed for it. Note the flag check does NOT catch that invocation — `--json ./dir` is legitimate — so this branch is what makes it visible, and it covers every other way of pointing at the wrong place. EXIT CODE, chosen against the CLI's existing convention rather than invented: 1 already means "I ran, and the thing you asked about is bad" (lint findings, a failing harness script); 2 already means "I could not do what you asked" (the top-level error handler, `eval`'s refusal-to-fan-out, `generate` with an unknown kind). Both new outcomes are the second. A script that greps for a grade needs "unhealthy" and "I was pointed at the wrong directory" to differ, and the second has no grade at all. `--json` still emits the report for an empty target (it self-describes with `score.empty: true`), so the machine contract is kept. TESTS, each watched failing first. With both guards reverted, 8 of src/scan-cli.test.ts fail (unknown flag on audit, the typo suggestion, the per-verb sweep, `audit --help`, every-verb `--help`, the empty-target outcome, its `--json` form, and the codex-config case below); with them, 48 pass. Plus 7 pure tests in src/cli-flag-check.test.ts, including the over-rejection guard that asserts EVERY flag in the registry is accepted by its verb, and a dogfood test that drives the real `vigiles --help` and fails if it advertises a flag the registry doesn't know. TWO EXISTING TESTS CHANGED, both because the behaviour they pinned WAS the defect: - cli.test.ts "old --pillars / --verify flags no longer scope a pillar" asserted that a REMOVED flag is silently ignored and `init` proceeds with defaults. It now asserts the removed flag is rejected — a cleaner break, and the exact "silently do something other than what was asked" this fixes. - scan-cli.test.ts "honors the .vigilesrc.json `harness` key" audits a CLAUDE.md-only fixture AS CODEX, which finds no AGENTS.md and no codex surface. That is now exit 2 rather than a silent grade F at exit 0; the harness-resolution assertion the test exists for is unchanged. Both `--help`/unknown-flag sweeps run from a THROWAWAY cwd: while proving the fix by reverting it, `init --definitely-not-a-flag` ran a real `vigiles init` and wrote 21 specs, a workflow and a config into the repo. The test now cannot do that to whatever directory it runs in. docs/cli.md documents the rejection, the `=`-only value form, and the exit-code table. * fix(report-view): mirror the split inventory fields, and let CI see the project The Evaluated ring added `untestedHarness` and `unevaluated` to the engine's inventory and not to the view's mirror, so `sample.ts` set two fields the type did not have. `tsc -p site/tsconfig.json` failed the whole time. It went unnoticed because CI runs three typecheck steps — root, `test/types`, and generated types — and none of them covers site/ or packages/report-view. Three green checks, none over the project that broke. That is the same shape as every defect in this PR: a check that exists, passes, and does not cover the thing that failed. So both halves. The fields are declared optional, because a report produced before the split carries neither and the view must still render an older artifact. And CI now typechecks that project, so the next field added on one side of the mirror fails where someone will see it instead of waiting for whoever next runs tsc by hand. Found by the agent that fixed #10-#12 while reporting what it had NOT fixed. * feat(audit): say so when a repo's vigiles skills are unreachable vigiles ships six user-facing skills, and `test-harness` alone answers the question the testing docs are otherwise organized around ("which tier do I want?"). They are wired by the GLOBAL plugin install; `npm install vigiles` ALSO drops them at `node_modules/vigiles/skills/`, which Claude Code never scans. A repo that took the dependency but never ran the plugin install has all six present on disk, unselectable, and silent — observed for a full day in a consumer repo whose author was writing harness tests by hand. `vigiles audit` now checks reachability and warns, following the `dialect-drift` shape: pure parsers + a best-effort local read that never throws + a formatter that returns null when there is nothing to say. No new verb, no new flag. Advisory only, never scored: reachability is a property of the machine, not the repo, and a score that differed between a laptop and CI for identical source would be a lie. The subtle part, and the reason this was misdiagnosed as an npm packaging bug: the authoritative record of `claude plugin install` is the global ~/.claude/plugins/installed_plugins.json, NOT the repo's .claude/settings.json (which carries PROJECT-level enabledPlugins, where a correctly-installed user-scope plugin does not appear). Judged from settings.json alone, a working install reads as broken. Verified live: with this box's real HOME the check is silent because vigiles@vigiles genuinely IS installed; with a fresh HOME the same repo warns. Also pins the trap that started this. A hand-rolled execFileSync runner returns stdout ALONE on success, so advisory output — including vigiles's own compiled hook notice() — vanishes and a healthy react hook reports as dead (hit three times in one repo). runHook already carries both streams and is the right tier for a plain helper script; test-harness now says so, including the honest limit that filesWritten is recorded only under confinement, so assertNoWrite passes vacuously on an unconfined run. Three tests hold that contract. Co-Authored-By: Claude <noreply@anthropic.com> * fix(audit): a project plugin declaration is not an install Investigating whether `init` should wire the plugin per-repo turned up the opposite of the premise, and a bug in the check shipped ina138826. Claude Code's team-marketplace docs, as of CC v2.1.195: "A plugin that only the project's .claude/settings.json enables, and that comes from an external source such as a GitHub repository or npm package, doesn't load until the team member installs it." vigiles ships from a GitHub marketplace, so that is our case. Committing extraKnownMarketplaces + enabledPlugins makes Claude Code PROMPT a collaborator to install; it does not install, and a fresh clone or CI job still has no plugin. Confirmed empirically on one machine: a repo whose committed settings.json declares an external plugin project-level had NO marketplace entry, NO cache dir and NO install record for it in ~/.claude/plugins/, while a plugin installed the normal way on that same machine had all three. Network was not the difference — the second install succeeded there, commit SHA recorded. The bug: the check treated a project `enabledPlugins` entry as proof of reachability, so it would fall silent for a repo that declares the plugin and has never installed it — precisely the state it exists to surface, and precisely the state observed. Project declaration is now a THIRD state, `declaredNotInstalled`, which still warns but with the correct fix line: the install is per machine, and the repo cannot perform it on the collaborator's behalf. Four tests were watched failing against the previous implementation before the fix. Advisory-and-unscored stands, and for a stronger reason than before: this is machine state that no repo-committed file can determine, by deliberate security design — plugins execute arbitrary code with user privileges. Co-Authored-By: Claude <noreply@anthropic.com> * feat(testing)!: split runScript out as the primitive under runHook `runHook` already WAS a general program runner — arbitrary shell command, stdin fed, exit code + both streams + writes + egress back — with a thin hook layer on top. The specialized name owned the general machinery, so someone testing an ordinary helper script had no name to look for. One did: he hand-rolled an execFileSync runner and hit the same bug three times, because execFileSync returns stdout ALONE on success while advisory output (including vigiles's own compiled-hook notice()) goes to stderr, so healthy react hooks reported as dead. The layering is now the right way up: runHook = runScript + event-to-stdin + exit-code-to-decision src/run-script.ts owns spawning, confinement, egress and write recording; none of that is about hooks. src/run-hook.ts is 250 lines of hook protocol. These are two questions, not two ways to ask one: a HOOK has a decision, a SCRIPT has effects. So ScriptRunResult carries no `decision` field rather than an always-undefined one, which would teach the reader the field means nothing. A test asserts its absence, so re-adding it fails. Also fixes a vacuous assertion, which is the same defect class this repo exists to catch in other people's harnesses. `filesWritten` is recorded by diffing the work dir, which only a confined run does — so an unconfined run reported `[]` and `assertNoWrite` returned GREEN having inspected nothing. "Nobody looked" and "nothing happened" are now structurally distinct: `undefined` vs `[]`. assertNoWrite/assertWroteOnly throw on the former and name the fix (`sandbox: "auto"`). Documenting this was too weak; the code distinguishes it now instead of the reader having to remember. BREAKING CHANGE: `HookRunResult.filesWritten` is optional (`readonly string[] | undefined`) — `undefined` when writes were never recorded, rather than a misleading `[]`. Code reading it directly needs `?? []`; code using assertNoWrite/assertWroteOnly gets a real assertion instead of a vacuous pass. Everything else is additive or structurally identical: HookRunResult now extends ScriptRunResult with the same members, and RunHookOptions is Omit<RunScriptOptions, "stdin">. runHook's signature and behaviour are unchanged. Watched failing first: 3 vacuity tests (assert.throws saw no throw — the vacuous pass itself), then 3 deliberate breaks — dropping stderr fails 3 tests, re-adding a decision field fails 1, collapsing unrecorded writes to [] fails 2. Co-Authored-By: Claude <noreply@anthropic.com> * feat(init): declare the plugin in the repo, turning silent absence into a prompt `vigiles init` now merges a two-key declaration into the repo's .claude/settings.json: an `extraKnownMarketplaces.vigiles` source and `enabledPlugins["vigiles@vigiles"]`. WHAT THIS BUYS, precisely: a collaborator who clones and never runs `init` currently gets SILENCE — the npm package is in node_modules, its six skills are unreachable, and nothing says so. That silence cost a full day. With the declaration, Claude Code tells them the project wants this plugin and prints the install command. WHAT IT DOES NOT BUY: the plugin. An external-source plugin declared project-level does not load until each person installs it on their own machine (CC docs, v2.1.195+), and the boundary is deliberate — plugins execute arbitrary code with the user's privileges, so a repo may not install one on your behalf. A fresh clone and a CI job still have no plugin. The README, agent-setup docs and the CLI output all say this in those terms; none of them claim it wires anything up. Nothing is vendored: the entry is a reference. Plugin content stays in the global cache, one copy shared across repos. MERGE, never clobber — that file holds the user's hooks, permissions and other plugins. The merge rules are pure and unit-tested: unrelated keys survive byte-identical; an existing `vigiles` marketplace entry is left alone (it may point at a fork); an explicit `false` is respected as a deliberate disable; the input object is never mutated; a second run is a no-op. Verified end to end against a synthetic repo whose settings.json carries hooks, permissions, a $schema and another marketplace — the diff was exactly the two additions. Invalid JSON is a loud skip, never a rewrite. Gated by the existing `--no-plugin`; skipped in the vigiles repo itself. `removeVigilesDeclaration` ships tested alongside it, deliberately not wired to `eject`: `eject [file]` is the per-file inverse of `compile`, and repo-scoped plugin wiring is not a property of any one file. Reasoning and the two-key manual removal are documented. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
81 lines
3.1 KiB
TypeScript
81 lines
3.1 KiB
TypeScript
/**
|
|
* Unit tests for the known-flag registry — the half of the "audit silently
|
|
* swallows unknown flags" defect that can be tested without spawning the CLI.
|
|
* The behavioural half (exit codes, `--help`, the nothing-to-audit outcome)
|
|
* lives in src/scan-cli.test.ts, which drives the real built binary.
|
|
*/
|
|
import { test } from "vitest";
|
|
import assert from "node:assert/strict";
|
|
|
|
import {
|
|
COMMAND_FLAGS,
|
|
formatUnknownFlag,
|
|
knownFlagsFor,
|
|
nearestFlag,
|
|
unknownFlags,
|
|
} from "./cli-flag-check.js";
|
|
import { VERBS } from "./cli-commands.js";
|
|
|
|
test("an unrecognised flag is reported, per verb", () => {
|
|
// The measured reproduction: this ran a complete audit and exited 0.
|
|
assert.deepEqual(unknownFlags("audit", ["--this-flag-does-not-exist"]), [
|
|
"--this-flag-does-not-exist",
|
|
]);
|
|
// …and a typo of a REAL flag, which is the dangerous case: `--no-htlm` used
|
|
// to write the HTML report while the author believed they had suppressed it.
|
|
assert.deepEqual(unknownFlags("audit", ["--no-htlm"]), ["--no-htlm"]);
|
|
});
|
|
|
|
test("every flag a verb actually reads is accepted", () => {
|
|
// Guards the other direction: over-rejecting is worse than the defect, since
|
|
// it breaks working invocations.
|
|
for (const verb of VERBS) {
|
|
for (const spec of COMMAND_FLAGS[verb]) {
|
|
const arg = spec.endsWith("=") ? `${spec}value` : spec;
|
|
assert.deepEqual(
|
|
unknownFlags(verb, [arg]),
|
|
[],
|
|
`${verb} must accept its own flag ${arg}`,
|
|
);
|
|
}
|
|
}
|
|
});
|
|
|
|
test("shared flags are accepted everywhere; positionals are never flags", () => {
|
|
for (const verb of VERBS) {
|
|
assert.deepEqual(unknownFlags(verb, ["--harness=codex", "--help"]), []);
|
|
}
|
|
assert.deepEqual(
|
|
unknownFlags("audit", ["./some/dir", "-", "--", "--wat"]),
|
|
[],
|
|
"a positional, a lone dash, and everything after `--` are not flags",
|
|
);
|
|
});
|
|
|
|
test("hook-runtime is exempt — its argv comes from the harness, not a human", () => {
|
|
assert.deepEqual(unknownFlags("hook-runtime", ["--anything-at-all"]), []);
|
|
});
|
|
|
|
test("a value flag rejects the space-separated form it cannot read", () => {
|
|
// `flagValue()` only ever reads `--out=x`. `--out dir` used to leave `--out`
|
|
// unread and `dir` interpreted as the scan target — the exact shape of the
|
|
// bug that made an audit run over the wrong directory.
|
|
assert.deepEqual(unknownFlags("audit", ["--out", "/tmp/x"]), ["--out"]);
|
|
assert.deepEqual(unknownFlags("audit", ["--out=/tmp/x"]), []);
|
|
});
|
|
|
|
test("the nearest-flag suggestion fires on a typo and stays quiet otherwise", () => {
|
|
const known = knownFlagsFor("audit");
|
|
assert.equal(nearestFlag("--no-htlm", known), "--no-html");
|
|
assert.equal(nearestFlag("--jsonn", known), "--json");
|
|
// Nothing close: a wrong suggestion invites "fixing" a flag never meant.
|
|
assert.equal(nearestFlag("--completely-unrelated-thing", known), undefined);
|
|
});
|
|
|
|
test("the message names the flag, the verb, and where to look", () => {
|
|
const msg = formatUnknownFlag("audit", "--no-htlm");
|
|
assert.ok(msg.includes('unknown flag "--no-htlm"'), msg);
|
|
assert.ok(msg.includes("Did you mean `--no-html`?"), msg);
|
|
assert.ok(msg.includes("vigiles audit --help"), msg);
|
|
});
|