* feat!: eval lock (CI staleness gate) + per-adapter harness versionKey Real-model evals run on your subscription, so they only run locally — never in CI. The eval LOCK closes the resulting gap: a committed integrity stamp that lets CI verify the committed numbers still match the current inputs WITHOUT a model call (distinct from the gitignored, local-speed eval cache). - `vigiles eval --update` (local): records each NAMED eval's report to a committed `.vigiles/eval-locks/<name>.lock.json` + prints the per-number delta. - `vigiles eval --check` (CI): recompute the input hash, fail "stale" on a mismatch — binary-free, deterministic. Green no-op until the first lock is committed (smooth adoption). - The clean split: the lock stores only the model's observed behavior; the script's own assertions re-run live against the replayed report, so a threshold-only edit is a valid replay while an input change is stale. - Honest scope: the harness version is provenance, NOT a hash input (CI's pinned claude ≠ a dev's local claude would false-trip), and there is no nightly run — model/harness drift is a local `--update` concern. Also extracts the harness version reduction to a per-adapter port method (`HarnessRuntime.versionKey`): Claude Code keys major.minor (~quarterly), Codex opts out with "" (its minor is patch-cadence, ~weekly). This fixes the local cache churning weekly for Codex and is where the cadence decision belongs. Whole-package coherence: the GHA gains `command: eval-check`, `vigiles init` scaffolds the eval-check CI job, and this repo dogfoods it. Wired at both eval seams (runEvalWith + measureTriggerRateWith), 100%-covered, docs updated (internal research + public guides). BREAKING CHANGE: `HarnessRuntime` (exported on `vigiles/adapter`) gains a required `versionKey(raw: string): string` method. Third-party adapters must add it; return "" to opt out of version partitioning. First-party adapters (claude-code, codex) are updated. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVmD8n7P62fuJ91aQh9yY9 * feat: eval-lock agent awareness (skill + nudge hook) + cohesive-delivery rule Close the loop on the eval lock so the AGENT knows it exists and maintains it — without the invasive option of editing the user's CLAUDE.md. - test-harness skill documents the lock lifecycle (--update/--check, commit the lock, re-run after editing a skill). - A PostToolUse nudge hook (hooks/eval-lock-nudge.sh → hook-runtime eval-lock-nudge): after editing a SKILL.md / *.eval.* with committed locks present, inject a NON-BLOCKING reminder to re-run `vigiles eval --update`. Self-gated (silent until a lock is committed), never blocks (the gate is CI's `eval --check`). Wired into the vigiles plugin; runHook + unit + .sh tests. - NO init edit to the user's CLAUDE.md — verified that's not how plugins deliver context (no plugin-level instructions file exists; the channel is a hook/skill). Docs + dedup: - research/agent-context-delivery.md captures the VERIFIED CC/Codex context-delivery mechanics ONCE (SessionStart additionalContext is the plugin channel; CLAUDE.md is the user's file; skill descriptions always-in-context; Codex AGENTS.md + hooks but inject deferred) so we stop re-researching it. - Per-adapter guides (harness-testing-claude-code/-codex.md) explain how the reminder reaches the agent per harness (CC injects; Codex inject is a documented follow-on), with the agnostic guide linking DOWN to them (no duplication). Cohesive-view rule: - New CLAUDE.md `cohesive-feature-delivery` rule — a definition-of-done CHECKLIST (CLI × GHA × init × agent-awareness × both adapters × per-adapter deduped docs × tests) so a feature's INTEGRATION across surfaces is checked as one thing, generalizing cohesive-cli-surface + harness-parity + great-agent-flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVmD8n7P62fuJ91aQh9yY9 * test: CLI e2e for the eval-lock invalidation flow (update → check → stale) Closes the gap flagged in review: the joined CLI flow wasn't tested as one thing. A real *.eval.cjs fixture (model-free — drives runEvalWith from dist with a fake runner) run through the built CLI proves the whole loop: `eval --update` writes a committed lock, `eval --check` with the same inputs replays green (no model), and an input change makes `eval --check` go red ("stale"). Binary-free, no subscription — exactly how it runs in CI. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVmD8n7P62fuJ91aQh9yY9 * docs: scannable eval-lock docs + prose-clarity & easy-adoption rules Make the eval-lock docs readable at a glance and frame upkeep as agent-driven, not a manual chore. Docs (scannable rewrite — ASCII flow diagram, comparison tables, ✅/❌/⚠️ signposts, short sentences, no walls of text): - docs/harness-testing.md — the lock how-to now leads with "you mostly won't touch this by hand; the test-harness skill records it + a hook reminds you," then a diagram + tables. - docs/harness-testing-claude-code.md / -codex.md — the per-harness nudge delivery as tables; the Codex inject caveat is one clear ⚠️ row. - docs/eval-architecture.md — the "what's stale vs a free replay" split as ✅/❌ bullets. - skills/test-harness/SKILL.md — Step 5 now tells the agent to run `eval --update` + commit the lock as part of its job (so it IS done for you). - README — the Quick-start reassurance leads with "Adoption is smooth: one command, then your agent does the rest," names skills AND hooks, no chores. Two new CLAUDE.md rules: - prose-clarity — public prose must be clear + scannable (lead with the point, one idea per sentence, define/drop jargon, tables/bullets/ASCII/Mermaid over walls, emoji as signposts, one bold per point). Prefer ASCII/Mermaid; never commit generated raster images into docs. - lead-with-easy-adoption — docs (README first screen especially) must make it clear up front that adoption is low-friction because skills + hooks do the work; frame every feature's upkeep as agent/skill/hook-driven, not homework. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVmD8n7P62fuJ91aQh9yY9 * docs: prose-clarity sweep across 13 user-facing guides Apply the new prose-clarity rule to the prose-heavy public guides: lead with the point, one idea per sentence, tables/bullets/ASCII over walls of text, ✅/❌/⚠️/ℹ️ signposts, one bold phrase per point, jargon glossed or dropped. Net −578 lines — shorter and far more scannable, with meaning unchanged. Swept: for-plugin-authors, what-vigiles-catches, measuring-skills, skills, harnesses, comparison, faq, sandboxing, safety, agent-setup, agent-workflows, verifying-instruction-files, compiled-hooks. The onboarding pair leads with the low-friction "one command, the agent does the rest" framing. Reference docs (cli, testing-api, spec-format, …) and the per-rule docs were deliberately left alone — they're already table-shaped, and churning them is high-risk / low-value. Verified: no heading text changed (anchors intact), no `research/` links added to public docs, the validation-rules matrix is complete (28 rule links → 28), safety claims retained, command refs resolve (self-command-refs), no orphan docs, fmt clean. Produced by parallel rewrite agents under strict preserve-anchors/links/ code constraints, then verified + spot-read. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVmD8n7P62fuJ91aQh9yY9 * feat!: encode hook inject support into HookProtocol.injectableEvents Codex inject was documented as "deferred" but the official Codex hooks docs confirm `additionalContext` is honored on SessionStart / UserPromptSubmit / PreToolUse / PostToolUse / SubagentStart — the same shape as Claude Code. The gap existed because inject support was a prose deferral, not a tested contract: nothing in the adapter conformance kit asserted a shellHooks harness could actually deliver an inject hook. Encode it as a required port field so the gap can't recur silently: - HookProtocol gains `injectableEvents: readonly string[]` — the events that honor `additionalContext` injection, per-adapter. - claudeCodeHookProtocol: [SessionStart, UserPromptSubmit, PostToolUse]; codexHookProtocol: + [PreToolUse, SubagentStart]. - Conformance rejects a shellHooks adapter with an empty list; the adapter-contract suite asserts every shellHooks adapter declares the events vigiles' shipped hooks use (PostToolUse, SessionStart) — the test that would have caught the original miss. - `compile` warns on an inject hook only when its event isn't injectable on the target harness (react output stays CC-confirmed-only). - Correct every "Codex inject deferred" reference across docs, research, and CLAUDE.md; document injectableEvents in the adapter API + authoring guides. BREAKING CHANGE: HookProtocol (exported on vigiles/adapter) gains a required `injectableEvents` field — a third-party adapter must declare it. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: refresh HANDOFF for the eval-lock branch + inject-port encoding Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: fan out hook install + the CI workflow to every declared harness A repo can enable BOTH harnesses (`harness: ["claude-code","codex"]`), and instruction files already mirror to both — but `compile` installed a hook into only ONE harness (`installHooks` ignored `config.harness` and auto-detected a single adapter), and the scaffolded CI workflow hardcoded the Claude Code binary. So an inject/gate hook that works on both could land in `.claude/settings.json` while silently missing from `.codex/config.toml` — the "works on CC, absent on Codex" failure the harness-parity rule exists to prevent. - `resolveHarnessAdapters` (adapter-registry.ts) returns the FULL declared set: `--harness=` flag → that one; else config `harness` list → all of them; else auto-detect → the detected one (de-duped by name). - `installHooks` loops over it, merging the SAME compiled hook into every enabled harness's native config (JSON / TOML), with per-harness warnings. - The `init` CI workflow is harness-aware: the deterministic harness job installs `@anthropic-ai/claude-code` and/or `@openai/codex` per declared harness (`harnessTestBinaries`). Skills compile byte-identical across harnesses and subagents are CC-only by design, so hooks + the CI binary were the real fan-out gaps. Tests: resolveHarnessAdapters units (fan-out / flag-singular / detect / alias-dedup), a both-harness hook-install e2e (the same hook in both configs), and harness-aware workflow e2es (codex-only, both). Documented as §4 in research/multi-harness-compile.md + the compiled-hooks guide. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * feat: wire vigiles's nudge hooks into .codex/config.toml during init Closes the last Codex-parity gap in agent-awareness: Claude Code gets the eval-lock + refs nudge hooks from the global marketplace plugin, but Codex has no global plugin store, so it got skills only — the proactive nudges never reached a Codex user (the "works on CC, silently absent on Codex" failure). Codex config is repo-committed (the idiomatic place), so `init` now wires the two PostToolUse nudges into `.codex/config.toml`: - `codexPluginHooks` (pure) declares them as DIRECT `npx vigiles hook-runtime …` commands — no plugin root, no vendored bash script. The runtime entrypoints read the event JSON on stdin and emit the `additionalContext` shape Codex honors on PostToolUse (confirmed + encoded in HookProtocol.injectableEvents). - `applyCodexPluginHooks` (pure, unit-tested) merges them idempotently: re-runs replace in place, and the user's own Codex hooks + every other config key are preserved. `wireCodexHooks` in cli.ts does only the IO. - Safety: only an INTENTIONAL exit 2 blocks an edit (the refs nudge under `unmarked-refs: error`); an npx-resolution failure exits non-2, so a missing dep never blocks. Loud, documented deferral (no-silent-skips): the SessionStart lint summary (plain stdout, unconfirmed on Codex) and the compile-on-edit / pre-edit guards (filename-gated bash, no harness-neutral entrypoint yet) stay manual on Codex. Also: the remaining pre-release Codex item — running the eval tier against the real `codex` binary once — is added to the launch-readiness P0 list in the roadmap. Tests: codexPluginHooks shape + applyCodexPluginHooks (fresh / idempotent / preserves user config); the planPluginInstall codex notes/vendors updated. Verified end-to-end that `init --harness=codex` writes the config and a pre-existing `.codex/config.toml` (model + a user hook) is preserved. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: update codex install e2e for the new vendors:true (repo hooks) The codex install plan now writes nudge hooks into .codex/config.toml, so `vendors` flipped to true. Updated the unit assertion last commit but missed this e2e (run by both the `e2e` and `test` CI jobs). The skills-are-global checks below are unchanged and still stand. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: cover the non-Actions lock message branch + address Codex review The `test` job's coverage gate failed: `emitLockMessage`'s console.warn/ console.log branches (the non-GITHUB_ACTIONS path) are unreachable in CI, where GITHUB_ACTIONS is globally set. Added a wiring test that DELETES the var to exercise both branches regardless of environment. Also addresses two Codex review findings on the eval-lock work: - P1 (soundness): `evalArmsInputs` omitted `spec.stubs` from the hash, though tool stubs are written onto PATH and are model-facing input — so a changed canned CLI output after `--update` left the same `inputsHash` and `--check` replayed stale results. Now folds a canonical (name-sorted) view of the stubs into the hash; a new wiring test proves a stub change → stale. - P2 (collision): two eval names normalizing to the same slug (`foo/bar` vs `foo bar` → `foo-bar.lock.json`) could let `--check` replay the WRONG eval's report. `readLock` now rejects a lock whose stored `name` differs from the requested one, degrading a collision to a safe miss (→ stale/re-run), never wrong data. Unit-tested. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: two more eval-lock soundness gaps from Codex review - An unnamed eval in `--check` mode called `produce()` (ran the model), breaking the no-model-in-CI contract: in CI there's no subscription, so it would hit the model or fail for missing auth. Now `check` mode throws a clear, actionable error for an unnamed eval (never reaching the model); `off`/`update` keep producing (the eval just isn't gated locally). - `spec.ephemeralEnv` toggles the trial's run environment (scrubbed env + throwaway HOME vs the inherited process env) — a model-facing input that was omitted from the lock hash, so a report recorded under one mode could be replayed under the other with the same inputsHash. Now folded into the hashed inputs. Both proven by new wiring tests (unnamed-check throws without calling the runner; an ephemeralEnv toggle → stale). Coverage holds at 100% lines/ statements/functions (verified with GITHUB_ACTIONS=true). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: fold the harness identity into the trigger-rate lock hash Codex review (P2): a trigger-rate eval recorded with one harness driver and later switched via `measureTriggerRate(spec, { evalDriver: codexEvalDriver })` hashed identically (prompts/model/plugin unchanged), so `--check` could replay a Claude-recorded report for a Codex run — defeating the gate for the Codex eval path this PR adds. `EvalDriver` gains an optional `harness` identity (defaults to "claude-code", so existing single-harness locks are unaffected); `claudeEvalDriver` and `codexEvalDriver` declare theirs, and `measureTriggerRate`/`...With` thread it into the hashed trigger inputs. A new wiring test proves a harness switch → stale. API surface regenerated (additive `harness?` on EvalDriver). This closes the input-hash bug class on the trigger-rate seam (the runEval seam's stubs + ephemeralEnv were fixed in the prior two commits). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs: refresh HANDOFF — PR #53 merge loop + the 5 review fixes Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: three more Codex-review findings (matchers + plugin-root path hash) - Codex nudge hooks used Claude tool names (`Edit|Write`) in their matcher, but Codex's edit tool is `apply_patch` — so the eval-lock + refs nudges wired into `.codex/config.toml` would never fire. Matcher → `^apply_patch$`. - The compiled-hook FAN-OUT to Codex regex-wraps the raw matcher without translating tool names, so a CC-authored `tools("Edit","Write")` gate emits `^(Edit|Write)$` into Codex config and silently never fires though compile reports success. vigiles has no cross-dialect tool-name map yet, so install now WARNS LOUDLY when a tool-matched hook is fanned to a non-CC harness (no silent non-firing success); a real mapping is the follow-up. - For eval arms using `plugin`, `resolveHarness` expands `${PLUGIN_ROOT}` to the checkout's ABSOLUTE path before the settings are hashed, so a lock recorded at `/home/dev/...` was falsely STALE when `--check` recomputed it at `/home/runner/...` in CI. `stripPluginRoot` normalizes the prefix back to a token → location-independent hash. New wiring test: the same plugin at two different abs paths replays (not stale). Coverage holds at 100% lines/statements/functions (verified GITHUB_ACTIONS=true). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix: guard stripPluginRoot against a hookless plugin's undefined settings A plugin that ships only instructions/skills (no hooks) resolves to settings:undefined; JSON.stringify(undefined) is undefined, so the prior .split() threw before the eval could run. Guard the non-string case (pass it through). Regression I introduced in the plugin-root normalization last commit; new test covers a hookless-plugin arm (update + check, no throw). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test: add a lock-hash COMPLETENESS guard for both seams The Codex review kept finding the same class of bug: a model-facing input omitted from the lock hash, so `--check` replays stale results (stubs, ephemeralEnv, driver-harness, plugin-root all slipped through). Our tests proved the MECHANISM (record→replay, task-change→stale) but never asserted the input set was COMPLETE. This adds a structural guard: for each input that steers the run, mutate ONLY it and assert the lock inputsHash CHANGES — one parametrized test per seam (runEval's evalArmsInputs + trigger-rate's triggerInputs). A new input added to EvalSpec/TriggerRateSpec without hashing it now fails a row HERE in CI, instead of a reviewer finding it later. Covers task/model/tools/fixture/ arm.files/arm.settings/arm.model/interceptTools/stubs/ephemeralEnv and prompts/irrelevantPrompts/model/harness. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
16 KiB
vigiles/adapter — API reference
The harness-adapter authoring API. Everything an adapter author needs — yours or
ours — in one import. This is the reference; for a guided walkthrough see
docs/authoring-an-adapter.md.
import {
// port interfaces (types)
type HarnessAdapter,
type HarnessDialect,
type PluginLayout,
type HarnessRuntime,
type HookProtocol,
type ModelMock,
// conformance
checkAdapterConformance,
assertAdapterConformance,
assertAdapterLoadsHooks,
type ConformanceResult,
// registry / detection
ADAPTERS,
defaultAdapter,
detectAdapter,
detectAdapterResult,
resolveAdapter,
getAdapter,
type DetectResult,
} from "vigiles/adapter";
Stability. The five port interfaces and HarnessAdapter are the stable
contract — changes to them are semver-major. The conformance and registry
functions are stable. The behaviour behind the descriptors (the renderers, the
mock HTTP server) is internal and may change between minors.
Port interfaces
A harness differs from Claude Code along two axes — format (what it reads and
writes) and transport (how it runs). Each port is a plain, fully-readonly
value object.
HarnessDialect (format)
The harness vocabulary the compiler verifies against.
| Field | Type | Meaning | Claude Code | Codex |
|---|---|---|---|---|
name |
string |
stable id | "claude-code" |
"codex" |
builtinAgentTools |
readonly string[] |
tools a subagent may list in its contract | ["Read","Write","Edit","Bash",…] |
["shell","apply_patch","update_plan","web_search"] |
neverAvailableTools |
readonly string[] |
tools the platform never exposes (a listed one is dead) | ["Agent","ExitPlanMode",…] |
[…] |
mcpToolPattern |
RegExp |
matches an MCP tool ref | /^mcp__[a-z0-9_-]+__[a-z0-9_-]+$/i |
same |
hookEvents |
readonly string[] |
hook event names the harness fires | ["PreToolUse","PostToolUse",…] |
[…,"PermissionRequest","SubagentStart"] |
instructionTargets |
readonly string[] |
instruction-file targets (also the h1) | ["CLAUDE.md","AGENTS.md"] |
["AGENTS.md"] |
pluginRootToken |
string |
env token expanded to the plugin root | "${CLAUDE_PLUGIN_ROOT}" |
"${PLUGIN_ROOT}" |
Consumed by: compileAgent(spec, { dialect }) — verifies the subagent
tool-contract against builtinAgentTools/neverAvailableTools/mcpToolPattern.
PluginLayout (format)
Where the harness keeps things on disk.
| Field | Type | Meaning | Claude Code |
|---|---|---|---|
name |
string |
stable id | "claude-code" |
manifestPath |
string |
plugin manifest | ".claude-plugin/plugin.json" |
hooksConventionPath |
string |
standalone hooks file convention | "hooks/hooks.json" |
settingsPath |
string |
repo settings carrying hooks | ".claude/settings.json" |
settingsFormat |
"json" | "toml" |
how the settings file is encoded | "json" (Codex: "toml") |
instructionFile |
string |
top-level instruction file | "CLAUDE.md" |
surfaceDirs |
readonly string[] |
surface dirs materialized into the sandbox | ["skills","agents","commands"] |
skillDir |
string |
dir holding <dir>/<name>/SKILL.md |
"skills" |
agentDir |
string |
subagent dir ("" = none); drives the subagent rules |
"agents" (OpenCode: ".opencode/agent") |
commandDir |
string |
slash-command dir (<dir>/<name>.md) |
"commands" (Codex: "prompts") |
materializeRoot |
string |
dir surfaces are materialized under | ".claude" |
pluginRootToken |
string |
the plugin-root token (must match the dialect's) | "${CLAUDE_PLUGIN_ROOT}" |
mcpConfigFile |
string |
standalone MCP config | ".mcp.json" |
mcpManifestKey |
string |
manifest key declaring MCP servers | "mcpServers" |
intraRefDirs |
readonly string[] |
dirs scanned for dangling intra-plugin refs | ["hooks","skills","agents","commands"] |
Consumed by: loadPlugin(path, layout) — reads hooks (JSON or TOML per
settingsFormat), materializes surfaces, expands pluginRootToken.
HarnessRuntime (transport)
How the test tiers drive the agent against a no-key mock.
| Field | Type | Meaning | Claude Code |
|---|---|---|---|
name |
string |
stable id | "claude-code" |
agentBinary |
string |
the CLI that runs the agent | "claude" |
modelBaseUrlEnv |
string |
env var pointing at the mock | "ANTHROPIC_BASE_URL" |
modelApiKeyEnv |
string |
env var carrying the (dummy) key | "ANTHROPIC_API_KEY" |
mockApiKey |
string |
a dummy key the mock ignores | "sk-vigiles-mock" |
HookProtocol (transport)
How a hook signals a block/deny, and which events can inject developer context.
| Field | Type | Meaning | Claude Code | Codex |
|---|---|---|---|---|
name |
string |
stable id | "claude-code" |
"codex" |
blockExitCode |
number |
exit code that blocks a tool call | 2 |
2 |
denyDecisionValues |
readonly string[] |
decision values meaning "deny" | ["block","deny"] |
["block","deny"] |
eventEnvVars |
readonly string[] |
env vars a synthesized event carries | [] |
[] |
injectableEvents |
readonly string[] |
events that honor additionalContext (inject/nudge hooks) |
["SessionStart","UserPromptSubmit","PostToolUse"] |
["SessionStart","UserPromptSubmit","PreToolUse","PostToolUse","SubagentStart"] |
injectableEvents makes "can this harness deliver an inject hook, and on which
events?" a tested contract rather than prose — a shellHooks adapter that
declares an empty list fails conformance, and compile warns when an inject hook
targets an event the harness doesn't honor. A code-module-hook adapter
(shellHooks:false, no hookProtocol) is exempt.
ModelMock (transport)
The mock model's wire format. (The HTTP/SSE renderer lives in the harness's mock module; this descriptor names the wire facts the bundle and tooling read.)
| Field | Type | Meaning | Claude Code | Codex |
|---|---|---|---|---|
name |
string |
stable id | "claude-code" |
"codex" |
wireApi |
string |
the model API wire format | "anthropic-messages" |
"openai-responses" |
modelEndpoint |
string |
URL substring of a turn-consuming request | "/v1/messages" |
"/v1/responses" |
countTokensEndpoint? |
string |
token-count endpoint a client probes | "count_tokens" |
(omit) |
HarnessAdapter
The bundle. name + a capabilities descriptor + the ports + a detect. The
two layer-1 ports (dialect, layout) are always required; the transport ports
are optional and gated by capabilities — present iff the matching capability
is declared, so a reference-only or code-module-hook harness isn't forced to ship
a fake transport (the conformance kit enforces this both ways).
interface AdapterCapabilities {
readonly referenceVerification: true; // layer 1 — always
readonly harnessTesting: boolean; // layer 2 — needs runtime + modelMock
readonly shellHooks: boolean; // shell-process hooks — needs hookProtocol
readonly subagents: boolean; // has subagents — gates the subagent lint rules
}
interface HarnessAdapter {
readonly name: string;
readonly capabilities: AdapterCapabilities;
readonly dialect: HarnessDialect; // always
readonly layout: PluginLayout; // always
readonly runtime?: HarnessRuntime; // iff capabilities.harnessTesting
readonly hookProtocol?: HookProtocol; // iff capabilities.shellHooks
readonly modelMock?: ModelMock; // iff capabilities.harnessTesting
/** Specificity score: 0 = not this harness; higher = a more specific match. */
detect(root: string): number;
}
assertHarnessTestable(adapter) is the guard the layer-2 runners call to refuse
a non-harnessTesting adapter up front (returning its narrowed runtime+modelMock).
detect contract. Return 0 when root is not this harness; otherwise a
positive specificity — a strong signal (a private config dir / manifest)
should outscore a weak one (a shared AGENTS.md). The registry picks the highest
scorer, so scores let a precise adapter win over a generic one regardless of
registration order.
Conformance
checkAdapterConformance(adapter): ConformanceResult
Pure (no IO). Returns { ok: boolean; failures: readonly string[] }. Checks:
every port populated; cross-port invariants (all port names equal
adapter.name; layout.instructionFile ∈ dialect.instructionTargets;
layout.pluginRootToken === dialect.pluginRootToken; settingsFormat is
"json"|"toml"); and a behavioural one — the dialect's own first built-in tool
passes compileAgent's tool-contract check.
assertAdapterConformance(adapter): void
Throws an Error listing every failure if checkAdapterConformance isn't ok.
Drop it in your adapter's test suite.
assertAdapterLoadsHooks(adapter): void
Does filesystem IO. Writes a minimal settings file in the adapter's
settingsFormat (with a hook) to a temp dir, loads it through the adapter's
layout, and throws if no hooks came back. This is what catches a layout that
points at the right file in the wrong format (the JSON-vs-TOML trap) — the
pure check passes it, but the agent would silently run zero hooks.
import { test } from "vitest";
import {
assertAdapterConformance,
assertAdapterLoadsHooks,
} from "vigiles/adapter";
import { myHarnessAdapter } from "./my-harness-adapter.js";
test("my adapter conforms", () => {
assertAdapterConformance(myHarnessAdapter);
assertAdapterLoadsHooks(myHarnessAdapter);
});
Registry & detection
The registry is how the CLI picks an adapter for a repo (the library picks by import). Claude Code is the default, so detection is backwards-compatible.
| Export | Signature | Behaviour |
|---|---|---|
ADAPTERS |
readonly HarnessAdapter[] |
all registered adapters |
defaultAdapter |
HarnessAdapter |
used when nothing is detected (Claude Code) |
detectAdapter(root) |
(string) => HarnessAdapter |
the highest-specificity match, else the default |
detectAdapterResult(root) |
(string) => DetectResult |
{ adapter, fallback, ambiguousWith } — ambiguousWith lists other top-scoring adapters (a repo that looks like several) |
resolveAdapter(root, harness?) |
(string, string?) => HarnessAdapter |
an explicit harness name wins (throws if unknown); else detectAdapter. The CLI's --harness flow |
getAdapter(name) |
(string) => HarnessAdapter | undefined |
lookup by name |
interface DetectResult {
readonly adapter: HarnessAdapter;
readonly fallback: boolean; // true = no markers, fell back to default
readonly ambiguousWith: readonly string[]; // other top-scoring adapter names
}
Applying an adapter (the programmatic path)
vigiles/adapter gives you the contract; you apply it with the verification
core. This path is fully supported for third-party adapters today:
import { compileAgent } from "vigiles/linting";
import { loadPlugin } from "vigiles/claude-code";
import { myHarnessAdapter } from "./my-harness-adapter.js";
// verify a subagent's tool contract under your dialect
const { markdown, errors } = compileAgent(spec, {
dialect: myHarnessAdapter.dialect,
specFile: "reviewer.md.spec.ts",
});
// load a repo/plugin under your layout (hooks parsed per settingsFormat)
const plugin = loadPlugin("./my-project", myHarnessAdapter.layout);
Third-party adapters
- Programmatic use — supported now. Implement the ports, validate with the
conformance kit, and pass
adapter.dialect/adapter.layoutto the core functions above. Nothing is gated on the adapter living in our tree. - CLI auto-detection — partial.
ADAPTERSis vigiles's internal registry; thevigilesCLI binary only auto-detects adapters compiled into it. A config- based mechanism for the CLI to load an external adapter package is planned Until then: use the programmatic path, or contribute the adapter upstream so it joinsADAPTERS.
See also
docs/authoring-an-adapter.md— the guided tutorial.docs/harnesses.md— how a consumer selects a harness.