Closes GAP-01, GAP-02, GAP-04 from the ADR-091 completion plan.
- Adds CommandEvalRunner (src/validation/command-eval-runner.ts) as a
sibling to ParallelEvalRunner. Detects shell-based eval suites by
shape (first test case has input.command), runs setup/main commands
sequentially, and evaluates exit_code, json_fields (dotted JSONPath),
severity_at_least (none < low < medium < high < critical), and
candidate_count_at_least assertions against script JSON envelopes.
- Wires aqe eval run to dispatch to CommandEvalRunner when the suite is
command-mode. LLM-prompt suites keep the pre-existing path unchanged.
- Activates .claude/skills/qe-browser/evals/qe-browser.yaml (status:
active, was design-spec) and removes the honesty note now that the
runner supports its structured assertions.
- Pins vibium@^26.3.18 via DEFAULT_VIBIUM_SPEC in browser-engine-
installer.ts so npm install -g can't silently upgrade past a version
we've verified against the 5 primitives + smoke tests.
- Drops qe-browser SKILL.md trust_tier from 3 to 2 pending eval
integration in CI; will re-promote once the full verification pass
runs green.
Tests: 31 new cases (pure helpers, runner integration with injected
runner, happy-path against the shipped qe-browser.yaml).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dragan Spiridonov