Promotion gate documented in ADR-091 (first green CI run of
test-qe-browser.yml against real Vibium) cleared at commit
fae9fd30 / workflow run 24776640163 on 2026-04-22:
- Unit tests (CommandEvalRunner + primitives): success
- Smoke test (real Vibium + httpbin): success
- aqe eval run against real Vibium: success
Changes:
- .claude/skills/qe-browser/SKILL.md: trust_tier 2 -> 3
- ADR-091 status: In Progress -> Implemented, gate marked cleared
- v3-adrs.md index row: In Progress -> Implemented, counts
updated 79->80 Implemented, 7->6 In Progress
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The smoke job is green (--no-sandbox wrapper works), but the eval
job failed because `aqe eval run` dispatches the yaml's setup
steps verbatim, and those steps used `vibium go URL` without the
global `--headless` flag. On a CI runner with no display that
tries to open a visible Chrome window and fails.
Six setup steps were affected (tc001, tc002, tc003, tc008, tc010,
tc011 equivalents). Tc006/tc007 already had --headless from commit
23416e15. Smoke-test.sh was unaffected because it prefixes
`vibium --headless` for all its invocations.
No logic changes; aligns yaml with smoke-test.sh convention.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Diagnostic output from commit a4cb96c5 confirmed the residual error
is chromedriver's classic "HTTP 500 create session" — Chrome's
setuid sandbox can't initialize in the GitHub Actions runner
environment (no user namespaces, restricted proc).
Fix: after `vibium install` lays down Chrome for Testing into
~/.cache/vibium/chrome-for-testing/<version>/chrome, replace the
binary with a tiny bash wrapper that execs the real Chrome with
--no-sandbox --disable-dev-shm-usage appended. The wrapper lives
inside the ephemeral runner cache; nothing is shipped to users.
This is the standard workaround used by every Chromium-based CI
runner (Puppeteer, Playwright, chromedp all do the equivalent).
Also removed the diagnostic step added in a4cb96c5 (job green is
the signal now).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause confirmed via diagnostic (commit 0f810e3f): Vibium's
auto-launch fails with "failed to create session: HTTP 500"
because the npm package ships only the Go binary — the actual
browser and chromedriver are downloaded on demand by
`vibium install`. The earlier VIBIUM_BROWSER_PATH fix was a red
herring: Vibium drives its own bundled Chrome for Testing over
WebDriver, not the runner's system google-chrome.
Reverted:
- "Discover browser path for Vibium" step (both jobs)
- "DIAGNOSTIC — probe Vibium failure" step (smoke job)
Added:
- "Download Chrome for Testing + chromedriver" step running
`vibium install` after the npm install, in both smoke and eval
jobs. This matches the documented setup flow for Vibium.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
TEMPORARY diagnostic step to surface the real error Vibium's
wrapper is hiding behind "stdout: null, stderr: null". Runs a
battery of probes with continue-on-error so the step never blocks
the smoke job itself, and prints enough to diagnose whether the
Go binary is crashing on sandbox, /dev/shm, missing lib, or
something else.
To be reverted in the next commit once the root cause is known.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Root cause of PR #431 smoke/eval job failures: Vibium installs
correctly (reports v26.3.18) but exits 1 when it tries to drive a
browser, because the runner has no Chrome on its default PATH for
Vibium's discovery — smoke-test tc002/tc004/tc006-8/tc010 all crash
with `vibium eval exited 1` in child_process, while tests that
don't need a real navigation (tc003, tc005, tc011) pass.
Fix: added a preflight step in both `smoke` and `eval` jobs that
probes the same prioritized list the installer uses
(google-chrome → google-chrome-stable → chromium-browser → chromium),
exports VIBIUM_BROWSER_PATH via $GITHUB_ENV, and fails loudly if
none is found. Ubuntu-24.04 runners have google-chrome preinstalled,
so this resolves without adding any third-party action.
This mirrors the remediation that `aqe init --auto` prints on Linux
ARM64 hosts — the mechanism is identical, just applied on AMD64 CI
where Vibium's auto-detection also fails.
Validated with actionlint 1.7.1 (clean).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Resolves 3 CodeQL findings (205/206/207) from github-advanced-security
on PR #431: "Workflow does not contain permissions".
Added workflow-level `permissions: contents: read`. The workflow only
reads the repo and uploads an artifact — no writes to code, issues,
PRs, or releases. The artifact upload uses the ephemeral job token,
not the `actions: write` permission.
Validated with actionlint 1.7.1 (clean).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-091 (qe-browser) promoted to Implemented, trust tier 3:
- new aqe eval run --skill <skill> CLI executes skill eval YAMLs as real CommandRunner evals
- aqe init --auto on Linux ARM64 now prints an exact VIBIUM_BROWSER_PATH hint
- docs/guides/qe-browser-getting-started.md for new users
- .github/workflows/test-qe-browser.yml gates PRs with vitest + smoke + eval jobs
- qe-browser eval YAML reworked to use httpbin.org (no fixture server needed)
- Vibium pin comment corrected: ^26.3.18 is a major-line pin, not major.minor
No breaking changes. CLI surface additions only.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Single commit covering the 5 concrete findings from the qe-devils-advocate
review. Does not change the underlying CommandEvalRunner design or the 5
primitives — only the claims around them and the fixture-server hack the
yaml had silently inherited.
1. Revert promotion (G4): trust_tier 3 -> 2 in SKILL.md; ADR-091 status
Implemented -> In Progress (2026-04-21); v3-adrs.md index row updated
to reflect the same; counts 80->79 Implemented, 6->7 In Progress.
The promotion gate is now documented on ADR-091: first green CI run
of test-qe-browser.yml (unit + smoke + eval against real Vibium)
before trust_tier 3 / Implemented are restored.
2. Fix yaml fixtures (G3): tc006/tc007 were referencing
http://localhost:8088/qe-browser/SKILL.md.html which required an
unstarted fixture server, so `aqe eval run --skill qe-browser` would
have failed on a fresh machine. Rewrote both to use https://httpbin.org/html
(matching scripts/smoke-test.sh tc006/tc007). Dropped tc009
(poisoned-page check-injection) from the yaml — it needs the local
serve-skills.js fixture which is out of scope for CommandEvalRunner.
Documented as an explicit GAP comment in the yaml, noted that
check-injection.js's severity path is unit-tested at
tests/unit/scripts/qe-browser-check-injection.test.ts. yaml test_cases
now 11 -> 10.
3. Correct pin comment wording (G1): the previous comment claimed
^26.3.18 was a "major.minor-line pin" — that's factually wrong under
npm semver (^26.3.18 accepts 26.99.x). Rewritten to accurately
describe the intent: major-line pin that blocks 27.0+ while
allowing auto-uptake of 26.x security patches and additive features,
with scripts/smoke-test.sh as the belt-and-suspenders API contract
gate. The spec itself stays ^26.3.18 — the correct choice for an
end-user tool under semver assumptions.
4. Add eval CI job (G2): third job in test-qe-browser.yml runs
`aqe eval run --skill qe-browser --model claude-3.5-sonnet` against
a real Vibium install after `npm run build`. This is the
end-to-end proof that CommandEvalRunner dispatch works that the
previous unit tests (all mocked CommandRunner) could not provide.
Uploads the eval JSON as a CI artifact for post-run inspection.
actionlint clean.
5. Scrub stray files (G5): git rm the coherence-comparison artifacts
from docs/reports/ that were swept in by a prior `git add -A`, and
added a focused pattern to .gitignore (not a blanket docs/reports/
ignore) so future sessions can't repeat the mistake while intentional
reports can still be committed.
Tests: 51 relevant tests still pass (3 files: command-eval-runner,
command-eval-runner-qe-browser, browser-engine-installer).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes GAP-05, GAP-08, and final verification gate.
- Linux ARM64 diagnostic (GAP-05): browser-engine-installer now probes
os.platform + os.arch on Linux aarch64 and checks a prioritized list
of system chromium locations (/usr/bin/chromium, chromium-browser,
google-chrome, google-chrome-stable, /snap/bin/chromium). When a
browser is found, the install result carries a PlatformHint with an
exact VIBIUM_BROWSER_PATH export the user can copy. When none exists,
the hint surfaces the `apt-get install chromium` remediation. Wired
into phase 09 so it prints after vibium install succeeds. 5 new tests.
- GAP-08: added explicit `vibium --headless viewport 1280 720` setup
steps to yaml tc006/tc007 so the eval runs are as deterministic in CI
as smoke-test.sh. Updated smoke-test.sh header to note the yaml is
the canonical spec and the bash script is its mirror (covers the F1
vibium-missing contract case the yaml can't express cleanly).
- Trust tier promoted 2 -> 3 now that (a) the eval is runnable via
CommandEvalRunner, (b) CI runs both unit tests + smoke gate, and
(c) all 11 dependent browser-using skills either reference qe-browser
or document their intentional Playwright fallback.
- ADR-091 status: Proposed -> Implemented (2026-04-21).
- v3-adrs.md index reflects 80 Implemented / 6 In Progress / 3 Proposed.
Verification: 643/643 tests pass across tests/unit/validation/,
tests/unit/init/, tests/unit/scripts/qe-browser-*.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes GAP-03a, GAP-03b, GAP-06, GAP-07, and revises GAP-09.
- Rephrased Playwright references in enterprise-integration-testing and
pentest-validation to point at qe-browser (batch/assert/check-injection)
so the AQE fleet's canonical browser runner is the recommended tool.
- accessibility-testing: added qe-browser / a11y-ally as the preferred
axe-core integration path ahead of the Playwright example; Playwright
kept as a documented fallback for Firefox/Safari or existing suites.
- GAP-09 revised: visual-testing-advanced and compatibility-testing are
intentional hybrids, not unfinished migrations. visual-testing-advanced
already labels its Playwright section "LEGACY: fallback"; compatibility
-testing notes that Vibium's BiDi backend is Chrome-only at v26.3.x so
Firefox/Safari coverage still needs Playwright. Both stay as-is.
- Added docs/guides/qe-browser-getting-started.md covering install,
smoke check, eval run, and the 5 primitives with example CLI calls.
- Added .github/workflows/test-qe-browser.yml gating PRs that touch the
skill, its installer, or CommandEvalRunner. Job 1 runs the full vitest
sweep; Job 2 installs a pinned Vibium and runs smoke-test.sh against
real httpbin fixtures.
- Workflow validated with actionlint 1.7.1 (clean).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes GAP-01, GAP-02, GAP-04 from the ADR-091 completion plan.
- Adds CommandEvalRunner (src/validation/command-eval-runner.ts) as a
sibling to ParallelEvalRunner. Detects shell-based eval suites by
shape (first test case has input.command), runs setup/main commands
sequentially, and evaluates exit_code, json_fields (dotted JSONPath),
severity_at_least (none < low < medium < high < critical), and
candidate_count_at_least assertions against script JSON envelopes.
- Wires aqe eval run to dispatch to CommandEvalRunner when the suite is
command-mode. LLM-prompt suites keep the pre-existing path unchanged.
- Activates .claude/skills/qe-browser/evals/qe-browser.yaml (status:
active, was design-spec) and removes the honesty note now that the
runner supports its structured assertions.
- Pins vibium@^26.3.18 via DEFAULT_VIBIUM_SPEC in browser-engine-
installer.ts so npm install -g can't silently upgrade past a version
we've verified against the 5 primitives + smoke tests.
- Drops qe-browser SKILL.md trust_tier from 3 to 2 pending eval
integration in CI; will re-promote once the full verification pass
runs green.
Tests: 31 new cases (pure helpers, runner integration with injected
runner, happy-path against the shipped qe-browser.yaml).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Previous regen (commit 80c8186a) still failed CI `npm ci` with the same
EUSAGE / Missing packages error. Root cause: the optional dependency
chain @claude-flow/browser + @claude-flow/codex transitively requires
@claude-flow/cli, @ruvector/*, agentdb, agentic-flow, and a wide set of
transitive packages (~880 entries). My earlier local `npm ci
--ignore-scripts` and `npm install` sequence produced a lockfile that
was missing those entries — likely because a mid-sequence step resolved
optional packages via a platform filter that excluded what CI actually
needs.
This commit: clean `rm -rf node_modules package-lock.json && npm
install` followed by strict `npm ci` verification (no --ignore-scripts).
Result is a 17874-line lockfile (up from 15239) with all transitive
deps properly enumerated, and `npm ci` exits 0 cleanly from an empty
node_modules.
Verified:
- Fresh `rm -rf node_modules package-lock.json && npm install` → 1117
packages resolved, lockfile stable at 17874 lines.
- `rm -rf node_modules && npm ci` → exit 0, no EUSAGE, no Missing errors.
- `node_modules/@claude-flow/cli` now present in lockfile as required by
@claude-flow/browser's peer/dep chain.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI (`npm ci`) on PR #430 failed with:
npm error `npm ci` can only install packages when your package.json
and package-lock.json or npm-shrinkwrap.json are in sync.
npm error Missing: @claude-flow/cli@3.5.80 from lock file
npm error Missing: @ruvector/attention@0.1.32 from lock file
... (dozens of transitive deps)
Root cause: when @claude-flow/guidance was moved from dependencies to
optionalDependencies, my local `npm install` ran in a Codespace where
@claude-flow/cli and related ruflo packages were already extraneous on
disk from a previous `claude mcp add ruflo` step. The npm install
reconciled node_modules to the new package.json but produced a lockfile
that assumed those extraneous packages were still available at resolve
time — so when CI did a clean `npm ci`, the lockfile claimed resolution
info that wasn't there.
Fix: remove node_modules + package-lock.json and regenerate from
package.json alone. New lockfile has +988 resolved packages (was missing
transitive deps pulled in by the optional @claude-flow/* chain) and
passes `npm ci --ignore-scripts` with exit 0.
Verified:
- `npm run build` — CLI + MCP bundles build
- `node dist/cli/bundle.js --version` → 3.9.14
- `npm audit --omit=dev` — 0 vulnerabilities (6 highs remain in dev-only
eslint/typescript-eslint chains — separate backlog item)
- `npm ci --ignore-scripts` — exit 0, 988 packages resolved cleanly
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Security + supply-chain hardening closes five P0 release blockers from
the v3.9.13 QE audit. Shipped tarball drops 52% (19.9 MB → 9.6 MB),
15 critical runtime npm vulnerabilities are eliminated, and a
command-injection path in `aqe learning repair` is closed.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Security (M-01):
- src/cli/commands/learning.ts: replace execSync shell-interpolation of
the user-controlled --file path with execFileSync + file-descriptor
redirect (dump) and stdin input (import). Closes the command-injection
vector where a crafted --file="x.db\"; rm -rf ~; \"" would escape
shell quoting.
Build / supply chain:
- Move @claude-flow/guidance from dependencies to optionalDependencies
alongside @claude-flow/browser. No stable successor exists on npm;
usage is already guarded by try/await-import/catch fallbacks so
demotion is safe. Documented in the follow-up research memo.
- Lazy-load @faker-js/faker via dynamic import in
test-data-generator.ts and mark it external in build-cli.mjs /
build-mcp.mjs. Faker is a devDep that previously got statically
bundled into shipped artefacts because of a top-level import. Tarball
drops 11.5 MB → 9.6 MB.
MCP API contracts:
- Add required:true to semantically load-bearing first parameters on
agent_metrics (agentId), coverage_analyze_sublinear (target),
accessibility_test (url), defect_predict (target), code_index
(target). Verified via live MCP tools/list handshake — required
arrays now match the intent documented in each tool's description.
CI hygiene:
- Rewrite the "Commit telemetry data" step in
qcsd-production-trigger.yml to push to a bot branch and open a PR
instead of `git push` direct to protected main. That push failed 8/10
runs on the last audit cycle because of branch protection. Artifact
upload (already present) continues to preserve the raw payload for 90
days regardless of whether the PR lands.
Verification:
- npm audit --omit=dev: 0 critical, 0 high (1 moderate in
@mapbox/node-pre-gyp — extraneous, pre-existing, unrelated).
- npm run typecheck: clean.
- CLI + MCP bundles rebuild successfully.
- 253/253 test-generation unit tests pass.
- Live MCP protocol smoke: new required flags visible on tools/list;
advisor_consult guard from v3.9.14 round 1 still fires.
Memory note: see feedback_cli_is_aqe.md — CLAUDE.md ruflo refs were
audited. Only 2 lines (204, 260-261) are stale AQE-product refs; the
other 11 swarm/memory/daemon ruflo commands are legitimate
orchestration-layer invocations and are intentionally retained.
Deferred to a dedicated docs PR so the user can review each.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Two targeted investigations opened while triaging the v3.9.13 QE audit
priority matrix:
1. @claude-flow/guidance strategy — recommendation: keep + demote to
optionalDependencies. No newer stable version exists on npm; usage is
already try/await-import/catch-fallback pattern across 62 refs in 17
files; the only real concern (protobufjs CWE-94 supply-chain) is
already neutralised by the v3.9.14 overrides.
2. QE agent count recount — verdict: the "60 specialized QE agents"
claim is honest, not an overcount. Primary fleet is 53 qe-*.md under
.claude/agents/v3/ plus 7 TDD/review subagents under
.claude/agents/v3/subagents/qe-*.md. Both sets mirror into
assets/agents/v3/ for npm distribution. The earlier honesty-audit
finding undercounted by 7 because subagents were not enumerated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Security (15 criticals → 0):
- Pin protobufjs to ^7.5.5 via overrides + resolutions to break the
@xenova/transformers → @claude-flow/{browser,guidance,embeddings}
→ onnxruntime-web → onnx-proto chain reaching the CVE-94 payload
(GHSA-xq3m-2v4x-88gg). npm audit --omit=dev now reports 0 vulns.
Tarball bloat (-42%):
- Add rmSync(dist/cli/chunks) to build-cli.mjs before the code-split
build so only fresh chunks ship. 799 → 240 chunks, dist/cli 44 → 11
MB, npm tarball 19.9 → 11.5 MB.
Retiring-model refs (ADR-093 tier-1 hardcodes):
- Replace 'claude-3-haiku-20240307' with 'claude-haiku-4-5' in
MODEL_TIERS[1] at constants.ts:608 and in five mirror implementations
across test-execution, contract-testing, chaos-resilience,
learning-optimization, and visual-accessibility services. Haiku 3
retirement will no longer 404 these tier-1 fallback paths. Backward-
compat refs retained intentionally in bedrock ARN table, cost
tracker, type unions, and RETIRING_MODELS schedule.
ESLint harness:
- package.json lint script invoked eslint on both `src` and `tests`
but .eslintrc.cjs explicitly ignores `tests`, causing ESLint to
abort. Narrow the script to `eslint src --ext .ts` so the harness
runs. (955 pre-existing rule violations remain as separate backlog.)
advisor_consult MCP contract (ADR-092 F-01):
- Mark `agent` and `task` parameters as required:true in the schema.
- Validate via trim() guard in the handler and return a structured
error when either is empty/whitespace instead of shelling out to
`aqe llm advise` with placeholder values. Verified via live MCP
stdio handshake: both empty-string and whitespace-only `agent`
produce the expected rejection without any child process spawn.
Verification:
- npm audit --omit=dev → 0 vulnerabilities (verified from node_modules,
not just lockfile).
- npm run typecheck → clean.
- npm run build → CLI + MCP bundles both build.
- 94/94 tests pass across the patterns touched (security-compliance
scanners + task-executor-errors).
- MCP protocol smoke test confirms advisor_consult guard fires.
Restores Dependency/Build from Grade C to Grade B and lifts the
composite quality score from 7.0 to ~7.3.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
QE Queen orchestrated parallel analysis across code complexity, security,
performance, test quality, product/QX, dependency health, API contracts,
DDD architecture, accessibility, brutal honesty, and CI pipelines.
Composite score 7.0/10 (+0.1 vs v3.8.13). First honesty-score reversal in
five cycles (68 → 74). CI fully recovered (+4.5) with test-gated npm
publish. Dual MCP server divergence resolved. Surfaced new release
blockers: 15 critical runtime npm vulns via protobufjs chain and 79%
tarball bloat from stale code-split chunks.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
These agents were updated in .claude/agents/v3/ for ADR-093 but the
corresponding files in assets/agents/v3/ (the directory shipped to
users via aqe init) still carried stale advisor model IDs
(claude-opus-4, claude-opus-4-6). Fresh installs would have received
the old model references.
Synced files:
qe-coverage-specialist, qe-fleet-commander, qe-pentest-validator,
qe-queen-coordinator, qe-risk-assessor, qe-root-cause-analyzer,
qe-security-auditor, qe-security-scanner, qe-test-architect
Verified in a fresh project: aqe init --auto now writes agent files
with claude-opus-4.7 / claude-opus-4-7 advisor models and the ADR-093
"effort: max" frontmatter for the three cyber-pinned agents.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships ADR-093 Opus 4.7 migration: Sonnet 4.6 becomes the fleet-wide
default, xhigh effort level is on everywhere, Opus 4.7 is the opt-in
escalation target via MultiModelExecutor, and cyber-sensitive agents
stay pinned to Sonnet 4.6 pending Cyber Verification Program approval.
Also fixes two stale integration-test assertions that still expected
the retiring claude-haiku-3-5 ID; both now assert the current
claude-haiku-4-5 per the model registry.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Status moves from Proposed to Accepted. Phase 3 thinking-config shim
dropped from scope after codebase sweep confirmed no raw-Messages-API
callers pass {type: enabled, budget_tokens: N}. Remaining work tracked:
Cyber Verification application submission (external) and 30-day
telemetry recalibration (~2026-05-17).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Four merge-blocking issues surfaced by the qe-devils-advocate audit.
C1 — effort-resolver YAML loading was silently broken in ESM
src/shared/llm/effort-resolver.ts: replaced
const { readFileSync } = require('fs') with a top-level
import { readFileSync } from 'fs'. The require() call threw
ReferenceError under ESM ("type": "module" in package.json),
caught by the bare try/catch, so the YAML priority tier in the
resolver chain was dead — callers always fell through to the
hardcoded xhigh default. config/fleet-defaults.yaml was a
decorative file. Proven fixed by runtime check:
YAML set to 'medium' → resolveEffortLevel() now returns 'medium'.
C2 — body.effort on every /v1/messages call was unverified shape
src/shared/llm/providers/claude.ts: the previous code
unconditionally set body.effort on every request, regardless of
whether the target model supported it, and used a top-level shape
that is not documented in Anthropic's public Messages API schema.
New behavior: effort is only set when the registry marks the
model supportsEffortXHigh=true (Opus 4.7 today) and is nested
under body.thinking = { type: 'adaptive', effort: <level> },
matching the Anthropic SDK's beta.messages thinking shape.
Non-xhigh models receive no thinking/effort field at all,
eliminating the risk of 400-ing Sonnet 4.6 / Haiku 4.5 calls.
Rationale inline at the call site.
C3 — CYBER_PINNED_AGENTS was missing qe-security-reviewer
qe-security-reviewer lives at .claude/agents/v3/subagents/
(not in .claude/agents/v3/ top level) and was present in the
submitted Cyber Verification application. Without it in the pin
list, that agent could route to Opus 4.7 freely. Added.
Cross-check test now parses the application file and asserts
every CYBER_PINNED_AGENTS entry is named in it — future edits
to either side cannot silently diverge.
C4 — cyber pin only fired in MultiModelExecutor.consult()
Direct HybridRouter.chat() calls from security agents bypassed
the pin. Extracted the pin helpers into src/routing/security/
cyber-pin.ts (applyCyberPin, shouldCyberPin, isOpus47,
CYBER_PINNED_AGENTS, CYBER_PIN_ADVISOR_FALLBACK,
CYBER_PIN_CHAT_FALLBACK). HybridRouter.chat() now applies the
pin post-selectProvider, before executeWithFallback, on both
decision.model and decision.providerModelId — covering
canonical, OpenRouter (dotted), and Bedrock ARN forms.
MultiModelExecutor.consult() imports from the same module so
both paths share one source of truth.
Tests moved tests/routing/advisor/cyber-pin.test.ts →
tests/routing/security/cyber-pin.test.ts and expanded: 19 tests
covering all 4 pinned agents × multiple model-id forms, the
application cross-check, env-flag semantics, non-security
passthrough, and non-4.7 model passthrough.
Verification:
* 0 tsc errors
* 128/128 targeted tests pass (effort-resolver + cyber-pin + advisor + providers)
* 2927/2927 broader regression pass (shared/llm + coordination + routing)
* Runtime proof of C1 fix: YAML 'medium' → resolveEffortLevel() returns 'medium'
Still outstanding from the devil's-advocate audit (deferred to
follow-up, not merge-blocking):
* C5 — Bedrock/OpenRouter model IDs need empirical verification
* C6 — no integration test hits real /v1/messages with 4.7
* H1–H5, M1–M5, L1–L3
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
91 files changed, +99/-99.
Fifth and final post-audit sweep. Addresses all gaps surfaced by
the third broader audit pass.
* packages/aqe-opencode-plugin/src/degradation/graceful-degradation.ts:
tier map updated — anthropic/claude-opus-4 → -4.7,
anthropic/claude-sonnet-4 → -4.6, anthropic/claude-haiku-3.5 → -4.5.
JSDoc example model ID also refreshed. This package is shipped
to OpenCode users.
* scripts/adr-092-multi-model-trial.ts: EXECUTOR_MODEL constant
updated to anthropic/claude-sonnet-4.6 (sibling of the harness
already fixed in commit 2068c293).
* .kiro/agents/*.json (87 files): generated Kiro agent configs
swept — claude-sonnet-4 → -4-6, claude-opus-4 → -4-7,
claude-haiku-4 / claude-haiku-3.5 → -4-5. These are dogfooding
artifacts in this repo from a prior init run. Users get fresh
(post-ADR-093) configs from the new kiro-installer.ts.
* .claude/commands/aqe-costs.md: example output in the slash
command docs refreshed to the post-ADR-093 baseline
(claude-sonnet-4-6, claude-haiku-4-5-20251001).
* tests/unit/cli/commands/llm-router.test.ts: 'should estimate
cost for a known model' — input changed from 'claude-sonnet-4'
to 'claude-sonnet-4-6'. The canonical was renamed in
DEFAULT_MODEL_MAPPINGS (commit 4f175638), so the old canonical
no longer resolves.
Verification:
* 0 tsc errors
* 25/25 llm-router.test.ts pass
* grep -rE 'claude-sonnet-4-20250514|claude-opus-4-20250514|claude-opus-4-5-20251101|claude-3-5-haiku-20241022' src/ returns 0 outside data-definition files (model-mapping, model-registry, cost-tracker), as intended.
Deliberately left alone (confirmed):
* .claude/settings.local.json — permission allowlist only
* Historical docs (CHANGELOG, ADR-056, SPEC-043-C, GOAP-ADR-043,
docs/releases, llm-independence-plan, ADR-089)
* tests/unit/shared/llm/model-mapping.test.ts and
model-registry.test.ts — deliberate retiring-ID regression coverage
* ADR-093 itself — references retiring IDs in Context and Options
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New user projects created via aqe init will now receive the post-ADR-093
model set (Opus 4.7 / Sonnet 4.6 / Haiku 4.5) instead of retiring IDs.
* src/init/kiro-installer.ts: 4 hardcoded model fallbacks updated.
claude-sonnet-4 → claude-sonnet-4-6 (default + priority=critical),
claude-opus-4 → claude-opus-4-7 (opus-detected path),
claude-haiku-4 → claude-haiku-4-5 (haiku-detected path).
Affects: runtime agent transformation, default QE agent template
(writeDefaultQEAgent).
* src/init/init-wizard-steps.ts, src/init/phases/12-verification.ts:
generated aqe.yaml advisor block comment updated to
anthropic/claude-opus-4.7 so wizard-produced configs point to the
right advisor model out of the box.
* .claude/settings.json:188-189: claudeFlow.modelPreferences
default claude-opus-4-5-20251101 → claude-opus-4-7,
routing claude-3-5-haiku-20241022 → claude-haiku-4-5-20251001.
* Skill eval YAMLs (both .claude/ and assets/ copies):
api-testing-patterns, contract-testing, database-testing,
testability-scoring — models_to_test list updated from
claude-sonnet-4 / claude-3-haiku to claude-sonnet-4-6 / claude-haiku-4-5.
* Skill eval schemas (3 copies: .claude, assets, docs):
skill-eval.schema.json enum extended with claude-opus-4-7,
claude-sonnet-4-6, claude-haiku-4-5 as accepted values. Old
entries retained for backward compat with existing eval files.
* Sample outputs (3 copies): modelUsed field updated to
claude-sonnet-4-6.
* a11y-ally SKILL.md code example (both .claude and assets):
model: 'claude-sonnet-4-20250514' → 'claude-sonnet-4-6' (this was
caught by Commit 2's sweep; landed here for audit completeness).
Verification:
* 0 tsc errors
* 392/392 init module tests pass
* npm run build: CLI + MCP bundles rebuilt clean
Not re-running aqe init against a real fixture in this commit —
minimal+non-interactive mode doesn't materialize integration files
(kiro/opencode); full wizard-mode verification deferred until after
PR opens and a maintainer runs init smoke.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Post-verification sweep found 4 categories of missed references:
(a) 6 qe-* agents had explicit advisor.model: anthropic/claude-opus-4
overriding the new 4.7 default from Commit 5:
qe-coverage-specialist, qe-fleet-commander, qe-queen-coordinator,
qe-risk-assessor, qe-root-cause-analyzer, qe-test-architect.
Updated each to anthropic/claude-opus-4.7.
(b) scripts/adr-092-phase0a-harness.ts: EXECUTOR_MODEL /
ADVISOR_MODEL constants for the ADR-092 Phase 0 eval harness
still hardcoded Sonnet 4 / Opus 4. Updated to Sonnet 4.6 and
Opus 4.7 so re-running the harness produces data consistent
with the post-migration fleet.
(c) scripts/eval-driven-workflow.ts: YAML template's models_to_test
list updated from claude-sonnet-4 / claude-3-haiku to
claude-sonnet-4-6 / claude-haiku-4-5.
(d) .claude/settings.json:188 claudeFlow.modelPreferences.default
updated from claude-opus-4-5-20251101 to claude-opus-4-7
(explicit user approval).
Not touched in this commit:
* .claude/settings.json:189 modelPreferences.routing still
'claude-3-5-haiku-20241022' (retiring 2026-06-15). Awaiting
explicit user direction.
* .claude/skills/*/evals/*.yaml, assets/skills/a11y-ally/SKILL.md,
assets/skills/testability-scoring/evals/*.yaml — skill eval
YAMLs with model references. Scope deferred to user review.
* Historical docs/tests that reference retiring IDs as regression
coverage (CHANGELOG, ADR-056, SPEC-043-C, model-mapping/registry
test files).
Verification:
* 0 tsc errors
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Verification commit (post Commit 6). User-perspective smoke caught a gap
that unit tests missed:
`aqe llm models` reads DEFAULT_MODEL_MAPPINGS from
src/shared/llm/router/types.ts, which is a separate registry from
MODEL_REGISTRY in model-registry.ts. Commit 2's sed sweep updated the
provider IDs inside DEFAULT_MODEL_MAPPINGS (anthropic path became
claude-opus-4-7 / claude-sonnet-4-6 / claude-haiku-4-5-20251001) but
left the canonicalId and canonicalName fields as 'claude-opus-4.5',
'claude-sonnet-4', 'claude-haiku-3.5'. Result:
* canonical claude-opus-4.5 → anthropic claude-opus-4-7, bedrock
anthropic.claude-opus-4-5-v1:0. Same canonical resolved to two
different models across providers (routing bug).
* `aqe llm models` output listed stale names (Opus 4.5 / Sonnet 4 /
Haiku 3.5) despite the backing model being 4.7 / 4.6 / 4.5.
Fix: rewrite the three Claude entries as first-class Opus 4.7 /
Sonnet 4.6 / Haiku 4.5 records with consistent canonicalId, canonical
names, all three provider IDs, 1M context for 4.7, correct pricing
($5/$25 for Opus, unchanged $3/$15 for Sonnet, $1/$5 for Haiku),
64k / 128k / 16k max output tokens respectively.
Verification (user perspective):
* npm run build: CLI + MCP bundles built clean
* node dist/cli/bundle.js llm models → shows Opus 4.7/Sonnet 4.6/Haiku 4.5
* node dist/cli/bundle.js llm cost claude-opus-4-7 --tokens 10000
→ "Claude Opus 4.7", $5/$25, $0.30 total (correct)
* node dist/cli/bundle.js llm cost claude-sonnet-4-6 --tokens 10000
→ "Claude Sonnet 4.6", $3/$15, $0.18 total (correct)
* MCP bundle loads ("MCP server starting v3.9.12")
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 6 of 6 for ADR-093 Opus 4.7 migration. Opt-out gate behind
AQE_CYBER_VERIFIED env var.
* src/routing/advisor/multi-model-executor.ts: new applyCyberPin()
helper + CYBER_PINNED_AGENTS list (qe-pentest-validator,
qe-security-auditor, qe-security-scanner). When the advisor model
resolves to Opus 4.7 for any of these agents, the pin downgrades
to anthropic/claude-sonnet-4.6 and logs a warning naming the env
var to flip after Cyber Verification Program approval lands.
Non-security agents pass through unchanged. Non-4.7 models pass
through unchanged. Only AQE_CYBER_VERIFIED === 'true' lifts the
pin — '1', 'yes', truthy-but-non-"true" values keep the pin.
* tests/routing/advisor/cyber-pin.test.ts (new): 10 tests covering
all 3 pinned agents with pin-on and pin-off, non-security passthrough,
non-Opus-4.7 passthrough, canonical claude-opus-4-7 form, and
strict "true" semantics.
Fleet inventory note: 3 qe-* security/pentest agents exist. The
Cyber Verification application draft listed 4 (including a
qe-security-reviewer that does not exist in .claude/agents/v3/).
Verification:
* 0 tsc errors
* 75/75 advisor tests pass (+10 new)
* 16555/16564 unit + routing tests pass (9 pre-existing skips)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 5 of 6 for ADR-093 (Commit 4 skipped — no thinking-config
callers to shim; documented in the implementation plan).
* src/routing/advisor/multi-model-executor.ts: DEFAULT_ADVISOR_MODEL
upgraded from 'anthropic/claude-opus-4' to 'anthropic/claude-opus-4.7'.
When TinyDancerRouter sets triggerMultiModel=true (confidence < 0.80,
or < 0.85 for security tasks), MultiModelExecutor.consult() now
dispatches to Opus 4.7 by default via HybridRouter.chat() +
OpenRouter provider.
* tests/routing/advisor/multi-model-executor.test.ts,
tiny-dancer-router-advisor.test.ts: update expected provider model IDs.
SWE-bench Verified 87.6% (vs 80.8% Opus 4.6), SWE-bench Pro 64.3%
(vs 53.4%), CursorBench 70% (vs 58%). Vendor-independence preserved —
consultations still flow through HybridRouter's 7 providers per
ADR-043.
Verification:
* 0 tsc errors
* 65/65 advisor unit tests pass
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Commit 3 of 6 for ADR-093 Opus 4.7 migration. Adds effort-level plumbing
with xhigh as the fleet-wide default and max override on security agents.
* src/shared/llm/effort-resolver.ts (new): resolveEffortLevel() implements
the ADR-093 priority chain:
runtime override > per-agent frontmatter > QE_EFFORT_LEVEL env
> config/fleet-defaults.yaml > DEFAULT_EFFORT_LEVEL (xhigh)
Plus downgradeEffort() for models that don't support xhigh.
Minimal regex-based YAML parse avoids a new dependency.
* config/fleet-defaults.yaml (new): effort_level: xhigh.
* src/shared/llm/interfaces.ts: GenerateOptions gains optional
`effort?: 'low' | 'medium' | 'high' | 'xhigh' | 'max'`.
* src/shared/llm/providers/claude.ts: resolve effort per request and
include in /v1/messages body; silently downgrade to 'high' when the
target model lacks supportsEffortXHigh capability.
* .claude/agents/v3/qe-security-auditor.md,
qe-pentest-validator.md,
qe-security-scanner.md: add `effort: max` frontmatter per user
direction. Also corrects stale `advisor.model: claude-opus-4-6`
(non-existent ID) to `claude-opus-4-7` on the two agents that had it.
Note: fleet has 3 security/pentest qe-* agents (qe-security-auditor,
qe-security-scanner, qe-pentest-validator). The Cyber Verification
application lists 4 (including qe-security-reviewer which does not
exist in .claude/agents/v3/). If the submitted application needs an
amendment, re-file with the correct 3-agent list.
* tests/unit/shared/llm/effort-resolver.test.ts (new): 13 tests
covering the priority chain, invalid-input fallback, YAML read,
and downgrade helper.
Verification:
* 0 tsc errors (npx tsc --noEmit)
* 677/677 LLM unit tests pass (+13 new)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- Add ruflo/bundle.js patterns to AQE_COMMAND_PATTERNS so mergeHooksSmart
recognizes ruflo-installed hooks and replaces them instead of duplicating
- Check root .mcp.json in Claude Flow detection (where ruflo/aqe actually
write MCP config) to avoid falling through to slow 5-10s npx binary probe
- Remove runPretrainAnalysis (120s timeout execSync) from aqe init — pretrain
is already handled by ruflo init, was the primary cause of init appearing stuck
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The /release skill's PR body template was missing the required
failure-modes checkbox from .github/PULL_REQUEST_TEMPLATE.md,
causing CI to reject release PRs. Updated Step 10 with the full
PR template structure (Summary, Verification, Failure modes,
Required check) and added PR body compliance to the Rules section.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dragan Spiridonov