Self-audit of the free-tier lane found two gaps:
- Multi-file: the free-tier path generated for sourceFiles[0] only, silently
skipping the rest AND the paid path. Now it engages only for single-file
requests; multi-file falls back to the normal path so every file is covered.
(+1 regression test.)
- Exports/docs mismatch: the guide + README showed
`import ... from 'agentic-qe/routing/free-tier'`, but package.json had no
./routing export — the import would fail for npm consumers. Added the
./routing/free-tier subpath export.
Verified: project `tsc --noEmit` clean; 236 routing/coordinator/plugin tests +
18 routing-feedback tests green; no sibling breakage from the generic tracker.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the cheap-first economics: local → repair → escalate the hard tail.
- coordinator: build a claudeRunner from the existing HybridRouter (tier →
complexity hint) and run the free-tier path with escalate:true. Falls back to
local-only automatically when no router is wired. patternsUsed now records the
winning tier.
- plugin (kernel layer): onInitialize constructs a live RoutingFeedbackCollector
(EMA calibration + auto-escalation, memory-only fallback) when the free tier is
opted in, and injects it as the coordinator's routingFeedback sink — so each
cheap-vs-escalated outcome drives the calibrator/escalation tracker/confidence
(lifts the stuck-at-40% routing metric). Built only when opted in.
- tests: +2 (escalation via router, D9 sink invocation); 113 green; existing
coordinator (47) and plugin (38) suites unchanged. Type-safe (MessageRole /
TaskComplexity / RoutingConfig fields all confirmed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply the free-tier escalation work to the test-generation coordinator as an
OPT-IN, default-off capability (Ruv's cheap-first economics). Zero impact unless
enabled via `enableFreeTier` config or `AQE_FREE_TIER=1`.
- coordinator: guarded `tryFreeTierGeneration()` tries the free local model first
(reads source, prompts, verifies test+assertion), falls through to the
unchanged paid path on any miss. 47 existing coordinator tests still green.
New optional `routingFeedback?` ctor param wires D9.
- D8 repair loop: executor gains same-tier repair (`repairAttempts`, verifier
feedback fed back) + `escalate:false` repair-only mode. Coordinator path runs
local-only + repair, no paid escalation yet.
- D9 sink: createRoutingFeedbackSink() maps executor outcomes onto the existing
RoutingFeedbackCollector (calibrator + escalation + confidence); cheap wins
raise the cheap tier's confidence, lifting the stuck-at-40% routing metric.
- docs: user guide docs/guides/free-tier-local-models.md (local/cloud Ollama,
OpenRouter, OpenAI-compatible) linked from README.
- tests: +11 new (executor repair, sink, coordinator opt-in); 111 green total;
free-tier modules strict-tsc clean.
Model: qwen3:8b default. Keys read from named env vars, never stored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Devil's-advocate review of today's branch surfaced three real gaps:
C1 (init-handler.ts): bare `aqe init --no-claude` (no --with-<platform>, no
--no-database) fell through to runStandardInit and installed the FULL Claude
surface it was meant to suppress — `noClaude` was only honored inside the
modular path. Add `options.claude === false` to the modular routing guard.
Verified: `aqe init --no-claude --minimal` now writes no .claude/, .mcp.json,
or CLAUDE.md (and the empty-install warning fires).
C2 (continuedev-installer.ts): the mergeExistingYamlConfig branch (existing
`mcpServers:` + --overwrite) appended a hardcoded AQE_MEMORY_PATH entry,
ignoring memoryBackend — re-introducing the #533 db leak in db-free installs
over an existing Continue config. Honor memoryBackend on this path too. Adds a
regression test.
H3 (entry.ts): main() ran fire-and-forget, so a startup rejection that escaped
its internal try hit the unhandledRejection handler (which intentionally keeps
the server alive at runtime) → on the in-process `agentic-qe mcp` path the CLI
wrapper would hang on its never-resolving promise with a zero exit. main() now
`.catch`es and exits non-zero on a failed startup. Re-verified #528/#535
integration tests still green (normal startup unaffected).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#533 — database-free mode now propagates AQE_MEMORY_BACKEND=memory to ALL
generated MCP-server configs, not just opencode.json. Previously `aqe init
--no-database` left .mcp.json (Claude Code) and every editor config pointed at
.agentic-qe/memory.db, so launching the server via any non-OpenCode client
recreated the db at runtime and defeated the db-free install.
- platform-config-generator.ts: generateMcpConfig(platformId, { memoryBackend })
emits AQE_MEMORY_BACKEND=memory and drops AQE_MEMORY_PATH for JSON/TOML/YAML;
covers the 8 shared-generator installers (cursor/cline/continuedev/kilocode/
roocode/windsurf/codex/copilot).
- 08-mcp.ts (.mcp.json) and kiro-installer.ts: same db-free env; learning/workers
not forced on in db-free mode (they persist to SQLite).
- 09-assets.ts threads memoryBackend into every platform installer.
Default (persistent) output is byte-for-byte unchanged.
#532 — new opt-in `--no-claude` suppresses the default Claude Code surface
(.claude/ skills+agents, .mcp.json, CLAUDE.md, governance, hooks) so
`--with-<platform>` flags become the only install targets — e.g. a clean
OpenCode-only install. Gated via shouldRun/run on phases 07-hooks, 08-mcp,
09-assets (core assets), 11-claude-md, 13-governance. Default install
(no flag) is unchanged; warns if --no-claude is passed with no platform.
Verified by real `aqe init` runs in temp dirs:
- `--no-database --with-codex --with-cursor`: .mcp.json/.cursor/.codex all emit
AQE_MEMORY_BACKEND=memory; no memory.db created.
- `--no-claude --with-opencode --no-database`: .opencode present; .claude/,
.mcp.json, CLAUDE.md all absent.
- default `--with-cursor`: full surface + persistent env intact (no regression).
Adds db-free + no-claude cases to platform-config-generator and mcp-phase tests;
all 406 init unit tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The 'reworded prose' mutation test used .replace('qe_agent_definition',
'qe_agent_definition') — a no-op replacing a string with itself (CodeQL
js/identity-replacement). Replaced with a real non-invariant edit
(frontmatter description + a <notes> block after the closing tag) that
genuinely exercises 'edits outside required sections must not trip the
gate'. 9/9 tests still pass.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
MCP-CLI parity bug found by a REAL protocol call during P5 acceptance:
the handler projected findings to a fixed 5-field shape, stripping
ADR-105's evidenceClass at the MCP boundary — unit tests passed because
they tested the projection itself. Projection now carries evidenceClass
+ evidenceArtifact per finding and the needsVerification queue; two
parity-pin tests added so the field can never be silently dropped again.
Verified: 9/9 handler tests; direct handler invocation (same code path
as the MCP tool) shows evidenceClass STATIC + needsVerification array.
Live MCP server reflects this after restart (long-running process has
pre-fix code loaded).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
RUBRIC-v1.md committed first per ADR-108: ground-truth primary outcome
(hidden acceptance test), style-neutral verbatim judge instruction (no
treatment-revealing framing in either direction), two-pass cross-family
judging with INFERRED-labeled fallback, clustered-stats reporting rule.
Provably rubric-before-data: this commit precedes any results-* artifact.
2 scenarios with seeded defects + personas + hidden tests (off-by-one
pagination; swallowed async rejection). Hidden tests verified both
directions: FAIL on seeded bug with precise reasons, PASS on the fix.
lib/stats.ts encodes the three Pattern Space harness fixes as code:
isGoodRow enforced inside aggregate() (no path to a number that skips
it), seeded per-item counterbalance (their null cell was 72% position-
skewed), exact scenario-clustered sign test (validated against the
known 27/11 → p≈0.0136 re-analysis).
Verified: 14/14 stats tests; hidden-test ground truth both directions;
tsc clean. Remaining: live conversation driver + judge integration
(API-gated, same decision as ADR-106 live mode).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
PatternNullStore: recordNull with ON CONFLICT consolidation,
getNullSummaries batch fetch, and the pure applyNullDiscount ranking
helper (context-matched nulls discount 5x harder than remote ones;
0.25 floor keeps patterns retrievable — nulls inform, never erase).
ExperienceCaptureService: appliedPatterns on TaskExperience (patterns
that GUIDED the task), setNullRecorder following the setWitnessChain
convention, and null capture in completeCapture — failure = negative
reward when a signal exists, else !success; recorder errors never break
capture; context fingerprint = domain:agent (v1, per ADR).
Verified: 11/11 tests run locally (London-school mocked backend — no
native deps) incl. reward-overrides-success, per-pattern fan-out,
recorder-throw survival, and discount floor/ordering; tsc clean.
Remaining for ADR-110: retrieval-time discount wiring in PatternStore
search, arena losing-strategy persistence, composition-root recorder
wiring.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Additive schema v10: qe_pattern_nulls (pattern_id FK, context_fingerprint,
failure_mode, trajectory_ref, evidence_class CHECK per ADR-105,
consolidated_count, UNIQUE(pattern_id, context_fingerprint)) + 2 indexes.
Wired into unified-memory v10 migration block; rollback refuses when
nulls exist (nulls are data — the un-committed v2.1 lesson).
Verified against a sqlite3-online-backup COPY of the real memory.db
(61MB, original never written): all 51 pre-existing table counts
byte-identical before/after (qe_patterns=298, qe_trajectories=377,
kv_store=5008), PRAGMA integrity_check ok, table+4 indexes created,
UNIQUE fires on duplicate, ON CONFLICT consolidation bumps count to 2.
tsc --noEmit clean. Unit tests (10) follow the repo's better-sqlite3
convention and run in CI — local native binding is the known macOS
binary (invalid ELF header).
Verification also caught a false-positive in its own first run: an
empty SQL pipe reported 'applied' with trivially-unchanged counts; the
new-objects query exposed it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Only EXECUTED/STATIC findings from failed blocking steps may block or
halt the pipeline. INFERRED findings queue in PipelineResult.
needsVerification (the ADR-102 adversarial-verification dispatch
surface) and downgrade overall to warn. CONJECTURE never gates. A
failed blocking step without gating evidence reads as warn, never as a
clean pass. Markdown report shows evidence class on blockers and a
'Needs Verification' section.
Verified: tsc clean; 8 new gating tests + 7 MCP handler tests, 15/15.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aqe arena run: seeded tournaments where test strategies compete on REAL
fitness — built-in first-order operator mutants (string/comment/arrow-safe
enumeration) executed per strategy via node --test, real line coverage
from --experimental-test-coverage, and a deterministic suite-cost term.
Tournament emits a Wolfram-style pairwise competitive array; --evolve
hill-climbs the winner by seeded group flips with real re-evaluation.
Runs persist fail-soft to memory.db kv_store (namespace 'arena');
aqe arena list reads them back. Self-contained: no core changes, no new
dependencies; fixtures/arena-demo ships 4 test groups of varying kill
power over a branch-dense pricing module.
Reproducibility contract: same seed + fixture => byte-identical envelope
(verified E2E by diffing two runs), with informational.runtimesMs the
only excluded field. Honest finding during verification: the original
measured-runtime rank penalty was jitter-dominated at fixture scale and
flipped rankings between identical-seed runs — replaced with the
deterministic suite-size ratio (documented in ADR-104).
Phase 2 (documented): LLM-generated strategies frozen as artifacts,
winners distilled into qe-test-architect priors; MCP arena_run
registration (CLI and MCP share runArena() — parity by construction).
10 tests green (incl. real-engine reproducibility); tsc + eslint clean.
Part of #520 (improvement 7/7 — initiative complete).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Versioned, additive-only verdict envelopes for agent/MCP handoffs:
RiskDecision (quality gates), FindingVerdict (review findings with
adversarial refutations), CoverageGap (risk-weighted gaps).
- src/contracts/verdicts.ts: dependency-free validators (source of
truth) + mirrored draft-07 JSON Schemas; no zod/ajv runtime deps
(light-install posture preserved)
- quality_assess MCP boundary now attaches a validated riskDecision
envelope derived from the gate outcome (approve/block, escalate when
indeterminate) — additive, existing payload unchanged; direct and
wrapped handlers share the same mapToResult so MCP parity is inherent
- schemas/*.schema.json published via scripts/generate-verdict-schemas.mjs
for external ajv validation and workflow agent({schema}) enforcement
(consumed by improvement 5)
17 contract tests (goldens validate; mutations rejected with field-level
errors; builder always emits valid envelopes). E2E: envelope emitted by
the compiled boundary builder validates with stock ajv against the
published schema; mutated sample rejected. tsc + eslint clean.
Part of #520 (improvement 6/7, Tier 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Anthropic announced nested subagent support (depth=5) on 2026-06-09; the
runtime currently strips Task at parent->child spawn (upstream denylist,
empirically confirmed by our committed probe: level=1 status=NO_AGENT_TOOL,
matching ruflo ADR-147). This lands the readiness infrastructure that is
independently useful today and activates with zero changes when the gate
lifts:
- aqe hooks post-task gains --parent-agent-id / --depth (validated:
identifier charset, integer 0 <= d <= 32; depth=0 survives as 0);
provenance persisted to qe_trajectories.metadata_json (stage 1 — no
schema migration) and forwarded into ReasoningBank feedback so learning
can segment patterns per hierarchy level
- TaskExecution gains optional parentAgentId/depth (additive types)
- scripts/probe-nested-spawn-depth.mjs: single-shot empirical probe,
committed transcript under docs/probes/ — re-run after every
`claude update`; TASK_TOOL_LIVE is the signal for follow-up phases
- fresh-project fix found by the E2E: persistTaskOutcome assumed
captured_experiences/experience_applications exist, but they are
middleware-created — Stream B silently no-oped on fresh installs;
canonical DDL now ensured in the hook path
Deliberately deferred (documented in ADR-101 P1): tools: frontmatter
retrofit on shipped qe-*.md agents — Claude Code frontmatter is a
restrictive allowlist and would sever inherited MCP tool access;
needs maintainer sign-off + wildcard verification first.
8 unit tests green; E2E on a fresh temp project verified persistence,
rejections, and depth=0 boundary. tsc + eslint clean.
Part of #520 (improvement 4/7, Tier 2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Breaks runaway retry loops: when the same Bash command fails consecutively
(normalized signature, cross-process via a small JSON sidecar under
.agentic-qe/), pre-command warns at 3 failures and emits a BLOCK
recommendation with a recovery hint at 5. Advisory by default;
AQE_STRICT_TOOL_LOOP=1 upgrades the block to a PreToolUse deny. A
successful run resets the breaker; an open breaker grants a half-open
probe after 2 minutes; stale entries pruned after 30 minutes.
Mirrors the ADR-064 domain-circuit-breaker semantics at command level.
Sidecar instead of memory.db: pre-command deliberately runs without a DB
open, and each hook call is a fresh process so in-process state is
impossible. Fail-open everywhere (corrupt state file => allow).
11 unit tests green; E2E verified with real aqe hooks pre/post-command:
warn after 3, BLOCK after 5, strict-mode deny envelope, reset on success.
Part of #520 (improvement 3/7, Tier 1 complete).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The enableCache flag existed since ADR-011 but was never read: every
advisor/LLM call paid full input price. Now:
- ClaudeProvider.generate() sends `system` as a content-block array with
cache_control {type: ephemeral} when enableCache (default on); plain
string when disabled. Anthropic-native provider only (ADR-092 — other
providers untouched).
- cache_creation_input_tokens / cache_read_input_tokens surfaced into
TokenUsage.cacheCreationTokens/cacheReadTokens; totalTokens now counts
the full prompt (input_tokens alone is the uncached remainder).
- CostTracker bills cache reads at 0.1x and writes at 1.25x input rate
instead of ignoring them.
5 new provider tests assert the serialized request shape (cache_control
present when enabled, absent when disabled, no system field when no
prompt) and the usage mapping. 64/64 LLM tests green, tsc clean.
Part of #520 (improvement 2/7, Tier 1).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Fable 5-era models emit large extended-thinking blocks inside task results
and step actions; embedding them verbatim contaminates the 384-dim pattern
vectors in memory.db and degrades HNSW retrieval.
- new scrubReasoningBlocks()/scrubReasoningDeep() (src/shared/reasoning-scrub.ts):
boundary-gated — only well-formed <think|thinking|reasoning|
REASONING_SCRATCHPAD>...</> pairs are removed; prose mentioning a tag
survives; clean input returned byte-identical
- applied at ingestion: TrajectoryTracker startTrajectory task,
recordStep action/result.data/result.error
- applied at distillation: ExperienceReplay.storeExperience task/strategy/
keyActions (insert + embedding text)
- applied at flow-forward: both init-time embedding backfills (legacy rows
cleaned as they get embedded — no memory.db rewrite) and retrieveSimilar
query embeddings (symmetric with stored vectors)
- 15 unit tests; E2E verified against a temp DB: trajectory + experience
rows persisted with all scratchpad blocks stripped
Part of #520 (improvement 1/7, Tier 1).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- aqe memory/agent/task/team/pipeline CLI commands always failed with
"Fleet not initialized": autoInitialize() built kernel+queen in its own
CLIContext while the shared MCP handlers guard on the core-handlers
fleet-state singleton. New adoptExternalFleet() bridges the CLI's
components into that state (idempotent, never clobbers an existing
fleet), restoring MCP-CLI parity. Verified E2E: store/get/delete green.
- devcontainer: node_modules named-volume overlay + npm ci in postCreate
so container installs Linux natives without touching the host's macOS
binaries (takes effect on next rebuild; devcontainer users only)
- docs/setup: drop stale ruflo@3.5.18 pins (CLAUDE.md quick setup,
install-tools.sh) in favor of ruflo@3 + global `ruflo mcp start`
- docs: add Fable 5 / ruflo-parity improvement plan (tracked in #520)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The v3.10.4 publish failed: rvf-pattern-store.test.ts expected RvfPatternStore
but got PatternStore. createPatternStore's #516 change used
`require('../kernel/unified-memory.js')` to get findProjectRoot; esbuild inlines
that in the shipped bundle, but under vitest (the CI publish gate) it throws
"Cannot find module", so the whole RVF block fell through to the plain store —
i.e. RVF pattern store was silently disabled under the test runner.
Fix: extract findProjectRoot/clearProjectRootCache into a lightweight,
sqlite-free module (src/kernel/project-root.ts) and import it STATICALLY from
the RVF hot paths. unified-memory.ts now imports + re-exports them, so existing
`import { findProjectRoot } from '.../unified-memory.js'` sites are unchanged.
A static import avoids both the vitest require-resolution failure and dragging
unified-memory's eager better-sqlite3 load into these modules.
- src/kernel/project-root.ts: new module (fs/path only)
- src/kernel/unified-memory.ts: import + re-export (internal callers keep a binding)
- src/learning/pattern-store.ts: static import; drop require()
- src/integrations/ruvector/shared-rvf-adapter.ts: same (latent same bug)
- rvf-pattern-store.test.ts: pin AQE_PROJECT_ROOT to a hermetic temp root with
.agentic-qe so the RVF branch is taken deterministically (was relying on a
prior test creating ./.agentic-qe in cwd)
Verified locally (native RVF available): rvf-pattern-store 16/16,
find-project-root 4/4, 181 non-sqlite ruvector tests pass, typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
When the parent (e.g. a Claude Code session) exited, aqe-mcp was reparented
to PID 1 and busy-looped at 70-98% CPU, ignoring SIGTERM (only kill -9 worked).
Root cause: on stdin EOF the stdio transport fired its ERROR handler, which
the protocol server routes to its reconnect path. Reconnect re-attached
readline to the already-ended stdin, immediately re-emitted 'close', and
span close → error → reconnect → close at high CPU. The async SIGTERM
handler, starved by the spin and able to hang on an unresolved await, never
reached process.exit — hence "SIGTERM ignored". Non-unref'd worker/daemon
timers meant nothing made the orphan exit on its own either.
Fix (defense-in-depth):
- stdio transport: stdin EOF is now TERMINAL — fire a new onClose handler,
never the error handler, so the reconnect spin can't start. reconnect()
is also a no-op on an ended stream.
- entry.ts: single idempotent shutdown() shared by SIGINT/SIGTERM and stdin
EOF, with an unref()'d 3s watchdog guaranteeing exit even if a graceful
step hangs. A non-TTY stdin 'end'/'close' now triggers shutdown so an
orphan exits when its parent dies.
- unref() the daemon health-check, worker tick, quality-daemon tick, and
git-watcher poll timers so they can't keep an orphaned event loop alive.
Verified: real spawned aqe-mcp now exits ~31ms after stdin EOF (code 0),
vs. lingering indefinitely before. Adds stdio-eof.test.ts covering the
terminal-close behavior and no-reconnect-on-ended-stream guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two root-resolution defects caused a project's learning to be written
outside its own .agentic-qe/.
Defect 1 — findProjectRoot() preferred the TOPMOST .agentic-qe, so an
ancestor store (e.g. ~/.agentic-qe, created by any `aqe` run from $HOME)
hijacked every descendant project. Now prefers the NEAREST .agentic-qe,
mirroring the existing .git nearest-wins logic in the same walk.
Defect 2 — the RVF substrate hardcoded a cwd-relative '.agentic-qe',
ignoring findProjectRoot(). A subfolder cwd (vendored builds, background
workers, `aqe code index docs/`) scattered stray patterns.rvf/brain.rvf
stores. All three sites now resolve via AQE_PROJECT_ROOT ?? findProjectRoot():
- shared-rvf-adapter.ts (relative dataDir anchored to project root;
explicit absolute args still honored)
- shared-rvf-dual-writer.ts (brain.rvf)
- pattern-store.ts (patterns.rvf + existence gate)
Adds find-project-root.test.ts: nearest-wins over ancestor store, env
override precedence, .git fallback, and .agentic-qe over ancestor .git.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Code-intelligence (index/deps/impact/search) required the multi-megabyte
`typescript` compiler at runtime; without it every TS/JS file errored
("TypeScript is required for code analysis") and produced 0 nodes/edges — i.e.
the feature was broken out of the box for the entire JS/TS ecosystem.
Bundle tree-sitter WASM grammars for typescript/tsx/javascript and extract
entities/imports from them instead:
- assets/grammars/tree-sitter-{typescript,tsx,javascript}.wasm (shipped via
package `files: assets/**`).
- treesitter-ts-extractor.ts: extracts functions (incl. async arrows), classes +
methods (with visibility), interfaces, and imports (incl. require()).
- knowledge-graph.ts: TS/JS extraction now prefers tree-sitter (typescript-free),
falling back to the TypeScript compiler API only if tree-sitter is unavailable
AND `typescript` happens to be installed.
- tree-sitter-wasm-parser.ts: export loadWasmGrammar() reusing the proven WASM
init + grammar-path resolution.
Built with tree-sitter-cli 0.26.9 (matching web-tree-sitter ~0.26) via the new
scripts/build-grammars.sh (docker + emscripten/emsdk; reproducible — byte
identical across runs). Grammars built with the pre-0.25 emscripten dylink format
do NOT load with web-tree-sitter 0.26 (getDylinkMetadata error), which is why the
build pins the toolchain.
Verified on a real fresh project with NO typescript installed: code index → 8
nodes / 7 edges, deps → 2 connecting edges, search → 3 results. 232
code-intelligence tests + new extractor unit tests pass.
Note: the pre-existing assets/grammars/*.wasm for python/java/c#/rust/swift were
built with the old toolchain and currently fail to load with web-tree-sitter
0.26 (silently falling back to regex). They can be rebuilt with
`scripts/build-grammars.sh all` — tracked as a follow-up.
Defense-in-depth after the QEReasoningBank init recursion. Pattern search + model
routing in createDomainHandler are optional hints, so they must never block a
tool call. Wrap routeDomainTask() in withTimeout (ROUTE_ADVISORY_TIMEOUT_MS,
default 5s, env-overridable): on timeout the tool proceeds with null routing
(mapToPayload already tolerates null). Also cache the learning engine only AFTER
a successful init, so a failed/slow init no longer serves a half-initialized
engine on the next call.
Adds unit tests for withTimeout (fallback on stall, real value when fast, fallback
on reject). MCP code_index still returns normally.
Root cause: initialize() seeds pretrained patterns via loadPretrainedPatterns() →
storePattern(), and storePattern() ensure-inits with
`if (!this.initialized) await this.initialize()`. Because `this.initialized = true`
was set AFTER seeding, the seed re-entered initialize() without bound — an async
self-recursion that pinned a CPU core and never returned.
Impact: EVERY MCP domain tool (code_index, quality_assess, defect_predict, …)
routes through the advisory pattern-search path (createDomainHandler →
routeDomainTask → searchPatternsForTask → getLearningEngine → AQELearningEngine →
QEReasoningBank.initialize). On a fresh install (patterns seeded for the first
time) that path recursed forever, so all MCP domain tools hung indefinitely while
the CLI — which calls domain services directly — worked. The repo and in-memory
unit tests never hit it (already seeded / bulk-load path).
Fix: set `this.initialized = true` BEFORE loadPretrainedPatterns()/seeding. The
core stores (patternStore + SQLite) are already up by then, so storePattern()
short-circuits instead of recursing.
Verified against a real tarball install: MCP code_index ∞-hang → 0.5s
(6 symbols / 5 relations); quality_assess / defect_predict 0.8s; returns cleanly
(0/0/0) when typescript is absent too. Adds a red-green regression test pinning
the init-order invariant.
Addresses the gaps a devil's-advocate audit surfaced in the #511 fixes:
1. Fail-loud warning (issue request #2): `index` now records how many files
were indexed under a language with no dependency extractor and returns them
as `IndexResult.warnings` (logged + printed by the CLI), so a downstream
0-edge `deps` result is explained rather than silently misleading.
2. Incremental stale state: `indexFile(..., incremental)` now calls
removeFileFromGraph() to delete a file's prior node/entity/outgoing-edge
keys before re-indexing, so `index --incremental` prunes removed imports and
renamed entities instead of leaving phantom dependencies. (Note: within a
single long-lived process the FileReader content cache can still serve stale
text; the CLI re-indexes in a fresh process so this is not user-visible.)
3. MCP parity: the `index-code` task handler emitted `nodesCreated`/`edgesCreated`
but the MCP result mapper reads `symbolsExtracted`/`relationsFound`, so
`code_index` always reported 0/0. Map the KG counts to those fields and pass
`warnings` through. Verified via a real MCP protocol call: filesIndexed 2,
symbolsExtracted 5, relationsFound 4 (was 0/0).
Adds regression tests: unsupported-language warning, incremental stale-edge
removal (fresh-instance, mirroring the CLI), and the MCP field mapping.
Known limitation (documented, not changed here): CLI persists the graph under
the 'qe-kernel' namespace and MCP under 'mcp-tools', so a graph indexed by one
tool is not read by the other. MCP exposes only `code_index` (no deps/search),
so this has no MCP-internal reader; reconciling the namespaces is entangled with
contradictory init/test-generation expectations and needs its own change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
indexCode() stored retrievable content with a {namespace} store option, but
the key is already fully namespace-qualified and every read path
(semanticSearch/findSimilar/exact/fuzzy) plus the storeVector() embedding write
use the default namespace. On a real backend that honors the option, the
content written with {namespace} could not be read back without it (issue #491
Bug 2), so each matched vector's content fetch returned undefined and the result
was silently dropped — search returned 0 results for obviously-present terms.
Fix: drop the redundant {namespace} store option on the content write so it
round-trips with the reads.
Verified via CLI: `code search "calculator compute"` now returns the indexed
files (was 0). Adds a regression test using a namespace-honoring mock backend
(the existing mock ignored the option, which is why it never caught this); the
test fails with the option reintroduced and passes without it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per request: aqe init now installs the cross-platform Node hook shim into each
user project and routes hooks through it (instead of `npx agentic-qe hooks`).
- package.json: add `.claude/hooks` to `files` so aqe-hook.cjs ships in the npm
tarball (verified via `npm pack --dry-run`).
- 07-hooks.ts: new installHookShim() copies aqe-hook.cjs into the project's
.claude/hooks/ (source: node_modules/agentic-qe, then the package's own copy
resolved relative to the compiled module — works for npx too).
- 07-hooks.ts + init-wizard-hooks.ts: generated hooks now run
`node "${CLAUDE_PROJECT_DIR:-.}/.claude/hooks/aqe-hook.cjs" <cmd> ...`
(README/help examples left as `npx agentic-qe hooks` for manual use).
- aqe-hook.cjs: added an npx fallback (`npx -y --prefer-offline agentic-qe`)
so npx-only installs with no project-local bundle still work; disable with
AQE_HOOK_NPX=0 (tests use this for determinism). Header/scope updated (now a
shipped, init-copied file, not repo-internal).
Migration: isAqeHookEntry already matches the new `aqe-hook.cjs` command (via
the `\baqe\b` pattern) AND the old `npx agentic-qe hooks` form, so re-running
`aqe init` replaces a user's existing hooks with the shim form — no duplicates.
Verified end-to-end (real `aqe init --auto` in a temp project):
- aqe-hook.cjs copied into .claude/hooks/ ("Installed aqe-hook.cjs (copied)")
- all 11 generated hooks use `node .../aqe-hook.cjs <cmd>` (0 npx commands)
- post-task retains the #508 fix (--agent/--success true/--description)
- isAqeHookEntry matches new+old forms, not unrelated user hooks
- shim 7/7, settings-merge 17/17, task-hooks 5/5 tests pass; typecheck clean
A .sh shim is the least portable option — it forces a .cjs Windows twin and a
node_modules/.bin shell-wrapper spawn. Node is already a hard dependency of AQE
(CLI is Node >=20), so a single .cjs runs identically on Windows/macOS/Linux.
aqe-hook.cjs resolves a project-local bundle and runs it via process.execPath
(this node) — no shell, no .bin/.cmd wrapper:
1. node_modules/agentic-qe/dist/cli/bundle.js (installed dep — user case)
2. dist/cli/bundle.js (this source repo)
else no-op (exit 0). Swallows stderr; emits only the top-level JSON object.
Repo .claude/settings.json rewired: all 10 bundle hooks now invoke
. Deleted
aqe-hook.sh. Tests rewritten to run the .cjs via node; 7/7 pass incl. installed-
dep precedence + stderr-swallow. This removes the Windows blocker that stood
against ever shipping the shim to users.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The shim hardcoded ${PROJECT}/dist/cli/bundle.js, which only exists in this
source repo — if ever copied into an installed project it would silently no-op.
Now resolves a PROJECT-LOCAL runner in priority order:
1. node_modules/.bin/aqe (installed project — the user case)
2. dist/cli/bundle.js (this source repo)
else no-op (exit 0, never block a turn)
No global-PATH / npx fallback by design (version drift + npx-per-fire crash).
Clarified in the header that the shim is REPO-INTERNAL: it is not in the npm
`files` list and `aqe init` does not copy it — user hooks are generated as
`npx agentic-qe hooks <cmd>` (cross-platform, npx-friendly). Portability here is
defense-in-depth, not a shipping decision.
Adds a regression test for branch-1 resolution + precedence (node_modules/.bin/aqe
preferred over dist). 6/6 shim tests pass.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reported by @ciprianmelian. ADR-094 moved dream cycles to the kernel-side
DreamScheduler, but the dream-scheduler:hook-state kv row that SessionStart
reads is only INCREMENTED by the post-task hook (experienceCount++) and was
never reset by the entity that actually dreams. Confirmed in source: the kernel
scheduler had ZERO references to that key (it persists its own
'dream-scheduler:state' instead). Result: SessionStart perpetually showed
lastDreamTime:null and a pendingExperiences counter that only grew (e.g.
11 -> 1747 -> 6489), reading like a stuck scheduler while the daemon dreamt fine.
Fix: after a successful executeDream cycle, reconcileHookState() stamps the
truth on the SAME key + DEFAULT namespace the hook path uses:
{ lastDreamTime: now, experienceCount: 0, totalDreamsThisSession: prev+1,
sessionStartTime: preserved }. Best-effort (never breaks a dream cycle). Key
is a local literal mirroring DREAM_STATE_KEY to avoid a learning->cli layering
import.
Verified: new unit test seeds the stale row (experienceCount: 2720,
lastDreamTime: null), triggers a dream, asserts the row becomes
experienceCount:0, lastDreamTime: <iso string>, totalDreamsThisSession:1,
sessionStartTime preserved. 39 dream-scheduler tests pass; typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Reported by @ciprianmelian. The aqe-init-generated PostToolUse hook ran
`post-task --task-id ... --success --json`, but --success was declared
`<bool>` (value REQUIRED), so Commander consumed --json as the success value:
options.success='--json' -> parsed false -> EVERY qe_trajectories /
captured_experiences row got success=0, the --json branch was swallowed, and no
--agent meant agent='unknown'. Net: Task trajectories were unpromotable
(distillation groups by agent/domain/success), so pattern count stayed pinned
near zero. (Confirmed in source; same class as the learning gaps #510 hardened.)
Fix (two layers):
1. Parser hardening — `--success <bool>` -> `--success [bool]` (optional value)
in task-hooks/command-hooks/routing-hooks, so a bare --success yields boolean
true and never eats the next token. Backward compatible: --success true/false
and the 'true' default all still parse identically.
2. Generated templates (07-hooks.ts + init-wizard-hooks.ts): post-task now emits
`--agent "$TOOL_INPUT_subagent_type" --success true --description
"$TOOL_INPUT_prompt" --json` (real agent attribution + explicit success +
pre-task bridge), and post-command emits explicit `--success true`.
Verified: isolated Commander proof — OLD <bool>: success="--json" (false);
NEW [bool]: success=true, --json fires. Plus 3 regression tests driving the EXACT
broken argv through registerTaskHooks: bare --success -> success=true, explicit
--success false -> false, --agent flows through. 5/5 task-hook tests pass
(incl. existing #449); typecheck clean. (Full post-task CLI run OOMs in Codespace
per CLAUDE.md — the parsing layer is the bug and is proven.)
NOTE for existing installs: re-run `aqe init` (or hand-edit the PostToolUse hook
in .claude/settings.json) to pick up the corrected command.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Pre-existing bug surfaced by the qe-devils-advocate audit and confirmed:
checkDeprecation() reads (pattern as PatternWithDeprecation).consecutive_failures
to trigger the 'failures' deprecation reason, but rowToPattern() built the
QEPattern WITHOUT that column (and without deprecated_at), so getPattern()
always surfaced undefined->0 and the 'failures' branch NEVER fired — a pattern
that kept failing was never deprecated for failures.
Fix (surgical): rowToPattern now surfaces consecutive_failures and deprecated_at
(added to the PatternRow interface; returns a typed PatternWithDeprecation). No
thresholds or deprecation logic changed.
Verified: new tests assert a pattern with consecutive_failures==3 (== default
threshold), not stale, confidence above min, deprecates with reason 'failures';
below threshold does not. Regression-proven (reverting rowToPattern fails the
new test). 12 EWC tests pass; typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audit (qe-devils-advocate) verdict: 10/10 items have real committed code + real
passing tests at claimed counts; production changes (#2/#6/#7) genuinely wired
into live paths; no real-store mutation; #8 honestly UNWIRED. Three fair
discrepancies, now corrected:
1. #2 had one TAUTOLOGICAL test (asserted (1/3)*0.4 > (1/11)*0.4 — arithmetic on
literals, not a code call). Rewritten to score a matching vs non-matching
agent through calculateAgentScores and assert the delta equals the CAPPED
domain term (~0.1333), which fails if the cap is removed. 4/4 still pass.
2. #2 status row softened to VERIFIED (PARTIAL): +5.6pp shipped, but the
structural Q-maturity / fixed-perf-term ceiling on confidence remains.
3. #10 status row corrected to PARTIAL: only the smoke contract was delivered;
the original multi-bundle 'subset packaging' scope is deferred.
No production code changed by this commit. Side-findings re-confirmed real
(dead consecutive_failures branch, MCP [object Object] error, RVF/SQLite drift).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ports ruflo ADR-077: seed the pattern store from the repo's OWN git history
(commits = success trajectories) through the SAME live-learning code path, so
a fresh project starts with a non-empty, project-specific pattern store instead
of only the generic PRETRAINED_PATTERNS.
scripts/pretrain-from-history.ts (npm run pretrain:history):
- git log --no-merges -> per commit: domain = deriveTaskType(subject),
success = subject !startsWith 'revert' (a revert is a failure signal);
- records each via persistTaskOutcome (the live captured_experiences +
experience_applications + Q-update path, honoring AQE_PROJECT_ROOT);
- then consolidateExperiencesToPatterns() distills them into qe_patterns.
- flags: --limit N (default 500), --dry-run (classify+histogram, write nothing),
--verbose. Extracted a pure classifyCommit() helper (19 unit tests).
Correctness fix found during verification (in the script, not prod): a brand-new
store only has core schema, so persistTaskOutcome failed with
'no such table: experience_applications'. The script now calls the canonical
ensureAllBrainTables() + initializeExperienceCapture() before recording — the
same tables production creates lazily.
Verified against a THROWAWAY AQE_PROJECT_ROOT temp store (data protection):
--limit 150 -> captured_experiences 0->150, qe_patterns 0->4, exit 0;
--dry-run -> nothing recorded (captured_experiences stays 0);
real .agentic-qe/*.rvf byte-identical (sha256) before/after; 19 tests pass.
Side-finding: AQE's real embedding (ONNX) costs ~2.3s/commit here vs ruflo's
~3ms/trajectory, so bootstrapping 500 commits takes minutes — logged as a perf
follow-up (embedding backend / batching).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ports ruflo's perf-M4 (sign-random-projection + Hamming popcount). Adds
src/shared/utils/rabitq.ts: signSignature (384-bit packed into 12 Uint32
words, 48 bytes), hammingDistance (SWAR popcount), hammingSimilarity, and
topKBySignature (Hamming prefilter gated at N>=100 + exact-cosine rerank of a
recall-aware pool, so output equals exact topK when the true top-k fall in the
pool). 16 tests + scripts/benchmark-rabitq.ts (npm run benchmark:rabitq).
HONEST FINDING (verified, does NOT match ruflo's optimistic 2.7x/10.9x):
N=1000: speedup 1.31x, recall@10 100%, index 1.46MB -> 47KB (32x smaller)
N=5000: speedup 1.18x, recall@10 99.9%, 7.32MB -> 234KB (32x smaller)
The 32x INDEX-MEMORY reduction is real and unconditional. The SPEEDUP is only
~1.2-1.5x in JS, not ruflo's figure: a 1-bit sign signature in 384-dim is
coarse, so preserving recall@10 >= 0.95 needs a large exact-rerank pool
(~40-55% of N), which eats most of the prefilter win. ruflo's big win came
from replacing a LINEAR scan; AQE's main search is already native HNSW
(sublinear), so that win doesn't transfer. Tests pin this tradeoff (recall==1.0
with a sufficient pool; small pool sacrifices recall) so it can't be
mis-sold later.
NOT INTEGRATED — no zero-regression consumer exists:
- unified-memory-hnsw search() IS the ranking (no separate exact rerank to
prefilter), so Hamming-prefiltering its beam would drop true neighbors;
- coherence-gate-energy O(N^2) SUM and k-means clustering are numerically
sensitive (Hamming would change results).
Committed as a tested, benchmarked, ready-to-use utility (real memory lever for
patterns.rvf and a future prefilter->exact-rerank consumer), not wired into any
production path. No production code or .agentic-qe stores touched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ports ruflo's hook-shim discipline. AQE's project hooks ran
`node dist/cli/bundle.js hooks <cmd> --json` directly, which printed init
diagnostics to STDOUT before the JSON on every SessionStart/UserPromptSubmit:
[hooks] CoherenceService initialized with WASM engines
[RvfPatternStore] Initialized: .agentic-qe/patterns.rvf (dim=384)
[RVF] Migration adapter active at stage 2 (dual-sqlite)
[hooks] System initialized
polluting the Claude Code transcript on every turn, and a crash/slow init could
surface a hook error mid-turn.
New .claude/hooks/aqe-hook.sh wraps every bundle hook with the ruflo contract:
* prefer the LOCAL built bundle (no npx / network);
* no-op if the bundle isn't built (never block a turn);
* swallow stderr;
* emit ONLY the top-level JSON object on stdout (sed '/^{/,/^}/p' drops both
leading init noise AND async daemon log lines printed after the JSON, so the
--json contract stays parseable);
* ALWAYS exit 0.
All 10 bundle hook commands in .claude/settings.json rewired to the shim
(diff touches only those 10 command strings).
Verified with real commands: route/session-start/guard via the exact settings
invocation -> exit 0, 0 noise lines, valid JSON; missing-bundle and crashing-
bundle -> exit 0, empty stdout, no stderr leak. 5 regression tests (resilience +
deterministic noise-stripping via a fake bundle) pass in ~120ms.
Note: Windows .cjs parity (ruflo also ships one) is a separate follow-up — the
entire AQE hook config is POSIX `sh -c`, so a .cjs twin needs the settings
rewrite too. Observed side concern: the route/session-start hooks cold-start the
full system (~30-60s) on each fire — logged for a perf follow-up.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
AQE's Phase-1 consolidation merged near-duplicate experiences by cosine
similarity, but two experiences with the SAME context and OPPOSITE outcomes
(large quality delta) are a conflict, not a duplicate. Merging silently
absorbed the conflict; worse, a *used* low-quality contradictory experience was
never merged (merge requires application_count===0) and stayed in retrieval,
poisoning recall.
Now, when similarity >= contradictionSimilarityThreshold (0.85) AND the quality
delta >= contradictionQualityDelta (0.4), the pair is treated as a contradiction:
the lower-quality loser is SUPPRESSED from retrieval (consolidated_into =
'contradicted', excluded by the IS NULL filter) instead of merged — even if it
was applied. The survivor is NOT boosted (a conflict is not a duplicate). The
action is logged distinctly ('contradiction'). New count surfaced as
ConsolidationResult.contradicted. Both thresholds are configurable.
PRODUCTION IMPACT: affects the dream/consolidation merge phase. Genuine
near-duplicates still merge exactly as before; only same-context/opposite-outcome
pairs are newly suppressed (soft, reversible — consolidated_into can be cleared).
Verified with 6 tests against an in-memory DB matching the production
captured_experiences schema (suppress loser, suppress even-when-applied, no
survivor boost, near-duplicate still merges, dissimilar pair untouched, distinct
log action). 13 total consolidation tests pass; typecheck clean.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Confidence decay (applyConfidenceDecay) and stale/age deprecation
(checkDeprecation) were uniform — a proven high-success pattern was forgotten
at the same rate as noise, so an influx of new low-value patterns could crowd
out historically valuable ones (catastrophic forgetting).
Inspired by Elastic Weight Consolidation, high-IMPORTANCE patterns now resist
forgetting, weighted by an importance proxy = clamped success_rate gated on
usage_count >= EWC_MIN_USAGE_FOR_PROTECTION (recency already implied: only
unused patterns decay/go stale). Honest framing (as in ruflo's EWC adapter):
this is a heuristic, NOT true Fisher information — no model gradients exist here.
- Decay: per-row factor shielded by EWC_DECAY_PROTECTION*importance. A
fully-proven pattern decays up to 10x slower; importance-0 patterns decay
EXACTLY as before (no regression — locked in by test).
- Stale: effective window = staleDays * (1 + importance*EWC_STALE_PROTECTION_BONUS),
so a proven pattern unused 30d isn't deprecated like noise.
Protection covers ONLY stale/age/decay forgetting. A pattern that drops below
minActiveConfidence (or starts failing) is STILL deprecated — we blend
retention, we don't permanently shield dead weight. All three EWC_* knobs are
exported constants; set to 0 to restore exact prior behavior (reversible).
PRODUCTION IMPACT: affects the dream/consolidation lifecycle. Low/zero-importance
patterns behave identically; high-success patterns persist longer. Verified with
10 tests against an in-memory DB matching the production qe_patterns schema
(decay deltas, stale window, no-regression, dead-weight still removed). 46 total
lifecycle tests pass.
Side-finding (pre-existing, NOT this change): checkDeprecation reads
consecutive_failures via a cast, but rowToPattern doesn't surface that column,
so the 'failures' deprecation branch never fires through getPattern. Logged for
a dedicated fix.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Routing confidence (QERoutingResult.confidence) is the top agent's combined
static score from calculateAgentScores. The domain term was
(domainMatch / detectedDomains.length) * 0.4, but AQE's domain detector emits
7-12 domains per task, so a perfect single-domain match scored (1/11)*0.4 ~ 0.04
— relevance washed out by detector breadth, leaving confidence dominated by the
fixed performance term.
Fix: cap the denominator at DOMAIN_DENOM_CAP (3), modeling the few truly-relevant
domains a task has. Precise detection (1-2 domains) is unchanged; broad detection
no longer dilutes a real match below usefulness. Proportionality preserved.
Tunable via the constant + the existing routingWeights.similarity.
PRODUCTION IMPACT: shifts every routing decision's confidence telemetry upward
for broad-domain tasks and strengthens domain relevance vs the fixed performance
term in ranking. Reversible (raise DOMAIN_DENOM_CAP toward old behavior).
Verified with real `aqe hooks pre-task` routes (not mocks):
broad multi-domain task: 27.7% -> 33.3% (+5.6pp); narrow tasks unchanged
(test 37.3%, security 30.3%); recommendations/ranking preserved.
77 existing routing tests pass; +4 regression tests lock in de-dilution.
NOTE: ruflo ADR-142 (per-bucket Beta priors) did NOT apply — AQE confidence is
not a Beta-bandit; root cause was this dilution. See docs/ruflo-adoption-plan.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Audited AQE routing against two router bugs ruflo fixed 2026-05-29:
- #2229 stale route-cache: NOT PRESENT — Q-values read live from SQLite
(qe-reasoning-bank buildQValueLookup) and in-memory qTable; no cached
routing decision exists to go stale, so no invalidation needed.
- #2239 state-encoder truncation: NOT PRESENT — state keys are plain
string joins (agent-routing buildRoutingStateKey: taskType|priority|domain),
no bit-folded quantized vector, no mask. Distinct keyword tasks map to
distinct states.
Adds a guard test locking in non-collapsing state encoding so a future
refactor toward a bit-folding encoder fails here. 4 tests passing.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Two follow-up fixes after re-running the broader test suite (2064 tests):
1. tests/unit/shared/llm/router/hybrid-router.test.ts:619 — the
pre-existing test "should not retry non-retryable errors" was
asserting the OLD short-circuit-on-non-retryable behavior that
Fix#3 in the audit corrected. Updated to assert the new (correct)
semantics: non-retryable errors don't re-try the SAME provider but
DO fall through to the NEXT provider in the fallback chain.
Renamed accordingly. Both providers fail non-retryable so the
request still rejects, preserving the original "rejects" assertion.
2. README.md "LLM Providers" section was under-documented relative to
the wired implementation. Updated to:
- List all 7 ExtendedProviderType providers with their env var names
(including the GOOGLE_API_KEY alias added in Phase 1)
- Document `aqe llm config --set` persistence path
- Document the AQE_LLM_ROUTER_DISABLED kill-switch
- Mention that apiKey can't be saved via CLI (env-only)
- Drop Groq from the table since it's not actually in the supported
provider list
Verification
- npx tsc --noEmit: clean
- 2063/2063 unit + integration tests pass (1 skipped, no failures)
- Includes 38/38 hybrid-router tests after the assertion update
A post-Phase-4 devil's-advocate audit identified 10 issues. All addressed
with code fixes + tests. Some were real bugs (silent billing regression,
2 unwired MCP tools, 2 HybridRouter routing bugs); others were robustness
improvements (split singletons, silent errors, decorative type fields).
Findings → fixes (severity tagged in audit report):
1. CRITICAL — silent billing regression on upgrade
Users with ANTHROPIC_API_KEY in env would silently start paying for
Claude API calls on every Fleet operation. The CHANGELOG's "opt out
via code changes" wasn't sufficient — needed an env-only escape.
Fix: AQE_LLM_ROUTER_DISABLED=1 env kill-switch honored in both
QEKernelImpl._initializeLLMRouter and src/mcp/tools/base.ts
getSharedLLMRouter. Truthy values disable; 'false'/'0'/'no'/'off'/''
pass. Unified parse rule between kernel and MCP base.
2. HIGH — 2 unwired MCP standalone tools (audit said 3; QualityCriteria
isn't LLM-aware so excluded)
src/mcp/tools/test-generation/generate.ts:82 was calling
createTestGeneratorService(memory) without router. Same for
visual-accessibility/index.ts:177. Both now use the
createXxxServiceWithDependencies factories with { memory, llmRouter }.
3. CRITICAL — E2E only tested manual routing mode, not production
Production default is rule-based. Adding two new E2E tests for
rule-based mode SURFACED two real bugs in HybridRouter that I fixed:
3a. executeWithFallback hardcoded ['claude','openai','ollama'] as the
only fallback providers; gemini/openrouter/azure-openai/bedrock
users had no fallback even though ProviderManager can construct
them. Plus the user's defaultProvider wasn't auto-included in the
chain. Both fixed at src/shared/llm/router/hybrid-router.ts:612.
3b. A non-retryable error from one provider (e.g. Claude "invalid
x-api-key") aborted the entire fallback loop via `throw error`,
defeating the whole point of a fallback chain. Fixed: skip the
inter-attempt delay for non-retryable but continue to the next
fallback entry.
4. HIGH — MCPToolContext.llmRouter field was decorative
Added the field but no code populated it. Every getLLMRouter(context)
call fell through to getSharedLLMRouter().
Fix: added llmRouter and memory to BaseMCPTool.invoke() options;
qe-tool-bridge.ts and registry.ts now explicitly forward the shared
singleton via { llmRouter } at invoke time.
5. HIGH — 14 of 15 wired services had only STRUCTURAL verification, not
behavioral (only TestExecutor had a chat() round-trip test)
Fix: new tests/integration/llm-router-behavioral.test.ts asserts
isLLMAnalysisAvailable()===true (the exact gate the audit named) for
all 14 services + 2 full behavioral round-trips. 16 tests total.
6. HIGH — MCP and kernel built SEPARATE HybridRouter instances
Different cost trackers, caches, circuit breakers per side.
routing_economics/routing_metrics would only see half the traffic.
Fix: kernel.ts._initializeLLMRouter calls setSharedLLMRouter(router)
on init so both paths share one instance. kernel.dispose() calls
resetSharedLLMRouter() so the next kernel doesn't inherit the
previous instance.
7. HIGH — Silent error swallowing in _initializeLLMRouter
console.warn'd then continued without LLM, with no signal for
monitoring. A misconfigured provider would silently degrade with no
alert.
Fix: now publishes kernel.llm-router.init-failed (caught error) and
kernel.llm-router.init-no-provider (enabled=true but nothing
available) events on the event bus. Test confirms event delivery.
8. MEDIUM — Service count mismatch in docs (ADR/CHANGELOG said "14
services / 12 domains"; test enumerates 15 services / 11
LLM-enhanced domains)
Fix: ADR-043 addendum + CHANGELOG corrected to 15/11. Added
explicit service list and noted the 2 non-LLM domains
(enterprise-integration, coordination).
9. MEDIUM — aqe llm config --set apiKey=... silently dropped the key
stripApiKeys() returned a successfully-saved file without the key,
user thought it was saved.
Fix: CLI updateConfig() now throws a clear error pointing to env
vars BEFORE calling save. stripApiKeys also now logs a console.warn
when it strips anything, so programmatic callers see the strip.
10. LOW — Env vs disk precedence was counterintuitive and undocumented
Force-enable-on-env overrides explicit `enabled: false` on disk.
Fix: doc comment in config-store.applyEnvProviderDetection +
ADR-043 "Precedence & env-var conventions" section explains.
Verification
- npx tsc --noEmit: clean
- 191 unit + integration tests pass (16 new behavioral tests for #5,
17 wiring tests, gemini provider, config-store, llm-router-service,
kernel-llm-router with 17 tests including all kill-switch cases,
CLI persistence with 28 tests)
- 5 real-provider E2E tests pass (Gemini 2.5 Flash, OpenAI gpt-4o-mini,
OpenRouter, plus 2 rule-based-mode tests that previously failed and
now pass with the HybridRouter fallback fixes)
Refs ADR-043 audit findings (2026-05-21)
Builds on 94033e3 (Phase 1), be1e60d (Phase 2), c206c81 (Phase 3+4)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the foundation for ADR-043's HybridRouter into the kernel boot
sequence so domain plugins can finally receive an llmRouter dependency.
Phase 1 lands the kernel/CLI/provider changes; Phase 2 will plumb the
router through all 13 domains.
Background: the LLM-enhancement code paths added in ADR-051
(isLLMAnalysisAvailable() across 14 services) were unreachable because
no construction site ever passed an llmRouter — services accepted the
optional dependency but no caller injected it. ProviderManager +
HybridRouter were only instantiated from the `aqe llm advise` CLI
subcommand.
Changes
- src/shared/llm/providers/gemini.ts: read GOOGLE_API_KEY as a third
fallback after GOOGLE_AI_API_KEY and GEMINI_API_KEY, so users with
the conventional Google key name don't need to alias it themselves.
- src/shared/llm/router/config-store.ts (new): load/save
.agentic-qe/llm-config.json with env-detection layering
(defaults < disk < env < explicit override). Strips apiKey fields
defensively before persisting. Filters out providers the
ProviderManager doesn't yet construct (onnx) to keep boot logs
clean.
- src/shared/llm/llm-router-service.ts (new): builds a HybridRouter
from the loaded config, picks primary/fallbacks from the available
set honoring the user's defaultProvider when possible. Accepts an
injected ProviderManager for tests.
- src/kernel/interfaces.ts: extends QEKernel with llmRouter? and
KernelConfig with llmRouter.enabled ('auto'|true|false) +
configOverride + providerManager test-injection.
- src/kernel/kernel.ts: kernel-singleton llmRouter built during
initialize(), exposed via kernel.llmRouter, forwarded to all 13
DOMAIN_FACTORIES as a 4th arg. Phase 2 wires each factory through
to its coordinator/services; for now the wrappers accept-and-discard.
- src/cli/commands/llm-router.ts: replaces the
`// would be persisted in real implementation` stub with real
read/write through the config-store. `aqe llm config --set` now
survives a process restart. Adds AQE_CONFIG_ROOT env var as a test
seam so tests don't mutate the real .agentic-qe/.
Tests (110 total, all passing)
- gemini.test.ts: +3 new tests for each env-var alias, existing
tests updated to clear all three names
- config-store.test.ts (new, 29 tests): load/save/merge precedence,
env detection, secret stripping, atomicity
- llm-router-service.test.ts (new, 10 tests): pickEnabledProviders,
pickPrimaryAndFallbacks, end-to-end with injected mock provider
- kernel-llm-router.test.ts (new, 4 tests): kernel boot with router
enabled=false / auto-no-providers / injected mock, chat() round-trip
- llm-router.test.ts (CLI): +2 persistence-to-disk tests, env-isolated
- tests/mocks/index.ts: shared createMockLLMProvider() helper
Verification
- npx tsc --noEmit: clean
- 110 tests pass; no regressions in existing kernel.test.ts (34/34)
- Stderr noise from PersistentSONAEngine init is pre-existing and
unrelated to this change.
What this commit does NOT do
- Does NOT change behavior for any user yet — DOMAIN_FACTORIES
wrappers still discard the llmRouter param. Phase 2 lights up the
domain services.
Refs ADR-043 (vendor-independent LLM), ADR-051 (LLM-enhanced services)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes 11 findings from a post-merge audit of the external embedder endpoint
implementation. The "no silent hash fallback" correctness promise was being
contradicted by two callers; the connect timeout was actually an idle timeout;
the identity probe could be bypassed in several paths; and identity drift
across runs was undetectable. All hardened.
Critical/High fixes:
- sqlite-persistence backfill no longer falls back to hash embeddings when the
endpoint is configured; re-raises to protect the HNSW index from non-comparable
vectors (was poisoning silently on every endpoint blip).
- qe-reasoning-bank.embed() propagates the throw under isUsingEndpoint() instead
of silently substituting a hash vector.
- EmbedderEndpointClient.embed() now gates on probe() — lazy probe on first use
with in-flight dedup so 20+ concurrent embeds share one probe.
- Circuit breaker recovery (window expires) clears cached identity so the next
embed re-probes — endpoint restarts often coincide with model swaps.
- True TCP-connect timeout enforced at the socket layer (was req.setTimeout,
which is an idle-after-write timeout — SYN blackholes hung ~75s instead of 5s).
- Endpoint identity persisted to memory.db kv_store (_system namespace) — cross-
run fingerprint drift now surfaces as a loud warning on next boot.
Medium fixes:
- encoding_format: 'float' pinned in request; base64 responses fail loud.
- TLS knobs (ca, cert, key, rejectUnauthorized, servername) passthrough for
self-hosted https endpoints without NODE_TLS_REJECT_UNAUTHORIZED=0.
- URL userinfo stripped + warned at parse; redacted from all error messages
and logs.
- Embedding cache key namespaced by mode (endpoint:<fingerprint>:<text> vs
inproc:<text>) — cross-mode collisions structurally impossible.
Tests:
- 33 unit tests pass (was 23): adds probe-gate, breaker re-probe, connect
timeout, DNS failure, concurrent embed dedup, base64 fail-loud, URL
redaction, encoding_format wire format.
- 6 live integration tests against real llama-server (all-MiniLM-L6-v2.Q8_0.gguf):
probe, single embed, batch, lazy probe gate, dim mismatch, 50-input batch.
Replaces the prior "9ms in-process fake" with honest numbers (cold 15ms,
warm 2ms against localhost llama.cpp).
- New tests/integration/embedder-endpoint-llamacpp.test.ts is gated on
AQE_LLAMA_EMBED_URL so it skips cleanly in CI without the binary.
ADR-097 updated with the per-caller audit table (12 writers verified
non-poisoning) and "drop-in for 6 providers" claim softened to "verified
against llama-server; OpenAI shape compat expected but unverified" per
provider.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add opt-in OpenAI-compatible /v1/embeddings client to eliminate duplicate
in-process model loads in ruflo/ruvector co-deployments and the per-hook
cold-load cost in fresh agentic-qe CLI processes.
When EmbeddingConfig.endpoint (or AQE_EMBEDDER_ENDPOINT) is set, the
@huggingface/transformers import is bypassed entirely and feature-extraction
routes through the remote service. Endpoint unset = today's behavior.
Wire format is OpenAI's /v1/embeddings so any existing embedder server (TEI,
vLLM, llama.cpp, Ollama, LocalAI, LM Studio, OpenAI) works as a drop-in,
not just bespoke ruvector servers. Transport is HTTP or HTTP-over-Unix-socket.
Errors hard-fail with a circuit breaker (3 failures / 60s window) — no
silent hash-embedding fallback, which would poison the HNSW index by mixing
non-comparable vectors. A startup probe asserts dim === 384 and computes a
stable identity fingerprint that catches model/quantization drift at the
boundary.
- src/learning/embedder-endpoint-client.ts: OpenAI client with keep-alive
agent, 5s/30s timeouts, Bearer auth from env, circuit breaker, probe
- src/learning/real-embeddings.ts: endpoint branch placed strictly before
the dynamic transformers import so the cold-load fix holds at runtime;
new isUsingEndpoint() / getEndpointIdentity() observability helpers
- 23 new unit tests (real http.createServer fixtures, no http mocking)
- End-to-end smoke: 9ms for import + init + probe + embed, transformers
never appears in the loaded-module list
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-096 fix#3 (Option A): the previous 4-dim state_key
taskType|priority|domain|complexityBucket
fragmented semantically identical tasks across cells. complexityBucket
was derived from raw description length (round(min(len/200,1)*10)) —
so a 55-char and 130-char phrasing of the same task landed in
different Q-rows, and QWEIGHT_RAMP_VISITS=20 was rarely reachable
per cell even after the fix #1/#2 loop closure.
State key is now 3-dim:
taskType|priority|domain
Same change applied at both ends of the loop:
- buildRoutingStateKey (agent-routing.ts) — parameter and key shape
- updateHookRouterQValue (hooks-dream-learning.ts) — state_key SQL
- qe-reasoning-bank.ts reader-side call
- task-hooks.ts post-task writer call
- routing-hooks.ts post-route writer call (new in fix#1)
`deriveComplexityBucket` stays exported (marked @deprecated for
state_key use) — the pre-task bridge payload still carries
`complexityBucket` for kv_store schema compatibility and any
downstream telemetry. `TaskBridgePayload` interface unchanged.
Migration: existing rl_q_values rows written under the previous
4-dim key are orphaned. Field reports show real installs have at
most ~2 such rows (the loop was broken; no real Q-data accumulated),
so we accept the orphans rather than ship a migration. If a future
install carries substantial 4-dim data, a one-shot script that
strips the trailing |N segment is straightforward.
Test contract extended:
- agent-routing-exploration.test.ts pins the 3-segment key shape
- route-q-loop-closure.test.ts asserts complexityBucket does NOT
leak into updateHookRouterQValue payloads (writer-side check) and
that bridge.complexityBucket is ignored even when present.
ADR-096 updated: complexity-bucket promoted from Open Questions to
Decision (Option A). Open Questions now lists "re-introducing a
coarse complexity tier if telemetry shows the 3-dim key conflating
decisions that benefit from differentiation" as a wait-and-measure
follow-up rather than a planned change.
Refs: #499
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The route hook (UserPromptSubmit) reads `rl_q_values` to produce qWeight,
but the only writer was task-hooks `post-task` gated on `outcome.bridge`.
On real projects, `routing_outcomes` reached 139 rows while `rl_q_values`
stayed at 2 — the route surface that displays qWeight had nothing to read,
so `qWeight: 0` was structurally pinned.
Two complementary producer additions, ADR-096:
Fix#1 — post-route trains rl_q_values
When the Stop hook resolves a routing_outcomes sentinel, derive a
state_key from the sentinel's `task_json.description` using the
same exported helpers the route hook used (`deriveTaskType`,
`deriveComplexityBucket`, `detectQEDomains`) and call
`updateHookRouterQValue`. Writer and reader address byte-identical
rows. Best-effort + try/catch: Stop hooks must never crash the host.
Fix#2 — post-task fires even when bridge is absent
Drop the `if (outcome.bridge)` gate. When bridge is missing,
derive state from `options.description` instead. Adds a new
`--description` option to `aqe hooks post-task` so the PostToolUse
hook command can pass `tool_input.description` for Task tools.
Skip the update only when neither bridge nor description is
available (would pollute an "unknown" sentinel state).
Tests: tests/unit/cli/commands/route-q-loop-closure.test.ts pins
the producer/consumer state_key alignment contract — if a future
refactor desynchronizes the three call sites
(routing-hooks.ts post-route, task-hooks.ts post-task,
qe-reasoning-bank.ts:530), assertions fail the build.
Closes the Open Question flagged in ADR-095. The deriveComplexityBucket
fragmentation Jordi noted is acknowledged in ADR-096's Open Questions
and tracked for a follow-up — orthogonal to closing the loop.
Refs: #499
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the qe-reasoning-bank split: the original 622-line file ran
~43 tests in a single fork, most of which spun up a full
AQELearningEngine (transformer + HNSW + WASM coherence) via
beforeEach. Distributing across three sequential forks (pool=forks,
maxForks=1, fileParallelism=false) caps per-fork heap.
- _aqe-engine-test-helpers.ts (NEW, not a test file)
Shared createMockMemoryBackend() and createMockEventBus() factories
pulled out of the original so the splits don't duplicate. Underscore
prefix keeps it out of vitest's test-file include glob.
- aqe-learning-engine-factory.test.ts (NEW, 4 tests, no init)
Factory Functions + DEFAULT_ENGINE_CONFIG. Tests only call the
constructor — no engine.initialize() — so transformer / HNSW /
WASM aren't loaded. Safe to include in test:unit:fast.
- aqe-learning-engine-patterns.test.ts (NEW, 15 tests, full engine)
The three heaviest sub-describes: Pattern Learning, Task Tracking,
Experience Capture Integration. Each test runs a full engine init.
Excluded from test:unit:fast until step 3's dispose hooks are
proven in CI.
- aqe-learning-engine.test.ts (SLIMMED, ~21 tests, full engine)
Init / Status / Task Routing / Model Routing / Codebase Analysis /
Guidance / Dispose. Header comment updated to reference the two
companion files. Inline mock factories replaced with imports from
the shared helper.
Coverage preserved: ~43 tests still run under test:ci, across three
forks instead of one.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The heavy QEReasoningBank / AQELearningEngine test files re-init a full
bank in every `beforeEach`, but several module-level singletons in the
transitive graph survive across tests and accumulate per run:
- real-embeddings.ts: `pipeline`, `embeddingModel`, `embeddingCache`
(the embedding Map grows for every distinct text routed through
`computeRealEmbedding()` and is never cleared).
- audit/witness-chain.ts: `_instance` retains the WitnessChain with
accumulated entries and its better-sqlite3 handle.
Wire-in:
afterEach(() => {
clearEmbeddingCache(); // existing export, just unused
_resetWitnessChainForTests(); // new test-only reset added here
});
afterAll(() => {
resetInitialization(); // existing export, full pipeline reset
});
`clearEmbeddingCache()` and `resetInitialization()` already existed in
real-embeddings.ts as test hooks but nothing was calling them. The
witness-chain singleton had no reset path — this commit adds a
`_resetWitnessChainForTests()` helper next to the existing factory
(underscore-prefixed to mark it as internal/test API).
The hooks are applied to all three files in the split (qe-reasoning-bank
core, qe-reasoning-bank-feedback-loop, aqe-learning-engine).
Local validation note: this Codespace's heap ceiling is too tight to
reproduce CI's cumulative-growth regime — a single bank init alone
OOMs at 6GB here. CI is the canonical environment for measuring
whether the dispose hooks restore acceptance #1. Re-enabling the heavy
files in test:unit:fast (acceptance #2) is held until CI confirms.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dragan Spiridonov