Wires the ADR-043 LLM router into the production swarm path: kernel
constructs HybridRouter singleton during initialize(), threads it
through all 11 LLM-enhanced domains and MCP standalone tools. Adds
persistent aqe llm config --set, Gemini GOOGLE_API_KEY alias, 2 real
fallback-chain bug fixes (provider-discriminatory hardcoded list +
non-retryable short-circuit), and a qe-devils-advocate audit hardening
sweep closing 10 findings.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Two follow-up fixes after re-running the broader test suite (2064 tests):
1. tests/unit/shared/llm/router/hybrid-router.test.ts:619 — the
pre-existing test "should not retry non-retryable errors" was
asserting the OLD short-circuit-on-non-retryable behavior that
Fix#3 in the audit corrected. Updated to assert the new (correct)
semantics: non-retryable errors don't re-try the SAME provider but
DO fall through to the NEXT provider in the fallback chain.
Renamed accordingly. Both providers fail non-retryable so the
request still rejects, preserving the original "rejects" assertion.
2. README.md "LLM Providers" section was under-documented relative to
the wired implementation. Updated to:
- List all 7 ExtendedProviderType providers with their env var names
(including the GOOGLE_API_KEY alias added in Phase 1)
- Document `aqe llm config --set` persistence path
- Document the AQE_LLM_ROUTER_DISABLED kill-switch
- Mention that apiKey can't be saved via CLI (env-only)
- Drop Groq from the table since it's not actually in the supported
provider list
Verification
- npx tsc --noEmit: clean
- 2063/2063 unit + integration tests pass (1 skipped, no failures)
- Includes 38/38 hybrid-router tests after the assertion update
A post-Phase-4 devil's-advocate audit identified 10 issues. All addressed
with code fixes + tests. Some were real bugs (silent billing regression,
2 unwired MCP tools, 2 HybridRouter routing bugs); others were robustness
improvements (split singletons, silent errors, decorative type fields).
Findings → fixes (severity tagged in audit report):
1. CRITICAL — silent billing regression on upgrade
Users with ANTHROPIC_API_KEY in env would silently start paying for
Claude API calls on every Fleet operation. The CHANGELOG's "opt out
via code changes" wasn't sufficient — needed an env-only escape.
Fix: AQE_LLM_ROUTER_DISABLED=1 env kill-switch honored in both
QEKernelImpl._initializeLLMRouter and src/mcp/tools/base.ts
getSharedLLMRouter. Truthy values disable; 'false'/'0'/'no'/'off'/''
pass. Unified parse rule between kernel and MCP base.
2. HIGH — 2 unwired MCP standalone tools (audit said 3; QualityCriteria
isn't LLM-aware so excluded)
src/mcp/tools/test-generation/generate.ts:82 was calling
createTestGeneratorService(memory) without router. Same for
visual-accessibility/index.ts:177. Both now use the
createXxxServiceWithDependencies factories with { memory, llmRouter }.
3. CRITICAL — E2E only tested manual routing mode, not production
Production default is rule-based. Adding two new E2E tests for
rule-based mode SURFACED two real bugs in HybridRouter that I fixed:
3a. executeWithFallback hardcoded ['claude','openai','ollama'] as the
only fallback providers; gemini/openrouter/azure-openai/bedrock
users had no fallback even though ProviderManager can construct
them. Plus the user's defaultProvider wasn't auto-included in the
chain. Both fixed at src/shared/llm/router/hybrid-router.ts:612.
3b. A non-retryable error from one provider (e.g. Claude "invalid
x-api-key") aborted the entire fallback loop via `throw error`,
defeating the whole point of a fallback chain. Fixed: skip the
inter-attempt delay for non-retryable but continue to the next
fallback entry.
4. HIGH — MCPToolContext.llmRouter field was decorative
Added the field but no code populated it. Every getLLMRouter(context)
call fell through to getSharedLLMRouter().
Fix: added llmRouter and memory to BaseMCPTool.invoke() options;
qe-tool-bridge.ts and registry.ts now explicitly forward the shared
singleton via { llmRouter } at invoke time.
5. HIGH — 14 of 15 wired services had only STRUCTURAL verification, not
behavioral (only TestExecutor had a chat() round-trip test)
Fix: new tests/integration/llm-router-behavioral.test.ts asserts
isLLMAnalysisAvailable()===true (the exact gate the audit named) for
all 14 services + 2 full behavioral round-trips. 16 tests total.
6. HIGH — MCP and kernel built SEPARATE HybridRouter instances
Different cost trackers, caches, circuit breakers per side.
routing_economics/routing_metrics would only see half the traffic.
Fix: kernel.ts._initializeLLMRouter calls setSharedLLMRouter(router)
on init so both paths share one instance. kernel.dispose() calls
resetSharedLLMRouter() so the next kernel doesn't inherit the
previous instance.
7. HIGH — Silent error swallowing in _initializeLLMRouter
console.warn'd then continued without LLM, with no signal for
monitoring. A misconfigured provider would silently degrade with no
alert.
Fix: now publishes kernel.llm-router.init-failed (caught error) and
kernel.llm-router.init-no-provider (enabled=true but nothing
available) events on the event bus. Test confirms event delivery.
8. MEDIUM — Service count mismatch in docs (ADR/CHANGELOG said "14
services / 12 domains"; test enumerates 15 services / 11
LLM-enhanced domains)
Fix: ADR-043 addendum + CHANGELOG corrected to 15/11. Added
explicit service list and noted the 2 non-LLM domains
(enterprise-integration, coordination).
9. MEDIUM — aqe llm config --set apiKey=... silently dropped the key
stripApiKeys() returned a successfully-saved file without the key,
user thought it was saved.
Fix: CLI updateConfig() now throws a clear error pointing to env
vars BEFORE calling save. stripApiKeys also now logs a console.warn
when it strips anything, so programmatic callers see the strip.
10. LOW — Env vs disk precedence was counterintuitive and undocumented
Force-enable-on-env overrides explicit `enabled: false` on disk.
Fix: doc comment in config-store.applyEnvProviderDetection +
ADR-043 "Precedence & env-var conventions" section explains.
Verification
- npx tsc --noEmit: clean
- 191 unit + integration tests pass (16 new behavioral tests for #5,
17 wiring tests, gemini provider, config-store, llm-router-service,
kernel-llm-router with 17 tests including all kill-switch cases,
CLI persistence with 28 tests)
- 5 real-provider E2E tests pass (Gemini 2.5 Flash, OpenAI gpt-4o-mini,
OpenRouter, plus 2 rule-based-mode tests that previously failed and
now pass with the HybridRouter fallback fixes)
Refs ADR-043 audit findings (2026-05-21)
Builds on 94033e3 (Phase 1), be1e60d (Phase 2), c206c81 (Phase 3+4)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Phase 3 — MCP tool path
- 8 standalone MCP tools (coverage-analyze, coverage-gaps, contract-validate,
chaos-inject, code-intel-analyze, defect-predict, learning-optimize) now
resolve the kernel HybridRouter via getLLMRouter(context) and pass it to
their service via the dependency-bag form.
- src/mcp/tools/base.ts: added getSharedLLMRouter() singleton, mirroring the
existing getSharedMemoryBackend() pattern. resetSharedLLMRouter() for
tests. Added MCPToolContext.llmRouter field so future tools (and the MCP
runtime when wiring per-request kernel context) can inject explicitly
rather than via the shared singleton.
Phase 4a — Mock-provider integration tests
- tests/integration/llm-router-wiring.test.ts (new, 17 tests).
- Per-domain wiring assertions (15): for each LLM-enhanced
plugin → coordinator → service path, asserts service.llmRouter ===
kernel.llmRouter after kernel boot with an injected mock provider.
This is the precise structural test for the gap the author's friend
flagged — a regression in any domain's wiring fails immediately.
- Behavioral end-to-end (1): TestExecutorService.analyzeFailuresWithLLM
actually invokes the mock provider; assert generateCalls counter
incremented and the response surfaced.
- Sanity (1): kernel.llmRouter is defined when a router was built.
Phase 4b — Real-provider E2E smoke test
- tests/e2e/llm-router-real-providers.test.ts (new, 3 tests).
- Gated by AQE_LLM_E2E=1; each provider auto-skips when its API key
isn't in env.
- Uses mode: 'manual' to bypass DEFAULT_QE_ROUTING_RULES (which would
otherwise route generic tasks to Claude even when defaultProvider
is set to something else — worth knowing).
- Verified live on 2026-05-21 against:
Gemini (gemini-2.5-flash) → "Hi!"
OpenAI (gpt-4o-mini) → "Hi!"
OpenRouter (openai/gpt-4o-mini) → "Hi!"
Claude intentionally skipped — the ANTHROPIC_API_KEY in the dev env
is a 24-char stub (real Anthropic keys are ~108 chars), would 401.
Phase 4c — Documentation
- docs/implementation/adrs/ADR-043-vendor-independent-llm.md: added
"Wiring Verified" status entry (2026-05-21) + full addendum documenting
the problem found, the four-phase resolution, the verification tests,
and the behavioral consequence for users.
- docs/implementation/adrs/ADR-051-agentic-flow-integration.md: added
"LLM-Enhancement Wiring Verified" status entry pointing to ADR-043's
addendum.
- CHANGELOG.md: added [Unreleased] section under Changed — behavioral,
explicitly noting the behavior shift (fleets with provider keys in env
will now start routing LLM calls through that provider), the opt-out
path (KernelConfig.llmRouter: { enabled: false }), and the persistence
fix for `aqe llm config --set`.
Test totals across all four phases on this branch
- 39 unit tests across 5 files (Phase 1)
- 17 integration tests in 1 file (Phase 4a)
- 3 real-provider E2E tests in 1 file (Phase 4b, AQE_LLM_E2E=1)
- 34 pre-existing kernel tests still pass (no regression)
- 22 pre-existing test-generation coordinator tests still pass (no regression)
- Clean tsc --noEmit across the full project
Refs ADR-043, ADR-051
Closes the wiring gap identified by user audit on 2026-05-21
Builds on 94033e3 (Phase 1) and be1e60d (Phase 2)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Plumbs the kernel-singleton HybridRouter (Phase 1) into every domain
plugin → coordinator → service that has an ADR-051 LLM-enhancement
branch, so isLLMAnalysisAvailable() finally returns true in production
instead of being unreachable dead code.
What changed (12 domains, same surgical pattern in each)
- plugin.ts: factory accepts `llmRouter?: HybridRouter` as a new last
arg; class constructor stores it as `private readonly`; onInitialize
forwards to the coordinator.
- coordinator.ts: constructor accepts `llmRouter?: HybridRouter`;
service construction switches from legacy `(memory)` form to the
dependency-bag `{ memory, llmRouter }` form that 14 service classes
already supported but no caller used.
- kernel.ts: each DOMAIN_FACTORIES wrapper updated from
`(eb, m, c, _llmRouter) => createXxxPlugin(eb, m, c)` to
`(eb, m, c, llmRouter) => createXxxPlugin(eb, m, c, undefined, llmRouter)`
so the kernel singleton actually reaches the plugin layer.
Domains wired
- test-execution → TestExecutorService
- test-generation → TestGeneratorService (via createTestGeneratorServiceWithDependencies)
- coverage-analysis → CoverageAnalyzerService + GapDetectorService
- quality-assessment → QualityAnalyzerService + DeploymentAdvisorService
- security-compliance → SecurityScannerService → SASTScanner (sub-component)
- contract-testing → ContractValidatorService
- chaos-resilience → ChaosEngineerService
- requirements-validation → RequirementsValidatorService
- code-intelligence → KnowledgeGraphService
- defect-intelligence → DefectPredictorService + RootCauseAnalyzerService
- learning-optimization → LearningCoordinatorService
- visual-accessibility → VisualTesterService (via createVisualTesterServiceWithDependencies)
Domains NOT wired (intentional)
- enterprise-integration: no LLM-enhanced services in its surface
- coordination: cross-cutting plugin, no LLM call sites
Their kernel wrappers stay at `_llmRouter` discard.
Behavior change
- Before this commit: a fleet booted with provider keys in env still
ran zero LLM calls from the swarm path — every `isLLMAnalysisAvailable()`
returned false because llmRouter was undefined in every service.
- After this commit: the same fleet routes LLM-enhanced branches
through Phase 1's HybridRouter. Setting GOOGLE_API_KEY +
`aqe llm config --set defaultProvider=gemini` now actually changes
which provider sees the tokens.
Verification
- `npx tsc --noEmit -p tsconfig.json`: clean
- 191 tests pass across 7 files including:
* 34/34 existing kernel.test.ts (no regression)
* 22/22 existing test-generation coordinator tests (no regression)
* 40/40 gemini.test.ts (Phase 1 alias)
* 27/27 cli/commands/llm-router.test.ts (Phase 1 persistence)
* 29/29 config-store.test.ts
* 10/10 llm-router-service.test.ts
* 4/4 kernel-llm-router.test.ts
- Phase 4 will add mock-provider integration tests per-domain proving
the analyzeXxxWithLLM branches actually fire end-to-end.
Refs ADR-043 (vendor-independent LLM), ADR-051 (LLM-enhanced services)
Builds on 94033e3 (Phase 1 — kernel singleton + persistence)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Wires the foundation for ADR-043's HybridRouter into the kernel boot
sequence so domain plugins can finally receive an llmRouter dependency.
Phase 1 lands the kernel/CLI/provider changes; Phase 2 will plumb the
router through all 13 domains.
Background: the LLM-enhancement code paths added in ADR-051
(isLLMAnalysisAvailable() across 14 services) were unreachable because
no construction site ever passed an llmRouter — services accepted the
optional dependency but no caller injected it. ProviderManager +
HybridRouter were only instantiated from the `aqe llm advise` CLI
subcommand.
Changes
- src/shared/llm/providers/gemini.ts: read GOOGLE_API_KEY as a third
fallback after GOOGLE_AI_API_KEY and GEMINI_API_KEY, so users with
the conventional Google key name don't need to alias it themselves.
- src/shared/llm/router/config-store.ts (new): load/save
.agentic-qe/llm-config.json with env-detection layering
(defaults < disk < env < explicit override). Strips apiKey fields
defensively before persisting. Filters out providers the
ProviderManager doesn't yet construct (onnx) to keep boot logs
clean.
- src/shared/llm/llm-router-service.ts (new): builds a HybridRouter
from the loaded config, picks primary/fallbacks from the available
set honoring the user's defaultProvider when possible. Accepts an
injected ProviderManager for tests.
- src/kernel/interfaces.ts: extends QEKernel with llmRouter? and
KernelConfig with llmRouter.enabled ('auto'|true|false) +
configOverride + providerManager test-injection.
- src/kernel/kernel.ts: kernel-singleton llmRouter built during
initialize(), exposed via kernel.llmRouter, forwarded to all 13
DOMAIN_FACTORIES as a 4th arg. Phase 2 wires each factory through
to its coordinator/services; for now the wrappers accept-and-discard.
- src/cli/commands/llm-router.ts: replaces the
`// would be persisted in real implementation` stub with real
read/write through the config-store. `aqe llm config --set` now
survives a process restart. Adds AQE_CONFIG_ROOT env var as a test
seam so tests don't mutate the real .agentic-qe/.
Tests (110 total, all passing)
- gemini.test.ts: +3 new tests for each env-var alias, existing
tests updated to clear all three names
- config-store.test.ts (new, 29 tests): load/save/merge precedence,
env detection, secret stripping, atomicity
- llm-router-service.test.ts (new, 10 tests): pickEnabledProviders,
pickPrimaryAndFallbacks, end-to-end with injected mock provider
- kernel-llm-router.test.ts (new, 4 tests): kernel boot with router
enabled=false / auto-no-providers / injected mock, chat() round-trip
- llm-router.test.ts (CLI): +2 persistence-to-disk tests, env-isolated
- tests/mocks/index.ts: shared createMockLLMProvider() helper
Verification
- npx tsc --noEmit: clean
- 110 tests pass; no regressions in existing kernel.test.ts (34/34)
- Stderr noise from PersistentSONAEngine init is pre-existing and
unrelated to this change.
What this commit does NOT do
- Does NOT change behavior for any user yet — DOMAIN_FACTORIES
wrappers still discard the llmRouter param. Phase 2 lights up the
domain services.
Refs ADR-043 (vendor-independent LLM), ADR-051 (LLM-enhanced services)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Closes 11 findings from a post-merge audit of the external embedder endpoint
implementation. The "no silent hash fallback" correctness promise was being
contradicted by two callers; the connect timeout was actually an idle timeout;
the identity probe could be bypassed in several paths; and identity drift
across runs was undetectable. All hardened.
Critical/High fixes:
- sqlite-persistence backfill no longer falls back to hash embeddings when the
endpoint is configured; re-raises to protect the HNSW index from non-comparable
vectors (was poisoning silently on every endpoint blip).
- qe-reasoning-bank.embed() propagates the throw under isUsingEndpoint() instead
of silently substituting a hash vector.
- EmbedderEndpointClient.embed() now gates on probe() — lazy probe on first use
with in-flight dedup so 20+ concurrent embeds share one probe.
- Circuit breaker recovery (window expires) clears cached identity so the next
embed re-probes — endpoint restarts often coincide with model swaps.
- True TCP-connect timeout enforced at the socket layer (was req.setTimeout,
which is an idle-after-write timeout — SYN blackholes hung ~75s instead of 5s).
- Endpoint identity persisted to memory.db kv_store (_system namespace) — cross-
run fingerprint drift now surfaces as a loud warning on next boot.
Medium fixes:
- encoding_format: 'float' pinned in request; base64 responses fail loud.
- TLS knobs (ca, cert, key, rejectUnauthorized, servername) passthrough for
self-hosted https endpoints without NODE_TLS_REJECT_UNAUTHORIZED=0.
- URL userinfo stripped + warned at parse; redacted from all error messages
and logs.
- Embedding cache key namespaced by mode (endpoint:<fingerprint>:<text> vs
inproc:<text>) — cross-mode collisions structurally impossible.
Tests:
- 33 unit tests pass (was 23): adds probe-gate, breaker re-probe, connect
timeout, DNS failure, concurrent embed dedup, base64 fail-loud, URL
redaction, encoding_format wire format.
- 6 live integration tests against real llama-server (all-MiniLM-L6-v2.Q8_0.gguf):
probe, single embed, batch, lazy probe gate, dim mismatch, 50-input batch.
Replaces the prior "9ms in-process fake" with honest numbers (cold 15ms,
warm 2ms against localhost llama.cpp).
- New tests/integration/embedder-endpoint-llamacpp.test.ts is gated on
AQE_LLAMA_EMBED_URL so it skips cleanly in CI without the binary.
ADR-097 updated with the per-caller audit table (12 writers verified
non-poisoning) and "drop-in for 6 providers" claim softened to "verified
against llama-server; OpenAI shape compat expected but unverified" per
provider.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add opt-in OpenAI-compatible /v1/embeddings client to eliminate duplicate
in-process model loads in ruflo/ruvector co-deployments and the per-hook
cold-load cost in fresh agentic-qe CLI processes.
When EmbeddingConfig.endpoint (or AQE_EMBEDDER_ENDPOINT) is set, the
@huggingface/transformers import is bypassed entirely and feature-extraction
routes through the remote service. Endpoint unset = today's behavior.
Wire format is OpenAI's /v1/embeddings so any existing embedder server (TEI,
vLLM, llama.cpp, Ollama, LocalAI, LM Studio, OpenAI) works as a drop-in,
not just bespoke ruvector servers. Transport is HTTP or HTTP-over-Unix-socket.
Errors hard-fail with a circuit breaker (3 failures / 60s window) — no
silent hash-embedding fallback, which would poison the HNSW index by mixing
non-comparable vectors. A startup probe asserts dim === 384 and computes a
stable identity fingerprint that catches model/quantization drift at the
boundary.
- src/learning/embedder-endpoint-client.ts: OpenAI client with keep-alive
agent, 5s/30s timeouts, Bearer auth from env, circuit breaker, probe
- src/learning/real-embeddings.ts: endpoint branch placed strictly before
the dynamic transformers import so the cold-load fix holds at runtime;
new isUsingEndpoint() / getEndpointIdentity() observability helpers
- 23 new unit tests (real http.createServer fixtures, no http mocking)
- End-to-end smoke: 9ms for import + init + probe + embed, transformers
never appears in the loaded-module list
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
ADR-096 fix#3 (Option A): the previous 4-dim state_key
taskType|priority|domain|complexityBucket
fragmented semantically identical tasks across cells. complexityBucket
was derived from raw description length (round(min(len/200,1)*10)) —
so a 55-char and 130-char phrasing of the same task landed in
different Q-rows, and QWEIGHT_RAMP_VISITS=20 was rarely reachable
per cell even after the fix #1/#2 loop closure.
State key is now 3-dim:
taskType|priority|domain
Same change applied at both ends of the loop:
- buildRoutingStateKey (agent-routing.ts) — parameter and key shape
- updateHookRouterQValue (hooks-dream-learning.ts) — state_key SQL
- qe-reasoning-bank.ts reader-side call
- task-hooks.ts post-task writer call
- routing-hooks.ts post-route writer call (new in fix#1)
`deriveComplexityBucket` stays exported (marked @deprecated for
state_key use) — the pre-task bridge payload still carries
`complexityBucket` for kv_store schema compatibility and any
downstream telemetry. `TaskBridgePayload` interface unchanged.
Migration: existing rl_q_values rows written under the previous
4-dim key are orphaned. Field reports show real installs have at
most ~2 such rows (the loop was broken; no real Q-data accumulated),
so we accept the orphans rather than ship a migration. If a future
install carries substantial 4-dim data, a one-shot script that
strips the trailing |N segment is straightforward.
Test contract extended:
- agent-routing-exploration.test.ts pins the 3-segment key shape
- route-q-loop-closure.test.ts asserts complexityBucket does NOT
leak into updateHookRouterQValue payloads (writer-side check) and
that bridge.complexityBucket is ignored even when present.
ADR-096 updated: complexity-bucket promoted from Open Questions to
Decision (Option A). Open Questions now lists "re-introducing a
coarse complexity tier if telemetry shows the 3-dim key conflating
decisions that benefit from differentiation" as a wait-and-measure
follow-up rather than a planned change.
Refs: #499
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The route hook (UserPromptSubmit) reads `rl_q_values` to produce qWeight,
but the only writer was task-hooks `post-task` gated on `outcome.bridge`.
On real projects, `routing_outcomes` reached 139 rows while `rl_q_values`
stayed at 2 — the route surface that displays qWeight had nothing to read,
so `qWeight: 0` was structurally pinned.
Two complementary producer additions, ADR-096:
Fix#1 — post-route trains rl_q_values
When the Stop hook resolves a routing_outcomes sentinel, derive a
state_key from the sentinel's `task_json.description` using the
same exported helpers the route hook used (`deriveTaskType`,
`deriveComplexityBucket`, `detectQEDomains`) and call
`updateHookRouterQValue`. Writer and reader address byte-identical
rows. Best-effort + try/catch: Stop hooks must never crash the host.
Fix#2 — post-task fires even when bridge is absent
Drop the `if (outcome.bridge)` gate. When bridge is missing,
derive state from `options.description` instead. Adds a new
`--description` option to `aqe hooks post-task` so the PostToolUse
hook command can pass `tool_input.description` for Task tools.
Skip the update only when neither bridge nor description is
available (would pollute an "unknown" sentinel state).
Tests: tests/unit/cli/commands/route-q-loop-closure.test.ts pins
the producer/consumer state_key alignment contract — if a future
refactor desynchronizes the three call sites
(routing-hooks.ts post-route, task-hooks.ts post-task,
qe-reasoning-bank.ts:530), assertions fail the build.
Closes the Open Question flagged in ADR-095. The deriveComplexityBucket
fragmentation Jordi noted is acknowledged in ADR-096's Open Questions
and tracked for a follow-up — orthogonal to closing the loop.
Refs: #499
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the qe-reasoning-bank split: the original 622-line file ran
~43 tests in a single fork, most of which spun up a full
AQELearningEngine (transformer + HNSW + WASM coherence) via
beforeEach. Distributing across three sequential forks (pool=forks,
maxForks=1, fileParallelism=false) caps per-fork heap.
- _aqe-engine-test-helpers.ts (NEW, not a test file)
Shared createMockMemoryBackend() and createMockEventBus() factories
pulled out of the original so the splits don't duplicate. Underscore
prefix keeps it out of vitest's test-file include glob.
- aqe-learning-engine-factory.test.ts (NEW, 4 tests, no init)
Factory Functions + DEFAULT_ENGINE_CONFIG. Tests only call the
constructor — no engine.initialize() — so transformer / HNSW /
WASM aren't loaded. Safe to include in test:unit:fast.
- aqe-learning-engine-patterns.test.ts (NEW, 15 tests, full engine)
The three heaviest sub-describes: Pattern Learning, Task Tracking,
Experience Capture Integration. Each test runs a full engine init.
Excluded from test:unit:fast until step 3's dispose hooks are
proven in CI.
- aqe-learning-engine.test.ts (SLIMMED, ~21 tests, full engine)
Init / Status / Task Routing / Model Routing / Codebase Analysis /
Guidance / Dispose. Header comment updated to reference the two
companion files. Inline mock factories replaced with imports from
the shared helper.
Coverage preserved: ~43 tests still run under test:ci, across three
forks instead of one.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The heavy QEReasoningBank / AQELearningEngine test files re-init a full
bank in every `beforeEach`, but several module-level singletons in the
transitive graph survive across tests and accumulate per run:
- real-embeddings.ts: `pipeline`, `embeddingModel`, `embeddingCache`
(the embedding Map grows for every distinct text routed through
`computeRealEmbedding()` and is never cleared).
- audit/witness-chain.ts: `_instance` retains the WitnessChain with
accumulated entries and its better-sqlite3 handle.
Wire-in:
afterEach(() => {
clearEmbeddingCache(); // existing export, just unused
_resetWitnessChainForTests(); // new test-only reset added here
});
afterAll(() => {
resetInitialization(); // existing export, full pipeline reset
});
`clearEmbeddingCache()` and `resetInitialization()` already existed in
real-embeddings.ts as test hooks but nothing was calling them. The
witness-chain singleton had no reset path — this commit adds a
`_resetWitnessChainForTests()` helper next to the existing factory
(underscore-prefixed to mark it as internal/test API).
The hooks are applied to all three files in the split (qe-reasoning-bank
core, qe-reasoning-bank-feedback-loop, aqe-learning-engine).
Local validation note: this Codespace's heap ceiling is too tight to
reproduce CI's cumulative-growth regime — a single bank init alone
OOMs at 6GB here. CI is the canonical environment for measuring
whether the dispose hooks restore acceptance #1. Re-enabling the heavy
files in test:unit:fast (acceptance #2) is held until CI confirms.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The original 1019-line file ran 58 tests in a single vitest fork, each
spinning up a full QEReasoningBank (transformer embeddings + HNSW + WASM
coherence). Cumulative per-test memory growth blew past the heap ceiling
and the publish workflow timed out on multiple v3.9.x releases.
This commit extracts the file into three forks so the heap pressure is
distributed across three sequential processes (vitest config:
pool=forks, maxForks=1, fileParallelism=false):
- qe-reasoning-bank-utilities.test.ts (NEW, 29 tests, no bank)
Pure-function tests for qe-patterns + qe-guidance. Module-graph weight
is negligible — runs in ~10ms with no transformer load. Safe to
include in test:unit:fast.
- qe-reasoning-bank-feedback-loop.test.ts (NEW, 9 tests, full bank)
recordOutcome / qe_pattern_usage feedback-loop tests with vi.doMock
churn over '../../../src/kernel/unified-memory.js'. Carries the
heaviest mocking surface, so isolated to its own fork.
- qe-reasoning-bank.test.ts (SLIMMED, 20 tests, full bank)
Init / Pattern Storage / Task Routing / Learning Outcomes /
Guidance Generation / Statistics / Embedding. Header comment
updated to reference the two companion files.
Both heavy files (-feedback-loop and the slim core) remain excluded from
test:unit:fast for now. Re-enabling them is acceptance #2 of the issue
and requires step 3 of the proposed solution (lazy / disposable hooks on
real-embeddings.ts and coherence-service.ts) — local repro in a 16GB
Codespace shows even the slim 9-test fork still OOMs at 6GB heap,
indicating per-test cumulative growth that file-splitting alone doesn't
cure. The split is preparatory and reduces the leak slope from
58×Δ to max(20,9,0)×Δ per fork, which should give CI headroom and
clear the way for the dispose-hooks fix in step 3.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Regression guard for PR #497: a shipped subpath missing from the
package.json `exports` map throws ERR_PACKAGE_PATH_NOT_EXPORTED under
Node ESM strict resolution, even when the compiled file is present.
Three assertions per declared subpath:
1. Entry exists with `import` and `types` keys.
2. Referenced files exist in dist/ (skipped when dist/ not built).
3. The documented symbol is reachable via dynamic import.
Negative-control verified: removing the `./kernel/hnsw-adapter` entry
causes the first assertion to fail with a "missing from exports map
(see PR #497)" message.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The HnswAdapter class is shipped in dist/kernel/hnsw-adapter.js but the
subpath was not registered in the package.json exports map. In Node.js
ESM strict mode, unlisted subpaths throw ERR_PACKAGE_PATH_NOT_EXPORTED,
making direct imports like:
import { HnswAdapter } from 'agentic-qe/kernel/hnsw-adapter'
fail at runtime even though the compiled file is present.
Fix: add the ./kernel/hnsw-adapter entry alongside the existing
./kernel barrel export.
Stops unbounded growth of patterns.rvf / brain.rvf.
- #495: AbortSignal in HybridMemoryBackend.initialize() so a 5s
createHybridBackendWithTimeout abort actually cancels the work
instead of leaking a 14-min runaway init (~20 GB / 12 MB/s leak
observed in the field). UnifiedMemoryManager.initialize() also
accepts {signal} with throwIfAborted() at each await seam.
- Post-dream compaction in DreamScheduler — threshold-gated compact()
against patterns.rvf and brain.rvf so append-only growth has a
steady-state janitor.
- Boot-time size guard in getSharedRvfAdapter() for one-shot recovery
of installations already in the bad state.
- Bounded SQLite→RVF backfill (top-1000 by last_used_at) so
`: > patterns.rvf` for disk recovery doesn't trigger an unbounded
replay (the "regrew to 59 GB" pathology).
- RvfDualWriter.compact() for brain.rvf parity.
Three operator env knobs added (no rebuild needed):
- AQE_RVF_SIZE_GUARD_BYTES (default 256 MB)
- AQE_RVF_DEAD_RATIO_THRESHOLD (default 0.30)
- AQE_RVF_BACKFILL_LIMIT (default 1000)
29 new unit tests. Performance gate 15/15 pass. test:ci 4035 pass
(3 pre-existing failures, verified identical on main).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Closes the gap surfaced by the @xenova/transformers → protobufjs
investigation: `npm audit` in this repo lies about user-facing CVEs
because the top-level `overrides` block in package.json forces patched
versions of vulnerable transitives — but npm `overrides` are root-only
by design and do NOT propagate to downstream consumers. A clean
`npm install agentic-qe` from a user's perspective can therefore see
high/critical CVEs that our in-repo audit reports as 0.
Two new gates run against an actual consumer install tree:
1. `consumer-audit` job in npm-publish.yml (BLOCKING pre-publish).
Packs the local build with `npm pack`, installs the tarball into a
fresh tmp project, runs `npm audit --audit-level=high` there. Now a
`needs:` of the publish job, so a CVE invisible to repo-level audit
still blocks publish.
2. Consumer-audit step + P1 issue creation in post-publish-canary.yml
(NON-BLOCKING). Same idea, but against the freshly-published tarball
from the registry — catches discrepancies between what we built and
what the CDN serves. Findings open an auto-labelled P1 issue
(security/canary/release) and surface in the run's step summary.
The two together replicate the actual user view at both the
pre-publish gate and the post-publish safety net.
Local simulation of the consumer-audit logic on the current head:
183 packages installed, 0 vulnerabilities — confirming the
@huggingface/transformers migration in 122f223c actually fixed the
consumer view (was 4 high + 1 critical pre-migration).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@xenova/transformers v2.17.2 (last publish 2024-05-29) pins
onnxruntime-web@1.14.0, which pulls onnx-proto → protobufjs <=7.5.5.
That chain has 8 GHSA advisories including a critical arbitrary-code-
execution one (GHSA-xq3m-2v4x-88gg). Our top-level `overrides` block
forced protobufjs@^7.5.6 in our own tree, hiding the vulns from
`npm audit` in this repo — but `overrides` are root-only by design and
do not propagate to downstream consumers. End users running
`npm install agentic-qe` were getting 5 vulns (4 high, 1 critical).
@huggingface/transformers@4.2.0 is the official maintained successor
(same project, renamed). It pulls onnxruntime-web@1.26 which depends
on protobufjs@^7.2.4 directly (no onnx-proto), so the version range
resolves cleanly to the patched line.
API changes handled:
- pipeline()/Tensor: drop-in compatible, just the import path changes
- `quantized: bool` → `dtype: DataType` (mapped from our QuantizationType
enum: 'none'→'fp32', 'fp16'→'fp16', 'int8'→'int8', 'binary'→'q8';
binary stays a post-process step downstream)
- progress_callback signature: ProgressInfo is now a discriminated union;
`progress` only exists on the "progress" variant — narrowed accordingly
Also added onnxruntime-node and onnxruntime-web to the CLI and MCP
bundle externals lists. @huggingface/transformers ships both runtimes
(node for Node.js, web as fallback); esbuild can't bundle the
native .node binary so it must remain external.
End-user verification (npm pack + clean install in /tmp/aqe-canary):
- 183 packages (was 180 pre-migration; same order of magnitude)
- 0 vulnerabilities (was 5: 4 high, 1 critical)
- aqe init --auto runs in 93 ms; all 13 phases pass; optional-peers
hint renders correctly.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
When @claude-flow/browser or @claude-flow/guidance is not installed
locally, append a short "Optional add-ons" section to the end of
`aqe init`'s success output with the install command for each. Skipped
silently when both are already present.
Why this matters: the previous two commits moved these from
optionalDependencies to peerDependenciesMeta{optional:true}, removing
~480 transitive packages and the upstream ERESOLVE noise. The trade-off
is that users no longer get them auto-installed, so the features they
gate (browser MCP URL/PII scanning; the guidance governance integrations)
silently fall back to JS paths. The init hint surfaces the opt-in at
exactly the moment the user is configuring their project, with a
concrete install command they can copy.
Detection uses `require.resolve` and treats ERR_PACKAGE_PATH_NOT_EXPORTED
as "installed" — @claude-flow/guidance ships only an ESM "import"
condition for its bare specifier, so a naive CJS-style resolve from
createRequire would report it missing even when it isn't.
Note: the dead-but-still-compiled src/cli/commands/init.ts contains a
similar displayNextSteps() that is NOT reached by the bundled CLI bin
(dist/cli/bundle.js dynamically loads dist/cli/chunks/init-handler-*.js,
which is built from src/cli/handlers/init-handler.ts). Editing
init-handler.ts is the only place that affects the actual `aqe init`
output. Leaving commands/init.ts alone — it's separately tracked tech
debt and out of scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Mirrors the previous @claude-flow/guidance treatment: moves
@claude-flow/browser from optionalDependencies to peerDependencies +
peerDependenciesMeta{optional:true}.
Why this is the high-leverage change in this cleanup pass:
@claude-flow/browser peers on @claude-flow/cli@^3.0.0-alpha.140, which
in turn pulls @claude-flow/embeddings whose peers disagree with cli's
own optional deps — that mismatch is the entire source of the ERESOLVE
warnings users see during `npm install agentic-qe`. With browser moved
to an optional peer, that whole transitive chain is no longer pulled by
default.
Tree shrinkage: 1207 → 725 packages (482 transitives removed,
~7,924 fewer lines in package-lock.json). Includes @claude-flow/cli,
@claude-flow/embeddings, agentic-flow, plus their long tail of
deprecated transitives (koa-router@14, prebuild-install@7, uuid@9,
node-domexception@1) — all of the deprecation warnings called out at
the start of this dep-cleanup pass.
Zero runtime risk: grep confirms zero `import`/`require` of
@claude-flow/browser anywhere in src/, tests/, or scripts/. The package
was only ever a marker for the separately-run @claude-flow/browser MCP
server; src/domains/visual-accessibility/services/browser-security-scanner.ts
talks to it over the MCP protocol, not via a Node import.
Verified: `npm run build` passes, `npm audit --audit-level=high
--omit=dev` clean, `node dist/cli/bundle.js --version` runs. `npm ls
@claude-flow/cli` is now empty.
Users who want the browser MCP server can still install it explicitly:
`npm install -g @claude-flow/browser@3.0.0-alpha.1`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Moves @claude-flow/guidance from optionalDependencies to:
- peerDependencies + peerDependenciesMeta{optional:true}: tells
consumers of agentic-qe that this is an optional integration they
can opt into, without forcing npm to install it for everyone.
- devDependencies: keeps our own build/CI deterministic without
depending on the transitive chain (@claude-flow/cli pulls guidance
as one of its own optional deps).
End-user impact: zero. The package was already transitively installed
via @claude-flow/browser → @claude-flow/cli, and runtime governance
integrations in src/governance/*-integration.ts already use lazy
`await import('@claude-flow/guidance/...')` with try/catch fallback,
so the declared-dep change does not alter what ends up in node_modules.
Type-surface check: dist/**/*.d.ts contains zero `import('@claude-flow/guidance')`
references — the internal type aliases compile away, so downstream
TypeScript consumers of agentic-qe/governance are unaffected.
Signals intent more cleanly. A bigger reducer of the upstream ERESOLVE
peer noise would be giving @claude-flow/browser the same treatment, but
that is out of scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds a new `security-audit` job to optimized-ci.yml that runs in parallel
with the journey shards and fails the build on any high/critical CVE in
production deps. devDep CVEs are reported as warnings — they don't ship
to users but are visible so we can fix them before they migrate.
Also adds the same `npm audit --audit-level=high --omit=dev` step to the
publish workflow's build job as a defense-in-depth gate: blocks `npm
publish` if a new prod-tree CVE has landed since the last main-branch CI
run.
Would have caught the 6 high `@typescript-eslint`/`minimatch` advisories
that Tier B just cleared the moment they were published, rather than
when someone happened to run `npm audit`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
#493 was not the last instance of the kv-Date hazard — it was the most
reachable. Audit of the rest of the learning pipeline found 11 more
unprotected kv-read sites where TypeScript declares `Date` but JSON
round-trip through `memory.get` yields an ISO string at runtime.
Two failure modes, both already documented at
learning-coordinator.ts:898-904 (#491 Bug 3):
1. THROW — `record.someDate.getTime()` raises TypeError. Surfaces
immediately on the first invocation once any record exists.
2. SILENT — `timeRange.contains(record.someDate)` or
`new Date() > record.expiresAt` coerces string vs Date to NaN; all
comparisons return false; records are silently filtered out or
expiry is never recognised. No error, just empty/stale analytics.
The silent variant is the worse one — #491 Bug 3 was the same class
and shipped for a release cycle undetected.
Strategic fix: a single `rehydrateDates<T>(record, dateFields[])` helper
at `src/shared/utils/kv-date-rehydrate.ts` applied at every kv-read seam
that touches Date-bearing types. Shallow-copies the record and coerces
named string fields back to Date; pass-through for already-Date, null,
undefined.
Sites patched (11 total):
src/domains/learning-optimization/services/
production-intel.ts — 4 sites (HIGH #2-#5 throw, MEDIUM #8 silent)
getMetricsHistory, getRecentIncidents (×2),
resolveIncident, getRecentMilestones
metrics-optimizer.ts — 1 site (HIGH #1 throw)
getMetricsHistory.sort
learning-coordinator.ts — 1 site (HIGH #6 throw)
getReplayBuffer.sort
transfer-specialist.ts — 4 sites (HIGH #7 throw, MEDIUM #9 silent)
queryKnowledge (×2), getKnowledgeById,
bulkTransfer
src/domains/learning-optimization/
coordinator-helpers.ts — 1 site (SILENT)
getExperiencesForDomain
coordinator.ts — 1 site (SILENT, duplicate of above)
getExperiencesForDomain (private)
Sites verified SAFE (kv-read but only access non-Date fields):
- coordinator.ts:1045/1134/1175 — uses .domain/.confidence/.version
- metrics-optimizer.ts:676 — uses .metrics map
- learning-coordinator.ts:379/438/483/547/830/842 — uses
.confidence/.successRate/.domain/.usageCount
- learning-coordinator.ts:901 — already has #491 Bug 3 inline
rehydration with instanceof guard
- experience-capture.ts:520/563 — TaskExperience uses number
timestamps, not Date
Sites NOT patched (deferred per scope):
- coordination/protocols/learning-consolidation.ts — defined but
never imported anywhere. Dead code; would resurrect the bug if
reactivated. Documented in the audit but not fixed.
Regression tests (29 new, 6 files):
tests/unit/shared/utils/kv-date-rehydrate.test.ts (10)
tests/unit/domains/learning-optimization/
production-intel-date-coercion.test.ts (5)
metrics-optimizer-date-coercion.test.ts (2)
learning-coordinator-date-coercion.test.ts (2)
transfer-specialist-date-coercion.test.ts (5)
coordinator-helpers-date-coercion.test.ts (2)
Each service test seeds its mock memory backend with kv-shape records
(ISO strings instead of Date instances — matching the post-JSON.parse
shape that better-sqlite3-backed memory yields in production) and
exercises the kv-read → consumer path that would fail pre-fix. Both
throw and silent failure modes are pinned: throws are asserted not to
fire, silent failures are caught by behavioural assertions (e.g. the
expired-knowledge test asserts `expired.score < live.score * 0.5 ±`,
which would fail pre-fix because the 0.5 penalty never triggered).
Verification:
- 29/29 new regression tests green
- 135/135 existing tests for patched services green (production-intel,
metrics-optimizer, transfer-specialist, learning-coordinator,
learning-consolidation, learning-consolidation-mining,
learning-consolidation-liveness)
- `tsc --noEmit` clean
- `npm run build` produces dist with rehydrateDates at all 17
expected call sites across the 6 patched files
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
`LearningConsolidationWorker.getDomainPatterns()` mapped
`lastSeen: p.lastUsedAt`, but `lastUsedAt` arrives from the
`learning:pattern:*` kv as an ISO string (not a Date) after the JSON
round-trip. The two downstream consumers — `consolidatePatterns` and
`pruneIneffectivePatterns` — both call `.getTime()` on it, throwing
`TypeError: lastSeen.getTime is not a function` every cycle on installs
with non-empty pattern kv.
This was dead code until v3.9.32: #491 Bug 1+2 starved the pattern store
so `collectPatterns()` always threw `No learning patterns to consolidate
yet` before reaching `consolidatePatterns`. Fixing the #491 chain made
this line reachable for the first time and the same `Date`-lost-across-
kv-JSON-round-trip class of bug as #491 Bug 3 surfaced. Other call sites
were hardened in v3.9.32 (`pattern-store.ts:1301-1304`,
`pattern-lifecycle.ts:592-594`, `sqlite-persistence.ts:891`); these two
were missed because they were unreachable at the time.
Fix: rehydrate at the kv-read boundary — `lastSeen: new Date(p.lastUsedAt)`.
`new Date(x)` is safe for `Date | string | null | undefined` (yields
Invalid Date — whose `.getTime()` returns `NaN` rather than throwing — on
null-ish input). One-line change covers both downstream consumers.
Test: `learning-consolidation-date-coercion.test.ts` exercises the full
post-mining half of the loop (the gap Jordi flagged in his release-gate
suggestion). Seeds a `learning:pattern:*` entry with ISO-string
`lastUsedAt`, runs the worker through `getDomainPatterns` →
`consolidatePatterns` → `pruneIneffectivePatterns` → full `doExecute`,
and asserts:
1. The seam rehydrates `lastSeen` to a real Date.
2. `consolidatePatterns` completes without throwing.
3. The recent-high-performer filter math actually counts the pattern
(not just "doesn't throw" — catches silent-suppression regressions).
4. `pruneIneffectivePatterns` completes and produces the stale finding.
5. Full `doExecute` reaches `Learning consolidation complete` and
`recordLoopHealth(success:true)` — the user-visible signals.
Verified pre-fix with 5/5 failing tests showing the exact error from
the issue; post-fix with 5/5 green, full learning-consolidation suite
(36 tests) green, and `tsc --noEmit` clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
- web-tree-sitter ~0.24.7 → ~0.26.8 (API moved Parser.init onto the
Parser class; adapter shim added in tree-sitter-wasm-parser.ts so we
also still work against the legacy 0.24 surface).
- @types/node ^20.19.17 → ^25.8.0 (engine "node >=18.0.0" is unchanged;
type widening only).
Deferred from Tier C (each needs its own migration PR, not a bump):
- @ruvector/attention 0.1.3 → 2.2.2: full API rewrite, classes
(FlashAttention, DotProductAttention, MultiHeadAttention,
HyperbolicAttention, LinearAttention, MoEAttention) replaced by
free functions (flashAttention, scaledDotProductAttention, …).
Requires rewriting integrations/ruvector/attention-wrapper.ts and
updating all 6 platform-binary optional deps in lockstep.
- vibium 0.1.8 → 26.x: package no longer ships dist/ — repurposed as a
CLI binary. `import { browser } from 'vibium'` no longer resolves.
Requires a different integration strategy.
- typescript 5.9 → 6.0 and eslint 8 → 10: released 1-4 weeks ago; need
their own validation pass (flat config migration for eslint).
- @ruvector/sona 0.1.5 → 0.1.6: published manifest references files
missing from the tarball (broken publish upstream).
Build passes (tsc + cli + mcp bundles). npm audit: 0 vulnerabilities.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #491 exposed a structural gap in the pre-publish init gate: it
runs `aqe init` against 4 pinned real repos and asserts file
existence, KG entry counts, double-init etc — 22 separate
assertions — but it NEVER starts the daemon. Every one of the four
bugs in #491 lives in the daemon-runtime seam, not the init seam:
the kernel→worker wiring opens during `fleet_init`, which only
happens at daemon startup. So a fully-broken learning loop sailed
through the gate as "OK".
This commit adds A23 to run_one():
- opt-in per fixture via the new `expectDaemonLiveness` manifest
flag (default false, so existing fixtures keep their ~5s budget)
- configurable wait via `daemonLivenessWaitSec` (default 20s)
- spawns the cleanroom's start-daemon.cjs, waits, greps stderr
and daemon.log for the failure shapes from the #491 chain:
"domain not available", "Domain not available", "not available for"
- asserts zero matches; lists offending lines + summary marker on fail
- cleans up the daemon process (SIGTERM, 2s grace, SIGKILL)
Pilot fixture: enabled on `tiny-ts` with a 15s wait window. tiny-ts
is the smallest fixture (~17 files, ~6s init), so adding 15s of
daemon observation puts the whole fixture at ~22s — well within
the 60s timeout and the 20-min job budget. If the pilot is stable
for two releases, we extend the flag to mid-ts and p-queue. The
larger multi-lang-real fixture stays opt-out until a separate
weekly workflow takes long-running daemon scenarios out of the
release path.
Manifest schemaVersion bumped 3→4 with a changelog entry documenting
the new fields. Run-gate header comments updated to document A23
alongside A1-A22.
The same script is invoked by post-publish-canary.yml against the
published npm version, so A23 doubles as a post-publish smoke for
any registry/CDN divergence that affects daemon-runtime — a class
of regression neither the build-job bundle smoke nor `test:unit:fast`
could detect.
This is the structural complement to the npm-publish.yml change in
the preceding commit (integration-tests-on-tag-sha runs the SOURCE
seam tests; A23 runs the INSTALLED PACKAGE seam check). Together
they close the daemon-runtime gap that let #491 ship.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #491 exposed a structural gap in the release process: integration
coverage of the daemon-runtime seam existed (bridge-end-to-end test was
already in the repo), but it was never run on the release path —
`tests-on-tag-sha` only ran `test:unit:fast`, which excludes
`tests/integration/` by design. Every one of the four bugs in #491
lived in that unrun surface.
Two changes:
1. `package.json` — define `test:integration:fast` and
`test:integration` scripts. The "fast" variant runs the focused
daemon-runtime seam suite in ~6s:
- tests/integration/mcp/fleet-init-wires-daemon.test.ts (Bug 1+4b)
- tests/integration/workers/workers-reach-domains.test.ts (Bug 1 generalized)
- tests/integration/learning/coordinator-roundtrip.test.ts (Bug 2+3)
- tests/integration/bridge/bridge-end-to-end.test.ts (existing)
The broader `test:integration` runs all of `tests/integration/`
excluding browser/e2e for local development use.
2. `.github/workflows/npm-publish.yml` — add a new
`integration-tests-on-tag-sha` job that runs `test:integration:fast`,
wired into the publish job's `needs` and `if` conditions in the same
shape as the existing `tests-on-tag-sha`. Picked the "fast" variant
deliberately: it covers exactly the regressions #491 reported and
keeps the release gate budget tight (~10-min job ceiling, but the
work itself is ~6s). The broader integration suite is too noisy for
a release gate; the fast subset is the floor.
This is the structural complement to issue #401's pre-publish init
gate: that one catches `aqe init` regressions before users, this one
catches daemon-runtime regressions before users.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The issue #491 fix wired the daemon WorkerManager to the kernel and
shipped one targeted integration test for the learning worker. The
same StubWorkerDomainAccess failure mode applies to every other
worker that declares targetDomains — test-health, flaky-detector,
coverage-tracker, defect-predictor, etc. — and the worker-level
unit tests all explicitly punt on this with comments along the
lines of "complex dependencies require integration tests for full
coverage. Unit tests focus on instantiation." That punt is what
let Bug 1 ship in the first place.
This test iterates EVERY registered production worker (enumerated
dynamically from `daemon.getWorkerManager().list()`), builds the
exact WorkerContext the scheduler hands to doExecute, and asserts
that each of the worker's declared targetDomains resolves to a real
domain API. If the kernel→worker wiring ever breaks again — for
any of the 11 workers, not just learning-consolidation — this test
fails before the daemon ticks for users.
Three assertions:
1. Every registered worker declares ≥1 targetDomain (catches the
"shipped a worker with no domains" misconfig).
2. For every (worker, targetDomain) pair, `getDomainAPI(domain)`
returns a defined value (the structural Bug 1 contract).
Failure list is formatted as a multi-line message so the
maintainer sees exactly which seam broke.
3. The worker manager's `context.memory` exposes `vectorSearch`
(only HybridMemoryBackend has it, InMemoryWorkerMemory does
not) — a cheap structural check for the Bug 4b memory swap.
Verified red→green: with the setKernel/setMemory wiring removed
from handleFleetInit, assertions 2 and 3 fail, listing every
(worker, domain) pair that lost its API. With the wiring restored,
3/3 pass in ~170ms.
The test deliberately does NOT call `daemon.runWorker(id)` to
exercise each worker's full body. Most workers raise legitimate
"no data yet" errors on fresh installs (test-health, flaky-detector
etc.) and asserting completion would couple this test to each
worker's empty-state behaviour. The contract is structural — the
seam to the kernel is alive — not behavioural.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #491 — even with the worker→kernel wiring restored (Bug 1/4b),
`mineExperiences` mined 0 for every domain on installs with hundreds
of indexed experiences. Two stacked bugs in the learning read path:
Bug 2 — namespace mismatch (the keystone)
=========================================
LearningCoordinator writes every experience with
`namespace: 'learning-optimization'`, but the MemoryBackend
interface only let writers specify a namespace; reads had no such
option. HybridBackend.get/search/has/delete therefore hard-coded
`defaultNamespace='default'`. The coordinator could not read back
its own data — `learning:experience:index:domain:<d>:*` returned 0
keys even when the DB held 81+ matching rows in the
`learning-optimization` namespace.
Fix:
- Add `RetrieveOptions { namespace? }` to kernel/interfaces.ts
(symmetric with the existing `StoreOptions.namespace`).
- Extend MemoryBackend.get/has/delete/search to accept the option
(backward-compatible — undefined still routes to defaultNamespace).
- HybridBackend honors the option in all four methods.
- InMemoryBackend likewise; its `search` now also scopes the scan
by namespace prefix when one is supplied, so unit-test fakes
behave the same as the production hybrid backend.
- All 15 read sites in LearningCoordinator pass
`LEARNING_NS = { namespace: 'learning-optimization' }`, matching
the writes that already specified it.
One pre-existing unit test (memory-backend.test.ts) was calling
`backend.get(key, 'test-ns')` against the undocumented positional
namespace param InMemoryBackend used to accept. Migrated it to the
options form — the positional shape was never part of the
MemoryBackend interface.
Bug 3 — Date vs ISO-string in time-window filter
================================================
`recordExperience` stores `timestamp: new Date()`. JSON.stringify
serializes it to an ISO string and JSON.parse does not rehydrate.
`TimeRange.contains` does `date >= start`; in `string >= Date`, the
string coerces to NaN and every comparison is false — so even with
Bug 2 fixed, the window filter would still silently drop every row.
Fix: in `getExperiencesByDomainAndTime`, coerce
`experience.timestamp` to a real `Date` at the kv-read boundary
before calling `timeRange.contains`. Downstream callers receive a
hydrated Experience with a proper Date. We deliberately fix this at
the boundary rather than making `TimeRange.contains` tolerant of
strings — the bug is "kv round-trip drops the Date type", not "the
value object is wrong", and a tolerant comparator would mask future
type drift everywhere else.
Round-trip integration test:
tests/integration/learning/coordinator-roundtrip.test.ts
pins three assertions:
1. recordExperience → mineExperiences finds the experience back
(this returned 0 before either fix).
2. The mined experience's timestamp is a real Date instance, not a
string (proves Bug 3 fix is at the boundary).
3. A time window deliberately in the past still excludes recent
writes (proves we didn't fix Bug 3 by making the filter a
no-op).
Verified red→green: with Bugs 2+3 reverted (interface kept), tests
1 and 2 fail with `experienceCount=0` and `rows.length=0` — exactly
the symptoms from the issue. With both fixes in place, 3/3 pass in
~30ms. 466/466 kernel unit tests and 335/335 adjacent suites stay
green; the only test that needed migration is documented above.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #491 — `aqe learning loop-health` permanently showed
`LearningConsolidationWorker ○ never-ran` even when the worker
executed every cycle. Root cause: in `doExecute`, the
`recordLoopHealth(success:true)` ping sat *after* `collectPatterns()`,
which throws on installs with nothing to consolidate (the common case
for fresh users). The error propagated up before liveness was ever
recorded, so the dashboard had no idea the worker was alive.
Fix: wrap the worker body in try/catch/finally — the same shape
CapturedExperienceBridge.drainSafe already uses (which is why the
bridge correctly shows `live` and the worker did not). The `finally`
emits `recordLoopHealth` with `success:true` on a clean tick and
`success:false + error` on a throw, then the catch rethrows so
BaseWorker's retry/error tracking sees the failure unchanged. The
liveness write is itself wrapped in try/catch so a memory-backend
hiccup can't shadow the worker's real error.
Liveness contract test:
tests/unit/workers/workers/learning-consolidation-liveness.test.ts
pins three behaviors:
1. Throw inside collectPatterns → recordLoopHealth still called
with `{success:false, error}` (THIS is what regressed before).
2. Successful tick → recordLoopHealth called with `{success:true}`.
3. recordLoopHealth itself throwing must not shadow the original
worker error.
Verified red→green: with the try/finally reverted, assertion (1)
fails with "expected number of calls: 1, received: 0" — exactly the
"never-ran" symptom from the issue. With the fix in place, 3/3 pass
in ~19ms.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Issue #491 (Jordi Izquierdo) — on a vanilla v3.9.31 install, every
domain-dependent worker tick fails with "<domain> not available" and
`aqe learning loop-health` shows `learningWorker: never-ran`. Root
cause: the MCP-hosted daemon constructs its WorkerManager in
`src/mcp/entry.ts` *before* the kernel exists, so it starts with
StubWorkerDomainAccess (getDomainAPI returns undefined for every
domain) and a private InMemoryWorkerMemory that no dashboard ever
reads from. The kernel is built inside `handleFleetInit` later in the
same process, but the daemon is never told about it.
Fix:
- Promote `setKernel`/`setMemory` from concrete-class-only methods
to the public WorkerManager interface so consumers can find them.
- Add `setMemory(memory: WorkerMemory)` to WorkerManagerImpl
(mirrors the existing `setKernel`). Broadens the internal
`memory` field from InMemoryWorkerMemory to the WorkerMemory
interface so a shared HybridMemoryBackend is type-correct.
- Hoist `KernelReference` to interfaces.ts as
`WorkerKernelReference` so the interface signature is properly
typed without re-introducing the kernel↔worker circular
dependency the local declaration was avoiding.
- In `handleFleetInit`, after `pluginLoader.loadAll()`, call
`getDaemon().getWorkerManager().setKernel(state.kernel)` and
`setMemory(state.kernel.memory)`. Wrapped in try/catch so a
workless smoke install (no daemon module bundled) still gets a
working kernel + queen.
Why an interface change, not the cast Bolivar used: the public
WorkerManager interface didn't expose setKernel/setMemory at all, so
the call site looked like it was reaching for a private method —
exactly the kind of seam that previous releases dropped on the
assumption that the bridge superseded it. Making it part of the
public contract means the next maintainer can't accidentally remove
the wiring without breaking a typed interface.
Integration test: tests/integration/mcp/fleet-init-wires-daemon.test.ts
pins three assertions:
1. Before fleet_init, the daemon's WorkerManager returns undefined
for `getDomainAPI('learning-optimization')` (the Bug 1 failure
mode).
2. After fleet_init, it returns a real plugin API.
3. The WorkerManager's memory is the same instance as the kernel's
memory (Bug 4b), so worker writes reach the dashboard's reads.
Verified red→green: with the wiring reverted, all three fail with
"expected undefined to be defined"; with the wiring in place, all
three pass in ~210ms.
Note: Bugs 2, 3, and 4a are addressed in follow-up commits in this
series.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Dragan Spiridonov