mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
benchmarks
36 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dragan Spiridonov
|
3bc09cda6d |
fix(llm-router): address 10 findings from qe-devils-advocate audit
A post-Phase-4 devil's-advocate audit identified 10 issues. All addressed
with code fixes + tests. Some were real bugs (silent billing regression,
2 unwired MCP tools, 2 HybridRouter routing bugs); others were robustness
improvements (split singletons, silent errors, decorative type fields).
Findings → fixes (severity tagged in audit report):
1. CRITICAL — silent billing regression on upgrade
Users with ANTHROPIC_API_KEY in env would silently start paying for
Claude API calls on every Fleet operation. The CHANGELOG's "opt out
via code changes" wasn't sufficient — needed an env-only escape.
Fix: AQE_LLM_ROUTER_DISABLED=1 env kill-switch honored in both
QEKernelImpl._initializeLLMRouter and src/mcp/tools/base.ts
getSharedLLMRouter. Truthy values disable; 'false'/'0'/'no'/'off'/''
pass. Unified parse rule between kernel and MCP base.
2. HIGH — 2 unwired MCP standalone tools (audit said 3; QualityCriteria
isn't LLM-aware so excluded)
src/mcp/tools/test-generation/generate.ts:82 was calling
createTestGeneratorService(memory) without router. Same for
visual-accessibility/index.ts:177. Both now use the
createXxxServiceWithDependencies factories with { memory, llmRouter }.
3. CRITICAL — E2E only tested manual routing mode, not production
Production default is rule-based. Adding two new E2E tests for
rule-based mode SURFACED two real bugs in HybridRouter that I fixed:
3a. executeWithFallback hardcoded ['claude','openai','ollama'] as the
only fallback providers; gemini/openrouter/azure-openai/bedrock
users had no fallback even though ProviderManager can construct
them. Plus the user's defaultProvider wasn't auto-included in the
chain. Both fixed at src/shared/llm/router/hybrid-router.ts:612.
3b. A non-retryable error from one provider (e.g. Claude "invalid
x-api-key") aborted the entire fallback loop via `throw error`,
defeating the whole point of a fallback chain. Fixed: skip the
inter-attempt delay for non-retryable but continue to the next
fallback entry.
4. HIGH — MCPToolContext.llmRouter field was decorative
Added the field but no code populated it. Every getLLMRouter(context)
call fell through to getSharedLLMRouter().
Fix: added llmRouter and memory to BaseMCPTool.invoke() options;
qe-tool-bridge.ts and registry.ts now explicitly forward the shared
singleton via { llmRouter } at invoke time.
5. HIGH — 14 of 15 wired services had only STRUCTURAL verification, not
behavioral (only TestExecutor had a chat() round-trip test)
Fix: new tests/integration/llm-router-behavioral.test.ts asserts
isLLMAnalysisAvailable()===true (the exact gate the audit named) for
all 14 services + 2 full behavioral round-trips. 16 tests total.
6. HIGH — MCP and kernel built SEPARATE HybridRouter instances
Different cost trackers, caches, circuit breakers per side.
routing_economics/routing_metrics would only see half the traffic.
Fix: kernel.ts._initializeLLMRouter calls setSharedLLMRouter(router)
on init so both paths share one instance. kernel.dispose() calls
resetSharedLLMRouter() so the next kernel doesn't inherit the
previous instance.
7. HIGH — Silent error swallowing in _initializeLLMRouter
console.warn'd then continued without LLM, with no signal for
monitoring. A misconfigured provider would silently degrade with no
alert.
Fix: now publishes kernel.llm-router.init-failed (caught error) and
kernel.llm-router.init-no-provider (enabled=true but nothing
available) events on the event bus. Test confirms event delivery.
8. MEDIUM — Service count mismatch in docs (ADR/CHANGELOG said "14
services / 12 domains"; test enumerates 15 services / 11
LLM-enhanced domains)
Fix: ADR-043 addendum + CHANGELOG corrected to 15/11. Added
explicit service list and noted the 2 non-LLM domains
(enterprise-integration, coordination).
9. MEDIUM — aqe llm config --set apiKey=... silently dropped the key
stripApiKeys() returned a successfully-saved file without the key,
user thought it was saved.
Fix: CLI updateConfig() now throws a clear error pointing to env
vars BEFORE calling save. stripApiKeys also now logs a console.warn
when it strips anything, so programmatic callers see the strip.
10. LOW — Env vs disk precedence was counterintuitive and undocumented
Force-enable-on-env overrides explicit `enabled: false` on disk.
Fix: doc comment in config-store.applyEnvProviderDetection +
ADR-043 "Precedence & env-var conventions" section explains.
Verification
- npx tsc --noEmit: clean
- 191 unit + integration tests pass (16 new behavioral tests for #5,
17 wiring tests, gemini provider, config-store, llm-router-service,
kernel-llm-router with 17 tests including all kill-switch cases,
CLI persistence with 28 tests)
- 5 real-provider E2E tests pass (Gemini 2.5 Flash, OpenAI gpt-4o-mini,
OpenRouter, plus 2 rule-based-mode tests that previously failed and
now pass with the HybridRouter fallback fixes)
Refs ADR-043 audit findings (2026-05-21)
Builds on
|
||
|
Dragan Spiridonov
|
94033e3500 |
feat(llm-router): Phase 1 — kernel singleton + persistence + Gemini key alias
Wires the foundation for ADR-043's HybridRouter into the kernel boot
sequence so domain plugins can finally receive an llmRouter dependency.
Phase 1 lands the kernel/CLI/provider changes; Phase 2 will plumb the
router through all 13 domains.
Background: the LLM-enhancement code paths added in ADR-051
(isLLMAnalysisAvailable() across 14 services) were unreachable because
no construction site ever passed an llmRouter — services accepted the
optional dependency but no caller injected it. ProviderManager +
HybridRouter were only instantiated from the `aqe llm advise` CLI
subcommand.
Changes
- src/shared/llm/providers/gemini.ts: read GOOGLE_API_KEY as a third
fallback after GOOGLE_AI_API_KEY and GEMINI_API_KEY, so users with
the conventional Google key name don't need to alias it themselves.
- src/shared/llm/router/config-store.ts (new): load/save
.agentic-qe/llm-config.json with env-detection layering
(defaults < disk < env < explicit override). Strips apiKey fields
defensively before persisting. Filters out providers the
ProviderManager doesn't yet construct (onnx) to keep boot logs
clean.
- src/shared/llm/llm-router-service.ts (new): builds a HybridRouter
from the loaded config, picks primary/fallbacks from the available
set honoring the user's defaultProvider when possible. Accepts an
injected ProviderManager for tests.
- src/kernel/interfaces.ts: extends QEKernel with llmRouter? and
KernelConfig with llmRouter.enabled ('auto'|true|false) +
configOverride + providerManager test-injection.
- src/kernel/kernel.ts: kernel-singleton llmRouter built during
initialize(), exposed via kernel.llmRouter, forwarded to all 13
DOMAIN_FACTORIES as a 4th arg. Phase 2 wires each factory through
to its coordinator/services; for now the wrappers accept-and-discard.
- src/cli/commands/llm-router.ts: replaces the
`// would be persisted in real implementation` stub with real
read/write through the config-store. `aqe llm config --set` now
survives a process restart. Adds AQE_CONFIG_ROOT env var as a test
seam so tests don't mutate the real .agentic-qe/.
Tests (110 total, all passing)
- gemini.test.ts: +3 new tests for each env-var alias, existing
tests updated to clear all three names
- config-store.test.ts (new, 29 tests): load/save/merge precedence,
env detection, secret stripping, atomicity
- llm-router-service.test.ts (new, 10 tests): pickEnabledProviders,
pickPrimaryAndFallbacks, end-to-end with injected mock provider
- kernel-llm-router.test.ts (new, 4 tests): kernel boot with router
enabled=false / auto-no-providers / injected mock, chat() round-trip
- llm-router.test.ts (CLI): +2 persistence-to-disk tests, env-isolated
- tests/mocks/index.ts: shared createMockLLMProvider() helper
Verification
- npx tsc --noEmit: clean
- 110 tests pass; no regressions in existing kernel.test.ts (34/34)
- Stderr noise from PersistentSONAEngine init is pre-existing and
unrelated to this change.
What this commit does NOT do
- Does NOT change behavior for any user yet — DOMAIN_FACTORIES
wrappers still discard the llmRouter param. Phase 2 lights up the
domain services.
Refs ADR-043 (vendor-independent LLM), ADR-051 (LLM-enhanced services)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
d698c58721 |
fix(routing): collapse Q-state to 3-dim (drop complexityBucket) (#499)
ADR-096 fix #3 (Option A): the previous 4-dim state_key taskType|priority|domain|complexityBucket fragmented semantically identical tasks across cells. complexityBucket was derived from raw description length (round(min(len/200,1)*10)) — so a 55-char and 130-char phrasing of the same task landed in different Q-rows, and QWEIGHT_RAMP_VISITS=20 was rarely reachable per cell even after the fix #1/#2 loop closure. State key is now 3-dim: taskType|priority|domain Same change applied at both ends of the loop: - buildRoutingStateKey (agent-routing.ts) — parameter and key shape - updateHookRouterQValue (hooks-dream-learning.ts) — state_key SQL - qe-reasoning-bank.ts reader-side call - task-hooks.ts post-task writer call - routing-hooks.ts post-route writer call (new in fix #1) `deriveComplexityBucket` stays exported (marked @deprecated for state_key use) — the pre-task bridge payload still carries `complexityBucket` for kv_store schema compatibility and any downstream telemetry. `TaskBridgePayload` interface unchanged. Migration: existing rl_q_values rows written under the previous 4-dim key are orphaned. Field reports show real installs have at most ~2 such rows (the loop was broken; no real Q-data accumulated), so we accept the orphans rather than ship a migration. If a future install carries substantial 4-dim data, a one-shot script that strips the trailing |N segment is straightforward. Test contract extended: - agent-routing-exploration.test.ts pins the 3-segment key shape - route-q-loop-closure.test.ts asserts complexityBucket does NOT leak into updateHookRouterQValue payloads (writer-side check) and that bridge.complexityBucket is ignored even when present. ADR-096 updated: complexity-bucket promoted from Open Questions to Decision (Option A). Open Questions now lists "re-introducing a coarse complexity tier if telemetry shows the 3-dim key conflating decisions that benefit from differentiation" as a wait-and-measure follow-up rather than a planned change. Refs: #499 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
d41bbc22b0 |
fix(routing): close Q-loop on routing surface (#499)
The route hook (UserPromptSubmit) reads `rl_q_values` to produce qWeight, but the only writer was task-hooks `post-task` gated on `outcome.bridge`. On real projects, `routing_outcomes` reached 139 rows while `rl_q_values` stayed at 2 — the route surface that displays qWeight had nothing to read, so `qWeight: 0` was structurally pinned. Two complementary producer additions, ADR-096: Fix #1 — post-route trains rl_q_values When the Stop hook resolves a routing_outcomes sentinel, derive a state_key from the sentinel's `task_json.description` using the same exported helpers the route hook used (`deriveTaskType`, `deriveComplexityBucket`, `detectQEDomains`) and call `updateHookRouterQValue`. Writer and reader address byte-identical rows. Best-effort + try/catch: Stop hooks must never crash the host. Fix #2 — post-task fires even when bridge is absent Drop the `if (outcome.bridge)` gate. When bridge is missing, derive state from `options.description` instead. Adds a new `--description` option to `aqe hooks post-task` so the PostToolUse hook command can pass `tool_input.description` for Task tools. Skip the update only when neither bridge nor description is available (would pollute an "unknown" sentinel state). Tests: tests/unit/cli/commands/route-q-loop-closure.test.ts pins the producer/consumer state_key alignment contract — if a future refactor desynchronizes the three call sites (routing-hooks.ts post-route, task-hooks.ts post-task, qe-reasoning-bank.ts:530), assertions fail the build. Closes the Open Question flagged in ADR-095. The deriveComplexityBucket fragmentation Jordi noted is acknowledged in ADR-096's Open Questions and tracked for a follow-up — orthogonal to closing the loop. Refs: #499 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6236a2cdc7 |
feat(kernel): kernel-side dream cycles + hooks-as-producers boundary (ADR-094, #488 Phase 2)
Moves dream cycles out of hook subprocesses into the long-lived kernel
process via the DreamScheduler that already existed but was never wired.
Single cutover, no feature flag — per maintainer decision on ADR-094.
Why this matters
================
Before this commit, every ~20 Edit/Write tool uses triggered a dream cycle
inside a short-lived `npx aqe hooks post-edit` subprocess. The cycle held
a SQLite write transaction for up to 10 seconds while the kernel-owning
MCP server, other hook subprocesses, and user MCP tool calls queued
behind it. stderr from the cycle was consumed by Claude Code's hook
reader and lost from operator visibility. Cold DreamEngine init paid
every ~20 edits.
Now the kernel's `QEKernelImpl.initialize()` boots a `DreamScheduler`
that runs cycles on its own cadence inside the long-lived process. Hook
subprocesses keep only `incrementDreamExperience` (a counter bump that
returns in ~50ms). Operators see `aqe learning loop-health` report the
`dreamScheduler` component as live within one tick.
What landed
===========
src/kernel/kernel.ts
- new private `_dreamScheduler?: DreamScheduler`
- new config flag `enableDreamScheduler` (default `true`, mirrors
`enableExperienceBridge`)
- initialize() constructs DreamEngine + DreamScheduler after the
bridge starts, wraps in try/catch (best-effort), failure logs a
warning and continues (kernel boot never blocks on this)
- dispose() stops the scheduler BEFORE bus/memory cleanup so a dream
in-progress doesn't write to a disposed backend
src/learning/dream/dream-scheduler.ts
- executeDream() now records loop-health on every tick (success or
failure) via the helper from #488 Phase 1 B.2. Operators see
`learning:loop-health.components.dreamScheduler.successesSinceBoot`
grow on each cycle.
src/cli/commands/hooks-handlers/editing-hooks.ts
src/cli/commands/hooks-handlers/task-hooks.ts
- dropped `checkAndTriggerDream` import + call
- kept `incrementDreamExperience` (the counter is the producer-side
signal the kernel scheduler reads)
- JSON output now emits `dreamTriggered: false, dreamReason:
'deferred-to-kernel'` so operator scripts grepping the field see
*where* the cycle actually runs
src/cli/commands/hooks-handlers/hooks-shared.ts
- removed the `checkAndTriggerDream` re-export. The function itself
is still defined in hooks-dream-learning.ts and exported there for
direct test access and the documented rollback path.
tests/unit/architecture/hooks-boundary.test.ts (NEW)
- greps `src/cli/commands/hooks-handlers/` for forbidden patterns:
imports from `src/learning/dream/` and references to
`checkAndTriggerDream`. Skips comment lines so explanatory mentions
don't false-positive. `hooks-dream-learning.ts` is in `allowedFiles`
since it DEFINES the function.
- This is the enforcement mechanism for the boundary contract. A
contributor accidentally re-adding heavy hook-side work gets a
clear CI failure pointing them at ADR-094.
tests/unit/kernel/dream-scheduler-wiring.test.ts (NEW)
- asserts `enableDreamScheduler` default, opt-out path, init+dispose
cleanliness with and without the scheduler, idempotent dispose
tests/unit/cli/commands/post-edit-dream-trigger.test.ts (REWRITTEN)
- was asserting checkAndTriggerDream was called from post-edit (the
#480 fix). That contract is now wrong — the kernel is the only
place that triggers dreams.
- now asserts: counter is bumped (the kernel reads it), no inline
cycle trigger, JSON emits `deferred-to-kernel`, hook stays robust
when the memory backend throws.
docs/implementation/adrs/ADR-094-kernel-side-dream-cycles.md
- status: Implemented
- migration section replaced with the single-cutover rationale and
a documented 4-line rollback path
583/583 tests pass across CLI, workers, bridge, learning, init/phases,
architecture, and kernel suites. Build clean.
Boundary contract (now enforced in CI)
======================================
Hook subprocesses MUST NOT do work that exceeds ~100ms. Acceptable
hook-side work: producer SQLite writes, counter increments, in-memory
routing lookups. Unacceptable: DreamEngine init, multi-second SQLite
transactions, anything that depends on long-lived state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
e266bae929 |
refactor(learning): single writer for pattern usage — qe_pattern_usage + qe_patterns columns (#486 Gap B)
Before this commit two parallel writers bumped pattern usage stats:
- SQLitePatternStore.recordUsage — did BOTH qe_pattern_usage INSERT
and qe_patterns UPDATE inside one transaction (correct).
- hooks-dream-learning.ts inline UPDATE — bumped qe_patterns columns
but SKIPPED the qe_pattern_usage INSERT, so the audit table stayed
EMPTY in production even as qe_patterns.usage_count climbed during
normal hook activity. V0.03 evidence: 0 rows in qe_pattern_usage
despite 33 hits on the dominant cli-hook-code-intelligence pattern.
Extracts the shared accounting into a new helper
src/learning/pattern-usage-recorder.ts with a single transaction:
- INSERT INTO qe_pattern_usage (audit trail per use)
- UPDATE qe_patterns.{usage_count, successful_uses, success_rate,
quality_score, last_used_at, updated_at}
Both writes are atomic — either both land or neither does.
Quality formula matches the canonical pattern-lifecycle.ts shape:
quality_score = confidence*0.3 + min(usage_count/100, 1)*0.2 + success_rate*0.5
Returns updated=false (no throw) when patternId is unknown, because hook
subprocesses iterate selectedPatternIds that may contain stale UUIDs
from a routing kv that drifted from qe_patterns. Callers wanting strict
semantics (SQLitePatternStore.recordUsage) check the flag and throw.
Refactors:
- SQLitePatternStore.recordUsage → thin wrapper around the helper.
Preserves the throw-on-missing contract.
- hooks-dream-learning.ts per-pattern inline UPDATE → replaced with
recordPatternUsage() call. Now passes meaningful metrics
(tokens_saved, source, experience_id) and feedback to the audit row.
Keeps the #455 inline promotion check (separate concern, fires
AFTER the helper has updated the qe_patterns columns).
Invariant established: COUNT(*) FROM qe_pattern_usage WHERE pattern_id=X
equals qe_patterns.usage_count for X. Before this commit that invariant
was systematically violated by the hook path.
Test infrastructure:
- 7 new tests in tests/unit/learning/pattern-usage-recorder.test.ts
cover atomicity, the missing-pattern no-throw contract, the formula
boundary at usage_count=100, null metrics/feedback handling, and
the audit/column count invariant.
- tests/unit/cli/commands/post-task-promotion.test.ts schema seeds
now include qe_pattern_usage (the test fixture was missing the
table that the production schema creates at boot).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
6655314120 |
fix(hooks): wire post-edit dream trigger (#480) + drop pre-task bridge pattern-count gate (#487)
post-edit was initializing `dreamTriggered = false` and only calling
`incrementDreamExperience` — `checkAndTriggerDream` was never invoked.
Since post-edit fires far more often than post-task during Claude Code
sessions, the hook-driven dream loop was starved: `experienceCount`
grew monotonically but `lastDreamTime` stayed null and
`dream_insights.applied` stayed at 0.
Mirrors task-hooks.ts: call `checkAndTriggerDream` after bumping the
counter and surface the full {triggered, reason, insightsGenerated,
insightsApplied} shape in JSON output so operators can distinguish
"not triggered because of state" from "not triggered because we
never tried."
The pre-task bridge was gated on `selectedPatternIds.length > 0`,
which silently dropped the `task-bridge:*` kv write for low-confidence
prompts. post-task's `if (outcome.bridge)` then skipped
`updateHookRouterQValue`, so `rl_q_values` never populated. The
Bellman update's state_key uses (taskType|priority|domain|
complexityBucket) and action_key uses `routing.recommendedAgent` —
neither requires non-empty patterns. The downstream per-pattern
fan-out at hooks-dream-learning.ts:428 already guards
`bridge.selectedPatternIds.length > 0` before iterating, so writing
the bridge with `[]` is structurally safe.
Tests:
- post-edit-dream-trigger.test.ts: 3 cases covering invocation,
full JSON shape on trigger, and dreamReason surfaced when
conditions are not met
- pre-task-bridge-empty-patterns.test.ts: parametrized over
patterns=[] and patterns=[p1,p2], asserts bridge row exists in
both; separate no-op test for missing --description
125/125 hook-handler unit tests pass; build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
bb34aeace4 |
chore(release): bump version to v3.9.26
Ten learning-pipeline fixes (#453, #454, #455, #456, #460, #461, #462, #463, #464, #465) plus dup closures (#457-#459). Tightens the #465 sentinel sweep with the same task_json discriminator the LIMIT-1 close uses, preserving the #451 ownership split (route sentinels owned by post-route, pre-task sentinels by post-task). Updates the regression suite to reflect the new sweep behavior. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
864766b399 |
fix(hooks): sweep stale routing_outcomes sentinels on every post-route (#465)
post-route closed routing_outcomes sentinels one at a time (LIMIT 1). Sessions that terminate without firing Stop — context compact, process kill, IDE crash — leave their sentinels permanently at quality_score=-1, and newer sessions keep pre-empting the old rows in ORDER BY DESC. The reporter saw 122 of 149 rows stuck at -1, inverting AVG(quality_score) from a real +0.666 to an observed -0.717. After the LIMIT-1 close, run a second UPDATE that closes any sentinel older than 5 minutes with quality_score=0.325 (conservative base — no success or duration bonus) and error='stale-sentinel'. Using the base score avoids attributing the current turn's outcome to unknown historical turns; the error tag lets precision-sensitive queries filter these rows. Surface the swept count in --json output (`staleSwept`) and a dim "swept N stale sentinel(s)" line in pretty output for observability. 4 regression tests cover: fresh + stale rows resolved together, no-op when no stale rows exist, already-resolved rows are untouched, and pre-task sentinels (task_json carries "taskId") still wait for post-task to close them via the existing LIMIT-1 path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6a91660863 |
fix(hooks): remove over-broad cli-hook agent exclusion from consolidation (#464)
consolidateExperiencesToPatterns() filtered out every row tagged agent='cli-hook' because issue #348 had observed low-quality Bash telemetry (quality ~0.40, success_rate ~0.24) flooding the pipeline. But post-edit also uses agent='cli-hook', and those rows have quality ~0.75 / success_rate ~1.0 — well above the HAVING thresholds. The agent-name filter blanket-excluded the dominant share of high-quality experiences (reporter saw 128 cli-hook rows ineligible, 0 patterns created). The HAVING clause below the WHERE is the real quality gate (avg_quality >= 0.5 AND success_rate >= 0.6 AND cnt >= 3) and already filters out the #348 shape. Drop the agent-name filter; tighten HAVING if the concern resurfaces. 4 regression tests verify: high-quality cli-hook rows now create a pattern, low-quality cli-hook rows are still gated by HAVING, non-cli-hook agents consolidate as before, and processed rows get application_count bumped. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
f0affe3520 |
fix(hooks): populate tokens_saved with qualityScore*100 in post-task (#463)
The base experience_applications insert hardcoded tokens_saved=0 on every row, so the learning-ROI column was permanently useless across every post-task invocation. qualityScore is already computed three lines above and ranges 0.335 (failure + slow) to 0.675 (success + fast). Replace the literal 0 with Math.round(qualityScore * 100) — gives a non-trivial 34-68 signal that downstream analytics can reason about, until a real per-task token-delta calculation is wired in. 4 regression tests cover the four canonical (success, duration) buckets and verify the value is never zero: - success + <100ms → 68 (reporter's exact evidence case) - failure + ≥10000ms → 34 - success + <2000ms → 64 - worst-case row is still > 0 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
b313297471 |
fix(hooks): guard dream cycle trigger against concurrent starts (#461)
When dream eligibility conditions are met, every concurrent Claude Code hook subprocess sees the same stale dreamState.lastDreamTime, races past the trigger, and starts its own ~10s dream cycle. They all hit the WAL writer at roughly the same moment and ~35% fail with `database is locked` (duration_ms ≈ 10050ms — the cycle ran, only the write-back failed). `busy_timeout` cannot help here: it serializes sequential contention, not simultaneous writers. Peek at dream_cycles before opening the engine: if a row with status='running' AND start_time > now-60s exists, return early with reason='already-running'. Fail-open — any error in the guard (table missing, unified memory not yet initialized) lets the dream proceed, so this can never make things worse than they were before. 3 regression tests cover: guard fires when a running cycle is recent, guard misses when none exists, and fail-open path when the guard query throws. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
79c9a99f9c |
fix(hooks): use bridge.agent as action_key in Q-learning when --agent is unavailable (#460)
The post-task hook command passes `--agent "$TOOL_RESULT_agent_id"`, but Claude Code does not expose that env var in PostToolUse context — so `options.agent` arrives empty, the local `agent` falls back to 'unknown', and every rl_q_values row used `action_key='unknown'`. The Q-learning router could never learn per-agent differentiation. The pre-task bridge payload already carries `agent: routing.recommendedAgent`, which is the right action key. Compute an `effectiveAgent` after persistTaskOutcome returns: prefer an explicit --agent value (user override), otherwise fall back to bridge.agent, and only land on 'unknown' when neither is available. Use it for both `updateHookRouterQValue` and `updateRoutingOutcomeQuality` (the latter uses agent as a tiebreaker in the routing-outcome sentinel match, so it benefits from the same correction). 4 regression tests cover: bridge fallback when --agent absent, bridge fallback for explicit empty --agent, explicit user override wins, and falls through to 'unknown' when neither source is available. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6d5d147146 |
fix(hooks): apply dream insights after each dream cycle (#456)
`checkAndTriggerDream()` called `dreamEngine.dream()` and persisted insights with applied=0, then closed the engine. The hook-driven path had no equivalent of `DreamScheduler.autoApplyInsights()` — so every hook-fired dream cycle left actionable insights piling up indefinitely (reporter saw 378 unapplied vs 9 applied; all unapplied rows had pattern_id=null). After `dream()` returns, iterate `result.insights` and call `engine.applyInsight(id)` for each `actionable && confidenceScore >= 0.5` (matching `DreamSchedulerConfig.insightConfidenceThreshold` default and the `InsightGenerator.actionable` gate). Best-effort try/catch so a single failing apply does not poison the rest of the loop. Surface `insightsApplied` through `checkAndTriggerDream`'s return value into `post-task --json` output and the dream-cycle console line so the behavior is observable. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
db2b7113be |
fix(hooks): promote patterns in bridge loop after usage update (#455)
`recordOutcome()` is always invoked with a synthetic `task:agent:taskId` patternId that never matches qe_patterns.id, so checkPatternPromotionWithCoherence() was skipped for the real pattern UUIDs. The post-task bridge loop in persistTaskOutcome updates usage_count / successful_uses / success_rate for each selected pattern but did not check promotion — patterns accumulated past the thresholds (successful_uses ≥ 3, success_rate ≥ 0.7, confidence ≥ 0.6) and stayed `tier='short-term'` forever. Add an inline re-read + guarded UPDATE inside the per-pattern bridge loop. The UPDATE's WHERE clause re-checks the thresholds atomically so it's idempotent (already-long-term rows are untouched, below-threshold rows stay short-term). Best-effort try/catch keeps a single corrupt row from killing the rest of the loop. Regression test seeds short-term patterns at the edge of each threshold and asserts (a) promotion fires when all thresholds cross, (b) it does NOT fire when success_rate or confidence is below threshold, (c) it's idempotent on already-long-term rows. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
53b3faf72c |
fix(hooks): post-edit reads file_path from stdin event when --file is empty (#453)
`$TOOL_INPUT_file_path` does not expand reliably in Claude Code PostToolUse context, so the init template's `--file "$TOOL_INPUT_file_path"` produced `--file ""` for ~88% of turns. Every captured_experiences row ended up tagged `task = "edit: "` with no path — embeddings were useless, pattern lookups matched nothing, and consolidation never fired. post-edit now mirrors the route hook: when --file is empty, read the Claude Code hook event JSON from stdin and pull tool_input.file_path. The shared readStdinJsonEvent helper moves from routing-hooks.ts into hooks-shared.ts alongside a new extractFilePathFromEvent helper that handles snake_case/camelCase envelopes and ignores non-string values. Verified end-to-end: piping a PostToolUse event to `aqe hooks post-edit --file "" --success --json` now produces `task = "edit: /workspaces/agentic-qe/src/example.ts"` in captured_experiences instead of the empty path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
b2e6090c6d |
chore(release): v3.9.25 (#452)
* feat(hooks): post-route Stop hook closes route sentinels 1:1 (#451) The `route` hook fires on UserPromptSubmit and writes a sentinel row to routing_outcomes with quality_score=-1, expecting `post-task` to close it. But `post-task` only fires on PostToolUse ^(Task|Agent)$, which is rare in direct-work sessions (Bash/Edit/Read without spawning sub- agents). Sentinels accumulated indefinitely — Jordi reported 123+ unresolved rows per session. Stream D never converged. Changes: 1. New `aqe hooks post-route` subcommand (src/cli/commands/hooks-handlers/routing-hooks.ts) Closes the most-recent unresolved route sentinel using a `task_json NOT LIKE '%"taskId"%'` discriminator that isolates route entries from pre-task entries. Quality formula collapses the 6-dim outcome formula for unknown duration: 0.325 + 0.25·success + 0.10·1.0 = 0.675 (success) / 0.425 (failure) 2. Inverse discriminator on updateRoutingOutcomeQuality (src/cli/commands/hooks-handlers/hooks-dream-learning.ts) Added `task_json LIKE '%"taskId"%'` to the WHERE clause so post-task only ever closes pre-task sentinels. Without this, post-task could grab a stale route sentinel by accident and miscredit duration to a row that wasn't about a Task() call. Symmetric counterpart to the post-route filter. 3. Stop-hook wiring in both init template paths (src/init/phases/07-hooks.ts + src/init/init-wizard-hooks.ts) Added `npx agentic-qe hooks post-route --success true --json` to the Stop block. Timeout 5000ms, continueOnError: true, so a failed post-route cannot block session teardown. 4. CLI help text (src/cli/commands/hooks.ts) Added example for the new subcommand. 5. Regression test (tests/unit/cli/commands/post-route.test.ts) 4 tests using in-memory better-sqlite3: - Closes most-recent route sentinel; leaves pre-task sentinels and resolved rows untouched - quality_score = 0.425 when --success false - No-op when only pre-task sentinels exist (discriminator works) - No-op on second invocation when only one sentinel was outstanding Bug-introduce verification: stashed routing-hooks.ts, ran new test — all 4 cases failed (Commander rejects unknown subcommand). Restored fix, all 4 pass. Test distinguishes pre-fix and post-fix states. Co-authored-by: Jordi Izquierdo <noreply@github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(release): bump version to v3.9.25 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
ca714ac1a7 |
chore(release): v3.9.24 (#450)
* fix(hooks): post-task runs Stream B/D/F when --task-id is missing (#449) The shipped PostToolUse hook command for Task/Agent matchers is `npx agentic-qe hooks post-task --task-id "$TOOL_RESULT_agent_id" --success --json` Claude Code does not populate $TOOL_RESULT_agent_id, so the flag arrives empty and options.taskId is falsy on every real hook invocation. The `if (options.taskId)` gate inside post-task then short-circuited the entire Stream B/D/F learning chain: - reasoningBank.recordOutcome — never called - persistTaskOutcome — never called (no captured_experiences, experience_applications, qe_trajectories, dream_insights.applied) - updateRoutingOutcomeQuality — never called (routing_outcomes sentinels accumulated unfilled at success=0/quality_score=-1) - updateHookRouterQValue — never called (rl_q_values stayed empty forever) Same regression was fixed in v3.9.21 (patch 030) and silently came back in v3.9.23. Fix: - Remove the `if (options.taskId)` gate in post-task. Use `hook-${Date.now()}` fallback for the patternId / experience id when the caller didn't supply one. The bridge lookup in persistTaskOutcome keys by `ORDER BY created_at DESC LIMIT 1` (not by taskId), so Stream F's Bellman Q-update still finds the bridge payload it needs. - Remove the parallel `&& options.taskId` clause from the pre-task routing_outcomes sentinel write at line 195 — same root cause. - Add `DO NOT REINTRODUCE` comment blocks at both sites pointing at #449 and patch 030. Test: tests/unit/cli/commands/task-hooks-no-taskid.test.ts mocks hooks-shared and asserts persistTaskOutcome / updateHookRouterQValue fire when --task-id is absent. Bug-introduce verification: re-introducing the gate makes the no-taskId test fail with the expected "called 1 times, but got 0 times". Co-authored-by: Jordi Izquierdo <noreply@github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(release): bump version to v3.9.24 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
5f711b5b3e |
fix(hooks): UserPromptSubmit reads stdin event JSON for routing (v3.9.17)
The shipped `aqe init` template wired the UserPromptSubmit hook to
`npx agentic-qe hooks route --task "$PROMPT" --json`, but Claude Code
never exposed `$PROMPT` as an env var — only stdin event JSON. The
shell expanded `"$PROMPT"` to `""`, so every prompt routed as empty
and `routing_outcomes.task_json` was persisted as `{"description":""}`,
silently breaking the routing learning loop.
- routing-hooks.ts: demote --task to optional, read stdin (500ms
timeout) and extract prompt/user_prompt/command/tool_input.{prompt,
description}; explicit error when neither source has content
- 07-hooks.ts and init-wizard-hooks.ts: drop --task "\$PROMPT", let
the CLI read stdin directly. isAqeHookEntry still matches both
shapes, so existing projects upgrade on `aqe init --upgrade`
- tests/unit/cli/commands/hooks-routing.test.ts: 10 unit tests
covering all event shapes, fallback ordering, malformed JSON, and
empty-field skip
Verified end-to-end: piped a Claude Code-shaped event into a fresh
project's configured route command and confirmed task_json captures
the real prompt instead of "". Backward-compat manual --task usage
preserved.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
2a6036c2b8 |
feat(completions): add brain + upgrade to bash/zsh/fish/PowerShell
Tab completion was not generating suggestions for the newly shipped
\`aqe brain diff\`, \`aqe brain search\`, or \`aqe upgrade\` commands on
any shell. This commit wires them in so users with installed completion
scripts get the same discoverability as for older commands.
Changes:
- src/cli/completions/index.ts
* COMMANDS map: add \`brain\` (with subcommands export/import/info/
diff/search/witness-backfill) and \`upgrade\` (with --json/--strict).
* Bash generator: extend top-level commands string and add case
branches for \`brain\`/\`upgrade\`, with a \`--domain\` completion
sourced from the existing domains list for \`brain search\`.
* Zsh generator: add entries to the \`commands\` array and two new
subcommand dispatch cases.
* Fish generator: add top-level completions plus per-subcommand
guards using __fish_seen_subcommand_from.
* PowerShell \$AQE_COMMANDS map: add both entries.
- tests/unit/cli/completions.test.ts
* New assertions: COMMANDS.brain + .upgrade shape, and that all four
shell generators contain both names.
Also cleaned up 22 pre-existing no-useless-escape lint errors in the
same file (unnecessary \\$ inside JS template literals — \$ alone is
already literal). Kept \${files[@]} because ${ would trigger JS
template interpolation. Output of generateBashCompletion() is
byte-identical after this cleanup; verified via 'bash -n' syntax check.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
3f743ada46 |
chore(adr-093): sweep remaining model refs in opencode plugin, scripts, .kiro, command docs
91 files changed, +99/-99. Fifth and final post-audit sweep. Addresses all gaps surfaced by the third broader audit pass. * packages/aqe-opencode-plugin/src/degradation/graceful-degradation.ts: tier map updated — anthropic/claude-opus-4 → -4.7, anthropic/claude-sonnet-4 → -4.6, anthropic/claude-haiku-3.5 → -4.5. JSDoc example model ID also refreshed. This package is shipped to OpenCode users. * scripts/adr-092-multi-model-trial.ts: EXECUTOR_MODEL constant updated to anthropic/claude-sonnet-4.6 (sibling of the harness already fixed in commit |
||
|
Dragan Spiridonov
|
da638f9079 |
feat(kernel): migrate NativeHnswBackend from @ruvector/router to hnswlib-node (#399)
@ruvector/router 0.1.28 was found to have four serious bugs that
together broke production code-intelligence semantic search and
forced the v3.9.5 hotfix to disable native HNSW entirely:
1. HNSW search returned essentially random results — recall@10 was
0% on the project's FNV fixture and 10% on textbook unit-Gaussian
random vectors. The graph walk could not even find self-vectors
when querying with a stored vector. Reproduced deterministically
on linux-arm64; matches the linux-x64 GitHub Actions failures
reported in #399.
2. The VectorDb constructor unconditionally wrote a `vectors.db`
redb file to the current working directory, polluting users'
project roots with multi-MB files outside .agentic-qe/, in a
non-SQLite format that no AQE migration or backup tooling knew
about. Direct violation of the unified memory architecture.
3. The redb file held a process-wide exclusive lock — only ONE
VectorDb instance could exist per process. Subsequent constructors
threw "Database error: Database already open. Cannot acquire lock."
4. NAPI dispose did not synchronously release the redb lock (NAPI GC
is not synchronous). The lock outlived dispose() and caused the
v3.9.5 futex deadlock when the indexer tried to recreate after
reset.
NativeHnswBackend is rewritten to wrap hnswlib-node@^3.0.0 — the
canonical Node.js binding for Yury Malkov's reference C++ Hnswlib
implementation, used by Pinecone, Weaviate, Qdrant, LangChain, and
ChromaDB. Empirical verification on the same fixture
(`scripts/diagnose-issue-399*.mjs`):
- hnswlib-node: recall@10 = 100%, top-1 = id 42 ✓
- @ruvector/router: recall@10 = 10%, top-1 = id 999 (random)
Insert is also faster (~324ms vs ~430ms for 1k vectors) and search
is comparable (0.35ms vs 0.21ms with correct vs wrong results).
No new dependency was added — `hnswlib-node` was already a direct
dependency in `package.json` from the legacy code path in
`src/integrations/embeddings/index/HNSWIndex.ts`.
Implementation notes:
- Drops the local `vectorStore` mirror that the previous backend
needed to re-score wrong native results — hnswlib-node returns
correct distances directly.
- Drops the `bruteForceSearch` private fallback — there is no
fallback path because hnswlib-node always returns correct results.
- `clear()` now recreates the underlying HierarchicalNSW from
scratch instead of just markDelete-ing every label. The
tombstone-only approach leaks state across long test runs that
share the singleton HnswAdapter registry, producing duplicate
labels from `searchKnn` after many cycles.
- `search()` overshoots k by 2x and dedups by id as a defensive
measure against any future hnswlib-node duplicate-label edge
case, then truncates back to k.
- `add()` doubles maxElements via `resizeIndex()` when the live id
count fills the current capacity, so callers don't need to know
the final index size at construction time.
- `useNativeHNSW` default flipped back to `true`. Users with large
codebases get sublinear HNSW search by default again.
- All four `@ruvector/router` failure modes have explicit regression
guards in the unit and integration test suites.
The two #399-skipped tests in `phase1-integration.test.ts`
(line 222 and 1185) are un-skipped and now pass green against the
new backend. The three flag-default tests that asserted
`useNativeHNSW: false` (broken by v3.9.5's hotfix) are updated to
assert `true` again.
Stale comments referencing `@ruvector/router`'s lock behavior are
updated in `src/kernel/unified-memory.ts`, `src/init/phases/06-code-intelligence.ts`,
`src/init/phases/10-workers.ts`, and `src/cli/index.ts`. The CLI
status output (`aqe ruvector status`) and the `useNativeHNSW` flag
description are updated to mention `hnswlib-node` instead of
`@ruvector/router`.
Refs #399, ADR-090.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
9e314ff383 |
fix(tests): unblock full unit suite and drop force-exit hack (#396)
The full `npx vitest run tests/unit/` suite was hanging >22 minutes
since v3.9.1 due to a futex deadlock in `@ruvector/router` native NAPI
calls during HNSW VectorDb operations. v3.9.5 already addressed the
root cause by flipping `useNativeHNSW` default from true to false
(commit
|
||
|
Dragan Spiridonov
|
73eab06585 |
feat: implement Six Hats Tier 3 Phase 1 — YAML pipelines, heartbeat CLI, economic routing (#334)
Three strategic features from Issue #334 with full CLI-MCP parity: Imp-9: YAML Deterministic QE Pipelines - Approval gate step type with auto-approve timeout and cancel cleanup - 4 deterministic actions (quality-gate, coverage, pattern-health, routing) with real SQL queries and graceful DB degradation - 7 CLI commands (aqe pipeline load/validate/run/list/status/approve/reject) - YAML loader validation for approval fields Imp-10: Token-Free Heartbeat CLI & MCP - 6 CLI commands (aqe heartbeat status/run-now/history/log/pause/resume) - 3 MCP tools (heartbeat_status/trigger/log) - History persistence, date validation, timeout support Imp-18: Economic Routing Model - EconomicRoutingModel with quality-per-dollar scoring and EMA estimates - Budget-aware tier selection with fallback strategies - Cost-adjusted rewards for neural router (opt-in) - CLI commands (aqe routing economics/accuracy/metrics) - routing_economics MCP tool with kv_store persistence Security: path traversal fix on CLI log reader, config weight validation 71 new tests across 6 test files, all passing. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
fdd32a42ce |
fix(tests): add vi.useFakeTimers() to 11 files using real setTimeout (#374 P2)
11 test files used real setTimeout without vi.useFakeTimers(), adding wall-clock time to sequential CI execution. Converted to fake timers with vi.advanceTimersByTime() for deterministic, instant execution. Files fixed: - error-paths/ (5 files): circuit breaker, workflow timeout, consensus timeout, task timeout, concurrent init — ~33 setTimeout patterns - kernel/ (3 files): TTL expiration tests that waited 1100ms real time - mcp/metrics (1 file): 7 timing-measurement tests - cli/progress (1 file): 3 ETA estimator tests - memory/crdt (1 file): convergence time tracking Estimated CI time savings: ~10s per sequential run. 11 files intentionally skipped (real I/O, production timer logic): websocket-transport, mcp-server, adr-040, swarm-skill-validator, plan-executor, parallel-eval-runner, failover, behavior-tree, etc. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
00e4d2cfa6 |
fix(tests): add afterEach cleanup to 104 test files missing it (#374 P2)
125 test files had beforeEach/beforeAll but no afterEach/afterAll, leaking state between tests and causing order-dependent failures. Added cleanup to 104 files (skipped 4 archived, 16 import-only): - 2 critical DB handle leaks (witness-chain): added db.close() - 7 high-risk service files: added vi.restoreAllMocks() - 7 beforeAll files: added matching afterAll - 88 medium/low risk: added afterEach with appropriate cleanup Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
c911aa88a8 |
feat: enable all 15 RuVector feature flags by default (opt-out instead of opt-in)
All new RuVector capabilities (native HNSW, neural routing, coherence gates, transfer learning, etc.) now ship enabled so users get immediate value on upgrade. Users can disable individual features via CLI or setRuVectorFeatureFlags() for opt-out. Update 11 test files to match new defaults. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6b249c034f |
feat: RuVector integration phases 1-4 — native HNSW, neural routing, safety gates, differentiation
Implements 23 tasks across 4 phases per docs/implementation/ruvector-integration-plan.md: Phase 1 — Foundation (make what we have faster): - Native HNSW backend via @ruvector/router (150x search speedup, ADR-081) - Metadata filtering layer with composable AND/OR/NOT/FIELD expressions - Temporal tensor compression (Int8Array quantization, 4x memory reduction) - Deterministic golden-ratio dithering for cross-platform reproducibility - 15 feature flags (all default false) with CLI profiles Phase 2 — Intelligence (make agents smarter): - Neural TinyDancer router with REINFORCE policy gradient (ADR-082) - SONA three-loop enhancement (MicroLoRA 0.18μs/adapt via WASM) - Cross-domain transfer learning with Thompson Sampling (ADR-084) - Regret tracking with log-log regression growth classification Phase 3 — Safety (make agents trustworthy): - Sheaf-gated test validation via prime-radiant CohomologyEngine - Coherence-gated agent actions (3-filter pipeline, ADR-083) - Witness chain audit trail with SHA-256 hash linking - HNSW health monitoring with spectral analysis Phase 4 — Differentiation (make AQE unique): - Cognitive container export/import (RVF v2) - DAG attention scheduler for dependency-aware test ordering - CNN visual regression with spatial pooling embeddings - Behavior tree orchestration (Sequence/Selector/Parallel) - Reasoning QEC with majority-vote consensus - Browser QE dashboard scaffolding Also: remove obsolete v2-to-v3-migration test (source already removed), add ADR-081 through ADR-085, research docs, benchmarks, and integration test suites for all 4 phases. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
20b098697e |
fix: add regression tests for march-fixes branch and fix 4 flaky tests
Move @ruvector/tiny-dancer-linux-arm64-gnu from dependencies to optionalDependencies to prevent npm install failures on non-ARM64. Add 26 regression tests across 5 files covering untested new paths: - task-executor coherence gate & reasoning QEC advisory paths - aqe-learning-engine witness chain & domain transfer wiring - experience-capture witnessHash & setWitnessChain integration - pattern-store filter-adapter module resolution & search compat - metrics-tracker getDashboardDataWithRegret method Fix 4 flaky tests failing under full-suite contention by resetting shared singletons (queenGovernanceAdapter, sharedMinCutGraph, UnifiedPersistence) in beforeEach/afterEach to prevent cross-test state contamination. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |
||
|
Profa
|
416c97ab39 |
feat(learning): implement Six Hats Tier 0-2 improvements and fix test flakiness
Tier 0 (Bug Fixes):
- Fix promotion threshold inconsistency (standardized on 3)
- Add embedding backfill script and ghost pattern health check
- Add FTS5 virtual table with hybrid vector/text search
Tier 1 (Quick Wins):
- Binary reward assignment (+1/-1/0) for outcome-based learning
- At-least-one learning guarantee per session
- Temporal decay on search results (30-day half-life)
- Configurable promotion activity window
- Enable coherence gate by default
Tier 2 (Medium Investments):
- Pre-compaction flush hook to prevent knowledge loss
- OPD remediation hints ("bad because X, fix by Y")
- Per-agent MCP tool scoping for security isolation
- Daily Markdown log tier for audit trail
- Proof-of-Quality CLI command with SHA-256 attestation
- Devil's advocate scoring fixes (FTS normalization, cap, decay, freshness)
Test Fixes:
- Fix queen governance flakiness in domain-handlers tests
- Fix cross-suite singleton contamination in handler-factory and e2e tests
- Fix mcp-server domain tool timeout and invocation failures
610 test files pass, 0 failures, 18,589 tests green.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
||
|
Profa
|
ec2c243172 |
fix(embeddings): complete 384-dim standardization across tests and providers
- Update default embed model from nomic-embed-text to all-MiniLM-L6-v2 in Ollama LLM provider - Fix 15 test files with hardcoded 768-dim references - Update mock model names in Ollama health checks - Update comments referencing old model name Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Profa
|
2253d05618 |
feat: flatten project structure — archive v2, promote v3 to root
Remove the nested v3/ subdirectory by moving all active source code, tests, configs, and assets directly to the project root. Archive v2/ (dead code, 847MB) to v2-archive.tar.gz. Key changes: - Move v3/src/ → src/, v3/tests/ → tests/, v3/packages/ → packages/ - Move v3/tsconfig.json, v3/vitest.config.ts to root - Merge v3/package.json into root package.json (ESM, scripts, deps, exports) - Merge v3/scripts/ into scripts/, v3/implementation/ into docs/implementation/ - Update all CI workflows to remove "cd v3 &&" and v3/ path prefixes - Fix all init installer path resolution (3-level-up → 2-level-up) - Fix hooks to use "node ./dist/cli/bundle.js" instead of bare "aqe" - Update settings.json ADR/DDD directories to docs/implementation/ - Fix helper scripts (statusline, adr-compliance, ddd-tracker, guidance-hooks) - Update 58 agent definitions: aqe/v3/ → aqe/ memory namespaces - Update 15 guidance shards: v3/src/ → src/ path references - Fix sync interfaces: remove v3/ and v2/ path references - Fix test path calculations (process.cwd() + '..' no longer needed) - Fix flaky perf test threshold (500ms → 1000ms for CI environments) - Fix complexity-analyzer test to match ADR-051 keyword-based eligibility Build passes, 17,901 tests pass, CLI and MCP bundles verified. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
Profa
|
fd2f37eaa0 |
fix(v3): update README claims and init hooks to use correct package name
Changes: - Fix README.md agent counts (48 QE agents = 41 main + 7 subagents) - Fix README.md skill counts (15 QE-specific skills) - Fix v3/README.md agent counts (47 → 48) - Fix init-wizard.ts hooks from 'npx @agentic-qe/v3' to 'npx agentic-qe' - Complete v2 → v2/ folder reorganization (54 items moved) - Update root package.json to point to v3 as main version Verified claims: - 48 QE agents (41 main + 7 TDD subagents) - 15 QE-specific skills - 12 DDD bounded contexts - ~5,600+ tests - HNSW, ReasoningBank, Queen Coordinator features exist Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Profa
|
8bb2ad1aee |
feat(v2.6.6): Web-Tree-Sitter migration, MinCut analysis, CriticalPathDetector
## Web-Tree-Sitter Migration (Phases 1-5) - New WebTreeSitterParser replacing native tree-sitter bindings - Eliminates npm install warnings about native compilation - Updated all language extractors (TS, JS, Python, Go, Rust) ## MinCut Analysis Integration (Phases 1-6) - New src/code-intelligence/analysis/mincut/ module - MinCutAnalyzer: Stoer-Wagner algorithm for graph partitioning - GraphAdapter: Code graph to MinCut format conversion - CircularDependencyDetector: Dependency cycle detection - ModuleCouplingAnalyzer: Module coupling analysis ## CriticalPathDetector for Coverage Analysis - New src/coverage/CriticalPathDetector.ts - Identifies structurally critical code paths using MinCut - Prioritizes coverage gaps by criticality score - Integrated with CoverageAnalyzerAgent ## Fleet Topology Management - New src/fleet/topology/ module - MinCut-based test file partitioning - Integrated with test-execute-parallel MCP handler ## Other Changes - MCP tool count corrected: 102 → 105 tools - New CLI commands: aqe kg mincut, aqe kg circular - 19 new tests for CriticalPathDetector 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Profa
|
cb27ee303f |
feat(v2.5.0): G4 BinaryCache + G6 OpenRouter hot-swap + ruvLLM integration
## Major Features ### G4: Unified Memory Architecture - BinaryCache Integration - Integrate BinaryMetadataCache with UnifiedMemoryCoordinator - Add TRM pattern caching with 6x faster pattern loading - New methods: cacheTRMPattern(), getCachedTRMPattern(), persistBinaryCache() ### G6: OpenRouter Provider with Model Hot-Swap - Full ILLMProvider implementation for OpenRouter API (300+ models) - Model hot-swapping at runtime without restart - Smart environment detection for auto provider selection ### ruvLLM Integration (Issue #131) - SONA Learning Strategy with MicroLoRA/BaseLoRA/EWC++ - TRM Learning Strategy for trajectory tracking - ReasoningBank integration for pattern storage - 104 new integration tests Closes #131 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> |
||
|
Profa
|
c17f2b612b |
feat: Complete AgentDB v1.2.0 implementation with vector search, QUIC sync, and neural training
This commit completes the full implementation of AgentDB v1.2.0 features that were previously documented but not implemented. All claimed features are now fully functional with real operations replacing metadata flags. Core Implementations: - AgentDBService: Production-ready service wrapping AgentDB operations - EmbeddingGenerator: Dual-mode embeddings (hash-based + ML with all-MiniLM-L6-v2) - QUICServer: TLS 1.3 secured synchronization with <1ms latency - NeuralTrainer: Integration of 9 RL algorithms (Decision Transformer, Q-Learning, SARSA, Actor-Critic, PPO, DDPG, TD3, SAC, DQN) Agent Updates: - BaseAgent: Real AgentDB storage in onPostTask hooks (no more fake flags) - TestGeneratorAgent: Vector search for similar test patterns - CoverageAnalyzerAgent: QUIC sync for gap sharing across agents - FlakyTestHunterAgent: Neural training for flakiness prediction New Features: - Vector embeddings with 384-dimensional MiniLM models - HNSW indexing for 150x faster pattern retrieval - Cross-agent pattern sharing via QUIC (<1ms sync) - Quantization support (4-32x memory reduction) - Graceful degradation with feature flags - Comprehensive error handling and logging Testing: - 170+ integration tests created and passing - Performance benchmarks verify 150x search speedup - Agent execution tests validate end-to-end workflows - QUIC sync tests confirm <1ms latency Documentation: - Complete API documentation for all new services - Architecture diagrams and integration guides - Implementation reports and verification evidence - Migration guide from v1.1.0 to v1.2.0 Code Quality: - Reviewed by specialized reviewer agent: 4.4/5 score - TypeScript strict mode compliance - Production-ready error handling - Comprehensive test coverage Status: ✅ APPROVED FOR v1.2.0 RELEASE 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> |