proffesor-for-testing__agentic-qe/docs/releases/v3.9.31.md

v3.9.31 Release Notes

Release Date: 2026-05-15

Highlights

The self-learning loop ships end-to-end. Before this release, four related issues — #480, #486, #487, #488 — meant the QE fleet's pattern catalog converged on whichever pattern won the first routing decision, post-edit hooks never triggered dream cycles despite the trigger function being correct, and rl_q_values populated but nothing read it.

This release closes all four issues, introduces two architectural decision records (ADR-094 for kernel-side dream cycles + the hooks-as-producers boundary, ADR-095 for three-signal routing with ε-greedy exploration), and adds the aqe learning loop-health operator dashboard. The catalog now genuinely diversifies in practice.

Fixed

• post-edit hook never triggered dream cycles (#480). The hook was initializing dreamTriggered = false as a literal and never calling checkAndTriggerDream. The experience counter grew monotonically but dream_insights.applied stuck at 0 forever. The hook now correctly invokes the trigger and surfaces the full result shape in JSON output. (Note: superseded by ADR-094 — see below — but the original bug fix is in the commit history for reference.)

• Q-learning never received signal for low-confidence prompts (#487). The pre-task hook gated the routing-bridge kv write on selectedPatternIds.length > 0, silently dropping the bridge for every low-confidence prompt. Post-task's Bellman update then had no state to update, so rl_q_values never populated despite routing decisions actively happening. The gate is removed; the bridge writes whenever the task has a description, regardless of pattern match count.

• qe_pattern_usage audit table stayed empty in production (#486 Gap B). Two parallel writers existed for pattern usage — the canonical SQLitePatternStore.recordUsage correctly wrote both the audit row and the aggregate columns in one transaction, but an inline UPDATE in the hook flow skipped the audit INSERT. Operators saw usage_count climbing while the audit table had zero rows. Extracted a shared single-writer helper; both call sites now delegate to it. The invariant COUNT(*) FROM qe_pattern_usage = SUM(qe_patterns.usage_count) is now structurally enforced.

Added

• Kernel-side dream cycles (ADR-094, #488 Phase 2). Dream cycles previously ran in short-lived hook subprocesses, holding the SQLite write transaction for up to 10 seconds while other writers queued behind. Errors went to stderr where Claude Code's hook reader swallowed them. The existing DreamScheduler (built in v3.7 but never wired) now starts from QEKernelImpl.initialize(). Hook subprocesses keep only the cheap experience-counter bump; the long-lived kernel owns the actual cycle. A boundary-enforcement test in tests/unit/architecture/hooks-boundary.test.ts fails CI if any hook handler re-introduces heavy work. Rollback path documented in ADR-094.

• Three-signal agent routing: static + Q-value blend + mincut-gated ε-greedy (ADR-095, #488 Phase 4). The Q-table from #487 is now actually consumed during routing decisions. A sigmoid-normalized Q-value blends into per-agent scores with a qWeight that ramps from 0 (no Q-data) to 0.4 (saturated at 20 visits). Crypto-random ε-greedy selection diversifies the catalog over the top-3 alternatives. The same getSharedMinCutMonitor infrastructure ADR-068 uses for model-tier routing now dampens exploration 5x when swarm topology is critical. Rollback knob: AQE_ROUTER_EXPLORATION_RATE=0.

• mineExperiences now auto-fires (#486 Gap A). The LearningConsolidationWorker runs every 30 minutes and now includes a per-domain mineExperiences sweep with a watermark cursor at learning:consolidation-cursor:{domain}. Cursor advances only on successful mining, so failures and empty windows don't silently consume fresh experiences. learning:pattern:* kv populates in default deployments instead of staying empty.

• aqe learning loop-health operator dashboard (#488 Phase 1 B.2). New CLI subcommand shows liveness of the three loop components (CapturedExperienceBridge, LearningConsolidationWorker, DreamScheduler) with per-component verdicts (live / stale / never-ran) and a routing-diversification dashboard (7-day exploit/explore counts, avg quality per bucket, avg mincut multiplier, avg Q-weight). Both --json and human-readable formats. Operators get a one-command view of whether the loop is closing.

• Daemon pidfile probes globally-installed agentic-qe directly (#488 Phase 1 B.1). The generated .agentic-qe/workers/start-daemon.cjs previously fell back to npx --yes agentic-qe mcp when no local install existed, but child.pid was then the npx wrapper PID (which exits as soon as it forks the real bundle), causing aqe daemon status to misreport liveness. The script now probes require.resolve('agentic-qe/dist/mcp/bundle.js') first to find the global install; if it falls back to npx anyway, a WARNING in daemon.log surfaces the degraded pidfile contract to operators.

• dream_insights retention sweep (#488 Phase 3 C.2). Stale unapplied insights older than 30 days now get pruned on every worker tick. Applied insights stay forever (audit trail); recently- created insights stay regardless of applied state. The table no longer grows unbounded.

Changed

• Bridge cursor is now monotonic (#488 Phase 3 C.3). CapturedExperienceBridge.drain reads back the persisted cursor before writing and uses max(this.cursor, persisted) so cross-process races (e.g., duplicate daemons) can't regress the cursor and cause duplicate event re-publication.

• Hooks no longer trigger dream cycles inline (ADR-094 boundary). The checkAndTriggerDream call is removed from post-edit and post-task; JSON output now emits dreamTriggered: false with dreamReason: 'deferred-to-kernel' so operators reading the JSON see where the cycle actually runs rather than a bare false literal.

• SQLitePatternStore.recordUsage is now a thin wrapper around the new shared recordPatternUsage helper. The throw-on-missing-pattern contract is preserved.

Architecture Decision Records

• ADR-094: Kernel-Side Dream Cycles + Hooks-as-Producers Boundary. Formalizes the architectural rule that hook subprocesses must not do work exceeding ~100ms. The CapturedExperienceBridge pattern proven in v3.9.27 is now applied to dream cycles too. Enforced by CI test.

• ADR-095: ε-Greedy Routing Exploration Policy. Documents the three-signal blend (static score + Q-value blend + mincut-gated ε), the rollback knob, and the post-deploy telemetry surfaces in aqe learning loop-health.

Verification

This release was unusual: a brutal-honesty self-audit of the initial implementation found one production-impacting bug (the mincut safety gate returned "critical" on an empty graph, dampening exploration 5x in the exact CLI-routing deployment case ADR-095 was meant to help). The bug was fixed before this release with 4 regression tests; the audit also caught a misleading dream-scheduler-wiring test (passed by way of failure rather than by the scheduler actually starting) which was rewritten to use a real SQLite backend and verify _dreamScheduler !== undefined after init.

Real-shop smoke verified aqe init --auto against a fresh tmpdir: the daemon script contains the new require.resolve probe, the routing_outcomes schema has all three new columns at CREATE TABLE time, and aqe learning loop-health correctly surfaces the routing-diversification section against hand-seeded data (including non-JSON output, after fixing an early-return bug found during the smoke).

Live telemetry observed in the Claude Code session itself shows exploration: false, criticality: 1, qWeight: 0 — confirming the CRITICAL fix landed correctly (empty-graph CLI hook gets full ε rate, not 0.2× dampening) and the Phase 4 telemetry pass-through works.

638 unit tests pass across 47 impacted suites; type-check clean; build clean. The qe-reasoning-bank.test.ts suite is excluded from test:unit:fast and runs in test:ci where heap headroom exists — it spins up real transformer embeddings per test and OOMs in 8GB codespaces.