mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
context-meetup
1530 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Dragan Spiridonov
|
c81078b7b7 | OpenHuman QE analysis | ||
|
Dragan Spiridonov
|
2c84e65465 |
Merge pull request #504 from proffesor-for-testing/feature/adr-097-external-embedder-endpoint
chore(release): v3.10.0 — optional external embedder endpoint (ADR-097, #503)v3.10.0 |
||
|
Dragan Spiridonov
|
e5585582cb |
chore(release): bump version to v3.10.0
Minor bump — new opt-in feature: external embedder endpoint (#503, ADR-097). |
||
|
Dragan Spiridonov
|
a9f7082f15 |
fix(embeddings): harden ADR-097 endpoint client against devil's-advocate audit
Closes 11 findings from a post-merge audit of the external embedder endpoint implementation. The "no silent hash fallback" correctness promise was being contradicted by two callers; the connect timeout was actually an idle timeout; the identity probe could be bypassed in several paths; and identity drift across runs was undetectable. All hardened. Critical/High fixes: - sqlite-persistence backfill no longer falls back to hash embeddings when the endpoint is configured; re-raises to protect the HNSW index from non-comparable vectors (was poisoning silently on every endpoint blip). - qe-reasoning-bank.embed() propagates the throw under isUsingEndpoint() instead of silently substituting a hash vector. - EmbedderEndpointClient.embed() now gates on probe() — lazy probe on first use with in-flight dedup so 20+ concurrent embeds share one probe. - Circuit breaker recovery (window expires) clears cached identity so the next embed re-probes — endpoint restarts often coincide with model swaps. - True TCP-connect timeout enforced at the socket layer (was req.setTimeout, which is an idle-after-write timeout — SYN blackholes hung ~75s instead of 5s). - Endpoint identity persisted to memory.db kv_store (_system namespace) — cross- run fingerprint drift now surfaces as a loud warning on next boot. Medium fixes: - encoding_format: 'float' pinned in request; base64 responses fail loud. - TLS knobs (ca, cert, key, rejectUnauthorized, servername) passthrough for self-hosted https endpoints without NODE_TLS_REJECT_UNAUTHORIZED=0. - URL userinfo stripped + warned at parse; redacted from all error messages and logs. - Embedding cache key namespaced by mode (endpoint:<fingerprint>:<text> vs inproc:<text>) — cross-mode collisions structurally impossible. Tests: - 33 unit tests pass (was 23): adds probe-gate, breaker re-probe, connect timeout, DNS failure, concurrent embed dedup, base64 fail-loud, URL redaction, encoding_format wire format. - 6 live integration tests against real llama-server (all-MiniLM-L6-v2.Q8_0.gguf): probe, single embed, batch, lazy probe gate, dim mismatch, 50-input batch. Replaces the prior "9ms in-process fake" with honest numbers (cold 15ms, warm 2ms against localhost llama.cpp). - New tests/integration/embedder-endpoint-llamacpp.test.ts is gated on AQE_LLAMA_EMBED_URL so it skips cleanly in CI without the binary. ADR-097 updated with the per-caller audit table (12 writers verified non-poisoning) and "drop-in for 6 providers" claim softened to "verified against llama-server; OpenAI shape compat expected but unverified" per provider. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
b17309b3f8 |
feat(embeddings): optional external embedder endpoint (ADR-097, #503)
Add opt-in OpenAI-compatible /v1/embeddings client to eliminate duplicate in-process model loads in ruflo/ruvector co-deployments and the per-hook cold-load cost in fresh agentic-qe CLI processes. When EmbeddingConfig.endpoint (or AQE_EMBEDDER_ENDPOINT) is set, the @huggingface/transformers import is bypassed entirely and feature-extraction routes through the remote service. Endpoint unset = today's behavior. Wire format is OpenAI's /v1/embeddings so any existing embedder server (TEI, vLLM, llama.cpp, Ollama, LocalAI, LM Studio, OpenAI) works as a drop-in, not just bespoke ruvector servers. Transport is HTTP or HTTP-over-Unix-socket. Errors hard-fail with a circuit breaker (3 failures / 60s window) — no silent hash-embedding fallback, which would poison the HNSW index by mixing non-comparable vectors. A startup probe asserts dim === 384 and computes a stable identity fingerprint that catches model/quantization drift at the boundary. - src/learning/embedder-endpoint-client.ts: OpenAI client with keep-alive agent, 5s/30s timeouts, Bearer auth from env, circuit breaker, probe - src/learning/real-embeddings.ts: endpoint branch placed strictly before the dynamic transformers import so the cold-load fix holds at runtime; new isUsingEndpoint() / getEndpointIdentity() observability helpers - 23 new unit tests (real http.createServer fixtures, no http mocking) - End-to-end smoke: 9ms for import + init + probe + embed, transformers never appears in the loaded-module list Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6b04dd9c2c |
Merge pull request #501 from proffesor-for-testing/fix/issue-499-route-q-loop
chore(release): v3.9.36v3.9.36 |
||
|
Dragan Spiridonov
|
eea53fcdef |
chore(release): bump version to v3.9.36
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
d698c58721 |
fix(routing): collapse Q-state to 3-dim (drop complexityBucket) (#499)
ADR-096 fix #3 (Option A): the previous 4-dim state_key taskType|priority|domain|complexityBucket fragmented semantically identical tasks across cells. complexityBucket was derived from raw description length (round(min(len/200,1)*10)) — so a 55-char and 130-char phrasing of the same task landed in different Q-rows, and QWEIGHT_RAMP_VISITS=20 was rarely reachable per cell even after the fix #1/#2 loop closure. State key is now 3-dim: taskType|priority|domain Same change applied at both ends of the loop: - buildRoutingStateKey (agent-routing.ts) — parameter and key shape - updateHookRouterQValue (hooks-dream-learning.ts) — state_key SQL - qe-reasoning-bank.ts reader-side call - task-hooks.ts post-task writer call - routing-hooks.ts post-route writer call (new in fix #1) `deriveComplexityBucket` stays exported (marked @deprecated for state_key use) — the pre-task bridge payload still carries `complexityBucket` for kv_store schema compatibility and any downstream telemetry. `TaskBridgePayload` interface unchanged. Migration: existing rl_q_values rows written under the previous 4-dim key are orphaned. Field reports show real installs have at most ~2 such rows (the loop was broken; no real Q-data accumulated), so we accept the orphans rather than ship a migration. If a future install carries substantial 4-dim data, a one-shot script that strips the trailing |N segment is straightforward. Test contract extended: - agent-routing-exploration.test.ts pins the 3-segment key shape - route-q-loop-closure.test.ts asserts complexityBucket does NOT leak into updateHookRouterQValue payloads (writer-side check) and that bridge.complexityBucket is ignored even when present. ADR-096 updated: complexity-bucket promoted from Open Questions to Decision (Option A). Open Questions now lists "re-introducing a coarse complexity tier if telemetry shows the 3-dim key conflating decisions that benefit from differentiation" as a wait-and-measure follow-up rather than a planned change. Refs: #499 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
d41bbc22b0 |
fix(routing): close Q-loop on routing surface (#499)
The route hook (UserPromptSubmit) reads `rl_q_values` to produce qWeight, but the only writer was task-hooks `post-task` gated on `outcome.bridge`. On real projects, `routing_outcomes` reached 139 rows while `rl_q_values` stayed at 2 — the route surface that displays qWeight had nothing to read, so `qWeight: 0` was structurally pinned. Two complementary producer additions, ADR-096: Fix #1 — post-route trains rl_q_values When the Stop hook resolves a routing_outcomes sentinel, derive a state_key from the sentinel's `task_json.description` using the same exported helpers the route hook used (`deriveTaskType`, `deriveComplexityBucket`, `detectQEDomains`) and call `updateHookRouterQValue`. Writer and reader address byte-identical rows. Best-effort + try/catch: Stop hooks must never crash the host. Fix #2 — post-task fires even when bridge is absent Drop the `if (outcome.bridge)` gate. When bridge is missing, derive state from `options.description` instead. Adds a new `--description` option to `aqe hooks post-task` so the PostToolUse hook command can pass `tool_input.description` for Task tools. Skip the update only when neither bridge nor description is available (would pollute an "unknown" sentinel state). Tests: tests/unit/cli/commands/route-q-loop-closure.test.ts pins the producer/consumer state_key alignment contract — if a future refactor desynchronizes the three call sites (routing-hooks.ts post-route, task-hooks.ts post-task, qe-reasoning-bank.ts:530), assertions fail the build. Closes the Open Question flagged in ADR-095. The deriveComplexityBucket fragmentation Jordi noted is acknowledged in ADR-096's Open Questions and tracked for a follow-up — orthogonal to closing the loop. Refs: #499 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
2f60d17d78 |
Merge pull request #500 from proffesor-for-testing/working-april
chore(release): v3.9.35v3.9.35 |
||
|
Dragan Spiridonov
|
91d9ab19a0 |
chore(release): bump version to v3.9.35
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
c6b3142f09 |
test(learning): split aqe-learning-engine.test.ts into three forks (issue #448 step 2)
Mirrors the qe-reasoning-bank split: the original 622-line file ran
~43 tests in a single fork, most of which spun up a full
AQELearningEngine (transformer + HNSW + WASM coherence) via
beforeEach. Distributing across three sequential forks (pool=forks,
maxForks=1, fileParallelism=false) caps per-fork heap.
- _aqe-engine-test-helpers.ts (NEW, not a test file)
Shared createMockMemoryBackend() and createMockEventBus() factories
pulled out of the original so the splits don't duplicate. Underscore
prefix keeps it out of vitest's test-file include glob.
- aqe-learning-engine-factory.test.ts (NEW, 4 tests, no init)
Factory Functions + DEFAULT_ENGINE_CONFIG. Tests only call the
constructor — no engine.initialize() — so transformer / HNSW /
WASM aren't loaded. Safe to include in test:unit:fast.
- aqe-learning-engine-patterns.test.ts (NEW, 15 tests, full engine)
The three heaviest sub-describes: Pattern Learning, Task Tracking,
Experience Capture Integration. Each test runs a full engine init.
Excluded from test:unit:fast until step 3's dispose hooks are
proven in CI.
- aqe-learning-engine.test.ts (SLIMMED, ~21 tests, full engine)
Init / Status / Task Routing / Model Routing / Codebase Analysis /
Guidance / Dispose. Header comment updated to reference the two
companion files. Inline mock factories replaced with imports from
the shared helper.
Coverage preserved: ~43 tests still run under test:ci, across three
forks instead of one.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
f0a7b5acaa |
test(learning): release module singletons between tests (issue #448 step 3)
The heavy QEReasoningBank / AQELearningEngine test files re-init a full
bank in every `beforeEach`, but several module-level singletons in the
transitive graph survive across tests and accumulate per run:
- real-embeddings.ts: `pipeline`, `embeddingModel`, `embeddingCache`
(the embedding Map grows for every distinct text routed through
`computeRealEmbedding()` and is never cleared).
- audit/witness-chain.ts: `_instance` retains the WitnessChain with
accumulated entries and its better-sqlite3 handle.
Wire-in:
afterEach(() => {
clearEmbeddingCache(); // existing export, just unused
_resetWitnessChainForTests(); // new test-only reset added here
});
afterAll(() => {
resetInitialization(); // existing export, full pipeline reset
});
`clearEmbeddingCache()` and `resetInitialization()` already existed in
real-embeddings.ts as test hooks but nothing was calling them. The
witness-chain singleton had no reset path — this commit adds a
`_resetWitnessChainForTests()` helper next to the existing factory
(underscore-prefixed to mark it as internal/test API).
The hooks are applied to all three files in the split (qe-reasoning-bank
core, qe-reasoning-bank-feedback-loop, aqe-learning-engine).
Local validation note: this Codespace's heap ceiling is too tight to
reproduce CI's cumulative-growth regime — a single bank init alone
OOMs at 6GB here. CI is the canonical environment for measuring
whether the dispose hooks restore acceptance #1. Re-enabling the heavy
files in test:unit:fast (acceptance #2) is held until CI confirms.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
ea3566f9a2 |
test(learning): split qe-reasoning-bank.test.ts into three forks (issue #448 step 2)
The original 1019-line file ran 58 tests in a single vitest fork, each
spinning up a full QEReasoningBank (transformer embeddings + HNSW + WASM
coherence). Cumulative per-test memory growth blew past the heap ceiling
and the publish workflow timed out on multiple v3.9.x releases.
This commit extracts the file into three forks so the heap pressure is
distributed across three sequential processes (vitest config:
pool=forks, maxForks=1, fileParallelism=false):
- qe-reasoning-bank-utilities.test.ts (NEW, 29 tests, no bank)
Pure-function tests for qe-patterns + qe-guidance. Module-graph weight
is negligible — runs in ~10ms with no transformer load. Safe to
include in test:unit:fast.
- qe-reasoning-bank-feedback-loop.test.ts (NEW, 9 tests, full bank)
recordOutcome / qe_pattern_usage feedback-loop tests with vi.doMock
churn over '../../../src/kernel/unified-memory.js'. Carries the
heaviest mocking surface, so isolated to its own fork.
- qe-reasoning-bank.test.ts (SLIMMED, 20 tests, full bank)
Init / Pattern Storage / Task Routing / Learning Outcomes /
Guidance Generation / Statistics / Embedding. Header comment
updated to reference the two companion files.
Both heavy files (-feedback-loop and the slim core) remain excluded from
test:unit:fast for now. Re-enabling them is acceptance #2 of the issue
and requires step 3 of the proposed solution (lazy / disposable hooks on
real-embeddings.ts and coherence-service.ts) — local repro in a 16GB
Codespace shows even the slim 9-test fork still OOMs at 6GB heap,
indicating per-test cumulative growth that file-splitting alone doesn't
cure. The split is preparatory and reduces the leak slope from
58×Δ to max(20,9,0)×Δ per fork, which should give CI headroom and
clear the way for the dispose-hooks fix in step 3.
Issue: #448
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
f97e33186d |
test(integration): pin package.json exports contract for kernel subpaths
Regression guard for PR #497: a shipped subpath missing from the package.json `exports` map throws ERR_PACKAGE_PATH_NOT_EXPORTED under Node ESM strict resolution, even when the compiled file is present. Three assertions per declared subpath: 1. Entry exists with `import` and `types` keys. 2. Referenced files exist in dist/ (skipped when dist/ not built). 3. The documented symbol is reachable via dynamic import. Negative-control verified: removing the `./kernel/hnsw-adapter` entry causes the first assertion to fail with a "missing from exports map (see PR #497)" message. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
dependabot[bot]
|
e69488eee5 |
chore(deps): bump ws in the npm_and_yarn group across 1 directory (#498)
Bumps the npm_and_yarn group with 1 update in the / directory: [ws](https://github.com/websockets/ws). Updates `ws` from 8.20.0 to 8.20.1 - [Release notes](https://github.com/websockets/ws/releases) - [Commits](https://github.com/websockets/ws/compare/8.20.0...8.20.1) --- updated-dependencies: - dependency-name: ws dependency-version: 8.20.1 dependency-type: indirect dependency-group: npm_and_yarn ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
JLMA
|
4123ef6774 |
fix: add ./kernel/hnsw-adapter to package exports map (#497)
The HnswAdapter class is shipped in dist/kernel/hnsw-adapter.js but the
subpath was not registered in the package.json exports map. In Node.js
ESM strict mode, unlisted subpaths throw ERR_PACKAGE_PATH_NOT_EXPORTED,
making direct imports like:
import { HnswAdapter } from 'agentic-qe/kernel/hnsw-adapter'
fail at runtime even though the compiled file is present.
Fix: add the ./kernel/hnsw-adapter entry alongside the existing
./kernel barrel export.
|
||
|
Dragan Spiridonov
|
2695b163e0 |
Merge pull request #496 from proffesor-for-testing/fix/patterns-rvf-compaction-and-size-guard
chore(release): v3.9.34v3.9.34 |
||
|
Dragan Spiridonov
|
c83c0345b3 |
chore(release): bump version to v3.9.34
Stops unbounded growth of patterns.rvf / brain.rvf. - #495: AbortSignal in HybridMemoryBackend.initialize() so a 5s createHybridBackendWithTimeout abort actually cancels the work instead of leaking a 14-min runaway init (~20 GB / 12 MB/s leak observed in the field). UnifiedMemoryManager.initialize() also accepts {signal} with throwIfAborted() at each await seam. - Post-dream compaction in DreamScheduler — threshold-gated compact() against patterns.rvf and brain.rvf so append-only growth has a steady-state janitor. - Boot-time size guard in getSharedRvfAdapter() for one-shot recovery of installations already in the bad state. - Bounded SQLite→RVF backfill (top-1000 by last_used_at) so `: > patterns.rvf` for disk recovery doesn't trigger an unbounded replay (the "regrew to 59 GB" pathology). - RvfDualWriter.compact() for brain.rvf parity. Three operator env knobs added (no rebuild needed): - AQE_RVF_SIZE_GUARD_BYTES (default 256 MB) - AQE_RVF_DEAD_RATIO_THRESHOLD (default 0.30) - AQE_RVF_BACKFILL_LIMIT (default 1000) 29 new unit tests. Performance gate 15/15 pass. test:ci 4035 pass (3 pre-existing failures, verified identical on main). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
b41adb569c |
Merge pull request #494 from proffesor-for-testing/working-april
chore(release): v3.9.33v3.9.33 |
||
|
Dragan Spiridonov
|
33096de66d | chore(release): bump version to v3.9.33 | ||
|
Dragan Spiridonov
|
a39d4ad010 |
ci(security): add consumer-side audit gate (npm overrides workaround)
Closes the gap surfaced by the @xenova/transformers → protobufjs
investigation: `npm audit` in this repo lies about user-facing CVEs
because the top-level `overrides` block in package.json forces patched
versions of vulnerable transitives — but npm `overrides` are root-only
by design and do NOT propagate to downstream consumers. A clean
`npm install agentic-qe` from a user's perspective can therefore see
high/critical CVEs that our in-repo audit reports as 0.
Two new gates run against an actual consumer install tree:
1. `consumer-audit` job in npm-publish.yml (BLOCKING pre-publish).
Packs the local build with `npm pack`, installs the tarball into a
fresh tmp project, runs `npm audit --audit-level=high` there. Now a
`needs:` of the publish job, so a CVE invisible to repo-level audit
still blocks publish.
2. Consumer-audit step + P1 issue creation in post-publish-canary.yml
(NON-BLOCKING). Same idea, but against the freshly-published tarball
from the registry — catches discrepancies between what we built and
what the CDN serves. Findings open an auto-labelled P1 issue
(security/canary/release) and surface in the run's step summary.
The two together replicate the actual user view at both the
pre-publish gate and the post-publish safety net.
Local simulation of the consumer-audit logic on the current head:
183 packages installed, 0 vulnerabilities — confirming the
@huggingface/transformers migration in
|
||
|
Dragan Spiridonov
|
122f223c9a |
fix(security): migrate from @xenova/transformers to @huggingface/transformers
@xenova/transformers v2.17.2 (last publish 2024-05-29) pins onnxruntime-web@1.14.0, which pulls onnx-proto → protobufjs <=7.5.5. That chain has 8 GHSA advisories including a critical arbitrary-code- execution one (GHSA-xq3m-2v4x-88gg). Our top-level `overrides` block forced protobufjs@^7.5.6 in our own tree, hiding the vulns from `npm audit` in this repo — but `overrides` are root-only by design and do not propagate to downstream consumers. End users running `npm install agentic-qe` were getting 5 vulns (4 high, 1 critical). @huggingface/transformers@4.2.0 is the official maintained successor (same project, renamed). It pulls onnxruntime-web@1.26 which depends on protobufjs@^7.2.4 directly (no onnx-proto), so the version range resolves cleanly to the patched line. API changes handled: - pipeline()/Tensor: drop-in compatible, just the import path changes - `quantized: bool` → `dtype: DataType` (mapped from our QuantizationType enum: 'none'→'fp32', 'fp16'→'fp16', 'int8'→'int8', 'binary'→'q8'; binary stays a post-process step downstream) - progress_callback signature: ProgressInfo is now a discriminated union; `progress` only exists on the "progress" variant — narrowed accordingly Also added onnxruntime-node and onnxruntime-web to the CLI and MCP bundle externals lists. @huggingface/transformers ships both runtimes (node for Node.js, web as fallback); esbuild can't bundle the native .node binary so it must remain external. End-user verification (npm pack + clean install in /tmp/aqe-canary): - 183 packages (was 180 pre-migration; same order of magnitude) - 0 vulnerabilities (was 5: 4 high, 1 critical) - aqe init --auto runs in 93 ms; all 13 phases pass; optional-peers hint renders correctly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
a7c0ad76c2 |
feat(init): hint optional peer deps after successful init
When @claude-flow/browser or @claude-flow/guidance is not installed
locally, append a short "Optional add-ons" section to the end of
`aqe init`'s success output with the install command for each. Skipped
silently when both are already present.
Why this matters: the previous two commits moved these from
optionalDependencies to peerDependenciesMeta{optional:true}, removing
~480 transitive packages and the upstream ERESOLVE noise. The trade-off
is that users no longer get them auto-installed, so the features they
gate (browser MCP URL/PII scanning; the guidance governance integrations)
silently fall back to JS paths. The init hint surfaces the opt-in at
exactly the moment the user is configuring their project, with a
concrete install command they can copy.
Detection uses `require.resolve` and treats ERR_PACKAGE_PATH_NOT_EXPORTED
as "installed" — @claude-flow/guidance ships only an ESM "import"
condition for its bare specifier, so a naive CJS-style resolve from
createRequire would report it missing even when it isn't.
Note: the dead-but-still-compiled src/cli/commands/init.ts contains a
similar displayNextSteps() that is NOT reached by the bundled CLI bin
(dist/cli/bundle.js dynamically loads dist/cli/chunks/init-handler-*.js,
which is built from src/cli/handlers/init-handler.ts). Editing
init-handler.ts is the only place that affects the actual `aqe init`
output. Leaving commands/init.ts alone — it's separately tracked tech
debt and out of scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
bb2e9a4090 |
chore(deps): declare @claude-flow/browser as optional peer
Mirrors the previous @claude-flow/guidance treatment: moves
@claude-flow/browser from optionalDependencies to peerDependencies +
peerDependenciesMeta{optional:true}.
Why this is the high-leverage change in this cleanup pass:
@claude-flow/browser peers on @claude-flow/cli@^3.0.0-alpha.140, which
in turn pulls @claude-flow/embeddings whose peers disagree with cli's
own optional deps — that mismatch is the entire source of the ERESOLVE
warnings users see during `npm install agentic-qe`. With browser moved
to an optional peer, that whole transitive chain is no longer pulled by
default.
Tree shrinkage: 1207 → 725 packages (482 transitives removed,
~7,924 fewer lines in package-lock.json). Includes @claude-flow/cli,
@claude-flow/embeddings, agentic-flow, plus their long tail of
deprecated transitives (koa-router@14, prebuild-install@7, uuid@9,
node-domexception@1) — all of the deprecation warnings called out at
the start of this dep-cleanup pass.
Zero runtime risk: grep confirms zero `import`/`require` of
@claude-flow/browser anywhere in src/, tests/, or scripts/. The package
was only ever a marker for the separately-run @claude-flow/browser MCP
server; src/domains/visual-accessibility/services/browser-security-scanner.ts
talks to it over the MCP protocol, not via a Node import.
Verified: `npm run build` passes, `npm audit --audit-level=high
--omit=dev` clean, `node dist/cli/bundle.js --version` runs. `npm ls
@claude-flow/cli` is now empty.
Users who want the browser MCP server can still install it explicitly:
`npm install -g @claude-flow/browser@3.0.0-alpha.1`.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
016eda1421 |
chore(deps): declare @claude-flow/guidance as optional peer
Moves @claude-flow/guidance from optionalDependencies to:
- peerDependencies + peerDependenciesMeta{optional:true}: tells
consumers of agentic-qe that this is an optional integration they
can opt into, without forcing npm to install it for everyone.
- devDependencies: keeps our own build/CI deterministic without
depending on the transitive chain (@claude-flow/cli pulls guidance
as one of its own optional deps).
End-user impact: zero. The package was already transitively installed
via @claude-flow/browser → @claude-flow/cli, and runtime governance
integrations in src/governance/*-integration.ts already use lazy
`await import('@claude-flow/guidance/...')` with try/catch fallback,
so the declared-dep change does not alter what ends up in node_modules.
Type-surface check: dist/**/*.d.ts contains zero `import('@claude-flow/guidance')`
references — the internal type aliases compile away, so downstream
TypeScript consumers of agentic-qe/governance are unaffected.
Signals intent more cleanly. A bigger reducer of the upstream ERESOLVE
peer noise would be giving @claude-flow/browser the same treatment, but
that is out of scope here.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
d8a04b22d2 |
ci(security): add npm audit gate on high/critical CVEs
Adds a new `security-audit` job to optimized-ci.yml that runs in parallel with the journey shards and fails the build on any high/critical CVE in production deps. devDep CVEs are reported as warnings — they don't ship to users but are visible so we can fix them before they migrate. Also adds the same `npm audit --audit-level=high --omit=dev` step to the publish workflow's build job as a defense-in-depth gate: blocks `npm publish` if a new prod-tree CVE has landed since the last main-branch CI run. Would have caught the 6 high `@typescript-eslint`/`minimatch` advisories that Tier B just cleared the moment they were published, rather than when someone happened to run `npm audit`. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
b62eb83ad4 |
fix(learning): rehydrate Date fields at all kv-read seams (#493 follow-up)
#493 was not the last instance of the kv-Date hazard — it was the most reachable. Audit of the rest of the learning pipeline found 11 more unprotected kv-read sites where TypeScript declares `Date` but JSON round-trip through `memory.get` yields an ISO string at runtime. Two failure modes, both already documented at learning-coordinator.ts:898-904 (#491 Bug 3): 1. THROW — `record.someDate.getTime()` raises TypeError. Surfaces immediately on the first invocation once any record exists. 2. SILENT — `timeRange.contains(record.someDate)` or `new Date() > record.expiresAt` coerces string vs Date to NaN; all comparisons return false; records are silently filtered out or expiry is never recognised. No error, just empty/stale analytics. The silent variant is the worse one — #491 Bug 3 was the same class and shipped for a release cycle undetected. Strategic fix: a single `rehydrateDates<T>(record, dateFields[])` helper at `src/shared/utils/kv-date-rehydrate.ts` applied at every kv-read seam that touches Date-bearing types. Shallow-copies the record and coerces named string fields back to Date; pass-through for already-Date, null, undefined. Sites patched (11 total): src/domains/learning-optimization/services/ production-intel.ts — 4 sites (HIGH #2-#5 throw, MEDIUM #8 silent) getMetricsHistory, getRecentIncidents (×2), resolveIncident, getRecentMilestones metrics-optimizer.ts — 1 site (HIGH #1 throw) getMetricsHistory.sort learning-coordinator.ts — 1 site (HIGH #6 throw) getReplayBuffer.sort transfer-specialist.ts — 4 sites (HIGH #7 throw, MEDIUM #9 silent) queryKnowledge (×2), getKnowledgeById, bulkTransfer src/domains/learning-optimization/ coordinator-helpers.ts — 1 site (SILENT) getExperiencesForDomain coordinator.ts — 1 site (SILENT, duplicate of above) getExperiencesForDomain (private) Sites verified SAFE (kv-read but only access non-Date fields): - coordinator.ts:1045/1134/1175 — uses .domain/.confidence/.version - metrics-optimizer.ts:676 — uses .metrics map - learning-coordinator.ts:379/438/483/547/830/842 — uses .confidence/.successRate/.domain/.usageCount - learning-coordinator.ts:901 — already has #491 Bug 3 inline rehydration with instanceof guard - experience-capture.ts:520/563 — TaskExperience uses number timestamps, not Date Sites NOT patched (deferred per scope): - coordination/protocols/learning-consolidation.ts — defined but never imported anywhere. Dead code; would resurrect the bug if reactivated. Documented in the audit but not fixed. Regression tests (29 new, 6 files): tests/unit/shared/utils/kv-date-rehydrate.test.ts (10) tests/unit/domains/learning-optimization/ production-intel-date-coercion.test.ts (5) metrics-optimizer-date-coercion.test.ts (2) learning-coordinator-date-coercion.test.ts (2) transfer-specialist-date-coercion.test.ts (5) coordinator-helpers-date-coercion.test.ts (2) Each service test seeds its mock memory backend with kv-shape records (ISO strings instead of Date instances — matching the post-JSON.parse shape that better-sqlite3-backed memory yields in production) and exercises the kv-read → consumer path that would fail pre-fix. Both throw and silent failure modes are pinned: throws are asserted not to fire, silent failures are caught by behavioural assertions (e.g. the expired-knowledge test asserts `expired.score < live.score * 0.5 ±`, which would fail pre-fix because the 0.5 penalty never triggered). Verification: - 29/29 new regression tests green - 135/135 existing tests for patched services green (production-intel, metrics-optimizer, transfer-specialist, learning-coordinator, learning-consolidation, learning-consolidation-mining, learning-consolidation-liveness) - `tsc --noEmit` clean - `npm run build` produces dist with rehydrateDates at all 17 expected call sites across the 6 patched files Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
daa9fcf0a0 |
fix(learning): rehydrate lastUsedAt at kv seam to unblock consolidation (#493)
`LearningConsolidationWorker.getDomainPatterns()` mapped `lastSeen: p.lastUsedAt`, but `lastUsedAt` arrives from the `learning:pattern:*` kv as an ISO string (not a Date) after the JSON round-trip. The two downstream consumers — `consolidatePatterns` and `pruneIneffectivePatterns` — both call `.getTime()` on it, throwing `TypeError: lastSeen.getTime is not a function` every cycle on installs with non-empty pattern kv. This was dead code until v3.9.32: #491 Bug 1+2 starved the pattern store so `collectPatterns()` always threw `No learning patterns to consolidate yet` before reaching `consolidatePatterns`. Fixing the #491 chain made this line reachable for the first time and the same `Date`-lost-across- kv-JSON-round-trip class of bug as #491 Bug 3 surfaced. Other call sites were hardened in v3.9.32 (`pattern-store.ts:1301-1304`, `pattern-lifecycle.ts:592-594`, `sqlite-persistence.ts:891`); these two were missed because they were unreachable at the time. Fix: rehydrate at the kv-read boundary — `lastSeen: new Date(p.lastUsedAt)`. `new Date(x)` is safe for `Date | string | null | undefined` (yields Invalid Date — whose `.getTime()` returns `NaN` rather than throwing — on null-ish input). One-line change covers both downstream consumers. Test: `learning-consolidation-date-coercion.test.ts` exercises the full post-mining half of the loop (the gap Jordi flagged in his release-gate suggestion). Seeds a `learning:pattern:*` entry with ISO-string `lastUsedAt`, runs the worker through `getDomainPatterns` → `consolidatePatterns` → `pruneIneffectivePatterns` → full `doExecute`, and asserts: 1. The seam rehydrates `lastSeen` to a real Date. 2. `consolidatePatterns` completes without throwing. 3. The recent-high-performer filter math actually counts the pattern (not just "doesn't throw" — catches silent-suppression regressions). 4. `pruneIneffectivePatterns` completes and produces the stale finding. 5. Full `doExecute` reaches `Learning consolidation complete` and `recordLoopHealth(success:true)` — the user-visible signals. Verified pre-fix with 5/5 failing tests showing the exact error from the issue; post-fix with 5/5 green, full learning-consolidation suite (36 tests) green, and `tsc --noEmit` clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
713652cd81 |
chore(deps): tier C — runtime-safe breaking bumps
- web-tree-sitter ~0.24.7 → ~0.26.8 (API moved Parser.init onto the
Parser class; adapter shim added in tree-sitter-wasm-parser.ts so we
also still work against the legacy 0.24 surface).
- @types/node ^20.19.17 → ^25.8.0 (engine "node >=18.0.0" is unchanged;
type widening only).
Deferred from Tier C (each needs its own migration PR, not a bump):
- @ruvector/attention 0.1.3 → 2.2.2: full API rewrite, classes
(FlashAttention, DotProductAttention, MultiHeadAttention,
HyperbolicAttention, LinearAttention, MoEAttention) replaced by
free functions (flashAttention, scaledDotProductAttention, …).
Requires rewriting integrations/ruvector/attention-wrapper.ts and
updating all 6 platform-binary optional deps in lockstep.
- vibium 0.1.8 → 26.x: package no longer ships dist/ — repurposed as a
CLI binary. `import { browser } from 'vibium'` no longer resolves.
Requires a different integration strategy.
- typescript 5.9 → 6.0 and eslint 8 → 10: released 1-4 weeks ago; need
their own validation pass (flat config migration for eslint).
- @ruvector/sona 0.1.5 → 0.1.6: published manifest references files
missing from the tarball (broken publish upstream).
Build passes (tsc + cli + mcp bundles). npm audit: 0 vulnerabilities.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
d90093544c |
chore(deps): tier B — tooling majors, fixes 6 high CVEs
- @typescript-eslint/eslint-plugin ^6.13.0 → ^8.59.3 - @typescript-eslint/parser ^6.13.0 → ^8.59.3 - commander ^12.1.0 → ^14.0.3 - esbuild ^0.27.2 → ^0.28.0 Clears all 6 high-severity advisories (minimatch chain via @typescript-eslint/typescript-estree). npm audit: 0 vulnerabilities. Deferred to Tier C: @ruvector/sona@0.1.6 (broken npm publish, manifest points to non-existent files) and web-tree-sitter@0.26 (breaking API move of Parser.init onto the Parser class). Build passes (tsc + cli + mcp bundles). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
9ba99d04e8 |
chore(deps): tier A — refresh within-caret patches (no package.json change)
Updates lockfile to latest patches for: - @vitest/coverage-v8, vitest 4.1.4 → 4.1.6 - axe-core 4.11.3 → 4.11.4 - better-sqlite3 12.9.0 → 12.10.0 - jose 6.2.2 → 6.2.3 - msw 2.13.4 → 2.14.6 - ora 9.3.0 → 9.4.0 - tsx 4.21.0 → 4.22.1 - yaml 2.8.3 → 2.9.0 Build passes (tsc + cli + mcp bundles). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
dfd02b32ce |
Merge pull request #492 from proffesor-for-testing/working-april
chore(release): v3.9.32v3.9.32 |
||
|
Dragan Spiridonov
|
c67942026b | chore(release): bump version to v3.9.32 | ||
|
Dragan Spiridonov
|
1d1190ca7e |
test(release-gate): add A23 daemon-liveness assertion (#491)
Issue #491 exposed a structural gap in the pre-publish init gate: it runs `aqe init` against 4 pinned real repos and asserts file existence, KG entry counts, double-init etc — 22 separate assertions — but it NEVER starts the daemon. Every one of the four bugs in #491 lives in the daemon-runtime seam, not the init seam: the kernel→worker wiring opens during `fleet_init`, which only happens at daemon startup. So a fully-broken learning loop sailed through the gate as "OK". This commit adds A23 to run_one(): - opt-in per fixture via the new `expectDaemonLiveness` manifest flag (default false, so existing fixtures keep their ~5s budget) - configurable wait via `daemonLivenessWaitSec` (default 20s) - spawns the cleanroom's start-daemon.cjs, waits, greps stderr and daemon.log for the failure shapes from the #491 chain: "domain not available", "Domain not available", "not available for" - asserts zero matches; lists offending lines + summary marker on fail - cleans up the daemon process (SIGTERM, 2s grace, SIGKILL) Pilot fixture: enabled on `tiny-ts` with a 15s wait window. tiny-ts is the smallest fixture (~17 files, ~6s init), so adding 15s of daemon observation puts the whole fixture at ~22s — well within the 60s timeout and the 20-min job budget. If the pilot is stable for two releases, we extend the flag to mid-ts and p-queue. The larger multi-lang-real fixture stays opt-out until a separate weekly workflow takes long-running daemon scenarios out of the release path. Manifest schemaVersion bumped 3→4 with a changelog entry documenting the new fields. Run-gate header comments updated to document A23 alongside A1-A22. The same script is invoked by post-publish-canary.yml against the published npm version, so A23 doubles as a post-publish smoke for any registry/CDN divergence that affects daemon-runtime — a class of regression neither the build-job bundle smoke nor `test:unit:fast` could detect. This is the structural complement to the npm-publish.yml change in the preceding commit (integration-tests-on-tag-sha runs the SOURCE seam tests; A23 runs the INSTALLED PACKAGE seam check). Together they close the daemon-runtime gap that let #491 ship. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
2f39f2210b |
ci(release): gate publish on daemon-runtime integration tests (#491)
Issue #491 exposed a structural gap in the release process: integration coverage of the daemon-runtime seam existed (bridge-end-to-end test was already in the repo), but it was never run on the release path — `tests-on-tag-sha` only ran `test:unit:fast`, which excludes `tests/integration/` by design. Every one of the four bugs in #491 lived in that unrun surface. Two changes: 1. `package.json` — define `test:integration:fast` and `test:integration` scripts. The "fast" variant runs the focused daemon-runtime seam suite in ~6s: - tests/integration/mcp/fleet-init-wires-daemon.test.ts (Bug 1+4b) - tests/integration/workers/workers-reach-domains.test.ts (Bug 1 generalized) - tests/integration/learning/coordinator-roundtrip.test.ts (Bug 2+3) - tests/integration/bridge/bridge-end-to-end.test.ts (existing) The broader `test:integration` runs all of `tests/integration/` excluding browser/e2e for local development use. 2. `.github/workflows/npm-publish.yml` — add a new `integration-tests-on-tag-sha` job that runs `test:integration:fast`, wired into the publish job's `needs` and `if` conditions in the same shape as the existing `tests-on-tag-sha`. Picked the "fast" variant deliberately: it covers exactly the regressions #491 reported and keeps the release gate budget tight (~10-min job ceiling, but the work itself is ~6s). The broader integration suite is too noisy for a release gate; the fast subset is the floor. This is the structural complement to issue #401's pre-publish init gate: that one catches `aqe init` regressions before users, this one catches daemon-runtime regressions before users. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
dd22dbddda |
test(workers): generalize #491 Bug 1 protection to all 11 workers
The issue #491 fix wired the daemon WorkerManager to the kernel and shipped one targeted integration test for the learning worker. The same StubWorkerDomainAccess failure mode applies to every other worker that declares targetDomains — test-health, flaky-detector, coverage-tracker, defect-predictor, etc. — and the worker-level unit tests all explicitly punt on this with comments along the lines of "complex dependencies require integration tests for full coverage. Unit tests focus on instantiation." That punt is what let Bug 1 ship in the first place. This test iterates EVERY registered production worker (enumerated dynamically from `daemon.getWorkerManager().list()`), builds the exact WorkerContext the scheduler hands to doExecute, and asserts that each of the worker's declared targetDomains resolves to a real domain API. If the kernel→worker wiring ever breaks again — for any of the 11 workers, not just learning-consolidation — this test fails before the daemon ticks for users. Three assertions: 1. Every registered worker declares ≥1 targetDomain (catches the "shipped a worker with no domains" misconfig). 2. For every (worker, targetDomain) pair, `getDomainAPI(domain)` returns a defined value (the structural Bug 1 contract). Failure list is formatted as a multi-line message so the maintainer sees exactly which seam broke. 3. The worker manager's `context.memory` exposes `vectorSearch` (only HybridMemoryBackend has it, InMemoryWorkerMemory does not) — a cheap structural check for the Bug 4b memory swap. Verified red→green: with the setKernel/setMemory wiring removed from handleFleetInit, assertions 2 and 3 fail, listing every (worker, domain) pair that lost its API. With the wiring restored, 3/3 pass in ~170ms. The test deliberately does NOT call `daemon.runWorker(id)` to exercise each worker's full body. Most workers raise legitimate "no data yet" errors on fresh installs (test-health, flaky-detector etc.) and asserting completion would couple this test to each worker's empty-state behaviour. The contract is structural — the seam to the kernel is alive — not behavioural. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
358990cc01 |
fix(kernel,learning): namespace-aware reads + timestamp rehydration (#491 Bug 2+3)
Issue #491 — even with the worker→kernel wiring restored (Bug 1/4b), `mineExperiences` mined 0 for every domain on installs with hundreds of indexed experiences. Two stacked bugs in the learning read path: Bug 2 — namespace mismatch (the keystone) ========================================= LearningCoordinator writes every experience with `namespace: 'learning-optimization'`, but the MemoryBackend interface only let writers specify a namespace; reads had no such option. HybridBackend.get/search/has/delete therefore hard-coded `defaultNamespace='default'`. The coordinator could not read back its own data — `learning:experience:index:domain:<d>:*` returned 0 keys even when the DB held 81+ matching rows in the `learning-optimization` namespace. Fix: - Add `RetrieveOptions { namespace? }` to kernel/interfaces.ts (symmetric with the existing `StoreOptions.namespace`). - Extend MemoryBackend.get/has/delete/search to accept the option (backward-compatible — undefined still routes to defaultNamespace). - HybridBackend honors the option in all four methods. - InMemoryBackend likewise; its `search` now also scopes the scan by namespace prefix when one is supplied, so unit-test fakes behave the same as the production hybrid backend. - All 15 read sites in LearningCoordinator pass `LEARNING_NS = { namespace: 'learning-optimization' }`, matching the writes that already specified it. One pre-existing unit test (memory-backend.test.ts) was calling `backend.get(key, 'test-ns')` against the undocumented positional namespace param InMemoryBackend used to accept. Migrated it to the options form — the positional shape was never part of the MemoryBackend interface. Bug 3 — Date vs ISO-string in time-window filter ================================================ `recordExperience` stores `timestamp: new Date()`. JSON.stringify serializes it to an ISO string and JSON.parse does not rehydrate. `TimeRange.contains` does `date >= start`; in `string >= Date`, the string coerces to NaN and every comparison is false — so even with Bug 2 fixed, the window filter would still silently drop every row. Fix: in `getExperiencesByDomainAndTime`, coerce `experience.timestamp` to a real `Date` at the kv-read boundary before calling `timeRange.contains`. Downstream callers receive a hydrated Experience with a proper Date. We deliberately fix this at the boundary rather than making `TimeRange.contains` tolerant of strings — the bug is "kv round-trip drops the Date type", not "the value object is wrong", and a tolerant comparator would mask future type drift everywhere else. Round-trip integration test: tests/integration/learning/coordinator-roundtrip.test.ts pins three assertions: 1. recordExperience → mineExperiences finds the experience back (this returned 0 before either fix). 2. The mined experience's timestamp is a real Date instance, not a string (proves Bug 3 fix is at the boundary). 3. A time window deliberately in the past still excludes recent writes (proves we didn't fix Bug 3 by making the filter a no-op). Verified red→green: with Bugs 2+3 reverted (interface kept), tests 1 and 2 fail with `experienceCount=0` and `rows.length=0` — exactly the symptoms from the issue. With both fixes in place, 3/3 pass in ~30ms. 466/466 kernel unit tests and 335/335 adjacent suites stay green; the only test that needed migration is documented above. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
7f13f129dd |
fix(workers): always emit loop-health from learning worker (#491 Bug 4a)
Issue #491 — `aqe learning loop-health` permanently showed `LearningConsolidationWorker ○ never-ran` even when the worker executed every cycle. Root cause: in `doExecute`, the `recordLoopHealth(success:true)` ping sat *after* `collectPatterns()`, which throws on installs with nothing to consolidate (the common case for fresh users). The error propagated up before liveness was ever recorded, so the dashboard had no idea the worker was alive. Fix: wrap the worker body in try/catch/finally — the same shape CapturedExperienceBridge.drainSafe already uses (which is why the bridge correctly shows `live` and the worker did not). The `finally` emits `recordLoopHealth` with `success:true` on a clean tick and `success:false + error` on a throw, then the catch rethrows so BaseWorker's retry/error tracking sees the failure unchanged. The liveness write is itself wrapped in try/catch so a memory-backend hiccup can't shadow the worker's real error. Liveness contract test: tests/unit/workers/workers/learning-consolidation-liveness.test.ts pins three behaviors: 1. Throw inside collectPatterns → recordLoopHealth still called with `{success:false, error}` (THIS is what regressed before). 2. Successful tick → recordLoopHealth called with `{success:true}`. 3. recordLoopHealth itself throwing must not shadow the original worker error. Verified red→green: with the try/finally reverted, assertion (1) fails with "expected number of calls: 1, received: 0" — exactly the "never-ran" symptom from the issue. With the fix in place, 3/3 pass in ~19ms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
eca8d86b37 |
fix(mcp,workers): wire daemon to kernel after fleet_init (#491 Bug 1+4b)
Issue #491 (Jordi Izquierdo) — on a vanilla v3.9.31 install, every domain-dependent worker tick fails with "<domain> not available" and `aqe learning loop-health` shows `learningWorker: never-ran`. Root cause: the MCP-hosted daemon constructs its WorkerManager in `src/mcp/entry.ts` *before* the kernel exists, so it starts with StubWorkerDomainAccess (getDomainAPI returns undefined for every domain) and a private InMemoryWorkerMemory that no dashboard ever reads from. The kernel is built inside `handleFleetInit` later in the same process, but the daemon is never told about it. Fix: - Promote `setKernel`/`setMemory` from concrete-class-only methods to the public WorkerManager interface so consumers can find them. - Add `setMemory(memory: WorkerMemory)` to WorkerManagerImpl (mirrors the existing `setKernel`). Broadens the internal `memory` field from InMemoryWorkerMemory to the WorkerMemory interface so a shared HybridMemoryBackend is type-correct. - Hoist `KernelReference` to interfaces.ts as `WorkerKernelReference` so the interface signature is properly typed without re-introducing the kernel↔worker circular dependency the local declaration was avoiding. - In `handleFleetInit`, after `pluginLoader.loadAll()`, call `getDaemon().getWorkerManager().setKernel(state.kernel)` and `setMemory(state.kernel.memory)`. Wrapped in try/catch so a workless smoke install (no daemon module bundled) still gets a working kernel + queen. Why an interface change, not the cast Bolivar used: the public WorkerManager interface didn't expose setKernel/setMemory at all, so the call site looked like it was reaching for a private method — exactly the kind of seam that previous releases dropped on the assumption that the bridge superseded it. Making it part of the public contract means the next maintainer can't accidentally remove the wiring without breaking a typed interface. Integration test: tests/integration/mcp/fleet-init-wires-daemon.test.ts pins three assertions: 1. Before fleet_init, the daemon's WorkerManager returns undefined for `getDomainAPI('learning-optimization')` (the Bug 1 failure mode). 2. After fleet_init, it returns a real plugin API. 3. The WorkerManager's memory is the same instance as the kernel's memory (Bug 4b), so worker writes reach the dashboard's reads. Verified red→green: with the wiring reverted, all three fail with "expected undefined to be defined"; with the wiring in place, all three pass in ~210ms. Note: Bugs 2, 3, and 4a are addressed in follow-up commits in this series. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
cfe4a154fe |
Merge pull request #490 from proffesor-for-testing/working-april
chore(release): v3.9.31v3.9.31 |
||
|
Dragan Spiridonov
|
33eb0020d6 |
chore(release): bump version to v3.9.31
Self-learning loop ships end-to-end. Closes #480, #486, #487, #488. Introduces ADR-094 (kernel-side dream cycles + hooks-as-producers boundary) and ADR-095 (three-signal routing with Q-blend + ε-greedy + mincut safety gate). 8 commits in this release: - |
||
|
Dragan Spiridonov
|
43ebcaa326 |
fix(routing,kernel,cli): audit follow-ups — CRITICAL #1 + HIGH #1/#2 + smoke
Addresses three findings from the brutal-honesty audit of the Phase 1-4 session (commits 66553141..70154c1b). Real integration smoke included. CRITICAL #1 — mincut safety gate inverted on empty graph ======================================================== Before this fix, `QEReasoningBank.routeTask` consulted `getSharedMinCutMonitor().isCritical()` unconditionally. The mincut calculator returns 0 on an empty graph; `isCritical()` is defined as `minCutValue < warningThreshold` — so an empty graph reported as critical and ε was dampened to 0.2x. CLI hook routing is the ONLY place ADR-095 exploration matters in practice, and the SwarmGraph is empty there (Queen coordinator activity is what populates it). So exploration was permanently dampened in the exact deployment case ADR-095 was meant to help. Fix: - new src/learning/routing-topology-gate.ts with `resolveTopologyCriticalFromSharedMincut()` — returns true ONLY when graph has actual vertices AND `isCritical()` returns true. Empty graph = no signal = full ε rate. - qe-reasoning-bank.ts routeTask now uses the helper. - 4 regression tests in tests/unit/learning/routing-mincut-safety-gate.test.ts cover: never-initialized, initialized-but-empty (regression case), populated-with-critical (legit dampening), populated-with-healthy. HIGH #2 — dream-scheduler-wiring test passed by way of failure ============================================================== The previous test used `memoryBackend: 'memory'` which made the DreamEngine.initialize() fail ("UnifiedMemoryManager not initialized") in every run. The kernel's try/catch caught the failure and the test asserted the kernel survived — NOT that the scheduler actually started. The happy-path was never exercised. Rewrite uses a real tmpdir SQLite backend AND resets BOTH singletons between tests (UnifiedMemoryManager + UnifiedPersistenceManager — the latter was missing). Adds an assertion that `_dreamScheduler !== undefined` after initialize() so we know the scheduler ACTUALLY STARTED. The previous "failure tolerance" test was removed: with proper singleton hygiene there's no clean way to force scheduler init to fail without invasive mocking. The behavior is enforced by the try/catch in kernel.ts and visible by inspection. Also added an `await scheduler.dispose()` in the kernel's catch path so a partially-constructed scheduler doesn't leak. HIGH #1 — Real integration smoke against a tmpdir ================================================== Ran (and documented): 1. `aqe init --auto` in /tmp/aqe-smoke-* — fresh shop 2. Inspected generated .agentic-qe/workers/start-daemon.cjs: ✓ contains `candidates.push(require.resolve('agentic-qe/dist/mcp/bundle.js'))` ✓ contains `WARNING: using npx fallback` message (B.1 verified in actual artifact, not just source template) 3. Confirmed routing_outcomes schema in the fresh DB: ✓ exploration INTEGER, criticality REAL, q_weight REAL all present (ADR-095 migration applies at CREATE TABLE time for new installs) 4. Hand-seeded 12 exploit + 3 explore rows; ran `aqe learning loop-health`: ✓ JSON path surfaces routingDiversification with correct counts, avg quality per bucket, criticality, q_weight ✓ Text path now surfaces the section even when loop-health kv is absent (bug fix below) CLI-bundle smoke for `aqe hooks route` and full daemon start OOMs in this 8GB codespace even at 4GB heap — that's an environment limitation documented under qe-reasoning-bank.test.ts; CI has the headroom. Bug found during smoke: aqe learning loop-health (non-JSON) returned early when no loop-health kv record existed, never reaching the routing-diversification render. Operators in CLI-only deployments (routing happens but daemon-side workers never tick) would see an empty dashboard. Fixed by removing the early return. Honest framing on qe-reasoning-bank.test.ts "flaky" claim ========================================================== It isn't flaky — it's resource-heavy. The project already excludes it from `test:unit:fast` (package.json) and runs it in `test:ci` where heap headroom exists. 58 tests × full transformer-embeddings init + HNSW indexing exceeds an 8GB codespace's default heap. I framed it as "flaky" in the audit. That was wrong. Added an extended header comment to the test file documenting the project convention so future readers don't repeat the same mistake. Test status =========== 638/638 tests pass across the 47 impacted unit suites. The +4 net additions (vs 634) are the new routing-mincut-safety-gate tests; dream-scheduler-wiring stayed at 5 tests (one removed, four expanded). Build clean. Type-check clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
70154c1b72 |
feat(routing): three-signal routing — static + Q-value blend + mincut-gated ε-greedy (ADR-095, #488 Phase 4)
Closes the self-learning loop: the Q-value Bellman updates that #487 populates now have a routing-side reader; structurally-aware exploration (via the mincut infrastructure from ADR-047 / ADR-068) gathers Q-table signal beyond the deterministic greedy winner; the catalog diversifies without sacrificing safety on bottleneck decisions. Three signals blended at the agent-routing decision point: 1. Static score (existing): domain match + capability match + performance prior + language boost + pattern similarity. Computed by `calculateAgentScores` in src/learning/agent-routing.ts. 2. Q-value blend (NEW — reads from rl_q_values): effectiveScore = staticScore * (1 - qWeight) + sigmoid(qValue) * qWeight qWeight ramps from 0 (no Q-data) to MAX_Q_WEIGHT=0.4 over QWEIGHT_RAMP_VISITS=20 visits per (state_key, action_key). Cold-start behavior is identical to pre-ADR-095 static-only routing; the learned signal contributes meaningfully only after the Q-table has matured for that state. 3. Mincut-gated ε-greedy exploration (NEW): baseEpsilon = AQE_ROUTER_EXPLORATION_RATE env or 0.05 default safetyMultiplier = getSharedMinCutMonitor().isCritical() ? 0.2 : 1.0 epsilon = baseEpsilon * safetyMultiplier With probability ε, swap the greedy winner with a uniformly-random pick from the top-3 alternatives. Uses crypto.randomInt (NOT Math.random) to match the codebase convention (randomUUID is the established crypto entry-point in the same modules). Maintainer decisions captured in ADR-095 v2: 1. Default ON (Option A): mincut-modulated exploration ships enabled. AQE_ROUTER_EXPLORATION_RATE=0 is the documented escape hatch. 2. Telemetry surface: extended `aqe learning loop-health` with a new "Routing diversification" section showing 7-day exploit/explore counts, avg quality_score per bucket, avg criticality, avg q_weight. 3. crypto.randomInt over Math.random — codebase convention preserved. 4. Q-value ramp constants are best-guess (MAX_Q_WEIGHT=0.4, QWEIGHT_RAMP_VISITS=20); tune post-deploy from telemetry. Source-level findings worth noting: - `rl_q_values` was populated by `updateHookRouterQValue` since #487 but NEVER read during routing. Verified via grep across src/learning/, src/routing/, src/cli/commands/hooks-handlers/. This commit adds the reader; closes that loop. - Mincut graph is populated by Queen coordinator swarm activity, NOT per-task. So the "criticality" signal is graph-level safety (the `isCritical()` boolean from MinCutHealthMonitor) rather than per-task structural weight. Honest framing reflected in the ADR. What landed: src/kernel/unified-memory-schemas.ts - routing_outcomes gains exploration (INTEGER NOT NULL DEFAULT 0), criticality (REAL), q_weight (REAL) columns + an exploration index for the loop-health bucket queries src/routing/routing-outcomes-migration.ts (NEW) - ensureRoutingOutcomesAdr095Columns(db) — idempotent ALTER TABLE for pre-existing databases. Process-local flag avoids repeat work. src/routing/routing-feedback.ts - hooks the migration helper into the existing schema-migration block src/learning/agent-routing.ts - new helpers: blendStaticAndQValue, resolveExplorationRate, applyExplorationPolicy (uses crypto.randomInt), sigmoid normalization - new helpers: deriveTaskType, buildRoutingStateKey, deriveComplexityBucket — moved from task-hooks.ts so the routing path and the post-task Q-update path share the same state_key derivation - calculateAgentScores accepts optional qValueLookup; backward-compat when undefined (identical pre-ADR-095 behavior) - ScoredAgent gains optional telemetry fields: staticScore, qWeight, qValue, qVisits, exploration src/learning/qe-reasoning-bank-types.ts - QERoutingResult gains optional exploration / criticality / qWeight src/learning/qe-reasoning-bank.ts - routeTask builds the Q-value lookup from rl_q_values keyed by stateKey = deriveTaskType(task) | normal | domain | complexityBucket - applies exploration policy after sort; reports avgQWeight + the mincut safetyMultiplier in the QERoutingResult - new private buildQValueLookup helper with table-exists guard so the method is a graceful no-op when the schema isn't applied yet (tests, fresh-init scenarios) src/cli/commands/hooks-handlers/task-hooks.ts - private deriveTaskType removed (DRY: now imported from agent-routing) - pre-task sentinel INSERT writes the three new columns - migration helper invoked before write src/cli/commands/hooks-handlers/routing-hooks.ts - JSON output passes exploration / criticality / qWeight through - INSERT writes the three new columns; migration helper invoked src/cli/commands/learning.ts - `aqe learning loop-health` queries routing_outcomes for 7-day rolling exploration stats and avg quality_score per bucket. Skipped silently when the columns don't exist (pre-ADR-095 databases). Tests: - tests/unit/learning/agent-routing-exploration.test.ts: 35 tests cover sigmoid blend math, qWeight ramp, env override priority, mincut safety multiplier, exploration rate statistical tolerance, only-top-3 alternative constraint, deriveTaskType regex coverage, stateKey format, complexityBucket scaling, end-to-end Q-blend - tests/unit/routing/routing-outcomes-migration.test.ts: 5 tests cover fresh-DB migration, idempotent re-run, missing-table tolerance, exploration index creation, column type/default checks 634/634 tests pass across CLI commands, workers, bridge, learning, init/phases, architecture, kernel, and routing suites; build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
874ae39c6a |
feat(learning,bridge): dream_insights retention + bridge cursor guard (#488 Phase 3)
Two small defensive changes that close the remaining cleanup-and-race gaps from #488. No new ADRs needed — both are additive within already-documented subsystem boundaries. C.2 — dream_insights retention pruner ===================================== The dream_insights schema has no expires_at column and no foreign-key cascade fires on regular cycle delete (cycles aren't routinely purged either). Without pruning the table grows linearly with hook activity — V0.03 evidence in #488 documents 120+ rows in 30h, most with applied=0. Over months `getPendingInsights` queries get slower and vector embeddings (if attached) grow unbounded. src/learning/dream/dream-insights-pruner.ts (NEW) - pruneStaleDreamInsights(db, { retentionDays = 30 }): DELETE FROM dream_insights WHERE applied = 0 AND created_at < datetime('now', '-N days') - Applied insights stay forever (audit trail). - Recently-created unapplied insights stay regardless of applied state — they might still be picked up by a future applyInsight. - Guards against the table not existing (fresh init before the dream schema is applied). Idempotent. src/workers/workers/learning-consolidation.ts - new Step 8 in runContinuousLearningLoop after the mineExperiences sweep: prune insights, emit a finding if pruned > 0, surface `dreamInsightsPruned` in ConsolidationResult and domainMetrics. - Failures isolated per-try-block (worker continues with the rest of its lifecycle work). C.3 — bridge cursor monotonic guard =================================== CapturedExperienceBridge.drain wrote `this.cursor` to the kv unconditionally. Cross-process race: two daemons (possible before #488 B.1 fixed the pidfile, still possible if user manually spawns extras) drain overlapping rows, then the "slower" process overwrites the kv with its older value. Next drain re-publishes already-drained rows; downstream plugins without event-ID dedup see duplicates. src/bridge/captured-experience-bridge.ts - Before persisting the cursor, read back what's stored and write Math.max(this.cursor, persisted). The cursor never regresses. - Read-then-max-write is a TOCTOU but it collapses the regression window to milliseconds — sufficient for the duplicate-daemon edge case. Stronger guarantees (SQL UPDATE WHERE, advisory lock) would bypass the MemoryBackend abstraction without a real-world benefit post-B.1. - Cursor persistence is now best-effort (catch and warn). On failure the next bridge boot re-reads from kv and resumes — a missed write just means we re-publish a small batch, idempotent for ID-dedup'd consumers. Tests ===== tests/unit/learning/dream/dream-insights-pruner.test.ts (NEW) - 8 tests: deletes stale unapplied, keeps applied regardless of age, keeps recent unapplied, honors custom retention, no-op on empty table, no-op when table doesn't exist, default 30-day retention, idempotent re-run tests/unit/bridge/captured-experience-bridge.test.ts (extended) - 3 new tests in describe('cursor monotonic guard'): - does not regress when another writer is ahead - advances normally when this bridge is ahead - sensible fallback when no persisted value exists 594/594 tests pass across CLI, workers, bridge, learning, init/phases, architecture, and kernel suites; build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
6236a2cdc7 |
feat(kernel): kernel-side dream cycles + hooks-as-producers boundary (ADR-094, #488 Phase 2)
Moves dream cycles out of hook subprocesses into the long-lived kernel
process via the DreamScheduler that already existed but was never wired.
Single cutover, no feature flag — per maintainer decision on ADR-094.
Why this matters
================
Before this commit, every ~20 Edit/Write tool uses triggered a dream cycle
inside a short-lived `npx aqe hooks post-edit` subprocess. The cycle held
a SQLite write transaction for up to 10 seconds while the kernel-owning
MCP server, other hook subprocesses, and user MCP tool calls queued
behind it. stderr from the cycle was consumed by Claude Code's hook
reader and lost from operator visibility. Cold DreamEngine init paid
every ~20 edits.
Now the kernel's `QEKernelImpl.initialize()` boots a `DreamScheduler`
that runs cycles on its own cadence inside the long-lived process. Hook
subprocesses keep only `incrementDreamExperience` (a counter bump that
returns in ~50ms). Operators see `aqe learning loop-health` report the
`dreamScheduler` component as live within one tick.
What landed
===========
src/kernel/kernel.ts
- new private `_dreamScheduler?: DreamScheduler`
- new config flag `enableDreamScheduler` (default `true`, mirrors
`enableExperienceBridge`)
- initialize() constructs DreamEngine + DreamScheduler after the
bridge starts, wraps in try/catch (best-effort), failure logs a
warning and continues (kernel boot never blocks on this)
- dispose() stops the scheduler BEFORE bus/memory cleanup so a dream
in-progress doesn't write to a disposed backend
src/learning/dream/dream-scheduler.ts
- executeDream() now records loop-health on every tick (success or
failure) via the helper from #488 Phase 1 B.2. Operators see
`learning:loop-health.components.dreamScheduler.successesSinceBoot`
grow on each cycle.
src/cli/commands/hooks-handlers/editing-hooks.ts
src/cli/commands/hooks-handlers/task-hooks.ts
- dropped `checkAndTriggerDream` import + call
- kept `incrementDreamExperience` (the counter is the producer-side
signal the kernel scheduler reads)
- JSON output now emits `dreamTriggered: false, dreamReason:
'deferred-to-kernel'` so operator scripts grepping the field see
*where* the cycle actually runs
src/cli/commands/hooks-handlers/hooks-shared.ts
- removed the `checkAndTriggerDream` re-export. The function itself
is still defined in hooks-dream-learning.ts and exported there for
direct test access and the documented rollback path.
tests/unit/architecture/hooks-boundary.test.ts (NEW)
- greps `src/cli/commands/hooks-handlers/` for forbidden patterns:
imports from `src/learning/dream/` and references to
`checkAndTriggerDream`. Skips comment lines so explanatory mentions
don't false-positive. `hooks-dream-learning.ts` is in `allowedFiles`
since it DEFINES the function.
- This is the enforcement mechanism for the boundary contract. A
contributor accidentally re-adding heavy hook-side work gets a
clear CI failure pointing them at ADR-094.
tests/unit/kernel/dream-scheduler-wiring.test.ts (NEW)
- asserts `enableDreamScheduler` default, opt-out path, init+dispose
cleanliness with and without the scheduler, idempotent dispose
tests/unit/cli/commands/post-edit-dream-trigger.test.ts (REWRITTEN)
- was asserting checkAndTriggerDream was called from post-edit (the
#480 fix). That contract is now wrong — the kernel is the only
place that triggers dreams.
- now asserts: counter is bumped (the kernel reads it), no inline
cycle trigger, JSON emits `deferred-to-kernel`, hook stays robust
when the memory backend throws.
docs/implementation/adrs/ADR-094-kernel-side-dream-cycles.md
- status: Implemented
- migration section replaced with the single-cutover rationale and
a documented 4-line rollback path
583/583 tests pass across CLI, workers, bridge, learning, init/phases,
architecture, and kernel suites. Build clean.
Boundary contract (now enforced in CI)
======================================
Hook subprocesses MUST NOT do work that exceeds ~100ms. Acceptable
hook-side work: producer SQLite writes, counter increments, in-memory
routing lookups. Unacceptable: DreamEngine init, multi-second SQLite
transactions, anything that depends on long-lived state.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
4c0bdfafb9 |
feat(observability): daemon pidfile fix + learning loop-health (#488 Phase 1)
Phase 1 of the #488 production-readiness work, scoped to additive changes that don't need new ADRs. Two parts: B.1 — Daemon pidfile points at the npx wrapper, not the MCP server ================================================================= When neither `node_modules/.bin/aqe-mcp` nor the local `node_modules/agentic-qe/dist/mcp/bundle.js` exists, the generated `.agentic-qe/workers/start-daemon.cjs` falls back to `npx --yes agentic-qe mcp`. `npx` exits as soon as it has forked the real bundle, so `child.pid` is the wrapper PID and the pidfile becomes stale within seconds. `aqe daemon status` then misreports liveness, and the idempotency check at the top of the script spawns duplicate daemons — the reporter hit this twice in two days. Adds `require.resolve('agentic-qe/dist/mcp/bundle.js')` as a 3rd candidate (try/catch-guarded) before the npx fallback. Globally-installed shops (`npm install -g agentic-qe` — the typical case the npx fallback was silently covering) now spawn the real bundle directly; `child.pid` is the long-lived MCP server. The npx fallback remains as a last resort for environments where neither local nor global install is reachable. When taken, it now logs a WARNING to daemon.log so operators can see the degraded pidfile contract. B.2 — `learning:loop-health` observability ========================================== Operators previously had no way to answer "is the self-learning loop closing end-to-end?" without DB-mining `dream_cycles`, `kv_store`, and `routing_outcomes` separately. Cumulative counters in `aqe learning stats` grow forever and can't distinguish "healthy loop" from "loop wedged 3 hours ago with stale totals". Adds a single `learning:loop-health` kv key with per-component liveness: { overallLastSuccess, bootedAt, components: { bridge?: { lastSuccessAt, ticksSinceBoot, ... }, learningWorker?: { lastSuccessAt, ticksSinceBoot, ... }, dreamScheduler?: { lastSuccessAt, ticksSinceBoot, ... }, } } Writers (best-effort — observability MUST NOT crash the observed component): - CapturedExperienceBridge.drainSafe — every poll, success or failure - LearningConsolidationWorker.doExecute — end of every tick - DreamScheduler reserved for #488 Phase 2 (the field stays undefined until Phase 2 wires the scheduler in-kernel) Surface: - new `aqe learning loop-health [--json]` CLI subcommand with component verdicts (live/stale/never-ran) based on per-component staleness thresholds (bridge: 30s; worker/scheduler: 2h) - distinct from the existing `aqe learning health` (per-domain regret analysis); the new command is pipeline liveness, not agent outcome quality Tests: - 12 tests in tests/unit/learning/loop-health.test.ts for the utility: fresh record creation, tick counters, error capture + clear-on-success, multi-component isolation, overallLastSuccess max-computation, throw-safety when the memory backend is broken, staleness verdicts - 4 tests in tests/unit/init/phases/10-workers-daemon-pidfile.test.ts asserting the require.resolve probe is present, try/catch-guarded, ordered before the npx fallback, and that the warning string is in place 541/541 tests pass across CLI commands, workers, bridge, learning, and init/phases suites; build clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Dragan Spiridonov
|
e266bae929 |
refactor(learning): single writer for pattern usage — qe_pattern_usage + qe_patterns columns (#486 Gap B)
Before this commit two parallel writers bumped pattern usage stats:
- SQLitePatternStore.recordUsage — did BOTH qe_pattern_usage INSERT
and qe_patterns UPDATE inside one transaction (correct).
- hooks-dream-learning.ts inline UPDATE — bumped qe_patterns columns
but SKIPPED the qe_pattern_usage INSERT, so the audit table stayed
EMPTY in production even as qe_patterns.usage_count climbed during
normal hook activity. V0.03 evidence: 0 rows in qe_pattern_usage
despite 33 hits on the dominant cli-hook-code-intelligence pattern.
Extracts the shared accounting into a new helper
src/learning/pattern-usage-recorder.ts with a single transaction:
- INSERT INTO qe_pattern_usage (audit trail per use)
- UPDATE qe_patterns.{usage_count, successful_uses, success_rate,
quality_score, last_used_at, updated_at}
Both writes are atomic — either both land or neither does.
Quality formula matches the canonical pattern-lifecycle.ts shape:
quality_score = confidence*0.3 + min(usage_count/100, 1)*0.2 + success_rate*0.5
Returns updated=false (no throw) when patternId is unknown, because hook
subprocesses iterate selectedPatternIds that may contain stale UUIDs
from a routing kv that drifted from qe_patterns. Callers wanting strict
semantics (SQLitePatternStore.recordUsage) check the flag and throw.
Refactors:
- SQLitePatternStore.recordUsage → thin wrapper around the helper.
Preserves the throw-on-missing contract.
- hooks-dream-learning.ts per-pattern inline UPDATE → replaced with
recordPatternUsage() call. Now passes meaningful metrics
(tokens_saved, source, experience_id) and feedback to the audit row.
Keeps the #455 inline promotion check (separate concern, fires
AFTER the helper has updated the qe_patterns columns).
Invariant established: COUNT(*) FROM qe_pattern_usage WHERE pattern_id=X
equals qe_patterns.usage_count for X. Before this commit that invariant
was systematically violated by the hook path.
Test infrastructure:
- 7 new tests in tests/unit/learning/pattern-usage-recorder.test.ts
cover atomicity, the missing-pattern no-throw contract, the formula
boundary at usage_count=100, null metrics/feedback handling, and
the audit/column count invariant.
- tests/unit/cli/commands/post-task-promotion.test.ts schema seeds
now include qe_pattern_usage (the test fixture was missing the
table that the production schema creates at boot).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
be32ff091e |
feat(workers): auto-trigger mineExperiences per-domain in learning-consolidation (#486 Gap A)
The producer side of the learning-optimization domain pipeline had no
auto-trigger for `mineExperiences`. After v3.9.29's CapturedExperienceBridge
landed, `learning:experience:*` kv populates correctly — but the chain
stopped there because `mineExperiences` was only callable via four manual
MCP tools or an explicit `task_submit`. Nothing scheduled it.
Verified consumer survey before choosing the fix shape:
- `learning:pattern:*` kv writers/readers are all scoped to the
learning-optimization domain + coordination layer
- `qe_patterns` table is the qualified multi-domain catalog with
distinct schema (tier, quality_score, embedding, deprecated_at,
token tracking, successful_uses) — NOT a duplicate of the kv
- The two stores represent sequentially different lifecycle phases:
kv = candidate working memory, table = qualified catalog
- Deprecating the kv would have wide cross-domain blast radius;
auto-triggering `mineExperiences` keeps the boundary intact
Implementation:
- new private `mineExperiencesPerDomain` step in
`runContinuousLearningLoop` (called as Step 7 after lifecycle stats)
- iterates ALL_DOMAINS, reads per-domain cursor from
`learning:consolidation-cursor:{domain}` kv, calls the canonical
`LearningCoordinatorService.mineExperiences(domain, timeRange)`
via `LearningOptimizationExtendedAPI.getLearningService()`
- advances cursor ONLY on `experienceCount > 0` — failures and
empty windows leave the cursor untouched so the next 30-min tick
retries the same window with new experiences
- clamps future/corrupted cursors to the default 1-day lookback
- per-domain try/catch isolates failures (one bad domain doesn't
block the others)
- graceful no-op when the learning-optimization API is unavailable
Why direct service call instead of `runLearningCycle`:
`runLearningCycle` also spawns a learning agent, runs strategy
optimization, and creates knowledge. Any of those can fail and block
mining. The direct service call is the minimum sufficient producer.
Why cursor at all:
`learnPattern` uses `uuidv4()` for pattern IDs and `Date.now()` for
names, so re-mining the same experiences would create duplicate
`learning:pattern:*` rows. The cursor is essential for idempotency
across worker ticks.
7 new tests cover: per-domain invocation, cursor advance on success,
no-advance on failure, throw isolation, cursor reuse as next window
start, future-cursor clamping, and graceful no-op when the API is
unavailable.
463/463 unit tests pass across workers + CLI suites; build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Dragan Spiridonov
|
6655314120 |
fix(hooks): wire post-edit dream trigger (#480) + drop pre-task bridge pattern-count gate (#487)
post-edit was initializing `dreamTriggered = false` and only calling
`incrementDreamExperience` — `checkAndTriggerDream` was never invoked.
Since post-edit fires far more often than post-task during Claude Code
sessions, the hook-driven dream loop was starved: `experienceCount`
grew monotonically but `lastDreamTime` stayed null and
`dream_insights.applied` stayed at 0.
Mirrors task-hooks.ts: call `checkAndTriggerDream` after bumping the
counter and surface the full {triggered, reason, insightsGenerated,
insightsApplied} shape in JSON output so operators can distinguish
"not triggered because of state" from "not triggered because we
never tried."
The pre-task bridge was gated on `selectedPatternIds.length > 0`,
which silently dropped the `task-bridge:*` kv write for low-confidence
prompts. post-task's `if (outcome.bridge)` then skipped
`updateHookRouterQValue`, so `rl_q_values` never populated. The
Bellman update's state_key uses (taskType|priority|domain|
complexityBucket) and action_key uses `routing.recommendedAgent` —
neither requires non-empty patterns. The downstream per-pattern
fan-out at hooks-dream-learning.ts:428 already guards
`bridge.selectedPatternIds.length > 0` before iterating, so writing
the bridge with `[]` is structurally safe.
Tests:
- post-edit-dream-trigger.test.ts: 3 cases covering invocation,
full JSON shape on trigger, and dreamReason surfaced when
conditions are not met
- pre-task-bridge-empty-patterns.test.ts: parametrized over
patterns=[] and patterns=[p1,p2], asserts bridge row exists in
both; separate no-op test for missing --description
125/125 hook-handler unit tests pass; build clean.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|