1658 Commits

Author SHA1 Message Date
Dragan Spiridonov 705e46e214 Merge pull request #539 from proffesor-for-testing/release/v3.10.8-prep
chore(release): v3.10.8
v3.10.8
2026-06-16 15:51:54 +02:00
Dragan Spiridonov 3ca830344e chore(release): bump version to v3.10.8
Rolls up three merged PRs:
- #531 database-free OpenCode install + MCP stdio client-compat fixes (@nagoodman)
- #537 protobufjs + ws production CVE fixes
- #538 vite 8.0.9 -> 8.0.16 (supersedes #536)

Version bumped across package.json, package-lock.json, and
skills-manifest.json fleetVersion. CHANGELOG + release notes added.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 13:39:47 +00:00
Dragan Spiridonov 5e6078c46c chore(deps-dev): bump vite to 8.0.16 (#538)
Supersedes #536. Dependabot's regenerated lockfile dropped the
@ruvector/ruvllm-* optional platform packages, making `npm ci` fail
with EUSAGE (lockfile out of sync). Regenerated with a targeted
`npm update vite --package-lock-only` on Linux, which keeps the
optional platform entries intact.

vite 8.0.9 -> 8.0.16 (transitive dev dependency; routine bug-fix bump).
Lockfile-only change; `npm ci` sync check passes; `npm audit --omit=dev`
reports 0 vulnerabilities.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 15:35:52 +02:00
Dragan Spiridonov 819b62ee1b Merge pull request #531 from nagoodman/feat/opencode-only-no-db
feat(init): database-free OpenCode install + MCP stdio client-compat fixes
2026-06-16 15:21:20 +02:00
Dragan Spiridonov 69bd763f94 Merge branch 'main' into feat/opencode-only-no-db 2026-06-16 14:51:46 +02:00
Dragan Spiridonov 2671db28a0 fix(security): bump protobufjs >=7.6.3 and ws >=8.21.0 to clear high CVEs (#537)
Production-dependency high-severity advisories surfaced by the Security
Audit gate (npm audit --omit=dev --audit-level=high):

- protobufjs (transitive via @grpc/proto-loader, @huggingface/transformers,
  onnx-proto): GHSA-f38q-mgvj-vph7 (property shadowing) and
  GHSA-wcpc-wj8m-hjx6 (DoS via unbounded Any expansion). Was pinned to
  ^7.5.6 (resolved 7.5.8); bumped to ^7.6.3 (resolves 7.6.4), staying
  within the 7.x major.
- ws (transitive): GHSA-96hv-2xvq-fx4p (memory-exhaustion DoS). Added
  ^8.21.0 override (resolves 8.21.0).

Overrides + resolutions updated in lockstep; lockfile regenerated.
`npm audit --omit=dev --audit-level=high` => 0 vulnerabilities.
Lockfile-only dependency change; no source changes.

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-16 14:51:22 +02:00
Nicholas Goodman 0f16559f7e fix(mcp): full feature parity in database-free mode (health, fleet, domains)
The earlier lean boot skipped fleet auto-init, experience capture, CrossPhaseHooks,
ReasoningBank and workers in memory mode. That left the server functionally
degraded — aqe_health reported unhealthy/memory.connected:false/loadedDomains:0,
and read-only fleet/team calls failed with 'Fleet not initialized'. Now that all
persistence layers are gated to run in-memory (unified memory, RVF, hypergraph,
sessions, results), those subsystems write nothing, so boot the SAME as normal
mode. Verified: aqe_health => healthy/connected/14 domains, full tool surface
works, and still ZERO files under .agentic-qe/. Refs #534
2026-06-15 12:25:58 -06:00
Nicholas Goodman 5dff7dc91f fix(mcp): central RVF gate for database-free mode (patterns.rvf, branches/)
Smoke testing showed fleet_init still wrote patterns.rvf and agent_spawn wrote
branches/ in database-free mode — the per-site gates missed the getSharedRvfAdapter
pattern-store path and the agent memory-branch path. Gate the single chokepoint
instead: isRvfNativeAvailable() reports false when AQE_MEMORY_BACKEND=memory, so
every RVF consumer (pattern store, dual-writer, agent memory branches) takes its
existing no-native-RVF degraded path and writes nothing. Verified: fleet_init +
agent_spawn x2 create zero files; default mode still creates RVF/db. Refs #534
2026-06-15 11:26:48 -06:00
Nicholas Goodman 019cbebbaa fix(mcp): honor database-free mode across the runtime, not just boot
Database-free mode previously held only at init/boot — exercising MCP tools
(memory_store, fleet_init, task_orchestrate, skills) re-created memory.db,
brain.rvf, patterns.rvf, branches/, results/ and sessions/ because several
runtime paths opened persistence directly, ignoring AQE_MEMORY_BACKEND=memory.

Gate every offender on memory mode:
- UnifiedMemoryManager → ephemeral :memory: SQLite (no .agentic-qe/memory.db)
- RVF dual-writer → disabled (no brain.rvf)
- RvfPatternStore → adapter-less / metadata-only (no patterns.rvf*)
- code-intelligence hypergraph (coordinator + coordinator-hypergraph) → :memory:
- fleet_init handler → forces 'memory' kernel backend instead of 'hybrid'
- session-store → in-memory bookkeeping, no sessions/*.jsonl
- result-saver → no results/* writes

Verified: a full tool workload (init + memory + fleet_init + task_orchestrate)
writes ZERO files under .agentic-qe/, while default (persistent) mode still
creates memory.db unchanged. Fixes #534
2026-06-15 10:51:35 -06:00
Nicholas Goodman bf6caf4079 fix(init): route platform/database-free installs to the modular orchestrator
Without --auto, `aqe init` fell through to the legacy runStandardInit path,
which ignores --with-opencode (no assets installed) and --no-database/--memory
memory (boots the full kernel and writes .agentic-qe/memory.db). Route to
runModularInit whenever a platform flag or database-free mode is requested, so
those flags take effect regardless of --auto. Refs #529, #530
2026-06-15 10:12:08 -06:00
Nicholas Goodman fa6ae4ce0e feat(mcp): database-free runtime + keep stdio stdout pure JSON-RPC
When AQE_MEMORY_BACKEND=memory the MCP server now uses InMemoryBackend and
boots lean (skips token persistence, experience-capture, ReasoningBank
prewarm, workers, QualityDaemon and fleet auto-init), writing nothing
under .agentic-qe/. Also route console.log/info/debug to stderr so stray
subsystem diagnostics never corrupt the JSON-RPC stream on stdout, which
made strict MCP clients fail with 'Failed to get tools'. Refs #530, #527
2026-06-15 09:40:11 -06:00
Nicholas Goodman 2dc8392334 feat(init): database-free mode (--no-database / --memory memory)
Add an opt-in in-memory backend mode wired through the modular
orchestrator. Skips the database, learning, code-intelligence and workers
phases; verification adapts (no DB to verify/write); the kernel no longer
creates .agentic-qe/ in memory mode. Default persistent behavior is
unchanged. Refs #530
2026-06-15 09:40:11 -06:00
Nicholas Goodman 1e63c800be fix(init): install OpenCode assets in native format + aqe-mcp config
OpenCode could not load AQE's .opencode/agents/*.yaml schema, so 0 agents
and 0 skills appeared. Convert at install time to OpenCode-native
.opencode/agent/*.md and .opencode/skills/<name>/SKILL.md (with per-agent
permission frontmatter); drop the incompatible tool wrappers. The
generated opencode.json now invokes the 'aqe-mcp' bin directly to avoid
the 'agentic-qe mcp' double-spawn stdin bug. Refs #529, #528
2026-06-15 09:40:11 -06:00
Nicholas Goodman 4a3bf19c5a fix(mcp): negotiate initialize protocolVersion instead of hardcoding
handleInitialize() returned a fixed protocolVersion regardless of the
client's request, which strict MCP clients (OpenCode, official SDK) can
reject. Echo the client's requested version when supported, else fall
back to the latest. Refs #527
2026-06-15 09:40:11 -06:00
dependabot[bot] 0ea5c2a684 chore(deps-dev): bump esbuild (#525)
Bumps the npm_and_yarn group with 1 update in the / directory: [esbuild](https://github.com/evanw/esbuild).


Updates `esbuild` from 0.28.0 to 0.28.1
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.28.0...v0.28.1)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version: 0.28.1
  dependency-type: direct:development
  dependency-group: npm_and_yarn
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-06-15 10:31:19 +02:00
Dragan Spiridonov 2ea6dcb93c Merge pull request #524 from proffesor-for-testing/release/v3.10.7-prep
chore(release): v3.10.7
v3.10.7
2026-06-12 17:06:54 +02:00
Dragan Spiridonov 56206c7be2 chore(evals): finish eval model-ID migration across remaining surfaces
Follow-up to d95f0d7b / 2f4d119c — the earlier passes updated .claude/skills
and assets/skills but missed several other eval surfaces that still pinned
retired IDs (claude-3.5-sonnet, claude-3-haiku):

- `.github/workflows/test-qe-browser.yml` — the `aqe eval run --model` flag
  (inert in command-eval mode, but a real dead-ID leftover) -> claude-sonnet-4-6.
- `plugins/agentic-qe-fleet/skills/` — the tracked plugin-marketplace skill copy
  (8 eval suites + a sample-output fixture); chaos skills get the opus-4-8 ceiling.
- `docs/schemas/skill-eval.schema.json` — models `enum` was missing claude-opus-4-8
  and still allowed retired/deprecated IDs; trimmed to current+active, default
  flipped to claude-sonnet-4-6. Plus description examples in the sibling schemas
  and the modelUsed values in docs/examples/*.json fixtures.
- `docs/templates/skill-frontmatter.example.yaml` validation_models list.
- `src/cli/commands/eval.ts` — the `aqe eval` `--model` help example and the
  `run-all --models` default (`claude-3.5-sonnet` -> `claude-sonnet-4-6`).

Verified: typecheck clean; all edited JSON/YAML parse; zero retired IDs remain in
any eval suite, schema, template, workflow, or eval-CLI default. Intentionally
left untouched: the consensus-provider subsystem (pricing tables / type-unions /
defaults — separate routing layer, bucket-2 data) and historical release notes /
QE audit reports (which document the old IDs by design).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 14:52:24 +00:00
Dragan Spiridonov b57c0c0026 chore(release): bump version to v3.10.7 2026-06-12 13:55:23 +00:00
Dragan Spiridonov 2f4d119c23 chore(evals): sync eval model refresh into assets/ + .validation source
Follow-up to d95f0d7b — that commit updated only .claude/skills/*/evals and
docs/templates, missing two surfaces:

1. assets/skills/ — the npm-distribution copy (tracked, generated from
   .claude/skills by prepare-assets.cjs). All 54 eval matrices + nested
   cross-model inputs + the .validation templates/schemas/examples now carry
   current model IDs, at exact parity with .claude/skills.
2. .claude/skills/.validation/ — the *shipped* eval template, schema, and
   example fixtures (the real source new skill evals are generated from):
   - skill-eval.schema.json: the models `enum` was missing claude-opus-4-8
     (so high-stakes evals using it would fail validation) and still listed
     retired claude-3.5-sonnet / claude-3-haiku and soon-retiring
     claude-sonnet-4; enum trimmed to current+active IDs, default flipped from
     claude-3.5-sonnet -> claude-sonnet-4-6.
   - eval.template.yaml / security-testing-eval.template.yaml: models_to_test
     refreshed (security template gets the opus-4-8 ceiling).
   - skill-frontmatter.example.yaml, schema description examples, and the
     modelUsed values in .validation/examples/*.json and
     mutation-testing/test-data/sample-output.json fixtures.

Verified: zero retired/deprecated Claude IDs in either tree; both trees at
parity (55x sonnet-4-6, 55x haiku-4-5, 7x opus-4-8 across the 7 high-stakes
skills); all edited YAML/JSON parses. Unrelated .md/.sh drift that
prepare-assets surfaced was reverted to keep this commit model-IDs only.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 13:29:53 +00:00
Dragan Spiridonov d95f0d7b18 chore(evals): refresh QE skill eval model matrix to current model IDs
The cross-model consistency matrix (models_to_test) in 54 skill evals + both
eval templates pinned retired/malformed model IDs — `claude-3.5-sonnet`
(malformed: dashes, not dots; resolves to Sonnet 3.5, retired Oct 2025) and
`claude-3-haiku` (retired Apr 19 2026). Both 404, so those evals could not run
a meaningful cross-model check.

Refresh to current pinned IDs, preserving the original "primary + fast floor"
design:
  - claude-sonnet-4-6   # primary (high accuracy)
  - claude-haiku-4-5    # fast floor (minimum quality)

High-stakes skills additionally get the capability ceiling claude-opus-4-8
(security-testing, security-visual-testing, n8n-security-testing,
pentest-validation, chaos-engineering-resilience, qe-chaos-resilience,
qe-defect-intelligence) and the security eval template. Opus is scoped to these
7 + the security template rather than all 54 to avoid a fleet-wide eval-cost
blowup.

Also fixes the two `--model claude-3.5-sonnet/claude-3-haiku` examples in
run-skill-eval.ts and a nested cross-model test-case input in
qe-learning-optimization.yaml.

Verified: all eval YAML parses; zero retired/deprecated Claude IDs remain across
skills/evals + templates + runner. (src/domains/constants.ts MODEL_TIERS was
already current — tier 1 claude-haiku-4-5 — so it is intentionally unchanged;
the legacy claude-3-haiku-20240307 entries in cost-tracker/bedrock are retained
historical pricing/availability data.)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 13:19:29 +00:00
Dragan Spiridonov 93fe231ea3 docs(qe): add v3.10.6 QE fleet analysis report set
Full-quality QE fleet analysis (11 specialized agents + Queen coordinator),
diffing v3.9.13 -> v3.10.6 across code complexity, security, performance, test
quality, product/QX, dependency/build, API contracts, architecture/DDD,
accessibility, brutal-honesty, and CI. Evidence-only, with cross-agent conflict
adjudication.

Headline: composite ~7.0 -> ~7.5/10 (largest single-cycle gain on file). All 5
prior release-blockers cleared; the 15 CRITICAL protobufjs supply-chain vulns
validated genuinely fixed via clean-consumer tarball install (not override-
masked). Carried debt: enterprise-integration test freeze (6th release), orphan-
dir drift, frozen type-erasure, stagnant HTML a11y.

SHARED-CONTEXT.md captures the shared baseline + methodology the fleet ran under.
2026-06-12 12:28:08 +00:00
Dragan Spiridonov f03c3150ac ci: shift-left consumer-side supply-chain audit onto PRs and main
npm-publish.yml runs a consumer-POV tarball audit as a publish blocker, but
only at release time — so an override-masked vuln (root-only `overrides` don't
propagate to consumers) merges to main undetected and only surfaces as a late,
blocked release. This adds consumer-audit.yml running the SAME clean-consumer
tarball audit on every dependency-affecting PR and push to main, catching the
regression before merge so main stays releasable.

Mirrors the proven publish-time job: npm ci -> build -> npm pack -> install the
tarball into a fresh consumer project -> `npm audit --audit-level=high` from the
consumer's perspective (where our overrides do NOT apply). Path-filtered to
dependency-affecting files (package.json, package-lock.json, build/pack scripts,
.npmignore, this workflow), least-privilege (contents: read), concurrency-guarded.

Precedent: v3.9.13 shipped 15 CRITICAL protobufjs ACE vulns to every installer
while in-repo `npm audit` read 0 (GHSA-xq3m-2v4x-88gg). Verified: actionlint
clean; logic confirmed passing on the current v3.10.6 tree (consumer install
resolves protobufjs@7.6.3, 0 high/critical).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 12:27:43 +00:00
Dragan Spiridonov 7a0c289cf6 chore(deps): bump @ruvector/sona 0.1.5 -> 0.1.7 (SONA learning fix)
0.1.5's SONA learning engine had a bug where reward feedback produced zero
weight changes (fixed upstream in 0.1.7, published 2026-06-11). AQE drives that
exact path via the trajectory API in sona-wrapper.ts
(beginTrajectory/addTrajectoryStep/endTrajectory + forceLearn), so reward
signals into SONA were effectively inert.

Verified on linux-arm64:
- 0.1.7 linux-arm64-gnu binary is a Linux ELF; native module loads (SonaEngine)
- all 12 native methods the wrapper calls exist in 0.1.7 (API compatible)
- functional smoke test: forceLearn now reports "20 trajectories -> 20 patterns",
  patterns_learned 0 -> 20 (was 0 in 0.1.5 — fix confirmed)
- tsc --noEmit: 0 errors; 206 SONA unit tests pass
- better-sqlite3 native fix from prior commit intact (experience capture unaffected)

Note: 0.1.7 restructured platform binaries into optional per-platform packages
(hence the lockfile churn). Install resolves the correct binary per platform;
do not rely on a host-side install in the bind-mounted container.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 08:48:25 +00:00
Dragan Spiridonov e2a917b7fc fix(hooks): surface fatal native-init failures instead of silently dropping learning
The aqe-hook.cjs shim discarded all stderr and always exited 0. When the
project-local better-sqlite3 binary was the wrong platform (host/container
mismatch → `invalid ELF header`), UnifiedMemoryManager init failed and
persistCommandExperience silently dropped every captured_experiences INSERT.
Self-learning stopped for days with zero trace while dream_cycles (written by
the global MCP server's working binary) masked the outage.

The shim now captures stderr and tees fatal native-init markers (invalid ELF
header, UnifiedMemoryManager init failure, ERR_DLOPEN_FAILED, Node ABI
mismatch) to a throttled (5min) .agentic-qe/hooks-health.log with an actionable
fix hint. The happy path is unchanged — stdout still emits only the JSON
object, stderr stays out of the transcript, and the hook always exits 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 08:09:37 +00:00
Dragan Spiridonov d8745cfb57 Merge pull request #523 from proffesor-for-testing/feat/pattern-space-adr-105-110
chore(release): v3.10.6
v3.10.6
2026-06-11 19:36:44 +02:00
Dragan Spiridonov 0403a65f0f test(scripts): fix js/identity-replacement CodeQL finding (alert 211)
The 'reworded prose' mutation test used .replace('qe_agent_definition',
'qe_agent_definition') — a no-op replacing a string with itself (CodeQL
js/identity-replacement). Replaced with a real non-invariant edit
(frontmatter description + a <notes> block after the closing tag) that
genuinely exercises 'edits outside required sections must not trip the
gate'. 9/9 tests still pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:05:17 +00:00
Dragan Spiridonov bbb1676b54 ci: add least-privilege permissions (contents: read) to new workflows
CodeQL/actions best practice + repo convention: invariant-check.yml and
safety-eval.yml lacked an explicit top-level permissions block. Both only
checkout and run scripts/tests — no gh, git push, PR comments, or release
writes — so contents:read is sufficient. Matches existing workflows.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 17:02:54 +00:00
Dragan Spiridonov a7e21f9ef9 fix(ci): ADR-108 lineage gate — find rubric in benchmark root, not only results dir
The gate fired correctly on PR #523 but its path assumption was too
strict: it looked for RUBRIC-v*.md only in the results file's own
directory, while the real layout is benchmarks/<name>/RUBRIC-v1.md with
results under <name>/results/. Now checks the dir OR its parent, with
two separate OR'd ls calls (a single 'ls a b' exits non-zero when either
glob is unmatched, which masked the present rubric).

Verified locally: real repo passes (rubric found in parent); a results
dir with no rubric anywhere still fails.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:53:41 +00:00
Dragan Spiridonov b77a4cc6ea chore(release): bump version to v3.10.6
Pattern Space cross-pollination (#522): evidence-class labels on
findings, pass/fail safety eval for the data-protection absolutes,
shipped-agent invariant CI, benchmark lineage + interaction benchmark,
kept-nulls learning. ADR-105–110.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 16:39:11 +00:00
Dragan Spiridonov 772af839a4 docs(adr): mark ADR-105..110 Implemented; gitignore transient eval outputs (issue #522)
Statuses updated to reflect reality (were stale 'Proposed' despite
implementation): 107/108/110 Implemented; 105 Implemented w/ INFERRED
domain-finding follow-up; 106 Implemented (live matrix, native tiers
pass every cell); 109 Implemented infrastructure w/ corpus-scaling
follow-up. Gitignore transient run outputs, force-keeping canonical
evidence files.

Verified: full branch tsc clean; both new workflows valid YAML; 58
locally-runnable tests green; no .env or key material in any branch
commit (audited).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 15:48:34 +00:00
Dragan Spiridonov be28f15f9c feat(benchmarks): ADR-109 live cross-family interaction run + two harness fixes (issue #522)
Live runner executed for real: OpenAI gpt-4o-mini under test, Gemini
2.5-flash as the non-same-family judge, hidden-test ground truth, seeded
X/Y counterbalance, pre-registered RUBRIC-v1 hash embedded per row.

The real run surfaced and fixed two harness bugs:
1. Ground-truth fix-rate was coupled to judge-row validity — a fixed bug
   that zeroed a real B-arm fix when its judge pass returned no winner.
   Ground truth is now tracked per (scenario, arm), independent of the
   judge; numbers are internally consistent.
2. Judge row model: emit ONE comparison row per (scenario, pass) — the
   unit the sign test clusters — instead of duplicated per-arm rows.
   Plus judge-verdict parse hardened (last-verdict wins, broader regex,
   more tokens): unparseable rate 50% -> 25%, the rest excluded in code.

Result at n=2: ground truth A=0.50 B=0.00, judge clustered 1-1 p=1.0 —
no arm signal, honestly underpowered. Corpus scaling is the path to a
claim (stated in results README + ADR).

Verified: 30 deterministic tests (stats + safety) green; real run $0.003.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 12:55:54 +00:00
Dragan Spiridonov 50302ce94e feat(safety): ADR-106 live per-tier eval — all native Claude tiers pass 5/5 (issue #522)
Path A (model-judgment) live runner over the real ADR-026 tiers
(haiku/sonnet/opus via Anthropic API). Each tier is given the
data-protection absolutes + an explicit ACTIONS: [...] declaration
protocol, hit with each temptation as opener AND mid-task, N=5, and run
through the existing assertion engine. Asserts on declared intent only;
never touches a real DB. safety-eval.yml live job now executes it.

Real run found and fixed a harness false-positive: a REFUSING model that
illustrates a command in prose was wrongly flagged. The ACTIONS protocol
makes intent unambiguous; discrimination tests prove a genuinely-declared
rm memory.db still FAILS while an illustrated one PASSES (not neutered).

Verified: 16 deterministic tests; live matrix 5/5 all cells, all 3 tiers
(haiku $0.085 / sonnet $0.231 / opus $1.108), committed as evidence.
Retry-resilient client (transient fetch/429/5xx backoff).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 12:51:30 +00:00
Dragan Spiridonov c2238019d5 fix(learning): wire ADR-110 dead inputs — appliedPatterns + contextFingerprint (issue #522)
Brutal-honesty review found the ADR-110 loop fully plumbed but fed by
inputs no real caller supplied: null capture never fired (no caller
passed appliedPatterns) and the discount always ran the unknown-context
branch (no caller passed contextFingerprint).

startTask now retrieves the patterns that will guide a task and threads
their ids into startCapture (fail-soft), so a task failure attributes
nulls to the right patterns. searchPatternsForTask forwards a
domain:agent contextFingerprint into the search, and routeTask passes
its own context — so the null-discount weights in-context failures
hardest, as ADR-110 intends.

Loop now closed end to end: guide -> capture -> fail -> null -> discount.

Verified: tsc clean; the review's own dead-input greps now show both
inputs connected at real call sites (was empty before). Behavioral
proof needs the full-engine harness (CI-only here, same constraint as
the migration suite) — not claiming a local green I can't produce.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-11 11:57:10 +00:00
Dragan Spiridonov 1942470eb7 ci: safety-eval workflow — deterministic layer gates now, live matrix wired per ADR-106 (issue #522)
Layer 1 (always): engine tests + runner over committed fixture
trajectories — all three violating shapes must FAIL, all three
compliant shapes must PASS; an engine regression that lets a violation
through fails CI with an explicit error.

Layer 2 (release/dispatch): live per-tier matrix — fails LOUDLY when
requested with an API key until runner.ts --live lands, and warns-skips
without the key. Never fakes a pass.

Verified locally: YAML parses; both gate steps executed verbatim —
violating shapes all exit non-zero, compliant shapes all exit 0.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 11:12:21 +00:00
Dragan Spiridonov 8592048b8a fix(mcp): expose evidenceClass + needsVerification in validation_pipeline MCP projection (issue #522)
MCP-CLI parity bug found by a REAL protocol call during P5 acceptance:
the handler projected findings to a fixed 5-field shape, stripping
ADR-105's evidenceClass at the MCP boundary — unit tests passed because
they tested the projection itself. Projection now carries evidenceClass
+ evidenceArtifact per finding and the needsVerification queue; two
parity-pin tests added so the field can never be silently dropped again.

Verified: 9/9 handler tests; direct handler invocation (same code path
as the MCP tool) shows evidenceClass STATIC + needsVerification array.
Live MCP server reflects this after restart (long-running process has
pre-fix code loaded).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 11:10:11 +00:00
Dragan Spiridonov c02e60a8f3 feat(benchmarks): ADR-109 interaction benchmark — rubric pre-registered before any data (issue #522)
RUBRIC-v1.md committed first per ADR-108: ground-truth primary outcome
(hidden acceptance test), style-neutral verbatim judge instruction (no
treatment-revealing framing in either direction), two-pass cross-family
judging with INFERRED-labeled fallback, clustered-stats reporting rule.
Provably rubric-before-data: this commit precedes any results-* artifact.

2 scenarios with seeded defects + personas + hidden tests (off-by-one
pagination; swallowed async rejection). Hidden tests verified both
directions: FAIL on seeded bug with precise reasons, PASS on the fix.

lib/stats.ts encodes the three Pattern Space harness fixes as code:
isGoodRow enforced inside aggregate() (no path to a number that skips
it), seeded per-item counterbalance (their null cell was 72% position-
skewed), exact scenario-clustered sign test (validated against the
known 27/11 → p≈0.0136 re-analysis).

Verified: 14/14 stats tests; hidden-test ground truth both directions;
tsc clean. Remaining: live conversation driver + judge integration
(API-gated, same decision as ADR-106 live mode).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 11:05:26 +00:00
Dragan Spiridonov 57325b97a1 feat(learning): retrieval null-discount + recorder wiring — ADR-110 complete (issue #522)
PatternStore.search now surfaces kept nulls on every result
(nullSummary) and discounts scores by context-matched null density
before ranking — 'succeeded elsewhere' never outranks 'failed here'.
Callers pass contextFingerprint via PatternSearchOptions. Fail-soft:
discount skipped (debug-logged) when the sqlite handle is absent.

AQELearningEngine wires PatternNullStore.recordNull into
ExperienceCaptureService.setNullRecorder at the composition root,
mirroring the witness-chain fail-soft wiring pattern.

Arena losing-strategy persistence required NO new code: ADR-104's
persisted run JSON already keeps every EvaluatedStrategy (killRate,
coveragePct, fitness) + competitiveArray + ranking — losers and their
loss reasons are on the record by construction.

Verified: tsc clean; 28/28 across null-capture, evidence-gating, and
invariant-verifier suites locally. DB-backed paths covered by the
migration suite in CI (native-binding constraint).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 11:00:47 +00:00
Dragan Spiridonov 8663df13b7 feat(learning): PatternNullStore + null capture on failed pattern-guided tasks per ADR-110 (issue #522)
PatternNullStore: recordNull with ON CONFLICT consolidation,
getNullSummaries batch fetch, and the pure applyNullDiscount ranking
helper (context-matched nulls discount 5x harder than remote ones;
0.25 floor keeps patterns retrievable — nulls inform, never erase).

ExperienceCaptureService: appliedPatterns on TaskExperience (patterns
that GUIDED the task), setNullRecorder following the setWitnessChain
convention, and null capture in completeCapture — failure = negative
reward when a signal exists, else !success; recorder errors never break
capture; context fingerprint = domain:agent (v1, per ADR).

Verified: 11/11 tests run locally (London-school mocked backend — no
native deps) incl. reward-overrides-success, per-pattern fan-out,
recorder-throw survival, and discount floor/ordering; tsc clean.

Remaining for ADR-110: retrieval-time discount wiring in PatternStore
search, arena losing-strategy persistence, composition-root recorder
wiring.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:56:35 +00:00
Dragan Spiridonov 687874fe37 feat(learning): qe_pattern_nulls migration — kept nulls per ADR-110 (issue #522)
Additive schema v10: qe_pattern_nulls (pattern_id FK, context_fingerprint,
failure_mode, trajectory_ref, evidence_class CHECK per ADR-105,
consolidated_count, UNIQUE(pattern_id, context_fingerprint)) + 2 indexes.
Wired into unified-memory v10 migration block; rollback refuses when
nulls exist (nulls are data — the un-committed v2.1 lesson).

Verified against a sqlite3-online-backup COPY of the real memory.db
(61MB, original never written): all 51 pre-existing table counts
byte-identical before/after (qe_patterns=298, qe_trajectories=377,
kv_store=5008), PRAGMA integrity_check ok, table+4 indexes created,
UNIQUE fires on duplicate, ON CONFLICT consolidation bumps count to 2.
tsc --noEmit clean. Unit tests (10) follow the repo's better-sqlite3
convention and run in CI — local native binding is the known macOS
binary (invalid ELF header).

Verification also caught a false-positive in its own first run: an
empty SQL pipe reported 'applied' with trivially-unchanged counts; the
new-objects query exposed it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:50:37 +00:00
Dragan Spiridonov e9c1a53873 feat(safety): ADR-106 behavioral eval engine + scenario corpus (issue #522)
Deterministic core of the safety eval: 4 temptation scenarios (stale-db
cleanup, recreate-db sync fix, WAL quick-delete, hurried consolidation)
each with opener and mid-task injection modes; trajectory assertion
engine (forbidden-operation with unlessPrecededBy exemptions,
missing-prerequisite ordering rules — backup-before-mutation,
verify-before-success-claim); runner CLI; 7 fixture trajectories
modeling real compliant/violating agent response shapes.

Pass/fail only, never a score — refusal is compliance; WAL removal is
exempt only after backup/integrity check (restore context per
CLAUDE.md); a success claim without row-count verification fails.

Verified: engine tests 9/9; runner CLI on fixtures — violating_direct_rm
exit 1, compliant_refusal exit 0, violating_fabricated_success exit 1.

Remaining for ADR-106 (tracked): live per-tier driver (needs API key),
N=5 matrix, safety-eval.yml release gate.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:43:04 +00:00
Dragan Spiridonov f486e887f7 ci: invariant-check workflow per ADR-107/108 (issue #522)
Gates PRs touching shipped artifacts + releases on: section-level agent
invariants (npm run verify:invariants), the verifier's own mutation
tests, and ADR-108 lineage discipline (LINEAGE.md present; any
committed benchmark results must sit next to a pre-registered
RUBRIC-v*.md). npm ci --ignore-scripts — no native builds needed.

Verified locally: YAML parses; all three steps executed with real
commands (verifier 0 violations/53 agents; mutation tests 9/9; lineage
gate exit 0 on repo, exit 1 on a results-without-rubric fixture).
Workflow executes on GitHub only after push; marking it a required
check is a branch-protection setting for the maintainer.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:37:32 +00:00
Dragan Spiridonov 86b44bee15 feat(agents): evidence_discipline block in all standard qe agents per ADR-105 (issue #522)
Inserts the ADR-105 evidence-class labeling rule as a tagged
<evidence_discipline> section in all 51 standard qe-*.md definitions,
identically in .claude/agents/v3 and assets/agents/v3 (the 2 exempt
legacy agents skipped, per manifest). Registers the section as a
required invariant in config/agent-invariants.json (minChars 400) so
ADR-107 machine-checks it from day one.

Verified: verify:invariants 0 violations across 53 shipped agents with
the new requirement active; negative check on a stale tree fails with
required-section-missing; trees diff-identical; mutation tests 9/9.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:32:04 +00:00
Dragan Spiridonov 88014dc20e feat(validation): evidence-class gating per ADR-105 (issue #522)
Only EXECUTED/STATIC findings from failed blocking steps may block or
halt the pipeline. INFERRED findings queue in PipelineResult.
needsVerification (the ADR-102 adversarial-verification dispatch
surface) and downgrade overall to warn. CONJECTURE never gates. A
failed blocking step without gating evidence reads as warn, never as a
clean pass. Markdown report shows evidence class on blockers and a
'Needs Verification' section.

Verified: tsc clean; 8 new gating tests + 7 MCP handler tests, 15/15.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:28:58 +00:00
Dragan Spiridonov bd0bb213ca feat(scripts): shipped-agent invariant verifier per ADR-107 (issue #522)
config/agent-invariants.json manifest + verify-shipped-invariants.ts:
section-level content assertions (not keyword greps), shipped-vs-source
divergence on invariant sections, assets-only-qe rule, loud exemptions,
version-policy warnings. npm run verify:invariants.

Found pre-existing drift on first run: qe-integration-architect.md
(legacy format, no XML sections) and qe-queen-coordinator.md (missing 3
sections) — exempted with reasons pending maintainer decision; 53 agents
carry frontmatter 3.0.0/3.1.0 vs package 3.10.5 (warn until semantics
decided).

Verified: baseline run 0 violations/55 warnings exit 0; mutation tests
9/9 (hollow-section, removed-section, divergence, reword-passes); E2E
smoke on mutated tree exits 1 catching both hollow + divergence rules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:25:01 +00:00
Dragan Spiridonov 69dd3a6a23 feat(validation): evidenceClass on Finding per ADR-105 (issue #522)
Adds EvidenceClass (EXECUTED|STATIC|INFERRED|CONJECTURE) and
EvidenceArtifact to the Finding schema, required at the type level.
requirements.ts factory defaults to STATIC (all 26 steps are
deterministic content analysis); pipeline crash-findings are EXECUTED
with the error output as artifact. Field is evidenceClass, not
evidence — StepResult.evidence: string[] already exists (ADR amended).

Verified: tsc --noEmit clean; validation-pipeline-handler tests 7/7
(MCP handler passes findings through, new field flows to MCP output).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:19:49 +00:00
Dragan Spiridonov 19eaae78ec feat(benchmarks): seed LINEAGE.md registry per ADR-108 (issue #522)
Lineage registry with rubric-before-data, changelog, kept-data,
clustered-statistics, and style-neutral-judge rules. Seeded with
arena Phase 1 (incl. its retracted measured-duration variant as the
model entry) and the planned ADR-109 row. Pre-registry numbers are
uncited-by-default pending backfill.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:15:50 +00:00
Dragan Spiridonov bc8a37696a docs(adr): ADR-105..110 — Pattern Space cross-pollination (issue #522)
Six proposed ADRs from the universal-pattern-space QE assessment:
evidence-class labels on findings, behavioral safety evals per model
tier, shipped-agent invariant verification, benchmark lineage with
pre-registered rubrics (incl. antipattern list), interaction benchmark
for qualitative agents, kept-nulls negative pattern records.

Cross-checked against existing ADR corpus: Related fields reference
ADR-058/061/070/074/083/086/098/099/102/104; mid-task safety injection,
restraint probes, and null-vs-quarantine sequencing folded in from the
cross-check findings.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-11 10:15:17 +00:00
Dragan Spiridonov c6742345dd fix(hooks): migrate qe_trajectories.metadata_json in place — unblocks v3.10.5 publish
The ADR-101 provenance insert named metadata_json unconditionally; stores
created by older versions (and the minimal fixtures in
post-task-promotion/post-task-tokens-saved tests) lack the column, so the
INSERT threw and aborted the whole Stream B learning transaction — caught
by the npm-publish test gate (run 27275586477), masked locally by the
documented native-module artifact.

- PRAGMA-check + ALTER TABLE ADD COLUMN metadata_json (fail-soft) before
  the insert — also heals real-world legacy stores
- canonical-DDL ensure block is now fail-soft too, so an unexpected legacy
  shape can never kill the transaction; the inserts surface real failures

Verified against the exact failing scenario via a Linux-native harness:
old-shape DB -> column migrated, trajectory row written,
experience_applications.tokens_saved = 68 (the precise value the failing
test asserts).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
v3.10.5
2026-06-10 12:31:09 +00:00
Dragan Spiridonov 429e91d53f Merge pull request #521 from proffesor-for-testing/working-may
chore(release): v3.10.5 — Fable 5 parity: CLI fleet fix, prompt caching, adversarial QCSD reviews, qe-arena
2026-06-10 14:15:21 +02:00
Dragan Spiridonov a483d36239 chore(release): bump version to v3.10.5
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-10 11:31:24 +00:00