CodeQL flagged 'polynomial regular expression on uncontrolled data' in two
branch-new files:
- ollama-url.ts: trailing-slash strip `/\/+$/` backtracks O(n^2) on many
'/'. Replaced with a linear charCodeAt scan.
- pattern-cache.ts: block-comment strip `/\/\*[\s\S]*?\*\//g` backtracks
O(n^2) on unterminated '/*' (46s on a 300k adversarial input). Replaced with
a single-pass indexOf('*/') scanner.
Both are behavior-identical to the originals on all normal/edge cases (verified)
and now run in <5ms on the adversarial inputs. 14 affected tests pass; typecheck
clean. Fixes code-scanning alerts 212 & 213.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
3.11.4 = the local-model-improvements branch (LLM/Ollama plumbing, free-tier
local test-gen, proof-gate ADR-116) + the already-merged dependency-security
system (ADR-115). Merged origin/main in first so the release carries the
grpc-js HIGH fix and moderate-CVE reductions rather than regressing them.
Bumps package.json + package-lock + skills-manifest fleetVersion to 3.11.4.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The proof-gate work cited only upstream RuVector 'ADR-194' — which is not an
AQE in-repo ADR. Adds ADR-116 (the AQE-side integration decision: port the
hash-chain to TS, opt-in/fail-soft/single-store wiring, and why NOT to depend
on the @ruvector npm packages — backed by the verified ruvllm mock-mode/demo
findings) and cross-references it from the four source/test files that cite
ADR-194, so the in-repo decision is discoverable from the code.
Comment/docs only — no behavior change.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Runs the new `npm run deps:update` (security mode: npm audit fix
--package-lock-only) — the system applied to its own repo. Fixes the
@opentelemetry/* advisory family (W3C Baggage unbounded-memory) and bumps
the @claude-flow/* alpha chain, all within existing semver ranges.
In-repo audit: moderate 29 -> 7; high/critical stay 0.
Large diff is npm dedup: the @claude-flow/memory bump (alpha.14 -> alpha.21)
collapsed nested duplicate transitive trees (844 -> 807 lock entries).
Optional @ruvector platform deps verified intact (15/15) by the updater guard.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wires the HashChainGate primitive into UnifiedMemoryManager so kv writes can
be cryptographically audited end-to-end. Design is audit-only and safe for the
production path:
- OPT-IN: off by default; enable per audit window via enableProofGate() or
AQE_PROOF_GATE=1 (the chain grows O(n) with writes, so it is meant for
bounded sync/migration/publish windows, not permanent always-on use).
- FAIL-SOFT: kvSet/kvDelete admit a content-bound receipt AFTER the row is
written, wrapped so an audit failure can never break or alter the write.
- NO RE-ENTRANCY: the chain persists via a direct write to a reserved
namespace (__proofgate__), never through kvSet, and that namespace is never
itself audited.
- UNIFIED STORE: the chain serializes into the same memory.db (kv_store), no
second store; it restores on the next process so audits survive restarts.
verifyMemoryIntegrity() re-derives the chain from genesis, turning the
"verify row counts before/after, never claim a sync succeeded without proof"
mandate into a tamper-evident guarantee. 7 tests: no-op-when-disabled,
one-receipt-per-write, reserved-ns-not-self-audited, root-advances,
persist+restore across restart, mutated-chain detection, and env auto-enable.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a dependency-free TS port of ruvector-proof-gate's HashChainGate
(ADR-194): admit one tamper-evident receipt per memory.db operation;
verifyIntegrity() cryptographically re-derives the chain to prove no
admitted entry was mutated, reordered, or dropped — turning the project's
"verify row counts before/after, never claim a sync succeeded without
proof" mandate into a guarantee. Uses Node crypto only, opens no DB (chain
serializes via toJSON/fromJSON for persistence through the unified store).
7 tests mirror the Rust crate's tamper cases + a persistence round-trip.
Delivered as a primitive; NOT yet wired into live memory.db paths.
Also records the 2026-06-29 small-coder oracle measurements in
value-score.ts: qwen2.5-coder:7b=0.50 (4/8, beats general 8B at 38%) and
1.5b=0.08 — the curve motivating a PEFT-on-7B spike.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-29 oracle bench (8 fixtures, mutation>=60%) on the same Mac Ollama:
qwen3-coder:30b and qwen3:30b-a3b score IDENTICALLY (both 6/8, 75% avg
mutation kill, same fixtures pass/fail) — same 30B MoE class and RAM — but
the code-tuned model is ~14x faster wall-clock (5.2s vs 71.1s/gen) because
it does not spend ~70s "thinking" before answering.
Records both measurements in value-score.ts (incl. stock qwen2.5-coder:1.5b
= 1/8, confirming size still dominates coder-tuning at tiny scale). qwen3:8b
stays off-default (3/8 across the same fixtures).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds TestGenPatternCache (exact normalized-source match + cosine-similarity
tiers, injectable embedder, in-memory — no new DB, respects the unified
memory ADR) and wires it into the free-tier executor as an opt-in gate.
Safety: a cache hit is only a candidate — it is re-verified by the same
objective oracle before use, so a stale/wrong hit falls through to real
generation and can never bypass the gate. Only verified outputs are cached.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Registers qwen3:8b, qwen3:30b-a3b, qwen3-coder:30b in model-mapping
(so normalizeModelId resolves the tags) and model-registry with correct
capability flags (coder = tool-capable, non-thinking).
Adds modelSupportsTools() (fail-open on unknown ids) and a guard in the
hybrid router: when a request requires tools, candidates KNOWN to be
tool-less are skipped (with a warning) instead of silently dropping tool
definitions. Never strands a request — falls open if no candidate qualifies.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
#2: single resolveOllamaBaseUrl() knob (AQE_OLLAMA_URL > OLLAMA_URL >
each call-site default) wired into all local clients — chat provider,
consensus provider, local judge, embeddings, free-tier. Unset = identical
behavior; one var now repoints every client at a remote/host Ollama.
#3: size each Ollama request timeout from num_predict and the model's
observed tok/s (EMA) instead of a fixed 120s, so slow dense/thinking
models (~15 tok/s measured) no longer silently time out mid-generation.
Explicit/configured timeouts still win.
Adds scripts/oracle-model-bench-local.ts (local mutation-oracle bench).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Close the biggest remaining drift gap (ADR-114 follow-up).
- skill-parity now gates plugins/agentic-qe-fleet/skills as a strict body-mirror alongside
assets/skills; reconcileBody + `check-skill-parity --sync` resync drifted mirrors while
preserving each tree's frontmatter (used to fix the plugins qe-quality-assessment drift)
- .kiro/skills is an intentionally divergent 86-skill variant (qe- prefixed, some large
.kiro-only bodies) — gated by PRESENCE via the conservation guard's kiro-skills surface,
not body-equality (which would destroy the divergence)
- both guards (verify:skill-parity blocking, verify:conservation non-blocking) wired into
invariant-check.yml with plugins/.kiro paths; policy doc updated
- tests: reconcileBody/splitFrontmatter covered; 16 validation tests pass
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Consolidate C4 diagram generation onto a single engine and expose it to
users via CLI and MCP, with a deterministic confidence gate and real
Knowledge-Graph-derived relationships.
- Consolidation (C1/C6): C4ModelService is the single render/analyze/store
engine; the bridge delegates and FAILS LOUD on a render error instead of
silently degrading. Duplicate inline generators removed.
- C2: real component relationships from the Knowledge Graph (AST import/call
edges) replace the naming heuristic; project-scoped via a new KG basePath
so repos outside cwd don't trip the path-traversal guard.
- C3: deterministic confidence gate (high/medium/low + reasons) on every
diagram — surfaces the detector's known limits instead of hiding them.
- C4/C5: `aqe code c4` CLI + `qe/code/c4` MCP tool (generate/search),
verified at MCP-CLI parity and through the protocol-server bridge.
- Search: generate persists embeddings (opt-in `enableC4Embeddings`, on for
MCP) so `qe/code/c4 search` returns hits in the standard flow.
- Fixed pre-existing lint in touched files (require->import, unused catches).
Docs: ADR-112 + docs/guides/c4-architecture-diagrams.md. tsc + lint clean;
C4 surface fully green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The absent-drill ran `npm ci --omit=optional` then vitest — but vitest's bundler
(rolldown) has its OWN native binary in optionalDependencies, so a blanket
--omit=optional removed it and the test runner crashed (MODULE_NOT_FOUND for
rolldown-binding) before testing anything. There's no selective omit.
Fix: install + build normally, then simulate ONLY the APP's native optionals
absent (rm @ruvector/* + hnswlib-node + rvlite), and prove the degraded path with
a PURE-NODE smoke (no test runner): the CLI boots and loadOptionalModule degrades
on a genuinely-absent app native while still loading present modules. Version read
from package.json (no hardcode). App code unchanged — this was a workflow flaw.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
ADR-150 graceful-degradation core. AQE's native/optional deps (@ruvector/*,
hnswlib-node, rvlite) must degrade — not crash — when a platform binary or a
Dependabot-pruned lockfile is missing.
- src/shared/optional-module.ts: loadOptionalModule(name, req?) → {available}|
{degraded}. Fixes the #528 anti-pattern in the existing ad-hoc
try{require}catch{} (6+ ruvector wrappers): degrades ONLY on genuine absence
(MODULE_NOT_FOUND) and RE-THROWS real load failures (ABI mismatch / corrupt
.node) instead of masking them. 8 tests incl. the rethrow property.
- .github/workflows/degraded-mode.yml: the absent drill — npm ci --omit=optional
--ignore-scripts, then run the degraded-path tests. Pairs with the regular
install job (proves both paths, #528).
Follow-up (held, CI-verified): migrate the ad-hoc sites + the static import in
sona-wrapper.ts onto loadOptionalModule (changes a production native-load path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigated A8 (pin @metaharness/darwin + retire the type-mirror):
- VERIFIED: AQE's DarwinScoreCard mirror is byte-identical to upstream ScoreCard
from @metaharness/darwin@0.2.1 THROUGH the published 0.7.0 (no drift, 5 minors).
- HELD the dep-swap deliberately: upstream is fast-moving pre-1.0 with napi/wasm
transitive deps; adding it changes every user's install + lockfile and can't be
verified locally (host-shared-node_modules breaks native rebuilds) — negative-EV
vs 74 stable, zero-coupling mirror lines.
- SHIPPED the version-contract half: pinned provenance (0.2.1–0.7.0) + a drift-
guard test (darwin-version-contract.test.ts) that fails CI on any field-set
divergence. 17 darwin tests green; tsc clean.
Swap only after upstream stabilizes (≥1.0) + a CI-verified install impact; needs OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The opt-in "verify before you emit" primitive a vertical/harness embeds: route
findings through the blind refuters, drop the refuted, surface every verdict for
witness, report overhead. Built in @ruvector/adversarial-verify (vendor-ready,
zero deps) so MetaHarness can embed it.
- gate.ts: verifyGate(findings, {judge, enabled, dropUncertain}) →
{emitted, blocked, all, latencyMs, judgeCalls}. Disabled ⇒ no-op, ZERO judge
calls (cost-neutral). Default-off-able per cheap-vertical posture.
- gate.test.ts (6): (a) false claim killed (b) true survives (c) no-op cost-
neutral when disabled (d) overhead measured. 31 green across the module; tsc clean.
MetaHarness-side embedding (template var + generated-harness wiring) is spec'd in
plan 05 — cross-repo (agent-harness-generator) + needs the package vendored/
published, both pending explicit OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Factor AQE's blind-refuter primitive out of qcsd-development-review.js into a
host-agnostic, ZERO-AQE-dependency module (publishable as @ruvector/* later;
not published per no-publish-without-OK). Closes G2.
- src/verification/adversarial-verify/: types (Finding/FindingVerdict/Judge),
blind refuterPrompt + lenses (ADR-074), pure k-of-n majority-kill synthesis,
adversarialVerify orchestrator (injected Judge — no agent()/Claude dep),
calibrate() (false-kill/false-keep vs ground truth, any Judge).
- 25 tests: synthesis math, blind orchestration, calibration (majority-of-3 ≤
single-refuter error), and PARITY — byte-identical to the workflow's inline
synthesis. The Workflow harness sandboxes .claude/workflows (no imports), so
the workflow inline-mirrors the package; annotated canonical + parity-guarded.
- Verification (plan A2): (a) synthesis tests ✓ (b) calibration mechanism ✓
(c) regression parity ✓ (d) zero AQE import (grep) ✓. strict-tsc clean.
Remaining (optional): empirical real-LLM false-kill on a labeled corpus (inject
a real local/OpenRouter Judge into calibrate()). Unblocks A5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A12 measured cross-model best-of-N at +6 composite over single-model (diverse
models cover each other's validity failures; the win is captured by first-pass
selection, no judge needed). Wire it as an opt-in capability:
- executor: candidateProviders?: FreeTierProviderConfig[] — when set, round-0
best-of-k at the START free tier draws each candidate from a DIFFERENT provider
(e.g. local qwen + an OpenRouter model) instead of one model's variants;
selection stays the objective verifier (first-pass = validity rescue). Per-
candidate attempt.model telemetry. Off (single-model) unless set.
- coordinator-support: FreeTierCoordinatorConfig.freeTierCandidateProviders
passthrough so coordinators can opt in.
- tests: cross-model rescue (falls through to the valid model, no escalation) +
single-model-unchanged. 87 green across the lane; strict-tsc clean.
No judge wired — A12 showed it does not beat first-valid on the cross-model pool.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured n=20 (qwen3:30b local + glm-5.2 OpenRouter, judge deepseek-v3.2):
- Cross-model best 84.8 vs qwen-alone 63.5 / glm-alone 71.2; the two models cover
each other's validity failures (qwen 0-valid on 4 instances, glm on 3, both on 11).
- Selector delivery (14 judged): judge 82.8 vs first-valid 82.2 (Δ0.6, noise) vs
qwen-alone 76.1 → cross-model gives +6 over single-model and barely needs the judge.
- A11 judge's same-model 88.9% did NOT transfer cross-model (33% best-pick).
- Production lever = cross-model GENERATION + cheap selection, NOT a judge.
Fixed a tautological metric: "union lift vs max(single models)" was always 0
(== cross-model best); now reports lift vs EACH model alone + validity rescue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code check of FreeTierEscalatingExecutor: each escalated tier's round-0 prompt is
built from req.messages (cold), not the prior tier's failed output (executor.ts:248-251).
The only warm context is within-tier repairMessages — correctly scoped. So the
plan's "executor escalates warm" claim was wrong; ADR-182 cold-start is already
satisfied. A12 reduces to the cross-model best-of-N lever (d3-xmodel.mjs, running).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A11 validated the judge (deepseek-v3.2 = 88.9%) but its aggregate gain was ~0
because a single model's candidates cluster. A12 generates candidates ACROSS
families to widen the spread so the judge can pay off.
- d3-xmodel.mjs: generators = local qwen3:30b-a3b + OpenRouter z-ai/glm-5.2
(cross-family); judge = deepseek-v3.2 (a third family — clean writer≠evaluator).
No Anthropic, per preference. Routes local↔OpenRouter by slug; scores every
candidate by REAL mutation-kill+coverage (ground truth, judge blind).
- Measures: union lift (cross-model best vs best-single-model), per-model win
share, and the judge's regret-vs-first-valid on the wider pool.
- Smoke (1 instance): GLM 82.5 > qwen 75.0; judge picked GLM → Δ +7.5 vs
first-valid (vs A11's clustered ~0.5). Full n=20 run pending.
Model note: Qwable-3.6-27b (local, Qwen3.5 dense) is diverse but too slow
(9.3 tok/s) for the loop — OpenRouter cross-family generator used instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The v1 "judge does not beat first-valid" was a measurement artifact: candidate
code was truncated to 2500 chars and the judge was the cheapest model. Retry
(full code, multi-judge, n=30):
- deepseek-v4-flash: 71.4% -> 77.8% best-pick (truncation fix alone).
- deepseek-v3.2 (stronger): 88.9% (8/9 discriminating) — clears Ruv's 88% bar.
So the judge IS reliable. But aggregate regret gain over first-valid is only
~0.5 pts (within noise) because qwen's candidates cluster — the judge's payoff is
gated on candidate DIVERSITY, which is the empirical case for A12 (cross-model
best-of-N). A11 proved the judge works; A12 gives it spread to act on.
- d3-judge.mjs: multi-judge (D3_JUDGE_MODELS), full candidate code, per-judge
3-way verdict; last-number parse for reasoning judges.
- plan 05: A11 corrected to VALIDATED-but-spread-gated; forward plan -> A12.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Precision-validated the writer≠evaluator LLM-judge selector against AQE's
mutation-kill ground truth (writer=local qwen3:30b, judge=OpenRouter
deepseek-v4-flash — no Anthropic, per preference).
Result (n=20, 7 discriminating): judge picks best 71.4% (< Ruv's 88% bar);
composite regret judge 2.1 vs first-valid 1.9 vs random 2.0 = within noise.
Verdict: do NOT wire the judge — qwen's QE-test candidates cluster, so first-valid
is already near-optimal and a cheap judge adds no measurable lift. Could flip with
a stronger judge or wider candidate spread (A12 cross-model best-of-N).
- d3-judge.mjs: honest 3-way verdict (INCONCLUSIVE within noise vs WIN/LOSE).
- plan 05: A11 status + forward plan (A1 done, A11 negative, A12 next).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Implements the asymmetric escalation lane corrections measured in D3 and makes
the opt-in reusable beyond test generation. All opt-in, off by default.
- executor: bestOfK diverse attempts per tier (§12 — single gated shots Goodhart,
best-of-k converts); oracleKind 'objective'|'self-authored' Goodhart guard (§10 —
a self-authored gate is withheld from routing-feedback so it can't lift confidence).
- coordinator-support: reusable buildFreeTierExecutor factory + runFreeTierTextTask;
default cheap model raised qwen3:8b -> qwen3:30b-a3b (D3: 8B below the gen floor).
- test-generation: refactored onto the factory; bestOfK=2 (the D3-validated config,
was running single-shot).
- requirements-validation: second adopter — opt-in cheap-first BDD/Gherkin gen with a
strengthened relevance oracle (rejects off-topic boilerplate, resists §10) + D9 sink.
- tests: executor (best-of-k, guard), coordinator-support factory, requirements opt-in
(incl. anti-Goodhart + D9). Full free-tier+coordinator sweep green; strict-tsc clean.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Closes G1 of the cross-pollination plan: AQE now governs its own ~90-tool
MCP surface with a default-deny policy + CI gate ("npm audit for agent tools").
- .harness/mcp-policy.json: truthful default-deny snapshot; _enforcement labels
what is wired (tool-scoping default-deny, witness-chain audit) vs posture/
stated-intent — not theater.
- scripts/mcp-policy-gate.mjs: self-contained mirror of MetaHarness mcp-scan's
HIGH checks; also reads .mcp.json (closing the scan false-negative where AQE's
server is declared there, not in .claude/settings.json). Exit 1 on any HIGH.
- .github/workflows/mcp-policy-gate.yml + `npm run security:mcp-policy`.
- tool-scoping: fix the interface doc-comment that claimed "empty/undefined ->
allowAll" (the code DENIES by default); export ALL_AGENT_ROLES.
- tests/unit/mcp/mcp-policy-gate.test.ts: drift guard (policy roles <-> enforcement),
default-deny posture. 46/46 green; scan = 0 HIGH; fails-closed verified.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Evolve the QE framework from PACT (Proactive, Autonomous, Collaborative,
Targeted) to PACTS by adding a fifth principle: Structured — governance,
observability, and explainability of agent behavior. We measure confidence,
not trust (trust is a human feeling; confidence can be mathematically
explained). Especially important in regulated industries. Inspired by DORA's
research on AI-assisted delivery and platform-engineering quality measurement.
Anti-pattern: bolting agentic AI onto chaos.
Scope:
- Source skills (.claude/skills): holistic-testing-pact + agentic-quality-
engineering SKILL.md, agentic-quality-engineering schema (added optional
`structured` dimension, kept slug keys), README, skills-manifest.json
- Regenerated derived artifacts: assets/skills (via prepare-assets equivalent
copy) and the two affected .opencode yaml (via generate-opencode-skills)
- Hand-maintained trees updated consistently: .kiro skills, docs (catalog,
glossary, intro), plugin/marketplace manifests, qe-learning-coordinator
Stability:
- Kept skill id/dir `holistic-testing-pact` (and `.kiro` `qe-holistic-testing-pact`)
unchanged — referenced across other repos. Only titles/descriptions/content
changed to PACTS.
Attribution: PACT originated with Reuven Cohen (Agentics Foundation); adapted
to QE by Dragan Spiridonov. PACTS adds Structured.
Co-Authored-By: claude-flow & Agentic QE fleet
Free-tier local test generation (opt-in) — cheap-first on local/free models
with a repair loop, auto-escalation to paid tiers, and self-learning routing
feedback. Plus #541 dependency fixes (ruvllm optional deps + hono security).
Version synced across all product manifests: package.json, fleetVersion in
both .claude/ and assets/ skills-manifests, and .claude-plugin/plugin.json
(the latter two had drifted at 3.10.4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dragan Spiridonov