Consolidate C4 diagram generation onto a single engine and expose it to
users via CLI and MCP, with a deterministic confidence gate and real
Knowledge-Graph-derived relationships.
- Consolidation (C1/C6): C4ModelService is the single render/analyze/store
engine; the bridge delegates and FAILS LOUD on a render error instead of
silently degrading. Duplicate inline generators removed.
- C2: real component relationships from the Knowledge Graph (AST import/call
edges) replace the naming heuristic; project-scoped via a new KG basePath
so repos outside cwd don't trip the path-traversal guard.
- C3: deterministic confidence gate (high/medium/low + reasons) on every
diagram — surfaces the detector's known limits instead of hiding them.
- C4/C5: `aqe code c4` CLI + `qe/code/c4` MCP tool (generate/search),
verified at MCP-CLI parity and through the protocol-server bridge.
- Search: generate persists embeddings (opt-in `enableC4Embeddings`, on for
MCP) so `qe/code/c4 search` returns hits in the standard flow.
- Fixed pre-existing lint in touched files (require->import, unused catches).
Docs: ADR-112 + docs/guides/c4-architecture-diagrams.md. tsc + lint clean;
C4 surface fully green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ADR-150 graceful-degradation core. AQE's native/optional deps (@ruvector/*,
hnswlib-node, rvlite) must degrade — not crash — when a platform binary or a
Dependabot-pruned lockfile is missing.
- src/shared/optional-module.ts: loadOptionalModule(name, req?) → {available}|
{degraded}. Fixes the #528 anti-pattern in the existing ad-hoc
try{require}catch{} (6+ ruvector wrappers): degrades ONLY on genuine absence
(MODULE_NOT_FOUND) and RE-THROWS real load failures (ABI mismatch / corrupt
.node) instead of masking them. 8 tests incl. the rethrow property.
- .github/workflows/degraded-mode.yml: the absent drill — npm ci --omit=optional
--ignore-scripts, then run the degraded-path tests. Pairs with the regular
install job (proves both paths, #528).
Follow-up (held, CI-verified): migrate the ad-hoc sites + the static import in
sona-wrapper.ts onto loadOptionalModule (changes a production native-load path).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Investigated A8 (pin @metaharness/darwin + retire the type-mirror):
- VERIFIED: AQE's DarwinScoreCard mirror is byte-identical to upstream ScoreCard
from @metaharness/darwin@0.2.1 THROUGH the published 0.7.0 (no drift, 5 minors).
- HELD the dep-swap deliberately: upstream is fast-moving pre-1.0 with napi/wasm
transitive deps; adding it changes every user's install + lockfile and can't be
verified locally (host-shared-node_modules breaks native rebuilds) — negative-EV
vs 74 stable, zero-coupling mirror lines.
- SHIPPED the version-contract half: pinned provenance (0.2.1–0.7.0) + a drift-
guard test (darwin-version-contract.test.ts) that fails CI on any field-set
divergence. 17 darwin tests green; tsc clean.
Swap only after upstream stabilizes (≥1.0) + a CI-verified install impact; needs OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The opt-in "verify before you emit" primitive a vertical/harness embeds: route
findings through the blind refuters, drop the refuted, surface every verdict for
witness, report overhead. Built in @ruvector/adversarial-verify (vendor-ready,
zero deps) so MetaHarness can embed it.
- gate.ts: verifyGate(findings, {judge, enabled, dropUncertain}) →
{emitted, blocked, all, latencyMs, judgeCalls}. Disabled ⇒ no-op, ZERO judge
calls (cost-neutral). Default-off-able per cheap-vertical posture.
- gate.test.ts (6): (a) false claim killed (b) true survives (c) no-op cost-
neutral when disabled (d) overhead measured. 31 green across the module; tsc clean.
MetaHarness-side embedding (template var + generated-harness wiring) is spec'd in
plan 05 — cross-repo (agent-harness-generator) + needs the package vendored/
published, both pending explicit OK.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Factor AQE's blind-refuter primitive out of qcsd-development-review.js into a
host-agnostic, ZERO-AQE-dependency module (publishable as @ruvector/* later;
not published per no-publish-without-OK). Closes G2.
- src/verification/adversarial-verify/: types (Finding/FindingVerdict/Judge),
blind refuterPrompt + lenses (ADR-074), pure k-of-n majority-kill synthesis,
adversarialVerify orchestrator (injected Judge — no agent()/Claude dep),
calibrate() (false-kill/false-keep vs ground truth, any Judge).
- 25 tests: synthesis math, blind orchestration, calibration (majority-of-3 ≤
single-refuter error), and PARITY — byte-identical to the workflow's inline
synthesis. The Workflow harness sandboxes .claude/workflows (no imports), so
the workflow inline-mirrors the package; annotated canonical + parity-guarded.
- Verification (plan A2): (a) synthesis tests ✓ (b) calibration mechanism ✓
(c) regression parity ✓ (d) zero AQE import (grep) ✓. strict-tsc clean.
Remaining (optional): empirical real-LLM false-kill on a labeled corpus (inject
a real local/OpenRouter Judge into calibrate()). Unblocks A5.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A12 measured cross-model best-of-N at +6 composite over single-model (diverse
models cover each other's validity failures; the win is captured by first-pass
selection, no judge needed). Wire it as an opt-in capability:
- executor: candidateProviders?: FreeTierProviderConfig[] — when set, round-0
best-of-k at the START free tier draws each candidate from a DIFFERENT provider
(e.g. local qwen + an OpenRouter model) instead of one model's variants;
selection stays the objective verifier (first-pass = validity rescue). Per-
candidate attempt.model telemetry. Off (single-model) unless set.
- coordinator-support: FreeTierCoordinatorConfig.freeTierCandidateProviders
passthrough so coordinators can opt in.
- tests: cross-model rescue (falls through to the valid model, no escalation) +
single-model-unchanged. 87 green across the lane; strict-tsc clean.
No judge wired — A12 showed it does not beat first-valid on the cross-model pool.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Measured n=20 (qwen3:30b local + glm-5.2 OpenRouter, judge deepseek-v3.2):
- Cross-model best 84.8 vs qwen-alone 63.5 / glm-alone 71.2; the two models cover
each other's validity failures (qwen 0-valid on 4 instances, glm on 3, both on 11).
- Selector delivery (14 judged): judge 82.8 vs first-valid 82.2 (Δ0.6, noise) vs
qwen-alone 76.1 → cross-model gives +6 over single-model and barely needs the judge.
- A11 judge's same-model 88.9% did NOT transfer cross-model (33% best-pick).
- Production lever = cross-model GENERATION + cheap selection, NOT a judge.
Fixed a tautological metric: "union lift vs max(single models)" was always 0
(== cross-model best); now reports lift vs EACH model alone + validity rescue.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Code check of FreeTierEscalatingExecutor: each escalated tier's round-0 prompt is
built from req.messages (cold), not the prior tier's failed output (executor.ts:248-251).
The only warm context is within-tier repairMessages — correctly scoped. So the
plan's "executor escalates warm" claim was wrong; ADR-182 cold-start is already
satisfied. A12 reduces to the cross-model best-of-N lever (d3-xmodel.mjs, running).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A11 validated the judge (deepseek-v3.2 = 88.9%) but its aggregate gain was ~0
because a single model's candidates cluster. A12 generates candidates ACROSS
families to widen the spread so the judge can pay off.
- d3-xmodel.mjs: generators = local qwen3:30b-a3b + OpenRouter z-ai/glm-5.2
(cross-family); judge = deepseek-v3.2 (a third family — clean writer≠evaluator).
No Anthropic, per preference. Routes local↔OpenRouter by slug; scores every
candidate by REAL mutation-kill+coverage (ground truth, judge blind).
- Measures: union lift (cross-model best vs best-single-model), per-model win
share, and the judge's regret-vs-first-valid on the wider pool.
- Smoke (1 instance): GLM 82.5 > qwen 75.0; judge picked GLM → Δ +7.5 vs
first-valid (vs A11's clustered ~0.5). Full n=20 run pending.
Model note: Qwable-3.6-27b (local, Qwen3.5 dense) is diverse but too slow
(9.3 tok/s) for the loop — OpenRouter cross-family generator used instead.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The v1 "judge does not beat first-valid" was a measurement artifact: candidate
code was truncated to 2500 chars and the judge was the cheapest model. Retry
(full code, multi-judge, n=30):
- deepseek-v4-flash: 71.4% -> 77.8% best-pick (truncation fix alone).
- deepseek-v3.2 (stronger): 88.9% (8/9 discriminating) — clears Ruv's 88% bar.
So the judge IS reliable. But aggregate regret gain over first-valid is only
~0.5 pts (within noise) because qwen's candidates cluster — the judge's payoff is
gated on candidate DIVERSITY, which is the empirical case for A12 (cross-model
best-of-N). A11 proved the judge works; A12 gives it spread to act on.
- d3-judge.mjs: multi-judge (D3_JUDGE_MODELS), full candidate code, per-judge
3-way verdict; last-number parse for reasoning judges.
- plan 05: A11 corrected to VALIDATED-but-spread-gated; forward plan -> A12.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Precision-validated the writer≠evaluator LLM-judge selector against AQE's
mutation-kill ground truth (writer=local qwen3:30b, judge=OpenRouter
deepseek-v4-flash — no Anthropic, per preference).
Result (n=20, 7 discriminating): judge picks best 71.4% (< Ruv's 88% bar);
composite regret judge 2.1 vs first-valid 1.9 vs random 2.0 = within noise.
Verdict: do NOT wire the judge — qwen's QE-test candidates cluster, so first-valid
is already near-optimal and a cheap judge adds no measurable lift. Could flip with
a stronger judge or wider candidate spread (A12 cross-model best-of-N).
- d3-judge.mjs: honest 3-way verdict (INCONCLUSIVE within noise vs WIN/LOSE).
- plan 05: A11 status + forward plan (A1 done, A11 negative, A12 next).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Evolve the QE framework from PACT (Proactive, Autonomous, Collaborative,
Targeted) to PACTS by adding a fifth principle: Structured — governance,
observability, and explainability of agent behavior. We measure confidence,
not trust (trust is a human feeling; confidence can be mathematically
explained). Especially important in regulated industries. Inspired by DORA's
research on AI-assisted delivery and platform-engineering quality measurement.
Anti-pattern: bolting agentic AI onto chaos.
Scope:
- Source skills (.claude/skills): holistic-testing-pact + agentic-quality-
engineering SKILL.md, agentic-quality-engineering schema (added optional
`structured` dimension, kept slug keys), README, skills-manifest.json
- Regenerated derived artifacts: assets/skills (via prepare-assets equivalent
copy) and the two affected .opencode yaml (via generate-opencode-skills)
- Hand-maintained trees updated consistently: .kiro skills, docs (catalog,
glossary, intro), plugin/marketplace manifests, qe-learning-coordinator
Stability:
- Kept skill id/dir `holistic-testing-pact` (and `.kiro` `qe-holistic-testing-pact`)
unchanged — referenced across other repos. Only titles/descriptions/content
changed to PACTS.
Attribution: PACT originated with Reuven Cohen (Agentics Foundation); adapted
to QE by Dragan Spiridonov. PACTS adds Structured.
Co-Authored-By: claude-flow & Agentic QE fleet
Free-tier local test generation (opt-in) — cheap-first on local/free models
with a repair loop, auto-escalation to paid tiers, and self-learning routing
feedback. Plus #541 dependency fixes (ruvllm optional deps + hono security).
Version synced across all product manifests: package.json, fleetVersion in
both .claude/ and assets/ skills-manifests, and .claude-plugin/plugin.json
(the latter two had drifted at 3.10.4).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Completes the cheap-first economics: local → repair → escalate the hard tail.
- coordinator: build a claudeRunner from the existing HybridRouter (tier →
complexity hint) and run the free-tier path with escalate:true. Falls back to
local-only automatically when no router is wired. patternsUsed now records the
winning tier.
- plugin (kernel layer): onInitialize constructs a live RoutingFeedbackCollector
(EMA calibration + auto-escalation, memory-only fallback) when the free tier is
opted in, and injects it as the coordinator's routingFeedback sink — so each
cheap-vs-escalated outcome drives the calibrator/escalation tracker/confidence
(lifts the stuck-at-40% routing metric). Built only when opted in.
- tests: +2 (escalation via router, D9 sink invocation); 113 green; existing
coordinator (47) and plugin (38) suites unchanged. Type-safe (MessageRole /
TaskComplexity / RoutingConfig fields all confirmed).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Apply the free-tier escalation work to the test-generation coordinator as an
OPT-IN, default-off capability (Ruv's cheap-first economics). Zero impact unless
enabled via `enableFreeTier` config or `AQE_FREE_TIER=1`.
- coordinator: guarded `tryFreeTierGeneration()` tries the free local model first
(reads source, prompts, verifies test+assertion), falls through to the
unchanged paid path on any miss. 47 existing coordinator tests still green.
New optional `routingFeedback?` ctor param wires D9.
- D8 repair loop: executor gains same-tier repair (`repairAttempts`, verifier
feedback fed back) + `escalate:false` repair-only mode. Coordinator path runs
local-only + repair, no paid escalation yet.
- D9 sink: createRoutingFeedbackSink() maps executor outcomes onto the existing
RoutingFeedbackCollector (calibrator + escalation + confidence); cheap wins
raise the cheap tier's confidence, lifting the stuck-at-40% routing metric.
- docs: user guide docs/guides/free-tier-local-models.md (local/cloud Ollama,
OpenRouter, OpenAI-compatible) linked from README.
- tests: +11 new (executor repair, sink, coordinator opt-in); 111 green total;
free-tier modules strict-tsc clean.
Model: qwen3:8b default. Keys read from named env vars, never stored.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Turns the dormant free-tier ladder into a running loop implementing Ruv's
SWE-bench economics for a QE task: run on the cheap local tier, verify with an
objective oracle, escalate this task up the ladder on failure, and record the
start-tier verdict so the base tier adapts across tasks.
- executor.ts: decoupled by injection — free tiers via freeTierChat, Claude
tiers via an injected ClaudeTierRunner (no Anthropic SDK dep; coordinator
delegates to its existing HybridRouter). Local-only mode when no runner.
onOutcome sink = the D9 seam into routing-feedback.
- tests: 8 new (53/53 green across free-tier + tracker), strict-tsc clean.
- prototype/d7-wire-proof.mjs: LIVE — qwen3:8b generated a correct vitest test
that passed the objective verifier in 8.65s ($0, no escalation); an
impossible verify climbed local->haiku->sonnet->opus and failed at the top.
Model: qwen3:8b (most productive worker in D0; 5.2GB fits the 8GB story).
Additive only — no shipped coordinator hot-path edited. Adoption is a one-call
snippet documented in docs/metaharness/06 (behind a flag; needs sign-off).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Ruv's SWE-bench Round-3 economics for AQE: cheap-local handles the bulk,
escalate only failures up the existing Claude tiers. Maps onto AQE's
existing AutoEscalationTracker, which previously bottomed out at haiku (paid).
- escalation/auto-escalation-tracker.ts: made generic over the tier-name type
with a configurable tierOrder (default resolved in ctor — DEFAULT_ESCALATION_
CONFIG unchanged, so the exact-match test stays green). 26/26 original tests
pass; zero behaviour change for existing callers.
- routing/free-tier/: configurable provider layer — local-ollama / cloud-ollama
/ openrouter / openai-compatible. Keys read from a named ENV VAR, never
stored. freeTierChat() is OpenAI-compatible + reasoning-model aware + never
throws (transport error = tier failure → escalate). defaultFreeTierLadder()
= local -> haiku -> sonnet -> opus.
- tests: 19 new (45/45 green with the tracker suite), strict-tsc clean.
- prototype/d7-proof.mjs: live proof vs M5 host Ollama — free local tier
answers ($0), escalates local->opus on failures, de-escalates back to local.
Dormant by design: not yet wired into a coordinator (next: D7-wire + D8 repair
loop + D9 feed outcomes into routing-feedback).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cross-pollination with Ruv's MetaHarness Darwin Mode: evolve the harness
around a frozen cheap local model, graded by AQE's objective QE metrics.
- docs/metaharness/06-...: GOAP action lane + ADR-style decision (→ ADR-111)
- A1: real mcp-scan snapshot of AQE's MCP surface (.harness/mcp-policy.json
mirror) → 2 HIGH (default-deny OFF, shell granted), 6 MED, 2 LOW
- D0: local-mutator spike benchmark on host M5 Ollama (qwen3:8b/30b > gemma4
as mutator; mock+LLM gives no fitness signal — documented)
- D1: src/integrations/darwin — QE→Darwin ScoreCard adapter folding ADR-104
arena metrics (kill-rate/coverage/cost) + ADR-072 promotion-gate mirror
(15/15 tests green, strict-tsc clean, zero coupling to @metaharness/darwin)
- D6: prototyped + proved a `customScore` hook in Darwin's evolve loop
(patch in prototype/, Ruv's tree left pristine). QE-native scoring now
DIFFERENTIATES variants: winner +0.1230 over baseline, 3 promotions
(vs flat 0.765 from the mock substrate)
Two upstream PRs to ruvnet/agent-harness-generator are staged but unsent:
the customScore hook + --ruvllm-timeout/max-tokens CLI flags.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
MCP server reliability + database-free completeness (community issues from
@nagoodman): in-process `agentic-qe mcp` (#528), memory_delete stale-cache fix
(#535), db-free propagation to all editor MCP configs (#533), and the opt-in
--no-claude exclusive install (#532). Plus the js-yaml dev bump (#540) and the
devil's-advocate follow-up fixes (routing/merge/startup + CI guard).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Records the 2026-06-17 evidence: ADR-038 (DRACO ceiling) is now Accepted
upstream → the A3 gate leans G-ABORT; MetaHarness v0.1.15 ships mcp-scan --json
+ scorecard, shrinking A1 to "consume, not build"; and two new standalone
candidate actions lifted from ADR-150 (A9 graceful-degradation, A10 KRR router).
Cross-links the ADR-150 spawnSync bug class to AQE #528.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Beta-test feedback for ruvnet/agent-harness-generator (MetaHarness):
- 00-04: executive summary, quality, product, technical, and six-hats
cross-pollination analyses.
- qcsd-development/: full QCSD Development Swarm pass (6 specialized QE
agents) — TDD, complexity, coverage, security, mutation, defect prediction,
plus executive summary and learning-persistence record. Gate: HOLD (2 HIGH
security findings in the provenance/secret-handling code).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Follow-up to d95f0d7b / 2f4d119c — the earlier passes updated .claude/skills
and assets/skills but missed several other eval surfaces that still pinned
retired IDs (claude-3.5-sonnet, claude-3-haiku):
- `.github/workflows/test-qe-browser.yml` — the `aqe eval run --model` flag
(inert in command-eval mode, but a real dead-ID leftover) -> claude-sonnet-4-6.
- `plugins/agentic-qe-fleet/skills/` — the tracked plugin-marketplace skill copy
(8 eval suites + a sample-output fixture); chaos skills get the opus-4-8 ceiling.
- `docs/schemas/skill-eval.schema.json` — models `enum` was missing claude-opus-4-8
and still allowed retired/deprecated IDs; trimmed to current+active, default
flipped to claude-sonnet-4-6. Plus description examples in the sibling schemas
and the modelUsed values in docs/examples/*.json fixtures.
- `docs/templates/skill-frontmatter.example.yaml` validation_models list.
- `src/cli/commands/eval.ts` — the `aqe eval` `--model` help example and the
`run-all --models` default (`claude-3.5-sonnet` -> `claude-sonnet-4-6`).
Verified: typecheck clean; all edited JSON/YAML parse; zero retired IDs remain in
any eval suite, schema, template, workflow, or eval-CLI default. Intentionally
left untouched: the consensus-provider subsystem (pricing tables / type-unions /
defaults — separate routing layer, bucket-2 data) and historical release notes /
QE audit reports (which document the old IDs by design).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cross-model consistency matrix (models_to_test) in 54 skill evals + both
eval templates pinned retired/malformed model IDs — `claude-3.5-sonnet`
(malformed: dashes, not dots; resolves to Sonnet 3.5, retired Oct 2025) and
`claude-3-haiku` (retired Apr 19 2026). Both 404, so those evals could not run
a meaningful cross-model check.
Refresh to current pinned IDs, preserving the original "primary + fast floor"
design:
- claude-sonnet-4-6 # primary (high accuracy)
- claude-haiku-4-5 # fast floor (minimum quality)
High-stakes skills additionally get the capability ceiling claude-opus-4-8
(security-testing, security-visual-testing, n8n-security-testing,
pentest-validation, chaos-engineering-resilience, qe-chaos-resilience,
qe-defect-intelligence) and the security eval template. Opus is scoped to these
7 + the security template rather than all 54 to avoid a fleet-wide eval-cost
blowup.
Also fixes the two `--model claude-3.5-sonnet/claude-3-haiku` examples in
run-skill-eval.ts and a nested cross-model test-case input in
qe-learning-optimization.yaml.
Verified: all eval YAML parses; zero retired/deprecated Claude IDs remain across
skills/evals + templates + runner. (src/domains/constants.ts MODEL_TIERS was
already current — tier 1 claude-haiku-4-5 — so it is intentionally unchanged;
the legacy claude-3-haiku-20240307 entries in cost-tracker/bedrock are retained
historical pricing/availability data.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Statuses updated to reflect reality (were stale 'Proposed' despite
implementation): 107/108/110 Implemented; 105 Implemented w/ INFERRED
domain-finding follow-up; 106 Implemented (live matrix, native tiers
pass every cell); 109 Implemented infrastructure w/ corpus-scaling
follow-up. Gitignore transient run outputs, force-keeping canonical
evidence files.
Verified: full branch tsc clean; both new workflows valid YAML; 58
locally-runnable tests green; no .env or key material in any branch
commit (audited).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
RUBRIC-v1.md committed first per ADR-108: ground-truth primary outcome
(hidden acceptance test), style-neutral verbatim judge instruction (no
treatment-revealing framing in either direction), two-pass cross-family
judging with INFERRED-labeled fallback, clustered-stats reporting rule.
Provably rubric-before-data: this commit precedes any results-* artifact.
2 scenarios with seeded defects + personas + hidden tests (off-by-one
pagination; swallowed async rejection). Hidden tests verified both
directions: FAIL on seeded bug with precise reasons, PASS on the fix.
lib/stats.ts encodes the three Pattern Space harness fixes as code:
isGoodRow enforced inside aggregate() (no path to a number that skips
it), seeded per-item counterbalance (their null cell was 72% position-
skewed), exact scenario-clustered sign test (validated against the
known 27/11 → p≈0.0136 re-analysis).
Verified: 14/14 stats tests; hidden-test ground truth both directions;
tsc clean. Remaining: live conversation driver + judge integration
(API-gated, same decision as ADR-106 live mode).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds EvidenceClass (EXECUTED|STATIC|INFERRED|CONJECTURE) and
EvidenceArtifact to the Finding schema, required at the type level.
requirements.ts factory defaults to STATIC (all 26 steps are
deterministic content analysis); pipeline crash-findings are EXECUTED
with the error output as artifact. Field is evidenceClass, not
evidence — StepResult.evidence: string[] already exists (ADR amended).
Verified: tsc --noEmit clean; validation-pipeline-handler tests 7/7
(MCP handler passes findings through, new field flows to MCP output).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lineage registry with rubric-before-data, changelog, kept-data,
clustered-statistics, and style-neutral-judge rules. Seeded with
arena Phase 1 (incl. its retracted measured-duration variant as the
model entry) and the planned ADR-109 row. Pre-registry numbers are
uncited-by-default pending backfill.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Six proposed ADRs from the universal-pattern-space QE assessment:
evidence-class labels on findings, behavioral safety evals per model
tier, shipped-agent invariant verification, benchmark lineage with
pre-registered rubrics (incl. antipattern list), interaction benchmark
for qualitative agents, kept-nulls negative pattern records.
Cross-checked against existing ADR corpus: Related fields reference
ADR-058/061/070/074/083/086/098/099/102/104; mid-task safety injection,
restraint probes, and null-vs-quarantine sequencing folded in from the
cross-check findings.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
aqe arena run: seeded tournaments where test strategies compete on REAL
fitness — built-in first-order operator mutants (string/comment/arrow-safe
enumeration) executed per strategy via node --test, real line coverage
from --experimental-test-coverage, and a deterministic suite-cost term.
Tournament emits a Wolfram-style pairwise competitive array; --evolve
hill-climbs the winner by seeded group flips with real re-evaluation.
Runs persist fail-soft to memory.db kv_store (namespace 'arena');
aqe arena list reads them back. Self-contained: no core changes, no new
dependencies; fixtures/arena-demo ships 4 test groups of varying kill
power over a branch-dense pricing module.
Reproducibility contract: same seed + fixture => byte-identical envelope
(verified E2E by diffing two runs), with informational.runtimesMs the
only excluded field. Honest finding during verification: the original
measured-runtime rank penalty was jitter-dominated at fixture scale and
flipped rankings between identical-seed runs — replaced with the
deterministic suite-size ratio (documented in ADR-104).
Phase 2 (documented): LLM-generated strategies frozen as artifacts,
winners distilled into qe-test-architect priors; MCP arena_run
registration (CLI and MCP share runArena() — parity by construction).
10 tests green (incl. real-engine reproducibility); tsc + eslint clean.
Part of #520 (improvement 7/7 — initiative complete).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
qcsd-development-swarm now runs as a deterministic workflow script
(.claude/workflows/qcsd-development-review.js) instead of prompt-driven
Task-tool fan-out:
- pipeline: one finder per quality dimension (TDD adherence, complexity,
coverage gaps; qe agents as agentType) -> 3 blind adversarial refuters
per finding -> deterministic synthesis, no agent editorializes the report
- Loki-mode constraints (ADR-074) encoded in refuter prompts: blind review
(bare claim + evidence only), anti-sycophancy (attack the claim;
uncertainty defaults to refuted), 3 distinct lenses
- majority kill (>= ceil(N/2) refuted votes); killed findings retained
with refutations for audit; output is finding-verdict@1 envelopes
(ADR-103)
- SKILL.md v1.1.0: execution.primary workflow, Task-tool fallback
documented (reports marked verification: none)
Validation run (wf_254f1271-022): 3 dimensions over src/, 48 agents,
15 raw findings -> 14 confirmed, 1 killed by unanimous refutation of a
factually-wrong claim — the adversarial gate demonstrably filtering a
false positive. Several confirmed findings target code added earlier in
this initiative (honest signal, queued as follow-ups).
Part of #520 (improvement 5/7, Tier 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Versioned, additive-only verdict envelopes for agent/MCP handoffs:
RiskDecision (quality gates), FindingVerdict (review findings with
adversarial refutations), CoverageGap (risk-weighted gaps).
- src/contracts/verdicts.ts: dependency-free validators (source of
truth) + mirrored draft-07 JSON Schemas; no zod/ajv runtime deps
(light-install posture preserved)
- quality_assess MCP boundary now attaches a validated riskDecision
envelope derived from the gate outcome (approve/block, escalate when
indeterminate) — additive, existing payload unchanged; direct and
wrapped handlers share the same mapToResult so MCP parity is inherent
- schemas/*.schema.json published via scripts/generate-verdict-schemas.mjs
for external ajv validation and workflow agent({schema}) enforcement
(consumed by improvement 5)
17 contract tests (goldens validate; mutations rejected with field-level
errors; builder always emits valid envelopes). E2E: envelope emitted by
the compiled boundary builder validates with stock ajv against the
published schema; mutated sample rejected. tsc + eslint clean.
Part of #520 (improvement 6/7, Tier 3).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Anthropic announced nested subagent support (depth=5) on 2026-06-09; the
runtime currently strips Task at parent->child spawn (upstream denylist,
empirically confirmed by our committed probe: level=1 status=NO_AGENT_TOOL,
matching ruflo ADR-147). This lands the readiness infrastructure that is
independently useful today and activates with zero changes when the gate
lifts:
- aqe hooks post-task gains --parent-agent-id / --depth (validated:
identifier charset, integer 0 <= d <= 32; depth=0 survives as 0);
provenance persisted to qe_trajectories.metadata_json (stage 1 — no
schema migration) and forwarded into ReasoningBank feedback so learning
can segment patterns per hierarchy level
- TaskExecution gains optional parentAgentId/depth (additive types)
- scripts/probe-nested-spawn-depth.mjs: single-shot empirical probe,
committed transcript under docs/probes/ — re-run after every
`claude update`; TASK_TOOL_LIVE is the signal for follow-up phases
- fresh-project fix found by the E2E: persistTaskOutcome assumed
captured_experiences/experience_applications exist, but they are
middleware-created — Stream B silently no-oped on fresh installs;
canonical DDL now ensured in the hook path
Deliberately deferred (documented in ADR-101 P1): tools: frontmatter
retrofit on shipped qe-*.md agents — Claude Code frontmatter is a
restrictive allowlist and would sever inherited MCP tool access;
needs maintainer sign-off + wildcard verification first.
8 unit tests green; E2E on a fresh temp project verified persistence,
rejections, and depth=0 boundary. tsc + eslint clean.
Part of #520 (improvement 4/7, Tier 2).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Breaks runaway retry loops: when the same Bash command fails consecutively
(normalized signature, cross-process via a small JSON sidecar under
.agentic-qe/), pre-command warns at 3 failures and emits a BLOCK
recommendation with a recovery hint at 5. Advisory by default;
AQE_STRICT_TOOL_LOOP=1 upgrades the block to a PreToolUse deny. A
successful run resets the breaker; an open breaker grants a half-open
probe after 2 minutes; stale entries pruned after 30 minutes.
Mirrors the ADR-064 domain-circuit-breaker semantics at command level.
Sidecar instead of memory.db: pre-command deliberately runs without a DB
open, and each hook call is a fresh process so in-process state is
impossible. Fail-open everywhere (corrupt state file => allow).
11 unit tests green; E2E verified with real aqe hooks pre/post-command:
warn after 3, BLOCK after 5, strict-mode deny envelope, reset on success.
Part of #520 (improvement 3/7, Tier 1 complete).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The enableCache flag existed since ADR-011 but was never read: every
advisor/LLM call paid full input price. Now:
- ClaudeProvider.generate() sends `system` as a content-block array with
cache_control {type: ephemeral} when enableCache (default on); plain
string when disabled. Anthropic-native provider only (ADR-092 — other
providers untouched).
- cache_creation_input_tokens / cache_read_input_tokens surfaced into
TokenUsage.cacheCreationTokens/cacheReadTokens; totalTokens now counts
the full prompt (input_tokens alone is the uncached remainder).
- CostTracker bills cache reads at 0.1x and writes at 1.25x input rate
instead of ignoring them.
5 new provider tests assert the serialized request shape (cache_control
present when enabled, absent when disabled, no system field when no
prompt) and the usage mapping. 64/64 LLM tests green, tsc clean.
Part of #520 (improvement 2/7, Tier 1).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Dragan Spiridonov