mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-09-14 14:01:28 +08:00
main
135 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 rUv
|
0b3cfb77d6 |
MetaHarness hardening: repair the dependency contract + strict sequential promotion evidence (#2956)
* feat(metaharness): repair dependency contract + strict sequential promotion evidence Item 1 — dependency & compatibility repair (the contract was silently broken): - @metaharness/darwin ^0.8.3, @metaharness/flywheel ^0.1.10, and @metaharness/radio ^0.1.0 are now explicit optionalDependencies of @claude-flow/cli (optional PEER deps are never auto-installed, so a clean ruflo install shipped with zero MetaHarness packages on disk) - check-metaharness-pins.mjs now searches dependencies, optionalDependencies, AND peerDependencies; UNDECLARED and PEER-ONLY (for installable pins) are fatal drift instead of silently passing; radio added to the watch list; --require-installed asserts the real published symbol contracts (RefineMutator, withSequentialEvidence, RadioBus, ...) - new scripts/metaharness-clean-install-test.mjs: installs the declared ranges into a pristine temp dir and asserts every advertised export; wired as a MANDATORY clean-install job in metaharness-ci.yml - doctor: new "MetaHarness declared packages" check FAILS (not warns) when a declared optional dep does not resolve at runtime; --component metaharness now runs upstream + declared-deps + integration checks - MH_DARWIN_PIN bumped 0.8.0 → 0.8.3 in lock-step with the declared range Item 2 — strict promotion evidence at the transaction authority: - receipts now carry task-level pairedOutcomes (taskId + per-task baseline/ candidate scores) behind heldOutDeltas; verifyFlywheelReceipt refuses rows that cannot reproduce their aggregate; evaluateFlywheelCandidate populates them; pre-existing receipts still verify byte-identically - new flywheel-sequential-evidence.ts: anytime-valid e-process over discordant paired outcomes (testing-by-betting) composed with per-candidate alpha allocation (alpha_k = alpha_total * 6/(pi^2 k^2)), so the family-wise false-promotion probability across an ADAPTIVE candidate stream is bounded by alpha_total = 5% - promoteFlywheelCandidate requires paired evidence by DEFAULT — aggregate- only receipts are refused, never silently downgraded (the upstream withSequentialEvidence fallback hole); explicit --allow-aggregate-evidence / allowAggregateEvidence escape hatch for pre-upgrade receipts; alpha spend is persisted per receiptId in the transaction state (looking spends alpha; retries reuse their index) - acceptance test: 1,000 null-improvement streams (20 adaptive candidates x 40 worst-case discordant pairs) — measured family-wise false promotion 0.6%, within the <=5% budget Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01FKWLVWewYaWmcUg2q8Y9wy * chore: gitignore node-compile-cache build artifact Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01FKWLVWewYaWmcUg2q8Y9wy * fix(ci): satisfy #2561 guard, dep-overlap audit, and smoke contract for metaharness optional deps Three CI failures from the dependency-contract repair, three fixes: - tilde-pin @metaharness/{darwin,flywheel,radio} (~0.8.3 / ~0.1.10 / ~0.1.0) per the ADR-150 anti-caret rule enforced by smoke step 17z73 — upstream is unstable, tilde absorbs patches but never minors - remove the three from peerDependencies/peerDependenciesMeta: an entry in BOTH optionalDependencies and peerDependencies crashes npm 11.x arborist on dedupe (#1147/#2018 dep-overlap audit); optionalDependencies alone is the declaration that actually installs - update the #2561 cold-startup guard per its own escape clause: budget 8 → 10 and drop @metaharness/darwin from the forbidden list — that entry dated from the pre-0.8 heavy-tree era; darwin@0.8.3 is 1.8M unpacked with ZERO dependencies and no lifecycle scripts (all three packages combined: 2.3M, 753ms cold install into an empty dir, measured 2026-08-10; the clean-install CI job re-verifies on every pin change) - smoke.sh 17h now accepts the componentMap ARRAY form ('metaharness': [checkMetaharness, ...]) introduced with the declared-packages doctor check Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01FKWLVWewYaWmcUg2q8Y9wy * fix(ci): remove apostrophe that broke the single-quoted guard script The #2561 guard runs as `node -e '...'` inside bash; an apostrophe in a comment ("guard's") terminated the quoted string and bash tried to execute the next // comment line (exit 126, '//: Is a directory'). Verified by executing the extracted run block end-to-end: all three checks OK, exit 0. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01FKWLVWewYaWmcUg2q8Y9wy * feat(flywheel): ADR-381 — alpha-stream governance, reset epochs, accept/v2+seq Completes the sequential-evidence governance PR #2956 opened: ADR-381 (new) records the decisions: - the ADR-322 promotion ledger IS the alpha stream (per project root) — receipt lineageIds default to fresh UUIDs and cannot scope the control - evidence epochs: resetSequentialEvidence({confirm, reason}) archives the spend into an append-only sequentialResets audit trail, EXPIRES every outstanding evaluated receipt (fresh-data enforcement — old-epoch evidence cannot be replayed against a reopened budget), and increments evidenceEpoch; surfaced as `flywheel evidence-reset --reason … --confirm` (CLI) and the metaharness_flywheel MCP op, both behind the same policy gate as promotion - accept/v2+seq: the ADR-176 generations loop now decides each bundle with a third conjunct — the e-process over the bundle's own embedded per-task holdout at alpha_k for its position in the attempts stream — with the full verdict recorded in the bundle and independently replayed by verifyReceiptBundle (v1 bundles keep v1 semantics; versions pin per bundle); flywheelStatus surfaces next test index / threshold / minimum pairs / remaining budget so exhaustion reads as plateau, not mystery - two-layer pre-flight: evaluateFlywheelCandidate annotates (never blocks) when the promotion holdout cannot clear the next threshold on a perfect sweep; promoteFlywheelCandidate refuses size-inviable receipts BEFORE allocating an alpha index — sample size is ancillary, so the refusal looks at no evidence and spends no budget Tests: reset semantics (archive/expire/epoch/fresh-promote), alpha-free size refusal vs alpha-spending e-process refusal, v2 promote/reject/replay including tamper + index-shopping detection, v1 compat, helper math anchors (min pairs = 9 at test 1), budget monotonicity. 82 tests green across the eight affected suites; tsc build clean. Co-Authored-By: RuFlo <ruv@ruv.net> Claude-Session: https://claude.ai/code/session_01FKWLVWewYaWmcUg2q8Y9wy * fix(flywheel): close 3 concurrency/epoch gaps in ADR-381 sequential evidence Code review of this PR found three real correctness bugs that can silently violate the <=5% family-wise false-promotion guarantee that is the PR's own core deliverable: - flywheel-transaction.ts: resetSequentialEvidence's "fresh data only" guarantee only expired receipts already registered at reset time; a receipt whose evidence predates a reset but is registered afterward was silently admitted into the new, cheaper epoch (index shopping). Now tracks evidenceEpochStartedAt and promoteFlywheelCandidate refuses any receipt whose payload.issuedAt predates it, before allocating an alpha index. - harness-flywheel-generations.ts: the daemon generations loop computed its sequential-evidence testIndex via an unlocked loadAttempts(root).length+1 read before appending. Two overlapping runFlywheelGeneration calls on the same root could be assigned the same test index and spend the same alpha_k twice. The read-index -> build-bundle -> append critical section now runs under the same O_EXCL lock pattern flywheel-transaction.ts already uses. - harness-flywheel.ts: evaluateFlywheelCandidate's sequentialPreflight (backing the `promotable` flag) was computed from an unlocked snapshot taken before the async retrieval/scoring work, so it could go stale by the time promoteFlywheelCandidate allocates the real index under its own lock. Moved to the latest possible read (after receipt registration) to minimize the window; documented as advisory, since promoteFlywheelCandidate remains the sole authority. Added a regression test for the epoch-boundary refusal and a concurrency test proving two racing generations get distinct sequential test indices. Co-Authored-By: RuFlo <ruv@ruv.net> --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
rUv
|
401e02d511 |
fix: complete reports and consistent initialization for v3.32.37 (#2851)
* fix(metaharness): preserve readiness verdict payloads * test(metaharness): cover blocked genome verdicts * fix(adr): parse bullet metadata and relationships (#2659) * fix(adr): align adr-create with AgentDB schema (#2651) * fix(adr): make index updates idempotent (#2660) * fix(memory): bound session-end graph consolidation (#2628) * fix(memory): align active row visibility (#2652) * fix(memory): honor database path during init * fix(hooks): keep all shim fallback tags aligned * fix(codex): omit unbacked full-template skills * fix(init): generate complete native dual projects * test(memory): isolate path and legacy-row regressions * chore(release): prepare v3.32.37 |
||
|
rUv
|
67ff9898c0 |
fix: resolve current runtime and verification defects for v3.32.36 (#2850)
* fix(cli): make runtime status and learning signals truthful * fix(metaharness): accept padded scan severities * fix(codex): make core hooks and status skill native * fix(security): make witness verification hermetic * fix(metrics): expose grounded learning outcomes * fix(hooks): deduplicate project and plugin events * fix(verification): preserve security audit payloads * fix(runtime): stabilize dual memory skills and MCP schemas * fix(cli): harden helper signing and reflexion health * test(release): align helper and container invariants * chore(release): prepare v3.32.36 * test(funnel): align gates with cold-start seed pool * fix(ci): document MCP environment precedence |
||
|
rUv
|
495a451cf6 |
feat: add ADR-324 agentic policy engine (#2822)
* feat(memory): typed provenance in AgentDB (ADR-323) Corrects and implements the 2026-07-28 dream-cycle research proposal (PR #2804, closed as superseded) for typed memory provenance in AgentDB, addressing MemIR/MemSyco-Bench's "provenance-role collapse" and memory-induced sycophancy findings. The dream-cycle PR's proposed schema change targeted the wrong table (`vector_indexes`, which is per-namespace HNSW index metadata, not per-entry) and its filename collided with the already-merged `ADR-322-metaharness-flywheel-integration.md` (#2817) — both corrected here as ADR-323 against `memory_entries`, the real per-record table. - memory-initializer.ts: `provenance_type` column (user_claim | agent_output | system_observation | tool_result | unknown, default unknown) on `memory_entries`, migrated for existing DBs via `ensureSchemaColumns()`. `storeEntry()`/`searchEntries()` accept and validate `provenanceType`/`provenanceFilter`. - memory-bridge.ts: same wiring for the native better-sqlite3 AgentDB v3 path (`bridgeStoreEntry`/`bridgeSearchEntries`). - commands/memory.ts: `memory store --provenance <type>`, `memory search --provenance-filter <types>` (comma-separated, OR semantics). - mcp-tools/memory-tools.ts + cli-core/mcp-tools/memory-defs.ts: `memory_store`'s `provenance_type` param, `memory_search`'s `provenance_filter` param (kept in sync across both tool-def sets). Found and fixed during implementation: an initial design skipped the RaBitQ/HNSW acceleration paths entirely whenever a provenance filter was requested (neither carries provenance_type on their candidates). This reproducibly crashed the CLI process on exit (a libuv assertion on Windows) after printing correct results. Fixed by never skipping those paths — RaBitQ now fetches provenance_type in its existing per-candidate query, HNSW does one batched (namespace, key) lookup afterward — eliminating the crash across repeated runs. See ADR-323's "Implementation note" for detail. 7 new end-to-end tests (adr-323-memory-provenance.test.ts), including a repeated-run regression guard for the crash. Full existing memory suite re-verified with no regressions (one pre-existing, environment- specific #2558 test failure confirmed identical against unmodified main — better-sqlite3's native bridge is unavailable in this dev environment, unrelated to this change). Closes the follow-up from dream-cycle PR #2804 (closed as superseded). Co-Authored-By: claude-flow <ruv@ruv.net> * fix(memory): enforce provenance across retrieval paths * feat: add agentic policy engine * fix: align Codex MCP integration * fix: keep MetaHarness dependencies optional * fix: declare MetaHarness as optional peers * fix: sync workspace lockfile * fix: document policy tool selection guidance * test: cover governed in-process flywheel * ci: make MetaHarness absence drill deterministic |
||
|
ruvnet
|
1fb874005c | test(plugins): preserve standalone MCP catalog assertions | ||
|
ruvnet
|
e332689b8c | fix(release): harden hooks, statusline, security, and plugin MCP integration | ||
|
rUv
|
1d5b06e4a2 |
fix(federation): unblock main — TS shim types + dispatcher import + witness re-sign (#2604)
* fix(federation): sync shim types + guard optional close() to unblock Build V3
Fixes 7 TS errors on main that block Build V3 across all 3 platforms:
- src/transport/midstream-aware-loader.ts:39-52 — local AgentMessage/
AgentTransport shim was over-minimized by #2578. Restored required
{id, type, payload, metadata?} on AgentMessage and made send/onMessage
2-arg (address, message) to match how plugin.ts actually calls them.
- src/transport/midstream-aware-loader.ts:84 — cast dynamic import()
through `unknown` since upstream's exported AgentTransport carries a
richer InboundMessageHandler surface we intentionally hide behind the
minimal shim.
- src/plugin.ts:481 — this.transport.close is optional (`?`) on the
shim; guard with optional chaining `close?.()`.
Local tsc build clean after fix.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(federation): dispatcher import + re-sign witness manifests (3.25.3 drift)
Two more preexisting main-red root causes, in the same fix:
(1) inbound-dispatcher.ts:22 still imported `AgentMessage` from
`agentic-flow/transport/loader` — the exact phantom subpath #2578
eliminated in midstream-aware-loader.ts. Repointed to the local
shim. This is the residual site #2578 missed; the graph schema smoke
(ADR-130 P1) failure was a cascade of this same TS type collision.
(2) Re-signed verification/{linux,macos,windows}/manifest.md.json via
plugins/ruflo-core/scripts/witness/regen.mjs. The 3.25.3 release
(PR #2602) edited the ADR-104-transport `desc` string in all 3 per-OS
manifests but did not re-run regen.mjs, so the stored
integrity.manifestHash no longer matched sha256(JSON.stringify(
manifest)) and Witness verify exited 1 across all 3 platforms.
Manifests now verify cleanly (117 pass, 0 drift, 0 regressed).
Note: witness/regen.mjs takes --manifest + --history and re-signs
deterministically from manifest.gitCommit; already wired but the
3.25.3 release process skipped it.
Local verify: pass=117 drift=0 regressed=0 missing=0 on all 3 OSes.
Local tsc build: clean.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(hooks): PreToolUse hook emits valid JSON verdict (#2613)
Cursor imports third-party Claude Code hooks under its stricter
`preToolUse` contract, which requires stdout to be a valid JSON
permission verdict (`{"permission":"allow"|"deny"|"ask"}`) and
fail-closes on any other text. The prior PreToolUse commands ran the
ruflo hook shim which prints telemetry to stdout — Cursor then blocked
every Bash/Write/Edit tool call across every workspace.
Fix:
1. ruflo-hook.sh (all three copies) — redirect stdout to /dev/null in
addition to stderr. Claude Code doesn't consume this stdout either,
so silencing it is a pure cleanup with no functional cost.
2. hooks.json PreToolUse commands — after invoking the shim, always
`printf` a well-formed `{"permission":"allow"}` verdict. Valid for
both Claude Code's JSON-hook contract and Cursor's stricter
preToolUse contract, so telemetry keeps working while nothing is
blocked. Also guard `${CLAUDE_PLUGIN_ROOT:-}` so an unset env var
collapses to the safe empty branch instead of a bash parse error.
Simulated Cursor invocation (CLAUDE_PLUGIN_ROOT unset) now emits:
{"permission":"allow"}
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(ci): indirect optional-dep imports in cli — @ruvector/*, @metaharness/router (#2608)
Same install-safety anti-pattern as #2586's prime-radiant fix, now
surfaced in @claude-flow/cli once plugin-agent-federation's TS2307
stopped masking it (
|
||
|
rUv
|
d012042800 |
feat: review-driven upgrades — perf, security, SOTA capabilities (6-agent concurrent implementation) (#2547)
* chore(arch): untrack the prepublish plugin mirror — build artifact, not source
v3/@claude-flow/cli/plugins/ is regenerated by prepublishOnly from
plugins/ruflo-metaharness/ at publish time. Keeping it committed caused
silent drift between publishes (4 files diverged this week alone) and
every source edit required a manual mirror sync. npm pack still bundles
it — the files array + prepublish copy are unaffected by gitignore.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(gaia): submission integrity checklist — fail-closed pre-submit audit (Berkeley RDI hardening)
Adds a 4-part integrity section to the gaia-validate pre-submit gate,
motivated by UC Berkeley RDI's 'Agents' Last Exam' (April 2026), which
gamed all 8 major agent benchmarks to near-100% without solving tasks.
- scripts/gaia-integrity.mjs: (A) answer-key-shaped path reads outside
the sanctioned dataset dir — FAIL-CLOSED exit 2; (B) eval()/new
Function()/exec-of-non-literal strings in gaia-bench runner code
paths — FAIL-CLOSED exit 2; (C) judge-prompt-injection markers in
produced answers/trajectories — WARN; (D) provenance stamp (results
+ git SHA + dataset content hash) written for gaia-submit to embed.
- --allow-integrity-override suppresses the exit code but records
'overridden: true' in the stamp — never silent.
- --self-test plants a fake answer-key read, an eval-of-task runner,
and injection markers in temp fixtures and proves fail-closed/pass/warn.
- gaia-validate.md: new check 7 documenting the checklist + flags.
- gaia-submit.md: package now embeds integrity.json under the Ed25519
witness manifest (tamper-evident attestation); stale/failed stamps block
packaging.
- smoke-gaia.sh: 3 new steps (script wired, self-test green, repo scan
clean); plugin bumped to 0.5.0.
Co-Authored-By: RuFlo <ruv@ruv.net>
* perf(cli): lazy ONNX embedder init + local-first ruvector resolution
Fix 1 — lazy ONNX init (measured, biggest win):
neural-tools.ts had a module-level try/catch with TOP-LEVEL AWAIT that ran
`await import('ruvector')` + `initOnnxEmbedder()` + a probe embed at import
time. Because mcp-client statically imports neural-tools, this added the
full ONNX startup to every CLI command. The block is now a memoized
`ensureEmbeddings()` (module-level promise guard) invoked from
generateEmbedding() and from the three handlers that report the provider
without embedding (neural_compress / neural_status / neural_optimize), so
degraded/fallback semantics and provider reporting are byte-identical.
The probe embed moved inside ensureEmbeddings() — it still guards the
ADR-086 type-load-success-but-runtime-fails trap, but only on first use.
Measured (warm, darwin-arm64):
node bin/cli.js --help 0.50s -> 0.09s
node bin/cli.js status 0.64s -> 0.24s
import(mcp-client.js) 438ms -> 30ms
"Loading ONNX model" no longer appears on non-embedding commands;
appears exactly once on first embed (neural_predict / memory search).
Fix 2 — ruvector pin + local-first resolution:
browser-session-tools.ts pinned `npx -y ruvector@0.2.25` while 0.2.27 is
what the package installs, forcing cold npx downloads of a second version
(4 per browser session). The pin is now 0.2.27 and a memoized
resolveLocalRuvectorCli() (createRequire + package.json bin walk) spawns
`node <local-cli>` directly, falling back to the npx pin only when no
local install exists.
While verifying the 0.2.27 subcommand surface (required before bumping),
found the trajectory-* flags passed here were never valid on 0.2.25 OR
0.2.27 (`--session-id/--task/--args/--verdict` vs the real
`--context/--agent`, `--action/--result`, `--success/--quality`) — every
trajectory call exited with commander "unknown option", same bug class as
the #2015 `rvf create --kind` fix documented in this file. Flags are now
mapped correctly; browser_session_record and browser_session_end verified
end-to-end (rvf create + trajectory-begin/step/end + rvf compact all pass).
Fix 3 — lazy heavy MCP tool groups: profiled and NOT converted.
Per-module timed dynamic imports of all 36 tool modules show neural-tools
was 469.6ms and every other group is 0.3-5.5ms (hooks-tools 3.5ms,
agentdb-tools 2.5ms, embeddings 0.9ms, wasm-agent 0.6ms). After fix 1 the
whole registry builds in ~30ms; converting handler dispatch to lazy
import() would add risk for <6ms of residual win, so mcp-client.ts is
intentionally untouched.
Validation:
- tsc build clean
- plugins/ruflo-metaharness/scripts/test-mcp-tools.mjs: 171/171 passed
- memory search / neural predict still work; ONNX loads lazily then
Co-Authored-By: RuFlo <ruv@ruv.net>
* security(deps): lift OTel floor-became-ceiling pins — core >=2.8.0, otlp family >=0.220.0
The exact pins @opentelemetry/{core,resources,sdk-trace-base}@1.25.1 and
the otlp exporter/transformer family@0.52.1 (root + ruflo wrapper
overrides) were added in #1997/#2112 as a 'fewer-knobs-for-arborist'
defense against the npm arborist Invalid Version crash (empty-version
placeholder from optionalDependencies/peerDependencies overlap). They
were never API-compat pins — but they became a ceiling that forces the
tree onto @opentelemetry/core <2.8.0, which npm audit flags for
GHSA-8988-4f7v-96qf (unbounded memory allocation in W3C Baggage
parsing), and they blocked every upstream fix.
Convert to floors while keeping the lockstep families coherent:
- stable line: core / resources / sdk-trace-base -> >=2.8.0
- otlp line: sdk-node, exporter-prometheus, the three
exporter-trace-otlp-* packages, otlp-exporter-base,
otlp-grpc-exporter-base, otlp-transformer -> >=0.220.0
(sdk-node@0.220.0 declares exactly core@2.9.0 + otlp@0.220.0, so the
floors resolve to one mutually consistent set; verified in both the
root lockfile and a fresh ruflo/ resolution: core 2.9.0 everywhere,
otlp 0.220.0 everywhere.)
The arborist defense is preserved — every package in both families is
still forced to a single resolution via overrides; the override set is
now broader (whole otlp family at root too), which gives arborist fewer
ambiguous nodes than before, not more.
Also adds the exporter family overrides to the root manifest so root
and the published ruflo wrapper stay aligned (#2112 lesson: root
overrides do not propagate to the published wrapper).
Validation:
- root: npm audit --omit=dev 30 moderate -> 0 vulnerabilities
- ruflo: npm install --package-lock-only succeeds, 0 vulnerabilities
- runtime probe: new NodeSDK({}) via @opentelemetry/sdk-node@0.220.0
with core@2.9.0 loads and exposes start(); OTLPTraceExporter loads
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): npm audit fix on v3 workspace lock — grpc-js, hono/node-server, brace-expansion
Plain `npm audit fix --omit=dev --package-lock-only` (no --force) on the
v3 npm workspace lockfile. Clears the three targeted advisories:
- @grpc/grpc-js 1.14.3 -> 1.14.4 GHSA-5375-pq7m-f5r2 (+GHSA-99f4-grh7-6pcq)
- @hono/node-server 1.19.9 -> 1.19.14 GHSA-wc8c-qw6v-h7f6 (+GHSA-92pp-h63x-v22m)
- @isaacs/brace-expansion 5.0.0 removed/deduped GHSA-7h2j-956f-4vf2
Audit summary: 82 vulnerabilities (2 low, 56 moderate, 24 high)
-> 55 vulnerabilities (50 moderate, 5 high).
The remainder needs --force / dependency swaps and is out of scope for
this pass.
Note: the reify was run with the two pnpm-style `workspace:*` devDep
specs (plugin-agent-federation, plugin-iot-cognitum) temporarily
substituted with the registry pins the lockfile already records
(@claude-flow/shared@3.0.0-alpha.8, @claude-flow/security@3.0.0-alpha.10),
because npm's arborist does not support the workspace: protocol; the
manifests themselves are unchanged. The v3 workspace is otherwise
pnpm-driven (`pnpm -r build`).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): align agentdb floor in @claude-flow/memory — ^3.0.0-alpha.17
Root overrides force agentdb >=3.0.0-alpha.17, but @claude-flow/memory
declared ^3.0.0-alpha.16, so its nested/published tree could still
resolve alpha.16 (the #2112 lesson: overrides do not propagate to
nested or published trees). The v3 pnpm lock indeed had memory pinned
to agentdb 3.0.0-alpha.16 — bumping the package's own floor makes both
branches resolve the same version regardless of the consumer's
override set.
pnpm-lock.yaml is regenerated by `pnpm install`; besides the
agentdb 3.0.0-alpha.16 -> 3.0.0-alpha.17 move under memory, it also
reconciles previously committed manifest bumps (metaharness family)
that had drifted from the stale lock.
Verified: `npm ls agentdb` from root shows a single deduped
agentdb@3.0.0-alpha.17 across direct, @claude-flow/memory,
@claude-flow/neural, and agentic-flow branches.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* security(deps): correct FEDERATION_BIND_HOST doc default — 127.0.0.1 (ADR-166 Phase 3d)
The bin.ts header JSDoc still advertised `default: 0.0.0.0`, but the
code (makeContext) has defaulted to 127.0.0.1 since the ADR-166
Phase 3d loopback-by-default hardening. Doc-only change.
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* fix(memory): drop vitest-4 style vi.fn generic that breaks tsc build
The hoisted vitest 1.x types Mock as Mock<Args extends any[], Returns>,
so the single-generic vi.fn<SearchFn> form fails the package build
(pre-existing baseline failure). Annotate the variable instead — the
inferred mock signature is identical.
Co-Authored-By: RuFlo <ruv@ruv.net>
* perf(memory): batch embeddings in the migration import path
MemoryMigrator.processBatch batched entries (batchSize=100) but then
awaited this.embeddingGenerator(content) ONE ENTRY AT A TIME inside the
loop — N sequential ONNX inferences per batch on the
memory_import_claude hot path.
Now:
- New BatchEmbeddingGenerator type ((contents: string[]) =>
Promise<Float32Array[]>) accepted as an optional 4th constructor /
factory param — one true batch call per migration batch when the
embedder supports it (e.g. bge-embedder's single padded forward pass).
- Fallback: bounded concurrency over the single-text generator
(MigrationConfig.embeddingConcurrency, default 8 — never unbounded;
set 1 to restore sequential behavior).
- If the batch call fails, falls back to per-entry embedding so one
backend hiccup doesn't strand the batch.
Per-entry error semantics preserved: validation failures skip, store
failures fail, embedding failures warn and store the entry without a
vector — one bad entry never fails the batch.
Measured (200 synthetic entries, 5ms simulated single-embed latency):
sequential 1220ms -> concurrency-8 156ms (7.8x) -> true batch 115ms
(10.6x).
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(memory): temporal validity on hierarchical memory (Zep/Graphiti-style)
Facts in hierarchical memory can now carry validity windows and are
INVALIDATED on conflict instead of overwritten:
- New TieredMemoryStore module (@claude-flow/memory) promotes the inline
controller-registry stub to a first-class, tested store with optional
validFrom / validUntil (ISO) + supersededBy fields.
- store accepts supersedes=<entryId|key>: the old entry is stamped
validUntil=now + supersededBy=<newId> and archived — never deleted.
- recall filters temporally-invalid entries (expired, superseded, or
not-yet-valid) by default; includeExpired=true is the audit escape
hatch.
- Backward compatible: entries without the fields behave exactly as
before (always valid); legacy store(key,value,tier) / recall(query,
topK) call shapes unchanged; same-key stores still overwrite; the
bridge's real-vs-stub detection signal (getStats+promote) is
preserved.
- CLI bridge passes the new optional fields through
bridgeHierarchicalStore/-Recall. On the native agentdb
HierarchicalMemory backend, validFrom/validUntil ride in metadata and
are filtered at recall; supersedes is reported as unsupported there
(no public update API) rather than silently dropped.
- MCP inputSchemas for agentdb_hierarchical-store (validFrom,
validUntil, supersedes) and agentdb_hierarchical-recall
(includeExpired) expose the fields to agents, with ISO-8601
validation.
Tests: supersede flow (including same-key supersede preserving
history), expired + not-yet-valid filtering, includeExpired escape
hatch, legacy entries untouched, unparseable timestamps fail open.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(routing): confidence-gated tier escalation — hooks_model-verify ($0 post-generation verifier)
ADR-026/143 classify BEFORE generation only. This adds the 2026-SOTA
post-generation gate: attempt the cheap tier, verify the output with
CHEAP structural signals (no LLM call), escalate only on failure.
- src/ruvector/output-verifier.ts — pure verifier: empty/truncated
output, refusal patterns, degenerate repetition, unbalanced
delimiters, and a real syntax parse for code/JSON tasks (TypeScript
transpileModule diagnostics / JSON.parse, graceful degradation when
typescript is absent). Returns {confident, reasons[], suggestedTier,
suggestedModel, escalate}; ladder haiku→sonnet→opus, tier 2→3.
- hooks_model-verify MCP tool (hooks-tools.ts, surgical) — wraps the
verifier and feeds the verdict into the SAME learning stream as
hooks_model-outcome (ModelRouter.recordOutcome: success when
confident, escalated when not) so the Thompson priors learn which
task shapes the cheap tier fails on. A dedicated tool (not a verify
mode on hooks_model-outcome) because verify is a mid-loop decision
point returning a verdict the agent acts on, while outcome is a
terminal write — the sequence is route → generate → verify →
(escalate) → outcome.
- __tests__/output-verifier.test.ts — refusal/truncation/parse/
escalation-ladder coverage + MCP handler tests (18 tests).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* feat(federation): A2A Agent Cards — spec-1.0 discovery adapter over the bespoke federation schema
Implements the Agent2Agent (A2A, Linux Foundation) Agent Card layer as an
ADAPTER over the existing federation registration/discovery shapes. Cards
only — A2A Tasks/messaging are deliberately out of scope this iteration.
Spec: A2A Protocol 1.0.0 (a2a-protocol.org; schema source of truth
specification/a2a.proto in a2aproject/A2A). Required AgentCard fields per
spec 4.4.1: name, description, supportedInterfaces, version, capabilities,
defaultInputModes, defaultOutputModes, skills. Well-known discovery path
per 8.2 + IANA registration: /.well-known/agent-card.json.
- src/a2a/agent-card.ts: toAgentCard() maps FederationManifest -> spec-
compliant card (agent types -> AgentSkills; federation endpoint -> open-
form RUFLO-FEDERATION AgentInterface; bespoke identity rides in a spec
4.6 extension urn:ruflo:federation:manifest:v1 for lossless round-trip);
validateAgentCard() enforces the 1.0 required-field shape;
fromAgentCard() maps a remote card into a FederationNode — ALWAYS
TrustLevel.UNTRUSTED (cards are self-asserted metadata; trust is earned
via the normal handshake + TrustEvaluator path).
- src/a2a/well-known.ts: minimal node:http endpoint serving GET/HEAD on
the exact well-known path with spec 8.6 Cache-Control + ETag/304.
Binds 127.0.0.1 by default and REFUSES non-loopback binds without an
explicit allowNonLoopback opt-in (ADR-166 posture preserved — no
loosening of federation bind or auth).
- src/a2a/consume.ts: fetchAgentCard (injectable fetch, timeout, size cap,
structural validation) + consumeAgentCard registering the peer into
DiscoveryService so A2A peers appear in federation discovery.
- DiscoveryService: new 'a2a-card' discovery mechanism +
registerExternalPeer() (refresh-not-replace preserves trust state).
- plugin.ts: opt-in wiring via config.a2aCard/a2aCardPort/a2aCardHost;
bind failure degrades gracefully; server closed on shutdown.
- 40 new unit tests incl. golden-file card fixture
(__tests__/fixtures/a2a-agent-card.golden.json), foreign-card mapping,
round-trip, loopback guard, and a live serve->consume e2e over HTTP.
- plugin.test.ts: fixed pre-existing stale assertion (10 -> 11 CLI
commands; 'federation trust elevate' was added without updating the
count) — baseline suite failed before this change.
Build clean; 600/600 tests green.
Co-Authored-By: RuFlo <ruv@ruv.net>
* feat(intelligence): execution-state-tree retrieval prototype (MAGE-style positional recall)
Research basis: MAGE (arXiv 2606.06090) — semantic-similarity retrieval
fragments decision trajectories on long-horizon tasks; retrieving by
POSITION in a hierarchical execution-state tree (root→current path)
preserves coherence.
- src/ruvector/trajectory-tree.ts — TrajectoryTree prototype:
trajectory-start opens a node under the session root (nesting under
the deepest open trajectory), steps append children, end closes and
pops the current position. recallPath({sessionId, depth}) returns
the exact root→current path + recent siblings — no embedding search.
Best-effort JSON snapshot at .claude-flow/intelligence/
trajectory-tree.json, ALONGSIDE existing trajectory storage (no
migration). Limitations documented in the module header.
- hooks-tools.ts (surgical): non-fatal tree mirrors in
hooks_intelligence_trajectory-start/step/end (+ optional sessionId
param on start), and an opt-in strategy:"state-tree" on
hooks_intelligence_pattern-search (+ sessionId/depth params).
Default strategy stays "semantic" — zero behavior change unless
opted in.
- __tests__/trajectory-tree.test.ts — synthetic 3-level trajectory
path assertions, nesting/pop, sibling window, persistence
round-trip, and default-strategy-untouched checks (12 tests).
Co-Authored-By: RuFlo <ruv@ruv.net>
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
* fix(metaharness): kill third-party @latest, extract shared _invoke.mjs, derive smoke counts
Three of the four converged security/perf/arch review fixes:
1. SECURITY (HIGH) + PERF — _harness.mjs no longer shells to
`npx -y metaharness@latest` (a compromised upstream publish would have
executed arbitrary code on user machines, and @latest forced a registry
check per call). Resolution is now: (a) locally installed metaharness
satisfying the PINNED range ~0.3.0 (walk-up node_modules), then (b) a
one-time `npm install --prefix ~/.ruflo/metaharness-cache-0.3.0`, then
plain `node <abs-path-to-cli>` spawns — zero network per call. Both bins
(metaharness + harness) are read from the resolved package.json bin map.
The {stdout, stderr, exitCode, json, degraded, reason, durationMs}
contract and ADR-150 graceful degradation are preserved; timeouts now
report 'metaharness-timeout' (matching _darwin/_redblue) instead of being
conflated with 'metaharness-not-available'.
Measured: `score.mjs --path .` completes in ~0.35s via the cache.
2. ARCH — new scripts/_invoke.mjs consolidates the copy-pasted plumbing:
DEGRADED_RX (superset incl. `npm ERR`), classifyDegraded (keeps the
-timeout vs -not-available distinction), injectJson, parseTrailingJson
(the LAST-{...}-block variant from _darwin — the old _harness first-block
parse was a live bug), ensureCachedInstall (generalizes gepa/_redblue,
cache-dir naming kept compatible so existing caches keep serving),
importOptionalLibrary, findLocalPackageDir, makeDegradedEmitter.
_harness/_darwin/_redblue/gepa are now thin adapters with IDENTICAL
exported signatures; their genuinely-different parts stay (redblue's
node-direct isMain workaround, darwin's async streaming for evolve,
harness's dual binaries). test-graceful-degradation.mjs gains
RUFLO_METAHARNESS_CACHE_BASE / RUFLO_METAHARNESS_SKIP_LOCAL seams so the
ADR-150 drill still exercises the degraded path on warm-cache machines.
3. ARCH — smoke.sh derives EXPECTED_TOOLS / EXPECTED_SUBS once from
metaharness-tools.ts / metaharness.ts and replaces the 7 independently
hardcoded literals (15 at :750/:829/:2046, 13 at :725/:851, 16 footnote,
15 success-pattern). Cross-surface value preserved: independent surfaces
(CLAUDE.md catalog, test-mcp-tools' own `tools.length === 15` literal,
footnote appends, runScript refs) are compared against the derived count,
never a surface against itself; a >=15/>=13 floor step catches
extraction-regex rot.
Validation: smoke.sh 122/122, test-graceful-degradation 16/16,
test-mcp-tools 171/171.
Co-Authored-By: RuFlo <ruv@ruv.net>
* fix(metaharness): add --diagnose GEPA failure diagnosis to evolve
GEPA's key trick — natural-language failure diagnosis from execution traces
feeding the next mutation, not just scalar fitness — scoped modestly into
the evolve wrapper as an opt-in flag.
`evolve.mjs --diagnose` now appends a `diagnosis` section to the emitted
JSON after the run completes: losing/failed variants' transcripts are run
through darwin's gepa `analyzeTranscript` + `classifyFailure` ops (via the
shared importGepa resolver, moved to _darwin.mjs because gepa.mjs runs
main() on import) and aggregated into failure classes + counts + dominant
class per variant.
Upstream shape findings (verified against @metaharness/darwin@0.8.0 with a
tiny mock-sandbox evolve):
- `metaharness-darwin evolve --json` emits a TEXT leaderboard — stdout
carries no JSON and no transcripts (r.json has always been null here).
- Per-variant run records live at <repo>/.metaharness/runs/<id>.json as
sandbox exec traces ({taskId, exitCode, stdout, stderr}) — NOT gepa
{actionRaw, obs} transcripts.
Diagnosis therefore layers: gepa-shaped transcripts when a run record
embeds them (agent sandbox / future upstream) → champion-transcript
fallback → `diagnosis: {available: false, reason, traceSummary}` with a
mechanical per-variant tally. Never fails the run — internal errors
degrade to {available: false, reason: 'diagnosis-failed: ...'}.
Both paths exercised end-to-end: mock evolve yields the traceSummary
fallback; a seeded gepa-shaped run record yields
{available: true, scope: 'losing-variants', failureClasses:
{exploration-loop: 1, edit-mechanics: 1}, dominantClass: 'exploration-loop'}.
Documented in skills/harness-evolve/SKILL.md.
Co-Authored-By: RuFlo <ruv@ruv.net>
* Revert "feat(gaia): submission integrity checklist — fail-closed pre-submit audit (Berkeley RDI hardening)"
This reverts commit
|
||
|
rUv
|
ae9bba523e |
feat(metaharness): learn + gepa integration — metaharness@0.3.0 / darwin@0.8.0 (15 MCP tools) (#2540)
* fix(mcp-bridge): streamable-HTTP session cleanup — DELETE + Mcp-Session-Id + 202 (#2425) Closes #2425. Cherry-picked from djimit's commit |
||
|
ruv
|
a63cdf0526 |
feat(metaharness): integrate @metaharness/redblue@~0.1.1 — adversarial red/blue LLM testing
ADR-150 sibling-package integration. `@metaharness/redblue` runs an
adversarial red-team → judge → blue-team patch → retest → report loop
against an LLM target you own. Covers OWASP LLM Top-10 / NIST AI RMF
families (prompt injection, tool misuse, data leakage, jailbreaks,
denial-of-wallet). Joins score/genome/mcp-scan/threat-model/oia-audit
as the run-time red/blue layer of the security plugin surface.
Why integrate here vs ruflo-aidefence or ruflo-security-audit:
- aidefence is detection (PII / prompt-injection scan); redblue is
*generation* — they complement, redblue doesn't replace it
- security-audit is dep CVEs / static scan; redblue is run-time LLM eval
- ruflo-metaharness already carries the @metaharness/* wrapper pattern
(`_darwin.mjs`, `_harness.mjs`), the locator, the MCP-tool registration
surface, and the smoke/CI gates — redblue slots in as one more sibling
Surface added:
- `npx ruflo metaharness redblue {init|run|patch|attack|report}` CLI
- `metaharness_redblue` MCP tool (subcommand + family + tests + patch +
mockJudge + alertOnFail + timeoutMs)
- Skill discoverability via CLAUDE.md (both CLI catalog + MCP tool catalog)
- 11 metaharness subcommands (was 10), 13 MCP tools (was 12)
Safety posture (inherited verbatim from upstream; wrapper does NOT relax):
- upstream `src/config/safety.ts` hard-enforces no real credentials, no
live external targets (loopback / *.test / *.internal only), no shell,
no arbitrary network, no eval at config-load time — a config that
enables any of these errors at load
- `--mock-judge` is the $0 marker-fixture path for CI / offline; the real
model judge requires `OPENROUTER_API_KEY` and is capped by
`max_cost_usd` / `max_runtime_minutes` / `max_tests`
- wrapper shells out with `shell: false` argv arrays; never string interp
- ADR-150 graceful-degradation rule #3 — on install/network failure the
wrapper returns `{degraded: true, reason: ...}` and exits 0 so ruflo
stays operational
Upstream bug workaround (track for revert):
`@metaharness/redblue@0.1.1`'s CLI bootstrap uses
`isMain = import.meta.url === \`file://\${process.argv[1]}\``. When npx
links `redblue` → `dist/cli/index.js`, argv[1] is the symlink path but
import.meta.url is the resolved real path, so isMain is false and the
binary silently exits 0 with no output, no file, no error. Darwin's CLI
calls `main()` unconditionally and is unaffected. Workaround: install the
package once into `~/.ruflo/redblue-cache` via `npm install --prefix` and
invoke `node <abs_path>/dist/cli/index.js` directly — argv[1] then equals
the real path and isMain becomes true. First install hits npm's cache so
the cold path is ~200ms; warm invocations are ~20ms.
Locator hardening (prevents recurrence of artifact-mirror staleness):
The publish artifact at `v3/@claude-flow/cli/plugins/ruflo-metaharness/`
is regenerated only at `prepublishOnly`. On a dev tree with a fresh
script in source but a stale mirror, the locator would pick the mirror
because `_harness.mjs` exists in both. Locators in `commands/metaharness.ts`
and `mcp-tools/metaharness-tools.ts` now accept an optional `requiredScript`
that narrows the match to dirs containing the dispatched script. The
artifact mirror is also synced with the two new redblue scripts so the
publish artifact stays usable until the next `prepublishOnly`.
Gates updated to lock new state:
- smoke.sh: 9 subcommands in dispatcher (was 8), 11 in description list
(was 10), 13 MCP tools (was 12), 13 mcp-script-count (was 12), 14
MCP_SUCCESS_SEMANTIC footnotes (was 13), 13 `success: r.success`
occurrences (was 12), tilde-pin check covers @metaharness/redblue
- test-mcp-tools.mjs: `tools.length === 13`, `metaharness_redblue`
enumerated, attack-preview Phase-3 input, 180s timeout (cold-cache npx
fetch class)
- CLAUDE.md: surfaced under both CLI catalog and MCP-tool catalog (gated
by smoke 17z56 + 17z57)
Verification:
- `bash plugins/ruflo-metaharness/scripts/smoke.sh` → 120 passed, 0 failed
- `node plugins/ruflo-metaharness/scripts/test-mcp-tools.mjs` → 153 passed, 0 failed
- `node v3/@claude-flow/cli/bin/cli.js metaharness redblue init --out <path>`
→ writes the safety-asserting sample config, exits 0
- attack preview, mock-judge run (3-test → real findings report), and
CLI dispatcher all exercised end-to-end before commit
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
76eca40429 |
fix(ci): mint.mjs probes metaharness availability before dry-run plan
The no-metaharness-smoke gate asserts every metaharness skill emits a
\`"degraded"\` field when metaharness is unreachable (ADR-150 #3 —
graceful degradation). All other skills did this already; mint did not
because its dry-run path printed the plan without touching the registry.
Result: smoke-without-metaharness reported \`FAIL: mint did not emit
degraded:true payload\`.
Added a fast \`runMetaharness(['--version'])\` probe at the top of main()
so the dry-run also reports degraded:true when the registry is
unreachable. The probe is the same call the --confirm path uses
internally, so we don't add a new failure mode; we just surface the
existing one earlier.
This was the final red check on PR #2468 — all other 112 checks are
green on commit
|
||
|
ruv
|
1d1b5ba726 |
fix(ci): 17o test-mcp-tools contract — enumerate 3 new MCP tools, expect 12
Final ruflo-metaharness smoke contract failure after the count bumps: test 17o anchors the contract test by enumerating every MCP tool name and asserting \`tools.length === 9\`. Both lists are deliberately spelled out so a refactor that drops a tool fails this gate. Added bench/evolve/ security_bench and bumped the count assertion to 12. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
4b9e86840f |
fix(ci): bump 3 ruflo-metaharness stale lock-counts for ADR-153 tools
3 more plugin-contract failures from no-metaharness-smoke, all the same
root cause: ADR-153 added bench/evolve/security_bench MCP tools but the
smoke contract had hardcoded counts pinned to the pre-ADR-153 tool set.
- 17z53: \`runScript('foo.mjs', ...)\` reference count 9 → 12
- 17z9: MCP_SUCCESS_SEMANTIC occurrence count 10 → 13 (1 decl + 12 uses)
- 17z7: \`success: r.success\` handler count 9 → 12
- 17z6: test-mcp-tools.mjs phase-1 \`tools.length === 9\` → 12, plus
the 3 new tool names added to the expectedNames set
These are anchor counts that exist deliberately so a future iter adding
a tool also touches the smoke and CLAUDE.md catalog. Bumping in lockstep
with the 3 new tools.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
575a7bdeed |
fix(ci): ruflo-metaharness smoke — fix brittle 17z61 lookback + iter-93 stale count
Two plugin-contract failures in the no-metaharness-smoke job for ruflo-metaharness: - **17z61** (drift-from-history dispatcher round-trip) used \`grep -B100\` to walk back from the step to its enclosing job header. metaharness-ci.yml grew past 100 lines between that step (now line 604) and its \`metaharness-real-data:\` job header (line 493). Replaced the fixed-window grep with an awk one-pass that finds the most-recent top-level job header above the step. - **17z56** (every MCP tool documented in CLAUDE.md, iter 93) asserted exactly 9 MCP tools. ADR-153 added 3 more (\`metaharness_bench\`, \`metaharness_evolve\`, \`metaharness_security_bench\`) without updating the lock count or the CLAUDE.md catalog. Documented the 3 tools in CLAUDE.md and bumped the count to 12. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
fdeacad7f8 |
feat(metaharness): integrate @metaharness/darwin@0.3.1 + bump umbrella to 0.2.6
Implements ADR-153 (Darwin Mode integration). Adds the WRITE layer that
closes the loop ADR-150's READ layer opens: score/genome describe a
harness; evolve changes one.
Three new surfaces, all honoring the four ADR-150 architectural constraints
(removable / optional / graceful degradation / CI-absent-path coverage):
- harness-evolve mutate 1 of 7 policy surfaces, sandbox-score, promote
- harness-security-bench upstream's "Darwin Shield" (their own ADR-155)
- harness-bench create/verify bench suites for evolve --bench
Wiring:
- plugins/ruflo-metaharness/scripts/_darwin.mjs — shared subprocess helper
- plugins/ruflo-metaharness/scripts/evolve.mjs — main verb (--confirm gate)
- plugins/ruflo-metaharness/scripts/security-bench.mjs — Darwin Shield wrapper
- plugins/ruflo-metaharness/scripts/bench.mjs — supporting verb
- plugins/ruflo-metaharness/skills/harness-{evolve,security-bench,bench}/SKILL.md
- v3/@claude-flow/cli/src/mcp-tools/metaharness-tools.ts — 3 new MCP tools:
metaharness_evolve, metaharness_security_bench, metaharness_bench
- v3/@claude-flow/cli/package.json optionalDependencies:
+ @metaharness/darwin ~0.3.1
bump metaharness ~0.1.11 -> ~0.2.6
- ruflo/package.json optionalDependencies: mirrored (per the #2112 lesson —
wrapper does NOT inherit root overrides; required for transitive pin)
Safety posture (matches mint.mjs convention):
- evolve requires --confirm; without it, returns a dry-run plan
- Ruflo caps: --generations 1..50, --children 1..20, --concurrency 1..8,
--population 1..20, --cycles 1..100 (upstream supports more)
- Upstream exit code 99 (safety-disqualified) propagates VERBATIM —
not remapped — so CI can distinguish "evolution failed" from
"evolution surfaced a safety-tripping mutation"
- All three scripts emit {degraded: true, reason: 'metaharness-darwin-not-available'}
and exit 0 when the optional dep is absent
Connection to ruflo's own ADR-155 (nightly self-learning security harness,
PR #2417, issue #2418): harness-security-bench is the closest reference
implementation for our Loop A reward-signal sanity check. Running it
periodically gives us the empirical floor — if Darwin Shield's champion
reaches TPR=1/FPR=0 on the seeded corpus, our loop A's gradient signal
is sound.
Tests:
- plugins/ruflo-metaharness/scripts/test-graceful-degradation.mjs extended
with the 3 new scripts; all 16 assertions pass (8 skills × 2 contracts:
exit 0 AND emit "degraded": true)
- npx tsc --noEmit clean on v3/@claude-flow/cli
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
daf39efa1b |
fix(bench): bench-recordpair-overhead emits markdown header in JSON mode (iter 137)
router-compat CI step pipes the bench's --format json output to JSON.parse, but the script always emitted a markdown header `# bench-recordpair-overhead` + iterations line BEFORE the JSON body. Result: SyntaxError: Unexpected token '#', "# bench-re"... is not valid JSON Same fix-class as iter 87 for bench-similarity. Guard the header output behind `ARGS.format !== 'json'` so --format json emits pure JSON. Local: pipes cleanly into JSON.parse — OK. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
95dc43be20 |
fix(ci): bump cold-cache timeouts (iter 131)
Two CI cold-cache problems hit the 180s ceiling — bump to 300s. (1) Windows hook execution smoke / macos-latest Observed running EXACTLY 180s before exit code null. macos runner's cold start of `ruflo init` triggers: ONNX model download, agentic-flow workspace init, MCP server spawn. CI's job timeout-minutes is 10, so 300s leaves room for assertion phase. Applied to both smoke-windows-hook-execution.mjs and smoke-windows-init-hooks.mjs. (2) test-mcp-tools chain-tool budget 90s → 180s metaharness_audit_list still timed out at 90s. Locally audit-list runs in ~4s; in CI, `npx @claude-flow/cli@latest memory list` pays the npx tarball fetch (warm) + full CLI startup (loads agentic-flow and ONNX). 180s gives 30x headroom over local cost. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
db1e965e19 |
fix(test-mcp-tools): audit_list also a chain-tool — 60s → 90s (iter 130)
In CI cold-cache, after oia_audit warms up the npx cache, metaharness_audit_list STILL timed out at 60s. The audit_list handler invokes `npx claude-flow memory list --namespace metaharness-audit` which has its own cold-cache cost on first invocation (memory backend init + ONNX embedder load if needed). Add audit_list to the chain-tool bracket alongside drift_from_history and oia_audit so it gets the 90s budget. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
8df666e6d5 |
fix(smoke): 17z17 accepts isChainTool refactor (iter 129)
iter 128 refactored test-mcp-tools.mjs from:
const timeoutMs = tool.name === 'metaharness_drift_from_history'
? 90_000 : 60_000;
to:
const isChainTool = tool.name === 'metaharness_drift_from_history'
|| tool.name === 'metaharness_oia_audit';
const timeoutMs = isChainTool ? 90_000 : 60_000;
The smoke step 17z17 ("drift_from_history MCP tool + CLAUDE.md
surfacing") still grep'd for the literal ternary form, so it failed
with "no-extended-timeout" after the refactor. Extended the OR-set
to include the isChainTool form.
Net: 528/528 plugin smoke steps now pass in no-metaharness-smoke CI.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
bc38e38108 |
fix(eject + mcp-tools-test): context.flags + oia_audit chain budget (iter 128)
Two related router-compat/eject-dryrun fixes. (1) eject — `--name is required` despite passing --name The eject action read from context.args via a local parseArgs, but the CLI parser already consumed declared flags into context.flags (NOT context.options or context.args). context.args was always empty in normal CLI invocation. Switched to read from context.flags first, fall back to parseArgs(context.args) for programmatic / test invocations where the parser wasn't used. Local: `node bin/cli.js eject --name my-test-harness --format json` now emits the expected dry-run plan with dryRun:true. (2) test-mcp-tools — `metaharness_oia_audit handler did not throw` oia_audit composes 5 sub-audits (oia-manifest + threat-model + mcp-scan + score + genome) in parallel — each shells out via npx. In CI cold-cache, the 60s budget was insufficient. Bumped its timeout to 90s (same bracket as drift_from_history) and added --dry-run input so the test doesn't pollute the audit namespace. Local: 117 passed, 0 failed (was 117/2 in CI cold-cache). Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
c4fc017f81 |
fix(smoke): 17z15 accepts test-mcp-tools SKIPPED-on-missing-dist (iter 126)
iter-125 added grep tolerance for ERR_MODULE_NOT_FOUND but test-mcp-tools.mjs self-detects missing dist BEFORE the dynamic import and emits this instead: # test-mcp-tools — SKIPPED Compiled dist not present: …/dist/src/mcp-tools/metaharness-tools.js Build the CLI first: ... Exit 0 — this script is meaningfully runnable only post-build. So the smoke's RTOUT in CI's score job contained "SKIPPED" / "Compiled dist not present", not ERR_MODULE_NOT_FOUND. Both grep targets added. Verified locally by hiding the dist file: greps match the actual SKIPPED-path stdout. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
14c36a4083 |
fix(smoke): tolerate CI degraded paths in 17z28 + 17z15 (iter 125)
Two metaharness smoke contracts still tripped in PR #2405 CI even after iter-124's contract-grep fixes. Root cause for both: CI's score job runs the smoke against a fresh checkout WITHOUT building the CLI dist or seeding the metaharness-audit memory namespace, so two runtime checks landed in legitimate degraded paths the smoke didn't cover. (1) 17z28 — drift-from-history runtime fields `node drift-from-history.mjs --dry-run --format json` was expected to emit `timing.parallelWallMs` + `timing.parallelSpeedup` unconditionally. But when the metaharness-audit namespace has zero records (always true in fresh CI), drift-from-history correctly exits 2 with `{error, stderrTail}` — the documented no-history degraded path (iter 57 + iter 58). Skip the runtime field check on that path. (2) 17z15 — test-mcp-tools.mjs runtime envelope The runtime arm of this gate runs `node test-mcp-tools.mjs` which imports MCP tool handlers from dist/src/mcp-tools/. In CI's score job the dist isn't built (only the metaharness-real-data job builds it via `pnpm -F "@claude-flow/cli..." run build`), so the import fails with ERR_MODULE_NOT_FOUND. The structural greps higher in the step already lock the Phase 4 assertions on the source file; runtime confirmation continues to run in metaharness-real-data which DOES build the dist. Local: 120 passed, 0 failed (was 119/0, lost no ground). Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
0215e7b633 |
fix(metaharness): 5 smoke contract failures (iter 124)
5 metaharness smoke contracts had grown out of sync with the source.
Local + CI smoke now goes 119/0 (was 5/120 failing).
(1) 17i — audit-trend severity rank grep was stale
Old check: literal `SEVERITY_RANK = { clean: 0, low: 1, medium: 2, high: 3 }`
But iter 63 refactored to shared `import { SEVERITY_RANK } from './_harness.mjs'`.
Updated grep to accept either form.
(2) 17z — ADR-152 §3.1 oia-audit fingerprint grep was stale
Old check: `score = runOne(['score'` / `genome = runOne(['genome'`
But iter 56 parallelized to `runMetaharnessAsync(['score', path])`.
Updated grep to accept either form.
(3) 17z14 — roundtrip Stage 7 hardcoded the assertion count
Old check: `node test-pipeline-roundtrip.mjs | grep "31 passed, 0 failed"`
But stages have been added since iter-51; count is now 66.
Updated to gate on "≥ 31 passed AND 0 failed" — no failures + floor.
(4) 17z6 — test-mcp-tools.mjs metaharness_audit_list intermittent 30s timeout
The first MCP tool that shells out via npx pays the cold-cache warmup
(~25s observed); 30s was tight. Bumped default per-tool timeout 30s
→ 60s (chain-tools like drift_from_history stay at 90s).
(5) test-mcp-tools.mjs mcp_scan first-finding message string
Upstream metaharness@0.1.x JSON path emits {id, severity, title, detail};
our parseMcpScanText emits {severity, message}. The test asserted
`message` only. Normalized both paths in mcp-scan.mjs to a unified
shape: {severity, message, title?, detail?} where `message` falls
back to `title || detail || ''` when the upstream path doesn't
supply one. Downstream consumers (audit-trend fingerprint diff,
test-mcp-tools assertions) now don't care which parser ran.
Local smoke runs: was 115/5, now 119/0 (and counts will grow further
as stages get added — the assertion-count floor protects against
regressions, the literal exact-match did not).
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
1de69daa95 |
test(metaharness): anchor oia-audit-weekly iter-108 hard-fail + iter-109 dispatch inputs (iter 116)
Two load-bearing invariants accumulated in the weekly cron workflow,
neither of which had a smoke anchor:
iter 108 — hard-fail when timing.path != "file"
The cron's --baseline-file flag is the load-bearing invariant for
cron-budget correctness. If it disappears, drift detection silently
falls back to slow path (~26s instead of ~1.4s), and the iter-7
weekly-cron budget exhausts on slower runners. Iter 108 added an
explicit `if [ "$PATH_LABEL" != "file" ]; then exit 1` block so
a regression produces a ::error:: annotation, not silent slowdown.
iter 109 — workflow_dispatch policy inputs
Parameterized `threshold` + `alert_on_new_severity` as
workflow_dispatch inputs so ad-hoc runs can explore stricter/looser
gates without editing YAML. The drift step references both via
`${{ inputs.threshold || '0.85' }}` and
`${{ inputs.alert_on_new_severity || 'high' }}`.
Both invariants are *removable* by a careless refactor; the YAML
still validates; the cron still runs; only the gate disappears.
That's exactly the failure mode smoke anchors are designed to catch.
WHAT SHIPPED — smoke step 17z79
Seven grep checks against .github/workflows/oia-audit-weekly.yml:
1. iter-108 fail-fast annotation string present
2. iter-108 PATH_LABEL != "file" check present
3. iter-109 workflow_dispatch: trigger present
4. iter-109 threshold: input declaration present
5. iter-109 alert_on_new_severity: input declaration present
6. iter-109 inputs.threshold wiring into drift step present
7. iter-109 inputs.alert_on_new_severity wiring present
Specific failure markers: "iter-108-hard-fail-removed",
"iter-109-threshold-input-removed", etc. — operator can see exactly
which invariant regressed.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-116)
| Category | Surfaces | Iters |
|------------------------|--------------------------------|-------------|
| Source → Target | 10 | 73-94, 106 |
| Target → Source | 1 | 102 |
| Internal consistency | 4 | 103-105, 107|
| ADR-150 invariants | 2 | 110, 111 |
| Workflow-job set | 1 | 115 |
| Workflow-step set | 1 (iter-108 + iter-109 anchors)| 116 | ◀ this iter
Nineteen smoke-gated surfaces.
GATES THIS ITER
smoke: 117 → 118 (step 17z79 green)
ci yml: 7 invariant strings × file-presence verified
PATTERN
Iter-115 anchored *which jobs* exist in a workflow. Iter-116 anchors
*which steps within a job* exist. Both are anti-deletion guards on
files that YAML-validate even after the gate is gone.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
b851f849a5 |
test(metaharness): metaharness-ci.yml contains all 6 expected jobs (iter 115)
metaharness-ci.yml accumulated 6 jobs across iters 1-48:
score (iter 1) — score harness against ruflo
mcp-scan (iter 1) — security finding scan
router-compat (iter 12) — @metaharness/router API tripwire
eject-dryrun (iter 4) — Phase-2 differentiator
similarity-tests (iter 40) — ADR-152 §3.1 contract
metaharness-real-data (iter 48) — load-bearing integration gate
A refactor that accidentally deletes a job header passes YAML
validation (the file is still valid YAML) but silently disables the
gate. The remaining jobs would still run, the CI status would be
green, and the deleted job's protection would just disappear.
WHAT SHIPPED — smoke step 17z78
For each of the 6 expected jobs:
- grep `^ jobname:$` (exact YAML job-header form)
- Specific failure: "missing-job-jobname"
Plus a count floor: ≥6 jobs expected (the count subtracts 1 for the
`push:` trigger block which also matches `^ ...:$`).
If a future iter:
- Drops a job header → "missing-job-eject-dryrun"
- Renames a job → "missing-job-old-name" + count stays same
- Adds a 7th job → all 6 pass, smoke still green (floor is
inclusive, not exact — additions OK)
The smoke gate is anti-deletion + anti-rename, not anti-addition.
That's the right asymmetry for a job-set assertion.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-115)
| Category | Surfaces | Iters |
|------------------------|--------------------------------|----------|
| Source → Target | 10 (description, SUBCOMMANDS, | 73-94, |
| | runScript, SKILL.md, MCP enum, | 106, 107 |
| | tool names + subcmd to CLAUDE, | |
| | CI tripwire refs, ADR refs) | |
| Target → Source | 1 (orphan-script gate) | 102 |
| Internal consistency | 4 (skill dir↔frontmatter, | 103-105, |
| | allowed-tools, desc+hint, | 107 |
| | ADR-151 scope) | |
| ADR-150 invariants | 2 (tilde pin, pin alignment) | 110, 111 |
| Workflow-job set | 1 (metaharness-ci jobs) | 115 | ◀ this iter
Eighteen smoke-gated surfaces.
GATES THIS ITER
smoke: 117 → 118 (step 17z78 green)
contract: 6 CI workflow jobs × file-presence verified
PATTERN
iter-115 is the workflow-layer equivalent of iter-89's SUBCOMMANDS-map
gate. Both verify that an enumerated set of identifiers (CLI subcommands
/ CI job names) all exist where they should.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
c712f4973a |
test(metaharness): bench JSON has results[] with numeric meanUs (iter 114)
iter-88 verified bench output JSON.parses. iter-114 goes deeper into
the schema: the bench JSON must have a non-empty `results[]` array
with `meanUs: number` and `label: string` per entry.
The bug-class iter-114 catches:
- Empty results array (e.g., wrong loop bounds): JSON.parse passes,
iter-82 GITHUB_STEP_SUMMARY table renders empty, iter-99 wall-budget
reads undefined meanUs
- Non-numeric meanUs (e.g., a bug stringifies it): downstream
consumers compute Number(meanUs) → NaN → broken comparisons
- Missing label: artifact analysis can't distinguish per-category
measurements
These are real failure modes that iter-88's parse-only gate misses.
WHAT SHIPPED — smoke step 17z77
For each of bench-similarity + bench-parse-mcp-scan:
- Run with --iters 500 --format json
- Parse + assert:
results is an array
results.length > 0
every entry has meanUs as number (int or float)
every entry has label as string
Specific failure form: ${bench}-empty / ${bench}-non-numeric-meanUs
/ ${bench}-not-array / ${bench}-non-string-label
GATES THIS ITER
smoke: 116 → 117 (step 17z77 green)
contract: 2 bench scripts × results[].meanUs + label shape verified
BENCH SCRIPT CONTRACT (post-iter-114)
| Gate | What it catches | Iter |
|-------------------------|------------------------------|------|
| `--format json` parses | output contamination | 88 |
| has generatedAt | timestamp emission | 112 |
| has results[] | array shape + numeric meanUs | 114 |
| --max-mean-us threshold | perf regression | 41+86|
| CI artifact upload | historical baseline tracking | 82+87|
Five gates per bench script. Future bench-scripts should adopt the
same shape contract.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
05b8e2f548 |
test(metaharness): oia-audit startedAt + finishedAt timestamp pair (iter 113)
Companion to iter-112's generatedAt gate. iter-112 documented oia-audit
as the multi-step exception: it emits startedAt + finishedAt instead
of a single generatedAt because it composes 5 subprocess calls and
the duration spans those calls.
iter-113 verifies the iter-7 / iter-56 contract — both timestamps
present and finishedAt >= startedAt.
WHAT SHIPPED — smoke step 17z76
Runs oia-audit --dry-run --format json and checks:
- startedAt is non-empty (≥20 chars ISO timestamp)
- finishedAt is non-empty
- finishedAt >= startedAt (sanity — composite never finishes before
it started; a clock-skew bug or a startedAt set AFTER the loop
would fail this check)
Failure form: "oia-audit-timestamp-${reason}" where reason is one of:
- missing-startedAt
- missing-finishedAt
- short-timestamps
- finished-before-started
iter-56's parallelization preserved the startedAt-set-before-runAllParallel
invariant; iter-113 anchors it at smoke-time.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-113)
iter-112 + iter-113 together codify the timestamp emission contract:
- Single-step scripts: generatedAt (iter 112)
- Multi-step composite (oia-audit): both (iter 113)
A future refactor that drops either field — or breaks the temporal
ordering — surfaces as a specific named failure.
GATES THIS ITER
smoke: 115 → 116 (step 17z76 green)
contract: oia-audit timestamp pair + ordering verified
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
38767da86f |
feat(metaharness): generatedAt timestamp on all --format json outputs (iter 112)
Pre-iter-112 audit of every --format json script revealed inconsistent
timestamp emission:
✓ score, threat-model, audit-list, audit-trend, similarity,
drift-from-history, bench-similarity, bench-parse-mcp-scan
✗ genome, mcp-scan (no generatedAt field)
⊘ oia-audit (uses startedAt/finishedAt — multi-step composite,
by design)
Downstream consumers (audit-trend's introduced/cleared diff, the
weekly cron's drift summary, ops scripts archiving artifacts) all
benefit from a canonical "when was this run" field. iter-112 fills
the two gaps.
WHAT SHIPPED
1. plugins/ruflo-metaharness/scripts/genome.mjs — payload now
includes `generatedAt: new Date().toISOString()`
2. plugins/ruflo-metaharness/scripts/mcp-scan.mjs — same field
appended to the --format json payload
3. plugins/ruflo-metaharness/scripts/smoke.sh step 17z75:
- Iterates 9 scripts (omits oia-audit which by design uses
startedAt/finishedAt) with minimal valid args
- For each: parses --format json output, asserts generatedAt
field exists with ≥20 chars (ISO timestamp format check)
- Failure form: "${name}-no-generatedAt"
DISCOVERY DURING IMPLEMENTATION
First-pass smoke step used `declare -A` (bash 4+ associative arrays).
macOS bash 3.2 doesn't support them — smoke threw "unbound variable"
immediately. Rewrote using the pipe-delimited list pattern from iter
88's same-style gate. Documented in the script-tests array shape.
The bash-portability bug-class is now showing up across multiple iters
(iter 94 BSD-sed `\s`, iter 112 declare -A). Future smoke contributors
should default to bash-3.2-compatible patterns.
GATES THIS ITER
smoke: 114 → 115 (step 17z75 green)
contract: 9 scripts × generatedAt ISO timestamp emitted
regression: none — purely additive field
CONSISTENT JSON CONTRACT (post-iter-112)
Every --format json output now emits:
{ ..., generatedAt: ISO-8601 timestamp }
oia-audit's startedAt/finishedAt remains the multi-step exception
(documented in iter-7 + the smoke-step exclusion comment).
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
82c8bacbc6 |
test(metaharness): pin versions consistent across 3 package.json files (iter 111)
iter-110 gated each pin's TILDE PREFIX. iter-111 gates the VERSION
ALIGNMENT across the 3 package.json files:
- package.json (root, dev install)
- ruflo/package.json (npm i ruflo)
- v3/@claude-flow/cli/package.json (npm i @claude-flow/cli)
Drift scenario:
- Someone updates ruflo/package.json to `metaharness: ~0.2.0`
- Forgets to update the other 2
- User installing ruflo gets metaharness 0.2.x
- CI tests against root's `~0.1.11` (still 0.1.x)
- Production behavior diverges from CI behavior — invisible until
production hits a metaharness 0.2.x-only bug
iter-111 enforces the pins match across all 3.
WHAT SHIPPED — smoke step 17z74
For each of metaharness, @metaharness/router, @metaharness/kernel:
- Read the optionalDependencies entry from all 3 package.json files
- Assert exactly 1 unique pin string (or 0, if package absent from all 3)
- Failure form: `metaharness-pin-drift:root=~0.1.11,ruflo=~0.2.0,cli=~0.1.11`
The failure message NAMES all 3 versions so the fixer immediately
sees which file is the outlier.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-111)
| Direction | Surface | Iter |
|--------------------------|-------------------------------|------|
| Source → Target (10 forward) |
| Target → Source (1 reverse) |
| Internal consistency (4) |
| ADR-150 invariants (2, iters 110 + 111) |
| package.json pins | tilde prefix | 110 |
| pin versions | consistent across 3 files | 111 | ◀ this iter
17 smoke-gated surfaces.
GATES THIS ITER
smoke: 113 → 114 (step 17z74 green)
contract: 3 metaharness packages × cross-file pin alignment verified
PIN-POLICY ARC (iters 110 + 111)
110 — pin SHAPE: must start with ~
111 — pin VALUE: must match across 3 package.json files
Together they encode the policy: "every metaharness pin uses tilde,
and the same tilde-pin appears wherever the package is referenced."
A drift in either shape OR value surfaces as a specific named smoke
failure.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
c1bc0c7408 |
test(metaharness): tilde-pin gate prevents caret regression (iter 110)
ADR-150's architectural-constraint review-round-1 mandated tilde pinning (`~0.1.11`) for @metaharness/* packages, not caret (`^0.1.11`). Rationale: upstream is unstable — iter-3 observed 5 releases in 2.7 hours. Tilde auto-absorbs PATCHES but stays on the same MINOR. Caret would auto-absorb minor bumps that might break ruflo's integration. Currently all 3 metaharness packages in package.json files use tilde: metaharness: ~0.1.11 @metaharness/router: ~0.3.2 @metaharness/kernel: ~0.1.0 A future contributor doing routine dep updates might run `npm install --save metaharness@latest` which writes caret pinning by default. That regression would be invisible at the package.json layer until upstream ships a 0.2.x release. WHAT SHIPPED — smoke step 17z73 For each of: - package.json (root) - ruflo/package.json - v3/@claude-flow/cli/package.json Iterates optionalDependencies and asserts every metaharness/@metaharness entry has a tilde-prefix version string. Caret, exact, or range-pinning all trip the gate. Failure form: "non-tilde-pin-in-ruflo:@metaharness/router=^0.3.2" Names BOTH the file AND the offending pin. CROSS-REFERENCE INTEGRITY MATRIX (post-iter-110) | Direction | Surface | Iter | |------------------------|-------------------------------|------| | Source → Target (10 forward, iters 73-94, 106, 107) | | Target → Source (1 reverse, iter 102) | | Internal consistency (4, iters 103-105, 107) | | ADR-150 invariants (1, iter 110) | | package.json pins | tilde-only for metaharness | 110 | ◀ this iter The "thing X must look like Y" pattern from iters 73-107 extends to the dependency-pinning policy. GATES THIS ITER smoke: 112 → 113 (step 17z73 green) contract: 3 package.json × 3 metaharness packages × tilde-prefix verified ADR-150 ARCHITECTURAL-CONSTRAINT ENFORCEMENT (post-iter-110) Constraint #1 (removable): iter 74 negative grep Constraint #2 (optional-only): iter 12 static-deps check Constraint #3 (graceful degrade): iter 19 + 55 drill Constraint #4 (CI gate): iter 7 no-metaharness-smoke.yml + Tilde pin (review-round-1): iter 110 ◀ this iter All four ADR-150 architectural constraints plus the review-mandated tilde pin are now smoke-gated. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
51db9b381e |
ci(metaharness): weekly cron exposes policy thresholds via workflow_dispatch inputs (iter 109)
Pre-iter-109 the cron's policy thresholds (--threshold 0.85,
--alert-on-new-severity high) were hardcoded. To test stricter or
looser gates ad-hoc, maintainers had to:
1. Edit oia-audit-weekly.yml
2. Commit
3. Push
4. Wait for the workflow trigger
5. Revert
Iter 109 adds workflow_dispatch inputs so maintainers can test
different thresholds via the Actions UI's "Run workflow" form without
touching the YAML.
WHAT SHIPPED — oia-audit-weekly.yml
workflow_dispatch:
inputs:
threshold:
description: 'Structural similarity threshold (default 0.85 —
scheduled). Lower = stricter drift alert.'
type: string
default: '0.85'
alert_on_new_severity:
description: 'Alert on any introduced finding ≥ this severity'
type: choice
options: [info, low, medium, warn, high, error, critical]
default: high
Drift step uses the inputs via fallback expressions:
THRESHOLD="${{ inputs.threshold || '0.85' }}"
ALERT_SEV="${{ inputs.alert_on_new_severity || 'high' }}"
Scheduled runs (no manual input) get the hardcoded defaults
(0.85 / high) — same behavior as pre-iter-109.
USAGE EXAMPLES
- Standard weekly run (Sundays 04:17 UTC):
Uses 0.85 + high (defaults)
- Ad-hoc stricter check via Actions UI "Run workflow":
threshold: 0.95
alert_on_new_severity: medium
→ Alerts on smaller drift + any new MEDIUM-or-higher finding
- Permissive ad-hoc check (e.g., investigating a known-noisy week):
threshold: 0.50
alert_on_new_severity: critical
→ Only structural collapse (overall < 0.5) OR a critical finding fires
The enum for alert_on_new_severity matches iter-78's MCP enum
(iter-92's smoke step keeps these aligned).
SMOKE STEP 17z72 anchors:
- threshold + alert_on_new_severity inputs present
- Both have safe defaults (scheduled runs preserved)
- Drift step uses the inputs via $THRESHOLD / $ALERT_SEV vars
- Choice list contains all 7 severity values from iter-78
PLUS iter-79's smoke step 17z42 relaxed: now accepts either the
pre-iter-109 literal `--alert-on-new-severity high` OR the post-iter-109
parameterized form `--alert-on-new-severity "$ALERT_SEV"`. Same forward-
compat pattern as iter-60's relaxation in iter 84.
GATES THIS ITER
smoke: 111 → 112 (step 17z72 green; 17z42 grep relaxed)
yaml: python yaml.safe_load passes
cron: scheduled run behavior unchanged; ad-hoc tuning unlocked
OBSERVATION
Iters 78+79+108+109 are the policy-control arc:
78 — added --alert-on-new-severity flag at script + MCP
79 — wired it into cron (hardcoded 'high')
108 — hard-fail when fastpath label drifts away from 'file'
109 — parameterize the policy thresholds via dispatch inputs
The cron now has hardcoded SAFETY invariants (path === 'file') and
TUNABLE POLICY (threshold + alert severity). The split reflects what
each value's purpose is: safety = static, policy = dynamic.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
08757d2db1 |
ci(metaharness): weekly cron hard-fails on slow-path regression (iter 108)
iter-69 + iter-79 wired the weekly cron to use --baseline-file fastpath.
iter-97 surfaced the path in $GITHUB_STEP_SUMMARY (informational). iter-99
added a wall-budget gate to the metaharness-ci dispatcher check. But the
WEEKLY CRON itself had no hard fail on slow-path regression.
The drift scenario:
- Someone refactors oia-audit-weekly.yml and accidentally drops
`--baseline-file /tmp/prior/oia-audit.json` from the drift step
- drift-from-history falls through to slow path
- timing.path === 'slow', wall ~26s instead of expected ~1.4s
- Workflow STILL PASSES (just slower)
- Path: slow line appears in summary but no one notices for weeks
- Eventually the 10-min job timeout-minutes catches it, but only
after MANY weeks of slowness
iter-108 makes the regression fail immediately and specifically.
WHAT SHIPPED — oia-audit-weekly.yml drift step extension
After the existing summary lines:
PATH_LABEL=$(node -e "
const j = JSON.parse(require('fs').readFileSync('/tmp/drift-trend.json'));
console.log(j.timing?.path || 'unknown');
")
if [ "$PATH_LABEL" != "file" ]; then
echo "::error::Cron drift step expected timing.path='file' but got
'$PATH_LABEL' — the iter-67 fastpath flag may have regressed"
exit 1
fi
The GHA `::error::` annotation surfaces in the Actions UI as a red-X
annotation pointing at the failing line, not just a generic step
failure. Maintainers see "iter-67 fastpath flag may have regressed"
and know exactly where to look.
WHY HARD-FAIL HERE BUT NOT IN ITER-99
iter-99 gates `wall < 30000ms` because CI runners are slow and 30s
is the conservative-safe upper bound. iter-108 gates the SPECIFIC
fastpath label because the cron's --baseline-file flag is explicit
in the YAML — there's no reason for the path label to be anything
other than 'file' on a healthy run.
If the path is 'key' (someone changed to --baseline-key) or 'slow'
(no fast-path flag), iter-108 fails. iter-99's wall gate is the
generic backstop; iter-108's path gate is the specific contract.
SMOKE STEP 17z71 anchors:
- PATH_LABEL variable
- timing?.path extraction with 'unknown' fallback
- The != "file" conditional
- The error message referencing iter-67
- GHA ::error:: annotation form
GATES THIS ITER
smoke: 110 → 111 (step 17z71 green)
yaml: python yaml.safe_load passes
cron: drift step hard-fails on path-label regression
ITER 65 + 95 + 96 + 97 + 99 + 108 — FAST-PATH OBSERVABILITY + ENFORCEMENT
65 — timing.parallelSpeedup field (observe)
95 — timing.path derived field (observe)
96 — CLI table mode surfaces Path: (observe — human)
97 — Weekly cron summary surfaces Path: (observe — UI)
99 — CI dispatcher wall budget < 30s (enforce — generic)
108 — Weekly cron path === 'file' exact (enforce — specific) ◀
Observability is complete; enforcement is now layered: generic
wall-budget catches gross regressions, specific path-label catches
the exact iter-67 fastpath drop.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
6638530ba8 |
test(metaharness): ADR-152 refs resolve + ADR-151 scope-refs documented (iter 107)
iter-106 gated ADR-150's cross-references. iter-107 extends to ADR-152
(which only references existing ADRs) AND adds a structural assertion
to ADR-151 (which references the not-yet-created ADR-153-156 by design).
THE TWO ASYMMETRIC ADRs
ADR-152 (Genome Similarity Search):
- References ADR-150 (parent: Phase 1+2)
- References ADR-151 (parent: Phase-3 scope)
- All refs MUST exist → standard cross-reference gate
ADR-151 (Phase-3 scope shell):
- References ADR-150 (parent)
- References ADR-152 (existing child)
- References ADR-153, 154, 155, 156 (future children, scope-only)
- The future ADRs are INTENTIONALLY referenced before they exist;
ADR-151's design is to scope-lock them
- File-existence gate would fail here for the WRONG reason
WHAT SHIPPED — smoke step 17z70
- ADR-152: iter-106 style file-existence check on every ADR ref
- ADR-151: structural assertion that:
(a) explicitly contains "scope-only" or "scope only" marker
(b) explicitly references ADR-153, 154, 155, 156 (the 4 future ADRs)
If a future iter:
- Drops the scope-only marker → "adr151-no-scope-only-marker"
- Forgets to mention any of 153-156 → "adr151-missing-ADR-NNN"
The dual-check pattern lets each ADR be gated by the rule that
applies to it:
- "all refs exist" for ADRs with hard cross-refs
- "scope-refs named" for the scope-shell ADR
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-107)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (10 forward) |
| ADR-150 refs | ADR files | 106 |
| ADR-152 refs | ADR files | 107 | ◀ this iter
| Target → Source (1 reverse, iter 102) |
| Internal consistency (4, iters 103-105 + 107) |
| ADR-151 scope marker | 153-156 references | 107 | ◀ this iter
Fourteen smoke-gated surfaces.
GATES THIS ITER
smoke: 109 → 110 (step 17z70 green)
contract: ADR-152's 3 refs verified + ADR-151's 4 scope markers anchored
OBSERVATION
Different ADRs have different reference semantics. Implementation ADRs
(150, 152) make hard cross-references. Scope-shell ADRs (151) make
forward references to future ADRs that don't exist yet. The smoke
gate distinguishes these via STRUCTURAL assertions specific to each
ADR's pattern, not a one-size-fits-all rule.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
f80977b566 |
test(metaharness): ADR-150 cross-references resolve (iter 106)
iter-91 gated SKILL.md → script-path references. iter-106 applies the
same pattern to the ADR layer: ADR-150 references ADR-151 (Phase-3
scope shell) and ADR-152 (Genome Similarity Search). If either gets
renamed/moved, ADR-150's links rot silently.
Drift scenario:
- Someone renames `ADR-151-harness-intelligence-layer.md` to
`ADR-151-phase-3-scope.md` (cleaner slug)
- ADR-150 still links to the old name
- Markdown renders but the link 404s
- Future maintainers reading ADR-150 hit a dead end finding the
Phase-3 scope
WHAT SHIPPED — smoke step 17z69
- Extract every `ADR-15[0-9]` reference from ADR-150's body
- For each ref, assert the matching file exists in v3/docs/adr/
using glob ADR-NNN-*.md (tolerates slug renames? No — the file
must literally be there with the ADR-NNN-...md naming).
- Floor count ≥ 3 (ADR-150 must reference itself + at least 151 + 152)
Current refs: ADR-150, ADR-151, ADR-152 (all resolve)
If a future iter:
- Renames ADR-151 file but not the ADR-150 references
→ "ADR-151-not-found"
- Adds an ADR-153 reference without creating the file
→ "ADR-153-not-found"
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-106)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (9 forward) |
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP runScript() | scripts/*.mjs | 90 |
| SKILL.md refs | scripts/*.mjs | 91 |
| MCP enum | SEVERITY_RANK | 92 |
| MCP tool names | CLAUDE.md catalog | 93 |
| SUBCOMMANDS keys | CLAUDE.md catalog | 94 |
| CI workflow refs | tripwire scripts | 84 |
| ADR-150 refs | ADR-NNN files | 106 | ◀ this iter
| Target → Source (1 reverse, iter 102) |
| Internal consistency (3, iters 103-105) |
Thirteen smoke-gated cross-reference surfaces.
GATES THIS ITER
smoke: 108 → 109 (step 17z69 green)
contract: 3 ADR references × 3 file-existence checks pass
DOCUMENTATION-LAYER INTEGRITY
Source-code references already gated at multiple layers. iter-106
extends to the ADR layer — the documentation form of cross-reference.
Now both code references AND ADR references have smoke-time existence
gates.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
d36c902133 |
test(metaharness): SKILL.md description + argument-hint populated (iter 105)
Companion to iter-104's allowed-tools gate. Three frontmatter fields
matter for an agent invoking a skill:
name: skill ID (gated iter 103)
description: agent uses this to PICK the right skill
argument-hint: agent uses this to KNOW what args to pass
allowed-tools: Claude Code uses this to authorize tools (iter 104)
Without `description:` the skill is invisible in tool registries.
Without `argument-hint:` the agent guesses at args and fails.
WHAT SHIPPED — smoke step 17z68
For each skill:
- description: ≥ 20 chars (catches stub descriptions like "TODO" or "X")
- argument-hint: field MUST exist (empty quoted-string is OK for no-arg
skills, but the FIELD must be declared so the agent knows there's
no surprise required positional arg)
If a future iter:
- Drops description → "harness-foo-description-too-short:0"
- Writes a stub → "harness-foo-description-too-short:3" (e.g. 'TODO')
- Drops argument-hint → "harness-foo-no-argument-hint"
The 20-char minimum is arbitrary but conservative: real descriptions
in current skills are 100-400 chars. 20 catches genuine stubs without
forcing verbosity.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-105)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (8 forward) |
| Target → Source (1 reverse) |
| Internal consistency (3, iters 103-105) |
| skill dir | SKILL.md frontmatter name | 103 |
| SKILL.md | allowed-tools content | 104 |
| SKILL.md | description + argument-hint | 105 | ◀ this iter
Twelve smoke-gated surfaces.
GATES THIS ITER
smoke: 107 → 108 (step 17z68 green)
contract: 8 skills × description + argument-hint each verified
SKILL-FRONTMATTER 3-FIELD GATE COMPLETE
name ← iter 103
description ← iter 105
argument-hint ← iter 105
allowed-tools ← iter 104
All four Claude-Code-recognized frontmatter fields for skill
discovery+invocation are now smoke-gated.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
5ca3c857c5 |
test(metaharness): SKILL.md allowed-tools must be non-empty + known (iter 104)
Each SKILL.md frontmatter has an `allowed-tools:` field that Claude
Code reads to decide which tools an agent invoking the skill may
call. If the field is missing or empty:
- Agents can't run the skill (no tools authorized)
- The skill becomes dead documentation that LOOKS callable but
silently fails when invoked
Iter 87's broader skill-audit ran during init/upgrade flows. iter 104
codifies the assertion at smoke-time for the metaharness skills
specifically.
WHAT SHIPPED — smoke step 17z67
For each subdirectory of skills/:
- Assert `allowed-tools:` field exists in SKILL.md
- Assert the value after the colon is non-empty
- Assert each comma-separated tool name matches the known set:
Bash | Read | Write | Edit | MultiEdit | Glob | Grep
| Task | TaskCreate
Current state: all 8 metaharness skills use `allowed-tools: Bash`
(they each shell out to a script). The gate fires if a future iter:
- Drops the field → "harness-foo-no-allowed-tools-field"
- Empties it → "harness-foo-allowed-tools-empty"
- Adds an unknown tool → "harness-foo-unknown-tool:GodMode"
Floor count ≥6 protects against mass-delete of skills.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-104)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (8 forward) |
| Target → Source (1 reverse, iter 102) |
| Internal consistency (2, iters 103 + 104) |
| skill dir | SKILL.md frontmatter | 103 |
| SKILL.md | allowed-tools content | 104 | ◀ this iter
Eleven smoke-gated surfaces.
GATES THIS ITER
smoke: 106 → 107 (step 17z67 green)
contract: 8 skills × 1 known tool each verified
KNOWN-TOOL ENUM
The `KNOWN_TOOLS` regex is conservative: only the tools currently
used by metaharness skills + commonly-needed alternates. If a
future iter wants to add Task tool authorization (e.g., for a skill
that spawns sub-agents), this gate fires "unknown-tool:Task" and
forces the maintainer to deliberately extend the regex. That's the
intended friction.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
3c9eed9e6b |
test(metaharness): SKILL.md frontmatter name ↔ directory name (iter 103)
Skills are auto-discovered by Claude Code via the .claude/skills
pattern. Each directory under skills/ is a skill; tooling that maps
dir → skill-id assumes the SKILL.md's frontmatter `name:` field
matches the directory name.
Drift scenario:
- Directory: plugins/ruflo-metaharness/skills/harness-similarity/
- SKILL.md: `name: harness-foo` (renamed without changing dir)
- Result: agents listing skills by directory see "harness-similarity"
but tooling looking up by frontmatter sees "harness-foo".
Two truths, conflicting answers, no test catches it.
WHAT SHIPPED — smoke step 17z66
For each subdirectory of skills/:
- Extract `name:` frontmatter from SKILL.md
- Assert it equals the directory basename
Floor count ≥6 (current set is 8: harness-drift-from-history,
harness-genome, harness-mcp-scan, harness-mint, harness-oia-audit,
harness-score, harness-similarity, harness-threat-model). If a
mass-delete drops below 6, the floor catches it.
If a future iter renames the directory without updating the
frontmatter (or vice versa):
→ "harness-foo-name-mismatch:harness-bar"
The output names BOTH ends of the mismatch so the fixer knows which
to align.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-103)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (8 forward, iters 73-94) |
| Target → Source (1 reverse, iter 102: orphaned-script gate) |
| Internal consistency (NEW, iter 103) |
| skill dir | SKILL.md frontmatter | 103 | ◀ this iter
Ten cross-reference surfaces now smoke-gated.
GATES THIS ITER
smoke: 105 → 106 (step 17z66 green)
contract: 8 skill directories × 8 frontmatter-name matches verified
OBSERVATION
Iters 102 + 103 explore two new directions beyond forward references:
- 102: reverse (target lacks any source)
- 103: internal (one source contradicts itself across keys)
Both produce specific failure messages when drift occurs. The
"specific named failure" pattern is the canonical defense against
the drift-class of bug.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
9bf72af589 |
test(metaharness): orphaned-script detector — every .mjs referenced (iter 102)
Iters 89/90/91 gated cross-references from source (SUBCOMMANDS, MCP
handlers, SKILL.md) → target (scripts/*.mjs). Iter 102 gates the
REVERSE direction: every .mjs file in scripts/ must be referenced by
at least one consumer.
The bug-class this catches:
- Someone adds scripts/foo.mjs as an experimental utility
- Never wires it into SUBCOMMANDS or runScript()
- Eventually forgets about it; the file lingers as dead code
- 6 months later a maintainer doing audit asks "what's this for?"
Negative result: the file rots in place, growing stale assumptions
about ruflo's surface.
WHAT SHIPPED — smoke step 17z65
For each .mjs in scripts/:
- Underscore-prefix files (_harness, _similarity, _spike-similarity):
accept either (a) imported by another script OR (b) referenced
by smoke.sh / CI workflows as standalone invocation
- Other files: must appear in SUBCOMMANDS map (CLI), runScript()
call (MCP), SKILL.md inline ref, smoke.sh test target, OR CI
workflow step
Counts ≥20 scripts (sanity floor — script set has been stable around
23 for many iters; below 20 means something was deleted en masse).
DISCOVERY DURING IMPLEMENTATION
First pass had a stricter underscore rule: "must be imported by
another script". _spike-similarity.mjs failed — it's a STANDALONE
regression anchor (iter 35) invoked directly via `node scripts/
_spike-similarity.mjs` from smoke + CI, never imported.
Relaxed the underscore rule to accept either import OR direct-
invocation reference. Documents the convention that underscore
prefix means "internal/utility" but doesn't mandate import-only
usage.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-102)
| Direction | Surface | Iter |
|------------------------|------------------------|------|
| Source → Target (8 surfaces, iters 73-94) |
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| runScript() | scripts/*.mjs | 90 |
| SKILL.md | scripts/*.mjs | 91 |
| MCP enum | SEVERITY_RANK | 92 |
| MCP tool names | CLAUDE.md catalog | 93 |
| SUBCOMMANDS keys | CLAUDE.md catalog | 94 |
| CI workflow refs | tripwire scripts | 84 |
| Target → Source (REVERSE, iter 102) |
| scripts/*.mjs | at least one consumer | 102 | ◀ this iter
Nine cross-reference surfaces now smoke-gated. A script that gets
added without being wired up surfaces as `orphaned-foo.mjs` in smoke.
GATES THIS ITER
smoke: 104 → 105 (step 17z65 green)
contract: 23 scripts × at least 1 reference each verified
the floor (≥20) tolerates intentional pruning while catching
accidental mass-delete
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
196b06496e |
test(metaharness): JSON-output gate covers all 10 scripts (iter 101)
Iter 88's family-wide JSON-output contract gate covered 8 scripts.
Two were deliberately omitted because they need multi-file args:
audit-trend (--baseline + --current) and similarity (--a + --b).
Iter 88's commit comment said "covered by roundtrip Stages 1-12
instead" — true for happy path, but the JSON-PARSE contract is a
distinct invariant that the roundtrip doesn't directly verify (it
parses the output but via a regex-extract, not strict parse-of-file).
Iter 101 fills the gap.
WHAT SHIPPED — smoke step 17z51 extension + new step 17z64
- Two new synthetic fixtures in the tmpdir:
/tmp/.../fixA.json — similarity input shape (score + genome)
/tmp/.../fixB.json — audit-trend input shape (composite +
components + fingerprint)
- SCRIPT_TESTS array extended:
similarity --a fixA.json --b fixA.json --format json
audit-trend --baseline fixB.json --current fixB.json --format json
- Both produce JSON.parse-able output
Smoke step 17z64 anchors:
- similarity + audit-trend fixture invocations in SCRIPT_TESTS
- The fixture-emit heredocs (TMP/fixA.json + TMP/fixB.json)
- Fixture shape requirements (agent_topology, composite.worst)
COVERAGE MATRIX (post-iter-101)
| Script | Iter 88 | Iter 101 |
|---------------------|:-------:|:--------:|
| oia-audit | ✓ | ✓ |
| score | ✓ | ✓ |
| genome | ✓ | ✓ |
| mcp-scan | ✓ | ✓ |
| threat-model | ✓ | ✓ |
| audit-list | ✓ | ✓ |
| bench-similarity | ✓ | ✓ |
| bench-parse-mcp-scan| ✓ | ✓ |
| similarity | | ✓ | ◀ NEW
| audit-trend | | ✓ | ◀ NEW
All 10 JSON-emitting scripts now contract-verified by parse-roundtrip.
The two remaining scripts NOT covered (drift-from-history, mint) are
intentionally excluded:
- drift-from-history: slow (~25s on real audit-list) — covered by
Stage 8 (iter 68) instead
- mint: filesystem-write protection — covered by graceful-
degradation drill (iter 19)
GATES THIS ITER
smoke: 103 → 104 (step 17z64 green)
contract: 10 scripts × JSON parse-roundtrip pass
The family-wide JSON-output contract is now exhaustively gated for
all scripts that downstream code (CI, ops, agents) consumes via JSON
parsing.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
3b3018b48a |
docs(adr): 100-iter milestone — ADR-150 reflects iters 83-99 (iter 100)
The symbolic 100th iteration of /loop on ADR-150's feat/metaharness-integration-research branch. This iter brings the canonical engineering record current through iter 99 and adds a retrospective section. WHAT'S NEW IN THE ADR 1. Status header: "Phase 3 §3.1 ✅ iters 33-82" → "iters 33-99" 2. Date marker: "82 iterations" → "**100 iterations of /loop**" 3. New section "Iters 83-99 — cross-reference integrity + observability" with three sub-arcs: - Cross-reference integrity matrix (iters 73, 84, 89-94) Eight surfaces where source A references source B by name. Each smoke-gated; specific failure message when references drift. - Fast-path observability arc (iters 95-99) The iter-66/67 fast-paths (~14x and ~19x speedups) are now observable at every consumption layer — JSON, CLI table, CI summary, CI dispatcher round-trip (shape + wall budget). - Anti-regression pattern reinforcement (iters 83-92) Positive-presence guards, perf characterization, JSON-output contract gate, MCP enum alignment. 4. Fleet status (post-iter-99): - Smoke step count: 85 → 102 - MCP runtime: 117 → 120 assertions - 8 cross-reference integrity surfaces - 3 negative-guard invariants - 3 compat tripwires - 2 bench scripts in CI with artifact tracking - 4 CI workflows - ~290 total ruflo-metaharness smoke assertions 5. 100-iter retrospective — names two reusable patterns: (a) BUG-DISCOVERY VIA END-TO-END TESTS Iter-47 schema bug hid for 9 iters. Iter-50 dead code hid for 28. Iter-87 JSON contamination hid for 5. All surfaced only when a test exercised the REAL chain — not when smoke greps verified source markers. Lesson: smoke-grep on source is a COMPLEMENT to, not a substitute for, real-output verification. (b) CROSS-REFERENCE INTEGRITY When source A references source B by name, drift is the default without an explicit gate. Eight surfaces are now gated; each produces a specific named failure when the reference rots. SMOKE STEP 17z63 anchors: - "iters 33-99" status marker - "100 iterations of /loop" date marker - "Iters 83-99 — cross-reference integrity" section header - Cross-reference integrity matrix table - Fast-path observability arc header - 100-iter retrospective section header - Fleet status post-iter-99 marker PLUS iter-60 + iter-83 smoke steps relaxed for forward-compat: - "Phase 3 §3.1 ✅ iters 33-N" with N as any number - "<N> iterations" with N as any digit-string or word-form Future ADR refreshes won't break these gates. GATES THIS ITER smoke: 102 → 103 (step 17z63 green; 17z23 + 17z46 grep relaxed) fleet: 33/33 (doc-only iter) no code changes — pure documentation milestone THE 100-ITER ARC Phase 1 (iters 1-3) — MVP shipped Phase 2 (iters 4-32) — expansion, eject, oia-audit composite Phase 3 §3.1 (iters 33-99) — Genome Similarity Search: - 35-41: spike + production module + CLI skill + MCP tool + CI - 42-44: bug-hardening arc (dispatcher flags, wrapper semantic) - 45-52: doctor + MCP runtime + roundtrip + tripwire scaffolding - 53-67: drift-from-history primitive + 3-tier fastpath ladder - 68-79: roundtrip stages 8-12 + alert-on-new-severity + cron auto - 80-99: compat tripwire family + cross-reference integrity matrix + observability arc + dispatcher CI cross-checks ADR-151 Phase 3 §3.2-§3.5 remain scope-locked. KRR retrain still blocked on production data. Two upstream issues filed at ruvnet/agent-harness-generator (#15, #16); both still OPEN as of iter 99. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
16150fd116 |
ci(metaharness): dispatcher round-trip enforces wall budget (iter 99)
iter-98 added a CLI-dispatcher round-trip for drift-from-history but
only asserted JSON shape. iter-67 promised ~19x speedup via
--baseline-file; iter-98 didn't verify that promise actually holds
in CI. A regression that reverts the fast-path to slow would still
produce the correct JSON shape (path field, skippedAuditList) but
take ~26s instead of ~1.4s.
WHAT SHIPPED — metaharness-ci.yml iter-98 step extended
After the iter-95 path + iter-66 skippedAuditList assertions:
WALL=$(node -e "
const j = JSON.parse(\`\$OUT\`);
console.log(j.timing?.parallelWallMs ?? 0);
")
if [ "$WALL" -gt 30000 ]; then
echo "FAIL: dispatcher fast-path wall ${WALL}ms > 30000ms (regression?)"
exit 1
fi
Budget rationale: iter-67 measured 1.4s on Apple Silicon. CI runners
are ~3-5x slower. 30s budget gives ~5x headroom from the local
baseline while still catching a 10x regression (slow path's 26s
fits inside the budget — so this catches the regression to slow path
ONLY when CI is already slow, but most CI runs are <30s for the
fast path).
A stricter budget (5s, 10s) would risk false positives on slow CI
runners. 30s is the lowest budget that's reliably safe across
GitHub-Actions runner classes.
The pass-message now includes the actual measured wall:
✓ drift-from-history dispatcher round-trip green (fast-path via CLI; wall 1342ms)
So even when the budget passes, the Actions UI shows what was measured —
ops users can spot creeping regressions (e.g., wall trending from 1.4s
to 5s over weeks) before they breach the 30s ceiling.
SMOKE STEP 17z62 anchors:
- Wall-budget failure message literal
- parallelWallMs extraction in node -e
- WALL variable assignment
GATES THIS ITER
smoke: 101 → 102 (step 17z62 green)
yaml: python yaml.safe_load passes
ci: iter-98 step now also gates wall < 30s
DISPATCHER-LAYER ARC + PERF (iters 42-66-67-95-97-98-99)
42 — dispatcher flag-drop fix
66 — --baseline-key fast-path (~14x)
67 — --baseline-file fast-path (~19x)
95 — timing.path field
97 — weekly cron summary surfaces path
98 — CI dispatcher JSON shape contract
99 — CI dispatcher wall budget ◀ this iter
The dispatcher path is now contract-anchored at BOTH the shape level
(iter 98) AND the performance level (iter 99). A regression in either
fails CI with a specific named error.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
fccee28486 |
ci(metaharness): drift-from-history dispatcher CI cross-check (iter 98)
The metaharness-real-data job tests the score subcommand through the
CLI dispatcher (iter-47 pattern). But the drift-from-history dispatcher
path was never CI-exercised. The iter-66/67 fast-paths + iter-95
timing.path field rely on:
- iter-42's dispatcher flag-roundtrip
- drift-from-history's synthetic listResult logic
- audit-trend's --baseline file dispatch
A regression in ANY of those would survive smoke (which tests at the
script layer) but fail real ops use (which goes through the CLI).
Iter 98 adds the missing dispatcher round-trip.
WHAT SHIPPED — metaharness-ci.yml, metaharness-real-data job
New step at job end:
1. Generate fresh audit as baseline-file input
2. Invoke through CLI: `node bin/cli.js metaharness drift-from-history
--baseline-file /tmp/drift-baseline.json --dry-run --format json`
3. Assert output contains `"path": "file"` (iter-95 + iter-67 contract)
4. Assert output contains `"skippedAuditList": true` (iter-66 contract)
VERIFIED LOCALLY
$ node bin/cli.js metaharness drift-from-history --baseline-file /tmp/baseline.json --dry-run --format json
"skippedAuditList": true,
"path": "file"
"verdict": "unchanged"
WHAT THIS CATCHES THAT EXISTING TESTS DON'T
smoke 17z23+ tests test-pipeline-roundtrip.mjs at the SCRIPT layer
smoke 17z61 tests the SAME path through the CLI DISPATCHER
A regression in:
- iter-42 dispatcher flag-drop (silent for 6 iters before iter 42 found it)
- iter-67 baseline-file branch (synthetic listResult drift)
- iter-95 timing.path derivation
would now surface in the metaharness-real-data job as
"FAIL: dispatcher did not propagate baseline-file or timing.path missing".
PARALLEL TO ITER 47'S CROSS-CHECK PATTERN
iter 47 — score dispatcher round-trip:
$ node bin/cli.js metaharness score --path . | grep harnessFit
iter 98 — drift-from-history dispatcher round-trip:
$ node bin/cli.js metaharness drift-from-history --baseline-file X
| grep '"path": "file"'
Both verify a specific CLI surface produces specific JSON keys.
GATES THIS ITER
smoke: 100 → 101 (step 17z61 green)
ci: new dispatcher round-trip step in metaharness-real-data job
yaml: python yaml.safe_load passes
runtime: dry-run produces all 3 expected JSON markers locally
ITER 42-66-67-95-97-98 DISPATCHER-LAYER ARC
42 — dispatcher flag-drop fix
66 — --baseline-key fast-path
67 — --baseline-file fast-path
95 — timing.path derived field
97 — weekly cron summary surfaces path
98 — CI dispatcher round-trip locked at job-level ◀ this iter
The dispatcher → drift-from-history path is now contract-anchored at
the CI level, not just the smoke level.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
de8b3e44e8 |
ci(metaharness): weekly cron summary surfaces timing.path (iter 97)
iter 96 surfaced timing.path in the drift-from-history CLI's
human-readable table mode. The same field is also useful in the
weekly cron's GITHUB_STEP_SUMMARY — it confirms which fastpath the
cron actually used.
Why this matters operationally:
- iter 79 wired the cron to use --baseline-file fastpath
- Expected wall: ~1.4s (iter 67's measured baseline)
- If a future bug accidentally falls through to the slow path,
wall would balloon to ~26s but the WORKFLOW WOULD STILL PASS
(just slower). The summary's Path: line makes the regression
visible at the Actions-UI level.
WHAT SHIPPED — .github/workflows/oia-audit-weekly.yml
Drift step's summary block extended:
## Drift vs prior week: `near-identical` (similarity=1)
Alert status: `OK`
Path: `file (wall 1342ms)` ◀ NEW (iter 97)
The PATH_TAKEN extraction reads:
- timing.path ('file' / 'key' / 'slow' / 'unknown' fallback)
- timing.parallelWallMs ('?' fallback if missing)
If a future drift-from-history version drops these fields, the
summary still renders cleanly with 'unknown' / '?'.
SMOKE STEP 17z60 anchors:
- PATH_TAKEN variable definition
- t.path || 'unknown' fallback
- t.parallelWallMs extraction
- echo "Path:" line written to $GITHUB_STEP_SUMMARY
GATES THIS ITER
smoke: 99 → 100 (step 17z60 green; 100-step milestone)
yaml: python yaml.safe_load passes
cron: summary now includes path observability
ITER 65 + 66 + 67 + 79 + 95 + 96 + 97 — FAST-PATH OBSERVABILITY ARC
65 — timing.parallelSpeedup field
66 — --baseline-key flag (~14x speedup)
67 — --baseline-file flag (~19x speedup)
79 — weekly cron uses --baseline-file
95 — derived timing.path field
96 — CLI table mode surfaces Path:
97 — weekly cron summary surfaces Path: ◀ this iter
The fast-path is now observable at every consumption layer: JSON,
table mode, and the CI Actions summary.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
1e856ab967 |
feat(metaharness): drift-from-history table surfaces path + wall (iter 96)
iter 95 added timing.path to the JSON payload. The human-readable table mode (--format != json) didn't show it — users running drift detection interactively had no signal whether the fast-path activated. WHAT SHIPPED — drift-from-history.mjs table mode New line between "Current:" and the Structural-similarity block: # drift-from-history Baseline: file:/tmp/baseline.json Current: 2026-06-17T04:06:50.180Z Path: file (wall 659ms) ◀ NEW (iter 96) Structural similarity: 1 (near-identical) Distance: 0 ✓ similarity ≥ 0.95 — OK The wall-clock annotation makes the speedup visible at a glance: - Path: file (wall 659ms) — iter-67 fastest path - Path: key (wall ~2s) — iter-66 audit-list-skip - Path: slow (wall ~26s) — default, audit-list ran A user running `drift-from-history` regularly sees their actual performance instead of having to JSON-parse the timing field. GATES THIS ITER smoke: 98 → 99 (step 17z59 green) regression: roundtrip 66/66 (no behavior change on default JSON path) ITER 95 + 96 OBSERVABILITY ARC 95 — timing.path derived field in JSON payload 96 — timing.path + wall surfaced in human table mode The fast-path observability story is now complete at both the machine-readable (iter 65 + 95) and human-readable (iter 96) layers. Co-Authored-By: RuFlo <ruv@ruv.net> |
||
|
ruv
|
b87d2b58f4 |
feat(metaharness): drift-from-history exposes derived timing.path label (iter 95)
Pre-iter-95 consumers had to combine two booleans to know which path
ran:
- skippedAuditList: bool (iter 66)
- usedBaselineFile: bool (iter 67)
- Implied path:
usedBaselineFile=true → 'file'
skippedAuditList=true, file=false → 'key'
both false → 'slow'
The 3-way derived label was inferable but every consumer (CI workflow,
agent code, ops scripts) had to compute it locally. Now they read
`timing.path` directly.
WHAT SHIPPED
drift-from-history.mjs — timing payload extended:
timing.path: 'slow' | 'key' | 'file'
derived: usedBaselineFile ? 'file' : (skippedAuditList ? 'key' : 'slow')
The two underlying booleans STAY in the payload (don't break iter-65/66/67
consumers). The new field is purely additive — a consumer-ergonomics
improvement.
VERIFIED VIA SMOKE STEP 17z58
- --baseline-file → timing.path === 'file' ✓
- --baseline-key → timing.path === 'key' (not gated at smoke level
because key resolution requires memory access; covered by Stage 9
+ manual verification at iter-95 commit time)
- default (slow) → timing.path === 'slow' (implicit by the truth
table; if either fast-path were active, file/key would be set)
GATES THIS ITER
smoke: 97 → 98 (step 17z58 green)
regression: roundtrip 66/66 (no behavior change on default path);
Stage 8 + 9 see timing.path field but don't assert it
(consumer-ergonomic, not contract-required)
CONSUMER VALUE
Before:
if (data.timing.usedBaselineFile) { ... }
else if (data.timing.skippedAuditList) { ... }
else { ... }
After:
switch (data.timing.path) {
case 'file': ...
case 'key': ...
case 'slow': ...
}
The booleans stay for backward compat. New consumers SHOULD use
timing.path. The truth-table comment in the source documents the
mapping.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
3bcebcfb39 |
test(metaharness): every CLI subcommand documented in CLAUDE.md (iter 94)
Companion to iter-93's MCP-tool documentation gate. CLAUDE.md also
serves as a CLI catalog — each subcommand from metaharness.ts's
SUBCOMMANDS map should appear as `npx ruflo metaharness <X>` so users
can discover it via the docs.
Pre-iter-94 coverage: only iter-40 verified `similarity` appears in
CLAUDE.md. The other 9 subcommands had no documentation gate. If a
future iter added a `metaharness compare` subcommand but forgot to
update CLAUDE.md, the CLI tool would exist but be undiscoverable.
WHAT SHIPPED — smoke step 17z57
- Extract SUBCOMMANDS keys from metaharness.ts (handles quoted +
unquoted forms: 'mcp-scan' vs score)
- For each key, grep `npx ruflo metaharness <key>` in CLAUDE.md
- Lock count at 10
BSD-sed portability fix:
- Pre-iter-94 used `\s` in sed expressions (GNU-only)
- Macos BSD sed treats `\s` literally → substitution silently fails
- Replaced with `[ \t]` character class throughout
DISCOVERY DURING IMPLEMENTATION
First-pass smoke regex used `\s` in BSD sed and silently produced no
substitution. The smoke step then "passed" by accumulating ALL 20
lines (10 keys + 10 values) as "subcommand keys" — each then failed
the CLAUDE.md grep with names like `'audit-list':-not-in-claude-md`.
The output exposed the bug. Without the iter-94 mismatch, the iter-89
smoke step's similar regex would have silently passed (its sed didn't
use `\s` in the substitution, only in the grep).
Lesson: BSD/GNU portability is a real concern even when "the test
passes" — silent substitution failures produce confused-looking diff
output that takes a beat to recognize.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-94)
| Source | Target | Iter |
|------------------------------|----------------------|------|
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP handler runScript() | scripts/*.mjs | 90 |
| SKILL.md inline refs | scripts/*.mjs | 91 |
| MCP enum | SEVERITY_RANK keys | 92 |
| MCP tool names | CLAUDE.md catalog | 93 |
| SUBCOMMANDS keys | CLAUDE.md catalog | 94 | ◀ this iter
| CI workflow run commands | tripwire scripts | 84 |
Eight cross-reference surfaces now smoke-gated.
GATES THIS ITER
smoke: 96 → 97 (step 17z57 green; portability fix surfaced)
contract: 10 CLI subcommands × 10 CLAUDE.md catalog entries verified
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
f79ba66f4c |
test(metaharness): every MCP tool documented in CLAUDE.md (iter 93)
CLAUDE.md is the agent-facing tool catalog. When agents are deciding
which MCP tool to call for a task, they read CLAUDE.md's listing. If
a future iter adds a tool to metaharness-tools.ts but forgets to
update CLAUDE.md, the tool exists in the registry but is INVISIBLE
to agents browsing the documentation.
The bug-class:
- metaharness-tools.ts has `name: 'metaharness_NEW_TOOL'`
- test-mcp-tools.mjs registers it and asserts the contract
- smoke 17z (MCP runtime) passes
- But CLAUDE.md still lists 9 tools, not 10
- Agents reading CLAUDE.md don't know to call metaharness_NEW_TOOL
- The tool is dead documentation
Iter 93 gates against this.
WHAT SHIPPED — smoke step 17z56
- Extract every `name: 'metaharness_X'` from metaharness-tools.ts
- For each, grep `mcp__claude-flow__metaharness_X` in CLAUDE.md
(the agent-facing tool-name format)
- Lock count at 9 (mint deliberately excluded per iter-73 §Sandboxing)
If a future iter:
- Adds a 10th MCP tool without CLAUDE.md update → "tool-X-not-in-claude-md"
- Removes a tool without CLAUDE.md update → mismatch
- Lists mint in CLAUDE.md as mcp__... → enforced via iter-73's anti-mint guard
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-93)
| Source | Target | Iter |
|------------------------------|-----------------------|------|
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP handler runScript() | scripts/*.mjs | 90 |
| SKILL.md inline refs | scripts/*.mjs | 91 |
| MCP enum | SEVERITY_RANK keys | 92 |
| MCP tool names | CLAUDE.md catalog | 93 | ◀ this iter
| CI workflow run commands | tripwire scripts | 84 |
Seven cross-reference surfaces now smoke-gated. The documentation
layer is no longer a dangling reference — every MCP tool registered
in source has a corresponding entry in the human/agent-facing catalog.
GATES THIS ITER
smoke: 95 → 96 (step 17z56 green)
contract: 9 MCP tools × 9 CLAUDE.md catalog entries verified
OBSERVATION
Documentation-as-contract is now a smoke-anchored idea, not just a
convention. ADR-150 and CLAUDE.md both have specific entries that
smoke verifies. Future contributors who update one without the other
get an immediate, named failure pointing at the missing surface.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
45c9650513 |
test(metaharness): MCP enum + SEVERITY_RANK vocabulary aligned (iter 92)
Two declarations of the same severity vocabulary exist:
1. _harness.mjs::SEVERITY_RANK keys (clean, info, low, medium, warn,
high, error, critical)
2. metaharness-tools.ts::metaharness_drift_from_history's
alertOnNewSeverity.enum
If a future iter adds 'urgent' to SEVERITY_RANK but forgets the MCP
enum (or vice versa), users see "alertOnNewSeverity must be one of..."
rejection of a supposedly-valid severity. The two sources drift
silently — no test catches it because each smoke step only looks at
ONE source.
WHAT SHIPPED — smoke step 17z55
- Extract SEVERITY_RANK keys via awk-range + grep
(8 keys: clean, info, low, medium, warn, high, error, critical)
- Extract enum values via grep on the alertOnNewSeverity bracket
(7 values: info, low, medium, warn, high, error, critical)
- Cross-check:
✓ Every enum value is in SEVERITY_RANK (subset)
✓ The only SEVERITY_RANK key NOT in the enum is 'clean'
(alerting on 'clean' is meaningless)
If a future iter:
- Adds 'urgent: 5' to SEVERITY_RANK only → "enum-missing-keys-mismatch"
- Adds 'urgent' to enum only → "enum-urgent-not-in-rank"
- Adds 'clean' to enum → mismatch (clean intentionally excluded)
DISCOVERY DURING IMPLEMENTATION
First-pass regex extracted unintended matches:
- `[a-z]+: [0-9]` matched 'Mode' (from recommendedMode) → 'ode'
- `'[a-z]+'` on the enum line captured 'string' from `type: 'string'`
Final regex: scoped to the SEVERITY_RANK literal range (awk pattern)
+ scoped to the bracket contents within `enum: [...]`. Documents the
right scoping pattern for future contributors who add similar gates.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-92)
| Source | Target | Iter |
|--------------------------|-------------------------|------|
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP handler runScript() | scripts/*.mjs | 90 |
| SKILL.md inline refs | scripts/*.mjs | 91 |
| MCP enum | SEVERITY_RANK keys | 92 | ◀ this iter
| CI workflow run commands | tripwire scripts | 84 |
Six cross-references now smoke-gated.
GATES THIS ITER
smoke: 94 → 95 (step 17z55 green)
contract: 7 enum values × 8 SEVERITY_RANK keys subset+complement check
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
8b3d11215a |
test(metaharness): SKILL.md script refs point at existing files (iter 91)
Third application of iters 89/90's cross-reference integrity pattern.
SKILL.md files embed paths like `scripts/foo.mjs` in their text. If a
future iter renames a script:
- SKILL.md still renders (markdown is permissive)
- The relative link 404s (a hidden break in the documentation)
- Ops users hunting for the implementation hit a dead end weeks
later, after the original committer has forgotten the context
WHAT SHIPPED — smoke step 17z54
Greps every SKILL.md for `scripts/[a-z_-]+\.mjs` references, asserts
each target exists in plugins/ruflo-metaharness/scripts/. Locks count
at ≥5 (at least one SKILL.md per skill references its implementation;
several reference multiple scripts).
If a future iter:
- Renames a script → "skill-ref-foo.mjs-missing"
- Deletes the SKILL.md set → "skill-ref-count-too-low:N"
The count floor (≥5 instead of exactly N) tolerates SKILL.md authors
adding cross-references between skills, which is legitimate and
shouldn't force a smoke update.
CROSS-REFERENCE INTEGRITY MATRIX (post-iter-91)
| Source | Target | Iter |
|--------------------------|-------------------------|------|
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP handler runScript() | scripts/*.mjs | 90 |
| SKILL.md inline refs | scripts/*.mjs | 91 | ◀ this iter
| CI workflow run commands | tripwire scripts | 84 |
Five distinct cross-reference surfaces are now smoke-gated. A rename
in scripts/ that misses ANY of the four reference sources produces a
specific failure message naming the missing file at smoke-time, not
a confused user weeks later.
GATES THIS ITER
smoke: 93 → 94 (step 17z54 green)
contract: 5+ SKILL.md script refs all resolve to existing files
THE FAMILY-WIDE INTEGRITY GATE
Smoke now anchors:
- Positive references: 4 surfaces × cross-check
- Negative references: 3 invariants (mint not-in-MCP, no from-repo,
no static metaharness imports)
Together they form a shape-of-codebase contract: smoke fails iff any
required reference goes stale OR any forbidden surface appears.
Co-Authored-By: RuFlo <ruv@ruv.net>
|
||
|
ruv
|
0212ec9690 |
test(metaharness): MCP-tool runScript() refs point at existing scripts (iter 90)
Companion to iter-89's SUBCOMMANDS check. metaharness-tools.ts has 9
MCP-tool handlers, each calling runScript('<name>.mjs', args). If a
future iter renames a script but forgets to update its handler, the
agent-callable MCP surface fails at runtime with "Script not found".
THE THREE-WAY CONTRACT (post-iter-90, fully anchored)
CLI dispatcher: SUBCOMMANDS['X'] = 'X.mjs' (iter 73 description ↔ key)
│ (iter 89 value ↔ file)
v
scripts/X.mjs ← file exists
^
│
MCP handler: runScript('X.mjs', args) (iter 90 ref ↔ file)
Every reference from BOTH the CLI dispatcher AND the MCP wrapper
now has its target verified at smoke-time.
WHAT SHIPPED — smoke step 17z53
Parses every runScript('foo.mjs', ...) call site in metaharness-tools.ts
via regex:
/runScript\('([a-z-]+\.mjs)'/
For each unique reference: asserts the .mjs file exists in
plugins/ruflo-metaharness/scripts/. Locks count at 9 (one per MCP
tool; mint deliberately excluded per ADR-150 §Sandboxing).
If a future iter:
- Renames a script → "mcp-script-foo.mjs-missing"
- Adds a 10th MCP tool → "mcp-script-count-stale:10-expected-9"
- Removes a tool → "mcp-script-count-stale:8-expected-9"
GATES THIS ITER
smoke: 92 → 93 (step 17z53 green)
contract: 9 MCP runScript() refs × 9 file-existence checks all pass
ITER 73 + 84 + 89 + 90 CROSS-REFERENCE INTEGRITY MATRIX
| Reference source | Reference target | Iter |
|-----------------------------|------------------|------|
| description string | SUBCOMMANDS keys | 73 |
| SUBCOMMANDS values | scripts/*.mjs | 89 |
| MCP handler runScript() | scripts/*.mjs | 90 |
| CLI workflow run commands | tripwire scripts | 84 |
Four cross-references now produce specific failure messages when
the references drift.
PATTERN: SAME REGEX, DIFFERENT SOURCE
Iters 89 + 90 both:
1. Parse a regex match from a source file (the reference list)
2. For each match, assert a target file exists
3. Assert the count matches the documented contract
Future iters that add new cross-reference surfaces (e.g., a third
MCP-tool registry, a SKILL.md → script-path mapping) can adopt this
3-step shape. The pattern is now self-documenting.
Co-Authored-By: RuFlo <ruv@ruv.net>
|