Commit Graph

2 Commits

Author SHA1 Message Date
rUv 67ff9898c0 fix: resolve current runtime and verification defects for v3.32.36 (#2850)
* fix(cli): make runtime status and learning signals truthful

* fix(metaharness): accept padded scan severities

* fix(codex): make core hooks and status skill native

* fix(security): make witness verification hermetic

* fix(metrics): expose grounded learning outcomes

* fix(hooks): deduplicate project and plugin events

* fix(verification): preserve security audit payloads

* fix(runtime): stabilize dual memory skills and MCP schemas

* fix(cli): harden helper signing and reflexion health

* test(release): align helper and container invariants

* chore(release): prepare v3.32.36

* test(funnel): align gates with cold-start seed pool

* fix(ci): document MCP environment precedence
2026-07-29 15:08:51 -04:00
ruv ca56902b27 test(metaharness): upstream mcp-scan text-format compat tripwire (iter 80)
Iter 50's parseMcpScanText reverse-engineered `harness mcp-scan` text
output to extract structured findings. If upstream changes the format
(drops `[]` markers, switches to JSON, changes the Result: regex), the
parser silently returns empty findings and iter-49's introduced/cleared
diff regresses to the original dead-code state.

This iter adds a compat tripwire mirroring scripts/check-metaharness-
compat.mjs (which protects the @metaharness/router API surface).

WHAT SHIPPED

1. scripts/check-mcp-scan-format.mjs — runs `harness mcp-scan` against
   ruflo and asserts 4 invariants:

      ✓ At least one `  [SEV] message` line in expected regex shape
      ✓ Result line matches `Result: SEV (N finding, M high)`
      ✓ parseMcpScanText loadable (sanity)
      ✓ parseMcpScanText extracts ≥ 1 finding from real output

   Exit 0 — invariants intact OR metaharness not installed (graceful skip)
   Exit 1 — invariant violated, parser will silently break
   Exit 2 — unexpected error

   The "extract ≥1 finding from real output" assertion is the
   load-bearing check. It catches drift the regex above might miss
   (e.g., if upstream switched to identical-looking but slightly
   different whitespace).

2. .github/workflows/metaharness-ci.yml — new step in the
   `metaharness-real-data` job runs the tripwire BEFORE the roundtrip.
   If upstream format drifted, CI fails with a clear "format invariant
   violated" message before chasing downstream symptoms.

3. plugins/ruflo-metaharness/scripts/smoke.sh — step 17z43 anchors:
   - tripwire script exists + parses
   - All 3 invariant text markers (finding/result/parser-roundtrip)
   - CI workflow references the script
   - Runtime: tripwire passes against the locally-installed metaharness

WHY TWO TRIPWIRES NOW EXIST

   scripts/check-metaharness-compat.mjs (iter 12)
      → Protects @metaharness/router public API (constructor, route(),
         fromExamples(), TrainedRouter.fromJSON())

   scripts/check-mcp-scan-format.mjs (THIS ITER)
      → Protects `harness mcp-scan` text format (regex + parser-roundtrip)

These cover different upstream surfaces. The router compat-check ran
at the API level; the new check is at the CLI-output level. Both are
necessary because ruflo depends on both surfaces in different ways.

VERIFIED LOCALLY
   $ node scripts/check-mcp-scan-format.mjs
   ✓ All 4 invariants intact.

GATES THIS ITER
   smoke:    82 → 83 (step 17z43 green)
   ci:       new pre-roundtrip step in metaharness-real-data job
   runtime:  4 format invariants pass against metaharness@0.1.14

PROTECTION RATIONALE

Iter 50 was a downstream workaround for issue ruvnet/agent-harness-
generator#16 (`harness mcp-scan` emits text only with --json). When
upstream eventually fixes that issue and emits structured JSON, the
parser's regex-based extraction will STILL work (the parser falls back
gracefully). But the tripwire will fail on the FIRST run after upstream
adds the JSON output AND drops the text format — which is a real risk
because upstream is in v0.1.x.

When the tripwire eventually fires, the fix is upstream-aware:
   - If text format dropped → switch parseMcpScanText to read r.json directly
   - If JSON added alongside text → prefer r.json, keep parser as fallback

The tripwire makes that decision visible at the PR that introduces it,
not 4 weeks later when someone notices drift detection is silent.

Co-Authored-By: RuFlo <ruv@ruv.net>
2026-06-16 22:37:57 -04:00