mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
Bo
2ef932b617
## RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated Provenance: running the loop for real across three trains on 2026-09-03 (#1099, #1100, #1101). The spine held; the flow around it did not. A risky-surface design shipped with no premortem and needed six repair passes. The open finding set grew three rounds running inside one class while the id-counting law watched. A same-family PASS and a cross-family FAIL had no tie-break until the caller named one afterwards. Every harness edit orphaned bound evidence with no warning until verify failed. A doc called an egress log published while the repository's `*.log` rule kept it out of the tree, and the same overclaim class came back four rounds. **What changes.** - **Premortem at plan exit.** When the write scope hits a risky surface, one fresh judge reads the frozen plan before Implement; a blocking finding returns NOT_PLANNED with the findings. The caller may declare `premortem: skip`; the report says so. - **A split is recorded, never elected and never closed inside the loop.** The law stands: a risky surface converges only when both legs PASS, and no finding leaves the open set because someone chose a judge. Plan declares `binding_judge` as the caller's disposition for a split that survives repair; it is bound in the plan identity, carried in the report, and never mutates the verdict. On a risky split that survives the repair budget (never after a law stop), one council leg receives a bounded packet marked untrusted and returns per-finding rulings, exactly one per id, which the traversal validates and records under `council.rulings` for the caller's next intent; the verdict and the open finding set stay exactly as repair left them. A council closure mechanism was built, drew a new finding class in three consecutive judge rounds (path containment, digest exclusion, presence-as-evidence, partial closure downgrading a FAIL), and was cut under the train's own class rule rather than hardened again. A non-risky split stays worst-of. - **Finding class in the convergence law.** Findings carry an optional stable `class`. A class carried by a resolved id that reappears on a new id with no surviving prior id carrying it is `class_reopened`, whether or not a classless round sits between; a blank class is an invalid round; a round can carry both a reopened id and a reopened class and stops on either. Repair stops, the traversal returns NOT_PROVEN, and the caller goes back to Plan, because the design is wrong, not the patch. Same rule in `run_once.py` and `workflows/rpi.js`, driven by one shared fixture file, with executed canaries in the cathedral gate. - **Plan identity, and the premortem's risky test.** On a risky scope the Plan carries `binding_judge` and the evidence it expects to orphan; `plan_digest`, when the plan declares one, must equal the script's digest over the acceptance, the write scope, the binding judge, and the intent-source digest, or the traversal stops before Implement. The risky test over a declared scope is glob intersection: witness paths, the risky regexes over the glob with wildcards instantiated, and any literal prefix that reaches a risky root; a bare `**` or `*` is risky. - **Intent is verified before anything is built.** An `intent-snapshot` receipt re-derives the digest and compares the snapshot bytes to the caller intent before premortem or Implement; absent, mis-hashed, or wrong stops the traversal with `plan_identity_mismatch`. The Python reference refuses on the same rule. Declared scopes are canonicalized (repeated `./` collapsed; absolute paths and `..` refused at plan exit), and the four spellings of `tests/**` share one plan digest while `tests/` does not. - **Orphaned evidence is a receipt, not a surprise.** After Implement and after every repair round the workflow runs `scripts/evidence-orphans.sh` over the runtime-derived path union, keyed on the round so a same-set repair still reruns it and appends every scorecard and fixture set whose bound evaluator or skill digests cover them to the check receipts the validator reads, each with a cause (`changed_path`, `digest_drift`, `both`, `skill_changed`). Plan budgets that recapture up front. On today's tree the receipt already lists the 08-16 and 08-26 sets whose bound harness digests moved long ago. - **Doc claims are gated.** The stronger scan already caught one stale claim in the tree (a 2026-06-22 harvest note calling evidence committed after Train 2 had removed that tree); the sentence now says where the evidence went. `docs.claims-tracked` (blocking) requires backticked repo paths in `evals/` and `docs/evals/` markdown to be tracked, and sentences that say published, tracked, or committed to name files that exist. - **Risky scope is a glob intersection, not a heuristic.** One segment-wise `globsIntersect` over one list of risky segment globs answers both the declared-scope question at Plan exit and the changed-path question for the cross-family rule; two heuristic designs missed in opposite directions and were replaced rather than patched. - **Contracts.** rpi gains a judgment dispatch table: risky scope routes to premortem, a risky split to council, an irreversible landing to one-way-door as a caller choice. plan, implement, validate, premortem, council, and learn carry their halves; learn proposes a deterministic check for any finding class that repeats. `docs/architecture/rpi-traversal.md` is the source of record and was updated with the flow. **Evidence on the tip.** Regen check clean; Go bar; full gate green with a HEAD-built binary (the new gate fires and passes); CI's literal bats command green; rpi unit tests; cathedral gate with executed canaries for the class law; routing goldens; token budgets; gate-tightening ratchet; coverage 1 of 12 unchanged. **Judgment, stated plainly.** Six judging rounds, each on a frozen tip. The caller elected Fable 5.1 as the binding validator; its round-6 verdict is PASS with no verdict-driving findings (lows: a council payload outside the schema throws a raw error instead of recording `invalid-rulings`; two stale comments; a dead parameter; a dead fixture element; the contracts say exactly one ruling per finding while the code enforces at most one). The cross-family Codex read of the same tip failed on three input-validation items, which the final commit closes: control characters are now refused on the original scope value with the full Unicode control class and the caller scope is canonicalized in place so one spelling reaches every prompt; the intent receipt no longer forgives a trailing-newline difference; and the Python reference accepts a digest string only from a verifier callback and matches digests with `fullmatch`. One half of one Codex item stays open and is disclosed: the workflow does not itself hash the caller intent's bytes to compare with the plan's declared digest (it relies on the receipt's exact-bytes match plus digest equality); binding that locally means every probe fixture carries a real digest of its intent text, which is the next pass. Also open, pre-existing and outside this train: the risky-surface list names `lib/**` while the preamble every gate sources lives under `scripts/lib/`. Across the rounds the loop behaved as designed once: the council closure mechanism drew a fresh finding class three rounds running and was cut under the train's own class rule instead of hardened a fourth time.