mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
efe427ebb2
--strict is the advertised strongest gate and CAN NEVER PASS today: it needs TWO distinct strict-eligible COLD families and only codex qualifies (agy is A7-benched; there is no cold claude adapter and never will be — LAW 0). An operator reaching for maximum rigor gets a guaranteed exit-5. Decided by a genuine CROSS-FAMILY QUORUM (no self-grading; the author is neither judge): - codex (gpt family) -> B: certify agy behind a measured bar - agy (gemini family) -> C: B now, warm-strict deferred Both reject warm-strict as the primary answer (it breaks strict's cold portability, and the measured warm data says the bar may be unpassable: the tri panel reached FULL agreement 0/22 times vs dual 10/19). Both accept certifying agy — but only after a measured bar, never an unmeasured flag flip. Grounded in live evidence from this session: with codex rate-limited, agy served as the sole cross-family cold reviewer for ~8 beads and produced BOTH a real, correct defect catch (reconcile falsely treated REBOUND verdicts as unbound) AND a false refute (wrong premise about source_hash, disproved by 4 deterministic gates). Useful but noisier. On a strict door a false REFUTE fails SAFE (HOLD) while a false CONFIRM would compromise the gate — so admit the noisy family IF its false-CONFIRM rate measures ZERO. The bar (all must pass before PAWL_STRICT_ELIGIBLE_FAMILIES is flipped): >=30 cold agy reviews over real held-out beads + seeded good/defective diffs, adjudicated against deterministic gates; ZERO false CONFIRMs; REFUTE precision >=80% (bounds the DoS risk both judges named); >=80% detection of seeded defects; schema-conforming evidence. Until then --strict stays honestly UNAVAILABLE (exit 5, never a silent single-family pass) — the status quo, so no code change holds the line. .9 (F8b) is rescoped to RUN that certification; if agy fails the bar, .9 closes with the measured evidence and strict stays unavailable — a real outcome, not a failure. Claude-Session: https://claude.ai/code/session_01RDWPYn4ZS2SyBkcBvAM18u