mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
main
626 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bo
|
d972fa2090 |
Prepare AgentOps 4.0.0 plugins, skills and CLI release (#1143)
## What Prepare AgentOps 4.0.0 across the Claude plugin, Codex plugin, skills and CLI. Claude writers capture the supplied check status during its original invocation, and plugin conformance verifies exact skill membership and link destinations. Full release security now scans the repository and blocks on Python collection failures that previously produced a false green result. ## Why The 3.6.0-to-current interval removes published commands and 20 skill names, so this is a major release with migration instructions. Release validation also exposed stale skill assertions and test prerequisites that need to match the current product contracts without weakening acceptance. ## How I tested - Native Claude Opus/Haiku success, failing-check and direct-writer trials: each check ran once, and the direct child returned plain JSON. - Actual fresh installs and upgrades from 3.6.0 in isolated Codex and Claude homes: 34 skills, expected agents, and exact installed package bytes. - Exact candidate `b721d02559e1495be6095ad97b820e88ceb4a049`: all 73 full repository gates, regeneration parity, and the complete local release rehearsal passed. All 12 security tools ran with zero skips, tool errors, critical findings or high-severity security findings. The unchanged advisory policy reports 35 quality-high findings on unchanged files. - Python: 327 tests and 72 subtests passed. Hosted Bats: 1,509 passed, 31 environment-dependent skips, zero failures. Go lint/build/vet/race/shuffle checks and CLI smoke/integration passed. - All 11 hosted checks passed, including Windows correctness, macOS/Linux installation, security, and the six-target no-publish GoReleaser snapshot. Local archive checksums and a real macOS CLI initialization/status/version smoke also passed. - Fresh author-distinct review passed all four acceptance criteria and all 35 changed paths with no unchecked acceptance. Canonical subject and caller-intent verification passed; verdict digest `68af2c935ed0106cd91b3950f5d168e662f4071f660fcbd113c36b7cd0f0426e` binds manifest `7affc77e25eaff69ba36c5ce05582b4f0385c954b76b62c02b97f97041f489b2`. ## Checklist - [x] Breaking changes documented in the migration guide and complete release notes. - [x] No credentials or private runtime proof included. - [x] Final full release checks pass on the exact candidate. - [x] Fresh author-distinct final PASS is recorded before merge. This prepares the release candidate; it does not publish a tag or release. Coverage limits remain explicit: native plugin tests used isolated macOS homes and local marketplaces, guard installation remains opt-in, and reader instructions do not prove sandbox confinement. Semgrep retains pre-existing warning-level parser diagnostics. Snapshot metadata follows the existing 3.6.0 tag; this is a packaging rehearsal, not a published 4.0.0 archive. |
||
|
Bo
|
86cdae4f3a |
Fix native Codex role registration paths (#1142)
A normal Codex session found both project role registrations but could not spawn either: native role loading rejected the symlink-backed configuration files with `Too many levels of symbolic links`. Point each registration directly at its canonical regular TOML source. Keep the role instructions and model pins in their existing owner files. Strengthen the registration regression test so `.resolve()` cannot hide this failure again. Validation: six focused role tests pass; projection regeneration check passes; `tests/run-all.sh` passes (10 passed, one unavailable optional OL integration suite skipped). A fresh interactive Codex session, without role or model overrides, launched both Luna roles; the reader returned three line-referenced decisions covering 1,203 lines, and the writer returned a successful one-file test receipt. Fresh independent transcript review passed all supplied smoke criteria, including exactly one writer check, receipt isolation, missing-reference refusal without writes, and the live hook denying two oversized reads while allowing a bounded read. The reader inherited the parent’s workspace-write sandbox as documented; this run proves bounded behavior, not enforced read-only access. |
||
|
Bo
|
937d6c8269 |
Add native Codex context-budget delegation and opt-in refusal (#1140)
## What Add Codex-native `bulk-reader` and `code-writer` roles pinned to `gpt-5.6-luna`, opt-in role/config installation, and an opt-in native `PreToolUse` Bash adapter for the shared read-budget guard. Source-owned guidance and role files ship through the existing generated Codex bundle; the menu stays at 34 skills. The installed Codex 0.154 runtime can refuse covered shell calls before execution. The adapter enforces that predicate; slice discipline, target-only writes and receipt-only replies are role instructions, not an output filter or per-file sandbox. Native exact-definition hook trust remains required. ## Why Follow-up to #1137 and its [fresh author-distinct review](https://github.com/boshu2/agentops/pull/1137#issuecomment-5648513520). Correct the unverified statement that Codex has no refusal-capable hooks and provide real native delegation without subprocess model execution. This branch incorporates the isolated fixes in #1139 and targets main; land the repairs first. The fixes PR contains no Codex-native changes. Work and original acceptance are recorded in private BD `age-z25n`; `bd context --json` resolves the existing private Dolt store. [Design and live evidence](https://github.com/boshu2/agentops/blob/codex/context-budget-native/docs/design/codex-context-budget.md) records the runtime/config contracts, exact available model identifiers and published comparable rates, source paths, invocation, transcripts, accounting and limits. ## How I tested Current head: `40edb5f216b71bfa659627dca10fc0a9e04c1821`. Fresh author-distinct review: **Job2 native PASS; combined subject FAIL**. Reviewer context `01a09778-bef6-7883-880b-6764fdd783b9`, observed `gpt-6-astra`/ultra, authored no candidate code. All 52 changed paths have identical start/end manifests (SHA-256 `b1f13ca731658bc699838189f928535a386ab6de6b381b0419bb0f7ebdd88cdc`); acceptance `not_checked: []`. The native repeated-refusal naming finding is fixed and independently reproduced as resolved. Local final checks pass. [Hosted Validate CI](https://github.com/boshu2/agentops/actions/runs/34722978404) remains pending at this update; no merge is performed. Real Claude Opus follow-up closes plugin-name resolution, inherited hook invocation, complete reader coverage and observed parent/child content separation. **It also finds a remaining Claude writer failure:** two of three final workers ran their supplied check twice; a direct receipt used Markdown fences. This combined PR does not claim a full Claude writer PASS. Details, excluded failed attempts and exact identities are in the design note. - `./cli/bin/ao gate check --scope range:origin/main..HEAD`: 33 passed; the earlier unchanged registry run `./cli/bin/ao gate check --full` passed **73 gates** at `90c8b31ee` (historical receipt). `bash scripts/regen-all.sh --check`: all 11 checks passed. - Door9, hookless cold-start, doc-hook drift, shellcheck on five changed shell files, and Node syntax checks on both workflows plus the native config editor passed. - A7 Bats suites plus workflow and all native suites: **223 passed, zero skipped**. All **21 documentation-reference tests** and the strict full-document reference scan also pass after clarifying the native hook-manager terminology. - `bash tests/run-all.sh`: **10 passed, 0 failed, 1 skipped** (optional OL directory absent). This is the default static tier. - `bash scripts/validate-codex-install-bundle.sh`: passed, **34 skill packages**. Changelogs are identical; diff check clean. - Live registered reader: parent `01a09776-004f-79a0-af85-c75b472a1e68` used only spawn/wait; child `01a09776-3c38-7281-b582-01fef5101f17` was natively identified as bulk-reader/Luna/low. Six separate slices covered 1,772 lines without truncation; parent received five findings and coverage only. - Live registered writer: parent `01a09771-8908-7a00-b101-919b558cf8c1` used only spawn/wait; child `01a09771-bbbf-7232-add4-cc9ec55ad759` was code-writer/Luna/medium. Required reference, seven-line Bats target, receipt only, child check passed; coordinating parent independently ran Bats successfully without reading the target. - Live hook: native parent `01a09769-14d2-7c22-9b7d-50847de07c90`, final turn `01a09771-25f2-7830-830b-498d7ca1945e`, refused a 400-line cat before execution and allowed a three-line sed slice. Real payload is PreToolUse/Bash/tool_input.command; hashed deny ledger schema verified. An explicit session hook was trusted in `/hooks` for this proof. - Credentials-free native config/read and hooks/list probes verify personal and ordinary project discovery. Codex 0.154 reads linked-worktree project hooks from the primary checkout; `--project` now rejects linked worktrees before writing, with a real Git regression. Known failed: Claude writer check-once behavior and direct receipt fencing. Not checked: arbitrary hosted/MCP read interception; adversarial enforcement of role instructions; other runtime versions/accounts; cost savings, comparative latency or ADR-0002 value-proof clearance. Earlier reader attempts with an overlarge final slice or truncated aggregate output are disclosed and excluded from the successful coverage proof. ## Checklist - [x] `make build && make test` passes (if Go changes) — no Go changes; gate runner built once. - [x] No secrets or credentials in code - [x] Breaking changes documented — opt-in additions; unsupported linked-worktree hook installation refuses before mutation. |
||
|
Bo
|
a3d6018f2c |
Fix read-budget refusals, installer backups and workflow boundaries (#1139)
## What
Repair the opt-in read-budget guard, installer and Claude workflows from
#1137. The fixes cover conservative literal shell parsing, preserved
installer backups, bounded worker returns, distinct filesystem targets,
actual plugin-qualified names, complete reader coverage, exact receipt
identity and measured writer line counts.
## Why
The [fresh review of
|
||
|
Bo
|
e32e88c338 |
Add opt-in read-budget guard and cheap-model bulk-read/code-write delegation (#1137)
Bring the Spotify Claude Code context-budget setup into AgentOps as one opt-in PreToolUse guard (skills/cc-hooks/hooks/read-budget-guard.sh, policy core.context:unbounded-read) with its opt-in installer, two Workflow-tool conveyors (bulk-read, code-write), two plugin subagents (bulk-reader, code-writer), their docs, regenerated projections, and three bats suites. Nothing ships wired by default; the skill menu is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2bDH2j6XDPPcLQS84PdzN |
||
|
Bo
|
c6558508d1 |
Consolidate AgentOps into a 34-skill engineering menu (#1133)
AgentOps' 55-skill catalog contained overlapping entry points, stale
routes and descriptions that could lose meaningful guidance in the Codex
projection. This change consolidates 21 roots into existing owners,
leaving 34 distinct skills and a generated, task-oriented menu. README
documents every retired name and its replacement.
Planning now establishes observable behavior in the caller's existing
intent, using proportional Given/When/Then examples and domain language.
Implementation and final validation carry those same examples forward.
Original adaptations informed by Matt Pocock's engineering skills
strengthen existing owners rather than adding a new workflow. Routine
edits need no mandatory plan, coverage report, mutation exercise or
learning artifact.
Codex retains complete source descriptions and translates explicit-only
invocation policy. All descriptions fit the existing 180-character
limit; the root instructions retain their 250-line limit. Generated
catalogs, projections, routers, moved references/helpers and their live
consumers are updated together. RPI remains explicitly selected.
Validation passed: projection/conformance checks, the local aggregate
(10 passed; one existing optional-directory skip), and exact-commit CI
covering the complete gate registry, Bats, Go build/vet/race/coverage,
Windows and security. A fresh author-distinct reviewer passed all
acceptance criteria over the complete 573-path subject at
|
||
|
Bo
|
4f5fd6301c |
Use CASS and MS for evidence-led instruction improvement (#1131)
## What Use CASS to discover session evidence and MS to find existing skill guidance. Reserve AO exact excerpts for an identified source-precision gap. Correct guidance and helpers that treated repetition as success, inferred user prompts from early line numbers, or refreshed the index before every search. Repeated failed prompts now remain unassessed. Native message roles identify user prompts, search helpers preserve unavailable reads, and recovery requires observed state before attempting repairs. Existing CASS/MS documentation explains their respective roles without adding another mining framework. ## Validation - Seven focused helper behavior tests and CASS/MS skill validators pass. - Generated projections are current. The local aggregate passed 10 groups with no failures; its absent OL integration suite was explicitly skipped. Full AO gates passed all 73 selected checks. - Fresh independent review caught an incomplete-status recovery edge case. The repair, regression test, and generated copies have been rechecked with no remaining findings; final local checks pass. - CI passed on the exact final commit, including Linux/Windows correctness, Go race tests, security and the required summary check. - A bounded private exercise used three CASS query families and MS search/full loads. It reached a justified no-change decision because selected native sources were unavailable or did not match returned locations. Retrieval success was not credited as learning, and no AO extraction was needed. No private session content or locators are included in this PR. ## Limits This establishes retrieval and evidence-handling behavior. It does not establish that a new instruction improves later task outcomes. The CASS artifact validator deliberately skipped its opt-in live corpus smoke test; the separately bounded retrieval exercise is described above. |
||
|
Bo
|
3213afcf1c |
Default to native execution and report independently accepted work (#1129)
## Change Make native coding-agent execution the default AgentOps entry path with zero mandatory skills. Preserve full bundles and add repeatable `ao skills link --skill NAME` selection, validating the entire selection before writes. Align product, installation, architecture and generated command documentation. Extend the existing trial readout to separate endpoint test results, execution state and independently accepted work. Bind supplied judgments to exact content, acceptance and native evidence. Reject empty implementation subjects and require the caller's complete criterion ID set before reporting acceptance. Preserve genuine nonempty and deletion-only subjects, valid failures and missing-proof outcomes. ## Validation - Native onboarding from empty home/consumer directories produces no setup files; selective/full linking and failure boundaries are covered. - Actual RED/GREEN regressions cover empty subjects and the partial-criterion omission found by independent review. - Full Go build, vet and race/shuffle tests; affected Go lint; 88 Python readout/statistics tests passed. - All 73 gates, generated projections, strict documentation build and local aggregate passed (10 passed; one documented optional absence). - All nine PR checks succeeded at `7df0d42b12f35ffc22008cc10a40339afcfbb6a0`. - Fresh author-distinct review passed all six acceptance criteria over all 59 changed paths, with no findings or unchecked scope, after repairing the criterion-coverage finding. ## Evidence limits The real native coding repair demonstrates usability, not comparative skill uplift. The strict live-session machine replay remains NOT_PROVEN where execution/identity observations are unavailable; the source review PASS is retained separately. Existing cohort limits and the historical aggregate-enforcement gap remain unwaived. No new comparative cohort, scheduler, skill-corpus deletion, memory migration or global installation is included. |
||
|
Bo
|
e1fae0dae6 |
Make the engineering harness lean and add optional topic memory (#1116)
RPI now owns the authorized outcome through finish, with Plan and Memory
loaded only when useful. Known defects get direct repair, and evidence
can change the approach under unchanged acceptance. Fresh exact-content
validation remains required. Memory provides optional recall, mining and
curation of reviewed topic pages; specialists and the fixed-dispatch
adapter remain optional.
The change reconciles current documentation and generated skill
projections. It preserves native budget and permission authority, BD
work ownership, protected external evidence storage and the distinction
between a supported lesson and demonstrated later benefit. It adds no
scheduler, work store, Go command or evidence schema.
Validation: required local Go/build/vet/race checks, aggregate suite,
generated-output check and 72 gates pass. The complete 44-test executor
suite passes; its shared-deadline fixture now tolerates CI scheduling
jitter while still requiring deadline exhaustion and preventing a third
launch. Fresh author-distinct review passed all 112 changed paths with
no findings; all seven exact-head CI checks passed at
|
||
|
Bo
|
57ece9fb7b |
Restore private context routes and verify native judgment receipts (#1112)
Add explicit, recoverable private context routing through `ao config context`, binding native source, owner, task, model and destination to existing policy and external storage. Recovery reads the original Beads maintenance anchor; configuration reports native access enforcement as unattested. Add `ao provenance verify-judgments` to check required review profiles against exact native transcript receipts, independent subject and acceptance, distinct contexts, completion and permitted providers. Requested identity and unreported effort do not count as runtime evidence. The verdict schema is unchanged. Repair the existing cleanup test: a 0.3-second budget could expire during preparation before either fixture process started. A separate controlled-delay test now proves preparation cannot renew that deadline. The running-cleanup case requires parent/child readiness, preserved partial output, the postlaunch cleanup result and both processes stopped within its existing four-second bound. Production timeout behavior is unchanged. Validation: fresh author-distinct review passed the exact 55-path final subject and all T05/T21 acceptance. The complete local Bats run passed (1,333 passed, two existing skips), as did Go build/vet/test/race, all 72 full-mode gates, the aggregate and generated-output checks. Ubuntu/Windows CI, security and both installation jobs passed on the final commit. The final evidence scan found no new orphaned bindings; 73 historical bindings remain preserved. Earlier failed results and private evidence remain outside the PR. |
||
|
Bo
|
8061085c89 |
Ship native evidence helpers and fresh-family review defaults (#1110)
AO now performs intent snapshots, subject manifests, strict evidence verification, atomic verdict storage, and orphan inspection through the Go binary. The command handler keeps verification separate from presentation so it meets the existing complexity limit. These operations preserve the existing evidence formats, require explicit protected storage where applicable, and run outside a checkout without Python. The unchanged Python implementation remains a developer oracle; agents still provide semantic judgment. Codex and Claude skills now default to a fresh reviewer from the author’s model family. Callers can explicitly request cross-model review or pin its model. Reviewer adapters use a finite caller timeout or remaining deadline instead of a fixed ten-minute default, while retaining output limits and abnormal-termination cleanup. Validation: Go build, vet, tests and race/shuffle tests; 1,334 shell tests; aggregate runner; regeneration check; 72 full-mode gates. Independent checks exercised 84 storage-boundary rejections and 21 evidence operations with an empty PATH. Both canonical and generated RPI reference suites pass all 48 tests after updating the migrated oracle import without weakening assertions. Change-sensitive checks explicitly compare the final committed candidate with the original PR base. Linux, Windows, installer, security, and required summary checks are green. |
||
|
Bo
|
af1e530159 |
Adopt CDLC contracts and reconcile BD tooling (#1108)
AgentOps now describes the selected Context Delivery Lifecycle alongside standalone RPI: external reviewed memory, native work and runtime ownership, independent validation, and explicit limits on what the pilots proved. The active ADRs name which historical prescriptions are superseded and which invariants remain. This integrates the BD foundation, including doctor diagnostics that distinguish BD from BR and protection against private tracker data entering outgoing Git history. It also fixes skills audit resolution of explicit relative reference paths. Source-owned projections were regenerated, and the landed stopping behavior is preserved. Validation: Go build/vet/test and race/shuffle, 1,325 Bats cases, the local aggregate, and full AO gates. The gate registry reports 71 passes and one nonblocking preamble-adoption failure on the existing report script; that advisory remains disclosed. Historical probe receipts whose source hashes changed remain historical evidence. This is foundation adoption. It does not claim that the later memory, goal, background, privacy-release, or benefit-measurement work is implemented. Independent Astra and Fable judgments both PASS over the exact integrated content. Earlier negative results and their factual resolution remain preserved in private work evidence. |
||
|
Bo
|
baa24e1b04 |
Bound headless execution and require acceptance-linked repair progress (#1107)
Headless invocations could run without a finite bound, and repair loops could count changed digests or finding counts as progress. The existing adapter now enforces positive finite deadlines, an optional inherited absolute deadline, bounded output capture and cleanup of its owned process group, refusing execution when protection is unavailable. The existing goal, RPI and validation skills now require acceptance-linked progress, distinguish newly discovered failures from regressions, and bound Andon recovery. Review effort follows the effect on acceptance and enforcement. These contracts do not claim native aggregate goal-budget or pause enforcement. Validation on the clean main baseline: - 1,317 Bats cases, zero failures, two skips; 46 RPI reference cases. - Go build, vet, tests and race/shuffle tests; local aggregate runner. - Generated projections current; all 72 AO gates pass. - Fresh independent Astra and Fable 5.1 xhigh judgments passed the exact clean source. Earlier uncommitted CDLC adoption, private work/evidence, and unrelated Go changes are excluded from this PR. |
||
|
Bo
|
021b41c951 |
Skills only: cut the RPI workflow script and the machinery around the loop (#1105)
## Skills only: cut the RPI workflow script and the machinery around the loop #1104 landed the right ideas in the wrong form. This repository is skills only: a skill is an instruction to a system, models route and judge, and deterministic checks exist where reality is deterministic (digests, tests, gates, "is this file tracked"). #1104 built a workflow script and a set of schemas, enums, and engines where instruction was wanted, and then spent four judge rounds hardening them. This PR deletes that and keeps the ideas as instruction. **Deleted.** `workflows/rpi.js` and everything that existed only to exercise it: its probe harness in the product-boundary suite, the convergence-law and glob-intersection fixture corpora, the cathedral canaries that pinned its mechanisms, and the pointers to it. **Reverted to the #1099 form.** `skills/rpi/scripts/run_once.py` and its tests: the repair law as accepted in the loop restore, without the class-law machinery, the snapshot verification, or the cross-leg checks. **Kept.** `docs.claims-tracked` (a blocking gate that caught a real stale claim on its first run), `scripts/evidence-orphans.sh` (a receipt a shell script computes), the optional `findings[].class` field on `verdict.v2` (a word a validator writes), and the corrected harvest-note sentence. **Kept as instruction.** The rpi, plan, implement, validate, council, learn, and boundaries contracts and `docs/architecture/rpi-traversal.md` now say, in plain sentences a model follows: a risky change gets one fresh judge on the frozen plan before Implement; validators name a class per finding and a returning class means go back to Plan; two judges disagreeing is the orchestrator's decision made in the open, with both reads in the report and a risky surface still needing both to pass; the orphan receipt runs after Implement and each repair and rides in the check receipts; a doc sentence that says published, pinned, or proven is a claim to check; Learn proposes a check for any repeating finding class. **Evidence on the tip.** Regen check clean; rpi unit tests; cathedral gate; routing goldens; token budgets; coverage 1 of 12; Go bar; full gate green with a HEAD-built binary; CI's bats command green; gate-tightening ratchet. One fresh read by Fable 5.1, asked in plain language for what it found, recorded in the PR thread. |
||
|
Bo
|
2ef932b617 |
RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated (#1104)
## RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated Provenance: running the loop for real across three trains on 2026-09-03 (#1099, #1100, #1101). The spine held; the flow around it did not. A risky-surface design shipped with no premortem and needed six repair passes. The open finding set grew three rounds running inside one class while the id-counting law watched. A same-family PASS and a cross-family FAIL had no tie-break until the caller named one afterwards. Every harness edit orphaned bound evidence with no warning until verify failed. A doc called an egress log published while the repository's `*.log` rule kept it out of the tree, and the same overclaim class came back four rounds. **What changes.** - **Premortem at plan exit.** When the write scope hits a risky surface, one fresh judge reads the frozen plan before Implement; a blocking finding returns NOT_PLANNED with the findings. The caller may declare `premortem: skip`; the report says so. - **A split is recorded, never elected and never closed inside the loop.** The law stands: a risky surface converges only when both legs PASS, and no finding leaves the open set because someone chose a judge. Plan declares `binding_judge` as the caller's disposition for a split that survives repair; it is bound in the plan identity, carried in the report, and never mutates the verdict. On a risky split that survives the repair budget (never after a law stop), one council leg receives a bounded packet marked untrusted and returns per-finding rulings, exactly one per id, which the traversal validates and records under `council.rulings` for the caller's next intent; the verdict and the open finding set stay exactly as repair left them. A council closure mechanism was built, drew a new finding class in three consecutive judge rounds (path containment, digest exclusion, presence-as-evidence, partial closure downgrading a FAIL), and was cut under the train's own class rule rather than hardened again. A non-risky split stays worst-of. - **Finding class in the convergence law.** Findings carry an optional stable `class`. A class carried by a resolved id that reappears on a new id with no surviving prior id carrying it is `class_reopened`, whether or not a classless round sits between; a blank class is an invalid round; a round can carry both a reopened id and a reopened class and stops on either. Repair stops, the traversal returns NOT_PROVEN, and the caller goes back to Plan, because the design is wrong, not the patch. Same rule in `run_once.py` and `workflows/rpi.js`, driven by one shared fixture file, with executed canaries in the cathedral gate. - **Plan identity, and the premortem's risky test.** On a risky scope the Plan carries `binding_judge` and the evidence it expects to orphan; `plan_digest`, when the plan declares one, must equal the script's digest over the acceptance, the write scope, the binding judge, and the intent-source digest, or the traversal stops before Implement. The risky test over a declared scope is glob intersection: witness paths, the risky regexes over the glob with wildcards instantiated, and any literal prefix that reaches a risky root; a bare `**` or `*` is risky. - **Intent is verified before anything is built.** An `intent-snapshot` receipt re-derives the digest and compares the snapshot bytes to the caller intent before premortem or Implement; absent, mis-hashed, or wrong stops the traversal with `plan_identity_mismatch`. The Python reference refuses on the same rule. Declared scopes are canonicalized (repeated `./` collapsed; absolute paths and `..` refused at plan exit), and the four spellings of `tests/**` share one plan digest while `tests/` does not. - **Orphaned evidence is a receipt, not a surprise.** After Implement and after every repair round the workflow runs `scripts/evidence-orphans.sh` over the runtime-derived path union, keyed on the round so a same-set repair still reruns it and appends every scorecard and fixture set whose bound evaluator or skill digests cover them to the check receipts the validator reads, each with a cause (`changed_path`, `digest_drift`, `both`, `skill_changed`). Plan budgets that recapture up front. On today's tree the receipt already lists the 08-16 and 08-26 sets whose bound harness digests moved long ago. - **Doc claims are gated.** The stronger scan already caught one stale claim in the tree (a 2026-06-22 harvest note calling evidence committed after Train 2 had removed that tree); the sentence now says where the evidence went. `docs.claims-tracked` (blocking) requires backticked repo paths in `evals/` and `docs/evals/` markdown to be tracked, and sentences that say published, tracked, or committed to name files that exist. - **Risky scope is a glob intersection, not a heuristic.** One segment-wise `globsIntersect` over one list of risky segment globs answers both the declared-scope question at Plan exit and the changed-path question for the cross-family rule; two heuristic designs missed in opposite directions and were replaced rather than patched. - **Contracts.** rpi gains a judgment dispatch table: risky scope routes to premortem, a risky split to council, an irreversible landing to one-way-door as a caller choice. plan, implement, validate, premortem, council, and learn carry their halves; learn proposes a deterministic check for any finding class that repeats. `docs/architecture/rpi-traversal.md` is the source of record and was updated with the flow. **Evidence on the tip.** Regen check clean; Go bar; full gate green with a HEAD-built binary (the new gate fires and passes); CI's literal bats command green; rpi unit tests; cathedral gate with executed canaries for the class law; routing goldens; token budgets; gate-tightening ratchet; coverage 1 of 12 unchanged. **Judgment, stated plainly.** Six judging rounds, each on a frozen tip. The caller elected Fable 5.1 as the binding validator; its round-6 verdict is PASS with no verdict-driving findings (lows: a council payload outside the schema throws a raw error instead of recording `invalid-rulings`; two stale comments; a dead parameter; a dead fixture element; the contracts say exactly one ruling per finding while the code enforces at most one). The cross-family Codex read of the same tip failed on three input-validation items, which the final commit closes: control characters are now refused on the original scope value with the full Unicode control class and the caller scope is canonicalized in place so one spelling reaches every prompt; the intent receipt no longer forgives a trailing-newline difference; and the Python reference accepts a digest string only from a verifier callback and matches digests with `fullmatch`. One half of one Codex item stays open and is disclosed: the workflow does not itself hash the caller intent's bytes to compare with the plan's declared digest (it relies on the receipt's exact-bytes match plus digest equality); binding that locally means every probe fixture carries a real digest of its intent text, which is the next pass. Also open, pre-existing and outside this train: the risky-surface list names `lib/**` while the preamble every gate sources lives under `scripts/lib/`. Across the rounds the loop behaved as designed once: the council closure mechanism drew a fresh finding class three rounds running and was cut under the train's own class rule instead of hardened a fourth time. |
||
|
Bo
|
1cebfee94d |
Measure one skill: sealed probe dispatch and the first current ledger row (#1101)
## Measure one skill: filesystem-sealed probe dispatch and the first current ledger row Provenance: the instrument-panel roadmap (`docs/plans/2026-08-26-instrument-panel-roadmap.md`) closed out with zero skills measured because control-arm reps read `skills/<skill>/SKILL.md` from the checkout and the operator's skill roots mid-run. Its named successor was filesystem-sealed dispatch. This PR builds the seal, then uses it, and the instrument caught one more leak on the way. **The seal.** Codex's own seatbelt does not nest inside an outer `sandbox-exec` profile, so a rep now runs under an outer profile with Codex's sandbox bypassed for exactly that reason: `file-read*` denied on the checkout and every skill root (`~/.agents`, `~/.claude/skills`, `~/.gemini/skills`, `~/.codex/skills`), `file-write*` denied everywhere except the rep's workspace, a scratch HOME, and the temp roots. HOME and CODEX_HOME are scratch (auth symlinked). Dispatch refuses without `sandbox-exec` unless `PROBE_SEAL=none` is set, which marks the run coverage-ineligible. The seal is recorded in the capture contract (`agentops-skill-probe-capture.v3`, a `seal` block), coverage eligibility requires `seal.mode == seatbelt`, and `verify-scorecard` re-checks that the denied roots include the repository and the four skill roots. Existing 08-26 sets become `legacy-unsealed` and ineligible, which changes nothing they were counting. **The leak the seal did not cover.** The first sealed capture showed the xhigh control-2 rep listing the shared live workspace, finding `treatment-1.prompt`, and reading it; the treatment prompt embeds the skill. So every rep now gets its own empty workspace, prompts reach a rep on stdin only, raw JSONL and stderr live in a harness-private dispatch directory the rep can only stat, and a `sibling-prompt-read` trap degrades any rep whose successful command names a prompt, contract, seal, or stage file or whose output lists a sibling's artifacts. Replaying the contaminated set under the new trap turns its REGRESSIVE into INERT with the contaminated rep degraded; that set is superseded and not committed. **The second pass, after the judges read the first.** A Fable validator passed the first sealed capture with four findings and a Codex read failed it with seven, all real: the whole per-user temp tree stayed readable (hundreds of stale probe directories held prompt files with the canonical skill bytes), sibling checkouts and the real `~/.codex` sessions were readable because seatbelt matches the traversed path and the skill roots are symlinks into the main checkout, the dispatch directory was writable so a rep could rename an earlier prompt into its workspace, the contract trusted asserted seal metadata (a hand-made Linux record was coverage-eligible), the only real-Darwin test nested seatbelt and proved nothing, and the operator's `config.toml` was symlinked in whole, so every rep started the operator's MCP servers. The seal is now one run directory removed on exit (`home/ ws/ tmp/ dispatch/`, reset per rep); reads are denied under the real HOME, the real TMPDIR, `/tmp`, the checkout, the git common directory, and each skill root's resolved target; `dispatch/` is write-denied with metadata-only reads; `file-link` and `file-clone` are denied on every denied root; the rep's config is rebuilt from top-level scalars only and `auth.json` is copied, not linked. The contract binds platform, mechanism, wrapper, the denied and allowed roots, the rep environment, and the sanitization, and coverage refuses any record that omits them; the Darwin test performs its reads inside the seal. One recorded hole remains by necessity: the codex launcher under `~/.local/bin` and `~/dev/dotfiles/bin` is re-allowed by literal path, and coverage refuses any allowed path inside the checkout. **The third pass, after the judges read the second.** Both judges failed the hardened tip again, and the headline was network: bypassing codex's own sandbox to nest the outer seal had dropped every inner deny, not only the filesystem ones, so a rep could fetch the canonical skill from GitHub while the disk was sealed. A rep now reaches the network only through a harness-owned CONNECT proxy (`scripts/lib/probe-connect-proxy.py`) with a host allowlist discovered from real runs (`chatgpt.com`, `ab.chatgpt.com`, the rotating `.oaiusercontent.com` suffix, and the API-key hosts), behind a seatbelt `network*` deny that admits only the proxy; every attempt is logged, a refused one degrades the rep, and web search is disabled in the rep's config. The profile is now rendered from the bound seal block by one function, so `verify-scorecard` reconstructs it and requires the digest to match; the recorded launcher chain is the only allowed read under a denied root; the wrap invokes `/usr/bin/sandbox-exec` by absolute path; the real CODEX_HOME and the Darwin cache directory are denied and bound; every non-stdio descriptor is closed and `/dev` writes narrowed to four devices; each rep runs in its own process group that must be empty before the next reset; one minimal config is generated per run, bound by text and digest, and re-parsed after each rep; one guarded trap covers the run root and any unpublished stage; and the coverage gate prints an eligibility row for every ledger row that names a scorecard. What stays open is stated in the README: allow-default still governs what the profile does not deny. **The fourth pass, after the judges read the third.** Both judges failed the third tip on properties the record carried but the verifier did not check. GNU `timeout` sets its own process group and ran outside the seal, so the per-rep reap was vacuous (four sleeping survivors after the test passed); the seal is now the outermost process, timeout runs inside it with `--foreground`, its absolute path is bound, and the reap counts survivors before signalling. The verifier now pins, not merely records: the egress host set (a module constant), port 443 only, no unix sockets, a loopback proxy, the generated config text equal to the renderer's output, the real CODEX_HOME and cache root inside the denied roots, writable roots under the run root, the four device paths, the environment allowlist, the launcher chain's digest against the producer identity, and it refuses unknown record fields. The proxy logs an attempt before dialing, captures the rep at accept time, refuses names that resolve to private or loopback ranges, and its source joins the evaluator hash set; the per-rep egress log is published with the fixture set. Withdrawn ledger rows get their own eligibility reason. The README lists which seal fields the verifier pins and which it only records; the cross-capture aggregate is recounted to 4 of 14. Still open and disclosed: Mach IPC and Apple Events beyond the filesystem and network seals, exact-byte environment sealing beyond the allowlist, and cleanup ownership by inode. **The fifth pass, after the judges read the fourth.** Both judges found the seal, proxy, reap ordering, transcripts, and scorecards sound and failed the tip on claims the tree did not hold: the egress log the docs called published was caught by the repository's `*.log` ignore rule, a `setsid()` child escaped the process-group reap, and one ledger sentence said no rep ran a command when a control rep had run `rg` over its empty workspace. The log is now tracked and required, strictly parsed, with both allowed and refused counts recomputed per rep; the reap is fatal to the capture when it cannot prove emptiness and an `lsof` sweep of the run root catches a session escape; the rep launches through a real `env -i` boundary; the timeout budget, the launcher chain's adjacency, every required root in both read and link denies, the evaluator match, and the trust-table value are pinned by the verifier; and every sentence in the ledger, README, and runbook names the check that enforces it. The remaining gap is stated: a child that both escapes the session and holds nothing under the run root is not detected. **The sixth pass, after CI read the fifth.** The first CI run of this PR failed on Linux: the launcher-chain pin walked the capturing Mac's filesystem, so on the runner both sets read as unsealed and the coverage headline fell to zero. A pin that holds on one host is not a pin. The chain is now bound as structure in the seal record (each link's path, kind, symlink target, and the final file's digest, with the invoked path as the head), the verifier checks that record on any host, and it cross-checks the live filesystem only where the head exists, reporting which happened. A gate-level test runs the coverage gate twice, once with the chain made absent, and requires byte-identical output. **The measurement.** `premortem-plan-shape-t2` (two seeded structural defects, band [1,4]) captured seven times on 2026-09-03, once under each seal; the first six sets were deleted with their scorecards when a judge or CI broke that seal. The rows that count are the seventh capture: low INERT (control 0/2, treatment 0/2) and xhigh INERT (control 1/2, treatment 1/2); no rep ran a command, no egress was refused, both per-rep egress logs are tracked and verified, and both scorecards verify with every pinned field, the launcher chain checked as a record and cross-checked on the capturing host. The superseded captures read, at low, BEHAVIORAL 1/1, INERT, INERT, BEHAVIORAL 1/2, INERT, BEHAVIORAL 1/2 and, at xhigh, INERT, BEHAVIORAL 2/2, INERT, INERT, INERT (control 1/2), INERT. The reversals across captures are recorded as an unresolved N=2 observation, not variance; across all seven the treatment arm put the marks in band in 7 of 26 usable reps and the control arm in 2 of 28, and the ledger marks that aggregate as disclosure, not a row. Headroom is SEPARATED at both levels. This is the first current, manifest-backed ledger row; it records response-shape behavior change, never quality uplift, and what it records today is INERT at both levels. **Evidence on the tip.** Probe, exec-lib, metadata, coverage, headroom, and adapter bats suites green with the Darwin seal, survivor, session-escape, and environment-boundary tests running; `verify-scorecard` on both scorecards; `check-skill-probe-coverage.sh` reports measured 1 of 12 with the 2026-08-26 row labelled `verdict-withdrawn`; regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go bar, lint, security, and the gate-tightening ratchet green. **Judgment, stated plainly.** The caller elected Fable 5.1 as the binding validator. Its fifth-round verdict on the pre-CI tip was PASS with no verdict-driving findings; its read of the portability delta failed on one test-only point (the coverage test had dropped the `measured` pin that CI tripped on) which this tip restores. The cross-family Codex reads of the same tips are FAIL. They closed the network seal, the environment boundary, the fatal reap, the data roots, the config binding, and the documentation, and they keep these open, recorded here as the next pass rather than hidden: the evaluator identity check still accepts the pre-proxy four-file shape for legacy sets; an inherited absolute `CODEX_EXEC_TIMEOUT_BIN` is trusted without a probe; a refusal logged under a rep label outside the expected set is not rejected; an empty top-level `[projects]` table passes the drift check; the `lsof` sweep treats a failing `lsof` as a clean result; and the launcher chain, now bound as a record that any host can check, is corroborated against the filesystem only on the capturing host, so a record whose head does not exist there is taken on its word (the docs say so; Codex wants the live walk required at capture and the invoked head bound into the producer identity). Fable also noted a test that mis-resolves when TMPDIR sits under `/private/tmp`, a `2026-09-04` date in one refusal message, and that the low control replies used a `S2: FINDING <reason>` shape the discriminator scores as "plan not walked" rather than by band (outcome-invariant). Each of those changes a harness or verifier file and would orphan the two rows again, so they are filed as the follow-up instead of an eighth capture today. Across six judging rounds the open set shrank only from round four on; the convergence law's stop signal was reached at round three and the run continued on the caller's instruction to land the instrument. The two ledger rows are honest about what they measure, and the seal they ran under is the one this tree can verify on any host. |
||
|
Bo
|
10f0277bdb |
Legible membrane, Train 2: what a stranger meets (#1100)
## Legible membrane, Train 2: what a stranger meets Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus, findings F5 through F9. This train is sized by a consumer inventory built with `rg` on the tip before any lane was written; the promoted-set directory move the audit proposed is deferred because that inventory shows skill-builder backing two blocking gates, swarm pinned by the cathedral gate and a routing golden, using-gc required by Go code, and `ao skills link` unable to install a second root. That inventory is the plan for a later train. **What changes.** - **Archival sweep by consumer disposition.** 172 audit snapshots, 29 pawl receipts, the `evals/workbench` and `evals/membrane` trees with their two bats consumers, four stray scratch docs, four retired eval contracts, and nine caller-less `scripts/check-*.sh` are deleted; git history is the archive. Every machine list that referenced them is pruned (evidence-grounding baseline, preamble grandfather, broken-links allowlist, `.gitattributes`, `.gitignore`, two eval fixtures, the workflow-coverage deferred list). `docs/audits/manifests/` and `.agents/ao/config.yaml` survive because they have live readers. About 48,000 lines. - **Three skills retired.** `goals` (alias of fitness), `shared` (tombstone), and `scope` (folded into plan step 3 as five write-scope checks). Consumers edited; the probe denominator exclusion for goals pruned; Codex package and golden count pins updated. - **Negative routing** on research, codebase-recon, reverse-engineer, premortem, one-way-door, and council, all within the 180-char budget, with a teardown golden (`rq-08`). One wording was changed after the router's prefix stemming showed "repository teardown" leaking into the wrong skill. - **Every promoted skill answers "It's working if"** with observable tells in backticks, and carries a paste-ready `## Prompt` with a concrete subject. Two fictional `ao` subcommands a draft prompt named were caught by the body-ref validator and replaced with real commands. - **Doctrine diet on the core five.** rpi, plan, implement, validate, and anti-ceremony drop from about 5,100 words to 3,600 (bodies from 4,700 to 3,150) by moving the shared ownership boundary, dated incidents, and mechanics tables into step-loaded references (`skills/rpi/references/boundaries.md`, `skills/validate/references/mechanics.md`, `skills/plan/references/ground-truth-routing.md`). Every cathedral canary and every skill validator grep survives unchanged. - **ADR-0018** records the goals, shared, and scope retirement; the cathedral gate tombstone and the routing goldens cite it instead of ADR-0017. - **Router and twins.** `ao skills find` holds a description's "Not for X; that is <sibling>." sentence out of its haystack, so premortem no longer ranks first for "is this live decision reversible" (golden `rq-10` pins the reciprocal of `rq-02`); a penalty variant was tried and reverted because it suppressed skills the caller named outright. A declared trigger phrase of two or more words quoted whole in the query now earns the name weight once, so "check this change" lands on validate rather than on reality-check's name token; a live-catalog test pins seven such queries. Single-quoted YAML descriptions unescape `''`. The Codex catalog keeps the exclusion sentence, and a closing `>` no longer turns `<run-id>/codebase-recon.json` into an invocation. - **Residue the judges found.** handoff, learn, and status open a `## Contract` heading after their tells; validate's prompt names its helper at `skills/validate/scripts/validate.py`; the explicit-skill prompt catalog names only live skills (five stale prompts replaced by nine, floor 20 restored, TESTING.md names the suite); the corpus-delta receipt binds the runner's path and SHA-256 and labels a `live_agent` claim as an unverified caller declaration; the probe README and ledger describe the 12-skill denominator; SKILL-API counts 30 of 54. **Evidence on the tip.** Regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go build/vet/test green; lint clean; security gate quick PASS; docs-build warnings did not rise. Fresh validation by Fable 5.1 (caller-elected) and a cross-family read by Codex, both recorded in the PR thread. --------- Co-authored-by: Bo <bofuller55@gmail.com> |
||
|
Bo
|
568e99d436 |
Loop restore: converge and crank as control flow under the verdict contract (ADR-0017) (#1099)
## Loop restore: converge and crank as control flow under the verdict
contract (ADR-0017)
Intent source: `docs/plans/2026-09-03-loop-restore.md` (in this PR).
Decision record:
`docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md`.
**Why.** The 2026-07-14 single-pass cut (`482307762`) removed the
iterate loop (discovery, crank, converge, evolve, the learn write-half)
together with the unproven compounding claim, although ADR-0011 demoted
only the latter. The control flow was never demoted, and its absence
showed on 2026-09-02, when a three-lane fix needed eight validators and
two stops because the contract had no repair phase. This restores the
loop as control flow and nothing else: no knowledge store, no `ao
converge`/`ao crank`, no evolve, no canary. ADR-0004 and ADR-0011 stay
in force.
**What changes.**
- **RPI gains a bounded repair phase.** On `FAIL` or `NOT_PROVEN` with
findings, repair and re-validate freshly under the convergence law:
caller-declared `repair_rounds` (default 2); open finding set keyed by
stable `findings[].id`, union across validator families, non-growing; no
closed id reopens; the subject digest changed or, for `NOT_PROVEN`, new
digest-bound evidence resolved a named gap. Converged = fresh PASS plus
cross-family PASS on risky surfaces. Plan and Implement keep their
single dispatch. `skills/rpi/scripts/run_once.py` models the law as pure
data (33 tests): rounds are validated for shape (digest required, no
duplicate ids, no PASS with findings, no FAIL without findings),
condition 4's evidence branch needs a NOT_PROVEN previous round, a
non-FAIL current round, new evidence, and a resolved finding, and a PASS
over unchanged bytes after a FAIL is a flip that reports NOT_PROVEN.
`workflows/rpi.js` runs validation as legs (spawned or external primary,
plus a caller-supplied `crossFamily.command` on risky surfaces) merged
worst-of with a union of stable ids; a risky surface without a
cross-family leg is `diversity_unsatisfied` and never converges or
enters repair; a failed repair or re-validation returns NOT_PROVEN with
no stale verdict. Validators return `subjectDigest`, stable finding ids,
and `evidenceRefs`.
- **crank returns as a thin wave executor** (113 lines): the caller
selects the wave and the repair bound, crank invokes RPI per lane
(parallel only on disjoint write and regen scopes), runs the wave
acceptance once, returns evidence, and stops. No retry, budget, queue,
claim, lease, Git, closure, or next-work ownership. Routing golden
`rq-07-wave-execution` ranks it first.
- **validate is cross-family by default on risky surfaces**
(`cli/internal/gates/**`, `scripts/check-*.sh`, `tests/**`,
`skills/*/scripts/**`, hook policies, `lib/**`, security-scanned paths)
with the LAW-0 dispatch table: Claude orchestrating uses read-only
`codex exec`; Codex orchestrating uses an interactive Claude session in
an NTM pane, never `claude -p`. No live adapter means
`diversity_unsatisfied`, which on a risky surface is `NOT_PROVEN`. The
full literal CI command set runs once on the final integrated subject;
routine rounds keep the receipt-driven freshness contract.
- **Conformance assertions flipped under ADR-0017 only:**
`scripts/check-cathedral-cut-conformance.py` (crank live; "Stop
regardless" replaced by positive canaries for the law's four conditions;
a bounded `for` loop that compares against `repair_rounds` is required
in `run_repair_phase`, and the gate executes the law's canaries against
the reference behavior), `workflows/rpi.js`,
`skills/rpi/scripts/validate.sh`,
`evals/agentops-core/rpi-behavior.json`,
`skills/rpi/references/rpi.feature`. Every single-pass public surface
(README, AGENTS.md, PRODUCT.md, CI-CD, agent-workflow-reference,
rpi-traversal, cli/README, quickstart and demo commands, the
operating-contract and product-boundary bats, the Codex-description
oracle) now states repair to convergence.
**Known approximation, disclosed.** The Claude conveyor has no
deterministic shell primitive, so changed paths are derived by the fresh
validator (git status and diff against the clean pre-run tree) and
unioned with the implementer's report; risk is classified over that
union and unreported paths are coverage findings. A validator is still a
model; runtime derivation outside every agent is a follow-up. Family
distinctness of the cross-family leg is asserted by the caller's choice
of command and not verified by the script.
**Not in scope.** Premortem stays a single advisory judge and Plan still
only names the first check (phase boundaries unchanged). No `verdict.v2`
or `rpi-report.v1` change. The loop's own effect on outcomes is
unmeasured and owed a seeded-defect probe, like the rest of the corpus.
**Evidence on the tip.** Regen check clean; full gate green with a
HEAD-built binary; CI's bats command green; Go build/vet/test green;
golangci-lint clean; security gate quick PASS; one fresh validator over
the whole diff; one cross-family read of the design before
implementation (13 findings folded) and two of the integrated diff (9
findings in round one, 11 by round two, 15 by round three, each round
repaired and re-reviewed; the fresh validator passed the tip after round
two and the final tip
|
||
|
Bo
|
e556834d3d |
Legible membrane, Train 1: Codex projection, executable entry points, green run-all, honest runtime docs (#1097)
## Legible membrane, Train 1: fix what ships Intent source: `docs/plans/2026-09-02-legible-membrane-plan.md` (added in this PR). Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus. Three defects a stranger hits today, all verified on `main` before the change: 1. **Codex projection truncated 51 of 56 skill descriptions mid-clause** (`scripts/codex-sync.sh` capped prose at 44 chars, so the Codex router read "Freshly judge whether a finished change is Triggers: …"). Now: first sentence of the source prose plus the full `Triggers:` clause, abbreviation- and quote-aware; the per-catalog bound is computed live as "Codex prose average may not exceed Claude's" (cross-multiplied, floor-free) with a 180-char hard ceiling. `using-flywheel` is exempted from runtime-phrase rewriting so its cross-runtime text projects verbatim. The dormant `ao codex ensure-start` emitter and the `# /x`→`# $x` title rewrite are removed. A literal oracle pins five twins and a rule oracle covers all 56. 2. **23 shebang-bearing shell entry points were committed non-executable**, including `scripts/regen-all.sh` and three gate scripts. Now `100755`, with an advisory gate `shell.exec-bits` (fast+full, fail-closed on enumeration error, reads the index blob, skips symlinks by stated policy). 3. **`tests/run-all.sh` was red on `main`**: the GOALS validator asserted a pre-08-25 file shape, and validate's description exceeded the 180-char budget. The validator now parses only the `## Gates` block (stops at any heading, like production), requires cited script paths to exist, takes its path via argv, and fails closed on tokenization; four negative fixtures. validate's description is 177 chars and gains the reality-check negative. `AGENTS.md` quotes CI's authoritative bats and Go commands. README and `docs/install-day2-ops.md` replace "No other runtime is required" with a runtime table derived by reading each skill's procedure (HARD / OPTIONAL / conditional), covered by a bats test that grounds every HARD row in an invocation line. Evidence on the tip: full gate 71/71 (HEAD binary), CI's bats command green across `tests/scripts/*.bats`, `tests/run-all.sh` green, Go build/vet/test green, golangci-lint clean, security gate quick PASS, `scripts/regen-all.sh --check` clean. Each lane was validated by a fresh context; the integrated train had two cross-family (Codex) review rounds and every finding is closed on this tip. Out of scope (successor intent): promoted set / `skills-internal/`, the process-artifact sweep, "It's working if" blocks, routing clusters, doctrine diet. One pre-existing drift noted for a follow-up: `packs/agentops-executor/agents/validator/skills/validate/SKILL.md` carries a stale third description that no gate binds. |
||
|
Bo
|
e69144d6d1 |
Train 2: seeded-defect probes + gate hardening — the instrument catches its own contamination (#1088)
> **⚠ Open review residues — merge is Bo's call (spiral stop: second
non-PASS on this train's intent).** The cross-family re-check affirmed
the substance (contaminated fixture reproducibly UNMEASURED with four
real reads; all six scorecards match fresh replay; coverage honestly
12/12; single-level rule strictly tightening; 70/70 gates; full suite,
lint, validators green) and states plainly that **the zero-result
acceptance miss is not the reason for failure**. What failed, each with
its fix:
> 1. `evals/skill-probes/LEDGER.md:71` + `RUNBOOK.md:75` — a summary
block my withdrawal sweep missed still says "Exactly one row was earned…
1/12… council SEPARATED… worked for one of four". Fix: rewrite that
block to the post-withdrawal truth (zero rows earned, 12/12, council
UNMEASURED pending a second level).
> 2. `scripts/lib/probe-fixture-metadata.py` — the contamination
docstring overclaims: "a failed attempt loaded no bytes" is false for
`cat …SKILL.md; false` (read succeeds, compound exits nonzero), and
glob/copy-then-read indirection evades. Fix: state the detector's honest
boundary (exact-string, final-exit heuristic; evasions listed) — the
mechanism is a floor, not a seal, which the RUNBOOK already says.
> 3. **Caller decision:** the `scripts/probe-skill.sh` stderr-literal
exclusion is validator-verified fail-closed but sits outside L2's
declared write scope — it needs your explicit scope blessing (or
reversion, which re-voids all live dispatch on codex-cli ≥ 0.14).
> 4. Minor, for the record: my commit message reports security-gate
quick PASS (true in my run); the re-checker's environment reported
WARN_QUALITY from a missing radon + a govulncheck toolchain mismatch
unrelated to this diff — and the grounding gate's dead-SHA class is
UNCHECKED on shallow clones, including both our checkouts.
> **Acceptance status, told straight: L2's plan clause ("≥ 2 skills move
off UNMEASURED") is NOT met — the honest count is zero.** The lane filed
one BEHAVIORAL row (premortem); its fresh validator dumped the committed
transcripts and proved every rep, both arms, had fetched the skill off
disk mid-run — the separation was band-spray variance between two arms
holding the same bytes. The row is withdrawn, the ledger honestly reads
12/12 unmeasured again, and the train's deliverable became something
better than the number: the harness now mechanically catches that
contamination class forever. Merge is Bo's call.
## What
Train 2 of the [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md):
seeded-defect probes for the judgment spine (L2) and gate hardening
(L3), plus one repair commit responding to L2's validator findings.
**L3 — gate hardening** (`instrument/gate-hardening`, lane verdict
**PASS 11/11**)
- `gate.tightening-ratchet` (advisory): diffs `BASE_REF...HEAD` and
fails on six textual loosening shapes in the gate surface unless a
`Gate-Loosen-Reason:` trailer justifies it; tightening always free.
Detection boundary documented in the script header — silent on semantic
weakening it cannot parse, fail-closed on what it can.
- `evidence.grounding` (advisory): mechanical scan of `docs/audits/**` +
`docs/evidence/**` for cited paths that don't exist, unresolvable
full-length hashes, and scaffold leaks. 45 real findings on the live
tree, each **baselined with its argument** (directory-pinned baseline
with the added-line rule as teeth: a finding on a newly-added line is
never excused). One genuine live defect surfaced:
`docs/evidence/membrane-receipts.md` claims a generator that no longer
exists.
- Registry 68 → 70; `ao gate check --full` 70/70.
**L2 — seeded-defect probes** (`instrument/seeded-probes` + repair
`927be2184`)
- Four tier-2 probe packages (validate, premortem, council,
one-way-door): sub-40-line realistic artifacts, two planted defects
each, `canonical-skill` treatment, act-grading discriminators calibrated
both directions (21 hand-written transcript cases) — the lane's
validator wrote its *own* counterexamples and the discriminators held.
- 28 live dispatches (gpt-5.6-luna, low + xhigh) through the sanctioned
executor with `CODEX_HOME` isolated. Six immutable v3 fixture sets
replay deterministically.
- **What the run actually measured:** `one-way-door` and `validate`
scenarios SATURATED (retired to the RUNBOOK, no ledger rows — the
headroom gate doing its job); `council` was measured at one level and
the lane *refused* to file the row the gate's labeling bug would have
allowed ("filing it would have laundered a void row into the acceptance
number"); `premortem`'s BEHAVIORAL row was filed, then **withdrawn**
when the validator proved contamination.
## The repair commit (one bounded round, per the plan's validation
strategy)
- **`skill-read-contamination`**: `classify_bytes` now degrades any rep
whose transcript shows a successful command reading a `SKILL.md` — one
insertion covering live and replay. RED proven against the committed
contaminated fixture (replayed BEHAVIORAL before; UNMEASURED 0/0 after),
bats case added (probe-skill 37/37). The six scorecards were false
records under the rule: deleted (the harness's immutability guard
rightly refuses overwrite; git keeps the bytes) and regenerated —
contamination turned out broader than the lane knew (validate lost both
arms at xhigh, one treatment rep at low; one-way-door lost an xhigh
rep).
- **Headroom gate single-level fix** (L2's disclosed gate finding): a
group measured at one effort level whose control aced it is now
UNMEASURED ("capture a second level before any verdict row"), not
SEPARATED. RED test first; strictly tightening; the council group now
reads honestly.
- LEDGER row → WITHDRAWN with the full story; RUNBOOK's two falsified
sentences corrected; `CODEX_HOME` isolation documented as proven
insufficient, the transcript-level trap as the floor until dispatch is
sealed at the filesystem.
- Ride-along accepted with justification (validator-verified
fail-closed): the harness's stderr rule now excludes exactly one
whole-line literal — codex-cli ≥ 0.14's own "Reading prompt from
stdin..." announcement, which was degrading 100% of live reps on a clean
exit.
## Evidence
Full Go suite exit 0 · go-lint clean · `ao gate check --full` **70/70**
· security-gate quick **PASS** · four skill validators · probe-skill
37/37 · headroom 10/10 · tightening-ratchet + grounding bats green ·
`regen-all.sh --check` current. Per-lane fresh validators (L3 PASS; L2
FAIL → this repair); cross-family Codex verdict on the integrated train
recorded in the session report.
## What this train proves
The membrane worked on its own measurement pipeline, three layers deep:
the lane refused to launder a void row; the fresh validator caught
contamination the lane's isolation missed; the repair turned that catch
into a permanent mechanical trap. The cost is the honest headline — zero
skills measured — and the asset is that the *next* live wave runs
against a harness that cannot silently repeat any of this.
## Follow-ups
- Seal live dispatch at the filesystem (arms must not be able to read
any skill corpus) — then re-run the premortem and council waves for real
measurements; council additionally needs its second effort level.
- `docs/evidence/membrane-receipts.md` generator claim (surfaced by
evidence.grounding).
- The `Gate-Loosen-Reason` ratchet does not yet cover
`scripts/probe-skill.sh` — the stderr-literal class belongs under it.
---------
Co-authored-by: Bo <bofuller55@gmail.com>
|
||
|
Bo
|
8cdcb5a903 |
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`): the round-2 findings were the same failure shape — round-1 repairs patched cited lines instead of sweeping the class — so this commit sweeps each file whole: every remaining SATURATED-row-append site in skill-eval now routes to RUNBOOK retirement, the human-only-skills *description* is runtime-conditional, premortem's "(MEASURED)" label is gone, SKILL-API's context table carries all 25 rows and the enforcement table gains `disable-model-invocation`, and the fixture-identity claim is stated precisely (probe id, honesty note, and control arm are the only differing fields — as the acceptance permits). Post-sweep: validators, full Go suite, 68/68 gates, goldens + headroom bats green, projections current, gemini in sync. Merging per Bo's instruction. ## What Train 1 of the accepted [instrument-panel roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent landed at `986a4feaf`): the measurement substrate, the skill-context diet, and the retrieval-eval contract. Three worktree-isolated lanes, each independently validated by a fresh context, plus one integration commit. 103 files, +5,510/−76. **L1 — measurement substrate** (`instrument/measurement-substrate`) - Gate `skill.probe-headroom` (advisory, Fast|Full): answers the question `skill.probe-coverage` cannot — not "does a probe result exist" but "could one have existed at all". The rule, ported to Go (`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin check script — the witness-crosscheck pattern, **no new `ao` root command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels ⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it; treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed fixture pairs read `INERT` to everything else in the repo; the failing separation test predates the implementation, and a bats negative-control swaps fixture bytes and asserts the gate flips. - **First reading on real data: 7 of 11 historical probe groups are SATURATED** — including both `validate-not-proven` runs. Those INERT rows were never honest nulls; they were void. The 0/12 ledger number now argues itself. - Declared denominator for probe-coverage: `scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry without an argument, stale slug, or duplicate ⇒ exit 2). One entry (`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12 → 0/12: alias left, `one-way-door` entered) — the gain is a declared number, not a better-looking one. - Re-landed from the recovered clean-room commit (`9872483bd`), re-validated against *current* main: `skill-eval` (defers saturation to the gate id; its shell scripts dropped, not shipped — ratchet intent), `route`, `one-way-door`, premortem reversibility check, council `caller_challenge` (schema + validator, per the agent-core boundary that the panel may challenge, never overrule). **L4 — context diet** (`instrument/context-diet`) - `disable-model-invocation: true` on 4 human-only skills (key verified verbatim against Anthropic's docs). The plan guessed 35 candidates; the graph said otherwise — 23 carry `user-invocable: true`, and 19 of those are excluded on cited evidence (rpi consumes anti-ceremony/implement/plan/validate; workflow scripts reach others; `goals` is a live migration tombstone). The exclusion evidence is retained in the lane report. - One router skill (`human-only-skills`) — the single always-loaded description that replaces four; it hints, never fires. - `.out-of-scope/` formalized with this week's three refusals (checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B as the measurement unit), each citing its evidence. - Deterministic proof, no model eval: before/after bytes of always-loaded description load reported in the lane summary. **L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane verdict PASS 10/10) - `AGENTS.md` federated row now names **ee (eidetic-engine)** as a concrete caller-selected memory system — consume, never build; symlink intact. - `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens + `scripts/check-routing-probe-goldens.sh` graded against `ao skills find`, wired as an **advisory** nightly job. Zero goldens is a failing state — no new zero-denominator green. - **The instrument caught a real miss on day one — and its own prescription fixed it.** Golden `rq-04` expects `validate` for "judge whether this finished change is actually proven before I merge it"; at authoring, `ao skills find` ranked the *forbidden* `premortem` first and `validate` nowhere in six natural phrasings. The pointer-wording-first repair (validate's description gained the caller's own words: finished, proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the golden pins the repair — a description regression reopens it. ## Integration `regen-all.sh` once over the merged lanes (catalog 52 → 56, four new codex twins, mesh, router, manifests); `skills/route/SKILL.md` catalog/router links became prose repo-root references (the projected twin cannot resolve `../catalog.json` — this was both the portable-conformance failure and the sole broken doc link); `codex-portable-conformance.bats` pin 52 → 56. ## Evidence - `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 · `ao gate check --full` **68/68** · four skill validators PASS · probe-headroom / routing-goldens / probe-coverage bats PASS · `regen-all.sh --check` all current. - Per-lane fresh validators re-ran every suite on detached content; L5 PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved for integration (their remaining acceptance observed green), settled above. Cross-family (Codex) review of the integrated diff recorded in the session report. - Two disclosed scope stretches accepted at integration: a one-line `.gitignore` entry mirroring the witness-crosscheck precedent, and the probe LEDGER.md fact-correction L1's own change made necessary (noted for Train 2's L2, which owns that file next). ## Cross-family review (Codex, fresh context) Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the control arm; the goldens grader was red where the plan's acceptance says green) and eight majors (contract contradictions in the re-landed skills, a converter-substitution false claim in the codex router twin, two overreaching `.out-of-scope` entries, stale SKILL-API counts). All repaired in one bounded round (`db68935a3`): fixtures now byte-identical outside the control arm, the routing miss actually fixed rather than tolerated, every cited contradiction reconciled at the source and re-projected. Post-repair: full Go suite exit 0, `gate check --full` 68/68, all validators and probe/goldens bats green, projections current, gemini byte-identity restored. Focused re-check verdict recorded in the session report. ## Follow-ups (Train 2, already planned) Seeded-defect probes for the judgment spine (every ledger row citing a passing headroom pre-screen) and the gate-hardening pair (`Gate-Loosen-Reason` tightening ratchet; mechanical grounding-validation over evidence docs). Plus, surfaced by this train: a latent `valid_keys`/schema divergence in `validate-skill-schema.sh` (two keys the schema defines are absent from the script's allowlist — pre-existing). |
||
|
Bo
|
2207c5b278 |
estate: goals fail-closed, contract map, verdict guard, workflow + docs honesty (5 lanes) (#1083)
> **Review findings closed.** The re-check's residue (`perl -pibak` bundled-cluster bypass) is applied in `2675bb383`: the perl alternation accepts any non-space run after the in-place flag, a new FIRE bats proves the deny, 48/48 policy-dispatch bats green, codex twins regenerated. Merging on Bo's instruction. ## What Five semantic lanes from the 2026-08-23 estate audit, each implemented in an isolated worktree and independently validated by a fresh context, plus one integration-fixup commit. **1. Goals: the zero-denominator green is dead** (`estate/goals-honest`) `GOALS.md` lost its Gates table in the 07-14 cut, so `ao goals validate` reported `Valid: true` with `GoalCount: 0` in the release smoke on every push. Now: zero parsed goals ⇒ `Valid: false` with an actionable error (RED test first); `GOALS.md` carries a real 6-row Gates table whose every `Check` was executed via `ao goals measure` before shipping; `goals-affects-files.yaml` pruned of dead ids; the smoke asserts `goal_count ≥ 1`. Two of the eight fitness properties (behavior-before-activity, stop-boundary) have no executable check yet — disclosed in GOALS.md rather than faked. **2. Front door: the contract file gets a map** (`estate/front-door`) `AGENTS.md` had zero build/test commands and zero layout — the Go mechanics lived only in `.claude/rules/go.md`, which Codex (the declared default runtime) never loads. Added a terse "Repository map and mechanics" section; compressed the architecture-poetry opener to plain language; README opens with what-it-does, the maintainer binary install moved to `docs/install-day2-ops.md`, and the quickstart now ends with an invitation to open an issue with your verdict. Deleted the dead `.claude/CLAUDE-base.md` + `build-claude.py` pipeline (zero consumers since June). `CLAUDE.md` remains a symlink to `AGENTS.md`. **3. Hooks: first policy that guards the user's invariant** (`estate/hooks-invariant`) All four shipped policies guarded this repo's own artifacts. New `core.verdicts:hand-edit` denies Edit/Write/Bash hand-writes into `.agents/ao/verdicts/**` — verdicts are content-addressed evidence written only by the validate flow. RED committed before GREEN (`d264682c0`); bats prove deny on hand-writes and allow on reads, intents, and the sanctioned `validate.py` writer. The lane shipped with in-place editors disclosed as un-matched; the cross-family review treated that as a blocker and the gap is closed in this PR (see below). **4. bdd-foundry stops instructing dead commands** (`estate/workflows-fix`) The R4 phase told agents to run `ao yield emit`; Phase 5 said pull holdouts via `ao scenario`; comments cited `ao membrane derive-checks` — all tombstoned, so R4 failed on every run. Emission is dropped per the tombstone's guidance (observation stays in the workflow result), Phase 5 grades in-run ground truth plus any caller-supplied register scenarios, and the lane's initially-overreaching "there is no external holdout register" wording was caught by its validator and corrected in integration. Bonus: `workflow-scripts-syntax.yml` globbed the gitignored `.claude/workflows/` (matched nothing in CI since inception) — now checks the tracked `workflows/` tree. **5. Docs honesty: retired vocabulary stops reading as live** (`estate/docs-honesty`) `intel_scope` removed from all 24 source SKILL.md + fixture + template + tutorial (declared by half the corpus, read by zero Go); the schema keeps accepting it for old-shape third-party skills but now self-describes as deprecated. `docs/SKILL-API.md` no longer presents `INTEL`/flywheel/forge/push/ratchet as live. `docs/knowledge-flywheel.md` states design-vs-implemented truthfully. ADR-0014's dead link fixed. 25 drifted `images/gemini/skills/` copies re-synced byte-identical per that bundle's own README contract. All codex projections regenerated once at integration (`regen-all.sh --check` passes). ## Evidence - `cd cli && go test ./...` exit 0 · 4 skill validators PASS · `policy-dispatch.bats` + `cross-runtime-hook-baseline.bats` PASS · `check-doc-hooks-drift` + `check-hookless-cold-start` PASS · `validate-goal-count` PASS · `node --check` on all 8 workflow scripts PASS · `regen-all.sh --check` PASS - From repo root: `ao goals validate --json` → `valid: true, goal_count: 6`; from a wrong cwd it now fails closed instead of reporting green. - Each lane branch carries its own commits; merge commits preserve the lane structure. Fresh per-lane validators re-ran every suite themselves; the one FAIL each (workflows wording, cross-lane doc fallout) was fixed in the integration commit, not waived. ## Cross-family review (Codex, fresh context) First pass: **FAIL** with one blocker — the new verdict guard explicitly excluded in-place editors, so `sed -i` silently mutated digest-named evidence (dispatcher probe returned exit 0). Repaired in `f53945c26` (one bounded round): three FIRE bats observed RED before the matcher landed (in-place sed, in-place perl, file deletion), two negative bats prove reads never fire; policy rationale and SKILL.md updated; codex twins regenerated; 47/47 bats green. Everything else in the first pass was affirmed (goals behavior, symlink/build bar, zero retired verbs in workflows, byte-identical gemini copies, current projections). Focused re-check: blocker confirmed closed via the dispatcher probe; one regex corner (`perl -pibak`, header note above) stopped the loop under the spiral rule. ## Follow-ups (not in this PR) - `workflows/bdd-foundry.js` `SKILL` constant points at `skills/behavior-first-planning/SKILL.md`, which was retired in v3.3 — every phase prompt references a dead path (pre-existing). Needs a design call: restore the phase discipline inline or repoint honestly. - Verdict-guard: the noclobber override redirect (`>|`) remains the one disclosed unmatched write shape. - `docs/UPGRADING.md` note for third parties still declaring `intel_scope` (schema accepts, nothing reads). |
||
|
Bo
|
ffb9f122af |
refactor(cli): delete the unconsumed eval/redact surfaces — the estate audit's mechanical cut (#1082)
> **Review findings closed.** The re-check's residue (app-seam family count) is applied in `9a2790ae7` along with the full-tier CI settlements: regenerated documentation index (generated file, hand-edit drifted it), regenerated CLI-surface count fixtures (top=18 sub=44 all=62), `Test-Removal-Reason` trailer for the deliberate test deletions, and the release-tag bats output list updated to the real changes-job set. 67/67 full-tier gates green locally. Merging on Bo's instruction. ## What Deletes the provably-dead 28% of the `ao` CLI and every reference to it, per the 2026-08-23 estate audit. −19.5K lines in the lane commit plus integration fixups. **Removed (each with zero live consumers, verified by consumer-grep + `go list -deps`):** - `ao eval` — 13 subcommands, ~10.9K LOC. Its would-be consumers were already tombstones (`scripts/eval-agentops.sh` printed `RETIRED`), `release.yml` hardcoded `--eval pass`, release evidence recorded `suite_count: 0`, and three of its module tests exercised subcommands that could never register (nil composition seats). - `ao redact` — its only declared caller (`skills/compile/scripts/compile.sh`) never existed. - `cli/internal/types/memrl_policy.go` + the orphan cascade it and eval left behind (`internal/scenario`, `internal/wiki`, `internal/runtimecmd`, `internal/redact`) — all with zero importers, verified before and after. - `scripts/check-memrl-health.sh` + `examples/schedules/feedback-drain-hourly.yaml` — a health check for the feedback loop amputated on 2026-07-14; it exits 1 on main today and the example instructs a verb (`ao feedback-loop`) that no longer exists. - `corpus.secret-scan` gate — vacuous: its file filter excluded the single tracked path its globs could match, so it scanned zero files; secrets are covered by the pinned gitleaks steps in nightly and release (validate's quick toolchain mode skips gitleaks). - Docs for the deleted surface: `docs/architecture/eval-architecture.md`, `docs/code-map/eval-lid-primitives.md`; `contracts/eval-baseline-ab.md` already carried a RETIRED banner and stays as history (delisted from the live index). **Kept, deliberately:** - `ao robot-docs` — the audit's "duplicate of `doctor robot-docs`" premise was false: they render different handbooks (whole-CLI vs doctor-scoped). Verified before acting. - `completion`, `demo`, `quick-start` — interactive human furniture, not dead code. - `corpus.witness-dolt-jsonl-crosscheck` gate — retargeted, not retired: its backing script is a hermetic self-test over real tracked fixtures; globs now point at the paths it actually exercises. - `cli/internal/evalsubstrate` — Go-dead but it is the declared mirror of `schemas/outcomes-rubric.v1.schema.json`; retiring it needs a paired schemas/docs/scripts decision (package doc comment records this). - `scripts/ci-local-release.sh` eval-evidence stanza — self-contained honest bookkeeping (`status: not_applicable`), invokes nothing removed. **Tombstones + migration:** `eval` and `redact` added to `removed_command_hint.go` and `docs/MIGRATION.md`; the now-false "(`ao eval` returned in 3.3 …)" parenthetical deleted; `go-cli.md` spine and the "Eval — the Learn seat" section updated; the dated research snapshot got a HISTORICAL banner via the docs-scope self-declaration mechanism (history not rewritten). ## Why v3.6.0 binary downloads: 4 darwin-arm64, 3 linux-amd64. Only 7 of 53 shipped skills invoke `ao` at all, and none of them touch this surface. The eval family was the single largest command surface in the CLI with zero live consumers — 28% of non-test Go maintained for nobody. ## Evidence - `cd cli && go build ./... && go vet ./... && go test ./...` — exit 0 (previously-failing `TestGoCLIDocSpineMatchesApprovedSpine` and `TestRemovedVerbsHaveMigrationRows` now pass) - `scripts/check-docs-cli-snippets.sh` PASS · `check-cmdao-surface-parity.sh` PASS (54 leaf commands) · `check-corpus-path-guard.sh` PASS · `check-new-scripts-use-preamble.sh` PASS · `ao gate check --dry-run` PASS - Implemented by a worktree-isolated lane, independently validated by a fresh context that re-ran the suite itself; the two failures it found were doc files outside the lane's write scope, fixed in the integration commit. Cross-family (Codex) review verdict included in the final session report. ## Cross-family review (Codex, fresh context) First pass: **FAIL** with two majors — (1) `quality.DeprecatedCommands` still mapped five rewrite entries onto the removed eval family, so `ao doctor --fix` would have introduced dead commands; (2) retained docs (formal-verification research links, applied-ood README run block, evalsubstrate hint strings) still prescribed removed commands. Both repaired in `4da85a0d4` (one bounded round), plus its two minors (types/AGENTS.md row, .gitignore unignore, family counts, gitleaks-coverage comment). Re-verified: full suite green, snippets gate PASS. Focused re-check: first-round findings confirmed closed; one new residue (the family count above) stopped the loop under the spiral rule. ## Follow-ups (not in this PR) - `cli/internal/quality/stale_refs.go` `DeprecatedCommands`: the five eval-target entries are pruned here; the older pre-existing dead targets (forge, inject, flywheel, ratchet, …) still need a map-wide reconciliation against the live registry. - `cli/internal/evalsubstrate` retirement decision (paired schemas/docs/scripts change). - `evals/scenarios/applied-ood/`, `evals/tier2-premortem/`, `evals/_stats/` retain historical `ao eval` mentions in prereg/holdout records — dated artifacts, left as history. |
||
|
Bo
|
c516dd55a9 |
fix(skills): preserve portable Codex activation metadata (#1069)
## What changed - preserve canonical trigger text while compacting generated Codex descriptions - harden portable skill validation for duplicate YAML keys, symlink containment, and current optional-field rules - regenerate all owned Codex projections and hashes - add focused negative and generator regression coverage ## Verification - portable conformance: 52/52 - portable Bats: 8/8 - generator acceptance: 18/18 - `scripts/regen-all.sh --check` - Bash syntax, blocking ShellCheck, and `git diff --check` The interrupted all-skills remediation snapshot is deliberately excluded: it remains preserved on `codex/all-skills-pass-20260816` and is not merge-safe. |
||
|
Bo
|
f9e246d9fd |
fix(skills): make audit grades evidence-honest (#1066)
Audit all canonical skills against a current evidence rubric, distinguish static readiness from safety and effectiveness, strengthen scorer/schema truthfulness, and publish bounded remediation. |
||
|
Bo
|
f3c6d0ecf2 |
Converge retained WIP and harden evidence boundaries (#1065)
Summary:
- lands the audited current WIP lanes and excludes stale/process-only
material
- hardens prune path confinement, probe-v3 evidence binding,
codebase-recon identity, handoff/release/reverse-engineer behavior, and
Codex prompt handling
- truth-labels static skill scoring and regenerates all owning
projections
Validation:
- fresh independent PASS on commit
|
||
|
Bo
|
8694b07ebf |
feat(skills): add anti-ceremony guard to RPI (#1064)
## Summary\n\n- add a clean-room, artifact-free anti-ceremony skill\n-
make RPI invoke its STOP/CONTINUE guard exactly once before Plan\n- STOP
before dispatching any core phase; CONTINUE preserves Plan → Implement →
fresh Validate\n- retain durable intent by reference and snapshot only
non-durable intent\n- update the canonical traversal, workflow,
standards, dependency contracts, tests, and generated projections\n\n##
Verification\n\n- RPI unit tests: 13/13\n- dependency/product-boundary
Bats: 15/15\n- RPI and Anti-Ceremony validators: PASS\n- Cathedral Cut
conformance: PASS\n- mesh, Codex/Gemini projections, parity, hashes,
manifests, and runtime formats: PASS\n- changed-scope
generation/contract checks: PASS\n- full exact-range gate: 68 pass, 0
warn, 0 fail, 0 unknown, 0 skip\n- fresh final semantic review on head
|
||
|
Bo
|
62cc3b6ee0 |
Clear the operations-layer alignment residuals (#1054)
Closes out the six residual items #1051 disclosed: terminology residue on non-authority surfaces, the eval command-surface fixture that failed when executed (#{3,4} -> #{3,5}), the vacuous retrieval-quality canary and the nightly job that ran it, the consumer-free dream config block and its exclusive helpers, the remaining knowledge-shaped writers moved to the scratch tier, and the MEMORY.md consumer audit. bin/ralph still resumes legacy .agents/ralph/ checkpoints so the documented backwards-compat contract holds without a migration; both paths and the outside-both refusal are now tested. Fresh author-distinct validation returned PASS with empty not_checked, after an earlier revision failed on a dangling nightly invoker and a back-compat test regression that were fixed and independently re-verified. Test-Removal-Reason: the dream config subsystem was deleted with its tests (operations-layer residuals) |
||
|
Bo
|
7a765cde19 |
Align AgentOps around its operations-layer identity (#1051)
Executes docs/plans/2026-08-07-agentops-operations-layer-alignment.md: AgentOps is the operations layer for agentic engineering; the federated integration graph is the topology, the semantic work-and-proof protocol is the contract, and RPI is the standard one-experiment traversal. Retires the ao flywheel command family and all knowledge-flywheel product state, tombstones the seven-move operating-loop workflow, narrows ao init and the .agents state writers to declared destinations, renames the core architecture page to rpi-traversal.md with a compatibility redirect, aligns AGENTS.md, 25 skills, public and package copy, regenerates every owned projection, and strengthens the conformance gates with planted-negative proofs. Both the alignment subject and the follow-up gate-bookkeeping commit carry fresh author-distinct validation PASS verdicts with empty not_checked scope. Test-Removal-Reason: dead knowledge-flywheel and session-store surfaces were deleted with their tests (operations-layer alignment) |
||
|
Bo
|
8462893486 |
ci(release): goreleaser snapshot smoke between releases (#1030)
Closes the release-record commitment carried across three releases
(`docs/audits/release-readiness-v3.3.0.md`, "Process lesson"): **CI must
exercise the goreleaser release path between releases.** A retired
`.goreleaser.yml` `before.hooks` entry once sat broken for 20 days
because only a real `v*` tag runs that path.
## What lands
- `.github/workflows/release-path-smoke.yml` — full GoReleaser snapshot,
publishing skipped.
- `tests/scripts/lib/release-snapshot-smoke.sh` — the one smoke path,
shared by CI and its negative witness.
- `tests/scripts/release-path-smoke.bats` — the negative witness (10
tests).
- `.github/workflows/nightly.yml` — calls the smoke via `workflow_call`,
wired into the summary table + failure-issue condition.
- `CHANGELOG.md` + `docs/CHANGELOG.md` — `[Unreleased]` entry
(byte-identical mirror).
## Exact invocation, and why
```
goreleaser release --snapshot --clean --skip=publish --config <cfg> --timeout 20m
```
Installed with the **same** pinned
`goreleaser/goreleaser-action@f06c13b6… # v7` and `version: '~> v2'`
that `release.yml` uses (`install-only: true`), so the smoke exercises
the binary the publisher would actually get.
**`goreleaser check` is deliberately not the gate.** Verified against
the repo's config on goreleaser 2.16.0: `goreleaser check` exits **1**
on deprecation warnings alone (the config's `brews:` block), so it
cannot gate without changing release behavior — and it never runs hooks,
builds, archives, or the Homebrew formula template, which are precisely
the stages that broke. The snapshot exits 0 on the same config. So the
evidence below is from the **real snapshot**, not the cheap proxy.
## Acceptance → evidence
**1. Nightly + release-plumbing-input triggers.**
- Nightly: `nightly.yml` gains `release-path-smoke: uses:
./.github/workflows/release-path-smoke.yml`, added to `summary.needs`,
the summary table, and both failure conditions.
- PR/push: `paths:` filter on `.goreleaser.yml`,
`.github/workflows/release.yml`, `cli/go.mod`, `cli/Formula/**`, plus
the smoke script and its bats file (so edits to the gate re-run the
gate). A **dedicated workflow** rather than a `validate.yml` filter
branch — same isolation rationale as `workflow-scripts-syntax.yml` ("no
shared job registry to drift"), and it keeps `validate.yml`'s `changes`
output list untouched.
- **Runtime choice: full six-target matrix, no split.** Measured
locally: `11.5s` wall for `darwin/linux/windows × amd64/arm64`
end-to-end (build + archive + checksums + formula render). A
single-platform snapshot would leave five publisher targets unexercised
for no meaningful time saving. Job `timeout-minutes: 20`.
**2. Negative witness.** `tests/scripts/release-path-smoke.bats` runs
**the CI smoke path itself** against deliberately broken temp copies of
`.goreleaser.yml`:
- `red: a retired before-hook makes the release path FAIL` — reproduces
the exact 20-day bug (config prepended with `before: hooks: -
./scripts/this-script-was-retired.sh`), asserts `status -eq 1`.
- `red: a structurally invalid config makes the release path FAIL` —
asserts `status -eq 1`.
- Plus usage-error legs asserting **exit 2** so "goreleaser missing /
bad flags" is never mistaken for "release path broken".
```
$ AGENTOPS_RELEASE_SMOKE_FULL=1 bats tests/scripts/release-path-smoke.bats
1..10 … ok 1 … ok 10 # all 10 pass, both RED legs fire
```
The job is workflow-only (no new registry gate), so per the intent I
confirmed the ratchet stays green:
```
$ cd cli && go test ./internal/gates/checks/ -run TestBlockingGatesHaveProvenNegativeWitness
ok (1 passed)
```
**3. Passes against current main's config.** The green leg is the real
snapshot, not `goreleaser check`:
```
$ bash tests/scripts/lib/release-snapshot-smoke.sh
… building binaries (6 targets) … archives … checksums … homebrew formula
release succeeded after 11s
OK release path built end-to-end (publish skipped) # rc=0
```
Also covered as the bats `green:` test (gated behind
`AGENTOPS_RELEASE_SMOKE_FULL=1` so the generic `bats
tests/scripts/*.bats` job stays fast; the smoke workflow sets it).
**4. CHANGELOG.** `[Unreleased] → Added` entry in `CHANGELOG.md`,
mirrored byte-identically to `docs/CHANGELOG.md` (`diff` clean).
## Local gates (all rc=0)
| Check | Result |
|---|---|
| `cd cli && go build ./...` | 0 |
| `cd cli && go vet ./...` | 0 |
| `cd cli && go test ./...` | 0 |
| `bash scripts/regen-all.sh --check` | 0 — "All generated projections
are current." |
| `bats --jobs 4 tests/scripts/*.bats` | 0 — 953 ok, 0 not ok |
| `ao gate check --full --workflow-coverage --require-workflow-parity` |
0 — 68/68 passed |
| `shellcheck tests/scripts/lib/release-snapshot-smoke.sh` | clean |
| YAML parse (both workflows) | OK |
## Scope
Write scope honored: `.github/workflows/**`, `tests/scripts/**`,
`CHANGELOG.md`, `docs/CHANGELOG.md`. **`.goreleaser.yml` is
read/verified only — not modified.** No `cli/**` Go code, no
`skills/**`.
## Residual risk
- The `brews:` deprecation is left in place (out of scope). It is a
warning today; if GoReleaser v3 removes it, this smoke is exactly what
will surface it between releases instead of at tag time — which is the
point.
- Nightly `workflow_call` inherits caller permissions; the reusable
workflow declares `permissions: contents: read` and the nightly job pins
the same.
|
||
|
Bo
|
e2e2e60aa2 |
chore(gc): decay retired Gas City prototype wiring to frozen bytes
The 2026-07-29 upstream-factories pivot retired packs/agentops-executor, packs/agentops-factory, and deploy/gc in place as migration evidence, but three live wires kept regenerating and gating them: - scripts/sync-gc-pack.py re-projected canonical skills into the retired executor pack on every regen. Remove the regen-all step and delete the projector + its test; the pack copy and manifest stay as frozen historical bytes (untouched by this commit). - adapter.gc-executor spanned packs/** and deploy/gc/**. Replace it with adapter.gc-maintainer, narrowed to the live surface only: scripts/gc-maintainer-ops.sh + tests/python/test_gc_maintainer_ops.py, backed by new scripts/check-gc-maintainer-ops.sh. - tests/python/test_gc33_thin_pack.py coupled the LIVE skills/using-gc skill to retired pack bytes (asserted matching strings in both), so any future skill edit would break against the frozen copy. Deleted with the gate that ran it. The new gate ships with a red bats case naming its backing script, so the adapter.gc-executor grandfather line is pruned rather than replaced — the negative-witness ratchet shrinks by one. Verified: go build/vet + gates suite (174 pass, includes the witness closure test), new bats twin 3/3, regen-all --check green, shellcheck + preamble ratchet green. workflow.install-drift fails pre-existing (dangling ~/.claude/workflows/bdd-foundry.js estate symlink, unrelated). |
||
|
Bo
|
520911899b |
test(gates): scrub inherited AGENTOPS_GATE_RANGE in pairing self-test (#1014)
Found during the v3.4.0 official-mode readiness run: exporting `AGENTOPS_GATE_RANGE=v3.3.0..HEAD` (the documented way to scope `check-go-command-test-pair.sh` to the release train) leaks into the gate's own self-test fixtures, where the range cannot resolve — expected-FAIL scenarios become SKIPs and the self-test fails. Fix: the self-test unsets the inherited range; fixtures own their scope. Same env-hermeticity class as `ScrubGitDiscoveryEnv`. Proof: `AGENTOPS_GATE_RANGE=v3.3.0..HEAD bash tests/scripts/test-go-command-test-pair.sh` → 5/5 PASS (previously failed); clean-env run unchanged 5/5. |
||
|
Bo
|
a6359795bf |
Make fresh validation persistence optional (#1012)
Keep fresh author-distinct validation mandatory while making verdict and report persistence consumer-driven. Align the RPI/Validate contracts, executable behavior, current guidance, regression coverage, and generated projections; preserve the Gas City 1.4 cutover. |
||
|
Bo
|
69f055f8a2 |
W3a judgment wave: real output contracts, honest effects, scratch-tier artifact dirs (#1000)
## Summary Wave W3a of the skill-overhaul reboot (`age-skill-overhaul-reboot-sjv7v.4`, judgment half) — six judgment skills, every checklist item verified against the live tree first. - **council** — the dangling `council-report.v1` contract is now real: skill-local schema + jq `validate-output.sh` (rejects smuggled verdict/readiness fields, single-judge panels, empty findings) + validator contract guard; boundary broadened to "does not mint a verdict of any version"; negative trigger, judge-timeout failure semantics, output location under `.agents/scratch/council/`. - **reality-check** — same treatment for the dangling `reality-check-report.v1`; adds no-verdict boundary and untestable-claim failure line. - **postmortem** — `output_contract` now names the real markdown report (was a `.feature` path); `effects: []` → `[write_postmortem_report]`; artifact dir moved to the ADR-0016 closed set (`.agents/scratch/postmortem/`). - **idea-genie** — artifact dir moved to `.agents/scratch/ideas/<run-id>/` with bats fixtures updated. - **premortem** — boundary fixed to deny any verdict version. - **scope** — zero changes: verified already-tight; its checklist enhances dispositioned reject/defer with reasons. Notable: the checklist's cross-cutting "live loop emits verdict.v3" claim was **verified stale** (main is verdict.v2; no validate_v3.py exists) and rejected — the reboot's verify-before-edit rule doing its job. Ledger (17 items: fixed 5 / applied 5 / rejected 4 / deferred 3, zero silent drops): `docs/audits/2026-07-28-skill-overhaul-reboot/wave-reports/w3a.md`. ## Validation - 23/23 bats (liveness + anti-spiral + native-skills fixtures); output validators smoke-tested both directions; contract guards strip-tested - 49/49 frontmatter; `regen-all.sh --check` clean; `check-skill-python-ratchet.sh` PASS (no new Python — validators are jq/shell); token budgets green - Cross-family (Codex) review in flight; two-round ceiling per plan Tracker: `age-skill-overhaul-reboot-sjv7v.4` (closes with W3b) |
||
|
Bo
|
5ed0cb2088 |
Skill overhaul reboot: lean plan, salvaged evidence, validator liveness, anti-spiral hardening (W0+W1) (#998)
## Summary Reboots the stalled 2026-07-24 skill-system overhaul as a lean wave program and lands its first two beads, plus the anti-spiral hardening from the 2026-07-28 incident brief. **W0 — salvage + plan** (`age-skill-overhaul-reboot-sjv7v.1`) - Lean reboot plan (`docs/plans/2026-07-28-skill-overhaul-reboot.md`): per-skill enhancement in 8 waves via normal RPI; proof-epoch machinery explicitly abandoned; 07-24 plan marked superseded. - Salvages the four Sol-reviewed per-skill audits (49 canonical skills) + program validation from `/tmp` into `docs/audits/2026-07-28-skill-overhaul-reboot/raw/`, the 07-24 plan/duel docs, and the 07-25 progress review; distills them into four actionable checklists (231 tagged items, all hypotheses to re-verify). **W1 — validator liveness** (`age-skill-overhaul-reboot-sjv7v.2`) - Seven skill validators (rpi, plan, implement, learn, ms, scaffold, security) had `! grep` forbidden-phrase guards that are silent no-ops under `set -euo pipefail`. Witnessed RED, converted to the `if grep …; exit 1` house form, reworded learn's boundary sentence. - `tests/scripts/skill-validator-liveness.bats`: whitespace-class static sweep + seeded behavioral liveness with unseeded pre-check. **Anti-spiral hardening** (incident: 326 control artifacts, zero implementation commits, RPI never invoked) - **rpi**: automatic admission (orchestration/delegation/"execute this plan"); phase lock after an accepted plan; spiral breaker — two consecutive control artifacts with no new implementation evidence → `NOT_BUILT`; subject-first reporting. All pinned by validator greps. - **validate**: nonempty-implementation-candidate precondition; `store-verdict` CLI mechanically refuses an empty subject manifest. - **AGENTS.md**: constraint floor (a plan/duel frozen without the active ADRs/blocking gates is invalid) + ADR-0016 pointer; phase-lock/spiral-stop lines in the core loop. - `tests/scripts/anti-spiral-contract.bats`: stripping any pinned phrase fails the owning validator; empty-manifest refusal is behaviorally tested. - Placement report with proposed host-level `CLAUDE.md` rule and `dueling-idea-wizards` packet-completeness patch: `docs/audits/2026-07-28-skill-overhaul-reboot/anti-spiral-hardening.md`. ## Validation - RED→GREEN witnesses for all seven guards; 15/15 bats across both new suites - 16/16 validate-skill unit tests; 9/9 rpi kernel tests; 8/8 skill validators - `scripts/regen-all.sh --check` clean; `shellcheck -S warning` clean - Cross-family (Codex) review round 1: 3 findings, all fixed; delta re-review in flight (two-round ceiling per plan) Known pre-existing: `validate-skill-frontmatter.sh --strict` warns on skill-builder's missing optional `context_rel` (present on `origin/main`; assigned to W4). Tracker: `age-skill-overhaul-reboot-sjv7v`; waves W2–W8 follow as separate PRs. |
||
|
Bo
|
f66ab953af |
test(gates): require every blocking gate to prove it can fail (#996)
P3 from the 2026-07-25 review: fix the inert-check CLASS, not just the two instances #995 patched. A gate verified only against a green tree is indistinguishable from `exit 0` -- delete its detector and CI stays green. Measured across the registry: 32 of 54 BLOCKING script-backed gates had no test anywhere under tests/ demonstrating they FAIL on what they detect. 17 had no test at all, including skill.schema and skill.triggers, which gate every SKILL.md in the corpus. The closure lives in Go because the registry does. An earlier bash/Python projection of seed.go undercounted by four -- some gates register outside that file. The authority is gates.Default.All(), and reading it any other way reproduces exactly the projection drift ADR-0016 is named after. Mechanism is the repo's proven shrink-only ratchet: current gaps pinned in scripts/.gate-negative-witness-grandfather; a NEW blocking gate must ship with a witness; a gate that gains one must be pruned; a gate that leaves the registry must be pruned; the growth guard reads HEAD so a change cannot exempt its own new gate. The surviving count prints on every run. Ratcheted down immediately to prove the mechanism moves rather than just counts: tests/scripts/validate-skill-schema.bats is a real witness for skill.schema (32 -> 31), driving the actual validator against a fixture tree. Two things worth recording: - The first draft's negatives passed for the WRONG reason. Descriptions in this corpus embed `Triggers: "x"`; unquoted, that colon makes YAML read a nested mapping, so the gate failed on a PARSE error rather than the schema violation each test claimed to witness. Caught only because the positive case failed too. A negative that fires for the wrong reason is not a witness. - go.test-isolation is ON the allowlist as lacking a proven negative, and it caught a real defect in this very PR (a git exec without per-command env scrubbing). The pin measures TEST COVERAGE of a gate, not its efficacy. An unproven gate may work fine; the point is that nothing would tell us if it stopped. All four of the meta-test's own failure directions were seeded and observed, so the check-liveness ratchet is not itself an unproven check. |
||
|
Bo
|
aceeb6f10f |
feat(gates): enforce ADR-0016 shipped-Python rule; fix RPI/Validate digest disagreement (#995)
Two fixes from the 2026-07-25 skill-overhaul progress review, both independent of the unlanded tranche fleet. 1. skill.python-ratchet — the gate ADR-0016 said it already had. Section 3 fixed "Python never ships in skills" on 2026-07-18 and called its own violation "a gate failure, not a style nit", while zero checks enforced it. Now a blocking shrink-only ratchet over skills/*/scripts/**/*.py: the 24 current files are pinned, new ones hard-fail, a promoted file must be pruned, and the growth guard rejects a change that allowlists itself. The surviving count prints on every run. ADR-0016 records the enforcing check by name and the skills/*/tests/** carve-out with its rationale, so the exemption cannot decay into the same inert prose. 2. The RPI/Validate acceptance-digest disagreement. run_once.py digested a canonical-JSON re-serialization of the parsed intent mapping, validate.py digested the raw intent bytes, and run_once.py hard-compared the two, so the composed contract could not succeed. Both suites were green because RPI's test mocked Validate with RPI's own digest function. RPI stops being a second digest authority: it carries the digest Plan declares over the bytes Plan snapshotted and cross-checks Validate's independently re-derived value. 3. The cathedral-cut probe, the same blind spot found again. It set intent_bytes = canonical_bytes(resolved_intent) — precisely the one input where the two digests coincide — so like the unit test it was built around the coincidence and could not observe the bug. Test shape is liveness-first: every gate negative is a seeded witness the gate must be shown to FAIL on (12/12 bats), and the digest fix landed against a RED witness that reproduces the real production error. |
||
|
Bo
|
cbe7ddbb5f |
feat(gc): ship thin native AgentOps factory pack (#980)
## Outcome Ships AgentOps 3.3 as a thin native Gas City pack and removes the accidental second orchestration control plane. - Removes the custom GC delivery command/package, packet/schema family, feeder/program engine, reducer/order, reliability registry, Beads capability mirror, and fork-baseline runtime checks (about 29k deleted lines). - Retains six bounded adapters for pinned toolchain materialization, clean bootstrap, native sling/status/doctor invocation, bead-isolated worktrees, moving-main PR delivery, and teardown. - Uses official Gas City and Beads behavior as the state owners, with required OTEL configuration. - Preserves Fable Mayor/Refiner, Sol-high planning and fresh validation, Terra-high default implementation, Opus-medium overflow, and support-only Luna. - Supports automatic Refiner merge after hosted CI or a manual-review toggle without locking `main`. ## Evidence - Exact official GC: `8ffc009ded781a2ada2077f3a29bd712b2def0bf` - Exact official BD: `8e4e59d39f3459a43cf21a3236a13eca4dd874f7` - Full pinned native boundary: 9/9 passed in 114.047s - Exact source-bead route replay: passed in 98.051s - Release replay: manifest integrity, thin tests and both pack lints, generated projections, contract compatibility, Bats contract, ShellCheck, test-removal ratchet, and `go test ./...` all passed - Fresh independent semantic verdict: PASS (Sol-high) - Exact delivered head: `e805f0e26ce21c2eae9e720f57fff2a705f3185b` - Subject manifest: `8554d065076f70d0e639ca353409d2a75dca7ac01e4cb65cb4a118e98c0770fc` - Verdict: `3f89376b675de854832f38b46ccfa7ad7ca54079ffde578769c33e1f118c68c1` ## Release boundary After this PR merges, the release qualification runs one fresh mixed Terra/Opus canary from merged `main`, with at most one external repair and one terminal retry. No live city repairs itself. The only known local fast-gate exception is `workflow.install-drift`: the installed workflow symlink resolves to the separate dirty primary checkout, outside this candidate. All candidate-owned gates passed. |
||
|
Bo
|
68e31ec3df |
fix(gc): route controls through primary rig dispatcher (#978)
## Summary - keep the city dispatcher and non-primary rig dispatchers suspended - activate exactly the selected primary rig dispatcher - seal Formula control records to that dispatcher before admission and fail closed on malformed or foreign bindings - verify the effective official GC agent inventory during bootstrap ## Evidence - fresh Sol-high pre-delivery binding verdict: PASS - candidate diff digest: c7362c701f417e259194b97572734aed18ebc022f17abcd907681926f9db6fc2 - Python workflow tests: 18/18 - bootstrap Bats: 30/30 - GC executor gate: PASS - full local release gate: PASS - two fresh official GC v1.3.5 / BD v1.1.0 zero-model qualifications: PASS ## Residual Stable GC teardown sometimes leaves its private tmux session after gc stop. Qualification cleanup was scoped and reached zero processes; this upstream-owned residual is disclosed and is not changed here. The post-merge mixed-provider canary remains a separate release gate. |
||
|
Bo
|
62642ebb52 |
fix(gc): bind stable formula workflows to deployed rig (#977)
## Summary - bind Formula cook to concrete targets derived from the digest-bound native rig - decode Gas City v1.3.5's exact relative Ralph-attempt refs into canonical AgentOps workflow keys - fail closed on root, control, source-spec, attempt, route, replay, and custom factory-binding drift - retain official GC v1.3.5 and Beads v1.1.0 unchanged ## Why The fifth release canary correctly stopped before model launch because `rig/agentops.plan-reviewer` was not a valid Formula target. Exact official qualification then exposed a second AgentOps assumption: stable GC persists top-level Ralph attempt refs as `mayor.iteration.1`, `plan.iteration.1`, `implement.iteration.1`, and `validate.iteration.1`, while AgentOps had expected formula-qualified values. This change supplies sealed concrete routes to the pinned stable release and uses explicit one-to-one persisted-to-canonical maps. It does not patch or depend on an unreleased Gas City/Beads fork. ## Evidence - candidate: `21cbd7693819ae047d4ce3da4f95ef9453e2e80a` - tree: `ede2b444217be5c8e2aa19bf5af81d231b70b360` - official GC: `8ffc009ded781a2ada2077f3a29bd712b2def0bf` (v1.3.5) - official BD: `8e4e59d39f3459a43cf21a3236a13eca4dd874f7` (v1.1.0) - two fresh configured-not-started roots passed independently: - route-fixture-g: build `rfg-bq0`, Terra `rfg-plb`, Opus `rfg-cio` - route-fixture-h: build `rfh-eez`, Terra `rfh-86h`, Opus `rfh-jgc` - both roots had zero sessions, zero running agents, clean Git, and clean managed teardown - fresh Sol-high binding verdict: PASS; file SHA-256 `0c9b14bcd0f04d782f67f815c029b267224a554a445c5d2a25feadac881d2e10` ## Checks - `bash scripts/check-gc-executor.sh` - 41 packet tests - 41 GC 3.3 tests - 4 projection tests - 35 bootstrap Bats tests - static contract PASS - `bash scripts/ci-local-release.sh` - LOCAL CI PASS in 89s - race, security, SBOM, release binary, smoke, install, and contract gates passed - advisory readiness is 7 only because HIL is intentionally deferred to the one post-merge mixed-provider canary ## Delivery boundary The final required-telemetry mixed-provider canary will run once from merged `main`. These stopped and qualification-only roots will not be replayed. |
||
|
Bo
|
d7224b87a3 |
fix(gc): bind delivery context in managed invocation (#976)
Summary\n\n- Bind the bootstrap-generated canonical native delivery context into the ready managed-city marker.\n- Make the managed invoker verify and export the complete fixed delivery/tool identity before imported pack execution.\n- Preserve the fourth bounded pre-admission stop and classify it as an AgentOps integration defect.\n\nValidation\n\n- scripts/check-gc-executor.sh: PASS\n- Full local release gate at .agents/releases/local-ci/20260722T160625Z: PASS on 8a3e3e066\n- Two clean official GC v1.3.5 / Beads v1.1.0 bootstrap-start-help-native-context-teardown cycles\n- Fresh Sol-high binding verdict: PASS; file SHA-256 0b42389409a5c7c9026d44d20c6b5c9e50d9a5763009d63baceeaef62fb720e5\n\nThe required-telemetry mixed-provider canary remains deferred until this exact subject passes hosted CI and lands. |
||
|
Bo
|
b8ad0c6851 |
fix(gc): omit root selector from pack dispatch (#975)
## Summary - keep validating the exact managed-city marker, GC digest, private supervisor, and telemetry policy - enter the exact managed city for eager imported-command discovery - execute GC without redundantly forwarding root `--city` or a literal separator into the imported leaf - make the parser-shaped regression reject either leaked root argument - record the third bounded pre-admission stop and correct its ownership classification ## Why The third fresh 3.3 canary reached the eagerly discovered `DisableFlagParsing` leaf, but the managed invoker still supplied `--city <path>` to GC. GC intentionally gives the discovered leaf ownership of its remaining argv, so `program_start.py` received that root selector and rejected it. AgentOps already classified this as a compatibility hazard it must handle; this correction makes the implementation match that known no-root-selector contract. No Gas City or Beads fork/source change is included. ## Evidence - exact candidate: `a8dc3511fb6d47b8860c706bddeb3e85706695ee` - exact tree: `ee9764561bebf0e196b5c2ac927dc866eb72d9c6` - focused invoker Bats: 4/4 PASS - focused + adjacent Bats under fresh Sol: 20/20 PASS - combined GC adapter gate: 33 Bats plus packet/factory/projection suites PASS - official-GC read-only imported help from `/private/tmp`: PASS with no stopped-cycle mutation - full local release: `.agents/releases/local-ci/20260722T152302Z` PASS, exact SHA bound - two fresh bootstrap/start/help/quiescence/teardown cycles: PASS - fresh Sol-high binding verdict: PASS (10/10), file SHA `af2a41a42c75c632a58adbeb9fa0ff6ec2349fcfc18d94e1eb478a032077ceb8` ## Deferred One wholly new required-telemetry mixed-provider canary remains after hosted CI, exact protected-main landing, and post-merge conformance. The stopped `ag-lb2` subject will not be replayed. |
||
|
Bo
|
d8c1ddc411 |
fix(gc): enter city before pack dispatch (#974)
## Summary - enter the exact marker-bound managed city before dispatching imported AgentOps pack commands - preserve direct leaf arguments without a literal `--` separator - document the verified official GC v1.3.5 discovery limitation and the bounded AgentOps workaround - add a parser-shaped regression test that fails when dispatch occurs outside the requested city ## Why The second bounded 3.3 live canary exposed an official GC v1.3.5 CLI-ordering limitation: imported pack commands are discovered before persistent `--city` is parsed. From outside the city, direct leaf flags are rejected by GC; the lazy fallback reached with `--` forwards that separator to the pack script. The managed invoker now enters the already validated marker city before exact GC exec, allowing eager pack discovery without changing the GC or Beads forks. ## Evidence - exact candidate: `6eff4767297178c4cd9d7945795526d2e61d0e0b` - exact tree: `8e57f213f9253e6880ee5f183783500dad729abb` - invoker Bats: 4/4 PASS - adjacent teardown/toolchain Bats: 16/16 PASS - real official-GC read-only pack-help proof from outside the city: PASS - full local release: `.agents/releases/local-ci/20260722T144624Z` PASS - two clean exact qualification cycles: PASS - fresh Sol-high binding verdict: PASS (10/10 criteria) ## Deferred One entirely fresh live mixed-provider canary remains after hosted CI and exact protected-main landing. The upstream CLI defect is retained under the canonical 3.3.1 bead for a bounded contribution only if current upstream still reproduces it. |
||
|
Bo
|
c0a68c29ea |
fix(gc): bind managed canary invocation (#973)
## Outcome Correct the one AgentOps-owned defect exposed by the first post-merge 3.3 canary: the operator README placed a literal argument boundary inside the discovered Gas City command, so Python never received source-bead. This adds a managed invocation boundary that: - consumes its own explicit wrapper separator and forwards discovered-command flags directly; - binds the schema-5 city marker, exact GC digest, private supervisor home, and isolated mode; - projects the marker-selected GC/BD telemetry endpoints or an explicit disabled state; - clears ambient generic OTLP fallback configuration; - records the stopped pre-admission canary without claiming a Gas City or Beads defect. ## Evidence - focused managed-invocation Bats: 4/4 PASS - combined Gas City contract gate: PASS (33 Bats plus Python/static contracts) - adjacent teardown/toolchain Bats: 16/16 PASS - exact full local release gate: PASS at .agents/releases/local-ci/20260722T141141Z - two fresh exact-toolchain bootstrap/start/status/teardown cycles: PASS, zero workers/sessions, clean teardown - fresh Sol-high pre-delivery binding: PASS, 11/11 criteria ## Deferred boundary This PR does not claim final 3.3 release readiness. After protected merge and post-merge conformance, exactly one new required-telemetry mixed Terra-high/Opus-medium live canary remains. |
||
|
Bo
|
faff55ac2a |
feat(gc): bind native 3.3 factory workflows (#972)
## Summary
- bind the GC 3.3 bead-native workflow: Fable Mayor, fresh Sol-high plan
and validation, Terra-high default or Opus-medium overflow
implementation, Luna support-only
- add the bounded one-shot graph feeder, strict packet/worktree
identity, semantic terminalization, and model-free protected delivery
sweep
- make official Gas City v1.3.5 plus Beads v1.1.0 bootstrap repeatable,
disable unstable event propulsion/hooks, and prove quiescent teardown
## Reliability boundaries
- no Gas City self-repair loop, daemon, private scheduler, model
fallback, or main mutex
- semantic completion is independent of delivery; protected
PR/CI/rebase/merge is deterministic and moving-main aware
- event hooks are disabled because 3.3 uses explicit program admission
and cooldown delivery; native controller bead observation remains active
## Evidence
- full local release CI PASS at
.agents/releases/local-ci/20260722T113355Z
- two pristine standalone-clone cycles with official GC v1.3.5 8ffc009d
and BD v1.1.0 8e4e59d: repeat bootstrap, exact rig Dolt context, absent
event hooks, bounded events, enabled sealed delivery, and delayed
zero-process teardown
- fresh independent Sol-high pre-delivery binding PASS over exact head
|
||
|
Bo
|
4d631207cf |
feat(gc): add bead-native crash-only delivery kernel (#970)
## Summary - replace the unreachable Python factory lifecycle with an optional typed Go delivery reducer - make Beads the delivery lifecycle authority with deterministic moving-main successor epochs - add native Git/worktree, PR, hosted-check, auto-merge, landing, and cold-replay boundaries - retire obsolete v1 role/factory schemas and preserve the pinned historical capability harness ## Validation - fresh Sol-high PASS over exact 43-path manifest ec6a2a4bf0b180782411b1ae4afd4a2ad8384129ff037ce8c119480317626b02 - go test ./... - go test -race ./internal/gcadapter/delivery - go vet ./... - scripts/check-gc-executor.sh - GC 3.3 schema, migration, provenance, factory-doctor, and bootstrap gates Bead: age-gc-scope-failclosed-release-gktia.1 |
||
|
Bo
|
00c4d4bfbc |
GC 3.3: prove writer isolation and routing firewall (#962)
Bead: ag-agentops-33-gc-refinery-4km81.5\n\nThis delivers the GC33-4 deterministic proof kernel without starting Gas City or adding a second agentic loop.\n\nWhat changed:\n- width-two per-bead lease and candidate isolation, with overlap serialization and no main/global mutex\n- exact candidate Git/worktree/index auditing and bounded PGID-scoped process receipts\n- Draft 2020-12 typed delivery/ambiguity publication with exhaustive construction-state routing\n- composed six-role native query/sling firewall executed by factory doctor\n- fake-store-derived loss, duplicate, expiry, and replay evidence\n- live Fable remains closed pending supported confinement and GC33-11\n\nFresh Sol-high binding evidence:\n- exact subject manifest: 9ec940be7b528fe9ed2181830d2261424af1865c2c4b9793d7f2c68158845790\n- verdict.v2: ef985788f1b28c66397a28e55e57cbaac620a51bdfad745812bcfe141df847f9\n- 8 of 8 criteria PASS, zero findings\n\nLocal gates:\n- 67 focused Python tests\n- 25 of 25 bootstrap Bats cases\n- factory doctor\n- Python and shell syntax\n- diff hygiene |
||
|
Bo
|
247fcccb5f |
GC 3.3: define the one-loop factory role pack (#961)
## Outcome Defines the GC 3.3 factory as one semantic intelligence loop with deterministic delivery: - Fable Mayor plus a bounded, nonbinding ambiguity adviser - Sol-high for plan binding and fresh exact-subject validation - Terra-high as the default writer and Opus 4.8 medium as explicit overflow/repair - Luna dormant and unroutable until later qualification - no model-driven delivery-policy/refinery loop - native GC sling/claim/scale-from-zero semantics retained - exact provider/model/reasoning/fallback runtime attestation - legacy composed program/refinery entry points and opposite-provider lifecycle roles removed Controller dispatch to the Fable adviser stays fail-closed until GC33-4 proves process/filesystem isolation. This does not claim that manual operator sling to the named on-demand target is impossible. ## Evidence Fresh Sol-high `verdict.v2`: PASS (`bb31806fc5e0714363a9e539121ee7aa19231431fb04ddfee1fce8376b57d272`) over subject manifest `119b7a6db2380af47d87a24aced60d96858dd9e413a54dc640376e27767e8d1e` (52 paths, 18 deletions). - focused acceptance: 124 tests PASS - projection owner: 4 tests PASS - bootstrap fixtures: 25 Bats tests PASS - complete GC static contract: PASS - factory/executor doctors: PASS - generated projection check: PASS - exact official GC v1.3.5 and Beads v1.1.0 provenance replay: PASS - `git diff --check`: PASS No city, model session, supervisor, tmux session, Dolt server, or canary was started for this pre-canary bead. |
||
|
Bo
|
c8f809595d |
feat(gc): pin the 3.3 bootstrap toolchain (#959)
## Summary - pin the qualified AgentOps Gas City bootstrap to official Gas City v1.3.5 and Beads v1.1.0 - materialize exact `gc`, `bd`, and repository-built `ao` binaries with a portable identity receipt - make bootstrap and teardown reject ambient substitution and persisted identity drift - prove repeat bootstrap projections and ownership-safe teardown Bead: `ag-agentops-33-gc-refinery-4km81.2` ## Binding validation Fresh Sol/high verdict: `PASS` - verdict digest: `abc7654f5e68d1d562a5e7b612d7d0cef3941f14bc7e02bbf22f4191a7f1c645` - subject manifest: `78011e248dda437bd0ac39a912cb6195df24d2789ed6c58016873f0f891775b2` - exact changed scope: nine paths in `deploy/gc` and `tests/scripts` A prior fresh verdict found that commit-only shallow fetches built Gas City as `dev`. The repair now fetches and verifies each locked release tag against the independently pinned commit before building. The prior FAIL remains preserved as evidence. ## Checks - real clean materialization: GC `1.3.5@8ffc009`, Beads `1.1.0@8e4e59d`, receipt-bound AO source/tree/digest - `37 tests, 0 failures` across toolchain, bootstrap, and teardown Bats suites - `bash -n deploy/gc/materialize-toolchain.sh deploy/gc/bootstrap.sh deploy/gc/teardown.sh` - `git diff --check` No Gas City, Dolt, or suspended v17 city was started. |
||
|
Bo
|
56190722e6 |
fix(gates): atomic-write ratchet fails closed on scan-helper death — no silent PASS (#946)
## Root cause (CI flake, run 29785505667 attempt 1, PR #944) The bats test `adding only the temp-signal half to a file with an EXISTING rename still trips` failed once on ubuntu with the ratchet printing a clean `PASS … grandfathered=0` where the fixture deterministically trips. Diagnosed by analysis per the test-isolation discipline (not repro-chasing): - bats errexit proves every fixture `git commit` succeeded; the fixture repo is fully isolated. - `grandfathered=0` in the PASS line fingerprints the run as in-fixture (real repo = 4). - Zero stderr in the captured output rules out every loud failure handler. What remains: **three silent paths where a scan helper (grep/awk/git) dying under 4-way-parallel runner load reads as "no finding"** — the gate then certifies an unchecked diff: 1. `file_trips`: strip/grep failure returned 1 ("does not trip") → file silently skipped. 2. Detector loop read from `< <(ratchet_changed_files | sort)` — a collector rc 2 inside process substitution is discarded, certifying an EMPTY change set (the exact fail-open the ratchet-lib header warns about). 3. `ratchet_added_hunk_matches`: helper death propagated as arbitrary nonzero, swallowed by `|| continue`. ## Fix - `file_trips` is tri-state (0 trips / 1 no / 2 refuse-to-certify, loud); signal greps read full input (no `-q` early-exit) so pipefail can never surface a SIGPIPE'd printf as a phantom failure on large files. - Changed files are collected into a variable under pipefail with an explicit `exit 2` handler. - `ratchet_added_hunk_matches` maps fallback-grep/awk death to loud rc 2 (0/1 unchanged; existing `|| continue` consumers skip on 2 exactly as before, now loudly). - `compute_grandfather_set` aborts `--regenerate` on helper failure — never a truncated grandfather list. ## Pinned Dying-awk PATH-shim tests at gate and lib level: helper death is rc 2 "refusing to certify", never PASS. Same fail-closed class as the pawl-refuted empty-stream certify (2026-07-10). ## Validation - 58/58 in `check-atomic-write-ratchet.bats` + `ratchet-lib.bats` on fresh origin/main - All six ratchet-lib consumer gate suites green - `shellcheck -S warning` clean; real-repo gate PASS (grandfathered=4); `--regenerate` byte-identical - 65 repetitions of the formerly-flaky test green Residual: the external kill source is unprovable from one occurrence; a recurrence now exits 2 naming the failed helper + file instead of printing PASS. |