mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
main
1054 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bo
|
d972fa2090 |
Prepare AgentOps 4.0.0 plugins, skills and CLI release (#1143)
## What Prepare AgentOps 4.0.0 across the Claude plugin, Codex plugin, skills and CLI. Claude writers capture the supplied check status during its original invocation, and plugin conformance verifies exact skill membership and link destinations. Full release security now scans the repository and blocks on Python collection failures that previously produced a false green result. ## Why The 3.6.0-to-current interval removes published commands and 20 skill names, so this is a major release with migration instructions. Release validation also exposed stale skill assertions and test prerequisites that need to match the current product contracts without weakening acceptance. ## How I tested - Native Claude Opus/Haiku success, failing-check and direct-writer trials: each check ran once, and the direct child returned plain JSON. - Actual fresh installs and upgrades from 3.6.0 in isolated Codex and Claude homes: 34 skills, expected agents, and exact installed package bytes. - Exact candidate `b721d02559e1495be6095ad97b820e88ceb4a049`: all 73 full repository gates, regeneration parity, and the complete local release rehearsal passed. All 12 security tools ran with zero skips, tool errors, critical findings or high-severity security findings. The unchanged advisory policy reports 35 quality-high findings on unchanged files. - Python: 327 tests and 72 subtests passed. Hosted Bats: 1,509 passed, 31 environment-dependent skips, zero failures. Go lint/build/vet/race/shuffle checks and CLI smoke/integration passed. - All 11 hosted checks passed, including Windows correctness, macOS/Linux installation, security, and the six-target no-publish GoReleaser snapshot. Local archive checksums and a real macOS CLI initialization/status/version smoke also passed. - Fresh author-distinct review passed all four acceptance criteria and all 35 changed paths with no unchecked acceptance. Canonical subject and caller-intent verification passed; verdict digest `68af2c935ed0106cd91b3950f5d168e662f4071f660fcbd113c36b7cd0f0426e` binds manifest `7affc77e25eaff69ba36c5ce05582b4f0385c954b76b62c02b97f97041f489b2`. ## Checklist - [x] Breaking changes documented in the migration guide and complete release notes. - [x] No credentials or private runtime proof included. - [x] Final full release checks pass on the exact candidate. - [x] Fresh author-distinct final PASS is recorded before merge. This prepares the release candidate; it does not publish a tag or release. Coverage limits remain explicit: native plugin tests used isolated macOS homes and local marketplaces, guard installation remains opt-in, and reader instructions do not prove sandbox confinement. Semgrep retains pre-existing warning-level parser diagnostics. Snapshot metadata follows the existing 3.6.0 tag; this is a packaging rehearsal, not a published 4.0.0 archive. |
||
|
Bo
|
937d6c8269 |
Add native Codex context-budget delegation and opt-in refusal (#1140)
## What Add Codex-native `bulk-reader` and `code-writer` roles pinned to `gpt-5.6-luna`, opt-in role/config installation, and an opt-in native `PreToolUse` Bash adapter for the shared read-budget guard. Source-owned guidance and role files ship through the existing generated Codex bundle; the menu stays at 34 skills. The installed Codex 0.154 runtime can refuse covered shell calls before execution. The adapter enforces that predicate; slice discipline, target-only writes and receipt-only replies are role instructions, not an output filter or per-file sandbox. Native exact-definition hook trust remains required. ## Why Follow-up to #1137 and its [fresh author-distinct review](https://github.com/boshu2/agentops/pull/1137#issuecomment-5648513520). Correct the unverified statement that Codex has no refusal-capable hooks and provide real native delegation without subprocess model execution. This branch incorporates the isolated fixes in #1139 and targets main; land the repairs first. The fixes PR contains no Codex-native changes. Work and original acceptance are recorded in private BD `age-z25n`; `bd context --json` resolves the existing private Dolt store. [Design and live evidence](https://github.com/boshu2/agentops/blob/codex/context-budget-native/docs/design/codex-context-budget.md) records the runtime/config contracts, exact available model identifiers and published comparable rates, source paths, invocation, transcripts, accounting and limits. ## How I tested Current head: `40edb5f216b71bfa659627dca10fc0a9e04c1821`. Fresh author-distinct review: **Job2 native PASS; combined subject FAIL**. Reviewer context `01a09778-bef6-7883-880b-6764fdd783b9`, observed `gpt-6-astra`/ultra, authored no candidate code. All 52 changed paths have identical start/end manifests (SHA-256 `b1f13ca731658bc699838189f928535a386ab6de6b381b0419bb0f7ebdd88cdc`); acceptance `not_checked: []`. The native repeated-refusal naming finding is fixed and independently reproduced as resolved. Local final checks pass. [Hosted Validate CI](https://github.com/boshu2/agentops/actions/runs/34722978404) remains pending at this update; no merge is performed. Real Claude Opus follow-up closes plugin-name resolution, inherited hook invocation, complete reader coverage and observed parent/child content separation. **It also finds a remaining Claude writer failure:** two of three final workers ran their supplied check twice; a direct receipt used Markdown fences. This combined PR does not claim a full Claude writer PASS. Details, excluded failed attempts and exact identities are in the design note. - `./cli/bin/ao gate check --scope range:origin/main..HEAD`: 33 passed; the earlier unchanged registry run `./cli/bin/ao gate check --full` passed **73 gates** at `90c8b31ee` (historical receipt). `bash scripts/regen-all.sh --check`: all 11 checks passed. - Door9, hookless cold-start, doc-hook drift, shellcheck on five changed shell files, and Node syntax checks on both workflows plus the native config editor passed. - A7 Bats suites plus workflow and all native suites: **223 passed, zero skipped**. All **21 documentation-reference tests** and the strict full-document reference scan also pass after clarifying the native hook-manager terminology. - `bash tests/run-all.sh`: **10 passed, 0 failed, 1 skipped** (optional OL directory absent). This is the default static tier. - `bash scripts/validate-codex-install-bundle.sh`: passed, **34 skill packages**. Changelogs are identical; diff check clean. - Live registered reader: parent `01a09776-004f-79a0-af85-c75b472a1e68` used only spawn/wait; child `01a09776-3c38-7281-b582-01fef5101f17` was natively identified as bulk-reader/Luna/low. Six separate slices covered 1,772 lines without truncation; parent received five findings and coverage only. - Live registered writer: parent `01a09771-8908-7a00-b101-919b558cf8c1` used only spawn/wait; child `01a09771-bbbf-7232-add4-cc9ec55ad759` was code-writer/Luna/medium. Required reference, seven-line Bats target, receipt only, child check passed; coordinating parent independently ran Bats successfully without reading the target. - Live hook: native parent `01a09769-14d2-7c22-9b7d-50847de07c90`, final turn `01a09771-25f2-7830-830b-498d7ca1945e`, refused a 400-line cat before execution and allowed a three-line sed slice. Real payload is PreToolUse/Bash/tool_input.command; hashed deny ledger schema verified. An explicit session hook was trusted in `/hooks` for this proof. - Credentials-free native config/read and hooks/list probes verify personal and ordinary project discovery. Codex 0.154 reads linked-worktree project hooks from the primary checkout; `--project` now rejects linked worktrees before writing, with a real Git regression. Known failed: Claude writer check-once behavior and direct receipt fencing. Not checked: arbitrary hosted/MCP read interception; adversarial enforcement of role instructions; other runtime versions/accounts; cost savings, comparative latency or ADR-0002 value-proof clearance. Earlier reader attempts with an overlarge final slice or truncated aggregate output are disclosed and excluded from the successful coverage proof. ## Checklist - [x] `make build && make test` passes (if Go changes) — no Go changes; gate runner built once. - [x] No secrets or credentials in code - [x] Breaking changes documented — opt-in additions; unsupported linked-worktree hook installation refuses before mutation. |
||
|
Bo
|
a3d6018f2c |
Fix read-budget refusals, installer backups and workflow boundaries (#1139)
## What
Repair the opt-in read-budget guard, installer and Claude workflows from
#1137. The fixes cover conservative literal shell parsing, preserved
installer backups, bounded worker returns, distinct filesystem targets,
actual plugin-qualified names, complete reader coverage, exact receipt
identity and measured writer line counts.
## Why
The [fresh review of
|
||
|
Bo
|
e32e88c338 |
Add opt-in read-budget guard and cheap-model bulk-read/code-write delegation (#1137)
Bring the Spotify Claude Code context-budget setup into AgentOps as one opt-in PreToolUse guard (skills/cc-hooks/hooks/read-budget-guard.sh, policy core.context:unbounded-read) with its opt-in installer, two Workflow-tool conveyors (bulk-read, code-write), two plugin subagents (bulk-reader, code-writer), their docs, regenerated projections, and three bats suites. Nothing ships wired by default; the skill menu is unchanged. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H2bDH2j6XDPPcLQS84PdzN |
||
|
Bo
|
c6558508d1 |
Consolidate AgentOps into a 34-skill engineering menu (#1133)
AgentOps' 55-skill catalog contained overlapping entry points, stale
routes and descriptions that could lose meaningful guidance in the Codex
projection. This change consolidates 21 roots into existing owners,
leaving 34 distinct skills and a generated, task-oriented menu. README
documents every retired name and its replacement.
Planning now establishes observable behavior in the caller's existing
intent, using proportional Given/When/Then examples and domain language.
Implementation and final validation carry those same examples forward.
Original adaptations informed by Matt Pocock's engineering skills
strengthen existing owners rather than adding a new workflow. Routine
edits need no mandatory plan, coverage report, mutation exercise or
learning artifact.
Codex retains complete source descriptions and translates explicit-only
invocation policy. All descriptions fit the existing 180-character
limit; the root instructions retain their 250-line limit. Generated
catalogs, projections, routers, moved references/helpers and their live
consumers are updated together. RPI remains explicitly selected.
Validation passed: projection/conformance checks, the local aggregate
(10 passed; one existing optional-directory skip), and exact-commit CI
covering the complete gate registry, Bats, Go build/vet/race/coverage,
Windows and security. A fresh author-distinct reviewer passed all
acceptance criteria over the complete 573-path subject at
|
||
|
Bo
|
e1fae0dae6 |
Make the engineering harness lean and add optional topic memory (#1116)
RPI now owns the authorized outcome through finish, with Plan and Memory
loaded only when useful. Known defects get direct repair, and evidence
can change the approach under unchanged acceptance. Fresh exact-content
validation remains required. Memory provides optional recall, mining and
curation of reviewed topic pages; specialists and the fixed-dispatch
adapter remain optional.
The change reconciles current documentation and generated skill
projections. It preserves native budget and permission authority, BD
work ownership, protected external evidence storage and the distinction
between a supported lesson and demonstrated later benefit. It adds no
scheduler, work store, Go command or evidence schema.
Validation: required local Go/build/vet/race checks, aggregate suite,
generated-output check and 72 gates pass. The complete 44-test executor
suite passes; its shared-deadline fixture now tolerates CI scheduling
jitter while still requiring deadline exhaustion and preventing a third
launch. Fresh author-distinct review passed all 112 changed paths with
no findings; all seven exact-head CI checks passed at
|
||
|
Bo
|
8061085c89 |
Ship native evidence helpers and fresh-family review defaults (#1110)
AO now performs intent snapshots, subject manifests, strict evidence verification, atomic verdict storage, and orphan inspection through the Go binary. The command handler keeps verification separate from presentation so it meets the existing complexity limit. These operations preserve the existing evidence formats, require explicit protected storage where applicable, and run outside a checkout without Python. The unchanged Python implementation remains a developer oracle; agents still provide semantic judgment. Codex and Claude skills now default to a fresh reviewer from the author’s model family. Callers can explicitly request cross-model review or pin its model. Reviewer adapters use a finite caller timeout or remaining deadline instead of a fixed ten-minute default, while retaining output limits and abnormal-termination cleanup. Validation: Go build, vet, tests and race/shuffle tests; 1,334 shell tests; aggregate runner; regeneration check; 72 full-mode gates. Independent checks exercised 84 storage-boundary rejections and 21 evidence operations with an empty PATH. Both canonical and generated RPI reference suites pass all 48 tests after updating the migrated oracle import without weakening assertions. Change-sensitive checks explicitly compare the final committed candidate with the original PR base. Linux, Windows, installer, security, and required summary checks are green. |
||
|
Bo
|
af1e530159 |
Adopt CDLC contracts and reconcile BD tooling (#1108)
AgentOps now describes the selected Context Delivery Lifecycle alongside standalone RPI: external reviewed memory, native work and runtime ownership, independent validation, and explicit limits on what the pilots proved. The active ADRs name which historical prescriptions are superseded and which invariants remain. This integrates the BD foundation, including doctor diagnostics that distinguish BD from BR and protection against private tracker data entering outgoing Git history. It also fixes skills audit resolution of explicit relative reference paths. Source-owned projections were regenerated, and the landed stopping behavior is preserved. Validation: Go build/vet/test and race/shuffle, 1,325 Bats cases, the local aggregate, and full AO gates. The gate registry reports 71 passes and one nonblocking preamble-adoption failure on the existing report script; that advisory remains disclosed. Historical probe receipts whose source hashes changed remain historical evidence. This is foundation adoption. It does not claim that the later memory, goal, background, privacy-release, or benefit-measurement work is implemented. Independent Astra and Fable judgments both PASS over the exact integrated content. Earlier negative results and their factual resolution remain preserved in private work evidence. |
||
|
Bo
|
baa24e1b04 |
Bound headless execution and require acceptance-linked repair progress (#1107)
Headless invocations could run without a finite bound, and repair loops could count changed digests or finding counts as progress. The existing adapter now enforces positive finite deadlines, an optional inherited absolute deadline, bounded output capture and cleanup of its owned process group, refusing execution when protection is unavailable. The existing goal, RPI and validation skills now require acceptance-linked progress, distinguish newly discovered failures from regressions, and bound Andon recovery. Review effort follows the effect on acceptance and enforcement. These contracts do not claim native aggregate goal-budget or pause enforcement. Validation on the clean main baseline: - 1,317 Bats cases, zero failures, two skips; 46 RPI reference cases. - Go build, vet, tests and race/shuffle tests; local aggregate runner. - Generated projections current; all 72 AO gates pass. - Fresh independent Astra and Fable 5.1 xhigh judgments passed the exact clean source. Earlier uncommitted CDLC adoption, private work/evidence, and unrelated Go changes are excluded from this PR. |
||
|
Bo
|
021b41c951 |
Skills only: cut the RPI workflow script and the machinery around the loop (#1105)
## Skills only: cut the RPI workflow script and the machinery around the loop #1104 landed the right ideas in the wrong form. This repository is skills only: a skill is an instruction to a system, models route and judge, and deterministic checks exist where reality is deterministic (digests, tests, gates, "is this file tracked"). #1104 built a workflow script and a set of schemas, enums, and engines where instruction was wanted, and then spent four judge rounds hardening them. This PR deletes that and keeps the ideas as instruction. **Deleted.** `workflows/rpi.js` and everything that existed only to exercise it: its probe harness in the product-boundary suite, the convergence-law and glob-intersection fixture corpora, the cathedral canaries that pinned its mechanisms, and the pointers to it. **Reverted to the #1099 form.** `skills/rpi/scripts/run_once.py` and its tests: the repair law as accepted in the loop restore, without the class-law machinery, the snapshot verification, or the cross-leg checks. **Kept.** `docs.claims-tracked` (a blocking gate that caught a real stale claim on its first run), `scripts/evidence-orphans.sh` (a receipt a shell script computes), the optional `findings[].class` field on `verdict.v2` (a word a validator writes), and the corrected harvest-note sentence. **Kept as instruction.** The rpi, plan, implement, validate, council, learn, and boundaries contracts and `docs/architecture/rpi-traversal.md` now say, in plain sentences a model follows: a risky change gets one fresh judge on the frozen plan before Implement; validators name a class per finding and a returning class means go back to Plan; two judges disagreeing is the orchestrator's decision made in the open, with both reads in the report and a risky surface still needing both to pass; the orphan receipt runs after Implement and each repair and rides in the check receipts; a doc sentence that says published, pinned, or proven is a claim to check; Learn proposes a check for any repeating finding class. **Evidence on the tip.** Regen check clean; rpi unit tests; cathedral gate; routing goldens; token budgets; coverage 1 of 12; Go bar; full gate green with a HEAD-built binary; CI's bats command green; gate-tightening ratchet. One fresh read by Fable 5.1, asked in plain language for what it found, recorded in the PR thread. |
||
|
Bo
|
2ef932b617 |
RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated (#1104)
## RPI flow: premortem on risky plans, finding class in the law, adjudicated splits, orphaned evidence, doc claims gated Provenance: running the loop for real across three trains on 2026-09-03 (#1099, #1100, #1101). The spine held; the flow around it did not. A risky-surface design shipped with no premortem and needed six repair passes. The open finding set grew three rounds running inside one class while the id-counting law watched. A same-family PASS and a cross-family FAIL had no tie-break until the caller named one afterwards. Every harness edit orphaned bound evidence with no warning until verify failed. A doc called an egress log published while the repository's `*.log` rule kept it out of the tree, and the same overclaim class came back four rounds. **What changes.** - **Premortem at plan exit.** When the write scope hits a risky surface, one fresh judge reads the frozen plan before Implement; a blocking finding returns NOT_PLANNED with the findings. The caller may declare `premortem: skip`; the report says so. - **A split is recorded, never elected and never closed inside the loop.** The law stands: a risky surface converges only when both legs PASS, and no finding leaves the open set because someone chose a judge. Plan declares `binding_judge` as the caller's disposition for a split that survives repair; it is bound in the plan identity, carried in the report, and never mutates the verdict. On a risky split that survives the repair budget (never after a law stop), one council leg receives a bounded packet marked untrusted and returns per-finding rulings, exactly one per id, which the traversal validates and records under `council.rulings` for the caller's next intent; the verdict and the open finding set stay exactly as repair left them. A council closure mechanism was built, drew a new finding class in three consecutive judge rounds (path containment, digest exclusion, presence-as-evidence, partial closure downgrading a FAIL), and was cut under the train's own class rule rather than hardened again. A non-risky split stays worst-of. - **Finding class in the convergence law.** Findings carry an optional stable `class`. A class carried by a resolved id that reappears on a new id with no surviving prior id carrying it is `class_reopened`, whether or not a classless round sits between; a blank class is an invalid round; a round can carry both a reopened id and a reopened class and stops on either. Repair stops, the traversal returns NOT_PROVEN, and the caller goes back to Plan, because the design is wrong, not the patch. Same rule in `run_once.py` and `workflows/rpi.js`, driven by one shared fixture file, with executed canaries in the cathedral gate. - **Plan identity, and the premortem's risky test.** On a risky scope the Plan carries `binding_judge` and the evidence it expects to orphan; `plan_digest`, when the plan declares one, must equal the script's digest over the acceptance, the write scope, the binding judge, and the intent-source digest, or the traversal stops before Implement. The risky test over a declared scope is glob intersection: witness paths, the risky regexes over the glob with wildcards instantiated, and any literal prefix that reaches a risky root; a bare `**` or `*` is risky. - **Intent is verified before anything is built.** An `intent-snapshot` receipt re-derives the digest and compares the snapshot bytes to the caller intent before premortem or Implement; absent, mis-hashed, or wrong stops the traversal with `plan_identity_mismatch`. The Python reference refuses on the same rule. Declared scopes are canonicalized (repeated `./` collapsed; absolute paths and `..` refused at plan exit), and the four spellings of `tests/**` share one plan digest while `tests/` does not. - **Orphaned evidence is a receipt, not a surprise.** After Implement and after every repair round the workflow runs `scripts/evidence-orphans.sh` over the runtime-derived path union, keyed on the round so a same-set repair still reruns it and appends every scorecard and fixture set whose bound evaluator or skill digests cover them to the check receipts the validator reads, each with a cause (`changed_path`, `digest_drift`, `both`, `skill_changed`). Plan budgets that recapture up front. On today's tree the receipt already lists the 08-16 and 08-26 sets whose bound harness digests moved long ago. - **Doc claims are gated.** The stronger scan already caught one stale claim in the tree (a 2026-06-22 harvest note calling evidence committed after Train 2 had removed that tree); the sentence now says where the evidence went. `docs.claims-tracked` (blocking) requires backticked repo paths in `evals/` and `docs/evals/` markdown to be tracked, and sentences that say published, tracked, or committed to name files that exist. - **Risky scope is a glob intersection, not a heuristic.** One segment-wise `globsIntersect` over one list of risky segment globs answers both the declared-scope question at Plan exit and the changed-path question for the cross-family rule; two heuristic designs missed in opposite directions and were replaced rather than patched. - **Contracts.** rpi gains a judgment dispatch table: risky scope routes to premortem, a risky split to council, an irreversible landing to one-way-door as a caller choice. plan, implement, validate, premortem, council, and learn carry their halves; learn proposes a deterministic check for any finding class that repeats. `docs/architecture/rpi-traversal.md` is the source of record and was updated with the flow. **Evidence on the tip.** Regen check clean; Go bar; full gate green with a HEAD-built binary (the new gate fires and passes); CI's literal bats command green; rpi unit tests; cathedral gate with executed canaries for the class law; routing goldens; token budgets; gate-tightening ratchet; coverage 1 of 12 unchanged. **Judgment, stated plainly.** Six judging rounds, each on a frozen tip. The caller elected Fable 5.1 as the binding validator; its round-6 verdict is PASS with no verdict-driving findings (lows: a council payload outside the schema throws a raw error instead of recording `invalid-rulings`; two stale comments; a dead parameter; a dead fixture element; the contracts say exactly one ruling per finding while the code enforces at most one). The cross-family Codex read of the same tip failed on three input-validation items, which the final commit closes: control characters are now refused on the original scope value with the full Unicode control class and the caller scope is canonicalized in place so one spelling reaches every prompt; the intent receipt no longer forgives a trailing-newline difference; and the Python reference accepts a digest string only from a verifier callback and matches digests with `fullmatch`. One half of one Codex item stays open and is disclosed: the workflow does not itself hash the caller intent's bytes to compare with the plan's declared digest (it relies on the receipt's exact-bytes match plus digest equality); binding that locally means every probe fixture carries a real digest of its intent text, which is the next pass. Also open, pre-existing and outside this train: the risky-surface list names `lib/**` while the preamble every gate sources lives under `scripts/lib/`. Across the rounds the loop behaved as designed once: the council closure mechanism drew a fresh finding class three rounds running and was cut under the train's own class rule instead of hardened a fourth time. |
||
|
Bo
|
1cebfee94d |
Measure one skill: sealed probe dispatch and the first current ledger row (#1101)
## Measure one skill: filesystem-sealed probe dispatch and the first current ledger row Provenance: the instrument-panel roadmap (`docs/plans/2026-08-26-instrument-panel-roadmap.md`) closed out with zero skills measured because control-arm reps read `skills/<skill>/SKILL.md` from the checkout and the operator's skill roots mid-run. Its named successor was filesystem-sealed dispatch. This PR builds the seal, then uses it, and the instrument caught one more leak on the way. **The seal.** Codex's own seatbelt does not nest inside an outer `sandbox-exec` profile, so a rep now runs under an outer profile with Codex's sandbox bypassed for exactly that reason: `file-read*` denied on the checkout and every skill root (`~/.agents`, `~/.claude/skills`, `~/.gemini/skills`, `~/.codex/skills`), `file-write*` denied everywhere except the rep's workspace, a scratch HOME, and the temp roots. HOME and CODEX_HOME are scratch (auth symlinked). Dispatch refuses without `sandbox-exec` unless `PROBE_SEAL=none` is set, which marks the run coverage-ineligible. The seal is recorded in the capture contract (`agentops-skill-probe-capture.v3`, a `seal` block), coverage eligibility requires `seal.mode == seatbelt`, and `verify-scorecard` re-checks that the denied roots include the repository and the four skill roots. Existing 08-26 sets become `legacy-unsealed` and ineligible, which changes nothing they were counting. **The leak the seal did not cover.** The first sealed capture showed the xhigh control-2 rep listing the shared live workspace, finding `treatment-1.prompt`, and reading it; the treatment prompt embeds the skill. So every rep now gets its own empty workspace, prompts reach a rep on stdin only, raw JSONL and stderr live in a harness-private dispatch directory the rep can only stat, and a `sibling-prompt-read` trap degrades any rep whose successful command names a prompt, contract, seal, or stage file or whose output lists a sibling's artifacts. Replaying the contaminated set under the new trap turns its REGRESSIVE into INERT with the contaminated rep degraded; that set is superseded and not committed. **The second pass, after the judges read the first.** A Fable validator passed the first sealed capture with four findings and a Codex read failed it with seven, all real: the whole per-user temp tree stayed readable (hundreds of stale probe directories held prompt files with the canonical skill bytes), sibling checkouts and the real `~/.codex` sessions were readable because seatbelt matches the traversed path and the skill roots are symlinks into the main checkout, the dispatch directory was writable so a rep could rename an earlier prompt into its workspace, the contract trusted asserted seal metadata (a hand-made Linux record was coverage-eligible), the only real-Darwin test nested seatbelt and proved nothing, and the operator's `config.toml` was symlinked in whole, so every rep started the operator's MCP servers. The seal is now one run directory removed on exit (`home/ ws/ tmp/ dispatch/`, reset per rep); reads are denied under the real HOME, the real TMPDIR, `/tmp`, the checkout, the git common directory, and each skill root's resolved target; `dispatch/` is write-denied with metadata-only reads; `file-link` and `file-clone` are denied on every denied root; the rep's config is rebuilt from top-level scalars only and `auth.json` is copied, not linked. The contract binds platform, mechanism, wrapper, the denied and allowed roots, the rep environment, and the sanitization, and coverage refuses any record that omits them; the Darwin test performs its reads inside the seal. One recorded hole remains by necessity: the codex launcher under `~/.local/bin` and `~/dev/dotfiles/bin` is re-allowed by literal path, and coverage refuses any allowed path inside the checkout. **The third pass, after the judges read the second.** Both judges failed the hardened tip again, and the headline was network: bypassing codex's own sandbox to nest the outer seal had dropped every inner deny, not only the filesystem ones, so a rep could fetch the canonical skill from GitHub while the disk was sealed. A rep now reaches the network only through a harness-owned CONNECT proxy (`scripts/lib/probe-connect-proxy.py`) with a host allowlist discovered from real runs (`chatgpt.com`, `ab.chatgpt.com`, the rotating `.oaiusercontent.com` suffix, and the API-key hosts), behind a seatbelt `network*` deny that admits only the proxy; every attempt is logged, a refused one degrades the rep, and web search is disabled in the rep's config. The profile is now rendered from the bound seal block by one function, so `verify-scorecard` reconstructs it and requires the digest to match; the recorded launcher chain is the only allowed read under a denied root; the wrap invokes `/usr/bin/sandbox-exec` by absolute path; the real CODEX_HOME and the Darwin cache directory are denied and bound; every non-stdio descriptor is closed and `/dev` writes narrowed to four devices; each rep runs in its own process group that must be empty before the next reset; one minimal config is generated per run, bound by text and digest, and re-parsed after each rep; one guarded trap covers the run root and any unpublished stage; and the coverage gate prints an eligibility row for every ledger row that names a scorecard. What stays open is stated in the README: allow-default still governs what the profile does not deny. **The fourth pass, after the judges read the third.** Both judges failed the third tip on properties the record carried but the verifier did not check. GNU `timeout` sets its own process group and ran outside the seal, so the per-rep reap was vacuous (four sleeping survivors after the test passed); the seal is now the outermost process, timeout runs inside it with `--foreground`, its absolute path is bound, and the reap counts survivors before signalling. The verifier now pins, not merely records: the egress host set (a module constant), port 443 only, no unix sockets, a loopback proxy, the generated config text equal to the renderer's output, the real CODEX_HOME and cache root inside the denied roots, writable roots under the run root, the four device paths, the environment allowlist, the launcher chain's digest against the producer identity, and it refuses unknown record fields. The proxy logs an attempt before dialing, captures the rep at accept time, refuses names that resolve to private or loopback ranges, and its source joins the evaluator hash set; the per-rep egress log is published with the fixture set. Withdrawn ledger rows get their own eligibility reason. The README lists which seal fields the verifier pins and which it only records; the cross-capture aggregate is recounted to 4 of 14. Still open and disclosed: Mach IPC and Apple Events beyond the filesystem and network seals, exact-byte environment sealing beyond the allowlist, and cleanup ownership by inode. **The fifth pass, after the judges read the fourth.** Both judges found the seal, proxy, reap ordering, transcripts, and scorecards sound and failed the tip on claims the tree did not hold: the egress log the docs called published was caught by the repository's `*.log` ignore rule, a `setsid()` child escaped the process-group reap, and one ledger sentence said no rep ran a command when a control rep had run `rg` over its empty workspace. The log is now tracked and required, strictly parsed, with both allowed and refused counts recomputed per rep; the reap is fatal to the capture when it cannot prove emptiness and an `lsof` sweep of the run root catches a session escape; the rep launches through a real `env -i` boundary; the timeout budget, the launcher chain's adjacency, every required root in both read and link denies, the evaluator match, and the trust-table value are pinned by the verifier; and every sentence in the ledger, README, and runbook names the check that enforces it. The remaining gap is stated: a child that both escapes the session and holds nothing under the run root is not detected. **The sixth pass, after CI read the fifth.** The first CI run of this PR failed on Linux: the launcher-chain pin walked the capturing Mac's filesystem, so on the runner both sets read as unsealed and the coverage headline fell to zero. A pin that holds on one host is not a pin. The chain is now bound as structure in the seal record (each link's path, kind, symlink target, and the final file's digest, with the invoked path as the head), the verifier checks that record on any host, and it cross-checks the live filesystem only where the head exists, reporting which happened. A gate-level test runs the coverage gate twice, once with the chain made absent, and requires byte-identical output. **The measurement.** `premortem-plan-shape-t2` (two seeded structural defects, band [1,4]) captured seven times on 2026-09-03, once under each seal; the first six sets were deleted with their scorecards when a judge or CI broke that seal. The rows that count are the seventh capture: low INERT (control 0/2, treatment 0/2) and xhigh INERT (control 1/2, treatment 1/2); no rep ran a command, no egress was refused, both per-rep egress logs are tracked and verified, and both scorecards verify with every pinned field, the launcher chain checked as a record and cross-checked on the capturing host. The superseded captures read, at low, BEHAVIORAL 1/1, INERT, INERT, BEHAVIORAL 1/2, INERT, BEHAVIORAL 1/2 and, at xhigh, INERT, BEHAVIORAL 2/2, INERT, INERT, INERT (control 1/2), INERT. The reversals across captures are recorded as an unresolved N=2 observation, not variance; across all seven the treatment arm put the marks in band in 7 of 26 usable reps and the control arm in 2 of 28, and the ledger marks that aggregate as disclosure, not a row. Headroom is SEPARATED at both levels. This is the first current, manifest-backed ledger row; it records response-shape behavior change, never quality uplift, and what it records today is INERT at both levels. **Evidence on the tip.** Probe, exec-lib, metadata, coverage, headroom, and adapter bats suites green with the Darwin seal, survivor, session-escape, and environment-boundary tests running; `verify-scorecard` on both scorecards; `check-skill-probe-coverage.sh` reports measured 1 of 12 with the 2026-08-26 row labelled `verdict-withdrawn`; regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go bar, lint, security, and the gate-tightening ratchet green. **Judgment, stated plainly.** The caller elected Fable 5.1 as the binding validator. Its fifth-round verdict on the pre-CI tip was PASS with no verdict-driving findings; its read of the portability delta failed on one test-only point (the coverage test had dropped the `measured` pin that CI tripped on) which this tip restores. The cross-family Codex reads of the same tips are FAIL. They closed the network seal, the environment boundary, the fatal reap, the data roots, the config binding, and the documentation, and they keep these open, recorded here as the next pass rather than hidden: the evaluator identity check still accepts the pre-proxy four-file shape for legacy sets; an inherited absolute `CODEX_EXEC_TIMEOUT_BIN` is trusted without a probe; a refusal logged under a rep label outside the expected set is not rejected; an empty top-level `[projects]` table passes the drift check; the `lsof` sweep treats a failing `lsof` as a clean result; and the launcher chain, now bound as a record that any host can check, is corroborated against the filesystem only on the capturing host, so a record whose head does not exist there is taken on its word (the docs say so; Codex wants the live walk required at capture and the invoked head bound into the producer identity). Fable also noted a test that mis-resolves when TMPDIR sits under `/private/tmp`, a `2026-09-04` date in one refusal message, and that the low control replies used a `S2: FINDING <reason>` shape the discriminator scores as "plan not walked" rather than by band (outcome-invariant). Each of those changes a harness or verifier file and would orphan the two rows again, so they are filed as the follow-up instead of an eighth capture today. Across six judging rounds the open set shrank only from round four on; the convergence law's stop signal was reached at round three and the run continued on the caller's instruction to land the instrument. The two ledger rows are honest about what they measure, and the seal they ran under is the one this tree can verify on any host. |
||
|
Bo
|
10f0277bdb |
Legible membrane, Train 2: what a stranger meets (#1100)
## Legible membrane, Train 2: what a stranger meets Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus, findings F5 through F9. This train is sized by a consumer inventory built with `rg` on the tip before any lane was written; the promoted-set directory move the audit proposed is deferred because that inventory shows skill-builder backing two blocking gates, swarm pinned by the cathedral gate and a routing golden, using-gc required by Go code, and `ao skills link` unable to install a second root. That inventory is the plan for a later train. **What changes.** - **Archival sweep by consumer disposition.** 172 audit snapshots, 29 pawl receipts, the `evals/workbench` and `evals/membrane` trees with their two bats consumers, four stray scratch docs, four retired eval contracts, and nine caller-less `scripts/check-*.sh` are deleted; git history is the archive. Every machine list that referenced them is pruned (evidence-grounding baseline, preamble grandfather, broken-links allowlist, `.gitattributes`, `.gitignore`, two eval fixtures, the workflow-coverage deferred list). `docs/audits/manifests/` and `.agents/ao/config.yaml` survive because they have live readers. About 48,000 lines. - **Three skills retired.** `goals` (alias of fitness), `shared` (tombstone), and `scope` (folded into plan step 3 as five write-scope checks). Consumers edited; the probe denominator exclusion for goals pruned; Codex package and golden count pins updated. - **Negative routing** on research, codebase-recon, reverse-engineer, premortem, one-way-door, and council, all within the 180-char budget, with a teardown golden (`rq-08`). One wording was changed after the router's prefix stemming showed "repository teardown" leaking into the wrong skill. - **Every promoted skill answers "It's working if"** with observable tells in backticks, and carries a paste-ready `## Prompt` with a concrete subject. Two fictional `ao` subcommands a draft prompt named were caught by the body-ref validator and replaced with real commands. - **Doctrine diet on the core five.** rpi, plan, implement, validate, and anti-ceremony drop from about 5,100 words to 3,600 (bodies from 4,700 to 3,150) by moving the shared ownership boundary, dated incidents, and mechanics tables into step-loaded references (`skills/rpi/references/boundaries.md`, `skills/validate/references/mechanics.md`, `skills/plan/references/ground-truth-routing.md`). Every cathedral canary and every skill validator grep survives unchanged. - **ADR-0018** records the goals, shared, and scope retirement; the cathedral gate tombstone and the routing goldens cite it instead of ADR-0017. - **Router and twins.** `ao skills find` holds a description's "Not for X; that is <sibling>." sentence out of its haystack, so premortem no longer ranks first for "is this live decision reversible" (golden `rq-10` pins the reciprocal of `rq-02`); a penalty variant was tried and reverted because it suppressed skills the caller named outright. A declared trigger phrase of two or more words quoted whole in the query now earns the name weight once, so "check this change" lands on validate rather than on reality-check's name token; a live-catalog test pins seven such queries. Single-quoted YAML descriptions unescape `''`. The Codex catalog keeps the exclusion sentence, and a closing `>` no longer turns `<run-id>/codebase-recon.json` into an invocation. - **Residue the judges found.** handoff, learn, and status open a `## Contract` heading after their tells; validate's prompt names its helper at `skills/validate/scripts/validate.py`; the explicit-skill prompt catalog names only live skills (five stale prompts replaced by nine, floor 20 restored, TESTING.md names the suite); the corpus-delta receipt binds the runner's path and SHA-256 and labels a `live_agent` claim as an unverified caller declaration; the probe README and ledger describe the 12-skill denominator; SKILL-API counts 30 of 54. **Evidence on the tip.** Regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go build/vet/test green; lint clean; security gate quick PASS; docs-build warnings did not rise. Fresh validation by Fable 5.1 (caller-elected) and a cross-family read by Codex, both recorded in the PR thread. --------- Co-authored-by: Bo <bofuller55@gmail.com> |
||
|
Bo
|
568e99d436 |
Loop restore: converge and crank as control flow under the verdict contract (ADR-0017) (#1099)
## Loop restore: converge and crank as control flow under the verdict
contract (ADR-0017)
Intent source: `docs/plans/2026-09-03-loop-restore.md` (in this PR).
Decision record:
`docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md`.
**Why.** The 2026-07-14 single-pass cut (`482307762`) removed the
iterate loop (discovery, crank, converge, evolve, the learn write-half)
together with the unproven compounding claim, although ADR-0011 demoted
only the latter. The control flow was never demoted, and its absence
showed on 2026-09-02, when a three-lane fix needed eight validators and
two stops because the contract had no repair phase. This restores the
loop as control flow and nothing else: no knowledge store, no `ao
converge`/`ao crank`, no evolve, no canary. ADR-0004 and ADR-0011 stay
in force.
**What changes.**
- **RPI gains a bounded repair phase.** On `FAIL` or `NOT_PROVEN` with
findings, repair and re-validate freshly under the convergence law:
caller-declared `repair_rounds` (default 2); open finding set keyed by
stable `findings[].id`, union across validator families, non-growing; no
closed id reopens; the subject digest changed or, for `NOT_PROVEN`, new
digest-bound evidence resolved a named gap. Converged = fresh PASS plus
cross-family PASS on risky surfaces. Plan and Implement keep their
single dispatch. `skills/rpi/scripts/run_once.py` models the law as pure
data (33 tests): rounds are validated for shape (digest required, no
duplicate ids, no PASS with findings, no FAIL without findings),
condition 4's evidence branch needs a NOT_PROVEN previous round, a
non-FAIL current round, new evidence, and a resolved finding, and a PASS
over unchanged bytes after a FAIL is a flip that reports NOT_PROVEN.
`workflows/rpi.js` runs validation as legs (spawned or external primary,
plus a caller-supplied `crossFamily.command` on risky surfaces) merged
worst-of with a union of stable ids; a risky surface without a
cross-family leg is `diversity_unsatisfied` and never converges or
enters repair; a failed repair or re-validation returns NOT_PROVEN with
no stale verdict. Validators return `subjectDigest`, stable finding ids,
and `evidenceRefs`.
- **crank returns as a thin wave executor** (113 lines): the caller
selects the wave and the repair bound, crank invokes RPI per lane
(parallel only on disjoint write and regen scopes), runs the wave
acceptance once, returns evidence, and stops. No retry, budget, queue,
claim, lease, Git, closure, or next-work ownership. Routing golden
`rq-07-wave-execution` ranks it first.
- **validate is cross-family by default on risky surfaces**
(`cli/internal/gates/**`, `scripts/check-*.sh`, `tests/**`,
`skills/*/scripts/**`, hook policies, `lib/**`, security-scanned paths)
with the LAW-0 dispatch table: Claude orchestrating uses read-only
`codex exec`; Codex orchestrating uses an interactive Claude session in
an NTM pane, never `claude -p`. No live adapter means
`diversity_unsatisfied`, which on a risky surface is `NOT_PROVEN`. The
full literal CI command set runs once on the final integrated subject;
routine rounds keep the receipt-driven freshness contract.
- **Conformance assertions flipped under ADR-0017 only:**
`scripts/check-cathedral-cut-conformance.py` (crank live; "Stop
regardless" replaced by positive canaries for the law's four conditions;
a bounded `for` loop that compares against `repair_rounds` is required
in `run_repair_phase`, and the gate executes the law's canaries against
the reference behavior), `workflows/rpi.js`,
`skills/rpi/scripts/validate.sh`,
`evals/agentops-core/rpi-behavior.json`,
`skills/rpi/references/rpi.feature`. Every single-pass public surface
(README, AGENTS.md, PRODUCT.md, CI-CD, agent-workflow-reference,
rpi-traversal, cli/README, quickstart and demo commands, the
operating-contract and product-boundary bats, the Codex-description
oracle) now states repair to convergence.
**Known approximation, disclosed.** The Claude conveyor has no
deterministic shell primitive, so changed paths are derived by the fresh
validator (git status and diff against the clean pre-run tree) and
unioned with the implementer's report; risk is classified over that
union and unreported paths are coverage findings. A validator is still a
model; runtime derivation outside every agent is a follow-up. Family
distinctness of the cross-family leg is asserted by the caller's choice
of command and not verified by the script.
**Not in scope.** Premortem stays a single advisory judge and Plan still
only names the first check (phase boundaries unchanged). No `verdict.v2`
or `rpi-report.v1` change. The loop's own effect on outcomes is
unmeasured and owed a seeded-defect probe, like the rest of the corpus.
**Evidence on the tip.** Regen check clean; full gate green with a
HEAD-built binary; CI's bats command green; Go build/vet/test green;
golangci-lint clean; security gate quick PASS; one fresh validator over
the whole diff; one cross-family read of the design before
implementation (13 findings folded) and two of the integrated diff (9
findings in round one, 11 by round two, 15 by round three, each round
repaired and re-reviewed; the fresh validator passed the tip after round
two and the final tip
|
||
|
Bo
|
e556834d3d |
Legible membrane, Train 1: Codex projection, executable entry points, green run-all, honest runtime docs (#1097)
## Legible membrane, Train 1: fix what ships Intent source: `docs/plans/2026-09-02-legible-membrane-plan.md` (added in this PR). Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus. Three defects a stranger hits today, all verified on `main` before the change: 1. **Codex projection truncated 51 of 56 skill descriptions mid-clause** (`scripts/codex-sync.sh` capped prose at 44 chars, so the Codex router read "Freshly judge whether a finished change is Triggers: …"). Now: first sentence of the source prose plus the full `Triggers:` clause, abbreviation- and quote-aware; the per-catalog bound is computed live as "Codex prose average may not exceed Claude's" (cross-multiplied, floor-free) with a 180-char hard ceiling. `using-flywheel` is exempted from runtime-phrase rewriting so its cross-runtime text projects verbatim. The dormant `ao codex ensure-start` emitter and the `# /x`→`# $x` title rewrite are removed. A literal oracle pins five twins and a rule oracle covers all 56. 2. **23 shebang-bearing shell entry points were committed non-executable**, including `scripts/regen-all.sh` and three gate scripts. Now `100755`, with an advisory gate `shell.exec-bits` (fast+full, fail-closed on enumeration error, reads the index blob, skips symlinks by stated policy). 3. **`tests/run-all.sh` was red on `main`**: the GOALS validator asserted a pre-08-25 file shape, and validate's description exceeded the 180-char budget. The validator now parses only the `## Gates` block (stops at any heading, like production), requires cited script paths to exist, takes its path via argv, and fails closed on tokenization; four negative fixtures. validate's description is 177 chars and gains the reality-check negative. `AGENTS.md` quotes CI's authoritative bats and Go commands. README and `docs/install-day2-ops.md` replace "No other runtime is required" with a runtime table derived by reading each skill's procedure (HARD / OPTIONAL / conditional), covered by a bats test that grounds every HARD row in an invocation line. Evidence on the tip: full gate 71/71 (HEAD binary), CI's bats command green across `tests/scripts/*.bats`, `tests/run-all.sh` green, Go build/vet/test green, golangci-lint clean, security gate quick PASS, `scripts/regen-all.sh --check` clean. Each lane was validated by a fresh context; the integrated train had two cross-family (Codex) review rounds and every finding is closed on this tip. Out of scope (successor intent): promoted set / `skills-internal/`, the process-artifact sweep, "It's working if" blocks, routing clusters, doctrine diet. One pre-existing drift noted for a follow-up: `packs/agentops-executor/agents/validator/skills/validate/SKILL.md` carries a stale third description that no gate binds. |
||
|
Bo
|
e69144d6d1 |
Train 2: seeded-defect probes + gate hardening — the instrument catches its own contamination (#1088)
> **⚠ Open review residues — merge is Bo's call (spiral stop: second
non-PASS on this train's intent).** The cross-family re-check affirmed
the substance (contaminated fixture reproducibly UNMEASURED with four
real reads; all six scorecards match fresh replay; coverage honestly
12/12; single-level rule strictly tightening; 70/70 gates; full suite,
lint, validators green) and states plainly that **the zero-result
acceptance miss is not the reason for failure**. What failed, each with
its fix:
> 1. `evals/skill-probes/LEDGER.md:71` + `RUNBOOK.md:75` — a summary
block my withdrawal sweep missed still says "Exactly one row was earned…
1/12… council SEPARATED… worked for one of four". Fix: rewrite that
block to the post-withdrawal truth (zero rows earned, 12/12, council
UNMEASURED pending a second level).
> 2. `scripts/lib/probe-fixture-metadata.py` — the contamination
docstring overclaims: "a failed attempt loaded no bytes" is false for
`cat …SKILL.md; false` (read succeeds, compound exits nonzero), and
glob/copy-then-read indirection evades. Fix: state the detector's honest
boundary (exact-string, final-exit heuristic; evasions listed) — the
mechanism is a floor, not a seal, which the RUNBOOK already says.
> 3. **Caller decision:** the `scripts/probe-skill.sh` stderr-literal
exclusion is validator-verified fail-closed but sits outside L2's
declared write scope — it needs your explicit scope blessing (or
reversion, which re-voids all live dispatch on codex-cli ≥ 0.14).
> 4. Minor, for the record: my commit message reports security-gate
quick PASS (true in my run); the re-checker's environment reported
WARN_QUALITY from a missing radon + a govulncheck toolchain mismatch
unrelated to this diff — and the grounding gate's dead-SHA class is
UNCHECKED on shallow clones, including both our checkouts.
> **Acceptance status, told straight: L2's plan clause ("≥ 2 skills move
off UNMEASURED") is NOT met — the honest count is zero.** The lane filed
one BEHAVIORAL row (premortem); its fresh validator dumped the committed
transcripts and proved every rep, both arms, had fetched the skill off
disk mid-run — the separation was band-spray variance between two arms
holding the same bytes. The row is withdrawn, the ledger honestly reads
12/12 unmeasured again, and the train's deliverable became something
better than the number: the harness now mechanically catches that
contamination class forever. Merge is Bo's call.
## What
Train 2 of the [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md):
seeded-defect probes for the judgment spine (L2) and gate hardening
(L3), plus one repair commit responding to L2's validator findings.
**L3 — gate hardening** (`instrument/gate-hardening`, lane verdict
**PASS 11/11**)
- `gate.tightening-ratchet` (advisory): diffs `BASE_REF...HEAD` and
fails on six textual loosening shapes in the gate surface unless a
`Gate-Loosen-Reason:` trailer justifies it; tightening always free.
Detection boundary documented in the script header — silent on semantic
weakening it cannot parse, fail-closed on what it can.
- `evidence.grounding` (advisory): mechanical scan of `docs/audits/**` +
`docs/evidence/**` for cited paths that don't exist, unresolvable
full-length hashes, and scaffold leaks. 45 real findings on the live
tree, each **baselined with its argument** (directory-pinned baseline
with the added-line rule as teeth: a finding on a newly-added line is
never excused). One genuine live defect surfaced:
`docs/evidence/membrane-receipts.md` claims a generator that no longer
exists.
- Registry 68 → 70; `ao gate check --full` 70/70.
**L2 — seeded-defect probes** (`instrument/seeded-probes` + repair
`927be2184`)
- Four tier-2 probe packages (validate, premortem, council,
one-way-door): sub-40-line realistic artifacts, two planted defects
each, `canonical-skill` treatment, act-grading discriminators calibrated
both directions (21 hand-written transcript cases) — the lane's
validator wrote its *own* counterexamples and the discriminators held.
- 28 live dispatches (gpt-5.6-luna, low + xhigh) through the sanctioned
executor with `CODEX_HOME` isolated. Six immutable v3 fixture sets
replay deterministically.
- **What the run actually measured:** `one-way-door` and `validate`
scenarios SATURATED (retired to the RUNBOOK, no ledger rows — the
headroom gate doing its job); `council` was measured at one level and
the lane *refused* to file the row the gate's labeling bug would have
allowed ("filing it would have laundered a void row into the acceptance
number"); `premortem`'s BEHAVIORAL row was filed, then **withdrawn**
when the validator proved contamination.
## The repair commit (one bounded round, per the plan's validation
strategy)
- **`skill-read-contamination`**: `classify_bytes` now degrades any rep
whose transcript shows a successful command reading a `SKILL.md` — one
insertion covering live and replay. RED proven against the committed
contaminated fixture (replayed BEHAVIORAL before; UNMEASURED 0/0 after),
bats case added (probe-skill 37/37). The six scorecards were false
records under the rule: deleted (the harness's immutability guard
rightly refuses overwrite; git keeps the bytes) and regenerated —
contamination turned out broader than the lane knew (validate lost both
arms at xhigh, one treatment rep at low; one-way-door lost an xhigh
rep).
- **Headroom gate single-level fix** (L2's disclosed gate finding): a
group measured at one effort level whose control aced it is now
UNMEASURED ("capture a second level before any verdict row"), not
SEPARATED. RED test first; strictly tightening; the council group now
reads honestly.
- LEDGER row → WITHDRAWN with the full story; RUNBOOK's two falsified
sentences corrected; `CODEX_HOME` isolation documented as proven
insufficient, the transcript-level trap as the floor until dispatch is
sealed at the filesystem.
- Ride-along accepted with justification (validator-verified
fail-closed): the harness's stderr rule now excludes exactly one
whole-line literal — codex-cli ≥ 0.14's own "Reading prompt from
stdin..." announcement, which was degrading 100% of live reps on a clean
exit.
## Evidence
Full Go suite exit 0 · go-lint clean · `ao gate check --full` **70/70**
· security-gate quick **PASS** · four skill validators · probe-skill
37/37 · headroom 10/10 · tightening-ratchet + grounding bats green ·
`regen-all.sh --check` current. Per-lane fresh validators (L3 PASS; L2
FAIL → this repair); cross-family Codex verdict on the integrated train
recorded in the session report.
## What this train proves
The membrane worked on its own measurement pipeline, three layers deep:
the lane refused to launder a void row; the fresh validator caught
contamination the lane's isolation missed; the repair turned that catch
into a permanent mechanical trap. The cost is the honest headline — zero
skills measured — and the asset is that the *next* live wave runs
against a harness that cannot silently repeat any of this.
## Follow-ups
- Seal live dispatch at the filesystem (arms must not be able to read
any skill corpus) — then re-run the premortem and council waves for real
measurements; council additionally needs its second effort level.
- `docs/evidence/membrane-receipts.md` generator claim (surfaced by
evidence.grounding).
- The `Gate-Loosen-Reason` ratchet does not yet cover
`scripts/probe-skill.sh` — the stderr-literal class belongs under it.
---------
Co-authored-by: Bo <bofuller55@gmail.com>
|
||
|
Bo
|
8cdcb5a903 |
Train 1: measurement substrate, context diet, retrieval-eval contract (instrument-panel roadmap) (#1087)
> **Residues closed on the caller's merge instruction** (`499d916a6`): the round-2 findings were the same failure shape — round-1 repairs patched cited lines instead of sweeping the class — so this commit sweeps each file whole: every remaining SATURATED-row-append site in skill-eval now routes to RUNBOOK retirement, the human-only-skills *description* is runtime-conditional, premortem's "(MEASURED)" label is gone, SKILL-API's context table carries all 25 rows and the enforcement table gains `disable-model-invocation`, and the fixture-identity claim is stated precisely (probe id, honesty note, and control arm are the only differing fields — as the acceptance permits). Post-sweep: validators, full Go suite, 68/68 gates, goldens + headroom bats green, projections current, gemini in sync. Merging per Bo's instruction. ## What Train 1 of the accepted [instrument-panel roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md) (intent landed at `986a4feaf`): the measurement substrate, the skill-context diet, and the retrieval-eval contract. Three worktree-isolated lanes, each independently validated by a fresh context, plus one integration commit. 103 files, +5,510/−76. **L1 — measurement substrate** (`instrument/measurement-substrate`) - Gate `skill.probe-headroom` (advisory, Fast|Full): answers the question `skill.probe-coverage` cannot — not "does a probe result exist" but "could one have existed at all". The rule, ported to Go (`cli/internal/probeheadroom` + `cli/cmd/probe-headroom` behind a thin check script — the witness-crosscheck pattern, **no new `ao` root command**): control arm ≥ 0.75 with ≥ 2 usable reps at ≥ 2 effort levels ⇒ SATURATED (void row, not an honest null); UNMEASURED outranks it; treatment-silent ⇒ FLOOR; else SEPARATED. RED first: both committed fixture pairs read `INERT` to everything else in the repo; the failing separation test predates the implementation, and a bats negative-control swaps fixture bytes and asserts the gate flips. - **First reading on real data: 7 of 11 historical probe groups are SATURATED** — including both `validate-not-proven` runs. Those INERT rows were never honest nulls; they were void. The 0/12 ledger number now argues itself. - Declared denominator for probe-coverage: `scripts/.skill-probe-denominator-exclusions`, fail-closed parser (entry without an argument, stale slug, or duplicate ⇒ exit 2). One entry (`goals`, a pure alias-of `fitness`). Net effect deliberately zero (0/12 → 0/12: alias left, `one-way-door` entered) — the gain is a declared number, not a better-looking one. - Re-landed from the recovered clean-room commit (`9872483bd`), re-validated against *current* main: `skill-eval` (defers saturation to the gate id; its shell scripts dropped, not shipped — ratchet intent), `route`, `one-way-door`, premortem reversibility check, council `caller_challenge` (schema + validator, per the agent-core boundary that the panel may challenge, never overrule). **L4 — context diet** (`instrument/context-diet`) - `disable-model-invocation: true` on 4 human-only skills (key verified verbatim against Anthropic's docs). The plan guessed 35 candidates; the graph said otherwise — 23 carry `user-invocable: true`, and 19 of those are excluded on cited evidence (rpi consumes anti-ceremony/implement/plan/validate; workflow scripts reach others; `goals` is a live migration tombstone). The exclusion evidence is retained in the lane report. - One router skill (`human-only-skills`) — the single always-loaded description that replaces four; it hints, never fires. - `.out-of-scope/` formalized with this week's three refusals (checked-in knowledge corpus; ee self-improvement loops; whole-skill A/B as the measurement unit), each citing its evidence. - Deterministic proof, no model eval: before/after bytes of always-loaded description load reported in the lane summary. **L5 — retrieval-eval contract** (`instrument/retrieval-contract`, lane verdict PASS 10/10) - `AGENTS.md` federated row now names **ee (eidetic-engine)** as a concrete caller-selected memory system — consume, never build; symlink intact. - `schemas/pack-quality-expectations.v1.schema.json` + 4 routing goldens + `scripts/check-routing-probe-goldens.sh` graded against `ao skills find`, wired as an **advisory** nightly job. Zero goldens is a failing state — no new zero-denominator green. - **The instrument caught a real miss on day one — and its own prescription fixed it.** Golden `rq-04` expects `validate` for "judge whether this finished change is actually proven before I merge it"; at authoring, `ao skills find` ranked the *forbidden* `premortem` first and `validate` nowhere in six natural phrasings. The pointer-wording-first repair (validate's description gained the caller's own words: finished, proven, verdict, merge) now ranks it #1 at 0.333; grader 6/6, and the golden pins the repair — a description regression reopens it. ## Integration `regen-all.sh` once over the merged lanes (catalog 52 → 56, four new codex twins, mesh, router, manifests); `skills/route/SKILL.md` catalog/router links became prose repo-root references (the projected twin cannot resolve `../catalog.json` — this was both the portable-conformance failure and the sole broken doc link); `codex-portable-conformance.bats` pin 52 → 56. ## Evidence - `cd cli && go build ./... && go vet ./... && go test ./...` exit 0 · `ao gate check --full` **68/68** · four skill validators PASS · probe-headroom / routing-goldens / probe-coverage bats PASS · `regen-all.sh --check` all current. - Per-lane fresh validators re-ran every suite on detached content; L5 PASS; L1/L4 NOT_PROVEN solely on the projection-regen clause reserved for integration (their remaining acceptance observed green), settled above. Cross-family (Codex) review of the integrated diff recorded in the session report. - Two disclosed scope stretches accepted at integration: a one-line `.gitignore` entry mirroring the witness-crosscheck precedent, and the probe LEDGER.md fact-correction L1's own change made necessary (noted for Train 2's L2, which owns that file next). ## Cross-family review (Codex, fresh context) Round 1: **FAIL** — two blockers (the RED fixtures didn't isolate the control arm; the goldens grader was red where the plan's acceptance says green) and eight majors (contract contradictions in the re-landed skills, a converter-substitution false claim in the codex router twin, two overreaching `.out-of-scope` entries, stale SKILL-API counts). All repaired in one bounded round (`db68935a3`): fixtures now byte-identical outside the control arm, the routing miss actually fixed rather than tolerated, every cited contradiction reconciled at the source and re-projected. Post-repair: full Go suite exit 0, `gate check --full` 68/68, all validators and probe/goldens bats green, projections current, gemini byte-identity restored. Focused re-check verdict recorded in the session report. ## Follow-ups (Train 2, already planned) Seeded-defect probes for the judgment spine (every ledger row citing a passing headroom pre-screen) and the gate-hardening pair (`Gate-Loosen-Reason` tightening ratchet; mechanical grounding-validation over evidence docs). Plus, surfaced by this train: a latent `valid_keys`/schema divergence in `validate-skill-schema.sh` (two keys the schema defines are absent from the script's allowlist — pre-existing). |
||
|
Bo
|
2207c5b278 |
estate: goals fail-closed, contract map, verdict guard, workflow + docs honesty (5 lanes) (#1083)
> **Review findings closed.** The re-check's residue (`perl -pibak` bundled-cluster bypass) is applied in `2675bb383`: the perl alternation accepts any non-space run after the in-place flag, a new FIRE bats proves the deny, 48/48 policy-dispatch bats green, codex twins regenerated. Merging on Bo's instruction. ## What Five semantic lanes from the 2026-08-23 estate audit, each implemented in an isolated worktree and independently validated by a fresh context, plus one integration-fixup commit. **1. Goals: the zero-denominator green is dead** (`estate/goals-honest`) `GOALS.md` lost its Gates table in the 07-14 cut, so `ao goals validate` reported `Valid: true` with `GoalCount: 0` in the release smoke on every push. Now: zero parsed goals ⇒ `Valid: false` with an actionable error (RED test first); `GOALS.md` carries a real 6-row Gates table whose every `Check` was executed via `ao goals measure` before shipping; `goals-affects-files.yaml` pruned of dead ids; the smoke asserts `goal_count ≥ 1`. Two of the eight fitness properties (behavior-before-activity, stop-boundary) have no executable check yet — disclosed in GOALS.md rather than faked. **2. Front door: the contract file gets a map** (`estate/front-door`) `AGENTS.md` had zero build/test commands and zero layout — the Go mechanics lived only in `.claude/rules/go.md`, which Codex (the declared default runtime) never loads. Added a terse "Repository map and mechanics" section; compressed the architecture-poetry opener to plain language; README opens with what-it-does, the maintainer binary install moved to `docs/install-day2-ops.md`, and the quickstart now ends with an invitation to open an issue with your verdict. Deleted the dead `.claude/CLAUDE-base.md` + `build-claude.py` pipeline (zero consumers since June). `CLAUDE.md` remains a symlink to `AGENTS.md`. **3. Hooks: first policy that guards the user's invariant** (`estate/hooks-invariant`) All four shipped policies guarded this repo's own artifacts. New `core.verdicts:hand-edit` denies Edit/Write/Bash hand-writes into `.agents/ao/verdicts/**` — verdicts are content-addressed evidence written only by the validate flow. RED committed before GREEN (`d264682c0`); bats prove deny on hand-writes and allow on reads, intents, and the sanctioned `validate.py` writer. The lane shipped with in-place editors disclosed as un-matched; the cross-family review treated that as a blocker and the gap is closed in this PR (see below). **4. bdd-foundry stops instructing dead commands** (`estate/workflows-fix`) The R4 phase told agents to run `ao yield emit`; Phase 5 said pull holdouts via `ao scenario`; comments cited `ao membrane derive-checks` — all tombstoned, so R4 failed on every run. Emission is dropped per the tombstone's guidance (observation stays in the workflow result), Phase 5 grades in-run ground truth plus any caller-supplied register scenarios, and the lane's initially-overreaching "there is no external holdout register" wording was caught by its validator and corrected in integration. Bonus: `workflow-scripts-syntax.yml` globbed the gitignored `.claude/workflows/` (matched nothing in CI since inception) — now checks the tracked `workflows/` tree. **5. Docs honesty: retired vocabulary stops reading as live** (`estate/docs-honesty`) `intel_scope` removed from all 24 source SKILL.md + fixture + template + tutorial (declared by half the corpus, read by zero Go); the schema keeps accepting it for old-shape third-party skills but now self-describes as deprecated. `docs/SKILL-API.md` no longer presents `INTEL`/flywheel/forge/push/ratchet as live. `docs/knowledge-flywheel.md` states design-vs-implemented truthfully. ADR-0014's dead link fixed. 25 drifted `images/gemini/skills/` copies re-synced byte-identical per that bundle's own README contract. All codex projections regenerated once at integration (`regen-all.sh --check` passes). ## Evidence - `cd cli && go test ./...` exit 0 · 4 skill validators PASS · `policy-dispatch.bats` + `cross-runtime-hook-baseline.bats` PASS · `check-doc-hooks-drift` + `check-hookless-cold-start` PASS · `validate-goal-count` PASS · `node --check` on all 8 workflow scripts PASS · `regen-all.sh --check` PASS - From repo root: `ao goals validate --json` → `valid: true, goal_count: 6`; from a wrong cwd it now fails closed instead of reporting green. - Each lane branch carries its own commits; merge commits preserve the lane structure. Fresh per-lane validators re-ran every suite themselves; the one FAIL each (workflows wording, cross-lane doc fallout) was fixed in the integration commit, not waived. ## Cross-family review (Codex, fresh context) First pass: **FAIL** with one blocker — the new verdict guard explicitly excluded in-place editors, so `sed -i` silently mutated digest-named evidence (dispatcher probe returned exit 0). Repaired in `f53945c26` (one bounded round): three FIRE bats observed RED before the matcher landed (in-place sed, in-place perl, file deletion), two negative bats prove reads never fire; policy rationale and SKILL.md updated; codex twins regenerated; 47/47 bats green. Everything else in the first pass was affirmed (goals behavior, symlink/build bar, zero retired verbs in workflows, byte-identical gemini copies, current projections). Focused re-check: blocker confirmed closed via the dispatcher probe; one regex corner (`perl -pibak`, header note above) stopped the loop under the spiral rule. ## Follow-ups (not in this PR) - `workflows/bdd-foundry.js` `SKILL` constant points at `skills/behavior-first-planning/SKILL.md`, which was retired in v3.3 — every phase prompt references a dead path (pre-existing). Needs a design call: restore the phase discipline inline or repoint honestly. - Verdict-guard: the noclobber override redirect (`>|`) remains the one disclosed unmatched write shape. - `docs/UPGRADING.md` note for third parties still declaring `intel_scope` (schema accepts, nothing reads). |
||
|
Bo
|
ffb9f122af |
refactor(cli): delete the unconsumed eval/redact surfaces — the estate audit's mechanical cut (#1082)
> **Review findings closed.** The re-check's residue (app-seam family count) is applied in `9a2790ae7` along with the full-tier CI settlements: regenerated documentation index (generated file, hand-edit drifted it), regenerated CLI-surface count fixtures (top=18 sub=44 all=62), `Test-Removal-Reason` trailer for the deliberate test deletions, and the release-tag bats output list updated to the real changes-job set. 67/67 full-tier gates green locally. Merging on Bo's instruction. ## What Deletes the provably-dead 28% of the `ao` CLI and every reference to it, per the 2026-08-23 estate audit. −19.5K lines in the lane commit plus integration fixups. **Removed (each with zero live consumers, verified by consumer-grep + `go list -deps`):** - `ao eval` — 13 subcommands, ~10.9K LOC. Its would-be consumers were already tombstones (`scripts/eval-agentops.sh` printed `RETIRED`), `release.yml` hardcoded `--eval pass`, release evidence recorded `suite_count: 0`, and three of its module tests exercised subcommands that could never register (nil composition seats). - `ao redact` — its only declared caller (`skills/compile/scripts/compile.sh`) never existed. - `cli/internal/types/memrl_policy.go` + the orphan cascade it and eval left behind (`internal/scenario`, `internal/wiki`, `internal/runtimecmd`, `internal/redact`) — all with zero importers, verified before and after. - `scripts/check-memrl-health.sh` + `examples/schedules/feedback-drain-hourly.yaml` — a health check for the feedback loop amputated on 2026-07-14; it exits 1 on main today and the example instructs a verb (`ao feedback-loop`) that no longer exists. - `corpus.secret-scan` gate — vacuous: its file filter excluded the single tracked path its globs could match, so it scanned zero files; secrets are covered by the pinned gitleaks steps in nightly and release (validate's quick toolchain mode skips gitleaks). - Docs for the deleted surface: `docs/architecture/eval-architecture.md`, `docs/code-map/eval-lid-primitives.md`; `contracts/eval-baseline-ab.md` already carried a RETIRED banner and stays as history (delisted from the live index). **Kept, deliberately:** - `ao robot-docs` — the audit's "duplicate of `doctor robot-docs`" premise was false: they render different handbooks (whole-CLI vs doctor-scoped). Verified before acting. - `completion`, `demo`, `quick-start` — interactive human furniture, not dead code. - `corpus.witness-dolt-jsonl-crosscheck` gate — retargeted, not retired: its backing script is a hermetic self-test over real tracked fixtures; globs now point at the paths it actually exercises. - `cli/internal/evalsubstrate` — Go-dead but it is the declared mirror of `schemas/outcomes-rubric.v1.schema.json`; retiring it needs a paired schemas/docs/scripts decision (package doc comment records this). - `scripts/ci-local-release.sh` eval-evidence stanza — self-contained honest bookkeeping (`status: not_applicable`), invokes nothing removed. **Tombstones + migration:** `eval` and `redact` added to `removed_command_hint.go` and `docs/MIGRATION.md`; the now-false "(`ao eval` returned in 3.3 …)" parenthetical deleted; `go-cli.md` spine and the "Eval — the Learn seat" section updated; the dated research snapshot got a HISTORICAL banner via the docs-scope self-declaration mechanism (history not rewritten). ## Why v3.6.0 binary downloads: 4 darwin-arm64, 3 linux-amd64. Only 7 of 53 shipped skills invoke `ao` at all, and none of them touch this surface. The eval family was the single largest command surface in the CLI with zero live consumers — 28% of non-test Go maintained for nobody. ## Evidence - `cd cli && go build ./... && go vet ./... && go test ./...` — exit 0 (previously-failing `TestGoCLIDocSpineMatchesApprovedSpine` and `TestRemovedVerbsHaveMigrationRows` now pass) - `scripts/check-docs-cli-snippets.sh` PASS · `check-cmdao-surface-parity.sh` PASS (54 leaf commands) · `check-corpus-path-guard.sh` PASS · `check-new-scripts-use-preamble.sh` PASS · `ao gate check --dry-run` PASS - Implemented by a worktree-isolated lane, independently validated by a fresh context that re-ran the suite itself; the two failures it found were doc files outside the lane's write scope, fixed in the integration commit. Cross-family (Codex) review verdict included in the final session report. ## Cross-family review (Codex, fresh context) First pass: **FAIL** with two majors — (1) `quality.DeprecatedCommands` still mapped five rewrite entries onto the removed eval family, so `ao doctor --fix` would have introduced dead commands; (2) retained docs (formal-verification research links, applied-ood README run block, evalsubstrate hint strings) still prescribed removed commands. Both repaired in `4da85a0d4` (one bounded round), plus its two minors (types/AGENTS.md row, .gitignore unignore, family counts, gitleaks-coverage comment). Re-verified: full suite green, snippets gate PASS. Focused re-check: first-round findings confirmed closed; one new residue (the family count above) stopped the loop under the spiral rule. ## Follow-ups (not in this PR) - `cli/internal/quality/stale_refs.go` `DeprecatedCommands`: the five eval-target entries are pruned here; the older pre-existing dead targets (forge, inject, flywheel, ratchet, …) still need a map-wide reconciliation against the live registry. - `cli/internal/evalsubstrate` retirement decision (paired schemas/docs/scripts change). - `evals/scenarios/applied-ood/`, `evals/tier2-premortem/`, `evals/_stats/` retain historical `ao eval` mentions in prereg/holdout records — dated artifacts, left as history. |
||
|
Bo
|
e8339105b7 |
fix(ci): golangci-lint v2.13.1 for Go 1.27; stop archcheck TempDir race (#1080)
## What Bumps the repo-pinned golangci-lint from v2.11.4 to v2.13.1 (in `scripts/golangci-lint-v2.sh` plus the toolchain installs in `validate.yml` and `nightly.yml`), and hardens `cli/internal/archcheck`'s fixture git helpers against a `t.TempDir` cleanup race by disabling background git maintenance (`gc.auto=0`, `gc.autodetach=false`, `maintenance.auto=false`). ## Why Fixes #1055. Fixes #1077. Two independent failures were blocking CI: - **go-gate-shadow red on the open Renovate PRs (#1076, #1074).** golangci-lint v2.11.4's vendored `x/tools` cannot decode Go 1.27's export data (`export data version 4 is greater than maximum supported version 2`), so the `go.lint` gate fails with `typecheck` errors on any PR that moves CI or the `toolchain` directive to go 1.27.0. v2.13.1 ships `x/tools` v0.49.0 with Go 1.27 support while keeping a `go 1.26.0` directive, so it still bootstraps under the current go 1.26.6 CI toolchain with `GOTOOLCHAIN=local`. - **Nightly failures #1055 (CLI tests, 2026-08-10) and #1077 (security toolchain go-test lane, 2026-08-21).** Both are the same root cause: `git commit` in `TestGoCLIArchitectureAcceptedBoundaryOwnsModuleIntroduction`'s fixture repo can detach an auto-maintenance child that outlives the test body and races `t.TempDir` cleanup, failing `RemoveAll` with `.git: directory not empty`. The fixture git helpers now disable auto gc/maintenance so no background child is ever spawned. ## How I tested - `cd cli && go build ./... && go vet ./... && go test ./...` — exit 0 (65 packages ok). - `scripts/check-go-lint.sh` with golangci-lint v2.13.1: clean (0 findings) under **both** `GOTOOLCHAIN=go1.26.6` (current CI) and `GOTOOLCHAIN=go1.27.0` (the environment of the Renovate PRs), on the final tree. - Reproduced the v2.11.4 failure locally under go1.27.0 (same two `typecheck` findings as CI) before the bump. - `go test ./internal/archcheck/ -run 'TestGoCLIArchitecture...' -count=3` — pass. ## Checklist - [x] `make build && make test` passes (if Go changes) - [x] No secrets or credentials in code - [x] Breaking changes documented (if any) — none --- _Generated by [Claude Code](https://claude.ai/code/session_011Kjsu6s8w9su671g6zCSQw)_ Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Bo
|
c516dd55a9 |
fix(skills): preserve portable Codex activation metadata (#1069)
## What changed - preserve canonical trigger text while compacting generated Codex descriptions - harden portable skill validation for duplicate YAML keys, symlink containment, and current optional-field rules - regenerate all owned Codex projections and hashes - add focused negative and generator regression coverage ## Verification - portable conformance: 52/52 - portable Bats: 8/8 - generator acceptance: 18/18 - `scripts/regen-all.sh --check` - Bash syntax, blocking ShellCheck, and `git diff --check` The interrupted all-skills remediation snapshot is deliberately excluded: it remains preserved on `codex/all-skills-pass-20260816` and is not merge-safe. |
||
|
Bo
|
f3c6d0ecf2 |
Converge retained WIP and harden evidence boundaries (#1065)
Summary:
- lands the audited current WIP lanes and excludes stale/process-only
material
- hardens prune path confinement, probe-v3 evidence binding,
codebase-recon identity, handoff/release/reverse-engineer behavior, and
Codex prompt handling
- truth-labels static skill scoring and regenerates all owning
projections
Validation:
- fresh independent PASS on commit
|
||
|
Bo
|
8694b07ebf |
feat(skills): add anti-ceremony guard to RPI (#1064)
## Summary\n\n- add a clean-room, artifact-free anti-ceremony skill\n-
make RPI invoke its STOP/CONTINUE guard exactly once before Plan\n- STOP
before dispatching any core phase; CONTINUE preserves Plan → Implement →
fresh Validate\n- retain durable intent by reference and snapshot only
non-durable intent\n- update the canonical traversal, workflow,
standards, dependency contracts, tests, and generated projections\n\n##
Verification\n\n- RPI unit tests: 13/13\n- dependency/product-boundary
Bats: 15/15\n- RPI and Anti-Ceremony validators: PASS\n- Cathedral Cut
conformance: PASS\n- mesh, Codex/Gemini projections, parity, hashes,
manifests, and runtime formats: PASS\n- changed-scope
generation/contract checks: PASS\n- full exact-range gate: 68 pass, 0
warn, 0 fail, 0 unknown, 0 skip\n- fresh final semantic review on head
|
||
|
Bo
|
62cc3b6ee0 |
Clear the operations-layer alignment residuals (#1054)
Closes out the six residual items #1051 disclosed: terminology residue on non-authority surfaces, the eval command-surface fixture that failed when executed (#{3,4} -> #{3,5}), the vacuous retrieval-quality canary and the nightly job that ran it, the consumer-free dream config block and its exclusive helpers, the remaining knowledge-shaped writers moved to the scratch tier, and the MEMORY.md consumer audit. bin/ralph still resumes legacy .agents/ralph/ checkpoints so the documented backwards-compat contract holds without a migration; both paths and the outside-both refusal are now tested. Fresh author-distinct validation returned PASS with empty not_checked, after an earlier revision failed on a dangling nightly invoker and a back-compat test regression that were fixed and independently re-verified. Test-Removal-Reason: the dream config subsystem was deleted with its tests (operations-layer residuals) |
||
|
Bo
|
7a765cde19 |
Align AgentOps around its operations-layer identity (#1051)
Executes docs/plans/2026-08-07-agentops-operations-layer-alignment.md: AgentOps is the operations layer for agentic engineering; the federated integration graph is the topology, the semantic work-and-proof protocol is the contract, and RPI is the standard one-experiment traversal. Retires the ao flywheel command family and all knowledge-flywheel product state, tombstones the seven-move operating-loop workflow, narrows ao init and the .agents state writers to declared destinations, renames the core architecture page to rpi-traversal.md with a compatibility redirect, aligns AGENTS.md, 25 skills, public and package copy, regenerates every owned projection, and strengthens the conformance gates with planted-negative proofs. Both the alignment subject and the follow-up gate-bookkeeping commit carry fresh author-distinct validation PASS verdicts with empty not_checked scope. Test-Removal-Reason: dead knowledge-flywheel and session-store surfaces were deleted with their tests (operations-layer alignment) |
||
|
Bo
|
51f5d64474 |
feat(evals): eval architecture + probe wave 1 — first BEHAVIORAL skill verdicts (#1033)
## What this is The eval program for the skill harness, end to end: SOTA research (two adversarially-verified deep-research passes) → a 12-decision eval architecture → harness completion → the first measured probe wave → skill improvements driven by the measurements. ## Wave-1 results (gpt-5.6-luna, xhigh + low effort, N=2/arm/config, directional) | Probe | Skill | xhigh C→T | low C→T | Verdict | |---|---|---|---|---| | premortem-self-validation | premortem | 0.5→1.0 | 0.0→1.0 | **BEHAVIORAL** | | standards-go-conventions | standards | 0.5→1.0 | 0.0→1.0 | **BEHAVIORAL** | | validate-not-proven | validate | 1.0→1.0 | 1.0→1.0 | INERT (ceiling) | | security-coverage-gap | security | 1.0→1.0 | 1.0→1.0 | INERT (ceiling) | | reality-check-gap | reality-check | 1.0→1.0 | 1.0→1.0 | INERT (ceiling) | | crank-luna | crank | 1.0→1.0 | 1.0→1.0 | INERT (3rd config) | The effect **grows as the producer weakens** — the SkillsBench gradient reproduced locally. Full report: `docs/evals/2026-08-04-probe-wave-1.md`. ## Verification (all in this branch's tree) - Go build + vet + 415 tests green (adapters/eval, eval, gates) - validate-skill-frontmatter 51/51; heal --check clean - shellcheck -S warning clean on all new/modified shell - probe-coverage gate: 4/11 measured (was 0/11); bats suite green - `_stats` vendored: 42 pytest green - Every wave cell replayable: `bash scripts/probe-skill.sh --probe <id> --replay` ## Notes for review - `evals/skill-probes/LEDGER.md` is the new hand-maintained MEASURED ledger — moved out of generated SKILL-TIERS.md because a regen wiped it there (measured results can't live in generated files). - skills/premortem + skills/standards gained front-loaded MEASURED blocks (the exact prelude content the probes proved behavioral); codex twins regenerated on this clean baseline. - The architecture doc is PROPOSED — ratification is the repo owner's call; nothing in it is load-bearing for this PR's code. |
||
|
Bo
|
83e1dd0cb5 |
feat(gc): pre-seed Codex trust for materialized Gas City homes (#1031)
## Defect
Gas City materializes Codex session homes with project-local hooks. The
first
Codex process in an untrusted home can stop at the interactive
workspace/hook
trust dialog, leaving the agent pane alive but unable to take dispatched
work.
Codex persists two independent decisions in `$CODEX_HOME/config.toml`:
1. workspace trust under `[projects."<dir>"]`
2. one content hash per hook under `[hooks.state."<hook-key>"]`
Trusting a parent directory does not trust a session home, and the hook
digest
input is intentionally owned by Codex rather than reimplemented here.
## Change
`ao gc prepare` now discovers the Gas City directories that exist when
it runs
(city and rig roots, materialized agent homes, and materialized rig
worktrees)
and pre-seeds both trust layers for those exact targets.
- Hook identities and current hashes come from Codex's `hooks/list`
app-server
method.
- Returned hooks are restricted to the discovered targets; user- or
plugin-level hooks are never granted trust by this command.
- Explicit operator decisions are preserved. An untrusted workspace,
modified
hook, disabled hook, unusable hash, malformed response, or malformed
TOML
fails loudly rather than being rewritten or accepted as complete.
- The merged TOML is validated in memory and installed with the CLI's
durable
atomic writer while preserving existing permissions.
- `ao gc check` verifies the same values from local files only. It
starts no
Codex subprocess and writes nothing.
## Deliberate boundary
Discovery is filesystem-based. A home Gas City creates *after* `prepare`
is not
pre-seeded by an earlier invocation. `prepare` compares configured agent
identities with materialized homes and warns about missing homes,
including the
real dotted-name shape (`gastown.mayor` → `.gc/agents/mayor`). The
operational
rule is documented explicitly:
```text
prepare → start the city → prepare again → dispatch
```
This PR does not claim that one pre-start invocation covers future homes
or
that every future pane can never encounter a prompt.
## Evidence
Automated tests cover:
- value-based workspace and hook trust, including `enabled = false`
- malformed/unexpected `hooks/list` responses
- regular local `hooks.json` files that derive zero hook identities
(`{}`,
`{"hooks":null}`, and `{"hooks":{}}`), keeping `prepare` and `check`
aligned
- real TOML spellings, invalid merges, idempotence, and mode
preservation
- target filtering and derived hook-key fidelity
- subprocess-free `check`
- missing-home identity reporting for nested and dotted qualified names
- operation with no Codex binary
- package-wide HOME isolation
An isolated real-Codex smoke on a disposable Gas City home established
the
behavioral differential: with the home's trust entries removed, Codex
rendered
the trust dialog; after seeding the same home, it reached the composer
without
the prompt. This proves the existing-home mechanism, not future-home
timing.
Final recovery checks on commit
`a4b52b2354b9f96e5e10e07b2916339c87190bfc`:
```text
go test -count=1 ./internal/gcmaintainer
ok github.com/boshu2/agentops/cli/internal/gcmaintainer 11.943s
go test -race -shuffle=on -count=2 ./internal/gcmaintainer
PASS
go test -count=1 ./internal/testsupport
PASS
go vet ./internal/gcmaintainer ./internal/testsupport
PASS
scripts/check-test-home-isolation.sh
PASS
scripts/check-test-isolation.sh
PASS (raw os.Setenv remains at the 10/10 baseline)
GOCACHE=/private/tmp/agentops-gocache \
GOLANGCI_LINT_CACHE=/private/tmp/agentops-golangci-cache \
WORKTREE_DISPOSITION_CI_SKIP=1 \
./bin/ao gate check --full --workflow-coverage --require-workflow-parity
PASS (68/68 full/head checks)
GOCACHE=/private/tmp/agentops-gocache bash scripts/regen-all.sh --check
PASS
git diff --check
PASS
```
Recovery fixed the prior CI findings with `storage.AtomicWriteFile`,
package-wide
HOME isolation, and `json.Encoder.Encode`. The first repaired CI replay
exposed
one further ratchet: raw `os.Setenv` calls in the new `_test.go`
TestMain raised
the repository baseline from 10 to 12. The final commit moves that
one-time
setup into the existing shared test-support boundary, keeps environment
changes
outside `m.Run`, and teaches the HOME-isolation gate only the exact safe
helper
shape. CI will rerun on the exact pushed commit.
|
||
|
Bo
|
efcf4879c8 |
feat(gc): port gc-maintainer-ops into the ao gc command family (#1016)
## What Ports `scripts/gc-maintainer-ops.sh` (425 lines of bash: prepare / check / recover-affinity for stock Gas City rigs) into the Go CLI as **`ao gc prepare|check|recover-affinity`**, per ADR-0016 (skill logic ships in Go via `ao`; shell stays thin glue). **Why:** skills ship via plugin/npx as SKILL.md only — a user without a repo checkout could not run the commands the shipped `using-gc` skill teaches. The skill said "From an AgentOps checkout", which was disclosed but weak. ## Changes - **`cli/internal/gcmaintainer`** — full port: rig/import pin verification, bundled pack-cache resolution, PyYAML-capable python selection, atomic runtime staging, managed check wrappers, skill links into city/rig Codex sinks, macOS LaunchAgent + doctor/status health checks, bounded affinity recovery. Output and refusal-message parity with the shell script (incl. refuse-before-mutation ordering). - **`cli/internal/commands/gc` + `cmd/ao/gc_composition.go`** — cobra module on the shared `clicontract.HostOptions` seam; global `--dry-run` always overrides `--apply`. - **Skills source resolution without a checkout**: `--skills-source` > enclosing agentops checkout > installed skills root (`~/.agents/skills`, `~/.claude/skills`). Existing rigs stay recognized: the `managed-by: agentops gc-maintainer-ops` wrapper marker is unchanged. - **Tests migrated**: `tests/python/test_gc_maintainer_ops.py` (7 cases) → Go L2 tests in `cli/internal/gcmaintainer` with the same fake-`gc` harness, plus module wiring tests. `scripts/check-gc-executor.sh` no longer runs the python suite. - **`scripts/gc-maintainer-ops.sh`** reduced to a thin wrapper exec'ing `ao gc`, pinning `--skills-source` to its checkout to preserve historical semantics (`--ao-bin` now selects the ao binary). - **Docs/projections**: `skills/using-gc/SKILL.md` now teaches `ao gc ...`; codex, gemini, and executor-pack projections regenerated via their owning generators; spine/COMMANDS.md/surface artifacts regenerated. ## Verification - `go build ./... && go vet ./... && go test ./...` — 2923 passed, 73 packages - `golangci-lint run` on new/touched packages — clean - `shellcheck -S warning` on wrapper + gate script — clean - `bash scripts/check-gc-executor.sh` — OK - Smoke: built `ao`, ran wrapper → `ao gc` delegation end-to-end |
||
|
Bo
|
e2e2e60aa2 |
chore(gc): decay retired Gas City prototype wiring to frozen bytes
The 2026-07-29 upstream-factories pivot retired packs/agentops-executor, packs/agentops-factory, and deploy/gc in place as migration evidence, but three live wires kept regenerating and gating them: - scripts/sync-gc-pack.py re-projected canonical skills into the retired executor pack on every regen. Remove the regen-all step and delete the projector + its test; the pack copy and manifest stay as frozen historical bytes (untouched by this commit). - adapter.gc-executor spanned packs/** and deploy/gc/**. Replace it with adapter.gc-maintainer, narrowed to the live surface only: scripts/gc-maintainer-ops.sh + tests/python/test_gc_maintainer_ops.py, backed by new scripts/check-gc-maintainer-ops.sh. - tests/python/test_gc33_thin_pack.py coupled the LIVE skills/using-gc skill to retired pack bytes (asserted matching strings in both), so any future skill edit would break against the frozen copy. Deleted with the gate that ran it. The new gate ships with a red bats case naming its backing script, so the adapter.gc-executor grandfather line is pruned rather than replaced — the negative-witness ratchet shrinks by one. Verified: go build/vet + gates suite (174 pass, includes the witness closure test), new bats twin 3/3, regen-all --check green, shellcheck + preamble ratchet green. workflow.install-drift fails pre-existing (dangling ~/.claude/workflows/bdd-foundry.js estate symlink, unrelated). |
||
|
Bo
|
1f8468429b | fix Gas City maintainer operations | ||
|
Bo
|
9dd6e7d3f9 |
3.4.0 release prep: upstream-factories pivot, version bumps, release notes (#1013)
## Summary Everything-but-the-tag for v3.4.0, in four commits: - **docs(gc)**: the factory pivot — README and `using-gc` present the upstream [Gas City build pack](https://github.com/gastownhall/gascity-packs/tree/main/gascity) and [Agentic Coding Flywheel](https://agent-flywheel.com/) as the supported factory choices; the in-repo prototype (`deploy/gc/`) is retired in place. AgentOps' lane is the skills + evidence discipline either factory executes. - **chore(release)**: version 3.3.0 → 3.4.0 across all six surfaces (claude/codex/gemini plugin manifests, marketplace, image verify pin, `ao` source fallback). - **fix(gates)**: `check-orchestration-skill-boundaries.sh` exited 2 on every run — it probed adapter files deleted by the 3.3 single-pass refactor and three contract phrases removed by the skill-overhaul waves. The live ratchets (retired-skill absence, ATM-era naming) are kept. - **docs(release)**: 3.4.0 CHANGELOG section (root + docs mirror) and curated release notes; `validate-release-notes.sh` passes (tier minor, full area coverage). ## Verification - Full Go gate in a clean worktree: build ✓ vet ✓ test **2902 passed / 0 failed** (71 packages) - `scripts/regen-all.sh --check`: all 11 projection/doc checks ✓ (including the doc-release freeze gate) - `scripts/validate-release-notes.sh v3.4.0 --since v3.3.0`: PASS - `scripts/check-orchestration-skill-boundaries.sh`: exit 0 (was exit 2 on main) ## Notes - The earlier read that `go.cli-reference` needed unpinning from the negative-witness grandfather list was a **false positive**: gitignored session logs under `tests/claude-code/logs/` pollute the witness scan in a dirty checkout. On a clean tree the pin is correct; a follow-up task exists to make the scanner read only tracked files. - Tagging + Release Publisher run happen after merge, separately; an official-mode readiness artifact gets produced at the merged SHA **before** any tag (binding rule from the v3.3.0 record). |
||
|
Bo
|
a6359795bf |
Make fresh validation persistence optional (#1012)
Keep fresh author-distinct validation mandatory while making verdict and report persistence consumer-driven. Align the RPI/Validate contracts, executable behavior, current guidance, regression coverage, and generated projections; preserve the Gas City 1.4 cutover. |
||
|
Bo
|
a0d9c3b8e1 |
W3b evidence wave: security suite repaired, write confinement, honest contracts across seven skills (#1002)
## Summary Wave W3b of the skill-overhaul reboot (`age-skill-overhaul-reboot-sjv7v.4`, evidence half) — ~40 verified fixes across seven evidence skills. Every checklist item verified against the live tree; heavyweights reproduced before fixing. - **security** — deleted the unrunnable skill-local gate duplicate (exit-1 reproduced); redteam pack refreshed against current docs — the `context-overexposure` case was **re-pointed at the surviving control** (its doc was deleted by `482307762`; the trust rule lives in AGENTS.md "Authority and trust"), pack 6/6; `tests/scripts/test-security-suite-redteam.sh` was red on `main`, now 8/8; validate.sh is behavioral (runs the redteam fail-closed); release/merge authority stripped; all three output artifacts declared per command surface. - **reverse-engineer** — writes confined under `output_dir`; FileNotFoundError canned-learning path deleted; `_TBD` gate scans the full seven-file audit bundle fail-closed (negative self-test proves it); ZIP extraction bounded; behavioral validator. - **cass** — three real effects declared; TMPDIR shadow + temp-dir leak fixed; always-failing `jq -se` extractor fixed; no-artifact-dir contract now true (retention path removed). - **codebase-recon** — resolves against the target repo root (`--repo-root` + `pwd -P`); claim evidence must be a file. - **standards** — false-`effects:[]` template annotated at its origin; dead anchor + missing owner rows fixed; bidirectional owner-table validator (matches table rows only, not prose). - **domain** — new liveness-proven validator (cited contracts must exist, term must resolve). - **research** — `write_research_report` declared; 3/3 scenario coverage. Ledger (~40 fixed / 2 rejected-stale / ~13 deferred-with-reason): `docs/audits/2026-07-28-skill-overhaul-reboot/wave-reports/w3b.md`. ## Validation - Redteam 6/6; security suite 8/8 (previously red on main); reverse-engineer self_test green end-to-end; per-skill validators PASS - 49/49 frontmatter; regen-check clean; liveness+anti-spiral bats green; python ratchet no-growth (all Python edits in grandfathered files); shellcheck clean - Cross-family review, two rounds (ceiling): round 1 six findings all fixed (incl. re-pointing the silenced redteam case with commit-traced lineage, `pwd -P` across validators, full-bundle `_TBD` scan); round 2 residual on the owner-row prose-match fixed with a witnessed table-row check; the remaining round-2 note (the redteam case covers the trust half of the old control; the loading-bounds half is metadata-enforced via per-skill `context:` blocks) is recorded in the ledger. Tracker: `age-skill-overhaul-reboot-sjv7v.4` (W3a merged in #1000; this completes W3) |
||
|
Bo
|
f66ab953af |
test(gates): require every blocking gate to prove it can fail (#996)
P3 from the 2026-07-25 review: fix the inert-check CLASS, not just the two instances #995 patched. A gate verified only against a green tree is indistinguishable from `exit 0` -- delete its detector and CI stays green. Measured across the registry: 32 of 54 BLOCKING script-backed gates had no test anywhere under tests/ demonstrating they FAIL on what they detect. 17 had no test at all, including skill.schema and skill.triggers, which gate every SKILL.md in the corpus. The closure lives in Go because the registry does. An earlier bash/Python projection of seed.go undercounted by four -- some gates register outside that file. The authority is gates.Default.All(), and reading it any other way reproduces exactly the projection drift ADR-0016 is named after. Mechanism is the repo's proven shrink-only ratchet: current gaps pinned in scripts/.gate-negative-witness-grandfather; a NEW blocking gate must ship with a witness; a gate that gains one must be pruned; a gate that leaves the registry must be pruned; the growth guard reads HEAD so a change cannot exempt its own new gate. The surviving count prints on every run. Ratcheted down immediately to prove the mechanism moves rather than just counts: tests/scripts/validate-skill-schema.bats is a real witness for skill.schema (32 -> 31), driving the actual validator against a fixture tree. Two things worth recording: - The first draft's negatives passed for the WRONG reason. Descriptions in this corpus embed `Triggers: "x"`; unquoted, that colon makes YAML read a nested mapping, so the gate failed on a PARSE error rather than the schema violation each test claimed to witness. Caught only because the positive case failed too. A negative that fires for the wrong reason is not a witness. - go.test-isolation is ON the allowlist as lacking a proven negative, and it caught a real defect in this very PR (a git exec without per-command env scrubbing). The pin measures TEST COVERAGE of a gate, not its efficacy. An unproven gate may work fine; the point is that nothing would tell us if it stopped. All four of the meta-test's own failure directions were seeded and observed, so the check-liveness ratchet is not itself an unproven check. |
||
|
Bo
|
aceeb6f10f |
feat(gates): enforce ADR-0016 shipped-Python rule; fix RPI/Validate digest disagreement (#995)
Two fixes from the 2026-07-25 skill-overhaul progress review, both independent of the unlanded tranche fleet. 1. skill.python-ratchet — the gate ADR-0016 said it already had. Section 3 fixed "Python never ships in skills" on 2026-07-18 and called its own violation "a gate failure, not a style nit", while zero checks enforced it. Now a blocking shrink-only ratchet over skills/*/scripts/**/*.py: the 24 current files are pinned, new ones hard-fail, a promoted file must be pruned, and the growth guard rejects a change that allowlists itself. The surviving count prints on every run. ADR-0016 records the enforcing check by name and the skills/*/tests/** carve-out with its rationale, so the exemption cannot decay into the same inert prose. 2. The RPI/Validate acceptance-digest disagreement. run_once.py digested a canonical-JSON re-serialization of the parsed intent mapping, validate.py digested the raw intent bytes, and run_once.py hard-compared the two, so the composed contract could not succeed. Both suites were green because RPI's test mocked Validate with RPI's own digest function. RPI stops being a second digest authority: it carries the digest Plan declares over the bytes Plan snapshotted and cross-checks Validate's independently re-derived value. 3. The cathedral-cut probe, the same blind spot found again. It set intent_bytes = canonical_bytes(resolved_intent) — precisely the one input where the two digests coincide — so like the unit test it was built around the coincidence and could not observe the bug. Test shape is liveness-first: every gate negative is a seeded witness the gate must be shown to FAIL on (12/12 bats), and the digest fix landed against a RED witness that reproduces the real production error. |
||
|
Bo
|
cbe7ddbb5f |
feat(gc): ship thin native AgentOps factory pack (#980)
## Outcome Ships AgentOps 3.3 as a thin native Gas City pack and removes the accidental second orchestration control plane. - Removes the custom GC delivery command/package, packet/schema family, feeder/program engine, reducer/order, reliability registry, Beads capability mirror, and fork-baseline runtime checks (about 29k deleted lines). - Retains six bounded adapters for pinned toolchain materialization, clean bootstrap, native sling/status/doctor invocation, bead-isolated worktrees, moving-main PR delivery, and teardown. - Uses official Gas City and Beads behavior as the state owners, with required OTEL configuration. - Preserves Fable Mayor/Refiner, Sol-high planning and fresh validation, Terra-high default implementation, Opus-medium overflow, and support-only Luna. - Supports automatic Refiner merge after hosted CI or a manual-review toggle without locking `main`. ## Evidence - Exact official GC: `8ffc009ded781a2ada2077f3a29bd712b2def0bf` - Exact official BD: `8e4e59d39f3459a43cf21a3236a13eca4dd874f7` - Full pinned native boundary: 9/9 passed in 114.047s - Exact source-bead route replay: passed in 98.051s - Release replay: manifest integrity, thin tests and both pack lints, generated projections, contract compatibility, Bats contract, ShellCheck, test-removal ratchet, and `go test ./...` all passed - Fresh independent semantic verdict: PASS (Sol-high) - Exact delivered head: `e805f0e26ce21c2eae9e720f57fff2a705f3185b` - Subject manifest: `8554d065076f70d0e639ca353409d2a75dca7ac01e4cb65cb4a118e98c0770fc` - Verdict: `3f89376b675de854832f38b46ccfa7ad7ca54079ffde578769c33e1f118c68c1` ## Release boundary After this PR merges, the release qualification runs one fresh mixed Terra/Opus canary from merged `main`, with at most one external repair and one terminal retry. No live city repairs itself. The only known local fast-gate exception is `workflow.install-drift`: the installed workflow symlink resolves to the separate dirty primary checkout, outside this candidate. All candidate-owned gates passed. |
||
|
Bo
|
c0a68c29ea |
fix(gc): bind managed canary invocation (#973)
## Outcome Correct the one AgentOps-owned defect exposed by the first post-merge 3.3 canary: the operator README placed a literal argument boundary inside the discovered Gas City command, so Python never received source-bead. This adds a managed invocation boundary that: - consumes its own explicit wrapper separator and forwards discovered-command flags directly; - binds the schema-5 city marker, exact GC digest, private supervisor home, and isolated mode; - projects the marker-selected GC/BD telemetry endpoints or an explicit disabled state; - clears ambient generic OTLP fallback configuration; - records the stopped pre-admission canary without claiming a Gas City or Beads defect. ## Evidence - focused managed-invocation Bats: 4/4 PASS - combined Gas City contract gate: PASS (33 Bats plus Python/static contracts) - adjacent teardown/toolchain Bats: 16/16 PASS - exact full local release gate: PASS at .agents/releases/local-ci/20260722T141141Z - two fresh exact-toolchain bootstrap/start/status/teardown cycles: PASS, zero workers/sessions, clean teardown - fresh Sol-high pre-delivery binding: PASS, 11/11 criteria ## Deferred boundary This PR does not claim final 3.3 release readiness. After protected merge and post-merge conformance, exactly one new required-telemetry mixed Terra-high/Opus-medium live canary remains. |
||
|
Bo
|
faff55ac2a |
feat(gc): bind native 3.3 factory workflows (#972)
## Summary
- bind the GC 3.3 bead-native workflow: Fable Mayor, fresh Sol-high plan
and validation, Terra-high default or Opus-medium overflow
implementation, Luna support-only
- add the bounded one-shot graph feeder, strict packet/worktree
identity, semantic terminalization, and model-free protected delivery
sweep
- make official Gas City v1.3.5 plus Beads v1.1.0 bootstrap repeatable,
disable unstable event propulsion/hooks, and prove quiescent teardown
## Reliability boundaries
- no Gas City self-repair loop, daemon, private scheduler, model
fallback, or main mutex
- semantic completion is independent of delivery; protected
PR/CI/rebase/merge is deterministic and moving-main aware
- event hooks are disabled because 3.3 uses explicit program admission
and cooldown delivery; native controller bead observation remains active
## Evidence
- full local release CI PASS at
.agents/releases/local-ci/20260722T113355Z
- two pristine standalone-clone cycles with official GC v1.3.5 8ffc009d
and BD v1.1.0 8e4e59d: repeat bootstrap, exact rig Dolt context, absent
event hooks, bounded events, enabled sealed delivery, and delayed
zero-process teardown
- fresh independent Sol-high pre-delivery binding PASS over exact head
|
||
|
Bo
|
4d631207cf |
feat(gc): add bead-native crash-only delivery kernel (#970)
## Summary - replace the unreachable Python factory lifecycle with an optional typed Go delivery reducer - make Beads the delivery lifecycle authority with deterministic moving-main successor epochs - add native Git/worktree, PR, hosted-check, auto-merge, landing, and cold-replay boundaries - retire obsolete v1 role/factory schemas and preserve the pinned historical capability harness ## Validation - fresh Sol-high PASS over exact 43-path manifest ec6a2a4bf0b180782411b1ae4afd4a2ad8384129ff037ce8c119480317626b02 - go test ./... - go test -race ./internal/gcadapter/delivery - go vet ./... - scripts/check-gc-executor.sh - GC 3.3 schema, migration, provenance, factory-doctor, and bootstrap gates Bead: age-gc-scope-failclosed-release-gktia.1 |
||
|
Bo
|
247fcccb5f |
GC 3.3: define the one-loop factory role pack (#961)
## Outcome Defines the GC 3.3 factory as one semantic intelligence loop with deterministic delivery: - Fable Mayor plus a bounded, nonbinding ambiguity adviser - Sol-high for plan binding and fresh exact-subject validation - Terra-high as the default writer and Opus 4.8 medium as explicit overflow/repair - Luna dormant and unroutable until later qualification - no model-driven delivery-policy/refinery loop - native GC sling/claim/scale-from-zero semantics retained - exact provider/model/reasoning/fallback runtime attestation - legacy composed program/refinery entry points and opposite-provider lifecycle roles removed Controller dispatch to the Fable adviser stays fail-closed until GC33-4 proves process/filesystem isolation. This does not claim that manual operator sling to the named on-demand target is impossible. ## Evidence Fresh Sol-high `verdict.v2`: PASS (`bb31806fc5e0714363a9e539121ee7aa19231431fb04ddfee1fce8376b57d272`) over subject manifest `119b7a6db2380af47d87a24aced60d96858dd9e413a54dc640376e27767e8d1e` (52 paths, 18 deletions). - focused acceptance: 124 tests PASS - projection owner: 4 tests PASS - bootstrap fixtures: 25 Bats tests PASS - complete GC static contract: PASS - factory/executor doctors: PASS - generated projection check: PASS - exact official GC v1.3.5 and Beads v1.1.0 provenance replay: PASS - `git diff --check`: PASS No city, model session, supervisor, tmux session, Dolt server, or canary was started for this pre-canary bead. |
||
|
Bo
|
56190722e6 |
fix(gates): atomic-write ratchet fails closed on scan-helper death — no silent PASS (#946)
## Root cause (CI flake, run 29785505667 attempt 1, PR #944) The bats test `adding only the temp-signal half to a file with an EXISTING rename still trips` failed once on ubuntu with the ratchet printing a clean `PASS … grandfathered=0` where the fixture deterministically trips. Diagnosed by analysis per the test-isolation discipline (not repro-chasing): - bats errexit proves every fixture `git commit` succeeded; the fixture repo is fully isolated. - `grandfathered=0` in the PASS line fingerprints the run as in-fixture (real repo = 4). - Zero stderr in the captured output rules out every loud failure handler. What remains: **three silent paths where a scan helper (grep/awk/git) dying under 4-way-parallel runner load reads as "no finding"** — the gate then certifies an unchecked diff: 1. `file_trips`: strip/grep failure returned 1 ("does not trip") → file silently skipped. 2. Detector loop read from `< <(ratchet_changed_files | sort)` — a collector rc 2 inside process substitution is discarded, certifying an EMPTY change set (the exact fail-open the ratchet-lib header warns about). 3. `ratchet_added_hunk_matches`: helper death propagated as arbitrary nonzero, swallowed by `|| continue`. ## Fix - `file_trips` is tri-state (0 trips / 1 no / 2 refuse-to-certify, loud); signal greps read full input (no `-q` early-exit) so pipefail can never surface a SIGPIPE'd printf as a phantom failure on large files. - Changed files are collected into a variable under pipefail with an explicit `exit 2` handler. - `ratchet_added_hunk_matches` maps fallback-grep/awk death to loud rc 2 (0/1 unchanged; existing `|| continue` consumers skip on 2 exactly as before, now loudly). - `compute_grandfather_set` aborts `--regenerate` on helper failure — never a truncated grandfather list. ## Pinned Dying-awk PATH-shim tests at gate and lib level: helper death is rc 2 "refusing to certify", never PASS. Same fail-closed class as the pawl-refuted empty-stream certify (2026-07-10). ## Validation - 58/58 in `check-atomic-write-ratchet.bats` + `ratchet-lib.bats` on fresh origin/main - All six ratchet-lib consumer gate suites green - `shellcheck -S warning` clean; real-repo gate PASS (grandfathered=4); `--regenerate` byte-identical - 65 repetitions of the formerly-flaky test green Residual: the external kill source is unprovable from one occurrence; a recurrence now exits 2 naming the failed helper + file instead of printing PASS. |
||
|
Bo
|
238964fdd2 |
feat(cli): workflows are canonical product artifacts — workflows/ + ao workflows link (#945)
Workflows get the skills treatment (operator decision): canonical source in the product tree, installed by a product verb, Claude-only labeled as such. **What moves:** all seven Claude workflow scripts + README migrate from force-added exceptions inside the gitignored `.claude/` to a tracked top-level `workflows/` (sibling of `skills/`) — the four existing conveyors plus `audit-dimensions`, `verify-fixes`, `implement-wave`: three thin, args-parameterized orchestration conveyors extracted from this session's hand-rolled waves, contract-reviewed, and smoke-proven through the real Workflow runtime (the smoke caught two contract gaps static review could not: an `export default` wrapper the runtime never invokes, and args arriving as a JSON string — both fixed, string-args tolerance now built in). **New verb:** `ao workflows link` / `unlink` mirror `ao skills link` semantics — dry-run `--json`, refuse to replace real files or foreign links, unlink only checkout-owned links — targeting the project-local `.claude/workflows/` where Claude Code resolves named workflows (`--into` overrides). Checkout identity reuses the skillsapp marker discipline, fail-closed. Claude-only runtime adapter, same doctrine as the Codex-only `skills-codex/`. **Legacy surfaces repointed:** `install-workflows.sh` (user-global $HOME installer), `check-workflow-drift.sh` + gate comment, `check-bdd-foundry-markers.sh`; spine allowlist + YAML-probe excuse + go-cli.md spine region gain the workflows group; COMMANDS.md, cli-surface projections, and surface-count fixture regenerated; new tests carry per-command git-env scrubbing (test-isolation ratchet back at baseline). **Built BY the workflow being canonized** — `implement-wave` orchestrated its own canonization: two disjoint-ownership lanes plus a seam-checking verifier that ran the real binary's link → resolve → unlink cycle in the live tree (both lanes RESOLVED). The lanes correctly *refused* to self-approve their command into the spine invariants and handed integration three flagged edits instead. **Expected local gate note:** `workflow.install-drift` correctly FAILS on machines whose user-global `~/.claude/workflows` links still point at the old location — that is the transition it exists to catch. CI stays green (absent→skip). **Post-merge operator step:** `cd ~/dev/agentops && git pull && bash scripts/install-workflows.sh`. **Verified:** full suite 63/63 pkgs; golangci-lint clean; `gate check --full` over this range = 66/67 with only the documented install-drift environment finding; workflows smoke-run evidence in session logs. |
||
|
Bo
|
3d24ee0e9d |
fix(cli): residue wave — doctor dev-version coherence, diff --only, config-models removal, dual-root stragglers (#944)
Wave 4 (residue) of the new-user happy-path arc. Three scoped implementer lanes + fresh adversarial verifier (4 RESOLVED; 1 INCOMPLETE = stale generated projections, closed in integration). **Doctor**: the dev-version detector now reuses the Binary Freshness resolution — a from-source build matching its checkout is healthy (a novice building from source can finally see `ao doctor` exit 0); findings fire only on genuine drift, shadowed duplicate `ao` binaries, or an informational from-source note outside any checkout. `ao doctor diff` gains `--only` so the fix-plan preview can be scoped the way remediation text implies. **Config**: the dead `ao config models` surface is removed end-to-end (lane re-verified zero consumers before deleting; `--show` proven byte-identical before/after; removed-child hint + MIGRATION row; existing `models:` config sections still parse and are ignored). **Dual-root stragglers**: learning-coherence gate globs, `quality.CountConstraints`, and the eval sandbox corpus deny-list now cover canonical `.agents/ao/<section>` alongside legacy roots. **Doc-link hygiene**: the strict docs-link backstop's allowlist was 100% stale (53/53 entries referenced Cathedral-Cut-deleted docs) — refreshed to 10 verified accepted-class entries; ROADMAP dead links fixed; documentation-index generator emits GitHub URLs for repo-root targets; doc-skill references instruct only shipped scripts; codex twins + CLI-surface projections + surface-count fixture regenerated. **Deferred by design**: the `3.3.0-rc` fallback version bump belongs inside the v3.3.0 tag-cut commit. **Verified**: full suite 61/61 pkgs (2825 tests); golangci-lint clean; `ao gate check --full` 67/67 over this range; Test-Removal-Reason trailer covers the 6 deliberately deleted models tests. |
||
|
Bo
|
d043390a0a |
fix(release): resolve 3.3 release-wrapper audit — blocker + 13 majors (#935)
Resolves every spellbreaking finding (the blocker + all 13 majors) from the 3.3.0 release-readiness audit ([docs/audits/release-readiness-3.3-2026-07-20.md](docs/audits/release-readiness-3.3-2026-07-20.md), included in this PR). ## Finding → fix map **CLI self-documentation (M1–M3)** - **M1** `ao robot-docs` prescribed removed `ao inject` → line removed from the canonical agent workflow; `inject` added to the removed-command hint table **and** the MIGRATION.md map (drift test `TestRemovedVerbsHaveMigrationRows` enforces the pair). - **M2** `ao config --help` documented ~14 env vars for removed subsystems (RPI/Dream/Council/tiers) → help text and the `--show` env panel pruned to the 5 vars the binary consumes; mirrored list in `internal/config` pruned identically. - **M3** `ao flywheel status` read only legacy `.agents/<section>` while `ao doctor fix` migrates learnings to canonical `.agents/ao/learnings` → new `quality.KnowledgeSectionDirs` dual-roots every knowledge reader (tier counts, new/stale artifacts, retros, health delta, utility, loop metrics, retrievable-citation stats — plus the golden-signals readers `ComputeResearchClosure`/`ComputeReuseConcentration` that the fresh verification pass caught as missed). Sandbox-proven twice: a learning existing only under `.agents/ao/learnings` appears in all metrics, and a research file only under `.agents/ao/research` flips closure from `starved/0` to `unmined/1 orphan`. **Release story (M4–M6)** - **M4** CHANGELOG `[3.3.0]` omitted post-07-17 surfaces → folded in `ao eval` (#921), default-build `ao flywheel`, the PreToolUse policy engine, and the #919 cleanup; date moved to 2026-07-20; `docs/CHANGELOG.md` re-synced (changelog.sync gate green). - **M5** MIGRATION.md attributed `ao eval` to a nonexistent "3.4" → now "returned in 3.3". - **M6** four release surfaces claimed a 50-skill corpus vs 48 everywhere real → all counts now 48 (CHANGELOG ×2, docs/3.3.md, release-notes page ×2). **Install story (M7 — product decision by Bo)** npx first (universal — installs into all coding agents), **plugins for Claude Code/Codex encouraged**, checkout + `ao skills link` as the source-tracked/contributor path; curl installers stay tombstones. Harmonized across README, UPGRADING, install-day2-ops, MIGRATION, 3.3.md, CHANGELOG, the release-notes page, all six installer tombstone messages (`install.sh` + claude/codex/agy/opencode/`codex.ps1`), and the site's CLI page. All "legacy migration-only / not the recommended path" plugin branding removed. **Docs site (B1, M8–M12)** - **B1** generated site CLI page instructed a tombstoned curl installer, nonexistent `ao rpi phased`, and wrong skills dir → `emit_index()` rewritten to the real install menu + a quickstart of commands that exist; semantic loop correctly attributed to skills. - **M8** deploy workflow's `--strict` contradicted mkdocs.yml's declared non-strict policy and aborted the build → flag dropped (lychee + validate-links.sh own link checking). - **M9** site banner said "AgentOps 2.x" → now 3.3. - **M10** ~176 internal files (audits/plans/handoffs/… + TEMP scratch doc) published and dominated search → `exclude_docs` extended; built site verified free of them; search index 4,751 → 2,444 entries. - **M11/M12** newcomer-guide skill links and all six SCHEMAS.md links 404'd on the site → absolute GitHub URLs (resolve on both GitHub and the site). The fresh verification pass found the same class on `docs/contracts/index.md` (nav-listed), `docs/contracts/corpus-learning-seam.md`, `docs/templates/slice-validation.md`, and five `docs/architecture/gas-city-factory.md` links into now-excluded `docs/audits/` — all repointed to absolute GitHub URLs. - Also from the verification pass: robot-docs exit-code table no longer says "bead claimed" (removed concept), and the docs.yml comment now cites the link checker that actually runs (`tests/docs/validate-links.sh` via doc-release checks) instead of lychee. **Skills corpus (M13)** - rch skill instructed 5 nonexistent scripts + 8 nonexistent reference files as recovery steps → pruned to the 7 real references; the wire-level `printf | rch` probe replaces the phantom `protocol_test.sh`; codex twin regenerated (parity gates green). ## Verification - `go vet` clean; **full test suite 60/60 packages pass** (includes the new-shape flywheel/quality/config tests and the inject↔MIGRATION drift test). - **`ao gate check --full --scope worktree`: 67/67 pass, 0 warnings** over this exact change set (changelog sync, shellcheck on the edited installer, skill mesh + codex parity, manifests/schema/triggers, provenance chain). - `scripts/regen-all.sh --check`: all generated projections current. - Rebuilt binary re-exercised: `robot-docs` clean, `ao inject` tombstone live, config help clean, flywheel sandbox proof above. - `mkdocs build` exit 0; warnings 84 → 46 (remainder is the accepted out-of-tree-link class per mkdocs.yml's declared policy). - Fresh-context adversarial verification workflow over all four fix groups (results in session log). ## Known residuals (deliberately out of scope) - `ao config models` subcommand still renders tier config (its two env vars ARE consumed; `COUNCIL_CLAUDE_MODEL` in its display list is not — follow-up). - Same-class single-rooted readers off the flywheel path: `learning.coherence` gate glob (`.agents/learnings/**` only), `quality.CountConstraints`, config `Paths` defaults feeding eval sandbox deny-lists. - `scripts/docs-build.sh` still uses `--strict` with its own allowlist (not wired into any workflow or gate). - Audit minors 1–8 (rc fallback version string, `config --show` legacy-fallback display, `ao init` vs doctor layout, doctor's `ao beads dir` hint, dead `tracker:` key in the example config, doc-skill phantom scripts, ROADMAP dead links, documentation-index root links). |
||
|
Bo
|
a5f7be8364 |
Qualify deterministic Gas City factory for AgentOps (#924)
## Summary - pin and materialize an exact colocated Gas City and official Beads toolchain without editing installed binaries - add deterministic isolated bootstrap, recovery, teardown, exact workspace binding, and tracked .gitignore preservation - add bead-native Mayor, plan review, isolated Codex/Claude workers, binding Validators, rejection-to-Mayor rescope, fenced Refinery, integration validation, qualification, and protected PR delivery - make the local quick release gate deterministic after CLI cleanup ## Validation - fresh GC-managed Codex Sol verdict: PASS (verdict d1a1e03808b4605e49cc97423c7570b04d581c9ee1fad9aa8f1daf6e72d6f0ef) - focused GC executor/factory/projection suite: PASS (32 packet + 77 factory + 4 projection + 21 bootstrap tests, plus doctors and static contracts) - teardown suite: 7/7 PASS - local-ci-fast: PASS - exact v16 mixed-provider live canary: Terra and Opus 4.8 workers in parallel, opposite-family candidate validators, Opus Refiner, Sol integration validator, all lifecycle beads closed, clean managed teardown ## Scope The follow-on Fable 5 Mayor/Refiner, Sol-every-bead validation, and Luna support-worker topology is deliberately not part of this historical v16 baseline; it will build on this foundation in a separate bead program. |
||
|
Bo
|
843aa4e61a |
feat(cc-hooks): hooks ship by default across every install path (age-4qw1) (#932)
## Doctrine flip: enforcement hooks on by default
Operator directive: hooks reach **every** user regardless of install
path. Injection hooks stay dead (#511, delta=0) — the
hookless-cold-start gate is untouched and passing.
### Delivery per install path
| Path | Delivery |
|---|---|
| Claude Code plugin (marketplace) | **Automatic** — new
`hooks/hooks.json` at plugin root; Claude wires the policy dispatcher on
install (`${CLAUDE_PLUGIN_ROOT}` paths, 10s timeout, Bash + Edit\|Write)
|
| npx skills / skills.sh copies | Skill package now carries its own
installer: `skills/cc-hooks/scripts/install-hooks.sh` (resolves
everything relative to the skill dir; works with no repo present) |
| git clone / brew | `scripts/install-policy-dispatch.sh` → thin
delegator to the skill-embedded installer |
### Also in this PR
- 4th registry policy `core.skills:edit-installed-copy` (Edit|Write,
`file_path`-only) — registry twin of the standalone guard, so dispatcher
users get the full protection set.
- Dispatcher hardening: malformed stdin now fully silent (fail-open,
zero stderr) — validator security-pass finding.
- README admission-control section; cc-hooks SKILL.md posture flip
(description, constraints, delivery table).
- Codex twin force-synced (incl. the new installer file), skill mesh +
catalog regenerated.
### Evidence
- 28/28 policy-dispatch bats (incl. plugin-layout registry-fallback
test); 16/16 on the injection-doctrine + standalone-guard suites.
- `ao gate check`: 28 pass / 0 fail on the exact subject.
- Fresh-context validation: first round returned FAIL (uncommitted regen
outputs + a stderr leak on malformed stdin) — repaired, amended,
re-validation in flight; merge follows a PASS verdict.
- Adversarial security pass on the dispatcher (it auto-activates for
plugin users): empty/malformed/1MB stdin, invalid registry, concurrent
fires — all fail-open, silent, correct.
Epic: age-4qw1 (H1/H2 landed previously as
|
||
|
Bo
|
dc84c1efd2 |
docs(cli): allowlist ao flywheel compare coverage under the flywheel carve-out
flywheel compare direct-test coverage moved from cmd/ao into internal/flywheelapp; mirror the provenance/skills carve-out allowlist pattern and regen cli-surface. |
||
|
Bo
|
d74c334b0f | docs(cli): reclassify carved init leaf as allowlisted in surface parity | ||
|
Bo
|
0eaf21f0cb |
feat(cc-hooks): PreToolUse policy engine + day-1 enforce cohort (age-bhsz, age-wnyt)
One dispatcher over a policies-as-data registry (hooks-manifest.v2): deny|route|audit modes, waivers, once-per-session messaging, hashed guardrail telemetry, fail-open on missing registry/jq. Predicate discipline (pure-only may block) enforced mechanically by lint-policies.sh. Day-1 cohort: git-add-_beads, provenance-ledger hand-append, cp-into- installed-skills. Ships INERT; opt-in installer wires Bash + Edit|Write PreToolUse matchers. 25 bats contract tests. |
||
|
Bo
|
65e3dd77f5 | docs(cli): reclassify carved provenance+skills leaves as allowlisted in surface parity |