mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
main
42 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bo
|
937d6c8269 |
Add native Codex context-budget delegation and opt-in refusal (#1140)
## What Add Codex-native `bulk-reader` and `code-writer` roles pinned to `gpt-5.6-luna`, opt-in role/config installation, and an opt-in native `PreToolUse` Bash adapter for the shared read-budget guard. Source-owned guidance and role files ship through the existing generated Codex bundle; the menu stays at 34 skills. The installed Codex 0.154 runtime can refuse covered shell calls before execution. The adapter enforces that predicate; slice discipline, target-only writes and receipt-only replies are role instructions, not an output filter or per-file sandbox. Native exact-definition hook trust remains required. ## Why Follow-up to #1137 and its [fresh author-distinct review](https://github.com/boshu2/agentops/pull/1137#issuecomment-5648513520). Correct the unverified statement that Codex has no refusal-capable hooks and provide real native delegation without subprocess model execution. This branch incorporates the isolated fixes in #1139 and targets main; land the repairs first. The fixes PR contains no Codex-native changes. Work and original acceptance are recorded in private BD `age-z25n`; `bd context --json` resolves the existing private Dolt store. [Design and live evidence](https://github.com/boshu2/agentops/blob/codex/context-budget-native/docs/design/codex-context-budget.md) records the runtime/config contracts, exact available model identifiers and published comparable rates, source paths, invocation, transcripts, accounting and limits. ## How I tested Current head: `40edb5f216b71bfa659627dca10fc0a9e04c1821`. Fresh author-distinct review: **Job2 native PASS; combined subject FAIL**. Reviewer context `01a09778-bef6-7883-880b-6764fdd783b9`, observed `gpt-6-astra`/ultra, authored no candidate code. All 52 changed paths have identical start/end manifests (SHA-256 `b1f13ca731658bc699838189f928535a386ab6de6b381b0419bb0f7ebdd88cdc`); acceptance `not_checked: []`. The native repeated-refusal naming finding is fixed and independently reproduced as resolved. Local final checks pass. [Hosted Validate CI](https://github.com/boshu2/agentops/actions/runs/34722978404) remains pending at this update; no merge is performed. Real Claude Opus follow-up closes plugin-name resolution, inherited hook invocation, complete reader coverage and observed parent/child content separation. **It also finds a remaining Claude writer failure:** two of three final workers ran their supplied check twice; a direct receipt used Markdown fences. This combined PR does not claim a full Claude writer PASS. Details, excluded failed attempts and exact identities are in the design note. - `./cli/bin/ao gate check --scope range:origin/main..HEAD`: 33 passed; the earlier unchanged registry run `./cli/bin/ao gate check --full` passed **73 gates** at `90c8b31ee` (historical receipt). `bash scripts/regen-all.sh --check`: all 11 checks passed. - Door9, hookless cold-start, doc-hook drift, shellcheck on five changed shell files, and Node syntax checks on both workflows plus the native config editor passed. - A7 Bats suites plus workflow and all native suites: **223 passed, zero skipped**. All **21 documentation-reference tests** and the strict full-document reference scan also pass after clarifying the native hook-manager terminology. - `bash tests/run-all.sh`: **10 passed, 0 failed, 1 skipped** (optional OL directory absent). This is the default static tier. - `bash scripts/validate-codex-install-bundle.sh`: passed, **34 skill packages**. Changelogs are identical; diff check clean. - Live registered reader: parent `01a09776-004f-79a0-af85-c75b472a1e68` used only spawn/wait; child `01a09776-3c38-7281-b582-01fef5101f17` was natively identified as bulk-reader/Luna/low. Six separate slices covered 1,772 lines without truncation; parent received five findings and coverage only. - Live registered writer: parent `01a09771-8908-7a00-b101-919b558cf8c1` used only spawn/wait; child `01a09771-bbbf-7232-add4-cc9ec55ad759` was code-writer/Luna/medium. Required reference, seven-line Bats target, receipt only, child check passed; coordinating parent independently ran Bats successfully without reading the target. - Live hook: native parent `01a09769-14d2-7c22-9b7d-50847de07c90`, final turn `01a09771-25f2-7830-830b-498d7ca1945e`, refused a 400-line cat before execution and allowed a three-line sed slice. Real payload is PreToolUse/Bash/tool_input.command; hashed deny ledger schema verified. An explicit session hook was trusted in `/hooks` for this proof. - Credentials-free native config/read and hooks/list probes verify personal and ordinary project discovery. Codex 0.154 reads linked-worktree project hooks from the primary checkout; `--project` now rejects linked worktrees before writing, with a real Git regression. Known failed: Claude writer check-once behavior and direct receipt fencing. Not checked: arbitrary hosted/MCP read interception; adversarial enforcement of role instructions; other runtime versions/accounts; cost savings, comparative latency or ADR-0002 value-proof clearance. Earlier reader attempts with an overlarge final slice or truncated aggregate output are disclosed and excluded from the successful coverage proof. ## Checklist - [x] `make build && make test` passes (if Go changes) — no Go changes; gate runner built once. - [x] No secrets or credentials in code - [x] Breaking changes documented — opt-in additions; unsupported linked-worktree hook installation refuses before mutation. |
||
|
Bo
|
8061085c89 |
Ship native evidence helpers and fresh-family review defaults (#1110)
AO now performs intent snapshots, subject manifests, strict evidence verification, atomic verdict storage, and orphan inspection through the Go binary. The command handler keeps verification separate from presentation so it meets the existing complexity limit. These operations preserve the existing evidence formats, require explicit protected storage where applicable, and run outside a checkout without Python. The unchanged Python implementation remains a developer oracle; agents still provide semantic judgment. Codex and Claude skills now default to a fresh reviewer from the author’s model family. Callers can explicitly request cross-model review or pin its model. Reviewer adapters use a finite caller timeout or remaining deadline instead of a fixed ten-minute default, while retaining output limits and abnormal-termination cleanup. Validation: Go build, vet, tests and race/shuffle tests; 1,334 shell tests; aggregate runner; regeneration check; 72 full-mode gates. Independent checks exercised 84 storage-boundary rejections and 21 evidence operations with an empty PATH. Both canonical and generated RPI reference suites pass all 48 tests after updating the migrated oracle import without weakening assertions. Change-sensitive checks explicitly compare the final committed candidate with the original PR base. Linux, Windows, installer, security, and required summary checks are green. |
||
|
Bo
|
baa24e1b04 |
Bound headless execution and require acceptance-linked repair progress (#1107)
Headless invocations could run without a finite bound, and repair loops could count changed digests or finding counts as progress. The existing adapter now enforces positive finite deadlines, an optional inherited absolute deadline, bounded output capture and cleanup of its owned process group, refusing execution when protection is unavailable. The existing goal, RPI and validation skills now require acceptance-linked progress, distinguish newly discovered failures from regressions, and bound Andon recovery. Review effort follows the effect on acceptance and enforcement. These contracts do not claim native aggregate goal-budget or pause enforcement. Validation on the clean main baseline: - 1,317 Bats cases, zero failures, two skips; 46 RPI reference cases. - Go build, vet, tests and race/shuffle tests; local aggregate runner. - Generated projections current; all 72 AO gates pass. - Fresh independent Astra and Fable 5.1 xhigh judgments passed the exact clean source. Earlier uncommitted CDLC adoption, private work/evidence, and unrelated Go changes are excluded from this PR. |
||
|
Bo
|
1cebfee94d |
Measure one skill: sealed probe dispatch and the first current ledger row (#1101)
## Measure one skill: filesystem-sealed probe dispatch and the first current ledger row Provenance: the instrument-panel roadmap (`docs/plans/2026-08-26-instrument-panel-roadmap.md`) closed out with zero skills measured because control-arm reps read `skills/<skill>/SKILL.md` from the checkout and the operator's skill roots mid-run. Its named successor was filesystem-sealed dispatch. This PR builds the seal, then uses it, and the instrument caught one more leak on the way. **The seal.** Codex's own seatbelt does not nest inside an outer `sandbox-exec` profile, so a rep now runs under an outer profile with Codex's sandbox bypassed for exactly that reason: `file-read*` denied on the checkout and every skill root (`~/.agents`, `~/.claude/skills`, `~/.gemini/skills`, `~/.codex/skills`), `file-write*` denied everywhere except the rep's workspace, a scratch HOME, and the temp roots. HOME and CODEX_HOME are scratch (auth symlinked). Dispatch refuses without `sandbox-exec` unless `PROBE_SEAL=none` is set, which marks the run coverage-ineligible. The seal is recorded in the capture contract (`agentops-skill-probe-capture.v3`, a `seal` block), coverage eligibility requires `seal.mode == seatbelt`, and `verify-scorecard` re-checks that the denied roots include the repository and the four skill roots. Existing 08-26 sets become `legacy-unsealed` and ineligible, which changes nothing they were counting. **The leak the seal did not cover.** The first sealed capture showed the xhigh control-2 rep listing the shared live workspace, finding `treatment-1.prompt`, and reading it; the treatment prompt embeds the skill. So every rep now gets its own empty workspace, prompts reach a rep on stdin only, raw JSONL and stderr live in a harness-private dispatch directory the rep can only stat, and a `sibling-prompt-read` trap degrades any rep whose successful command names a prompt, contract, seal, or stage file or whose output lists a sibling's artifacts. Replaying the contaminated set under the new trap turns its REGRESSIVE into INERT with the contaminated rep degraded; that set is superseded and not committed. **The second pass, after the judges read the first.** A Fable validator passed the first sealed capture with four findings and a Codex read failed it with seven, all real: the whole per-user temp tree stayed readable (hundreds of stale probe directories held prompt files with the canonical skill bytes), sibling checkouts and the real `~/.codex` sessions were readable because seatbelt matches the traversed path and the skill roots are symlinks into the main checkout, the dispatch directory was writable so a rep could rename an earlier prompt into its workspace, the contract trusted asserted seal metadata (a hand-made Linux record was coverage-eligible), the only real-Darwin test nested seatbelt and proved nothing, and the operator's `config.toml` was symlinked in whole, so every rep started the operator's MCP servers. The seal is now one run directory removed on exit (`home/ ws/ tmp/ dispatch/`, reset per rep); reads are denied under the real HOME, the real TMPDIR, `/tmp`, the checkout, the git common directory, and each skill root's resolved target; `dispatch/` is write-denied with metadata-only reads; `file-link` and `file-clone` are denied on every denied root; the rep's config is rebuilt from top-level scalars only and `auth.json` is copied, not linked. The contract binds platform, mechanism, wrapper, the denied and allowed roots, the rep environment, and the sanitization, and coverage refuses any record that omits them; the Darwin test performs its reads inside the seal. One recorded hole remains by necessity: the codex launcher under `~/.local/bin` and `~/dev/dotfiles/bin` is re-allowed by literal path, and coverage refuses any allowed path inside the checkout. **The third pass, after the judges read the second.** Both judges failed the hardened tip again, and the headline was network: bypassing codex's own sandbox to nest the outer seal had dropped every inner deny, not only the filesystem ones, so a rep could fetch the canonical skill from GitHub while the disk was sealed. A rep now reaches the network only through a harness-owned CONNECT proxy (`scripts/lib/probe-connect-proxy.py`) with a host allowlist discovered from real runs (`chatgpt.com`, `ab.chatgpt.com`, the rotating `.oaiusercontent.com` suffix, and the API-key hosts), behind a seatbelt `network*` deny that admits only the proxy; every attempt is logged, a refused one degrades the rep, and web search is disabled in the rep's config. The profile is now rendered from the bound seal block by one function, so `verify-scorecard` reconstructs it and requires the digest to match; the recorded launcher chain is the only allowed read under a denied root; the wrap invokes `/usr/bin/sandbox-exec` by absolute path; the real CODEX_HOME and the Darwin cache directory are denied and bound; every non-stdio descriptor is closed and `/dev` writes narrowed to four devices; each rep runs in its own process group that must be empty before the next reset; one minimal config is generated per run, bound by text and digest, and re-parsed after each rep; one guarded trap covers the run root and any unpublished stage; and the coverage gate prints an eligibility row for every ledger row that names a scorecard. What stays open is stated in the README: allow-default still governs what the profile does not deny. **The fourth pass, after the judges read the third.** Both judges failed the third tip on properties the record carried but the verifier did not check. GNU `timeout` sets its own process group and ran outside the seal, so the per-rep reap was vacuous (four sleeping survivors after the test passed); the seal is now the outermost process, timeout runs inside it with `--foreground`, its absolute path is bound, and the reap counts survivors before signalling. The verifier now pins, not merely records: the egress host set (a module constant), port 443 only, no unix sockets, a loopback proxy, the generated config text equal to the renderer's output, the real CODEX_HOME and cache root inside the denied roots, writable roots under the run root, the four device paths, the environment allowlist, the launcher chain's digest against the producer identity, and it refuses unknown record fields. The proxy logs an attempt before dialing, captures the rep at accept time, refuses names that resolve to private or loopback ranges, and its source joins the evaluator hash set; the per-rep egress log is published with the fixture set. Withdrawn ledger rows get their own eligibility reason. The README lists which seal fields the verifier pins and which it only records; the cross-capture aggregate is recounted to 4 of 14. Still open and disclosed: Mach IPC and Apple Events beyond the filesystem and network seals, exact-byte environment sealing beyond the allowlist, and cleanup ownership by inode. **The fifth pass, after the judges read the fourth.** Both judges found the seal, proxy, reap ordering, transcripts, and scorecards sound and failed the tip on claims the tree did not hold: the egress log the docs called published was caught by the repository's `*.log` ignore rule, a `setsid()` child escaped the process-group reap, and one ledger sentence said no rep ran a command when a control rep had run `rg` over its empty workspace. The log is now tracked and required, strictly parsed, with both allowed and refused counts recomputed per rep; the reap is fatal to the capture when it cannot prove emptiness and an `lsof` sweep of the run root catches a session escape; the rep launches through a real `env -i` boundary; the timeout budget, the launcher chain's adjacency, every required root in both read and link denies, the evaluator match, and the trust-table value are pinned by the verifier; and every sentence in the ledger, README, and runbook names the check that enforces it. The remaining gap is stated: a child that both escapes the session and holds nothing under the run root is not detected. **The sixth pass, after CI read the fifth.** The first CI run of this PR failed on Linux: the launcher-chain pin walked the capturing Mac's filesystem, so on the runner both sets read as unsealed and the coverage headline fell to zero. A pin that holds on one host is not a pin. The chain is now bound as structure in the seal record (each link's path, kind, symlink target, and the final file's digest, with the invoked path as the head), the verifier checks that record on any host, and it cross-checks the live filesystem only where the head exists, reporting which happened. A gate-level test runs the coverage gate twice, once with the chain made absent, and requires byte-identical output. **The measurement.** `premortem-plan-shape-t2` (two seeded structural defects, band [1,4]) captured seven times on 2026-09-03, once under each seal; the first six sets were deleted with their scorecards when a judge or CI broke that seal. The rows that count are the seventh capture: low INERT (control 0/2, treatment 0/2) and xhigh INERT (control 1/2, treatment 1/2); no rep ran a command, no egress was refused, both per-rep egress logs are tracked and verified, and both scorecards verify with every pinned field, the launcher chain checked as a record and cross-checked on the capturing host. The superseded captures read, at low, BEHAVIORAL 1/1, INERT, INERT, BEHAVIORAL 1/2, INERT, BEHAVIORAL 1/2 and, at xhigh, INERT, BEHAVIORAL 2/2, INERT, INERT, INERT (control 1/2), INERT. The reversals across captures are recorded as an unresolved N=2 observation, not variance; across all seven the treatment arm put the marks in band in 7 of 26 usable reps and the control arm in 2 of 28, and the ledger marks that aggregate as disclosure, not a row. Headroom is SEPARATED at both levels. This is the first current, manifest-backed ledger row; it records response-shape behavior change, never quality uplift, and what it records today is INERT at both levels. **Evidence on the tip.** Probe, exec-lib, metadata, coverage, headroom, and adapter bats suites green with the Darwin seal, survivor, session-escape, and environment-boundary tests running; `verify-scorecard` on both scorecards; `check-skill-probe-coverage.sh` reports measured 1 of 12 with the 2026-08-26 row labelled `verdict-withdrawn`; regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go bar, lint, security, and the gate-tightening ratchet green. **Judgment, stated plainly.** The caller elected Fable 5.1 as the binding validator. Its fifth-round verdict on the pre-CI tip was PASS with no verdict-driving findings; its read of the portability delta failed on one test-only point (the coverage test had dropped the `measured` pin that CI tripped on) which this tip restores. The cross-family Codex reads of the same tips are FAIL. They closed the network seal, the environment boundary, the fatal reap, the data roots, the config binding, and the documentation, and they keep these open, recorded here as the next pass rather than hidden: the evaluator identity check still accepts the pre-proxy four-file shape for legacy sets; an inherited absolute `CODEX_EXEC_TIMEOUT_BIN` is trusted without a probe; a refusal logged under a rep label outside the expected set is not rejected; an empty top-level `[projects]` table passes the drift check; the `lsof` sweep treats a failing `lsof` as a clean result; and the launcher chain, now bound as a record that any host can check, is corroborated against the filesystem only on the capturing host, so a record whose head does not exist there is taken on its word (the docs say so; Codex wants the live walk required at capture and the invoked head bound into the producer identity). Fable also noted a test that mis-resolves when TMPDIR sits under `/private/tmp`, a `2026-09-04` date in one refusal message, and that the low control replies used a `S2: FINDING <reason>` shape the discriminator scores as "plan not walked" rather than by band (outcome-invariant). Each of those changes a harness or verifier file and would orphan the two rows again, so they are filed as the follow-up instead of an eighth capture today. Across six judging rounds the open set shrank only from round four on; the convergence law's stop signal was reached at round three and the run continued on the caller's instruction to land the instrument. The two ledger rows are honest about what they measure, and the seal they ran under is the one this tree can verify on any host. |
||
|
Bo
|
10f0277bdb |
Legible membrane, Train 2: what a stranger meets (#1100)
## Legible membrane, Train 2: what a stranger meets Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus, findings F5 through F9. This train is sized by a consumer inventory built with `rg` on the tip before any lane was written; the promoted-set directory move the audit proposed is deferred because that inventory shows skill-builder backing two blocking gates, swarm pinned by the cathedral gate and a routing golden, using-gc required by Go code, and `ao skills link` unable to install a second root. That inventory is the plan for a later train. **What changes.** - **Archival sweep by consumer disposition.** 172 audit snapshots, 29 pawl receipts, the `evals/workbench` and `evals/membrane` trees with their two bats consumers, four stray scratch docs, four retired eval contracts, and nine caller-less `scripts/check-*.sh` are deleted; git history is the archive. Every machine list that referenced them is pruned (evidence-grounding baseline, preamble grandfather, broken-links allowlist, `.gitattributes`, `.gitignore`, two eval fixtures, the workflow-coverage deferred list). `docs/audits/manifests/` and `.agents/ao/config.yaml` survive because they have live readers. About 48,000 lines. - **Three skills retired.** `goals` (alias of fitness), `shared` (tombstone), and `scope` (folded into plan step 3 as five write-scope checks). Consumers edited; the probe denominator exclusion for goals pruned; Codex package and golden count pins updated. - **Negative routing** on research, codebase-recon, reverse-engineer, premortem, one-way-door, and council, all within the 180-char budget, with a teardown golden (`rq-08`). One wording was changed after the router's prefix stemming showed "repository teardown" leaking into the wrong skill. - **Every promoted skill answers "It's working if"** with observable tells in backticks, and carries a paste-ready `## Prompt` with a concrete subject. Two fictional `ao` subcommands a draft prompt named were caught by the body-ref validator and replaced with real commands. - **Doctrine diet on the core five.** rpi, plan, implement, validate, and anti-ceremony drop from about 5,100 words to 3,600 (bodies from 4,700 to 3,150) by moving the shared ownership boundary, dated incidents, and mechanics tables into step-loaded references (`skills/rpi/references/boundaries.md`, `skills/validate/references/mechanics.md`, `skills/plan/references/ground-truth-routing.md`). Every cathedral canary and every skill validator grep survives unchanged. - **ADR-0018** records the goals, shared, and scope retirement; the cathedral gate tombstone and the routing goldens cite it instead of ADR-0017. - **Router and twins.** `ao skills find` holds a description's "Not for X; that is <sibling>." sentence out of its haystack, so premortem no longer ranks first for "is this live decision reversible" (golden `rq-10` pins the reciprocal of `rq-02`); a penalty variant was tried and reverted because it suppressed skills the caller named outright. A declared trigger phrase of two or more words quoted whole in the query now earns the name weight once, so "check this change" lands on validate rather than on reality-check's name token; a live-catalog test pins seven such queries. Single-quoted YAML descriptions unescape `''`. The Codex catalog keeps the exclusion sentence, and a closing `>` no longer turns `<run-id>/codebase-recon.json` into an invocation. - **Residue the judges found.** handoff, learn, and status open a `## Contract` heading after their tells; validate's prompt names its helper at `skills/validate/scripts/validate.py`; the explicit-skill prompt catalog names only live skills (five stale prompts replaced by nine, floor 20 restored, TESTING.md names the suite); the corpus-delta receipt binds the runner's path and SHA-256 and labels a `live_agent` claim as an unverified caller declaration; the probe README and ledger describe the 12-skill denominator; SKILL-API counts 30 of 54. **Evidence on the tip.** Regen check clean; full gate green with a HEAD-built binary; CI's bats command green; Go build/vet/test green; lint clean; security gate quick PASS; docs-build warnings did not rise. Fresh validation by Fable 5.1 (caller-elected) and a cross-family read by Codex, both recorded in the PR thread. --------- Co-authored-by: Bo <bofuller55@gmail.com> |
||
|
Bo
|
e556834d3d |
Legible membrane, Train 1: Codex projection, executable entry points, green run-all, honest runtime docs (#1097)
## Legible membrane, Train 1: fix what ships Intent source: `docs/plans/2026-09-02-legible-membrane-plan.md` (added in this PR). Provenance: the 2026-09-02 field audit of this repo against mattpocock/skills, compound-engineering, and the jsm corpus. Three defects a stranger hits today, all verified on `main` before the change: 1. **Codex projection truncated 51 of 56 skill descriptions mid-clause** (`scripts/codex-sync.sh` capped prose at 44 chars, so the Codex router read "Freshly judge whether a finished change is Triggers: …"). Now: first sentence of the source prose plus the full `Triggers:` clause, abbreviation- and quote-aware; the per-catalog bound is computed live as "Codex prose average may not exceed Claude's" (cross-multiplied, floor-free) with a 180-char hard ceiling. `using-flywheel` is exempted from runtime-phrase rewriting so its cross-runtime text projects verbatim. The dormant `ao codex ensure-start` emitter and the `# /x`→`# $x` title rewrite are removed. A literal oracle pins five twins and a rule oracle covers all 56. 2. **23 shebang-bearing shell entry points were committed non-executable**, including `scripts/regen-all.sh` and three gate scripts. Now `100755`, with an advisory gate `shell.exec-bits` (fast+full, fail-closed on enumeration error, reads the index blob, skips symlinks by stated policy). 3. **`tests/run-all.sh` was red on `main`**: the GOALS validator asserted a pre-08-25 file shape, and validate's description exceeded the 180-char budget. The validator now parses only the `## Gates` block (stops at any heading, like production), requires cited script paths to exist, takes its path via argv, and fails closed on tokenization; four negative fixtures. validate's description is 177 chars and gains the reality-check negative. `AGENTS.md` quotes CI's authoritative bats and Go commands. README and `docs/install-day2-ops.md` replace "No other runtime is required" with a runtime table derived by reading each skill's procedure (HARD / OPTIONAL / conditional), covered by a bats test that grounds every HARD row in an invocation line. Evidence on the tip: full gate 71/71 (HEAD binary), CI's bats command green across `tests/scripts/*.bats`, `tests/run-all.sh` green, Go build/vet/test green, golangci-lint clean, security gate quick PASS, `scripts/regen-all.sh --check` clean. Each lane was validated by a fresh context; the integrated train had two cross-family (Codex) review rounds and every finding is closed on this tip. Out of scope (successor intent): promoted set / `skills-internal/`, the process-artifact sweep, "It's working if" blocks, routing clusters, doctrine diet. One pre-existing drift noted for a follow-up: `packs/agentops-executor/agents/validator/skills/validate/SKILL.md` carries a stale third description that no gate binds. |
||
|
Bo
|
e69144d6d1 |
Train 2: seeded-defect probes + gate hardening — the instrument catches its own contamination (#1088)
> **⚠ Open review residues — merge is Bo's call (spiral stop: second
non-PASS on this train's intent).** The cross-family re-check affirmed
the substance (contaminated fixture reproducibly UNMEASURED with four
real reads; all six scorecards match fresh replay; coverage honestly
12/12; single-level rule strictly tightening; 70/70 gates; full suite,
lint, validators green) and states plainly that **the zero-result
acceptance miss is not the reason for failure**. What failed, each with
its fix:
> 1. `evals/skill-probes/LEDGER.md:71` + `RUNBOOK.md:75` — a summary
block my withdrawal sweep missed still says "Exactly one row was earned…
1/12… council SEPARATED… worked for one of four". Fix: rewrite that
block to the post-withdrawal truth (zero rows earned, 12/12, council
UNMEASURED pending a second level).
> 2. `scripts/lib/probe-fixture-metadata.py` — the contamination
docstring overclaims: "a failed attempt loaded no bytes" is false for
`cat …SKILL.md; false` (read succeeds, compound exits nonzero), and
glob/copy-then-read indirection evades. Fix: state the detector's honest
boundary (exact-string, final-exit heuristic; evasions listed) — the
mechanism is a floor, not a seal, which the RUNBOOK already says.
> 3. **Caller decision:** the `scripts/probe-skill.sh` stderr-literal
exclusion is validator-verified fail-closed but sits outside L2's
declared write scope — it needs your explicit scope blessing (or
reversion, which re-voids all live dispatch on codex-cli ≥ 0.14).
> 4. Minor, for the record: my commit message reports security-gate
quick PASS (true in my run); the re-checker's environment reported
WARN_QUALITY from a missing radon + a govulncheck toolchain mismatch
unrelated to this diff — and the grounding gate's dead-SHA class is
UNCHECKED on shallow clones, including both our checkouts.
> **Acceptance status, told straight: L2's plan clause ("≥ 2 skills move
off UNMEASURED") is NOT met — the honest count is zero.** The lane filed
one BEHAVIORAL row (premortem); its fresh validator dumped the committed
transcripts and proved every rep, both arms, had fetched the skill off
disk mid-run — the separation was band-spray variance between two arms
holding the same bytes. The row is withdrawn, the ledger honestly reads
12/12 unmeasured again, and the train's deliverable became something
better than the number: the harness now mechanically catches that
contamination class forever. Merge is Bo's call.
## What
Train 2 of the [instrument-panel
roadmap](docs/plans/2026-08-26-instrument-panel-roadmap.md):
seeded-defect probes for the judgment spine (L2) and gate hardening
(L3), plus one repair commit responding to L2's validator findings.
**L3 — gate hardening** (`instrument/gate-hardening`, lane verdict
**PASS 11/11**)
- `gate.tightening-ratchet` (advisory): diffs `BASE_REF...HEAD` and
fails on six textual loosening shapes in the gate surface unless a
`Gate-Loosen-Reason:` trailer justifies it; tightening always free.
Detection boundary documented in the script header — silent on semantic
weakening it cannot parse, fail-closed on what it can.
- `evidence.grounding` (advisory): mechanical scan of `docs/audits/**` +
`docs/evidence/**` for cited paths that don't exist, unresolvable
full-length hashes, and scaffold leaks. 45 real findings on the live
tree, each **baselined with its argument** (directory-pinned baseline
with the added-line rule as teeth: a finding on a newly-added line is
never excused). One genuine live defect surfaced:
`docs/evidence/membrane-receipts.md` claims a generator that no longer
exists.
- Registry 68 → 70; `ao gate check --full` 70/70.
**L2 — seeded-defect probes** (`instrument/seeded-probes` + repair
`927be2184`)
- Four tier-2 probe packages (validate, premortem, council,
one-way-door): sub-40-line realistic artifacts, two planted defects
each, `canonical-skill` treatment, act-grading discriminators calibrated
both directions (21 hand-written transcript cases) — the lane's
validator wrote its *own* counterexamples and the discriminators held.
- 28 live dispatches (gpt-5.6-luna, low + xhigh) through the sanctioned
executor with `CODEX_HOME` isolated. Six immutable v3 fixture sets
replay deterministically.
- **What the run actually measured:** `one-way-door` and `validate`
scenarios SATURATED (retired to the RUNBOOK, no ledger rows — the
headroom gate doing its job); `council` was measured at one level and
the lane *refused* to file the row the gate's labeling bug would have
allowed ("filing it would have laundered a void row into the acceptance
number"); `premortem`'s BEHAVIORAL row was filed, then **withdrawn**
when the validator proved contamination.
## The repair commit (one bounded round, per the plan's validation
strategy)
- **`skill-read-contamination`**: `classify_bytes` now degrades any rep
whose transcript shows a successful command reading a `SKILL.md` — one
insertion covering live and replay. RED proven against the committed
contaminated fixture (replayed BEHAVIORAL before; UNMEASURED 0/0 after),
bats case added (probe-skill 37/37). The six scorecards were false
records under the rule: deleted (the harness's immutability guard
rightly refuses overwrite; git keeps the bytes) and regenerated —
contamination turned out broader than the lane knew (validate lost both
arms at xhigh, one treatment rep at low; one-way-door lost an xhigh
rep).
- **Headroom gate single-level fix** (L2's disclosed gate finding): a
group measured at one effort level whose control aced it is now
UNMEASURED ("capture a second level before any verdict row"), not
SEPARATED. RED test first; strictly tightening; the council group now
reads honestly.
- LEDGER row → WITHDRAWN with the full story; RUNBOOK's two falsified
sentences corrected; `CODEX_HOME` isolation documented as proven
insufficient, the transcript-level trap as the floor until dispatch is
sealed at the filesystem.
- Ride-along accepted with justification (validator-verified
fail-closed): the harness's stderr rule now excludes exactly one
whole-line literal — codex-cli ≥ 0.14's own "Reading prompt from
stdin..." announcement, which was degrading 100% of live reps on a clean
exit.
## Evidence
Full Go suite exit 0 · go-lint clean · `ao gate check --full` **70/70**
· security-gate quick **PASS** · four skill validators · probe-skill
37/37 · headroom 10/10 · tightening-ratchet + grounding bats green ·
`regen-all.sh --check` current. Per-lane fresh validators (L3 PASS; L2
FAIL → this repair); cross-family Codex verdict on the integrated train
recorded in the session report.
## What this train proves
The membrane worked on its own measurement pipeline, three layers deep:
the lane refused to launder a void row; the fresh validator caught
contamination the lane's isolation missed; the repair turned that catch
into a permanent mechanical trap. The cost is the honest headline — zero
skills measured — and the asset is that the *next* live wave runs
against a harness that cannot silently repeat any of this.
## Follow-ups
- Seal live dispatch at the filesystem (arms must not be able to read
any skill corpus) — then re-run the premortem and council waves for real
measurements; council additionally needs its second effort level.
- `docs/evidence/membrane-receipts.md` generator claim (surfaced by
evidence.grounding).
- The `Gate-Loosen-Reason` ratchet does not yet cover
`scripts/probe-skill.sh` — the stderr-literal class belongs under it.
---------
Co-authored-by: Bo <bofuller55@gmail.com>
|
||
|
Bo
|
f3c6d0ecf2 |
Converge retained WIP and harden evidence boundaries (#1065)
Summary:
- lands the audited current WIP lanes and excludes stale/process-only
material
- hardens prune path confinement, probe-v3 evidence binding,
codebase-recon identity, handoff/release/reverse-engineer behavior, and
Codex prompt handling
- truth-labels static skill scoring and regenerates all owning
projections
Validation:
- fresh independent PASS on commit
|
||
|
Bo
|
7a765cde19 |
Align AgentOps around its operations-layer identity (#1051)
Executes docs/plans/2026-08-07-agentops-operations-layer-alignment.md: AgentOps is the operations layer for agentic engineering; the federated integration graph is the topology, the semantic work-and-proof protocol is the contract, and RPI is the standard one-experiment traversal. Retires the ao flywheel command family and all knowledge-flywheel product state, tombstones the seven-move operating-loop workflow, narrows ao init and the .agents state writers to declared destinations, renames the core architecture page to rpi-traversal.md with a compatibility redirect, aligns AGENTS.md, 25 skills, public and package copy, regenerates every owned projection, and strengthens the conformance gates with planted-negative proofs. Both the alignment subject and the follow-up gate-bookkeeping commit carry fresh author-distinct validation PASS verdicts with empty not_checked scope. Test-Removal-Reason: dead knowledge-flywheel and session-store surfaces were deleted with their tests (operations-layer alignment) |
||
|
Bo
|
56190722e6 |
fix(gates): atomic-write ratchet fails closed on scan-helper death — no silent PASS (#946)
## Root cause (CI flake, run 29785505667 attempt 1, PR #944) The bats test `adding only the temp-signal half to a file with an EXISTING rename still trips` failed once on ubuntu with the ratchet printing a clean `PASS … grandfathered=0` where the fixture deterministically trips. Diagnosed by analysis per the test-isolation discipline (not repro-chasing): - bats errexit proves every fixture `git commit` succeeded; the fixture repo is fully isolated. - `grandfathered=0` in the PASS line fingerprints the run as in-fixture (real repo = 4). - Zero stderr in the captured output rules out every loud failure handler. What remains: **three silent paths where a scan helper (grep/awk/git) dying under 4-way-parallel runner load reads as "no finding"** — the gate then certifies an unchecked diff: 1. `file_trips`: strip/grep failure returned 1 ("does not trip") → file silently skipped. 2. Detector loop read from `< <(ratchet_changed_files | sort)` — a collector rc 2 inside process substitution is discarded, certifying an EMPTY change set (the exact fail-open the ratchet-lib header warns about). 3. `ratchet_added_hunk_matches`: helper death propagated as arbitrary nonzero, swallowed by `|| continue`. ## Fix - `file_trips` is tri-state (0 trips / 1 no / 2 refuse-to-certify, loud); signal greps read full input (no `-q` early-exit) so pipefail can never surface a SIGPIPE'd printf as a phantom failure on large files. - Changed files are collected into a variable under pipefail with an explicit `exit 2` handler. - `ratchet_added_hunk_matches` maps fallback-grep/awk death to loud rc 2 (0/1 unchanged; existing `|| continue` consumers skip on 2 exactly as before, now loudly). - `compute_grandfather_set` aborts `--regenerate` on helper failure — never a truncated grandfather list. ## Pinned Dying-awk PATH-shim tests at gate and lib level: helper death is rc 2 "refusing to certify", never PASS. Same fail-closed class as the pawl-refuted empty-stream certify (2026-07-10). ## Validation - 58/58 in `check-atomic-write-ratchet.bats` + `ratchet-lib.bats` on fresh origin/main - All six ratchet-lib consumer gate suites green - `shellcheck -S warning` clean; real-repo gate PASS (grandfathered=4); `--regenerate` byte-identical - 65 repetitions of the formerly-flaky test green Residual: the external kill source is unprovable from one occurrence; a recurrence now exits 2 naming the failed helper + file instead of printing PASS. |
||
|
Bo
|
81e6380510 |
feat(cli): wire ao eval — the Learn seat's measurement surface (#921)
Registers the fully-built-but-never-wired eval module into the CLI spine and repairs its one real instrument bug. Three commits, RPI-shaped: wire + fix, drift guards + docs, fixture counts. ## What - **`ao eval` joins the spine** (cmd/ao/eval_composition.go): deterministic suite runs, run-record compare/baseline/scorecard/coverage, locked Tasks, holdout scenarios, suite A/B verdicts with power-derived n-required, and the holdout-safe Outcomes projection. This is the operating contract's *Learn* seat — a read/measure consumer of evidence. It reports numbers and owns no retry, scheduling, promotion, or lifecycle authority. The retired Aliases/Bench seats stay nil and their subcommands are omitted. - **Instrument bug fixed**: both scenario-ab arms were corpus-denied at runtime — the with-gold treatment was only a prompt injection through the retired `ao lookup`, so the A/B silently guaranteed a zero delta. Treatment is now environment-shaped: the with-gold arm runs with the corpus readable, the control arm stays sandbox-confined (the fail-closed empty-deny guard is untouched), and judges are always corpus-denied. - **Schema drift guards**: a run record persisted by the production writer must validate against eval-run.v1.schema.json (and the minimal suite against eval-suite.v1.schema.json) — closing the same silent-fork class the verdict.v2 golden corpus closed. - **Hermetic test builds hardened**: keep the real HOME's Go caches (TestMain isolation was forcing module re-downloads) and skip VCS stamping. - docs/architecture/go-cli.md gains the eval section; MIGRATION.md notes `ao eval`'s return; COMMANDS.md, cli-surface, and surface counts regenerated; new leaves carry honest allowlist reasons; new cmd/ao L2 tests drive eval run/compare/help through the full production wiring. ## Evidence - Full cli suite 2,801 green; race green on cmd/ao + the eval island; vet clean - Full-tier gate: 65/66 on the penultimate commit; the single failure (surface counts) is the final fixture commit, re-verified PASS standalone — no other check reads those fixtures - Fresh independent validation (Codex, frozen subject 86022a3c6, clean tree at start and end): **OVERALL PASS**, 6/6 — including a live end-to-end `ao eval run` producing a passing run record - Rubric-relocation decision honored: true holdout rubrics live in the external measurement register; the repo keeps only leak-guarded projections |
||
|
Bo
|
5d61bf8a3f |
fix: kill the shared-config gremlin class (hook-safe repo-root, git-env scrubbing, config-hygiene gate) (#922)
Fixes the three gremlins observed 2026-07-18 (bead age-gate-scripts-worktree-gitdir-p62wo): 1. **Gate scripts broke under hook env from linked worktrees** (#919 regression): new scripts/lib/repo-root.sh (GIT_*-scrubbed resolution, BASH_SOURCE anchor, worktree-correct) swept across 32 scripts. Proof: this PR's own push ran the pre-push gates from a linked worktree and passed. 2. **core.bare=true recurrence**: mechanism proven (leaked GIT_DIR + no-arg git init --bare rewrites the SHARED config); writer traced with high confidence to a live Gas City city process (concrete suspect: gascity cmd_rig_test.go runGitInTest, unscrubbed env + no-arg init --bare — fork-side fix filed separately). Defense: new always.git-config-hygiene gate fails fast on core.bare/test identities with the repair command inline (--self-test proves fail-closed). 3. **Test/test@test.com identity leak**: writer convicted — our own pre-push chain leaked GIT_DIR into go test, whose helpers wrote the shared config via git -C (mechanism proven). Cut at both ends: validate-go-fast scrubs env for children; every test git-helper set-site scrubs; check-test-isolation gains a WARN-ratchet rule (baseline 3 → FAIL at 0). Gates: shellcheck rc=0 across 36 scripts; go build/vet clean; 655+379+149 tests; hygiene self-test; both originally-failing validators pass under the repro env. |
||
|
Bo
|
23c0026500 |
fix: close v3.3.0 release-audit findings (#915)
## Summary Closes the findings from the v3.3.0 release-viability audit of main. **Surface truth (High):** `ao constraint`, `ao goals trace`, `ao session memory`, and `ao skills edit` are production tombstones but docs/cli-surface.md documented them as `public-tested`/`covered` (the test binary keeps constraint live, and the generated doc reflected the test build). Added `deprecated` allowlist rows, dropped the stale constraint-subcommand rows, regrouped constraint with its tombstone peers in the default spine, regenerated both surface projections, and added `TestCathedralCutTombstonesSurvivePruning` pinning the invariant that every Cathedral Cut verb keeps an executable tombstone. **Installer hardening (Medium):** `install-bd.sh` now fails closed when no published checksum exists for a release asset (explicit `--no-verify` overrides), and the curl|bash path verifies the fetched `installer-common.sh` against a pinned SHA-256 before sourcing it (drift-guarded by a new install-bd.bats test). **Docs (Medium/Low):** UPGRADING.md no longer links the MIGRATION-3.0.md file the Cathedral Cut deleted; CHANGELOG 3.3.0 `### Removed` now names `yield`, `constraint`, and the 3.2 `verify` front door and links the MIGRATION.md map. **Gate scripts (Low):** flywheel/provenance gate scripts build `ao` into `mktemp -d` scratch dirs instead of fixed world-known /tmp paths (symlink/TOCTOU on shared hosts). **Bonus (surfaced by the push gate):** `scripts/lib/preamble.sh` mis-resolved REPO_ROOT under git hook env (GIT_DIR without GIT_WORK_TREE), which made `go.cli-architecture` refuse every worktree-origin push touching `cli/cmd/ao/**`. Fixed by scrubbing the hook-injected discovery env around the rev-parse. ## Validation - `go build && go vet && go test ./...`: 5,085 tests green, golangci-lint clean - `ao gate check --fast`: 27/27 pass, including under simulated hook env - bats install-bd suite 6/6 incl. new pin drift test; shellcheck clean on all touched scripts - markdownlint clean; doc-release gate passes; tombstone exit codes verified on the built binary |
||
|
boshu
|
5bdbb5fd09 | refactor: simplify AgentOps loop and harden CLI | ||
|
boshu
|
e4dbef98f6 |
Collapse AgentOps install to ao skills link
Retire the 3.x plugin curl/PowerShell installers as tombstones, delete the plugin-cache helpers, and rewire CI/doctor/conformance onto the canonical checkout + ao skills link path. Keep install-bd and the opt-in operator scripts. Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
boshu
|
cb2a7ac398 | refactor: complete the cathedral cut | ||
|
boshu
|
4823077621 | refactor: cut AgentOps to a single-pass evidence loop | ||
|
Bo
|
1d4c003e05 |
Cut plan: gate registry diet, legacy bash gate removal, CI collapse, pawl warm-service removal (#906)
## Summary - **Wave 1** — remove the `always.wiring-closure` meta-gate, demote `docs.skill-refs` to Full tier, delete 8 orphan scripts. - **Wave 2** — delete the legacy bash pre-push gate (`pre-push-gate.sh`, 2,279 lines) and its parity machinery; collapse `validate.yml` from 18 jobs to 5. - **Wave 3** — remove the pawl warm NTM standing service; the cold review spine (pawl-review / pawl-verdict / pre-push check) is untouched. - **Wave 4** — gate registry diet: cut 24 doctrine/meta check IDs and ~70 backing scripts/workflows/bats; trim nightly knowledge-cycle job. Net: gate registry 112 → 83 checks (incl. main's additions), ~24k lines of scripts/CI/bats deleted. Routine one-file fast gate now runs in ~19s. ## Test plan - [x] `go build` / `go vet` / gates+goals+ports package tests green - [x] `ao gate check --fast --scope range:origin/main..HEAD` — 66 passed, 0 failed, 1 skipped (199 changed files) - [x] `validate-ci-policy-parity.sh` PASS; validate.yml / nightly.yml parse - [x] `regen-changed-scope --check` green; registry.json regenerated - [x] Grepped tree for dangling references to every deleted script/check ID Made with [Cursor](https://cursor.com) --------- Co-authored-by: boshu <241868352+boshu2@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> |
||
|
boshu
|
9c6a662203 |
feat(loop): refactor the four-umbrella operating loop
Bead: age-four-umbrella-loop-refactor-xz4ps.1 |
||
|
boshu
|
0514f82ec3 |
feat(pawl): present review as the user front door; warm verbs grouped operator-only (age-hk5zg.2)
'ao pawl --help' listed the warm standing-service verbs as flat peers of review,
pushing users toward operator machinery (NTM, projects_base) they cannot run.
Cobra groups now split the surface: review under 'Use the membrane (the front
door — needs no NTM, no setup)', up/down/reap/health/doctor/smoke/route/metrics
under 'Operate the standing warm service (operator-only; requires the NTM swarm
substrate)'. Grouping/labeling only — every verb keeps its name and behavior.
docs/contracts/pawls.md carries the same operator-only note.
Fix swept in: scripts/lib/sku_extract.py's cobra help parser only recognized
'*Commands:' section titles, so EVERY grouped command's subcommands were
invisible to the SKU catalog ('ao goals' was already affected; this change made
'ao pawl' the second victim and surfaced the bug). The parser now enters any
non-reserved column-0 title block and requires the 2+-space name/short gap, so
grouped subcommands resolve precisely (registry.json reflects the corrected
projection for ao goals too). Locked by TestPawlHelpPresentsReviewAsFrontDoor.
|
||
|
boshu
|
c0d12e4004 |
feat(gates): extract shared shrink-only ratchet lib (age-ratchet-lib-extraction-bv7d.1)
scripts/lib/ratchet.sh: the one implementation of the grandfather/baseline ratchet mechanics 7 gates hand-roll today — per-consumer parse modes (raw|cr-strip|strip|trailing-comment), scope-aware changed-file collection incl. upstream, fail-closed growth guard (comm -13 + intersection authority, RATCHET_GROWTH_GUARD=off escape for migrated gates), parameterized stale predicates, generic regenerate (header-fn + LC_ALL=C sorted body), added-hunk guard with ENVIRON-passed ERE. tests/scripts/lib/ratchet-difftest.bash: byte-parity differential harness (stdout, stderr, exit code separately) for the migration slices. tests/scripts/ratchet-lib.bats: 24 cases — real pinned-shape round-trips per parse mode, growth rejection, self-allowlist refusal, stale by set-diff and by predicate, regenerate golden, added-hunk guard, harness smoke. cli/internal/gates/checks/ratchet_routing_test.go: self-extending FM3 closure — every check script sourcing the lib must route lib edits via its Match globs. Pre-mortem: .agents/council/2026-07-10-pre-mortem-ratchet-lib.md (3 rounds). Lib is FROZEN for the rest of the arc per FM6. Claude-Session: https://claude.ai/code/session_01RDWPYn4ZS2SyBkcBvAM18u |
||
|
boshu
|
f7864bff30 | feat(frontier): LKG frontier with compensated-ancestry RESOLVED + 5 liveness acceptance tests (age-ekam) | ||
|
boshu
|
46c802d7ab |
fix(pawl): tip-coherence guard + canonical evidence/catch rooting (age-pawl-bead-tip-coherence-wckn, age-pawl-evidence-worktree-loss-np1e)
Two guards for the 2026-07-09 wrong-tree/evidence-loss incident: wckn: pawl_tip_coherence refuses (exit 3, fix-and-re-run) when the review target commit cites bead ids sharing the reviewed bead's prefix and none is the bead or a dotted relative - a verdict can no longer bind a tree that belongs to a different bead. Positive-mismatch only; PAWL_ALLOW_TIP_MISMATCH=1 opts out. 7 bats. np1e: evidence transcripts and membrane catches now root at the CANONICAL checkout (git-common-dir parent) via pawl_evidence_dir/pawl_canonical_root, so reviews run in disposable land worktrees no longer write artifacts that die with 'git worktree remove', and ledger evidence_path never dangles. 5 bats. Both libs registered in validate-embedded-sync + embedded copies synced. Claude-Session: https://claude.ai/code/session_01BkgdP88SiccCSyYdi8GHNa |
||
|
boshu
|
e7acfd9fa9 |
refactor(skills): repoint operating-loop CRUD to ao beads exec for bd+br dual-support + trim rpi (age-jgf9)
Dual-support increment 3 (the user-facing completion). AgentOps ships to users who mostly track with bd; operating-loop skills hardcoded `br`, breaking them. Repoint bead CRUD (ready/update/close/create/dep/show/list, non-jq) and the children-extraction pattern to the tracker-agnostic `ao beads exec` (bd-or-br) across skills/**/SKILL.md + references. In THIS repo (br) there is no behavior change; the win is bd users. - children pattern `br show <epic> --json | jq '…parent-child….id'` → `ao beads exec children <epic>` (post-mortem closure-integrity-audit + the latent invalid `br children` in rpi phase-data-contracts). - probe blocks re-gated `command -v br` → `command -v ao` (bd users have ao). - parity_only codex twins refreshed via codex-sync + hashes; bespoke twins are hand-maintained (codex-sync skips them by design) and left for a deliberate follow-up. - jq-shape `br … --json | jq '<shape>'` sites left on br (increment 4 will normalize the --json shape across trackers). - rpi/SKILL.md trimmed 280→249 lines (meta-tier ≤250): receipt JSON folded into the existing phase-data-contracts.md, rpi_state block compacted, redundant Reference-Documents entries dropped. - snippet validators (ao_snippet_resolve.py lib + validate-skill-body-refs.sh) taught that `ao beads exec` is a flag-forwarding passthrough, so tracker flags after it are not validated against ao's help. |
||
|
boshu
|
74aefa8312 |
feat(economics): amend-into-#trivial-bind trap guard (age-verification-economics-ebec.11)
Kills the operator-error class that fired ~4x on 2026-07-07, each costing a wasted review round: after the pawl auto-binds a #trivial provenance commit on the feat, a 'git commit --amend' folds the fix INTO the bind — HEAD becomes a #trivial commit carrying code, the reviewer's trivial-waiver walk steps PAST it to review the STALE original feat, and the verdict binds the wrong tree (surfacing only as an opaque 'hidden non-trivial commit' gate failure at push). scripts/lib/pawl-amend-guard.sh (new, sourceable, unit-tested): pawl_amend_guard returns 2 when HEAD is a trailing-#trivial commit changing non-provenance files, printing the diagnosis + the rebuild-don't-amend recovery. Wired EARLY into pawl-review.sh (scope=head, before the review — where it saves the round) and as a land-time backstop in pawl-land.sh; embedded pawl bundle + Makefile + embed validator updated. Safe: provenance-only #trivial binds, plain feats, and mid-subject #trivial mentions all pass; opt out PAWL_NO_AMEND_GUARD=1. bats 5/5; guard proven to fire on the trap and stay silent on legitimate binds; race suite green. |
||
|
boshu
|
69d5e7c392 |
feat(economics): pre-flight gate — run the deterministic battery BEFORE the model reviewer (age-verification-economics-ebec.9)
THE bottleneck fix. On 2026-07-07, closing skill-doc changes took 4-7 model-review rounds each because ~90% of the defects were DETERMINISTIC (format/link/twin/ gate-hint) — but the expensive cross-family verdict was requested BEFORE the local deterministic checks were green, so the membrane re-discovered them serially, each round a full re-review + re-land. scripts/lib/pawl-preflight.sh (new, sourceable, unit-tested): pawl_preflight <scope> <repo> runs the reviewed repo's deterministic battery before the reviewer and fails fast (exit 3, same fix-and-re-run disposition as the --smoke/REFUTED path) on a CONFIRMED-red battery — WITHOUT spending a reviewer round (0 tokens). Command: PAWL_PREFLIGHT_CMD if set (operator owns exit semantics), else the default 'ao gate check --fast --scope head' (ao resolved AO_BIN -> repo cli/bin/ao -> PATH, mirroring check-pawl-pre-push.sh). Wired into pawl-review.sh after the HEAD snapshot, before packet assembly + the meter clock. SAFETY — pure ACCELERATOR, never a false blocker: a nonzero that cannot be CONFIRMED as a real gate run (RCE trust-guard, missing/stale ao, build error — distinguished by the absence of the gate's 'checks —' summary marker) is SKIP-and-proceed, not RED. Worst case it no-ops and behavior is byte-identical to today (the pre-push gate still backstops). Disabled for PAWL_UNTRUSTED_REPO=1 (never run a repo's own battery over an untrusted checkout) and scope != head (no committed HEAD). Opt out: PAWL_NO_PREFLIGHT=1. Converts the inert prose lesson 'run the battery before the verdict' into MECHANISM (the skills audit proved prose lessons are inert). bats 9/9; embedded pawl bundle + Makefile sync + embed-sync validator updated; Go embed test green. |
||
|
boshu
|
9a23ba9cc5 |
refactor(skills): execute the audit retire wave — 8 skills retired/merged, 66 -> 58, spine 15 -> 13 (age-skills-audit-fable-l6ic.12)
Executes the age-e3zk decision (fresh disposition pass: docs/audits/ skills-audit-2026-07-06.md; council 2026-07-06 parked this behind age-p2c7, which landed as d7f950ca8; operator directive 2026-07-07 authorized finishing all filed work). RETIRED: red-team (validate --debate absorbs), perf (frontier-generic, zero repo bindings), flywheel (ao flywheel status CLI is the surface). MERGED: eval-outcomes -> validate (--mode=pre-impl --target=scenario), review -> validate (--mode=pr), compile + curate -> post-mortem (mining half; mechanical surfaces stay ao compile / ao lookup), recover -> status (--recover mode; deep playbook preserved at status/references/ recovery-playbook.md; delivers l6ic.7 and the l6ic.13 row reconcile). Mechanics: ao skills retire x8 (trees incl. images/*/skills, terminal ledger rows via --into, .agy-plugin review bundle removed); absorption tombstones in docs/SKILLS.md + SKILL-TIERS.md (56 user-facing + 2 internal = 58, counted); spine gate re-anchored 15 -> 13 (l6ic.11 minimal consistency: review + red-team leave; membrane 7 + bookkeeper 6); bespoke twins post-mortem/status hand-mirrored per AGENTS-CODEX; manifest pruned of orphan twin rows (62 -> 57); overrides catalog pruned of 8 retired rows; gemini verify core_skills + claude/codex image manifests updated; retired-subject eval red-team-adversarial-validation.json removed (not canary-listed). Gates: spine-integrity 13 PASS; wiring-closure PASS; skill-frontmatter 58/58; codex manifest+artifacts+parity PASS; regen-check ALL GREEN; SKILL-TIERS also fixed Opus 4.6 -> 4.8, GOALS.yaml -> GOALS.md, flywheel diagram to CLI truth (l6ic.6 pre-work). Residual prose mentions in non-gated docs are the l6ic.2 debris sweep's scope. |
||
|
boshu
|
fe1d8fee8c |
feat(pawl): patch-id REBOUND verdicts — authorize a byte-identical rebase without a full re-review, gate-green + lineage required (age-rk3r.9)
# Conflicts: # skills/catalog.json |
||
|
boshu
|
3c4e4d6d54 | feat(pawl): source per-repo verify config once at pawl-review entry (age-rk3r.17) | ||
|
boshu
|
b4c74afd02 |
feat(pawl): reviewer adapter contract — codex byte-compatible, agy cold, local-mlx eval-only; folds age-iian timeout re-snapshot (age-rk3r.1)
Post-refutation fixes (cross-family pawl caught two defects in the first cut): - DEFECT 1 (fail-open): the agy/local-mlx genuine-run marker 'VERDICT:' appears in the packet itself, so a run that merely cat/echoed the packet classified GENUINE and the last-verdict parser could extract a CONFIRMED from echoed diff-context lines. Fixed: sentinel-wrapped agy packets (random-nonce boundary lines) + a marker-independent packet-line containment check (reviewer_packet_echoed) classify packet echo as ECHO. - DEFECT 2 (wrong family certified): the verdict write hardcoded --refuter codex:...; REVIEWER=agy now certifies the canonical 'gemini' family (roster: agy|google->gemini), codex stays byte-identical, local-mlx gets a deliberately off-roster label so an opted-in eval verdict can never pass the prod roster check. |
||
|
boshu
|
dbbcbde1b8 |
feat(gates): preamble ratchet — new scripts must source the hardened preamble, grandfathered tree only shrinks (age-gate-the-ungated-egwt.10)
Advisory-first rollout, deliberately: shell.preamble-ratchet seeds Blocking:false for ONE clean cycle on main, then flips Blocking — the same staged pattern the epic already landed for docs.cli-snippets (egwt.4) and docs.demoted-claims (egwt.6), recorded in the RepairHint. The backing check itself is fail-closed (exit 1) on every violation class: - new/changed top-level scripts/*.sh must source scripts/lib/preamble.sh or carry a non-empty '# preamble-exempt: <reason>' (empty reason rejected) - grandfather authority = base-ref ∩ working snapshot, so appending an entry in the same diff grants nothing (both halves fail independently) - the snapshot is shrink-only vs. its base-ref version; only the initial snapshot commit may introduce it - scripts/lib/** exempt as a class (sourced libs, not entrypoints) Also folds with_tmpdir (trap-EXIT mktemp cleanup, ~79 re-rolls) and require_cmd (command -v + install-hint die, ~62 re-rolls) into the preamble; 18 preamble bats + 10 ratchet bats. |
||
|
boshu
|
322d419572 | feat(ci): verdict backstop — report-only tip-verdict verification reusing the pre-push waiver (age-wedge-all-in-dyr0.9) | ||
|
boshu
|
55186e5460 | refactor(scripts): codex-exec lib — one fail-closed hardened runner, migrate all non-pawl callers (age-gate-the-ungated-egwt.8) | ||
|
boshu
|
b7503b4519 | feat(gates): docs.cli-snippets gate — resolve ao commands in live docs against the cobra tree (age-gate-the-ungated-egwt.4) | ||
|
boshu
|
38aa215222 | feat(gates): extract docs-scope lib — shared live-doc resolution + banner exemption (age-gate-the-ungated-egwt.1) | ||
|
boshu
|
0096a050c3 |
refactor(skills): demote evolve/autodev/acfs to a new experimental tier (age-focus-membrane-bookkeeper-m1wg.21)
Heavy legacy RPI chains with no measured uplift. Add a new `experimental` tier and demote the three. The tier value is registered at ALL FOUR enum sites — the authoritative schemas/skill-frontmatter.v1.schema.json (enforced by the blocking skill.manifests gate), validate-skill-schema.sh, sku_catalog.py, and generate-registry.sh — plus a Tier Values row in SKILL-TIERS.md. Frontmatter metadata.tier and the SKILL-TIERS row cell flipped in lockstep for all three. acfs is demoted, NOT cut: its cut is external (~/acfs) and a deferred operator one-way-door decision (noted in the disposition ledger). Codex twins refreshed by regen (not dropped — .18 owns twin cuts). registry drift, manifests schema, sku-catalog, sync-counts all green; go build + internal tests pass. |
||
|
boshu
|
974f351d6f |
feat(scripts): add scripts/lib/preamble.sh — strict-mode + REPO_ROOT + portable stat/find (age-0dq9.1)
Sourced library centralizing two recurring portability hazards (recon P6a): the macOS find->bfs shim (-printf support diverges from real /usr/bin/find) and stat -f %m (BSD) vs stat -c %Y (GNU). Provides strict mode, REPO_ROOT (git toplevel + non-git fallback), portable_mtime, portable_find (real system find), newest_by_mtime (global sort, not find-exec-ls), newest_in_dir. Additive only — callers migrate in P6b (deferred). bats covers the real-find path + proves portable_find bypasses a broken PATH find; shellcheck -S error clean. Closes age-0dq9.1 |
||
|
Boden Fuller
|
cf90ec392a |
feat(skills): consolidate beads 4->2 — retire beads umbrella, fold lifecycle doctrine into beads-br (ag-ez7y6 #beads-consolidate-4to2)
Salvaged the uncommitted work from wt-ag-ez7y6 and completed the 4->2 beads skill consolidation the salvaged test asserts: - Fold the "Issue-Lifecycle Discipline" doctrine (live-reads-authoritative, scoped-closure-proof, parent-reconciliation, narrow-the-umbrella-issue, normalize-stale-queue) from the retired `beads` umbrella into beads-br. - Retire skills/beads --into beads-br via `ao skills retire` (trees removed, ledger row moved to historical: merged-into beads-br). - Re-target every retired-skill reference: crank/implement/plan/post-mortem/ refactor frontmatter+deps, SKILL-TIERS rows, docs/SKILLS.md, GLOSSARY link, sku_catalog LOOP_MOVES[2], image manifests (counts adjusted). - Codex twin: mirror the doctrine into skills-codex/beads-br, prune `beads` from the codex catalog/manifest, regen hashes (parity gate age-yxl). - Add cli/cmd/ao/beads_consolidation_test.go (L2 structural invariants). Uses a leak-proof repo-root finder (skills/+docs/contracts/) because the shared findRepoRoot keys on .agents/, which the cmd/ao suite leaks into the pkg dir. Fixed an ao-skills-retire bug surfaced here: the historical row was appended at the end of the `workflows:` section (between historical: and dispositions:) instead of inside historical:. Corrected manually; full fix tracked separately. Bounded-context: BC3-Loop Evidence: cli/cmd/ao/beads_consolidation_test.go Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Boden Fuller
|
692f420ac3 |
feat(contracts): additive artifact-classification schema + BC6 Orchestration (ag-4akl8 #backfilled-fields)
S0 foundation slice (v4 additive, no-rename — quorum-ratified). Adds the artifact-classification fields to every active `- skill:` row + workflows entry and introduces BC6 Orchestration, without renaming the ledger file, the `- skill:` row key, or the `domain: "BCn Name"` string format. - skill-dispositions.yaml: backfill kind/runtime_targets/parity_policy/ capability_class/path/aliases/supersedes on all 71 active rows + 4 workflows; re-bin the 6 orchestration skills (ntm/swarm/agent-mail/using-atm/ vibing-with-ntm/continuity-loop) into BC6. - bounded-contexts.yaml: add BC6 Orchestration + OrchestrationPort. - sku_catalog.py: BC6 enum + cli-command-per-BC carve-out (require >=1 active artifact per BC; cli coverage only where a BC owns commands). - generate-registry.sh: emit the new fields (+ derived runtime_reach) and a first-class `workflows` surface. - generate-skill-domain-map.sh / check-bounded-contexts-drift.sh: BC-count sites 5->6; both registry docs carry BC6. - new gate validate-skill-disposition-schema.sh (+ bats) rejects unknown kind/capability_class and missing fields, naming the offending row; wired into regen-all, pre-push-gate, and the Go gate registry. Renamed-consumer parsers (resolve-skill-path.sh, skills_retire.go, heal.sh) and the `- skill:` fixtures are byte-untouched. Closes-scenario: ag-4akl8#backfilled-fields-preserve-the-skill-count Bounded-context: BC4-Factory Evidence: docs/contracts/skill-dispositions.yaml Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Boden Fuller
|
85b7b97b1d |
fix(scripts): validators resolve skill paths via dispositions ledger (ag-2vz5v)
Shared scripts/lib/resolve-skill-path.sh routes skills/<slug>/ and skills-codex/<slug>/ paths through the historical: section of docs/contracts/skill-dispositions.yaml: merged-into rewrites the slug segment to the target, cut skips visibly (stderr warn), no row / unknown slug / missing ledger degrade to identity. Exact-slug awk parse (plan does not match plan-foundry). The 4 validators that hardcoded skill paths (rpi-contract, lifecycle-guards, cold-start, pre-push-gate) route their fixed entries through it so folds/cuts auto-retarget instead of needing hand-repointing. Identity-preserving on the current repo (no historical row matches any routed active skill): 3 standalone validators byte-identical before/after; 16 bats incl. 4 validator-integration cases. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Bo
|
092b99dad9 |
feat(skills): add skill-eval.sh ms-backed quality gate (ag-yzoz #skill-eval) (#726)
## What
Adds `scripts/skill-eval.sh` — a thin gate that runs one skill's
`SKILL.md` through Jeff Emanuel's `ms` (meta_skill) `lint` + `validate`.
The wrapper decides the gate purely on whether a **blocking** `rule_id`
appears (ms's own severity/exit is not trusted):
- Blocking: `no-secrets`, `no-injection`, `safe-paths`,
`required-metadata`, `no-cycle`, `valid-version` → non-zero exit.
- Everything else → annotated only (`::warning::`), never fails.
- `ms` not on PATH (or `MS_BIN` missing) → **loud `::error::` hard-fail
(exit 3)**, never skip-and-pass — a silent skip would recreate the
no-evaluation gap this gate closes.
Planted `skills/_fixtures/{bad,good}-skill/SKILL.md` fixtures +
`tests/scripts/skill-eval.bats` (10 tests) exercise the
bad/good/ms-absent contracts.
## Fixture scoping
`skills/_fixtures/` is non-skill scaffolding (planted fixtures). The
skill-enumerating gates/generators are taught to skip `skills/_*` so the
fixtures never inflate the skill count (stays 82) or fail registry /
schema / wiring / runtime / count gates. Touched enumerators:
`generate-registry`, `validate-skill-schema`, `sync-skill-counts`,
`validate-skill-count`, `generate-skill-catalog`,
`generate-context-map`, `validate-manifests`, `check-wiring-closure`,
`validate-headless-runtime-skills`, `lib/sku_catalog.py`,
`check-alias-collisions`, `run-all`, `test-runtime-opencode-smoke`,
`release-smoke-test`, `smoke-test`, `validate-skill`.
Does **not** touch `validate.yml` — wiring skill-eval.sh into CI is a
separate bead.
Epic: ag-czzf
## Verification
- `bats tests/scripts/skill-eval.bats` → 10/10 pass (ms 0.1.2 on PATH)
- `shellcheck scripts/skill-eval.sh` → clean
- `check-registry-drift`, `validate-skill-schema`, `sync-skill-counts`,
`run-all`, opencode-smoke → all PASS with fixtures present, skill count
= 82 (fixtures excluded)
Closes-scenario: ag-yzoz#skill-eval
Bounded-context: BC2-Validation
Evidence: scripts/skill-eval.sh
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
|
||
|
Bo
|
e90d6d79f9 |
feat(registry): SKU capability catalog (schema v2) — generated skill×command×BC join (ag-cbm) (#525)
## What Extends `registry.json` into a **generated SKU capability catalog** (schema_version 2) — the flagship 3.0 inventory feature. It is a 4th DERIVED projection (alongside DDD-vocab / Hex-frontmatter / Gherkin-acceptance), a JOIN of existing sources — **never hand-authored** (repo bans hand-edited inventory maps). Oracle-validated design: `.agents/discovery/2026-05-24-capability-inventory-and-gaps.md`. ## Catalog 214 SKU entries: **81 skills + 66 cli-commands + 64 gates + 3 reference-impls**. Each entry: `sku` (stable id), `name`, `type`, `bounded_context`, `hex_role`, `tier`, `purpose`, `status` (active | deprecated | planned | alias-of:<sku>), `disposition`, `consumes`, `produces`, `drives_commands`, `driven_by_skills`. `registry.json` schema v2 is a strict **superset** of v1 — existing `summary`/`surfaces`/`cadence_recommendations` consumers (the Go `ao registry` command) keep working; new `capabilities`/`capability_summary`/`cli_top_level_commands` blocks are added. ## The join key — `drives_commands` Derived from each skill **BODY** (SKILL.md + `references/*`), resolved against the live cobra tree (`scripts/lib/sku_extract.py`). Stale refs that resolve to nothing (e.g. the removed `ao schedule`) never become edges. `driven_by_skills` is the cmd→skill reverse. This closes oracle gap #1 (no skill↔command linkage anywhere). ## New gate — `validate-sku-catalog-drift` Three required checks: **(a) drift** (regenerate + diff), **(b) linkage integrity** (every `drives_commands` edge resolves to a real `ao` command — closes gaps #1/#2/#3), **(c) coverage** (every BC + every operating-loop move has an active skill; every BC has a cli-command). Wired into `validate.yml` + the verdict aggregator. ## Stale refs fixed `ao schedule` in `skills/crank` + `skills/rpi` best-practices and `agentopsd` in `skills/swarm` shared-checkout-discipline → reframed to the hookless / out-of-session (cron-driven `ao rpi`/`ao evolve` via Gas City) model. Disjoint from PR #524 (which fixed the hooks/inject refs); rebased on it — `validate-skill-cli-snippets.sh` is now 0. ## Retired The bogus "163 cli_commands" count (Go-file count, oracle gap #6) → **66 real top-level cobra nodes** from the live tree. ## Gates (all green locally) registry-check · validate-sku-catalog-drift (3/3) · validate-context-map-drift · skill-domain-map-golden · bounded-contexts-drift · codex-parity-drift · regen-codex-hashes --check · contract-compatibility (sku-catalog.md catalogued) · doc-release · validate-skill-cli-snippets (0) · skill-frontmatter · heal --strict · markdownlint (0) · shellcheck (0 errors) · mkdocs --strict · `go test ./cmd/ao` (9207) · new bats suite (5/5). Closes-scenario: ag-cbm#sku-catalog Bounded-context: BC1-Corpus Evidence: .agents/discovery/2026-05-24-capability-inventory-and-gaps.md |