152 Commits

Author	SHA1	Message	Date
Bo	bb8d747d3a	feat(ao): add harness-neutral bead ports (#810) ... ## Summary - add harness-neutral BR/git ports: `ao ready`, `ao close ...`, and bare `ao claim <id>` while preserving `ao claim bind/list` - keep top-level gate commands from #809 as the shared gate surface; `ao close` reuses the typed commit-persist close path and returns success when the ledger already proves the bead is closed - regenerate command docs, command surface inventory, matrix counts, and registry metadata ## Validation - `go test ./cmd/ao -run TestTick -count=1` - `bash scripts/regen-all.sh && bash scripts/regen-all.sh --check` - `go test ./cmd/ao -count=1` - `go vet ./...` - `go test ./... -count=1 -short` - `git diff --check` - `/tmp/ao-cp-ljp ready` from `/Users/bo/dev/control-plane` returned `state_source=br`, `ready_count=20`, and counts `{ready:20, open:30, in_progress:1, closed:19}` - `/tmp/ao-cp-ljp claim cp-ljp` from `/Users/bo/dev/control-plane` succeeded and did not dirty the Git ledger - `PRE_PUSH_SKIP_MKDOCS=1 bash scripts/pre-push-gate.sh --fast` passed; warning-only executable-spec drift remains pre-existing (`scenarios --lint`, `trace --orphans`) BR: cp-ljp	2026-06-07 02:03:39 -04:00
Bo	a617df3028	feat(ao): expose membrane gate aliases (#809) ... ## Summary - expose the membrane gate surfaces as top-level `ao` aliases: `verdict-gate`, `council-gate`, `install-guards`, `guard-status`, and `chaos-test` - keep the existing `ao tick ...` implementation as the shared backend, so the shell-oracle port and public aliases cannot drift - regenerate CLI command docs, surface inventory, matrix counts, and registry metadata ## Validation - `go test ./cmd/ao -run TestTick -count=1` - `bash scripts/check-cmdao-surface-parity.sh --write-surface` - `bash scripts/regen-all.sh --check` - `go test ./cmd/ao -count=1` - `go vet ./...` - `go test ./... -count=1 -short` - `/tmp/ao-cp-ltj chaos-test` from `/Users/bo/dev/control-plane` => `SMOKE PASS` - `./tick.sh smoke` from `/Users/bo/dev/control-plane` => `SMOKE PASS` - top-level alias exit matrix from `/Users/bo/dev/control-plane`: `council-gate` mixed => 8, unverified => 6 - `PRE_PUSH_SKIP_MKDOCS=1 bash scripts/pre-push-gate.sh --fast` passed; warning-only executable-spec drift remains pre-existing (`scenarios --lint`, `trace --orphans`) BR: cp-ltj	2026-06-07 01:45:16 -04:00
Bo	9ddffab410	feat(ao): add typed tick oracle port (#808) ... ## Summary - add `ao tick` as a typed Go port of the control-plane tick oracle - preserve BR/git boundaries for next/status/claim/reopen/close and port verdict/council/guard/smoke behavior - update generated command inventory, registry, CLI docs, and command-surface fixtures ## Validation - cd cli && go test ./cmd/ao -run TestTick -count=1 - cd cli && go test ./cmd/ao -count=1 - cd cli && go vet ./... - cd cli && go test ./... -count=1 -short - bash scripts/check-cmdao-surface-parity.sh - bash scripts/regen-all.sh --check - git diff --check - PRE_PUSH_SKIP_MKDOCS=1 bash scripts/pre-push-gate.sh --fast - built /tmp/ao-cp7k6 and compared against /Users/bo/dev/control-plane/tick.sh for verdict-gate, council-gate, next; `ao tick smoke` returned SMOKE PASS Note: fast pre-push gate still reports existing warn-only executable-spec drift for scenarios --lint and trace --orphans.	2026-06-07 01:26:56 -04:00
Bo	8ad6747913	chore(ao): delete the ao evolve CLI surface (ag-llni slice 1) (#724) ... **Slice 1 of ag-llni** — retire `ao rpi`/`ao evolve` (orchestration is no longer an in-CLI engine). This PR removes the **`ao evolve`** half; the `rpi*.go` + phased web is slice 2. **Architecture (operator-confirmed 2026-06-04):** orchestration is two-tier — *inner loop* (one session) = skills/CLIs/native-subagents + **Workflows** (`.claude/workflows/operating-loop.js` + `bead-crank.js` — the Claude encoding of inner-orchestration that was mis-shaped as `rpi`/`evolve`); *outer loop* (many sessions) = **NTM + mcp-agent-mail**. Codex inner-loop = NTM + agent-mail for now. Leaf CLIs (`corpus`/`goals`/flywheel/knowledge loops = BC1 corpus), NTM, and agent-mail all stay. ### Changes - Deleted `ao evolve` + 4 subcommands (`config`, `next-work`, `blocked`, `write-stop-marker`) — 10 files. - Relocated the one non-evolve coupling, `countCronHistoryRows`, into its sole consumer `cron_self_adjust.go`. - Removed `evolve` from the `cobra_commands_test.go` golden `expectedCmds` list (2 sites). - Regenerated `cli/docs/COMMANDS.md` (−125 lines) + `docs/cli-skills-map.md`. ### Verification - `go build ./...` green · `go vet ./cmd/ao` clean · **`go test ./cmd/ao` = 9251 pass**. - Net: 13 files, −2192 LOC. Bounded-context: BC5-Runtime Closes-scenario: ag-llni#delete-evolve-cli Evidence: cli/cmd/ao/cron_self_adjust.go Test-Removal-Reason: ao evolve CLI surface retired (ag-llni) — orchestration moved to .claude/workflows + NTM; the 5 deleted evolve_*_test.go covered now-deleted command code, not kept behavior. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>	2026-06-05 00:21:53 +00:00
Bo	3d99d4bf69	fix(corpus): unify secret redaction + compile/CI git-safety (ag-sz3h ag-zi90 ag-y2xy #w1-git-safety) (#720) ... Completes the **W1 git-safety arc** of epic ag-lhu0 (memory-system re-architecture) — three beads, one coherent surface, after #719 wired scrub-on-write into corpus_fs. - **ag-zi90 (dedup):** `redactContent` (`ao context assemble`) now runs canonical `llm.Redact` first — a single source of truth for credential patterns (AWS / GitHub / GitLab / Slack / GCP / PEM, previously uncovered in this path) — then keeps the supplements `llm.Redact` does *not* cover: 2-segment app JWTs, sensitive env-var assignments, high-entropy strings. - **ag-sz3h (compile chokepoint):** new `ao redact` (stdin→stdout canonical scrub) bridges the shell compile path; `compile.sh` pipes every rendered article through it before `os.WriteFile` (loud fallback if `ao` absent). - **ag-y2xy (CI backstop):** `check-corpus-secret-scan.sh` scans **git-tracked** (committed/allowlisted) `.agents` corpus + canon only — repo-root `.agents` is local-only by policy, so scrub-on-write (#719 + `ao redact`) is the primary defense and this is the backstop. Wired into `validate.yml`; bats covers clean / leak-with-file:line / untracked-not-scanned. Scrub-on-write is primary; CI is the backstop. Verified: go build/vet/test (incl. `TestRunRedact`, JWT-dedup regression test green), bats 3/3, shellcheck clean, conformance regenerated. Closes-scenario: ag-sz3h#compile-render-scrub Closes-scenario: ag-zi90#dedup-redactor Closes-scenario: ag-y2xy#ci-secret-scan Bounded-context: BC1-Corpus Evidence: scripts/check-corpus-secret-scan.sh --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>	2026-06-04 14:11:55 -04:00
Bo	215fcef720	feat(eval): held corpus-delta tasks cd-ci-1 + cd-ci-4 (ag-nfux #approved-tasks-ci1-ci4) (#703) ... ## What Materializes the **two operator-vetted** W1b tasks (the only 2 the adversarial audit passed as clean: `.agents/`-grounded, principle-not-answer, deterministically gradeable). - **cd-ci-1** — author an advisory-tier guard gate. Grounds in `.agents/learnings/2026-05-17-ci-advisory-killed.md`; the agent must build `check-no-advisory-tier.sh`. - **cd-ci-4** — author a removed-job assertion pre-flight gate. Grounds in `.agents/learnings/2026-05-17-delete-grep-asserts.md`; the agent must build `check-removed-job-assertions.sh`. Each task ships `prompt.md` + `setup.sh` + `score.sh` (deterministic, no-LLM grader) + a `golden-solution.sh` reference (**not** given to the agent). The grounding learning lives in `.agents/`, so the W1a sandbox (ag-5apc) surfaces it in `context_on` and strips it in `context_off` → a real delta. ## Grader-discrimination test The task fixtures' own validity: each grader scores the **golden solution PASS** and an **empty workspace FAIL** — a grader that can't tell the difference would make the A/B meaningless. 5/5 bats pass; shellcheck clean on all 6 task scripts. ## N note This is the honest **N=2** starting set. The adversarial audit showed N≥10 isn't reachable from the first drafting slate without a second `.agents/`-only pass; a rigorous small-N beats a large rigged one (per the product panel). **ag-nfux stays open** for N expansion. Closes-scenario: ag-nfux#approved-tasks-ci1-ci4 Bounded-context: BC2-Validation Evidence: evals/workbench/tasks/cd-ci-1/score.sh --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>	2026-06-03 19:09:36 +00:00
Bo	3702761e75	feat(canon): earned-promotion team-knowledge tier (ag-bw5s #earned-promotion-gate) (#672) ... ## What A team-knowledge **canon**: learnings *earned* into a shared trusted set, not self-asserted. Adapts Doug Dan's warmind (#671) to the post-ACFS direction — moat = knowledge earned by verification, not self-certification. Promotion requires **two independent, self-guarded signals**: - **citation** — another engineer/agent USED it (proves useful) - **verification** — another engineer/agent CHECKED it (proves true; receipt-bearing) An independent refutation hard-blocks. The anti-self-certification guard is generalized to *all* promotion signals: the factory may not grade its own homework. ## Slices (3 commits, one coherent arc) 1. **Earned-promotion gate** — `internal/canon` (identity guard, citation + verification JSONL ledgers, gate); `ao canon cite/verify/status/promote`. 2. **Inject trusted tier** — `collectLearnings` surfaces `.agents/canon/learnings/` deduped + boosted 1.15× over equal-utility local; flagged `Canon`. 3. **IdentityResolver port** — `--as` > `AGENTOPS_ACTOR` env > git. Env tier is the seam to Agent Mail (`whois`) with **no MCP coupling** — `ao` stays portable. Makes the cross-actor guard hold for humans AND agents. ## Deliberately dropped from #671 Resurrected `hooks/` + session-end auto-sync (hookless regression), the "Gas City Dispatch (CRITICAL)" doctrine (contradicts prune-the-coordination-plane), the parallel scoring/maturity engine (duplicated inject primitives), and the "warmind" name. ## Verification - `go build`/`vet` clean, gofmt clean - 9,388 tests green (`cmd/ao` + `internal/canon` + `internal/search`) - e2e: apollo self-attesting excluded; witness (distinct agent via `AGENTOPS_ACTOR`) earns promotion; `ao inject` ranks canon first at equal utility ## Deferred (next) Wire `verify` → `ao verify`/council (currently operator-attested — the load-bearing gap); tier-aware gate (heuristics earn citation, not verification). Closes-scenario: ag-bw5s#earned-promotion-gate Closes-scenario: ag-bw5s#inject-trusted-tier Closes-scenario: ag-bw5s#agent-identity-resolver Bounded-context: BC1-Corpus Evidence: cli/internal/canon/promote.go	2026-06-01 00:57:12 -04:00
Bo	486cb99be5	feat(turn): ao turn verify Evidenced-Turn DoD predicate (ag-lmdx.5 #ao-turn-verify) (#657) ... ## What Adds `ao turn verify <bead>` — a legible Definition-of-Done predicate for an **Evidenced Turn** (ag-lmdx.5). Where the live AP#7 gate checks Evidence *presence*, this checks turn *sufficiency*: one verdict answering "can this bead legally transition validated->closed?" with a clear pass/fail per predicate and a reason for every gap (`--json` too). ## Definition of "Evidenced Turn" A turn is DONE iff all six predicates pass: | predicate | meaning | |---|---| | `chain_intact` | the bead's state_transition log hash-verifies and folds (turnstate `FoldVerified`) | | `terminal_state` | the folded lifecycle state is at the validated->closed boundary | | `scenarios_covered` | every Closes-scenario claim has a passing test | | `evidence_resolves` | every Evidence line resolves to a gate log that exercised THAT scenario (presence->sufficiency) | | `provenance_event` | a provenance edge in the ledger references the bead | | `no_orphan` | no artifact in the turn's trace-graph is a provenance orphan | ## How - New pure, in-memory predicate core `cli/internal/evidencedturn` (no Dolt / no shell-out) — the testable kernel that defines what "done" MEANS. - Builds directly on **#654** `cli/internal/turnstate` (`Seal/Append/Fold/FoldVerified/VerifyChain`) and `cli/internal/provenancegraph` (edges + `FindOrphans`). - `cli/cmd/ao/turn_verify.go` wires `--input` (turn-input JSON: state log + scenario coverage), `--ledger` (provenance EDGE ledger), and `--graph` (optional trace-graph for orphan audit). Non-zero exit on not-done makes it usable as a transition guard. ## Scope note (ag-lmdx.4) The author!=validator guard is a **separate** concern and is NOT implemented. It is left as a documented extension point (TODO on `Evaluate`); the verdict shape already accommodates an extra predicate row without a reshape. ## Tests `cli/internal/evidencedturn` (14 cases: complete-turn-done, each missing predicate fails with a legible reason, multi-gap, blank-bead error) + `cli/cmd/ao/turn_verify_test.go` (complete/incomplete/json/orphan/no-graph/missing-input/bead-mismatch). `go build ./...`, `go vet`, full `cmd/ao`+`evidencedturn` suites (9288) green. ## Derived surfaces regenerated COMMANDS.md, cobra `expectedCmds`, command-surface smoke (top=75 sub=188 all=263) + matrix.json, registry.json (cli_commands 72->73), cli-skills-map. Closes-scenario: ag-lmdx.5#ao-turn-verify Bounded-context: BC4-Factory Evidence: cli/internal/evidencedturn/evidencedturn.go	2026-05-31 14:40:05 -04:00
Bo	a6336b7c34	feat(provenance): ao provenance trace --orphans --strict gate (ag-x31t.6 #provenance-orphan-gate) (#653) ... ## What Adds `ao provenance trace --orphans --strict`: detects provenance orphans — artifact nodes with **no inbound authored/inferred edge** — by generalizing the `goals_trace_orphans` no-inbound chain-gap detection onto the provenance graph. Wires it as a blocking CI gate. ## How - **`cli/internal/provenancegraph/orphans.go`** — `ReadGraphRecords` (parses the goalstrace `Node`/`Edge` JSONL contract that the seeded fixtures use) + `FindOrphans` (an artifact node is an orphan iff no edge's `to_id` targets it; it flips green the moment any inbound edge is added). - **`cli/cmd/ao/provenance_trace.go`** — the `trace --orphans [--strict] [--json] [--graph <path>]` subcommand. `--strict` exits non-zero when orphans exist; `--json` emits one finding per line. - **`scripts/check-provenance-orphans.sh`** + a new **blocking** step in `.github/workflows/validate.yml` (in the existing goals/spec-linkage job, right after the warn-only `ao goals trace --orphans`). The gate asserts the strict audit **catches** each seeded orphan fixture and **passes** once an inbound edge wires the artifact back to a directive — so it is deterministically green while proving the detector is wired. ## Tests (TDD-first) - `cli/cmd/ao/provenance_trace_test.go` (L2): drives the real `tests/fixtures/provenance/` fixtures against `expected-orphans.json` — `--strict` catches each of the 3 seeded orphans (`gate:scenario-hash-stability`, `artifact:scripts/pre-push-gate.sh`, `claim:65-jobs`), and a wired graph exits 0. Plus mode/flag-guard cases. - `cli/internal/provenancegraph/orphans_test.go` (L1): detection, non-artifact nodes never orphaned, deterministic sort, JSONL parsing + malformed/missing-file rejection. - `tests/scripts/check-provenance-orphans.bats`: gate-script behavior. ## Derived surfaces regenerated - `cli/docs/COMMANDS.md` (new `ao provenance trace` subcommand, cobra-conformance passes) - CLI surface smoke fixture + matrix (sub 186→187, all 260→261) - `registry.json` **unchanged** (provenance is already a top-level command; subcommands aren't counted) — timestamp-only churn reverted. ## Gates `go build ./... && go vet ./... && go test ./...` (11959 pass) · gosec clean on new files (one `#nosec G304` on the operator/CI-supplied graph path, with reason) · `scripts/validate-ci-policy-parity.sh` PASS. Closes-scenario: ag-x31t.6#provenance-orphan-gate Bounded-context: BC4-Factory Evidence: .github/workflows/validate.yml	2026-05-31 14:07:45 -04:00
Bo	2fecf2b304	feat(provenance): ao provenance export deterministic hash-chained (ag-x31t.5 #provenance-export) (#651) ... ## What Adds `ao provenance export`: a deterministic, hash-chained rendering of the committed provenance ledger (`docs/provenance/ledger.jsonl`). - Reads the ledger, **canonically sorts** edges by `(ts, from_id, to_id, relation)` (with type/trust/evidence tie-breakers for a total order), then **re-seals** them into a fresh per-record hash chain. - Output is **byte-identical on re-run** regardless of the ledger's physical append order. Default = JSONL (one compact edge per line); `--json` = indented array; `--verify` = one-line OK summary, no varying body. - The re-chained export **verifies with no Dolt server** — the committed JSONL is the audit authority, and re-chaining uses only the in-process hashing in `cli/internal/provenancegraph` (same `prev_hash` discipline as the rpi ledger). No reinvented Edge/hash logic: new `CanonicalSort`/`ReChain` helpers reuse `Seal`/`VerifyChain`. ## Tests (TDD-first) - `chain_test.go`: canonical-sort ordering + stability + non-mutation, order-independent re-chain, byte-identical serialization, empty ledger, invalid-edge rejection, tamper detection. - `provenance_export_test.go`: deterministic bytes across runs, chain verifies, empty ledger (`[]` not `null`), `--verify` summary, tampered-ledger rejection. ## Derived surfaces regenerated - `cli/docs/COMMANDS.md` (cobra conformance) - `evals/agentops-core/fixtures/cli-command-surface-smoke.sh` + `cli-command-surface-matrix.json` (sub 185→186, all 259→260) - `registry.json` unchanged: it counts top-level commands only (subcommands not tracked; top-level count = 72). ## Gates `go build` / `go vet` / `go test ./...` green; gosec clean on the new files; CLI-skills-map, JSON-flag-consistency, and `generate-cli-reference.sh --check` all pass. Closes-scenario: ag-x31t.5#provenance-export Bounded-context: BC4-Factory Evidence: cli/cmd/ao/provenance_export.go	2026-05-31 13:34:45 -04:00
Bo	427c25b248	feat(provenance): provenance_edges write-model + ao provenance add (ag-x31t.4 #provenance-write-model) (#649) ... ## What Builds the WRITE side of the SDLC provenance/intent graph (ag-x31t slice 1). Adds `ao provenance add` / `ao provenance list` plus a new `cli/internal/provenancegraph` package that seals typed, evidence-backed edges onto the per-record hash-chained ledger at `docs/provenance/ledger.jsonl`. Consumes the schema merged in #637 (`schemas/agentops-sdlc-provenance.v1.schema.json`). Does not touch the schema (.2), export (.5), or gate (.6). ## CQRS doctrine Per CLAUDE.md and the council architecture, the committed JSONL ledger is the **audit authority and source of truth**; any Dolt `provenance_edges` table is a rebuildable projection that loses on disagreement. So this command appends the JSONL ledger directly. Hashing reuses the `cli/internal/rpi/ledger.go` discipline: `payload_hash = sha256(canonical payload)`, `hash = sha256(payload_hash + "\n" + prev_hash)`, genesis `prev_hash=""`. ## Behavior - `ao provenance add <from-id> <to-id> --relation ... [--from-type --to-type --trust-tier --evidence --ts --json]` — seals + appends one schema-valid edge. **Idempotent** on edge identity (endpoints+relation+evidence+trust-tier), so a re-run with a different timestamp is a no-op. - `ao provenance list [--json --from-id --relation]` — reads edges back in chain order. ## Tests (TDD, table-driven, exact-value asserts) - `edge_test.go`: field/enum validation, deterministic hash chain, schema-version forcing, tamper + chain-link detection, identity stability, JSON field-name parity with the v1 schema. - `store_test.go`: append→read round-trip + persisted-chain verify, idempotency, reject-invalid-before-write (no file created), corrupt-line rejection, and **validation of emitted edges against the merged schema via `scripts/validate-provenance-ledger.sh`**. - `provenance_add_test.go`: add produces a schema-valid sealed edge + list reads it back, idempotent no-op, invalid-relation error, list filters. ## Gates - `cd cli && go build ./... && go vet ./... && go test ./...` — all green (11949 pass). - Command-surface bumped: regenerated `cli/docs/COMMANDS.md`, updated `cli-command-surface-smoke.sh` + `cli-command-surface-matrix.json` (top 73→74, sub 183→185, all 256→259), added `provenance` to cobra expectedCmds; `validate-cli-skills-map.sh` PASS. Closes-scenario: ag-x31t.4#provenance-write-model Bounded-context: BC4-Factory Evidence: cli/cmd/ao/provenance_add.go	2026-05-31 13:02:55 -04:00
Bo	1307eb0e0a	feat(skills): queryable skill catalog JSON + ao skills (soc-vuu6.4 #skill-catalog-json) (#640) ... ## What Slice 2 of the skill-catalog bead (soc-vuu6.4): the **queryable surface** over the generated `skills/catalog.json`. Slice 1 (the generator, schema, drift gate, and `catalog.json` itself) landed in PR #379; this adds the `ao skills` query commands the bead's acceptance criterion calls for. - New `cli/internal/skills/catalog.go` — loads `skills/catalog.json` and a **pure, table-tested** query engine (`List`/`Consumers`/`Producers`/`Mermaid`). No SKILL.md re-parsing; reads the committed, CI-synced catalog. - New `cli/cmd/ao/skills_query.go` — four cobra subcommands: - `ao skills list [--role --produces --consumes --practice --user-invocable] [--json]` - `ao skills consumers <skill> [--json]` - `ao skills producers <output> [--json]` - `ao skills graph [--format mermaid]` - Regenerated `cli/docs/COMMANDS.md`; bumped the cli-command-surface fixtures (`sub` 179→183, `all` 252→256) for the 4 new subcommands. ## Why `registry.json` and `catalog.json` existed but had no query API — agents grepping "which skill produces X?" had to scan markdown. This turns the catalog into a queryable surface (the bead blocks `ao skill new`, which needs sibling-skill defaults). ## Acceptance > `ao skills list ... --json` returns the matching skill set in <100ms. Verified: `ao skills list --produces result.json --json` → `[beads, discovery]`; `ao skills consumers rpi` → 5 skills; `ao skills graph` emits deterministic Mermaid. Latency well under 100ms. ## Scope notes - Did **not** regenerate `skills/catalog.json` — it carries pre-existing drift on `main` (the `check-skill-catalog-drift` gate is I0/advisory). This PR only *consumes* the catalog; regenerating would sweep in unrelated drift. - The bead's illustrative `--tier judge` example does not map to the real generated schema (which has no `tier` field — it uses `hexagonal_role`, `produces`, `consumes`, `practices`). Filters implemented against the actual catalog fields. ## Tests `cli/internal/skills/catalog_test.go` (engine, exact-value table tests + load round-trip) and `cli/cmd/ao/skills_query_test.go` (JSON shape, sort invariants, flag validation, mermaid header) — all green. `go build`/`go vet`/`go test ./...` pass; conformance + command-surface smoke pass. Closes-scenario: soc-vuu6.4#skill-catalog-json Bounded-context: BC1-Corpus Evidence: cli/internal/skills/catalog.go	2026-05-31 12:36:45 -04:00
Bo	4131e7c720	feat(rpi): materialize harvested next-work into durable beads (ag-9jle.3 #lessons-to-beads) (#633) ... ## What Closes the missing half of the BDD-Gherkin-wave loop — **lessons → beads**. Harvested follow-ups landed in `.agents/rpi/next-work.jsonl` as a queue but never became durable beads, so the flywheel executed without compounding into the tracker. New **`ao next-work materialize`**: - Reads unmaterialized items from `next-work.jsonl` and `bd create`s one durable bead each. - Carries `source_epic` + `proof_ref` on **bd's native `--metadata`** — *not* a forked provenance format. The real provenance graph edge is deferred to `ag-x31t.4`'s future `ao provenance add`. - **Idempotent** via the existing per-item `bead_id` back-reference. Re-runs skip already-materialized / consumed / held-for-review items. - Flags: `--file`, `--dry-run`, `--json`, `--source-epic`, `--materialized-by`. - `skills/post-mortem/references/harvest-next-work.md` now invokes it after the queue-write (CLI owns the deterministic core; the skill just calls it). ## First failing test (Gherkin contract) > Given a completed wave with a harvested follow-up · When materialize runs · Then a durable bead exists via `bd create` carrying `source_epic`/`proof_ref`, not only a `next-work.jsonl` queue line. `TestNextWorkMaterialize_CreatesDurableBeadWithProvenance` + 6 more (idempotency, skips-consumed/held, dry-run-no-mutation, source-epic filter, 2 mapping tables) — all green. ## Compass-not-maps pivots vs the locked plan Verifying each premise before executing (the W1-reconciliation overstated-scope lesson) caught two beats that were **already done**: 1. The schema fields `source_epic`/`proof_ref` already exist in `next-work-{item,batch}.v1.schema.json` — dropped the "add fields" beat. 2. The idempotency anchor `NextWorkItem.BeadID` already exists — no new field invented. ## Boundary Zero `ag-x31t` files touched; `validate.yml` untouched (parked per handoff). `ao next-work` is collision-free (distinct from the existing `ao evolve next-work` ladder recommender). ## Validation - `cli/cmd/ao` package: all tests green except pre-existing `TestCronSelfAdjust_RoundTrip` (macOS-only `/tmp` vs `/private/tmp` path bug — fails identically on clean `main`, passes on CI Linux). - `TestCobraConformance` + `TestCobraExpectedCmdsMatchRegistration` updated for the new command; `COMMANDS.md` regenerated. - `go vet` + `gofmt` clean. Closes-scenario: ag-9jle.3#lessons-to-beads Bounded-context: BC3-Loop Evidence: cli/cmd/ao/next_work_materialize_test.go	2026-05-30 13:55:39 -04:00
Bo	159d620506	fix(eval): unstale pre-push-gate-governance canary positional assertion (ag-teb1) (#625) ... ## What The `agentops-core.pre-push-gate-governance` contract canary (case `pre-push-bats-suite`) hardcodes the positional assertion `ok 45 pre-push-gate.sh test-home-isolation lint stub exists in fake repo (paired stub guard)`. A bats test was inserted before it, shifting that paired-guard test to **`ok 46`**. This bumps the expectation 45 → 46. ## Why it was latent (not caught earlier) The contract-canaries step is gated `if: changes.outputs.contracts || shell || ci`. Main's recent pushes didn't touch those paths, so the step was **skipped** (latent green at a5ecfa78 — verified via the jobs API). PR #624 (which adds a schema + contract + validator) triggers the step and surfaced the drift as a `contracts-sync` FAIL. **This blocks every contract/shell-touching PR until fixed.** ## Scope / safety - One-line change to one repo CI fixture. `evals/agentops-core/` is a **repo CI fixture, NOT** the locked `~/.agents/evals/` holdout substrate — a positional bump is sanctioned maintenance, not relitigating the locked schema. - The other three positional assertions (`ok 6/21/22`) are unchanged and confirmed still correct against current bats numbering. - `tests/scripts/pre-push-gate.bats` and `scripts/pre-push-gate.sh` are **not** touched. ## Evidence - Before: governance canary `status=fail aggregate=0.9549` (`stdout does not contain "ok 45 ...stub exists..."`) - After: `scripts/test-agentops-contract-canaries.sh --suite-list <gov-only>` → `status=pass verdict=pass aggregate=1` (failures=0) - `jq -e .` → valid JSON; diff is 1 insertion / 1 deletion Closes-scenario: ag-teb1#canary-positional-assertion-45-to-46 Bounded-context: BC4-Evaluation Evidence: evals/agentops-core/pre-push-gate-governance.json	2026-05-30 06:53:16 -04:00
Bo	c4552483be	feat(cli): ao mcp serve --print-tools — curated MCP tool-descriptor surface (ag-h1mk #mcp-print-tools) (#619) ... ## What New `ao mcp` noun + `serve` verb exposing **`--print-tools --json`** — the curated, read-mostly MCP tool surface a hosted/SDK Claude loop uses to orient and self-check. It's the descriptor `ao agent bundle --runtime managed` (#618) already references. **Claude-only** (Codex shells `ao` directly). **Slice 1 of ag-3ucpd** (split per the evolve scope filter — the full MCP server is genuinely oversized: a from-scratch JSON-RPC stdio transport + live integration test, no MCP library vendored). This ships the deterministic, high-value, fully-testable descriptor surface now; the live transport + transport-round-trip integration test is carved to the **ag-3ucpd** umbrella for a fresh context. - `mcpToolDescriptors()` — 6 curated tools (`session_bootstrap`, `inject`, `corpus_inject`, `standards`, `validate`, `goals_measure`) with name + description + input_schema + holdout-sensitive flag. - `mcpToolDenied()` — deterministic **NOT-ZDR holdout refusal**: a tool call whose args reference `holdout` / `ground_truth` / `.agents/evals` is denied (cloud MCP surface; eval substrate LOCKED; `AGENTOPS_HOLDOUT_EVALUATOR` does **not** unlock the cloud surface). - Bare `ao mcp serve` (no `--print-tools`) errors loudly pointing at the ag-3ucpd live-transport follow-up — never a silent no-op. ## Tests (TDD-first) `cli/cmd/ao/mcp_serve_test.go` — 6 cases: curated surface (6 tools, schemas present), holdout refusal (query + path-escape), clean-call allowed, `--print-tools` JSON shape, live-transport-not-yet error. `go vet` clean, gocyclo < 15, full `cmd/ao` suite **9210 pass**. ## New-`ao`-command surfaces (all 5 regenerated up-front) The corpus-encoded checklist from #618's three-round discovery — applied in one shot, expecting one-shot-green: 1. `expectedCmds` ×2 (`mcp` inserted alphabetically) · 2. `cli/docs/COMMANDS.md` (TestCobraConformance) · 3. `registry.json` (69→70 commands, 164→165 SKU) · 4. `cli-command-surface-matrix` eval fixtures (71/177/248 → 72/178/250; smoke PASS) · 5. `docs/cli-surface.md`+`.json` (`ao mcp serve` covered). Scope-only diff (10 files, all mcp-attributable; no unrelated drift). Closes-scenario: ag-h1mk#mcp-print-tools Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/mcp_serve_test.go	2026-05-30 01:51:19 -04:00
Bo	7fc46eb76e	feat(cli): ao agent bundle — emit runtime-specific AgentOps-native Agent definitions (ag-eguw0 #agent-bundle) (#618) ... ## What New `ao agent` cobra noun + `bundle` verb — emits a **runtime-specific Agent definition** that stitches the AgentOps skill set + the `ao` tool surface so an out-of-session loop (Managed Agent / Codex-NTM swarm) runs under the same guardrails. Third child of epic **ag-7s9fo** (after ag-2wln skill #616, ag-mptr CI gate #617). - `--runtime managed` → Managed Agents JSON (model + stitched instructions + `skills[]` + `ao` MCP tool descriptor; self-hosted-sandbox block on `--sandbox self-hosted`). - `--runtime codex-ntm` → NTM bundle (skills-codex/agent-native ref + pane bootstrap running `ao session bootstrap` + `ao inject`); no MCP — Codex shells `ao` directly. - Default skills: `session-bootstrap, standards, validation, provenance`. Flags: `--skills`, `--sandbox`, `--out`, `--json`. ## Security — HARD REFUSAL (NOT-ZDR) Managed Agents are **not** ZDR. `buildAgentBundle` refuses (non-zero exit + explicit message) if any selected skill **names/paths a holdout/eval surface** or its `SKILL.md` body carries a holdout marker (`private_holdout` / `ground_truth` / `holdout target`). `AGENTOPS_HOLDOUT_EVALUATOR` does **not** authorize leaking holdout to a cloud agent. The eval substrate stays LOCKED (no relitigation of `~/.agents/evals/SCHEMA.md`). Fails closed. ## Tests (TDD-first) `cli/cmd/ao/agent_bundle_test.go` — 7 cases on the pure `buildAgentBundle` seam: managed defaults + `ao` MCP descriptor; self-hosted sandbox block; codex-ntm shape (bootstrap + reference, no MCP); **holdout refusal**; unknown runtime; default-skill set; managed JSON required-keys (= the schema contract). All funcs CC < 15 (budget 25). `go vet` clean; full `cmd/ao` suite 9204 pass. ## Gates handled - Added `agent` to **both** `expectedCmds` lists (TestCobraExpectedCmdsMatchRegistration). - Regenerated `cli/docs/COMMANDS.md` (TestCobraConformance) + `docs/cli-skills-map.md`. - `ao agent bundle` reads as `public-tested`/`covered`. `docs/cli-surface.md` left at its standing tolerated state (non-blocking inventory; reverted to avoid sweeping in unrelated #613 eval-outcomes drift). ## Notes for review - **Naming**: `ao agent` (singular, new) sits beside `ao agents` (plural, AGENTS.md doctor/lint) — different domains, but the singular/plural proximity is a UX smell. Proceeded per epic spec; trivial rename if you'd prefer (e.g. `ao agentdef bundle`). - **Schema follow-up**: the managed-output structural contract is test-enforced (required-keys); a formal `schemas/agent-definition.schema.json` + validator wiring can land later if a downstream consumer needs strict validation. Kept out to bound this PR. - Instructions are stitched as skill **names + preamble**, not full bodies — bounded payload + the agent loads full skills at runtime via the `ao` tool surface (also avoids over-inlining). Closes-scenario: ag-eguw0#agent-bundle Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/agent_bundle_test.go	2026-05-30 01:28:41 -04:00
Bo	a3900ad0dc	feat(eval): ao eval outcomes compile — holdout-safe rubric payload (ag-hdqu0 #compile-strip) (#601) ... ## What Adds `ao eval outcomes compile <input.json>` under `eval` — the second slice of ag-hdqu0, building directly on the merged `evalsubstrate.ProjectRubric` (#599). Projects a locked Task + criteria into an Outcomes rubric payload. ## Holdout isolation (Managed Agents are NOT ZDR) - Projection via `ProjectRubric` (strips ground truth by construction). - **Deny-by-default re-scan** (`ContainsAny`, guard layer 3): `compileOutcomesRubric` REFUSES to emit any rubric that would carry a holdout value (input `holdout_values` feeds the scan; never copied to output). - Carries `judge_content_hash` for stale-rubric self-invalidation. Outcomes is a derived projection of the locked SCHEMA.md — never an alternate authority. ## Tests (TDD) - `TestCompileOutcomesRubric_StripsHoldoutTarget` — criteria carried, zero holdout leak. - `TestCompileOutcomesRubric_RefusesLeak` — deny-by-default errors on a leaking criterion, names the value. - 9184 `cmd/ao` tests pass, `go vet`/`build` clean, `COMMANDS.md` regenerated for the new subcommand. ## Next in ag-hdqu0 `.2` ingest→verdict; `.4` re-parse/stale-hash; `.9` gate#3 burn ledger (remaining half); the future GT-loader slice wires `holdout_values` from the substrate automatically. Closes-scenario: ag-hdqu0.1#compile-strip Bounded-context: BC1-Corpus Evidence: cli/cmd/ao/eval_outcomes_test.go	2026-05-29 15:49:22 -04:00
Bo	e4e32a3374	feat(orchestration): dual-runtime OrchestrationPort foundation (ag-nk67 #orchestration-foundation) (#598) ... ## What The **dual-runtime OrchestrationPort foundation** — the seam that gates the `[EPIC] AgentOps × Claude Managed Agents integration (dual-runtime)` application epics. Implements the 3-category model (Claude Workflow / NTM swarm / plain skill) behind a single `OrchestrationPort` with safe degradation **NTM → Claude-native → beads floor** and a global `AGENTOPS_ORCHESTRATION=off` opt-out. Derived from a live 3-legged spike (`~/dev/agentops-3cat-spike/`): safe degradation is solvable at the selection layer; parallel buys quality/independence not wall-clock at small N; NTM is a multi-vendor control plane that runs Claude/Codex as panes. ## What's in it (10 slices, agent-team driven) - **Port + adapters** — `cli/internal/ports/orchestration.go` (idiomatic to existing ports pattern) + new `cli/internal/orchestration/` pkg: `Selector`, `ntm_probe` (capability detection via `ntm --robot-capabilities`, not `command -v`), `beads_floor` + `OrchestrationResult` parity type, degradation-conformance test. - **Live CLI** — `ao orchestrate select` (`--pin`/`--opt-out`/`--json`); COMMANDS.md regenerated. - **Contracts** — `schemas/orchestration-{backend,result}.v1` + paired `docs/contracts/` docs (structural-floor gated: 41 contracts). - **Prose** — swarm/shared/crank rewritten NTM>native>beads; **gc demoted** (soc-2rtm0); `rpi_phased_stream.go` doc-comment fixed. - **Skills** — `automation-shape-routing` (the 3-category router) + `workflow-builder` + meta-skill authoring chain wired (`context_rel`); skill counts synced. - `lib/orchestrate-select.sh` selector seam. ## Verification (local) `go build ./...` clean · `go vet` clean · **56 Go tests pass** (orchestration 31, ports 10, cmd/ao orchestrate 15) · structural-floor PASS (41) · `lib/orchestrate-select.sh --self-test` 6/6 · live command: `--json`→claude (NTM absent), `--opt-out`→beads, `--pin claude`→claude. ## Notes - Bundles the foundation + the routing/authoring skills as one coherent dual-runtime arc. - NTM/Claude executors are stubs (the parity contract is real); concrete executors land in the application epics this gates. - Two-ladders distinction preserved (spawn-backend ladder A vs CLI phase-executor ladder B) — see `docs/contracts/orchestration-ports.md`. Closes-scenario: ag-nk67#orchestration-foundation Bounded-context: BC5-Runtime Evidence: cli/internal/orchestration/conformance_test.go	2026-05-29 18:42:45 +00:00
Bo	4f2ef54179	fix(evals): drop deleted validate-hooks-doc-parity.sh ref from docs-release-governance canary (ag-jov #eval-references-only-existing-scripts) (#575) ... ## Summary - Hooks were removed in 3.0 and `scripts/validate-hooks-doc-parity.sh` was deleted. `ag-s3z` removed the Nightly workflow step that ran it but the critical eval `evals/agentops-core/docs-release-governance.json` still referenced the deleted script in 3 spots: a shell command (`docs-policy-parity-gates`), a `HOOKS_DOC_PARITY: PASS` stdout expectation, and an `artifact_contains` target (`doc-release-script-contracts`) — so the canary fails when run. - Removed those 3 references plus the now-inaccurate "hook/docs parity" prose in the suite description and two case objectives. - Added `cli/internal/eval/governance_refs_test.go`: a regression gate that loads this suite and asserts every shell script token and every `artifact_contains` target resolves on disk, and that no reference to the deleted script / `HOOKS_DOC_PARITY` marker remains. ## Test plan - [x] `cd cli && go test ./internal/eval/ -run TestDocsReleaseGovernanceEval` — both new tests fail on `main` (dangling ref) and pass after the edit (test-first verified) - [x] `cd cli && go build ./... && go vet ./internal/eval/` - [x] `cd cli && go test ./internal/eval/... ./cmd/ao/` - [x] JSON re-validated (`python3 -m json.tool`); `LoadSuite` strict-decode + `ValidateSuite` pass Out-of-scope dangling refs (different bounded context: docs + local gate) tracked in `ag-c2i` (discovered-from `ag-jov`). Closes-scenario: ag-jov#eval-references-only-existing-scripts Bounded-context: BC1-Corpus Evidence: cd cli && go test ./internal/eval/ -run TestDocsReleaseGovernanceEval	2026-05-28 08:21:38 -04:00
Bo	a31999a061	feat(beads): ao beads scenarios extract → Gherkin to stdout (ag-ljd #scenarios-extract-stdout) (#567) ... ## Summary Slice 1 of **ag-dwq** (child bead **ag-ljd**). Adds `ao beads scenarios extract <bead-id>`: a deterministic, dependency-free, **dry-run** command that converts a bead's free-text acceptance criteria into a candidate Gherkin `## Scenarios` block printed to stdout. The bead is never modified. - **`cli/internal/scenarios`** (new, pure domain): `Extract(acceptance) ([]Scenario, error)` + `Render([]Scenario) string`. Word-boundary `given`/`when`/`then` matching so `forgiven`/`whenever` don't false-match; each clause must carry a letter so prose merely *mentioning* "Given/When/Then" isn't mistaken for a scenario. - **`cli/cmd/ao/beads_scenarios.go`** (new): subcommand under the existing `ao beads` surface, fetching via the `execBD show --json` seam. Agent-ergonomic (Directive 13): `--json` emits data on stdout / diagnostics on stderr; `bd`-unavailable degrades gracefully (warn + exit 0); errors name the corrective action (manual authoring). - **`cli/docs/COMMANDS.md`** regenerated. `capabilities` / `robot-docs` pick the command up automatically via the live command-tree walk. ### Out of scope (later slices of ag-dwq) `--write` / operator-confirmed update · idempotent `## Scenarios` guard + `--force` · `scenarios validate` + Gherkin-parser dep · LLM fallback for non-deterministic prose. ## Test plan - [x] `cd cli && go test ./internal/scenarios/...` — 12 cases (single/multi-bullet, sentence-form, mixed casing, skip-unparseable, empty→error, no-keywords→error, word-boundary, prose-mention→error, Render exactness, round-trip) - [x] `cd cli && go test ./cmd/ao -run BeadsScenarios` — L2 with stubbed `bd`: stdout Gherkin, `--json`, unparseable→error, bd-unavailable graceful, `parseAcceptanceFromBDJSON` (array/object/fallback/empty/malformed) - [x] `cd cli && go build ./... && go test ./...` — 11950 passed, 68 packages, 0 failures - [x] Real binary against `ag-dwq` → 5 clean scenarios; `--json` valid Closes-scenario: ag-ljd#scenarios-extract-stdout Bounded-context: BC1-Corpus Evidence: cd cli && go test ./internal/scenarios/... ./cmd/ao	2026-05-28 03:42:43 -04:00
Bo	6c5a88694c	feat(cli): ao skills find <intent> skill discovery (ag-a97 #find-ranks) (#563) ... ## Summary Turns the 75-skill catalog from oral tradition into a queryable surface. `ao skills find <intent>` scores every `skills/*/SKILL.md` against a free-text intent and returns the top matches (name, one-line description, score) — so an agent gets a discovery API instead of memorizing skill names. - **Scoring engine** (`cli/internal/skills/find.go`) — pure, deterministic token-overlap. A query word hitting the skill **name** counts most, a declared **trigger** next, a **description** word least, with light plural/stem tolerance (`loop` ↔ `loops`). Scores normalized to `[0,1]`; ties broken by name. No filesystem access, so it is fully table-testable. - **Loader** (`cli/internal/skills/load.go`) — reads `skills/*/SKILL.md`, parses frontmatter (`name`, `description`, best-effort top-level / `metadata` triggers). No static index file: a newly added skill is found on the next run. - **Command** (`cli/cmd/ao/skills_find.go`) — `--json` / `--limit` (default 5), stdout-as-data / stderr-as-diagnostics. An unmatched intent **exits 0** with a stderr note, not an error. Unknown-flag typo hints inherited from the root cobra config; `--limit < 1` returns a usage error naming the fix. - Regenerated `cli/docs/COMMANDS.md`. ### Notes / corrections - The bead's premise ("each SKILL.md has a populated `triggers:` array") is inaccurate — phase-1 discovery found **0/75** top-level `triggers:` arrays; intent lives as prose in `description`, with `metadata.triggers` in ~10 skills. Scoring therefore treats name+description as the primary signal and triggers as an optional boost. - Bounded context: the slice reads the **skills corpus** (SKILL.md content), so `BC1-Corpus` (the plan's draft `BC4-Practice` does not exist — BC4 is Evidence). - Deferred to follow-ups: `ao skills list --by-trigger` / `--list-triggers` (ag-0r0) and backfilling structured triggers across SKILL.md (ag-piv). ## Test plan - [x] `cd cli && go test ./internal/skills/... ./cmd/ao` (green) - [x] `cd cli && go build ./... && go vet ./internal/skills/... ./cmd/ao` - [x] `scripts/generate-cli-reference.sh` (COMMANDS.md in conformance) - [x] `bash skills/heal-skill/scripts/heal.sh --strict` (All clean) - [x] `ao autodev validate --file PROGRAM.md --json` → `valid: true` - [x] Manual smoke: `ao skills find "close the loop"`, `--json --limit 3`, unmatched intent Closes-scenario: ag-a97#find-ranks-relevant-skills Bounded-context: BC1-Corpus Evidence: cli/internal/skills/find_test.go + go test ./internal/skills/... ./cmd/ao	2026-05-28 00:15:43 -04:00
Bo	3f4b67be49	feat(cli): add reconciliation report (#550) ... ## Summary - add `ao reconcile` for a read-only git/GitHub release/beads/.agents decision report - emit both human and `--json` output with ranked findings and next actions - anchor external checks at the git root and disable bd Dolt auto-start for this read-only scan - update generated CLI docs and JSON flag consistency coverage ## Validation - `cd cli && go test ./cmd/ao -run 'TestReconcile|TestCapabilities'`\n- `cd cli && make build`\n- `cd cli && bin/ao reconcile --json | jq ...` (live run detected green main Validate plus failed `v3.0.1` tag Validate)\n- `bash tests/cli/test-json-flag-consistency.sh`\n- `bash scripts/validate-ci-policy-parity.sh`\n- `./scripts/generate-cli-reference.sh --check`\n- `git diff --check`\n- `scripts/pre-push-gate.sh --fast` (passes; existing warn-only executable-spec warnings for `scenarios --lint` and `trace --orphans`)\n\nNote: a broad local `go test ./cmd/ao` run was stopped after it appeared hung with no child test process; the focused command tests and build passed, and CI should provide the full-package signal.	2026-05-26 17:03:27 -04:00
Bo	b4c6aec361	feat(skills)!: 3.0 consolidation — merge 4, cut 2, resolve 6 keep-flags (81→75) (#533) ... ## AgentOps 3.0 skill consolidation — reconcile-and-tighten Resolves all 12 skills flagged `merge-review`/`cut-review` in `docs/contracts/skill-dispositions.yaml` per the operator-approved dispositions. **Zero capability loss** — every merged skill's content lives in its target as a mode/section. ### MERGE (4) — content folded into target, source removed | Source | Target | How | |---|---|---| | expert-council | council | `--mode=debate` (alias triggers documented in council) | | oss-docs | doc | `--mode=oss` (refs + `audit-oss-docs.sh` copied into `doc/`) | | readme | doc | `--mode=readme` (`readme-craft.md`; `bootstrap` consumer repointed readme→doc) | | pr-retro | post-mortem | `--scope=pr` (`pr-scope.md` + gh PR-outcome mining) | ### CUT (2) — genuinely redundant, no live consumer - **pr-plan** — folds into `/plan` + `/pr-research`; pr-* chain repointed. - **update** — install-script concern; refresh note added to `using-agentops` + install one-liner. ### KEEP — resolved 6 flags (rationale in `skill-dispositions.yaml`) `validate` + `curate` (designed-future canonical validator/miner; epic `soc-cp7pv` GO/REVERT is a separate decision) · `security` (release-gate driver, distinct from security-suite) · `converter` (packaging adapter skill-builder consumes) · `knowledge-activation` (outer-loop operationalization) · `grafana-platform-dashboard` (marketplace specialization). ### Result **81 → 75 skills** (65 user-facing + 10 internal). Trunk regenerated once: skill-counts, skill-catalog, skill-domain-map, context-map, registry, sku-catalog, codex hashes, embedded-sync. Evals repointed to absorbed locations. ### Gates verified locally (all green) lint-skills · heal.sh --strict · skill-schema · skill-frontmatter · skill-dependency-check · scenario-test-linkage · validate-context-map-drift · check-registry-drift · generate-registry --check · skill-domain-map golden · validate-sku-catalog-drift · bounded-contexts-drift · ci-policy-parity · codex parity/override-coverage/manifest/generated-artifacts/skill-parity/runtime-sections/backbone/lifecycle · regen-codex-hashes --check · embedded-sync · validate-skill-count · markdownlint · shellcheck (--severity=error) · `cd cli && go build && go vet && go test ./cmd/ao` (9126 passed). Closes-scenario: ag-cons#consolidation Bounded-context: BC1-Corpus Evidence: .agents/discovery/2026-05-24-skill-consolidation-plan.md	2026-05-24 23:17:27 -04:00
Bo	dde214825b	docs(3.0): ag-p7j W1 — delete daemon contracts, regen generated docs, reframe skills (ag-p7j) (#529) ... W1 (structural) of the 3-way ag-p7j doc-sweep. Owns shared/generated/structural artifacts + skills + contracts. Disjoint from W2 (#527, core narrative) and W3 (#528, peripheral docs), both already merged; this branch is rebased on top of them. ## Deleted (deleted-surface docs) - `docs/contracts/{agentops-daemon,ao-watch,daemon-idempotency,dream-run-contract,agent-worker,ao-outcomes,openclaw-consumer-api,gascity-integration}.md` - `docs/code-map/agentopsd-codebase-map.md` - `gascity-integration.md` deleted because it described the OLD daemon↔GasCity adapter (`ao daemon run --executor-policy=gascity`, daemon ledger, OpenClaw consumer endpoints). The current GC story is the using-gc skill + the reference City. - Removed every deleted doc's row from `docs/documentation-index.md`, `docs/contracts/index.md`, and `mkdocs.yml` nav. ## Regenerated (generators, not hand-edited) - `docs/cli-surface.{md,json}` via `check-cmdao-surface-parity.sh --write-surface` — daemon/schedule/overnight/plans/watch/factory rows gone (live `ao` binary no longer registers them; the docs were stale). - `registry.json`, `skills/catalog.json`, SKU catalog, codex `.agentops-generated.json` hashes + manifest. - Skill counts verified in sync (81 skills). ## Reframed to in-session / GC / hookless - Skills: `dream` (+codex twin, neutralized literal `agentopsd`/`ao schedule`/`ao overnight`/`ao watch`/`ao plans` while keeping the retirement message), `compile` (+codex), `scaffold/recommended-reading` (+codex), `discovery/best-practices`, `validation/best-practices`. - Generated narrative: `docs/SKILLS.md` /dream block, `docs/ARCHITECTURE.md` dream paragraph, `skills/SKILL-TIERS.md` dream row. - Contracts prose: `docs/contracts/ubiquitous-language.md`, `docs/contracts/agents-write-surfaces.md` (kept catalog rows for surviving load-bearing-lane subdirs; dropped deleted-command references in descriptions). ## 3.0.md:150 Restored the strong claim: "...gone from the CLI **and from live doc prose** (only historical surfaces — ADRs, the CHANGELOG, release notes, and the 3.0 migration record — still name them, by design)"; kept the `validate-sku-catalog-drift` linkage-gate mention. ## Collateral (minimal dead-link repairs to my deleted targets) - `docs/daemon-migration.md` (gascity-integration link → dependencies.md), `cli/README.md` (dropped dream-run-contract link). The comparisons-file dead link was resolved by W3's merge. ## Gates (all green locally) doc-release (2765 links, 0 broken) · contract-compatibility (0 failures) · structural-floor · agents-write-surfaces · registry --check · validate-sku-catalog-drift · validate-ci-policy-parity · validate-skill-cli-snippets · validate-codex-runtime-sections · codex parity (5 skills) + regen-codex-hashes --check · skill-catalog drift · cli-skills-map · markdownlint (changed files) · mkdocs --strict · `cd cli && go build ./...` Closes-scenario: ag-p7j#w1-structural Bounded-context: BC1-Corpus Evidence: docs/3.0.md	2026-05-24 21:55:17 -04:00
Bo	010ade393c	feat(rip)!: carve the daemon — delete internal/daemon + agentopsd, GC is the substrate (soc-2rtm0 #daemon-carve wave5) (#515) ... Wave 5 of 5 (the finale) of the orchestration-substrate retirement (soc-2rtm0). AgentOps stops being the orchestration substrate; GC is the substrate. The always-on out-of-session supervisor (the daemon) is deleted; GC's controller replaces it. Features that ran AS daemon jobs (wiki build, llmwiki loop) keep their BUILDER CORE (in-process / GC-callable) but lose the daemon-job-executor wrapper. ## Deleted - `cli/internal/daemon/` — the whole package (71 .go files + testdata + AGENTS.md). - `cli/cmd/ao/`: `agentopsd.go`, `daemon_jobs.go`, `daemon_soak.go`, `daemon_fake_policy.go`, `rpi_phased_daemon.go` (+ their tests). - Daemon-only tests: `rpi_status_daemon_test.go`, `rpi_integration_test.go`, `doctor_product_runtime_test.go`. - `internal/doctor/fix_daemon.go` (+test) + the daemon runtime/ledger-health/telemetry/openclaw-consumer checks and the `--product-runtime` flag in `doctor.go` (the `ao doctor` command itself is KEPT). - The `ao daemon`/`agentopsd` command registration (went with `agentopsd.go`). - Daemon test scripts: `scripts/validate-daemon-product-e2e.sh`, `tests/scripts/test-daemon-product-e2e.sh`. No daemon CI gates existed in `validate.yml` / `ci-jobs.yaml` to remove (prior waves had already cleared them). ## Severed (KEEP features — builders stay, daemon-job-executor wrappers removed) - **wiki** (`internal/wiki/pipeline.go`): removed the `daemon.JobExecutor` impl (`JobTypes`/`RunJob`/`PipelineJobSpec`/`parsePipelineJobSpec`/`jobResultFromOutcome`) + the daemon import. The builder core — `WikiPipeline`, `RunStage`, `SelectStage`, the ingest/query/lint runners — is untouched; `ao wiki lint/query/promote` run in-process. - **llmwiki** (`internal/llmwiki/executor.go`): removed `LLMWikiLoopExecutor` + `LoopJobSpec` + the daemon import. KEPT the leaf builder core in the same file (`Stage`, stage constants, `StageHandler`, `StageResult`, `DefaultLintIntervalHours`, `SelectStage`, `rawHasNewerFiles`, `lintIsStale`) — `stages.go` (the real handlers) and `internal/scope` depend on these. No new package needed; the shared types already lived in this leaf file. - **rpi status** (`cmd/ao/rpi_status.go`): severed the `--daemon`/`--daemon-url`/`--daemon-fallback` read path. `ao rpi status` now reads only the local run registry (`.agents/rpi/runs/`) — a KEEP feature. Relocated the tiny `cobraContext` helper here (it lived in the deleted `agentopsd.go`, only consumer is this file). - **rpi phased** (`cmd/ao/rpi_phased.go`, `rpi_phased_context.go`): removed the `--daemon-submit` short-circuit + flags + the `DaemonSubmit/URL/Token/Fallback` opts fields (only consumer was the deleted submit path). ## Lost capabilities (intentional, daemon-only) - `ao rpi status --daemon` (read RPI job state from agentopsd) and `ao rpi phased --daemon-submit` (queue an RPI run to agentopsd). - llmwiki `LoopJobSpec.AllowPlaceholderOutputs` opt-in gating (the placeholder-output-disabled skip) — that gate lived in the removed daemon-job wrapper, not the stage handlers. - `ao doctor` daemon runtime/ledger-health/telemetry/openclaw-consumer checks + `--product-runtime`. ## Verification (all green) - `gofmt -l` clean (my files), `go build ./...`, `go vet ./...`, `go test ./...` → 11920 passed, 0 failed, 65 packages. - `scripts/release-smoke-test.sh` → 108 pass / 0 fail (EXPECTED_COMMANDS never listed daemon; no scrub needed). - `tests/windows/test-windows-smoke.ps1` → `Invoke-GoTest` list has no `./internal/daemon` ref (nothing to scrub). - `internal/quality/stale_refs.go` → no daemon/agentopsd deprecated-command entries (`TestDoctorStaleReplacementsExist` passes). - `tests/smoke-test.sh`, `tests/cli/test-json-flag-consistency.sh` (15/0), `scripts/test-agentops-contract-canaries.sh` (failures=0 after fixing the cli-command-surface heading-count fixture/eval 68/179/247 → 67/172/239 and the `ao daemon` refs in `skills/dream` + `skills-codex/dream`). - Regenerated `cli/docs/COMMANDS.md` + `registry.json`; `regen-codex-hashes.sh --check` clean; `validate-ci-policy-parity.sh` PASS; doc link/learning-ref checks pass; `heal.sh --strict` clean. Diff stat: 113 files changed, 95 insertions, 34,978 deletions. Closes-scenario: soc-2rtm0#cascade-rip Bounded-context: BC5-Runtime Evidence: .agents/discovery/2026-05-24-rip-caller-map.md	2026-05-24 17:11:20 -04:00
Bo	d52144ed40	feat(cli): ao validate --gate — exit-code verdict for GC retry + CI (#ao-validate-gate) (#516) ... ## What New top-level `ao validate` umbrella command with `--gate` (exit-code-as-verdict) mode. This is the dependency the GC reference City needs — GC's `check={max_attempts}` retry, CI steps, and `ao rpi`'s internal validate phase can run one command and branch on the exit code. Spec: `.agents/discovery/2026-05-24-gvkj6-finalize-runbook.md` ("`ao validate --gate` — SPEC" section). ## Exit-code contract (`--gate`) | Code | Meaning | |---|---| | `0` | PASS or WARN (gate passes) | | `1` | FAIL (gate fails) | | `2` | internal/setup error (distinct so a broken gate is never read as a clean pass) | - WARN exits `0` by default (advisory); `--strict` / `--warn-as-fail` promotes WARN → exit `1`. - Deterministic: **no network, no LLM**. Safe for GC `check.max_attempts`, CI, and `ao rpi`. - `--json` emits `{"verdict","issues","warnings","gate_exit"}` in both modes. ## How (composed, not reinvented) Reuses the existing `ratchet.Validator` (`ValidateWithOptions`) over the resolved file set (`--changes`, `--bead`, or auto-located RPI outputs); aggregates each result's `Valid`/`Issues`/`Warnings` into a single PASS/WARN/FAIL verdict; maps verdict → exit code via a `gateExitError` typed error caught in `Execute()` (same pattern as `doctorExitError`/`AgentsLintError`). The existing `ao ratchet/scenario/goals validate` sub-surfaces are untouched. New logic is just the aggregation + mapping (well under the complexity budget). ## Residue handled (command-surface +1) - Regenerated `cli/docs/COMMANDS.md`, `registry.json` (166 commands), `docs/cli-skills-map.md`. - Bumped `cli-command-surface` canary (`cli-command-surface-matrix.json`) + smoke fixture heading counts: top 68→69, all 247→248 (sub unchanged — flat command). - Added `validate` to release-smoke `EXPECTED_COMMANDS` + a `test_help "ao validate --help"` line. - Registered `validate` in `cobra_commands_test.go` `expectedCmds` (both lists). ## Tests L1 (`aggregateVerdict`, `gateExitForVerdict`, `gateExitError`) + L2 (full `runValidate` over real temp-dir artifacts) asserting exact exit codes: PASS=0, WARN=0, WARN+`--strict`=1, FAIL=1, internal=2, and the `--json` contract. ## Verified green `gofmt` clean · `go build`/`go vet`/`go test ./cmd/ao ./internal/ratchet` · manual exit-code check (0/1/2) · `release-smoke-test.sh` (0 failed) · `test-json-flag-consistency.sh` (0 errors) · `validate-cli-skills-map.sh` (PASS) · `cli-command-surface` canary (failures=0) · `regen-codex-hashes.sh --check` clean · `validate-ci-policy-parity.sh` PASS. Closes-scenario: soc-ba1un#ao-validate-gate Bounded-context: BC4-Evidence Evidence: .agents/discovery/2026-05-24-gvkj6-finalize-runbook.md	2026-05-24 15:48:04 -04:00
Bo	1ba0ca8120	feat(rip)!: kill schedule/plans/watch/overnight commands+engines — GC owns scheduling (soc-2rtm0 #kill-schedule-overnight wave3-4) (#514) ... Wave 3+4 of the soc-2rtm0 orchestration-substrate rip. Operator decision: **kill the schedule/plans/watch/overnight commands AND their engines entirely** — GC owns all scheduling/overnight/watch; AgentOps keeps only rpi/evolve/crank/swarm as loops. ## Killed (commands + engines) - `cli/cmd/ao/`: `overnight*.go` (overnight, council, curator, packets, setup, longhaul, daemon, tier1), `dream_subcycle.go`, `schedule.go`, `watch.go`, `plans*.go` (+ all tests). - `cli/internal/schedule/` — clean delete (only `schedule.go` + `agentopsd.go` imported it). - `cli/internal/overnight/` — the overnight engine, deleted whole. - Daemon refs severed (daemon stays, wave 5): removed `agentopsd.go` dream-executor + crash-recovery + `--schedule-file` loading; removed the `dream` soak scenario. - Removed `ao init --with-schedule` (seeded config for the now-dead daemon scheduler). - Retired the `dream` skill (claude + codex) to a GC-pointer stub. ## KEEP-feature severance (HARD CONSTRAINT) - **harvest** (KEEP): lock helpers (`LockIsStale`/`ReadLockPID`/`ProcessAlive`/`WriteLockPID`) **extracted** from `overnight/recovery.go` into new `cli/internal/lockfile` package; `harvest.go` repointed. Generic process-lock machinery, not overnight-specific. - **forge** (KEEP): `writeJSONAtomic` + `buildCuratorID` + curator-queue types **extracted** into `cli/cmd/ao/forge_curator_id.go`; forge curator-queue enqueue path preserved. - **evolve** (KEEP): the `--dream-first`/`--dream-only` Phase-1 dream sub-cycle was **severed** (capability genuinely overnight/dream-only). **LOST CAPABILITY** — flagged below. - `forge/runmine.go` + `wiki/artifact.go` only referenced overnight in comments (no real import) — untouched. ## CI gates / docs / fixtures reconciled - Removed `dream-end-user-coverage` GOALS gate + `scripts/check-schedule-example.sh` + `schedule.yaml.example` + `goals-affects-files.yaml` entry; updated GOALS Directive #8. - Fixed `cli-command-surface` canary: heading counts 72/194/266 → 68/179/247, dropped deleted `plans.go` artifact check. - Deleted `dream-software-factory-operator` eval; dropped deleted `plans.go` check from `session-continuity-context` eval. - Regenerated `COMMANDS.md`, `cli-skills-map.md`, `registry.json`; regenerated codex hashes. - No dedicated validate.yml/ci-jobs.yaml job existed for these surfaces (verified) — `validate-ci-policy-parity` stays PASS. ## Lost capability (flagged) - `ao evolve --dream-first` / `--dream-only` (run the knowledge-compounding dream loop before/instead of code cycles) is gone. Knowledge compounding now runs out-of-session via GC. The nightly-CI dream-cycle proof job (built on KEEP harvest/forge/inject/defrag primitives) is unaffected. ## Verification (all green) `cd cli && gofmt -l` clean · `go build ./...` · `go vet ./...` · `go test ./...` (zero FAIL) · `tests/smoke-test.sh` · `tests/cli/test-json-flag-consistency.sh` (0 errors) · `scripts/test-agentops-contract-canaries.sh` (**failures=0**) · doc-release gate · `heal --strict` clean · skill-lint · codex parity · ci-policy-parity. ## Follow-up (out of scope, build-green-safe to defer) Narrative docs (`docs/scheduling.md`, `docs/ARCHITECTURE.md`, comparisons, etc.) still mention the killed `ao overnight`/`ao schedule` commands in prose. No CI gate validates ao-subcommand existence in narrative docs, so these do not block; a doc-sweep is a separate follow-up. Closes-scenario: soc-2rtm0#cascade-rip Bounded-context: BC5-Runtime Evidence: .agents/discovery/2026-05-24-rip-caller-map.md	2026-05-24 14:56:20 -04:00
Bo	ee748d580d	feat(rip)!: retire factory command + contract corpus (soc-2rtm0 #cascade-rip) (#512) ... **Wave 1 of the orchestration-substrate cascade rip** (soc-2rtm0). The full cascade (daemon + dream/overnight + schedule/plans/watch + gc-glue + factory) is ~50K LOC across deeply-entangled packages — far past one reviewable PR. Per the bead's slice directive and the recon map's serial-by-wave verdict, this ships as sequential PRs by wave. **This is the factory wave** (the only cleanly-disjoint slice — no KEEP-feature entanglement). ## What this retires (operator-confirmed FULL factory retire) **Go command surface** (no external non-test caller — clean delete): - `cmd/ao/factory.go`, `factory_admit.go`, `factory_pilot.go`, `factory_admission_adapter.go` (+ all tests) - `internal/ports/factory_admission.go` + `inmemory_factory_admission.go` (+ test) — the port was only wired to the daemon by the deleted production adapter - severed factory flag save/restore + command-tree assertions from `cobra_commands_test.go` / `testutil_test.go` **Contract corpus** (scripts, schemas, contract docs, runbooks, fixtures, tests): - `scripts/check-factory-{admission,claim-ledger,yield-ledger}.sh` - `schemas/factory-{admission,work-order,yield}.v1.schema.json` - `docs/contracts/factory-{admission,claim-ledger,yield-ledger}.*` - `docs/runbooks/{factory-manual-merge,daemon-factory-admission,cloud-frontier-pilot}.md` - `tests/e2e/factory-operator-canary.sh`, `tests/integration/test-factory-claim-ledger-ci-job.sh`, `tests/scripts/test-factory-admission-contracts.py`, `tests/fixtures/factory-*` - orphaned `aggregate-observation-log` subsystem (script + integration test + fixtures) — its only data source was the deleted `factory-claim-ledger-strict` CI job **CI / generated reconciliation:** - removed 3 factory jobs from `validate.yml` (+ both `needs:` arrays + verdict echoes); removed entries from `ci-jobs.yaml`; regenerated `AGENTS-CI.md` (63 jobs) - removed factory gate-ids from `GOALS.md`; trimmed `contracts-structural-floor` factory references - regenerated `registry.json` (177→174 CLI commands), `COMMANDS.md` (−70 lines), `cli-skills-map.md` (73→72 headings) - updated `cli-command-surface-matrix` canary spec + fixture counts (`top=72 sub=194 all=266`); scrubbed deleted-file links from `documentation-index.md`, `SCHEMAS.md`, `contracts/index.md`, `agentopsd-control-plane.md`, `insert-schema-practices.py` ## Deliberately left for the daemon-carve wave Daemon-owned factory job types (`factory.admission`, `factory.local-pilot`) and the `agentopsd` routing-policy schema remain — they belong to `internal/daemon/` and retire when the daemon does. ## Gates verified green locally `gofmt`/`go build`/`go vet`, full Go test (**11830 pass**), `validate-ci-policy-parity`, `contract-compatibility`, `contracts-structural-floor`, `registry --check`, `doc-release` (0 broken links), `smoke-test`, `json-flag-consistency` (0 errors), **`contract-canaries` (failures=0)**, codex-hashes, `pre-push-gate.bats` (55). Diff: 66 files changed, ~16 insertions / ~5680 deletions. Closes-scenario: soc-2rtm0#cascade-rip Bounded-context: BC5-Runtime Evidence: .agents/discovery/2026-05-24-rip-caller-map.md	2026-05-24 12:29:22 -04:00
Bo	e431339c40	feat(hookless)!: remove all hooks — skills + CLI only (soc-57b7f #remove-all-hooks) (#511) ... ## What AgentOps 3.0 goes **fully hookless**. This removes the entire hook product surface — the product is now skills + the `ao` CLI with **zero hooks**. CI is the authoritative gate. ## Deleted - `hooks/` — 46 hook scripts + `hooks.json` + examples (whole directory) - `cli/embedded/hooks/` + embedded `hooks.json` + `embedded/lib/hook-helpers.sh` - `lib/hook-helpers.sh` - `ao hooks` CLI command: `cli/cmd/ao/hooks.go`, `hooks_run.go` + 5 hook test files; `cli/internal/bridge/hooks.go`; `cli/internal/doctor/fix_hooks.go` + test - Codex hook manifest (`hooks/codex-hooks.json`) + handler/audit scripts (`scripts/audit-codex-hooks.sh`, `install-dev-hooks.sh`, `test-hooks-output.sh`, `test-codex-hookless-lifecycle.sh`, `test-hookless-rpi-phased.sh`) - Hook lease surface: `docs/contracts/hook-lease-*`, `schemas/hook-lease.v1.schema.json`, `scripts/check-hook-lease-inventory.sh`, `check-hook-port-replacements.sh`, `generate-hook-lease-inventory.py` - Hook docs: `docs/HOOKS.md`, `cli/docs/HOOKS.md`, `docs/architecture/hook-noise-audit.md`, `docs/contracts/hook-runtime-contract.md` - `tests/hooks/` (whole dir) + hook test scripts; 5 hook eval suites + fixtures ## CI gates removed `hook-preflight`, `validate-hooks-doc-parity`, `hook-output-schema-lint` (+ scripts/bats), the hook eval suites, the `test-runtime-codex-smoke` / `test-codex-native-install` / `test-codex-plugin-install` hook-handler assertions, the cli-integration hook-lifecycle steps, the bats `tests/hooks/*` glob + orphan-hooks audit, and the `--with-hooks` install path. `embedded-sync` now validates only lib/skills. Both summary `needs:` lists + the summary echo + `docs/contracts/ci-jobs.yaml` updated; `validate-ci-policy-parity` passes (66 rows). `.codex-plugin/plugin.json` drops the "Hooks" capability. ## Load-bearing PRESERVED (reported, not forced) - **`cli/cmd/ao/codex_runtime.go`** — the codex runtime *lifecycle profile* (`HookCapable`/`HookConfigured` detection) is a runtime-capability abstraction used by 6 `validate-codex-*` CI gates, distinct from the hook product. Deleting it is a separate refactor. - **`hooks-authoring` skill** + `schemas/hooks-manifest.v1.schema.json` — reframed as an opt-in "author your own hooks" guide. Deleting the skill cascades into catalog/domain-map/dispositions/context-map golden files (skill-count sync), out of scope for the hook-product removal. - **`scripts/pre-push-gate.sh`** — retired local gate (not in CI); self-skips deleted hook scripts via `[[ -x ]]` guards; its 55 bats tests still pass. - Domain-enforcement audit keeps the `enforced`-mode schema for backward-compatible JSON readers, but the hookless resolver only returns `audited`/`unavailable`. ## Verification `go build` / `go vet` clean; `go test ./...` = **11849 passed, 0 failed** (68 pkgs). gofmt clean on all touched files. Green: registry `--check` (0 hooks), `regen-codex-hashes --check`, `validate-ci-policy-parity`, `check-wiring-closure`, `validate-embedded-sync`, `validate-manifests`, `validate-agents-split`, `doc-release` (0 broken links), `generate-cli-reference --check`, `heal.sh --strict`, codex/claude/plugin smoke tests, baseline-audit (0 stale). Diff: 231 files, +354 / −39803. Closes-scenario: soc-57b7f#remove-all-hooks Bounded-context: BC5-Runtime Evidence: .github/workflows/validate.yml	2026-05-24 11:19:37 -04:00
Bo	580e2d2422	fix(evals): clean fixtures referencing deleted hooks (soc-t40ai #eval-fixture-cleanup) (#497) ... ## Summary S1-S3 deleted 7 noise-injector hooks (research-loop-detector, context-monitor, write-time-quality, edit-knowledge-surface, go-vet-post-edit, standards-injector, commit-review-gate) but left eval/test fixtures asserting on them. This cleans the three drifted fixtures to current reality. ## Changes - **`evals/agentops-core/fixtures/hook-context-value-smoke.sh`**: removed `test_context_monitor` and `test_edit_knowledge_surface` (both hooks deleted) plus the now-dead `assert_empty` helper. Keeps the valid `test_precompact_snapshot`. Exit 1 -> exit 0. - **`evals/agentops-core/hook-context-value.json`**: dropped stale `standards_go_bytes=` / `commit_review_bytes=` stdout expectations (those test functions were removed in S2/S3 deletion PRs); keeps `precompact_bytes=`. - **`evals/agentops-core/hook-manifest-runtime-contracts.json`**: dropped the three deleted-hook `artifact_contains` (write-time-quality, go-vet-post-edit, research-loop-detector); refreshed counts to current `hooks.json` reality: - preflight `Registered: 32 of 47` -> `31 of 46` - manifest command counts `length==44 / unique==37` -> `35 / 30` - bats plan `1..104` -> `1..75`, renumbered ok-lines `19/34/48/104` -> `8/19/33/75` ## Verification - `hook-context-value-smoke.sh`: exit 1 -> **exit 0** - `hook-context-value` eval suite: **pass** (aggregate 1.0) - `hook-manifest-runtime-contracts`: 5/6 cases pass (was failing 4 deleted-hook cases) - CI gate `agentops-eval-baseline-audit`: `stale_suite_hashes=0` -> **passes** - All edited JSON validates ## Out of scope (noted) The `embedded-hook-inventory-parity` case still fails because `cli/embedded/hooks/` still contains the 5 deleted hook scripts (needs `cd cli && make sync-hooks`). This is pre-existing drift, fails identically on main, and lands in `cli/` which is owned by another worker. Not touched here. Closes-scenario: soc-t40ai#eval-fixture-cleanup Bounded-context: BC4-Evidence Evidence: evals/agentops-core/hook-context-value-smoke.sh	2026-05-23 21:55:17 -04:00
Bo	211f04cfbe	chore(hooks): delete commit-review-gate — hookless S3 (soc-vpmzg #rm-commit-review-gate) (#494) ... ## Summary Deletes `commit-review-gate` — a value-negative noise-injector. Despite its name it never blocked: it always exited 0 and injected the staged diff + a "SELF-REVIEW" prompt on every `git commit` (A/B showed Δ=0). Hookless S3 of the go-hookless epic (`soc-e2ju0`). The "+7 conditional injectors" the original audit grouped with it were reclassified as kept-gates (they speak only on a real violation) — **out of scope, not deleted**. S3 removes only `commit-review-gate`. ## Changes - **Hook scripts:** `git rm` of `hooks/commit-review-gate.sh` + `cli/embedded/hooks/commit-review-gate.sh` - **Go runtime:** removed `runCommitReviewHook` managed-backend + its exclusive helpers (`gitOutput`, `countDiffFiles`, `hookCountLines`, `firstLines`, `redactSensitiveDiff` + credential regexes) and the now-dead `TestHooksRunRedactsSensitiveDiff` test - **Manifests:** pruned from `hooks/hooks.json`, `hooks/codex-hooks.json`, `registry.json`, `hook-lease-classification.v1.json` (lease-inventory regenerated) - **Codex handler count 15 → 14:** assertions in `test-runtime-codex-smoke.sh` + `test-codex-native-install.sh`, plus require-handler arrays in `install-codex-plugin.sh` / `install-codex.ps1` / `test-codex-native-install.sh` / `audit-codex-hooks.sh` - **Test blocks:** removed commit-review @test blocks from `test-hooks.bats`/`.sh`, `hook-output-schema.bats`, `hook-stdin-contracts.bats`, the eval fixture, and manifest-runtime-contracts - **Docs:** `HOOKS.md`, `software-factory.md`, `how-it-works.md`, `update-principles.md`, `SDK-INVENTORY.md`; `hook-noise-audit.md` records the S3 deletion (46 hooks remaining) - **Drive-by:** fixed pre-existing stale `length == 20` assertions in `test-audit-codex-hooks.sh` (codex bundle is 14 post-delete; this test is not CI-gated and was already failing on main) ## Verification (all green) `check-hook-lease-inventory` · `validate-hooks-doc-parity` · `check-hook-port-replacements` · `validate-hook-preflight` · `generate-registry --check` · `smoke-test` · `test-runtime-codex-smoke` (33/0) · `bats tests/hooks/*.bats tests/scripts/pre-push-gate.bats` (297/0) · `test-agentops-contract-canaries` (failures=0, incl pre-push-gate-governance + runtime-smoke-matrix) · `test-codex-native-install` · `test-audit-codex-hooks` (4/4) · `go build/vet/test ./cmd/ao/` Diff stat: 30 files, +26 / −569 (deletion-heavy, as expected). Closes-scenario: soc-vpmzg#rm-commit-review-gate Bounded-context: BC5-Runtime Evidence: docs/contracts/hook-lease-inventory.md	2026-05-23 21:43:03 -04:00
Bo	29e0acc26e	chore(hooks): delete standards-injector + retire its CI gate — hookless S2 (soc-khev6 #rm-standards-injector) (#488) ... ## What Hookless epic S2 (`soc-khev6`): delete the `standards-injector` PreToolUse hook and **fully retire** its dedicated CI gate. `standards-injector` is a value-negative noise-injector — it injects language standards on every Edit/Write regardless of relevance (A/B Δ=0, per `docs/architecture/hook-noise-audit.md`). The replacement is the `standards` skill, read on demand: a PRODUCT surface, not hook magic. ## Teardown (all of it) - **Hook + tests:** `git rm hooks/standards-injector.sh`, its embedded copy, and `tests/hooks/standards-injector.bats`. - **Manifests:** pruned the Edit + Write entries from `hooks/hooks.json` and `hooks/codex-hooks.json`, dropping the now-empty matcher groups. **Codex native handlers 17 → 15** (two entries removed, not one — Edit retains `codex-parity-warn.sh`, Write group dropped). - **CI gate retired:** removed the `standards-injector-completeness` job from `validate.yml` + both `needs:` lists (`validate-pr-evidence-claims`, `summary`) + the summary echo; `git rm` the check script + its bats; removed from `ci-jobs.yaml` manifest and regenerated `AGENTS-CI.md`; removed from `pre-push-gate.sh` + its bats stub. - **Codex test assertions (the S1 bite):** `test-runtime-codex-smoke.sh` and `test-codex-native-install.sh` length `17 → 15`; removed `standards-injector.sh` from the require-handlers array. - **Shared iterators:** pruned the `@test`/section blocks from `test-hooks.bats`, `hook-output-schema.bats`, `hook-stdin-contracts.bats`, `test-hook-chain.sh`, `test-hook-lifecycle.sh`, `test-hooks.sh`, and the eval fixture `hook-context-value-smoke.sh`. - **Other surfaces:** `validate-hook-preflight.sh`, `install-codex-plugin.sh`, `install-codex.ps1` owned-script lists; `hooks_test.go` (swapped synthetic name to a live hook); `context_compiler.go` port doc comment. - **Regen:** hook-lease classification + inventory, registry, embedded sync. - **Docs:** `HOOKS.md`, `hook-noise-audit.md` (records S2 deletion), `ENV-VARS.md` (`AGENTOPS_STANDARDS_*` vars), `LOOP-BUDGET.md`, `cli-skills-map.md`, `software-factory.md`, `SDK-INVENTORY.md`. ## Verify (all green locally) `check-hook-lease-inventory`, `validate-hooks-doc-parity`, `check-hook-port-replacements`, `validate-hook-preflight`, `generate-registry --check`, `validate-ci-policy-parity`, `tests/smoke-test.sh` (codex smoke + native install), `test-runtime-codex-smoke.sh`, the 3 bats hook suites, `go build/vet/test`. `grep standards-injector validate.yml ci-local-release.sh` → empty. Diff stat: 39 files, +32 / -1045 (deletion-heavy, as expected for a teardown). Closes-scenario: soc-khev6#rm-standards-injector Bounded-context: BC5-Runtime Evidence: docs/contracts/hook-lease-inventory.md	2026-05-23 20:54:24 -04:00
Bo	ce9ec94f41	fix(evals): unstale 3 release-gate eval hard-fails behind legit refactors (soc-2gd6 #eval-hard-fails) (#402) ... ## Why The v2.42.0 release gate (`scripts/ci-local-release.sh`) was red on 8 evals. The 3 score-0/near-0 hard fails are all **eval-staleness behind legitimate recent refactors** — verified, not gaming or security weakening. Operator decision: update eval to match source of truth (executable > contract). | Eval | Was | Cause | Fix | |---|---|---|---| | `hook-manifest-command-counts` | 0 | `session-pr-counter.sh` (PR #362) is the legit 37th hook script; eval hardcoded 43/36 | bump expected counts 43→44, 36→37 | | `push-worktree landing-plane` | 0.14 | #387 tiered-AGENTS split moved "Landing the Plane" to `AGENTS-WORKFLOW.md` (+ dropped 2 lines) | redirect eval target `AGENTS.md`→`AGENTS-WORKFLOW.md` + restore the 2 dropped policy lines | | `security-toolchain ci-soft-gate-policy` | 0 | gate is intentionally **HARD** (no `continue-on-error`); job already runs `security-gate.sh --mode quick` + uploads artifacts | drop the stale `continue-on-error` requirement (security stays HARD) | **Security note:** `security-toolchain-gate` stays a HARD blocking gate. Only the stale "soft gate" assertion was removed from the eval; the actual scan + artifact upload + summary-blocking are unchanged. ## How tested - hook-manifest jq → `hook-manifest-counts-ok` - security smoke `ci-policy` → `security-toolchain-ci-policy-ok` - all 7 landing-plane strings present in `AGENTS-WORKFLOW.md` - shellcheck clean on edited smoke ## Scope honesty This fixes the 3 **hard** fails only. The release gate still has **5 minor evals (0.71–0.99)** + the **vil/release-smoke** lane — a separate remediation, deliberately NOT in this PR (no green-washing). Sibling pattern: same "update eval to match legitimately-changed source of truth" move as the cli-command-surface canary bumps in #396/#397. Fitness: release-gate eval hard-fails 3 → 0. Closes-scenario: soc-2gd6#eval-hard-fails Bounded-context: BC4-Validation Evidence: evals/agentops-core/fixtures/security-toolchain-governance-smoke.sh	2026-05-22 12:12:05 -04:00
Bo	0759a740e6	feat(evolve): Phase 2 of /evolve --mode=loop — ladder + cron self-adjust + typed blocked events (soc-g2qd #phase-2) (#397) ... ## Why Phase 2 of soc-g2qd: ship the CLI enforcement primitives the skill's prompt-text alone can't guarantee. | Bead | What it gives the operator-loop | |---|---| | soc-mlbm | `ao evolve next-work` — 5-step programmatic ladder; agent stops guessing what to claim next | | soc-un0m | `ao cron self-adjust` — renders cron template + emits JSON spec; replaces manual CronList/Delete/Create per cycle | | soc-g34d | `ao evolve blocked` — typed blocked events at `.agents/evolve/blocked.jsonl`; agent logs rather than halts | ## What changed | Surface | Change | |---|---| | `cli/cmd/ao/evolve_next_work.go` + `_test.go` | New subcommand + L2 integration tests | | `cli/internal/evolve/ladder/` | 5-step ladder package (shape_filter, grep_siblings, primitive_test, cross_hop_pickup, bug_fallback) with table-driven unit tests | | `cli/cmd/ao/cron.go` | New top-level `ao cron` command | | `cli/cmd/ao/cron_self_adjust.go` + `_test.go` | New subcommand; calls `evolve.VerifyMarkers` + `evolve.Render` from #394; writes audit row to `.agents/evolve/cron-history.jsonl`; emits JSON spec to stdout (harness orchestrates CronCreate) | | `cli/cmd/ao/evolve_blocked.go` + `_test.go` | New subcommand: `--reason` (write), `--list [--tail N] [--json]` (read), `--clear <cycle>` (operator) | | Generated: `cli/docs/COMMANDS.md`, `registry.json`, `docs/cli-skills-map.md` | Regen for 3 new subcommands | | `evals/agentops-core/cli-command-surface-matrix.json` + smoke fixture | Counts bumped 73/199/272 → 74/202/276 | ## How tested - L2 integration: each new subcommand has L2 tests using fixture workspaces - L1 unit: 5-step ladder per-step table tests + JSONL schema validation on blocked records - Mechanical: `go test ./cli/...` 0 → 0 failures; cli-command-surface-smoke.sh `cli-help-matrix-ok`; check-no-tracked-agents.sh exits 0 ## Counts CLI heading counts: top 73 → 74, sub 199 → 202, all 272 → 276. Sibling pattern: `cron-history.jsonl` + `blocked.jsonl` follow the cycle-history.jsonl JSONL append-only shape from soc-5qit. Ladder structure mirrors the in-prompt cascade in `references/scout-mode.md` — making it programmatic per §A5. [no-sibling for cron-self-adjust] First-of-kind: no prior subcommand emits a cron-spec JSON for harness orchestration. The CLI does the safe work (template render + marker verify + audit row); the harness owns CronCreate. See: `docs/plans/2026-05-21-evolve-loop-epic-design.md` §A4, §A5, §A6 Closes-scenario: soc-mlbm#next-work-ladder Closes-scenario: soc-un0m#cron-self-adjust Closes-scenario: soc-g34d#typed-blocked-events Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/evolve_next_work.go	2026-05-21 14:09:41 -04:00
Bo	2ea3c12b7a	feat(evolve): --mode=burst|loop flag + mechanical write-stop-marker refusal (soc-hwax #mode-loop-flag) (#395) ... ## Why /evolve self-stops in operator-driven cron sessions because its defaults assume agent-as-operator. `--mode=loop` flips the contract: operator owns stop signals; CLI mechanically refuses self-written STOPs. ## What changed | Surface | Change | |---|---| | `cli/cmd/ao/evolve.go` | New `--mode string` flag (burst\|loop, default burst) + `validateEvolveMode` helper called from `runEvolve` | | `cli/cmd/ao/evolve_write_stop_marker.go` | New subcommand; refuses with exit 1 under `--mode=loop`, writes `.agents/evolve/<MARKER>` under burst | | `cli/cmd/ao/evolve_test.go` | L1: `--mode=invalid` rejected; mode flag registered with burst default | | `cli/cmd/ao/evolve_write_stop_marker_test.go` | L2 table tests: burst writes (DORMANT body assertion), loop refuses {dormant,stop,kill}, no file written under refusal | | `skills/evolve/SKILL.md` | Mode-aware HTML comments in Steps 1/3/7 + Flag table row (only edited Steps 1/3/7 + Flag table; verified via grep `^### Step`) | | `skills/evolve/references/loop-mode.md` | New reference doc with invariants + CLI primitives | | `docs/plans/2026-05-21-evolve-loop-epic-design.md` | Epic design memo (consumed by all 6 sub-beads of soc-g2qd) | | `cli/docs/COMMANDS.md` | Regenerated via `scripts/generate-cli-reference.sh` | | `skills-codex/{.agentops-manifest.json,evolve/.agentops-generated.json}` | Post-edit hook source-hash refresh; `check-codex-parity-drift.sh` PASS | ## How tested - `go test -count=1 ./cmd/ao/` — 10175 pass, 1 skip (full suite green after CLI-docs regen; pre-regen `TestCobraConformance` correctly flagged the doc drift) - `go build ./... && go vet ./...` — green - `grep '^### Step' skills/evolve/SKILL.md` confirms only Steps 1/3/7 edited (12 Step markers preserved at same positions) - `bash scripts/check-codex-parity-drift.sh` — PASS - Table tests assert exact marker filename (DORMANT/STOP/KILL) + body content for burst; for loop they assert error substring AND that no marker file was created. ## §A7 fallback decision **Chose in-file HTML comments** (not two-file split). Each Step comment is one line; no conditional sections needed. Two-file split would have ballooned the surface for a single-line contract reminder. CLI mechanical enforcement (`write-stop-marker` refusal) is the real invariant; SKILL.md comments only document intent for human readers. See: docs/plans/2026-05-21-evolve-loop-epic-design.md §A1 + §A7 ## TODOs the worker leaves for the epic - Wire `cli/internal/evolve.preferences.Load()` into `detectEvolveWriteStopMarkerMode` once soc-6svt lands (currently flag-only). - `ao evolve operator-stop` (Wave 2) — separate code path explicit operator override. - `ao evolve blocked` (Wave 2) — typed blocked event instead of STOP. Closes-scenario: soc-hwax#mode-loop-flag Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/evolve_write_stop_marker.go	2026-05-21 12:21:51 -04:00
Bo	2d2b806ab8	feat(evolve): per-repo preferences.yaml schema + loader + ao evolve config --show (soc-6svt #preferences-yaml) (#396) ... ## Why Different repos have different /evolve physics. Hardcoded defaults force ADRs per-repo to override. Per-repo YAML separates skill-doctrine from repo-policy. ## What changed | Surface | Change | |---|---| | `schemas/evolve-preferences.v1.schema.json` | New JSON Schema draft-07 for the v1 preferences shape | | `cli/internal/evolve/preferences.go` | New loader: `Load(ctx) (*Prefs, error)` with file:line error context | | `cli/cmd/ao/evolve_config.go` | New `ao evolve config --show [--json]` subcommand | | `.agents/evolve/preferences.yaml.template` | New starter template | | `.gitignore` | Negation rules for the template path | ## Resolution order defaults < preferences.yaml < CLI flag (caller-applied) Invalid keys/types -> exit 1 with `preferences.yaml:line:col` context. Never silent fallback. ## How tested - L2: malformed YAML -> `ao evolve config --show` exits 1, stderr has file:line - L1: `preferences.Load()` table tests cover defaults, valid, invalid types, out-of-range, unknown keys - `go test ./cli/internal/evolve/ ./cli/cmd/ao/` green (299+ tests across both) See: docs/plans/2026-05-21-evolve-loop-epic-design.md §A2 (created by parallel worker W1a in soc-hwax PR) Closes-scenario: soc-6svt#preferences-yaml Bounded-context: BC5-Runtime Evidence: cli/internal/evolve/preferences.go	2026-05-21 11:56:04 -04:00
Bo	1a7e2cb748	feat(ao session bootstrap): universal init prompt (soc-vuu6.25 #ao-session-bootstrap) (#390) ... ## Summary `ao session bootstrap` — every agent spawned into an AgentOps repo runs this first. Output shape is identical regardless of model (the fungibility contract: swap Claude for Codex on any bead without re-orienting). ## What Four fail-open substeps emit fields in `schemas/session-bootstrap.v1.schema.json`: | Field | Source | Fallback | |---|---|---| | `agents_md_read` + `agents_siblings_read` | `AGENTS.md` + post-soc-vuu6.3 siblings | `false` / empty array | | `onboard_phase` | `onboard --auto` (soc-vuu6.9 future) | `skipped:not-implemented` | | `ready_beads_count` | `bd ready --json` | `null` | | `mail_unread_count` | mcp-agent-mail | `null` (wiring is follow-up bead) | ## Why The fungibility philosophy mandates one init prompt for every agent regardless of type. Today every operator hand-rolls orientation; two agents spawned by two different operators don't share a starting frame. `ao session bootstrap` is the operational primitive that closes the gap. Tiny (~270 LOC including helpers), fail-open, deterministic. ## Files - New: `cli/cmd/ao/session_bootstrap.go` + `_test.go` (8 cases) - New: `schemas/session-bootstrap.v1.schema.json` (draft-07) - New: `skills/session-bootstrap/SKILL.md` + `skills-codex/session-bootstrap/SKILL.md` (mirrored) - Catalog: `skills/SKILL-TIERS.md`, `skills/using-agentops/SKILL.md`, `skills-codex/using-agentops/SKILL.md` - Regenerated: `cli/docs/COMMANDS.md` (sub 196→197, all 269→270), `registry.json` (cli_commands 173→174), `PRODUCT.md`/`docs/ARCHITECTURE.md`/`docs/SKILLS.md` (count sync 79→80), `evals/agentops-core/cli-command-surface-{matrix.json,fixtures/cli-command-surface-smoke.sh}` (fixture bump) ## Verification ``` cd cli && go build ./... && go test ./cmd/ao/ -run TestSessionBootstrap -count=1 # 8 passed bash evals/agentops-core/fixtures/cli-command-surface-smoke.sh # cli-command-headings: top=73 sub=197 all=270 / cli-help-matrix-ok bash tests/docs/validate-doc-release.sh # PASS: doc-release gate succeeded ``` Closes-scenario: soc-vuu6.25#ao-session-bootstrap Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/session_bootstrap.go Evidence: cli/cmd/ao/session_bootstrap_test.go Evidence: schemas/session-bootstrap.v1.schema.json Evidence: skills/session-bootstrap/SKILL.md	2026-05-21 04:27:09 +00:00
Bo	3655c55356	feat(scripts): auto-resume-stale-claims driver (soc-vuu6.27 slice 4a #auto-resume-driver) (#385) ... ## Why Slice 4a of soc-vuu6.27 (fungible-swarm death recovery). The bead originally specified an "agentopsd job spec" but the operational essence is a periodic driver that calls slice 2 + slice 3 — that intent is met by a shell wrapper today, faster and with smaller surface than touching agentopsd internals. Slice 4b (`` filed) tracks the agentopsd-native variant for later. **Stacked on slice 3's branch** (which is stacked on slice 2). When the chain merges, this rebases onto main. ## What `scripts/auto-resume-stale-claims.sh` — one-shot driver suitable for cron, systemd timers, or future agentopsd job wrapping: 1. Calls `ao beads stale-claims --threshold=<N>h --json` (slice 2). 2. For each stale bead, calls `ao beads resume <id> --agent <recovery-agent>` (slice 3). 3. Per-bead OK/FAIL output + summary line. Exit code summarizes the run. Flags: `--threshold` `--agent` `--dry-run` `--max` (cap per-run transfers; default 25) `--quiet`. Exit 0 = success or no candidates; exit 1 = at least one failure; exit 2 = usage/env error. ## Test 9 bats tests, all passing. Stubs `ao` via PATH so the harness is deterministic. Covers: zero candidates, multi-candidate happy path, --dry-run, partial failure, --max cap, --threshold passthrough, --quiet suppression, unknown-flag error, missing-ao error. Self-dogfood: With slices 2 and 3 landed plus a stale-claim simulator, running this driver against the real bd backend transfers stale claims and emits one claim_transferred event per transfer (slice 3 contract). ## Follow-up - Slice 4b (``) — agentopsd-native job spec wrapping this driver. Closes-scenario: soc-vuu6.27#auto-resume-driver Bounded-context: BC2-Loop Evidence: scripts/auto-resume-stale-claims.sh Evidence: tests/scripts/auto-resume-stale-claims.bats --------- Co-authored-by: Codex <codex@example.invalid>	2026-05-21 00:37:27 +00:00
Bo	57b2eabd7e	feat(ao beads): stale-claims list command (soc-vuu6.27 slice 2 #stale-claims-list) (#383) ... ## Why Slice 2 of soc-vuu6.27 (fungible-swarm death recovery). The fungibility doctrine ("Dead agent? Start another.") needs an operational primitive. This is the read-only listing half — surfacing stale claims so an operator (or future slice-4 daemon job) can decide what to do. Slice 1 (PR #382) shipped the contract; this slice consumes it. ## What `ao beads stale-claims [--threshold=4h] [--json]` — walks `bd list --status in_progress --json`, applies a staleness heuristic (last activity > threshold), emits a sorted listing (oldest first) or a JSON array of records conforming to `schemas/stale-claim-event.v1.schema.json` (event_type `stale_detected`). Files: - `cli/cmd/ao/beads_stale.go` — cobra subcommand + pure `computeStaleEvents` helper + exec seam for tests - `cli/cmd/ao/beads_stale_test.go` — 5 test functions, 7 table sub-cases + L2 integration via the seam - `cli/cmd/ao/beads.go` — one-line registration (`beadsCmd.AddCommand(beadsStaleCmd)`) Read-only: this command DOES NOT transfer claims. Slice 3 handles that. Slice scope honors complexity budget — the orchestration function is two short helpers (RunE wrapper + pure `computeStaleEvents`), each well under the warn-at-15 threshold. ## Test `cd cli && go test ./cmd/ao -run 'TestComputeStaleEvents|TestRunBeadsStale'`: - TestComputeStaleEvents_TableDriven (7 sub-cases) - TestComputeStaleEvents_AssigneeDefaultUnknown - TestRunBeadsStale_JSON_OutputShape (L2 integration via seam) - TestRunBeadsStale_HumanOutput_HasZeroMessage - TestRunBeadsStale_BdMalformed - TestComputeStaleEvents_ClaimAgeRounding All passing. `gofmt -l` clean, `go vet ./...` clean, `go build ./...` clean. Self-dogfood (in this worktree, smoke): ``` $ ao beads stale-claims --threshold=4 ao beads stale-claims: 47 in_progress bead(s) stale (threshold 4.0h) soc-t3z1 claim_age=21.5h last_touch=2026-05-19T22:30:00Z ... ``` ## Follow-ups already filed - Slice 3 — `ao beads resume <id>` atomic transfer (blocked on this PR) - Slice 4 — agentopsd auto-resume daemon job (blocked on slice 3) Closes-scenario: soc-vuu6.27#stale-claims-list Bounded-context: BC5-Runtime Evidence: cli/cmd/ao/beads_stale.go Evidence: cli/cmd/ao/beads_stale_test.go --------- Co-authored-by: Codex <codex@example.invalid>	2026-05-21 00:22:31 +00:00
Bo	c8edc80143	feat(wiki): wiki bounded context waves 1-4 (epic soc-behj) (#291) ... ## Summary Phased strangler consolidating `ao`'s `.agents/`-touching logic into one `cli/internal/wiki` bounded context. Waves 1-4 of epic **soc-behj** (8 beads, 5 waves). Strictly accretive — no `ao` command removed, legacy command surface untouched. Each wave boundary was gated on the 102 `cmd/ao` integration tests staying green. | Wave | Bead | Deliverable | |------|------|-------------| | W1 | soc-1lju | `FrontmatterCodec` — all 5 frontmatter parsers delegate to one codec | | W2 | soc-36lw | `CorpusLocator` — `agentsDirIn` moved, 17 call sites migrated | | W2 | soc-f4tr | `Artifact` + `Claim` domain types with subtype invariants | | W3 | soc-vot0 | persistent `WikiIndex` — JSONL, incremental by content-hash | | W3 | soc-r08p | `FreshnessPolicy` — claim-level, evidence-driven | | W4 | soc-ijp8 | `WikiPipeline` — subsumes the llmwiki loop, 3 stages wired | | W4 | soc-q50a | port conformance suite | Also anchors `.gitignore`'s bare `wiki/` pattern with a `!cli/internal/wiki/` negation — the new package was being silently ignored. Wave 5 (`ao wiki` command group) is **not** in this PR. ## Test plan - [x] `go build ./...` clean - [x] `go vet ./internal/wiki/` clean - [x] `go test ./internal/wiki/` — 96 pass - [x] `go test ./cmd/ao/ -run Integration` — 102 pass (regression net green) - [ ] CI `validate.yml` --------- Co-authored-by: Codex <codex@example.invalid>	2026-05-18 13:29:12 +00:00
Bo	e7f45c92ba	chore(ci): require claude-review check + remove always-failing agentops-eval-advisory (soc-7ltz #claude-review-required) (#300) ... ## What Two coupled CI surface changes shipped as one PR (soc-7ltz closes both scenarios). ### 1. `claude-review-required` Branch protection on `main` now requires **`claude-review`** in addition to `summary`. The Claude GitHub App's Code Review workflow (`.github/workflows/claude-code-review.yml`) posts the `claude-review` check on every PR. This turns the AGENTS.md/CLAUDE.md `## Workflow` block's "0 reviews + required `claude-code-review` check" line from **aspirational** to **enforced**. **Already applied** before this commit was pushed; verifiable now: ``` gh api repos/boshu2/agentops/branches/main/protection -q '.required_status_checks.contexts' # → ["summary", "claude-review"] ``` ### 2. `eval-advisory-removed` Deleted the `agentops-eval-advisory` job from `validate.yml`. It was `continue-on-error: true` + `(advisory)` — never blocked, but failed on 3+ consecutive `main` runs (eval canaries: `goals-management-lifecycle`, `security-toolchain-governance`, `workbench-behavioral-v1`). Pure noise on every PR's check list. **Eval coverage preserved:** - `agentops-eval-baseline-audit` (required) — drift-only gate, continues to run. - `eval-workbench-verify` (required) — workbench correctness. - `eval-skill-delta` (required) — skill-change deltas. **Stale references stripped:** - `validate.yml`: `summary` job's `needs:` list and `echo` debug line. - `AGENTS.md`: 3 separate stale mentions (the "summary gates on all except..." sentence, the Advisory SLA table row, the CI gate detail row). ## How this dogfoods the discipline - Branch off `origin/main`, named `chore/soc-7ltz-claude-review-required` (≤80 chars, bead-id + scenario-token). - PR title: `<type>(<scope>): <subject> (<bead-id> #<scenario-slug>)` per the rule. - Body trailers: `Closes-scenario:` (×2), `Bounded-context:`, `Evidence:`. - Squash-merge will close the bead (both scenarios shipping in this PR). - The PR itself will be gated by the new `claude-review` requirement (it's added to the required-checks list before this PR opens, so the PR is the first one bound by it). ## Pre-existing CI parity drift (NOT this PR) Local `validate-ci-policy-parity.sh` flags drift on `doctor-check` and `executable-spec-link-integrity` (both unrelated to this PR; they showed up on `main` before any of my edits). Filed as a follow-up bead. ## Acceptance - `gh api .../main/protection` returns `claude-review` in required contexts ✅ - `gh pr checks` no longer shows `agentops-eval-advisory` ✅ (will verify on this PR) - `summary` job still runs cleanly (its `needs:` list no longer references the removed job) - `claude-review` posts and passes on this PR Co-authored-by: Codex <codex@example.invalid>	2026-05-17 21:14:57 -04:00
Bo	fc16cf31df	feat(release): require zero-trust audit evidence (#297) ... ## Summary - add eval fast, eval baseline-audit, VIL, and digital-twin evidence to local release artifact manifests - require resolver/audit validation to reject incomplete SBOM/security/eval/HIL/VIL/twin proof bundles - document the zero-trust release proof bundle and add focused BATS coverage ## Validation - bash -n scripts/ci-local-release.sh scripts/resolve-release-artifacts.sh scripts/validate-release-audit-artifacts.sh - shellcheck --severity=error scripts/ci-local-release.sh scripts/resolve-release-artifacts.sh scripts/validate-release-audit-artifacts.sh - bats tests/scripts/ci-local-release.bats tests/scripts/release-artifacts.bats - git diff --check - markdownlint docs/RELEASING.md docs/release-e2e-checklist.md docs/contracts/release-readiness.md - bash scripts/validate-ci-policy-parity.sh - bash tests/docs/validate-doc-release.sh - scripts/pre-push-gate.sh --fast (passed; warn-only executable-spec link-integrity reports unknown flags --lint/--orphans) - bd dep cycles - scripts/bd-audit.sh --json Bead: soc-owed.7 --------- Co-authored-by: Codex <codex@example.invalid>	2026-05-18 01:14:33 +00:00
Bo	1f0e545a51	Executable spec layer (epic soc-58nt, F2–F5): GOALS as BDD acceptance criteria (#292) ... * feat(goals): F2.0 scenario-result artifact contract + producer/writer bead: soc-58nt.2.6 * docs(goals): F4.0 trace link convention ADR-0005 bead: soc-58nt.4.8 * docs(goals): F3.0 domain-slice manifest contract + ADR-0004 bead: soc-58nt.3.8 - schemas/domain-slice-manifest.v1.schema.json: JSON Schema (draft/2020-12, additionalProperties:false) for domain-slice manifests. All required fields: domain, version, bounded_context, directive_ids (d-<slug> pattern), scenario_ids, context_roots, allowed_read_globs, denied_read_globs, validation_commands, owner. - docs/adr/ADR-0004-domain-slice-manifest-contract.md: records four decisions: (A) command shape is ao rpi phased --domain <name>; (B) manifest is durable tracked artifact at docs/domains/<name>/manifest.yaml; (C) reconciles with the domain skill (vocabulary), context-map.md (architecture view), and skill frontmatter — no overlap; (D) Go model named domainSliceManifest, explicitly distinct from rpi_phased_manifest.go's phaseManifest. Cross-references ADR-0003. - docs/domains/README.md: explains the directory, field table, relationship to the three pre-existing domain surfaces, and how to add a new slice. - docs/domains/example/manifest.yaml: fully populated example; validated against the schema via jsonschema (PASSED). * feat(goals): F2.1 scenario-result aggregation reader bead: soc-58nt.2.1 * feat(rpi): F3.1 domainSliceManifest model + loader Implements the domainSliceManifest Go model and loader in cli/internal/domainslice/ per the F3.0 contract (ADR-0004, Decision D). Uses gopkg.in/yaml.v3 with KnownFields(true) to reject unknown fields, mirrors the schema's additionalProperties:false. Validates all required fields, directive_id pattern (^d-[a-z0-9][a-z0-9-]*$), context_roots minItems:1, and validation_commands sub-fields with field-named errors. Explicitly distinct from phaseManifest (rpi_phased_manifest.go). 28 tests pass (L1 unit + L2 fixture round-trip via docs/domains/example/manifest.yaml). bead: soc-58nt.3.1 * feat(goals): F4.1 read-only executable-spec trace graph walker bead: soc-58nt.4.1 * feat(goals): F2.2 per-directive scenario_satisfaction + threshold verdict bead: soc-58nt.2.2 * feat(rpi): F3.2 ao rpi phased --domain scoping bead: soc-58nt.3.2 * feat(goals): F4.2/F4.3 ao goals trace --from / --orphans beads: soc-58nt.4.2 soc-58nt.4.3 * feat(goals): F4.4 ao goals render — GOALS.md to Gherkin bead: soc-58nt.4.4 * feat(rpi): F3.3a domain-scope audit evidence bead: soc-58nt.3.3 * feat(goals): F2.3 scenario_satisfaction JSON field + --scenarios-only Per-directive scenario satisfaction added to ao goals measure: 0 -> 8 new JSON fields and a --scenarios-only mode that skips gate execution. bead: soc-58nt.2.3 * feat(rpi): F3.4 ao rpi phased --scaffold-domain bead: soc-58nt.3.4 * test(goals): commit missing goals-trace scenario-results fixture The F4.1/F4.2 walker tests reference a scenario-results.json fixture under a gitignored .agents/ subpath; it was never force-added and a clean checkout (and CI) lacks it, failing 5 scenario_result tests. Force-add it. bead: soc-58nt.4.1 * fix(goals): F4.1 tighten goalstrace scenario-claim heuristic bead: soc-58nt.4.9 - Tighten scenarioTokenRe: auto- tokens now require at least two hyphen-separated slug segments (auto-X-Y…), so single-word English compounds like auto-merge and auto-update are no longer matched. - Downgrade broken_bead_scenario_claim from error to warning when the claim comes from a heuristic (ConfidenceLow) free-text match; per ADR-0005 §4.1 only an explicit Scenarios: line constitutes a broken explicit link and may produce an error. - Add beads_test.go with table-driven tests covering: English auto-* false-positive rejection, real multi-segment auto-* ID detection, explicit missing scenario → error, heuristic missing → warning only, resolvable scenario → no defect, end-to-end Walk with auto-merge bead produces no error-severity finding. * test(goals): F2.T1 regression-coverage audit + gap fill bead: soc-58nt.2.4 * ci(goals): F1.6 wire executable-spec link lint + trace-orphans (warn-first) bead: soc-58nt.1.9 * feat(rpi): F3.3b runtime hook enforcement of domain scope bead: soc-58nt.3.9 * test(goals): F2.T2 e2e script for scenario-satisfaction gate bead: soc-58nt.2.5 * docs(goals): F5.0 re-steer policy + mutation-safety contract (ADR-0006) bead: soc-58nt.5.9 * docs(goals): fix streak-reset prose in ADR-0006 (cleanup) bead: soc-58nt.5.9 [no-sibling] prose fixup only, no structural change 0 → 0 schema fields changed * test(goals): F4.T1 regression-coverage audit + gap fill bead: soc-58nt.4.6 * feat(wiki): land wiki bounded context waves 1-4 (epic soc-behj) Phased strangler consolidating ao's .agents/-touching logic into one cli/internal/wiki bounded context. Each wave gated by the 102 cmd/ao integration tests staying green; legacy command surface untouched. - W1 soc-1lju FrontmatterCodec — all 5 frontmatter parsers delegate - W2 soc-36lw CorpusLocator — agentsDirIn moved, 17 call sites migrated - W2 soc-f4tr Artifact + Claim domain types with subtype invariants - W3 soc-vot0 persistent WikiIndex — JSONL, incremental by content-hash - W3 soc-r08p FreshnessPolicy — claim-level evidence-driven freshness - W4 soc-ijp8 WikiPipeline — subsumes llmwiki loop, 3 stages wired - W4 soc-q50a port conformance suite Also anchors .gitignore's bare wiki/ pattern with a !cli/internal/wiki/ negation — the new package was being silently ignored. Wave 5 (ao wiki command group) is not included in this commit. * test(goals): F4.T2 e2e script for trace chain bead: soc-58nt.4.7 * test(rpi): F3.T1 regression-coverage audit + gap fill bead: soc-58nt.3.6 * test(rpi): F3.T2 e2e script for domain-scoped RPI bead: soc-58nt.3.7 * feat(goals): F5.1 verdict ledger schema + writer bead: soc-58nt.5.1 * feat(goals): F5.2 re-steer policy engine + verdict-ledger producer hookup bead: soc-58nt.5.2 * feat(goals): F5.4 feedback-to-learning compiler Adds cli/internal/feedbackcompiler — scans the verdict ledger for fail->pass directive transitions and drafts a learning entry in docs/learnings/ for each transition found. Drafts carry status: draft and directive_id frontmatter (ADR-0005 §2.6). Never auto-promotes. Idempotent: skips existing drafts on re-run. Also documents the auto-draft workflow in skills/forge/references/feedback-compiler-drafts.md with a compact reference-link in skills/forge/SKILL.md (under 248-line limit). Synced to skills-codex/forge/ and regenerated codex hashes. bead: soc-58nt.5.4 * docs(goals): F5.5 compound-engineering retro in /post-mortem Extend the /post-mortem skill with a Compound-Engineering Retro section that compares iteration N vs N-1 for a domain slice using the F5.1 verdict ledger (.agents/goals/verdict-ledger.json). The mode emits a comparative delta — directives that improved (fail→pass, satisfaction up), regressed (pass→fail, satisfaction down), or held stable — plus learning yield since N-1, written as a status:draft learning to .agents/learnings/. Detailed step-by-step procedure (CE.0–CE.5) lives in references/compound-engineering-retro.md; SKILL.md carries a compact trigger/commands summary and links to it. Both files synced to skills-codex/post-mortem/ with updated codex hashes. bead: soc-58nt.5.5 * feat(goals): F5.3 ao goals steer --auto with human-gated confirmation bead: soc-58nt.5.3 * test(goals): F5.T1 regression-coverage audit + gap fill bead: soc-58nt.5.7 * test(goals): F5.T2 e2e script for auto re-steer bead: soc-58nt.5.8 * docs(goals): F2.4 docs regen for scenario-satisfaction gate + epic CLI artifacts Regenerate cli/docs/COMMANDS.md and registry.json from a clean worktree at HEAD so the soc-58nt command surface is documented without leaking unrelated peer WIP into the generated docs. F2 (soc-58nt.2.7): document scenario_satisfaction JSON shape, --scenarios-only, result-artifact resolution, and exit codes in skills/goals/SKILL.md + references/executable-spec-chain.md; cross-ref ao goals measure/trace from skills/scenario/SKILL.md. Epic-wide artifacts also landed here because they regenerate atomically across F2-F5: COMMANDS.md/registry.json (all new ao goals/rpi surface), cli-skills-map heading count, skills-codex hashes, ADR-0004/0005/0006 documentation-index links, and the F2-F5 e2e CI lanes in validate.yml. ao capabilities and ao robot-docs need no manual edits — both build their command/flag surface from the live cobra tree, so new commands register automatically (Global Rule G5 satisfied by construction). bead: soc-58nt.2.7 * docs(domain): F3.5 domain-as-loop docs for domain-scoped RPI Document the domain-slice runtime in skills/domain/SKILL.md (a "Domain as a scoped RPI loop" section connecting the Slice primitive to ao rpi phased --domain / --scaffold-domain / --force) and add the Domain-Slice row to the modes table in skills/scaffold/SKILL.md, mirroring the existing Project/ Component/CI mode-row shape. The scaffold skill's Domain-Slice Mode section already covered the workflow; this lands the missing modes-table entry so the mode count is 3 -> 4. The COMMANDS.md/registry/CI-lane regen for F3 landed in soc-58nt.2.7. skills-codex hashes for the two changed skills refreshed via scripts/regen-codex-hashes.sh; codex-parity audit clean. bead: soc-58nt.3.5 * docs(goals): F5.6 re-steer loop docs in /post-mortem Cross-reference the auto re-steer loop from the Compound-Engineering Retro section of skills/post-mortem/SKILL.md: when the compound retro names a chronically regressing directive, ao goals steer recommend prints policy-driven directive mutations from the same verdict ledger and ao goals steer apply writes the mutation to GOALS.md, human-gated via the non-lossy patcher (ADR-0006). Mirrors the existing "closing the loop" cross-ref shape used elsewhere in the skill's See Also prose. The ao goals steer recommend/apply COMMANDS.md surface, the F5 e2e CI lane, and the ADR-0006 documentation-index link landed atomically in soc-58nt.2.7's epic-wide regen; the re-steer contract detail lives in skills/goals/references/executable-spec-chain.md (also in 2.7). skills-codex/post-mortem hash refreshed; codex-parity audit clean. bead: soc-58nt.5.6 * docs(goals): F4.5 docs regen for goals trace + render (epic-subsumed) F4's docs deliverables — the `ao goals trace` and `ao goals render` entries in cli/docs/COMMANDS.md, the F4 trace-chain e2e CI lane in validate.yml, the ADR-0005 documentation-index link, and the Trace/Render mode sections plus the trace contract in skills/goals/SKILL.md + references/executable-spec-chain.md — all co-landed atomically in soc-58nt.2.7's epic-wide CLI-reference regen (commit 37798abe), because COMMANDS.md/registry.json regenerate as one unit across F2-F5 and cannot be split per-bead. This empty commit records soc-58nt.4.5 as complete with no further file changes. Matches the epic-subsumed-bead convention: regen artifacts land once, later beads point at the lead regen commit. bead: soc-58nt.4.5 * test(goals): fix flag-global leak breaking full goals-measure run TestGoalsIntegration ran 'goals measure --directives' via cobra, which sets the package-global goalsMeasureDirectives and never resets it. soc-58nt's new TestGoalsMeasure_FullMode* tests branch on that global, so the unfiltered 'go test ./cmd/ao/...' run failed (gates skipped, directives array emitted instead of the full snapshot+scenarios object). -run filters masked it. Restore the global after Execute. bead: soc-58nt.2.4 * fix(ci): correct soc-58nt skill/codex artifacts and CI-policy parity Resolves 6 red CI checks on PR #292 (crank/soc-58nt) after merging main: - skill-integrity: reword the "no scaffold subcommand" note in skills/scaffold/SKILL.md so it no longer trips the heal INVALID_AO_CMD substring check; copy skills/forge/references/examples.md into skills-codex/forge/references/ so the codex forge dead-link clears. - validate-codex-generated-artifacts: strip leaked non-Codex frontmatter from skills-codex/forge/SKILL.md down to name + description. - validate-codex-runtime-sections: remove the residual mixed-runtime "Cross-vendor analog" line (Anthropic Managed Agents) from codex forge; convert /forge slash-command refs to $forge Codex invocation style. - validate-ci-policy-parity: add executable-spec-link-integrity (non-blocking) to the AGENTS.md non-blocking list so it matches the workflow's continue-on-error classification (warn-only F1.6 gate added by soc-58nt without parity co-update). - doc-release-gate: the link-validation broken link was the same codex forge references/examples.md dead link, fixed above. - agentops-contract-canaries: update the cli-command-surface canary fixture + eval JSON for the 185->187 / 257->259 subcommand count after soc-58nt added `ao goals render` and `ao goals trace`. Also ports the soc-58nt "Closing the loop with re-steer" paragraph into skills-codex/post-mortem/SKILL.md and regenerates codex artifact hashes and registry.json after the merge. * fix(test): set repo-local git identity in rpi-phased-domain e2e CI runners have no global git identity; the temp-repo 'git commit' in the F3.7 e2e exited 128 (empty ident name). Configure a repo-local identity. bead: soc-58nt.3.7 * fix(goals): use canonical practice slug bdd-gherkin soc-58nt files cited the practice slug 'bdd'; the canonical slug in PRACTICE-REGISTRY.md is 'bdd-gherkin'. Clears the practice-citations advisory check (9 invalid slug citations -> 0). bead: soc-58nt.2.7 * fix(goals): add missing practices field to F5 gap-test file goals_steer_auto_gap_test.go lacked a // practices: header; the practice-citations --strict gate flags missing fields. Matches its sibling goals_steer_auto_test.go. bead: soc-58nt.5.7 * fix(security): resolve gosec G122 + golangci-lint errcheck/staticcheck HIGH findings Drive scripts/security-gate.sh --mode quick to 0 HIGH findings. gosec G122 (CWE-367 TOCTOU): goalstrace/artifacts.go reads files inside a filepath.WalkDir callback via an os.Root-scoped handle (os.OpenRoot), closing the check-to-use window and blocking symlink escape. golangci-lint errcheck (~25): unchecked defer Close / os.Remove returns. Resource-cleanup closes use explicit-ignore (defer func(){ _ = X.Close() }()). Durability-gating closes before a rename (llm/review.go tmp.Close) are checked and wrapped. golangci-lint staticcheck (~21): ST1005 (drop trailing punctuation from error strings), QF1001 (De Morgan's law), QF1002 (tagged switch), QF1012 (fmt.Fprintf over WriteString+Sprintf), S1016 (struct-literal -> conversion), S1017 (strings.TrimSuffix), S1040 (drop redundant type assertion), SA1012 (context.TODO over nil Context), SA4032 (drop dead GOOS branch under build constraint), SA9003 (remove empty branch). * ci(security): make security-toolchain-gate blocking Remove continue-on-error: true from the security-toolchain-gate job and drop the "(advisory)" name suffix so a CRITICAL/HIGH security-gate finding fails the validate summary job. The job is already in summary.needs, so removing continue-on-error promotes it into the contains(needs.*.result,'failure') fail-set. AGENTS.md: drop security-toolchain-gate from the (non-blocking) prose list and the Advisory Job Triage SLA table; update its CI-jobs-table row to describe the blocking failure mode. validate-ci-policy-parity confirms the AGENTS blocking set matches the workflow summary fail-set (7 non-blocking). --------- Co-authored-by: Codex <codex@example.invalid>	2026-05-17 15:31:15 -04:00
Bo	a984b742d9	feat(skills): productize skill factory workflow (#290)	2026-05-17 09:56:32 -04:00
Boden Fuller	cf9de949d6	fix(eval): sync canaries + json-flag test to the ao doctor surface ... The ao doctor engine (PRs #281-#284) drifted three offline canaries that CI path-filtering skipped on the doctor PRs themselves: - tests/cli/test-json-flag-consistency.sh hard-failed on any non-zero exit, but `ao doctor --json` exits 1 by contract when findings exist. test_json_cmd now takes --diagnostic (accepts exit 0/1; >=2 still fails), mirroring the diagnostic handling in flag_matrix_test.go. Errors 1 -> 0 (16/16 pass). - cli-command-surface-matrix: the 9 new `ao doctor` subcommands grew the generated COMMANDS.md heading count from 173/243 to 184/254. Updated both the fixture's internal guard and the eval's stdout_contains. - distribution-install-update: PR #283 changed test-windows-smoke.ps1 from `$builtAO doctor --json` to plain `$builtAO doctor`; updated the artifact_contains expectation to match.	2026-05-16 16:20:16 -04:00
Bo	2973ba05ea	fix(canaries): repair contract-canary drift in pre-push + hook suites (#280) ... Two unrelated failures kept agentops-contract-canaries (required) red: 1. pre-push-gate-governance pinned TAP line "ok 42 ...test-home- isolation lint stub..." but three bats tests were inserted ahead of it, shifting the test to position 45 (55 @test total). Bumped the pin: ok 42 → ok 45. 2. test-hooks.sh aborted intermittently with exit 141. Three `VAR=$(ls -t ... | head -1)` assignments run under `set -euo pipefail`; when `ls` emits >1 line, `head` closes the pipe early and the SIGPIPE'd `ls` makes the pipeline exit 141, tripping `set -e` before the suite prints its summary. Added `|| true` to the three pipelines — captured value is unchanged, exit status neutralized. Suite now completes: Total 213 | Fail 0 | ALL PASSED. Canary failures fixed: 2 → 0.	2026-05-16 11:23:07 -04:00
Boden Fuller	b8c5a3761b	feat(agentops): make 3.0 hookless-first ... Add DDD skill-domain contracts, hook lease inventory, replacement ports, artifact-first RPI skill boundaries, and release exact-SHA validation. Demote ao inject to a compatibility adapter and remove prompt-only default hooks.	2026-05-16 09:43:08 -04:00
Boden Fuller	5428adbc5c	fix(ci): clear 3 validate failures shipped red in v2.41.0 ... The v2.41.0 release commit (eb119f48) shipped with three CI checks already red — they were `skipped` in the intervening push (path filter didn't trigger them), so the release validation missed them. registry-check — registry.json was stale. The v2.41 BC arc added 10 new `ao` subcommands (cycles 144-153) without regenerating the registry. Ran scripts/generate-registry.sh: 158 CLI commands, 77 skills, 46 hooks now recorded. skill-integrity — `references/flags.md not linked in SKILL.md`. The cycle-d59c0d60 size-fix extracted the flags table to references/flags.md but linked it with a backtick code-span (`` `references/flags.md` ``) instead of a markdown link. heal.sh --strict requires a real link or a Read instruction. Changed to `[references/flags.md](references/flags.md)`. contract-canaries (two suites): - cli-command-surface-matrix: the heading-count baseline was stale (top=64 sub=159 all=223) — the v2.41 arc grew the surface to top=70 sub=173 all=243. Updated both the eval expectation and the fixture script's hardcoded guard. - distribution-install-update: the codex-native install canary flags `/skillname` slash-command references in skills-codex/**/SKILL.md (Codex uses `$skillname` notation). The mandatory-release-gates commit (176d45a4) copied `/release` and `/evolve` verbatim from the source skills into skills-codex/evolve + validation. Converted all 7 refs to `$release` / `$evolve`. Codex hashes regenerated for evolve + validation. Verified locally: heal.sh --strict clean; cli-command-surface-smoke.sh passes; test-codex-native-install.sh "PASS: Codex-native install flow verified (77 skills)"; generate-registry.sh diff committed.	2026-05-15 07:44:01 -04:00
Boden Fuller	cfc59bdcd4	fix(evals): cli-command-surface-matrix canary count drift ... Closes soc-l4yt — pre-push-gate-governance canary flake investigation. Why - Cycle 22 (D11) added `ao corpus snapshot` and `ao corpus restore` subcommands, which changed cli/docs/COMMANDS.md's heading counts: sub 157→159, all 221→223 (top unchanged at 64). - evals/agentops-core/fixtures/cli-command-surface-smoke.sh asserts the counts as hardcoded literals on 3 lines. - evals/agentops-core/cli-command-surface-matrix.json asserts the emitted stdout string `cli-command-headings: top=64 sub=157 all=221`. - Both went stale on the cycle-22 commit. The result: any push that triggered the canary suite failed on documented-cli-help-matrix with aggregate=0.7917. This was misdiagnosed as the pre-push-gate- governance flake (soc-l4yt) for most of the session — the actual pre-push-gate-governance canary was passing. What - evals/agentops-core/fixtures/cli-command-surface-smoke.sh: updated the 3 hardcoded count checks (sub: 157→159 at line 20, all: 221→223 at line 20, commands[@] size: 221→223 at line 28). Fixed two printf quoting errors collateral to the sed replacement (lines 21, 42) — shellcheck clean. - evals/agentops-core/cli-command-surface-matrix.json: updated the expected stdout assertion to top=64 sub=159 all=223. Verification - bash evals/agentops-core/fixtures/cli-command-surface-smoke.sh prints "cli-command-headings: top=64 sub=159 all=223" and "cli-help-matrix-ok" -- PASS. - shellcheck on the modified smoke script -- clean. - 25 consecutive runs of bash scripts/test-agentops-contract- canaries.sh after the fix -- all 25 PASS. Total session evidence 73 PASS / 2 FAIL (the 2 fails clustered immediately after the fix landed; subsequent 50+ runs all green). - bash scripts/pre-push-gate.sh --fast WITHOUT the documented PRE_PUSH_SKIP_EVAL=1 override -- green for the first time this session. The original soc-l4yt flake description (pre-push-gate-governance returning aggregate=0.9549 score=0.8333 with just exit_code failing) did NOT reproduce in 5 direct bats runs + 75 canary-suite runs. Either the flake required corpus state from the cycle-17 session that no longer exists, or the count-drift fix incidentally cleared the failure path. Bead annotated with both possibilities; bats 5/5 green stretch + 25 consecutive canary suite passes are sufficient evidence to close on "not reproducing + adjacent canary drift fixed". Operator note: the documented PRE_PUSH_SKIP_EVAL=1 override stays in place as a runtime release valve for future canary regressions; do NOT delete it. Companion: D11 (soc-ymph.11) shipped the corpus subcommands that caused the drift; this fix closes the eval surface that didn't get updated alongside.	2026-05-12 03:53:47 -04:00
Boden Fuller	8dcfd92d47	feat(eval): D10 head-to-head delta gate against baseline scorecard ... Closes soc-ymph.10 — Directive D10 (Behavioral eval as default blocking gate). Why - eval-workbench-verify was structural-only: it confirmed task count ≥ 9 and that suites parse. A PR that deleted half the task fixtures or downgraded `evidence_kind` would have passed because there were no comparison points. - Bead acceptance: "PR that introduces a skill regression has its CI fail with a delta scorecard artifact." What - evals/workbench/baseline-scorecard.json (NEW, tracked): structural baseline at HEAD 0c0b79e2. Tracks 9 fields — task_count, prompt_count, agent_eval_cases, behavioral_eval_cases, evidence kind, scoring dimensions, component presence flags. - scripts/check-eval-workbench.sh: extended. After the existing structural checks, the script now generates evals/workbench/ scorecard-latest.json (a snapshot of the current state) and compares to the baseline. Any decrease in numeric fields, any flip of a tracked bool from true to false, or evidence_kind drift fires a REGRESSION block and exits 1 with the delta payload. - evals/workbench/Makefile: new `head-to-head` target invokes the same check so an operator can reproduce the gate locally. - .github/workflows/validate.yml: existing eval-workbench-verify job picks up the upgraded script automatically (gate body is the same script). Added always-on artifact upload of scorecard-latest. json so a CI reviewer sees the delta when a regression fires. - .gitignore: scorecard-latest.json is regenerated, not tracked. Refresh baseline (when an intentional drop is approved): make -C evals/workbench head-to-head cp evals/workbench/scorecard-latest.json evals/workbench/baseline-scorecard.json git commit -m "chore(eval): refresh workbench baseline scorecard" Verification - Smoke: gate PASS at HEAD ("12 tasks, 12 agent cases, 12 behavioral cases; D10 delta check OK"). - Regression smoke: synthesize a baseline with inflated counts → gate exits 1 with REGRESSION block. Restore → gate PASS. - validate-ci-policy-parity: PASS (54 jobs; no new top-level job, the upgrade is in-place to existing eval-workbench-verify). - pre-push --fast: green. Companion: soc-ymph.10 (D10).	2026-05-11 18:29:30 -04:00