mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
ao-pre-reduction@5b334a13f
665 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Bo
|
f4162649b8 |
fix(agentops): remove api-print worker paths (#804)
## Summary - replace AgentOps headless Claude print worker paths with Codex exec or non-CLI providers - switch compile auto-detect/default headless CLI runtime from claude-cli to codex-cli - fix two shellcheck blockers from the merged skill corpus and refresh generated docs/hash metadata ## Validation - bash /Users/bo/acfs/skill-pipeline/scripts/no-api-print-scan.sh /Users/bo/dev/agentops/skills /Users/bo/dev/agentops/scripts /Users/bo/dev/agentops/hooks - shellcheck --severity=error skills/compile/scripts/compile.sh scripts/eval-agent-harness.sh scripts/run-rpi-phases.sh skills/agy-rules-workflows/scripts/close-guard.sh skills/rch/scripts/mine_rch_history.sh - cd cli && go test ./cmd/ao -run 'TestResolveCompileRuntime|TestPreflightCompileRuntime|TestCompileScriptOptionsPassesBatchFlags|TestRunCompile_EndToEnd_FixtureCorpus' -count=1 - bash scripts/regen-all.sh --check - PRE_PUSH_SKIP_MKDOCS=1 bash scripts/pre-push-gate.sh --fast Note: unscoped /Users/bo/acfs/skill-pipeline/scripts/no-api-print-scan.sh still flags ACFS conformance-selftest fixture strings outside this repo; AgentOps scoped roots are clean. |
||
|
Bo
|
69b8ba0a4f |
feat(liveness): add inbound admission core (ag-c3c7) (#757)
## Summary - Add a substrate-neutral inbound admission primitive in `cli/internal/liveness`. - Gate inbound work messages by source class and intent: authenticated operator directives may execute; peer/other-host/tmux directives are downgraded to proposals and cannot execute reflexively. - Require ratified quorum authority for quorum-sourced directives, with cross-model ACK verification for significant actions. - Extend the liveness composition test so the inbound admission path composes with role, quorum, judge, and lease primitives. ## Validation - `go test ./internal/liveness -count=1` - `go test ./...` from `cli/` - `PRE_PUSH_FAIL_FAST=false bash scripts/pre-push-gate.sh` ran and blocked on existing repo/environment gates: - `test-fixture parity break`: missing pre-push helper Bats stubs for `scripts/check-author-judge-convergence.sh` and `scripts/gc-stale-worktrees.sh` - `worktree disposition`: canonical root `/home/boful/dev/agentops` has pre-existing dirty/untracked operator state - `goals validate`: reported `open GOALS.md: no such file or directory` despite `GOALS.md` existing in this worktree - `corpus freshness`: newest corpus snapshot is 8d old, over the 7d threshold - `contract canaries`: distribution README install-line drift and pre-push governance Bats failure Branch-specific gate sections passed, including `go build`, `go vet`, changed-scope `go test -race`, command/test pairing, mutation-route guard, author!=judge convergence, and the full `cli` Go test suite above. ## Review Cross-model review required before merge. This is not self-graded. Refs: `ag-c3c7` |
||
|
Bo
|
def8fc9bd4 |
chore(doctor): drop dead ao agentopsd/GasCity strings (ag-9hhm #dead-text-removed-surfaces) (#793)
## What Two dead-text remnants pointed operators at **retired surfaces** (discovered in the 2026-06-03 feature-map analysis). Pure string hygiene — no behavior change. 1. **`cli/internal/doctor/fix_bridges.go`** — `openclawHealthReport` and `snapshotStaleReport` instructed `ao agentopsd start/status/restart` and `ao agentopsd projection rebuild --consumer openclaw`. The in-repo daemon was **removed (ADR-0009)** and no such command exists. Reworded to: bring the OpenClaw daemon up out-of-band / regenerate the projection from the out-of-session substrate, naming the daemon removal instead of a dead command. (Also fixed two stale code comments naming `agentopsd`.) 2. **`cli/cmd/ao/forge.go:43`** — the `--legacy-local-llm` deprecation warning named *"AgentWorker/GasCity-backed Codex or Claude workers"*. Gas City was pruned (ag-124p) and the GasCity worker adapter removed. Reworded to point at a **Dream worker queue** (consistent with the existing language elsewhere in `forge.go`). ## Why These strings mislead operators toward commands/substrates that no longer exist; the `gc`/`agentopsd` prune was code-clean but the user-facing strings survived. ## Verification - `gofmt` clean; `go build ./...`; `go vet ./internal/doctor/ ./cmd/ao/` - `go test ./internal/doctor/` → 76 passed; `go test ./cmd/ao/ -run Forge` → 379 passed - `git grep "ao agentopsd"` / `GasCity` over live (non-test) code in touched files → empty - Updated the two doctor tests that asserted the old substrings. Closes-scenario: ag-9hhm#dead-text-removed-surfaces Bounded-context: BC5-Runtime Evidence: cli/internal/doctor/fix_bridges_test.go |
||
|
Bo
|
93bc54d401 |
fix(goals): drop unresolved heuristic scenario claims to kill prose false positives (soc-artx7 #orphans-prose-false-positive) (#779)
## What
`ao goals trace --orphans` reported **7 `broken_bead_scenario_claim`
warnings that were false positives** — bead-description prose mis-read
as scenario references. All 7 were multi-segment `auto-` slugs
(`auto-merge-update-branch`, `auto-safe-pull`, `auto-picks-up`,
`auto-regen-on-skill-change`, …) that match the heuristic free-text
token grammar but are lexically **indistinguishable** from genuine
auto-generated scenario IDs like `auto-nightly-evolution-dry-run`. No
regex can separate them — the only real discriminator is **resolution**
(does the token name an actual `spec/scenarios/<id>.json` or
`.agents/holdout/<id>.json` file?).
## Fix
`emitBeadClaimEdges` now drops heuristic (free-text) claims that resolve
to no scenario file *before* emitting an edge: a token matching the slug
grammar but naming no real scenario is prose, not a claim. This aligns
with ADR-0005 §4.1 — **only explicit `Scenarios:`-line claims are
defects.** Explicit claims are untouched: a broken explicit link stays
an **error**.
## Evidence (live repo, before → after)
| class | old | fixed |
|---|---|---|
| `broken_bead_scenario_claim` | **7** | **0** |
| `no_learning_yield` | 15 | 15 |
| `scenario_no_bead_claim` | 14 | 14 |
Surgical: only the 7 prose false positives drop; every other warning
class is unchanged. Acceptance ("`ao goals trace --orphans` returns 0
`broken_bead_scenario_claim` on current main, no false positives from
bead prose") met. This unblocks promoting `trace --orphans` to a
required gate.
## Tests
- `go test ./internal/goalstrace/...` — 78 pass (incl. new
`TestWalk_MultiSegmentProseAutoSlugProducesNoClaim` regression).
- Existing heuristic/explicit contract tests stay green; `gofmt`/`go
vet` clean.
Closes-scenario: soc-artx7#orphans-prose-false-positive
Bounded-context: BC5-Runtime
Evidence: cli/internal/goalstrace/beads_test.go
|
||
|
Bo
|
7265fda1b1 |
chore(test): migrate redundant os.Setenv to t.Setenv; lower isolation ratchet 22->12 (ag-k38x #bulk-migration-setenv) (#772)
## What Advances the test-isolation ratchet (`scripts/check-test-isolation.sh`, shipped in #699) by driving the `os.Setenv` baseline from **22 → 12**. Pure test-hygiene; no production-code change. `t.Setenv` auto-restores on cleanup **and** fails fast under `t.Parallel` — the raw `os.Setenv` form is the latent-flake landmine the ratchet exists to retire (cf. ag-jfzs). ## Sites migrated (10 eliminated) | File | Δ | How | |---|---|---| | `internal/config/config_test.go` | -6 | Collapse the `origX := os.Getenv(...)` + `defer os.Setenv(... restore)` block into 3× `t.Setenv` | | `internal/goals/measure_test.go` | -1 | Delete the redundant `origPath`/`defer os.Setenv` — the existing `t.Setenv("PATH","")` already auto-restores | | `cmd/ao/rpi_reliability_test.go` | -3 | Delete 3 redundant restore-defers wrapped around existing `t.Setenv` calls | ## The remaining 12 are intentional (not migratable) Documented inline in the ratchet script. They cannot use `t.Setenv`: - **TestMain** (`cmd/ao/main_test.go`, `internal/doctor/doctor_testmain_test.go`) — no `*testing.T` in scope (4) - **String-literal fixtures** (`internal/doctor/fix_cliconfig_test.go`) — `os.Setenv` appears inside generated test-program source, not a real call (2) - **Unset-semantics helpers** (`cmd/ao/handoff_test.go`, `internal/lifecycle/repo_readiness_test.go`, `internal/wiki/locator_test.go`) — these `os.Unsetenv`+restore vars; `t.Setenv` has no unset form (6) ## Verification - `bash scripts/check-test-isolation.sh` → `PASS … os.Setenv=12/12` - `go build ./...` ✓ · `go vet` (touched pkgs) ✓ - `go test ./internal/config/ ./internal/goals/` → 526 passed; touched cmd/ao tests → 3 passed Closes-scenario: ag-k38x#bulk-migration-setenv Bounded-context: BC5-Runtime Evidence: scripts/check-test-isolation.sh PASS 12/12 |
||
|
Bo
|
ba2d0380b2 |
feat(eval): holdout-leak key-name backstop for outcomes compile (ag-uux93 #holdout-key-guard) (#770)
## What Adds the key-name leak backstop that **ag-uux93** explicitly calls for on `ao eval outcomes compile`: re-parse the emitted Outcomes rubric payload and refuse any object key (at any depth) matching `/target|ground_truth|expected_output|answer/i`. ## Why The compile command itself already shipped (under sibling **ag-hdqu0** / #601) with a value-based guard (`ContainsAny` scans for known holdout *values* — layer 3). ag-uux93 additionally requires a **structural key-name** backstop: "re-parse the emitted payload and assert no key matches `/target |ground_truth|expected_output|answer/i`; abort + non-zero exit if found." This is **additive defense-in-depth** at a load-bearing boundary — Managed Agents are not ZDR, so a holdout leak across the cloud boundary is permanent. The allowlisted `Rubric` struct can't carry such a key today; the guard *pins that invariant* so a future field that smuggles ground truth fails closed instead of leaking silently. ## Changes - `evalsubstrate.ScanForbiddenKeys(payload []byte)` — walks decoded JSON for leak keys at any nesting depth - `Rubric.HasForbiddenKey()` — marshals + scans (mirrors the existing `ContainsAny`) - Wired into `compileOutcomesRubric` after the value-scan, before return (structured non-zero error on hit) - Unit tests: positive (nested / case-insensitive / substring), clean rubric, unparseable input, allowlist-by-construction invariant ## Verification `cd cli && go build ./... && go vet ./... && go test` — build + vet clean; evalsubstrate 92 pass, cmd/ao outcomes/rubric 30 pass; gofmt clean. ## Scope note for maintainer ag-uux93's spec also describes a `--suite/--split/--out` **flag interface** reading locked YAML from `~/.agents/evals/`. The shipped #601 command deliberately took a **JSON-input** design (`ao eval outcomes compile <input.json>`) instead. I did **not** unilaterally rewrite that working interface in an unattended tick — that's an intentional design fork needing your call. Queued to `bo-queue`. This PR closes only the `#holdout-key-guard` scenario. Closes-scenario: ag-uux93#holdout-key-guard Bounded-context: BC4-Evals Evidence: cli/internal/evalsubstrate/holdout_guard_test.go |
||
|
Bo
|
f56f5b1521 |
chore(lint): clear 13 golangci-lint findings (ag-tcf0.1 #golangci-lint-clear) (#777)
## What
Clears all outstanding `golangci-lint run` findings in `cli/` — **13
issues** across 8 files. Pure mechanical lint cleanup, no behavior
change. (Several were hidden behind golangci-lint's default
`max-same-issues=3` display cap; running with caps disabled surfaced the
full set.)
| Linter | Count | Fix |
|---|---|---|
| errcheck | 7 | unchecked `f.Close()` → `defer func() { _ = f.Close()
}()` / `_ = f.Close()` |
| staticcheck S1016 | 2 | struct-literal copy → type conversion
(`reconcileCIRun(run)`, `reconcileBeadSummary(issue)`) |
| staticcheck QF1012 | 2 | `b.WriteString(fmt.Sprintf(...))` →
`fmt.Fprintf(&b, ...)` |
| staticcheck QF1001 | 1 | De Morgan: `!(a && b)` → `a' || b'` |
| copyloopvar | 1 | drop redundant `fx := fx` (Go 1.22+ per-iteration
loop var; go.mod is 1.26) |
Files: `loop_blocked.go`, `loop_next_work.go`, `cron_self_adjust.go`,
`reconcile.go`, `provenance_trace_test.go`, `internal/canon/ledger.go`,
`internal/scenarios/validate.go`, `internal/skills/catalog.go`.
## Verification
- `go build ./...` ✅ · `go vet ./...` ✅
- `golangci-lint run --max-same-issues 0 --max-issues-per-linter 0` →
**0 issues** ✅
- `go test ./internal/canon/... ./internal/scenarios/...
./internal/skills/...` → 116 passed ✅
- targeted `cmd/ao` tests (provenance/reconcile/cron/blocked/next-work)
→ ok ✅
- `scripts/pre-push-gate.sh --fast` → passed ✅
Closes-scenario: ag-tcf0.1#golangci-lint-clear
Bounded-context: BC5-Runtime
Evidence: golangci-lint run --max-same-issues 0 → 0 issues (verified
locally; CI re-runs the omnibus gate)
|
||
|
Bo
|
a3457c001d |
test(posture): restore stance-claim enforcement lost when internal/gascity removed (ag-ddrb #posture-tests) (#791)
## What
Adds `cli/internal/posture/posture_test.go` — the package had **zero**
tests.
## Why
`internal/posture/posture.go` is pure and deterministic but its doc
comment promises:
> the AgentOps stance claim "must either update this value or fail the
integration test in `cli/internal/gascity/bridge_test.go`"
`internal/gascity` was **deleted** (ag-hfc, 3.1 teardown — confirmed in
CLAUDE.md), so that enforcing test no longer exists. The `AgentOps`
stance declaration has been **unguarded** since. `TestAgentOpsStance`
here is the replacement regression guard: a silent edit to the stance
now fails CI.
## Coverage
Behavioral, table-driven, **100.0% statement coverage** (10 tests):
- `Layer.String()` — all six + sentinel/out-of-range
- Layer bit-position values (the shift-amount contract `LayerSet`
depends on)
- `LayerSet` `NewLayerSet`/`Has`/`Union` — idempotence, commutativity,
empty-set identity
- `AllLayers` / `ExternalLayers` exact membership ("Bead is universal")
- `Kind.String()` — all families + out-of-range
- `AgentOps` stance invariant: `Kind=Distribution`, owns exactly
`{Role,Skill,Ledger,Distribution}`, does **not** own `Loop`, `Bead` is
external
- owned ⊆ `AllLayers` structural invariant
No coverage-padding: every assertion checks exact behavior or a
documented invariant.
## Verification
```
go test ./internal/posture/ -cover → ok 100.0% of statements (10 pass)
go vet ./internal/posture/ → clean
golangci-lint run ./internal/posture/ → 0 issues
gofmt -l → clean
```
Closes-scenario: ag-ddrb#posture-tests
Bounded-context: BC5-Runtime
Evidence: go test ./internal/posture/ -cover → 100.0%
|
||
|
Bo
|
466842d9bd |
feat(memory): add citation outcome taxonomy (#741)
## Summary Implements `ag-rbx8` W3.1 citation outcome taxonomy and reviewer identity plumbing for memory feedback. - Adds canonical citation types: `retrieved`, `used-in-final-artifact`, `helpful`, `harmful`, `refuted`, while preserving legacy `applied`, `reference`, and `skill_loaded`. - Extends citation events with `artifact_author_id`, `cited_by_agent_id`, and `cited_by_model_family`. - Parses legacy identity aliases in citation JSONL (`author_id`, `reviewer_id`, `judge_id`, `model_family`). - Updates `ao metrics cite` with non-author identity flags and rejects self-authored outcome judgments. - Teaches citation pipeline and feedback-loop logic that retrieved-only is exposure, helpful/final-artifact/applied are rewardable, and harmful/refuted are explicit non-reward outcomes. - Regenerates CLI command docs and `registry.json`. ## Validation Focused tests after rebase over #737 (`origin/main` @ `7bbc6569`): - `go test ./internal/types ./internal/ratchet ./internal/quality ./internal/lifecycle` - `go test ./cmd/ao -run 'TestMetricsCite|TestProcessCitationFeedback|TestCitationConfidenceBuckets|TestValidateFeedbackCitationType|TestFlagCompletions_Registered|TestComputeCitationPipeline|TestCanonicalCitationType'` - `go test ./cmd/ao` - `git diff --check` Required regen/gate: - `bash scripts/regen-all.sh` completed generated artifact updates, then exited on known CMDAO surface parity red (`wiki query`; 30 uncovered commands). - `PRE_PUSH_FAIL_FAST=false bash scripts/pre-push-gate.sh` completed with 5 known non-diff failures: - test-fixture parity: `scripts/gc-stale-worktrees.sh` - worktree disposition: canonical root dirty/untracked operator state - goals validate: local `GOALS.md` lookup red - corpus freshness: newest local snapshot over threshold - contract canaries: known `agentops-core.pre-push-gate-governance` failure Diff-owned checks passed in the gate, including go build, go vet, changed-scope race tests, command/test pairing, registry drift, CLI docs parity, mkdocs strict build, contract compatibility, CI policy parity, context-map drift, shellcheck, skill citation parity, and executable-spec link integrity. ## Review Gate No self-grade: requesting cross-model review from WindyCastle and RubyMoose before merge. Bead: `ag-rbx8` |
||
|
Bo
|
76af462b45 |
refactor(evidencedturn): converge author≠judge onto liveness.Disjoint (DRIFT fix, impl #2) (#744)
## What WindyCastle's vibe-check (Agent Mail #149) found the **no-self-grade / author≠judge invariant fragmented across 3 divergent implementations** on main: 1. `liveness.Disjoint` (ag-xdrw/#737) — the canonical one. 2. `evidencedturn`'s `evalAuthorNeqValidator` (ag-lmdx.4) — **this PR**. 3. #741's inline `EqualFold` — SageHarbor converges in parallel. That divergence (empty-handling / case / waiver) is **why the empty-ID self-grade bypass recurred** (#737 PG3/PG4 → #741 citation reward). This PR converges **impl #2**: `evalAuthorNeqValidator` now routes its core author≠judge decision through `liveness.Disjoint`. ## Behavior-preserving `Disjoint` denies on `empty-author OR empty-judge OR author==judge` — **identical** to the prior inline checks. Verified: all existing `evidencedturn` tests pass unchanged. Preserved: the `--allow-self` waiver (explicit pre-check) + the specific legible reason strings. ## Anti-recurrence lock Adds `TestAuthorNeqValidator_AgreesWithDisjoint` — asserts the validator's decision matches `liveness.Disjoint` across every identity case (distinct / self-graded / empty-author / empty-judge / both-empty) + the waiver. **A future re-divergence fails CI** — the structural fix WindyCastle's finding asks for. ## Scope 2 files (`evidencedturn.go` +28/−10, `evidencedturn_test.go` +37). `evidencedturn`→`liveness` import, **no cycle** (liveness doesn't import evidencedturn). No overlap with #741 (different package — SageHarbor owns impl #3). `go build ./...` + vet + race + package tests green; 4 reds are the known non-diff set. > Provenance: bead pending Dolt write recovery (down per #149); vibe-check #149 + Agent Mail #151 carry the claim. File the p1 DRIFT bead when Dolt's back. Bounded-context: BC5-Runtime Evidence: TestAuthorNeqValidator_AgreesWithDisjoint |
||
|
Bo
|
9c6b4036bf |
test(liveness): end-to-end kernel composition test (ag-c3c7 #compose) (#743)
## What An **L2 integration test** proving the 5 liveness files — built across separate PRs by separate contexts (#733 roles, #737 quorum/guards, #739 work_lease) — compose into **one admission lifecycle on one vocabulary**, not five parallel ones. `TestKernelComposition_FullAdmissionFlow` walks the full flow: 1. **Role-matrix SoD** (`roles`) — worker edits, worker can't self-vote, orchestrator can't edit (injection defense). 2. **External role source** (`request`+`guards`) — leased role passes `Check`; self-asserted (source==actor) → `Denied`. 3. **Work-lease** (`work_lease`) — the just-authorized identity takes a lease. 4. **Author≠judge** (`guards.Disjoint` via `Check` `VerbJudge`) — self-judge `Denied`, distinct verifier `Allowed`; high-level `Check` and low-level `Disjoint` must **agree**. 5. **Cross-model quorum** (`quorum`) — solo→NeedsAdmission, actor self-ACK doesn't count, two same-family ACKs don't clear (collusion guard), two distinct families clear. 6. **Hard expiry + takeover** (`work_lease`) — late renewal `ErrLeaseExpired`, deadman sweep surfaces it, distinct owner takes over via `Create`. ## Why WindyCastle's vibe-checks *assert* the kernel composes; this *proves* it mechanically and **locks it against regression** — if a future change breaks the composition (e.g. `Check VerbJudge` stops routing through `Disjoint`, or quorum drops family-diversity), this fails where the per-file unit tests wouldn't. ## Scope New test file ONLY (`cli/internal/liveness/kernel_composition_test.go`). **Zero production code touched.** Exercises only public APIs already on main. `go build ./...` + `go vet` + full liveness package green; the 4 local reds are the known pre-existing non-diff set. Closes-scenario: ag-c3c7#compose Bounded-context: BC5-Runtime Evidence: TestKernelComposition_FullAdmissionFlow |
||
|
Bo
|
7bbc6569c4 |
feat(liveness): enforce author-judge quorum gate (ag-xdrw) (#737)
## Summary Implements `ag-xdrw` independent-verification gates for claim-evidence and significant orchestration actions. - Adds `author_id` / `judge_id` metadata to claim-evidence bindings. - Rejects self-graded bindings at the ClaimEvidenceBinder port. - Requires non-empty, distinct author/judge metadata for high-assurance PG3/PG4 evidence while preserving legacy anonymous PG1/PG2 behavior. - Carries reviewer metadata through in-memory + production claim-evidence derive paths and durable JSONL binding records. - Adds `liveness.CheckSignificantAction` for orchestration-tier significant actions requiring two distinct non-actor ACKs across at least two model families. - Regenerates CLI docs/surface/registry after adding `ao claim bind --author-id/--judge-id` flags. ## Validation Current head: `e4456538d89660c954ca12d347a186bd1cf68edf` rebased over #740 (`853d781e`). Focused checks after rebase: - `go test ./internal/liveness ./internal/ports` - `go test ./cmd/ao -run 'TestProductionClaimEvidenceBinder|TestClaimBind|TestClaimList|TestProductionClaimEvidence|TestClaim'` Required regen after rebase: - `bash scripts/regen-all.sh` regenerated artifacts, then exited nonzero on the known CMDAO surface parity red: `wiki query` / 30 uncovered commands. Required pre-push after rebase: - `PRE_PUSH_FAIL_FAST=false bash scripts/pre-push-gate.sh` completed with 4 known non-diff failures: - test-fixture parity: `scripts/gc-stale-worktrees.sh` - worktree disposition: canonical root dirty/untracked operator state - corpus freshness: newest local snapshot older than threshold - contract canaries: known `agentops-core.pre-push-gate-governance` Bats failure Diff-relevant gates passed in the pre-push run, including changed-scope build/vet/race, command/test pairing, `goals validate`, registry drift, CLI docs parity, command-surface matrix canary, shellcheck, mkdocs, contract compatibility, CI policy parity, and context-map drift. ## Review Gate No self-grade: cross-model ACKs are recorded in Agent Mail. - WindyCastle re-confirmed ACK on `547eb36f` and then explicitly called to rebase once more over #740 and merge this keystone next (#134). - RubyMoose re-confirmed ACK on `547eb36f`; current `e4456538` is the same reviewed semantic diff rebased over #740 plus regenerated registry timestamp. Review attacks covered: - PG3/PG4 anonymous reviewer metadata rejects; PG1/PG2 anonymous legacy remains allowed. - `AuthorID == JudgeID` rejects. - Significant-action actor ACK does not count toward quorum. - Two same-family ACKs do not satisfy quorum. - Non-quorum path returns `NeedsAdmission`, never `Allowed`. Bead: `ag-xdrw` Evidence: TestProductionClaimEvidenceBinder_SelfGradeRejected Evidence: TestProductionClaimEvidenceBinder_HighAssuranceWithoutReviewersRejected Evidence: TestCheckSignificantActionTwoCrossModelACKsAllowed Evidence: TestCheckSignificantActionActorACKDoesNotCount |
||
|
Bo
|
6f3b0aac04 |
feat(liveness): typed work-lease tracking (ag-c3c7.2 #work-lease) (#739)
## What
A deterministic liveness-domain **work-lease tracker** — the substrate
record of who owns an active work arc and when the claim lapses.
deadman/fallback (later slice) consumes expired leases to know which
work is abandoned and eligible for quorum takeover. Makes no admission
decision itself.
`cli/internal/liveness/work_lease.go` (+ test):
- `WorkLease{BeadID, Branch, OwnerAgentID, ModelFamily, LeaseReason,
IssuedAt, ExpiresAt, RenewedAt, EvidenceRef}` + `Status(now)`.
- `LeaseTracker` (injected `Clock`): `Create / Renew / Expire / Get /
List / Expired`.
## Invariants (the kernel's principles, mechanical)
- **No-self-grade on liveness:** `Renew` requires a **non-empty
`evidence_ref`** → `ErrEvidenceRequired`. Alive = externally-visible
evidence, not self-report.
- **Quorum takeover:** `Create` rejects an *active* duplicate
(`ErrLeaseActive`) but **re-leases an expired bead** (new owner takes
over abandoned work).
- **Deterministic:** every time decision routes through the injected
`Clock` — reproducible, testable; tests use a fixed clock.
## Scope (per SageHarbor's carve, Agent Mail #115)
New `work_lease{,_test}.go` ONLY. **Not** admission classification,
pulse/quorum schema, deadman escalation, or the #737 author/judge gate.
Untouched: `ports/claim_evidence_binder.go`, `session_bootstrap.go`,
`search/*`, `registry.json`, generated CLI docs.
## Validation
- 6 tests pass (create/takeover, evidence-gated renewal, status/expired,
force-expire) with exact-value assertions + fake clock.
- Local gate: `go build` / `go vet` / `go test -race (changed scope)` /
command-test pairing / HOME-isolation all **green**. The 4 remaining
local reds (gc-stale-worktrees fixture, canonical-root-dirty [shared
checkout], goals-validate, corpus-freshness) are the **pre-existing
non-diff** set (same as #730/#736/#737).
Requesting **≥2-of-3 cross-model review** — this is liveness-kernel
code, review hard.
Closes-scenario: ag-c3c7.2#work-lease
Bounded-context: BC5-Runtime
Evidence: cli/internal/liveness/work_lease.go
|
||
|
Bo
|
cd5b8acc1e |
feat(liveness): role-capability matrix — deny-and-escalate RBAC (ag-c3c7.1 #role-matrix) (#733)
Role-capability matrix for the liveness-kernel (ag-c3c7.1, assigned by SageHarbor as sequencing lead). `cli/internal/liveness/roles.go` defines each role's allowed verbs — orchestrator: route/vote/shepherd/synthesize · worker: edit · verifier: judge · scribe: record · heartbeat: nudge. Every out-of-capability action resolves to `needs-admission` — **deny-and-escalate by default, never allow**. Key separation semantics (all tested): - **orchestrator CANNOT edit** — the prompt-injection defense; an orchestrator can't perform a worker's edit - **verifier CANNOT edit what it judges** — no-self-grade - worker / scribe CANNOT vote (execute/record, don't decide) - heartbeat does nothing but nudge - unknown role/verb → needs-admission New isolated package. No-touch list respected: validate.yml, registry.json, pre-push-gate.sh, skill-eval.sh, .agents/design/*, cli/internal/search/*, cli/cmd/ao/* all untouched. 24 tests pass; full `go build ./...` + vet clean. Closes-scenario: ag-c3c7.1#role-matrix Bounded-context: BC2-Validation Evidence: cli/internal/liveness/roles.go |
||
|
Bo
|
01dc1568cf |
feat(memory): compute always reach from canon maturity
Implements ag-oqha W2.2. Authored reach=always is sanitized to pull; effective always is computed only from canon-collected maturity=established learnings. Cross-model ACKs recorded in Agent Mail messages #70 and #71. |
||
|
Bo
|
8ad6747913 |
chore(ao): delete the ao evolve CLI surface (ag-llni slice 1) (#724)
**Slice 1 of ag-llni** — retire `ao rpi`/`ao evolve` (orchestration is no longer an in-CLI engine). This PR removes the **`ao evolve`** half; the `rpi*.go` + phased web is slice 2. **Architecture (operator-confirmed 2026-06-04):** orchestration is two-tier — *inner loop* (one session) = skills/CLIs/native-subagents + **Workflows** (`.claude/workflows/operating-loop.js` + `bead-crank.js` — the Claude encoding of inner-orchestration that was mis-shaped as `rpi`/`evolve`); *outer loop* (many sessions) = **NTM + mcp-agent-mail**. Codex inner-loop = NTM + agent-mail for now. Leaf CLIs (`corpus`/`goals`/flywheel/knowledge loops = BC1 corpus), NTM, and agent-mail all stay. ### Changes - Deleted `ao evolve` + 4 subcommands (`config`, `next-work`, `blocked`, `write-stop-marker`) — 10 files. - Relocated the one non-evolve coupling, `countCronHistoryRows`, into its sole consumer `cron_self_adjust.go`. - Removed `evolve` from the `cobra_commands_test.go` golden `expectedCmds` list (2 sites). - Regenerated `cli/docs/COMMANDS.md` (−125 lines) + `docs/cli-skills-map.md`. ### Verification - `go build ./...` green · `go vet ./cmd/ao` clean · **`go test ./cmd/ao` = 9251 pass**. - Net: 13 files, −2192 LOC. Bounded-context: BC5-Runtime Closes-scenario: ag-llni#delete-evolve-cli Evidence: cli/cmd/ao/cron_self_adjust.go Test-Removal-Reason: ao evolve CLI surface retired (ag-llni) — orchestration moved to .claude/workflows + NTM; the 5 deleted evolve_*_test.go covered now-deleted command code, not kept behavior. --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com> |
||
|
Bo
|
6c8cb223ea |
feat(corpus): add reach blast-radius tier {bead|pull|always} (ag-bsf6 #reach-field) (#721)
W2.1 of epic ag-lhu0 (memory-system re-architecture). Adds the **reach**
blast-radius axis, orthogonal to maturity, per the locked duel
architecture.
- `reach {bead|pull|always}` on FrontMatter + Learning; `reach:`
frontmatter round-trips.
- `SanitizeReach` normalizes + defaults absent/invalid → `pull`, and
**never** promotes to `always` (the computed-not-authored invariant —
`always` is derived from maturity==established ∩ canon in ag-oqha).
- `learning.v1.schema.json`: reach enum + default pull; domain vocab
`skills/domain/references/reach.md`.
- Tests: SanitizeReach + frontmatter round-trip/default (88 search-pkg
tests green).
Closes-scenario: ag-bsf6#reach-field
Bounded-context: BC1-Corpus
Evidence: cli/internal/search/learnings.go
|
||
|
Bo
|
dd6cff74d8 |
fix(corpus): scrub secrets on corpus write (ag-6vlm #scrub-on-write) (#719)
Wire `llm.RedactBytes` into `corpus_fs.Writer.Capture` so every byte that lands in the `.agents/` corpus (frontmatter + body) is scrubbed before `os.WriteFile`. Activates the previously-dead redactor on the load-bearing local corpus-write path — the W1 git-safety scrub-on-write chokepoint (primary defense; CI scan is the backstop). **Scenario (Gherkin):** Given a learning body with an sk-ant- token / AWS key / PEM block, When `corpus_fs.Writer.Capture` writes it, Then the persisted file has the secret scrubbed to `[REDACTED]` and non-sensitive content preserved. Proven by `TestWriter_CaptureScrubsSecrets` (L2). Epic ag-lhu0 (memory-system re-architecture), Wave 1. Closes-scenario: ag-6vlm#scrub-on-write Bounded-context: BC1-Corpus Evidence: cli/internal/adapters/corpus_fs/corpus_fs.go |
||
|
Bo
|
6bd788d8af |
chore(cli): gofmt sweep — 14 unformatted files (ag-ozac #gofmt-sweep) (#717)
Mechanical `gofmt -w` across `cli/` — fixes the 14 unformatted files surfaced by the **ag-tcf0** Go-cohesion audit. No logic change (diff is whitespace/grouping only; `git diff -w` is ~empty). ## Verification - `gofmt -l cli/` → empty - `go build ./...` → passes - Rebased onto current `origin/main` Closes-scenario: ag-ozac#gofmt-sweep Bounded-context: BC5-Runtime Evidence: cli/internal/eval/types.go |
||
|
Bo
|
8be8d678f2 |
fix(corpus): relevance-rank ao inject — strip stopwords from query tokens (ag-32gx #stopword-relevance) (#704)
## What Fixes the retrieval-relevance gap the corpus-delta **W1c first signal** located: `ao inject --context "<verbose query>"` returned generic recency-ranked learnings, not the task-relevant one. **Root cause:** `QueryTokens` did no stopword filtering — a long query became ~80 tokens (incl. the/if/any/that), so `MatchRatio` (matched ÷ total) collapsed for every learning and ranking fell back to freshness. **Fix** (mirrors the proven `skills/find.go` tokenizer): `QueryTokens` now splits on any non-alphanumeric rune (`continue-on-error:true` → continue/error/true), drops a ~80-word English stopword set + sub-2-char tokens, and dedups. Domain words (gate/job/exit/ci/run/check) are deliberately kept. ## Validation - **Empirical, against the real corpus:** a focused query's hits on the target learning rose (2 → 4); the learning is NOT quality-gate-filtered (it surfaces with a salient query). - **Tests:** QueryTokens strips stopwords/punctuation + dedups; a verbose query ranks a salient-matching learning above a generic one; all-stopword query degrades safely (→ no relevance signal → freshness, preserving prior behavior). 86 search + 580 cmd/ao pass; build + vet clean. ## Honest residual (separate, NOT an inject defect) A raw whole-prompt query still dilutes via `MatchRatio` length-normalization. inject's job is to rank *given* a query; it now does. The **caller** should pass a focused query — noted on **ag-epgk**: the W1c runner should derive a focused retrieval query (task title/salient line), not pass `prompt.md` verbatim. That, plus this fix, unblocks a clean W1c re-run. Closes-scenario: ag-32gx#stopword-relevance Bounded-context: BC1-Corpus Evidence: cli/internal/search/learnings.go |
||
|
Bo
|
28b18a8523 |
fix(corpus): handle silent G104 parse/write swallows (ag-chvc #silent-error-handling) (#697)
## What Fixes the behavioral `#nosec G104` silent-error swallows surfaced by the codebase-audit (ag-chvc). - **Sscanf** (`lifecycle/temper.go` ×3, `cmd/ao/temper.go` ×4, `storage/searchindex.go`): a *malformed* Utility/Confidence/SchemaVersion silently parsed to **zero**, corrupting decay-rank inputs. Now parse into a local and assign only on a clean parse → malformed value keeps the prior default. The error is handled, so the `#nosec G104` is gone. - **WriteFile/MkdirAll** (`cmd/ao/quickstart.go`, `cmd/ao/demo.go` ×6): silently dropped write failures while still printing ✓. Now warn to stderr on failure, print ✓ only after the write lands. - **Left untouched** (legitimate best-effort): `rpi_phased_processing` `os.Remove` cleanup, demo interactive `Scanln`. ## Verification `go build` + `go vet` clean; 276 lifecycle/storage tests pass under `-race`; 3 new regression tests (malformed → keeps default; valid → parses). Zero G104 left in the 5 touched files. Closes-scenario: ag-chvc#silent-error-handling Bounded-context: BC1-Corpus Evidence: cli/internal/lifecycle/temper.go |
||
|
Bo
|
ff4c726a60 |
refactor(rpi): remove orphaned GasCity phase-evidence dead types (ag-82pv #strip-deadtypes) (#685)
## What Removes the orphaned **GasCity phase-evidence** dead types from `cli/internal/rpi/artifacts.go` (**ag-82pv**). After the #679/#683 Gas City strip, these had **no production caller** — only their own unit test referenced them. Flagged by post-mortem #6 (Judge 2) and Bo's #682-close note ("pending rpi/artifacts.go cleanup"). ## Removed (−122 lines, 2 files) - `GasCityPhaseEvidenceFileFmt` (const), `GasCityPhaseEvidence` + `GasCityTranscriptArtifact` (structs), `GasCityPhaseEvidencePath` + `WriteGasCityPhaseEvidence` (funcs). - The now-unused `time` import. - `TestWriteGasCityPhaseEvidence` (the only caller) + its now-unused test imports. ## Kept (intentional) `ClassifyRPIArtifact`'s `-gascity-evidence.json` case and its test — a **live, defensive classifier** for any legacy on-disk evidence files. It keys off a string literal, not the removed types, so it's safe and out of scope for this dead-type strip. ## Deletion safety Repo-wide grep confirmed the removed symbols were referenced **only** in `artifacts.go` + `artifacts_test.go` — no other Go callers, no schema/doc/string-literal anchors, no indirect wiring. ## Verification (local, full gate) - `go build ./...` ✓ · `go vet ./...` ✓ · `go test ./...` ✓ (**11,947 tests / 73 pkgs**) · `golangci-lint` clean on rpi. - `regen-all.sh --check`: ALL GATES GREEN (no derived-surface drift). Closes-scenario: ag-82pv#strip-deadtypes Bounded-context: BC3-Loop Evidence: cli/internal/rpi/artifacts.go |
||
|
Bo
|
900fde103e |
refactor(doctor): strip gc-bridge detector/fixer subsystem (ag-gdns) (#683)
## What The final load-bearing Gas City **code** coupling. Removes the residual gc-bridge detector/fixer subsystem from `ao doctor`. AgentOps no longer references Gas City (`ag-124p`/#679, `ag-h93p`/#680 merged); this completes the prune on the code side. This is the **bounded, single-purpose** replacement for the closed mega-PR #682 — `ag-gdns` only, separately reviewable. ## Changes (3 files, +15 / −832) - **`cli/internal/doctor/fix_bridges.go`** — removed the 4 gc detectors + 4 gc fixers (binary-missing, version-incompatible, controller-down, status-parse-error), their registrations, all gc-only helpers, gc consts, and now-unused imports (`os/exec`, `sort`). **Preserved verbatim** the OpenClaw consumer subsystem (health-unreachable + snapshot-stale FMs) and the shared helpers it depends on (`truncatePayload`/`reportPath`/`writeReport`/`detectOnlyRemediation`). `ao doctor` no longer probes the `gc` binary/version/controller/status. - **`cli/internal/doctor/fix_bridges_test.go`** — removed the 4 gc tests; trimmed registration + auto-fixable-flag tests to the 2 OpenClaw IDs; kept all OpenClaw tests. - **`docs/dependencies.md`** — dropped `gc` from the `ao doctor` probe list. ## Verification `gofmt` clean · `go build ./...` + `go vet` · `go test ./...` green (**11946 pass**) · doctor package green · repo-wide grep finds **zero** residual gc-bridge symbol/ID refs · `process-hygiene` (doc-release + quarantine) + `contracts-sync` drift gates green locally. ## Scope note The OpenClaw subsystem and shared helpers were deliberately kept — `ag-gdns` is gc-only. The #682 attempt's `correctness` red was the known flaky `TestFlagMatrix_QuietMode/memory-sync` (tracked separately as `ag-z8c4`), not this strip. Closes-scenario: ag-gdns#strip-gc-bridge-doctor Bounded-context: BC2-Validation Evidence: cli/internal/doctor/fix_bridges.go |
||
|
Bo
|
fbc80c68b9 |
feat(canon): tier-aware promotion gate (ag-n2r4 #heuristic-earns-on-citations) (#675)
## What gap#2 — **not all knowledge is mechanically verifiable.** A learning declares `canon_tier:` in its frontmatter; the promotion gate branches on it: | Tier | Requirement | Why | |---|---|---| | `falsifiable` (default) | 1 cross-engineer citation **AND** 1 independent verification | claims about code/gates are checkable — and must be checked | | `heuristic` | 3 cross-engineer citations, no verification | judgment can't be mechanically verified; breadth of independent reuse is the only honest trust signal | - **Default is the stricter tier** (`falsifiable`) — `heuristic` is an explicit opt-out, so an unmarked learning can't dodge verification by omission. - Refutation **hard-blocks both** tiers; unknown tier (no path) falls back to strict falsifiable. - `status`/`promote` surface the tier; `EvaluateEntry` resolves it from the learning file. ## Verification - 8 new unit tests (TierOf default/garbage/missing, per-tier thresholds, heuristic-earns-on-3-citations, falsifiable-still-needs-verification, unknown-path-strict) + a `cmd/ao` CLI test; canon package green (43) - e2e: heuristic EARNED on 3rd citation (0 verifs); falsifiable PENDING on 3 citations until verified - pre-push gate clean except the pre-existing bd-closeout/AGENTS.md drift (untouched file, out of scope) Closes-scenario: ag-n2r4#heuristic-earns-on-citations Bounded-context: BC1-Corpus Evidence: cli/internal/canon/tier.go |
||
|
Bo
|
7bd90eab3b |
feat(canon): verify --council for independent cross-vendor verdicts (ag-mluj #council-verdict-independent) (#674)
## What gap#1 — the load-bearing **"earned by verification, not assertion"** gap. `ao canon verify --council` delegates the verdict to an **independent cross-vendor judge** instead of taking an operator-asserted one: - `CouncilVerifier` port + `CommandVerifier` adapter, configurable via `AGENTOPS_CANON_VERIFIER_CMD` (codex = the cross-vendor lane in this fleet) - Verdict is attributed to the **judge**, not whoever ran the command → counts toward promotion even when the author invokes it (independence by construction) - No judge configured → **loud error**, never fabricates a verdict - Judge output must carry `VERDICT: confirmed|refuted`; absence is an error (never defaults to confirmed) - The judge transcript is recorded as the verification **receipt** ## Honest scope This wires the cross-vendor **judgment** lane (council's P9 cross-vendor rule). How deeply the judge gathers its own evidence is bounded by what the judge command does — a codex judge has shell access and can read cited files; the receipt records what it examined. The full "gather-your-own-evidence" keystone (mo-qz3srw) remains a further step. ## Verification - 8 new unit tests (verdict parse, empty-cmd-fails-loud, confirmed/refuted exec, no-verdict-errors, council-is-independent); canon package green - e2e: no judge → loud error; fixture cross-vendor judge → independent `confirmed` attributed to `codex:gpt-5.3`; learning EARNED even when author-invoked - command-surface canary unchanged (flag, not a new heading) Closes-scenario: ag-mluj#council-verdict-independent Bounded-context: BC1-Corpus Evidence: cli/internal/canon/verifier.go |
||
|
Bo
|
b962a0ae29 |
docs(canon): credit warmind lineage (ag-bw5s #trivial) (#673)
## What Records canon's lineage: the team-knowledge-sharing pipeline, citation-gated promotion, and the self-citation guard originate in **Douglas J. Dan's warmind prototype (#671)**. Canon adapted them — re-aiming promotion to be earned by independent verification and folding retrieval into the existing `ao inject` path. Co-authored to Doug so his contribution lands on `main`'s history and contributor graph. #671 stays open for reference. Closes-scenario: ag-bw5s#earned-promotion-gate Bounded-context: BC1-Corpus Evidence: cli/internal/canon/canon.go Co-authored-by: Douglas J. Dan <dougd920@gmail.com> |
||
|
Bo
|
3702761e75 |
feat(canon): earned-promotion team-knowledge tier (ag-bw5s #earned-promotion-gate) (#672)
## What A team-knowledge **canon**: learnings *earned* into a shared trusted set, not self-asserted. Adapts Doug Dan's warmind (#671) to the post-ACFS direction — moat = knowledge earned by verification, not self-certification. Promotion requires **two independent, self-guarded signals**: - **citation** — another engineer/agent USED it (proves useful) - **verification** — another engineer/agent CHECKED it (proves true; receipt-bearing) An independent refutation hard-blocks. The anti-self-certification guard is generalized to *all* promotion signals: the factory may not grade its own homework. ## Slices (3 commits, one coherent arc) 1. **Earned-promotion gate** — `internal/canon` (identity guard, citation + verification JSONL ledgers, gate); `ao canon cite/verify/status/promote`. 2. **Inject trusted tier** — `collectLearnings` surfaces `.agents/canon/learnings/` deduped + boosted 1.15× over equal-utility local; flagged `Canon`. 3. **IdentityResolver port** — `--as` > `AGENTOPS_ACTOR` env > git. Env tier is the seam to Agent Mail (`whois`) with **no MCP coupling** — `ao` stays portable. Makes the cross-actor guard hold for humans AND agents. ## Deliberately dropped from #671 Resurrected `hooks/` + session-end auto-sync (hookless regression), the "Gas City Dispatch (CRITICAL)" doctrine (contradicts prune-the-coordination-plane), the parallel scoring/maturity engine (duplicated inject primitives), and the "warmind" name. ## Verification - `go build`/`vet` clean, gofmt clean - 9,388 tests green (`cmd/ao` + `internal/canon` + `internal/search`) - e2e: apollo self-attesting excluded; witness (distinct agent via `AGENTOPS_ACTOR`) earns promotion; `ao inject` ranks canon first at equal utility ## Deferred (next) Wire `verify` → `ao verify`/council (currently operator-attested — the load-bearing gap); tier-aware gate (heuristics earn citation, not verification). Closes-scenario: ag-bw5s#earned-promotion-gate Closes-scenario: ag-bw5s#inject-trusted-tier Closes-scenario: ag-bw5s#agent-identity-resolver Bounded-context: BC1-Corpus Evidence: cli/internal/canon/promote.go |
||
|
Bo
|
a36abd3675 |
chore(provenance): align relation enum to W3C PROV-O vocabulary (ag-lmdx.7 #prov-o-vocabulary) (#660)
## What Rename the provenance ledger relation enum from AgentOps-local `subject_verb_object` names to the standard **W3C PROV-O / PROV-DM** verbs so an external auditor recognizes the term, and document the **columns-not-JSON-paths** guard-read principle in the schema. Schema changes are single-writer on `main`. Implements ag-lmdx.7. ### Relation mapping (atomic contract change) | Prior AgentOps-local | W3C PROV-O | |---|---| | `decision_produces_artifact` | `wasGeneratedBy` | | `decision_authorizes` | `wasAssociatedWith` | | `artifact_derived_from` | `wasDerivedFrom` | | `scenario_covers_artifact` | `wasInformedBy` | | `verdict_attests_artifact` | `wasAttributedTo` | | `bead_scopes_decision` | `wasInfluencedBy` | | `commit_implements_decision` | `wasRevisionOf` | | `learning_revises_decision` | `wasInvalidatedBy` | ## Scenarios - **Relations use PROV-O vocabulary** — the enum now contains only PROV-O verbs; an edge with the colloquial `derives_from` (or the prior `artifact_derived_from`) is REJECTED in favor of `wasDerivedFrom`. Enforced by two new bats cases (accept PROV-O, reject legacy vocabulary). - **Guard-read field is a column not a JSON path** — the schema description now states that guard-read/queryable fields (`from_id`, `to_id`, `relation`, `trust_tier`, the hash-chain anchors) are first-class top-level columns, never nested JSON payload paths, because Dolt JSON-path generated-column indexing is unreliable. The `judge_id` verdict guard already reads a first-class struct field in `evidencedturn`; **no Dolt migration is invented** (no Dolt projection schema exists in this repo — the principle is encoded in the contract). ## Ripple (every merged consumer, atomic) - `schemas/agentops-sdlc-provenance.v1.schema.json` (enum + description) - `cli/internal/provenancegraph/edge.go` (`Relations` + godoc) - schema-driven validator via `tests/scripts/validate-provenance-ledger.bats` (PROV-O accept + legacy reject) - consumers: `drrebuild`, `drwitness`, `evidencedturn`, `cmd/ao/provenance_*`, `cmd/ao/turn_verify` - all hash-chained fixtures **re-sealed via the canonical hasher** (`drrebuild` ledger + frozen `expected-graph-hash.txt`, `committed-witness.jsonl`); witness dolt-rows + provenance JSON fixtures updated - generated `cli/docs/COMMANDS.md` regenerated ## Verification - `cd cli && go build ./... && go vet ./...` clean; **`go test ./...` → 11942 passed in 72 packages** - `bats validate-provenance-ledger.bats witness-dolt-jsonl-crosscheck.bats` → 20/20 - `bats provenance-orphan-fixtures.bats check-provenance-orphans.bats` → 7/7 - `scripts/check-contracts-structural-floor.sh` → PASS (45 contracts) - `docs/provenance/ledger.jsonl` does not exist (no real entries to migrate); `.agents/ao/provenance/graph.jsonl` is a separate transcript-mining graph (no `relation` field) and out of scope. Closes-scenario: ag-lmdx.7#prov-o-vocabulary Bounded-context: BC4-Factory Evidence: schemas/agentops-sdlc-provenance.v1.schema.json |
||
|
Bo
|
554eb8c306 |
feat(turn): enforce author!=validator no-self-grading invariant (ag-lmdx.4 #author-validator-guard) (#658)
## What Enforce the no-self-grading invariant (`ag-lmdx.4`): a Turn graded by its own author is autocorrelated, so an Evidenced Turn is not done unless an independent judge context produced the acceptance verdict. - **`cli/internal/evidencedturn`** — new `author_neq_validator` predicate + `AuthorID`/`JudgeID`/`AllowSelf` fields on `Input`. The predicate FAILS when `judge_id == author_id` (self-graded) or when no judge identity was recorded (independence that was never recorded cannot be asserted); the documented, default-OFF `--allow-self` waives it for the inline fallback. This fills the `TODO(ag-lmdx.4)` extension point the #657 author left. - **`cli/cmd/ao/turn verify`** — `--allow-self` flag + `author_id`/`judge_id` on the turn-input file, wired into the DoD predicate so a self-graded verdict fails the `validated->closed` guard. - **`/pre-mortem` + `/vibe` SKILL.md** (and `skills-codex` twins) — refuse a self-graded PASS verdict; require the judge to be a blind, context-isolated sub-agent (`judge_id != author_id`); document `--allow-self` as the default-OFF escape and point at `ao turn verify` as the enforcement predicate. ## Tests (TDD-first) `evidencedturn`: self-graded refused / independent accepted / `--allow-self` waives / missing-judge refused. `cmd/ao`: `turn verify` self-graded exits non-zero, `--allow-self` passes. ## Gates `go build ./...` + `go vet` + `go test ./...` green (11845 pass). gosec: 0 new findings in touched files. Codex parity audit + generated-manifest/artifacts validators pass. `tests/skills/run-all.sh`: 87/87. COMMANDS.md regenerated (the one `--allow-self` line); codex manifest spliced to pre-mortem+vibe only (no drift-sweep). ## Dedup note (ag-9jle.5) This bead is the **invariant/guard** (data shape + predicate + skill refusal). The skill text states the blind-sub-agent requirement, but the actual blind-judge SPAWNING MECHANISM in `/vibe` + `/validation` (ag-9jle.5) is NOT implemented here. Partial overlap on the skill contract; the runtime mechanism remains for ag-9jle.5. Closes-scenario: ag-lmdx.4#author-validator-guard Bounded-context: BC2-Validation Evidence: cli/internal/evidencedturn/evidencedturn.go |
||
|
Bo
|
486cb99be5 |
feat(turn): ao turn verify Evidenced-Turn DoD predicate (ag-lmdx.5 #ao-turn-verify) (#657)
## What Adds `ao turn verify <bead>` — a legible Definition-of-Done predicate for an **Evidenced Turn** (ag-lmdx.5). Where the live AP#7 gate checks Evidence *presence*, this checks turn *sufficiency*: one verdict answering "can this bead legally transition validated->closed?" with a clear pass/fail per predicate and a reason for every gap (`--json` too). ## Definition of "Evidenced Turn" A turn is DONE iff all six predicates pass: | predicate | meaning | |---|---| | `chain_intact` | the bead's state_transition log hash-verifies and folds (turnstate `FoldVerified`) | | `terminal_state` | the folded lifecycle state is at the validated->closed boundary | | `scenarios_covered` | every Closes-scenario claim has a passing test | | `evidence_resolves` | every Evidence line resolves to a gate log that exercised THAT scenario (presence->sufficiency) | | `provenance_event` | a provenance edge in the ledger references the bead | | `no_orphan` | no artifact in the turn's trace-graph is a provenance orphan | ## How - New pure, in-memory predicate core `cli/internal/evidencedturn` (no Dolt / no shell-out) — the testable kernel that defines what "done" MEANS. - Builds directly on **#654** `cli/internal/turnstate` (`Seal/Append/Fold/FoldVerified/VerifyChain`) and `cli/internal/provenancegraph` (edges + `FindOrphans`). - `cli/cmd/ao/turn_verify.go` wires `--input` (turn-input JSON: state log + scenario coverage), `--ledger` (provenance EDGE ledger), and `--graph` (optional trace-graph for orphan audit). Non-zero exit on not-done makes it usable as a transition guard. ## Scope note (ag-lmdx.4) The author!=validator guard is a **separate** concern and is NOT implemented. It is left as a documented extension point (TODO on `Evaluate`); the verdict shape already accommodates an extra predicate row without a reshape. ## Tests `cli/internal/evidencedturn` (14 cases: complete-turn-done, each missing predicate fails with a legible reason, multi-gap, blank-bead error) + `cli/cmd/ao/turn_verify_test.go` (complete/incomplete/json/orphan/no-graph/missing-input/bead-mismatch). `go build ./...`, `go vet`, full `cmd/ao`+`evidencedturn` suites (9288) green. ## Derived surfaces regenerated COMMANDS.md, cobra `expectedCmds`, command-surface smoke (top=75 sub=188 all=263) + matrix.json, registry.json (cli_commands 72->73), cli-skills-map. Closes-scenario: ag-lmdx.5#ao-turn-verify Bounded-context: BC4-Factory Evidence: cli/internal/evidencedturn/evidencedturn.go |
||
|
Bo
|
81c9f305ac |
feat(ci): witness Dolt->JSONL re-derive + hash-compare (ag-lmdx.3 #witness-crosscheck) (#656)
## What The **watcher that watches the watcher** (ag-lmdx.3). A blocking CI gate that re-derives the hash-chained JSONL witness **from the Dolt projection** and hash-compares it against the git-committed witness ledger — proving Dolt is a *faithful projection* of the canonical event log. This is the **inverse** of drrebuild (#646): drrebuild rebuilds the Dolt context graph FROM the witness (JSONL + git blobs); this rebuilds the witness FROM Dolt rows. Together they pin both ends of the projection. Dolt history is rewritable (reset/--force/root) so it is NOT tamper-evidence; the git-anchored JSONL witness is. A Dolt rewrite that did not also rewrite the committed witness is caught here, and nowhere else. ## How - `cli/internal/drwitness/` — re-derive + cross-check core. Reuses drrebuild's exact hash discipline (`ComputeEventHashes`: payload_hash = sha256(canonical-JSON), hash = sha256(payload_hash + "\n" + prev_hash)) and canonical edge ordering. Re-derivation is order-independent (canonical sort then re-seal), which is what makes the hash-compare meaningful. - `cli/cmd/witness-crosscheck/` — standalone helper main (like `cmd/skill-frontmatter-json`), **NOT** an `ao` subcommand (no CLI surface added). - `scripts/witness-dolt-jsonl-crosscheck.sh` — hermetic CI gate. CI has no live Dolt, so it drives committed fixtures through the helper and asserts the gate fires correctly. - `tests/scripts/` + `tests/fixtures/witness/` — bats behavioral proof. ## Scenarios (acceptance) - **Clean state passes** — the faithful Dolt projection re-derives to the committed witness; gate exits 0. - **Tampered Dolt fails** — a rewritten Dolt row no longer matches the committed witness head; helper reports `DIVERGED`, gate exits non-zero. ## Verification - `go build ./... && go vet ./... && go test ./...` (full suite: 11948 passed / 71 pkgs; drwitness: 7/7) - `bats tests/scripts/witness-dolt-jsonl-crosscheck.bats` — 6/6 - `shellcheck -S warning` — clean - `scripts/validate-ci-policy-parity.sh` — PASS - diff is scope-only (10 files, additions); the sibling wave-pregate (#655) is present and untouched. Closes-scenario: ag-lmdx.3#witness-crosscheck Bounded-context: BC4-Factory Evidence: scripts/witness-dolt-jsonl-crosscheck.sh |
||
|
Bo
|
280f5d601a |
feat(turnstate): event-log canonical source of truth, state as fold (ag-lmdx.2 #eventlog-source-of-truth) (#654)
## What
Establishes the **state-machine projection** half of the ag-lmdx
inversion ("append-only event log canonical, state is a rebuildable
projection"). Where `cli/internal/provenancegraph` is the write model
for the provenance *graph* and `cli/internal/drrebuild` proves the
*graph* rebuilds from the ledger, this PR establishes the same invariant
for an artifact's lifecycle **state**:
> `context_artifact.state` is NOT authored. It is a FOLD over the
append-only, hash-chained `state_transition` log.
## How
New pure, in-memory write+fold kernel `cli/internal/turnstate`, reusing
the `rpi/ledger.go` + `provenancegraph` hash discipline verbatim
(`payload_hash = sha256(canonical-JSON sans chain fields)`, `hash =
sha256(payload_hash + "\n" + prev_hash)`):
- `Seal` / `Append` — the **only** path that advances state, via an
appended, hash-linked transition. There is no setter for state itself.
- `VerifyChain` — tamper-evidence (mirrors
`provenancegraph.VerifyChain`).
- `Fold` / `FoldVerified` — deterministic replay deriving current state
per artifact, with an **in-band `from_state` guard** (a transition's
`from_state` must equal the current folded state, so state cannot jump
without a contiguous transition). This is what makes the author!=judge /
orphan-gate invariants true *by construction*.
- `StateOf` — single-artifact projection.
Property tests cover: replay determinism, **order-independence**
(`replay(events) == replay(reverse(events))`), tamper detection,
broken-state-chain rejection, genesis-not-from-initial rejection, and
tie-break determinism.
## Scope (bounded slice, single rollback semantic)
This is the smallest credible testable slice that establishes "state is
a fold over the append-only log". Per ag-lmdx ("strangler-migrate, NO
big-bang rewrite"), two pieces are **deferred to a follow-up migration
bead**:
- Forbidding direct `context_artifact.state` writes at every existing
call site.
- Wiring the kernel to a durable JSONL/Dolt ledger.
No `ao` subcommand, no schema, no skill changes -> no derived-surface
regen.
## Gates
- `go build ./...` clean
- `go vet ./...` clean
- `go test ./...` -> 11949 passed / 69 packages; new package 20 tests
- gosec on the new package -> 0 issues
Closes-scenario: ag-lmdx.2#eventlog-source-of-truth
Bounded-context: BC4-Factory
Evidence: cli/internal/turnstate/turnstate.go
|
||
|
Bo
|
a6336b7c34 |
feat(provenance): ao provenance trace --orphans --strict gate (ag-x31t.6 #provenance-orphan-gate) (#653)
## What Adds `ao provenance trace --orphans --strict`: detects provenance orphans — artifact nodes with **no inbound authored/inferred edge** — by generalizing the `goals_trace_orphans` no-inbound chain-gap detection onto the provenance graph. Wires it as a blocking CI gate. ## How - **`cli/internal/provenancegraph/orphans.go`** — `ReadGraphRecords` (parses the goalstrace `Node`/`Edge` JSONL contract that the seeded fixtures use) + `FindOrphans` (an artifact node is an orphan iff no edge's `to_id` targets it; it flips green the moment any inbound edge is added). - **`cli/cmd/ao/provenance_trace.go`** — the `trace --orphans [--strict] [--json] [--graph <path>]` subcommand. `--strict` exits non-zero when orphans exist; `--json` emits one finding per line. - **`scripts/check-provenance-orphans.sh`** + a new **blocking** step in `.github/workflows/validate.yml` (in the existing goals/spec-linkage job, right after the warn-only `ao goals trace --orphans`). The gate asserts the strict audit **catches** each seeded orphan fixture and **passes** once an inbound edge wires the artifact back to a directive — so it is deterministically green while proving the detector is wired. ## Tests (TDD-first) - `cli/cmd/ao/provenance_trace_test.go` (L2): drives the real `tests/fixtures/provenance/` fixtures against `expected-orphans.json` — `--strict` catches each of the 3 seeded orphans (`gate:scenario-hash-stability`, `artifact:scripts/pre-push-gate.sh`, `claim:65-jobs`), and a wired graph exits 0. Plus mode/flag-guard cases. - `cli/internal/provenancegraph/orphans_test.go` (L1): detection, non-artifact nodes never orphaned, deterministic sort, JSONL parsing + malformed/missing-file rejection. - `tests/scripts/check-provenance-orphans.bats`: gate-script behavior. ## Derived surfaces regenerated - `cli/docs/COMMANDS.md` (new `ao provenance trace` subcommand, cobra-conformance passes) - CLI surface smoke fixture + matrix (sub 186→187, all 260→261) - `registry.json` **unchanged** (provenance is already a top-level command; subcommands aren't counted) — timestamp-only churn reverted. ## Gates `go build ./... && go vet ./... && go test ./...` (11959 pass) · gosec clean on new files (one `#nosec G304` on the operator/CI-supplied graph path, with reason) · `scripts/validate-ci-policy-parity.sh` PASS. Closes-scenario: ag-x31t.6#provenance-orphan-gate Bounded-context: BC4-Factory Evidence: .github/workflows/validate.yml |
||
|
Bo
|
2fecf2b304 |
feat(provenance): ao provenance export deterministic hash-chained (ag-x31t.5 #provenance-export) (#651)
## What Adds `ao provenance export`: a deterministic, hash-chained rendering of the committed provenance ledger (`docs/provenance/ledger.jsonl`). - Reads the ledger, **canonically sorts** edges by `(ts, from_id, to_id, relation)` (with type/trust/evidence tie-breakers for a total order), then **re-seals** them into a fresh per-record hash chain. - Output is **byte-identical on re-run** regardless of the ledger's physical append order. Default = JSONL (one compact edge per line); `--json` = indented array; `--verify` = one-line OK summary, no varying body. - The re-chained export **verifies with no Dolt server** — the committed JSONL is the audit authority, and re-chaining uses only the in-process hashing in `cli/internal/provenancegraph` (same `prev_hash` discipline as the rpi ledger). No reinvented Edge/hash logic: new `CanonicalSort`/`ReChain` helpers reuse `Seal`/`VerifyChain`. ## Tests (TDD-first) - `chain_test.go`: canonical-sort ordering + stability + non-mutation, order-independent re-chain, byte-identical serialization, empty ledger, invalid-edge rejection, tamper detection. - `provenance_export_test.go`: deterministic bytes across runs, chain verifies, empty ledger (`[]` not `null`), `--verify` summary, tampered-ledger rejection. ## Derived surfaces regenerated - `cli/docs/COMMANDS.md` (cobra conformance) - `evals/agentops-core/fixtures/cli-command-surface-smoke.sh` + `cli-command-surface-matrix.json` (sub 185→186, all 259→260) - `registry.json` unchanged: it counts top-level commands only (subcommands not tracked; top-level count = 72). ## Gates `go build` / `go vet` / `go test ./...` green; gosec clean on the new files; CLI-skills-map, JSON-flag-consistency, and `generate-cli-reference.sh --check` all pass. Closes-scenario: ag-x31t.5#provenance-export Bounded-context: BC4-Factory Evidence: cli/cmd/ao/provenance_export.go |
||
|
Bo
|
dacd6d2be9 |
feat(dr): rebuild Dolt context graph from JSONL+git blobs proof (ag-lmdx.1 #dr-rebuild-proof) (#646)
## What
Disaster-recovery **proof** for the ag-lmdx event-log-canonical design:
demonstrates that the Dolt context graph (nodes + provenance_edges) is a
**rebuildable projection** of two durable sources — the committed,
per-record hash-chained provenance ledger
(`agentops-sdlc-provenance.v1`) + content-addressed git blobs — and is
therefore never a valid sole source of truth. *"If you can't rebuild
from the audit, you have a hope, not a backup."*
This is a **PROOF artifact**, not the production `ao provenance rebuild`
service. New hermetic package `cli/internal/drrebuild/` + a committed
static fixture.
## Scenarios (from the bead)
- **Rebuild matches original** — replay the verified ledger against the
blob store; reconstructed graph hash-matches the original. Asserted both
in-memory and from a **static committed fixture** (fresh-clone path, no
live Dolt).
- **Missing blob is detected not silently dropped** — a dangling
content_ref (git-GC simulation) aborts loudly with `*DanglingRefError`
naming the ref; never returns a partial graph as complete.
## Why it's grounded, not a toy
- Models the **real** `agentops-sdlc-provenance.v1` ledger-event shape
and reuses its hash discipline (`payload_hash = sha256(canonical
payload)`; `hash = sha256(payload_hash + "\n" + prev_hash)`), mirroring
`cli/internal/rpi/ledger.go`.
- `GitBlobOID` matches `git hash-object` exactly (`sha1("blob
<len>\0<content>")`) — fixture blob **filenames equal their git OIDs**
(verified with `git hash-object`), so "rebuild from git blobs" is
literal.
- Chain verification (tamper + reorder detection) gates the rebuild: a
tampered witness is refused, not trusted.
## Gates
- `cd cli && rtk go build ./... && rtk go vet ./internal/drrebuild/`
clean
- `go test ./...` → 11947 passed, 0 failed (includes the repo-wide
`TestJSONMarshal_NoSwallowedErrors` guard)
- `gofmt -l` clean · diff is scope-only (6 new files)
## Scope boundary
Other ag-lmdx beads (.2 event-log, .3 witness, .4 author-guard, .5
turn-verify, .6 wave-pregate) untouched. The production rebuild service
would wrap `RebuildFromLedger`; this PR proves the rebuild is possible
and faithful.
Closes-scenario: ag-lmdx.1#dr-rebuild-proof
Bounded-context: BC4-Factory
Evidence: cli/internal/drrebuild/drrebuild.go
|
||
|
Bo
|
427c25b248 |
feat(provenance): provenance_edges write-model + ao provenance add (ag-x31t.4 #provenance-write-model) (#649)
## What Builds the WRITE side of the SDLC provenance/intent graph (ag-x31t slice 1). Adds `ao provenance add` / `ao provenance list` plus a new `cli/internal/provenancegraph` package that seals typed, evidence-backed edges onto the per-record hash-chained ledger at `docs/provenance/ledger.jsonl`. Consumes the schema merged in #637 (`schemas/agentops-sdlc-provenance.v1.schema.json`). Does not touch the schema (.2), export (.5), or gate (.6). ## CQRS doctrine Per CLAUDE.md and the council architecture, the committed JSONL ledger is the **audit authority and source of truth**; any Dolt `provenance_edges` table is a rebuildable projection that loses on disagreement. So this command appends the JSONL ledger directly. Hashing reuses the `cli/internal/rpi/ledger.go` discipline: `payload_hash = sha256(canonical payload)`, `hash = sha256(payload_hash + "\n" + prev_hash)`, genesis `prev_hash=""`. ## Behavior - `ao provenance add <from-id> <to-id> --relation ... [--from-type --to-type --trust-tier --evidence --ts --json]` — seals + appends one schema-valid edge. **Idempotent** on edge identity (endpoints+relation+evidence+trust-tier), so a re-run with a different timestamp is a no-op. - `ao provenance list [--json --from-id --relation]` — reads edges back in chain order. ## Tests (TDD, table-driven, exact-value asserts) - `edge_test.go`: field/enum validation, deterministic hash chain, schema-version forcing, tamper + chain-link detection, identity stability, JSON field-name parity with the v1 schema. - `store_test.go`: append→read round-trip + persisted-chain verify, idempotency, reject-invalid-before-write (no file created), corrupt-line rejection, and **validation of emitted edges against the merged schema via `scripts/validate-provenance-ledger.sh`**. - `provenance_add_test.go`: add produces a schema-valid sealed edge + list reads it back, idempotent no-op, invalid-relation error, list filters. ## Gates - `cd cli && go build ./... && go vet ./... && go test ./...` — all green (11949 pass). - Command-surface bumped: regenerated `cli/docs/COMMANDS.md`, updated `cli-command-surface-smoke.sh` + `cli-command-surface-matrix.json` (top 73→74, sub 183→185, all 256→259), added `provenance` to cobra expectedCmds; `validate-cli-skills-map.sh` PASS. Closes-scenario: ag-x31t.4#provenance-write-model Bounded-context: BC4-Factory Evidence: cli/cmd/ao/provenance_add.go |
||
|
Bo
|
1307eb0e0a |
feat(skills): queryable skill catalog JSON + ao skills (soc-vuu6.4 #skill-catalog-json) (#640)
## What Slice 2 of the skill-catalog bead (soc-vuu6.4): the **queryable surface** over the generated `skills/catalog.json`. Slice 1 (the generator, schema, drift gate, and `catalog.json` itself) landed in PR #379; this adds the `ao skills` query commands the bead's acceptance criterion calls for. - New `cli/internal/skills/catalog.go` — loads `skills/catalog.json` and a **pure, table-tested** query engine (`List`/`Consumers`/`Producers`/`Mermaid`). No SKILL.md re-parsing; reads the committed, CI-synced catalog. - New `cli/cmd/ao/skills_query.go` — four cobra subcommands: - `ao skills list [--role --produces --consumes --practice --user-invocable] [--json]` - `ao skills consumers <skill> [--json]` - `ao skills producers <output> [--json]` - `ao skills graph [--format mermaid]` - Regenerated `cli/docs/COMMANDS.md`; bumped the cli-command-surface fixtures (`sub` 179→183, `all` 252→256) for the 4 new subcommands. ## Why `registry.json` and `catalog.json` existed but had no query API — agents grepping "which skill produces X?" had to scan markdown. This turns the catalog into a queryable surface (the bead blocks `ao skill new`, which needs sibling-skill defaults). ## Acceptance > `ao skills list ... --json` returns the matching skill set in <100ms. Verified: `ao skills list --produces result.json --json` → `[beads, discovery]`; `ao skills consumers rpi` → 5 skills; `ao skills graph` emits deterministic Mermaid. Latency well under 100ms. ## Scope notes - Did **not** regenerate `skills/catalog.json` — it carries pre-existing drift on `main` (the `check-skill-catalog-drift` gate is I0/advisory). This PR only *consumes* the catalog; regenerating would sweep in unrelated drift. - The bead's illustrative `--tier judge` example does not map to the real generated schema (which has no `tier` field — it uses `hexagonal_role`, `produces`, `consumes`, `practices`). Filters implemented against the actual catalog fields. ## Tests `cli/internal/skills/catalog_test.go` (engine, exact-value table tests + load round-trip) and `cli/cmd/ao/skills_query_test.go` (JSON shape, sort invariants, flag validation, mermaid header) — all green. `go build`/`go vet`/`go test ./...` pass; conformance + command-surface smoke pass. Closes-scenario: soc-vuu6.4#skill-catalog-json Bounded-context: BC1-Corpus Evidence: cli/internal/skills/catalog.go |
||
|
Bo
|
0f16a574bc |
feat(eval): outcomes-rubric projection contract + schema (ag-hguuf #outcomes-rubric) (#624)
## What
Defines the machine-checkable contract for projecting the **locked eval
substrate** into an **Outcomes-style grading payload** (Anthropic
Managed Agents Outcomes analog) — holdout-safe by construction *and* by
schema.
The schema validates the **actual** output of `ao eval outcomes compile`
(`evalsubstrate.Rubric`), not the bead's pre-implementation prose sketch
(which imagined `suite_id`/`metric`/`decision_rule`/`split`). Per
source-of-truth precedence (executable > narrative), the shipped
projection deliberately emits *fewer* fields — a smaller cloud boundary
is a smaller leak surface. The "deliberately NOT projected" rationale is
documented in the contract.
## The load-bearing invariant
**Managed Agents are not ZDR** — a rubric crossing the cloud boundary
must never carry a ground-truth answer / holdout value / `target` /
`expected_output`. Enforced three ways:
1. **By construction** — `ProjectRubric` can't copy ground truth
(existing).
2. **By re-scan** — `Rubric.ContainsAny` guard (existing).
3. **By schema (this PR)** — `additionalProperties:false` at **every
nesting level** → a malformed compiler emission that smuggles a leak
field **fails schema validation**, not merely a code check.
## Files
- `schemas/outcomes-rubric.v1.schema.json` — the contract schema
-
`tests/fixtures/outcomes-rubric/{valid-dev,valid-holdout-criteria-only,invalid-contains-target}.json`
- `scripts/validate-outcomes-rubric.sh` — standalone validator (python
`jsonschema`, mirrors `validate-next-work.sh`); `--selftest` asserts
pass/pass/fail
- `tests/scripts/validate-outcomes-rubric.bats` — 9 acceptance tests
- `cli/internal/evalsubstrate/rubric_schema_test.go` — Go schema↔struct
drift guard (reflection)
- `docs/contracts/outcomes-rubric-projection.md` —
SCHEMA-field→Outcomes-field mapping; catalogued in doc index;
cross-linked from `eval-verdict-pipeline.md`
The contract **references** the LOCKED `~/.agents/evals/SCHEMA.md`
§rc2/§4 — it never edits it. Substrate EXTENDED, never relitigated.
## Evidence
- `scripts/validate-outcomes-rubric.sh --selftest` → PASS/PASS/FAIL
(exit 0)
- `bats tests/scripts/validate-outcomes-rubric.bats` → 9/9 ok
- `go test ./internal/evalsubstrate/` → 82 passed; `go vet` clean; `go
build ./...` success
- `check-contract-compatibility.sh` → contract catalogued + schema
reference OK
- `shellcheck scripts/validate-outcomes-rubric.sh` → clean
Closes-scenario: ag-hguuf#outcomes-rubric-projection-contract
Bounded-context: BC4-Evaluation
Evidence: schemas/outcomes-rubric.v1.schema.json
|
||
|
Bo
|
6617303639 |
feat(eval): gate #3 holdout-burn ledger — refuse re-grading spent holdout (ag-hdqu0 #gate3-burn-ledger) (#607)
## What Builds the **remaining half of `ag-hdqu0.9`** — SCHEMA §6 **gate #3, the holdout-burn ledger** — deferred in `gates.go` since Day-2 (`// Day 4 will extend with … holdout-burn-ledger`). The rubric-model half landed in #599; this completes the operator-chosen substrate prereq that unblocks `ag-hdqu0.5`. ## How - **`holdout_burn.go`** — `HoldoutBurnLedger` + `BurnRecord` projection model. `Spent(suiteRef, gtVersion)` counts only burns matching that pair (the ledger is global, so other suites/versions never consume this run's quota). - **`gate3HoldoutBurn`** — refuses a holdout-split run once the `(suite, gt_version)` budget is spent (reusing spent holdout lets the loop overfit to held-out answers, voiding the split's statistical guarantee). **Dev-split runs never burn**; **no-op** when no ledger is wired or no budget is configured. - Wired into `RunGates` in §6 order (`1, 3, 6, 7, 8, 9`); `GateInputs` gains `BurnLedger *HoldoutBurnLedger` — a pointer with `nil` default, so the live `ao eval task run` path (`eval_task.go`) keeps compiling and gate #3 is dormant there until the Dolt read-adapter wires it. ## Boundaries respected - **EXTENDS** the locked substrate (`internal/evalsubstrate`); `~/.agents/evals/SCHEMA.md` untouched — gate #3 is a slice the SCHEMA already specifies (§6 #3), not a relitigation. - Gate takes the **projected** ledger the same way other gates take injected `GroundTruth`; the global-Dolt read/write adapter stays a separate concern — the **write-side is `ag-hdqu0.5`**. - Holdout integrity: this is the *enforcement* side of NOT-ZDR holdout isolation — no holdout values are emitted anywhere. ## Tests (TDD, red→green) `cli/internal/evalsubstrate/holdout_burn_test.go` — 6 cases: ledger `Spent` scoping, exhausted→refuse, within-budget→pass, dev-split-never-burns, no-ledger-skips, and `RunGates` surfaces gate #3. Full package **78 pass**, `go vet` clean, `go build ./...` success, fast pre-push gate passed. Closes-scenario: ag-hdqu0.9#gate3-burn-ledger Bounded-context: BC4-eval Evidence: cli/internal/evalsubstrate/holdout_burn_test.go |
||
|
Bo
|
6658c2bdd5 |
feat(eval): stale-rubric rejection guard via judge_content_hash (ag-hdqu0 #stale-reject) (#604)
## What Adds `evalsubstrate.Rubric.FreshAgainst(currentJudgeHash)` — the **drift-parity guard** for ag-hdqu0 (SCHEMA gate-#2 parity). An Outcomes rubric is a derived artifact content-addressed off the judge; `FreshAgainst` refuses any rubric whose carried `judge_content_hash` ≠ the current judge, so a divergent/stale cloud rubric **self-invalidates** rather than grading against an outdated bar. Empty hash on either side is unverifiable → rejected. Pure-Go internal guard — **no CLI surface**, so no canary/allowlist involvement. ## Tests (TDD) - `TestRubric_FreshAgainst` — match→fresh · mismatch→error naming both hashes · empty-rubric-hash→reject · empty-current→reject. - 72 pass in evalsubstrate, vet/build clean. Closes-scenario: ag-hdqu0.4#stale-reject Bounded-context: BC1-Corpus Evidence: cli/internal/evalsubstrate/rubric_freshness_test.go |
||
|
Bo
|
e4e32a3374 |
feat(orchestration): dual-runtime OrchestrationPort foundation (ag-nk67 #orchestration-foundation) (#598)
## What
The **dual-runtime OrchestrationPort foundation** — the seam that gates
the
`[EPIC] AgentOps × Claude Managed Agents integration (dual-runtime)`
application epics.
Implements the 3-category model (Claude Workflow / NTM swarm / plain
skill) behind a
single `OrchestrationPort` with safe degradation **NTM → Claude-native →
beads floor**
and a global `AGENTOPS_ORCHESTRATION=off` opt-out.
Derived from a live 3-legged spike (`~/dev/agentops-3cat-spike/`): safe
degradation is
solvable at the selection layer; parallel buys quality/independence not
wall-clock at
small N; NTM is a multi-vendor control plane that runs Claude/Codex as
panes.
## What's in it (10 slices, agent-team driven)
- **Port + adapters** — `cli/internal/ports/orchestration.go` (idiomatic
to existing ports pattern) + new `cli/internal/orchestration/` pkg:
`Selector`, `ntm_probe` (capability detection via `ntm
--robot-capabilities`, not `command -v`), `beads_floor` +
`OrchestrationResult` parity type, degradation-conformance test.
- **Live CLI** — `ao orchestrate select` (`--pin`/`--opt-out`/`--json`);
COMMANDS.md regenerated.
- **Contracts** — `schemas/orchestration-{backend,result}.v1` + paired
`docs/contracts/` docs (structural-floor gated: 41 contracts).
- **Prose** — swarm/shared/crank rewritten NTM>native>beads; **gc
demoted** (soc-2rtm0); `rpi_phased_stream.go` doc-comment fixed.
- **Skills** — `automation-shape-routing` (the 3-category router) +
`workflow-builder` + meta-skill authoring chain wired (`context_rel`);
skill counts synced.
- `lib/orchestrate-select.sh` selector seam.
## Verification (local)
`go build ./...` clean · `go vet` clean · **56 Go tests pass**
(orchestration 31, ports 10, cmd/ao orchestrate 15) · structural-floor
PASS (41) · `lib/orchestrate-select.sh --self-test` 6/6 · live command:
`--json`→claude (NTM absent), `--opt-out`→beads, `--pin claude`→claude.
## Notes
- Bundles the foundation + the routing/authoring skills as one coherent
dual-runtime arc.
- NTM/Claude executors are stubs (the parity contract is real); concrete
executors land in the application epics this gates.
- Two-ladders distinction preserved (spawn-backend ladder A vs CLI
phase-executor ladder B) — see `docs/contracts/orchestration-ports.md`.
Closes-scenario: ag-nk67#orchestration-foundation
Bounded-context: BC5-Runtime
Evidence: cli/internal/orchestration/conformance_test.go
|
||
|
Bo
|
0ff68d64be |
feat(eval): holdout-safe Rubric projection for Outcomes (ag-hdqu0 #rubric-model) (#599)
## What Adds `evalsubstrate.Rubric` + `Criterion` + `ProjectRubric()` — the holdout-safe projection `ao eval outcomes compile` will emit. The rubric-model half of the operator-chosen substrate prereq (ag-hdqu0.9). ## Holdout isolation (the load-bearing invariant; Managed Agents are NOT ZDR) - **By construction:** `ProjectRubric` never receives ground-truth rows, so it physically cannot copy `GroundTruthRow.Value`. - **Defense-in-depth:** `Rubric.ContainsAny` re-scan guard (holdout-leak guard layer 3) for callers to run on the emitted payload before any cloud boundary crossing. - Carries `judge_content_hash` so a stale rubric self-invalidates like a stale local judge (SCHEMA rc2 parity). EXTENDS the locked eval substrate; SCHEMA.md stays the sole authority. ## Tests (TDD) - `TestProjectRubric_StripsHoldoutValues` — keystone: projected rubric leaks none of the holdout answers. - `TestRubric_ContainsAny` — guard passes clean, catches a planted value, ignores empty strings. - 71 pass in evalsubstrate, `go vet` clean, `go build ./...` success. ## Remaining for ag-hdqu0.9 The gate#3 global-Dolt holdout **burn ledger** is the other half (separate slice). Closes-scenario: ag-hdqu0.9#rubric-model Bounded-context: BC1-Corpus Evidence: cli/internal/evalsubstrate/rubric_test.go |
||
|
Bo
|
68b0b93d6f |
refactor(bridge): delete orphaned gascity cluster, keep live semver/codex helpers (ag-hfc #gascity-orphan-cluster) (#590)
3.1 teardown **S2** (ag-hfc). Deletes the orphaned GasCity compat cluster. ### Why it's an orphan (verified repo-wide) - `agentworker.GasCityClientAdapter` / `NewGasCityWorker` — **zero callers** anywhere (grep, incl. packs/plugins). - `internal/gascity` — imported only by that dead adapter. - `bridge/gc.go`'s `GC*` symbols — used only by `gascity/fallback.go` (deleted here). - `packs/`/`plugins/` pre-gate: clean. ### Corrects the stale premise (closed ag-2oi / old ag-hfc text) Both prior beads claimed "2 doctor-hint importers, pure `rm`, −3,399". Reality: - `bridge/gc.go` is **mixed** — it also held `CompareSemver`/`ParseSemverParts`, used **live** by `cmd/ao/codex_runtime.go`. Extracted to `bridge/semver.go` before deleting `gc.go`. - The doctor gc-detection (`doctor/fix_bridges.go`) is **independent** — it shells out to the `gc` binary and imports neither package. **No rewire needed.** - `agentworker/gascity.go` also defined a generic `firstNonEmpty` used by `cli_fallback.go` + `quarantine.go` — relocated to `agentworker/strings.go` (the build caught this cross-file coupling that static grep missed). ### What changed - **Deleted:** `internal/gascity` (whole pkg + testdata), `agentworker/gascity.go` + `gascity_test.go`, `bridge/gc.go` + its 12 gascity test fns. - **Preserved:** `bridge` codex helpers, `bridge.CompareSemver`/`ParseSemverParts` (→`semver.go`), `agentworker.QuarantineWriter`, `agentworker.firstNonEmpty` (→`strings.go`). - Updated the now-false `internal/gascity` "untouched here" note in CLAUDE.md. ### Verification - `go build ./... && go vet ./...` green - 76 bridge + agentworker tests pass; full `go test ./...` green - −4,713 / +47 lines Closes-scenario: ag-hfc#gascity-orphan-cluster Bounded-context: BC5-Runtime Evidence: spec/teardown-2026-05-28/ |
||
|
Bo
|
471537ed2d |
refactor(agentworker): delete dead wikiworker, relocate quarantine anchor to live writer (ag-jqx #wikiworker-delete) (#589)
Continues #581 (ag-nxm) S1 teardown. #581 split `wikiworker` to ag-jqx as "contract-entangled" — 0-import but the only production anchor for the `.agents/quarantine` write surface. This PR resolves that entanglement. ### What - Delete `cli/internal/wikiworker/` — 0 importers (verified `grep -rl`), dead caller code. - Relocate the surface anchor to its correct home: add `agentworker.DefaultQuarantineDir()` so the **live** writer (`agentworker.QuarantineWriter`, kept) self-documents its standard `.agents/quarantine/agentworker` location. ### Why this is safe The only runtime thing wikiworker contributed was `worker.go:280` `filepath.Join(".agents","quarantine","agentworker")` — the production reference that `TestAgentsWriteSurfaces_EachAllowlistEntryHasProductionReference` requires for the documented `quarantine` allowlist entry. Relocating it to the writer package keeps the contract satisfied without keeping a dead caller alive. Zero behavior change. ### Verification - `go build ./... && go vet ./...` green - `TestAgentsWriteSurfaces_EachAllowlistEntryHasProductionReference` green (RED on naive delete) - `go test ./internal/agentworker/` green - −1403 / +9 lines ### Scope ag-jqx residual: the `safety/` (`.agents/swarm-role` anchor) contract decision remains operator-gated and stays open on ag-jqx. Closes-scenario: ag-jqx#wikiworker-delete Bounded-context: BC5-Runtime Evidence: spec/teardown-2026-05-28/ |
||
|
Bo
|
51424bea8c |
fix(cli): error-handling correctness — HTTP error leak, silent bd update, lost error chain (ag-1pp #cli-error-handling) (#587)
## Summary Three verified error-handling correctness fixes in `cli/`, each with a focused regression test. - **ag-1pp** — `cmd/ao/rpi_serve.go` `serveRPIArtifact`: `http.Error(w, err.Error(), 500)` leaked the raw internal error (carrying filesystem paths) into the HTTP body. Now logs the real error to stderr and returns a generic `"internal error"` to the client. The sibling `http.Error` sites in this handler are static client-input 4xx/5xx messages (`invalid run-id`, `invalid path`, `artifact not found`, `SSE not supported`) — left unchanged, as instructed. - **ag-3lt** — `cmd/ao/beads_audit_cluster.go` `autoCloseLikelyFixed`: `_, _ = execBD(...)` silently swallowed errors on a tracker state mutation (closing a bead). It now returns a wrapped error; both callers in `recordLikelyFixedAuditFinding` surface it via the file's existing `fmt.Fprintf(os.Stderr, "WARN: ...")` pattern. - **ag-uha** — `cli/internal/mine/mine.go` `MineGitLogContext`: the `*exec.ExitError` branch formatted only stderr with `%s`, dropping the chain so `errors.As` could no longer recover the underlying `*exec.ExitError`. Added `: %w` (stderr text preserved in the message). ## Tests - `TestServeRPIArtifact_InternalErrorNotLeaked` — asserts a 500 read failure returns the generic body and does not leak `open ...`/the artifact path. - `TestAutoCloseLikelyFixed_SurfacesError` + `TestRecordLikelyFixed_AutoCloseInvokesBD` — assert the bd error is returned/wrapped and the close still fires through the audit flow. - `TestMineGitLogContext_ExitErrorPreservesChain` — asserts `errors.As` recovers `*exec.ExitError` and the message keeps both context prefix and git stderr. ## Local validation `cd cli && go build ./... && go vet ./... && go test ./...` — all green (11946 tests). Closes-scenario: ag-1pp#cli-error-handling Closes-scenario: ag-3lt#cli-error-handling Closes-scenario: ag-uha#cli-error-handling Bounded-context: BC5-Runtime Evidence: cd cli && go test ./cmd/ao/ -run 'TestAutoCloseLikelyFixed_SurfacesError|TestServeRPIArtifact_InternalErrorNotLeaked' |
||
|
Bo
|
00a7d5e53b |
chore(cli): delete 2 zero-import dead packages (ag-nxm #dead-pkgs) (#581)
## Wave 1 — AgentOps teardown (epic ag-097)
Deletes two genuinely-dead Go packages: `cli/internal/plans` +
`cli/internal/worker` (verified 0 importers, no blank imports, no non-Go
refs). Build + vet + full test suite green (EXIT=0, 0 failures).
### Scope discipline (why 2, not 6)
Discovery + execution narrowed the candidate list:
- **`internal/domain` excluded** — `domain/packet` has 4 live importers
(ports/storage, adapters/storage_fs, rpi/artifacts). Not dead.
- **`safety`, `wikiworker`, `feedbackcompiler` split to ag-jqx** —
they're 0-import but **contract-entangled**: `safety` is the only
production anchor for the `.agents/swarm-role` agents-write-surfaces
entry; `wikiworker` for `.agents/quarantine` (a live surface —
agentworker has 26 active quarantine tests); `feedbackcompiler` is the
documented forge F5.4 capability. Deleting them naively fails
`TestAgentsWriteSurfaces_EachAllowlistEntryHasProductionReference`. They
need owner decisions, not a naive rm.
### Evidence
- spec/teardown-2026-05-28/{spine,slop-map,strangler-plan}.md (added in
this PR)
- .agents/research/2026-05-28-teardown-discovery.md (local)
Closes-scenario: ag-nxm#dead-pkgs
Bounded-context: BC5-Runtime
Evidence: spec/teardown-2026-05-28/
|
||
|
Bo
|
465f3b77a6 |
fix(rpi): resolve merge lock via git common dir
Fix RPI merge lock placement for linked Git worktrees.\n\nCloses-scenario: ag-pmd#linked-worktree-merge-lock\nBounded-context: BC5-Runtime\nEvidence: scripts/pre-push-gate.sh --fast |
||
|
Bo
|
4f2ef54179 |
fix(evals): drop deleted validate-hooks-doc-parity.sh ref from docs-release-governance canary (ag-jov #eval-references-only-existing-scripts) (#575)
## Summary - Hooks were removed in 3.0 and `scripts/validate-hooks-doc-parity.sh` was deleted. `ag-s3z` removed the Nightly workflow step that ran it but the critical eval `evals/agentops-core/docs-release-governance.json` still referenced the deleted script in 3 spots: a shell command (`docs-policy-parity-gates`), a `HOOKS_DOC_PARITY: PASS` stdout expectation, and an `artifact_contains` target (`doc-release-script-contracts`) — so the canary fails when run. - Removed those 3 references plus the now-inaccurate "hook/docs parity" prose in the suite description and two case objectives. - Added `cli/internal/eval/governance_refs_test.go`: a regression gate that loads this suite and asserts every shell script token and every `artifact_contains` target resolves on disk, and that no reference to the deleted script / `HOOKS_DOC_PARITY` marker remains. ## Test plan - [x] `cd cli && go test ./internal/eval/ -run TestDocsReleaseGovernanceEval` — both new tests fail on `main` (dangling ref) and pass after the edit (test-first verified) - [x] `cd cli && go build ./... && go vet ./internal/eval/` - [x] `cd cli && go test ./internal/eval/... ./cmd/ao/` - [x] JSON re-validated (`python3 -m json.tool`); `LoadSuite` strict-decode + `ValidateSuite` pass Out-of-scope dangling refs (different bounded context: docs + local gate) tracked in `ag-c2i` (discovered-from `ag-jov`). Closes-scenario: ag-jov#eval-references-only-existing-scripts Bounded-context: BC1-Corpus Evidence: cd cli && go test ./internal/eval/ -run TestDocsReleaseGovernanceEval |
||
|
Bo
|
2c5ac99e4d |
feat(beads): ao beads scenarios validate <id> — well-formedness check (ag-5cz #scenarios-validate) (#569)
## Summary Slice 4 of **ag-dwq** (child bead **ag-5cz**). Adds `ao beads scenarios validate <bead-id>`: a read-only, **dependency-free** validator for an authored `## Scenarios` Gherkin block. It is the inverse acceptance gate of the `extract` slice (#567) — extract turns acceptance into a candidate block; validate confirms an authored block is well-formed. - **`cli/internal/scenarios/validate.go`** (new, pure domain): `ParseBlock(text) ([]Scenario, error)` + `Validate(text) error`. A deterministic in-repo parser — **no `github.com/cucumber/gherkin-go` dependency** (the bead's open design question, resolved toward the repo's established dependency-free pattern). Each scenario must declare a name and a Given/When/Then step in that order with a non-empty body; `And`/`But` lines are accepted as continuations. Refactored into `groupScenarios`/`assemble`/`checkStepOrder` to stay under the cyclomatic-complexity budget (max 14 < 25). - **`cli/cmd/ao/beads_scenarios.go`**: `validate` subcommand under the existing `ao beads scenarios` surface. Agent-ergonomic (Directive 13): exits 0 when well-formed, non-zero **naming the parse error** otherwise; `--json` emits a verdict object (`{bead_id,valid,scenarios}` on success / `{bead_id,valid:false,error}` on failure) on **stdout** with diagnostics on **stderr**; `bd`-unavailable degrades gracefully (warn + exit 0). - **`cli/docs/COMMANDS.md`** regenerated. `capabilities` / `robot-docs` pick the command up automatically via the live command-tree walk. Surface heading counts are **unchanged** (validate is a `#####` leaf under the existing beads-scenarios group, so the `#{3,4}` command-surface canary is unaffected). ### Out of scope (remaining ag-dwq slices) `--write` / operator-confirmed bead update (**ag-3gm**) · LLM fallback for non-deterministic prose. Parent **ag-dwq** stays OPEN. ## Test plan - [x] `cd cli && go test ./internal/scenarios/...` — 14-case `TestParseBlock` table (well-formed single/multi, And/But continuations, block bounded by following heading, prose-prefixed block, no-block, no-Scenario, missing-name, missing Given/When/Then, out-of-order, empty-body, And-before-primary) + Validate nil/error + Render round-trip - [x] `cd cli && go test ./cmd/ao -run Scenarios` — L2 with stubbed `bd`: well-formed exit 0, malformed error, no-block error, `--json` success/failure verdicts, bd-unavailable graceful, read-only contract (no `bd update`) - [x] `cd cli && go build ./... && go vet ./... && go test ./...` — 11951 passed, 69 packages, 0 failures - [x] Real binary: `ao beads scenarios validate ag-3gm` → exit 0 (1 scenario well-formed); `ag-dwq` (free-text) → exit 1 naming the missing block; `--json` verdicts valid - [x] `gocyclo -over 15 internal/scenarios/` — clean; `scripts/generate-cli-reference.sh --check` — up to date; command-surface smoke — `top=69 sub=174 all=243`, exit 0 Closes-scenario: ag-5cz#scenarios-validate Bounded-context: BC1-Corpus Evidence: cd cli && go test ./internal/scenarios/... ./cmd/ao -run Scenarios |
||
|
Bo
|
70b6d638dc |
feat(beads): idempotent '## Scenarios' guard + --force for scenarios extract (ag-kyn #scenarios-guard) (#568)
## Summary Next slice of the open epic **ag-dwq** (Gherkin scenarios extractor). Slice 1 (`ag-ljd`, the dry-run `extract` command) merged in #567; this adds the **idempotent guard** (ag-dwq scenario 3): `ao beads scenarios extract <id>` now **refuses** when the bead already carries a `## Scenarios` block — prints a diagnostic to stderr naming `--force` as the corrective action, exits 0, emits nothing on stdout — **unless `--force`** is passed, in which case it re-extracts and prints normally. - New pure `scenarios.HasScenariosBlock(text)`: detects a markdown `## Scenarios` section heading (plural-only, case-insensitive). A bare `Scenario:` line or singular `## Scenario:` heading does **not** match. - Fetch now returns the bead's acceptance **and** description (`fetchedBead`) so the guard can inspect the description, where authored scenario blocks live. - Stays a **dry-run**; no bead mutation on any path. Out of scope (remain in parent ag-dwq): `--write`, `validate` subcommand + Gherkin parser dep, LLM fallback. > **Bead note:** the canonical follow-up bead for this work is **ag-kyn** (filed by a concurrent RPI cycle as a child of ag-dwq). A duplicate bead `ag-gam` was created before ag-kyn was visible; `ag-gam` is closed as duplicate-of `ag-kyn`. Branch name retains the `ag-gam` token but the work closes `ag-kyn`. ## Test plan - [x] `cd cli && go build ./... && go vet ./... && go test ./...` — clean (66 pkg ok, 0 fail) - [x] `go test ./internal/scenarios/ ./cmd/ao` — targeted, incl. new `TestHasScenariosBlock` (11 cases) + 2 new L2 guard tests - [x] Real binary: refuses on `ag-ljd` (has block), extracts on `ag-dwq` (free-text bullets) - [x] `cli/docs/COMMANDS.md` regenerated; `cli-command-surface-matrix` smoke passes (69/174/243 — flags aren't headings) Closes-scenario: ag-kyn#scenarios-guard Bounded-context: BC1-Corpus Evidence: cd cli && go test ./internal/scenarios/ ./cmd/ao |