Session mining could overwrite its source when the checkpoint referred
to the same file, consume pending events during `--dry-run`, and miss
native Codex custom-tool input edits when checking for transcript
rewrites. The CLI now rejects source/checkpoint aliases before output or
writes, previews from the current watermark without persisting it, and
preserves native custom input so edits invalidate the checkpoint.
Regression controls retain normal incremental mining and legacy argument
handling.
Postmortem now accepts real goal/session evidence without requiring a
fabricated verdict file, uses current Memory ownership and protected
external storage, and separates structural checks from causal support.
Implement guidance emphasizes early representative checks and exact
repair verification. The Go standard and its local rule now point to the
actual CI complexity gate and explain its committed-file scope. Normal
projections are regenerated; no skill roots, framework or gate
thresholds were added or weakened.
The bounded evaluation used three useful coding tasks with fixed
behavior acceptance. Trial two exposed a real workflow defect: lint and
fast checks passed while CI rejected complexity 25. Checkpoint
construction was extracted, reducing MineSession to 20, and the Go
guidance was corrected before the third task. The failed attempt stays
in the evidence; this is an observational result, not proof of token
savings or superiority over native prompting. Final local Go
build/vet/test, lint, strict whole-campaign complexity and 37 selected
gates pass. [CI run
34628874411](https://github.com/boshu2/agentops/actions/runs/34628874411)
is green on `62205367e91946f84723147763662ca39ad4b580`, including Linux
race/shuffle tests, Windows, Bats and security. The independent review
checked all 27 changed paths and returned PASS with no unresolved
findings or unchecked acceptance; its earlier complexity FAIL remains
preserved. The local aggregate retains its existing skip for the absent
legacy OL test directory.
AgentOps previously relied on behavioral probes and retrospective
summaries to assess skills. This adds a development-only evaluator that
runs a frozen installed skill package on isolated Go tasks, preserves
failed and interrupted attempts, and rebuilds a comparison readout from
native results without another model call.
The suite contains six task families, separate executable verifiers,
frozen launch identities, native session accounting, and a focused
`skill-eval` maintenance workflow. The readout separates passing code
from completed trials, retains incomplete cost information, and reports
missing evidence without claiming equivalence or uplift. `ao eval`
remains retired; no new runtime controller or required core skill is
introduced.
Validation: Go build/vet/race checks and all repository CI passed.
Focused reader/statistics, receipt integrity, verifier integrity,
fixture calibration, generated projections, and the local aggregate
runner passed. A real two-variant Docker preparation check verifies that
frozen worker and verifier images survive later staging.
The bounded coding pilot retained all 24 starts and produced eight
comparable pairs across six task families, with no observed paired
endpoint difference. The separate eight-start memory experiment did not
demonstrate incremental benefit and does not promote another guidance
rule. Individual runtime limits were enforced; aggregate desktop
deadline enforcement remains unproven. Raw trial evidence and
credentials stay outside Git.
The same gate set failed on every open PR and drove all 16 nightly-failure
issues. Each root-caused via isolated-worktree reproduction of the live CI
jobs, then fixed minimally:
- correctness: check + propagate the discarded json.Marshal error in
plan-pawl --json and yield tokens --json (planning-rule f-2026-04-29-002).
- lint: 3 markdownlint violations in pre-existing audit docs (MD004 leading
"+" continuations, MD037 skills/* glob emphasis).
- security: dual #nosec + nosemgrep on 4 gosec/semgrep false positives in
local-CLI code (G404 e-exploration RNG, G101 message-type sentinel, G122
corpus-walk I/O) -- not a dependency CVE; per .claude/rules/go.md.
- skills-integrity: scope the broken-symlink scan to the shipped dirs
(skills hooks lib scripts), matching the pre-push gate, so the intentional
root CLAUDE.md -> AGENTS.md symlink is not falsely flagged.
- go-gate-shadow: add the 4 post-parity job-scoped blocking scripts to the
deferredWorkflowScripts allowlist (--require-workflow-parity missing_blocking
4 -> 0); strengthen the plan-pawl JSON test (command/test pair).
contracts-sync + skill-gates were already green on main (no change).
Round-2 cross-family review (Codex) caught that the command advertised
"Claude Code or Codex" transcripts but deriveTranscriptTokens only summed
message.usage (the Claude shape). A real Codex transcript returned 0 0 —
and Codex is the default runtime on this machine, so reconcile would have
silently recorded zero spend for Codex-produced work.
Codex reports a CUMULATIVE total_token_usage on each token_count event
(input_tokens already includes cache; total = input + output), so the LAST
one is the session total — NOT a per-message sum.
- parser: capture token_count's total_token_usage into the last-wins
ParseResult.FinalUsage (folded in processLine, never added to the summed
message list).
- ParseResult.TokenTotals() picks the right aggregation per runtime: Codex
cumulative FinalUsage, else Claude per-message SumUsage (dedup-by-id).
- deriveTranscriptTokens + forge_tier1 use TokenTotals(); the malformed
guard now treats a Codex transcript with FinalUsage (few messages) as parseable.
Verified on a real Codex session (32397/40 == ground-truth last total) and
the real Claude fixture (10946330/228 unchanged). Tests use real Codex +
Claude persisted shapes. Full build/vet/test green.
Refs: age-membrane-memory-arch-tz2s.3.2
A fresh-context cross-family review (Codex) REFUTED the E4 producer-truth
diff and caught three real defects self-review missed — the membrane
working as designed:
1. OVERCOUNT (severe): real Claude Code transcripts write one model
response as several rows (thinking/text/tool_use) that ALL repeat the
same usage block under one message.id. Summing every row inflated
tokens 2-3x (real fixture: 26781956 vs correct 10946330). Fix: capture
message.id (fallback requestId) in the parser; new types.SumUsage dedups
by response id; forge_tier1 + deriveTranscriptTokens both use it.
2. reconcile HOLD/REFUTED path ignored AO_YIELD_TRANSCRIPT -> rejected
spend silently recorded 0 (broke L/R loss accounting). Fix: shared
derive_yield_tokens() helper used by BOTH the accepted and rejected
emit paths.
3. malformed transcript returned success 0 0 (SkipMalformed) instead of a
visible fail-open. Fix: deriveTranscriptTokens errors when content
yields zero messages, so reconcile keeps the honest default.
Plus: dropped misleading omitempty on value-type StreamEvent.Usage;
real-fixture regression test locking the deduped total (10946330/228);
dedup + malformed + message.id capture tests using real persisted shapes
(the synthetic one-row fixtures hid the overcount — go.md fixture-fidelity).
Full build/vet/test green; reconcile bash + shellcheck clean.
Refs: age-membrane-memory-arch-tz2s.3.1, age-membrane-memory-arch-tz2s.3.2
Producer truth, E4.1. Tokens were silently dropped to 0 at every
observation point, making the medallion's bronze tier lossy.
- stream_events.go: parse the result event's usage block (StreamUsage:
input/output + cache-creation/cache-read) with TotalInputTokens/
TotalTokens helpers.
- stream_parser.go: wire usage into PhaseProgress.Tokens (live status
now shows real tokens, not 0).
- types.TranscriptMessage: carry a per-message *TokenUsage.
- parser: capture the assistant message usage block from transcripts.
- forge_tier1.deriveSessionMeta: sum real TokensIn/TokensOut across
messages (killed the 'not tracked in v1' hardcoded 0) — the bronze
capture E4.2 (bronze->silver) will emit.
TDD: token tests added at stream, parser, and bronze-ingest layers; all
red before, green after. Full cmd/ao + parser + types + llm green.
Refs: age-membrane-memory-arch-tz2s.3.1
Completes the sweep started in cycles 1-4. Applies the same byte-length
upper-bound fast-path to the four remaining rune-safe truncation sites:
- internal/search/search_cass.go TruncateContext
- internal/parser/parser.go Parser.truncate
- cmd/ao/contradict.go truncateSnippet
- cmd/ao/extract.go truncateForPrompt
All call sites now follow the same pattern: if len(s) <= max, return s
without allocating a []rune slice. Behavior is unchanged for inputs
whose byte length exceeds max (the rune-count branch still fires).
When the input byte length already fits within maxLen, the rune count
cannot exceed it either (bytes >= runes). Short-circuit before the
[]rune allocation. Adds multibyte test cases proving both the fast-path
(bytes==runes), the rune-count fit case (bytes>maxLen, runes<=maxLen),
and the truncation case.
Closes finding: ag-yht / Add ASCII fast-path comment to truncate()
- markdown.go: replace strings.Trim(s, "`") with paired-backtick check in
parseGateRow; asymmetric/lone backticks are now preserved instead of
being silently corrupted or data-lost
- parser.go: truncate() now slices at rune boundaries ([]rune) to avoid
splitting multi-byte UTF-8 chars (CJK, emoji) and producing \uFFFD
- Tests: +3 adversarial cases (asymmetric, single-backtick, matched-pair)
+1 Unicode truncation case (CJK at rune boundary)
Closes: ag-yht.1, ag-yht.2
Apply version-aware modern Go patterns across cli/cmd/ao/ and cli/internal/:
- sort.Slice → slices.SortFunc with .Compare() (19 in cmd/ao, 15 in internal)
- C-style for loops → for i := range n / for range n (18 conversions)
- default-value chains → cmp.Or (19 conversions across 11 files)
- HasPrefix+TrimPrefix → strings.CutPrefix (2 in internal)
- strings.Index+slice → strings.Cut (1 in internal)
- sort.Strings → slices.Sort (3 in internal)
All 18 packages pass with -race. Sourced from JetBrains go-modern-guidelines RPI.
Replace repeated string literals with named constants:
- context: item type constants (ItemTypeFailingTest, etc.)
- parser: message type and error classification constants
- goals: drift delta and measurement result constants
- vibecheck: severity and health classification constants
- storage: default slug constant
- ratchet: unknown value constant
Addresses goconst lint warnings for production code.
Split content extraction logic into a dedicated helper that handles
both string and array-of-text-blocks formats. parseToolResult drops
from complexity 8 to 3; new helper is 6.
Replace all 135 occurrences of interface{} with the any alias
introduced in Go 1.18. The module uses Go 1.23 so this is safe.
Zero behavior change -- any is a built-in alias for interface{}.
Move line parsing, error recording, and message accumulation into
processLine. Parse now focuses on scanning and checksum, dropping
from complexity 9 to 6.
Bo