Files
boshu2__agentops/docs/knowledge-flywheel.md
T
Bo 46ed7fee5a docs(3.0): sweep remaining daemon/schedule/hook prose + MIGRATION-3.0 (ag-qbw) (#520)
## What

Wave 2 of the AgentOps 3.0 honest-docs sweep (post-#518 audit, bead
ag-qbw). Makes the `docs/` prose match the shipped code: the daemon,
scheduler, factory, and all hooks were **DELETED** (not deprecated).
AgentOps is the in-session loop + context compiler; out-of-session
orchestration is delegated to a substrate (Gas City reference City /
Olympus). Mirrors the already-correct `docs/3.0.md`.

This makes `docs/3.0.md:150` ("the standalone
daemon/scheduler/overnight-runner surfaces are gone from ... live doc
prose") true for the enumerated wave-2 surfaces.

## Deleted (whole docs about deleted commands)

- `docs/scheduling.md` — entire `ao schedule` + `ao daemon` guide
-
`docs/contracts/{agentopsd-control-plane,agentopsd-schedule,routing-policy}.md`
- `docs/contracts/jobspec-openapi-v0.yaml`
- `docs/templates/dark-factory-schedule.yaml.example`
- `docs/design/olympus-v4-specs/*.md` — v4-port reference for the dead
Olympus → agentopsd extraction

## Reframed (prose → in-session + Gas City)

- Daemon/schedule/overnight: `agentops-3-youtube-starter-series.md`,
`agentops-3-pmf-evidence-loop.md`, `wiki-for-agents.md`,
`activation-profiles.md` (+ `profiles/README.md` profile rename),
`CI-CD.md` (overnight lane)
- Hooks → hookless + CI-as-authoritative-gate (hooks removed #511):
`knowledge-flywheel.md`, `troubleshooting.md`, `reference.md`,
`CI-CD.md`, `release-e2e-checklist.md`, `seed-definition.md`

## Catalog + cross-link cleanup (no broken links)

- `documentation-index.md` — removed deleted-doc rows + the v4 section
- `contracts/index.md` — removed deleted-contract cards
- `rfcs/0002-jobspec-openapi-v0.md` marked superseded; `SCHEMAS.md`,
`templates/README.md` cross-links fixed

## Created

- `docs/MIGRATION-3.0.md` (from the approved draft) + catalog row +
mkdocs nav entry

## Gates (green for this PR's surfaces)

- Link check: **2723 links, 0 broken**
- Contracts structural floor: **PASS (45 contracts)**
- Contract compatibility (orphans): **PASS**
- markdownlint (all changed/new files): **clean**
- `heal.sh --strict`: **All clean**
- mkdocs-strict: my new doc + nav entry build cleanly. The strict build
is **red on main for pre-existing reasons outside this scope** —
`docs/_hooks/gen_cli_reference.py` FATALs on missing `cli/docs/HOOKS.md`
(hooks removed #511, generator not updated; release-mechanics domain)
and `docs/3.0.md:172` links `../PRODUCT.md`/`../GOALS.md` (root files
outside the docs tree). Both confirmed present on `origin/main`; filed
for wave 3.

## Scope notes

- Did **not** touch `CHANGELOG.md`, `cli/`, or version files (parallel
worker owns release mechanics).
- The audit grep still matches ~40 more out-of-scope `docs/` files
(contracts/*, code-map, comparisons, narrative pages, runbooks). Filed
as **wave 3 (ag-p7j)**.

Closes-scenario: ag-qbw#doc-sweep-2
Bounded-context: BC1-Corpus
Evidence: docs/3.0.md
2026-05-24 18:27:24 -04:00

11 KiB
Raw Permalink Blame History

The Knowledge Flywheel

Agents are stateless. The repo learns.

The Problem

Coding agents forget everything between sessions. Notes alone do not fix that. If a solved problem is not extracted, curated, retrieved, and reused, the repo keeps paying for the same lesson.

AgentOps frames this as two of the three gaps in the Context Lifecycle Contract:

  • Durable learning (Gap 2) — solved problems recur because knowledge is not extracted, scored, and surfaced.
  • Loop closure (Gap 3) — completed work does not produce better next work because learnings are not harvested, promoted, or fed back into future sessions.

The flywheel is the mechanism that closes both gaps. Each stage below maps to one or both.

The Solution

AgentOps turns session output into durable environment state. AgentOps 3.0 ships zero hooks — the flywheel runs through explicit lifecycle commands, and CI is the authoritative gate. The same start/closeout stages work on every runtime without depending on hook side effects; if you want a bounded gate of your own, author it with the hooks-authoring skill (AgentOps does not ship one).

Runtime Modes

Mode Start path Closeout path What runs the stages
Any runtime (hookless default) ao inject / ao codex start / ao rpi phased ao forge transcript + ao flywheel close-loop (or ao codex stop) Startup context assembly, transcript discovery, citation capture, and close-loop status through explicit commands — portable across Claude, Codex, and OpenCode
Self-authored gate (optional) A hook you write with the hooks-authoring skill A hook you write with the hooks-authoring skill Only what you choose to wire; AgentOps ships no hooks, so nothing fires unless you author it

The Flywheel

┌───────────────────────────────────────────────────────────────────────┐
│                     THE KNOWLEDGE FLYWHEEL                            │
│                                                                       │
│  ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐       │
│  │  1. WORK   │─>│  2. FORGE  │─>│  3. POOL   │─>│ 4. PROMOTE │       │
│  │  Session   │  │  Extract   │  │  Score &   │  │  Graduate  │       │
│  │            │  │            │  │  Queue     │  │            │       │
│  └────────────┘  └────────────┘  └────────────┘  └────────────┘       │
│       ^                                                  │            │
│       │         ┌────────────┐  ┌────────────┐           │            │
│       └─────────│  6. INJECT │<─│5. LEARNINGS│<──────────┘            │
│                 │  Surface   │  │  Permanent │                        │
│                 │  & Cite    │  │  Knowledge │                        │
│                 └────────────┘  └────────────┘                        │
│                                                                       │
│  Each citation feeds back: utility scores update, high-utility        │
│  knowledge surfaces more often, low-utility decays. This is the       │
│  compounding effect — sessions get smarter because the best           │
│  knowledge rises and the noise sinks.                                 │
└───────────────────────────────────────────────────────────────────────┘

The Six Stages

Each stage maps to the gaps it closes: L = Durable Learning (Gap 2), C = Loop Closure (Gap 3).

Stage 1: Work (source material)

You build, debug, research, or plan. In hook-capable runtimes, transcripts are typically available directly from the runtime. In Codex, AgentOps prefers archived session transcripts and can fall back to ~/.codex/history.jsonl when no archived transcript exists.

Stage 2: Forge — L (extraction)

At closeout, ao forge transcript or ao codex stop parses the transcript and extracts structured knowledge — decisions, solutions, learnings, failures, and references. Each becomes a markdown file in .agents/knowledge/pending/. In hook-capable runtimes, the SessionEnd hook (session-end-maintenance.sh) triggers this automatically; the compile-session-defrag.sh hook runs deduplication and defrag in the same event.

Stage 3: Pool — L (curation)

ao flywheel close-loop ingests pending files and scores each on five dimensions:

Dimension What it measures
Specificity Names concrete files, functions, error messages
Actionability A future session can act on this without more context
Novelty New knowledge, not repetition
Context Explains WHY, not just WHAT
Confidence How certain the extraction is

Candidates are tiered: Gold (>0.85), Silver (0.70–0.85), Bronze (0.50–0.70), or Discard (<0.50).

Stage 4: Promote — L + C (graduation)

Candidates that pass the promotion gate graduate to permanent knowledge:

  • Age gate: Must be >24h old (prevents promoting noise from the current session)
  • Citation gate: Must have been cited at least once (proves another session found it useful)
  • Tier gate: Gold and Silver auto-promote. Bronze requires 3+ citations.

This is where durable learning and loop closure intersect: only knowledge that a later session actually cited gets promoted, proving the loop closed at least once.

Stage 5: Learnings — L (permanent store)

Promoted knowledge lives in .agents/learnings/ and .agents/patterns/. The maturity lifecycle:

provisional → established → archived

AgentOps maturity controls (ao maturity --expire, ao maturity --evict, ao dedup, ao contradict) prevent the corpus from decaying into stale noise. Maintenance runs through the active lifecycle path: hook-capable runtimes run it from hooks, while Codex runs the same hygiene from ao codex start / ao codex stop.

Stage 6: Inject — C (retrieval closes the loop)

At session start and during work, ao inject, ao lookup, or ao codex start retrieves the most relevant learnings for the current task. Startup retrieval prefers task-scoped context such as handoff goals and active beads instead of generic commit-subject fallbacks. ao lookup records citations automatically. When ao search results are actually adopted, use ao search --cite retrieved|reference|applied to record that decision in-band instead of relying on tribal workflow knowledge. Each citation is the signal that drives the feedback loop.

Citations with positive feedback increase the learning's utility score → higher utility → ranked higher in next injection → cited more → utility increases more → compounding. This is the loop closure mechanism: completed work produces better next work because the flywheel feeds validated knowledge back into future sessions.

The Compounding Math

The flywheel equation:

dK/dt = I(t) - δ·K + σ·ρ·K
  • σ — Retrieval coverage: unique surfaced artifacts / total retrievable artifacts, scale 0.0–1.0
  • ρ — Decision influence rate: unique surfaced artifacts later evidenced by reference or applied citations / surfaced artifacts, scale 0.0–1.0
  • δ — Knowledge age: average age of active learnings in days. The theoretical decay rate (0.17/week from Darr 1995) motivates the metric, but the CLI implementation (metrics_health.go) measures delta as days, not a weekly rate.
  • Escape velocity: When σ × ρ > δ/100, knowledge compounds faster than it ages out. The /100 normalizes delta (days) to a ratio comparable with sigma and rho.

Golden Signals

Escape velocity is necessary, but not sufficient. Four golden signals measure whether the flywheel is actually compounding:

ao flywheel status
Signal Question Healthy
Velocity Trend Is σρ-δ improving over time? Positive slope
Citation Pipeline Are citations delivering value? >60% high-utility
Research Closure Is research being mined into learnings? <10% orphaned
Reuse Concentration Is the whole pool active or just a few items? Gini < 0.4

Knowledge Stores

Store Content Updated By
.agents/knowledge/pending/ Forge output awaiting pool ingestion ao forge, ao codex stop
.agents/knowledge/pending/.quarantine/ Low-quality or unsafe pending extracts held out of promotion Pool hygiene, promotion gates, and close-loop maintenance
.agents/pool/ Scored candidates awaiting promotion ao flywheel close-loop
.agents/learnings/ Promoted, permanent knowledge Pool promotion pipeline
.agents/patterns/ Promoted decision patterns Pool promotion pipeline
.agents/research/ Scoped investigations /research
.agents/findings/registry.jsonl Reusable findings /pre-mortem, /vibe, /post-mortem
.agents/ao/citations.jsonl Citation trail ao inject, ao lookup, ao search --cite, ao codex start
.agents/ao/feedback.jsonl Utility feedback ao flywheel close-loop
.agents/ao/metrics/ Baseline snapshots for trend tracking ao metrics baseline
.agents/ao/codex/startup-context.md Explicit startup context assembled for hookless Codex sessions ao codex start
.agents/ao/codex/state.json Last Codex start/stop lifecycle state ao codex start, ao codex stop

The Compounding Effect

Gap Without the flywheel With AgentOps flywheel
Durable learning The same bug is rediscovered each session ao lookup retrieves the prior failure before planning starts
Durable learning Notes accumulate without pressure AgentOps promotion gates ensure only cited, high-quality knowledge survives
Durable learning Stale knowledge pollutes retrieval ao maturity, ao dedup, and ao contradict keep the corpus current
Loop closure Handoffs rely on chat memory AgentOps stores handoffs and phased state on disk in .agents/
Loop closure Session 50 starts from scratch Session 50 starts with 50 sessions of flywheel-promoted wisdom
Loop closure Completed work teaches nothing /post-mortem + finding compiler + ao-flywheel-close.sh harvest and compile learnings automatically

See Also