Files
boshu2__agentops/docs/adr/ADR-0017-loop-as-control-flow-not-knowledge.md
Bo 3213afcf1c Default to native execution and report independently accepted work (#1129)
## Change

Make native coding-agent execution the default AgentOps entry path with
zero mandatory skills. Preserve full bundles and add repeatable `ao
skills link --skill NAME` selection, validating the entire selection
before writes. Align product, installation, architecture and generated
command documentation.

Extend the existing trial readout to separate endpoint test results,
execution state and independently accepted work. Bind supplied judgments
to exact content, acceptance and native evidence. Reject empty
implementation subjects and require the caller's complete criterion ID
set before reporting acceptance. Preserve genuine nonempty and
deletion-only subjects, valid failures and missing-proof outcomes.

## Validation

- Native onboarding from empty home/consumer directories produces no
setup files; selective/full linking and failure boundaries are covered.
- Actual RED/GREEN regressions cover empty subjects and the
partial-criterion omission found by independent review.
- Full Go build, vet and race/shuffle tests; affected Go lint; 88 Python
readout/statistics tests passed.
- All 73 gates, generated projections, strict documentation build and
local aggregate passed (10 passed; one documented optional absence).
- All nine PR checks succeeded at
`7df0d42b12f35ffc22008cc10a40339afcfbb6a0`.
- Fresh author-distinct review passed all six acceptance criteria over
all 59 changed paths, with no findings or unchecked scope, after
repairing the criterion-coverage finding.

## Evidence limits

The real native coding repair demonstrates usability, not comparative
skill uplift. The strict live-session machine replay remains NOT_PROVEN
where execution/identity observations are unavailable; the source review
PASS is retained separately. Existing cohort limits and the historical
aggregate-enforcement gap remain unwaived. No new comparative cohort,
scheduler, skill-corpus deletion, memory migration or global
installation is included.
2026-09-10 20:28:22 +00:00

16 KiB
Raw Permalink Blame History

ADR-0017: The Loop Is Control Flow, Not Knowledge

  • Status: Accepted, amended for native execution 2026-09-10 (2026-09-03)
  • Author: AgentOps maintainers
  • Builds on: ADR-0004 (corpus moat unproven, position on the verification system), ADR-0011 (escape-corpus compounding demoted to hypothesis)
  • Origin: docs/plans/2026-09-03-loop-restore.md (this decision's intent source), and the 2026-09-02 Train 1 run, where the repair loop had to be improvised by hand

Active native-default amendment — 2026-09-10

The caller selected native agent execution with zero mandatory AgentOps skills. This supersedes RPI as the default entrypoint, not the exact-content judgment bar. A native goal carries continuity, BD carries work and handoffs, and the model chooses useful actions within accepted scope. AO provides deterministic tools on demand. A fresh author-distinct reviewer judges the final change; invoking the Validate skill is optional.

Keep the existing skill library and explicit full installations. Selective linking makes individual guidance available without requiring a bundle. RPI remains an explicitly selected workflow; its dependency graph is unchanged. No new scheduler, aggregate controller, knowledge platform or lifecycle command is authorized. Native limits must be real runtime controls; goal prose cannot claim enforcement. Historical missing-proof outcomes remain unwaived.

This amendment responds to observed process overhead and the caller's decision to test direct execution. A native coding demonstration establishes usability, not comparative skill uplift. Evaluation must distinguish endpoint checks from independently accepted work, retain failures and missing proof, and make association and evidence-verification limits visible. Subsequent comparative evidence may justify selectively adding or removing guidance.

Lean-harness amendment — 2026-09-09

The caller approved a short RPI charter with on-demand Plan, Implement, Validate and Memory. This supersedes the once-only phase lock, mandatory anti-ceremony/premortem dispatch, mandatory entry Recall, Learn's verdict-only input/scratch-TTL limit, and a default two-round native stop. Those earlier prescriptions below are historical; the pure fixed-dispatch reference retains its narrower contract only for explicit adapter callers.

Own the authorized outcome through finish. Plan may revise an approach when evidence disproves an assumption under unchanged accepted outcome and scope; acceptance changes need caller authority. Implement fixes ordinary known defects directly. Use cheap checks, reserve finishing capacity, respect real native/caller bounds, and obtain fresh author-distinct final validation. Same-family remains default, cross-model opt-in, with no fixed ten-minute cap. Exact subject, all acceptance, empty not_checked and prior findings remain required.

Unknown cause, recurrence, no progress or wrong objective warrants causal examination. A genuine stall gets at most one authorized bounded fresh helper per incident inside remaining bounds. An unhelpful answer ends the attempt; known failures need direct repair, never a helper chain. Cancellation, refusal and spent real time/cost/quota skip help. No new invocation or compaction resets limits. Optional outer-goal guidance remains outside the core; no scheduler, evolve root, new AO command or process ledger is restored.

Memory is an optional skill root with recall, mine/learn and curate/qualify/retire references. Use reviewed caller-selected external topic pages and update existing pages. Mining is separately budgeted off-path. Single incidents support narrow observations; stronger rules need stronger support, and benefit needs later work. Learning may remove rules; no-change and rare useful constraints remain valid. ADR-0016 still protects legacy evidence, external staging and review before Git. The lean trials use public/already-cleared inputs only and claim no native restricted-source enforcement. No worksheet is owed for trivial work.

The observed failure motivating this change was process work and repeated judgment displacing implementation and finishing. The guardrail is the smallest acceptance-advancing action and bounded causal help. The counterexample is a falsified implementation assumption: revising that approach is useful work, not a forbidden return to planning. This amendment establishes behavior contracts; only subsequent task evidence can establish benefit.

Historical amendments retained for provenance

Prior disposition — 2026-09-06 selected CDLC

Retain the bounded repair law, once-only Plan/Implement, exact subject and fresh author-distinct Validate. Replace the blanket exclusion of knowledge writes and outer re-planning with selected Context Delivery Lifecycle contracts: maintained external context/environment around disposable agents, no weight training and no deterministic-inference promise. Discovery (Plan shapes), Implement and Validate are the delivery phases. An explicitly selected bounded outer goal may authorize a new experiment within its accepted envelope; RPI itself neither replans nor owns native budgets, stops, queues or delivery.

Replace Decision 3's NTM-only and blanket headless-Claude ban with the single authorized bounded model-dispatch recipe. Both fresh and required cross-family exact-subject legs remain, with independent initial inputs and actual model/context receipts. Caller-required unavailable diversity stays diversity_unsatisfied / NOT_PROVEN; no preferred judge, majority vote or author can supply binding PASS. Native runtimes own finite input/output, timeout and verified cleanup. Exit/process facts are not semantic verdicts. Executable Door9 and specialist provider guards stay intact.

The selected new roots are Recall first and a small evolve only after native stop/continuation proof. Recall and broader Learn source/curation behavior have later owners; T25 alone may restore evolve and amend REMOVED_SKILLS. Thus "What stays removed" below describes the current executable inventory, not a permanent prohibition on the selected later learning contract. No Recall or evolve entrypoint is advertised as runnable by T04; existing entrypoint checks and dependency invariants remain enforced. Selected recall is instruction-level composition, not a new hard edge or Discovery root. The old evolve implementation, operator-only unlimited rules, CLI/controller and packet machinery stay retired.

ADR-0016 admits caller-owned external reviewed Markdown/OKF memory and protected non-Git drafts/evidence. This replaces scratch-only curation without restoring a transcript lake or a second work store. Factual support, destination disclosure before any Git object/index/stash, and later utility are separate decisions. Source authors cannot approve their own knowledge, and learning cannot change completed product verdicts.

ADR-0004/0011's honesty remains: measured mechanism and net benefit are distinct. The current cold-reuse trial requested no optional knowledge body and showed no memory benefit. Trial acceptance preserves its failed worker and null result; it cannot imply improvement. The selected skills-plus-Go evidence path needs later implementation and shared conformance, not a shipped Python expansion. The original removal rationale and repair evidence below remain historical.

Prior stopping amendment — 2026-09-07

Replace finding-count monotonicity and digest movement as repair admission proxies with acceptance evidence: a fresh digest-bound receipt proves closure of a named acceptance finding or proof gap. New ids need causal evidence of pre-existing discovery; introduced regression or unknown cause stops repair. Reopened ids and recurring closed classes warrant causal HOLD, never an automatic design-failure diagnosis. The existing pure reference consumes those receipt facts without creating a runtime, persisted schema, or budget account.

Informative red can justify a different experiment in an explicitly selected bounded outer goal under unchanged acceptance. Its causal HOLD gets exactly one bounded fresh helper per incident inside the remaining allowance; unhelpful advice stops implementation. Cancellation, explicit refusal/judgment, or a spent hard time/cost/quota skips the helper. New subjects, compaction, and repeated continuations never reset allowance or helper use. RPI itself stops and reports.

Size evidence inspection by effect on acceptance/enforcement, as owned by Validate. The 2026-09-08 amendment below replaces this amendment's automatic cross-family requirement. Exact subject, all acceptance, and empty not_checked remain necessary for binding PASS. Reuse applicable exact-input receipts and fast discriminating checks; no additional progress ledger is owed.

Native objective text demonstrates neither pause nor aggregate allowance operations. Report observed controls and missing measurement truthfully. Evidence and provenance are retained while beliefs may be revised; knowledge is not monotonically true or useful. The historical rationale below is preserved.

Prior review-selection amendment — 2026-09-08

At the caller's request, default to one fresh author-distinct validator in the author's model family: Codex/OpenAI for Codex/OpenAI work, Claude/Anthropic for Claude/Anthropic work. Risk determines evidence depth, not automatic model multiplicity. Validate and RPI accept --cross-model [model] or an explicit cross-model request to add a different-family judge; a supplied model pins it. This supersedes the earlier risk-triggered family rule, including Decision 3. An independently explicit caller requirement stays required until the caller changes it. Historical findings and missing reviews remain recorded.

Remove the fixed ten-minute invocation default. Native runtime bounds come from an explicit caller timeout or the remaining caller deadline, taking the earlier when both exist. Headless calls retain finite bounds, output limits, cancellation and verified cleanup; retries never renew the allowance. A timeout is an incomplete review, not a semantic objection. The observed motivation was repeated reviews expiring before returning judgment. This amendment changes selection and delivery behavior; it does not claim measured model superiority.

Historical context

The 2026-07-14 cut (482307762, 2,433 files, −416K lines) removed the compounding-knowledge machinery and the iterate loop in the same pass. It took out converge, crank, discovery, evolve, and the learn write-half together, as one class.

They were not one class. ADR-0004 and ADR-0011 demoted a knowledge claim: that an escape corpus accrues and compounds into an advantage. Neither ADR said anything about control flow. Converge was not a knowledge store: it was the criterion for when repair stops. Crank was not a corpus: it was a wave executor. The cut removed them because they sat next to the unproven claim, not because anything demoted them.

What that costs showed up on 2026-09-02. The Train 1 run needed repair and re-validation, and the contract had no repair phase, so the loop was improvised by hand: 8 validators and 2 stops, with the stopping rule living in an orchestrator's judgment instead of in the contract. An improvised loop is not reproducible and cannot be judged.

Decision

Restore the control flow the cut over-reached on, and only that.

  1. Converge's criterion returns as RPI's repair phase, bounded by the convergence law from the plan. A repair round is admitted only while all hold:

    1. rounds_used < repair_rounds (caller-declared, default 2).
    2. New digest-bound evidence proves closure of a named acceptance finding or proof gap. Changed bytes or a reduced finding count alone do not count.
    3. No closed id reopens, closed class recurs, or introduced regression is admitted. A new finding needs causal evidence of prior existence under the same acceptance; unknown cause stops repair. The open set remains the union of required judges' stable ids, including necessary discoveries. The precise current law and receipt bindings live in skills/rpi/SKILL.md.

    Converged means the fresh validator returns PASS and, when the caller selects cross-model review, that validator also returns PASS. On any violation RPI stops and reports the current status. No third judge, no escalation, no auto-replan.

  2. Crank's shape returns as a thin wave executor. One invocation executes one caller-selected wave, runs the wave's acceptance once, returns wave evidence, and stops. It owns no wave selection, retry, budget, queue, claim, lease, Git, closure, or next work.

  3. Fresh same-family validation is the default, as owned by skills/validate/SKILL.md. Cross-family validation is caller-selected under the active 2026-09-08 amendment. Dispatch uses the authorized bounded model-dispatch recipe; required unavailable diversity remains diversity_unsatisfied / NOT_PROVEN. Same-family agreement cannot satisfy an explicitly required second family.

Conformance assertions flipped

This ADR is the sole authority for flipping the assertions below. They encode the cut's over-reach, not a demoted claim, and they are flipped by lane L-A of the plan:

  • scripts/check-cathedral-cut-conformance.py: crank listed as removed; RPI required to contain "Stop regardless"; loops in run_once.py rejected.
  • workflows/rpi.js: single-pass traversal.
  • skills/rpi/scripts/validate.sh: single-pass assertion.
  • skills/rpi/tests/test_run_once.py: stop-on-FAIL.
  • evals/agentops-core/rpi-behavior.json: single-pass behavior expectation.

What stays unproven

  • Compounding stays unproven. ADR-0004 and ADR-0011 remain in force; nothing here reopens a corpus, a knowledge store, or an accrual claim.
  • The loop's own effect on outcomes is unproven. Restoring a repair phase is a contract change, not evidence that repair produces better subjects. That is owed a seeded-defect probe, and until one runs, the loop is justified by the improvisation it replaces, not by measured lift.

What stays removed

  • ao converge and ao crank as root commands.
  • evolve, and the ADR-0007 operator-stop rules.
  • The learn write-half.
  • Discovery's packet machinery.
  • Converge's Go command, its canary, and its findings registry; crank's flags, lifecycle tiers, and Sisyphus markers.

Two narrow authorizations ride on this decision: the skill mesh permits exactly one non-core hard dependency, crank on rpi (scripts/generate-skill-mesh.py), and the Claude conveyor's cross-family leg is a caller-supplied read-only command whose model family the script cannot verify and does not claim to.

Consequences

  • RPI can end in a repaired PASS instead of only a first-pass verdict, and the stopping rule is in the contract where a validator can check it, rather than in an orchestrator's judgment.
  • A repair round without new acceptance-relevant proof stops even when bytes or counts move. Newly discovered pre-existing defects do not become regressions merely because they increase the count; unknown cause still stops for examination. Control artifacts alone never justify repair.
  • Caller-selected cross-model review costs a second judge leg. When its legal adapter is unavailable, the combined result is NOT_PROVEN. The default fresh same-family review retains the full acceptance and evidence bar.
  • The removed surfaces stay removed. A future proposal to bring back a knowledge store or a lifecycle command does not inherit this ADR's permission; it needs its own, with evidence.