Files
2026-08-04 12:09:27 -07:00
..

Implementation Plans

CAMPAIGN COMPLETE 2026-07-07: all 15 plans executed by dispatched agents and approved after review; all branches merged and battery-proven on advisor/integration @ d88f979 (worktree agent-a2c73990543bdce51): 456 PASS / 1 XFAIL / 0 FAIL over 490 rows, 25 gates, both runner modes equivalent, silhouette separation unchanged, ruff clean, composed conflict sites satisfy both parent plans' criteria. Merging advisor/integration → main is the maintainer's decision. Known post-merge follow-up: restore 011's full speed by giving the in-process stdin shim a .buffer (BytesIO in a TextIOWrapper) — 002's buffer-reads currently trigger 235 designed subprocess fallbacks (36s vs the ~10s target; correctness unaffected, equivalence verified).

Generated by the improve skill on 2026-07-06, against commit 217c218. Execute in the order below unless dependencies say otherwise. Each executor: read the plan fully before starting, honor its STOP conditions, and update your row when done.

Repo verification baseline (used by every plan): python3 evals/run_adversarial.py → exit 0, PASS 439 XFAIL 1 FAIL 0 (counts grow as plans add rows). Repo rule: eval rows land red-first (CLAUDE.md, "Add a New Pattern").

Execution order & status

Plan Title Priority Effort Depends on Status
001 Make the Python 3.8+ floor true, test it in CI P1 S — DONE — approved 2026-07-07; branch advisor/001-python-floor @ 91a99c4 (worktree agent-af5b97e83617cb586); 19 files, +40/−3
002 Harden input surfaces (bad bytes, unreadable files) P1 M — DONE — approved 2026-07-07; branch advisor/002-input-robustness @ 1cdfc1c; 6 ENC rows red-first, suite 445/1/0
003 Contribute slug validation + exclamation ReDoS fix P1 S — DONE — approved 2026-07-07; branch advisor/003-slug-and-redos @ 460339e; 32k probe 5.7s→0.0007s; REGX-01 deviation approved (in-process, guards live pattern)
004 Eval-suite schema gate + CI driven from gate matrix P1 S — DONE — approved 2026-07-07; branch advisor/004-suite-self-validation @ 52c581b; 25 gates; placeholder-skip deviation approved (convention-anchored)
005 Violation offsets carried once; linear containment P2 M 004 (soft) DONE — approved 2026-07-07; branch advisor/005-scan-offsets @ 060b3ba; 10k-sentence scan 11s→1.3s (reviewer-verified), CRLF/case-fold fixed, containment equivalence proven + fixture-diffed
006 Silhouette English-decline parity P2 S — DONE — approved 2026-07-07; branch advisor/006-silhouette-decline @ 561a684; suite 441/1/0, separation byte-identical
007 wiki_sync offline fixture coverage P2 M 004 (soft) DONE — approved 2026-07-07; branch worktree-agent-a3f0cd923d0e7c49f @ 6a8cb45; 3 WIKI rows, offline 0.057s, state untouched; merge after/with 001 (annotation coverage)
008 Unify drifted prose-view helpers (parameterized) P2 M — DONE — approved 2026-07-07; branch advisor/008-shared-prose-view @ c426065; byte-identity independently verified by reviewer
009 Standardize check-script import seam + document it P3 S not concurrent with 004/007 DONE — approved 2026-07-07; branch advisor/009-import-convention @ 3e0c547; AGENTS.md mirror deviation approved (DOC-06 forced); CHECKS.md prose scored for the first time and cleaned
010 DX polish: --help ×3, ruff baseline P3 S — DONE — approved 2026-07-07; branch advisor/010-dx-polish @ 7982f24; found real F601 dup-key bug (see findings)
011 In-process eval runner (26.6s → target <10s) P3 L 004 DONE — approved 2026-07-07; branch advisor/011-inprocess-runner; 30.0s→10.1s verified by reviewer, equivalence diff empty, --subprocess escape hatch kept
012 SPIKE: live mimic-fidelity eval (operator gate) P2 M–L — DONE (steps 1–2) — approved 2026-07-07; branch advisor/012-live-mimic-spike @ 1b9c1af; protocol + mock dry-run committed; STEP 3 AWAITS OPERATOR (live spend + local voice paths). Found 4 harness gaps — see Dependency notes
013 Adversarial-refresh procedure + DECISIONS.md P3 M 007 (soft) DONE — approved 2026-07-07; branch worktree-agent-a11122607e1f9d14d @ 4cd3ea0; all 9 decisions located and cited (zero unlocated); ran full behavioral tune per CLAUDE.md
014 Contributor on-ramp (CONTRIBUTING + templates) P3 S — DONE — approved 2026-07-07; branch worktree-agent-ae64085b11aba9fa4 @ 4a4bced; 5 files +133, scanners 0/0, links resolve
015 SPIKE: per-author silhouette stability P3 M — DONE — approved 2026-07-07; verdict NO-GO (independent-author discrimination at chance; degenerate IQRs; leakage caveat); report in worktree agent-ab171a4f32bc06c17 plans/015-report-silhouette-stability.md

Status values: TODO | IN PROGRESS | DONE | BLOCKED (with one-line reason) | REJECTED (with one-line rationale).

Dependency notes

  • 004 before 005/007/011: the schema gate validates rows those plans add; 011 additionally reshapes the runner 004's gate lives in.
  • 006 before or after 008: both touch silhouette_scan.py's import block; whichever lands second rebases trivially. Do not run concurrently.
  • 009 not concurrent with 004/007: same evals/ files.
  • 012 step 3 requires operator approval (live model spend + local voice profile paths) — hard stop built into the plan.
  • 005 and 003 both edit banned_phrase_scan.py (different regions: scan_for_violations vs one pattern string). Sequential execution recommended; either order works.
  • 015's "go" outcome feeds a future plan that would depend on 012's corpus protocol — read both reports together before scheduling it.

New findings surfaced during execution (candidate future plans)

From plan 012's mock rehearsal (documented in evals/LIVE-MIMIC-PROTOCOL.md §7 on its branch):

  1. Profile scoring discrepancy — in-memory vs JSON-round-tripped profiles can score the same candidate differently (dict-order tie-breaking in z_function_vector top-N; JSON uses sort_keys, in-memory doesn't). Real numerical bug; deserves an eval-first fix plan.
  2. Refine hard gates ⊂ documented battery — the loop never runs readability_metrics/diff_check; diff_check flags even accepted winners. Reconcile before trusting live gate-pass rates.
  3. Live path always injects the card (no true no-card baseline via --generate-cmd); baselines must be generated externally.
  4. Synthetic voice fixtures too small for the protocol's own corpus spec (5 docs vs ≥8 needed).

From plan 010's ruff baseline: "needless to say" is a DUPLICATE KEY in BANNED_PHRASES (scripts/banned_phrase_scan.py, F601) — the second literal silently overwrites the first, so one definition is dead. Micro-fix candidate: keep the intended entry, delete the shadowed one, confirm its FN/REC rows still pass.

From plan 015: per-author silhouette references are a NO-GO on current evidence (discrimination at chance for independent authors) — the silhouette_scan.py docstring's anticipated integration should not be built without new data.

Findings considered and rejected

So nobody re-audits these:

  • Per-suite gate double-execution (HARV/CONTRIB/CAL/MIMIC slices re-run inside dedicated gates): deliberate — granular pass/fail attribution in the gate battery. Cost ~12% of script cases; accepted.
  • Doc-system consolidation (packs / commands / deep references as three parity-gated systems): audited, judged not worth doing — the parity gates already prevent drift and the systems serve different consumers.
  • Parity-harness key handling: audited clean — keychain key reaches only the Authorization header; error paths never echo it.
  • run_mimic_refine --generate-cmd shell injection: audited clean — shlex.split to argv list, never shell=True.
  • "No unit tests" as a bare finding: the eval-suite-as-tests design is the product's deliberate architecture; only specific uncovered surfaces (see plans 002/007) were actionable.
  • Harvest memory footprint / whole-file reads (audit CORRECT-03): real but low-priority; revisit if harvest targets multi-GB archives. Streaming refactor sketch lives in the audit record.
  • CRLF/case-fold offset drift: NOT rejected — folded into plan 005 (same root cause as the quadratic).
  • Firing the sealed holdback split: a launch-day decision for the maintainer, not an engineering plan. One-shot by design; coordinate with the last pattern additions before any public announcement.

Audit scope note

Standard-effort audit, hotspot-weighted: scanners, voice stack, harvest/ contribute pipelines, eval runner + gates, docs, CI. NOT audited in depth: behavioral judge prompt quality, fixture prose quality, presets/ content, deep references' claims beyond spot checks, git history hygiene.