mirror of
https://github.com/theclaymethod/unslop.git
synced 2026-09-19 01:45:09 +08:00
Clayton Kim
ad33b1cb97
FINDING: the cheapest model that recovers macro structure under the climb is claude-haiku-4-5 -- it converges in 5 rounds given --max-rounds 8 (was capped-at-1-residual at cap 4; one round short, not a hard ceiling). On the GPT side only the frontier (gpt-5.5) converges; gpt-5.4-mini oscillates (trajectories 4->1->5->2 at cap 4 and 3->3->2->1->6->2->1->5 at cap 8) and never reaches clean. "Cheap tiers partially recover" was model-dependent, not a property of cheap: read the two families separately. Codex CLI as the GPT-side generator (OpenRouter still 402-blocked): evals/model_generate.py gains a `codex` kind. codex exec is an agent CLI whose stdout is an interleaved transcript, so call_codex extracts ONLY the final message via -o/--output-last-message (verified clean by hand), runs read-only/ephemeral, and wraps the call in its own process group with a hard timeout that SIGKILLs the group -- the known silent-hang failure mode becomes an honest failed round. CLIMB-07 gates the extraction and the hang kill offline against a fake codex binary (evals/fixtures/climb/fake_codex.py). All four live runs (gpt-5.5 cap 4 converged 4->1->1->0; gpt-5.4-mini caps 4 and 8 capped; haiku-4-5 cap 8 converged 1->1->1->1->0) preserved every source fact every round, on the same SKILL-MACRO-01 fixture and settings as the recorded matrix. pipeline.md's climb section gains the follow-up table and the cheapest-viable doctrine; the tiering row and PRODUCT.md bullet updated to the model-dependent reading. Gates: adversarial 469 pass / 1 documented xfail (CLIMB-07 new), schema, gates-doc, taboo parity, build --check, strict-leakage all green; 0 scanner hits (hard or soft) on both edited docs. Behavioral tune: 33/34 tasks ran; the one failure was a transient API server error mid-response on SKILL-MACRO-01/with_skill (runner infra, not a model answer), which aborted the non-blocking judge step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01K6CYksdLbXbTAxcAQjvHz5