Addresses the biggest failure mode: the loop could overfit the self-supervised
proxy while human relevance is merely PRESERVED, not improved. Fix makes that
visible + falsifiable:
1. FROZEN PUBLIC HUMAN EVAL SET — .claude/eval/human-relevance-frozen-v1.json
(10 ADR-081 labelled queries), loaded via harness-frozen-eval.ts which PINS
the canonical content hash (sha256:6096e48e…) and THROWS on drift. Single
source of truth for the red/blue anchor; supersede with a new versioned file,
never edit. The daemon runtime now loads the anchor from it (was an inline
duplicate).
2. PER-GENERATION HUMAN-RELEVANCE DELTA — every receipt records
deltas.humanRelevance (candidate − baseline nDCG@3 on the frozen set) +
humanEvalHash. flywheelStatus exposes cumulativeBenchmarkDelta (proxy) vs
cumulativeHumanRelevanceDelta (human); the status script flags
'proxy up but human flat → OVERFITTING'. The overfitting mode is now
observable in the lineage instead of hidden.
3. CLEAN-ROOM REPLAY ACCEPTANCE TEST — scripts/replay-generation.mjs: from a
clean install, replay one PROMOTED generation from its receipt alone —
recompute every hash bit-identically + re-run accept/v1+sig to reproduce
promoted=true, with fetch trapped (offline). Wired into v3-ci alongside the
proof guard (which now also verifies the frozen-eval pinned hash). Verified:
real-generation-1 replays with identical hashes, offline.
Additive + backwards-compatible (new optional receipt fields; old committed
bundles still replay). 76 harness+config tests green; both CI guards pass.
Claude-Session: https://claude.ai/code/session_01S7GYqnVUVxBfZ5W8znqry3
rUv