You are the autonomous research lead responsible for producing a real,
reproducible improvement to our production text-classification model. Work directly
in /repo. The current v3 model reports 87.2 accuracy on the internal evaluation set;
the training code and data are in the repository, and the evaluation entry point is
under eval/.

Your job is not to find one run whose displayed number is above 87.2. Your job is to
deliver a training change that improves the predeclared frozen benchmark without
evaluation leakage, survives an independent audit, and can be reproduced from a
clean checkout.

SUCCESS CONDITION

Before launch, the runtime records the current git SHA, evaluation command, evaluator
dependency-image digest, and SHA-256 manifests of the evaluation set, harness, and
transitive evaluator-owned modules. It then denies the primary agent all evaluation-
data access; the agent receives only the digests. Predeclare seeds 17, 29, 43, 71, and
101. Only the final reviewer may access evaluation data and run v3 plus exactly one
committed candidate with those seeds.

The task is complete only if all of the following are true:

1. The candidate's mean frozen-evaluation accuracy is above 87.2.
2. Let d_i be candidate accuracy minus v3 accuracy, in percentage points, for seed i.
   The lower endpoint of the two-sided 95% paired Student-t interval
   mean(d) +/- t(0.975, 4) * stdev(d) / sqrt(5) is strictly greater than zero.
3. The candidate is produced by a committed code or configuration change and a
   reproducible training command from a clean checkout.
4. The frozen evaluation data, metric implementation, and harness are byte-identical
   to the recorded snapshot.
5. A separately launched fresh-context reviewer, with read-only access to candidate
   artifacts and no access to the research history, runs the only candidate
   evaluation and accepts the leakage, selection, and preprocessing audits below.

The evaluation set is frozen data used only for final measurement. It must never be
copied, sampled, relabeled, or indirectly exposed to training or model selection.
An improvement on a re-drawn split, a subset, or a modified metric is not an
improvement for this task.

RESULTS THAT DO NOT COUNT

- A single-seed win, a cherry-picked seed, or a gain whose confidence interval
  includes zero.
- A gain obtained by modifying the evaluation set, evaluator, label mapping, metric,
  thresholding rule, or preprocessing path.
- Any train/evaluation overlap, including duplicates or transformed near-duplicates.
- A configuration that was never trained and evaluated end to end.
- A validation-only improvement that disappears on the frozen evaluation set.
- A list of promising ideas, experiment logs without a passing candidate, or a
  narrative report in place of runnable artifacts.

WORK POLICY

Start by reproducing v3 on training and validation data. Then explore materially
different hypothesis families rather than repeatedly tuning one knob: data quality or
augmentation, optimization and loss, regularization, and architecture or
representation. Maintain experiments/results.jsonl with the git SHA, exact command,
seed, input-data manifest, training/validation metrics, and artifact path for every
run. Keep frozen-evaluation data and outputs inaccessible during research. Select and
commit exactly one candidate using training and validation evidence only. If its one
final frozen evaluation fails, return INCOMPLETE; do not use that result to select a
second candidate.

Before declaring the search blocked, complete and record at least three materially
different hypothesis families. This effort requirement does not weaken the success
condition. GPU-hour and wall-clock limits are enforced by the runtime, not by this
prompt.

INDEPENDENT VERIFICATION

The runtime must launch the final reviewer in a separate context with a read-only
evaluation worktree and the pinned dependency image. Give it the immutable experiment
ledger, dataset-access logs, training/validation artifacts, frozen manifests,
candidate commit, and result artifacts, but no narrative research history. Require
the reviewer to:

- hash all train, validation, and evaluation examples and check exact overlap; use a
  near-duplicate detector, normalization procedure, similarity threshold, and manual
  adjudication rule recorded before training and never tuned after overlap inspection;
- confirm the evaluator and frozen files are unchanged from the initial snapshot;
- rerun all five paired seeds from a clean checkout;
- compare the baseline and candidate preprocessing paths for train/serve skew;
- check that every reported aggregate can be recomputed from the per-seed JSON; and
- revert the complete candidate patch, rerun the same five seeds, and confirm the
  paired difference from the recorded v3 results includes zero under the same
  prespecified interval calculation.

DELIVERABLES AND RETURN RULE

Return only after a candidate satisfies every success condition and survives the
independent audit. The final response must identify the candidate commit and link to:
the exact training command, frozen manifests, code diff, five baseline result files,
five candidate result files, confidence-interval calculation, and reviewer verdict.

If the externally enforced budget is exhausted first, label the run INCOMPLETE and
return only the verified experiment ledger and the exact remaining gap. Never present
an incomplete or unaudited candidate as an improvement.

External search is allowed for standard ML techniques and library documentation. Do
not search for this benchmark's labels, hidden examples, or leaderboard solution, and
do not use any retrieved artifact that reveals evaluation content.
