Commit Graph

3 Commits

Author SHA1 Message Date
Jude Gao 3e57444699 Add run-all command with housekeeping (#41)
New run-all command auto-discovers experiments/*.ts and orchestrates
the full pipeline: fingerprint reuse, failure classification,
auto-retry of infra failures, and housekeeping.

Housekeeping runs after each experiment: keeps only the latest valid
result per (experiment, eval) pair, removes duplicates and incomplete
results, cleans up empty timestamp directories.

Supports --dry (preview), --force (ignore fingerprints), --smoke
(1 eval per experiment), and optional experiment name/glob arguments.
2026-02-08 00:21:05 -05:00
Jude Gao 59826e7b9a [Agents] Add AI SDK Harness for evaluating any AI Gateway model (#8) 2026-02-03 19:10:44 -05:00
Jude Gao 97b9cc5ce4 [Config] Move sandbox backend selection from env var to experiment config (#6) 2026-02-04 03:07:00 +08:00