2 Commits

Author	SHA1	Message	Date
Boden Fuller	81066b6a16	fix(eval): remove unused vars flagged by shellcheck ... Drop JSON_OUTPUT (parsed but unused) from harness, WORKBENCH (unused) from suite runner, and self-assignment in single-mode case.	2026-05-04 15:04:20 -04:00
Boden Fuller	e5defbb8e9	feat(eval): enhance agent harness with industry-proven eval patterns ... Borrows four patterns from the eval landscape (Anthropic pass@k, SkillsBench A/B, Aider retry-with-feedback, SWE-bench task prompts) and adapts them to our 12-task workbench: - Add task-specific prompt.md to all 12 tasks (lifts pass rate 50%→92%) - Add --runs N flag for pass@k / pass^k statistical tracking - Add --compare flag for with-skill vs without-skill A/B testing - Add --retry flag for two-attempt with test failure feedback - Add --generic-prompt flag for prompt sensitivity measurement - New eval-agent-suite.sh orchestrator with mode selection - Expand agent suite JSON from 3 cases to all 12 - Add prompt.md and agent suite coverage checks to CI gate	2026-05-04 14:31:48 -04:00