Profa fe9b4d9a19 feat(eval-runner): implement P1 grading improvements — negative control, finding count, rubric scoring ...

Implement three P1 items from skill-evaluation-research.md that deepen
eval grading in the runner's validateOutput method:

- P1-6: Negative control test cases (inverted pass logic for "skill
  should decline" tests)
- P1-5: Finding count enforcement (JSON parse + regex fallback),
  schema_path validation, and weighted grading rubric scoring
  (completeness/accuracy/actionability)
- P1-4: Adaptive rubric that extracts keywords from prompts (quoted
  strings, format words, named standards) as dynamic must_contain checks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

2026-02-18 10:17:14 +00:00

42 KiB

Raw Permalink Blame History

View Raw