* Add grade-tests skill for per-test quality grading
Adds a new polyglot skill that grades a provided list of test methods on a
letter scale (A-F) with a score band and 1-line notes, suitable for posting
in a PR comment table. Complements existing audit skills (assertion-quality,
test-anti-patterns, test-smell-detection) which produce suite-wide reports.
The skill:
- Takes an explicit list of tests (refuses to grade a whole workspace silently)
- Uses a 3-dimension rubric (Assertion strength, Structure & focus, Anti-pattern hygiene)
- Combines via 0.45/0.30/0.25 weighting with safety caps
- Starts at A and only deducts for observable issues
- Does not flag idiomatic patterns (table-driven sub-tests, bare assert, etc.)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback
- OrderProcessorTests.cs: fix 4 invocations that mixed named arg
customerId: with subsequent positional args -- invalid C# syntax
(Copilot review on PR #727)
- README.md: update 'five test-analysis skills' to 'six' in three
places now that grade-tests is added to the polyglot list, and
clarify that only the original 5 load from test-analysis-extensions
while grade-tests has an inline rubric
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* eval: drop 'grade-tests skill' wording from scenario prompts
The skill-validator's evaluate pre-flight check ValidateEvalPrompts
rejects any prompt that names the target skill, because mentioning the
skill biases baseline (skill-not-loaded) runs and inflates measured
lift. The two scenarios that asked the agent to 'use the grade-tests
skill' now just ask it to grade test methods individually -- the agent
is still expected to pick the right skill organically.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix rubric consistency per review feedback
- README: drop incorrect 'inline rubric instead' claim (grade-tests also
loads test-analysis-extensions for language-specific guidance)
- Assertion sub-grade: literal always-true assertions are now F (was D),
matching the eval expectation that 'assert True' produces overall F
- Anti-pattern catalog: split always-true literals (now -> F) from
self-referential assertions (still -> D) — they were conflated
- Combination rule: overall grade is now capped at the worst sub-grade
(was capped at D), so an F in any one dimension yields overall F
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Amaury Levé