* Add TUnit-focused eval scenarios for dotnet-test-frameworks
Adds 5 new evaluation scenarios and tightens 2 existing scenarios to
exercise the expanded TUnit coverage in dotnet-test-frameworks/SKILL.md
(see #677).
New scenarios:
- Convert cross-framework assertions to TUnit syntax (await / IsEqualTo /
IsTrue / IsNull / IsAssignableTo / Contains / Throws<T>)
- Diagnose silently-passing TUnit test with missing await (debugging
mystery framed without naming the pitfall)
- Refactor TUnit try/catch to native exception assertion (Throws<T>()
/ ThrowsExactly<T>() / WithMessage)
- TUnit lifecycle hooks at test / class / assembly / session scope
- TUnit skip mechanisms — attribute, assembly-wide [assembly: Skip],
and dynamic Skip.Test(...)
Updated scenarios:
- Identify TUnit framework: rubric now distinguishes [ClassDataSource]
as a fixture/data source rather than a class marker (TUnit classes
are convention-based, like xUnit)
- Identify integration tests: adds a TUnit Project D using live
SqlConnection and requires recommending [Category("Integration")] for
TUnit
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Mirror TUnit eval scenarios into Vally spec
Apply the same updated and new scenarios as in eval.yaml to the parallel
eval.vally.yaml so both pipelines exercise the new TUnit-focused checks.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Amaury Levé <amauryleve@github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix dotnet-test-frameworks skill activation for evaluations
The skill had user-invocable: false and a description saying 'DO NOT USE
directly', which prevented the model from activating it during evaluation.
All 6 eval scenarios default to expect_activation: true, causing
SkillNotActivated failures on CI.
- Remove user-invocable: false from both dotnet-test and experimental copies
- Rewrite description with activation keywords matching eval prompts
(assertion mapping, framework identification, skip annotations, exception
assertions, lifecycle conversion, integration test markers)
- Preserve note that other analysis skills also load this as reference data
* Restore user-invocable: false, fix cross-refs, add expect_activation: false
Address review comment: remove exp-* skill references from dotnet-test-
frameworks description since those skills reference exp-dotnet-test-
frameworks instead.
Optimize eval results: this reference data skill's content is knowledge
the model already has (baseline scores 5.0/5 on most scenarios). Direct
activation adds token/time overhead without quality improvement:
- Restore user-invocable: false (correct for reference data skill)
- Shorten description to not trigger activation
- Add expect_activation: false to all 6 scenarios
The skill's value is realized when loaded by consumer skills (e.g.
test-anti-patterns), not via direct user queries.
Amaury Levé