* Update target framework to net10.0 and upgrade TUnit package to version 1.45.8
* Fix and expand TUnit coverage in dotnet-test-frameworks reference
Cross-referenced against tunit.dev/llms.txt and the official attributes
comparison page. Corrects the test class marker (TUnit is convention-based,
not [ClassDataSource]), and fills in the previously-missing TUnit rows for
the assertion APIs, setup/teardown methods, exception-handling examples,
and integration test category marker.
* Refine TUnit assertion table accuracy
The Type row's TUnit cell now uses IsAssignableTo<T>() to match the
assignable-type semantics of the MSTest/xUnit/NUnit equivalents, with a
note pointing at IsTypeOf<T>() for exact-type checks. The Skip row now
documents that [Skip] also applies at class and assembly scope and that
Skip.Test("reason") handles dynamic in-test skipping.
Verified against the TUnit migration guides and the type-assertions docs.
* Spell out TUnit assertion alternatives as complete awaited expressions
The Boolean, Null, Exception, and Type rows in the assertions table
previously used a `await Assert.That(x).IsTrue()` / `.IsFalse()` shorthand
for the alternative form. A reader copy-pasting just the second fragment
would end up with an un-awaited assertion that silently passes — which
contradicts the note immediately below the table about always awaiting.
Each alternative is now a complete, independently copy-pasteable expression.
Addresses the inline review on dotnet/skills#677.
---------
Co-authored-by: Amaury Levé <amauryleve@microsoft.com>
* run-tests: use prompt-shaped multi-TFM keywords for plugin activation
The 'Run tests in a multi-TFM project targeting a specific framework' eval
scenario kept failing plugin-mode activation after #667. The skill description
had 'multi-TFM projects (--framework)' as a brief parenthetical, but the
scenario prompt uses different language: 'This test project targets both
net8.0 and net9.0. I only want to run the tests on net9.0 using dotnet test.'
Per the InvestigatingResults.md guidance for 'skill not activated' failures,
expand that clause to include keywords from the prompt — 'running tests
against a single target framework when a project targets multiple TFMs'
plus a concrete '<TargetFrameworks>net8.0;net9.0</TargetFrameworks>' example
and the '--framework <TFM>' invocation. Per-skill description is 947 chars,
well under the 1024 spec cap.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Replace simple multi-TFM scenario with a harder MTP+SDK 9 variant that requires skill knowledge
The previous 'Run tests in a multi-TFM project targeting a specific
framework' scenario tested a trivial prompt (single --framework flag)
that the model already answers perfectly from training (baseline 4.7/5).
This matches pattern #8 (baseline already good, no headroom) in
InvestigatingResults.md, and explains why two prior attempts to fix
plugin-mode activation via description tweaks (PR #667, #688) failed:
the router rationally skips loading the skill for a trivial question.
This change replaces the scenario with a meaningfully harder one that
combines several signals the model often confuses:
- Multi-TFM project (TargetFrameworks plural)
- MTP runner (TestingPlatformDotnetTestSupport)
- .NET SDK 9 (-- separator required for MTP extension args)
- TRX report (MTP extension flag)
The correct answer requires knowing that --framework is a dotnet
test/MSBuild flag that always goes BEFORE --, while --report-trx is an
MTP extension flag that goes AFTER -- on SDK 9:
dotnet test --framework net9.0 -- --report-trx
The ordering assertion in eval.yaml enforces this on a single line, so
swapping --framework after the separator no longer satisfies the
scenario.
Also fix a documentation bug in SKILL.md surfaced by this work: the
'Common MTP flags > Built-in flags' table previously listed --framework
and --no-build under 'On SDK 8/9, pass after --', which is wrong. Those
are dotnet test/MSBuild flags consumed by dotnet test itself; they
always go before --. The table now lists only flags genuinely consumed
by MTP, and an Important note clarifies the rule.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add eval scenario for filter-syntax [trait] query language pattern
Add a run-tests scenario that exercises the xUnit v3 query filter language
with [traitName=traitValue] syntax (--filter-query), covering the uncovered
[CodePattern] [trait] at line 113 of filter-syntax SKILL.md.
New fixture: mtp-xunit-v3-query with multiple test classes and traits
(Smoke, Regression, Unit) to test combined class-pattern + trait filtering.
* Address review feedback for xUnit v3 query eval coverage
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Add rollForward: latestFeature to all global.json entries that pin SDK 9.0.200
so that any 9.0.xxx SDK satisfies the requirement. Without this, machines with
only 9.0.3xx installed fail all dotnet commands during evaluation, causing the
model to burn its entire timeout budget on failed bash retries.
Tom Longhurst