mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Amaury Levé
4f78daf583
* run-tests: add platform-detection keywords for activation
The 'Detect test platform from Directory.Build.props' eval scenario
regressed from 2.7/5 to 2.0/5 with run-tests not activated (skill
loaded column shows NOT ACTIVATED). The scenario prompt asks 'figure
out which test platform it uses' but the previous description framed
detection only as 'chooses the correct platform/SDK/framework syntax'
and 'selecting VSTest vs Microsoft.Testing.Platform command syntax' --
both focused on syntax selection, not on platform detection from
project files.
Per InvestigatingResults.md guidance for 'skill not activated'
failures, surface the detection use case explicitly with the file
names the scenario hinges on:
detecting test platform/framework from \global.json\, \.csproj\,
\Directory.Build.props\
Trim the syntax clause ('command syntax' -> 'syntax', 'including' ->
'incl.') and shorten the multi-TFM example (drop redundant
\dotnet test\ prefix) to stay under the 1,024-char description cap.
Folded description: 948 -> 1,010 chars.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* run-tests: rephrase platform detection with prompt-shaped language
The previous fix (021b0a343) activated the Detect test platform from
Directory.Build.props scenario in isolated mode but plugin-mode activation
still failed (1.3/5 → 1.7/5, isolated ✅ / plugin ⚠️ NOT ACTIVATED).
The scenario prompt asks the agent to `figure out which test platform`
a project uses, but the description used `detecting test
platform/framework` — the router did not bridge these phrasings when
competing sibling skills were loaded in plugin mode.
Changes:
- Replace `detecting test platform/framework from \global.json\...\` with
`figuring out which test platform (VSTest vs Microsoft.Testing.Platform)
a project uses from \global.json\, \.csproj\, \Directory.Build.props\
and selecting the matching syntax\\. This mirrors the prompt verb
`figure out\\ and merges the two adjacent clauses (detection +
selection) into one stronger USE FOR entry.
- Trim verbose tokens elsewhere to stay within the 1024-char folded
description cap (\SkillProfiler.cs\ MaxDescriptionLength): `TRX and
other reporting` -> `TRX/reporting`, `writing or generating` ->
`writing/generating`, `CI/CD pipeline configuration` -> `CI/CD
config`, drop `e.g., ` from the VSTest/MTP mixups list.
Folded description length: 1010 -> 1007 / 1024.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* run-tests: lead description with platform detection as primary value
Plugin-mode activation still failed after the second iteration (2.0/5 →
2.3/5, isolated activates / plugin does NOT). Eval artifact analysis
(run 26581320083) shows the plugin agent actually correctly read
`Directory.Build.props` and identified MTP (rubric: 5/5 + 5/5 on
detection), but did NOT load the skill — so it missed the SDK 9 MTP
`--` separator rule and integration-test filtering (rubric: 1/5 + 1/5
on syntax), capping the score at 2/5.
The router declined to load the skill because the prior opening
sentence `Runs .NET tests with \dotnet test\ and chooses the correct
platform/SDK/framework syntax\\ sounded like base knowledge — the model
felt it could answer `run dotnet test` directly. The detection clause
was buried as the 2nd item in USE FOR, where the router did not weight
it strongly enough.
This rewrite leads the description with platform detection as the
primary value proposition:
`For \dotnet test\: figures out which test platform (VSTest vs
Microsoft.Testing.Platform) a project uses from \Directory.Build.props\,
\global.json\, and \.csproj\, then picks the matching command
syntax.\\
Other changes:
- Add a parallel USE FOR entry `identifying the test runner/platform
from project files\\ to widen verb/noun coverage (`figures out` /
`identifying`, `platform\\ / `runner`) for prompt phrasings the
router may otherwise miss.
- Promote `\--\ separator rules on .NET SDK 8/9 vs 10+\\ from a
parenthetical aside to a standalone USE FOR entry so it shows up as
distinct specialized knowledge.
- Drop redundant `and selecting the matching syntax\\ since it is now
in the opening sentence.
Folded description length: 1007 -> 994 / 1024. `skill-validator check`
green.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>