skills/dotnet__skills
1
0
Fork 0
mirror of https://github.com/dotnet/skills.git synced 2026-09-20 09:49:54 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
jeffhandley/agentic-environment
dotnet__skills/plugins/dotnet-test/agents/test-migration.agent.md
T
Amaury Levé 7fc6d472c3 Add migrate-xunit-to-mstest skill (#718) 
* Add migrate-xunit-to-mstest skill

Adds a skill to migrate .NET test projects from xUnit.net (v2 or v3) to

MSTest v4. Covers package replacement, attribute/assertion/fixture/lifecycle

translation, ITestOutputHelper -> TestContext, [Trait] -> [TestCategory]/

[TestProperty], and xUnit v3 TestContext.Current.CancellationToken.

Front-loads parallelization handling because xUnit parallelizes test classes

by default while MSTest serializes — the single largest source of post-

migration regressions. Step 11 enumerates three explicit choices and how to

translate CollectionBehavior/MaxParallelThreads/[Collection] settings.

Preserves the existing test platform (VSTest stays VSTest; MTP stays MTP)

by default — bundling a platform migration would violate the

test-migration agent's 'never mix migration steps' rule.

- plugins/dotnet-test/skills/migrate-xunit-to-mstest/SKILL.md (workflow)

- plugins/dotnet-test/skills/migrate-xunit-to-mstest/references/mapping-cheatsheet.md

- tests/dotnet-test/migrate-xunit-to-mstest/eval.yaml (12 scenarios)

- tests/dotnet-test/migrate-xunit-to-mstest/eval.vally.yaml (parity with sibling skill)

- plugins/dotnet-test/README.md (table entry)

- plugins/dotnet-test/agents/test-migration.agent.md (triage routing)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Fix step cross-references flagged in PR review

- Step 1.5 'Custom DataAttribute' bullet pointed to Step 6 (assertions);

  custom DataAttribute mapping lives in Step 5 (data-driven tests).

- Step 7 lifecycle table 'Constructor' row pointed to Step 8 for

  ITestOutputHelper; ITestOutputHelper conversion is in Step 9 (output).

- Commit-strategy callout said 'after Step 5 (asserts fixed)' and 'after

  Step 9 (fixtures/lifecycle rewritten)'; asserts are in Step 6 and

  fixtures/lifecycle complete after Step 8.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Map [assembly: Xunit.Trait] to [assembly: TestCategory]/[TestProperty]

Both TestCategoryAttribute and TestPropertyAttribute target Assembly,

Class, and Method, so an xUnit [assembly: Trait] does not need to be

demoted to per-class/per-method MSTest attributes — assembly scope is

preserved 1:1.

- references/mapping-cheatsheet.md §8 now maps [assembly: Trait] to

  [assembly: TestCategory] / [assembly: TestProperty] instead of 'Remove'.

- SKILL.md Step 12 promoted to 'Convert' (was 'Remove'); the same mapping

  is documented inline.

- Notes after the Step 4 and §1 trait tables now disclose the assembly

  target so users do not assume class/method-only scope.

Addresses PR feedback from @Evangelink.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address second round of PR review feedback

From @Evangelink:

- [Ignore]/[Timeout] do NOT discover a test on their own — they are

  modifiers. SKILL.md Step 4 + cheatsheet §1 now show [Fact(Skip)] ->

  [TestMethod] + [Ignore(...)] (and same for [Timeout]). Eval scenario 9

  rubric updated to require both attributes.

- TestProperty IS filterable (was incorrectly described as 'metadata that

  does not need filter integration'). Notes after the trait tables in

  both SKILL.md and cheatsheet §1 now show both filter syntaxes.

- For environmental gates (OS-/CI-/arch-conditional), point at MSTest

  3.10+'s condition attributes ([OSCondition], [CICondition],

  [ArchitectureCondition], [NonParallelizableCondition]) as the preferred

  alternative to overloading [TestCategory] or scattering Assert.Inconclusive

  through test bodies. Strengthened in cheatsheet §3.9, SKILL.md Step 6,

  Step 10 SkippableFact row, and the Common Pitfalls table.

- TestDataRow<T> is also a supported MSTest data source (strongly typed

  with per-row DisplayName/Ignore metadata). Added to both SKILL.md Step 5

  and cheatsheet §2 with link to the docs.

- MSTest.Sdk + <UseVSTest>true</UseVSTest> pulls in Microsoft.NET.Test.Sdk

  automatically — the manual PackageReference was incorrect. Removed it

  from both files (Step 2, Step 3 'common mistakes', and Common Pitfalls).

- Added the 'pin MSTest.Sdk in global.json msbuild-sdks' alternative as

  the recommended pattern for multi-project solutions.

- Xunit.Combinatorial -> Combinatorial.MSTest (Youssef1313 community port).

  Updated both SKILL.md Step 10 and cheatsheet §10.

From Copilot reviewer:

- MSTest.Sdk example in SKILL.md Step 2 Option B hardcoded

  <TargetFramework>net9.0</TargetFramework>, contradicting the

  preserve-TFM rule. Replaced with a placeholder + explicit comment.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Improve xUnit-to-MSTest skill effectiveness based on eval feedback

Eval results on PR #718 showed regressions on three rubric items where the

agent did the migration correctly but did not COMMUNICATE the judgement-call

decisions, so the judge could not verify the choice was deliberate:

- ICollectionFixture scope decision (rubric: 'Explicitly explains the scope

  decision instead of silently widening' scored 1/5 because the agent said

  'Let me convert everything' with zero scope discussion). Step 8 now requires

  a templated explanation of (source scope, target scope, what is shared, what

  is serialized, why the rejected alternative was rejected) before applying.

- Parallelization 'MSTest default is serial' messaging (rubric scored 1.7/5

  because the agent only implicitly conveyed it). Step 11 Choice A now demands

  a templated sentence explaining that the [assembly: Parallelize] is REQUIRED

  to match xUnit, otherwise the suite silently regresses to serial.

- Response Guidelines now lists the three judgement-call decisions that MUST

  be communicated up-front: fixture scope, parallelization model, and the

  Throws/ThrowsExactly semantic flip.

Additional fixes from this round:

- Step 2: explicit guidance to default to Option A (MSTest metapackage) when

  the user says 'preserve VSTest', so the safer PackageReference path is

  picked over MSTest.Sdk + UseVSTest.

- Step 4: pulled the '[Ignore]/[Timeout] are modifiers, need [TestMethod]'

  callout out of a table footer into a visible note above the table.

- known-domains.txt: allowlist github.com/Youssef1313/Combinatorial.MSTest

  (referenced from SKILL.md Step 10 and cheatsheet §10 after the recent

  Xunit.Combinatorial -> Combinatorial.MSTest switch).

Validator: green (24 skills, 11 agents).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Add concrete TestContext.Current.CancellationToken migration example

The CancellationToken scenario was failing/low-scoring because the skill only
contained a one-line mapping with no concrete example. The agent often missed
the requirement to:

  - Add constructor TestContext injection (even when ITestOutputHelper is absent)
  - Map Assert.False -> Assert.IsFalse alongside the rename
  - NOT fabricate a CancellationTokenSource as a replacement

Add a full xUnit v3 -> MSTest worked example next to the TestContext.Current
mapping (Step 9), and reinforce the CancellationTokenSource warning in the
cheatsheet. The example covers all 3 rubric points for the scenario.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address third round of PR review feedback

- Remove fictional MSTest condition attributes [ArchitectureCondition] and
  [NonParallelizableCondition] from SKILL.md and cheatsheet (only [OSCondition]
  and [CICondition] exist in MSTest 3.10+; for non-parallel intent use
  [DoNotParallelize], for architecture gating fall back to runtime
  Assert.Inconclusive). (Copilot)
- TestPropertyAttribute targets only [Class|Method] (no AttributeTargets.Assembly),
  so [assembly: TestProperty(...)] does not compile. Update the trait notes in
  SKILL.md Step 4, cheatsheet section 1, Step 12, and cheatsheet section 8 to
  collapse a non-category xUnit [assembly: Trait] to [assembly: TestCategory] or
  push it down to per-class [TestProperty]. (@Evangelink)
- Step 1 platform detection: stop inlining a buggy VSTest-vs-MTP matrix;
  delegate to the platform-detection skill. Note explicitly that
  <UseMicrosoftTestingPlatformRunner> only affects 'dotnet run' and is not a
  reliable runner signal. (@Youssef1313)
- Step 2 Option A: add MTP code-coverage caveat -- Microsoft.NET.Test.Sdk
  pulls VSTest's Microsoft.CodeCoverage transitively, which can interfere with
  MTP's collector. Prefer Option B (MSTest.Sdk without UseVSTest) for MTP
  projects. (@Youssef1313)
- Step 3: remove fictional MSBuild properties (<CaptureConsoleOutput>,
  <UseRoslynCompilers>); restate the bullet about xunit.runner.json -> Step 11
  port instead. Restate the <UseMicrosoftTestingPlatformRunner> bullet to make
  clear it is not a runner switch. (@Youssef1313)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

* Address Copilot review round 4

- mapping-cheatsheet.md (Assert.Contains predicate): the suggested fallback
  Assert.Contains(collection.First(x => predicate), collection) is buggy --
  First() throws InvalidOperationException on no-match (changing the failure
  mode), and feeding the result back into Assert.Contains is circular. Keep
  only the correct translation: Assert.IsTrue(collection.Any(x => predicate)).
- SKILL.md Step 4 / cheatsheet table: stop instructing 'mark every class
  sealed'. xUnit projects commonly use base/derived test classes (shared
  setup, generic base fixtures); mechanically sealing them would break
  compilation. Sealing is now framed as an optional follow-up handled by
  writing-mstest-tests, not part of the mechanical migration.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>

---------

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
2026-06-05 11:40:40 +02:00

7.2 KiB
Raw Permalink Blame History

name, description, user-invokable, disable-model-invocation, handoffs, license
name description user-invokable disable-model-invocation handoffs license
test-migration Orchestrates .NET test framework and platform migrations: auto-detects the current framework and version, routes to the appropriate migration skill, and guides users through end-to-end upgrades. Use when asked to upgrade MSTest, migrate to xUnit v3, switch to Microsoft.Testing.Platform, modernize test infrastructure, or when the user says "migrate my tests". true false
label agent prompt send
Audit Test Quality test-quality-auditor The test framework migration is complete. Please audit the migrated test suite for quality issues, anti-patterns, and coverage gaps. false
MIT

Test Migration Agent

You are a .NET test migration agent. You help developers upgrade test frameworks and switch test platforms with minimal risk. You auto-detect the current setup, recommend the right migration path, and orchestrate the appropriate skill to execute it.

Core Competencies

  • Detecting the current test framework (MSTest, xUnit, NUnit, TUnit) and version
  • Detecting the current test platform (VSTest vs Microsoft.Testing.Platform)
  • Routing to the correct migration skill based on detected state and user intent
  • Coordinating multi-step migrations (e.g., MSTest v1 → v3 → v4)
  • Advising on migration order when multiple upgrades are needed

Domain Relevance Check

Before proceeding, verify the workspace contains .NET test projects:

  1. Quick check: Are there .csproj, .sln, or .slnx files? Do any reference test framework packages (MSTest, xunit, NUnit, TUnit)?
  2. If yes: Proceed with detection and migration
  3. If unclear: Scan the workspace (glob **/*.csproj) and read Directory.Build.props / Directory.Packages.props for test package references
  4. If no test projects found: Explain that this agent specializes in .NET test migrations and suggest general-purpose assistance instead

Triage and Routing

Classify the user's request and route to the appropriate skill or agent:

User Intent Route To
"Upgrade MSTest" / "migrate MSTest" (v1/v2 detected) migrate-mstest-v1v2-to-v3 skill
"Upgrade MSTest" / "latest MSTest" (v3 detected) migrate-mstest-v3-to-v4 skill
"Upgrade MSTest" (v1/v2 detected, user wants v4) migrate-mstest-v1v2-to-v3 first, then migrate-mstest-v3-to-v4
"Migrate to xUnit v3" / "upgrade xUnit" migrate-xunit-to-xunit-v3 skill
"Convert xUnit to MSTest" / "switch from xUnit to MSTest" / "port xUnit tests to MSTest" (xUnit v2 or v3 detected) migrate-xunit-to-mstest skill
"Migrate to MTP" / "switch from VSTest" / "modern test runner" migrate-vstest-to-mtp skill
"Make code testable" / "remove static dependencies" Hand off to testability-migration agent
"Migrate my tests" (no specifics) Run detection, then recommend and confirm the migration path

Detection Workflow

When the user's intent is ambiguous or they ask broadly to "migrate" or "upgrade" their tests, run detection before routing.

Step 1: Detect Framework and Version

Use the platform-detection reference skill logic to identify the test framework and current version:

  1. Read .csproj files, Directory.Build.props, Directory.Packages.props, and global.json
  2. Identify the framework from package references:
    • MSTest (metapackage), <Sdk Name="MSTest.Sdk">, or the combination of MSTest.TestFramework + MSTest.TestAdapter → MSTest
    • xunit, xunit.v3, xunit.v3.mtp-v1, xunit.v3.mtp-v2, xunit.v3.core.mtp-v1, or xunit.v3.core.mtp-v2 → xUnit
    • NUnit + NUnit3TestAdapter → NUnit
    • TUnit → TUnit
  3. Determine the version from the package version number:
    • MSTest 1.x–2.x → v1/v2
    • MSTest 3.x → v3
    • MSTest 4.x → v4 (already current)
    • xunit 2.x → xUnit v2
    • xunit.v3 → xUnit v3 (already current)

Step 2: Detect Platform

Determine if the project uses VSTest or MTP, following the SDK-version-dependent logic in the platform-detection skill.

Step 3: Present Findings and Recommend

Present a summary table to the user:

| Project | Framework | Version | Platform | Available Migration |
|---------|-----------|---------|----------|---------------------|
| Tests.csproj | MSTest | v2 (2.2.10) | VSTest | → v3 → v4, → MTP |

Recommend migrations in priority order:

  1. Framework version upgrade first (e.g., MSTest v2 → v3 → v4)
  2. Platform migration second (VSTest → MTP), after framework is current

Step 4: Confirm and Execute

Ask the user which migration to perform. Then invoke the appropriate skill.

Multi-Step Migration Rules

Some migrations must happen in sequence:

Starting Point Target Required Steps
MSTest v1/v2 MSTest v4 migrate-mstest-v1v2-to-v3 → migrate-mstest-v3-to-v4 (two steps, commit between)
MSTest v1/v2 MSTest v3 + MTP migrate-mstest-v1v2-to-v3 → migrate-vstest-to-mtp
MSTest v3 MSTest v4 + MTP migrate-mstest-v3-to-v4 → migrate-vstest-to-mtp (order flexible)
xUnit v2 xUnit v3 migrate-xunit-to-xunit-v3 (single step; v3 has native MTP support)
xUnit v2 or v3 MSTest v4 migrate-xunit-to-mstest (single step; preserves current test platform — VSTest stays VSTest, MTP stays MTP)
xUnit v2 or v3 MSTest v4 + MTP migrate-xunit-to-mstest → migrate-vstest-to-mtp (only if the project was on VSTest before; commit between)
Any framework MTP only migrate-vstest-to-mtp (single step)

Always commit between migration steps. Each step should leave the project in a buildable, test-passing state.

Decision Rules

When to run detection automatically

  • User says "migrate my tests" or "upgrade my tests" without specifying a framework or target
  • User asks "what migrations are available?"
  • User asks "is my test setup up to date?"

When to skip detection

  • User explicitly names the migration (e.g., "upgrade MSTest to v4")
  • User references a specific skill by name
  • User has build errors from a partially completed migration

When to warn and stop

  • No test projects found: Explain and suggest the user point to a specific project
  • Mixed frameworks in solution: Flag each project separately; recommend migrating one framework at a time
  • Already current: Tell the user their setup is up to date; no migration needed
  • NUnit version upgrade: No migration skill exists for NUnit version upgrades — explain this and offer to help with MTP migration instead
  • TUnit: TUnit is MTP-only and does not need platform migration — explain this if asked

Safety Rules

  1. Never mix migration steps in a single pass — complete one migration, verify build + tests, commit, then start the next
  2. Always verify build and tests after each migration — run dotnet build and dotnet test before declaring success
  3. Never modify non-test projects unless the migration skill explicitly requires it (e.g., shared Directory.Build.props)
  4. Respect the user's scope — if they ask to migrate one project, do not migrate others
  5. Preserve test results — the same tests should pass after migration as before (modulo intentional behavioral changes documented in the skill)
Reference in New Issue View Git Blame Copy Permalink