* CTA: invocation-only baseline (dispatch mechanics, no content rules)
Experiment branch to isolate the impact of "invoke prompted subagents
more often" from the impact of "make those subagents do richer work."
Same baseline as dev/ykovalova/cta-prompt-tuning (main = 66628b6), but
strips out every content/quality rule and keeps only the dispatch
plumbing.
Comparison branch: dev/ykovalova/cta-prompt-tuning (HEAD: bd530be)
which contains both the dispatch mechanics AND content/quality rules.
Files modified (3 vs 5 in cta-prompt-tuning):
- code-testing-generator.agent.md +110 lines
- code-testing-implementer.agent.md +11 lines
- code-testing-fixer.agent.md +1 line
- code-testing-researcher.agent.md UNTOUCHED (baseline)
- code-testing-planner.agent.md UNTOUCHED (baseline)
KEPT (invocation / dispatch mechanics):
Generator:
- Rule 1: every task() call MUST use agent_type "dotnet-test:code-testing-..."
(without this, calls dispatch generic built-ins and never reach the
named CTA agents)
- Rule 2: routing table -- which named agent for which job
- Rule 3: prefer one named-agent dispatch over many tool calls
- Rule 4: orchestrator MUST NOT edit/create test files itself
(forces implementer dispatch)
- Rule 5: orchestrator MUST NOT run builds/tests via terminal
(forces builder/tester dispatch)
- Rule 6: every run MUST dispatch the planner (no exceptions for "small"
scope; Direct still goes through planner)
- Rule 7: every build/test failure MUST dispatch the fixer
- Step 1b: mandatory initial researcher dispatch (every strategy)
- Direct strategy rewritten: dispatches planner -> implementer -> builder
-> tester -> fixer -> linter (was "Skip Steps 3-5, write tests inline")
- All Step 3/4/5/6/7/8/9 dispatches converted from runSubagent({agent:...})
to task({ agent_type: "dotnet-test:code-testing-...", name:..., prompt:...})
- Step 9 validator dispatch (forces builder dispatch for cleanup)
- Steps 6/7 mandatory builder/tester dispatch wrapper
Implementer:
- Section 5: "you MUST dispatch fixer for build errors" + no-inline-edit
block (forces fixer dispatch on build failures)
- Section 6: "you MUST dispatch fixer for test failures" + no-inline-edit
block (forces fixer dispatch on test failures)
- Section 7: "Format Code (mandatory if a lint command exists)"
(was "Optional"; mandatory firing of linter)
- Rule 6: never declare SUCCESS while build/tests fail (gates SUCCESS on
fixer dispatch)
- Rule 7: no inline test-file edits between failed dispatch and fixer
Fixer:
- Frontmatter description widened to advertise handling of failing tests
(without this, the orchestrator's routing logic does not select the
fixer for test failures, so even Rule 7's mandate produces no firing
-- this is the change that took fixer firing from 0.00/inst to 0.39/inst
in earlier iterations)
DROPPED (content / quality rules -- in cta-prompt-tuning, NOT here):
Generator:
- Test-strength rules embedded in implementer dispatch prompt
- Test-design rules embedded in implementer dispatch prompt (OFAT,
mutation self-check, never mock subject under test)
- File-location rules embedded in implementer dispatch prompt
- TARGET ENTITIES / PHASE CHECKLIST / TEST TRACEABILITY blocks in
implementer dispatch prompt
- CHECKLIST format spec in planner dispatch prompt
- Step 9 validator's detailed cleanup classification
Implementer:
- Section 4b "Verify CHECKLIST coverage" pre-completion check
- Section 8 "CHECKLIST COVERAGE" report block
- "Honor the CHECKLIST" rule
Fixer:
- "Process -- Failing Tests" section (5-step diagnosis flow)
- All anti-weakening / anti-skipping rules
- "Re-derive expected from production source" guidance
Planner:
- CHECKLIST format ("one item per TARGET BEHAVIOR, Source/Variants/
Expected mandatory")
- "Test name from research.md conventions" rule
- "At least 2 phases" rule
Researcher:
- Section 8 "Extract Local Test Naming & Style Conventions"
- TARGET ENTITIES / TARGET BEHAVIORS / TEST INFRASTRUCTURE structure
in research.md
- Test naming pattern extraction
WHAT THE SUBAGENTS WILL ACTUALLY DO:
The researcher / planner / implementer / fixer all operate at baseline
behavior -- they receive the same prompts they receive in the upstream
"vanilla" runs. The only difference vs vanilla is that the orchestrator
ACTUALLY DISPATCHES THEM (where vanilla often inlines the work or skips
sub-agent dispatch entirely).
EXPECTED COMPARISON:
If quality on this branch is similar to or higher than dev/ykovalova/
cta-prompt-tuning (bd530be), then "more dispatches" is the dominant
quality lever and the content/quality rules in cta-prompt-tuning are
adding marginal or noise-level value.
If quality on this branch is materially lower than cta-prompt-tuning,
then the content/quality rules are doing the heavy lifting and the
dispatch mechanics alone are insufficient.
If quality on this branch matches or exceeds vanilla but trails
cta-prompt-tuning, then the dispatch mechanics provide a baseline lift
and the content rules add an incremental quality layer on top.
Rubber-duck check passed (validated dispatch-vs-content classification;
fixer frontmatter is routing metadata not a runtime gate; surviving
dispatch prompts contain no dangling references to removed CHECKLIST /
TARGET ENTITIES / TEST STRENGTH / naming-convention concepts).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* CTA: extract unit-under-test + behaviors quality cue (no caps)
* Address PR #646 review feedback
- Step 1b prompt: explicitly request unit-under-test (file:line) and behaviors
so the verification gate rarely needs re-dispatch.
- Step 3: rename to 'Deep Research Phase', mark skipped for Direct strategy,
and switch from overwriting research.md to extending it (no double-research).
- Step references: update '6-9' -> '6-10' and 'Step 9' -> 'Steps 9-10' in the
strategy table and the All-strategies-MUST line, since reporting is Step 10.
- Step 6 builder prompt: drop '*.sln' glob (could expand to multiple args);
use 'dotnet build --no-incremental' (auto-discovers .sln) per dotnet.md.
- Step 9: stop overloading the builder agent; perform diff/cleanup directly
in the orchestrator (Rule 5 forbids inline build/test, not git/fs hygiene).
- Fixer agent: update mission text to cover failing tests and assertion
correction (front-matter description already mentioned this; body now
matches), with explicit no-Ignore/no-Skip/no-production-rewrite guardrails.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Yuliia Kovalova <ykovalova@example.com>
Cover previously uncovered skill aspects:
- [Validation] All .cs files in scope were scanned (check count)
- [Validation] Report includes category totals, top patterns, and affected files
- [Validation] obj/ and bin/ directories were excluded
- [Pitfall] Scanning obj/ or generated code
- [Pitfall] Missing statics inside lambdas/LINQ
- [WorkflowStep] Step 4: Present the report
Three new scenarios with supporting fixtures:
- Structured report with file count, categories, and top patterns
- Exclude obj/bin directories (with Generated.cs decoy in obj/)
- Detect statics inside lambdas and LINQ expressions
* Add Python and TypeScript extension files for code-testing pipeline
First-draft language extension files following the dotnet.md pattern.
These guide the polyglot test agent on build/test/lint/fix for each language.
Python: pytest-focused, environment/runner detection (Poetry/PDM/uv/Hatch),
public-API testing philosophy, common errors, mocking guidelines.
TypeScript: package-manager detection (npm/pnpm/yarn/bun), Jest+Vitest+Mocha
support, ESM/CJS guidance, TS-specific considerations, framework detection.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Revise python.md and typescript.md based on 10-agent review
Key changes:
- Add 'Rule #1: Investigate the repo first' as top section
- Remove generic templates (anchor to wrong conventions)
- Remove testing philosophy (unactionable for agent)
- Fix command inconsistency (parameterize with <prefix>/<exec>)
- Move dependency install to 'last resort' section
- Add error-driven fixer playbook with concrete fixes
- Fix Mocha --grep (test name, not file filter)
- Fix ESM guidance (don't change package.json type field)
- Add monorepo/workspace guidance (Nx, Turborepo)
- Add framework notes (React, Express, NestJS)
- Add Jest mock hoisting warning
- Cut from 224+289 lines to 117+136 lines (~57% reduction)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add powershell.md extension for Pester v5 test generation
Key sections:
- Repo investigation first (shell target detection)
- Pester v5 discovery vs run phase rules
- Import patterns for modules, library scripts, and executable scripts
- Cross-platform guidance (pwsh vs powershell.exe, Join-Path, casing)
- Non-terminating error handling with Should -Throw
- Mock scoping, -ModuleName, PesterBoundParameters
- TestDrive: for file-based tests
- Non-obvious assertion gotchas (Contain vs Be, Throw needs scriptblock)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Reword test infra guidance; add framework detection to dotnet.md
- Reword 'do not add' to 'use existing, only introduce if none exist'
- Add test framework detection table to dotnet.md (MSTest/xUnit/NUnit)
- Tailor wording per language (Python defaults to pytest, TS follows
scripts.test, PS defaults to Pester)
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Make Python extension framework-adaptive instead of pytest-centric
Rule #1 now searches for ALL test file formats (not just test_*.py),
explicitly calls out custom frameworks like UTscapy, and emphasizes
adopting repo conventions fully rather than layering pytest on top.
Test Commands section now has a custom framework block before the
pytest block, and Test File Naming defers to repo conventions.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update .NET 10→11 migration skill for Preview 2 and 3
Add breaking changes from .NET 11 Preview 2 and Preview 3 to the
migrate-dotnet10-to-dotnet11 skill. The skill previously covered only
Preview 1.
Changes:
- Add 15 new breaking changes across all reference files
- Create new aspnetcore-dotnet10to11.md reference (OpenAPI v3, Blazor)
- Update SKILL.md workflow steps, assessment checklist, and patterns
- Add 4 new eval scenarios covering high-impact P2/P3 changes
- Add github.com/dotnet/aspnetcore to known-domains.txt
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback
- Replace prompt-echoable 'corrupt' in ZipArchive eval regex with 'checksum'
- Update EF Core reference note to say 'Preview 1 through Preview 3'
- Add missing Source link for Zstandard entry
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Address review feedback: tighten eval regex, fix raw HTML
- Tighten OverscanCount grader regex to require 'OverscanCount' with
a number, instead of matching bare '3' anywhere
- Wrap Blazor component snippet in backticks so it renders as code
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add eval scenario for filter-syntax [trait] query language pattern
Add a run-tests scenario that exercises the xUnit v3 query filter language
with [traitName=traitValue] syntax (--filter-query), covering the uncovered
[CodePattern] [trait] at line 113 of filter-syntax SKILL.md.
New fixture: mtp-xunit-v3-query with multiple test classes and traits
(Smoke, Regression, Unit) to test combined class-pattern + trait filtering.
* Address review feedback for xUnit v3 query eval coverage
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* fix(groom): prune resolved rows, add First Seen column, handle duplicate sections
- Rename 'Status' column to 'Investigation' for clarity
- Add 'First Seen' column showing when finding was first detected
- Remove resolved investigation rows instead of keeping them forever
- Handle duplicate Investigation Results sections by merging into one
- Update health-check workflow to use matching column schema
- Drop resolved rows from health-check table preservation logic
* fix(health-check): remove duplicate Investigation Results section
The health-check and groom workflows both created a '## Investigation
Results' section, causing duplicates in issue #288. The groom owns this
section via its replace-island block — remove it from the health-check
template and update the guideline accordingly.
* fix(health-check): restore Investigation Results as placeholder section
The health-check replaces the entire issue body, so removing the section
entirely caused it to disappear. Instead, keep it as a placeholder that:
- Preserves existing rows from the groom's island markers
- Appends new 'Dispatched' rows for findings dispatched in this run
- Does NOT add island markers itself (the groom adds those)
This ensures the section survives health-check body rewrites while the
groom still owns the detailed content via replace-island.
* fix: address PR review comments on groom/health-check workflows
- Fix duplicate section guidance: clarify replace-island only updates
first occurrence, doesn't auto-remove later duplicates
- Match example dispatched row text with health-check template
- Fix First Seen source: parse from summary line (first seen YYYY-MM-DD)
instead of non-existent **First seen:** field
- Preserve all non-resolved rows (not just Done rows) in health-check
- Fix Step 4.2 to align with prune behavior in Step 4.3
- Add schema migration for legacy 4-column tables in health-check
- Add link to dispatched investigation rows
* fix(groom): use MCP tools with min-integrity:none instead of unauthenticated gh CLI
The previous fix (PR #627) instructed the agent to use 'gh api' for reading
issue comments. However, the gh CLI is NOT authenticated inside the sandbox —
the framework deliberately strips credentials via the 'Clean git credentials'
step. The system prompt also explicitly states 'gh is not authenticated'.
Root cause analysis of run 25481332080 (pre-fix) shows the MCP issue_read
tool DID return comments — only 1 item out of many was filtered by integrity
policy. The agent overreacted to the [Filtered] warning and called
report_incomplete instead of working with the available data.
This fix:
- Sets min-integrity: none under tools.github so the integrity filter
does not remove any comments. This is safe because the workflow only
processes comments from github-actions[bot] (Step 2 filters by author).
- Reverts to MCP issue_read with get_comments (the proper tool for reading
comments in the sandbox).
- Removes gh from bash tools (not usable without auth).
- Adds explicit guidance: if filtered items appear, continue working with
available data — do NOT call report_incomplete or missing_tool.
- Documents that gh CLI is NOT authenticated in the sandbox.
Closes#625
* fix: address review comments - scope repos, bound comment fetch, fix jq guideline
The 'Fix Assert.AreEqual object overload errors after v3 upgrade' evaluation
scenario was failing because:
1. The prompt said 'latest major version' without mentioning 'v3' or 'MSTest v3',
missing the skill's activation keywords.
2. The project file showed MSTest 3.8.0 (post-upgrade state) with no v2 context,
while the SKILL.md 'When Not to Use' said 'Project already uses MSTest v3'.
3. The prompt's 'fix compilation issues' phrasing competed with writing-mstest-tests
skill activation.
Changes:
- Updated prompt to explicitly mention upgrading from v2 to v3 and reference
MSTest v3 compatibility issues
- Added XML comment in csproj showing the pre-upgrade v2 package versions
- Clarified SKILL.md 'When Not to Use' to distinguish fully-migrated projects
from projects with active migration errors
Move assertion-quality, test-gap-analysis, test-smell-detection, and
test-tagging from dotnet-experimental to dotnet-test. These skills have
demonstrated sufficient eval coverage, unique value beyond baseline model
capabilities, and are already integrated into the test-quality-auditor
agent pipeline.
Changes:
- Move skills and tests, dropping the exp- prefix
- Update all cross-references in skill descriptions and routing tables
- Update test-quality-auditor agent and eval scenarios
- Update CODEOWNERS for the moved paths
- Delete duplicate exp-dotnet-test-frameworks (identical to dotnet-test-frameworks)
- Fix stale exp-dotnet-test-frameworks reference in exp-test-maintainability
Skills held back for further iteration:
- exp-mock-usage-analysis: baseline model scores 5/5 on key scenarios
- exp-test-maintainability: quality gains offset by token overhead
The scenario was timing out on CI because:
- The 180s timeout left insufficient headroom for CI machines
(local runs take 120-135s, CI machines are slower due to load,
network NuGet restores, and parallel execution)
Changes:
- Add 'dotnet restore' setup command to pre-warm NuGet cache before
the agent starts (same pattern used by migrate-mstest-v3-to-v4)
- Increase timeout from 180s to 300s to accommodate CI variability
* Fix groom workflow: use gh api to bypass MCP integrity filter
The MCP gateway's integrity policy intermittently blocks access to issue
comments (report_incomplete: 'Resource issue:dotnet/skills#288 has lower
integrity than agent requires'). This caused issue #625.
Fix: Add 'gh' to allowed bash tools and instruct the agent to use
'gh api --paginate' for fetching comments. This bypasses the MCP gateway
entirely and also provides node_id fields required by hide-comment
(which the MCP issue_read tool did not return).
* Address PR review: fix --paginate JSON, add author filter, narrow guideline scope
* Fix triage workflow: separate label and assignee operations
- Use add_labels tool for labeling (Step 6) instead of update_issue
- Reserve update_issue exclusively for assignee assignment (Step 4)
- Increase update_issue max to 2 for assignment flexibility
- Add explicit workflow_dispatch guidance: agent must pass
issue_number/item_number on all safe-output calls when there
is no triggering issue context
- Fixes issues where owners were mentioned in comments but not
actually assigned to the issue
* Apply suggestions from code review
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
* Dashboards series fix (followup)
* Fix deploy-dashboard condition for workflow_dispatch
The deploy-dashboard job required needs.discover.result == 'success',
but the discover job's condition does not include workflow_dispatch,
so it is always skipped for manual triggers. This made workflow_dispatch
unable to deploy the dashboard.
Split the condition so workflow_dispatch from main bypasses the
discover dependency, while scheduled runs still require it.
* Update deploy-dashboard comment to reflect workflow_dispatch support
* add a new dotnet11 plugin and a new system text json skill to it.
* Update plugins/dotnet11/skills/system-text-json-net11/SKILL.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* added the option to run dotnet11 skills. by default they don't get included
* Update tests/dotnet11/system-text-json-net11/eval.yaml
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Add JSON output assertions to system-text-json-net11 eval.yaml
Agent-Logs-Url: https://github.com/dotnet/skills/sessions/8e302f3f-4328-44a0-845d-4eb0c930af63
Co-authored-by: ManishJayaswal <9527491+ManishJayaswal@users.noreply.github.com>
* addressed CR comments
* addressed more CR comments
* fixed agument to pass while enabling dotnet11 skills
* Remove dotnet11-specific filtering from skill-validator and evaluation workflow
Now that .NET 11 is the default SDK for the repo (global.json updated on main),
there is no need for special dotnet11 exclusion logic. The dotnet11 plugin skills
are treated like any other plugin and included in all evaluation runs.
Changes:
- Remove --include-dotnet11 CLI option and filtering logic from EvaluateCommand.cs
- Remove IncludeDotnet11 from ValidatorConfig model
- Remove --dotnet11 flag parsing from evaluation workflow gate job
- Remove dotnet11 exclusion logic from discover job
- Remove conditional .NET 11 preview SDK install step (global.json handles it)
- Remove dotnet11 from excludeFromSchedule list
- Remove all dotnet11-specific warning/status messages
- Remove section 9 from InvestigatingResults.md
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address code review comments after rebase
* added the plugin to cursor and added MIT license to skill
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: [Manish Jayaswal] <[manishj@microsoft.com]>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: ManishJayaswal <9527491+ManishJayaswal@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
/-prefix paths bypassed the flag and hit the depth check unconditionally. Now treated symmetrically with .. traversal: allowed only with the flag, otherwise produce a clear absolute (repo-rooted) path error. Also classify // as a protocol-relative URL.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Improve test quality for benchmark performance
Based on SWE Atlas benchmark analysis (48.8% vanilla vs 19-34% CTA):
1. Default to Direct strategy for single-task requests — reduces
multi-agent overhead that costs tokens without improving results
2. Run tests immediately in Direct strategy — catches assertion
errors early instead of accumulating failures
3. Read source thoroughly before writing tests — trace actual logic
and return values, not just function signatures
4. Quality over Quantity guidelines — cover stated requirements first,
fewer focused tests beat many shallow ones
5. Verify tests are implementation-specific — tests that pass with
an empty function body aren't testing anything useful
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Combine redundant implementation-specificity bullets into one
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add fsharp-project-structure skill for .fsproj file ordering and .fsi files
Teaches agents F# compilation order rules, signature file placement,
and common FS0039/FS0010/FS0034 fixes. Includes eval with three scenarios
(add module, fix broken order, add signature file) and test fixtures.
Validated +64.6% improvement on gpt-4.1 (significant, CI [+17.2%, +63.0%]).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Add fsharp-project-structure skill for .fsproj file ordering and .fsi files
Teaches agents F# compilation order rules, signature file placement,
and common FS0039/FS0010/FS0034 fixes. Includes eval with three scenarios
(add module, fix broken order, add signature file) and test fixtures.
Validated +64.6% improvement on gpt-4.1 (significant, CI [+17.2%, +63.0%]).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Update plugins/dotnet/skills/fsharp-project-structure/SKILL.md
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fold F# project-structure into msbuild-antipatterns AP-05
Scraps the standalone fsharp-project-structure skill and instead adds a
short F# exception to AP-05 (Manual File Listing in SDK-Style Projects):
F# compilation is order-dependent, so .fsproj files must list Compile
items in dependency order (utilities first, entry point last), and .fsi
signature files must appear immediately before their companion .fs.
The three F# eval scenarios (add module, fix broken order, add signature
file) are ported into tests/dotnet-msbuild/msbuild-antipatterns/eval.yaml
with fixtures isolated under a fsharp/ subfolder (with empty
Directory.Build.{props,targets} to prevent the existing C#-antipattern
fixture files from leaking into the F# builds).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Fix dotnet-test-frameworks skill activation for evaluations
The skill had user-invocable: false and a description saying 'DO NOT USE
directly', which prevented the model from activating it during evaluation.
All 6 eval scenarios default to expect_activation: true, causing
SkillNotActivated failures on CI.
- Remove user-invocable: false from both dotnet-test and experimental copies
- Rewrite description with activation keywords matching eval prompts
(assertion mapping, framework identification, skip annotations, exception
assertions, lifecycle conversion, integration test markers)
- Preserve note that other analysis skills also load this as reference data
* Restore user-invocable: false, fix cross-refs, add expect_activation: false
Address review comment: remove exp-* skill references from dotnet-test-
frameworks description since those skills reference exp-dotnet-test-
frameworks instead.
Optimize eval results: this reference data skill's content is knowledge
the model already has (baseline scores 5.0/5 on most scenarios). Direct
activation adds token/time overhead without quality improvement:
- Restore user-invocable: false (correct for reference data skill)
- Shorten description to not trigger activation
- Add expect_activation: false to all 6 scenarios
The skill's value is realized when loaded by consumer skills (e.g.
test-anti-patterns), not via direct user queries.
Adds a new agentic workflow (.github/workflows/markdown-linter.md) that
runs Super Linter on weekdays at 2 PM UTC and on manual dispatch. The
agent analyzes linter output and creates GitHub issues with categorized
findings, severity breakdown, and fix recommendations.
Adapted from https://github.com/githubnext/agentics/blob/main/workflows/markdown-linter.md
with repo-specific conventions: PAT pool rotation, fork protection,
pinned action SHAs, scoped bash tools, and safe-outputs.
Add end-to-end input/output examples to anchor expected behavior for the
LLM, addressing the Example Quality (2/5) review feedback.
Changes:
- New extensions/dotnet-examples.md with .NET-specific examples: sample
source code, research output, plan output, generated test file, fix
cycle walkthroughs, and final report
- SKILL.md: replace one-liner examples with strategy selection table,
pipeline walkthrough, and pointer to language-specific extensions
- Generator agent: add strategy decision examples and sample final report
- Researcher/Planner/Implementer agents: add references to extension
examples for concrete output shapes
Architecture keeps core agents language-agnostic — all language-specific
examples live in the extensions/ folder.
The 'Migrate MSTest.Sdk v3 project using ManagedType and TestTimeout'
scenario times out on CI because MSTest.Sdk/4.1.0 resolution takes 40+
seconds on cold NuGet cache. Each agent session may call dotnet build
multiple times, easily exceeding the 240s timeout.
Add a 'dotnet restore' setup command that runs before the agent starts
(outside the timeout window), pre-warming the NuGet cache and obj/
directory so subsequent builds complete in ~2s.
* Remove agentic workflows
The agentic workflows were useful when CCA didn't support loading skills. That's now solved and supported. The agentic workflows come with a couple of issues (limited chars) which make it hard to use. Remove them in favor of the plugin system.
* Remove Agentic Workflows section from README
Removed section about Agentic Workflows from README.
* Remove stale /agentic-workflows references from CONTRIBUTING.md, CODEOWNERS, and docs
Agent-Logs-Url: https://github.com/dotnet/skills/sessions/8f129e2f-156e-4ee9-9897-896cd1981513
Co-authored-by: ViktorHofer <7412651+ViktorHofer@users.noreply.github.com>
---------
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Harden the test generator agent to ensure final build validation, test
validation, and git commit always run — even when using the Direct
strategy for small requests.
Changes:
- Step 1: Mandate reading language-specific extensions (e.g., dotnet.md)
before writing any code. The extension contains critical project
registration (dotnet sln add) and build validation guidance.
- Step 2: Direct strategy now skips only Steps 3-5 (research, plan,
implement sub-agents), NOT Steps 6-9. Added explicit note that
Steps 6-9 are mandatory for ALL strategies.
- Step 6: Clarify full-solution build must use no --framework flag to
catch multi-target framework issues (e.g., net472 errors).
- Step 7: Require fresh build for final test validation (no --no-build).
- New Step 9: Explicit git commit with verification that the diff is
non-empty.
- New rules 9-11: Read extensions first, always validate+commit,
preserve existing tests.
Motivated by MSBench investigation showing:
- Agent used Direct strategy and skipped final validation entirely
- Agent never read extensions/dotnet.md (missing dotnet sln add)
- Agent forgot to git commit (300 tests created, Changes +0 -0)
- Agent deleted pre-existing tests while adding new ones
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Fix timeout in MSTest.Sdk v3-to-v4 eval scenario 9
The fixture csproj had MSTest.Sdk/3.8.0 while the prompt claimed the user
already upgraded to 4.1.0. This mismatch caused the model to edit the
csproj and run dotnet build, triggering slow MSTest SDK NuGet downloads
that exceeded the 240s timeout on CI.
- Update fixture csproj to MSTest.Sdk/4.1.0 to match the prompt
- Add specific CS error codes to the prompt so the model provides direct
fixes instead of building
- Update rubric item to reflect the project already being on 4.1.0
* Fix CI timeout in migrate-mstest-v3-to-v4 eval scenario
Reduce scope of the 'Fix multiple v4 breaking changes' scenario:
- Remove 3 lower-signal breaking changes from the fixture (AreEqual params,
IsInstanceOfType out param, Properties.Contains) to cut agent work
- Add max_turns: 5 to cap agent iterations
- Lower timeout from 240 to 180 seconds
* Strengthen net6.0 scenario prompt for reliable skill activation
YuliiaKovalova