mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
YuliiaKovalova
5f1170152b
* Add 4 MSBuild target authoring skills for dotnet-msbuild plugin New skills: - target-authoring: three-level target chain, DependsOn extension, naming conventions - property-patterns: conditional defaults, composition, path normalization, TFM helpers - item-management: Include/Remove/Update, batching, transforms, FileWrites registration - extension-points: CustomBefore/After hooks, import gating, NuGet build extensions Each skill includes eval.yaml tests with anti-pattern fixtures. Closes #668 * Address Copilot review feedback - target-authoring: Reword DO NOT USE clause to exclude only deep incremental-build diagnostics, not basic Inputs/Outputs usage - property-patterns: Close unclosed XML elements in String Functions snippet (PropertyGroup, TargetFrameworkMoniker) - extension-points: Add missing CustomAfterMySDK property definition to match the import at bottom of example * Remove accidental pr-body.md file * Fix wildcard import placeholder path and quote NormalizePath args - extension-points: Use full MSBuildExtensionsPath expression and consistent Exists() casing in wildcard import example - property-patterns: Quote property arguments in NormalizePath call * Tune eval prompts to reduce overfitting Rewrite all 4 eval prompts as natural developer problem descriptions instead of skill-aligned checklists. Softens technique-prescriptive rubric items. Result: overfitting scores drop from 0.15-0.31 to 0.06-0.08 (all green). property-patterns now passes eval. * Fix Copilot review round 3: quote property function args, fix fixture - property-patterns: Quote in IsPathRooted call - extension-points: Quote in GetDirectoryNameOfFileAbove - item-management fixture: Use literal semicolon instead of %3B in WriteLinesToFile * Add harder multi-file eval scenarios for all 4 skills Each skill now has a second scenario with multi-file setups containing interacting bugs that require cross-file analysis: - target-authoring: SDK .targets + Directory.Build.targets with 3 bugs (target redefinition, fragile BeforeTargets, duplicate DependsOn chain) - property-patterns: nested Directory.Build.props with 5 bugs (import order, unconditional overwrite, unquoted condition, missing trailing slash, NoWarn overwrite) - item-management: csproj with 5 interacting bugs (Include vs Update, cross-product batching, eval-time FileWrites, too-broad Remove glob) - extension-points: Directory.Build.props/targets + NuGet package with 6 bugs (inverted guard, late ImportByWildcard, CustomBefore overwrite, missing Exists guard, target name collision, internal target hook) Timeouts increased to 180s for all scenarios. * Fix review round 4: revert unintended README change, quote intrinsic call in condition * Improve eval quality for 4 new MSBuild authoring skills Eval improvements across target-authoring, property-patterns, item-management, and extension-points skills: - Rewrite prompts to describe observable symptoms without naming MSBuild concepts, inspired by real dotnet/msbuild and dotnet/sdk issues (msbuild#2470, #12894, #13056, #4109, sdk#43908) - Broaden assertion regex patterns to test for diagnostic outcomes rather than skill-specific vocabulary - Improve rubric items to test understanding and diagnosis flow (diagnosis -> root cause -> fix) instead of skill terminology - Add task-completion 'fix it' scenario to each skill with file_contains assertions that verify the agent applied correct patterns (CompileDependsOn append, FileWrites, Exists guard, etc.) - Remove leading 'Anti-pattern:', 'BUG N:' comments from fixture files that were giving away answers to the baseline agent, making scenarios more realistic and better at measuring skill value Each skill now has 3 scenarios: basic review, hard multi-file review, and a task-completion fix scenario. Total: 12 scenarios across 4 skills. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix review comments: quote TFM intrinsic, fix PrivateAssets example - property-patterns SKILL.md: Quote the GetTargetFrameworkIdentifier intrinsic call result in the Condition to match the 'always quote both sides' guidance and avoid brittle parse when empty - item-management SKILL.md: Replace PackageReference Update on Microsoft.NETCore.App (not a real user-added item) with Include on Microsoft.CodeAnalysis.NetAnalyzers (a concrete analyzer package) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix eval activation and timeouts based on eval run results - Add MSBuild domain terms to prompts (e.g., 'MSBuild targets', 'MSBuild property patterns', 'MSBuild item groups', 'MSBuild extension points') to improve skill activation rates — skills were NOT ACTIVATED in plugin mode due to prompts being too symptom-focused without domain keywords - Increase timeouts: review scenarios 180s -> 240s, fix-it scenarios 180s -> 300s to avoid timeout-related scoring penalties - Keep prompts symptom-driven but include enough domain context for the skill router to activate the correct skill Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix PR review comments and improve eval quality Review comment fixes: - TargetAuthoring.csproj: Replace %22/%3B URL encoding with XML escaping ("/;) so generated C# code is valid - target-authoring eval: Fix prompt referencing 'schema files' when fixture has endpoint definitions — now says 'endpoint list' - property-patterns eval: Reword LangVersion rubric — unconditional assignment in .props IS overridable by csproj but prevents override from earlier imports and command-line properties - extension-points eval: Add stronger activation terms to Fix scenario prompt (explicit mention of CustomBeforeMicrosoftCommonTargets) Eval quality improvements based on CI results: - Shorten target-authoring rubric items to reduce overfitting (was 0.30) - Remove 'Focus on target authoring patterns' instruction from hard scenario prompt (was directing agent too specifically) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Improve eval: increase timeout, add activation keywords, explicit fix instructions - property-patterns 'Diagnose multi-level': timeout 240s -> 360s (was hitting limit at 249s) - target-authoring 'Diagnose custom target': add CompileDependsOn/incremental build keywords for better skill activation - property-patterns 'Fix': add 'edit ... directly' to prompt to encourage file edits - target-authoring 'Fix': add 'edit ... directly' to prompt - item-management 'Fix': add 'edit ... directly' to prompt Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix target-authoring skill description and eval prompts for plugin activation The skill description said 'DO NOT USE FOR: deep incremental-build diagnostics' which caused the plugin agent to skip it for scenarios about build regressions. Clarified that the skill IS for diagnosing target authoring mistakes (missing Inputs/Outputs, broken dependency chains) that cause full rebuilds. Also updated eval prompts to use 'target authoring mistakes' framing and increased Fix scenario timeout to 360s. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Increase eval timeouts: extension-points NuGet (240->360s), item-management cascading (240->360s) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix 3 fresh PR review comments + update eval prompts for target-authoring - property-patterns/SKILL.md: add warning that TargetFramework-conditioned PropertyGroups must be in .targets not .props (empty in single-target .props) - extension-points/SKILL.md: add Exists() guard to GetPathOfFileAbove Import to prevent build failure when no parent file is found - target-authoring/SKILL.md: fix comment mismatch — validation target runs via dependency chain, not BeforeTargets; update comment accordingly - target-authoring/eval.yaml: reframe 'Diagnose' and 'Fix' prompts to use 'dependency chain' / 'authoring patterns' language (avoids routing to incremental-build skill which also claims Inputs/Outputs diagnostics) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Improve skill descriptions for reliable plugin-mode activation Add 'Only activate in MSBuild/.NET build context.' prefix to all 4 skills for consistency with existing skills in the plugin. Add explicit 'diagnosing and fixing' and 'reviewing' keywords to USE FOR sections so the SDK selects these skills over the broader msbuild-antipatterns skill when prompts ask to fix or review specific item/extension/property/target patterns. Add 'general MSBuild anti-pattern catalog (use msbuild-antipatterns)' to DO NOT USE FOR sections to help the SDK disambiguate. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix skill descriptions: shorten to under 1024-char SDK limit All 4 new skills had descriptions exceeding the 1024-character maximum enforced by the skill-validator check command. The Copilot SDK silently ignores skills with over-length descriptions, causing 0% activation in both isolated and plugin evaluation modes. Shortened all 4 descriptions while preserving key activation keywords (USE FOR / DO NOT USE FOR / 'Only activate in MSBuild/.NET build context'). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Address PR review comments - target-authoring eval.yaml: fix prompt to say 'Outputs attribute' (not 'Returns') since the fixture uses Outputs on a query target, which is the actual bug that causes MSBuild to skip re-execution - property-patterns eval.yaml: reword prompt and rubric — the real issue is unconditional assignment breaking parent-child Directory.Build.props inheritance, not project files being unable to override properties - extension-points SKILL.md: fix GetPathOfFileAbove example to factor the path into a property so Project= and Condition= use the same value consistently (avoiding the ..\\ vs ..\ discrepancy) - CustomSdk.targets: add WriteLinesToFile to CoreCodeGen so it actually creates the .g.cs output files (prevents compilation failure in fixture) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * Fix property-patterns fixtures to match rubric claims Scenario 1 (path merge bug): - PropertyPatterns.csproj: add OutputPath using CustomOutputDir without a trailing separator, making the merge bug observable (artifacts\binPropertyPatterns\ instead of artifacts\bin\PropertyPatterns\) Scenario 2 (multi-level hierarchy bugs): - hard/Directory.Build.props: make LangVersion unconditional so it overwrites the child src/Directory.Build.props LangVersion=preview, matching the rubric claim about parent unconditional assignments - hard/Directory.Build.props: add NoWarn=NU1702 (conditional default) so parent suppressions are visible but lost when the child's unconditional <NoWarn>CS1591;IDE0005</NoWarn> overwrites them after import Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
6 lines
111 B
XML
6 lines
111 B
XML
<Project>
|
|
<PropertyGroup>
|
|
<RepoRoot>$(MSBuildThisFileDirectory)</RepoRoot>
|
|
</PropertyGroup>
|
|
</Project>
|