mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Aditya Mandaleeka
84 lines
3.9 KiB
YAML
84 lines
3.9 KiB
YAML
name: template-smart-defaults
|
|
description: Evaluates the dotnet-template-engine/template-smart-defaults skill
|
|
type: capability
|
|
config:
|
|
timeout: 3m
|
|
stimuli:
|
|
- name: AOT implies a compatible framework
|
|
prompt: "Create a native AOT worker service. I haven't picked a target framework — choose an appropriate one for me and explain why. Don't write any files yet, just tell me the parameters you'd use."
|
|
graders:
|
|
- type: exit-success
|
|
- type: output-matches
|
|
config:
|
|
pattern: (?i)(aot)
|
|
- type: output-matches
|
|
config:
|
|
pattern: (?i)net(8|9|10)\.0
|
|
- type: prompt
|
|
rubric:
|
|
- The agent used the template-smart-defaults skill to fill the cross-parameter gap
|
|
- Because AOT was requested, the agent also selected a target framework
|
|
- The agent chose the latest AOT-compatible framework available rather than an arbitrary one
|
|
- The agent explained the rationale (native AOT needs a recent AOT-capable TFM)
|
|
- The agent confirmed it would verify parameter names against `dotnet new <template> --help`
|
|
constraints:
|
|
expect_tools:
|
|
- skill
|
|
- name: Auth implies HTTPS stays enabled
|
|
prompt: "I want to create a webapi with Individual authentication. First output the exact single `dotnet new` command line you would run — include only the flags you will actually pass, and do not mention any flags you are not using. Then give a brief explanation. Don't create anything yet."
|
|
graders:
|
|
- type: exit-success
|
|
- type: output-matches
|
|
config:
|
|
pattern: (?i)individual
|
|
- type: output-not-matches
|
|
config:
|
|
pattern: (?i)dotnet new[^\n]*--no-https
|
|
- type: prompt
|
|
rubric:
|
|
- The agent used the template-smart-defaults skill
|
|
- Because auth is not None, the agent did NOT add a --no-https flag
|
|
- The agent explained that authentication flows require HTTPS
|
|
- The agent only filled gaps and logged each applied default
|
|
constraints:
|
|
expect_tools:
|
|
- skill
|
|
- name: Controllers exclude the minimal-API flag
|
|
prompt: "Create a web API that uses controllers. First output the exact single `dotnet new` command line you would run — include only the flags you will actually pass, and do not mention any flags you are not using. Then explain. Do not create files."
|
|
graders:
|
|
- type: exit-success
|
|
- type: output-matches
|
|
config:
|
|
pattern: (?i)(use-controllers|controllers)
|
|
- type: output-not-matches
|
|
config:
|
|
pattern: (?i)dotnet new[^\n]*--[a-z-]*minimal
|
|
- type: prompt
|
|
rubric:
|
|
- The agent used the template-smart-defaults skill
|
|
- The agent passed the controllers flag
|
|
- The agent did NOT also pass a minimal-API flag (the two are mutually exclusive)
|
|
- The agent explained the reasoning
|
|
constraints:
|
|
expect_tools:
|
|
- skill
|
|
- name: Never override an explicit user value
|
|
prompt: "Create a native AOT console app, but I specifically want it targeting net8.0 — keep it on net8.0 no matter what. First output the exact single `dotnet new` command line you would run — include only the flags and framework value you will actually pass, and do not mention any flags or framework versions you are not using. Then tell me the parameters."
|
|
graders:
|
|
- type: exit-success
|
|
- type: output-contains
|
|
config:
|
|
substring: net8.0
|
|
- type: output-not-matches
|
|
config:
|
|
pattern: (?i)dotnet new[^\n]*--framework[ =]+net(9|10)\.0
|
|
- type: prompt
|
|
rubric:
|
|
- The agent used the template-smart-defaults skill
|
|
- The agent respected the explicit net8.0 framework the user set
|
|
- The agent did NOT override the user value with a newer framework despite AOT being requested
|
|
- The agent explained that smart defaults only fill gaps and never override explicit user choices
|
|
constraints:
|
|
expect_tools:
|
|
- skill
|