mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Aditya Mandaleeka
64 lines
2.6 KiB
YAML
64 lines
2.6 KiB
YAML
name: mcp-csharp-test
|
|
description: Evaluates the dotnet-ai/mcp-csharp-test skill
|
|
type: capability
|
|
config:
|
|
timeout: 2m
|
|
stimuli:
|
|
- name: Write unit and integration tests for an MCP server
|
|
prompt: |
|
|
I have a C# MCP server with tools that call an external REST API.
|
|
I want to write tests that verify the tools work correctly without
|
|
calling the real API. How do I structure the test project?
|
|
graders:
|
|
- type: output-contains
|
|
config:
|
|
substring: xunit
|
|
- type: output-matches
|
|
config:
|
|
pattern: (Mock|Fake|mock|fake|HttpMessageHandler)
|
|
- type: output-matches
|
|
config:
|
|
pattern: (CallToolAsync|McpClient)
|
|
- type: prompt
|
|
rubric:
|
|
- Creates a test project with xUnit and references the server project
|
|
- Shows how to mock HttpClient or external dependencies
|
|
- Demonstrates an integration test using McpClient to call tools in-memory
|
|
- Separates unit tests from integration tests using traits or folders
|
|
- name: Test an HTTP MCP server with WebApplicationFactory
|
|
prompt: |
|
|
I have an HTTP-based C# MCP server built with ASP.NET Core.
|
|
How do I write integration tests that test the full MCP protocol
|
|
over HTTP without spinning up a real server?
|
|
graders:
|
|
- type: output-contains
|
|
config:
|
|
substring: WebApplicationFactory
|
|
- type: output-matches
|
|
config:
|
|
pattern: (initialize|MCP|jsonrpc)
|
|
- type: prompt
|
|
rubric:
|
|
- Uses WebApplicationFactory<Program> for in-process HTTP testing
|
|
- Shows how to send an MCP initialize request to verify the server responds
|
|
- Mentions InternalsVisibleTo or making Program public for test access
|
|
- Tests tool invocation through the HTTP endpoint
|
|
- name: Create evaluations for an MCP server
|
|
prompt: |
|
|
I have a C# MCP server that provides tools for querying a product catalog.
|
|
I want to create evaluations to measure how well an LLM uses the tools.
|
|
What format should I use and what makes a good evaluation question?
|
|
graders:
|
|
- type: output-matches
|
|
config:
|
|
pattern: (evaluation|qa_pair|question.*answer)
|
|
- type: output-matches
|
|
config:
|
|
pattern: (read.only|non.destructive|deterministic|verifiable)
|
|
- type: prompt
|
|
rubric:
|
|
- Shows the XML evaluation format with qa_pair elements
|
|
- Explains that questions should require multiple tool calls and reasoning
|
|
- Emphasizes that questions must be read-only with deterministic answers
|
|
- Provides example evaluation questions appropriate for a product catalog
|