dotnet__skills/tests/dotnet-ai/mcp-csharp-publish/eval.vally.yaml

name: mcp-csharp-publish
description: Evaluates the dotnet-ai/mcp-csharp-publish skill
type: capability
config:
  timeout: 2m
stimuli:
  - name: Publish an MCP server as a NuGet tool package
    prompt: |
      I have a C# MCP server using stdio transport at src/MyMcpServer/.
      I want to publish it to NuGet.org so users can install it as a
      dotnet tool. What do I need to configure?
    graders:
      - type: output-contains
        config:
          substring: PackAsTool
      - type: output-contains
        config:
          substring: dotnet pack
      - type: output-matches
        config:
          pattern: (dotnet nuget push|nuget\.org)
      - type: prompt
    rubric:
      - Configures PackAsTool and ToolCommandName in the csproj
      - Shows how to build, pack, and push to NuGet.org
      - Includes local testing with dotnet tool install --global
      - Mentions the dnx tool runner for MCP client configuration
  - name: Deploy an HTTP MCP server to Azure Container Apps
    prompt: |
      I have an HTTP-based C# MCP server that I want to deploy to Azure.
      It uses environment variables for API keys. What's the best way
      to containerize and deploy it?
    graders:
      - type: output-matches
        config:
          pattern: (Dockerfile|docker)
      - type: output-matches
        config:
          pattern: (Container Apps|containerapp)
      - type: output-matches
        config:
          pattern: (secret|Secret|KEY)
      - type: prompt
    rubric:
      - Provides a multi-stage Dockerfile with a non-root user
      - Shows Azure Container Apps deployment commands
      - Configures secrets using Container Apps secrets, not environment variables in plain text
      - Includes a health check endpoint
  - name: Publish to the MCP Registry
    prompt: |
      I already published my C# MCP server to NuGet.org. Now I want to
      register it in the official MCP Registry so it's discoverable.
      How do I do this?
    graders:
      - type: output-contains
        config:
          substring: server.json
      - type: output-contains
        config:
          substring: mcp-publisher
      - type: output-matches
        config:
          pattern: io\.github\.
      - type: prompt
    rubric:
      - Shows the server.json schema with the correct format
      - Uses the mcp-publisher CLI for publishing
      - Explains the naming convention (io.github.<username>/<server>)
      - Emphasizes version consistency between csproj and server.json