mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Amaury Levé
04348a46ef
The grader expected coverage.cobertura.xml under a workspace-root TestResults/ directory, but `dotnet test --collect:"XPlat Code Coverage"` writes the report under <TestProject>/TestResults/<guid>/coverage.cobertura.xml. With the previous path, `find TestResults …` failed with no-such-directory before even searching, and the glob `TestResults/**/coverage.cobertura.xml` could not match the nested location either, so the assertion failed on otherwise-successful runs. Search recursively from cwd in both eval.vally.yaml (`find . …`) and eval.yaml (`**/TestResults/**/coverage.cobertura.xml`) so the assertion matches the actual output location of XPlat Code Coverage regardless of the test project layout the agent chose. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
42 lines
2.4 KiB
YAML
42 lines
2.4 KiB
YAML
scenarios:
|
|
# ============================================================================
|
|
# Goal: Generate comprehensive tests for a real-world ASP.NET Core MVC app
|
|
# ContosoUniversity is a non-trivial web app with EF Core (SqlServer), TPH
|
|
# inheritance, multiple CRUD controllers, a notification service, pagination,
|
|
# and a complex DbContext. The agent must figure out how to set up an in-memory
|
|
# or SQLite test database, mock/arrange controller dependencies, and cover
|
|
# controllers + services + data layer logic.
|
|
# ============================================================================
|
|
|
|
- name: "Generate tests for ContosoUniversity ASP.NET Core MVC app"
|
|
prompt: |
|
|
I have an ASP.NET Core MVC application under ContosoUniversity/ — a university
|
|
enrollment system with multiple controllers, EF Core data access, and a
|
|
notification service. This is a project-wide, multi-file test generation task.
|
|
|
|
Please scaffold a new test project, then write comprehensive unit tests
|
|
across the controllers (Students, Courses, Departments, Instructors), the
|
|
services, and the data layer. Achieve high code coverage. The generated
|
|
tests should compile and pass, and the test project should be configured so
|
|
`dotnet test --collect:"XPlat Code Coverage"` produces a Cobertura XML
|
|
report under TestResults/ (add `coverlet.collector` to the test csproj if
|
|
it is not already there).
|
|
setup:
|
|
copy_test_files: true
|
|
assertions:
|
|
# The generated tests must compile, pass, and produce a coverage report
|
|
- type: run_command_and_assert
|
|
command_to_run: "dotnet"
|
|
command_arguments: "test ContosoUniversity.Tests/ContosoUniversity.Tests.csproj -v:q --collect:\"XPlat Code Coverage\""
|
|
expected_exit_code: 0
|
|
expected_std_output_contains: "Passed!"
|
|
command_timeout: 300
|
|
- type: file_exists
|
|
path: "**/TestResults/**/coverage.cobertura.xml"
|
|
rubric:
|
|
- "Achieved high line coverage (>70%) across the ContosoUniversity source files as reported by the Cobertura XML in TestResults/"
|
|
- "Tests are organized logically with clear naming that describes test scenarios"
|
|
- "Generated tests that cover multiple controllers (Students, Courses, Departments, Instructors)"
|
|
- "Set up the EF Core SchoolContext correctly for testing without a real SQL Server (e.g., InMemory provider or SQLite)"
|
|
timeout: 3600
|