Files
Amaury Levé 1270722c95 Fix migrate-static-to-wrapper skill activation on evals (#864)
Sharpen the description boundary between migrate-static-to-wrapper and
generate-testability-wrappers so prompts that replace call sites when the
abstraction is already registered in DI activate the migration skill.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
2026-07-07 14:18:11 +02:00

218 lines
9.3 KiB
YAML

scenarios:
- name: "Migrate DateTime.UtcNow to TimeProvider in a service class"
prompt: |
I have a CouponService that uses DateTime.UtcNow and DateTime.Now in
several places. TimeProvider is already registered in my DI container.
Can you replace all the DateTime calls with TimeProvider and add the
constructor parameter?
setup:
copy_test_files: true
assertions:
- type: "file_contains"
path: "**/CouponService.cs"
value: "TimeProvider"
- type: "file_not_contains"
path: "**/CouponService.cs"
value: "DateTime.UtcNow"
- type: "file_not_contains"
path: "**/CouponService.cs"
value: "DateTime.Now"
- type: "exit_success"
rubric:
- "Replaced all DateTime.Now and DateTime.UtcNow calls in CouponService.cs with TimeProvider calls"
- "Preserved local-vs-UTC time semantics — the local-time call (DateTime.Now) maps to a local TimeProvider call and the UTC calls map to a UTC call, rather than conflating them (which would silently change IsRedeemableToday's behavior across time zones)"
- "Added TimeProvider as a constructor parameter to CouponService"
- "Stored TimeProvider in a readonly field following the class's existing naming convention"
- "Added any required `using` directives so the migrated file still compiles"
- "Left the project in a state where the build still succeeds after the migration"
- "Introduced no behavioral changes — the migrated code delegates directly to the injected TimeProvider in place of the static DateTime.UtcNow"
- "Did not break the original static class or alter it in a way that breaks existing callers"
- "Did not modify Program.cs DI registration since TimeProvider was already registered"
timeout: 180
- name: "Migrate only in scoped files, leaving others untouched"
prompt: |
I want to migrate DateTime.Now and DateTime.UtcNow to TimeProvider, but
only in CouponService.cs for now. Don't touch AuditLogger.cs yet — I'll do
that in a separate pass. TimeProvider is already in my DI container.
setup:
copy_test_files: true
assertions:
- type: "file_contains"
path: "**/CouponService.cs"
value: "TimeProvider"
- type: "file_not_contains"
path: "**/CouponService.cs"
value: "DateTime.UtcNow"
- type: "file_not_contains"
path: "**/CouponService.cs"
value: "DateTime.Now"
- type: "file_contains"
path: "**/AuditLogger.cs"
value: "DateTime.UtcNow"
- type: "exit_success"
rubric:
- "Migrated DateTime.Now and DateTime.UtcNow to TimeProvider in CouponService.cs"
- "Preserved local-vs-UTC time semantics — the local-time call (DateTime.Now) maps to a local TimeProvider call and the UTC calls map to a UTC call, rather than conflating them"
- "Left AuditLogger.cs completely untouched with DateTime.UtcNow calls still present"
- "Migrated incrementally rather than too much at once, keeping the change to the single file the user scoped"
- "Respected the scope boundary the user specified"
- "Reported what was migrated and what remains to be done"
timeout: 180
- name: "Update test doubles when migrating a service to TimeProvider"
prompt: |
Please migrate DateTime.UtcNow to TimeProvider in my InvoiceService, and
update the unit tests so they use a controllable time source and stay
deterministic. TimeProvider is already registered in my DI container.
setup:
files:
- path: "BillingApp/BillingApp.csproj"
content: |
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
</Project>
- path: "BillingApp/InvoiceService.cs"
content: |
namespace BillingApp;
public class InvoiceService
{
public Invoice Create(decimal amount)
{
return new Invoice
{
Amount = amount,
IssuedAt = DateTime.UtcNow
};
}
}
public class Invoice
{
public decimal Amount { get; set; }
public DateTimeOffset IssuedAt { get; set; }
}
- path: "BillingApp.Tests/BillingApp.Tests.csproj"
content: |
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.11.1" />
<PackageReference Include="xunit" Version="2.9.2" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.8.2" />
<PackageReference Include="Microsoft.Extensions.TimeProvider.Testing" Version="9.0.0" />
</ItemGroup>
<ItemGroup>
<ProjectReference Include="..\BillingApp\BillingApp.csproj" />
</ItemGroup>
</Project>
- path: "BillingApp.Tests/InvoiceServiceTests.cs"
content: |
using Xunit;
using BillingApp;
namespace BillingApp.Tests;
public class InvoiceServiceTests
{
[Fact]
public void Create_SetsIssuedAt()
{
var service = new InvoiceService();
var invoice = service.Create(100m);
Assert.True(invoice.IssuedAt <= DateTimeOffset.UtcNow);
}
}
assertions:
- type: "file_contains"
path: "**/InvoiceService.cs"
value: "TimeProvider"
- type: "file_not_contains"
path: "**/InvoiceService.cs"
value: "DateTime.UtcNow"
- type: "file_contains"
path: "**/InvoiceServiceTests.cs"
value: "TimeProvider"
- type: "exit_success"
rubric:
- "Updated the affected test files with an appropriate test double for the time source (such as FakeTimeProvider) instead of relying on the real clock"
- "Kept the migrated production code free of behavioral changes — it delegates directly to the injected TimeProvider rather than the static DateTime.UtcNow"
- "Left both the production and test projects in a state where the build still succeeds after the migration"
- "Added any required `using` directives to the files it changed"
timeout: 240
- name: "Handle a genuinely static class that cannot take constructor injection"
prompt: |
I want to migrate DateTime.UtcNow to TimeProvider in my TimestampFormatter
helper. It's a static class with only static members. TimeProvider is
registered in DI. How should I do this?
setup:
files:
- path: "Utils/Utils.csproj"
content: |
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
<Nullable>enable</Nullable>
<ImplicitUsings>enable</ImplicitUsings>
</PropertyGroup>
</Project>
- path: "Utils/TimestampFormatter.cs"
content: |
namespace Utils;
public static class TimestampFormatter
{
public static string Now() => DateTime.UtcNow.ToString("O");
}
assertions:
- type: "output_matches"
pattern: "(static class|ambient|constructor|TimeProvider)"
- type: "exit_success"
rubric:
- "Recognized that the static class cannot receive constructor injection without breaking it"
- "Used an ambient context approach for the time source instead of adding a constructor to the static class"
- "Did not produce code that fails to compile by adding instance constructors or fields to a static class"
timeout: 180
- name: "Decline migration when wrapper does not exist yet"
prompt: |
Replace all File.ReadAllText calls in my project with IFileSystem. Just do
the bulk replacement across all files.
expect_activation: false
setup:
files:
- path: "MyProject/MyProject.csproj"
content: |
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>net10.0</TargetFramework>
</PropertyGroup>
</Project>
- path: "MyProject/DataLoader.cs"
content: |
namespace MyProject;
public class DataLoader
{
public string Load(string path) => File.ReadAllText(path);
}
assertions:
- type: "output_matches"
pattern: "(System\\.IO\\.Abstractions|package|install|not.*registered|not.*referenced|add.*package)"
rubric:
- "Identified that System.IO.Abstractions NuGet package is not referenced in the project"
- "Did not blindly replace File.ReadAllText with IFileSystem calls that would not compile"
- "Recommended installing the package and registering IFileSystem before migrating"
timeout: 180