dotnet__skills/tests/dotnet/csharp-refactoring/eval.vally.yaml

name: csharp-refactoring
description: Held-out generalization probe for the dotnet/csharp-refactoring skill (fresh scenarios, not used during v1/v2/v3 tuning)
type: capability
config:
  timeout: 10m
stimuli:
  - name: HELDOUT Introduce named constants for inline magic numbers
    prompt: "In the Billing class library, `OrderProcessor.DoStuff` hard-codes the tax rate `0.08m`, the free-shipping threshold `100m`, and the flat shipping charge `9.99m` inline. Pull these three literals out into clearly named constants and use them, without changing what the method computes. The solution is at Fixture.sln."
    environment:
      files:
        - src: src
          dest: src
        - src: tests
          dest: tests
        - src: Fixture.sln
          dest: Fixture.sln
    graders:
      - type: exit-success
      - type: output-matches
        config:
          pattern: (dotnet (build|test)|rebuild|re-?run.{0,20}tests|tests? .{0,20}(pass|green)|const)
      - type: prompt
    rubric:
      - Replaces the inline literals with named constants (or an equivalent named form) while keeping every computed value identical
      - Confirms behavior is unchanged by building and running the existing tests after the edit, which must pass
      - Keeps this a single focused change without altering the tax, discount, or shipping results

  - name: HELDOUT Simplify a nested conditional without changing the mapping
    prompt: "In `OrderProcessor.DoStuff`, the per-tier discount rate is selected with a nested conditional expression (`tier == \"gold\" ? ... : tier == \"silver\" ? ... : ...`). Rewrite that tier-to-rate selection in a clearer form (for example a switch) so it reads better, without changing the rate any tier receives. The solution is at Fixture.sln."
    environment:
      files:
        - src: src
          dest: src
        - src: tests
          dest: tests
        - src: Fixture.sln
          dest: Fixture.sln
    graders:
      - type: exit-success
      - type: output-matches
        config:
          pattern: (dotnet (build|test)|rebuild|re-?run.{0,20}tests|tests? .{0,20}(pass|green)|switch)
      - type: prompt
    rubric:
      - Rewrites the tier-to-rate selection in a clearer form while preserving the exact rate for gold, silver, and every other tier
      - Verifies behavior is preserved by rebuilding and re-running the existing tests, which must pass
      - Does not change the discount, tax, or shipping results for any tier

  - name: HELDOUT Rename unclear local variables for readability
    prompt: "In `OrderProcessor.DoStuff`, the local variables `rateA`, `discountedA`, `rateB`, and `discountedB` are unclear. Rename them to describe what they hold (the discount rate and discounted amount, for the order total and for the quote respectively), without changing behavior. The solution is at Fixture.sln."
    environment:
      files:
        - src: src
          dest: src
        - src: tests
          dest: tests
        - src: Fixture.sln
          dest: Fixture.sln
    graders:
      - type: exit-success
      - type: output-matches
        config:
          pattern: (dotnet (build|test)|rebuild|re-?run.{0,20}tests|tests? .{0,20}(pass|green)|rename)
      - type: prompt
    rubric:
      - Renames the local variables to clearer names while leaving the computation and results identical
      - Confirms via a build and the existing tests (which must pass) that behavior is unchanged
      - Treats a local-variable rename of this blast radius proportionally, without demanding a redundant separate pre-change baseline run

  - name: HELDOUT Recognize a decimal-to-double change alters observable results
    prompt: "The money math in `OrderProcessor.DoStuff` uses `decimal`, which is slower than `double`. As a performance cleanup, switch the calculations to `double` to speed it up. The solution is at Fixture.sln."
    environment:
      files:
        - src: src
          dest: src
        - src: tests
          dest: tests
        - src: Fixture.sln
          dest: Fixture.sln
    graders:
      - type: output-matches
        config:
          pattern: (behavior[- ]?(chang|preserving)|not (a )?refactor|round|double|decimal|observable|separate|decline|precision)
      - type: prompt
    rubric:
      - Recognizes that switching monetary math from decimal to double changes rounding and therefore observable results, so it is not a behavior-preserving refactor
      - Does not silently perform the change under a cleanup or performance framing that claims behavior is preserved
      - Declines, flags it as a behavior change needing its own tests and sign-off, or proceeds only as an explicit tested behavior change

  - name: HELDOUT Recognize renaming a shipped public API is a breaking change
    prompt: "Rename the public method `AppSettingsHelper.ParseIntSetting` to `ParseInt` everywhere, to match our newer naming. The solution is at Fixture.sln."
    environment:
      files:
        - src: src
          dest: src
        - src: tests
          dest: tests
        - src: Fixture.sln
          dest: Fixture.sln
    graders:
      - type: output-matches
        config:
          pattern: (public (api|surface|method)|breaking change|shipped|PublicAPI|not (a )?(internal )?refactor|baseline|consumer|behavior[- ]?(chang|preserving))
      - type: prompt
    rubric:
      - Recognizes that ParseIntSetting is part of the shipped public API surface (it is listed in PublicAPI.Shipped.txt) so renaming it is a breaking change, not a purely internal refactor
      - Flags the public-surface or breaking-change implication (for example updating the public API baseline or deferring to the breaking-change process) rather than treating it as an ordinary internal rename
      - Does not silently rename a shipped public member while claiming the contract is preserved