Commit Graph

9 Commits

Author SHA1 Message Date
Abhitej John adae199c67 dbc breadth study: point trimmed arm at dotnet-breaking-changes-v3 2026-07-21 00:40:38 -07:00
Abhitej John 63b4c11073 v3 grid (opus48): ultra-lean judgment-first skill; trimmed arm -> csharp-refactoring-v3 2026-07-20 02:35:13 -07:00
Abhitej John aa08a4a2c6 Breadth grid: executor=claude-opus-4.8 n=5 (baseline vs current-skill vs trimmed)
Eval-branch only; not on PR #873.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-19 23:47:59 -07:00
Abhitej John 5429c2f621 Add 2 fresh mixed-intent boundary fixtures; revert executor to opus-4.6
Closes the 'refactoring-only endpoint too narrow' risk flagged by the
cross-family rubber-duck. Adds two boundary stimuli never used in trim tuning:
  9. Separate a smuggled behavior change from a requested rename (rename +
     smuggled 10%->12% discount change) -> should separate/flag, not bundle.
 10. Recognize a behavior-changing simplification is not a refactor (remove
     free-shipping-over-\ rule under a 'cleanup' framing) -> should not
     silently change observable behavior.
Executor reverted to claude-opus-4.6 / judge gpt-5.5 (re-judge) to match the
stringent Confirm A family where boundary behavior looked weakest.
Eval-branch only; not on PR #873.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-19 23:24:41 -07:00
Abhitej John bfe4075469 Hardening run: cross-family executor (gpt-5.5) + Claude judge
Robustness probe for the csharp-refactoring trim: all prior CI trajectories
were executed by claude-opus-4.6. Flip executor to gpt-5.5 (GPT family) and
CI judge to claude-opus-4.8 so the (executor, judge) pair stays cross-family.
Tests whether trimmed-vs-current non-inferiority holds under a different model
family. Eval-branch only; not on PR #873.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-17 16:14:41 -07:00
Abhitej John bd1bc5a920 Confirmation run: n=10 (freeze v2 skill; pre-registered analysis)
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-17 13:29:05 -07:00
Abhitej John a10a7ae714 Trim v2: clarify bug-fix/feature guidance; n=5 experiment
Rewrite the 'not behavior-preserving' escalate bullet so an explicitly
requested bug fix or feature is completed via normal development (not
refused / 'nothing to change'), addressing the one scenario where v1
regressed. Bump experiment runs 3->5 to tighten the aggregate CI.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-17 12:01:41 -07:00
Abhitej John 99dcf9edeb Add csharp-refactoring-trim variant + 3-arm experiment (eval study)
Trim study for PR #873: csharp-refactoring-trim is a ~30% shorter rewrite of csharp-refactoring (drops When-to-use/Inputs/Outputs boilerplate + verbose LSP launch prose, compresses the dbc-overlapping hazards section) while preserving every rubric-rewarded behavior. Adds a 'trimmed' experiment arm and bumps runs to 3 so CI generates baseline/skilled/trimmed trajectories for local cross-family re-judging. Eval-only scratch branch; not part of the PR.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot-Session: 3c9f9823-7f2f-4f7d-9d1b-f2b9e7a20c60
2026-07-17 11:03:01 -07:00
Aditya Mandaleeka 52ba152ed6 Update to Vally 0.7.0 (#854) 2026-07-10 02:19:48 +00:00