mirror of
https://github.com/dotnet/skills.git
synced 2026-09-20 09:49:54 +08:00
Abhitej John
0723ac2b44
Run every skill against a 3-model matrix (Opus 4.8, GPT 5.5, Sonnet 4.6) with a cross-family judge that is never the executing model (default judge = latest Opus). Adds eval-models.json as the single source of truth, resolve-judge.mjs to derive model/judge pairs, preflight-models.mjs and a ListModels command for availability checks (without exposing the full model roster), and publish-vally-data.mjs for carrying run history to skills-data. - eng/eval-models.json: model matrix + judge map (Sonnet 4.6) - eng/resolve-judge.mjs: judge/matrix/required/selftest resolver - eng/preflight-models.mjs + ListModelsCommand.cs: privacy-safe preflight - vally-evaluation.yml: multi-model fan-out; redirect verbose trajectories to per-eval vally.log + tail-on-failure; 8-token selection pool parity - .NET CLI defaults + README bumped claude-opus-4.6 -> claude-opus-4.8 - vally-adapter: adapt.mjs/run-vally-evals.sh default Sonnet 4.6 - dashboard + InvestigatingResults.md updates Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>