19b885551e
Revert Vally CLI 0.16 upgrade
...
Restore Vally CLI 0.14.0 because 0.16.0 rejects legacy config, executionShard metadata, and tool constraints used by most repository evals.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-16 16:30:19 -07:00
d8ebaed63a
Bump @microsoft/vally-cli from 0.14.0 to 0.16.0 in /eng/evaluation-tools
...
Bumps @microsoft/vally-cli from 0.14.0 to 0.16.0.
---
updated-dependencies:
- dependency-name: "@microsoft/vally-cli"
dependency-version: 0.16.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
2026-09-16 05:44:10 +00:00
4c72b17fa2
Merge pull request #1154 from dotnet/abhitejjohn-agentic-workflow-repair
...
fix(ci): use Sol for triage and Sonnet 5 for PR evaluations
2026-09-11 17:01:18 -07:00
b8083dd46b
fix(ci): cover base-branch SDK startup compatibility
...
Validate the same startup guard against SDK 1.0.11 used by the PR merge checks and SDK 1.0.13 on this branch. Keep unknown SDK versions fail-closed. Mock the runtime path explicitly in transport-only tests.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-11 15:04:02 -07:00
a226ead6ff
fix(ci): serialize evaluation SDK startup before sessions
...
Guard the pinned SDK's concurrent lazy startup and wait for filesystem-provider readiness before create/resume. Use the trusted launcher for evaluation and comparison without changing model routing, trial concurrency, or result gates.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-11 15:00:58 -07:00
c26f32a702
Pin Copilot SDK for Vally concurrency
...
Restore the Vally 0.14.0 lock graph that resolves @github/copilot-sdk 1.0.11 and avoids concurrent session lifecycle failures.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-09 17:25:14 -07:00
65e48c9b3f
Pin Vally CLI to 0.14.0
...
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
2026-09-09 10:11:02 -07:00
af059f7469
Bump @microsoft/vally-cli from 0.14.0 to 0.15.0 in /eng/evaluation-tools
...
Bumps @microsoft/vally-cli from 0.14.0 to 0.15.0.
---
updated-dependencies:
- dependency-name: "@microsoft/vally-cli"
dependency-version: 0.15.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
2026-09-09 05:45:24 +00:00
3774d08eb3
Bump @microsoft/vally-cli from 0.13.0 to 0.14.0 in /eng/evaluation-tools
...
Bumps @microsoft/vally-cli from 0.13.0 to 0.14.0.
---
updated-dependencies:
- dependency-name: "@microsoft/vally-cli"
dependency-version: 0.14.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
2026-08-26 05:44:37 +00:00
e156bb30c9
Bump @microsoft/vally-cli from 0.9.0 to 0.13.0 in /eng/evaluation-tools
...
Bumps @microsoft/vally-cli from 0.9.0 to 0.13.0.
---
updated-dependencies:
- dependency-name: "@microsoft/vally-cli"
dependency-version: 0.13.0
dependency-type: direct:production
update-type: version-update:semver-minor
...
Signed-off-by: dependabot[bot] <support@github.com >
2026-08-17 19:47:21 +00:00
ca093a06ef
Automate Vally dependency updates ( #1010 )
...
* Automate Vally dependency updates
Move the evaluation CLI pins into a trusted npm manifest and configure weekly Dependabot updates for Vally.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: d4443741-2ca2-47b3-a2db-0f91e937dbf9
* Smoke test evaluation tool updates
Run a secretless install and CLI smoke test whenever the evaluation tool manifest changes.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: d4443741-2ca2-47b3-a2db-0f91e937dbf9
* Generate evaluation tools lockfile in CI
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: 8779074c-8123-4bfc-98ff-6a9c2da72cf0
* Make evaluation tool installs reproducible
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com >
Copilot-Session: 8779074c-8123-4bfc-98ff-6a9c2da72cf0
---------
Copilot-Session: d4443741-2ca2-47b3-a2db-0f91e937dbf9
Copilot-Session: 8779074c-8123-4bfc-98ff-6a9c2da72cf0
2026-08-17 21:46:24 +02:00
Abhitej John