Commit Graph

5 Commits

Author SHA1 Message Date
Jordan Ritter 25c7fa368b fix(showcase): add --ci flag to eval workflow command
The eval workflow was missing --ci, causing it to try Docker Compose
lifecycle in CI (which fails because there's no .env file). The --ci
flag skips Docker and assumes services are already running or uses
native execution.
2026-04-30 15:58:42 -07:00
Jordan Ritter 1a37ca36c7 feat(showcase): Check Run button UX for eval trigger
New showcase_eval_check.yml creates a Check Run with "Run Showcase
Eval" action button on every PR. Modified showcase_eval.yml adds
workflow_dispatch trigger with dispatch-gate job, Check Run update
in post-result, and devops bot token for Checks API calls.
2026-04-30 11:13:13 -07:00
Jordan Ritter db62915362 fix(showcase): restore eval system accidentally deleted by #4449
PR #4449 removed the eval tier system while unaware #4448 had just
merged. Restoring orchestrator, matrix, scope, config, GHA workflow,
and CLI wiring with all 8 CR fixes intact.
2026-04-29 20:11:33 -07:00
Jordan Ritter 5abf0409a8 fix(showcase): D5 test infrastructure and CLI improvements
- Rewrite cmd-test.sh to use harness CLI for D5 probes
- Add run-e2e-with-aimock.sh helper (removed stale version)
- Update fail-baseline hash for D5 coverage
- Fix provider base URLs in .env.example
- Update local-ports.json for spring-ai
- Remove eval tier system (superseded by D5 depth probes)
- Add D5 debugging runbook
2026-04-29 19:40:10 -07:00
Jordan Ritter bcb8dda1e4 feat(showcase): add /eval PR comment-triggered GitHub Actions workflow
Comment-triggered CI workflow for per-PR evaluation:
- Permission gate (write collaborator only)
- /eval d5 [slug,...] comment parsing with slug validation
- Depot 16-core runner with full git history
- Posts running/result comments on the PR
- Scope detection, baseline comparison, JSON output
- 45-minute timeout with configurable parallel runners
2026-04-29 18:48:03 -07:00