dan 8564d8a7d2 feat(evals): reactor harness evaluation suite (phase 1b) (#102) ...

* feat(cli): add evaluation runner core

* feat(cli): add eval suite loading and cost mapping

* feat(cli): add public harness eval adapters

* feat(cli): add eval suite registry and runner helper

* fix(cli): harden eval runner safety gates

* feat(cli): harden eval evidence and isolation

* feat(cli): add eval claim eligibility gates

* Harden reactor eval claim eligibility

* Harden eval claim eligibility gates

* feat(eval): add reactor timeline case contract

* feat(eval): add reactor proof verdict library

* feat(eval): add timeline baseline adapters

* feat(eval): add docker isolation substrate

* feat(eval): add phase 1b reactor-independent tier 1

* feat(eval): add phase 1b cost and canary lanes

* feat(eval): add phase 1b statistical pilot

* feat(eval): reconcile openrouter costs and containerize competitors

* feat(evals): run isolated competitor harnesses locally

* feat(evals): add phase 1b report pilot runner

* feat(evals): add phase 1b confidence intervals

* fix(evals): harden provider host matching against URL prefix spoofing

CodeQL flagged "Incomplete URL substring sanitization" in
providerFromUrl: host.endsWith("anthropic.com") also matches an
attacker-controlled host like "evilanthropic.com". Match the exact
host or a proper subdomain (".anthropic.com") instead, applied to all
three provider domains via a hostMatches helper.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

2026-05-28 16:17:42 -07:00

435 B

Raw Permalink Blame History

View Raw