The Codex CLI defaults both `model_reasoning_effort` and `model_verbosity`
to "low", but `gpt-5.2-codex` (the default Codex model) only accepts
"medium" for both. Out-of-the-box `codex exec` against the AI Gateway
fails with:
Unsupported value: 'low' is not supported with the 'gpt-5.2-codex'
model. Supported values are: 'medium'.
The error covers both the `reasoning.effort` and `text.verbosity`
request parameters, depending on which the model rejects first.
Set both fields to "medium" in two places:
- the generated profile config in ~/.codex/default.config.toml
- explicit -c flags on `codex exec`, since CLI flags have the highest
precedence and we observed the profile-only setting being silently
overridden by the CLI's "low" default in some Codex versions.
`generateCodexConfig` now accepts an optional `reasoningEffort`
parameter so callers can override per-run via
`model: "gpt-5.2-codex?reasoningEffort=high"`.
Verified end-to-end against the Vercel AI Gateway: a previously-failing
`codex exec` smoke run now completes in ~31s and returns a real
response instead of erroring at `turn.failed`.
Also added `vercel-agent-eval-*.tgz` to .gitignore so local `npm pack`
artifacts don't leak into commits.
Allows experiment config files to pass agent-specific options (like
binaryUrl and extraProviders) at runtime via a new agentOptions field.
Previously these could only be set at agent registration time, making
configs for unreleased models non-replicable.
The options flow: ExperimentConfig → runner → AgentRunOptions → agent.run().
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Use built-in OpenAI provider for Codex
* Strip openai/ prefix from --model CLI flag for direct API path
The config.toml correctly strips the provider prefix for direct OpenAI API
usage, but the --model CLI flag still passed the prefixed name (e.g.
"openai/gpt-5.2-codex"), causing a "model not found" error.
* feat: support CLAUDE_CODE_OAUTH_TOKEN for Claude Code agent
Allow Claude Pro/Max subscribers to authenticate using their OAuth token
instead of requiring a separate ANTHROPIC_API_KEY. When CLAUDE_CODE_OAUTH_TOKEN
is set in the environment, it takes precedence over ANTHROPIC_API_KEY for
non-AI-Gateway configurations.
Closes#54
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* refactor: address PR review comments for OAuth token support
Refactor nested ternary for sandbox env to if/else for readability,
add clarifying comment about credential consistency, and add unit
tests for getApiKeyEnvVar() precedence (gateway > oauth > direct).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
* [Classifier] Upgrade classification model to Claude Sonnet 4.6
* [Classifier] Parallelize classification with p-limit and add dashboard progress
Run up to 4 classifications concurrently using p-limit instead of a
sequential for-loop. The dashboard now shows "classifying N/M…" so
users can track per-eval progress during the classification phase.
* [Classifier] Switch to Haiku 4.5 for classification
Haiku is faster and cheaper while still capable enough for the
classification task, especially with 4 concurrent classifications.
github-actions[bot]