Extract skills/<name> from a git ref and run the same prompt on the CLIs already on PATH: claude, codex, grok. Bills those products (whatever you already use to run the harness). No extra Anthropic/OpenAI API key. No Vercel AI SDK.

Default hosts are the other two from the calling harness (Claude Code → Codex+Grok, Codex → Claude+Grok, Grok → Claude+Codex). --hosts overrides that. A missing CLI prints warning: skipping <host>: … and the run continues with whatever is left. If no peers are installed it falls back to the current harness and prints warning: own-eval only …. Exit 2 only when nothing can run. summary.json records current_harness, hosts_wanted, hosts_run, own_eval_only, and warnings. Not in default bun test except mechanical pins (hosts.test.ts, extract.test.ts, path-shim.test.ts). Cursor is not a default host (cursor-agent -p hangs without prior trust).

Run

bun run test:skill-eval-cell -- \
  --skill ce-debug \
  --fixture tests/skill-eval-cell/fixtures/seat-cap \
  --git-init \
  --shim-git-push \
  --task "mode:pipeline the seat cap test is failing. Run node tests/seat-cap.check.js."

--git-remote (catalog: git_remote: true) adds a fake origin whose main is the seed commit, so a shipping tail takes the push/PR path — where --shim-git-push then fails — instead of the local-commit path it takes when no remote exists.

--read-only enforces the fake boundary, it does not merely suggest it: Codex drops --dangerously-bypass-approvals-and-sandbox and runs --sandbox read-only (the two contradict each other), and Claude pairs --allowedTools Read,Glob,Grep with a --disallowedTools list that also names Task,Skill,WebFetch,WebSearch,NotebookEdit — under --dangerously-skip-permissions those stay callable, so allow-listing alone leaves the boundary open.

Prints a summary.json path. Each host gets its own workspace copy plus stdout/stderr, git status/log, and a file list. PATH shims live beside that workspace, never inside it, so the skill under test never sees harness files as its own dirty tree. Grade those; Grok narrates before the answer (grep FILES_READ:). Codex transcript is stderr, final message is stdout. claude -p is one-tick only.

Each invocation requires a new or empty --out directory and records input and evidence fingerprints. Packs freeze their scenario criteria and grader hashes. Regrading applies current criteria by default; --mode original reproduces the recorded assessment. Both write separate reports and preserve original grades. See reproducible evaluation evidence for regrading commands, partial collection outcomes, legacy-pack compatibility, and snapshot limits.

Gotchas baked in (see docs/solutions/skill-design/size-driven-skill-restructure.md): Codex stdin /dev/null, CLAUDECODE unset, NO_COLOR=1.

Sweep A/B pack

Cases live in catalog.ts, authored from the skill bodies before the 8KB merges (PRE_SWEEP_REF = parent of #1433). The same prompt runs against that ref, then against the working tree (POST_SWEEP_REF = the WORKTREE sentinel, the default --ref). git archive only ever sees committed content, so the post arm copies skills/<name> off disk — that is what lets you grade a skill edit before committing it. Pass a real git ref to --ref for a committed arm. See scenarios.md for the inventory.

bun run test:skill-eval-pack -- --list
bun run test:skill-eval-pack -- --wave1 --arm ab
bun run test:skill-eval-pack -- --id ce-babysit-pr/refuse-unasked-update --arm ab
bun run test:skill-eval-pack -- --id lfg/plan-first --arm ab

--arm ab is pre+post for every catalog skill (the 8KB sweep is fully merged). --wave1 is the cheap read-only decision set, not every scenario. Live mutation and oracle dispatch are separate ids. The pack exits non-zero when any arm failed, after writing pack.json, so it can be used as a check. ok is the only verdict: a listed files_read_post miss fails the cell; unlisted references are not graded. Not in default bun test.