- Switch eval source from all.yaml to wba-all.yaml (W&B-specific tasks)
- Run update_all_eval.py before each eval to pick up new tasks
- Replace raw eval-log artifact with results-summary.json (counts only)
- Generate per-agent badge JSON files (.badges/codex.json, .badges/claude-code.json)
- Add Codex and Claude Code shield badges to README
- Keep eval_tasks input override for manual dispatch
- No task names, rubric content, or eval details leak to public artifacts
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add install.sh for easy skill installation (local or global Claude Code)
- Replace skills/wandb with skills/wandb-primary (comprehensive W&B + Weave skill
with helper scripts and reference docs)
- Update README with installation instructions (npx skills + install script)
- Fix CI matrix: use codex and claude-code as base agents (not codex-mcp)
- Fix all wandb-primary-v2 path references to wandb-primary
- Remove stale .badges directory
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- ref: wbaf-refact → ref: main
- Matrix strategy: benchmark both codex-mcp and claude-code
- Add ANTHROPIC_API_KEY for Claude Code agent
- Use all.yaml instead of codex-all.yaml (which no longer exists)
- Unique temp agent per matrix cell to avoid config collisions
- Separate README update job that aggregates results across agents
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Parses the eval summary from the log output to extract:
- passed count / total tasks
- score percentage from correctness or rubric scorer
Badge now shows e.g. "13/24 passing (54%)" instead of just "passing".
Per-agent badges will come later when WBAF supports multi-agent eval reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The workflow no longer depends on pre-existing agent configs in WBAF.
Instead it:
1. Copies skills from this repo into WBAF's skills/
2. Writes a temp agent config (ci-skill-eval) extending codex-mcp with those skills
3. Writes a temp eval config pointing at that agent
4. Runs the eval
Inputs:
- skills: comma-separated list (blank = all)
- base_agent: agent to extend (default: codex-mcp)
- eval_tasks: task source (default: all from codex-all.yaml)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Workflow status badge at top of README
- Skill table with markers (<!-- BEGIN/END SKILL TABLE -->) auto-updated by CI
- CI reads skill descriptions from frontmatter and writes status into table
- Replaces JSON badge files with direct README updates (works for private repos)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GitHub Actions workflow that runs on push/PR to main (when skills/ changes)
- Clones WandBAgentFactory (wbaf-refact branch) for the eval framework
- Copies skills into the eval framework and runs Codex agent with MCP
- Generates badge data on main merges
- Weekly scheduled run on Sundays at 11 PM PT
- Updated README with skill format docs and contributing guide
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Zubin Aysola