Replace the placeholder in CONTRIBUTING.md with the actual contributing
instructions from README, and link to CONTRIBUTING.md from README.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Set up contributor-assistant GitHub Action to require CLA signatures
on pull requests, matching the wandb/weave CLA setup.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Update all references from wandb/agent-skills to wandb/skills in
README and CI workflow to reflect the repo rename.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Python blocks run with cwd=wbaf/ (via uv --directory wbaf), so
paths must not include the wbaf/ prefix. Shell keeps the prefix.
- Use WBAF_PAT for update-readme checkout to bypass branch protection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
uv run --directory wbaf sets cwd to wbaf/, so Python paths must
not include the wbaf/ prefix. Shell commands (mkdir, echo fallback)
run from workspace root and keep the prefix.
Verified end-to-end in a tempdir simulating the CI checkout layout.
The printf command with single quotes and > redirect was being
parsed as YAML syntax rather than a shell command. Switch to
block scalar (|) to avoid ambiguity.
* Fix eval workflow: target main, add Claude Code agent matrix
- ref: wbaf-refact → ref: main
- Matrix strategy: benchmark both codex-mcp and claude-code
- Add ANTHROPIC_API_KEY for Claude Code agent
- Use all.yaml instead of codex-all.yaml (which no longer exists)
- Unique temp agent per matrix cell to avoid config collisions
- Separate README update job that aggregates results across agents
* Add install.sh, replace wandb skill with wandb-primary, update CI
- Add install.sh for easy skill installation (local or global Claude Code)
- Replace skills/wandb with skills/wandb-primary (comprehensive W&B + Weave skill
with helper scripts and reference docs)
- Update README with installation instructions (npx skills + install script)
- Fix CI matrix: use codex and claude-code as base agents (not codex-mcp)
- Fix all wandb-primary-v2 path references to wandb-primary
- Remove stale .badges directory
* Overhaul CI: per-agent badges, wba-all eval source, no task name leaks
- Switch eval source from all.yaml to wba-all.yaml (W&B-specific tasks)
- Run update_all_eval.py before each eval to pick up new tasks
- Replace raw eval-log artifact with results-summary.json (counts only)
- Generate per-agent badge JSON files (.badges/codex.json, .badges/claude-code.json)
- Add Codex and Claude Code shield badges to README
- Keep eval_tasks input override for manual dispatch
- No task names, rubric content, or eval details leak to public artifacts
-- AI was used to create large parts of this PR
Parses the eval summary from the log output to extract:
- passed count / total tasks
- score percentage from correctness or rubric scorer
Badge now shows e.g. "13/24 passing (54%)" instead of just "passing".
Per-agent badges will come later when WBAF supports multi-agent eval reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The workflow no longer depends on pre-existing agent configs in WBAF.
Instead it:
1. Copies skills from this repo into WBAF's skills/
2. Writes a temp agent config (ci-skill-eval) extending codex-mcp with those skills
3. Writes a temp eval config pointing at that agent
4. Runs the eval
Inputs:
- skills: comma-separated list (blank = all)
- base_agent: agent to extend (default: codex-mcp)
- eval_tasks: task source (default: all from codex-all.yaml)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Workflow status badge at top of README
- Skill table with markers (<!-- BEGIN/END SKILL TABLE -->) auto-updated by CI
- CI reads skill descriptions from frontmatter and writes status into table
- Replaces JSON badge files with direct README updates (works for private repos)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GitHub Actions workflow that runs on push/PR to main (when skills/ changes)
- Clones WandBAgentFactory (wbaf-refact branch) for the eval framework
- Copies skills into the eval framework and runs Codex agent with MCP
- Generates badge data on main merges
- Weekly scheduled run on Sundays at 11 PM PT
- Updated README with skill format docs and contributing guide
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Zubin Aysola