Instead of hardcoding uv as the package manager, the skill now instructs
agents to detect the user's Python environment first and write the
finding into the skill itself. Falls back to uv only when the
environment cannot be determined.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace the placeholder in CONTRIBUTING.md with the actual contributing
instructions from README, and link to CONTRIBUTING.md from README.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Set up contributor-assistant GitHub Action to require CLA signatures
on pull requests, matching the wandb/weave CLA setup.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Update all references from wandb/agent-skills to wandb/skills in
README and CI workflow to reflect the repo rename.
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
- Python blocks run with cwd=wbaf/ (via uv --directory wbaf), so
paths must not include the wbaf/ prefix. Shell keeps the prefix.
- Use WBAF_PAT for update-readme checkout to bypass branch protection.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
uv run --directory wbaf sets cwd to wbaf/, so Python paths must
not include the wbaf/ prefix. Shell commands (mkdir, echo fallback)
run from workspace root and keep the prefix.
Verified end-to-end in a tempdir simulating the CI checkout layout.
The printf command with single quotes and > redirect was being
parsed as YAML syntax rather than a shell command. Switch to
block scalar (|) to avoid ambiguity.
* Fix eval workflow: target main, add Claude Code agent matrix
- ref: wbaf-refact → ref: main
- Matrix strategy: benchmark both codex-mcp and claude-code
- Add ANTHROPIC_API_KEY for Claude Code agent
- Use all.yaml instead of codex-all.yaml (which no longer exists)
- Unique temp agent per matrix cell to avoid config collisions
- Separate README update job that aggregates results across agents
* Add install.sh, replace wandb skill with wandb-primary, update CI
- Add install.sh for easy skill installation (local or global Claude Code)
- Replace skills/wandb with skills/wandb-primary (comprehensive W&B + Weave skill
with helper scripts and reference docs)
- Update README with installation instructions (npx skills + install script)
- Fix CI matrix: use codex and claude-code as base agents (not codex-mcp)
- Fix all wandb-primary-v2 path references to wandb-primary
- Remove stale .badges directory
* Overhaul CI: per-agent badges, wba-all eval source, no task name leaks
- Switch eval source from all.yaml to wba-all.yaml (W&B-specific tasks)
- Run update_all_eval.py before each eval to pick up new tasks
- Replace raw eval-log artifact with results-summary.json (counts only)
- Generate per-agent badge JSON files (.badges/codex.json, .badges/claude-code.json)
- Add Codex and Claude Code shield badges to README
- Keep eval_tasks input override for manual dispatch
- No task names, rubric content, or eval details leak to public artifacts
-- AI was used to create large parts of this PR
Parses the eval summary from the log output to extract:
- passed count / total tasks
- score percentage from correctness or rubric scorer
Badge now shows e.g. "13/24 passing (54%)" instead of just "passing".
Per-agent badges will come later when WBAF supports multi-agent eval reporting.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The workflow no longer depends on pre-existing agent configs in WBAF.
Instead it:
1. Copies skills from this repo into WBAF's skills/
2. Writes a temp agent config (ci-skill-eval) extending codex-mcp with those skills
3. Writes a temp eval config pointing at that agent
4. Runs the eval
Inputs:
- skills: comma-separated list (blank = all)
- base_agent: agent to extend (default: codex-mcp)
- eval_tasks: task source (default: all from codex-all.yaml)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Workflow status badge at top of README
- Skill table with markers (<!-- BEGIN/END SKILL TABLE -->) auto-updated by CI
- CI reads skill descriptions from frontmatter and writes status into table
- Replaces JSON badge files with direct README updates (works for private repos)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- GitHub Actions workflow that runs on push/PR to main (when skills/ changes)
- Clones WandBAgentFactory (wbaf-refact branch) for the eval framework
- Copies skills into the eval framework and runs Codex agent with MCP
- Generates badge data on main merges
- Weekly scheduled run on Sundays at 11 PM PT
- Updated README with skill format docs and contributing guide
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Zubin Aysola