mirror of https://github.com/wshobson/agents.git synced 2026-09-14 18:04:20 +08:00

Files

T

History

dependabot[bot] f2c440fd95 deps(plugin-eval): bump the python-minor-and-patch group (#699 )

Bumps the python-minor-and-patch group in /plugins/plugin-eval with 6 updates:

| Package | From | To |
| --- | --- | --- |
| [pydantic](https://github.com/pydantic/pydantic) | `2.13.4` | `2.13.5` |
| [typer](https://github.com/fastapi/typer) | `0.27.1` | `0.27.2` |
| [claude-agent-sdk](https://github.com/anthropics/claude-agent-sdk-python) | `0.2.147` | `0.2.152` |
| [anthropic](https://github.com/anthropics/anthropic-sdk-python) | `1.2.0` | `1.3.0` |
| [ruff](https://github.com/astral-sh/ruff) | `0.16.5` | `0.16.6` |
| [ty](https://github.com/astral-sh/ty) | `0.0.75` | `0.0.78` |


Updates `pydantic` from 2.13.4 to 2.13.5
- [Release notes](https://github.com/pydantic/pydantic/releases)
- [Changelog](https://github.com/pydantic/pydantic/blob/v2.13.5/HISTORY.md)
- [Commits](https://github.com/pydantic/pydantic/compare/v2.13.4...v2.13.5)

Updates `typer` from 0.27.1 to 0.27.2
- [Release notes](https://github.com/fastapi/typer/releases)
- [Changelog](https://github.com/fastapi/typer/blob/master/docs/release-notes.md)
- [Commits](https://github.com/fastapi/typer/compare/0.27.1...0.27.2)

Updates `claude-agent-sdk` from 0.2.147 to 0.2.152
- [Release notes](https://github.com/anthropics/claude-agent-sdk-python/releases)
- [Changelog](https://github.com/anthropics/claude-agent-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/claude-agent-sdk-python/compare/v0.2.147...v0.2.152)

Updates `anthropic` from 1.2.0 to 1.3.0
- [Release notes](https://github.com/anthropics/anthropic-sdk-python/releases)
- [Changelog](https://github.com/anthropics/anthropic-sdk-python/blob/main/CHANGELOG.md)
- [Commits](https://github.com/anthropics/anthropic-sdk-python/compare/v1.2.0...v1.3.0)

Updates `ruff` from 0.16.5 to 0.16.6
- [Release notes](https://github.com/astral-sh/ruff/releases)
- [Changelog](https://github.com/astral-sh/ruff/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ruff/compare/0.16.5...0.16.6)

Updates `ty` from 0.0.75 to 0.0.78
- [Release notes](https://github.com/astral-sh/ty/releases)
- [Changelog](https://github.com/astral-sh/ty/blob/main/CHANGELOG.md)
- [Commits](https://github.com/astral-sh/ty/compare/0.0.75...0.0.78)

---
updated-dependencies:
- dependency-name: pydantic
  dependency-version: 2.13.5
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-minor-and-patch
- dependency-name: typer
  dependency-version: 0.27.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-minor-and-patch
- dependency-name: claude-agent-sdk
  dependency-version: 0.2.152
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-minor-and-patch
- dependency-name: anthropic
  dependency-version: 1.3.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: python-minor-and-patch
- dependency-name: ruff
  dependency-version: 0.16.6
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-minor-and-patch
- dependency-name: ty
  dependency-version: 0.0.78
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: python-minor-and-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

2026-09-13 07:37:03 -04:00

.claude-plugin

fix(plugin-eval): add missing plugin description (#630 )

2026-07-18 16:59:59 -04:00

.codex-plugin

fix(plugin-eval): add missing plugin description (#630 )

2026-07-18 16:59:59 -04:00

agents

fix(plugin-eval): address code review findings

2026-03-26 00:54:01 -04:00

commands

fix(plugin-eval): address code review findings

2026-03-26 00:54:01 -04:00

scripts

ci: add plugin eval reporting workflow with per-plugin scores

2026-04-16 13:16:47 -04:00

skills/evaluation-methodology

improve(plugin-eval): tune evaluation-methodology skill for higher score

2026-03-26 13:05:51 -04:00

src/plugin_eval

feat(antigravity)!: migrate from Gemini CLI to Google Antigravity CLI harness (#669 )

2026-08-18 11:42:59 -04:00

tests

fix(plugin-eval): populate model_usage from judge and Monte Carlo layers (#660 ) (#668 )

2026-08-18 10:31:31 -04:00

pyproject.toml

deps(plugin-eval): bump the python-minor-and-patch group (#659 )

2026-08-18 08:55:57 -04:00

README.md

docs(plugin-eval): add CLAUDE.md, full evaluation framework docs, and README updates

2026-03-26 17:40:56 -04:00

uv.lock

deps(plugin-eval): bump the python-minor-and-patch group (#699 )

2026-09-13 07:37:03 -04:00

README.md

plugin-eval

Three-layer quality evaluation framework for Claude Code plugins.

Quick Start

cd plugins/plugin-eval
uv sync

# Evaluate a skill (static only, instant)
uv run plugin-eval score path/to/skill --depth quick

# Evaluate with LLM judge (~30s)
uv run plugin-eval score path/to/skill --depth standard

# Full certification (all layers, ~5 min)
uv run plugin-eval certify path/to/skill

Layers

Static Analysis — Structural checks, anti-pattern detection. Instant, free.
LLM Judge — Semantic evaluation (triggering, orchestration, output, scope). ~30s, 4 calls.
Monte Carlo — Statistical reliability via 50–100 simulated runs. ~2–5 min.

Commands

CLI	Claude Code	Description
`plugin-eval score`	`/eval`	Score a plugin or skill
`plugin-eval certify`	`/certify`	Full certification with badge
`plugin-eval compare`	`/compare`	Head-to-head comparison
`plugin-eval init`	—	Build corpus for Elo ranking

Documentation

See docs/plugin-eval.md for the full reference: layers, dimensions, scoring formula, anti-patterns, statistical methods, and project structure.

README.md Unescape Escape

plugin-eval

Quick Start

Layers

Commands

Documentation

README.md