* feat(antigravity): add Google Antigravity CLI harness adapter (#644)
* feat(antigravity)!: retire Gemini CLI harness (#644)
Google deprecated the Gemini CLI in May 2026. This drops the Gemini adapter,
validator, and doc-gardener drift pairs, and removes the committed
gemini-extension.json / .gemini/ / GEMINI.md artifacts and the local
build-only skills/, agents/, commands/ trees they produced.
The Google Antigravity CLI (agy), added in the prior commit, is now the
harness those users should migrate to: native plugins at
.antigravity/plugins/<name>/, reading AGENTS.md directly (no context-file
redirect needed), with its own marketplace, tier-based model aliases
(pro/flash/inherit), and `make install-antigravity` for global installs.
- tools/adapters/gemini.py deleted; capabilities.py/generate.py/
validate_generated.py/doc_gardener.py/Makefile lose their Gemini
dispatch, targets, and drift pairs.
- Tests: TestGeminiAdapter, TestGeminiValidator, TestGeminiRoundTrip,
TestGeminiSmoke removed along with now-unused imports.
- CI: cli-smoke-test now installs the Antigravity CLI instead of the
Gemini CLI; multi-harness-generate uploads .antigravity/ instead of the
legacy top-level skills/agents/commands/ output.
- Docs (AGENTS.md, ARCHITECTURE.md, docs/harnesses.md, docs/authoring.md,
docs/round-trip-results.md, docs/plugin-eval.md, README.md,
CONTRIBUTING.md, issue/PR templates) swept to describe Antigravity as
the fifth harness in place of Gemini.
BREAKING CHANGE: the Gemini CLI harness is no longer generated, validated,
or supported. Existing gemini-extension.json / .gemini/ / GEMINI.md
consumers should switch to `make generate HARNESS=antigravity` and
`make install-antigravity`.
* fix(antigravity): mirror skill support dirs, translate $ARGUMENTS, harden validator (#644)
Address CodeRabbit + Codex review feedback on PR #669:
- antigravity.py: mirror every skill support file (scripts/, assets/,
resources/, examples/), not just references/ — matches OpenCode's pattern.
Excludes hidden files.
- antigravity.py: translate $ARGUMENTS to {{args}} in place within command
bodies; only append a trailing {{args}} block when the source has none.
- antigravity.py: serialize frontmatter with YAML-safe scalar quoting and
preserve dict-valued fields (e.g. metadata) as nested mappings instead of
stringifying the Python repr.
- validate_generated.py: guard against non-dict plugin.json and non-string
command description/prompt fields so malformed input is reported as a
finding instead of crashing with AttributeError/TypeError.
- Sync stale plugin/agent/skill/command counts in claude-code-review.yml and
ARCHITECTURE.md to the canonical 92/202/181/105.
- CONTRIBUTING.md: add the missing Antigravity entry to the six-harness
portability checklist.
- docs/authoring.md: add fable to ARCHITECTURE.md's valid model list; correct
the TodoWrite/hooks support matrix for Antigravity.
- harness_portability.py: fix the bare-model-alias comment — Antigravity maps
aliases to tier values, not full model IDs.
- .cursor/rules/020-agent-skill-authoring.mdc (source in
tools/adapters/cursor_rules/, regenerated): Antigravity lacks TodoWrite but
does support Task-spawn and hooks via native equivalents.
- README.md: narrow the Pensyve integration claim to the harnesses it
actually covers.
- .gitignore: document that Antigravity follows OpenCode's clone+generate
install pattern; give .antigravity/ its own comment.
- Extend adapter and validator test suites for both fixes.
* fix(antigravity): quote comma-containing items in flow-style YAML lists
CodeRabbit follow-up on the frontmatter YAML-safety fix: _yaml_scalar() didn't
treat ',' or ']' as needing quotes, so a list item containing a comma (e.g.
tags: ["foo, bar", baz]) split into two list entries on round-trip since flow
sequences use ',' as the item delimiter. Add _yaml_flow_scalar() for list
items specifically (top-level scalars don't need this — commas are only
ambiguous inside [...]). Regression test added.
* fix(plugin-eval): populate model_usage from judge and Monte Carlo layers (#660)
* fix(plugin-eval): attribute judge usage to SDK model, scope it per-call
Addresses CodeRabbit + Codex review on #668:
- query_llm now keys usage_sink by the SDK-reported model
(collect_sdk_output(messages).model), falling back to the requested
model only when the stream reported none. Keeps judge attribution
consistent with the Monte Carlo layer, which already aggregates by
SDK-reported model, and stops routing/fallback substitutions from
misattributing tokens to the wrong model.
- JudgeAnalyzer no longer accumulates model_usage as instance state.
analyze_skill() now creates a local dict and threads it through the
four assess_* calls, so a reused analyzer (or concurrent
analyze_skill calls) can no longer leak or mix token counts between
runs.
- Parameterized a bare `dict` annotation introduced by this feature in
test_judge.py's _result() helper.
- Added tests: SDK-reported model differing from the requested model,
and repeated analyze_skill() calls on one analyzer not leaking usage.
_simresult_from_messages applied the ResultMessage.result fallback
without checking is_error, so an errored SDK run whose result carries
diagnostic text ("API error") came back activated=True and
errored=True. The same run was then counted in both n_activated and
n_errored, inflating the activation rate and the triggering
contribution to the composite score.
That also left the activation path as the odd one out: output
consistency and token efficiency already filter on `not r.errored`,
and run_simulation's own except branch reports a failed run as
activated=False with quality_score 0.0. An SDK-reported error now
looks the same as a transport exception.
Errored runs still count once, against the failure rate.
* feat(adapters): map sonnet tier to Claude Sonnet 5; Copilot fable to claude-fable-5
Claude Sonnet 5 is GA (Copilot GA 2026-06-30) and Copilot now serves
Fable 5 natively, so the sonnet/inherit aliases move off Sonnet 4.6 and
the Copilot fable alias no longer falls back to Opus 4.8. Catalog
verification date bumped to 2026-07.
* fix(plugin-eval): judge sonnet tier resolves to claude-sonnet-5
* docs: refresh model alias table and tier counts (Sonnet 5, five-tier, 66/22/52)
* docs(plugins): update model examples to Sonnet 5 / Opus 4.8
Replace claude-sonnet-4-6 with claude-sonnet-5 and Opus 4.7 mentions
with 4.8 in llm-application-dev; replace retired claude-3-5-sonnet in
performance-testing-review's ai-review. Drop temperature args from
migrated examples — Sonnet 5 rejects non-default sampling params.
- judge: guard _measured_score against non-dict JSON; strip whitespace text
before the result fallback (typed result param as Any for ty).
- monte_carlo: use the shared result fallback for activation/quality; sum real
SDK usage token fields via _sdk.usage_total_tokens (no fake total_tokens=0).
- tests: importorskip claude_agent_sdk so dev-only (no llm extra) collection
doesn't fail; add tests for non-dict scores, whitespace/result fallback,
result-only activation, and token summing.
- run-tests.sh: cd guard + multiline-tolerant in-on-String detection.
- docs: cedar-validate if/else, repo-root-safe path, and #598 narrative
corrected to the verified conclusion (reporter essentially correct;
protect-mcp evaluate/sign subcommands absent -> separate issue).
De-duplicate the AssistantMessage/TextBlock/ResultMessage walk that judge.py
and monte_carlo.py each carried (collect_sdk_output in layers/_sdk.py), and
collapse the duplicated grep in the review-agent-governance policy test.
ResultMessage has no .content attribute, so the judge always read empty
text and silently fell back to 0.5. Extract from AssistantMessage TextBlocks
with ResultMessage.result fallback; mark errored/empty/non-JSON as unmeasured.
Add Claude Fable 5 (model: fable) as the tier above opus across the
adapter framework - native passthrough in Claude Code, mapped to each
harness's top model elsewhere. No agents are promoted yet; tier criteria
and cost guidance are documented in docs/agents.md and docs/authoring.md.
Refresh MODEL_ALIASES against current harness catalogs (June 2026):
- Codex: gpt-5 family -> gpt-5.5 / gpt-5.4-mini (per Codex model docs)
- Copilot: GPT-5 stand-ins -> native Claude models (claude-opus-4.8,
claude-sonnet-4.6, claude-haiku-4.5) now that Copilot CLI serves them
- OpenCode: opus -> anthropic/claude-opus-4-8; haiku dated pin ->
latest-tracking anthropic/claude-haiku-4-5
- Gemini: intentionally stays on GA gemini-2.5-* (3.x is preview-gated)
- plugin-eval judge opus tier -> claude-opus-4-8
Gates: generate-all (no committed drift), validate STRICT=1, garden,
test (442 passed), smoke-test all green.
Review feedback:
- codex.py: Codex marketplace entries now conform to the openai/codex
core-plugins schema — source {source: "local", path}, policy
{installation, authentication}, and category (was missing required
fields; flagged P1). Top-level trimmed to {name, plugins}.
- codex.py: interface.shortDescription cuts on a word boundary + ellipsis
instead of a mid-word [:120] slice in committed .codex-plugin manifests.
- cursor.py: top-level .cursor-plugin/plugin.json displayName is now the
short "Claude Code Workflows" instead of the long marketplace description.
Tooling (local drives CI):
- Bump ruff 0.15.14 -> 0.15.15 and ty 0.0.39 -> 0.0.40 in plugin-eval's
uv.lock so CI installs the same versions we run locally, and reformat the
tools/ + plugin-eval sources to ruff 0.15.15 (fixes the failing
ruff format --check; the lock-hash change also busts CI's stale cache).
make validate STRICT=1 clean / garden 0 errors / 442 tests pass; regeneration
remains drift-free.
Add lean, native plugin-install entry points so each harness's own plugin
manager can install this marketplace (mirroring obra/superpowers) — committing
only small JSON registries, not duplicated skill/agent content trees.
- Codex: committed marketplace registry (.agents/plugins/marketplace.json) +
per-plugin manifests (plugins/*/.codex-plugin/plugin.json). Entries point at
source ./plugins/<name>; Codex reads SKILL.md directly. Transformed
.codex/skills|agents stay gitignored.
- Cursor: commit the existing .cursor-plugin/ marketplace + .cursor/rules/
(these already point at source plugins/).
- Gemini: gemini-extension.json already committed (contextFileName: AGENTS.md);
transformed trees stay gitignored (install via clone + make generate).
- OpenCode: unchanged — install via `make install-opencode` (transformed tree
stays gitignored).
- CLAUDE.md is now a symlink to AGENTS.md; Claude-Code addenda moved to
docs/harnesses.md.
- CI: new step fails if `make generate-all` drifts from the committed registries.
Net new committed: ~720 KB of manifests (no skill/agent duplication). Adds
round-trip tests for the new registries + the symlink. Docs updated across
README, docs/harnesses.md, ARCHITECTURE.md, CONTRIBUTING.md, GEMINI.md,
docs/authoring.md, and the PR template.
* fix(plugin-eval): exempt slash-only and path-triggered skills from MISSING_TRIGGER
Skills can be invoked through three mechanisms in Claude Code:
1. Model-driven auto-invocation based on the description (the default,
and what MISSING_TRIGGER is designed to gate).
2. User-driven slash invocation only — opt-out via
`disable-model-invocation: true` in the SKILL.md frontmatter.
3. Path-triggered auto-load — opt-in via the `paths:` frontmatter glob,
where the skill is loaded when the model opens a matching file.
The current `MISSING_TRIGGER` check runs against every skill regardless
of invocation mechanism. Skills in (2) cannot be auto-invoked at all, so
a trigger phrase in the description is irrelevant. Skills in (3) are
triggered by file paths, not description matching, so the description
serves as documentation rather than as a discovery surface.
Add a `_skill_uses_description_trigger(skill)` predicate that consults
`skill.frontmatter` and short-circuits the MISSING_TRIGGER check for
the two non-description invocation paths. Behaviour for ordinary
model-invocable skills is unchanged.
Tests:
- `test_disable_model_invocation_exempts_skill` — slash-only, no trigger,
no flag.
- `test_paths_auto_load_exempts_skill` — path-triggered, no trigger,
no flag.
- `test_disable_model_invocation_false_still_checks_trigger` — explicit
`false` does not exempt.
- `test_empty_paths_value_still_checks_trigger` — empty string is not
a valid auto-load configuration.
Full plugin-eval suite (73 tests) passes.
* fix: address ruff SIM103/SIM102 and guard paths check against non-string values
Simplify conditional returns (SIM103), collapse nested ifs (SIM102),
and use isinstance(paths, str) to prevent paths: [] from incorrectly
exempting a skill from MISSING_TRIGGER.
---------
Co-authored-by: Seth Hobson <wshobson@gmail.com>
* chore: move toolchain to uv-native (no pip, no requirements.txt)
The repo previously mixed uv (for plugin-eval) with pip + requirements.txt
(for yt-design-extractor) and raw `python3` invocations in the Makefile and
CI workflows. This makes the toolchain uniformly uv-managed.
## yt-design-extractor
Moved `tools/yt-design-extractor.py` into `tools/yt-design-extractor/` with its
own `pyproject.toml` + `uv.lock`. EasyOCR (and its ~2 GB torch dependency)
becomes an optional extra (`uv sync --extra easyocr`). Deleted
`tools/requirements.txt`.
## Makefile
All targets now route through uv:
- `make install` / `make install-easyocr` / `make deps` / `make check` / `make run*`:
`cd tools/yt-design-extractor && uv run/uv sync`
- `make generate` / `make validate` / `make garden` / `make clean-generated`:
`uv run --project plugins/plugin-eval python tools/...` (reuses plugin-eval's
venv — it already has pyyaml and `extra-paths = ["../.."]` for tools/adapters)
- `make test` / `make smoke-test`: `uv run --project plugins/plugin-eval pytest`
## CI
- `.github/workflows/validate.yml`:
- `multi-harness-generate` swapped from `actions/setup-python@v5` to
`astral-sh/setup-uv@v5` + `uv sync` of plugin-eval before `make generate-all`
- Workflow-level `permissions: contents: read` + `persist-credentials: false`
on every checkout (carrying the security hardening forward consistently)
- `.github/workflows/code-quality.yml`:
- `json-lint` job swapped from `setup-python` to `setup-uv`
- YAML validator now uses `uv run --with pyyaml python` (no pre-step install)
- JSON/TOML validators use `uv run python` (stdlib `json.tool`, `tomllib`)
- Dropped the standalone `pip install pyyaml --quiet` line
## Inline hints
- `plugins/plugin-eval/src/plugin_eval/layers/judge.py`: error message
recommends `uv sync --extra llm` instead of `pip install plugin-eval[llm]`
- `tools/yt-design-extractor/yt-design-extractor.py`: usage docstring and
install hints now reference `make install` / `make install-easyocr` / `uv run`
The pip refs inside plugin-authored commands (deps-audit.md, doc-generate.md,
error-trace.md) describe scanning **user** Python projects — those legitimately
use pip and are out of scope.
Local gates green: make test (385 pass), make garden (0 errors), make validate
clean, ruff + format + ty all pass, markdownlint clean.
* chore: address PR #543 review feedback
- checkmake (Makefile): consolidate the multi-line `.PHONY:` declaration
onto a single line. The previous backslash-continued form was readable
but checkmake couldn't parse it, falsely flagging `validate` and
`clean-generated` as missing from .PHONY. They were already declared —
just invisible to the linter.
Other CodeRabbit feedback declined:
- pyproject.toml dep version constraints (CodeRabbit self-tagged "Low value"):
uv.lock pins exact versions for reproducibility; upper bounds on yt-dlp,
Pillow, etc. would invite stale-pin churn without changing the locked
installation. Tracking upstream aggressively is the right default for a
utility tool.
- SHA-pinning for GitHub Actions (CodeRabbit "Major" but defensible):
Workflow has no write scope (permissions: contents: read on both files),
no secrets are exposed, and the actions involved are first-party Anthropic
(astral-sh, actions/*, DavidAnson). Same policy decision as PR #542 —
blanket SHA pinning is a bigger commitment than this PR's scope warrants.
* chore: pin all GitHub Actions to commit SHAs
Addresses CodeRabbit's Major finding (zizmor `unpinned-uses`). Reverses the
earlier policy decision now that the workflow scope has grown — pins both
workflows consistently in one pass:
- actions/checkout v4 → 34e114876b0b11c390a56381ad16ebd13914f8d5
- actions/setup-node v4 → 49933ea5288caeca8642d1e84afbd3f7d6820020
- actions/upload-artifact v4 → ea165f8d65b6e75b540449e92b4886f43607fa02
- astral-sh/setup-uv v5 → e58605a9b6da7c637471fab8847a5e5a6b8df081
- DavidAnson/markdownlint-cli2-action v18 → eb5ca3ab411449c66620fe7f1b3c9e10547144b0
- oven-sh/setup-bun v2 → 0c5077e51419868618aeaa5fe8019c62421857d6
Each pin keeps the version tag in a trailing comment for human readability
and dependabot/renovate compatibility.
Both workflows YAML-validated locally; functionality unchanged.
* feat: AGENTS.md canonical context + OpenAI harness-engineering layout
Promote AGENTS.md to the committed cross-harness context file (per the
agents.md convention and OpenAI's harness-engineering blog). Harness-
specific files become thin redirects:
- AGENTS.md — canonical, committed (~74 lines, table-of-contents)
- CLAUDE.md — `@AGENTS.md` import + Claude-specific addenda
- GEMINI.md — Gemini-specific setup only
- .gemini/settings.json — redirects Gemini CLI's context to read AGENTS.md
- ARCHITECTURE.md — new at root, top-level architectural map
- gemini-extension.json — bumps version to 1.7.0, sets contextFileName: AGENTS.md
- .gitignore — drops the AGENTS.md entry (file is now committed)
Harness support verified:
- Codex CLI reads AGENTS.md natively (root → cwd walk, 32 KiB cap)
- Cursor 2.5+ reads AGENTS.md natively
- OpenCode reads AGENTS.md natively (wins over CLAUDE.md if both exist)
- Claude Code: `CLAUDE.md` first line is `@AGENTS.md` (Anthropic's
documented interop pattern)
- Gemini CLI: `.gemini/settings.json` context.fileName redirect
(Gemini doesn't support @-imports)
Codex adapter no longer generates AGENTS.md — `emit_global` instead
validates the committed file fits Codex's 32 KiB cap and the 150-line
table-of-contents convention. Tests updated. Clean-output target no
longer touches AGENTS.md.
## Auxiliary files updated for multi-harness reality
- `.github/ISSUE_TEMPLATE/bug_report.yml` — dropdown for harness +
component path; renames "subagent" → "plugin/agent/skill/command"
- `.github/ISSUE_TEMPLATE/feature_request.yml` — scope dropdown covers
framework / harness / tooling / docs / CI in addition to components
- `.github/ISSUE_TEMPLATE/new_subagent.yml` — relabeled "New Component
Proposal" with component-type dropdown (plugin/agent/skill/command/
harness adapter) and cross-harness portability field
- `.github/ISSUE_TEMPLATE/config.yml` — links to AGENTS.md, authoring
guide, per-harness docs; updated Contributing link to root
- `.github/CONTRIBUTING.md` — thin pointer to canonical root CONTRIBUTING.md
- `.github/PULL_REQUEST_TEMPLATE.md` — new; scope + affected-harness
checklists, test-plan checklist, portability-notes section
- CONTRIBUTING.md (root) — updated to reference AGENTS.md / ARCHITECTURE.md
- gemini-extension.json — version 1.6.0 → 1.7.0, count fixes, redirects
to AGENTS.md as contextFileName
## Code-quality CI
New `.github/workflows/code-quality.yml` with three jobs:
- `python-lint` — `ruff check`, `ruff format --check`, `ty check` on
the adapter framework + plugin-eval. yt-design-extractor.py legacy
code excluded.
- `markdown-lint` — markdownlint-cli2 against README, AGENTS, ARCHITECTURE,
CLAUDE, top-level guides, and docs/. Config in `.markdownlint.json`.
- `json-lint` — validates every JSON / TOML / YAML in the repo (excluding
generated trees).
Required ty environment config added to plugin-eval/pyproject.toml so
`tools.adapters.*` resolves from outside the package.
Fixed one ty error in `tools/adapters/base.py:HarnessAdapter.capabilities`
(return-type annotation didn't match the `Capability` dataclass returned).
Fixed two ruff SIM108 ternary suggestions in codex.py and doc_gardener.py.
ruff format applied across all in-scope files (formatting-only diffs).
## Tests + verification
- 387 pytest tests pass (1 new test for the AGENTS.md validate-don't-overwrite behavior)
- `make validate STRICT=1` clean
- `make garden` 0 errors (10 warnings — remaining oversize source skills)
- `make smoke-test` clean against locally installed OpenCode/Gemini/Codex/Claude Code
- Real-CLI round-trip: `opencode agent list` discovers 193 subagents,
`gemini extensions validate .` succeeds, all 191 Codex agent TOMLs parse
## Tag recommendations (separate task — for repo About panel)
Top 20 by reach + relevance (from `gh api search/repositories?q=topic:<tag>`):
automation mcp ai-agents developer-tools claude-code anthropic agentic-ai
agents prompt-engineering cursor multi-agent agent-skills orchestration
opencode workflows gemini-cli codex-cli claude-code-skills cursor-rules
claude-code-plugins
* fix(ci): YAML multi-doc + markdownlint scope/rules
Two CI failures on PR #542, both fixed:
## JSON/TOML/YAML syntax job (3 false-positive YAML errors)
The job's YAML validation used `yaml.safe_load` which only reads the first
document in a multi-document YAML stream. Three Kubernetes manifest
templates use the standard `---` document separator (valid YAML) and were
mis-flagged:
plugins/kubernetes-operations/skills/k8s-manifest-generator/assets/configmap-template.yaml
plugins/kubernetes-operations/skills/k8s-manifest-generator/assets/service-template.yaml
plugins/kubernetes-operations/skills/k8s-security-policies/assets/network-policy-template.yaml
Switched to `list(yaml.safe_load_all(...))` so multi-doc YAML is accepted.
## Markdown lint job (lots of pre-existing plugin-README violations)
Two changes:
1. **Narrow the lint glob** — markdownlint now runs against top-level
guides (README, AGENTS, ARCHITECTURE, CLAUDE, per-harness setup,
CONTRIBUTING) and our authored `docs/` only. Per-plugin READMEs
(`plugins/*/README.md`) are owned by their plugin authors and not
lint-gated as part of this framework PR. Lint enforcement for those
belongs at the plugin-author layer, not the framework PR layer.
2. **Tighten `.markdownlint.json`** — disable two rules that produce
noise without catching real defects:
- MD040 (fenced-code-language) — terminal output / shell command
blocks frequently omit a language by convention
- MD060 (table-column-style) — cosmetic table-pipe spacing; doesn't
affect rendering
Genuine formatting rules kept: MD029 (ol-prefix), MD031 (blanks-
around-fences), MD032 (blanks-around-lists), MD056 (table-column-
count), MD058 (blanks-around-tables).
## Real defects caught and fixed
The narrower scope still caught 5 real issues:
- `docs/agent-skills.md:397` — code fence inside an ordered list item
needed a blank line before the fence
- `docs/authoring.md:90` — bulleted list needed a blank line above
- `docs/plugin-eval.md:87` — table needed a blank line above
- `OPENCODE.md:39` — table-column-count error caused by literal `|`
inside backticks: ``mode: primary|subagent|all`` (3 cells reads as 5)
Rewrote as ``mode:` one of `primary` / `subagent` / `all``
## Verification
- `npx markdownlint-cli2 "*.md" "docs/*.md"` → 0 errors
- `yaml.safe_load_all` accepts all multi-doc YAMLs (0 errors)
- All other CI jobs already passing (Python ruff/ty, multi-harness
generate, CLI smoke test, plugin-eval pytest, tools pytest)
* fix: address PR #542 bot feedback
- Codex P2 (chatgpt-codex-connector): emit_global now reads AGENTS.md from
the repo root (WORKTREE), not output_root. Previously `--output-root <scratch>`
produced a false "missing" warning even when AGENTS.md was committed at the
real root, breaking --strict generation outside the repo. Added a constructor
arg `repo_root` so tests can stage a fake AGENTS.md without touching the
committed file, plus a regression test that proves the two paths are decoupled.
- CodeRabbit nitpick (code-quality.yml): added workflow-level `permissions:
contents: read` and `persist-credentials: false` on every checkout. Skipped
the SHA-pinning recommendation — it's a heavier blanket-policy decision and
the workflow has no write scope to abuse.
- CodeRabbit nitpick (pyproject.toml): consolidated the duplicate `dev` groups
by moving `ty` into `[project.optional-dependencies].dev` and removing the
now-empty `[dependency-groups]` block. Dropped `--group dev` from
`code-quality.yml`'s `uv sync` since `--all-extras` now covers it.
- Verified gemini-extension.json counts (82/191/155/102) against the actual
source-of-truth: 81 local plugins + 1 external = 82 in marketplace.json,
191 agent .md files, 155 SKILL.md files, 102 command .md files. Counts are
correct as-is — CodeRabbit's quick-win was a regex miscount.
* feat(adapters): multi-harness framework + harness_portability eval dimension
Turn this Claude Code plugin marketplace into a generic agentic-harness
marketplace. Adapters under tools/adapters/ emit harness-native artifacts
for OpenAI Codex CLI, Cursor, OpenCode, and Gemini CLI from a single
Markdown source. Source-of-truth stays under plugins/ — Claude Code is
unchanged.
Framework (tools/adapters/):
- base.py — PluginSource parser, HarnessAdapter ABC, write/mirror helpers
(path-traversal guard, UTF-8-safe), inline-list + block-list + block-scalar
YAML-ish parser, _utf8_safe_cut, _split_inline_list, _normalize_author
- capabilities.py — per-harness capability matrix, TOOL_NAME_MAPS,
MODEL_ALIASES, resolve_model() with explicit warnings
- codex.py — emits .codex/{skills,agents}/ + AGENTS.md (≤150-line
table-of-contents). Fence-aware body splitter, _utf8_safe_cut for
multibyte safety, _yaml_scalar with reserved-word + special-char quoting.
Skill/command name collision detection (and second-order __cmd fallback).
- cursor.py — emits .cursor-plugin/{plugin,marketplace}.json + curated
.cursor/rules/*.mdc. _validate_mdc_frontmatter handles YAML block scalars
(no false positives on colons in description body). _normalize_author
handles dict, npm-style strings, and author lists.
- opencode.py — transpiles agents to .opencode/agents/<id>.md with
mode:subagent + permission: deny-everything-else block (skill/task always
allowed as base capabilities — Claude's implicit defaults).
- gemini.py — emits native skills/, agents/, and commands/ at extension
root (April 2026 spec). Tool-allowlist remapped via TOOL_NAME_MAPS.
CLI + tooling:
- tools/generate.py — unified `make generate HARNESS=<x> [PLUGIN=<y>]`,
with --clean (containment-guarded; case-insensitive on Darwin/Win32),
--prune (orphan removal across all per-harness output trees), --strict
(warnings fail), per-plugin error aggregation, refuses --clean --plugin
(would silently wipe other plugins' artifacts).
- tools/validate_generated.py — structural validation across all four
harness outputs. Codex 8KB cap → error. _extract_permission_block
correctly handles nested permission keys (column-0 only).
- tools/doc_gardener.py — recurring drift detection per OpenAI harness-
engineering principle. STALE_ARTIFACT (info), DEAD_LINK (error),
MARKETPLACE_ORPHAN (error), SKILL_OVER_CODEX_CAP (warning), grouped
output sorted by severity.
plugin-eval (extends existing framework):
- New harness_portability dimension (6% weight, rebalanced from existing
static sub-scores). Surfaces non-portable patterns with concrete
remediation hints: SKILL_OVER_CODEX_CAP, CLAUDE_TOOL_REFS,
CLAUDE_TOOL_PROSE, AGENT_NAME_COLLISION, BARE_MODEL_ALIAS.
- _CAMEL_TOOL_PATTERN requires Claude-tool context (no false positives
on Rust's `Task` etc.). _TOOL_PROSE_PATTERN case-sensitive on tool
names, case-insensitive on the leading article.
- Findings do NOT also feed anti_pattern_penalty (no double-counting).
Documentation:
- Top-level guides: CODEX.md, CURSOR.md, OPENCODE.md (≤150 lines each,
table-of-contents pattern per OpenAI harness-engineering post)
- docs/harnesses.md — capability matrix, graceful-degradation table,
generated output paths
- docs/authoring.md — portable-content style guide (tools, models,
collision rules, fence-respect)
- docs/round-trip-results.md — real-CLI verification recipes (OpenCode
discovers 193 subagents, Gemini extensions validate passes, Codex
TOMLs all parse)
- CONTRIBUTING.md — new file pointing at docs/authoring.md
- README.md — rewritten for multi-harness (145 lines, was 460)
- CLAUDE.md — trimmed to 60-line table-of-contents
- GEMINI.md — trimmed from 1500 to 500 tokens (3× over budget previously)
Tests: 181 passing (103 plugin-eval + 78 tools/tests). Real-CLI round-trip
verified for OpenCode, Gemini, and Codex (TOML parses).
Replaces tools/generate_gemini_commands.py with the unified CLI.
* refactor(skills): extract detail to references/details.md (~75 skills)
Apply Anthropic's canonical SKILL.md progressive-disclosure pattern across
the marketplace: SKILL.md body becomes a navigation tier (trigger phrasing
+ quick start), detailed templates and worked examples move to
references/details.md (loaded on demand by the agent).
Motivation: OpenAI Codex CLI hard-truncates skills at 8 KB. Before this
change, ~90 skills exceeded that cap and would silently break on Codex.
The progressive-disclosure pattern is also Anthropic's documented
recommendation for token efficiency — Claude Code reads references/ files
on demand when the body navigation says to.
What's extracted, by pattern:
- Pass 1 (## Templates section): 19 skills — full template libraries
moved to references/details.md
- Pass 2 (## Implementation Patterns / ## Advanced Patterns): 13 skills
- Pass 3 (everything between nav-tier and wrap-tier headings): 53 skills
- Conservative re-extraction for 8 skills that got over-reduced — kept
~6-7 KB inline (most of the quick-start tier) plus references/ overflow
What stays inline (SKILL.md navigation tier):
- description: frontmatter (triggering — unchanged for all skills)
- ## When to Use This Skill / ## Core Concepts / ## Quick Start
- ## Best Practices / ## Troubleshooting / ## See Also wrap-ups
- A pointer note ("see references/details.md") so the agent knows where
to look for detail
What goes to references/details.md (detail tier, on-demand load):
- ## Templates (full code template libraries)
- ## Implementation Patterns / ## Advanced Patterns (deep examples)
- Mid-skill walkthroughs that exceed the inline budget
Also in this commit:
- plugins/brand-landingpage description trimmed from 958→543 chars
(preserves trigger phrasing, drops verbose example-quote list)
Net effect:
- SKILL_OVER_CODEX_CAP findings: 90 → 10 (88% reduction)
- All triggers unchanged — discovery behavior identical across harnesses
- 75 new references/details.md files with the extracted content
- Same depth of guidance, loaded progressively
Remaining 10 oversized skills are complex multi-section docs (e.g.
postgresql, code-review-excellence, evaluation-methodology) that need
per-skill manual judgment — flagged by `make garden` for future work.
* chore: bump all plugin versions (multi-harness release)
Patch-bump every local plugin (81) in both .claude-plugin/marketplace.json
entries and each plugins/<name>/.claude-plugin/plugin.json. Minor-bump the
top-level marketplace metadata.version (1.6.0 → 1.7.0) to signal the
multi-harness adapter framework addition.
The external git-subdir entry (qa-orchestra) is unaffected — its version
is governed by its upstream repo.
* fix(opencode): preserve explicit tools:[] + word-boundary subtask match
Addresses two Codex review findings on PR #541.
## P1 — `tools: []` silently upgraded to permissive (privilege escalation)
Before: `_build_permission_block` returned `{}` for any empty list, which
omits the `permission:` block entirely from the emitted agent. An author
who explicitly wrote `tools: []` to lock down an advisory-only agent got
an UNRESTRICTED agent in OpenCode. Affected agent in this tree:
`plugins/arm-cortex-microcontrollers/agents/arm-cortex-expert.md`.
Fix: `_build_permission_block` now takes a `has_tools_field` flag so the
caller can distinguish "tools: key missing" (Claude default permissive)
from "tools: []" (explicit lock-down). The lock-down case emits a
deny-everything block that allows ONLY the base capabilities (skill, task)
that Claude Code always grants implicitly. Verified against the real
arm-cortex-expert agent — now emits read/edit/write/bash/grep/glob/list:
deny, task/skill: allow.
## P2 — `"agent" in cmd.body.lower()` false-positives on substrings
Before: a command body containing `PerformanceReviewAgent` (class name
in a code snippet) or `useragent` triggered `subtask: true`, changing
runtime behavior based on incidental text.
Fix: switch to a compiled word-boundary regex `\b(agent|subagent)s?\b`
(case-insensitive). Tests confirm the substring `PerformanceReviewAgent`
no longer fires, while a real "spawn a subagent" sentence still does.
## Tests
3 new regression tests in tools/tests/test_adapters.py:
- `test_explicit_empty_tools_yields_locked_permission_block` (P1)
- `test_missing_tools_field_yields_no_permission_block` (P1 boundary)
- `test_subtask_inference_word_boundary` (P2)
184 total tests pass (was 181). OpenCode round-trip still discovers all
193 subagents; arm-cortex-expert agent is now properly locked down.
* test: behavioral verification + CI gates for multi-harness pipeline
Adds three layers of automated verification that pure-Python parser tests
miss, plus the CI jobs that turn them into hard gates. Catches the kinds
of issues that previously only surfaced when a real user installed the
marketplace and tried to use it.
## test_real_world.py — real-source structural tests
Runs against the actual `plugins/` tree (not synthetic fixtures). Catches
issues that only appear on real content:
- every marketplace entry resolves to a plugins/<name>/ dir
- every local plugin dir appears in marketplace.json
- marketplace.json version == per-plugin plugin.json version (catches drift)
- every plugin loads via load_plugin() without error
- no plugin name contains `__` (adapter namespace separator)
- every agent has name + description; every skill has a trigger phrase
(same regex plugin_eval's MISSING_TRIGGER check uses)
- no agent name collides with Codex built-ins
- every refactored skill (with `references/details.md`) has:
- meaningful detail content (>=500 B in details.md)
- a pointer to references/ in the SKILL.md body
- a navigation-tier heading preserved (When to Use, Overview, etc.)
- body >= 600 B (not a stub)
- every plugin.json has name + version matching the dir
This test pass found and fixed three real defects before commit:
- ship-mate/skills/scan: description had no trigger phrase ("Use when…")
- reverse-engineering/skills/memory-forensics: nav-tier section lost
during extraction
- reverse-engineering/skills/binary-analysis-patterns: same
All three are now fixed (preserved trigger phrasing, added When-to-Use
sections back to the skills my extraction over-trimmed).
## test_round_trip.py — generate→parse→verify
CI runs this AFTER `make generate-all`. Catches generation-time regressions:
- OpenCode/Codex/Gemini agent counts match source agent count (no skips)
- every Codex SKILL.md under 8 KB (the cap that would silently truncate)
- every Codex agent TOML has required fields + valid sandbox_mode
- every OpenCode agent has mode in {primary,subagent,all} and
provider-prefixed model
- locked agents (source `tools: []`) emit proper deny-everything permission
block with skill/task allow (regression guard for PR-541 P1)
- every Gemini @{path} injection resolves to a real source file
- every Gemini command TOML has prompt + {{args}} placeholder
- every context file (CLAUDE.md, AGENTS.md, GEMINI.md, etc.) within
150-line cap
- Cursor marketplace + per-plugin manifests cover all local plugins
- .cursor/rules/*.mdc only use the 3 documented frontmatter keys
## test_cli_smoke.py — real-CLI subprocess tests
Invokes the actual harness binaries (OpenCode, Gemini, Codex, Claude Code)
against the generated artifacts. Catches CLI-level issues pure-Python
parsing can't see: schema-loader drift, plugin-discovery bugs, version
incompatibilities.
- `opencode agent list` — must succeed AND discover every source agent
(currently 191 + 2 OpenCode built-ins)
- `gemini extensions validate <repo>` — must return success
- `codex doctor` — must report healthy install
- every Codex agent TOML must parse with stdlib `tomllib`
- `claude --version` — sanity check the Claude Code CLI loads
- marketplace.json must have owner + metadata.version for Claude Code's loader
Per-CLI tests skip gracefully when the binary isn't on PATH, so local
devs only exercise what they have installed. CI installs OpenCode +
Gemini and turns those skips into hard gates.
## Makefile + CI
- `make test` — full pytest suite (plugin-eval + tools/tests/)
- `make smoke-test` — generates if needed, then runs real-CLI smoke tests
- `.github/workflows/validate.yml` extended with:
- `tools-tests` job — runs pytest tools/tests/
- `multi-harness-generate` job — `make generate-all && make validate
STRICT=1 && make garden`, uploads generated artifacts on every run
- `cli-smoke-test` job — installs OpenCode + Gemini, runs test_cli_smoke.py
## Test counts
- Before: 184 tests
- After: 386 tests (parameterized real-source tests over all 82 plugins)
- All passing locally on OpenCode 1.15.7 + Gemini 0.42.0 + Codex 0.133.0
+ Claude Code 2.1.148
dependabot[bot]