2 Commits

Author	SHA1	Message	Date
Bo	092b99dad9	feat(skills): add skill-eval.sh ms-backed quality gate (ag-yzoz #skill-eval) (#726) ... ## What Adds `scripts/skill-eval.sh` — a thin gate that runs one skill's `SKILL.md` through Jeff Emanuel's `ms` (meta_skill) `lint` + `validate`. The wrapper decides the gate purely on whether a **blocking** `rule_id` appears (ms's own severity/exit is not trusted): - Blocking: `no-secrets`, `no-injection`, `safe-paths`, `required-metadata`, `no-cycle`, `valid-version` → non-zero exit. - Everything else → annotated only (`::warning::`), never fails. - `ms` not on PATH (or `MS_BIN` missing) → **loud `::error::` hard-fail (exit 3)**, never skip-and-pass — a silent skip would recreate the no-evaluation gap this gate closes. Planted `skills/_fixtures/{bad,good}-skill/SKILL.md` fixtures + `tests/scripts/skill-eval.bats` (10 tests) exercise the bad/good/ms-absent contracts. ## Fixture scoping `skills/_fixtures/` is non-skill scaffolding (planted fixtures). The skill-enumerating gates/generators are taught to skip `skills/_*` so the fixtures never inflate the skill count (stays 82) or fail registry / schema / wiring / runtime / count gates. Touched enumerators: `generate-registry`, `validate-skill-schema`, `sync-skill-counts`, `validate-skill-count`, `generate-skill-catalog`, `generate-context-map`, `validate-manifests`, `check-wiring-closure`, `validate-headless-runtime-skills`, `lib/sku_catalog.py`, `check-alias-collisions`, `run-all`, `test-runtime-opencode-smoke`, `release-smoke-test`, `smoke-test`, `validate-skill`. Does **not** touch `validate.yml` — wiring skill-eval.sh into CI is a separate bead. Epic: ag-czzf ## Verification - `bats tests/scripts/skill-eval.bats` → 10/10 pass (ms 0.1.2 on PATH) - `shellcheck scripts/skill-eval.sh` → clean - `check-registry-drift`, `validate-skill-schema`, `sync-skill-counts`, `run-all`, opencode-smoke → all PASS with fixtures present, skill count = 82 (fixtures excluded) Closes-scenario: ag-yzoz#skill-eval Bounded-context: BC2-Validation Evidence: scripts/skill-eval.sh --------- Co-authored-by: Cursor Agent <cursoragent@cursor.com>	2026-06-04 16:38:57 -04:00
Bo	e90d6d79f9	feat(registry): SKU capability catalog (schema v2) — generated skill×command×BC join (ag-cbm) (#525) ... ## What Extends `registry.json` into a **generated SKU capability catalog** (schema_version 2) — the flagship 3.0 inventory feature. It is a 4th DERIVED projection (alongside DDD-vocab / Hex-frontmatter / Gherkin-acceptance), a JOIN of existing sources — **never hand-authored** (repo bans hand-edited inventory maps). Oracle-validated design: `.agents/discovery/2026-05-24-capability-inventory-and-gaps.md`. ## Catalog 214 SKU entries: **81 skills + 66 cli-commands + 64 gates + 3 reference-impls**. Each entry: `sku` (stable id), `name`, `type`, `bounded_context`, `hex_role`, `tier`, `purpose`, `status` (active | deprecated | planned | alias-of:&lt;sku&gt;), `disposition`, `consumes`, `produces`, `drives_commands`, `driven_by_skills`. `registry.json` schema v2 is a strict **superset** of v1 — existing `summary`/`surfaces`/`cadence_recommendations` consumers (the Go `ao registry` command) keep working; new `capabilities`/`capability_summary`/`cli_top_level_commands` blocks are added. ## The join key — `drives_commands` Derived from each skill **BODY** (SKILL.md + `references/*`), resolved against the live cobra tree (`scripts/lib/sku_extract.py`). Stale refs that resolve to nothing (e.g. the removed `ao schedule`) never become edges. `driven_by_skills` is the cmd→skill reverse. This closes oracle gap #1 (no skill↔command linkage anywhere). ## New gate — `validate-sku-catalog-drift` Three required checks: **(a) drift** (regenerate + diff), **(b) linkage integrity** (every `drives_commands` edge resolves to a real `ao` command — closes gaps #1/#2/#3), **(c) coverage** (every BC + every operating-loop move has an active skill; every BC has a cli-command). Wired into `validate.yml` + the verdict aggregator. ## Stale refs fixed `ao schedule` in `skills/crank` + `skills/rpi` best-practices and `agentopsd` in `skills/swarm` shared-checkout-discipline → reframed to the hookless / out-of-session (cron-driven `ao rpi`/`ao evolve` via Gas City) model. Disjoint from PR #524 (which fixed the hooks/inject refs); rebased on it — `validate-skill-cli-snippets.sh` is now 0. ## Retired The bogus "163 cli_commands" count (Go-file count, oracle gap #6) → **66 real top-level cobra nodes** from the live tree. ## Gates (all green locally) registry-check · validate-sku-catalog-drift (3/3) · validate-context-map-drift · skill-domain-map-golden · bounded-contexts-drift · codex-parity-drift · regen-codex-hashes --check · contract-compatibility (sku-catalog.md catalogued) · doc-release · validate-skill-cli-snippets (0) · skill-frontmatter · heal --strict · markdownlint (0) · shellcheck (0 errors) · mkdocs --strict · `go test ./cmd/ao` (9207) · new bats suite (5/5). Closes-scenario: ag-cbm#sku-catalog Bounded-context: BC1-Corpus Evidence: .agents/discovery/2026-05-24-capability-inventory-and-gaps.md	2026-05-24 20:47:04 -04:00