092b99dad9
feat(skills): add skill-eval.sh ms-backed quality gate (ag-yzoz #skill-eval) ( #726 )
...
## What
Adds `scripts/skill-eval.sh` — a thin gate that runs one skill's
`SKILL.md` through Jeff Emanuel's `ms` (meta_skill) `lint` + `validate`.
The wrapper decides the gate purely on whether a **blocking** `rule_id`
appears (ms's own severity/exit is not trusted):
- Blocking: `no-secrets`, `no-injection`, `safe-paths`,
`required-metadata`, `no-cycle`, `valid-version` → non-zero exit.
- Everything else → annotated only (`::warning::`), never fails.
- `ms` not on PATH (or `MS_BIN` missing) → **loud `::error::` hard-fail
(exit 3)**, never skip-and-pass — a silent skip would recreate the
no-evaluation gap this gate closes.
Planted `skills/_fixtures/{bad,good}-skill/SKILL.md` fixtures +
`tests/scripts/skill-eval.bats` (10 tests) exercise the
bad/good/ms-absent contracts.
## Fixture scoping
`skills/_fixtures/` is non-skill scaffolding (planted fixtures). The
skill-enumerating gates/generators are taught to skip `skills/_*` so the
fixtures never inflate the skill count (stays 82) or fail registry /
schema / wiring / runtime / count gates. Touched enumerators:
`generate-registry`, `validate-skill-schema`, `sync-skill-counts`,
`validate-skill-count`, `generate-skill-catalog`,
`generate-context-map`, `validate-manifests`, `check-wiring-closure`,
`validate-headless-runtime-skills`, `lib/sku_catalog.py`,
`check-alias-collisions`, `run-all`, `test-runtime-opencode-smoke`,
`release-smoke-test`, `smoke-test`, `validate-skill`.
Does **not** touch `validate.yml` — wiring skill-eval.sh into CI is a
separate bead.
Epic: ag-czzf
## Verification
- `bats tests/scripts/skill-eval.bats` → 10/10 pass (ms 0.1.2 on PATH)
- `shellcheck scripts/skill-eval.sh` → clean
- `check-registry-drift`, `validate-skill-schema`, `sync-skill-counts`,
`run-all`, opencode-smoke → all PASS with fixtures present, skill count
= 82 (fixtures excluded)
Closes-scenario: ag-yzoz#skill-eval
Bounded-context: BC2-Validation
Evidence: scripts/skill-eval.sh
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com >
2026-06-04 16:38:57 -04:00
8e59cf1389
fix(skills-codex): audit + fix 60+ findings, strengthen CI gates
...
Skills content fixes:
- Add metadata.tier to all 54 skills-codex/ frontmatter
- Fix wrong-directory cross-refs in 11 skills (skills/ → ../)
- Remove Claude primitive leakage from 13 skills (TaskCreate,
AskUserQuestion, read_file, Agent tool, claude-code-latest-features)
- Fix dead refs, link missing references, remove orphaned Claude files
- Fix malformed ..$X link syntax in brainstorm, codex-team, rpi, validation
- Minor skills/ fixes: release refs, using-agentops count, push sections
CI gate hardening (false-pass elimination):
- heal.sh: scan skills-codex/, add MISSING_TIER/MALFORMED_LINK/DUPLICATE_REF
- audit-codex-parity.py: +7 Claude primitive patterns, +WRONG_XREF_DIR
- validate-links.sh: add skills-codex/ to scan scope (726 → 1117 links)
- validate-codex-generated-artifacts.sh: frontmatter check, invoke audit
2026-03-15 13:51:49 -04:00
62ce8b0d1c
feat(skills): ag-wug epic + codex converter parity regen
...
- Add output_contract, effort tables, sparse checkout guidance to skills
- Update claude-code-latest-features.md across all skills
- Regenerate all 54 codex skills with improved converter rules
- Codex parity: 107 → 8 findings (92% reduction)
2026-03-14 02:27:59 -04:00
Bo