Commit Graph

3 Commits

Author SHA1 Message Date
Bo ab138a5418 feat(skills): add scan_descriptions --probe deterministic trigger ranker (ag-7led #trigger-probe) (#722)
## Summary

Adds a `--probe "<phrase>"` mode to
`skills/skill-builder/scripts/scan_descriptions.py` that ranks every
skill against a lexical trigger phrase **deterministically** (no live
model, byte-stable output). Introduces an optional `trigger_probes:`
skill-frontmatter field so a skill can self-declare phrases it must rank
#1 for; the probe asserts the declaring skill wins its own phrases
against the corpus.

Epic: ag-czzf.

**INFORMATIONAL-only tooling — not a blocking gate.** The probe is a
self-check authors can run; no CI gate is wired to it.

## Changes
- `skills/skill-builder/scripts/scan_descriptions.py` — `--probe` ranker
+ `trigger_probes` parser (flow + block YAML forms).
- `schemas/skill-frontmatter.v2.schema.json` — optional `trigger_probes`
array field.
- `tests/scripts/scan-descriptions-probe.bats` — 4 scenarios (rank-#1,
determinism, demotion-on-mutation, usage-error exit codes).

## Validation
- New bats suite: 4/4 pass.
- `scripts/pre-push-gate.sh` reds are pre-existing main-red
(context-packet-ab canary + its downstream pre-push-gate-governance
nested bats), not caused by this diff; CI is changed-files-scoped.

Closes-scenario: ag-7led#trigger-probe
Bounded-context: BC2-Validation
Evidence: skills/skill-builder/scripts/scan_descriptions.py

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-06-04 15:40:07 -04:00
Bo a5a0cdcb0b feat(skills): encode external skill-authoring standard + corpus trigger scanner (ag-vb3q #authoring-standard-trigger-scan) (#663)
## What

RPI cycle on **mining how AgentOps makes skills** (bead `ag-vb3q`),
using the external `meta_skill` best-practices doc as the scoring lens
(methodology only — no Rust `ms` build).

**Research:** codebase-archaeology over the 83-skill corpus + the skill
factory
(`skill-builder`/`skill-auditor`/`heal-skill`/`converter`/`forge`,
`SKILL-TIERS`, `score_agentops_skill.py`, CI gates). The corpus is
structurally excellent: **100% frontmatter conformance, 100% Codex
parity, ~94% references usage**.

**The one finding that matters:** skill selection is pure LLM reasoning
over the `description` field, yet **73 of 81 skills (90%) carry no
trigger marker**. The auditor *has* the checks
(`description-has-triggers`, `trigger-clarity`) but they were downgraded
**FAIL→WARN**, so the gap accumulated silently — a textbook
doctrine-vs-enforcement drift.

## This PR (Implement)

- **`references/skill-authoring-standard.md`** — clean-room distillation
of the external standard, cross-walked to AgentOps's *actual* enforced
rules (best-practice → gate → severity table).
- **`scripts/scan_descriptions.py`** — corpus-wide trigger scanner that
mirrors `audit.sh`'s exact three-form detection (so it never contradicts
the per-skill auditor) and adds the prioritized remediation backlog + a
suggested `Triggers:` stub the auditor never produced. `--json` robot
mode, `--strict` exit code.
- **`tests/python/test_scan_descriptions.py`** — 11 behavioral tests
(all three forms, suggestion quality, CLI exit codes, live-corpus
smoke).
- **`skill-builder/SKILL.md`** — "Corpus authoring health" section +
reference link (194/250 lines).
- Regenerated `registry.json`; patched `catalog.json` reference count.

## Plan (follow-on beads filed)

- `ag-okfu` — add trigger markers to the 73 flagged skills (wave by
tier; acceptance = scanner `--strict` exits 0)
- `ag-cx7d` — ratchet `trigger-clarity`/`description-has-triggers`
WARN→FAIL once the corpus is clean (blocked by `ag-okfu`)
- `ag-mm6q` — fix `generate-skill-catalog.sh` BSD-awk incompatibility

## Evidence

\`\`\`
python3 skills/skill-builder/scripts/scan_descriptions.py skills #
scanned 81, missing 73 (90%)
python3 -m unittest tests.python.test_scan_descriptions # 11 ok
bash scripts/check-registry-drift.sh # PASS — no drift
bash scripts/validate-context-map-drift.sh # exit 0
heal --strict skills/skill-builder # All clean
\`\`\`

Closes-scenario: ag-vb3q#authoring-standard-trigger-scan
Bounded-context: BC4-Factory
Evidence: .agents/audits/meta-skill-archaeology.md
2026-05-31 20:16:38 +00:00
Claude 5240d0dcda test: expand coverage for context, quality, search, overnight, python
Add tests for previously untested surface area:
- internal/context: pure helpers for readiness classification, action
  mapping, display formatting, timestamp parsing, tmux target parsing,
  file-tail reading (23.9% -> 41.2%)
- internal/quality: golden signals math (linear regression, Gini coefficient,
  top-10/bottom ratio, citation counting, verdict computation) (15.3% -> 27.9%)
- internal/search: findings parsing (frontmatter, title extraction,
  freshness scoring), bead context resolution (env + cache + keywords),
  constraint index (load, save, lock, filter stale) (8.7% -> 22.4%)
- internal/overnight: loop helpers (validateRunLoopOptions, summary
  round-trips, snapshot partitioning, test fault injector hook) (77.4% -> 78%)
- internal/goals: platform-specific process-group tests for
  measure_unix.go (track/untrack/killAllChildren)
- tests/python: unittest-based tests for generate-context-shards.py
  covering is_binary, count_lines, units_for_file, pack_shards,
  verify_manifest, build_manifest (22 tests; runs under vanilla python3)
- tests/scripts: behavioral tests for ci-local-release.sh flag parsing
  and validation (17 bats cases)

https://claude.ai/code/session_01EMusoGMSUSbtPqkotJy3U7
2026-04-22 11:53:01 +00:00