Commit Graph

12 Commits

Author SHA1 Message Date
Bo 092b99dad9 feat(skills): add skill-eval.sh ms-backed quality gate (ag-yzoz #skill-eval) (#726)
## What

Adds `scripts/skill-eval.sh` — a thin gate that runs one skill's
`SKILL.md` through Jeff Emanuel's `ms` (meta_skill) `lint` + `validate`.
The wrapper decides the gate purely on whether a **blocking** `rule_id`
appears (ms's own severity/exit is not trusted):

- Blocking: `no-secrets`, `no-injection`, `safe-paths`,
`required-metadata`, `no-cycle`, `valid-version` → non-zero exit.
- Everything else → annotated only (`::warning::`), never fails.
- `ms` not on PATH (or `MS_BIN` missing) → **loud `::error::` hard-fail
(exit 3)**, never skip-and-pass — a silent skip would recreate the
no-evaluation gap this gate closes.

Planted `skills/_fixtures/{bad,good}-skill/SKILL.md` fixtures +
`tests/scripts/skill-eval.bats` (10 tests) exercise the
bad/good/ms-absent contracts.

## Fixture scoping

`skills/_fixtures/` is non-skill scaffolding (planted fixtures). The
skill-enumerating gates/generators are taught to skip `skills/_*` so the
fixtures never inflate the skill count (stays 82) or fail registry /
schema / wiring / runtime / count gates. Touched enumerators:
`generate-registry`, `validate-skill-schema`, `sync-skill-counts`,
`validate-skill-count`, `generate-skill-catalog`,
`generate-context-map`, `validate-manifests`, `check-wiring-closure`,
`validate-headless-runtime-skills`, `lib/sku_catalog.py`,
`check-alias-collisions`, `run-all`, `test-runtime-opencode-smoke`,
`release-smoke-test`, `smoke-test`, `validate-skill`.

Does **not** touch `validate.yml` — wiring skill-eval.sh into CI is a
separate bead.

Epic: ag-czzf

## Verification

- `bats tests/scripts/skill-eval.bats` → 10/10 pass (ms 0.1.2 on PATH)
- `shellcheck scripts/skill-eval.sh` → clean
- `check-registry-drift`, `validate-skill-schema`, `sync-skill-counts`,
`run-all`, opencode-smoke → all PASS with fixtures present, skill count
= 82 (fixtures excluded)

Closes-scenario: ag-yzoz#skill-eval
Bounded-context: BC2-Validation
Evidence: scripts/skill-eval.sh

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
2026-06-04 16:38:57 -04:00
Bo d4d92c8dfb chore(gates): gate ASCII-diagram skill counts + drop dead hooks/ scan (ag-c4wn #lk80-papercuts) (#715)
Three low-severity **ag-lk80** post-mortem papercuts, one chore arc
(single rollback semantic).

## 1. ASCII-diagram skill-count drift guard — `sync-skill-counts.sh`
`docs/agentops-system-map.md` and `docs/agentops-brief.md` hardcode the
skill count inside box-drawing borders whose column widths are
alignment-sensitive. `patch_file` deliberately won't auto-rewrite them
(a digit-width change, e.g. `99→100`, silently breaks the box). Added
the sanctioned **tolerant check**: every `<N> [shared] skills` token
must equal the on-disk skill TOTAL, else FAIL so a human re-pads the
box. Closes the drift class auto-patch left open. Works in both
`--check` and apply modes (reuses the existing `errors` counter).

## 2. `bd hooks` commit noise — not repo-actionable
The help-text dumped on every commit comes from `.git/hooks/*`
bd-installed shims (**local, untracked git state**) — fixable only by a
bd-tool change or a local `bd hooks` reinstall, nothing committable in
agentops. Documented as resolved-here-not-applicable (the dump even
appears in this PR's own commit step).

## 3. Drop dead `hooks/` scan — `check-agents-write-surfaces.sh`
`hooks/` doesn't exist post-3.0-teardown, so the `[[ -d hooks ]]` scan
was a guaranteed no-op. Removed it + its comment. **Kept**
`writer["hooks"]` — `hooks` remains a valid *opt-in* writer-surface
label per `docs/contracts/agents-write-surfaces.md` (only the dead
directory scan was vestigial, not the classifier vocabulary).

## Verification
- `sync-skill-counts.sh --check` → PASS; guard FAILs on an injected
`82→99` drift (confirmed it bites)
- `check-agents-write-surfaces.sh` → PASS (68 referenced subdirs, all
documented); **24/24 bats green**
- `shellcheck` clean on both scripts

Closes-scenario: ag-c4wn#lk80-papercuts
Bounded-context: BC2-Validation
Evidence: scripts/sync-skill-counts.sh
2026-06-04 08:09:24 -04:00
Bo 02659e7b44 docs: fix GLOSSARY skill count (66→82, gated) + portable test-debt path (ag-j4tw #counts) (#709)
## What

Verified-clean subset of the second-wave counts/misc bead (ag-j4tw).

- **GLOSSARY.md:170** "ships 66 shared skills" → 82, and **extended
`sync-skill-counts.sh`** to cover the call-site so it self-heals and CI
catches future drift (same gate pattern as ag-bcl4).
- **test-architecture-debt-analysis.md:3** hardcoded Mac path
`/Users/fullerbt/…/cli/cmd/ao/` → repo-relative `cli/cmd/ao/`.

## Verify-first dropped FALSE POSITIVES
Applying the session's recurring lesson (audit agents over-flag):
- ENV-VARS `COUNCIL_EXPLORER_MODEL/TIMEOUT`, `COUNCIL_R2_TIMEOUT` are
**live** (consumed by `skills/council/`, not `cli/` — the audit only
grepped cli/).
- `MEMRL_MODE` exists (3 files); `rg-optimized.md:70` is blank; ROADMAP
"hookless S2–S5" is a real follow-on; `ao inject` is live.

## Still open (ag-j4tw stays open)
- agentops-brief.md / agentops-system-map.md counts are entangled in
**hook-diagram lines** → handle with ag-t1ca.
- TESTING.md `tests/hooks/` (hooks) + testing-skills.md
`tests/_quarantine/claude-code` → `tests/claude-code` (larger path
rework).

## Verification
- `scripts/sync-skill-counts.sh --check` → clean (GLOSSARY now covered)

Closes-scenario: ag-j4tw#counts-glossary
Bounded-context: BC2-Validation
Evidence: docs/GLOSSARY.md
2026-06-03 21:44:22 -04:00
Bo b8731a4378 docs(audit): remediate stale root + key docs — hookless refs, skill counts, retired CI jobs (ag-lk80 #staleness-fixes) (#705)
## What

Remediates verified staleness from the 2026-06-03 doc audit of root
governance/product docs + the key `docs/` cluster. Two systemic rot
patterns drove almost every finding: **skill-count drift** (73/78/~80 vs
truth 82) and **hooks references that survived the 3.0 hookless
teardown**.

All fixes independently verified against `cli/**`, `scripts/**`,
`.github/workflows/validate.yml`, and `skills/**`.

## Changes (by bead)

| Bead | Sev | Fix |
|---|---|---|
| ag-swi1 | CRIT | `newcomer-guide.md`: removed hooks-as-runtime-layer
framing (3.0 ships zero hooks); 5→4 layers; fixed dead
`scripts/install-dev-hooks.sh` ref; reframed hooks as opt-in via
`hooks-authoring` skill |
| ag-99xv | CRIT | `index.md`: skill count 73→82. **The "broken catalog
link" was a false positive** — `docs/skills/catalog.md` is generated at
build by `gen_skill_pages.py`; link left unchanged |
| ag-xa2b | CRIT | `cli-skills-map.md`: removed 5 phantom skills
(`pr-plan`, `pr-retro`, `update`, `readme`, `oss-docs`) an agent could
mis-invoke. Deep hooks-decoupling of this file (title, "Hook Callers"
column, "Hooks → Commands" section) **decomposed to a follow-up** —
half-editing leaves it internally inconsistent |
| ag-w4t8 | CRIT | `AGENTS-CI.md`: dropped retired advisory rows
`practice-citations` (RETIRED 2026-05-19) and
`factory-claim-ledger-strict` (no such job; references nonexistent file)
|
| ag-bcl4 | HIGH | `index.md` + `documentation-index.md`: counts → 82,
**and extended `sync-skill-counts.sh`** to cover both call-sites so the
count self-heals and CI (`validate-doc-release.sh` → `sync --check`)
catches future drift — the durable fix, not just retyping the number |
| ag-p7zg | HIGH | `CLAUDE.md`: aligned Ship-phase pre-push command to
`scripts/pre-push-gate.sh --fast`, matching `AGENTS-WORKFLOW.md` |
| ag-y5ii | MED | `documentation-index.md`: dropped stale "hooks" from
the how-it-works description |

## Deferred (filed as follow-up beads, not in this PR)

- PRODUCT.md "April 2026 Claude Code source analysis" reframe — product
positioning, operator's call
- Full hooks-decoupling of `cli-skills-map.md` — structural rework,
needs its own verification
- `bd worktree create --branch` syntax is wrong in
CLAUDE.md/AGENTS-WORKFLOW.md (needs a positional name) — discovered this
session
- `validate-bd-closeout-contract.sh` requires `bd dolt push` conditional
wording in AGENTS.md that the AGENTS split dropped (pre-existing main
red, local-only gate)
- Second-wave sweep of the remaining ~80 `docs/` files (ag-bz8n)

## Verification

- `scripts/sync-skill-counts.sh --check` → clean (both new call-sites
covered)
- `scripts/validate-cli-skills-map.sh` → PASS
- `tests/docs/validate-doc-release.sh` → PASS
- No residual `hooks/` / `hooks.json` runtime refs in
`newcomer-guide.md`
- All referenced paths exist (`skills/hooks-authoring/`,
`scripts/pre-push-gate.sh`)

Closes-scenario: ag-lk80#staleness-fixes
Bounded-context: BC1-Corpus
Evidence: docs/index.md
2026-06-03 20:44:06 -04:00
Bo 8af560f678 fix(ci): reconcile hookless gates and flywheel snapshot
Squash merge PR #546. Local evidence: doc-release, smoke, snapshot gate, pre-push Bats, snapshot Bats, structural gates, and pre-push --fast passed. PR GitHub Actions were blocked in runner setup by GitHub codeload/action download failures before repository code executed. Snapshot records current corpus as DECAYING rather than preserving stale compounding evidence.
2026-05-26 08:19:50 -04:00
Bo b4c6aec361 feat(skills)!: 3.0 consolidation — merge 4, cut 2, resolve 6 keep-flags (81→75) (#533)
## AgentOps 3.0 skill consolidation — reconcile-and-tighten

Resolves all 12 skills flagged `merge-review`/`cut-review` in
`docs/contracts/skill-dispositions.yaml` per the operator-approved
dispositions. **Zero capability loss** — every merged skill's content
lives in its target as a mode/section.

### MERGE (4) — content folded into target, source removed
| Source | Target | How |
|---|---|---|
| expert-council | council | `--mode=debate` (alias triggers documented
in council) |
| oss-docs | doc | `--mode=oss` (refs + `audit-oss-docs.sh` copied into
`doc/`) |
| readme | doc | `--mode=readme` (`readme-craft.md`; `bootstrap`
consumer repointed readme→doc) |
| pr-retro | post-mortem | `--scope=pr` (`pr-scope.md` + gh PR-outcome
mining) |

### CUT (2) — genuinely redundant, no live consumer
- **pr-plan** — folds into `/plan` + `/pr-research`; pr-* chain
repointed.
- **update** — install-script concern; refresh note added to
`using-agentops` + install one-liner.

### KEEP — resolved 6 flags (rationale in `skill-dispositions.yaml`)
`validate` + `curate` (designed-future canonical validator/miner; epic
`soc-cp7pv` GO/REVERT is a separate decision) · `security` (release-gate
driver, distinct from security-suite) · `converter` (packaging adapter
skill-builder consumes) · `knowledge-activation` (outer-loop
operationalization) · `grafana-platform-dashboard` (marketplace
specialization).

### Result
**81 → 75 skills** (65 user-facing + 10 internal). Trunk regenerated
once: skill-counts, skill-catalog, skill-domain-map, context-map,
registry, sku-catalog, codex hashes, embedded-sync. Evals repointed to
absorbed locations.

### Gates verified locally (all green)
lint-skills · heal.sh --strict · skill-schema · skill-frontmatter ·
skill-dependency-check · scenario-test-linkage ·
validate-context-map-drift · check-registry-drift · generate-registry
--check · skill-domain-map golden · validate-sku-catalog-drift ·
bounded-contexts-drift · ci-policy-parity · codex
parity/override-coverage/manifest/generated-artifacts/skill-parity/runtime-sections/backbone/lifecycle
· regen-codex-hashes --check · embedded-sync · validate-skill-count ·
markdownlint · shellcheck (--severity=error) · `cd cli && go build && go
vet && go test ./cmd/ao` (9126 passed).

Closes-scenario: ag-cons#consolidation
Bounded-context: BC1-Corpus
Evidence: .agents/discovery/2026-05-24-skill-consolidation-plan.md
2026-05-24 23:17:27 -04:00
Boden Fuller b8c5a3761b feat(agentops): make 3.0 hookless-first
Add DDD skill-domain contracts, hook lease inventory, replacement ports, artifact-first RPI skill boundaries, and release exact-SHA validation. Demote ao inject to a compatibility adapter and remove prompt-only default hooks.
2026-05-16 09:43:08 -04:00
Boden Fuller 00c4ff2204 docs: rewrite product to three layers (compiler + gates + flywheel)
Replace four-layer model (bookkeeping/validation/primitives/flows) with
three product layers that map to the CDLC loop: Context Compiler,
Validation Gates, Knowledge Flywheel. Update sync-skill-counts.sh
patterns to match the new PRODUCT.md heading format.
2026-05-04 14:09:04 -04:00
Boden Fuller 2156c627ca test(runtime): add runtime proof gates 2026-04-23 23:54:40 -04:00
Boden Fuller bdf112ba7f feat: align codex skill docs and hook command mappings 2026-02-26 05:48:41 -05:00
Boden Fuller 98d83674eb evolve: cycle 2 — improve rpi package coverage 69%->82% and fix shellcheck
- Add MergeWorktree happy-path, path inference, and error tests
- Add EnsureAttachedBranch edge cases (empty prefix, trailing dash)
- Add CreateWorktree and GetCurrentBranch error path tests
- Add NormalizeRuntimeMode, ValidateRuntimeMode, and flag override tests
- Fix shellcheck SC2206 in claude-freeze-repro.sh (quoting)
- Fix shellcheck SC2034 in sync-skill-counts.sh (unused variable)
2026-02-21 23:17:19 -05:00
Boden Fuller 37d4b5b2ae Merge branch 'codex/rpi-supervisor-cli' into main 2026-02-21 19:41:50 -05:00