Commit Graph

1 Commits

Author SHA1 Message Date
Bo 03e365bb92 feat(validate): generate+check tests-from-scenarios coverage (ag-9jle.4 #tests-from-scenarios) (#648)
## What

Adds the C2 "tests-from-scenarios" leaf gate (ag-9jle.4): a per-bead,
forward-from-Gherkin scenario→test coverage check, plus wiring into the
leaf validators `/vibe` and `/test`.

`scripts/check-bead-scenario-coverage.sh` parses a bead's `## Scenarios`
block (via `--bead <id>`), a `.feature` file, or stdin, and **FAILS if
any scenario lacks a `@covered-by:<test-path>` link**. With `--run` it
also executes each covering test and requires it to PASS — not merely
exist. This is the distinction the bead calls out: "tests exist" or a
coverage percentage is not sufficient.

This is the **per-slice sibling** of the existing corpus-wide
`scripts/check-scenario-test-linkage.sh` (which sweeps every `.feature`
under `skills/*/references/` in CI). They share the `@covered-by:`
convention so a scenario authored once carries linkage to both. No new
`ao` subcommand (avoids cli-command-surface churn); no symlinks (codex
twins copied).

## Why

Audit ag-9jle.1 found C2 weak: vibe/validate check test-pyramid
*presence*, `test` checks coverage %, review uses a checkbox; only
`/validation` enforced scenario→passing-test mapping and only at the
bypassable roll-up. This makes the check mechanical and moves it to the
leaf (vibe Step 2g.1, test Step 0a), working forward from behavior.

## How it's tested

`tests/scripts/check-bead-scenario-coverage.bats` — 19 hermetic cases:
M>=N gate, bead-body + `.feature` inputs, `--run` pass vs fail, dangling
`@covered-by`, `::Name` resolution, threshold override, `--json`, stdin,
`## Scenarios`-block scoping, misuse exits. The gate is dogfooded: it
passes on its own bead ag-9jle.4 (`--run` executes this very bats file).

## Gates

- `bats tests/scripts/check-bead-scenario-coverage.bats` — 19/19 pass
- `shellcheck scripts/check-bead-scenario-coverage.sh` — clean
- `bash scripts/check-codex-parity-drift.sh` — no drift (codex twins
mirrored)
- `bash tests/skills/run-all.sh` — 87/0 pass

Closes-scenario: ag-9jle.4#tests-from-scenarios
Bounded-context: BC2-Validation
Evidence: scripts/check-bead-scenario-coverage.sh
2026-05-31 13:00:06 -04:00