 Bo
|
585b3da5f9
|
test(e2e): no-mocks harness — factory, JSON-line logger, prod guards (soc-acte) (#342)
Adopts the `testing-real-service-e2e-no-mocks` skill in `tests/e2e/`.
The agentops e2e suite was already mock-free in behavior; this PR makes
the contract explicit, adds the missing harness pieces, and proves the
wiring on two scripts.
**Bead:** soc-acte
**Closes-scenario:** soc-acte#e2e-harness-adoption
**Bounded-context:** BC4-Tests
**Evidence:** tests/e2e/README.md (Mock Risk Assessment Matrix); local
proof-run.sh 20/20 PASS; goals-scenarios-link.sh F1 PASS; shellcheck -S
warning clean
## What landed
- `tests/lib/e2e-guards.sh` — refuse-to-run if `$HOME` / repo dir / `ao`
binary look real. Strict defaults per the skill's "never hit prod from
tests" rule. Documented `AGENTOPS_E2E_ALLOW_UNSAFE=1` bypass logs loudly
to stderr.
- `tests/lib/e2e-logger.sh` — JSON-line CI-parseable sidecar. Stable
schema: `{ts, suite, phase, event, message, data}` where `event ∈
{suite_start, phase_start, pass, fail, assert, artifact, db_snapshot,
suite_end}`. jq-graceful degrade.
- `tests/lib/e2e-factory.sh` — sandbox + repo + ao binary + `.agents/`
skeleton + fixture-copy builders. `PROOF_AO_BIN` / `PROOF_FORCE_BUILD`
resolution preserved verbatim.
- `tests/e2e/proof-run.sh` — adopts all three across all 6 phases.
**20/20 checks PASS** end-to-end.
- `tests/e2e/goals-scenarios-link.sh` — adopts factory + guards. **F1
e2e PASS.**
- `tests/e2e/README.md` — e2e contract + Mock Risk Assessment Matrix (11
paths scored).
## Mock Risk audit
Score≥8 mock-free coverage: **6/11 → 11/11**. The 2 acceptable ⚠ rows
(openclaw `httptest` fixture, Claude CLI non-determinism) score <8 /
advisory respectively — see README for the rationale on each.
## Sibling pattern
Mirrors the shared-lib + per-script-adoption shape from
`tests/lib/colors.sh` (the existing convention for cross-script shell
helpers in this repo).
## Follow-up (not in this PR)
The remaining `goals-*.sh` and `closure-integrity-grace.sh` adopt the
same harness pattern (~5 lines each); mechanical and low-risk.
Co-authored-by: Codex <codex@example.invalid>
|
2026-05-18 22:18:23 -04:00 |
|