mirror of
https://github.com/boshu2/agentops.git
synced 2026-09-14 15:08:13 +08:00
Bo
0c42512fdd
## What Reusable, authoritative CI gate over **agent-produced output** — the hookless guardrail for out-of-session agents (Claude Managed Agent, Agent SDK loop, or NTM/Codex swarm). A `workflow_call` job runs `ao validate --gate` (the umbrella composing ratchet/standards/scenario) against an agent's PR/artifact and propagates PASS/WARN/FAIL to the job exit code. **CI is the gate, not a ported hook** (AgentOps 3.0 is hookless). Second child of epic **ag-7s9fo** (Make Managed/SDK agents AgentOps-native), after ag-2wln `/agent-native` (#616). Inherits the richer spec absorbed from superseded ag-toat4; the optional SDK hook adapter was split out to ag-50m3. ## Files (3, +184/−0) - `scripts/agent-output-validate.sh` — testable gate logic. Resolves `ao` (`AO_BIN` override), runs `ao validate --gate "$@"`, propagates exit (0 pass/warn, 1 fail, 2 error). **Fails loudly if `ao` is missing** — a missing gate never reads as a clean pass. - `.github/workflows/agent-output-validate.yml` — `workflow_call` wrapper (inputs: `changes`, `strict`); builds `ao`, runs the script. Runtime-agnostic (serves Managed-Agent + NTM-swarm PRs). `workflow_call`-only **by design** — it's adopted by a caller, not triggered standalone, so it does not run on this PR. - `tests/scripts/agent-output-validate.bats` — 7 cases. The **passing+failing fixture evidence** the bead requires, exercised deterministically via an injectable fake `ao` (exit 0/1/2), plus scope-flag forwarding, missing-binary, strict-mode, and workflow-wiring assertions. ## Evidence - `bats tests/scripts/agent-output-validate.bats` → **7/7 green** - `shellcheck scripts/agent-output-validate.sh` → clean (severity=error) - Real-`ao` smoke: against the live binary the script faithfully propagates the gate verdict (an empty dir yields ao's exit-2 "no artifacts" → ERROR, proving no silent pass) - yaml parses valid ## Note for review The workflow is `workflow_call`-only, so CI here can't execute it (nothing `uses:` it yet). The **bats suite is the regression proof** for the gate logic; the yaml is the thin Actions wrapper a caller adopts. Bare invocation (no `changes`/`bead`) intentionally surfaces ao's exit-2 — callers scope the gate via the `changes` input. Closes-scenario: ag-mptr#agent-output-validate Bounded-context: BC5-Runtime Evidence: tests/scripts/agent-output-validate.bats
48 lines
1.9 KiB
Bash
Executable File
48 lines
1.9 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# agent-output-validate.sh — the authoritative gate over agent-produced output.
|
|
#
|
|
# Runs `ao validate --gate` (the umbrella that composes the ratchet / standards /
|
|
# scenario validators) against whatever an out-of-session agent produced — a PR
|
|
# branch checkout or an artifact bundle — and propagates the PASS/WARN/FAIL
|
|
# verdict to this process's exit code. This is the SAME authoritative gate
|
|
# interactive work hits in validate.yml; CI is the gate, not a hook (AgentOps
|
|
# 3.0 is hookless). Called by .github/workflows/agent-output-validate.yml and
|
|
# usable directly by an NTM lead pane or a self-hosted sandbox.
|
|
#
|
|
# Usage:
|
|
# agent-output-validate.sh [ao validate flags...]
|
|
# agent-output-validate.sh --changes plan.md
|
|
# agent-output-validate.sh --bead ag-123 --strict
|
|
#
|
|
# Exit codes (mirrors `ao validate --gate`):
|
|
# 0 PASS or WARN (gate passes)
|
|
# 1 FAIL (gate fails — a violation was found)
|
|
# 2 internal/usage error (could not run the gate, e.g. ao not found)
|
|
#
|
|
# Env:
|
|
# AO_BIN override the `ao` binary (default: `ao` on PATH). Lets CI point at a
|
|
# freshly built binary and lets tests inject a fixture.
|
|
set -euo pipefail
|
|
|
|
AO_BIN="${AO_BIN:-ao}"
|
|
|
|
# Resolve the ao binary loudly — a missing gate must never read as a clean pass.
|
|
if ! command -v "$AO_BIN" >/dev/null 2>&1 && [ ! -x "$AO_BIN" ]; then
|
|
echo "agent-output-validate: ao binary not found (AO_BIN=$AO_BIN). Build it first (cd cli && go install ./cmd/ao) or set AO_BIN." >&2
|
|
exit 2
|
|
fi
|
|
|
|
echo "agent-output-validate: running the authoritative gate on agent-produced output"
|
|
echo "agent-output-validate: \$ $AO_BIN validate --gate $*"
|
|
|
|
rc=0
|
|
"$AO_BIN" validate --gate "$@" || rc=$?
|
|
|
|
case "$rc" in
|
|
0) echo "agent-output-validate: PASS — agent output cleared the gate" ;;
|
|
1) echo "agent-output-validate: FAIL — agent output violated the gate (not mergeable)" >&2 ;;
|
|
*) echo "agent-output-validate: ERROR — gate could not run (exit $rc)" >&2 ;;
|
|
esac
|
|
|
|
exit "$rc"
|