Companion to e16da42a. The agentops-eval-advisory job calls
scripts/eval-agentops.sh --fast directly, which has its own copy of the
strict baseline-audit logic — untouched by the pre-push-gate.sh / new CI
job softening in e16da42a. Result: advisory job still failed on the 54
missing_compare_baselines under the no-tracked-.agents/ policy from
3f1566fd, drowning out the 5 real eval-suite regressions in the noise.
Mirror the same drift-only logic into eval-agentops.sh:
- stale_suite_hashes>0 → record_failure (real drift)
- policy_mismatch_count>0 → record_warning (substrate-info)
- both 0 → unchanged success line
The 5 real suite regressions (distribution-install-update,
hook-manifest-runtime-contracts, pre-push-gate-governance,
push-worktree-closeout, release-security-gates) remain visible as
suite-level FAIL records. They are real signal worth a separate
investigation but unrelated to the substrate inconsistency this change
addresses.
Verified locally: shellcheck pre-push-gate-style → ok; bash -n → ok.
The 2026-05-03 overnight /evolve session shipped a baseline-audit pre-push
gate (commit d5d38ceb) but post-mortem council verdict was FAIL: gate had
0% enforcement (core.hooksPath was dangling), CI ran the eval suite with
continue-on-error: true (drift undetected), and eval-agentops.sh's per-suite
loop could promote baselines for suites that had just regressed. This commit
lands all three critical fixes from .agents/learnings/2026-05-03-overnight-evolve-post-mortem.md (L2, L4) and phase-3-summary-2026-05-03-overnight-evolve.md
items 1-3.
Item 1 — wire pre-push baseline-audit gate to a real git hook:
- scripts/check-pre-push-gate-wired.sh: asserts .githooks/pre-push exists,
is executable, and references scripts/pre-push-gate.sh; verifies the
baseline-audit block exists with a non-zero exit path; --dry-run-smoke
runs `git push --dry-run` against a sandbox bare remote and asserts
the hook actually fires.
- install-dev-hooks.sh already sets core.hooksPath=.githooks but had not
been run on this checkout; ran it to repair the local config.
Item 2 — promote ao eval baseline-audit to required CI:
- .github/workflows/validate.yml: new agentops-eval-baseline-audit job
(required, gated through summary needs/checks) builds ao, runs
`ao eval baseline-audit --root evals/agentops-core --json`, and fails
on stale_suite_hashes>0 OR policy_mismatch_count>0. Mirrors
pre-push-gate.sh block 24d.
- Also adds pre-push-gate-wired job (required) that runs
check-pre-push-gate-wired.sh --dry-run-smoke on every PR.
Item 3 — eval-agentops.sh two-phase promotion:
- Per-suite loop now queues promotions via queue_promotion() instead of
executing them. After the loop, run_promote_queue() aborts if
failures>0 OR working tree is dirty outside .agents/ and .beads/.
Override: EVAL_AGENTOPS_ALLOW_DIRTY=1.
Verified locally:
- check-pre-push-gate-wired.sh --dry-run-smoke --check-activation: 7/7 ok
- jq parsing of ao baseline-audit output: stale=0 mismatch=0
- eval-agentops.sh --fast --advisory: still runs (caught a real
regression on agentops-core.beads-issue-tracking; with the new
two-phase logic, --promote-baseline would now refuse to overwrite
that suite's baseline)
- python yaml.safe_load on validate.yml: 36 jobs, both new jobs in
summary.needs
Refs: .agents/council/2026-05-03-validate-overnight-evolve-claude-skeptic.md
.agents/learnings/2026-05-03-overnight-evolve-post-mortem.md (L2, L4)
.agents/rpi/phase-3-summary-2026-05-03-overnight-evolve.md
Boden Fuller