Files
boshu2__agentops/cli/cmd/ao/cobra_conformance_test.go
Bo 64e93c0428 refactor(cli): complete the post-Cathedral-Cut Go CLI cleanup (#919)
Completes the Go CLI migration: one composition model, no tombstones, no
dead packages, hermetic tests, and a cross-language verdict contract
corpus. Net −88k lines.

## What
- **Tombstones + prune deleted**: the 21 registered 'Removed in the
Cathedral Cut' stubs, `zzz_default_spine.go`'s runtime prune, and the
~30 registered-but-pruned legacy command families are gone. The
registered tree IS the production tree; retired verbs fail with exit 1 +
a data-driven replacement hint pointing at docs/MIGRATION.md (now
covering nested verbs). Non-runnable parents reject unknown subcommands
instead of printing help with exit 0.
- **~35 unreachable packages deleted** (cliapp, canon, pawl-era verdict
parser, pool, harvest, knowledge, wiki engine, vibecheck, cli/embedded,
…) after import analysis + script/CI consumer verification. Retained
with named consumers: archcheck (arch gate scripts), eval island (rubric
schema drift guards), drwitness/drrebuild (witness-crosscheck).
- **Composition decision recorded**: single cmd/ao root +
internal/commands/<family> modules + clicontract; the never-adopted
cliapp.BuildRoot is removed. New overview: docs/architecture/go-cli.md
(repairs two broken links).
- **Hermetic tests**: flag_matrix_test.go builds the exec'd binary from
package source — the stale cli/bin/ao failure class is closed;
TestHermeticBinaryMatchesSourceSurface pins source↔binary
correspondence.
- **Verdict contract consolidated**: verification extracted from
status.go into cli/internal/verdictcheck; 19-case golden corpus
(tests/fixtures/verdict-contract) runs through Go + Python + JSON schema
via blocking gate `contract.verdict-corpus`. One divergence closed
(empty-string context ids now rejected by all three).
- **Fossil guard chains retired**: embedded-sync (mirror had zero
importers), retired-family compat baselines, dead
go_cli_compatibility.bats, stale gate globs/grandfather entries.

## Evidence
- Full-tier deterministic gate: **66/66 PASS** on the rebased tip (fresh
source-built binary)
- cli suite 2,795 tests green; race suite green on touched packages; vet
+ pinned golangci-lint clean
- Cathedral Cut conformance PASS; CLI reference/surface projections
regenerated and checked
- Independent cross-family validation (fresh Codex context, frozen SHA):
5/6 claims CONFIRMED; 6th NOT_PROVEN only due to sandbox networking (its
cached rerun of the same suite passed)
- Test-count decrease is deliberate and carried by a Test-Removal-Reason
trailer
2026-07-18 16:55:24 -04:00

100 lines
2.2 KiB
Go

// practices: [pragmatic-programmer, twelve-factor-app]
package main
import (
"fmt"
"os"
"path/filepath"
"regexp"
"sort"
"strings"
"testing"
"github.com/spf13/cobra"
)
var commandHeadingPattern = regexp.MustCompile("(?m)^#{3,6} `(ao(?: [^`]+)+)`$")
func TestCobraConformance(t *testing.T) {
// COMMANDS.md documents the published CLI surface; the registered tree
// IS the production tree (no prune step).
rootCmd.InitDefaultHelpCmd()
docsPath := filepath.Join("..", "..", "docs", "COMMANDS.md")
data, err := os.ReadFile(docsPath)
if err != nil {
t.Fatalf("read generated CLI reference: %v", err)
}
liveCommands := visibleCobraCommandPaths(rootCmd)
documentedCommands := documentedCommandPaths(string(data))
missingDocs := difference(liveCommands, documentedCommands)
staleDocs := difference(documentedCommands, liveCommands)
if len(missingDocs) > 0 || len(staleDocs) > 0 {
t.Fatalf("cli/docs/COMMANDS.md is not in conformance with the live Cobra tree.\nMissing from docs:%s\nStale in docs:%s\nRun scripts/generate-cli-reference.sh to regenerate.",
formatCommandList(missingDocs),
formatCommandList(staleDocs),
)
}
}
func visibleCobraCommandPaths(root *cobra.Command) map[string]struct{} {
paths := map[string]struct{}{}
var walk func(*cobra.Command)
walk = func(parent *cobra.Command) {
for _, child := range parent.Commands() {
if child.Hidden {
continue
}
path := strings.TrimSpace(child.CommandPath())
if path != "" {
paths[path] = struct{}{}
}
walk(child)
}
}
walk(root)
return paths
}
func documentedCommandPaths(docs string) map[string]struct{} {
paths := map[string]struct{}{}
for _, match := range commandHeadingPattern.FindAllStringSubmatch(docs, -1) {
paths[strings.TrimSpace(match[1])] = struct{}{}
}
return paths
}
func difference(left, right map[string]struct{}) []string {
var diff []string
for path := range left {
if _, ok := right[path]; !ok {
diff = append(diff, path)
}
}
sort.Strings(diff)
return diff
}
func formatCommandList(paths []string) string {
if len(paths) == 0 {
return "\n - none"
}
var builder strings.Builder
for _, path := range paths {
fmt.Fprintf(&builder, "\n - %s", path)
}
return builder.String()
}