3 Commits

Author SHA1 Message Date
Dicklesworthstone 5f05938484 feat(pages,doctor,status,indexer,sources,gate): pages key CLI, WAL hygiene, segment pressure, setup sync, install fall-through, docs validator, and end-to-end proofs for #439/#440/#441 (coding_agent_session_search-ctigq, -2w1sc, -hr9rn, -q2vn9, -z2uon, -wpe6s, -vg2e1, -tcrxi, -c2xxq, -xwuon, -jwox6, -vdyxd, -apriu)
Correction to 1f642a24's message: the golden leak was not caused by the golden
harness's own stale-on-read refresh — verify7 reproduced it with that opt-out
in place. The writer is tests/cli_robot.rs: 52 call sites use
tests/fixtures/search_demo_data IN PLACE through a base_cmd() that never
disabled auto-refresh, and on a fleet worker the checkout is not under the OS
temp dir, so a robot-CLI search spawned `cass index --background` into the
committed fixture and ingested that worker's real sessions before the golden
tests copied it. base_cmd() now sets CASS_AUTO_REFRESH=0. Proof: verify9 ran
the in-place robot tests before golden regeneration in one job and
search_robot/stats_full_payload came back byte-identical to HEAD.

pages key (ctigq): `cass pages key list|add-password|add-recovery|revoke|
rotate --archive <bundle>` over the complete key_management engine (3,915
lines, zero callers until now). Passwords via prompt or --password-stdin
(current on line 1, new on line 2), never argv; --json; exit 1 engine refused,
2 usage, 3 not a bundle, 6 password required. Four unit tests on a real
encrypted bundle plus a CLI suite (tests/pages_key_cli.rs). docs/RECOVERY.md
rewritten to the real surface; README row.

WAL hygiene (z2uon): forget --apply, dedup --apply and analytics validate
--fix close through the index run's checkpointing close; doctor `archive_wal`
(pass ≤ 64 MiB; warn with size and remedy; warn-and-defer under an index run;
--fix checkpoints and reports pass / blocked / failed truthfully) with a
sparse-WAL e2e in tests/cli_doctor.rs. status/health e2e for db_bytes /
wal_bytes / shm_present, now on the `status` surface too (q2vn9).

Segment pressure (wpe6s, GH #441): quill_bridge::segment_file_count
(metadata-only upper bound), status/health `index.segment_files`, doctor
`index_segments` (warn above 8× the merge threshold, `cass index --full`
remedy), CASS_TEST_SKIP_POST_RUN_LEXICAL_MAINTENANCE hook and an e2e that
builds a 40-segment generation, sees the warning, and proves a plain
`cass index` consolidates it.

GH #439 (vg2e1) and GH #440 (tcrxi) end-to-end proofs: FTS-repair hooks
(sleep per page after the heartbeat; park before the first page) — a slow
repair survives a 20 s abort window, a parked one exits 70 with the
`index-stalled` envelope; a hook parks a force-rebuild between the staged
engine commit and the checkpoint write, records the gap, and a plain
`cass index` resumes with exit 0 and no duplicate identities.

sources (vdyxd, G.1): setup no longer marks the final sync complete without
running it — interactive setup runs `cass sources sync` afterwards and records
it only on success; --json setup defers it and reports sync.status=pending.
Remote install walks candidate_methods() (binstall → verified prebuilt → cargo
install → bootstrap) and falls through on failure instead of stopping at the
first choice. `cass sources sync --all` exists (jwox6): doctor, robot-docs and
the fleet rehearsal recommended it while clap rejected it.

installer (apriu, G.2a): install.sh probes host glibc and falls back to
build-from-source below 2.38; proven on ubuntu:22.04 (2.35 → source route)
and ubuntu:24.04 (2.39 → prebuilt).

docs validator (xwuon, A.9): scripts/validate_docs.sh --keys/--flags/--env/
--truth check README key bindings against the TUI key map (123/123, planted
fake key fails), every `cass … --flag` usage against `cass … --help`, and
every README env var against the code; found and fixed `sources sync --all`,
`timeline --days`, the `<mark>` highlight claim and a dead CASS_UI_METRICS
row. gate.sh gains an opt-in docs-truth stage.

gate (c2xxq): scripts/gate.sh had a false green — verify8 was cut by the
fleet's 30-minute SSH ceiling after fmt+clippy and the script still said
GREEN. Now a non-zero rch exit is RED, every expected stage (plus a terminal
job-complete marker) must report, the receipt captures both streams and
survives a RED run, --lib-only/--no-goldens split runs under the ceiling, the
lib stage runs under `timeout`, and the fleet target dir no longer inherits
the shell's CARGO_TARGET_DIR.

contract (hr9rn): introspect/capabilities has_json_output looks through the
subcommand tree (11 commands flip to true and gain the robot-format arg).
TUI: the unreachable `Alt+W => swarm cockpit` arm removed. CHANGELOG
[Unreleased], AGENTS swarm/issue rules, plan §7b progress log.

Verification (fleet, scripts/gate.sh STAGE receipts, RUST_MIN_STACK=16777216):
- verify9 (2026-09-02 00:35-00:58): fmt 0, clippy 0, goldens-regen 0,
  goldens 0 with search_robot/stats_full_payload byte-identical to HEAD
  after the in-place cli_robot tests ran first (leak fix proven);
  cli_index/cli_robot/cli_status 101 (Codex fixture names, thresholds,
  health nesting) fixed in tree and re-run below.
- verify10 (01:04-01:19): `cargo test --lib` to completion, 6,847 passed /
  6 failed / 38 ignored, no wedge; the six (bookmarks missing from
  CANONICAL_TOP_LEVEL_COMMANDS, four sqlite3-CLI tests on a worker without
  sqlite3, daemon socket path over sun_path on a long TMPDIR) fixed in
  tree and re-run below.
- verify11 (05:08-05:23): fmt 0, clippy 0, lib-tests 0 (canonical
  commands, daemon, sqlite, sources install/setup, pages key,
  quill_bridge, stall diagnostics, fingerprint cache), pages_key_cli 0,
  cli_robot 0, cli_doctor 0, tui_flows 0, bookmarks_cli 0, goldens-regen
  0, goldens 0; cli_index 101 (gh439 reads the last stdout JSON line now)
  and cli_status 101 (health nests state.database) fixed in tree.
- verify12 (05:41-05:47, full lib suite on this tree): fmt 0, clippy 0,
  lib-tests 0 — 6,855 passed / 0 failed / 38 ignored in 144 s.
- verify13 (05:47-05:55, --regen-goldens): fmt 0, clippy 0, lib-tests 0
  (quill_bridge, stall diagnostics), cli_status 0, cli_doctor 0,
  pages_key_cli 0, goldens-regen 0, goldens 0; cli_index 101 because the
  gh439 slow-repair test parsed the last LINE of a pretty-printed summary
  (harness bug) - it now stream-parses the last JSON document.
- verify14 (05:57-06:00, cli_index gh439 after that fix): fmt 0, clippy 0,
  test-cli_index 0 (gate: GREEN).
Goldens: 12 reviewed diffs (segment_files / db_bytes / wal_bytes /
shm_present fields, has_json_output flips, pages key and sources sync
--all in the command trees, doctor archive_wal / index_segments checks);
the four that differ only by the regeneration writer's dropped final
newline were left at HEAD.

Measurement (owner's archive: 10.3 GB db, 200 MB WAL, 267 Quill segment
files; `search "borrow checker lifetimes" --robot --robot-meta --limit 5`,
five warm runs, CASS_AUTO_REFRESH=0): v0.7.1 median wall 15.3 s (search_ms
6.3-7.5 s, other_ms 1.0-1.9 s); this tree (release build) median wall
1.9 s (search_ms 337-406 ms, other_ms 754-810 ms after a first-run 2.6 s
sidecar miss). hits=1 on both. Not yet isolated: the engine-time drop
happened with the segment count unchanged, so the per-open seal in the old
Quill config is the likelier cause than fragmentation on this archive.
Consolidation of the owner's 267 segments was NOT achieved: the documented
path (`cass index`, and `cass doctor --fix` for the WAL) never leaves the
frankensqlite writable open on that archive - one core, 6.7 GB RSS,
~30k 24-byte header reads/s on the 200 MB WAL, no progress in 7 min -
while read-only opens take 1.5 s. On a COPY whose WAL C SQLite checkpointed
losslessly (1,012 conversations / 538,807 messages before and after), the
same `cass index` gets through the open and indexes, so the WAL is the
trigger; C SQLite's quick_check on that copy also reports a doubly-
referenced freelist page and free-space corruption on ~35 pages that
`cass doctor` calls healthy. The archive's last completed index is
2026-08-14. Recorded as a new P0 in the plan (section 7b); the live
archive was not modified.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-02 06:18:55 -04:00
Dicklesworthstone c59c5da9b8 fix(scripts): rch-wrap docs validation cargo 2026-05-08 04:16:06 -04:00
Dicklesworthstone 3ecbf19ca9 Add documentation testing infrastructure (P6.12)
- tests/docs/readme.rs: Tests for README.md generation accuracy
- tests/docs/help.rs: CLI help and generated HTML validation tests
- scripts/validate_docs.sh: Shell script for link and content validation
- docs/DOCUMENTATION_STYLE.md: Documentation style guide
- Cargo.toml: Added [[test]] entry for docs test crate

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-21 13:20:15 -05:00