177 Commits

Author SHA1 Message Date
Jeff Emanuel bdf80e0090 fix(search): preserve native component module layout in the execution harness
The first native WAL Actions run reached rustc but failed before executing
any tests: an absolute path attribute made split ANN child modules resolve
under query/ rather than query/ann_shards/. Mirror the actual module tree
byte-for-byte into the temporary extracted module and declare it normally.
Include every mirrored child in the source-hash receipt. Reject destinations
inside either checkout; never change shipping source to fit the harness.

Apply the compiler-reported formatting changes to the two WAL files from
this work. No implementation behavior, dependency, workflow, test assertion
or production module visibility changes. Preexisting formatting findings in
the unchanged ann_shards.rs parent are not claimed fixed by this commit.

Validation: Python syntax and four-file byte-identical module-layout checks
passed locally; current base and uploaded blob identities verified. Actions
35646917646 failed with E0583 and cascading missing-method diagnostics, so
it provides no native test pass. The corrected harness still requires fresh
Rust execution; full CASS, Clippy, UBS and RCH qualification remain pending.
No gate was disabled or downgraded. Preserve concurrent main through 6a0a834.

Direct main via GitHub, no PR and no force update.
2026-09-21 16:05:16 -04:00
Jeff Emanuel 31375c9a63 fix(build): unblock recovery qualification without disabling SQLite readers
The full locked CI build stopped in build.rs before reaching CASS tests:
its exact FAD feature contract omitted copilot-vscdb and openclaw-sqlite,
which Cargo.toml intentionally enables for SQLite transcript ingestion.
Admit both existing features without weakening equality, changing sources,
or removing connector functionality.

Add a dependency-free feature preflight and six regression tests so this
class of drift is caught before compiling the dependency graph. Add a
read-only main-push CI job running the actual locked artifact lifecycle,
approval, recovery binary and separate-process CLI suites; require nonzero
executed tests and preserve the full production dependency graph.

Validation: six Python tests passed; YAML parsing and embedded bash syntax
checked; original and uploaded Git blobs verified; diff whitespace clean.
Rust/cargo unavailable locally. Compilation and recovery test execution
remain unverified until the dedicated CI job completes.
2026-09-20 21:11:30 -04:00
Jeff Emanuel bd1345fb78 fix(semantic): harden artifact ownership and cover killed-run recovery (#490)
Replace Deref delegation with explicit forwarding so associated-method calls
remain compatible and callers cannot bypass the backfill artifact lease.
Reuse the existing bounded generation-selection validator for cleanup authority;
require observed metadata to match the durably pinned pointer and manifest.

Add subprocess hard-kill/startup recovery, shard/ANN ownership, orphan WAL,
invalid selected generation, symlink/alias and associated-method regressions.
The crash test kills the lease-owning child without running destructors and
asserts that live and resumable FSVI files remain byte-for-byte intact.
Keep the source-extraction verification harness pointed at the unchanged
semantic engine after its module extraction.

Preserve the intervening lexical repair commit. Validation: static review,
Rust lexical delimiter/whitespace checks, Python AST parsing and exact uploaded
blob matching. Rust/cargo/rustfmt are unavailable; compilation, formatting gates
and Rust tests NOT RUN.
2026-09-20 16:59:39 -04:00
Jeff Emanuel 3357a3353e test(openclaw): complete native and legacy consumer qualification
Add the existing legacy target's chrono dependency to the disposable probe.
All four wrapper unit tests, including real WAL recovery through the registry,
passed on Linux; integration compilation was blocked solely by that missing
harness dependency. Preserve every production module and assertion unchanged.

Retain compiled engine artifacts even on failed checks so subsequent native
qualification does not rebuild the full SQLite stack from a registry-only
cache. No production dependency, registry or runtime behavior changes here.
2026-09-19 22:54:56 -04:00
Jeff Emanuel 35d40ce30b fix(openclaw): honor isolated state profiles across detection and scanning
Support the documented OPENCLAW_STATE_DIR override, including whitespace and
home-component expansion. Detection returns a root covering both native and
legacy stores; default discovery and scanning select the same profile, and
explicit caller roots remain authoritative. Never fall back to the real home
when an explicit profile is missing. Keep schema/event parsing upstream.

Use the explicit state contract in the native-only subprocess regression,
instead of assuming HOME/USERPROFILE redirect Windows known-folder APIs.
The first native run passed all eight tests plus Clippy on Linux and seven
of eight tests on Windows; only that home-redirection fixture failed.

Fix the qualification script's registry anchor (match the return statement,
not closure arguments). Retain all native, legacy and registered WAL tests,
use line-free debug builds only in the disposable probe to avoid huge linker
symbols, and allow the cold Windows engine build to finish.
2026-09-19 22:39:23 -04:00
Jeff Emanuel 81eaf5d3f7 test(openclaw): retain the legacy consumer suite in native qualification
Run the existing JSONL integration target unchanged alongside the eight
native regressions and the registered WAL adapter tests. Use the existing
CASS 128 MiB stack reservation rather than lowering its engine test budget.
No assertions, reader logic, dependency versions or source pins change.
2026-09-19 22:29:00 -04:00
Jeff Emanuel 77fd9cbbd4 test(openclaw): qualify the runtime factory adapter with WAL recovery
Exercise the same final registry adapter from the WAL-only regression and
ensure unrelated factories remain unchanged. Prepare the one-line registry
routing change alongside the native feature manifest; preserve every other
CASS-specific wrapper. The focused probe verifies the production seam and
compiles its actual adapter, without pretending to build the other wrappers.

Retain the formatted wrapper, consumer regression, candidate registry and
manifest only after both native platforms pass. The final GitHub commit
will land those exact blobs together; no runner updates branches.
2026-09-19 22:24:17 -04:00
Jeff Emanuel 9b50f9dcd9 fix(openclaw): discover and ingest committed WAL-only updates
Recover native databases that FAD 0.3.0's main-file mtime gate skips even
though their WAL is fresh. Parse each selected database through the existing
schema-admitted read-only connector, then apply its session freshness rule
while keeping the complete message prefix. Suppress migration leftovers
only within the same physical agent directory, including stale native-owned
sessions. Preserve legacy fallback when a native store yields no history.

Declare existing WAL files as required reconstruction sidecars in discovery.
Add a real committed-WAL regression that makes only the main database stale,
checks discovery and full session recovery, and verifies DB/WAL bytes and
mtimes are unchanged. Continue qualification before enabling GH #487's feature.
2026-09-19 22:20:10 -04:00
Jeff Emanuel e60357f434 test(openclaw): qualify native SQLite ingestion through the CASS consumer
Add eight unconditional consumer regressions for GH #487: native source
selection, session ordering and metadata, complete incremental sessions,
legacy migration deduplication, agent identities and narrow scopes, tool
arguments, schema admission, isolated native-only detection, and committed
WAL reads without modifying the source DB or WAL.

Use the real CASS OpenClaw module and synchronous SQLite bridge with the
published engine versions from CASS's lockfile. First qualify a candidate
manifest enabling openclaw-sqlite; GitHub will land the tested manifest in
a follow-up direct-main commit. The runner only stages blobs, never refs.
This is a connector consumer gate, not a full CASS archive/TUI qualification.
2026-09-19 22:13:03 -04:00
Jeff Emanuel d3a07e991a test(raw-mirror): include the real database bridge in capture qualification
The first native run exposed raw-mirror prune's compile-time dependency on
CASS's SQLite facade. Stage the real franken_sync module and use the pinned
FrankenSQLite/Asupersync packages, preserving their manifest features.
Use a real read-only engine connection for keep-tag fixture queries; this
focused harness does not qualify the full storage open/recovery policy.

Add --full to run the actual CASS raw-mirror and CLI acceptance targets with
no temporary crate. The production capture engine and guards stay unchanged.
2026-09-19 20:52:18 -04:00
Jeff Emanuel 1b0be16cae fix(raw-mirror): enforce scan exclusions before fallback capture
GH #486 still had a privacy bypass: an empty filtered Codex inventory makes
the indexer fall back to explicit-root raw capture, which did not consult
CASS_EXCLUDE_PATHS. Guard both public and chunk-policy capture entry points
before source I/O, mirror initialization, locks, cache reuse, or manifests.

Preserve the storage engine byte-for-byte behind the admission facade; do
not alter format, pruning, sensitive-provider guards, or existing captures.
Add isolated real-Codex/real-storage regressions for all-excluded fallback,
legacy archives, prefix siblings, missing files, warm caches, and both
capture routes. No dependency pins or stored user data are changed.
2026-09-19 20:43:51 -04:00
Jeff Emanuel 8619fa7a21 feat(codex): ingest archived rollouts without losing incremental history
Discover Codex's archived_sessions alongside active sessions from default,
CODEX_HOME, full-home and mirrored roots. Keep narrow session/file scopes
narrow and use the same inventory for discovery and guarded recovery.
Archive moves preserve mtime, so inventory archives independently of the
active cutoff and leave unchanged-source reuse to the existing host hook.
Preserve full-home native dated IDs across archive/unarchive moves while
retaining explicitly scoped archive identities and original provenance.

Separate the public source-selection surface from the byte-identical
existing parsing/enrichment implementation. Add real-consumer regressions
for both collections, old archived files, archive-only homes, exclusions,
root scopes, remote provenance, identity stability and failed archives.
Use existing published dependencies; no pin or lockfile change.
Also correct the import ordering left by the prior resume-test commit.
2026-09-19 19:45:36 -04:00
Jeff Emanuel a897707f28 test(codex): validate upstream reader changes in the real CASS consumer
Add --fad-checkout for pre-publication compatibility testing without changing
CASS's production registry pin or lockfile. Override only the temporary test
crate and verify Cargo metadata resolves the requested checkout, so an unused
patch cannot silently test the old published connector instead.

Keep the default published-dependency mode and all existing consumer tests.
2026-09-19 16:43:07 -04:00
Jeff Emanuel 687e5afc85 test(codex): preserve module layout in the native consumer harness
An absolute #[path] changes Rust's nested-module lookup base. Stage byte-
identical production files with their normal layout instead, without editing
or mocking the connector. Include the existing real-source containment tests
so the exclusion fix is checked alongside oversized-source error semantics.
2026-09-19 15:42:39 -04:00
Jeff Emanuel b2ce6c3b5d test(codex): exercise the production consumer on Windows and Linux
Build the actual CASS Codex module, its unit tests and the GH #486 public-API
regressions in a focused temporary crate. Read dependency versions and the
initial lockfile from CASS, rather than silently testing an upstream checkout
or maintaining a second pin. Check source-budget and enrichment behavior too.

This complements, and does not claim to replace, the full archive/TUI suite.
2026-09-19 15:38:08 -04:00
github-actions[bot] 98f30eff75 fix(search): bind partial semantic tiers to the authenticated selected corpus
Require partial fast/quality tiers to be subsets of the complete companion corpus through retained FSVI-order merge walks. No extra corpus-sized ID set or file opens; two partial tiers still rely on publication commitments. Prove both previously accepted foreign-document cases RED against unchanged parent production code, then GREEN with the same engine and dependency lock. Add sparse-coverage, failed-refresh, cross-version rollback, retained-lease and fresh-process multi-shard retrieval tests. All 147 component unit tests and four existing public-manifest process contracts passed. These actual production modules in a separate package do not qualify the full CASS binary; shipping dependency publication and progressive CLI integration remain separate. No model, archive, dependency-pin or repository lockfile changes.
2026-09-18 21:34:20 +00:00
github-actions[bot] cf10034c79 feat(search): publish and retain complete ordered semantic shard cohorts
Add explicit version-2 partition metadata while preserving v1 JSON and digest behavior. Validate complete bounded lexical partitions and exact ordinal-specific ANN bases; admit every selected shard with immutable owner and checked live-row extrema. Reconstruct whole-corpus commitments by a bounded-head merge in native FSVI hash order, including multi-row and tombstoned images. Keep partial quality coverage explicit and missing graph receipts on exact fallback. All 140 component tests passed against pinned real upstream code in run 35396526034, including 14 new real-file regressions; all committed bytes match its immutable artifact. This separate dependency graph does not qualify the shipping CASS binary or enable its progressive CLI route. Shipping frankensearch =0.6.1 remains unpublished. No repository dependency pins, lockfile, models or user archives changed.
2026-09-18 21:28:36 +00:00
Jeff Emanuel 140e3492d2 test(index): reproduce destructive failure cleanup in the live direct writer
Normal CLI/watch and daemon indexing use the direct writer, not the prototype shard builder. Prepare seven real-file regressions through its public build entry points and extend the existing independent harness to extract those exact methods. Require two parent failures proving main-file deletion, with and without pending WAL, then all 32 component tests GREEN before the source-only runtime commit.

The reviewed fix removes caller-owned deletion of a destination the buffered engine writer has not replaced and preserves typed causes with anyhow context. No source is silently substituted, no dependency pin changed, and no full CLI/model/RCH/UBS pass is claimed.
2026-09-18 11:16:48 -04:00
github-actions[bot] 2b27d992d1 fix(index): isolate semantic shard builds before manifest publication
Allocate a unique generation directory for every build, including rebuilds of the same database fingerprint. Publish only completed vector/ANN paths; failed builds cannot overwrite installed files. Strictly reject malformed/future manifests at preflight and publication, preserve other tiers, and avoid allocation from the shard-size limit before input arrives. Failed unreferenced generations remain for explicit inspection/GC; no old artifacts are removed.

RED reproduced installed-path reuse with the unchanged production writer. GREEN ran 25 tests of exact writer/manifest methods and the actual hash control producer against real pinned FSVI/HNSW. Component Clippy exit=101; locked CASS test exit=101. This independent graph is not full CLI/model/RCH/UBS or archive-scale qualification. Concurrent manifest publication remains separate.
2026-09-18 15:08:37 +00:00
Jeff Emanuel 449f971da7 test(index): enable real hash producer bindings in shard publication qualification
The first execution stopped on missing harness exports and a disabled upstream hash feature, before the regression ran. Enable that actual upstream feature and reexport the real async adapter/trait required by the existing CASS hash tests. Preserve the production patch and all assertions, and rerun the same RED/GREEN source-only landing. No production source has been committed by the failed runner.
2026-09-18 11:01:49 -04:00
Jeff Emanuel ac385aed0e test(index): reproduce unsafe semantic shard replacement before isolated publication
Prepare the manually reviewed build-isolation patch and six real-file regressions. Require the unchanged parent writer to reproduce installed path reuse before the patched writer and actual hash control producer pass real FSVI/HNSW component tests. Only then does the guarded source-only runner publish runtime wiring directly to main.

No existing source, archive, dependencies or lockfile changed in this preparation commit. Full CASS qualification is recorded independently; native semantic relevance and concurrent manifest publication are not implied by this component check.
2026-09-18 10:52:49 -04:00
Jeff Emanuel d66d05d1c5 test(search): reproduce false exact completeness after initial-window deduplication
Prepare two real-FSVI RED/GREEN cases: a short first window hiding later messages and a filled multi-shard page with the wrong second result. Require the parent to fail at the intended completeness assertions, then require all 43 component tests to pass after the narrow production patch before source publication.

Use the retained exact readers, unchanged engine pin and same temporary dependency lock. No full-CASS qualification is claimed; no archive or dependency manifests are modified.
2026-09-18 09:38:49 -04:00
github-actions[bot] e36ad641e9 fix(search): recover filtered and chunk-starved ANN pages through exact retrieval
When a valid native cohort underfills its message page with unsearched candidates remaining, execute the existing bounded exact driver over every retained shard with the same filter. Replace rather than mix native/exact scores, preserve native work counters, and emit a typed exact-fallback receipt. Preserve full native pages, exhausted scopes, zero requests, invalid-owner rejection, and read-only artifact ownership.

Executed 41 production-component tests against real pinned FSVI/HNSW, including six new recovery regressions and complete production receipt serialization. Component Clippy exit=0. This separate dependency graph does not qualify the full CASS binary; its locked build remains independently blocked by frankensearch =0.6.1 publication. No model/CLI/RCH or archive-scale pass is claimed.
2026-09-18 13:36:15 +00:00
Jeff Emanuel 97eb4ccd80 feat(search): prepare exact recovery for filtered and chunk-starved ANN pages
Include the reviewed source patch and six real-index regression cases. A hash-guarded source-only runner must execute all 41 component tests before committing the runtime wiring to main. Native counters remain separate from the typed exact-fallback receipt. Full CASS qualification remains a separate blocking check; no dependency pins or archive files are changed.

Preserves current main and concurrent indexing work; no PR or forced branch update.
2026-09-18 09:27:03 -04:00
Jeff Emanuel 0c8243ca38 fix(index): preserve contention retries for uncached agent and workspace registration 2026-09-18 02:20:48 -04:00
Jeff Emanuel 212ba663c2 fix(index): retry cold writer setup without replaying canonical or lexical side effects 2026-09-18 02:15:55 -04:00
Jeff Emanuel d70e8e6874 fix(search): handle session-scoped exact fallback in the CLI warning match 2026-09-18 02:09:02 -04:00
Jeff Emanuel a2464e31a6 feat(search): implement retained all-shard native ANN admission and merging
Add a native ANN cohort retaining every explicitly paired FSVI owner. Admission uses the existing no-rebuild validator and is all-or-nothing; missing, swapped, invalid, or mismatched sidecars never drop a shard. Validate shared dimensions and explicit v2 embedding spaces, then query every graph and bound intermediate message merging to the requested page. Aggregate actual per-graph work without presenting the recall heuristic as measured recall.

Add ten real-native-graph tests covering all-shard retrieval, duplicate and metadata handling, failed admission, retained owners after rename, stale contexts, invalid input, and read-only behavior. Extend the existing independent real-FSVI harness with native HNSW enabled. Tests are authored and queued, not yet claimed passed. Production routing will be switched in the next meaningful commit after qualification; dependency pins and archive formats are unchanged.
2026-09-18 02:03:24 -04:00
Jeff Emanuel bbf068ea9a test(search): qualify bounded message refills with real FSVI and scoped multi-shard cases
Add seven real vector-file regressions: dominant chunks beyond both old windows, metadata/session intersection, cross-shard duplicate collapse, message-ID tie ordering, unchanged proven fast path, one shared refill budget, and bounded initial-merge equivalence. Preserve the 12 executed selection-kernel tests.

Provide a reproducible narrow harness extracting exact CASS driver/filter/artifact bodies against pinned upstream FSVI 6852e79, without modifying CASS dependency pins or its lockfile. The guarded application wires tests and bounds the initial shard map, then commits only after all 19 harness tests pass and retires writer permissions in the same source commit. Full CASS qualification is attempted and reported separately. New integration tests are authored, not yet claimed passed.

Based on current main 8c4fd784; concurrent GH473 indexing changes preserved.
2026-09-18 01:08:26 -04:00
github-actions[bot] 8c4fd78437 fix(index): distinguish typed permanent preflight errors from contention (GH473) 2026-09-18 05:02:06 +00:00
Jeff Emanuel a8ecedfd4e test(index): resolve GH473 probe dependencies from the production manifest section 2026-09-18 00:56:41 -04:00
Jeff Emanuel 8594623581 feat(search): implement bounded exact message refill for chunk-dominated results
Add a page-bounded selection kernel that excludes retained message IDs while refilling chunk windows and resolves score ties using a decreasing message-ID ceiling. Backend or work-budget failures never become incorrectly ranked exact successes. Preserve the existing proven first-window fast path and all session/metadata predicates in the accompanying guarded source patch.

Twelve authored Rust tests include 1,920 deterministic exhaustive-oracle comparisons. A narrowly scoped runner must execute those exact kernel tests before applying production wiring directly to main. No full CASS build or runtime pass is claimed by this preparation commit. No dependency or archive changes.

Based on the latest 6bd4129 main; concurrent indexer work is preserved.
2026-09-18 00:54:23 -04:00
Jeff Emanuel fd8c318fb5 test(index): separate exact-engine contention evidence from the full CASS qualification gate 2026-09-18 00:51:07 -04:00
Jeff Emanuel 607a9a1133 fix(search): prepare hash-guarded landing of session-scoped vector selection
Carry the reviewed session-scope implementation as an ordinary source patch and apply it once on a GitHub runner to avoid retranscribing the nearly 1 MB query module through the contents API. The workflow verifies exact source and patch blobs, rejects existing module/dirty checkout, stages only three source paths, and pushes a normal fast-forward commit directly to main. It does not open a PR, force-push, change branch protection, or alter existing CI/release workflows.

The patch resolves bounded authoritative message IDs through the existing read snapshot before vector top-k, intersects all metadata predicates, uses exact retained owners for scoped ANN requests, and suppresses unused ANN/two-tier setup. Nine authored tests cover candidate starvation, metadata and source restrictions, malformed/negative IDs, literal Unicode paths, bounded collection, rollback, and multi-shard merge. Correct the old patch's mismatched io::Error/FrankenError row mapper.

Local validation: git apply on synthetic contexts produces the exact reviewed edits and new module; whitespace and test-count checks pass. No full checkout application or Rust compilation/tests/Clippy/rustfmt/UBS/RCH execution has occurred in this editing environment. The workflow is transport, not runtime validation.
2026-09-17 23:57:00 -04:00
Dicklesworthstone e5005bf140 fix(gate): install pinned UBS language routing contract
55wh0: fresh private UBS runners require modules/contract.json before classifying explicit targets. Fetch the version-matched contract and verify its reviewed SHA256 before scanning; preserve strict scanner flags and existing module verification.

Normal RCH ovh-a regression 2026-09-16 15:53:47: baseline EXIT=3 reproduced; fixed clean EXIT=0 and fixed critical EXIT=1, both executing rust/python/bash. Probe job EXIT=0, source SHA stable; receipt .release-format/scarletstone-ubs-routing-20260916/results/receipt.json. git diff --check passed. Full release UBS findings remain blocking.
2026-09-16 11:53:59 -04:00
Dicklesworthstone 88f1b8e7ca chore(gate): pin qualified UBS 5.4.4
Use the release with indexed suppression lookup and exact finding parity. RCH scoped receipt: STAGE=syntax EXIT=0; STAGE=runner-digest EXIT=0; STAGE=ubs EXIT=0; STAGE=source-stability EXIT=0. Upstream all 27 suite stages and strict self-scan passed. This scoped gate does not clear the remaining CASS Rust findings.
2026-09-15 21:21:50 -04:00
Dicklesworthstone ae4dd92e5a chore(quality): pin qualified UBS 5.4.3 runner and digest
Use tagged scanner fixes for source matching and scoped suppressions. The exact GitHub tag runner digest matches 656f1a449754531ae3f5362ddb96566bb2ab0a884e5f3d0074a79bc22318a598. Remote ovh-a receipt .release-format/greenfinch-ubs543-pin: STAGE=shell-syntax EXIT=0; STAGE=ubs EXIT=0 (one gate script, 0 critical, 0 warning); STAGE=source-stability EXIT=0. Upstream full suite stages and strict self-scan are qualified. This validates the pin update only; CASS product runtime and release gates remain pending.
2026-09-15 20:24:04 -04:00
Dicklesworthstone cc80307b62 fix(gate): pin UBS into a private module dir for the scan
A shared UBS module cache can be replaced by another pinned invocation
while this gate is scanning. Always copy or download the pinned runner
into a private temp directory, pass --module-dir there, and fail if
cd into PROJECT_ROOT fails. The dpfvr e2e script now also fails closed
on those cd steps.
2026-09-12 12:45:25 -04:00
Dicklesworthstone 42f35b74c4 fix(gate): count nested cargo test summaries without inflating binaries
Inherited subprocess output from a lib test that self-spawns appears
between the outer `running N tests` line and that binary's final
summary. The previous counter treated every `running` / `test result:
ok` pair as a top-level binary, so a nested child either inflated
PASSED/BINARIES or flagged a complete parent as incomplete.

Track announcement depth: count a binary only when depth returns to
zero, require the ok-summary's passed+failed+ignored+measured to match
the announced count, and treat any other `test result:` line (including
FAILED or a truncated parent) as incomplete.

The dpfvr e2e harness now covers nested children counted once,
deep nesting across two binaries, a completed child with a truncated
parent, a parent closing a missing child, nested empty/failed children,
announced-vs-accounted mismatches, ignored+measured still totaling N,
a false `ok` with nonzero failures, and a malformed terminal summary.
2026-09-12 12:39:23 -04:00
Dicklesworthstone b8fc786ae4 fix(index): preserve incremental work and complete pending analytics safely
Retain Devin commits from the native watermark second, preserve source
completion across bounded and signal-driven stops, and repair stale initial
analytics cursors without double-counting. Expose pending OMP analytics in
status/doctor/validation and complete all projections under the existing
maintenance lock while preserving published search and canonical identities.

Keep statistics errors and ordering truthful; combine unfiltered conversation
scans without claiming a measured speedup. Extend real CLI coverage for
deferred analytics, WAL appends, replay, interrupted resume and timeout output.

RCH v8c: STAGE=fmt EXIT=0; STAGE=clippy EXIT=0;
STAGE=lib-tests EXIT=0 (225 passed); selected integrations86 passed;
STAGE=goldens EXIT=0 (68 passed). Original OMP helper failed by reading the
wrong error stream; its retained test-only follow-up passed1/1 with
STAGE=fmt EXIT=0, STAGE=clippy EXIT=0 and STAGE=test-connector_omp EXIT=0.
All remaining runtime inputs and application ELF were unchanged.

Full gate is RED: STAGE=ubs EXIT=1 (Rust300s timeout; original Python warning).
The separately reviewed Python line pragma passed its pinned strict scan and
16 controls. This does not clear Rust UBS or authorize publication.
Large-copy stats10.25s/1498792KiB showed no clear improvement.
All26 original issue acceptances remain open; no release or native proof.

Broad receipt SHA256: d9552cf8bd1bf7b31766b141594513c19212c7fd575cc8ebc1a9c149117014ef
OMP receipt SHA256: f51bc7bb92563b1f574779024377f939be13f0e4ee9c4592ffe33a0c758f2135
Application SHA256: 1182c93d1ba643a1988cb628796d6e33a29c2fed6fd81ca16d5f9af4f75d4241
2026-09-11 04:14:37 -04:00
Dicklesworthstone 8ec967ae32 Parse live-fleet JSON without echoing private inventory on decode failure.
json_document raises ValueError with line/column only, so a malformed
inventory or remote command receipt stays in its existing file. Sync,
discovery, Tailscale baseline, and mirror-reingest all use it. UBS
acceptance asserts that require JSON booleans and exact identity sets
are marked ubs:ignore so the scanner does not rewrite them.
2026-09-11 01:45:18 -04:00
Dicklesworthstone 5d187f8760 Unreleased: Shelley connector, reingest --source, schema-only opens, backfill skip cache.
Enable FAD 0.2.4 `shelley` in Cargo.toml/build.rs and document CASS_SHELLEY_DB
defaults, live WAL/SHM watch, CASS_SKIP_SUBAGENTS parent-ID skip, and the
sensitive-container policy. Raw mirroring and remote DB ingest stay disabled
for Shelley; doctor will not promote a Shelley file into a repairable mirror
(GH#415 / 3kde4).

sources reingest --source now selects remote roots instead of only narrowing
the report, preserves local watermarks so later ordinary indexing still finds
local history, fails mixed valid/unknown names before archive creation, hashes
configured paths including a bare `~`, and no longer opens the database before
the index lock (av59c). Live-fleet harness resolves TMPDIR before the
containment check so a symlink cannot plant the private inventory in-tree.

Current-schema archive opens use schema-only admission and skip whole-database
hydration while keeping required migration/repair checks (#443/#450). Semantic
backfill rebuilds the known legacy hash-vector format; on Unix an unchanged
completed backfill can skip canonical replay, bound to archive/WAL/vector and
producer identity. That cache is a maintenance skip hint, never search-serving
authority (#458). Combined remote validation and remaining issue closure are
not claimed.
2026-09-10 22:46:14 -04:00
Dicklesworthstone 28f65e9fd6 Treat null TailscaleIPs as empty and compare live-search content plus provenance (av59c).
merge_tailscale_hosts deserializes TailscaleIPs as Option<Vec<IpAddr>> so JSON
null or a missing field skips that peer instead of rejecting the whole status
document. The mixed-peer unit test now includes an online peer with null
addresses.

The live harness compares (source_id, path, line, origin_host, content)
across global, source-scoped, and default-hybrid hits, not just identity
triples. CHANGELOG/CHANGELOG_RESEARCH record the 02:45 UTC review: 91 tests,
nine authenticated machines, tenth still needs auth, UBS MODULE_TIMEOUT.
Final ELF 7241298dc93353fe3aef6456c092bef5ac87c36831e43a84b9c6f990551268d4.

Does not close av59c. Skip .release-format/.
2026-09-10 03:12:44 -04:00
Dicklesworthstone 04f0163fdf Bound SSH/Tailscale child stdout/stderr and keep draining overflow (av59c).
wait_for_child_output_with_timeout now wraps wait_for_child_output_with_limit
(unlimited). When max_bytes is set, each pipe reads at most limit+1 then
drains the rest to sink so a chatty child cannot deadlock on a full pipe;
overflow is InvalidData. Tailscale status already calls the limited helper.

Unix tests: 4-byte limit keeps 1234/abcd, rejects 5-byte stdout or stderr,
and still times out after overflow plus sleep. The live harness catches
Exception (not a short list) so unexpected failures write failure.json
without printing private paths.

Skip .release-format/.
2026-09-09 22:07:15 -04:00
Dicklesworthstone da54a759f7 Treat Match as a new SSH block and skip already-configured hosts by address (av59c).
parse_ssh_config now closes the current Host set on Match so conditional
options cannot attach to the preceding literal alias. Host tokens are split
with shell_words (quoted aliases), stop at comments, and reject glob/negate/
unsafe/leading-dash names. Hostname/User/Port/IdentityFile keep the first
value so a later HostName cannot overwrite a quoted address.

sources discover --skip-existing treats a host as configured when the source
name matches or the SSH target host (after user@) matches the discovered
name/hostname, so a Tailscale IPv4 peer that is already operator@example
is not offered again. JSON already_configured uses the same predicate.
will_reindex is true only when indexing will actually run (files > 0).
The discover next-step hint uses cass sources add <user>@<host> --name --path.

Tests: Match+quoted HostName first-wins still surfaces the extra tailnet
peer; --skip-existing --json reports all_existing for a configured target.
The live harness requires source-scoped identities to equal the global hit
set, refuses PYTHONOPTIMIZE, and maps invalid inventory/config to a parser
error without printing private paths.

Skip .release-format/. Does not close av59c.
2026-09-09 22:07:15 -04:00
Dicklesworthstone a8655107b3 Document --tailscale in CHANGELOG and require a Tailscale-only inventory alias (av59c).
CHANGELOG notes optional sources discover/setup --tailscale IPv4 merge with
SSH-config fallback. CHANGELOG_RESEARCH records that the new capability is
not covered by the earlier nine-host SSH proof; a frozen remote gate and
private live tailnet run are still pending.

The live harness --tailscale path now compares against SSH-only discover and
requires at least one inventory target that is absent from configured SSH
aliases, so a run cannot pass on SSH-config discovery alone.

Does not close av59c. Skip .release-format/.
2026-09-09 18:42:05 -04:00
Dicklesworthstone cc32c30df5 Prove --tailscale discover fallback and opt-in live Tailscale fleet discovery (av59c).
sources discover --tailscale --json with an empty PATH keeps the SSH Include
hosts and reports discovery_warning containing "could not start", using a
missing executable directory rather than a fake tailscale binary.

The live-fleet harness gains --tailscale: discovery then requires
--tailscale and an empty discovery_warning. Inventory SSH targets are
expected to be tailnet IPs when that flag is set. Default runs stay
SSH-config-only.

Does not close av59c. Skip .release-format/.
2026-09-09 18:36:07 -04:00
Dicklesworthstone 182c197b1c Honor CASS_SSH_CONFIG Includes and emit one JSON document for sync/reingest indexing (av59c).
Discovery previously always read ~/.ssh/config and ignored Include files, so a
private override that transport already used produced an empty host list while
direct SSH through the same file worked. Discovery now follows the same
CASS_SSH_CONFIG / ssh_config_override path, concatenates Include fragments with
bounded recursion (depth 16, 256 files, 1 MiB), skips duplicate aliases, roots
relative Includes at ~/.ssh per ssh_config(5), and never executes Match exec.

sources sync --json and sources reingest --json used to print a premature sync
document and then a second indexing document. run_index_with_data now accepts
an optional captured_result so robot sync/reingest emit one nested payload.
Indexing lock refusal reports status index_failed with indexing.code 7 and
keeps the nonzero exit, rather than advertising completed indexing.

The live-fleet harness wraps the operator SSH config with Host
cass-live-unreachable, records discovery-pending ordinals, holds
index-run.lock via fcntl, asserts expected_exit 7/8, numbers append events,
resolves ssh -G users instead of substituting the runner, and checks default
hybrid identity against lexical hits. README documents the opt-in inventory
outside git (example hosts laptop/workstation, not css/csd/yto). CHANGELOG
and CHANGELOG_RESEARCH record the nine-reachable-host proof (18/18/18/27/36
hits; busy exit 7; refused-SSH exit 8) and the 9lz4y rustfmt-only doctor
collector already on main.

Tests: sources_discover_uses_private_ssh_config_override_and_includes and
sources_reingest_json_preserves_index_result_and_busy_exit.

Does not close av59c: UBS MODULE_TIMEOUT, auth-blocked tenth host, and
setup-timeout host selection remain open. Skip .release-format/ scratch.
2026-09-09 18:36:07 -04:00
Dicklesworthstone 761e4e5374 Add an opt-in real-SSH fleet search harness that never stores host inventory in git (av59c).
scripts/e2e/live_fleet_search.py takes --inventory and --cass-bin. The
inventory file must live outside the repo with mode 0600. Each host gets
a fresh cass-live-fleet temp dir and a synthetic Codex rollout session;
only those sessions are transferred. Strict host-key checks, no archive
edits, no file deletes. Console output uses ordinal host labels, never
identities. Unreachable hosts fail the run rather than skip. Skip
.release-format/.
2026-09-09 17:24:51 -04:00
Dicklesworthstone 73db130584 Refresh verified compile inputs after source-identity so Cargo cannot reuse a stale fingerprint.
F6 failed before tests because a warm worker Cargo fingerprint still
rejected Devin while the frozen build.rs already listed it. rch transfers
preserve mtimes, so an input whose bytes changed can still look older than
the cached fingerprint.

prepare_compile_inputs now SHA-256s the admitted path set (source-identity),
then touch -c those exact paths and re-digest (source-freshness). touch -c
never recreates a missing input. A concurrent edit or a missing file fails
closed before any compiler stage. EXPECTED_STAGES includes source-freshness.

The dpfvr e2e harness proves: old inputs become newer than a retained
fingerprint, bytes stay identical, a mutated input is refused before
source-freshness, and a renamed input is not recreated. No Cargo
invocation is admitted in that test.
2026-09-08 18:13:20 -04:00