Two bugs in flush_clone/post-processing fixed:
1. Overlap guard off-by-one: the strict check fired for adjacent
duplicate blocks sharing exactly one boundary token (ex_end == cur_start).
This caused the primary loop to discard legitimate consecutive self-clones
(e.g. haxe/file1.hx blocks at lines 6-47 and 47-88). Fixed to use strict
less-than on both ends: ex_end > cur_start for the overlap condition.
2. Secondary pass over-generation: the open_clone SM suppresses sub-windows
for cross-file detection but the secondary pass still generates overlapping
sub-clones for intra-file matches (repeated_windows has all adjacent
windows; coverage tracking misses them when primary produces no primary
clone). Re-add suppress_subclones (O(n^2) on clones, not on tokens) to
remove fully-contained sub-clones after dedup_exact_clones.
Regenerate tests/golden.json from the corrected binary: 218 stable pairs.
Regression gate exits 0 on three consecutive runs.
- Sort format groups and files within each group by id before detection to
eliminate FxHashMap iteration non-determinism; output is now stable across runs
- Remove &mut store argument from detect() test call sites (API changed in wave 2)
- Regenerate golden.json from deterministic binary (283 stable pairs)
Wave 1 of the perf-rust-optimization heist. Algorithmic-only wins, zero
public-API breakage, identical detection results (290-pair golden gate).
Changes:
- hash.rs: `roll()` now takes a precomputed `window_power: u64` instead of
recomputing `base_pow(window_size - 1)` on every call. Removes an
O(window_size) loop from the inner sliding-window hot path. `base_pow`
is now `pub` so callers can hoist the power once per format group.
- detect.rs: hoist `window_power = base_pow(min_tokens - 1)` once before
the outer file loop in `detect_in_group()`. Pass it into every `roll()`
call.
- detect.rs: switch the per-call `by_format` map from `std::HashMap` to
`FxHashMap` for consistency with the rest of the detection path.
- store.rs: add `Store::reserve(n)` as a default-noop trait method
(performance hint only — correctness must not depend on it). Implement
on `MemoryStore` as `self.inner.reserve(n)`. Trait remains `dyn`-safe
(verified by new test that calls `reserve` through `&mut dyn Store`).
- detect.rs: call `store.reserve(total_windows)` once before the detection
loop, where `total_windows` is the sum of `hashes.len() - min_tokens`
across all prepared files. Eliminates `FxHashMap` rehashing mid-detect.
Tooling:
- Add `rust/scripts/regression_check.sh` — golden-baseline gate run after
each Wave change. Normalizes file paths to `../fixtures/...` form so the
gate is independent of absolute vs relative path emission.
- Add `rust/tests/golden.json` — frozen 290-pair clone set captured from
the pre-Heist baseline.
Verification:
- `cargo test -p cpd-core` — 22 pass
- `scripts/regression_check.sh` — 290 pairs match golden, exit 0
Andrey Kucherenko