Commit Graph

61 Commits

Author SHA1 Message Date
apk 2c8a2f418f fix: derive crate versions from Cargo.toml at publish time instead of hardcoding 2026-06-05 12:24:16 +02:00
apk 924a632c38 fix: update sync-version.mjs sub-crate versions to 0.1.1 2026-06-05 12:23:24 +02:00
apk 1f0e9662dd fix: hardcode verify version in publish script trailing output 2026-06-05 12:14:07 +02:00
apk 43cd7a45ed fix: skip already-published crates in publish script
Check if crate version exists on crates.io before attempting to publish.
2026-06-05 12:13:45 +02:00
apk 47fb351ae2 fix: bash compatibility for negative array subscript in publish script
Use ${#PUBLISH_ORDER[@]}-1 instead of -1 which is not portable.
2026-06-05 12:13:17 +02:00
apk 9024105e44 fix: add description and license to all crates, use trap for cleanup in publish script
crates.io requires description and license fields. Added description to
all 5 crates and license.workspace = true. publish-crates.sh now uses a
trap EXIT to restore the working tree even on failure.
2026-06-05 12:12:51 +02:00
apk 467170f22f fix: add cargo generate-lockfile after version sync, remove --locked from cargo publish
--locked conflicts with version changes in Cargo.toml. --allow-dirty
handles the uncommitted changes. Regenerating Cargo.lock after sync
ensures the lock file matches the updated versions.
2026-06-05 12:09:03 +02:00
apk 4f301063b8 fix: sync-version and publish-crates scripts handle two-tier versioning
- sync-version.mjs: only bump jscpd to npm version, keep sub-crates at 0.1.0
  and sync inter-crate dependency version refs
- publish-crates.sh: use per-crate versions for index-wait polling instead
  of npm version for all crates, handle GNU/BSD sed differences
2026-06-05 12:05:50 +02:00
apk 2dabdc766a feat: add publish pipelines for Rust jscpd (npm + crates.io)
- Add npm package scaffold for cpd@5.0.0 with prebuilt binary distribution
  - rust/package.json: main cpd package with optionalDependencies for 6 platforms
  - rust/run-cpd.js: CLI shim with platform detection and binary resolution
  - rust/platform-map.js: platform detection (6 targets including musl)
  - rust/npm/prebuilt-targets.json: platform matrix configuration
  - rust/npm/cpd-*/package.json: platform package templates
  - rust/scripts/npm-prebuilt-package.mjs: assembles platform packages from binaries

- Add GitHub Actions workflows
  - .github/workflows/crates-publish.yml: publishes all 5 crates to crates.io
    in dependency order with OIDC Trusted Publishing and index-wait polling
  - .github/workflows/npm-rust-publish.yml: cross-compiles for 6 platforms,
    publishes platform packages with idempotency checks, then main cpd package
  - .github/workflows/release-rust.yml: orchestrator triggered by push to
    release/rust branch, creates GitHub Release with cpd-v<version> tag

- Configure Rust crates for publishing
  - Change binary name from jscpd to cpd (crate name stays jscpd)
  - Convert inter-crate deps to dual version+path format for crates.io
  - Update integration tests for cpd binary name

- Add build and release scripts
  - rust/scripts/sync-version.mjs: syncs version from package.json to Cargo.toml
  - rust/scripts/publish-crates.sh: manual crates.io publish with dependency ordering

- Document pnpm workspace exclusion for rust/ directory
2026-06-05 11:37:08 +02:00
Andrey Kucherenko 7b3fe41950 perf(rust): implement design spec — DetectionToken, open_clone SM, FxHashSet dedup, globset walker
- Add DetectionToken struct (no heap alloc per token in detection hot path)
- Add hash_token(kind, value, ignore_case) parity with jscpd-rs hash interface
- Add TokenizeOptions + push_token: filter+hash at tokenize time, eliminating
  the separate apply_mode pass
- Add tokenize_to_detection: detection path bypasses SourceFile.tokens entirely
- Rewrite detect.rs: Store trait + MemoryStore replaced by FxHashMap<u64,Occurrence>
  type alias; open_clone state machine replaces O(N^2) suppress_subclones;
  FxHashSet<CloneDedupKey> dedup replaces sort+dedup; secondary pass cap 16->2
- Add detect_prepared(): orchestrate.rs calls this with pre-hashed PreparedSource,
  skipping redundant re-hashing of SourceFile.tokens
- skip_local logic moved into flush_clone; skip_local.rs deleted
- Walker: globset replaces substring ignore-pattern check (correctness+speed);
  single fs::read eliminates double-read on line-filter paths; mpsc::channel
  replaces Arc<Mutex<Vec>> for parallel collection
- Delete store.rs, filter.rs, skip_local.rs

220 tests pass, 3 ignored; regression gate: 283 pairs match golden baseline
2026-06-04 18:06:57 +02:00
Andrey Kucherenko 425f57b142 perf(rust/core): wave 1 — eliminate hash O(N×k) loop and FxHashMap rehashing
Wave 1 of the perf-rust-optimization heist. Algorithmic-only wins, zero
public-API breakage, identical detection results (290-pair golden gate).

Changes:
- hash.rs: `roll()` now takes a precomputed `window_power: u64` instead of
  recomputing `base_pow(window_size - 1)` on every call. Removes an
  O(window_size) loop from the inner sliding-window hot path. `base_pow`
  is now `pub` so callers can hoist the power once per format group.
- detect.rs: hoist `window_power = base_pow(min_tokens - 1)` once before
  the outer file loop in `detect_in_group()`. Pass it into every `roll()`
  call.
- detect.rs: switch the per-call `by_format` map from `std::HashMap` to
  `FxHashMap` for consistency with the rest of the detection path.
- store.rs: add `Store::reserve(n)` as a default-noop trait method
  (performance hint only — correctness must not depend on it). Implement
  on `MemoryStore` as `self.inner.reserve(n)`. Trait remains `dyn`-safe
  (verified by new test that calls `reserve` through `&mut dyn Store`).
- detect.rs: call `store.reserve(total_windows)` once before the detection
  loop, where `total_windows` is the sum of `hashes.len() - min_tokens`
  across all prepared files. Eliminates `FxHashMap` rehashing mid-detect.

Tooling:
- Add `rust/scripts/regression_check.sh` — golden-baseline gate run after
  each Wave change. Normalizes file paths to `../fixtures/...` form so the
  gate is independent of absolute vs relative path emission.
- Add `rust/tests/golden.json` — frozen 290-pair clone set captured from
  the pre-Heist baseline.

Verification:
- `cargo test -p cpd-core` — 22 pass
- `scripts/regression_check.sh` — 290 pairs match golden, exit 0
2026-06-04 15:42:23 +02:00