Closes#1002 (the --history half; --ratchet is deliberately not
implemented, the report shows the headroom instead).
`jscpd src --history v5.0.0..HEAD` (or `--history-since 2026-01-01`)
lists the commits with `git log`, checks each one out into a temporary
detached worktree (the --baseline-from-ref machinery, now shared through
baseline_ref::map_scan_paths), scans it with the run's own configuration
and keeps the totals; the current run is the series' last point.
`--history-every N` keeps every Nth commit counted from the newest,
`--history-limit N` (default 30) samples a long series evenly keeping
both ends. Nothing runs unless one of the flags is given.
Output: the console reporter appends a sparkline of the duplication
percentage, a table (commit, date, files, lines, clones, dup lines,
dup%, change, subject) with the change column red when duplication rose
and green when it fell, the overall trend line, and, when --threshold is
set and the latest value is under it, the headroom with the value the
threshold could be tightened to. The json reporter adds a `history`
object; the ai reporter prints one compact line per point. Other
reporters are unchanged.
New cpd_core::history (model, sparkline, change/headroom helpers with
unit tests), cpd::history (git log parsing, sampling, worktree scans),
cpd_reporter::history_render. Config keys history / historySince /
historyEvery / historyLimit; action input `history`. Integration tests
build a four-commit repo with fixed dates and check the JSON series,
the console block, every/limit thinning and the error paths. 969
workspace tests pass, clippy clean.
Docs: History section and option rows in docs/rust.md, README bullet,
action input row, fixtures/history-demo/README.md with a script that
builds the demo repository and the expected output.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Closes#1047.
Four situations used to write an empty report and exit 0, so a CI job
that checks only the exit code saw a green run:
- `--format` naming a format that is not supported (a warning since #964)
is now `Error: --format: 'x' is not a supported format`, exit 1. Names
introduced through --formats-exts or --formats-names stay accepted.
- A scan path that does not exist is `Error: path does not exist: <p>`,
exit 1, before anything runs.
- A reporter that fails to write its output still prints the existing
`Reporter '<name>' error: ...` line; the run now ends with
`ERROR: a reporter failed to write its output` and exit 1, after the
other reporters have run.
- A scan that analyzes no files (paths exist, but nothing matched the
--format/--ignore/--pattern filters or every file was below
--min-tokens) prints `Warning: jscpd analyzed no files` and still
exits 0, so an intentionally empty tree passes. The new
`--fail-on-empty` flag (config key `failOnEmpty`, action input
`fail-on-empty`) makes it exit 1. Reports are written before the check.
Docs: an "Exit codes" section in docs/rust.md, option rows in
docs/rust.md and docs/ci-and-hooks.md, a README feature bullet.
Fixtures: fixtures/fail-on-empty-demo with a `code/` clone pair, a
`nothing/` comment-only file, and a README quoting every command's
output and exit code. Tests: the #964 warning test becomes an error
test, plus integration tests for each new exit path and for the config
key; 531 tests pass, clippy clean.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Closes#1037.
Adds rust/scripts/build-pypi-wheels.py, which repacks the eight
jscpd-<platform>.tar.gz release tarballs into platform wheels: the same
binary under <pkg>.data/scripts/ as both `jscpd` and `cpd`, one wheel
per target, stdlib only, no maturin and no recompilation. glibc wheels
take their manylinux floor from the GLIBC_x.y symbols in the binary
(manylinux_2_34 today), musl builds are static so they tag as
musllinux_1_2, macOS and Windows use the Rust minimums. Semver
pre-releases map to PEP 440 (5.3.0-beta.1 -> 5.3.0b1); other labels
fail rather than guess.
release.yml gains a rust-publish-pypi job after the npm publish: it
takes the binary artifacts from this run, refills any missing target
from the published npm platform package (resumed releases, or a version
already on npm), builds the wheels, pip-installs the matching one and
checks `jscpd --version` and `cpd --version`, then uploads with PyPI
trusted publishing (environment `pypi`, no token, skip-existing). The
detect job treats a version missing from PyPI as pending, so the wheels
for the current release publish on the next push to master without a
new release commit. The GitHub release lists the PyPI package and waits
on the upload the same way it waits on crates.io and npm.
Docs: PyPI in the README install table and packages table, rust/README
and docs/rust.md install sections, and a `language: python` pre-commit
option in docs/ci-and-hooks.md so the hook no longer needs Node.js.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Off by default. `--similarity RATIO` (config `similarity`, Action input
`similarity`, MCP `check_duplication` argument) summarizes every function,
method and arrow function of a JavaScript/TypeScript/JSX/TSX file by the
bag of 4-grams over the pre-order sequence of its AST node types (oxc),
indexes the bags with MinHash + LSH banding, and reports pairs whose
weighted Jaccard index reaches RATIO as `similar` clones with a
`similarity` value. Names and literal values are not part of the summary:
a renamed copy scores 1.0, one inserted line ~0.9, two inserted statements
plus renames ~0.75.
Functions must clear --min-tokens/--min-lines, nested functions are never
paired with their parent, and pairs already covered by an exact, renamed
or merged clone are skipped. Values outside (0, 1] warn and disable the
option. Reporting reuses the Type-3 vocabulary (console `similar ~0.75`,
JSON kind/similarity, SARIF jscpd/near-miss-code).
New: cpd-core::similarity (FunctionSig, SimilarityIndex, bag_jaccard,
minhash), cpd-tokenizer::functions (oxc visitor extractor), MCP function
index built lazily per scan. fixtures/type3-demo/similar-functions with
README numbers, docs, unit + integration + MCP stdio tests. Also adds the
missing max_gap_lines/similarity fields to the --debug config output.
Off by default (0). When set to N, clones of the same file pair whose
fragments follow each other in both files with at most N unmatched lines
between them are merged into one clone of kind `similar`, with `tokens` =
matched tokens and `similarity` = matched tokens over the tokens of the
longer merged span. Adjacent exact windows that share a boundary token are
counted once; chains merge transitively; overlapping or reordered fragments
never merge. The pass runs after every other filter and returns its input
untouched at 0, so default output is unchanged.
Reporting: console `Clone found (fmt, similar ~0.91)`, ai `[~0.91]`, JSON
"kind"/"similarity" (rounded to three decimals), SARIF rule
jscpd/near-miss-code with a similarity property (declared only when
referenced), Code Climate check_name.
Config key maxGapLines (kebab alias), GitHub Action input max-gap-lines,
docs section, fixtures/type3-demo with a README of runnable commands, unit
tests for the merge pass and reporters, and a binary-level integration test.
Closes#998.
Three opt-in flags (config keys ignoreIdentifiers, ignoreLiterals,
ignoreAnnotations) normalize token classes before hashing so that blocks
which differ only in names, values or annotations are detected:
- identifiers hash as `$id`; keywords are kept (the JS/TS tokenizer
classifies them, other languages use a built-in common-keyword list)
- string literals hash as `$str`, numeric literals as `$num`
- `@Name`, `@a.b.Name` and `@Name(...)` runs are dropped in Java, Kotlin,
Scala, Groovy, Python, Dart, Swift, JavaScript and TypeScript
Each clone now carries `kind: exact | renamed`. DetectionToken gains a
raw_hash (equal to hash unless normalization rewrote the token); the
PreparedSource materializes raw hashes only when a token was normalized,
and the detector compares raw hashes over the matched run to classify a
pair. A dropped annotation salts the raw hash of the token before it, so a
pair differing only in annotations inside the matched run is `renamed`.
Reporters: console prints `Clone found (fmt, renamed)`, ai appends
`(renamed)`, json adds "kind", sarif uses rule jscpd/similar-code (declared
only when referenced), codeclimate the same check_name. Default runs are
unchanged apart from the additive "kind": "exact" JSON field.
Also: GitHub Action inputs, docs section, fixtures/type2-demo with a README
of runnable commands, unit tests in tokenizer/core/reporter, CLI flag
tests, and a binary-level integration test.
First release from the v5-only master. Bumps jscpd to 5.1.2 and every
sub-crate (cpd-core 0.1.11, cpd-tokenizer 0.1.13, cpd-finder 0.1.14,
cpd-reporter 0.1.12) so the new crates.io metadata from #988 and the
Windows drive-path fix in cpd-finder are republished. Adds the 5.1.2
changelog section (linux-arm64-musl package, cargo-binstall, Docker
image, jscpd --version naming, pre-commit hook args, repository split)
and refreshes the version examples in docs/.
Pre-release checks: cargo fmt, clippy -D warnings, cargo test --locked
(22 suites), cargo check --all-targets --target x86_64-pc-windows-msvc,
release-notes extraction dry-run against rust/CHANGELOG.md.
Everything belonging to the v4 TypeScript engine moves to the master-v4
branch: apps/, packages/, build-utils/, changesets, pnpm workspace, turbo,
the Node.js CI workflow and the ts-* release jobs. What remains is the
Rust workspace under rust/, the npm wrapper and platform packages, the
GitHub Action, Docker image, flake, skills, benchmark and fixtures/.
- rust.yml gains a smoke job that runs the release binary against
fixtures/ (json, sarif, html) and asserts clones and reporters
- release.yml keeps only the Rust jobs; pnpm removed from every workflow;
audit.yml keeps cargo audit + cargo deny; dependabot drops the root npm entry
- root package.json becomes a thin private package depending on the exact
jscpd@5 version so the pre-commit hook keeps installing the binary;
sync-version.mjs keeps it in step
- .pre-commit-hooks.yaml used v4-only flags that the v5 CLI rejects; fixed
- FORMATS.md is now generated from the Rust tokenizer (224 formats)
- README, docs, CONTRIBUTING, SECURITY, package and crate descriptions,
benchmark and skills rewritten for v5 on its own terms: no v5-vs-v4 speed
multipliers anywhere, a single "Looking for v4?" pointer to master-v4
Rust / crates.io
- report the invoked binary name in --version/--help (jscpd vs cpd)
- add repository, documentation, keywords, categories to all crates
- exclude tests/ from the jscpd crate package; add cargo-binstall metadata
- expand the crates.io README and render it on docs.rs
npm
- add funding field to jscpd, cpd and jscpd@4 packages
- add jscpd-linux-arm64-musl platform package and clearer unsupported-platform error
- enable provenance for the v4 changeset publish
Pipelines
- rust.yml: windows-latest in the matrix, drop dead schedule guard
- release.yml: cosign-sign release assets, floating v5 tag, LICENSE in tarballs,
invoke the new docker.yml (GHCR multi-arch image) after each release
- audit.yml: nightly cargo audit, cargo deny and pnpm audit
- Dockerfile: multi-stage image from release binaries (distroless), mcp target for Glama
Docs / community
- README: shields badges, single primary install per OS, GitHub Action snippet,
link to jscpd.dev, v5.1 features, 15 reporters, 224 formats, 8 platforms
- docs: @v5 action tag, Docker usage, baseline section, current crate versions
- YAML issue forms with engine selector, config.yml, CODEOWNERS
- remove stale .github/CONTRIBUTING.md stub, custom.md, TODO.md; untrack rust/jscpd-report.json
- examples: mark as v4 API, add a Rust cpd-finder example
apk