Single-file component support (Vue/Svelte/Astro), bundler alias
resolution (vite.config, svelte.config, tsconfig paths) and monorepo
workspace package-name resolution landed on master after basta 0.1.1
was tagged (#1062); the published crate and npm packages don't have
them yet.
`jscpd rust/` reported 48 clones. This folds every one that was repetition
rather than a deliberate copy, leaving 34 across `rust/` and 14 in
`rust/crates`, all of the kinds the #1053 pass kept on purpose: fixture
corpora, `use` blocks, the TypeScript file and its JavaScript twin,
`impl Reporter` headers and the `ts_strip` visitor pair.
- basta's JSONC reader copied a string-skipping loop into both `strip_jsonc`
and `drop_trailing_commas`. It is now `strip_comments` followed by a comma
pass over `lang::outside_strings`, the quote-aware scanning the bundler
config readers already share.
- The JavaScript and Python analyzer tests each built their own module
index, import lookup, binding-reference lookup and member-read list. They
live once in `test_scan` as `index_of`, `import`, `binding_reference` and
`member_reads`.
- `test_scan::TempTree` writes a test's files and removes them when dropped,
so a failing assertion no longer leaves its directory behind; the script
and `extends` tests that repeated the setup use it.
- Four tsconfig `paths` tests with one shape are one table-driven test.
- The CLI integration tests share `duplicate_shapes`, `history_stdout` and
`assert_only_the_real_file_was_scanned`.
- `scripts/publish-npm.sh` repeated a 43-line "publish the platform package,
then the main packages under --force-main" tail in two branches, and the
"publish unless already live, or show the dry run" block five times. They
are `publish_platform_target` and `publish_package`. Run against the old
script with the build, the registry check and `npm` stubbed, all eleven
combinations of --target, --all, --force-main, --dry-run, --provenance, a
failed build and missing platforms print the same lines and exit the same.
Behaviour is unchanged: the fixtures corpus reports in every mode, the
summary, the console output and the dead-code demo are byte-identical.
basta reads JavaScript, TypeScript and Python, so `jscpd --dead-code` has no
view of the Rust crates, and the compiler does not warn about an unused `pub`
item. Every crate in the workspace is `publish = false`, though, so a `pub`
item nothing in the workspace uses is dead as surely as a private one. A scan
of every `pub` item's uses across the crates, their tests and the fuzz
targets, checked by hand, found:
- `CloneKind::is_exact`: used nowhere.
- `ConfigResult::has_diagnostics`, `basta::lang::language_of`,
`TokenizeOptions::normalizes` and `TokenizeOptions::with_code_ignore_patterns`:
called only by tests written for them. The language test now asks
`analyzer_for` directly.
- `stats_with_pct` is a reporter test fixture that was compiled into the
library; it moves inside the `#[cfg(test)]` fixtures module that re-exported it.
- Fields kept alive by `#[allow(dead_code)]`: `MarkdownFence::front_matter`
and `SfcBlock::tag` are written and never read, and `Options::list` is never
read because `main` reads `cli.list`.
- `scripts/regression_check.sh` and its 1528-line `tests/golden.json`: a gate
for the Wave 2/3 migration that nothing runs, against a report schema that
has since changed.
Behaviour is unchanged: the JSON reports for the fixtures corpus in mild, weak
and strict modes, with Type-2 normalization, and with the complexity summary,
the console output and the dead-code demo output are byte-identical before and
after.
`cargo publish` verifies by building the packaged crate against the
*published* versions of its dependencies, not the local paths. basta uses
`cpd_core::deadcode`, `cpd_reporter::deadcode` and a public `LineIndex` —
none of which exist in the versions on crates.io — so every attempt failed
with "failed to verify package tarball", and basta could not reach crates.io
until a jscpd release happened to carry those three crates along.
That is the coupling this workflow exists to avoid. It now walks basta's own
dependency graph, publishes in topological order whatever is missing, and
waits for the index between crates. jscpd is excluded by construction: it is
a sibling, not a dependency, and releasing it is not this workflow's job.
Bumps the three crates that actually changed: cpd-core 0.1.13 -> 0.1.14
(the `deadcode` module), cpd-tokenizer 0.1.15 -> 0.1.16 (`LineIndex` made
public, `.pyi`), cpd-reporter 0.1.14 -> 0.1.15 (the dead-code reporters).
cpd-finder is untouched and stays where it is.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
`basta@0.1.0` published with `files: ["run-basta.js", "platform-map.js"]`,
so npmjs.com/package/basta shows an empty page — the one place most people
will look before installing anything. npm takes the README from the
published tarball, so a version bump is the only way to put one there.
The README is npm-facing rather than a copy of the crate's: it leads with
`npm install -g basta` and `npx basta src`, and it names the known gaps —
no Vue/Svelte/Astro, class members matched by name, PEP 420 namespace
packages — so a reader can judge whether the tool fits before running it.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
basta is a separate product from jscpd — a different question about the
same tree — so it gets its own version, its own prebuilt packages and its
own trigger. jscpd goes out on every push to master that leaves a version
unpublished; basta goes out only when a `basta-v*` tag is pushed, or by
hand through workflow_dispatch. Neither release can drag the other along
or hold it up.
`basta-release.yml` owns the whole thing: it checks that the crate, the npm
manifest and the tag all name the same version and that the working tree is
already synced, publishes the crate, builds the eight targets and publishes
`basta-<platform>` plus the `basta` wrapper. It ends by asserting that npm's
`latest` really points at this release — the package carries 1.x versions
from an unrelated tool of the same name, and a dist-tag that quietly stayed
on those would hand every `npm i basta` the wrong program.
The one place the two cannot be separated is crates.io: `jscpd` depends on
`basta`, so cargo refuses to publish jscpd until a matching basta exists.
The jscpd crates job therefore still publishes basta when it is missing, and
that is now commented as the dependency requirement it is. Both paths skip a
version already on the registry, so whichever runs first wins.
npm needed the wrapper pattern `jscpd` and `cpd` already use: `rust/basta/`
holds the manifest, a platform map naming `basta-<platform>`, and a launcher
that falls back to a local `cargo build` when no platform package is
installed. `npm-prebuilt-package.mjs` grew a `--product` flag; its output for
jscpd is byte-identical to before. `prebuilt-targets.json` now carries a
platform label rather than a jscpd-specific blurb, so one table serves both
products. `sync-version.mjs` syncs and verifies the basta wrapper against the
basta crate version — the same independent version/pin update the jscpd block
does, for the same reason it does it.
Nothing is published by this commit.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Adds a second question about the same tree: not "what is duplicated" but
"what does nothing reach". A new `basta` crate builds the import graph from
the project's entry points and walks it, reporting unused files, unused
exports, unused module-private declarations, unused imports and — opt-in —
unused class members, across JavaScript, TypeScript, JSX, TSX and Python.
It ships two ways: `jscpd --dead-code` inside the existing binary, and a
standalone `basta` command. Both reuse the walker, the filters, all fifteen
reporters and the `--threshold` / `--exit-code` gates, so nothing new has to
be learned to run it.
Because it is a graph traversal rather than a reference count, dead code
cascades: a helper whose only caller is itself dead is reported too. Every
finding carries a confidence score and, below 100, the reasons it might be
wrong — `eval`, an unrecognised decorator, a wildcard re-export, a name seen
only in a string, a file in the scan that did not parse. `--min-confidence`
sets the floor at 60.
Entry points come from `package.json`, `pyproject.toml`, framework and
file-name conventions, and anything a shell script, CI workflow, Makefile or
Dockerfile names by path. CommonJS is read alongside ESM. A project that
renames its own import paths is read on its own terms: `paths` and `baseUrl`
from `tsconfig.json` and `jsconfig.json` are resolved, through a relative
`extends` chain and per package in a monorepo. A file named only by a path
string — a worker spawned through `new URL`, a build entry, a vitest setup
file — counts as used.
Languages plug in through one trait, `basta::lang::Analyzer`, which owns
parsing, specifier resolution, entry-point conventions, manifests, alias
configs and path traits. Nothing outside `lang/` is language-specific;
`docs/basta-extending.md` walks through adding one.
Validated against knip, fallow, vulture, skylos, fossil-mcp and repowise on
trending repositories. On a corpus of ten of them — 11k files, 3.3M lines —
every one of basta's 1,806 findings was checked against an import graph
rebuilt independently of it: no unused-file, unused-import or unused-symbol
claim survived as wrong.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
The PyPI long description pointed the repo-hosted pre-commit hook at
"v5.3.0 or later"; the python hook ships with the next release, which
is 5.2.1. The PEP 440 examples use the same version.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
CITATION.cff (CFF 1.2.0, validated with cffconvert) gives GitHub's
"Cite this repository" button BibTeX and APA output for the project.
The README gets a short Citation section with a ready-made @software
entry. sync-version.mjs keeps the file's version in step with
rust/package.json and stamps date-released whenever the version
changes, and verifies the version like the other manifests.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
The PyPI long description said 150+ formats and little else. It now
matches the README: 224 formats, 15 reporters, exact / Type-2 / Type-3
clones, baseline and threshold gating, the pip/pipx/uv install paths,
the Node-free pre-commit hook and the repo-hosted hook, plus links.
Metadata of a published release cannot change, so this lands with the
next upload.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
The `jscpd` hook in .pre-commit-hooks.yaml switches from `language: node`
to `language: python`: pre-commit now `pip install`s this repository,
whose new root pyproject.toml is an empty project that depends on the
`jscpd` wheel of the same version, so the hook needs no Node.js on the
machine. pre-commit is itself a Python tool, so Python is always there.
The root package.json existed only for the node hook and is removed;
sync-version.mjs now keeps pyproject.toml's version and `jscpd==` pin
in step with rust/package.json instead (mapped to PEP 440 the same way
build-pypi-wheels.py does) and verifies both before a release.
Verified with `pre-commit try-repo . jscpd` against locally built
5.2.0 wheels. Tags up to v5.2.0 keep the node hook, since a hook is
read from the pinned rev.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Closes#1037.
Adds rust/scripts/build-pypi-wheels.py, which repacks the eight
jscpd-<platform>.tar.gz release tarballs into platform wheels: the same
binary under <pkg>.data/scripts/ as both `jscpd` and `cpd`, one wheel
per target, stdlib only, no maturin and no recompilation. glibc wheels
take their manylinux floor from the GLIBC_x.y symbols in the binary
(manylinux_2_34 today), musl builds are static so they tag as
musllinux_1_2, macOS and Windows use the Rust minimums. Semver
pre-releases map to PEP 440 (5.3.0-beta.1 -> 5.3.0b1); other labels
fail rather than guess.
release.yml gains a rust-publish-pypi job after the npm publish: it
takes the binary artifacts from this run, refills any missing target
from the published npm platform package (resumed releases, or a version
already on npm), builds the wheels, pip-installs the matching one and
checks `jscpd --version` and `cpd --version`, then uploads with PyPI
trusted publishing (environment `pypi`, no token, skip-existing). The
detect job treats a version missing from PyPI as pending, so the wheels
for the current release publish on the next push to master without a
new release commit. The GitHub release lists the PyPI package and waits
on the upload the same way it waits on crates.io and npm.
Docs: PyPI in the README install table and packages table, rust/README
and docs/rust.md install sections, and a `language: python` pre-commit
option in docs/ci-and-hooks.md so the hook no longer needs Node.js.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
First release from the v5-only master. Bumps jscpd to 5.1.2 and every
sub-crate (cpd-core 0.1.11, cpd-tokenizer 0.1.13, cpd-finder 0.1.14,
cpd-reporter 0.1.12) so the new crates.io metadata from #988 and the
Windows drive-path fix in cpd-finder are republished. Adds the 5.1.2
changelog section (linux-arm64-musl package, cargo-binstall, Docker
image, jscpd --version naming, pre-commit hook args, repository split)
and refreshes the version examples in docs/.
Pre-release checks: cargo fmt, clippy -D warnings, cargo test --locked
(22 suites), cargo check --all-targets --target x86_64-pc-windows-msvc,
release-notes extraction dry-run against rust/CHANGELOG.md.
CodeQL flagged the cell escaper for handling '|' but not '\'. Escape both, and
unescape when descriptions are carried over from the previous file so the
generator stays idempotent.
Everything belonging to the v4 TypeScript engine moves to the master-v4
branch: apps/, packages/, build-utils/, changesets, pnpm workspace, turbo,
the Node.js CI workflow and the ts-* release jobs. What remains is the
Rust workspace under rust/, the npm wrapper and platform packages, the
GitHub Action, Docker image, flake, skills, benchmark and fixtures/.
- rust.yml gains a smoke job that runs the release binary against
fixtures/ (json, sarif, html) and asserts clones and reporters
- release.yml keeps only the Rust jobs; pnpm removed from every workflow;
audit.yml keeps cargo audit + cargo deny; dependabot drops the root npm entry
- root package.json becomes a thin private package depending on the exact
jscpd@5 version so the pre-commit hook keeps installing the binary;
sync-version.mjs keeps it in step
- .pre-commit-hooks.yaml used v4-only flags that the v5 CLI rejects; fixed
- FORMATS.md is now generated from the Rust tokenizer (224 formats)
- README, docs, CONTRIBUTING, SECURITY, package and crate descriptions,
benchmark and skills rewritten for v5 on its own terms: no v5-vs-v4 speed
multipliers anywhere, a single "Looking for v4?" pointer to master-v4
Rust / crates.io
- report the invoked binary name in --version/--help (jscpd vs cpd)
- add repository, documentation, keywords, categories to all crates
- exclude tests/ from the jscpd crate package; add cargo-binstall metadata
- expand the crates.io README and render it on docs.rs
npm
- add funding field to jscpd, cpd and jscpd@4 packages
- add jscpd-linux-arm64-musl platform package and clearer unsupported-platform error
- enable provenance for the v4 changeset publish
Pipelines
- rust.yml: windows-latest in the matrix, drop dead schedule guard
- release.yml: cosign-sign release assets, floating v5 tag, LICENSE in tarballs,
invoke the new docker.yml (GHCR multi-arch image) after each release
- audit.yml: nightly cargo audit, cargo deny and pnpm audit
- Dockerfile: multi-stage image from release binaries (distroless), mcp target for Glama
Docs / community
- README: shields badges, single primary install per OS, GitHub Action snippet,
link to jscpd.dev, v5.1 features, 15 reporters, 224 formats, 8 platforms
- docs: @v5 action tag, Docker usage, baseline section, current crate versions
- YAML issue forms with engine selector, config.yml, CODEOWNERS
- remove stale .github/CONTRIBUTING.md stub, custom.md, TODO.md; untrack rust/jscpd-report.json
- examples: mark as v4 API, add a Rust cpd-finder example
Republishes the jscpd npm wrapper with platform pins that match the
release. jscpd@5.1.0 shipped optionalDependencies pinned to the 5.0.16
binaries, so `npm i jscpd@5.1.0` resolved an engine one release behind
and `jscpd --version` printed `cpd 5.0.16` — every 5.1.0 fix, including
the Windows --baseline-from-ref fix, was absent for jscpd users. The cpd
package, the platform packages and the crates.io release were all correct;
only the wrapper's pins were stale.
Root cause in scripts/sync-version.mjs: the wrapper's version and its
platform pins were written behind one `version !== npmVersion` guard, so
once anything set version before the script ran, the guard read "already
up to date" and skipped the pins. Version and pins are now updated
independently, and the script ends by verifying that every npm version and
platform pin equals the release version, exiting non-zero otherwise.
release.yml runs this script, so the same mismatch now fails the release
rather than publishing. This is the 5.0.13 defect again — that fix covered
the cpd package but not the jscpd wrapper.
Sub-crates bumped for the code that changed since the published 5.1.0
(cpd-tokenizer 0.1.12, cpd-finder 0.1.13, cpd-reporter 0.1.11); cpd-core
is unchanged at 0.1.10. The only source delta is the clippy collapsible_if
rewrites unmasked by the MSRV correction, which are behaviour-preserving.
Pre-release verification: cargo fmt --check, clippy --workspace
--all-targets -D warnings, 847 tests, cargo +1.96 check at the declared
MSRV, and — per the Windows-only blind spot that blocked 5.0.15, 5.0.16
and 5.1.0 — cargo check --workspace --all-targets --target
x86_64-pc-windows-msvc, all clean. Release-notes extraction for 5.1.1
dry-run against rust/CHANGELOG.md returns the right section.
Revert the 5.1.1 bump (its release run was cancelled before publishing
anything; crates.io already has 5.1.0 and npm has nothing newer than
5.0.16, so the registry-based detect resumes the 5.1.0 release on this
push) and fold the fingerprint fix into the 5.1.0 changelog section.
Claude-Session: https://claude.ai/code/session_01VvWzWfEK6oV4HTe6EoGkff
Bump cpd/jscpd to 5.0.14 and changed sub-crates (cpd-core 0.1.7,
cpd-tokenizer 0.1.8, cpd-finder 0.1.9). Includes --cross-formats (#810)
and the prose-only markdown fix (#883).
sync-version.mjs now also syncs the cpd package optionalDependencies
(cpd@5.0.12 shipped pointing at the 5.0.11 binaries, which forced the
manual npm-only cpd@5.0.13) and its inter-crate dependency regex now
matches the actual Cargo.toml formatting (space before closing brace),
so dependency refs are bumped along with crate versions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Renames the compiled binary from cpd(.exe) to jscpd(.exe) and the
platform-specific optional-dependency npm packages from cpd-* to
jscpd-* to avoid collision with an antivirus-flagged executable name.
Fixes#826
- Add rust/jscpd/ npm package: same Rust binary as cpd, published as
'jscpd' on npm with version 5.x (matching cpd). Exposes 'jscpd'
CLI command, depends on cpd-* platform packages.
- Fix --exit-code: changed from boolean to Option<i32> with
default_missing_value=1, matching jscpd v4's --exitCode behavior.
--exit-code (no value) exits 1 on clones; --exit-code 2 exits 2.
--threshold and --exit-code are now independent (matching v4).
- Update CI: add publish-jscpd job after cpd with fixture verification
gate (installs cpd, runs on ./fixtures, then publishes jscpd).
- Update publish-npm.sh: publish jscpd wrapper after cpd with
fixture check in --all, --target, and default modes.
- Update sync-version.mjs: sync jscpd/package.json version alongside
cpd's.
- Update README docs across cpd, jscpd, and npm packages to document
v5.x as Rust-based and v4.x as TypeScript, and fix --exit-code docs.
apk