Single-file component support (Vue/Svelte/Astro), bundler alias
resolution (vite.config, svelte.config, tsconfig paths) and monorepo
workspace package-name resolution landed on master after basta 0.1.1
was tagged (#1062); the published crate and npm packages don't have
them yet.
Dependabot opened five separate PRs bumping oxc_syntax, oxc_parser,
oxc_allocator, oxc_semantic and oxc_ast individually (#1066-#1070), but the
oxc crates share AST/Visitor trait definitions across the family and only
compile against matching versions of each other — bumping one alone breaks
the build (Visit trait signature mismatches, missing .span() methods) on
every CI job. oxc_ast_visit and oxc_span also needed the same bump, but
dependabot had not opened PRs for those two.
Bumps all seven together instead, and fixes the one real API change
between 0.147 and 0.150: ParserReturn::panicked was renamed to
::fatal_error (same meaning, confirmed against oxc_parser's source).
cpd-tokenizer's own oxc_parser/oxc_ast pins stay at 0.147, coexisting
fine as a separate resolution in the same Cargo.lock (no basta/
cpd-tokenizer API overlap), so this stays scoped to basta.
serde, anyhow, thiserror and log from cpd-finder; thiserror from
cpd-reporter; xxhash-rust and dev serde_json from cpd-tokenizer; anyhow,
log and env_logger from cpd. None of them is referenced in code.
`cargo publish` verifies by building the packaged crate against the
*published* versions of its dependencies, not the local paths. basta uses
`cpd_core::deadcode`, `cpd_reporter::deadcode` and a public `LineIndex` —
none of which exist in the versions on crates.io — so every attempt failed
with "failed to verify package tarball", and basta could not reach crates.io
until a jscpd release happened to carry those three crates along.
That is the coupling this workflow exists to avoid. It now walks basta's own
dependency graph, publishes in topological order whatever is missing, and
waits for the index between crates. jscpd is excluded by construction: it is
a sibling, not a dependency, and releasing it is not this workflow's job.
Bumps the three crates that actually changed: cpd-core 0.1.13 -> 0.1.14
(the `deadcode` module), cpd-tokenizer 0.1.15 -> 0.1.16 (`LineIndex` made
public, `.pyi`), cpd-reporter 0.1.14 -> 0.1.15 (the dead-code reporters).
cpd-finder is untouched and stays where it is.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
`basta@0.1.0` published with `files: ["run-basta.js", "platform-map.js"]`,
so npmjs.com/package/basta shows an empty page — the one place most people
will look before installing anything. npm takes the README from the
published tarball, so a version bump is the only way to put one there.
The README is npm-facing rather than a copy of the crate's: it leads with
`npm install -g basta` and `npx basta src`, and it names the known gaps —
no Vue/Svelte/Astro, class members matched by name, PEP 420 namespace
packages — so a reader can judge whether the tool fits before running it.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
Adds a second question about the same tree: not "what is duplicated" but
"what does nothing reach". A new `basta` crate builds the import graph from
the project's entry points and walks it, reporting unused files, unused
exports, unused module-private declarations, unused imports and — opt-in —
unused class members, across JavaScript, TypeScript, JSX, TSX and Python.
It ships two ways: `jscpd --dead-code` inside the existing binary, and a
standalone `basta` command. Both reuse the walker, the filters, all fifteen
reporters and the `--threshold` / `--exit-code` gates, so nothing new has to
be learned to run it.
Because it is a graph traversal rather than a reference count, dead code
cascades: a helper whose only caller is itself dead is reported too. Every
finding carries a confidence score and, below 100, the reasons it might be
wrong — `eval`, an unrecognised decorator, a wildcard re-export, a name seen
only in a string, a file in the scan that did not parse. `--min-confidence`
sets the floor at 60.
Entry points come from `package.json`, `pyproject.toml`, framework and
file-name conventions, and anything a shell script, CI workflow, Makefile or
Dockerfile names by path. CommonJS is read alongside ESM. A project that
renames its own import paths is read on its own terms: `paths` and `baseUrl`
from `tsconfig.json` and `jsconfig.json` are resolved, through a relative
`extends` chain and per package in a monorepo. A file named only by a path
string — a worker spawned through `new URL`, a build entry, a vitest setup
file — counts as used.
Languages plug in through one trait, `basta::lang::Analyzer`, which owns
parsing, specifier resolution, entry-point conventions, manifests, alias
configs and path traits. Nothing outside `lang/` is language-specific;
`docs/basta-extending.md` walks through adding one.
Validated against knip, fallow, vulture, skylos, fossil-mcp and repowise on
trending repositories. On a corpus of ten of them — 11k files, 3.3M lines —
every one of basta's 1,806 findings was checked against an import graph
rebuilt independently of it: no unused-file, unused-import or unused-symbol
claim survived as wrong.
Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
`resolve_config_paths` treated `ignorePattern` entries without `*` or `?`
as relative paths and joined them onto the config directory, a leftover
from when `ignore` and `ignorePattern` were conflated. A plain regex such
as `"Copyright 2026 Example Authors"` in `.jscpd.json` therefore matched
nothing, while the same string via `--ignore-pattern` worked. Config
entries are now passed through untouched.
Regexes that fail to compile were also dropped silently by the tokenizer;
the CLI now prints a `Warning: --ignore-pattern: invalid regex ...` line
for each one so a typo no longer disables a pattern without feedback.
Adds integration tests for both behaviors.
First release from the v5-only master. Bumps jscpd to 5.1.2 and every
sub-crate (cpd-core 0.1.11, cpd-tokenizer 0.1.13, cpd-finder 0.1.14,
cpd-reporter 0.1.12) so the new crates.io metadata from #988 and the
Windows drive-path fix in cpd-finder are republished. Adds the 5.1.2
changelog section (linux-arm64-musl package, cargo-binstall, Docker
image, jscpd --version naming, pre-commit hook args, repository split)
and refreshes the version examples in docs/.
Pre-release checks: cargo fmt, clippy -D warnings, cargo test --locked
(22 suites), cargo check --all-targets --target x86_64-pc-windows-msvc,
release-notes extraction dry-run against rust/CHANGELOG.md.
Republishes the jscpd npm wrapper with platform pins that match the
release. jscpd@5.1.0 shipped optionalDependencies pinned to the 5.0.16
binaries, so `npm i jscpd@5.1.0` resolved an engine one release behind
and `jscpd --version` printed `cpd 5.0.16` — every 5.1.0 fix, including
the Windows --baseline-from-ref fix, was absent for jscpd users. The cpd
package, the platform packages and the crates.io release were all correct;
only the wrapper's pins were stale.
Root cause in scripts/sync-version.mjs: the wrapper's version and its
platform pins were written behind one `version !== npmVersion` guard, so
once anything set version before the script ran, the guard read "already
up to date" and skipped the pins. Version and pins are now updated
independently, and the script ends by verifying that every npm version and
platform pin equals the release version, exiting non-zero otherwise.
release.yml runs this script, so the same mismatch now fails the release
rather than publishing. This is the 5.0.13 defect again — that fix covered
the cpd package but not the jscpd wrapper.
Sub-crates bumped for the code that changed since the published 5.1.0
(cpd-tokenizer 0.1.12, cpd-finder 0.1.13, cpd-reporter 0.1.11); cpd-core
is unchanged at 0.1.10. The only source delta is the clippy collapsible_if
rewrites unmasked by the MSRV correction, which are behaviour-preserving.
Pre-release verification: cargo fmt --check, clippy --workspace
--all-targets -D warnings, 847 tests, cargo +1.96 check at the declared
MSRV, and — per the Windows-only blind spot that blocked 5.0.15, 5.0.16
and 5.1.0 — cargo check --workspace --all-targets --target
x86_64-pc-windows-msvc, all clean. Release-notes extraction for 5.1.1
dry-run against rust/CHANGELOG.md returns the right section.
Revert the 5.1.1 bump (its release run was cancelled before publishing
anything; crates.io already has 5.1.0 and npm has nothing newer than
5.0.16, so the registry-based detect resumes the 5.1.0 release on this
push) and fold the fingerprint fix into the 5.1.0 changelog section.
Claude-Session: https://claude.ai/code/session_01VvWzWfEK6oV4HTe6EoGkff
oxc_span alone at 0.147 left the other oxc crates at 0.144, so the
GetSpan trait no longer matched the AST types and cpd-tokenizer failed
to compile. The oxc crates only work when versioned together.
Claude-Session: https://claude.ai/code/session_01VvWzWfEK6oV4HTe6EoGkff
New `codeclimate` reporter (alias: `gitlab`) writes
gl-code-quality-report.json in the CodeClimate issue format, restricted
to the subset GitLab defines as its Code Quality report format, ready to
be declared as an `artifacts:reports:codequality` artifact. Unlike the
SARIF reporter, which GitLab ingests as security vulnerability findings,
this surfaces duplicates as code quality issues in merge requests.
Each clone yields one issue per fragment (each describing the other
location, plus the CodeClimate `other_locations` field), with a
deterministic fingerprint derived from the clone's content hash — the
same snippet_pair_hash identity used by SARIF partialFingerprints and
the baseline file, now shared via `shared::clone_pair_hash`. Severity is
"minor", escalating to "major" for clones absent from a configured
baseline or when the run exceeds --threshold.
Closes#958
Claude-Session: https://claude.ai/code/session_01UkyKpExH2hcHawXqGp2geA
Coordinated bump of all six oxc crates (parser, allocator, span, ast,
ast_visit, syntax) — supersedes #920, which bumped only oxc_parser and
failed CI because oxc 0.144 requires rustc >= 1.95.
Adaptations for oxc 0.144 breaking changes:
- ParserReturn.errors renamed to .diagnostics
- TSModuleDeclaration split into TSExternalModuleDeclaration and
TSNamespaceDeclaration (oxc-project/oxc#25284); ts_strip visitor
updated to handle both with the same erasure semantics
Closes the gap reported in #891: the HTTP-only jscpd-server cannot be
used by MCP clients that spawn and manage a stdio process themselves
(Claude Desktop/Code, Cursor, APM, ...). The Rust binary now serves the
Model Context Protocol directly:
cpd --mcp /path/to/project
Transport & protocol:
- newline-delimited JSON-RPC 2.0 on stdin/stdout; stdout carries
protocol messages only, logging goes to stderr; exits 0 on stdin EOF
- protocol revision 2025-06-18, accepting 2025-03-26 and 2024-11-05
clients via version negotiation; ping, tools/list, tools/call;
notifications ignored; parse errors reported (-32700) without
killing the session
Tools (mirroring the HTTP jscpd-server):
- check_duplication(code, format) — checks a snippet against the
scanned project; results are compact JSON in a text content block
with file/snippet line ranges; unknown formats and sub-threshold
snippets produce informative results
- get_statistics() — files, clones, full Statistics of the last scan
- check_current_directory() — re-scan the configured paths
Performance: the project is scanned once at startup and cached as
detection-ready token hashes (PreparedSource pools per format /
cross-format group), so snippet checks only tokenize the snippet and
run one pool detection — milliseconds even on large codebases. All
standard detection options (--min-tokens, --format, --cross-formats,
--ignore, ...) apply to the scan and to snippet checks.
Supporting refactor in cpd-finder: orchestrate::run() split into
build_thread_pool() + prepare_scan_in() (walk + tokenize, returning
PreparedScan) so the MCP server can keep prepared sources around;
run() behavior is unchanged and detect_prepared is untouched.
strip_types_formats made pub for snippet tokenization parity;
PreparedSource now derives Clone. hyperfine A/B vs a master-built
baseline shows no regression on the normal CLI path (39.0ms vs
41.2ms, within noise).
Tests: 12 unit tests for the protocol handler and tools, plus 2
end-to-end tests that spawn the real binary and drive a full stdio
session (initialize → tools/list → tools/call → EOF), asserting
stdout is protocol-only. Full workspace suite green.
Docs: docs/ai-ready.md MCP section restructured into stdio (Rust v5)
and Streamable HTTP (Node v4) transports with client config examples;
docs/rust.md options table; root README; rust/CHANGELOG.md 5.0.16.
The jscpd-server (Node) stdio subcommand asked for literally in #891
can follow separately; this provides the stdio transport via the v5
binary that ships with jscpd@5.
Refs #891
Bump cpd/jscpd to 5.0.14 and changed sub-crates (cpd-core 0.1.7,
cpd-tokenizer 0.1.8, cpd-finder 0.1.9). Includes --cross-formats (#810)
and the prose-only markdown fix (#883).
sync-version.mjs now also syncs the cpd package optionalDependencies
(cpd@5.0.12 shipped pointing at the 5.0.11 binaries, which forced the
manual npm-only cpd@5.0.13) and its inter-crate dependency regex now
matches the actual Cargo.toml formatting (space before closing brace),
so dependency refs are bumped along with crate versions.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Detect duplicates across related formats via format equivalence groups
sharing one comparison pool, e.g. --cross-formats "javascript,typescript"
or the js-ts preset (javascript,jsx,typescript,tsx).
When a group mixes TypeScript with JavaScript, TS files are compared with
erasable type-only syntax stripped from the detection token stream (type
annotations, generics, interface/type declarations, as/satisfies, ?/!
markers, access modifiers, implements clauses, type-only imports/exports,
overload signatures, declare statements) so that
`function f(a: number): void` matches `function f(a)`. Stripping is
AST-driven (oxc_ast_visit over the already-parsed program) and only drops
tokens, so reported clone positions always reference the original source.
- cpd-tokenizer: new ts_strip module, tokenize_js_stripped,
TokenizeOptions.strip_types_formats (empty by default)
- cpd-finder: RunConfig.cross_formats, build_pools() pool grouping,
strip-set computation for TS-family formats grouped with JS
- cpd: --cross-formats flag, crossFormats config field (string,
array-of-strings, or array-of-arrays), js-ts preset, overlap merging,
unknown-format warnings, --debug output
- docs: rust.md option + Cross-Format Groups section, CHANGELOG entry
- tests: tokenizer strip tests, pool unit tests, cross-pool core test,
cpd-finder integration tests, CLI parse/config tests, e2e binary tests
with fixtures; default behavior unchanged (golden parity intact)
Non-erasable TS syntax with runtime semantics (enum, non-declare
namespace, parameter properties, import =, export =) is left intact and
will not cross-match. Cross-format clones are attributed to one member
format in per-format statistics.
Closes#810
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
apk