kucherenko__jscpd/docs/rust.md

jscpd v5 (Rust Engine)

jscpd v5 is a Rust engine shipped as a self-contained binary: no runtime required, same CLI flags, reporters and .jscpd.json config as the earlier Node.js versions.

It is distributed under three names:

Package	Installs commands	Notes
jscpd	jscpd	Default install; installs the jscpd command
cpd	cpd	Lighter package, shorter command only
jscpd (crates.io)	jscpd and cpd	Rust-native install; both binaries

All three install the identical Rust binary and accept the same CLI options. Only the crates.io install exposes both command names from a single package.

Performance

A comparison against other copy/paste detectors (jscpd-rs, Duplo, Fallow, Simian, PMD CPD) on the repository's fixtures/ corpus — timing, detection counts, cross-format detection and AI-token efficiency — is in benchmark/BENCHMARK.md.

Installation

# npm — installs the jscpd command
npm install -g jscpd
jscpd /path/to/code

# npm — installs only the cpd command (lighter)
npm install -g cpd
cpd /path/to/code

# crates.io — Rust-native install (exposes both jscpd and cpd commands)
cargo install jscpd
jscpd /path/to/code
cpd /path/to/code

# Nix — run without installing
nix run github:kucherenko/jscpd -- /path/to/code

# Nix — install permanently
nix profile install github:kucherenko/jscpd

# Homebrew (macOS/Linux)
brew install jscpd

# PyPI — platform wheels with the same binary (pip, pipx, uv)
pip install jscpd
uvx jscpd /path/to/code

The npm packages ship prebuilt binaries for 8 platforms — no Node.js runtime is required, the binary is self-contained:

Platform	npm package	Rust target
macOS arm64	jscpd-darwin-arm64	aarch64-apple-darwin
macOS x64	jscpd-darwin-x64	x86_64-apple-darwin
Linux arm64 (glibc)	jscpd-linux-arm64-gnu	aarch64-unknown-linux-gnu
Linux arm64 (musl)	jscpd-linux-arm64-musl	aarch64-unknown-linux-musl
Linux x64 (glibc)	jscpd-linux-x64-gnu	x86_64-unknown-linux-gnu
Linux x64 (musl)	jscpd-linux-x64-musl	x86_64-unknown-linux-musl
Windows arm64	jscpd-windows-arm64-msvc	aarch64-pc-windows-msvc
Windows x64	jscpd-windows-x64-msvc	x86_64-pc-windows-msvc

The same binaries are attached to every GitHub Release as jscpd-<platform>.tar.gz with a checksums.txt and SLSA provenance, and packaged as a multi-arch Docker image at ghcr.io/kucherenko/jscpd (see CI docs).

CLI Usage

The jscpd command is available after installing jscpd from npm; the cpd command is available after installing either cpd (npm) or jscpd (crates.io). Both commands accept the same options and are identical:

jscpd [OPTIONS] [PATH]...
cpd [OPTIONS] [PATH]...

Options

Option	Short	Description	Default
--min-tokens	-k	Minimum tokens in a clone	50
--min-lines	-l	Minimum lines in a clone	5
--max-lines	-x	Maximum source file lines	—
--max-size	-z	Skip files larger than SIZE (e.g. 1kb, 1mb, 100kb)	no limit
--mode	-m	Detection mode: mild, weak, strict	mild
--ignore-pattern		Comma-separated regular expressions; source text matched by a pattern is excluded from clone detection. See Ignoring source regions	—
--workers		Number of worker threads for parallel tokenization/detection	auto (all CPU cores)
--no-colors		Disable ANSI color output	off
--absolute	-a	Use absolute paths in reports	off
--follow-symlinks		Follow symbolic links while walking. A file reached through a link is reported by the path it was found at (relative to the scan root, like any other file), and a file reachable through several paths is scanned once. v4 followed links by default	off
--ignore-case		Ignore case of symbols in code (experimental)	off
--ignore-identifiers		Treat all identifiers as equal, so clones that differ only in variable, function or type names are found. See Type-2 clones	off
--ignore-literals		Treat all string literals as equal and all numeric literals as equal	off
--ignore-annotations		Skip annotations and decorators (@Name, @Name(...)) before detection	off
--max-gap-lines		Merge clones of one file pair separated by at most N unmatched lines in both files into one near-miss clone reported as similar. See Type-3 clones	0 (off)
--similarity		Report JavaScript/TypeScript function pairs whose syntax-tree similarity reaches RATIO, a number in (0, 1], as similar clones; 1 means exact matches only. See function similarity	1 (off)
--formats-exts		Custom format-to-extension mapping (e.g. javascript:es,es6;dart:dt)	—
--formats-names		Custom format-to-filename mapping	—
--cross-formats		Detect clones across formats: ;-separated groups of ,-separated formats (e.g. javascript,typescript). Preset js-ts = javascript,jsx,typescript,tsx	—
--list		List all supported formats and exit	—
--skip-local		Skip clones where both fragments are in the same directory	off
--skip-isolated		Skip clones between different folders of the same isolation group: ,-separated groups of |-separated folders (e.g. packages/a|packages/b). Useful in monorepos where teams own separate packages	—
--baseline		Clone baseline file (e.g. .jscpd-baseline.json): clones whose fingerprint is absent from it are reported as new. See Baseline	—
--update-baseline		Rewrite the baseline file from the current run, creating it if missing (requires --baseline)	off
--fail-on-new-clones		Exit 1 when more than N new clones are found (--fail-on-new-clones alone means N=0; requires --baseline or --baseline-from-ref)	—
--fail-on-empty		Exit 1 when the scan analyzes no files: the paths exist but nothing matched the --format, --ignore and --pattern filters, or every file was below --min-tokens. See Exit codes	off
--baseline-from-ref		Compare against an ephemeral baseline built from a git ref's tree (e.g. origin/main). Conflicts with --baseline	—
--sarif-error-tokens		Report SARIF results as error for clones with at least this many tokens (smaller clones stay warning). When overall duplication exceeds --threshold, all SARIF results become error regardless of size.	— (all warning)
--min-duplicated-lines		Minimum percentage of duplication to report (0-100)	0
--mcp		Serve the Model Context Protocol over stdio: scan PATHs once, then expose check_duplication / get_statistics / check_current_directory tools to MCP clients	off
--summary		Print a codebase summary: top files and folders by tokens, lines, size, and a complexity estimate. See Summary	off
--summary-top		Number of entries in each summary top list	10
--summary-by		Summary sort metric: tokens, lines, size, complexity	tokens
--history		Duplication trend over git history: scan every commit in RANGE (e.g. v5.0.0..HEAD) with the same configuration and print a sparkline and a table. See History	—
--history-since		Select commits since DATE (e.g. 2026-01-01); alone it walks HEAD, with --history it bounds the range	—
--history-every		Keep every Nth commit of the series, counted from the newest	1
--history-limit		Maximum number of commits in the series, sampled evenly with both ends kept	30
--silent	-s	Suppress console output	off
--no-tips		Suppress tips and promotional messages. Tips are also skipped automatically when stdout is not a terminal (a pipe, a file, a CI log) or when CI or JSCPD_NO_TIPS is set; NO_COLOR only removes the colours	off
--version	-V	Print version	—
--help	-h	Print help	—

Reporters

15 built-in reporters:

Reporter	Output
console	Clone list + statistics table (default)
console-full	Clone list with source snippets; with --blame shows side-by-side author comparison
json	report/jscpd-report.json
xml	report/jscpd-report.xml
csv	report/jscpd-report.csv
html	report/jscpd-report.html
markdown	report/jscpd-report.md
badge	report/jscpd-badge.svg + report/jscpd-lines-badge.svg
sarif	report/jscpd-report.sarif (GitHub Code Scanning)
codeclimate (alias gitlab)	report/gl-code-quality-report.json — CodeClimate issue format, ready for GitLab artifacts:reports:codequality
openmetrics	report/jscpd-metrics.txt — OpenMetrics text format, ready for GitLab artifacts:reports:metrics
ai	Token-efficient output for LLM pipelines
xcode	Xcode-compatible warnings
threshold	Exit 1 if duplication percentage exceeds --threshold
silent	No console output

The xml report is always well-formed XML 1.0: a ]]> inside a snippet is split across two CDATA sections, and characters XML cannot represent at all (NUL, ANSI escapes, form feeds and other control bytes, U+FFFE/U+FFFF) are replaced with U+FFFD in snippets and file paths, so xmllint and XML parsers accept reports built from files that contain them. See fixtures/xml-report-demo.

File reporters write into the --output directory (default report/) using the jscpd-report.* prefix (e.g. jscpd-report.json, jscpd-report.sarif).

Summary

--summary appends a codebase summary to the run output — the statistics jscpd already collects while scanning, aggregated to answer "where should I refactor first":

Summary (by tokens; 321 files, 129 folders analyzed)
Top files:
  TOKENS  LINES   SIZE  CX  DUP%  PATH
    2052    363  11.4K  80   0.0  files.ts
    ...
Top folders:
  FILES  TOKENS  LINES   SIZE  CX  PATH
      8    5264    843  26.5K  15  src/core
      ...

• Top files lists the top --summary-top files ranked by the --summary-by metric. Every row carries all metrics, so re-ranking by another lens is a --summary-by size (or lines, complexity) away.
• Top folders aggregates files into their direct parent directory (each file counted exactly once; no cumulative ancestor totals).
• CX is a language-agnostic cyclomatic-complexity estimate computed from the token stream: 1 + the number of decision-point tokens (if, elif/elsif/elseif, unless, for, foreach, while, until, case, cond, when, catch, rescue, except, and, or, andalso, orelse, &&, ||, ?, ??). Matching is case-insensitive, so uppercase-keyword languages (SQL, PL/SQL, Fortran, COBOL, BASIC) count too. For folders it is the per-file mean. Languages that branch without such keywords (Smalltalk ifTrue: messages, Prolog clauses) stay at 1 — treat CX as a ranking signal, not an exact metric.
• DUP% is the share of the file's lines covered by detected clone fragments (both fragments of a clone count toward their files; display is capped at 100%).

The summary is fully opt-in and computed after detection from data already in memory, so runs without --summary are unaffected. It integrates with:

• console / console-full — the block shown above
• ai — a compact, LLM-token-efficient variant (one line per file/folder)
• json — an additive summary key in jscpd-report.json (absent when the flag is off, so the schema is unchanged for existing consumers)

Config file equivalents: "summary": true, "summaryTop": 10, "summaryBy": "tokens".

# Refactoring hotspots: biggest files by tokens plus duplication share
cpd ./src --summary

# Agent-friendly: compact clone list + compact summary
cpd ./src --summary --reporters ai --no-tips

# Focus on the most complex files, top 5 lists, machine-readable
cpd ./src --summary --summary-by complexity --summary-top 5 --reporters json

History

--history <range> answers "is duplication going up or down?" without a hosted dashboard. Every commit git log yields for the range is checked out into a temporary detached worktree (the same machinery as --baseline-from-ref), scanned with the run's own configuration, and its totals become one point of a series; the current run is the last point. Commits are scanned one after another, since each scan is already parallel across files.

jscpd src --history v5.0.0..HEAD                 # every commit in the range
jscpd src --history-since 2026-01-01             # every commit since a date, up to HEAD
jscpd src --history main --history-since 2026-06-01 --history-every 5 --history-limit 12

The console and console-full reporters append a block with a bar chart of the duplication percentage (its y-axis spans the series' own min and max, so small drifts stay visible), the table, the change from the previous point (red when duplication rose, green when it fell), the overall trend, and, when --threshold is set and the latest value sits below it, the headroom:

History (since 2026-01-01: 4 commits + working tree)
  duplicated lines, % of all lines: min 0.0%  max 58.1%  now 42.9%
    58.1% ┤       ██
          │       ██
          │    ▇▇ ██ ▇▇ ▇▇
    29.0% ┤    ██ ██ ██ ██
          │    ██ ██ ██ ██
          │    ██ ██ ██ ██
          │    ██ ██ ██ ██
     0.0% ┤ ▁▁ ██ ██ ██ ██
          └────────────────
            1  2  3  4  5

  #  COMMIT   DATE        FILES  LINES  CLONES  DUP LINES   DUP%  CHANGE  SUBJECT
  1  0a3e3d5  2026-08-01      1     11       0          0   0.0%          initial helpers
  2  6728f91  2026-08-08      2     21       1          9  42.9%   +42.9  copy total() into b.js
  3  aff51e5  2026-08-15      3     31       2         18  58.1%   +15.2  and again into c.js
  4  817b39d  2026-08-22      2     21       1          9  42.9%   -15.2  b.js imports total() instead
  5  working  2026-09-12      2     21       1          9  42.9%       =  (uncommitted changes)
Trend: +42.9 points since 0a3e3d5 (2026-08-01)
Threshold 50.0% has 7.1 points of headroom: the series never needed it, tighten it with --threshold 42.9

That last line is the manual form of a ratchet: jscpd reports how far the threshold could be tightened and leaves the change to you. The json reporter adds a history object (range, threshold, points[] with commit, short, date, subject, sources, lines, tokens, clones, duplicatedLines, percentage), and the ai reporter prints one compact line per point. Other reporters are unchanged.

--history-every N keeps every Nth commit counted back from the newest, --history-limit N (default 30) samples a longer series evenly while keeping both ends. Paths that did not exist at a commit count as zero. The range needs the commits locally: in a shallow CI checkout fetch them first (fetch-depth: 0). Config keys: history, historySince, historyEvery, historyLimit. See fixtures/history-demo for a runnable example.

Baseline

Gate CI on new duplication only, tolerating clones that already exist. A baseline file records a content-hash fingerprint per accepted clone (the same hash the SARIF reporter emits as partialFingerprints["jscpdCloneHash/v1"]); clones absent from it are reported as new.

# Create or refresh the committed baseline
jscpd --baseline .jscpd-baseline.json --update-baseline .

# Fail when new clones appear (independent of --threshold)
jscpd --baseline .jscpd-baseline.json --fail-on-new-clones .

# Allow up to 3 new clones
jscpd --baseline .jscpd-baseline.json --fail-on-new-clones 3 .

# Stateless variant for PR gates: compare against a git ref instead of a file
jscpd --baseline-from-ref origin/main --fail-on-new-clones .

--baseline-from-ref checks the base ref's tree out into a temporary detached worktree, scans it with the same configuration, and compares fingerprints in memory. It costs a second scan of the corpus; the committed file needs only one. In CI, fetch the ref first (fetch-depth: 0 or git fetch origin main).

New-clone information flows through the reporters: [NEW] markers in console/console-full, per-clone isNew plus newClones / newDuplicatedLines statistics in json, level error in sarif, severity major in codeclimate, and jscpd_new_clones / jscpd_new_duplicated_lines gauges in openmetrics.

Config file keys: baseline, baselineFromRef, failOnNewClones.

Blame Output

With --blame --reporters console-full, clones are displayed with a side-by-side author comparison:

176 │ Andrii Kucherenko │ <= │ 196 │ Josh Soref │ ## TODO
177 │ Andrii Kucherenko │ <= │ 197 │ Josh Soref │
180 │ Andrii Kucherenko │ == │ 200 │ Andrii Kucherenko │ ## License

== means both lines were written by the same author; <= means different authors (potential copy).

Exit codes

Code	When
0	The scan ran and no gate fired; clones may still have been found and reported
1	--threshold exceeded; --fail-on-new-clones exceeded; --fail-on-empty and no file was analyzed; a reporter failed to write its output; a scan path does not exist; --format names a format that is not supported; an invalid flag combination or an unreadable --config file
N	--exit-code N (default 1) when at least one clone was found
2	Command-line parse errors: an unknown flag or a missing value

A scan that analyzes no files, because the paths exist but nothing matched the --format, --ignore and --pattern filters or every file was below --min-tokens, prints Warning: jscpd analyzed no files and still exits 0, so an intentionally empty tree does not break a pipeline. --fail-on-empty (config key failOnEmpty) turns that into an error for CI jobs where an empty result means a misconfigured scan. Reports are written before the check, so the empty report is still there to inspect. Unknown reporter names remain a warning. See fixtures/fail-on-empty-demo for a runnable example.

Examples

# Scan a directory
jscpd /path/to/source
# or
cpd /path/to/source

# Tune sensitivity and pick reporters
cpd /path/to/source --min-tokens 30 --min-lines 3 --reporters console,json,html

# Git blame with side-by-side author comparison
cpd /path/to/source --blame --reporters console-full

# List supported formats
cpd --list

# Use multiple reporters with custom output
cpd ./src -r console,json,sarif -o ./reports

# Skip clones within the same directory
cpd --skip-local /path/to/source

# Monorepo: don't compare team-owned packages with each other
cpd . --skip-isolated "packages/team-a|packages/team-b"

Config File

Options can also come from a .jscpd.json config file (camelCase keys; existing v4 config files work unchanged):

{
  "path": ["./src"],
  "reporters": ["console", "json"],
  "minLines": 5,
  "minTokens": 50,
  "threshold": 0,
  "format": ["javascript", "typescript"],
  "ignore": ["**/node_modules/**"],
  "ignorePattern": ["generated by .*"],
  "ignoreIdentifiers": false,
  "ignoreLiterals": false,
  "ignoreAnnotations": false,
  "maxGapLines": 0,
  "gitignore": true,
  "mode": "mild"
}

Isolation groups use the nested-array form in the config file: "skipIsolated": [["packages/a", "packages/b"]].

Config discovery order: --config <path> → .jscpd.json → .config/jscpd.json (the dot-config convention, also accepts .config/.jscpd.json) → the jscpd key in package.json.

Ignoring source regions

Use --ignore-pattern (or ignorePattern in .jscpd.json) when only part of a file should be excluded. Each regular expression is matched against the raw source text before tokenization, and tokens that overlap a match are omitted from clone detection. This matches the v4 behavior; v5 uses Rust regex syntax, which does not support look-around or backreferences. A pattern that fails to compile is skipped with a warning.

The CLI flag splits its value on commas, so a regular expression that itself contains a comma (a {1,3} repetition, a character class such as [,;]) must be set in the config file instead.

For a one-off region, place jscpd:ignore-start and jscpd:ignore-end in comments that are valid for the source language:

// jscpd:ignore-start
const generatedLookup = {
  alpha: 1,
  beta: 2,
};
// jscpd:ignore-end

All source between the markers is excluded during tokenization. This works in every detection mode.

License headers are a common use case in languages whose comments are tokenized: C#, Java, Go, Python and every other format handled by the generic tokenizer. JavaScript and TypeScript comments never produce tokens, so a header in those files is never part of a clone and needs no exception. Mark a header explicitly when only a few files need it:

// jscpd:ignore-start
/*
 * Copyright 2026 Example Authors
 * Licensed under the Example License
 */
// jscpd:ignore-end

For the same block-comment header across many files, configure one anchored, dot-all regular expression instead:

{
  "ignorePattern": ["(?s)\\A/\\*.*?\\*/"]
}

The JSON escaping above produces the Rust regular expression (?s)\A/\*.*?\*/, which matches the first block comment only when it starts at the beginning of the file. Adjust the expression to the comment style and exact license text used by the project.

--mode weak is a blunter alternative: it drops every comment from detection in every language, so duplicated comments elsewhere in the code are ignored too.

Type-2 clones: renamed identifiers, literals and annotations

By default two blocks are a clone only when their tokens are identical. Three opt-in flags normalize token classes before hashing so that blocks which differ only in names or values match as well (Type-2 clones, the same idea as PMD CPD's --ignore-identifiers / --ignore-literals / --ignore-annotations):

Flag	Config key	What is normalized
--ignore-identifiers	ignoreIdentifiers	Every identifier hashes as $id. Keywords are kept, so if never matches while: the JavaScript/TypeScript tokenizer classifies keywords itself, other languages use a built-in list of common keywords
--ignore-literals	ignoreLiterals	String literals hash as $str, numeric literals as $num; true, null and regex literals keep their value
--ignore-annotations	ignoreAnnotations	@Name, @a.b.Name and @Name(...) runs are dropped in Java, Kotlin, Scala, Groovy, Python, Dart, Swift, JavaScript and TypeScript. Formats where @ prefixes a variable or directive (Ruby, Perl, Razor, T-SQL, CSS) are untouched

Positions in reports still point at the original source. Each clone carries a kind: exact when the raw tokens of both fragments are identical, renamed when they match only after normalization. The console prints Clone found (javascript, renamed), the ai reporter appends (renamed), the JSON report adds "kind": "renamed" to each duplicate, SARIF reports renamed clones under the rule jscpd/renamed-code (exact ones stay jscpd/duplicate-code), and Code Climate uses the same two check_name values. A run without these flags reports only exact clones and its output is unchanged. When --max-gap-lines merges two renamed halves, the result is similar: a merged clone is no longer identical even after normalization, so similar takes precedence over renamed.

jscpd --ignore-identifiers src/                       # function a(x) {…} matches function b(y) {…}
jscpd --ignore-identifiers --ignore-literals src/     # …and 10 matches 25, 'dev' matches 'prod'
jscpd --ignore-annotations src/main/java/             # @Override / @Deprecated no longer split a clone

Normalized runs find more and longer clones than exact runs, so their snippet fingerprints (jscpdCloneHash/v1) differ from an exact run's. Keep a separate --baseline file per configuration. See fixtures/type2-demo for a runnable example of each flag.

Type-3 clones: near-miss merging with --max-gap-lines

A copy with a line inserted, removed or changed in the middle shows up as two shorter exact clones with a gap between them, each of which must clear --min-tokens and --min-lines on its own. --max-gap-lines N (config key maxGapLines) merges clones of the same file pair whose fragments follow each other in both files with at most N unmatched lines in between, and reports the result as one clone of kind similar:

jscpd src/                       # a.js 1-4 ↔ b.js 1-4, a.js 4-9 ↔ b.js 5-10: two exact clones
jscpd --max-gap-lines 1 src/     # Clone found (javascript, similar (gap) ~0.85): a.js 1-9 ↔ b.js 1-10

A merged clone's tokens is the number of matched tokens and similarity is that number divided by the tokens of the longer merged span, so a single inserted line in a 60-token block gives roughly 0.9. Chains of matches merge transitively, and the merge is applied after every other filter (--min-lines, --skip-local, --skip-isolated). Because merging only ever joins clones the exact run already reported, it cannot introduce a match that was not there; it removes fragmentation. It applies to every language.

Reporting: the console prints Clone found (javascript, similar (gap) ~0.85), the ai reporter appends [~0.85 gap], the JSON report adds "kind": "similar", a "similarity" value and "method": "gap", SARIF files merged clones under jscpd/similar-code with similarity and similarity_method properties, and Code Climate uses the same check_name. The method is shown because the two near-miss mechanisms score on different scales: gap is matched tokens over the merged span, ast (from --similarity, below) is structural overlap of whole functions. A merge is refused when its similarity would fall below 0.5, that is when the gap holds more tokens than the halves match (one very long inserted line, say); the halves are then reported separately as before. Duplicated-line statistics count only the matched lines of a merged clone, so enabling the merge does not move --threshold. With the default 0 the merge pass is skipped entirely and output is identical to earlier releases. See fixtures/type3-demo for a runnable example.

Function-level similarity with --similarity

Edits spread through a function rather than concentrated in one gap still escape a token window. --similarity RATIO (config key similarity, a number in (0, 1]; the default 1 means exact matches only, so the pass never runs until you set a lower value) compares whole functions instead: every function declaration, function expression, method and arrow function in a JavaScript, TypeScript, JSX or TSX file is summarized by the bag of 4-grams over the pre-order sequence of its syntax-tree node types, and two functions are reported as one similar clone when the weighted Jaccard index of their bags reaches RATIO. Names and literal values are not part of the summary, so a renamed copy scores 1.0; one inserted line scores about 0.9; two inserted statements plus renames score about 0.75. Candidates come from a MinHash index, so the search stays close to linear in the number of functions.

jscpd --similarity 0.85 src/        # near-identical structure: renames, literal changes, a one-line edit
jscpd --similarity 0.7 src/         # looser: a couple of added or removed statements

Functions must clear --min-tokens and --min-lines on their own, nested functions are never paired with their parent, and a pair that an exact, renamed or merged clone already covers is not reported again. Reporting is the same as for merged clones except for the method: the console prints Clone found (javascript, similar (ast) ~0.75), the ai reporter [~0.75 ast], JSON carries "method": "ast" and SARIF similarity_method; tokens is the smaller function's token count and the fragments span the whole functions. Values outside (0, 1] print a warning and fall back to 1.

Scoring needs a syntax tree, and today only JavaScript/TypeScript have one (oxc). Each language plugs in through the FunctionExtractor trait in cpd-tokenizer (functions.rs): a grammar id, the formats it serves, and a walk that opens a function at every function-like node and records the node-type sequence inside it. Signatures carry their grammar id and are only compared within one grammar, so a tree-sitter-backed extractor for another language is a self-contained addition; the scoring, CLI, MCP tool and reporters need no change. Formats without an extractor are a silent no-op. The MCP check_duplication tool accepts the same similarity argument and returns the structurally similar project functions for each function in the snippet.

How detection works

jscpd is a token-based detector, but the tokens come from each language's own syntax rather than from splitting text on whitespace. A scan goes through these stages:

1. Format detection. The file extension (or a name such as Makefile / Dockerfile, see --formats-names) selects one of the 224 formats, and the format selects the tokenizer.
2. Tokenization. Every format has its comment style (// and /* */, #, --, Lua's --[[ ]], ;, Visual Basic's ', none for Markdown) and string-literal rules, so a comment marker inside a string, or a /* in Markdown prose, does not swallow the rest of the file. Each token is classified as identifier, literal, punctuation, comment or whitespace.
   • JavaScript, TypeScript, JSX, TSX are tokenized by the oxc parser: template literals, regular expressions, JSX and decorators come out as the tokens the language defines, and a recoverable parse error does not change the stream. With --cross-formats, erasable TypeScript-only syntax (type annotations, interfaces, type aliases, generics, access modifiers) is removed via the syntax tree so the remaining tokens match the equivalent JavaScript; constructs with runtime meaning (enum, non-declare namespace, parameter properties) are kept.
   • Vue, Svelte, Astro files are split into <template>, <script> and <style> blocks, and each block is tokenized as the language its lang attribute names. Markdown fences are tokenized as the language of the fence. Razor views separate C# from HTML. Clones are reported with the line range of the block, and a <script lang="ts"> block matches .ts files.
3. What counts. --mode mild (default) drops whitespace tokens, --mode weak also drops comments, --mode strict keeps every token. jscpd:ignore-start / jscpd:ignore-end comments exclude a region and --ignore-pattern regular expressions exclude whatever they match; skipped tokens leave the stream without shifting the positions reported for the rest.
4. Normalization (opt-in). --ignore-identifiers hashes every identifier as the same placeholder but leaves keywords alone (the oxc token kinds for JS/TS, a shared keyword table for other languages); --ignore-literals does the same for strings and numbers; --ignore-annotations drops @Name and @Name(...) sequences only in the languages where @ means an annotation or decorator (Java, Kotlin, Scala, Groovy, Python, Dart, Swift, JavaScript, TypeScript), never in Ruby, Perl, T-SQL, Razor or CSS, where it means something else.
5. Matching. A rolling Rabin-Karp hash over the token stream finds every repeated window of at least --min-tokens tokens and --min-lines lines, within a format and across the formats that share a pool.
6. Near-miss passes (opt-in). --max-gap-lines merges clone pieces separated by a few edited lines into one similar clone; --similarity extracts JavaScript/TypeScript functions from the syntax tree and compares their node-type sequences, so two functions with the same structure match regardless of names, literals or scattered edits.

What jscpd does not do is semantic analysis: two functions that compute the same result with different code (Type-4 clones) are out of scope, as they are for every token-based detector. See Types of Code Clones for where the line sits.

Dead code detection (--dead-code)

jscpd answers two questions about a codebase. jscpd . asks what is written twice; jscpd --dead-code . asks what is never run. The second is a separate engine — basta — that ships inside the same binary and also stands alone as a basta command.

# Anything nothing runs, in the current project
jscpd --dead-code .

# `--basta` is the same flag
jscpd --basta src

# Or the standalone binary, with no duplication half
basta src

It supports JavaScript, TypeScript, JSX, TSX and Python — ESM and CommonJS alike: require('./x'), const { a } = require('./x'), module.exports = { a, b }, exports.a = … and a literal import('./x') are all edges in the graph, not just import and export. A run walks with the same filters as a duplication run — --ignore, --format, .gitignore handling, --max-size, --follow-symlinks — and reports through the same reporter names, so -r sarif -o report means the same thing in both modes.

What it reports

Category	What it means
unused-file	No entry point reaches the file through the import graph
unused-export	An exported name no reachable module imports
unused-symbol	A module-private declaration nothing reaches
unused-import	An import binding with no references
unused-member	A class or enum member whose name is never read (opt-in)

Pick a subset with --dead-code-categories, or ask for everything:

jscpd --dead-code src --dead-code-categories unused-export,unused-import
jscpd --dead-code src --dead-code-categories all

unused-member is off by default: without type information, x.render() could be a call to any render in the project, so the rule is the least certain of the five.

How it decides

Everything rests on the entry points, because every finding is the answer to nothing reaches this. They come from three places, in decreasing order of authority:

1. Manifests. package.json's main, module, bin, exports, files and scripts; pyproject.toml's [project.scripts] and entry-point tables. A manifest that names a built file (./dist/index.js) is mapped back to the source it was built from, since that is what the repository holds. A source file listed under files ships to every consumer, so it is a public surface whether or not the package's own entry imports it.
2. Conventions. src/index.ts, __main__.py, manage.py, a framework's pages/ and app/ routes, *.config.ts, a .d.ts ambient declaration, a file with a shebang, a Python if __name__ == "__main__" guard, and a package's __init__.py.
3. Scripts. A shell script, a CI workflow, a Makefile or a Dockerfile in the tree that names a source file by path runs it, copies it or ships it. Those files are not JavaScript or Python, so nothing imports from them — but publish.sh requiring ./platform-map.js is as real a use as any import, and the file it names is an entry point.
4. You. --entry <glob>, repeatable, which adds entry points and never removes one.

From there it is two breadth-first walks: over import edges to decide which files run, and over reference edges to decide which declarations run. Because it is a traversal and not a reference count, dead code cascades — a helper whose only caller is dead is reported too.

Test files are always entry points, so a test file is never "unused". Dead code inside one is off by default; --include-tests turns it on. An export only the test suite imports is reported separately, and says so.

Confidence

Static analysis of JavaScript and Python cannot be certain, so basta does not pretend. Every finding carries a score from 0 to 100 and, below 100, the reasons it might be wrong:

Unused exports (1)
 - function src/registry.ts:12:17 registerPlugin  medium 60%  14 lines
   ↳ carries an unrecognised decorator; the name appears in a string literal

The score starts from a base set by how much inference the rule needs — an unreferenced import binding is a fact, an unused class member is a guess — and loses points for each piece of contrary evidence: a file that calls eval or getattr, a decorator the analyzer does not recognise, a wildcard re-export, a name that appears in a string, a file in the scan that did not parse.

--min-confidence sets the floor; the default is 60.

# Only what basta is sure of
jscpd --dead-code src --min-confidence 90

# Everything, including the guesses, with their reasons
jscpd --dead-code src --min-confidence 0

Raising the floor is the first thing to try on a codebase that does something unusual, and --entry is the second.

A file that fails to parse lowers the confidence of every finding in the run, because its references are unknown. The console trailer names such files (up to ten; the JSON report carries the full list under statistics.unparsedFiles), so a broken test fixture can be told apart from a real gap in the parser.

In CI

# Fail when dead code exceeds 2% of the codebase
jscpd --dead-code src --threshold 2

# Fail on any finding at all
jscpd --dead-code src --exit-code 1

# Publish to GitHub code scanning
jscpd --dead-code src -r sarif -o report

See fixtures/dead-code-demo for a runnable example of every category in both languages.

What it does not do

• No type inference. A member access matches members by name across the whole project, which is why unused-member is opt-in.
• No runtime resolution. getattr(obj, name), import(expr) and a module object passed as a parameter are recorded as uncertainty, not resolved.
• An export used only inside its own file is not reported. The export keyword is then unnecessary, but the code is not dead, and the two are different conversations.
• Only the five categories above. Unused dependencies, unused files in other languages, and unused local variables are out of scope; a linter already finds the last of those.

Format Support

JavaScript, TypeScript, JSX and TSX are tokenized by the oxc lexer; a parse diagnostic (a redeclaration, a recoverable syntax error) does not change the token stream, so such files still match files that parse cleanly. Only a source the parser gives up on entirely falls back to a word-split tokenizer, and that file then matches only other fallback-tokenized files. See fixtures/parse-errors-demo.

jscpd supports 224 formats. Use cpd --list to see the full list, or see FORMATS.md for names, file extensions and descriptions.

Cross-Format Detection

Vue SFC (.vue), Svelte (.svelte), Astro (.astro), and Markdown (.md) files are tokenized per-block/per-section, enabling duplicate detection across file types. In a Vue file only the block bodies are scanned; the wrapper tags around <template>, <script> and <style> are left out, so a template clone is reported with the template's own line range. See fixtures/sfc-demo for a runnable example.

Cross-Format Groups (--cross-formats)

By default every format is compared in its own isolated pool, so a TypeScript file never matches a near-identical JavaScript file. --cross-formats declares format equivalence groups that share one comparison pool — useful for finding leftover .js copies during a TypeScript migration:

cpd --cross-formats "javascript,typescript" ./src
cpd --cross-formats js-ts ./src                      # preset: javascript,jsx,typescript,tsx
cpd --cross-formats "js-ts;css,scss" ./src           # multiple groups

When a group mixes TypeScript (typescript/tsx) with JavaScript (javascript/jsx), TypeScript files are compared with erasable type syntax stripped from the detection token stream — type annotations, generics, interface/type declarations, as/satisfies, ?/! markers, access modifiers, implements clauses, type-only imports/exports, overload signatures, and declare statements. Reported clone positions always reference the original sources.

Config file equivalents (all three shapes are accepted):

{ "crossFormats": "javascript,typescript;css,scss" }
{ "crossFormats": ["javascript,typescript", "css,scss"] }
{ "crossFormats": [["javascript", "typescript"], ["css", "scss"]] }

Notes:

• TypeScript syntax with runtime semantics is not erased and will not cross-match: enum, non-declare namespace, parameter properties (constructor(private x)), import x = require(), export =.
• A cross-format clone is attributed to one member format in the per-format statistics.
• Overlapping groups are merged; groups with fewer than two formats are ignored.

Migrating from jscpd v4

jscpd v4 (TypeScript engine) is maintained on the master-v4 branch and published as jscpd@4. Moving to v5 needs no changes to flags or config in most projects; the differences:

Feature	jscpd v4 (Node.js)	jscpd v5 (Rust)
--blame	Calls git CLI for each file	Same output (==/<= markers), calls git blame --porcelain per file
--store (LevelDB/Redis)	Persistent store for large repos	Not supported; the flag is ignored with a warning. Available on the master-v4 line.
--formats-exts	Custom format-to-extension mapping	Same flag name, same behavior
--formats-names	Custom format-to-filename mapping	Same flag name, same behavior
Programming API	jscpd() Promise API, detectClones()	Rust API via cpd-finder crate; no Node.js API
Config file	.jscpd.json with camelCase keys	Same — .jscpd.json with camelCase keys
Cross-format detection	Vue SFC, Svelte, Astro, Markdown	Same — per-block tokenization
Token counts	Varies by tokenizer	May differ by 1-2% due to Rust tokenizer; clone detection matches
--reporters	All v4 reporters	All v4 reporters except full (use console-full)
--no-gitignore	Default respects .gitignore	Same behavior, same flag name
Symbolic links	Followed by default; --noSymlinks to skip them	Skipped by default; --follow-symlinks (config followSymlinks: true) to follow them. A v4 config with noSymlinks: false still maps to following
--workers	Not available	Available — control parallelism for file tokenization/detection
Output filenames	jscpd-report.json, html/ directory	jscpd-report.json, jscpd-report.html, jscpd-report.sarif, jscpd-report.csv, jscpd-report.md, jscpd-badge.svg, jscpd-lines-badge.svg

Rust API

For integration in Rust applications:

use cpd_finder::orchestrate::{RunConfig, run};

let config = RunConfig {
    paths: vec!["./src".into()],
    min_tokens: 50,
    ..Default::default()
};

let result = run(&config).unwrap();
println!("Found {} clones", result.clones.len());
println!("Analyzed {} files", result.statistics.total.sources);

Architecture

cpd / jscpd (CLI binary)        basta (CLI binary)
 ├── cpd-core      — Detection algorithm (Rabin-Karp rolling hash), report models
 ├── cpd-tokenizer — Language tokenization (224 formats)
 ├── cpd-finder    — File walking, orchestration, git blame
 ├── cpd-reporter  — Output formatting (15 reporters, for both modes)
 └── basta         — Dead code: per-language analyzers, module graph, reachability

Both binaries walk with cpd-finder and report through cpd-reporter; the finding types live in cpd-core beside the clone models, so a reporter can render a dead-code run without linking the analyzer that produced it.

Inside basta, one file per language under src/lang/ implements the Analyzer trait: it turns a source file into declarations, imports and references, resolves the language's import specifiers against the index of scanned modules, and declares the language's entry-point conventions, manifests and path traits. Nothing outside lang/ knows a language. graph.rs merges every file's facts into one address space and runs the two reachability passes, confidence.rs scores what is left, and classify.rs decides what is worth saying. Adding a language is a new file under lang/ plus an entry in the ANALYZERS registry — see docs/basta-extending.md.