apk
b6dc11fce0
Closes #375. A snippet containing a NUL, an ANSI escape, a form feed or any other control byte made the xml report unparseable: XML 1.0 has no way to represent those characters, not even inside CDATA, so xmllint stopped at "PCDATA invalid Char value 27" and every parser refused the file. The reporter now replaces every character outside XML's Char production (tab, LF, CR and U+0020 and up, minus U+FFFE/U+FFFF) with U+FFFD in snippets and in path attributes. The existing "]]>" split is kept. Path attributes were also escaped twice: quick-xml escapes attribute values on its own, so the reporter's escape_xml() turned "r&d" into "r&amp;d". The manual escape is gone and the sanitized path goes in raw. Tests parse the generated report to the end with quick-xml and check: control characters become U+FFFD and none survive, a "]]>" round-trips through the split sections, a path with & < " > is escaped exactly once and reads back unchanged, and legal text is left borrowed. Fixtures: fixtures/xml-report-demo with a clone whose body holds real escape bytes, a form feed and a "]]>" literal; the README shows the xmllint and Python checks. Verified the published 5.2.0 fails xmllint on it and the fixed binary passes. Claude-Session: https://claude.ai/code/session_01WztdFKZAwW9b51iJe9zhtV
xml-report demo
Shows that the xml reporter produces well-formed XML even when a clone
contains bytes XML cannot represent (issue #375). All commands run from the
repository root at default thresholds; xmllint ships with macOS and
libxml2 on Linux.
| Directory | Contents | Default scan |
|---|---|---|
escapes/ |
Two JavaScript files sharing a function whose body holds real ANSI escape bytes (0x1B), a form feed (0x0C) and a "]]>" literal |
Found 1 clones. |
escapes/: a clone the XML report has to carry
banner.js and banner-copy.js differ only in the function name, so the
clone spans the whole body, escapes included.
jscpd fixtures/xml-report-demo --reporters xml,console --output report
# Clone found (javascript)
# - escapes/banner-copy.js [5:33 - 18:2] (14 lines, 115 tokens)
# escapes/banner.js [5:28 - 18:2]
# Found 1 clones.
# XML report saved to report/jscpd-report.xml
xmllint --noout report/jscpd-report.xml
# (no output: the document is well-formed)
Inside the report the escape and form-feed bytes are replaced by U+FFFD, the
replacement character, because XML 1.0 has no way to encode them, not even in
CDATA. Every ]]> is split across two CDATA sections, so a parser reads the
snippet text back unchanged:
grep -c '<!\[CDATA\[' report/jscpd-report.xml
# 6
python3 -c "import xml.dom.minidom as m; m.parse('report/jscpd-report.xml'); print('ok')"
# ok
Before the fix (jscpd 5.2.0 and earlier) the same command wrote the bytes
verbatim and xmllint reported PCDATA invalid Char value 27 on the first
escape and refused the file.