`\ No newline at end of file` describes the line directly above it. The
parser never renders context lines, so a marker git attached to an
unchanged last line landed under the last `-`/`+` line rendered and
claimed THAT line lacked a newline -- the one place the output stated
something untrue about the diff. The marker is now kept exactly when the
line above it was emitted (in-hunk after a `-`/`+`, or on the line right
after the budget closed on one), which keeps the witness of a
newline-only change and drops a marker that would describe the wrong
line. `emitted_at` replaces `closed_at`: the position test is the same
for both rules. Real fixture `git_diff_no_newline_raw.txt` carries both
shapes.
`# HG changeset patch` opens a message region like an mbox `From`, so
`hg export`'s column-0 headers and message no longer latch
`dropped_prologue` and send the whole changeset raw through its own
`diff -r <a> -r <b> <file>` echo. Real fixture `hg_export_raw.txt`
(Mercurial 7.0.1) joins the corpus.
`rename to`/`copy to` (and their `from`) go through `dequote`, so a
`core.quotepath` name has one spelling across a stream. The submodule
note keeps the range's direction qualifier:
`[file] sub (submodule e139196..b0ac9b1 rewind)`.
The body-line replay in `corpus_every_marked_body_line_survives` shares
the parser's per-parent presence rule (there is no second formula for
combined-diff columns); its doc now says so and names the oracle -- git's
header counts -- and it asserts a budget is never still owed at a
non-body line or at EOF, so under-consumption fails there too.
GNU diff translates `Only in` and `Binary files … differ` (git translates
none of its own lines). Rule 6 read the English spelling only, so on a
French machine `diff -ru g1 g2 | rtk diff -` dropped the three translated
fact lines as prose and, because a text section still parsed, returned
`Some` -- four files became one, silently.
Rule 3b now reads GNU diff's per-file `diff <opts> X Y` echo (printed
whenever it compares directories) as the mark of a `diff -r` stream. Such
a stream carries no prose, so a column-0 line no arm read there is a fact
in a language the parser does not speak, and rule 9 returns `None`
instead of dropping it. A line dropped before the echo settled the
question counts the same way once it does -- `diff -r` may list an
`Only in` first. Real fixture `diff_ru_fr_raw.txt` (`LC_ALL=fr_FR.UTF-8`,
diffutils 3.10) pins the raw fallback; the English `diff_ru_raw.txt`
still parses in full, as do `git log -p` streams whose prologue is prose.
The `Submodule <path> <a>..<b>` note now carries the range --
`[file] sub (submodule e139196..b0ac9b1)` -- for parity with the dirty
arm's `(submodule, modified content)`.
Review round 3 on the file-level-fact arms, plus a cold pass on top:
- `Only in <dir>: <file>` splits on the FIRST `: ` (GNU diff's separator
follows the directory; a filename may carry its own).
- Standalone `Binary files X and Y differ` splits at the ` and ` where
both sides agree past their first component, so a filename containing
` and ` is not cut, and is named through `header_name` so the `a/`/`b/`
strip follows the same decision as a header pair.
- `* Unmerged path X` folds into the section git emits for X right after
it: one `[file] X (unmerged) (+A -B)` entry instead of two.
- `Submodule <path> <a>..<b>` keeps only the path in the name slot (the
tokens before the LAST `..` token, so a path containing `..` survives),
and the sha-less `Submodule <path> contains ... content` dirty line is a
fact too -- it used to be dropped silently beside a condensed sibling.
Real fixture `git_diff_submodule_dirty_raw.txt` joins the corpus.
- `rename to` / `copy to` is git's exact path; the `+++` pair no longer
re-derives it, which stripped a real `b/` directory under `--no-prefix`.
- `core.quotepath` wraps non-ASCII paths in quotes on every header line;
names are dequoted before the prefix strip.
- The header-pair hunk gate applies to open hunkless sections as well:
git only emits `---`/`+++` before the first `@@`, so a stray pair there
is lost content and falls back raw instead of renaming the section.
- `diff --cc X` carries one path and leaves `prefixed` to its header pair.
- `past_diffstat` comment states the first-section-mangled shape it alone
catches; the ` name | 3` bound says it fires always (the `---` separator
is the marked line that follows); the stdin doc says the never-worse
check is inlined and why.
A/B against 9106d48: 400 `git show` streams byte-identical; 290
format-patch series identical except 12 that fell back raw through a
prose pipe and now condense, zero new fallbacks.
Structure is read through a per-line ANSI/CR-stripped view and content is
pushed raw, so escapes that are part of the user's content survive (the
file path was already byte-faithful on the same input). Rule 8's mbox
tolerance now ends at the first diffstat line: an orphaned hunk body in a
hand-edited series falls back raw instead of being dropped as prose, while
version notes between `---` and the diffstat stay prose. `diff --git X Y`
with X == Y marks a --no-prefix stream whose header names keep their
leading a/ b/ (creations and deletions included); the check is confined
to the two-path form. The new-side `\ No newline` marker is kept only on
the line right after its hunk closes. The dead `!in_prologue` conjunct
and the variable behind it are gone.
Real `git diff --cached --no-prefix` fixture added to the corpus.
- '\ No newline at end of file' survives to the output (in-hunk and the
common new-side position just after the budget closes): it is the only
witness of a trailing-newline-only change, which otherwise renders as
two byte-identical -/+ lines. Real fixture; proven load-bearing by
revert.
- Non-UTF-8 stdin now always takes the raw-bytes branch, even when the
stream parses: condensing would rewrite content bytes to U+FFFD, and
byte fidelity outranks savings. (Base hard-errored here; raw
passthrough is strictly better on both counts.)
- Doc-comment bounds recorded for the mbox header-pair spoof and the
raw-fallback newline append.
- File-level facts outside hunks become note-only entries instead of
vanishing while the stream condenses: 'Only in <dir>: <file>' and
standalone 'Binary files ... differ' (GNU diff -r), '* Unmerged path'
(git diff --ours during a merge), and 'Submodule <a>..<b>' headers.
Suppressed inside mbox message regions, where column-0 prose is
indistinguishable by value. Real fixtures captured for all four shapes;
fix proven load-bearing by revert.
- is_mbox_from accepts 64-hex separators: SHA-256 repos' format-patch
streams no longer fall back raw wholesale.
- .context() on the stdin read and raw-fallback write paths.
- Savings comment corrected to the measured corpus numbers (52-87%
metadata-heavy, single digits content-heavy); exemption escalation
unchanged.
- new file mode / deleted file mode sections (an empty file added or
removed) get their own notes; previously they vanished silently whenever
another file in the stream parsed cleanly. Real fixture captured, fix
proven load-bearing by revert.
- Signature tolerance (rule 6) now requires the stream to have carried an
mbox From separator; a bare '--' in a plain diff falls back raw.
- never_worse discrimination by token estimate instead of pointer
identity; stdin decoded once (Cow) instead of twice; hunk-body prefix
checked on the byte view without a per-line allocation.
- Structural fallback now emits the caller's exact bytes (a lossy UTF-8
decode or ANSI-stripped 'raw' is not raw); the decode-strip-parse-guard
pipeline moved into condense_stdin so it is unit-testable.
- Dropped the mailer-padding tolerance: a line shorter than the hunk's
prefix width now falls back instead of being guessed into context.
- Plain-diff fixtures recaptured with relative paths (no local usernames
or session ids); copy-only and mode-only sections get real fixtures and
a vanish-class test.
Parse the stream into (prologue)(file-header)(hunk)* regions before
classifying: hunk extent comes from the @@ line budget (one old-budget per
parent, so @@@ combined headers parse too), content is classified only
inside hunks, and any structural disagreement (invalid body prefix, budget
over/under-run, budget owed at EOF, malformed @@) falls back to raw
passthrough. Detector precedence is a documented total order on
condense_unified_diff_strict.
Fixes, each with its reproducer from
claudedocs/diff-classifier-review-2026-08-29.md as a test:
- SQL/Lua '-- comment' removals (--- on the wire) no longer dropped
- added '++ text' lines no longer parsed as file headers
- format-patch mbox prose and the '-- ' signature no longer counted
- +++ /dev/null deletions named by their --- side
- binary / rename-only / mode-only sections reported instead of vanishing
- --color input stripped (was: silently empty output), non-UTF-8 stdin
read lossily (was: hard error, no fallback)
- CRLF content bytes survive verbatim (split('\n'), not lines())
- b/ prefix stripped exactly once; timestamps tab-split from header names
- the '... +N more' trailer removed: it claimed truncation that never
happened (all lines were printed above it)
Fixture corpus: 13 real-producer captures (git 2.54, GNU diff) in
tests/fixtures/diff -- multi-file, -U0, -W, rename/delete/binary,
log -p, show --cc, format-patch single/series/cover, diff -u/-ru/-rN,
CRLF. Property tests: every marked body line survives to the output,
counters equal rendered lines, the fallback fires on zero corpus fixtures.
kylehgc