The pipeline diagram, the discover walkthrough and the `rtk read` option
table still described the `--max-lines` mapping this branch replaced, and
the repo's own suggest hook still proposed `--max-lines N` for `head -N`
— the mapping this branch exists to correct.
Drop the `head -` disjunct in the rewrite guard: every string with that
prefix also has the `head ` prefix that sits next to it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Two defects in `filter_ast_grep`, both verified against ast-grep 0.45.3.
`ast-grep scan` diagnostics parse only on their locator line, so filtering
line by line kept ` ┌─ a.rs:2:13` and dropped the rule id, severity,
message and source line — at exit code 0, so it read as success. The
whole-output fallback never fired because `order` was not empty.
`unparsed_signal()` now passes any shape through untouched when a single
non-blank line fails to parse, which is what `search.rs` already does for
grep/rg. This also covers `--heading` mode and Windows drive-letter paths,
where `[^:]+` cannot match `C:\src\a.rs`.
The per-file overflow hint was computed as `entries.len() - max_per_file`,
which ignores `max_total` cutting a file short: a file under its own cap
lost its remainder with no hint at all, and a file over it under-reported
the drop. Both now count against what was actually printed.
The hint said "matches" while counting lines. ast-grep prints one line per
matched source line and a structural match spans several, so a repo search
reported "19 more matches" where five matches remained.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
A large blob is windowed to a preview with a recovery hint, so redirecting a
manual 'rtk git show HEAD:x > file' can silently truncate (exit stays 0). Note
it in the git-show user docs and point at plain 'git show' / the recovery hint.
A hunk ran until the next header, so any trailing text was read as its body.
`gh pr diff --patch` routes here, and GitHub's `.patch` is an mbox: each patch
carries a bare `---` before the diffstat and a `-- ` signature after the diff,
both at column 0. With hunk bodies now anchoring, a two-commit mbox with two
real deletions reported `grep -c "^-"` = 5. `@@ -a,b +c,d @@` declares its own
line counts, so the hunk now ends when they are consumed and anything after it
is outside every hunk. That covers trailing prose and truncated input too,
rather than the shapes we happen to know about, and a heuristic on `---` / `-- `
could not have worked: a deleted line whose content is `- ` renders as `-- `.
`--patch` also joins `has_non_diff_format_flag`, so an mbox passes through
whole. Someone asking for a patch wants one that applies.
Leading context keeps the last lines before the change rather than the first.
With `-U10` or `--function-context` the first three sit ten lines above it, and
emitting those tells the reader that ctx3 precedes the deletion when ctx10 does.
A three-slot ring buffer costs the same lines. The diff-wide budget is charged
when the buffer drains, so a line the ring evicted costs nothing.
Combined diffs carry one marker column per parent, so a line changed against
only one of them holds its marker in column 2. Those were counted as context:
neither in the `+N -M` tally nor reachable by an anchored grep. The tally reads
both columns now. The grep gap is real and cannot be closed without rewriting
the line, so FEATURES.md states it alongside the per-hunk cap. Against
`git diff --numstat` on three complete commit ranges the tally is now exact,
where master under-counted one of them by +2 -1.
`diff_header_path` keeps paths containing spaces: git does not quote them, so
`diff --cc my file.txt` needs the whole remainder, not the last token.
FEATURES.md said 15 lines for a 24-line block and showed the stat's first line
with a leading space it does not have. It now also carries a measured example
with real compression, since the two-file one saves 11% in a document
advertising 60-90%.
Dropping the `+++` / `---` guards assumed those headers only appear before the
first `@@`. That holds for `diff --git` sections. It does not hold for the
`diff --cc` sections git emits for unmerged paths: `diff --cc` never matched
the `diff --git` branch, so `in_hunk` stayed set from the previous section and
the `--- a/z.txt` / `+++ b/z.txt` headers of every later file were printed at
column 0 and counted in the tally. On a two-file conflict with no deletions,
`rtk git diff | grep -c "^-"` returned 1. The state reset now fires on any
`diff --` prefix, and the path falls back to the trailing token for the
`diff --cc <path>` shape, so the headers land before the first `@@` again.
Same repo now returns 0.
Leading context no longer counts against `max_lines`, so it cannot displace
change lines. Its previous justification was wrong: before this branch, leading
context was never emitted, so it could not have crowded anything out of
`max_hunk_lines`. The cost it does carry is on the global budget, and on a
five-commit diff of this repo it cost 57 change lines (144 shown, against 201
with the exemption). A diff-wide cap of `max_lines / 10` bounds what the
exemption gives back: without it, a diff of many small hunks would spend three
exempt lines on every one of them. The same diff comes out at 71.7% byte
reduction against 71.8% on master, which showed zero anchorable change lines.
`docs/usage/FEATURES.md` showed change lines indented under a
`src/main.rs (+5/-2)` header, which no version of rtk emits. It now carries
real captured output, plus the two limits of the anchored audit: the 100-line
per-hunk cap, and that the output is not an applyable patch.
Condense the Grep variant and its tests to the comment rules in
CONTRIBUTING.md (explain why, not what), and carry the -l/-t change
through the files that still assumed the old behavior.
- scripts/test-aristote.sh: add -r to the two grep assertions, since
`grep PATTERN <dir>` exits 2, and move the type filter to `rtk rg`,
the engine where -t is valid.
- docs/usage/FEATURES.md: extra args go to the engine actually invoked,
not always to rg; drop the per-row shortcut justifications.
- tests/search_compress_test.rs: guard grep-engine tests on
grep_available() instead of rg_available(), which skipped them into a
false green on a box without ripgrep.
- .claude/hooks/rtk-suggest.sh: reverted; the rg -> rtk rg fix is
correct but belongs in its own PR.
BREAKING CHANGE: `rtk grep --file-type` and its `-t` short are removed.
The option never reached the engine, so it was a silent no-op; `-t` now
flows through, which means `rtk rg -t rust` filters by type while
`rtk grep -t rust` returns grep's own `invalid option -- 't'`. `-l` and
`-m` under `rtk grep` are likewise the native grep flags now rather than
rtk's --max-len and --max.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
scripts/test-all.sh asserted `rtk grep "pub fn" src/ -t rust` exits 0. That
only held because --file-type swallowed `-t` and never passed it to the
engine, so the assertion was pinning a silent no-op. With --file-type gone,
`-t` reaches GNU grep, which has no such flag:
$ rtk grep "pub fn" src/ -t rust
grep: invalid option -- t
exit=2
Replaced with an assert_fails pinning that rejection, plus an assert_ok
against rg (which does have -t), guarded on rg being installed. Verified:
rtk grep "pub fn" src/ -t rust -> exit=2 (assert_fails wants non-zero)
rtk rg "pub fn" src/ -t rust -> exit=0 (assert_ok wants 0)
$ bash -n scripts/test-all.sh
syntax OK
Also drops a `--line-numbers | -n` row from the FEATURES.md grep table. No
such option exists on Commands::Grep -- its fields are max_len, max,
context_only, extra_args. The `line_numbers` field at src/main.rs:118
belongs to Read. The row predates this PR but sits in the table it edits.
Note, NOT fixed here as it is pre-existing and unrelated: the adjacent
`assert_ok "rtk grep pattern" rtk grep "pub fn" src/` on the line above
also fails, because grep without -r on a directory exits 2. The smoke
suite is not wired into CI (referenced only from
.github/copilot-instructions.md), which is how it rotted unnoticed.
rtk's own tuning options on `Commands::Grep` declared short forms that
collide with native grep/rg flags of the same letter. This removes the two
that remain, and deletes one option outright.
`--max-len` loses `-l`. `-l` is grep's --files-with-matches, and the old
binding split behavior on whether the PATTERN parsed as a usize:
- Non-numeric: clap rejected it, run_fallback re-ran raw grep, and the
user got the right answer. Wasteful, not harmful.
- Numeric: clap ACCEPTED it, and the result was silently wrong.
# hit.txt contains "listen on port 8080 today"; miss.txt does not
$ rtk grep -l 8080 hit.txt miss.txt
before: (no output) exit=1 <- claims nothing matched
after: hit.txt exit=0
`-l 8080` set max_len=8080, leaving hit.txt as the pattern and miss.txt
as the only path. With a single file the filename becomes the pattern,
no path is left, and rtk falls back to stdin: exit 1 against /dev/null,
hangs on a terminal. Port numbers, years and error codes are ordinary
patterns, and the Claude Code hook rewrites plain `grep` into `rtk grep`
transparently -- so this reached users who never opted in.
Note this is not a savings story. `-l` is a has_format_flag passthrough
on the fixed path too (search.rs:548), so there is no compaction for
this flag either way. The bug was a wrong answer.
`--file-type` is removed entirely, with its `-t` short, rather than left as
a parse-only no-op. It was destructured as `file_type: _` at the call site
and never reached search::run, so it parsed and then did nothing.
`rtk rg -t rust` is unaffected: Rg has no such field and forwards `-t` to
real ripgrep.
Deliberately typed `fix`, not `fix!`, even though a documented long option
disappears. The option was inert -- it never filtered anything -- so no
working behavior is lost, and #3259 set the precedent for removing a
colliding grep short under plain `fix`. Recording the choice here because
silence would read as an oversight.
User-visible consequence: `rtk grep -t rust` goes from silently returning
unfiltered results to surfacing the engine's own error. BSD/macOS grep
prints `invalid option -- t`; GNU grep prints `invalid option -- 't'`.
`--max`/`-m` is NOT part of this change. #3259 removed that short already;
this branch is rebased onto it and preserves its rationale comment verbatim.
The struct NOTE deliberately does not claim to finish the job. clap's auto
`-h` still shadows grep/rg's --no-filename (that is #2532's scope), and
removing a clap short does not make a letter safe end to end: search.rs
shares one VALUE_FLAGS_SHORT table across both engines, so a letter taking
a value in rg but not grep still eats the next token inside a cluster --
`rtk grep -rt FOO .` consumes FOO as -t's value regardless of clap.
Adds clap-layer tests pinning the routing, including characterization tests
for behavior that was previously untested.
`exclude_commands` entries name a tool, but a command can spell that tool with a
wrapper (`npx playwright test`), an interpreter (`python3 -m pytest tests/`) or a
path (`vendor/bin/phpunit tests/`). Those spellings are absorbed by each rule's own
pattern rather than stripped beforehand, so the anchored `^playwright($|\s)` never
matched and the exclusion silently did nothing — the README shipped
`exclude_commands = ["curl", "playwright"]` as the example, and `playwright` is a
tool almost nobody invokes bare.
Peel the wrapper off the command and match what remains, alongside the existing
check on the typed command. The peeled form keeps the arguments, so an anchored
entry still narrows the way it was written: `"^ls$"` excludes a bare `ls` without
swallowing `ls -la`.
Peeling uses the rule's own `rewrite_prefixes`, taking the shortest token-suffix of
the matched prefix that is itself a prefix of that rule. That drops `npx` and
`python3 -m` while keeping a subcommand the rule treats as part of the tool, so
`golangci-lint run` does not collapse to `run`.
Peeling reuses the PHP normalization the rewrite path already applies (`php`
wrapper and ini flags, leading `./`, vendor/composer bin dir), extracted into
`php_tool_form` and shared by both, so `php vendor/bin/phpunit tests/` is excluded
by `["phpunit"]` the same way `vendor/bin/phpunit tests/` is.
The peeled check is gated on a non-empty `exclude_commands`, keeping the default
config off the `RULES` scan on the hook rewrite path.
Matching the resolved `rtk` target instead would have been shorter but wrong in
both directions: it misses tools whose target differs from the binary (`["eslint"]`
would still rewrite `npx eslint .`, since the target is `lint`), and it leaks
across tools sharing a target (`["read"]` would exclude `cat`, `["git"]` would
exclude `yadm`). Peeling has neither failure.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`-m` was derived as the short for RTK's `--max` (a display cap), colliding
with GNU grep where `-m N` = `--max-count` (stop after N matches per file).
`rtk grep -m N` set RTK's display cap and never forwarded --max-count, so
grep didn't actually stop early — a silent semantic mismatch (no parse error).
Drop the short. `-m` now flows to extra_args and is forwarded verbatim to
grep/rg (it is not a has_format_flag letter, so it takes the normal engine
path, not passthrough), applying genuine --max-count while RTK still compacts.
This makes Grep consistent with Rg, which never had the short and already
forwarded `-m` correctly. `--max` keeps its long form and default 200.
- src/main.rs: drop `short` from Grep::max + clap-parse regression test.
- tests/grep_faithful_format_test.rs: end-to-end `rtk grep -m N` == `grep -n
-m N` (leading, trailing, N >= total) plus the per-file multi-file case.
- docs/usage/FEATURES.md: drop the `-m` short; fix stale default 50 -> 200.
The tree was copied into 11 files, so every future correction to it meant
11 edits in 7 languages. It now lives only in savings-explained.md, which
each of those pages already links to.
The surrounding prose stays: it carries the dilution point in the reader's
own language, which is the part that matters at a glance. Three pages
introduced the diagram with a trailing colon, reflowed into the following
paragraph. TRACKING.md gained the link it was missing.
Drop the 33 standalone "**Economies :** ~N%" claims, which asserted a
number with no context next to each command.
Rename the metric everywhere it remains. "Economies" on its own reads as
money saved; the tables measure bash output bytes removed. Column headers,
the section heading and its anchor, the rtk gain description, the telemetry
field list and the category summary now all say so explicitly.
The arrow diagram read as a pipeline where bash output becomes input tokens
which become cost. The real relationship is containment: bash output is
part of input tokens, and input tokens are part of cost alongside output
tokens.
Replace the arrow chain with a tree in all 12 places it appeared, including
the six translated READMEs:
Cost
├─ Input tokens
│ ├─ Bash output <- the only part RTK filters
│ ├─ Your prompt
│ ├─ System prompt
│ └─ Conversation history
└─ Output tokens <- what the model writes
This also makes the dilution self-evident: RTK shrinks one leaf, so the
effect on the root is bounded by that leaf's share.
RTK was documented as delivering "60-90% token savings", which reads as a
cost reduction. What RTK actually reduces is bash output bytes. Those are
one contributor to input tokens, which are themselves only part of a bill
that also counts output tokens, so the reduction dilutes at every step.
- add docs/guide/resources/savings-explained.md as the canonical explainer:
the savings chain, both estimators, and what RTK does not reduce
- rescope the headline claim across README (7 languages), the guide, hook
rules, agent definitions and module READMEs
- relabel per-command tables as bash output reduction, keeping every figure
- document that reported tokens are estimates: rtk gain uses bytes/4
(src/core/tracking.rs), filter tests use split_whitespace().count().
Neither is a real tokenizer, so ratios hold but absolute counts do not
Remove figures that had no source: the $3/Mtok constant and its $36
example, the +/-10% tokenization accuracy claim, the 99.5% hook-install
figure, the invented session tables in README and INSTALL, and the 30-50%
parser range.
CHANGELOG is untouched. Shipped release notes stay as a historical record.
Adds reset flag documentation to the upstream file path:
- Quick Reference: reset usage examples with and without --yes
- Command Options: Reset Flag table with --reset and --yes descriptions
- Warning note about irreversibility and non-interactive behaviour
https://claude.ai/code/session_015WXbDdxEU3pXPBtheDkXcd
Nicolas Le Cam