Luke Sandberg
ce14ca88a4
## What Replaces the `@babel/code-frame` dependency with a new Rust-based implementation (`next-code-[frame](https://github.com/arthurprs/qfilter/pull/20#issuecomment-3986882055)` crate) for rendering code frames in error messages. ### Why - **Crash fix**: `@babel/code-frame` uses the `js-tokens` library for syntax highlighting, which has [known issues](https://github.com/lydell/js-tokens?tab=readme-ov-file#known-failures) with large string literals and long lines. This can cause Next.js to throw RangeErrors when rendering errors, hiding the original issue! - **Long line support**: The old implementation had no concept of terminal width, dumping entire lines into the output. The new implementation uses "horizontal scrolling" — truncating lines and centering the error location in the visible window. - **Performance**: The Rust implementation only processes the visible line range (typically ~6 lines), not the entire file. Syntax highlighting uses a skip-scan heuristic to start tokenizing near the visible window rather than from byte 0. - **Dependency reduction**: Drops the semi-unmaintained `@babel/code-frame` bundled dependency in favor of code we control. ### Benchmarks In-process benchmarks comparing `render_code_frame()` (Rust, via criterion) against `codeFrameColumns()` (Babel, via hrtime with DCE prevention). Both have syntax highlighting and color output enabled. No process startup or file I/O is included in the measurement. | Scenario | `next-code-frame` (Rust) | `@babel/code-frame` | Speedup | |---|---|---|---| | Small file (~490 lines TSX) | **5.4 µs** | 507 µs | **~94x** | | Large file (~39k lines JS) | **143 µs** | 82.9 ms | **~580x** | | Large file minified | **51 µs** | - | **-** | The gap widens with file size because Babel's `highlight()` runs a regex tokenizer over the **entire source** before slicing to the visible window, while the Rust implementation uses a windowed line index and skip-scan heuristic — only processing the visible window regardless of file size. However, for minified files we do end up tokenizing the whole thing so we end up being only 8x faster ### How - New `crates/next-code-frame/` Rust crate with: - Frame rendering with terminal-width-aware horizontal scrolling - Regex-based syntax highlighting (matching Babel's color scheme) - Skip-scan heuristic for O(1)-ish highlighting regardless of file size - Windowed line index that only scans/stores offsets for the visible region - Comprehensive test suite (800+ lines) - Exposed via both NAPI (native) and WASM bindings - JS wrappers in `packages/next/src/shared/lib/errors/`: - `code-frame.ts` — primary wrapper using native bindings - `optional-code-frame.ts` — graceful fallback returning `undefined` if bindings are unavailable - All existing callsites (`diagnosticFormatter`, `parseScss`, dev overlay, turbopack utils) updated - for `patch-error-inspect.ts` i adopted an `injection` style approach to avoid coupling to the native dependency ### Concerns / review focus areas - **Reliability**: The regex-based tokenizer is best-effort and language-agnostic — it should never crash on invalid syntax, but highlighting accuracy may differ from Babel's `js-tokens` in edge cases. - **Native dependency**: This moves code frame rendering into the native binary. Performance should be better, but worth verifying there are no regressions in environments where native bindings behave differently (e.g. WASM fallback path). - **Regressions**: The output format and color scheme closely match Babel's, but there may be subtle differences. The horizontal scrolling behavior is new. Fixes #85357 Closes PACK-5754
3.5 KiB
next-code-frame
Fast, scalable code frame rendering for Next.js error reporting, written in Rust.
This crate provides functionality similar to @babel/code-frame but with several improvements:
- Scalability: Handles arbitrarily large files efficiently
- Long line handling: Gracefully scrolls long lines to keep error positions visible and avoid overwhelming the terminal with long lines
- Syntax highlighting: Uses a language-agnostic regex tokenizer for best-effort syntax highlighting
Design
Following the next-taskless pattern, this crate:
- Has no dependency on turbo-tasks, allowing use in webpack/rspack codepaths
- Is compilable to WASM for environments without native bindings
- Follows "sans-io" patterns - the library accepts file content as arguments rather than performing IO
CLI
A code_frame binary is included for quick testing. Pass a filename and start/end positions (1-indexed line:column):
# Highlight a single position
cargo run -p next-code-frame --bin code_frame -- src/app.tsx 10:5
# Highlight a range
cargo run -p next-code-frame --bin code_frame -- src/app.tsx 10:5 10:20
# With an error message
cargo run -p next-code-frame --bin code_frame -- -m "Unexpected token" src/app.tsx 10:5 10:20
Syntax highlighting
The highlighter uses a single compiled regex pass over the visible content to
tokenize strings, comments, numbers, regex literals, and identifiers. Keywords
are identified via a compile-time perfect hash set (phf).
Skip-scan heuristic
For large files, scanning from byte 0 is expensive — the regex tokenizer
dominates runtime. To avoid this, extract_highlights() walks backwards from
the visible window looking for a blank line and starts the scan there.
A blank line is a safe restart point for single-line constructs (strings,
line comments, regex literals) because they cannot span blank lines.
Known limitation: The heuristic can produce incorrect highlighting when a multi-line construct (block comment or template literal) contains a blank line that falls between the scan start and the visible window. In this case the scanner misses the opening delimiter and the closing delimiter / trailing code may lose its expected coloring. For example:
/** sneaky
*/
const after = 1; // `*/` may lose comment highlighting
This is a deliberate tradeoff — blank lines inside block comments or template literals that span the window boundary are vanishingly rare in practice, and the consequence is only slightly wrong highlighting, never a failure or missing output.
Byte-level skip for long lines
When the visible window starts far into a long line (>200 bytes from the
line-level scan start), the heuristic additionally scans backwards from the
visible start for a ; and restarts the tokenizer there. This is critical for
minified files where the entire source may be a single line — without it the
scanner would tokenize hundreds of kilobytes of invisible content.
Known limitation: The ; can land inside a string literal, causing an
unbalanced quote that cascades incorrect highlighting across the visible window.
In practice minified code has frequent ; between statements so this rarely
triggers, and the consequence is only incorrect highlighting, never a failure.
Features
- Caller-provided output width (no terminal detection — sans-io)
- Syntax highlighting for JS, TS, JSX, TSX
- Graceful degradation for non-JS files or parsing errors
- ANSI color support matching babel-code-frame aesthetics
- Support for single-line and multi-line error ranges