`ParserReturn` contains a property `panicked`.
The name `panicked` is misleading. The parser did _not_ panic. When `true`, it means that the parser encountered a fatal error and is unable to parse the source, because it contains an unrecoverable syntax error.
Rename this property to `fatal_error`, to better describe what it means.
This is a breaking change, but one that's trivial to fix in downstream code - the property doesn't change its meaning or behavior, just its name.
All the changes in this PR in other crates are just reacting to this change and making everywhere use the term "fatal error" instead of "panicked", so the codebase is consistent.
Fix tokens produced when `<<` opens a type argument list. e.g.:
```typescript
type Bar = ReturnType<<T>(x: T) => number>;
```
Previously only 1 `<` token was produced between `ReturnType` and `T`. This PR corrects that so 2 `<` tokens are produced.
TS-ESLint get this wrong too - hence why this wasn't noticed before, as our conformance tests compare to TS-ESLint.
PR submitted to fix TS-ESLint: https://github.com/typescript-eslint/typescript-eslint/pull/12821
In meantime, this PR skips the erroneously failing ESTree tokens conformance tests. We can can re-enable those tests once the fix lands in TS-ESLint.
#26263 added the ability to run tests and conformance for both versions of `oxc_lexer` (SIMD and scalar fallback) on any platform.
Run tests and conformance on CI (both SIMD and scalar) on any PR which touches `oxc_lexer` crate or other files which affect conformance runner.
Additionally, run Clippy on the SIMD version (main Lint CI task doesn't lint it, due to the `target_feature = "avx2"` gate on the code).
All of this sets up `oxc_lexer` crate for further development, catching any regressions in CI.
`oxc_lexer` has 2 implementations:
1\. SIMD-based on x86_64 with AVX2.
2\. Scalar/SWAR fallback which works on all platforms.
Both produce the same output.
This PR enables running the lexer conformance on either implementation. Which implementation is tested depends on what build flags are passed to `cargo coverage`.
It turns out that it's possible to test the SIMD implementation on Mac by compiling as x86_64, and running it under Rosetta 2.
This PR also adds `just` recipes to run tests and conformance on either implementation:
- `just test-lexer`
- `just test-lexer-simd`
- `just conformance-lexer`
- `just conformance-lexer-simd`
These commands should work correctly on any of Linux x86, Windows x86, Mac aarch64.
Refactor the conformance tester for lexer.
The diff is large, as I've reordered the code so it now reads top-to-bottom, but it's pure refactor. The actual logic is exactly as it was before - just moved things around, renamed vars, shortened code etc.
This also fixes some Clippy warnings which were present before, but hidden because this code is gated behind `target_feature = "avx2"`. That changes in next PR #26263, at which point CI would have started failing if the warnings weren't fixed here first.
Small fix to `cargo coverage`.
Previously a typo e.g. `cargo coverage prser` would fall through to the catch-all arm and would run all coverage tasks.
Instead, this PR makes it panic with an error `Unknown coverage task: "prser"`. This ensures you are running what you think you are.
Oxc can shorten variable names and private class names, but it cannot shorten property names such as `obj._field`. This change adds optional property mangling. It replaces #23772 and gives bundlers a way to use the same property names across several files.
### Feature
- `include` selects names to mangle. `exclude` and `reserved` keep names unchanged.
- The cache can give a property a fixed output name or keep it unchanged.
- The mangler supports member access, object and class keys, destructuring, optional chains, the left side of `in`, and JSX property names.
- `mangle_quoted` also handles strings and template literals used directly as property keys. `/* @__KEY__ */` and `/* #__KEY__ */` can mark one other string or template literal as a property key.
- Short names are assigned in a stable order. More common properties get shorter names. Existing and reserved names are avoided.
- Bundlers can collect names from several programs, create one shared map, and apply it to every program.
- Numeric keys, `__proto__`, `constructor`, `prototype`, TypeScript-only keys, and private names are never changed by this pass.
### Design
Property names are part of JavaScript syntax, not local scope. For this reason, the code lives in `oxc_minifier`. It only reuses the short-name generator from `oxc_mangler`.
For one program, the order is:
```text
collect names → assign names → rewrite once → compress → mangle variables and private names
```
Property names are rewritten before compression because compression can remove information about quotes and comments.
For several programs, a bundler collects names from each program, merges the results, assigns one shared map, and rewrites each program once. Collection can run in parallel. All programs must use the same options.
The shared map keeps property names consistent inside one build. The cache has a different job: it keeps fixed names and reservations across separate minify calls.
### Example
With `include: "^_"` and `mangle_quoted: true`:
```js
obj[`_field`] = 1;
obj._field = 2;
```
becomes:
```js
obj[`e`] = 1;
obj.e = 2;
```
### Limitation
- Property mangling is unsafe when other code still uses the old property name. Public APIs and names shared with external code, host APIs, imported namespace objects, or separate builds must be excluded or reserved.
- Quote handling is per use. Without `mangle_quoted`, `obj._field` may change while `obj["_field"]` does not.
- The mangler cannot find names built at runtime or stored in arbitrary strings. Code that uses `eval`, `Function`, or `with` must reserve any affected names.
- Compression may create a matching property after this pass has finished. For example, `'f' + 'oo_'` may later become `foo_` and will not be mangled.
- TypeScript enum reverse mappings are not supported because the transform creates quoted strings.
- Multi-program support is a caller API. The caller must collect every program that shares mangled properties before assigning names.
The public `oxc-minify` API is stacked in #24741. The downstream multi-chunk integration is shown in rolldown/rolldown#10374.
Supersedes #23772.
AI assistance: OpenAI Codex and Claude were used for research, implementation, review, and testing. The contributor is responsible for reviewing and understanding the changes before submission.
- dispatcher
- string_embedder
- tailwind_sorter
Group these 3 together as `SessionServices`, make them available to `FormatSession`, and pass them around.
Review feedback was that the token kinds should be a real enum rather than a module of u8 constants. The values were listed twice, once as the constants and again in the token_kind_name match, so they now come from one declaration, and tok_kinds and the new Lexer::kinds hand out &[TokenKind] instead of raw bytes.
The discriminants do not move. The pipeline computes kinds arithmetically and blends them in SIMD registers, so it keeps working on u8. Only the aliases in pipeline and the compile-time tables in opmap are re-expressed in terms of the enum, and those fold away at compile time. Reading the bytes back as TokenKind assumes every kind written is a declared discriminant, which a debug assertion checks on every accessor. Output is identical across both corpora and an interleaved A/B against the branch below lands inside the noise floor.
The token tape carried a u32 start per entry and kept whitespace and comments on it as ordinary tokens, so anything reconstructing a token's end had to reach for the next entry and every consumer had to step over trivia. Because whitespace was itself a token, the end of each significant token was already in the array as the start of the following entry, just stored under a different name and then discarded. This publishes one oxc_span::Span per significant token instead, start in the low half and end in the high half, bit-identical to the AST's own Span, with trivia gone from the tape entirely. A span turns out to be a misaligned eight-byte window onto positions the pipeline already had, so build_spans widens and compacts them with a qword permute and a byte shuffle rather than recomputing anything, and compress now fills a fixed staging buffer a chunk of blocks at a time instead of a full-length array, carrying one entry across chunks so the last token of a chunk can still see its end. Eight EOF sentinels spanning (n, n) follow the last token so a consumer can peek a short way past the end without a bounds check, and the hashbang stays on the tape, since its kind sits inside the trivia range and the drop mask excludes it explicitly.
Value lanes, cooked bytes, comment records and diagnostics are byte-identical across both fixture corpora, and the span stream was checked against a scalar reference on every fixture. It costs about 0.22 cyc/byte, roughly 1.0 to 1.5 cycles per significant token, because the tape writes nine bytes per token where it used to write five per entry across about a third more entries than there are real tokens. Worth saying plainly that this one costs throughput rather than winning it. It is a contract change that moves work out of the consumer, and it only pays off once the consumer stops reconstructing ends, skipping trivia and scanning for token boundaries.
#24349 made unused-class removal conservatively keep every class whose heritage resolves to a class, lexical, or `var` binding. That preserves class-evaluation errors which the minifier assumptions explicitly allow optimizations to ignore, and it leaves `(class extends Base {});` residue in Rolldown chunks after tree-shaking ([rolldown#10274](https://github.com/rolldown/rolldown/issues/10274)).
The minifier already documents both `No TDZ Violation` and `No side effects from extending a class`, and `oxc_ecmascript` explicitly ignores invalid-superclass errors so removable derived classes are not broadly deoptimized. Treat resolved identifier heritage like other pure reads while retaining the pre-existing literal-arrow exception, including its paren/sequence unwrap. Effects from evaluating the heritage expression, decorators, and effectful class elements remain observable.
The runtime harness already has minifier-only exclusions for unsupported inputs. Six Test262 fixtures covered by these assumptions are skipped only before the minifier variant, with the reason documented beside the list; codegen and transformer still execute, and `runtime.snap` remains unchanged.
## Example
Input:
```js
export var Base = class {};
export var Keep = class extends Base {};
var REMOVE = class extends Base {};
```
Before:
```js
var Base = class {}, Keep = class extends Base {};
(class extends Base {});
export { Base, Keep };
```
After:
```js
var Base = class {}, Keep = class extends Base {};
export { Base, Keep };
```
Verified with `cargo test -p oxc_minifier`, `cargo coverage runtime`, Clippy with warnings denied for `oxc_minifier` and `oxc_coverage`, `just minsize`, `just allocs`, formatting, and `just ready`. Minsize and `runtime.snap` are unchanged; the allocation snapshot records fewer allocations for `kitchen-sink.tsx`.
Implemented with AI assistance (Claude Code and OpenAI Codex). The contributor remains responsible for reviewing, understanding, and submitting the changes under the repository AI usage policy.
Migrate the transform-pipeline `Visit` implementors to `VisitJs` (#24499): `oxc_transformer` (3), `oxc_minifier` (4), `oxc_react_compiler` (1), plus the parser regex example and the coverage driver. Part of #24317; linter half in #24530. Behavior is unchanged.
Together with #24530: `oxlint` −129.4 KiB (−0.94%) stripped, `oxc_transform` napi −16.2 KiB (−0.46%).
## Why each visitor can switch
A visitor is safe on `VisitJs` when its hooks cannot observe anything inside the pruned type grammar — either the hooked node kinds are value-space-only, or the AST it walks cannot contain type nodes at that pipeline stage.
| Visitor | File | Why equivalent |
|---|---|---|
| `PrivateInExpressionDetector` | `crates/oxc_transformer/src/decorator/legacy/mod.rs` | Hooks `visit_private_in_expression` (+`visit_decorators`); `#x in y` is runtime-only and cannot occur in type grammar, even inside decorator expressions carrying casts. |
| `BindingMover` | `crates/oxc_transformer/src/es2017/async_to_generator.rs` | Its formal-parameter overrides visit only pattern + initializer (`BindingPattern` carries no type field — annotations are sibling fields it never touches); runs in the exit phase after TS strip. |
| `UsedInJSXBindingsCollector` | `crates/oxc_transformer/src/jsx/refresh.rs` | Runs on un-stripped TSX, but its hooks (JSX-shaped calls, `visit_jsx_opening_element`, declarations/imports) cannot fire inside type grammar; its hand-written `visit_ts_type_annotation` skip-stub is deleted — `VisitJs` prunes that natively. |
| `Finder` | `crates/oxc_react_compiler/src/react_compiler/entrypoint/program.rs` | Matches functions by `scope_id`; real JS function scopes never live in pure type grammar, and `VisitJs` still walks the places one could hide (enum initializers, namespace bodies, cast inner expressions). |
| `KeepVar` | `crates/oxc_minifier/src/keep_var.rs` | Hooks `visit_statement`/`visit_variable_declaration`; statements and `var` declarations don't exist in type space, and namespace/enum bodies (which can contain them) are still walked by `VisitJs`. |
| `DirectEvalFlagCheck` | `crates/oxc_minifier/src/peephole/mod.rs` | Debug-only; hooks `visit_call_expression` only — call expressions cannot occur in valid type grammar. |
| `LiveDirectEvalCollector` | `crates/oxc_minifier/src/peephole/mod.rs` | Same single call-expression hook; type-space-immune regardless of input. |
| `TestVisitor` | `crates/oxc_minifier/tests/ecmascript/prop_name.rs` | Test helper parsing `SourceType::default()` (plain JS); single non-recursing `visit_object_expression` hook. |
| `RegularExpressionVisitor` | `crates/oxc_parser/examples/regular_expression.rs` | Finds `RegExpLiteral` / `new RegExp(...)`; neither node kind can occur in type grammar. |
| `CheckASTNodes` | `tasks/coverage/src/driver.rs` | Its only active override is `visit_reg_exp_literal` (regex print idempotency); `RegExpLiteral` cannot occur in type space. (The commented-out `visit_span` full-AST check would need `Visit` if ever revived.) |
Left on `Visit`: the minifier's incremental-scoping bookkeeping (`DropDiff`, `OverPruneCheck`, `LiveRefCollector`) — dropping `let x: Foo = …` must unmark the type-space `Foo` reference, which the `peephole::remove_unused_declaration` tests verify (they fail under `VisitJs`; confirmed and reverted during this work). `ChildScopeCollector` also stays: TS type nodes (mapped/conditional/function types) carry real `scope_id`s it must collect.
---
🤖 This PR was written with the assistance of Claude Code (Claude Fable 5).
Adds the x86_64 version of my Research `SIMD For Context Sensitive
Languages`.
(CPB = Cycles per byte)
- Adds new create crates/oxc_lexer; Which eventually lead into an
unfused parser design inside oxc_parser
- Fully test262 compliant JS standalone Lexer; SIMD used has
counterparts on every platform.
- Performs at 2.5CPB on checker.js on my ZEN 3 CPU; or around 1.5-1.75
gbs (3.8GHz).
- Across aarch64, + other platforms performance will hold which I've
seen in my POCs but those targets will come later.
- Supports JSX/TS options, Which don't have much impact on overall perf
results.
------
The expected results for seeing this work through [including on parser
side on oxc_parser]
1. P95 improvements of 80-100% over current `oxc_parser`; 12-18% of this
will come from the unfused contract where JS grammar specifically loves
being able to see 'ahead' into `kinds` in various places. The rest comes
from the perf improvements to the lexer itself and some wins that will
come inside the parser from work now done in the lexer.
## Summary
Self-review follow-up to #24034 (stacked on #24052), hardening the runtime suite's server transport:
- **Ephemeral ports + stdout handshake**: the fixed 32055..N port range let an orphaned server from a killed run pass the readiness probe (silently testing against a stale binary) and made two concurrent runs in different worktrees cross-talk — one run's shutdown DELETE kills the other's servers. `runtime.js` now listens on an OS-assigned ephemeral port and reports it as the first stdout line, which the runner reads as both the port and the readiness signal (the polling loop and port math are gone); remaining server stdout is drained by a background thread so `DEBUG=1` output can't deadlock a full pipe.
- **Transport retry**: monitor-oxc's first live Test262 run flipped 13 for-of cases to `timeout: global` — ureq's 10s client timeout tripping under CI load while requests queued, not a semantic change. A localhost-specific agent (no proxy, one-minute timeout) plus a single retry fixes it; the server responds even for failing tests, so a transport `Err` never carries a verdict.
Result-neutral: 17279/18277, `runtime.snap` byte-identical across repeated runs.
## Summary
Self-review follow-up to #24034 (stacked on #24049): the runtime suite now fits the standard `run_tool` framework — `runtime::run` is a plain `fn(&[Test262File]) -> Vec<CoverageResult>` runner, deleting its hand-copied print/snapshot tail so reporting-convention changes can't silently skip it. `cargo coverage all` no longer reads the ~52k-file test262 tree twice (`run_all_with` shares one loaded `TestData`); the standalone `runtime` command still loads only test262, which monitor-oxc CI relies on. Also hoists the per-request `workspace_root()` walk to once per file.
Result-neutral: 17279/18277, `runtime.snap` byte-identical, verified via both `cargo coverage runtime` and `cargo coverage all`.
## Summary
Follow-up to #24034 from self-review: `available_parallelism()` spawned N Node servers even when `--debug` caps the rayon pool at 1 thread, leaving N−1 servers idle (only `current_thread_index() % num_servers` = server 0 is ever used). Size the server count from `rayon::current_num_threads()` instead. Full suite verified unchanged: 17279/18277, byte-identical `runtime.snap`.
## Summary
#18478 reduced the test262 runtime runner to a stub ("not yet implemented"), and since oxc's own CI never runs `cargo coverage runtime`, the transformer/minifier semantics coverage it provided has been dark — monitor-oxc's Test262 job (the only consumer) has been disabled since July 2025.
This restores execution on the simplified architecture: each eligible test262 case runs through codegen / transform / minify and executes in the Node runtime server, driven by rayon with blocking HTTP instead of the old tokio design. Skip lists and per-variant semantics are ported verbatim from the pre-#18478 runner.
- `main.rs` now loads `TestData` lazily, so `cargo coverage runtime` works with only the test262 fixtures checked out (which is how monitor-oxc CI invokes it).
- Execution is parallelized: one Node server per core with each rayon worker pinned to its own server, and the test262 harness is compiled once per distinct include list as a reusable `vm.Script` instead of re-parsed per request. Full suite: ~10s wall locally (~45s single-server).
- Results at test262 `de8e621c`: 17279/18277 positive tests pass (94.54%); `runtime.snap` is committed and byte-stable across repeated runs; clippy/fmt clean.
- monitor-oxc side: oxc-project/monitor-oxc#178 (must merge after this one).
Note: oxc CI still does not run the runtime suite, so `runtime.snap` can go stale and turn monitor-oxc's job red — the failure mode that led to the original "skip 262". Possible follow-ups: run it in conformance CI, or add a regression-only mode (fail on new failures, tolerate improvements).
## Summary
Ports typescript-go's symbol/type/error baseline harnesses
(`internal/testutil/tsbaseline/{type_symbol_baseline,error_baseline}.go`)
into `tasks/coverage` as new conformance suites, measuring oxc against
the `.symbols`, `.types`, and `.errors.txt` baselines already shipped in
the TypeScript submodule (~14k each). All report through the shared
`CoverageResult` machinery, so their summaries/snapshots look exactly
like the other suites.
Run with `cargo coverage symbols` / `cargo coverage types` / `cargo
coverage errors`.
## What's measured
- **symbols** — walk the AST (`oxc_ast_visit`), resolve lexical bindings
via `oxc_semantic`, emit `Symbol(name, Decl(file, line, col), ...)` with
declaration positions recovered from the parser token stream (to match
TypeScript's trivia-inclusive `node.pos`), then diff against the
baseline.
`AST Parsed 8364/8364 (100%)`, `Positive Passed 1765/8364 (21.10%)`.
- **errors** — match oxc parser diagnostic positions + TS codes (oxc's
parser attaches real `TSxxxx` codes via `ts_error`) against the
`.errors.txt` summary headers.
`AST Parsed 7699/7699 (100%)`, `Positive Passed 7251/7699 (94.18%)`.
- **types** — **scaffold**. Same harness parameterized by kind
(mirroring typescript-go's `isSymbolBaseline`), but oxc has no type
checker, so every type payload is `<<unresolved>>`. The full runner
(baseline discovery, walk, output format, comparison, snapshot) is wired
and ready for a checker to fill in the walker's `Types` arm.
`AST Parsed 8364/8364 (100%)`, `Positive Passed 78/8364 (0.93%)` — the
passes are files with empty type baselines.
## Notes
- oxc has no type checker, so `.types` and the type-checker surface of
`.symbols`/`.errors.txt` (member/`this` symbols, qualified names, lib
globals, `TS2xxx`/`TS18xxx` errors) are out of reach — that gap is the
measurable target for a future checker, which is exactly what the
`.types` scaffold sets up.
- The suites are opt-in subcommands (like `transpiler`), not part of
`cargo coverage all`.
---
🤖 Written with [Claude Code](https://claude.com/claude-code) (AI
assistance); reviewed by the author, per oxc's AI usage disclosure
policy.
Previously, whether to serialize AST to ESTree JSON with/without TS fields was a compile-time option - via 2 different configs `CompactTSSerializer` and `CompactJSSerializer` (and their pretty-printing and with-fixes counterparts).
Turn this into a runtime option instead -
This make serialization a little slower (more branches in either mode, and dead code in JS-only mode) but reduces the size of `oxc-parser` binary as only one `ESTreeSerializer` is generated, instead of 2 (JS-only, and with-TS).
Summary:
- Upgrade oxc_sourcemap to 8.0.1.
- Return borrowed SourceMap from CodegenReturn.
- Support no-sourcemap feature builds used by benchmarks.
AI-assisted: yes
## Summary
Introduces `oxc_diagnostics::Diagnostics` — a `Vec<OxcDiagnostic>` newtype that knows the difference between errors and warnings — and makes it the standard diagnostics return value across oxc.
- **New type**: `Diagnostics` with `has_errors()`, `has_warnings()`, `errors()`, `warnings()`, plus `Vec`-like ergonomics (Deref, `IntoIterator`, `Extend`, `From`/`Into<Vec>`) so most call sites are unchanged.
- **Return types**: `ParserReturn`, `SemanticBuilderReturn`, `IsolatedDeclarationsReturn`, the React Compiler `TransformResult`/`LintResult`, and `TransformerReturn` now expose `diagnostics: Diagnostics` instead of `errors: Vec<OxcDiagnostic>` (the React Compiler's separate `errors`/`warnings` are merged into one severity-tagged list).
- **Behaviour**: a React Compiler **error** now stops the transform (no output), while **warnings** flow through and the transform continues. `compile()` aborts codegen only when `diagnostics.has_errors()` — warnings alone no longer abort (previously any diagnostic did).
## Breaking change
Consumers reading `*.errors` on these return values must read `*.diagnostics`. React Compiler consumers that read `.errors`/`.warnings` separately should use `diagnostics.has_errors()` / `diagnostics.errors()` / `diagnostics.warnings()`.
Parser/semantic/isolated-declarations fatality is otherwise unchanged. Builds on the React Compiler transform feature (#23201).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## What
Make the full `AstNodes` store **opt-in**, so the compiler pipeline (transformer / codegen / minifier) can run semantic analysis without building it at all.
Previously every `SemanticBuilder` run allocated and populated the full `AstNodes` store, even for consumers that only need scoping + the syntax checker. This separates the node-related builder state into three independent pieces:
- a standalone `NodeId` counter — the source of truth for node ids, valid in either mode;
- an **ancestry stack** — an upward-walk-only stack of the current node's ancestors, the only ancestry the syntax checker and binder actually read;
- the full **`AstNodes`** store — random access to every node, needed by the linter and mangler.
## How
- A new `AstNodeStore` (in `crates/oxc_semantic/src/node/store.rs`) bundles the node-id counter, the current-node cursor/flags, and the storage. The storage is an enum (`AstNodeStoreKind`) with two variants — `Full(AstNodes)` and `Ancestry(AncestryStack)` — so the full store and the ancestry stack are **mutually exclusive by construction**; they can never both be maintained at once.
- The checker and binder read ancestry through an `Ancestry` view (`node/ancestry.rs`) that dispatches between the full store (linter / mangler) and the stack (compiler pipeline). They no longer do random `NodeId` access into the store — all ancestry access is upward-walk-only.
- `SemanticBuilder` gains `with_build_nodes(bool)`. The default is **off** (ancestry stack only). `new_linter()` turns it on (the linter needs the full store); `new_compiler()` leaves it off.
- `CompilerInterface::build_semantic_nodes()` lets `oxc::Compiler` consumers opt back in if their `after_semantic` hook reads `Semantic::nodes`.
## Notes
- Behaviour-neutral: semantic conformance snapshots are unchanged.
- Allocation snapshots updated — the default (ancestry) path allocates slightly less (the growing `Stack` trades a couple of fixed allocations for a few reallocs).
- Builds on the already-merged `SymbolFlags`-based redeclaration checks (#23060) and the `new_linter`/`new_compiler` preset adoption (#23057), which removed the last random node reads from the redeclaration path so the checker works without the store.
The Printer never consumes these; they belong with each language formatter.
- Evict `Quote` (each formatter owns `QuoteStyle`)
- `normalize_string` takes a quote byte
- Evict `Expand` / `BracketSpacing` / `TrailingCommas` into oxc_formatter and oxc_formatter_json
- JSON's `TrailingCommas` is now two-variant (`Always`/`Never`)
- collapse Prettier's `all`/`es5` at each translation layer
- Rename `util/` -> `spec/` (pure shared helpers)
- Split `SourceText`: mechanical access stays in core, JS/TS lexis moves to `SourceTextExt`
- Drop the `oxc_syntax` dependency from core
All behaviros, Prettier conformance are unchanged.
Align `oxc_formatter` with `oxc_formatter_json`, both now expose `format(...) -> Result<Formatted<'a, _>, OxcDiagnostic>`.
- Add text-in `format` and `format_fragment`, also `format_program`, `parse_for_format` and the `FragmentContext` enum
- Parse options, JSX enabling, js-in-xxx context aware handlings are applied internally
- Make the `Formatter` struct and AST/IR primitives crate-internal
- Fold `get_parse_options` / `enable_jsx_source_type` and the `service` module into `parse_for_format`
Inside an internal `namespace N {}` / `module N {}`, statements that reference a
module are not permitted — this matches both babel and TypeScript:
- `import ... from "m"` / `import x = require("m")` → TS1147
- `export ... from "m"` / `export * from "m"` → TS1194
- `export default ...` → TS1319
- `export = ...` → TS1063
Bare `export { x }` (re-export of locals, no module source) and `import x = A.B`
(entity alias) stay valid, as do external modules (`declare module "x" {}`) where
these forms are allowed — the check only runs for internal namespaces.
Each statement is validated inline as the namespace body is parsed (no second pass).
Also removes TS1147 from the coverage denylist (`NOT_SUPPORTED_ERROR_CODES`), since
oxc now detects it.
Conformance: parser_babel +8 and parser_typescript +9 negative-passes vs main, with
positive/AST staying 100% across all parser suites (no regressions).
Split `oxc_formatter` into:
- `oxc_formatter(_js)`: JS/TS specific formatter, comment handling, sortImports, jsdoc, etc
- `oxc_formatter_core`: Language agnostic IR, printer, etc
The naming convention is different from Biome or Ruff (they do not have `_core`, and `_formatter` is equivalent to the core), but this is because in Oxc, items without `_js` are the default.
As a basic task, I have added `Context` as a generic to all relevant parts to ensure support for multiple languages.
There might still be other components that could be refactored into the core, but I would like to finish them as I implement support for other languages.
Also moved some shared utilities to the core in advance.
## Summary
`cargo coverage` (a.k.a. `just c`) crashes locally on macOS with `fatal runtime error: stack overflow` while running `semantic_typescript` on `typescript/tests/cases/compiler/binderBinaryExpressionStressJs.ts`. That fixture builds a ~1500-deep left-leaning `BinaryExpression` chain (`0 + 1 + 2 + ... + 1499`).
macOS's `std::thread` default stack is 2 MB; Linux is 8 MB, which is why CI doesn't see this. The file used to fit because `OxcDiagnostic` was 8 bytes (`Box<OxcDiagnosticInner>`); after #22406 inlined the ~200-byte inner, every recursive frame holding a diagnostic grew, pushing past 2 MB on macOS.
This sets the rayon pool's `stack_size` to 8 MB to match Linux's default. Scope is limited to the dev coverage task — the perf win from #22406 is preserved, and no runtime consumers (parser, semantic, oxlint, oxfmt) are affected.
Verified `just c` runs end-to-end on macOS after the change.
This PR contains the following updates:
| Package | Update | Change |
|---|---|---|
| [rust](https://redirect.github.com/rust-lang/rust) | minor | `1.94.1` → `1.95.0` |
---
### Release Notes
<details>
<summary>rust-lang/rust (rust)</summary>
### [`v1.95.0`](https://redirect.github.com/rust-lang/rust/compare/1.94.1...1.95.0)
[Compare Source](https://redirect.github.com/rust-lang/rust/compare/1.94.1...1.95.0)
</details>
---
### Configuration
📅 **Schedule**: (in timezone Asia/Shanghai)
- Branch creation
- At any time (no schedule defined)
- Automerge
- At any time (no schedule defined)
🚦 **Automerge**: Enabled.
♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 **Ignore**: Close this PR and you won't be reminded about this update again.
---
- [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box
---
This PR was generated by [Mend Renovate](https://mend.io/renovate/). View the [repository job log](https://developer.mend.io/github/oxc-project/oxc).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0My4xMjMuOCIsInVwZGF0ZWRJblZlciI6IjQzLjEyMy44IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6W119-->
Tighten up parsing of TS snapshot files for ESTree conformance tests. Previously if the snapshot files were malformed and didn't match expected format, those tests would be silently skipped. Instead, panic if snapshot files are malformed.
Refactor of ESTree conformance tests.
These conformance tests check both AST and tokens for each category (Test262, Acorn JSX, TS). The functions that test AST and tokens for each category have a lot of shared logic.
Move that shared logic into a separate function for each category, so the main test functions for AST and tokens contain only the logic that's specific to them. This makes the tests less verbose and easier to understand.
ESTree conformance tests for tokens were doing unnecessary work. Converting all spans in the AST from UTF-8 to UTF-16 is unnecessary, since the AST is never looked at in these tests.
Pure refactor. Re-order testing functions in ESTree conformance tests so each pair of AST test and tokens test functions are together. Does not alter any code, just re-order it.
Introduce `ESTreeTokenConfig` trait and allow option for the "flavor" of tokens to be set at compile time.
When producing Espree-style tokens, this allows cutting out all the special-case logic related to `JSXIdentifier` tokens and removing 2 branches from `emit_identifier_at`.
Small effect on perf - only +0.3% on ESTree tokens benchmark.
Previously we used a 3-pass process for serializing tokens in ESTree format.
* Pass 1: Walk AST and produce a list of "overrides" - which tokens need their `Kind` changed.
* Pass 2: Loop through all tokens, apply any overrides, convert to `ESTreeToken`s, and save them in a `Vec`.
* Pass 3: Serialize the `Vec<ESTreeToken>` with `ESTree` trait (JSON serializer).
Compress this all to a single pass:
* Create an `ESTreeSequenceSerializer` which can have items sent to it to serialize one-by-one (streaming).
* Walk AST.
* When encounter a token which needs its `Kind` altered:
* Flush all previous tokens, serializing each in turn, using its default `Kind`.
* Serialize the current token with the amended `Kind`.
* Continue walking.
* At the end, flush any remaining tokens to the serializer.
This avoids multiple passes, and also removes the `Vec<ESTreeToken>` which was just intermediate data, and grew very large.
+18% perf improvement on ESTree tokens benchmark.
Refactor.
Previously we converted spans from UTF-8 to UTF-16 in AST first, then serialized tokens. This meant that when serializing tokens, we had to convert spans to UTF-16 before comparing offsets of overrides to the token's start offset (because the override offsets are taken from the AST nodes during the AST walk pass).
Instead, convert token spans first, *then* the AST. This removes span conversion from the critical path. On a CPU with instruction-level parallelism, other work can start without being blocked on waiting for the result of span conversion, and the two can happen in parallel.
This should be a slight perf improvement, though it likely won't show up on CodSpeed benchmarks as CodSpeed's CPU simulation to basic to emulate instruction-level parallelism.
But mainly, the motivation is that this makes the process simpler and easier to reason about.
Oxc's AST on JS side has a non-standard `hashbang` property. Previously in ESTree conformance tests, we skipped files with a hashbang.
Bump `estree-conformance` submodule to include https://github.com/oxc-project/estree-conformance/pull/183 which adds `hashbang` property to AST in snapshots.
We can now enable the tests which we previously skipped.
Perf optimization to converting tokens to ESTree JSON. Previously we serialized to pretty-printed JSON. Instead use compact JSON - it's just passed to `JSON.parse` on JS side, so there's no need for whitespace.
Fix a bug where we didn't produce any tokens for files which start with a BOM.
As a side effect, re-use the existing `Utf8ToUtf16` span converter created for converting spans in the AST again for tokens, rather than creating another one.
Refactor. We had pointlessly complex logic where caller would call `collect_token_context` and then pass the result back into some other tokens code `to_estree_tokens_json`. Combine the 2 functions to make the call sites simpler.
Pure refactor of conformance tests for ESTree and ESTree tokens. Use the same order of statements, and same variable names in each test function, to make it easier to follow.
We were skipping some ESTree tokens conformance tests for files containing hashbangs. We have to skip these tests for ESTree AST tests due to our non-standard `hashbang` field on `Program`, but we don't have to skip them for tokens tests - hashbangs are handled correctly.
Fix tokens when parsing JSX (not TSX) so they match `acorn-jsx` test cases. TS-ESLint's tokens are clearly wrong. We continue to match TS-ESLint bug-for-bug, but process plain JSX differently, so both the TS and JSX tests pass.
The only 2 remaining failing JSX test cases are due to #9667. Both these cases are also failing in the ESTree AST conformance tests.
### What this PR does
Introduce `ParserConfig` trait (another try at #16785).
The aim is to remove the large performance regression in parser that #19497 created, by making whether the parser generates tokens or not a compile-time option.
`ParserConfig::tokens` method replaces `ParseOptions::collect_tokens` property. The former can be const-folded at compile time, where the latter couldn't.
### 3 options
This PR also introduces 3 different config types that users can pass to the parser:
* `NoTokensParserConfig` (default)
* `TokensParserConfig`
* `RuntimeParserConfig`
The first 2 set whether tokens are collected or not at compile time. The last sets it at runtime.
All 3 implement `ParserConfig`.
`NoTokensParserConfig` is the default, and is what's used in compiler pipeline. It switches tokens off in the parser, and makes all the tokens-related code dead code, which the compiler eliminates. This makes the ability of the parser to generate tokens zero cost when it's not used (in the compiler pipeline).
`TokensParserConfig` is the one to use where you *always* want tokens. This is probably the config that linter will use.
`RuntimeParserConfig` is the one to use when an application decides whether to generate tokens or not at runtime. This config avoids compiling the parser twice, at the cost of runtime checks. This is what NAPI parser package will use.
### Future extension
#### Supporting additional features
In future we intend to build the UTF-8 to UTF-16 offsets conversion table in the parser. This will be more performant than searching through the source text for unicode characters in a 2nd pass later on. But this feature is only required for uses of the parser where we're interacting with JS side (NAPI parser package, linter with JS plugins).
`ParserConfig` can be extended to toggle this feature on/off at compile time or runtime, in the same way as you toggle on/off tokens.
#### Options and configs
This PR introduces `ParserConfig` but leaves `ParseOptions` as it is. So we now have 2 sets of options, passed to `Parser` with `with_options(...)` and `with_config(...)`. This is confusing.
We could merge the 2 by making `ParseOptions` implement `ParserConfig`, so then you'd define all options with one `with_options` call.
This would have the side effect of making all other parser options (e.g. `preserve_parens`) able to be set at either runtime or compile time, depending on the use case.
For users consuming `oxc_parser` as a library, with specific needs, they could also configure `Parser` to their needs e.g. create a parser which only handles plain JS code with all the code paths for JSX and TS shaken out as dead code. This would likely improve parsing speed significantly for these use cases.
### Implementation details
Why a trait instead of a cargo feature?
IMO a trait is preferable for the following reasons:
1. We have 3 different use cases we need to support (the 3 provided configs). 3 different Cargo features would be unwieldy.
2. This situation would become far worse once we introduce more features e.g. UTF-8 -> UTF-16 conversion.
3. No problems around feature unification. We found Cargo features caused headaches when we used them in linter for toggling on/off JS plugins support.
4. No clippy errors which only appear when the feature is/isn't disabled, requiring complex `#[cfg_attr(feature = "whatever", expect(clippy::unused_async))]` etc.
The introduction of a trait does not seem to significantly affect compile time:
```
# Before this PR
cargo build -p oxc_parser 15.33s user 3.27s system 254% cpu 7.316 total
cargo build -p oxc_parser --release 17.36s user 2.26s system 231% cpu 8.477 total
cargo build -p oxc_parser --example parser 18.43s user 3.75s system 271% cpu 8.156 total
cargo build -p oxc_parser --example parser --release 32.52s user 2.59s system 180% cpu 19.454 total
# After this PR
cargo build -p oxc_parser 15.00s user 3.24s system 272% cpu 6.692 total
cargo build -p oxc_parser --release 16.71s user 2.12s system 287% cpu 6.539 total
cargo build -p oxc_parser --example parser 18.50s user 3.91s system 285% cpu 7.845 total
cargo build -p oxc_parser --example parser --release 33.48s user 2.63s system 169% cpu 21.263 total
```
Measured on Mac Mini M4 Pro, `cargo clean` run before each. The difference appears to be mostly within the noise threshold.
overlookmotel