## Summary
- Groups `Class::super_class` and `Class::super_type_arguments` into `Class::heritage`.
- Introduces `ClassHeritage` containing the superclass expression and its optional TypeScript type arguments.
- Makes superclass type arguments without a superclass expression unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub super_class: Option<Expression<'a>>,
#[ts]
pub super_type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
```
to:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub heritage: Option<ClassHeritage<'a>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
pub struct ClassHeritage<'a> {
pub expression: Expression<'a>,
#[ts]
pub type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
}
```
## Why?
ECMAScript defines the complete `extends` clause as `ClassHeritage`:
```text
ClassHeritage : extends LeftHandSideExpression
```
TypeScript additionally permits type arguments on the heritage expression:
```ts
class Derived extends Base<T> {}
```
Previously, the AST stored the expression and type arguments independently:
```text
Class {
super_class: Some(Base),
super_type_arguments: Some(<T>),
...
}
```
This allowed an invalid state to be constructed:
```text
Class {
super_class: None,
super_type_arguments: Some(<T>),
...
}
```
Superclass type arguments cannot exist without an `extends` expression.
The parser already treated these fields as a pair. It parsed a class heritage entry and then split its expression and type arguments into two independent fields. Builders and transforms therefore had to preserve their relationship manually.
Grouping them into `ClassHeritage` makes the syntactic dependency explicit and prevents consumers from observing or constructing contradictory class heritage.
## Class heritage forms
A class without an `extends` clause has no heritage:
```js
class Base {}
```
```text
Class {
heritage: None,
...
}
```
A class extending an expression has heritage without type arguments:
```js
class Derived extends Base {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: None,
}),
...
}
```
TypeScript type arguments belong to the same heritage structure:
```ts
class Derived extends Base<T> {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: Some(<T>),
}),
...
}
```
The heritage expression may be any valid class heritage expression:
```js
class Derived extends mixin(Base) {}
class NullDerived extends null {}
```
## How to migrate
Code that previously accessed the superclass expression directly:
```rust
if let Some(super_class) = &class.super_class {
// ...
}
```
can use the compatibility accessor:
```rust
if let Some(super_class) = class.super_class() {
// ...
}
```
Code that needs both parts should access the grouped heritage:
```rust
if let Some(heritage) = &class.heritage {
let expression = &heritage.expression;
let type_arguments = heritage.type_arguments.as_deref();
}
```
Mutating consumers should update the nested fields:
```rust
if let Some(heritage) = &mut class.heritage {
heritage.type_arguments = None;
}
```
Common migrations:
| Previous representation | New representation |
| --- | --- |
| `class.super_class` | `class.heritage.as_ref().map(\|h\| &h.expression)` or `class.super_class()` |
| `class.super_type_arguments` | `class.heritage.as_ref()?.type_arguments` or `class.super_type_arguments()` |
| `class.super_class.is_some()` | `class.heritage.is_some()` |
| Mutate `class.super_class` | Mutate `class.heritage.expression` |
| Clear `class.super_type_arguments` | Clear `class.heritage.type_arguments` |
| Pass superclass and type arguments separately to a `Class` builder | Construct and pass `ClassHeritage` |
| Clone both fields independently | Clone `class.heritage` |
```ts
superClass: Expression | null;
superTypeArguments?: TSTypeParameterInstantiation | null;
```
No `heritage` property or `ClassHeritage` node is added to the ESTree output.
fixes https://github.com/oxc-project/backlog/issues/216
## Summary
Narrow `TSInterfaceHeritage` from a general JavaScript `Expression` to `TSTypeName`.
```rust
pub struct TSInterfaceHeritage<'a> {
pub type_name: TSTypeName<'a>,
pub type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
}
```
## Breaking Change
The Rust AST field changes from:
```rust
expression: Expression<'a>
```
to:
```rust
type_name: TSTypeName<'a>
```
## Why?
The current AST can represent shapes such as:
```ts
interface A extends foo() {}
interface A extends A + B {}
interface A extends new Foo() {}
interface A extends true {}
```
Which is an overly wide type since all of the above variants are invalid.
This PR tightens up the AST, making these invalid variants irrepresentable.
## How to migrate?
This should be a trivial migration:
1. instead of accessing `expression` on `TSInterfaceHeritage`, now access `type_name`
2. change any pattern matching on the `type_name` field to use `TypeName` (this is a smaller subset so should allow deleting code!)
closes https://github.com/oxc-project/backlog/issues/215
## Summary
- Introduces `ArrowFunctionBody` to represent concise expression and block bodies explicitly.
- Removes the `ArrowFunctionExpression::expression` boolean.
- Makes invalid arrow body combinations unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct ArrowFunctionExpression<'a> {
pub expression: bool,
pub body: Box<'a, FunctionBody<'a>>,
}
```
to:
```rust
pub enum ArrowFunctionBody<'a> {
FunctionBody(Box<'a, FunctionBody<'a>>) = 64,
INHERIT(Expression<'a>),
}
pub struct ArrowFunctionExpression<'a> {
pub body: ArrowFunctionBody<'a>,
}
```
Concise arrow bodies are now stored directly as expressions instead of synthetic `FunctionBody` and `ExpressionStatement` nodes. Block bodies continue to use `FunctionBody`.
## Why?
The previous representation required consumers to maintain this implicit invariant:
```text
expression == true
→ no directives
→ exactly one statement
→ that statement is an ExpressionStatement
```
Invalid combinations could cause panics, skipped work, or fallback behavior. The enum encodes the grammar directly and makes those invalid states unrepresentable.
Concise expressions also now have `ArrowFunctionExpression` as their direct traversal ancestor, without synthetic `FunctionBody` or `ExpressionStatement` nodes.
## How to migrate
Use the arrow body accessors or match on `ArrowFunctionBody` rather than inspecting `expression` and synthetic statements.
Common field migrations:
| Previous representation | New representation |
| --- | --- |
| `arrow.expression` | `arrow.is_expression()` |
| `arrow.get_expression()` | `arrow.get_expression()` |
| `&arrow.body.statements` | `arrow.get_function_body().map(\|body\| &body.statements)` |
| `&mut arrow.body.statements` | `arrow.get_function_body_mut().map(\|body\| &mut body.statements)` |
| Synthetic expression statement in `arrow.body` | Direct `ArrowFunctionBody` expression variant |
| `arrow.body = function_body` | `arrow.body = ArrowFunctionBody::FunctionBody(function_body)` |
| Expression-to-block conversion | Create a `FunctionBody` containing a return statement |
| Block-to-expression conversion | Assign the expression directly to `arrow.body` |
| Builder `(expression, body)` arguments | Pass a single `ArrowFunctionBody` |
closes https://github.com/oxc-project/backlog/issues/211
Follow-on after #24557. Handle when `ImportMeta` / `NewTarget` has an empty span (`SPAN`). This code path is not exercised at present, but it seems a good idea to be prepared for when we may serialize AST after transformation.
## Summary
- replace the generic Rust MetaProperty AST node with dedicated ImportMeta and NewTarget nodes
- preserve the ESTree MetaProperty shape and generated public TypeScript types
- update parser, formatter, codegen, lint, transform, minifier, traversal, and React compiler consumers
## Summary
Remove `MetaProperty` AST node, in favour of two separate nodes (`ImportMeta` and `NewTarget`)
## Breaking Change
The Rust AST field changes from:
```rust
pub struct MetaProperty<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub meta: IdentifierName<'a>,
pub property: IdentifierName<'a>,
}
```
to:
```rust
/// /// `import.meta` in `console.log(import.meta);`.
pub struct ImportMeta {
pub node_id: Cell<NodeId>,
pub span: Span,
}
/// `new.target` in `function F() { return new.target; }`.
pub struct NewTarget {
pub node_id: Cell<NodeId>,
pub span: Span,
}
```
## Why?
The current AST could represent shapes such as:
```
foo.bar
```
By constructing the node manually. However this doesn't make sense, and is invalid.
## How to migrate?
1. Replace any `Expression::MetaProperty` matches/checks with the dedicated `Expression::ImportMeta` and `Expression::NewTarget` varients
2. Replace any `new_meta_property` construction with the new `new_import_meta` or `new_new_target` APIs.
3. Inspecting the `meta`/`property` identifier names is no longer required.
Oxlint's raw transfer implementation stores the source text at end of the buffer. Prior to this PR `napi/parser` stored source text at the start of the buffer.
Bring `napi/parser` into line with Oxlint, by also storing source text at end of the buffer.
Making all versions of raw transfer that we use align with each other removes complication (all versions now have the same implementation of `deserializeStr`) and also removes some fragile code where it'd be easy to accidentally trigger UB (see the updated comments in `arena/fixed_size/windows.rs` and `arena/alloc_impl.rs`).
Unfortunately, storing source at end of buffer is slightly less efficient than storing it at the start. We'll switch all implementations to storing source at start of buffer when we move to storing all strings (source text and all other strings) together in a single contiguous block. But in meantime, this slight inefficiency is outweighed by the gain in safety.
Large change to how raw transfer stores metadata about the allocations it uses.
Previously the two metadata structures `RawTransferMetadata` and `FixedSizeAllocator` after the chunk's `ChunkFooter`.
This had a few problems:
1. It was confusing and unwieldy - a recipe for bugs.
2. `ChunkFooter`'s memory was exposed on JS side - a bit unsafe as altering bytes in this region could easily trigger UB.
3. It entwined `Arena` (which is just the thing that allocates) with the details of exactly _what_ raw transfer allocates.
4. Imposed annoying alignment requirements, because `ChunkFooter` must be aligned on 16, and so anything after it must ensure it doesn't break that invariant.
Old layout:
```
WHOLE BLOCK - aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ALLOCATOR
<-----------------------------------------> `Allocator` chunk (`CHUNK_SIZE` bytes)
<----> `ChunkFooter` (aligned on 16)
<-----------------------------------> `Allocator` chunk data storage (for AST)
(`ACTIVE_SIZE` bytes)
METADATA
<----> `RawTransferMetadata`
<----> `FixedSizeAllocatorMetadata`
BUFFER SENT TO JS
<-----------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
This PR moves `RawTransferMetadata` and `FixedSizeAllocatorMetadata` into the chunk itself. New layout:
```
WHOLE BLOCK - size 2 GiB - 16, aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ARENA
<-----------------------------------------------------> Chunk (fills whole block)
<--------------------------------------> Allocatable region for AST (`ACTIVE_SIZE` bytes)
<---> `RawTransferMetadata`
<---> `FixedSizeAllocatorMetadata`
<---> `ChunkFooter` (aligned on 16, last in block)
BUFFER SENT TO JS
<-------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
`FixedSizeAllocatorMetadata` and `ChunkFooter` are no longer in the region which is shared with JS side. As far as `Arena` is concerned, they're now just some data (like any other data) which is allocated in the arena.
Also:
- Introduce more consistency to the naming of constants which specify the size and position of these various data structures in the arena.
- Add more const assertions to ensure everything is laid out and aligned as it should be.
#21653 revealed that `TSTypePredicateName` has an unboxed variant `This`.
Before the introduction of `NodeId` fields, this was fine as `TSThisType` contained only a `Span` (8 bytes). But with `NodeId` field added, it's 16 bytes, which makes `TSTypePredicateName` larger too - 24 bytes.
Box `TSThisType` in this variant, bringing `TSTypePredicateName` back down to 16 bytes.
This also has the side-effect of making `TSTypePredicateName`'s `node_id`, `span`, and `span_mut` methods branchless.
#21653 revealed that `JSXExpression` has an unboxed variant `EmptyExpression`.
Before the introduction of `NodeId` fields, this was fine as `JSXEmptyExpression` contained only a `Span` (8 bytes). But with `NodeId` field added, it's 16 bytes, which makes `JSXExpression` larger too - 24 bytes.
Box `JSXEmptyExpression` in this variant, bringing `JSXExpression` back down to 16 bytes.
This also has the side-effect of making `JSXExpression`'s `node_id`, `span`, and `span_mut` methods branchless.
#21653 revealed that `ArrayExpressionElement` has an unboxed variant `Elision`.
Before the introduction of `NodeId` fields, this was fine as `Elison` contained only a `Span` (8 bytes). But with `NodeId` field added, it's 16 bytes, which makes `ArrayExpressionElement` larger too - 24 bytes.
Box `Elison` in this variant, bringing `ArrayExpressionElement` back down to 16 bytes.
This also has the side-effect of making `ArrayExpressionElement`'s `node_id`, `span`, and `span_mut` methods branchless.
Small perf optimization to raw transfer. When calculating a `u32` offset, shift first then add (`(pos >> 2) + 4`) instead of adding first (`(pos + 16) >> 2`). If a function contains multiple `pos >> 2` calculations, V8 may be able to combine them into a single calculation. Previously it couldn't prove that `pos` is a multiple of 4, so couldn't make this optimization.
Continuation of #21132. Get span `start` and `end` from the `Int32Array` view of the buffer, instead of the `Uint32Array`view. This allows V8 to statically see that they are SMIs, and skip type checks.
Utilize the `#[estree(raw_deser_inline)]` attribute introduced in #21134 to mark all custom deserializers which do not depend on `parent` as inline-able.
This reduces deserializer code size, as well as being more performant in some cases, by avoiding write boundary checks.
Use `int32` (`Int32Array`) instead of `uint32`(`Uint32Array`) for getting data from buffer in raw transfer deserializer. The buffer is 2 GiB in size, so all offsets within the buffer can be represented as a `u31` which can be stored in an `i32` without any loss, and with no risk of being interpreted as negative numbers.
Same as in #21129, the advantage of fetching offsets from an `Int32Array` is that V8 statically knows the value can be stored in an SMI, its native integer type, without any range checks.
Refacfor. In a strange pattern, we were injecting `getNodeLoc` into `deserialize.js` module on every call to `deserializeProgramOnly`. It's a static function, so just import it.
There's no need to defer clearing vars containing buffers and source text in raw transfer deserializer module until end of linting. It was a left-over from when this module contained code which lazily deserialized comments, but that's now done elsewhere.
Clear these vars as soon as deserialization is complete. Buffer and `sourceText` are still held in vars in other modules until linting completes, but `sourceTextLatin` (introduced in #21021) can be freed immediately.
Optimize raw transfer string decoding.
Replace `TextDecoder("utf8")` with `Buffer.prototype.utf8Slice` to decode Rust UTF-8 strings to JS strings.
Benchmarks show this is an average 5% speed-up in `deserializeStr`, though the reason why is unclear. It skips various checks, and avoids creating a temporary `Uint8Array` (which is required with `TextDecoder.decode`) which trims 60 bytes of the size of the assembly created for `deserializeStr`. But that's not enough to explain the speed-up 🤷. Anyway, take the win.
Improve perf of deserializing strings in raw transfer. This PR combines several optimizations, which have been tested and benchmarked in https://github.com/overlookmotel/oxc-raw-str-bench. This PR implements the version "latin-slice-onebyte64" from that repo, which is the current winner.
String deserialization is the main bottleneck in raw transfer, so speeding it up will likely make a large impact on deserialization overall.
This work follows on from #20834 which produced a major speed-up in many files by making files which contain some non-ASCII characters take the fast path of slicing `sourceText` more often.
This PR tackles the remainder - speeding up the fallback path where the fast path can't be taken.
## Optimizations
The optimizations in this PR are:
### Latin1
When source is not 100% ASCII, decode source text from buffer as Latin1.
A Latin1-decoded string represents each UTF-8 byte as a single Latin1 character, so it can be indexed into using UTF-8 offsets.
So when we can't slice the string from `sourceText` because the UTF-8 and UTF-16 offsets differ (after any non-ASCII character), loop through the string's bytes and check if they're all ASCII. If they are, the string can be sliced from `sourceTextLatin` instead, with the original UTF-8 offsets.
This is way faster than calling `textDecoder.decode`, as it avoids a call into C++. [Benchmarks show](https://github.com/overlookmotel/oxc-raw-str-bench/blob/4f96275efa9a35d5d27615abb27f21a137149cc0/README.md#apply30-vs-latin-vs-latin-source64) speed up of 55% on average, and up to 70% on some files.
### Latin1 decoding method
It turns out that `new TextDecoder("latin1").decode(arr)` doesn't actually decode to Latin1!
Per the WHATWG Encoding Standard, "latin1" is mapped to "windows-1252".
The result is that with `TextDecoder("latin1")`:
1. `decode` is quite complicated, requiring a 2-pass scan of the bytes to determine if they're all ASCII, followed by a 2nd pass to do the actual `windows-1252` decoding. If the string *does* contain any non-ASCII characters (which it always does in our usecase), NodeJS implements the decoding in JS, not native code. Slow.
2. `decode` produces a 2-byte-per-char string (`TWO_BYTE` in V8), which takes more memory, and is slower for all operations on it e.g. string comparison, hashing for use as an object key etc.
Instead, use `Buffer.prototype.latin1Slice` which:
1. Does a pure Latin1 decode, which is just a single `memcpy` call.
2. Produces a 1-byte-per-char string (`ONE_BYTE` in V8).
`latin1Slice` involves a call into C++, but we only do it once per file, so this cost is tiny in context of deserializing the whole AST.
### Latin1 string slicing
In the fast path, slice from the Latin1-decoded string, instead of `sourceText`. In the fast path, we know that all bytes of source comprising the string are ASCII, so no further checks are required.
This makes no difference on benchmarks for `deserializeStr` itself, but it may have beneficial effects downstream for code (e.g. lint rules) which access strings in the AST, e.g. `Identifier` names.
Because Latin1-decoded source text is `ONE_BYTE`-encoded, slices of it are too. In comparison, slices of `sourceText` may be `ONE_BYTE` or `TWO_BYTE`. If a file's source is pure ASCII, it'll be `ONE_BYTE`, if source contains any non-ASCII characters, it'll be `TWO_BYTE`. Files in a repo will likely be a mix of both, which makes strings returned from `deserializeStr` and placed in the AST a mix too. This in turn makes functions (e.g. lint rule visitors) polymorphic. V8 cannot optimize them as aggressively as if they see only `ONE_BYTE` strings.
We cannot make sure that all strings returned by `deserializeStr` are `ONE_BYTE`. Some string may contain non-ASCII characters, and they *have* to be represented in `TWO_BYTE` form. But we can minimize it - now only strings which *themselves* contain non-ASCII characters are `TWO_BYTE`, whereas before they would be if the source text as a whole contains a single non-ASCII byte.
Code which accesses `Identifier` names, for example will exclusively see `ONE_BYTE` strings and will be more heavily optimized, because Unicode `Identifier`s are rarer than hen's teeth in real-world code.
### Remove string-concatenation loop
Previously strings which are outside of source text were assembled byte-by-byte in a loop via concatenation.
Instead, check that all the bytes are ASCII first, copy them into an array and pass that array to `String.fromCharCode` with `fromCharCode.apply(null, array)`.
To avoid allocating a fresh array every time, hold a stock of arrays for all string lengths that this path can require, and reuse them.
This is a variation on the approach that #20883 took, but without the massive switch. This produces much tighter assembly, and avoids regressing the fast path due making `deserializeStr` a very large function.
Despite the complexity, and multiple operations, [this is up to 3x faster](https://github.com/overlookmotel/oxc-raw-str-bench/blob/4f96275efa9a35d5d27615abb27f21a137149cc0/README.md#apply30-vs-switch30) than the switch approach, and gives an average 30% speed-up.
### Increase native call threshold
The above optimizations make the slow path much faster. This shifts the tipping point at which it's faster to make a native call to `TextDecoder.decode` from 9 bytes to 64 bytes. Most strings now avoid the native call and stay in JS code which is heavily optimized by Turbofan.
The tipping point of 64 is something of a guesstimate. Benchmarking shows its in the right ballpark, but we could finesse it, and probably squeeze out another couple of %.
## Credit
The Latin1 string technique was cooked up by @joshuaisaact in https://github.com/overlookmotel/oxc-raw-str-bench/pull/1. All credit to him for this masterstroke which cracks the whole problem!
Tiny optimization to raw transfer string deserialization. Initialize vars which contain integers as `0`. This may help V8 optimize usage of these variables as they always have SMI type.
The effect, if there is one, is too small to register on benchamarks. But it's certainly not a regression, and may be a tiny gain, so why not?
Follow-on after #20834.
Simplify the branch condition in `deserializeStr` for detemining if can take the fast path of just slicing `sourceText`. There's no need to check `sourceIsAscii`, just compare the offset to `firstNonAsciiPos` (the position in buffer of first non-ASCII byte in source code). When source is 100% ASCII, `firstNonAsciiPos = sourceEndPos`, so `pos < firstNonAsciiPos` passes for all positions in source.
The implementation is different for parser and for Oxlint, as the source text sits in a different location in buffer - at the start in parser, at the end in Oxlint - but the principle is the same in both.
[Benchmarking](https://github.com/overlookmotel/oxc-raw-str-bench) showed this speeds up `deserializeStr` by a small percentage.
Small optimization to raw transfer string deserialization. Use `String.fromCharCode`instead of `String.fromCodePoint`. [Benchmarking](https://github.com/overlookmotel/oxc-raw-str-bench) showed it's slightly faster, as it's simpler - doesn't need to handle astral code points.
`raw` field of `NumericLiteral`, `StringLiteral`, `BigIntLiteral`, `RegExpLiteral`, and `JSXText` are slices of source text.
So in raw transfer deserializer, skip going through `deserializeStr`, which can be slow when source contains any non-ASCII characters. Instead just slice the string from the source text directly with `sourceText.slice(start, end)`.
String decoding is the slowest part of raw transfer, so this should be a significant speed gain.
**AI Disclosure:** Developed with Claude Code (Opus). The winning
approach came out of an automated experiment loop
([auto-claude](https://github.com/joshuaisaact/auto-claude)) — I was
looking for a tight feedback loop to test the tool on and the `TODO:
Find best switch-over point` comment in `deserializeStr` caught my eye.
20 experiments, keep-or-revert on each (all 20 summarized in an
expandable section at the bottom). All code reviewed and understood.
Happy to close this if it's not useful or doesn't meet the bar.
## Why
I was profiling the raw transfer deserialization path (`node --prof` on
`checker.ts`) and noticed `StringAdd_CheckNone` at 13.7% of time — the
single hottest function. It comes from the byte-by-byte `out +=
fromCodePoint(c)` loop in `deserializeStr` when `sourceIsAscii` is
false.
The thing is, `sourceIsAscii` is false for almost everything. All 5 NAPI
bench fixtures are non-ASCII. `checker.ts` has literally one Bengali
character at position 2.1M out of 2.9M. That one character disables the
fast `substr` path for all ~148K strings.
## What
Two changes to the generator
(`tasks/ast_tools/src/generators/raw_transfer.rs`) — that's the only
file with real changes. The 9 generated JS files in the diff are the
mechanical output of `cargo run -p oxc_ast_tools`.
**1. `firstNonAsciiPos` scan at init** — On non-ASCII sources, find the
first non-ASCII byte once upfront. Strings ending before that position
can still use `sourceText.substr()` since byte offsets equal char
offsets in the ASCII prefix. For `checker.ts` this covers 73% of the
file, for `pdf.mjs` 98%.
**2. Lower TextDecoder threshold from 50 to 9** — The existing TODO
asked for the right switch-over point. Experimentally, 9 is the sweet
spot: `TextDecoder` beats the `fromCodePoint` concat loop for strings of
10+ bytes, and the concat loop is still faster for very short strings
where the native call overhead dominates.
Benchmarked on the `complicated()` test set (5 rounds of 30 iters,
dropping round 1 for JIT warmup):
```
Before After
checker.ts 26.7ms 13.0ms -51%
cal.com.tsx 15.4ms 9.1ms -41%
antd.js 53.8ms 44.7ms -17%
pdf.mjs 3.8ms 4.3ms noise
binder.ts 0.7ms 0.5ms noise
Total 100.5ms 71.6ms -29%
```
Also verified across 15 files — the 10 non-ASCII files above plus 5
ASCII files (`react.development.js`, `binder.ts`, `moment.js`,
`jquery.js`, `vue.js`). ASCII files are unchanged (our code only touches
the non-ASCII path). -16% across non-ASCII files, no regressions.
## References
- Addresses the `TODO: Find best switch-over point` in
`STR_DESERIALIZER_BODY`
- Related to perf goals in #19918
<details>
<summary>I appreciate this is already a bloated PR description (sorry
@overlookmotel) but given the fairly unusual approach I thought you
might want to see a very short summary of all 20 experiments Claude
clauded through:</summary>
The loop works like this: edit code, benchmark, keep if faster, `git
reset --hard` if not. Metric is total deserialization time across the
benchmark corpus.
**Baseline: 97.7ms** (5-file corpus, all non-ASCII)
| # | Idea | Result | Verdict |
|---|------|--------|---------|
| 1 | Always use TextDecoder, delete the loop entirely | 99.2ms |
Revert. TextDecoder's ~78ns fixed overhead kills short strings. |
| 2 | Lower TextDecoder threshold from 50 to 10 (ts.js only) | 89.8ms |
**Keep.** First real win — moves 50% of strings off the concat loop. |
| 3 | Threshold 5 | 91.3ms | Revert. Too aggressive, too many short
strings go to TextDecoder. |
| 4 | Threshold 8 | 95.0ms | Revert. Worse than 10. |
| 5 | Threshold 12 | 91.2ms | Revert. Worse than 10. |
| 6 | Threshold 10 + unrolled `switch` on len for inline
`String.fromCharCode(uint8[pos], ...)` | 90.9ms | Revert. Switch
dispatch overhead eats the gain. |
| 7 | Threshold 10 + special fast path for len=1 | 94.8ms | Revert.
Extra branch hurts more than the 1-byte optimization helps. |
| 8 | Threshold 10 + accumulate char codes in array, single
`fromCharCode.apply` at end | 90.5ms | Revert. Array allocation
overhead. |
| 9 | Threshold 15 | 93.2ms | Revert. 10 is still the sweet spot. |
| 10 | Unrolled ASCII check for bytes 1-4, TextDecoder for 5+ | 94.7ms |
Revert. Branching overhead. |
| 11 | **Apply threshold 10 to js.js too** (had only been changing
ts.js) | 82.5ms | **Keep.** Facepalm moment — antd.js uses the JS
deserializer. |
| 12 | Threshold 9 in both files | 77.9ms | **Keep.** New best. |
| 13 | Threshold 7 | 79.0ms | Revert. 9 wins. |
| 14 | Threshold 11 | 84.8ms | Revert. 9 confirmed. |
| 15 | Replace `fromCodePoint` with `String.fromCharCode` in the loop |
82.5ms | Revert. V8 optimizes the pre-extracted `fromCodePoint` better.
|
| 16 | Various unrolled `fromCharCode` approaches for short strings | —
| Abandoned, too complex for marginal gain. |
| 17 | Always TextDecoder for non-source strings (remove loop) | 96.1ms
| Revert. Confirms the short-string loop IS valuable for 1-9 bytes. |
| 18 | `Buffer.from().toString()` instead of TextDecoder | 80.4ms |
Revert. TextDecoder is faster. |
| 19 | `firstNonAsciiPos` only (use substr before it, TextDecoder after,
no loop) | 84.9ms | Revert. checker.ts loved it (-34%) but antd.js hated
it (+20%) because its first non-ASCII byte is at 1.3%. |
| 20 | **`firstNonAsciiPos` + threshold 9 + keep the loop** | 76.4ms |
**Keep.** Best of both worlds — substr where possible, TextDecoder for
medium strings, loop for short. |
Three things I (Claude) learned:
- Experiment 11 was the biggest single win and it was just... applying
the change to the other file. Embarrassing.
- Every attempt to replace the `fromCodePoint` loop for short strings
(1-9 bytes) made things worse. The loop is genuinely good for that
range.
- `firstNonAsciiPos` only works when combined with the threshold change.
On its own it hurts files where non-ASCII appears early (antd.js,
cal.com.tsx).
</details>
Alter the code in `ast_tools` which re-orders struct fields, to make `NodeId` always occupy a consistent "slot" in every AST node - at byte position 8, just after `Span`.
This will be a sizeable gain once we start utilizing the `NodeId` fields more. Because, for example, every variant of `Expression` has its `NodeId` stored in same location as all the other variants, `Expression::node_id()` is just 1 operation - a pointer read - rather than a nest of branches, or a lookup table.
Also alter the algorithm for ordering struct fields to fill in the 4-byte gap after `NodeId` field with other field(s), to avoid excess padding.
The new algorithm also prioritizes keeping fields in definition order as much as possible, rather than sorting them strictly in order of size and alignment. This is mildly advantageous because field definition order is the order the AST is walked in, so it avoids bouncing between cache lines while iterating through the fields of a struct when visiting the AST.
No types change size in the process. Fields remain packed to keep type sizes the minimum they can be - they just change order.
One annoyance with `ast_tools`, our codegen, has been that when you want it to generate code from types in some file, you had to manually add the file paths to a list in `ast_tools` itself.
This PR removes that list and instead finds crates in the monorepo which depend on `oxc_ast_macros` crate, via `cargo metadata`, and searches all files in those crates for types with `#[ast]` attributes etc - types that it need to act on.
There is no longer a list to keep updated, you just mark types `#[ast]` and the codegen will find them.
This produces some churn in generated files. None of the generated code changes, it just gets re-ordered in some places, due to the different order that files get found in now. This order is deterministic, and it should not change again.
Belated follow-on after #15925.
`VariableDeclarator` doesn't need a whole-struct ESTree converter. Switch to a converter just for the `id` field, to reduce custom conversion code.
Builds on #19497. Use the tokens generated by Oxc's parser in linter plugins, instead of running TS parser to tokenize source.
This is a sizeable perf gain, and also allows us to remove TS parser from `oxlint`'s bundle (#19531).
Additionally, our parser is more accurate than TypeScript - we handle HTML comments and space before slash in closing JSX elements (`< /div>`) - so this fixes a few conformance tests.
## Summary
- Add `pub node_id: NodeId` as the first field in all AST structs (~190 structs across js.rs, ts.rs, jsx.rs, literal.rs)
- Update codegen to special-case `node_id`:
- `Dummy`: Use `NodeId::DUMMY` instead of `Dummy::dummy(allocator)`
- `CloneIn`: Use `NodeId::DUMMY` for regular clone, preserve ID for semantic clone
- `ContentEq`: Skip `node_id` from equality comparisons
- `ESTree`: Skip `node_id` from serialization
- `AstBuilder`: Exclude `node_id` from builder parameters, use `NodeId::DUMMY`
- Update pattern matching in minifier and transformer to use `..` for new field
- Update traverse script to include `NodeId` import
This is the foundational step for assigning unique IDs to AST nodes. The `node_id` defaults to `NodeId::DUMMY` (value 0) for uninitialized nodes.
## Test plan
- [x] `cargo build` succeeds
- [x] `cargo test` passes
- [x] `just ast` regenerates code correctly
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Add `decorators` field to `FormalParameterRest` to support decorators on rest parameters in TypeScript
- Parse decorators before checking for rest parameters in formal parameter lists
- Update semantic builder to visit decorators on rest parameters
Fixes parsing of `typescript/tests/cases/compiler/sourceMapValidationDecorators.ts`
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Removes `ThisExpression` variant from `TSModuleReference` since `import x = this` is invalid TypeScript (error TS1359)
- Adds parser error when `this` is used in import equals declaration
- Updates codegen, transformer, and linter rules to handle the new enum structure
Closes#16841🤖 Generated with [Claude Code](https://claude.com/claude-code)
Previous raw transfer required that the source text start exactly at the start of the buffer. In linter, relax this restriction and handle when source text is stored elsewhere in the buffer.
This is necessary for stripping BOM from source (#18376), as then the source text used on JS side starts 3 bytes after the start of the buffer.
## Summary
- Use inline assignment pattern (`let x; { x: x = value }`) for `start` and `end` fields instead of declaring them in the preamble
- This allows minifiers to eliminate unused variables when RANGE is false
- Restores original file size for deserializers that don't use range fields
**Before (js.js with RANGE=false):**
```javascript
function deserializeIdentifierName(pos) {
let start = deserializeU32(pos),
end = deserializeU32(pos + 4);
return { type: "Identifier", name: deserializeStr(pos + 8), start, end };
}
```
**After (js.js with RANGE=false):**
```javascript
function deserializeIdentifierName(pos) {
return { type: "Identifier", name: deserializeStr(pos + 8), start: deserializeU32(pos), end: deserializeU32(pos + 4) };
}
```
Closes#14332
## Test plan
- [x] `just ast` regenerates deserializers correctly
- [x] `pnpm test --filter='@oxc-project/napi-parser'` passes (109 tests)
- [x] `cargo clippy -p oxc_ast_tools` passes
🤖 Generated with [Claude Code](https://claude.ai/code)
## Summary
- Replace `type_parameter: Box<'a, TSTypeParameter<'a>>` with two dedicated fields:
- `key: BindingIdentifier<'a>` - the parameter name (e.g., `P` in `[P in keyof T]`)
- `constraint: TSType<'a>` - the constraint type (e.g., `keyof T`)
- Improves type safety by preventing invalid AST states - `TSTypeParameter` contains `const`, `in`, `out`, and `default` fields that are syntactically invalid in mapped types
- Matches TSESTree structure more closely
Closes#15101
## Test plan
- [x] All parser, codegen, semantic, and linter tests pass
- [x] Conformance tests show no regressions
- [x] Semantic snapshots updated to reflect the new structure
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Add `ModuleKind::CommonJS` variant to support CommonJS module type in the parser
- Enable top-level `return` statements (CommonJS files are wrapped in a function)
- Enable top-level `new.target` (allowed in CommonJS function wrapper)
- Add proper errors for `import`/`export` statements in CommonJS
- Add proper errors for `import.meta` in CommonJS
The CommonJS source type can be set via:
- `SourceType::with_commonjs(true)` method
- Babel's `sourceType: "commonjs"` option
Closes#16200
## Test plan
- [x] Added test cases in `tasks/coverage/misc/pass/` for valid CommonJS features
- [x] Added test cases in `tasks/coverage/misc/fail/` for invalid CommonJS usage
- [x] Enabled Babel conformance tests for `sourceType: "commonjs"`
- [x] All existing tests pass
🤖 Generated with [Claude Code](https://claude.com/claude-code)
## Summary
- Move `scope_id: Cell<Option<ScopeId>>` from `TSEnumDeclaration` to `TSEnumBody`
- Since `TSEnumBody` was introduced as a dedicated type in PR #10284, the scope logically belongs on the body rather than on the parent declaration node
Closes#10331
## Test plan
- [x] Unit tests pass (`just test`)
- [x] Conformance tests pass (`just conformance`)
- [x] Updated semantic test snapshots reflect scope now on `TSEnumBody`
🤖 Generated with [Claude Code](https://claude.ai/code)
fixes#11489
Note, in quite a few places i changed the span we report on to look something like this, the only reason for this is to reduce the diff in some of the snapshots
```
decl.type_annotation.as_ref().map_or_else(
|| decl.id.span(),
|type_annotation| decl.id.span().merge(type_annotation.span()),
),
```
Boshen