mirror of
https://github.com/oxc-project/oxc.git
synced 2026-09-14 19:36:11 +08:00
codex/parser-grouped-expression-allocation
11 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Cameron Clark
|
7784ad92d5 | perf(parser): avoid allocation for single grouped expressions | ||
|
camc314
|
e4208164e1 |
perf(parser): optimize trailing comma tracking (#25376)
## Summary - Track trailing-comma spans only for array and object literals ending in a spread. - Consume the stored span while covering a rest assignment target. ## Rationale Avoids parser-state map work for ordinary trailing commas while retaining constant-time diagnostics for invalid rest-target trailing commas. |
||
|
camc314
|
a33788e13e |
feat(ast)!: group class heritage into ClassHeritage (#25360)
## Summary
- Groups `Class::super_class` and `Class::super_type_arguments` into `Class::heritage`.
- Introduces `ClassHeritage` containing the superclass expression and its optional TypeScript type arguments.
- Makes superclass type arguments without a superclass expression unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub super_class: Option<Expression<'a>>,
#[ts]
pub super_type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
```
to:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub heritage: Option<ClassHeritage<'a>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
pub struct ClassHeritage<'a> {
pub expression: Expression<'a>,
#[ts]
pub type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
}
```
## Why?
ECMAScript defines the complete `extends` clause as `ClassHeritage`:
```text
ClassHeritage : extends LeftHandSideExpression
```
TypeScript additionally permits type arguments on the heritage expression:
```ts
class Derived extends Base<T> {}
```
Previously, the AST stored the expression and type arguments independently:
```text
Class {
super_class: Some(Base),
super_type_arguments: Some(<T>),
...
}
```
This allowed an invalid state to be constructed:
```text
Class {
super_class: None,
super_type_arguments: Some(<T>),
...
}
```
Superclass type arguments cannot exist without an `extends` expression.
The parser already treated these fields as a pair. It parsed a class heritage entry and then split its expression and type arguments into two independent fields. Builders and transforms therefore had to preserve their relationship manually.
Grouping them into `ClassHeritage` makes the syntactic dependency explicit and prevents consumers from observing or constructing contradictory class heritage.
## Class heritage forms
A class without an `extends` clause has no heritage:
```js
class Base {}
```
```text
Class {
heritage: None,
...
}
```
A class extending an expression has heritage without type arguments:
```js
class Derived extends Base {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: None,
}),
...
}
```
TypeScript type arguments belong to the same heritage structure:
```ts
class Derived extends Base<T> {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: Some(<T>),
}),
...
}
```
The heritage expression may be any valid class heritage expression:
```js
class Derived extends mixin(Base) {}
class NullDerived extends null {}
```
## How to migrate
Code that previously accessed the superclass expression directly:
```rust
if let Some(super_class) = &class.super_class {
// ...
}
```
can use the compatibility accessor:
```rust
if let Some(super_class) = class.super_class() {
// ...
}
```
Code that needs both parts should access the grouped heritage:
```rust
if let Some(heritage) = &class.heritage {
let expression = &heritage.expression;
let type_arguments = heritage.type_arguments.as_deref();
}
```
Mutating consumers should update the nested fields:
```rust
if let Some(heritage) = &mut class.heritage {
heritage.type_arguments = None;
}
```
Common migrations:
| Previous representation | New representation |
| --- | --- |
| `class.super_class` | `class.heritage.as_ref().map(\|h\| &h.expression)` or `class.super_class()` |
| `class.super_type_arguments` | `class.heritage.as_ref()?.type_arguments` or `class.super_type_arguments()` |
| `class.super_class.is_some()` | `class.heritage.is_some()` |
| Mutate `class.super_class` | Mutate `class.heritage.expression` |
| Clear `class.super_type_arguments` | Clear `class.heritage.type_arguments` |
| Pass superclass and type arguments separately to a `Class` builder | Construct and pass `ClassHeritage` |
| Clone both fields independently | Clone `class.heritage` |
```ts
superClass: Expression | null;
superTypeArguments?: TSTypeParameterInstantiation | null;
```
No `heritage` property or `ClassHeritage` node is added to the ESTree output.
fixes https://github.com/oxc-project/backlog/issues/216
|
||
|
leaysgur
|
a56b7d059b |
refactor(tasks/allocations): do not print redundunt traling blank line (#25268)
Otherwise, our new YAML formatter trims it, then remains as diff. |
||
|
camc314
|
067da8c4e7 |
feat(ast)!: store single parameter in TSIndexSignature::parameter (#25154)
|
||
|
camc314
|
1bdedd11ca |
feat(ast)!: introduce ExportDeclaration, ExportFromDeclaration (#25095)
## Summary
Split the overloaded Rust `ExportNamedDeclaration` into syntax-specific AST nodes:
- `ExportDeclaration` for exported declarations
- `ExportNamedDeclaration` for local exports
- `ExportFromDeclaration` for re-exports
## Breaking Change
The Rust AST changes from:
```rust
pub struct ExportNamedDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub declaration: Option<Declaration<'a>>,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub source: Option<StringLiteral<'a>>,
pub export_kind: ImportOrExportKind,
pub with_clause: Option<Box<'a, WithClause<'a>>>,
}
pub enum ModuleDeclaration<'a> {
// ...
ExportNamedDeclaration(Box<'a, ExportNamedDeclaration<'a>>),
// ...
}
```
to:
```rust
pub struct ExportDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub declaration: Declaration<'a>,
}
pub struct ExportNamedDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub export_kind: ImportOrExportKind,
}
pub struct ExportFromDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub source: StringLiteral<'a>,
pub export_kind: ImportOrExportKind,
pub with_clause: Option<Box<'a, WithClause<'a>>>,
}
pub enum ModuleDeclaration<'a> {
// ...
ExportDeclaration(Box<'a, ExportDeclaration<'a>>),
ExportNamedDeclaration(Box<'a, ExportNamedDeclaration<'a>>),
ExportFromDeclaration(Box<'a, ExportFromDeclaration<'a>>),
// ...
}
```
## Why?
The previous node represented three distinct syntax forms through optional and mutually exclusive fields:
```text
export const value = 1; -> ExportDeclaration
export { value }; -> ExportNamedDeclaration
export { value } from "mod"; -> ExportFromDeclaration
```
This allowed invalid combinations such as declarations with specifiers or sources, and import attributes without a source. The new nodes make these states unrepresentable and remove repeated `declaration`/`source` branching from consumers.
## How to migrate
Match the syntax-specific `ModuleDeclaration`, `Statement`, or `AstKind` variant instead of inspecting optional fields. Generated builders, visitors, traversal ancestors, and formatter wrappers now expose corresponding APIs for all three nodes.
Common field migrations:
| Previous representation | New representation |
| --- | --- |
| `declaration: Some(declaration)` | `ExportDeclaration { declaration }` |
| `declaration.as_ref()` | `&export.declaration` |
| `source.is_none()` for local exports | Match `ExportNamedDeclaration` |
| `source: Some(source)` | `ExportFromDeclaration { source }` |
| `source.as_ref().unwrap()` | `&export.source` |
| `with_clause` | Only available on `ExportFromDeclaration` |
| Stored `export_kind` on exported declarations | `ExportDeclaration::export_kind()` |
| `ExportNamedDeclaration::boxed_plain_declaration(...)` | `ExportDeclaration::boxed(...)` |
| `ExportNamedDeclaration::boxed(..., None, specifiers, None, ...)` | `ExportNamedDeclaration::boxed(..., specifiers, ...)` |
| `ExportNamedDeclaration::boxed(..., None, specifiers, Some(source), ...)` | `ExportFromDeclaration::boxed(..., specifiers, source, ...)` |
closes https://github.com/oxc-project/backlog/issues/219
|
||
|
camc314
|
c917f204a7 |
feat(ast)!: introduce ArrowFunctionBody enum (#24987)
## Summary
- Introduces `ArrowFunctionBody` to represent concise expression and block bodies explicitly.
- Removes the `ArrowFunctionExpression::expression` boolean.
- Makes invalid arrow body combinations unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct ArrowFunctionExpression<'a> {
pub expression: bool,
pub body: Box<'a, FunctionBody<'a>>,
}
```
to:
```rust
pub enum ArrowFunctionBody<'a> {
FunctionBody(Box<'a, FunctionBody<'a>>) = 64,
INHERIT(Expression<'a>),
}
pub struct ArrowFunctionExpression<'a> {
pub body: ArrowFunctionBody<'a>,
}
```
Concise arrow bodies are now stored directly as expressions instead of synthetic `FunctionBody` and `ExpressionStatement` nodes. Block bodies continue to use `FunctionBody`.
## Why?
The previous representation required consumers to maintain this implicit invariant:
```text
expression == true
→ no directives
→ exactly one statement
→ that statement is an ExpressionStatement
```
Invalid combinations could cause panics, skipped work, or fallback behavior. The enum encodes the grammar directly and makes those invalid states unrepresentable.
Concise expressions also now have `ArrowFunctionExpression` as their direct traversal ancestor, without synthetic `FunctionBody` or `ExpressionStatement` nodes.
## How to migrate
Use the arrow body accessors or match on `ArrowFunctionBody` rather than inspecting `expression` and synthetic statements.
Common field migrations:
| Previous representation | New representation |
| --- | --- |
| `arrow.expression` | `arrow.is_expression()` |
| `arrow.get_expression()` | `arrow.get_expression()` |
| `&arrow.body.statements` | `arrow.get_function_body().map(\|body\| &body.statements)` |
| `&mut arrow.body.statements` | `arrow.get_function_body_mut().map(\|body\| &mut body.statements)` |
| Synthetic expression statement in `arrow.body` | Direct `ArrowFunctionBody` expression variant |
| `arrow.body = function_body` | `arrow.body = ArrowFunctionBody::FunctionBody(function_body)` |
| Expression-to-block conversion | Create a `FunctionBody` containing a return statement |
| Block-to-expression conversion | Assign the expression directly to `arrow.body` |
| Builder `(expression, body)` arguments | Pass a single `ArrowFunctionBody` |
closes https://github.com/oxc-project/backlog/issues/211
|
||
|
Boshen
|
884d9eb6eb |
perf(parser): pre-size cover-grammar assignment target buffers (#24683)
The array/object destructuring assignment-target cover conversion (`ArrayAssignmentTarget::cover` / `ObjectAssignmentTarget::cover`) built its target buffer from empty and let it grow via arena reallocs as targets were pushed. Each source element or property yields at most one target, so the final length is a known upper bound — reserve it up front with `with_capacity_in`. Removes a handful of arena reallocs on the tracked parser fixtures: - `checker.ts`: −2 - `pdf.mjs`: −17 - `kitchen-sink.tsx`: −7 No change to arena size or system allocations. |
||
|
Boshen
|
bcc9de077e |
perf(parser): defer diagnostic creation until parse exit (#24663)
> [!NOTE] > Draft. Builds on the merged diagnostics cleanup (#24661), and has been through two `/simplify` review passes. > > One design point a perf reviewer may ask about: `ParserCheckpoint` (copied on every `checkpoint`/`rewind`/`lookahead`) embeds `FatalError` **inline**, not boxed. Boxing it would shrink the checkpoint from 3 → 2 cache lines, but only by adding a heap allocation — counter to this PR's goal. The deferral already shrank the checkpoint substantially for free: `FatalError` previously held an inline `OxcDiagnostic` (~150 B), now a 72 B POD `ParserDiagnostic`. Parser diagnostics are now stored in a deferred (unmaterialized) form during parsing and materialized into `OxcDiagnostic` only once, at parse exit. ## Design A `parser_diagnostics!` macro in `diagnostics.rs` generates, from one entry per diagnostic: - an enum variant `ParserDiagnostic::Foo { .. }` holding only the POD arguments (`Span`, `&'a str`, `Copy` types); - a `#[cold] into_diagnostic()` match arm holding the eager construction (box + label strings + `format!`) — all the allocation/formatting work concentrated in one cold function; - an `#[inline]` constructor with the **same name and signature as before**, so the ~330 `diagnostics::foo(...)` call sites are unchanged. The parser/lexer error vecs and `FatalError` now carry `ParserDiagnostic<'a>`, so pushing an error, snapshotting the error set on `checkpoint()`, and discarding it on `rewind()` never allocate. Materialization happens at the two `parse()` / `parse_expression()` exit points. Foreign already-materialized diagnostics (from `oxc_regular_expression`) ride an `Eager(Box<OxcDiagnostic>)` escape hatch; the two module-record diagnostics that are built once at exit stay eager. ## Why This eliminates the eager boxing of diagnostics that are created and then thrown away during speculative parsing (arrow-function / type-vs-expression backtracking). On real-world files — all valid, so nearly every one of these allocations was a speculation-discarded diagnostic: | file | sys allocs before → after | |---|---| | checker.ts | 1818 → 23 | | antd.js | 3661 → 190 | | kitchen-sink.tsx | 2051 → 190 | | App.tsx | 360 → 23 | | binder.ts | 85 → 1 | Diagnostic output is byte-identical — parser conformance (Test262, Babel, TypeScript) snapshots are unchanged. 🤖 Generated with [Claude Code](https://claude.com/claude-code) |
||
|
Boshen
|
2338c13f60 |
feat(track-memory-allocations): track heap deallocs, alloc bytes, and peak growth (#24619)
Adds three heap-side metrics to the allocation snapshots, using the metric machinery from #24587: - `sys deallocs` (exact count) — system deallocations per stage; read against `sys allocs` it shows whether a stage churns or retains heap. - `sys alloc bytes` — cumulative bytes requested from the system allocator per stage (allocation sizes plus reallocation growth). - `sys peak growth` — high-water mark of live heap during a stage relative to the live bytes at stage start, via live-bytes accounting in the `GlobalAlloc` wrapper. Captures transient heap a stage needs even when it frees it before finishing. Arena chunk operations are excluded from all `sys` metrics at the source: `oxc_allocator` marks each chunk allocation/deallocation immediately before calling the system allocator (task-only `track_allocations` feature, compiled out everywhere else), and the task's tracking allocator consumes the marker and skips the call. This replaces the previous global chunk counter + per-stage subtraction: a subtraction can correct counts but not a high-water mark, and chunk sizes are quantized and platform-dependent (whether an arena needs one more chunk depends on byte totals that vary across architectures with type layout — the #22621 mechanism). The first CI run demonstrated this concretely: kitchen-sink's minifier peak differed deterministically by 720 kB between aarch64 and x86_64 because the mangler's temporary arena crossed a chunk-growth boundary only on x86_64; and antd's formatter "peak" was 762 MB of main-arena chunk-doubling history rather than formatter behavior (now 13.8 MB of actual heap use). Cross-platform tolerances: the cumulative byte metrics still drift with the number of heap hashbrown tables allocated (their control-group width is architecture-dependent), so they use a relative tolerance (1%, 4 kB floor) via the committed-value comparison from #24565; `arena size` keeps its tight 1024-byte tolerance. Sample values for checker.ts (2.9 MB source): parser needs only 2.4 kB of transient heap (fully arena-based), semantic peaks at 3.9 MB, the minifier is the pipeline's largest heap consumer at 10.2 MB, and the formatter requests 5.9 MB total while peaking at 3 MB (heavy alloc/free churn). 🤖 Implemented with [Claude Code](https://claude.com/claude-code). |
||
|
Boshen
|
b1165f9c63 |
fix(track-memory-allocations): make arena size snapshots platform-independent (#24565)
The `Allocations` CI job fails whenever the snapshots are regenerated on a different CPU architecture than CI (e.g. https://github.com/oxc-project/oxc/actions/runs/29420345285/job/87368903834): every `arena size` value differs by 32–128 bytes between aarch64 and x86_64. The cause is target-dependent type layout — hashbrown's table control groups are 16 bytes wide on x86_64 (SSE2) but 8 bytes on aarch64 (NEON) — so each live `HashMap` in the arena shifts `used_bytes()` slightly. This is the same root cause that #22621 handled for chunk counts; the `arena size` gauge added in #24553 re-exposed it. Allocation counts are unaffected and stay exact. Exact byte equality across architectures isn't achievable (the arena genuinely occupies a different number of bytes), so instead the snapshot writer tolerates layout noise: - Snapshots are now YAML (`allocs_*.yaml`) with the same layout as before: one value per line, file names at column 0 for git hunk headers, exact numbers as the diffed values, and human-readable sizes moved to trailing `#` comments. - When regenerating, `cargo allocs` parses the committed snapshot (with `saphyr`, already a workspace dependency used by `oxc_coverage`) and keeps the committed `arena size` when the measured value is within `ARENA_SIZE_TOLERANCE` (1024 bytes, ~8× the observed worst-case cross-arch drift). Differences beyond the tolerance — real regressions such as growing an AST node type change sizes by orders of magnitude more — rewrite the value and fail the `git diff --exit-code` check as before. - The trailing comments are rendered from the recorded value, so a snapshot regenerated on any platform stays byte-identical while within tolerance. No CI workflow changes are needed, and drift can't accumulate silently: each run compares against the committed value, so creeping sub-tolerance changes eventually cross the threshold and surface. The committed values here are aarch64-measured; x86_64 CI measures 32–128 bytes higher and keeps them. Open PRs that touch the old `.snap` files will need a rebase and a `cargo allocs` rerun. 🤖 Implemented with [Claude Code](https://claude.com/claude-code). |