mirror of
https://github.com/oxc-project/oxc.git
synced 2026-09-14 19:36:11 +08:00
codex/parser-grouped-expression-allocation
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Cameron Clark
|
7784ad92d5 | perf(parser): avoid allocation for single grouped expressions | ||
|
camc314
|
a33788e13e |
feat(ast)!: group class heritage into ClassHeritage (#25360)
## Summary
- Groups `Class::super_class` and `Class::super_type_arguments` into `Class::heritage`.
- Introduces `ClassHeritage` containing the superclass expression and its optional TypeScript type arguments.
- Makes superclass type arguments without a superclass expression unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub super_class: Option<Expression<'a>>,
#[ts]
pub super_type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
```
to:
```rust
pub struct Class<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub r#type: ClassType,
pub decorators: Vec<'a, Decorator<'a>>,
pub id: Option<BindingIdentifier<'a>>,
#[ts]
pub type_parameters: Option<Box<'a, TSTypeParameterDeclaration<'a>>>,
pub heritage: Option<ClassHeritage<'a>>,
#[ts]
pub implements: Vec<'a, TSClassImplements<'a>>,
pub body: Box<'a, ClassBody<'a>>,
#[ts]
pub r#abstract: bool,
#[ts]
pub declare: bool,
pub scope_id: Cell<Option<ScopeId>>,
}
pub struct ClassHeritage<'a> {
pub expression: Expression<'a>,
#[ts]
pub type_arguments: Option<Box<'a, TSTypeParameterInstantiation<'a>>>,
}
```
## Why?
ECMAScript defines the complete `extends` clause as `ClassHeritage`:
```text
ClassHeritage : extends LeftHandSideExpression
```
TypeScript additionally permits type arguments on the heritage expression:
```ts
class Derived extends Base<T> {}
```
Previously, the AST stored the expression and type arguments independently:
```text
Class {
super_class: Some(Base),
super_type_arguments: Some(<T>),
...
}
```
This allowed an invalid state to be constructed:
```text
Class {
super_class: None,
super_type_arguments: Some(<T>),
...
}
```
Superclass type arguments cannot exist without an `extends` expression.
The parser already treated these fields as a pair. It parsed a class heritage entry and then split its expression and type arguments into two independent fields. Builders and transforms therefore had to preserve their relationship manually.
Grouping them into `ClassHeritage` makes the syntactic dependency explicit and prevents consumers from observing or constructing contradictory class heritage.
## Class heritage forms
A class without an `extends` clause has no heritage:
```js
class Base {}
```
```text
Class {
heritage: None,
...
}
```
A class extending an expression has heritage without type arguments:
```js
class Derived extends Base {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: None,
}),
...
}
```
TypeScript type arguments belong to the same heritage structure:
```ts
class Derived extends Base<T> {}
```
```text
Class {
heritage: Some(ClassHeritage {
expression: Base,
type_arguments: Some(<T>),
}),
...
}
```
The heritage expression may be any valid class heritage expression:
```js
class Derived extends mixin(Base) {}
class NullDerived extends null {}
```
## How to migrate
Code that previously accessed the superclass expression directly:
```rust
if let Some(super_class) = &class.super_class {
// ...
}
```
can use the compatibility accessor:
```rust
if let Some(super_class) = class.super_class() {
// ...
}
```
Code that needs both parts should access the grouped heritage:
```rust
if let Some(heritage) = &class.heritage {
let expression = &heritage.expression;
let type_arguments = heritage.type_arguments.as_deref();
}
```
Mutating consumers should update the nested fields:
```rust
if let Some(heritage) = &mut class.heritage {
heritage.type_arguments = None;
}
```
Common migrations:
| Previous representation | New representation |
| --- | --- |
| `class.super_class` | `class.heritage.as_ref().map(\|h\| &h.expression)` or `class.super_class()` |
| `class.super_type_arguments` | `class.heritage.as_ref()?.type_arguments` or `class.super_type_arguments()` |
| `class.super_class.is_some()` | `class.heritage.is_some()` |
| Mutate `class.super_class` | Mutate `class.heritage.expression` |
| Clear `class.super_type_arguments` | Clear `class.heritage.type_arguments` |
| Pass superclass and type arguments separately to a `Class` builder | Construct and pass `ClassHeritage` |
| Clone both fields independently | Clone `class.heritage` |
```ts
superClass: Expression | null;
superTypeArguments?: TSTypeParameterInstantiation | null;
```
No `heritage` property or `ClassHeritage` node is added to the ESTree output.
fixes https://github.com/oxc-project/backlog/issues/216
|
||
|
leaysgur
|
a56b7d059b |
refactor(tasks/allocations): do not print redundunt traling blank line (#25268)
Otherwise, our new YAML formatter trims it, then remains as diff. |
||
|
camc314
|
1bdedd11ca |
feat(ast)!: introduce ExportDeclaration, ExportFromDeclaration (#25095)
## Summary
Split the overloaded Rust `ExportNamedDeclaration` into syntax-specific AST nodes:
- `ExportDeclaration` for exported declarations
- `ExportNamedDeclaration` for local exports
- `ExportFromDeclaration` for re-exports
## Breaking Change
The Rust AST changes from:
```rust
pub struct ExportNamedDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub declaration: Option<Declaration<'a>>,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub source: Option<StringLiteral<'a>>,
pub export_kind: ImportOrExportKind,
pub with_clause: Option<Box<'a, WithClause<'a>>>,
}
pub enum ModuleDeclaration<'a> {
// ...
ExportNamedDeclaration(Box<'a, ExportNamedDeclaration<'a>>),
// ...
}
```
to:
```rust
pub struct ExportDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub declaration: Declaration<'a>,
}
pub struct ExportNamedDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub export_kind: ImportOrExportKind,
}
pub struct ExportFromDeclaration<'a> {
pub node_id: Cell<NodeId>,
pub span: Span,
pub specifiers: Vec<'a, ExportSpecifier<'a>>,
pub source: StringLiteral<'a>,
pub export_kind: ImportOrExportKind,
pub with_clause: Option<Box<'a, WithClause<'a>>>,
}
pub enum ModuleDeclaration<'a> {
// ...
ExportDeclaration(Box<'a, ExportDeclaration<'a>>),
ExportNamedDeclaration(Box<'a, ExportNamedDeclaration<'a>>),
ExportFromDeclaration(Box<'a, ExportFromDeclaration<'a>>),
// ...
}
```
## Why?
The previous node represented three distinct syntax forms through optional and mutually exclusive fields:
```text
export const value = 1; -> ExportDeclaration
export { value }; -> ExportNamedDeclaration
export { value } from "mod"; -> ExportFromDeclaration
```
This allowed invalid combinations such as declarations with specifiers or sources, and import attributes without a source. The new nodes make these states unrepresentable and remove repeated `declaration`/`source` branching from consumers.
## How to migrate
Match the syntax-specific `ModuleDeclaration`, `Statement`, or `AstKind` variant instead of inspecting optional fields. Generated builders, visitors, traversal ancestors, and formatter wrappers now expose corresponding APIs for all three nodes.
Common field migrations:
| Previous representation | New representation |
| --- | --- |
| `declaration: Some(declaration)` | `ExportDeclaration { declaration }` |
| `declaration.as_ref()` | `&export.declaration` |
| `source.is_none()` for local exports | Match `ExportNamedDeclaration` |
| `source: Some(source)` | `ExportFromDeclaration { source }` |
| `source.as_ref().unwrap()` | `&export.source` |
| `with_clause` | Only available on `ExportFromDeclaration` |
| Stored `export_kind` on exported declarations | `ExportDeclaration::export_kind()` |
| `ExportNamedDeclaration::boxed_plain_declaration(...)` | `ExportDeclaration::boxed(...)` |
| `ExportNamedDeclaration::boxed(..., None, specifiers, None, ...)` | `ExportNamedDeclaration::boxed(..., specifiers, ...)` |
| `ExportNamedDeclaration::boxed(..., None, specifiers, Some(source), ...)` | `ExportFromDeclaration::boxed(..., specifiers, source, ...)` |
closes https://github.com/oxc-project/backlog/issues/219
|
||
|
camc314
|
c917f204a7 |
feat(ast)!: introduce ArrowFunctionBody enum (#24987)
## Summary
- Introduces `ArrowFunctionBody` to represent concise expression and block bodies explicitly.
- Removes the `ArrowFunctionExpression::expression` boolean.
- Makes invalid arrow body combinations unrepresentable.
## Breaking Change
The Rust AST changes from:
```rust
pub struct ArrowFunctionExpression<'a> {
pub expression: bool,
pub body: Box<'a, FunctionBody<'a>>,
}
```
to:
```rust
pub enum ArrowFunctionBody<'a> {
FunctionBody(Box<'a, FunctionBody<'a>>) = 64,
INHERIT(Expression<'a>),
}
pub struct ArrowFunctionExpression<'a> {
pub body: ArrowFunctionBody<'a>,
}
```
Concise arrow bodies are now stored directly as expressions instead of synthetic `FunctionBody` and `ExpressionStatement` nodes. Block bodies continue to use `FunctionBody`.
## Why?
The previous representation required consumers to maintain this implicit invariant:
```text
expression == true
→ no directives
→ exactly one statement
→ that statement is an ExpressionStatement
```
Invalid combinations could cause panics, skipped work, or fallback behavior. The enum encodes the grammar directly and makes those invalid states unrepresentable.
Concise expressions also now have `ArrowFunctionExpression` as their direct traversal ancestor, without synthetic `FunctionBody` or `ExpressionStatement` nodes.
## How to migrate
Use the arrow body accessors or match on `ArrowFunctionBody` rather than inspecting `expression` and synthetic statements.
Common field migrations:
| Previous representation | New representation |
| --- | --- |
| `arrow.expression` | `arrow.is_expression()` |
| `arrow.get_expression()` | `arrow.get_expression()` |
| `&arrow.body.statements` | `arrow.get_function_body().map(\|body\| &body.statements)` |
| `&mut arrow.body.statements` | `arrow.get_function_body_mut().map(\|body\| &mut body.statements)` |
| Synthetic expression statement in `arrow.body` | Direct `ArrowFunctionBody` expression variant |
| `arrow.body = function_body` | `arrow.body = ArrowFunctionBody::FunctionBody(function_body)` |
| Expression-to-block conversion | Create a `FunctionBody` containing a return statement |
| Block-to-expression conversion | Assign the expression directly to `arrow.body` |
| Builder `(expression, body)` arguments | Pass a single `ArrowFunctionBody` |
closes https://github.com/oxc-project/backlog/issues/211
|
||
|
Boshen
|
2338c13f60 |
feat(track-memory-allocations): track heap deallocs, alloc bytes, and peak growth (#24619)
Adds three heap-side metrics to the allocation snapshots, using the metric machinery from #24587: - `sys deallocs` (exact count) — system deallocations per stage; read against `sys allocs` it shows whether a stage churns or retains heap. - `sys alloc bytes` — cumulative bytes requested from the system allocator per stage (allocation sizes plus reallocation growth). - `sys peak growth` — high-water mark of live heap during a stage relative to the live bytes at stage start, via live-bytes accounting in the `GlobalAlloc` wrapper. Captures transient heap a stage needs even when it frees it before finishing. Arena chunk operations are excluded from all `sys` metrics at the source: `oxc_allocator` marks each chunk allocation/deallocation immediately before calling the system allocator (task-only `track_allocations` feature, compiled out everywhere else), and the task's tracking allocator consumes the marker and skips the call. This replaces the previous global chunk counter + per-stage subtraction: a subtraction can correct counts but not a high-water mark, and chunk sizes are quantized and platform-dependent (whether an arena needs one more chunk depends on byte totals that vary across architectures with type layout — the #22621 mechanism). The first CI run demonstrated this concretely: kitchen-sink's minifier peak differed deterministically by 720 kB between aarch64 and x86_64 because the mangler's temporary arena crossed a chunk-growth boundary only on x86_64; and antd's formatter "peak" was 762 MB of main-arena chunk-doubling history rather than formatter behavior (now 13.8 MB of actual heap use). Cross-platform tolerances: the cumulative byte metrics still drift with the number of heap hashbrown tables allocated (their control-group width is architecture-dependent), so they use a relative tolerance (1%, 4 kB floor) via the committed-value comparison from #24565; `arena size` keeps its tight 1024-byte tolerance. Sample values for checker.ts (2.9 MB source): parser needs only 2.4 kB of transient heap (fully arena-based), semantic peaks at 3.9 MB, the minifier is the pipeline's largest heap consumer at 10.2 MB, and the formatter requests 5.9 MB total while peaking at 3 MB (heavy alloc/free churn). 🤖 Implemented with [Claude Code](https://claude.com/claude-code). |
||
|
Boshen
|
b1165f9c63 |
fix(track-memory-allocations): make arena size snapshots platform-independent (#24565)
The `Allocations` CI job fails whenever the snapshots are regenerated on a different CPU architecture than CI (e.g. https://github.com/oxc-project/oxc/actions/runs/29420345285/job/87368903834): every `arena size` value differs by 32–128 bytes between aarch64 and x86_64. The cause is target-dependent type layout — hashbrown's table control groups are 16 bytes wide on x86_64 (SSE2) but 8 bytes on aarch64 (NEON) — so each live `HashMap` in the arena shifts `used_bytes()` slightly. This is the same root cause that #22621 handled for chunk counts; the `arena size` gauge added in #24553 re-exposed it. Allocation counts are unaffected and stay exact. Exact byte equality across architectures isn't achievable (the arena genuinely occupies a different number of bytes), so instead the snapshot writer tolerates layout noise: - Snapshots are now YAML (`allocs_*.yaml`) with the same layout as before: one value per line, file names at column 0 for git hunk headers, exact numbers as the diffed values, and human-readable sizes moved to trailing `#` comments. - When regenerating, `cargo allocs` parses the committed snapshot (with `saphyr`, already a workspace dependency used by `oxc_coverage`) and keeps the committed `arena size` when the measured value is within `ARENA_SIZE_TOLERANCE` (1024 bytes, ~8× the observed worst-case cross-arch drift). Differences beyond the tolerance — real regressions such as growing an AST node type change sizes by orders of magnitude more — rewrite the value and fail the `git diff --exit-code` check as before. - The trailing comments are rendered from the recorded value, so a snapshot regenerated on any platform stays byte-identical while within tolerance. No CI workflow changes are needed, and drift can't accumulate silently: each run compares against the committed value, so creeping sub-tolerance changes eventually cross the threshold and surface. The committed values here are aarch64-measured; x86_64 CI measures 32–128 bytes higher and keeps them. Open PRs that touch the old `.snap` files will need a rebase and a `cargo allocs` rerun. 🤖 Implemented with [Claude Code](https://claude.com/claude-code). |