In ESTree serializer, reduce repeated code by introducing a `serialize_span` method which outputs `start` and `end`, and also `range` if the `range` option is enabled.
Related to #10307.
Re-arrange fields in ESTree AST, so `start` and `end` are last, instead of first.
Also bump `acorn-test262` submodule to include https://github.com/oxc-project/acorn-test262/pull/54, which rearranges the fields in test snapshots.
Integers `>= 4294967295` are treated as string properties, not numerical array indexes. Make `NodeArray` behave the same as `Array` for such properties, and allow user to set them.
Part of #10307. Add `range` option to `oxc-parser`, to add `range` property to nodes in ESTree AST.
---------
Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>
Make `NodeArray` iterators (e.g. `NodeArrayValuesIterator`) constructors take a `Proxy<NodeArray>`.
This makes them safe to construct from user code. Therefore can remove the `TOKEN` check which previously blocked user code from constructing these classes. This makes iterating over a `NodeArray` very slightly faster.
`NodeArray` store length as an internal property, rather then creating an underlying `Array` with that length. This is much faster when the array is large.
Produces an 18x speed-up in getting `program.body` for `cal.com.tsx` benchmark fixture, because it has 3942 statements.
Improve the performance of `NodeArray` by removing the `WeakMap` which was used to get the inner `NodeArray` from a proxy. Use a private `Symbol` instead to perform the same function.
Tiny optimization to lazy visitor. It's marginally cheaper in a loop to compare to 0, rather than to a constant. I also assume that `!== 0` is cheaper than `> 0` as it doesn't involve type coercion.
Improve raw transfer benchmarks. Use `require` to load internal modules instead of `import`. `oxc-parser` uses `require` internally, so using `import` here was resulting in a lot of code being loaded and instantiated twice.
Add lazy visitor to `oxc-parser`.
This API is experimental and not ready for use yet. AST is not ESTree-compatible at present (same as lazy deserializer).
Usage:
```js
import { parseSync, experimentalGetLazyVisitor } from 'oxc-parser';
// Get `Visitor` class.
// `Visitor` is not exported directly from `oxc-parser` as all code
// related to lazy deserialization is lazy-loaded.
const Visitor = experimentalGetLazyVisitor();
// Define visitor.
// This visitor counts how many identifiers in AST.
let identCount = 0;
const visitor = new Visitor({
BindingIdentifier(ident) {
identCount++;
},
IdentifierReference(ident) {
identCount++;
},
IdentifierName(ident) {
identCount++;
},
});
const code = 'let x, y; x = { z: 2 };';
const { visit, dispose } = parseSync('test.js', code, { experimentalLazy: true });
visit(visitor);
assert(identCount === 4);
// Dispose of AST now we're done with it.
// This returns buffer storing AST data to the pool for reuse.
dispose();
```
Pure refactor. Adapt lazy deserialization codegen in preparation for adding lazy visitor. Involves introducing a `State` struct, renaming things, and moving code around. No substantive changes.
Tiny perf optimization to raw transfer. #11807 made this whole file lazy-loaded, so no need to lazily create the `TextEncoder` any more. If the file is imported, it's going to be used.
Pure refactor. Move `parseSync` to be before `parseAsync`, for consistency with all other code in `napi/parser`. Hoist all exports to top of file to make it a little more readable.
Small perf optimization to `oxc-parser`. Destructure `bindings` object when importing it. Calling those methods repeatedly later on avoids overhead of an object property lookup each time.
The effect is infinitesimal, except when using lazy deserialization, which is so fast for small files that it makes sense to optimize every step of its call path.
Raw transfer and raw lazy deserialization are experimental and not publicly advertised. Lazy-load all code for these features only when they're first used.
Lazy deserialization in particular requires a non-trivial amount of code.
`rawTransferSupported` still needs to be eagerly loaded, as user would call it before calling `parseSync` or `parseAsync`. So move it into a separate module.
Pure refactor. Add more `#[estree]` attributes to `RegExpFlagsAlias` type. These have no effect on generated code for JSON serialization or raw transfer deserialization, as `RegExpFlagsAlias` already has a custom converter. But it simplifies the codegen for lazy deserialization.
Small perf optimization to raw transfer. Don't use 2 different counters in the loop when deserializing `Vec`s. Do what Rust's slice iterators do, and use current and end pointers.
Remove the custom ESTree serializer for `MethodDefinition`'s `key` field. This serializer was required to replicate a bug in TS-ESLint, which they've now fixed. This fixes the test case which was broken by #11762.
Move `getElement` function into `NodeArray` class definition, to avoid extra function call to intermediate `getInternal`. This is a hot path, so streamlining it is important for perf.
Simple object shapes are faster. Adding extra property to `NodeArray.prototype` alters its shape. Prevent this by defining a placeholder method in class definition, and then mutating this property later, rather than adding a new property.
Implement an optimized `slice` method for `NodeArray` in lazy-deserialized AST. It returns a another `NodeArray`, instead of a plain `Array`, and doesn't deserialize any of the contents.
Lazy deserialization return `NodeArray`s which deserialize their contents lazily, instead of plain JS arrays.
`program.body.find(stmt => stmt.type === 'ImportDeclaration')` now only deserializes the nodes up until first `ImportDeclaration`, rather than deserializing every statement in the program into an array, and then searching it. Similarly getting `program.body.length` does not deserialize any nodes, and all other array methods are similarly optimized.
Also provide an optimized iterator for the common case of:
```js
for (const stmt of program.body) {
doStuff(stmt);
}
```
Pure refactor. We prefix some property names on `internal` objects with `$` to avoid clashes between static properties (`pos`, `ast`) and cached fields. Switch the prefixing around - now prefix keys for cached fields, so the internal properties are called e.g. `pos` not `$pos`. This shortens generated code, and makes writing implementations simpler.
Fix a bug in lazy deserializer where `Vec`s were not getting cached because they were wrapped in `Option`. When determining if a node's field needs to be locally cached, drill down through any wrapping `Option`s / `Box`es / `Cell`s / enums.
Improve the output from `console.log` when printing a lazy-deserialized AST.
Previously `console.log(program)` printed `Program { type: 'Program' }`. Now it prints much the same as a normal (eagerly deserialized) AST:
```
Program {
type: 'Program',
start: 0,
end: 17,
sourceType: SourceType { sourceType: 'module' },
hashbang: null,
body: [
VariableDeclaration {
type: 'VariableDeclaration',
start: 0,
end: 10,
kind: 'let',
declarations: [Array],
declare: false
}
]
}
```
In lazy-deserialized AST, cache node objects, so accessing the same field twice returns the same object. Previously `assert(binaryExpression.left == binaryExpression.left)` would fail, as each access returned a new object.
`pos` (i.e. memory address) is used as the cache key, with 2 exceptions: `RegExp` and `Program`.
`RegExp` doesn't have a `span: Span` field, so its first field `RegExpPattern` has the same memory address as the `RegExp`. So `pos` would not be a unique key.
```rs
pub struct RegExp<'a> {
pub pattern: RegExpPattern<'a>,
pub flags: RegExpFlags,
}
pub struct RegExpPattern<'a> {
pub text: Atom<'a>,
pub pattern: Option<Box<'a, Pattern<'a>>>,
}
```
The codegen identifies these cases, and applies a workaround - `pos + 1` is used as the cache key instead of `pos`, so the parent and child have different cache keys.
oxc-bot