Oxlint's raw transfer implementation stores the source text at end of the buffer. Prior to this PR `napi/parser` stored source text at the start of the buffer.
Bring `napi/parser` into line with Oxlint, by also storing source text at end of the buffer.
Making all versions of raw transfer that we use align with each other removes complication (all versions now have the same implementation of `deserializeStr`) and also removes some fragile code where it'd be easy to accidentally trigger UB (see the updated comments in `arena/fixed_size/windows.rs` and `arena/alloc_impl.rs`).
Unfortunately, storing source at end of buffer is slightly less efficient than storing it at the start. We'll switch all implementations to storing source at start of buffer when we move to storing all strings (source text and all other strings) together in a single contiguous block. But in meantime, this slight inefficiency is outweighed by the gain in safety.
Small perf optimization. When deserializing tokens in `oxc-parser`, use only the `Int32Array` view of the buffer, instead of both `Int32Array` and `Uint8Array`. Less values to keep in registers, and `int32` doesn't need to be obtained with a property lookup on each call.
Implement the beginnings of support for tokens in `napi/parser`.
This PR only adds support for tokens via raw transfer, and behind an undocumented `experimentalTokens` option.
Add tests checking that tokens received on JS side via raw transfer match snapshots for all Test262, AcornJSX, and TypeScript test cases. They do!
The tests are the main purpose of this PR, making sure it works before we switch over to tokens via raw transfer in Oxlint.
We can add full tokens support to `napi/parser` (including via JSON transfer) later on.
Similar to #16442.
There's no need to remove the `experimental*` properties from `options` object on JS side. They'll be ignored by NAPI-RS on Rust side anyway, so creating a new object on JS side is pointless overhead.
Add an option to `oxc-parser` to add `parent` to all AST nodes. This option is only available with raw transfer (not really feasible with JSON transfer, as JSON doesn't support circular references).
This is really for the purposes of JS plugins in Oxlint, so have not included the new deserializers which add `parent` in the `oxc-parser` NPM package, to avoid bloating download size. But it's ideal home is in `napi/parser` so can add it to the conformance tests.
`parent` is correct for all nodes in all Test262 and TypeScript test cases.
We had a runtime `preserveParens` option in deserializers to cater for linter, but it was not used in parser. Convert it to a compile-time option instead.
Now that `apps/oxlint` and `napi/parser` are both ESM packages (#13723 and #14042), we can use `.js` file extensions for all ESM files, rather than the mix of `.js` and `.mjs` file we had previously.
It's less confusing having to remember what's `.js` and what's `.mjs`, and avoids ugly workarounds like #14038 (`.d.mts`??).
overlookmotel