Fix tokens produced when `<<` opens a type argument list. e.g.:
```typescript
type Bar = ReturnType<<T>(x: T) => number>;
```
Previously only 1 `<` token was produced between `ReturnType` and `T`. This PR corrects that so 2 `<` tokens are produced.
TS-ESLint get this wrong too - hence why this wasn't noticed before, as our conformance tests compare to TS-ESLint.
PR submitted to fix TS-ESLint: https://github.com/typescript-eslint/typescript-eslint/pull/12821
In meantime, this PR skips the erroneously failing ESTree tokens conformance tests. We can can re-enable those tests once the fix lands in TS-ESLint.
Pin the Codegen benchmark fixture to an immutable `benchmark-files` commit. This makes benchmark inputs reproducible and prevents `main` updates from silently changing results.
#25488 copied the code from `napi/parser` for splitting TypeScript test fixtures.
AI reviewer correctly pointed out that it was clearly broken: https://github.com/oxc-project/oxc/pull/25488#discussion_r3754102027
Turns out all this code was pointless. It looks up the error codes TypeScript provides for its fixtures, but that's not needed here. And in any case (according to Claude at least) the logic wasn't correctly ported from the Rust version, so it wouldn't work anyway.
So this PR just deletes it all.
If #25488 is merged, this will keep the 2 copies in sync with each other.
A little breakdown of what was going on:
- the failing test case was testing that raw transfer vs non raw transfer was the same. This was parsing checker.ts (a 3.2mb file)
- this test was running concurrently (via vitest's describe.concurrently) with a memory exhaustion test that was running parse with raw transfer 10,000x concurrently
- as a result of these concurrent tests, it resulted in a large number of JS objects being created (and hence needing to be GC-ed)
- since CI is not deterministic, it caused a flakiness roughly 2% of the time
- this can clearly be seen from the attached flamegraph - the readFile call (reading checker.ts from the disk) was sometimes taking up to 3 seconds to read the file from the disk
This seems to be fixed by no longer using describe.concurrent, as it reduces the amount of concurrent work and reduces the chance of the timeout.
<img width="4096" height="1724" alt="image" src="https://github.com/user-attachments/assets/0fd5bc59-8d28-4453-adda-bf33834aafa8" />
## Summary
Follow-up to #22590. That PR replaced the synthetic `cal.com.tsx` fixture — a concatenation of Next.js pages that declares top-level names 70–113× and emits thousands of redeclaration diagnostics, inflating semantic `sys_allocs` — with excalidraw's real, single-file `App.tsx`. But it only touched `TestFiles::complicated()` (used by `track_memory_allocations`).
This applies the same swap to every remaining benchmark that still used `cal.com.tsx`, and unifies **all** excalidraw `App.tsx` references on one pin (`f6d85bc8`):
| File | Change |
| --- | --- |
| `tasks/common/src/test_file.rs` | `minimal()` (feeds every `tasks/benchmark` criterion bench): `cal.com.tsx` → `App.tsx@f6d85bc8`. `formatter()` re-pinned `@v0.18.0` → `@f6d85bc8` |
| `napi/parser/bench.bench.js` | parser benchmark fixture → `App.tsx@f6d85bc8` |
| `napi/parser/test/parse-raw.test.ts` | bench-fixture list + size comment → `App.tsx@f6d85bc8` |
| `tasks/benchmark/benches/semantic.rs` | rewrote the now-false "`cal.com.tsx` has many errors" comment |
| `crates/oxc_ast/src/serialize/mod.rs` | removed the stale `cal.com.tsx` row from the capacity-ratio doc table |
## Why a single pin
`get_source_text` caches downloads by the last URL segment only (`App.tsx`). With two different pins live, a single `cargo bench` run would download both into the same `target/App.tsx` and silently bench whichever landed first. Re-pinning `formatter()` to `f6d85bc8` removes that collision — and the pre-existing latent one between `cargo bench` (`@v0.18.0`) and `cargo allocs` (`@f6d85bc8`).
## Notes
No committed snapshots change: these fixtures feed only timing benches and a `programRaw.toEqual(programStandard)` self-consistency test; the snapshotted `complicated()` / `minifier()` sets are untouched.
---
This PR was prepared with AI assistance; I reviewed, tested, and verified the changes.
Implement the beginnings of support for tokens in `napi/parser`.
This PR only adds support for tokens via raw transfer, and behind an undocumented `experimentalTokens` option.
Add tests checking that tokens received on JS side via raw transfer match snapshots for all Test262, AcornJSX, and TypeScript test cases. They do!
The tests are the main purpose of this PR, making sure it works before we switch over to tokens via raw transfer in Oxlint.
We can add full tokens support to `napi/parser` (including via JSON transfer) later on.
`napi/parser`'s tests for raw transfer ranges, parents, and lazy deserialization were incorrect. They were testing parsing the entire TS fixture file, which can contain multiple parts, instead of testing each part individually. Fix that.
Pure refactor.
Remove a few remaining instances of the pattern where we destructure global objects at top level e.g. `const { isArray } = Array;`.
In Oxlint, TSDown plugin already performs this optimization. In `napi/parser`, we intend to build the package with TSDown, and we'll apply the same optimization in build process then too.
#18089 added support in parser for `commonjs` source type.
Extend this support to the NAPI packages - accept `sourceType: 'commonjs'` in options for `oxc-parser` and `oxc-transform`.
Refactor tests. The tests for `ImportDeclaration` and `ImportExpression` had somehow got nested within the tests for `TemplateExpression`. Move them to a better place.
## Summary
- Implements speculative/unambiguous parsing for `.js`, `.jsx`, `.ts`, and `.tsx` files
- Instead of assuming module or script upfront, parses with `ModuleKind::Unambiguous` and upgrades to module mode when ESM syntax is detected
- ESM indicators: `import`, `export`, `import.meta` (NOT top-level await alone)
- Defers top-level await errors until module type is resolved
- Vue loader upgrades unambiguous scripts to module mode
- Updates linter tests to use explicit ESM extensions (`.mjs`/`.mts`) where needed
## Test plan
- [x] All parser tests pass
- [x] All linter tests pass
- [x] All semantic tests pass
- [x] Coverage conformance tests updated
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Enable `prefer-destructuring` rule in our linting setup.
Not related to JS plugins, I just like this rule, and it's useful not to make nitty comments on PRs, when linter can do it for me...
Follow-on after #16383 and #16403.
While working on those I discovered an option NAPI-RS has to represent `Option::None` as `null`, instead of omitting the field entirely.
Use that option on the structs used for transferring errors and module record over to JS.
I think this is likely more performant because:
1. NAPI-RS can create all properties of the objects using only the faster `node_api_create_object_with_properties` API.
2. It produces consistent object shapes, so JS engine can better optimize code using these objects.
It also brings the shape of the data perfectly into line between standard transfer and raw transfer, and is consistent with how empty fields in the AST are represented as `null`.
I would have used this option before if I'd known it existed.
### Breaking change
I've marked this as a breaking change because code consuming these objects would now need to check for empty fields with `value === null` instead of `value === undefined`.
However in practice, most people probably use `!value` or `value ?? ...`, so it's unlikely to affect many users. This only affects module record and errors anyway, not the AST itself, as we transfer that as JSON, not via NAPI.
NAPI-RS 3.6.0 makes a change to the ordering of objects. It now puts optional fields last.
This made tests in `napi/parser` for module record fail when we tried to update (#16383) because the `module_request` field of `ExportEntry` moves to last which breaks the snapshots.
The change in NAPI-RS is unlikely to be reverted, because it's a sizeable perf optimization: https://github.com/napi-rs/napi-rs/pull/2990
To work around this problem:
1. Move the field in the `ExportEntry` intermediate struct in `napi/parser` to last, so NAPI's output matches what you'd expect from the struct definition.
2. Alter the `#[estree]` attr on `ExportEntry` struct in `oxc_syntax` crate to match.
Note: The `ExportEntry` struct in `napi/parser` is just an intermediate structure used for serialization. So I think it's fine to fiddle with its field order. The actual `ExportEntry` struct used in the module record is in `oxc_syntax` crate, and it remains unaltered.
`napi/parser` tests utilize hidden `experimentalRawTransfer` and `experimentalParent` options. Patch the types of `parse` and `parseSync` in the tests so using these options is not a type error. This replaces a bunch of `// @ts-ignore` and `// @ts-expect-error` comments.
Use `const` where variables are not reassigned instead of `let`. Preparation for enabling ESLint's `prefer-const` rule (running as an Oxlint JS plugin) in our linting setup.
Tiny style change. Use `// oxlint-disable-next-line` comment instead of `// eslint-disable-next-line`. Either works, but obviously Oxlint is superior. :)
The raw transfer test on `antd.js` fixture is timing out sometimes on CI. We already disabled the "range & parent" test on this fixture. Disable the other test on it too. But only skip these tests in CI.
(alternative to #14438)
The raw transfer range + parent test on `antd.js` fixture is still hitting 5 second timeout on CI. For some reason the `timeout` option isn't increasing the timeout. Just skip this test.
Add an option to `oxc-parser` to add `parent` to all AST nodes. This option is only available with raw transfer (not really feasible with JSON transfer, as JSON doesn't support circular references).
This is really for the purposes of JS plugins in Oxlint, so have not included the new deserializers which add `parent` in the `oxc-parser` NPM package, to avoid bloating download size. But it's ideal home is in `napi/parser` so can add it to the conformance tests.
`parent` is correct for all nodes in all Test262 and TypeScript test cases.
#13465 aligned the version of `antd.js` used in different Rust tasks to v4.16.1. But it missed a couple of places in `napi/parser` tests/benchmarks. Align these to same version too.
Misalignment was problematic as the tasks using these fixtures all download the file to same location (`target` dir) and don't re-download if the file is already present. This lead to unpredictable results depending on which task you ran first, and so which version was downloaded.
Now that `apps/oxlint` and `napi/parser` are both ESM packages (#13723 and #14042), we can use `.js` file extensions for all ESM files, rather than the mix of `.js` and `.mjs` file we had previously.
It's less confusing having to remember what's `.js` and what's `.mjs`, and avoids ugly workarounds like #14038 (`.d.mts`??).
overlookmotel