Commit Graph

130 Commits

Author SHA1 Message Date
overlookmotel 48d51e31bc test(napi): add tests for hashbang field (#9386)
ESTree conformance tester does not test `hashbang` field, so add tests to `napi/parser` instead.
2025-02-26 13:07:35 +00:00
Boshen 4a5a7cf921 feat(napi/parser)!: remove magic string; enable utf16 span converter by default (#9291)
Benchmark reveals utf16 span converter is not that slow:

```
 ✓ bench.bench.mjs 1491ms
     name                                                  hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · parser_napi[checker.ts] convertSpanUtf16: true   31.3194  30.8477  34.4288  31.9291  32.3740  34.4288  34.4288  34.4288  ±1.61%       16
   · parser_napi[checker.ts] convertSpanUtf16: false  34.1307  28.6449  29.9385  29.2991  29.6852  29.9385  29.9385  29.9385  ±0.70%       18   fastest

 BENCH  Summary

  parser_napi[checker.ts] convertSpanUtf16: false - bench.bench.mjs
    1.09x faster than parser_napi[checker.ts] convertSpanUtf16: true
```

```
 ✓ bench.bench.mjs 2127ms
     name                                               hz      min      max     mean      p75      p99     p995     p999     rme  samples
   · parser_napi[antd.js] convertSpanUtf16: true   15.1191  65.1335  67.4651  66.1414  66.6032  67.4651  67.4651  67.4651  ±0.70%       10
   · parser_napi[antd.js] convertSpanUtf16: false  17.3037  57.0110  58.6842  57.7911  58.1348  58.6842  58.6842  58.6842  ±0.69%       10   fastest

 BENCH  Summary

  parser_napi[antd.js] convertSpanUtf16: false - bench.bench.mjs
    1.14x faster than parser_napi[antd.js] convertSpanUtf16: true
```

Remove the option and enable the conversion by default.

Magic string API becomes useless at this point. 

This is a trade off between dx, maintenance and performance. dx and
maintainance wins this time.

---------

Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2025-02-22 14:27:35 +08:00
overlookmotel 216b33fd2a feat(ast/estree)!: replace serde with custom ESTree serializer (#9256)
Replace `serde` with a custom serializer specialized for writing JSON.

`impl Serialize` -> `impl ESTree`. I've called it `ESTree` in case we
want to add e.g. `impl Babel` later on.

The main motivation is that with control of the serializer, we can make
it stateful, so we can extend it to, for example, omit TS fields based
on a flag. This is pretty much impossible with `serde`'s interface.

But a side-effect of the simpler implementation is that it's also more
performant.

The usage of `ESTree` trait is very similar to `serde::Serialize`. Main
differences:

* `serialize` method is infallible. Does not return a `Result`.
* `state = serializer.serialize_struct();` instead of `map =
serializer.serialize_map();`.
* `state.serialize_field(...)` instead of `map.serialize_entry(...)`.
* Struct keys must be strings, and JSON-safe (not including `"` or other
characters that require escaping in JSON).

This is a breaking change, as it's possible downstream consumers are
using `serde` to serialize our AST to other formats. I'd guess unlikely,
but possible. We can always implement `serde::Serialize` as well as
`ESTree` further down the line if there's a demand.
2025-02-21 12:20:26 +08:00
hi-ogawa eaff3d985f test(napi/parser): split tests for convertSpanUtf16 (#9113)
Minor cleanup. I was smashing all assertions in a single `it`, but I've wanted to split them.
2025-02-14 12:58:58 +00:00
hi-ogawa 0937a55b3b fix(napi/parser): utf16 span for errors (#9112) 2025-02-14 12:58:58 +00:00
hi-ogawa 15f23f1556 fix(napi/parser): utf16 span for module record (#9093)
I (and mostly copilot) wrote `VisitMutModuleRecord` to visit all spans as I wasn't sure how `VisitMut` is generated. I'm looking into visit generator now if it's something I can do. Please let me know if manually writing it down is fine for now or I should try macro (or maybe a completely different approach).
2025-02-14 08:00:57 +00:00
hi-ogawa 9edfb1d330 fix(napi/parser): fix unicode comment panic (#9084)
- closes https://github.com/oxc-project/oxc/issues/9079

I'm still leaving an existing TODO for `module_record` and `errors` span not being converted as they won't cause an immediate issue, but I'll work on that shortly.
2025-02-13 10:02:37 +00:00
overlookmotel 41dba62780 fix(ast/estree): set value for BigIntLiterals and RegExpLiterals on JS side (#9044)
Add a "reviver" function to `JSON.parse` in NAPI parser module, to correctly set `value` field of `Literal`s for `RegExp`s and `BigInt`s.

Surprisingly, judging by a local run of NAPI parser benchmarks (added in #9045), this does not seem to hurt performance. In fact the benchmarks were showing a ~2% speed-up. That's clearly nonsense - just noise - but it does suggest at least that this isn't hurting performance significantly.
2025-02-11 14:48:43 +00:00
hi-ogawa 81c81a7cb5 feat(napi/parser): add convert_span_utf16 option (#8983)
For starter, I added opt-in `convert_span_utf16` for `napi/parser` level option, so rolldown-vite can start to battle test estree output.
2025-02-11 06:21:54 +00:00
hi-ogawa 4803059925 test(ast): remove old ast snapshot tests (#8976) 2025-02-09 04:14:36 +00:00
Hiroshi Ogawa a520986e1a fix(ast): estree compat Program.sourceType (#8919)
part of https://github.com/oxc-project/oxc/issues/2854
ref: https://github.com/estree/estree/blob/master/es2015.md#programs

I tried to move `via` customization on `struct SourceType`, but couldn't
find a way maybe because struct is defined in `oxc_span`. For now, I
made this as `via` on `Program::source_type` field.
2025-02-06 13:18:32 +00:00
Hiroshi Ogawa e30cf6aeb1 fix(ast): estree compat MemberExpression (#8921)
Part of https://github.com/oxc-project/oxc/issues/2854

I added a bit weird macro `add_entry(computed = true)` to add constant.
Likely this is not the best way. I would appreciate any suggestion for a
better approach 🙏

---------

Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>
2025-02-06 11:50:31 +00:00
Hiroshi Ogawa 0c55dd684e fix(ast): serialize Function.params like estree (#8772)
- part of https://github.com/oxc-project/oxc/issues/2854

This PR attempts to handle estree ast incompatibility of
`Function.params: FormalParameters` as mentioned in the above issue:

> `FormalParameters` is closer to ESTree now, but should be inlined
directly into `<node>.params` in multiple nodes.

Estree spec has `Function.params: Pattern[]`
https://github.com/estree/estree/blob/master/es5.md#functions, but oxc
already has `interface Pattern`, so I named it to `Function.params:
ParamPattern[]` for now.

Also I'm not sure about the testing (I suppose that's a part of
https://github.com/oxc-project/oxc/issues/8630), so I snapshoted one
example code. For comparison, here is acorn's output
https://astexplorer.net/#/gist/25138c0605f82dcfc1a8fd363dc2a681/5ad30d36c9f276519063e6fd2e340c113d8c85b0

---------

Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>
2025-02-05 02:40:02 +00:00
Boshen ee8ee55cda feat(napi/parser): add .hasChanged() to MagicString (#8586) 2025-01-18 15:29:17 +00:00
Boshen 1bef911e59 feat(napi/parser): add source map API (#8584) 2025-01-18 23:06:42 +08:00
Boshen c479a58718 feat(napi/parser): expose dynamic import expressions (#8540)
closes #8369
2025-01-16 22:25:22 +08:00
Boshen 9157a0ed0f feat(napi/parser): add getUtf16ByteOffset API; update README (#7772) 2024-12-10 14:15:39 +00:00
Boshen f3cfa7f3dd feat(napi/parser): add get_line_column_number API (#7765) 2024-12-10 12:49:11 +00:00
Boshen 4bd3d103d3 feat(napi/parser): introduce experimental MagicString (#7529)
Hold magic string instance on the Rust side for utf8 string manipulation.
2024-12-10 08:33:47 +00:00
Boshen 85eec3c82e feat(napi/transform,napi/parser): return structured error object (#7724)
closes #7261
2024-12-08 14:11:56 +00:00
Boshen 00fea9285b feat(napi/parser): expose span positions of import.meta (#7677) 2024-12-05 11:03:52 +00:00
Boshen 1de2842f8e feat(napi/parser): return EcmaScriptModule::hasModuleSyntax (#7647) 2024-12-04 13:00:47 +00:00
Boshen 40792b4440 feat(napi/parser): change parse API to accept mandatory filename and optional lang (#7605) 2024-12-03 12:09:48 +00:00
Boshen 7c62a33a06 feat(napi/parser): return esm info (#7602)
The parser now returns import / export statement information, which can be used for parser plugins.
2024-12-03 11:21:54 +00:00
Boshen 943462f22c feat(module_lexer)!: remove oxc_module_lexer (#7595)
This crate will no longer be relevant after we export esm information
directly from the parser.

Besides, `ex-module-lexer`'s output data is too limited for plugin
authors to use.
2024-12-03 04:22:51 +00:00
Boshen 24c585a20a feat(napi/parser): reexport @oxc-project/types (#7214)
closes #7207
2024-11-09 17:24:07 +08:00
Boshen d1d187417b feat(ast)!: change comment.span to real position that contain // and /* (#7154)
closes #7150
2024-11-06 05:10:33 +00:00
Boshen 9ed9501bcd fix(module_lexer): add missing export * from 'foo'; case (#7103)
fixes #7039
2024-11-03 16:31:19 +00:00
ottomated 67a7bde4c6 feat(napi/parser)!: add typings to napi/parser (#6796)
- Added TypeScript annotation for `ParseResult.program`
- Modified the entrypoint for `oxc-parser` to allow wrapping the napi functions
- Updated `index.js` to parse the `program` string into a JSON object
- Updated tests
- Added a dependency on `@oxc/types`
2024-10-24 13:08:59 +00:00
Boshen 5b5daec392 refactor(napi): use vitest (#6307) 2024-10-06 02:57:48 +00:00