Closes#17068. Supersedes #18095 / #18118.
Adds `CodegenOptions::ascii_only` (and `codegen.asciiOnly` in
`oxc_minify_napi`): escape every non-ASCII character in string literals,
untagged template literals, regexp literals, directives and identifier
names so the output is 7-bit clean — like esbuild `--charset=ascii` /
terser `format.ascii_only`. Off by default; the default path is
untouched (codegen bench and conformance unchanged).
```js
// Input
let café = "naïve 😀";
let re = /[–—]/u;
// Output with ascii_only: true
let caf\u00E9 = "na\u00EFve \u{1F600}";
let re = /[\u2013\u2014]/u;
```
Characters above U+FFFF use ES2015 code-point escapes. Regex patterns
use escaped surrogate pairs instead; RegExp.prototype.source reflects
the escaped spelling.
Tagged template quasis, JSX names/text/attribute strings, hashbangs and
preserved comments may still contain non-ASCII characters. JavaScript
expressions inside tagged templates and JSX are escaped normally.
Performance; with the option disabled, codegen codspeed benchmarks are
withing noise (<1%). With the option on, performance is only affected
for files that contain non ascii characters.
---
Our use case is a large Vite/Rolldown app served through proxies that
mangle non-ASCII JS (previously handled by terser's `ascii_only`);
#17068 also lists deno_core snapshots and Chrome extension content
scripts.
AI-assisted (Claude Code); I reviewed the code, wrote the tests, and
have been running this in production builds via a vendored
`oxc_codegen`.
---------
Co-authored-by: Samuel Attard <6634592+MarshallOfSound@users.noreply.github.com>
Co-authored-by: Cameron Clark <cameron.clark@hey.com>
Co-authored-by: Dunqing <dengqing0821@gmail.com>
Block comments containing U+2028 or U+2029 were marked `SingleLineBlock` unless they also contained LF or CR. Switch to the multiline-comment scanner after either Unicode separator so downstream consumers receive the correct comment kind.
This prevents the formatter from dropping parentheses around a return argument with such a leading comment, which could change the returned value to `undefined`.
Only bind a private field when it is the result of an optional chain being called. In `(this.#items.get(key)?.find(fn))?.()`, the intermediate `#items` field must not receive `.bind(this)`.
Normalize non-null assertions in the TypeScript transform before other transforms inspect the chain. This preserves the receiver for `(this?.#fn!)()` and `(this?.#fn!!)()` without mixing TypeScript assertion removal into the class transform.
Fixes#26528
The parser consumes `source`, `defer`, and sometimes `from` as binding identifiers before resolving their role in an import declaration. Validate escapes once they are known to be keywords, so declarations such as `import s\u006furce wasm from "m"` and `import source fr\u006fm "m"` report an error.
Escaped binding names remain valid, including `import s\u006furce from "m"` and `import source fr\u006fm from "m"`.
Recognize `await` and `yield` as binding names when looking ahead for TypeScript function types and index signatures. This allows valid syntax such as `(await: number) => void` and `interface I { [await: string]: unknown }`, while leaving contextual legality to actual parsing.
Iterate JSX name segments directly instead of collecting them into a temporary vector. Namespaced names still split on `:`, member expressions on `.`, and simple names remain a single segment.
Require `=>` when parsing TypeScript function and constructor types, while retaining `:` for function, method, and arrow return annotations. Previously, any token could be consumed as the separator, accepting invalid types such as `type F = (): number` and `type F = new () + number`.
This also prevents a valid conditional expression from being silently grouped into the wrong arrows:
```ts
const a = true, b = 1, c = 2;
const f = a ? (b) : (): any => b ? c : (x = 1): any => x;
console.log(typeof f);
```
The transformed program now prints `number`; previously it printed `function`.
Include label text when sorting diagnostics for the optimized/unoptimized runtime assertion. Diagnostics with identical spans and messages but different labels previously compared equal, so sorting could pair different diagnostics and trigger a false assertion failure.
Fixes the assertion seen in the [Kibana ecosystem CI job](https://github.com/oxc-project/oxc-ecosystem-ci/actions/runs/34459300249/job/102815162719).
Allow classic `for` initializers such as `for (await using;;)` when `using` is an ordinary identifier. Reuse the statement parser’s `await using` lookahead so resource declarations require a following binding identifier, preserving the existing line-break restrictions.
Allow parenthesized comma expressions such as `<A>{(a, b)}</A>` and `<A x={(a, b)} />` when `preserve_parens` is false.
Compare the sequence expression's start with the first token's start to retain the grouping distinction after parentheses are omitted from the AST. Ungrouped comma expressions such as `{a, b}` and `{(a, b), c}` remain invalid.
Constructor overloads and ambient signatures previously bypassed the return type check because it required a body. Reject their return annotations with TS1093, matching constructor implementations.
Highlight only the return type itself, excluding the colon and preceding whitespace.
Mainly for blockquote for Markdown.
```md
> If this line is long and to be broken, how printer know new line needs `> ` prefix?
```
This will also help JSDoc formatting (`* ` prefix) if we update it via IR, currently just splitting lines.
Pure refactor.
Split `find` module into parts.
- `find` directory contains find primitives (`find1`, `find2` etc) and specialized `find_*` functions.
- `scan` directory contains `scan_*` functions.
- `chunk` module contains primitives for reading and manipulating chunks (`load256`, `mm`, `veq` etc).
In `find` and `scan`, AVX2 and generic code is split into separate files, which both export the same functions.
Each component is now small enough to read as a whole.
This is pure refactor in the strictest sense. The ASM the crate compiles to remains identical to before this PR.
Follow up on #26469 for CSS.
Updated `oxc-css-parser` to `v0.0.15`.
---
- Keep parens for `calc((a + b))` by adding `CalcParenthesized` AST
- Do not follow JS formatter approach (remove all, then add back), only works for CSS, not for SCSS/Less
- Fix Less operand spacing: `2em*1em` -> `2em * 1em`
- Rework `@import` prelude fill
- Leave glued `and(` and `when(` as-is
- `@media screen and (max-width:1280px) { }` and `@media screen and(max-width:1280px) { }` is different
- We do not alter behavior
- Format `LessMixinDefinition` and `LessConditionalQualifiedRule`
- We still have a room to go better than Prettier here, but keep for now
- `substitute_typeof_undefined`: do not recreate binnary expression and
instead just update right/left argument and operator
- `substitute_rotate_binary_expression`: replace `take_in*2` and
reassign with `std::mem::swap`
- `substitute_object_or_array_constructor`,
`substitute_global_new_expression`: introduce new helper
`transform_new_expr_to_call_expr` to convert new to call expr
- `substitute_iife_call`: use array to alloc all items in sequence
without reallocs
Pure refactor. Small change to `oxc_lexer`.
`coalesce` stage selects a `KwSet` depending on source type - JS or TS. Move making this selection from `lex_raw` into `coalesce` itself. This just removes a little clutter from `lex_raw` and makes it clear that only `coalesce` stage uses it.
Join formatted termination methods directly when building diagnostic and help text, avoiding temporary vectors while preserving method order and wording.
Read the first role token directly instead of collecting every token into a vector. The rule continues to inspect only the first role, preserving its existing behavior for multi-role values.
Join the allowed attribute names directly when formatting missing-href help text, avoiding a temporary vector while preserving the diagnostic wording and attribute order.
Join formatted dependency names directly when building the missing-dependency diagnostic, avoiding a temporary vector while preserving dependency order and wording.
Compute deletion spans while building shared-code suggestions instead of collecting them into temporary vectors. This also defers span computation until the suggestion closure runs, preserving deletion order and replacement text.
Enable all 43 commented-out number-printing cases ported from esbuild, with expectations matching Oxc's existing output.
The old expectations preserved the input spelling, but Oxc uses the same value-based number printer with and without whitespace minification. It removes numeric separators and compares decimal, exponent, and hexadecimal spellings by length, keeping the current candidate on ties. For example, `1e-3` becomes `.001` (equal length), while `1000000000001` becomes `0xe8d4a51001` (shorter).
Large integer literals also reflect JavaScript `Number` rounding already applied during parsing. For example, `0xFFFF_FFFF_FFFF_FFFF` rounds to `0x10000000000000000`, so the changed spelling preserves the runtime value.
Reject `await using` in non-async functions and ordinary generators, including `for` and `for...of` loop heads. Treat top-level `await using` as module syntax when the source type is unambiguous.
Track line offsets while splitting multiline JSX text instead of collecting newline positions in a separate vector. Offsets remain byte-based, preserving fixes for Unicode text, blank lines, and trailing newlines.
Iterate matching module requests directly instead of collecting a temporary vector before counting loaded modules. Export traversal order and diagnostics remain unchanged.
Borrow the receiver source text and build argument text inside the suggestion closure. Join argument slices directly instead of collecting a temporary vector, preserving the generated replacement.
Reject invalid initializers such as `for (using x = a in b;;);` by parsing `using` and `await using` declarations with `Context::In` disabled, matching ordinary for-loop declarations. Restore the outer context afterward so parenthesized `in` expressions and the remaining loop expressions keep their existing behavior.
Pure refactor. No code changes, just moving code around.
Move `unicode_ws_len` out of `classify` stage. It's used across `carve`, `misc`, `find`, and `regex_div` - but not `classify` (since #26482).
Move it into `find.rs`, which contains all the other primitives reading from the source buffer.
`misc_pre` and `misc_post` both have early exit branches. `misc_pre` also dispatches to 2 different implementations depending on whether UTF-8 validation is required.
This logic was all in `Lexer::lex_raw`, mixed around the calls into other stages. Move it into the `misc` module, simplifying `lex_raw`.
The early exits are handled in the `misc_pre` / `misc_post` entry points (which are marked `#[inline]`), and only pass control into the main implementations `misc_pre_impl` / `misc_post_impl` if the early exit checks pass. ASM after this PR is completely unchanged from before, so this PR is pure refactor in the strictest sense.
Pure refactor. No code changes, just moving code around.
`misc_pre` and `misc_post` were part of `classify` stage. Move them into their own separate stage.
Splitting it up into `pre.rs` and `post.rs` files is likely overkill - they could live within a single file. It's just a bit tidier for each stage to have its own directory (following the pattern of `classify`, `carve`, `coalesce` and `compress`) and it'd be weird if that directory just contained a single `mod.rs` file.
Splitting it does have the advantage of making it clear which primitives each uses from the `use` statements at top of each file.
Join argument source slices directly when building the `Buffer.from` or `Buffer.alloc` suggestion, avoiding an intermediate vector while preserving the replacement text.
Stop checking JSDoc blocks once a non-ignored block is found instead of counting all matching blocks. In the yield-expression path, pass tags directly to the missing-tag check rather than collecting a temporary vector.
Allow escaped value names such as `import { t\u0079pe as x } from "m"` and the equivalent named exports. Reject escapes only after resolving `type` as a type-only modifier.
Preserve the decoded identifier name in the ambiguous `type as as` form so its AST name remains `type`.
`class C { async x; }` was accepted in JavaScript and TypeScript, silently discarding `async` from the resulting field. Reject the modifier after parsing a class field, once fields have been distinguished from methods.
Async methods, fields named `async`, and newline-separated fields remain valid.
Return early for calls whose last argument is not `undefined`, avoiding unnecessary callee and `bind` checks. Collect trailing undefined spans with pushes followed by one reversal instead of repeated front insertions, preserving diagnostic and fix order.
Require a comma after a TypeScript `this` parameter when the parameter list continues. Previously, malformed syntax such as `function f(this: T x: T) {}` parsed without a diagnostic.
Pure refactor.
Add `bm_clear` function that clears a bit in a bitmap at specified index. This is the converse of `bm_set`, and single-bit version of `bm_clear_range`.
This operation is performed in various places across `classify`, `carve`, and `coalesce` stages. Replace the duplicated code with `bm_clear` calls. This makes the logic easier to read.
`bm_clear` is `#[inline(always)]` so this change makes zero difference to codegen. The changes in `carve_jsx` look more substantive, but compiling to ASM and diffing it before/after this PR shows 0 difference. The compiler is able to identify the repeated terms across the 4 calls - there's no need to perform that optimization manually.
Code to read the nth bit of a bitmap was duplicated in several places across pipeline code:
- `wordbit` in `carve`
- `bit_at` in `regex_div`
- Various places inline in `carve`, `classify`, and `coalesce`
Add a function `bm_get` for this purpose to `bitmap` module (which feels like its natural home) and replace all usages with calls to this function.
This reduces repetition, and makes the call sites easier to read.
`bm_get` is marked `#[inline(always)]`, so extracting it into a function should have no effect on perf.
Samuel Attard