mirror of https://github.com/oxc-project/oxc.git synced 2026-09-14 19:36:11 +08:00

Files

T

History

Shanon Jackson 275c2f8302 fix(lexer): regex vs division when ASI ends the previous statement (#25355 )

`prev_is_regex` walks back one token to decide whether `/` opens a
regex. When a newline forces ASI but the previous token still looks
expression-final, it answered division. The JSX `<` decision goes
through the same function, so TSX was wrong in the same places.

```ts
let x: T
/re/.exec(s)      // was lexed as division

let x
/re/.test(s)

import y from 'm'
/re/.exec(s)

break outer
/re/.test(x)

let x: 1
/re/.exec(s)

let x!: T
/re/.exec(s)
```

Also `type A = T`, `declare function f(): T`, `function f(): void {}`,
template literal types, and `continue label`.

Free on the hot path: ASI needs a newline before the `/`, so the
backward walks sit behind an `lt_in_range` gate that straight-line `a /
b` never passes. No measurable change over the fixture corpus.

18 tests added.

2026-08-07 14:04:14 +01:00

src

fix(lexer): regex vs division when ASI ends the previous statement (#25355 )

2026-08-07 14:04:14 +01:00

tests

refactor(lexer): replace token kind constants with a TokenKind enum (#24931 )

2026-07-29 00:15:12 +00:00

Cargo.toml

feat(lexer): emit trivia-free spans in place of token starts (#24910 )

2026-07-29 00:15:11 +00:00

README.md

feat(lexer): add scalar lexer and close the remaining pathological regex vs division cases. (#24376 )

2026-07-12 00:08:57 +01:00

README.md

Oxc Lexer

A spec compliant standalone lexer for JS/JSX/TS.

Overview

Test262 compliant.
TS/JSX Working
UTF-8 validation costs 0.15CPB (Cycles per byte), optional parameter.
SIMD in x86_64 AVX2, Scalar for every other platform currently (in progress).
Regex vs Division cases should be handled even in pathological cases.