mirror of https://github.com/oxc-project/oxc.git synced 2026-09-14 19:36:11 +08:00

Files

T

History

Shanon Jackson fed8ff8753 fix(lexer): close remaining grammar gaps in context-dependent tokenization (#26377 )

Closes out remaining spec compliance gaps in the lexer, Regex vs
Division cases, `>>` cases, + many more.

Fully passes on Test262, All Typescript tests, Oxc tests, SWC tests,
VSCode (codebase), And 1_000 machine hours of differential fuzzing;
Confidence is high that it's either 'correct' or very close.

This PR specifically needs AI Disclosure in both finding and fixing the
cases now because the bitwise logic is complex; Something that I'll
start addressing once 'correctness' is locked in (and performance).

---------

Co-authored-by: overlookmotel <theoverlookmotel@gmail.com>

2026-09-07 11:12:57 +01:00

src

fix(lexer): close remaining grammar gaps in context-dependent tokenization (#26377 )

2026-09-07 11:12:57 +01:00

tests

fix(lexer): close remaining grammar gaps in context-dependent tokenization (#26377 )

2026-09-07 11:12:57 +01:00

Cargo.toml

feat(lexer): emit trivia-free spans in place of token starts (#24910 )

2026-07-29 00:15:11 +00:00

README.md

feat(lexer): add scalar lexer and close the remaining pathological regex vs division cases. (#24376 )

2026-07-12 00:08:57 +01:00

README.md

Oxc Lexer

A spec compliant standalone lexer for JS/JSX/TS.

Overview

Test262 compliant.
TS/JSX Working
UTF-8 validation costs 0.15CPB (Cycles per byte), optional parameter.
SIMD in x86_64 AVX2, Scalar for every other platform currently (in progress).
Regex vs Division cases should be handled even in pathological cases.