skills/oxc-project__oxc
1
0
Fork 0
mirror of https://github.com/oxc-project/oxc.git synced 2026-09-14 19:36:11 +08:00
Code Issues Packages Projects Releases Wiki Activity
19,977 Commits 481 Branches 542 Tags
codex/wtf8-str-poc
Commit Graph

9 Commits

Author SHA1 Message Date
overlookmotel 603b904d72 perf(packages/codegen): check sourceText only once (#25887) 
Follow-on after #25585. Small optimization.

Sourcemaps can't be generated unless `options.sourceText` is provided. This is already checked at the start. So everywhere else can then rely on `state.sourceText != null`, rather than re-checking it on each write.
 2026-08-19 08:12:02 +00:00
overlookmotel 8783524015 perf(packages/codegen): remove loop from growMappingBuffer (#25886) 
Follow-on after #25585. Small optimization.

Simplify `growMappingBuffer` by relying on existing invariants. Buffer always has at least 64 bytes capacity, and no caller needs to reserve more than 64 bytes. So doubling capacity is always sufficient.

Add a debug assert to ensure this invariant continues to hold.
 2026-08-18 17:25:19 +00:00
overlookmotel b6b14c0629 perf(packages/codegen): remove redundant branch (#25885) 
Follow-on after #25585. Small optimization.

The function exits early if `sourceText === undefined`, so no need to check this again later.
 2026-08-18 17:25:18 +00:00
overlookmotel 6a7eb602ed feat(packages/codegen): alter capitalization of sourceFilename option (#25854) 
Follow-on after [#25585](/github/pr-or-issue/oxc-project/oxc/25585).

Rename `options.sourceFileName` to `sourceFilename`. We use both capitalizations internal Rust code, but always `filename` in external JS APIs (Oxlint plugins etc).

NOTE: this is not marked as a breaking change as it was merged before the feature was released.
 2026-08-18 13:16:15 +00:00
overlookmotel e1085e5185 refactor(packages/codegen): move consts to top of file (#25850) 
Follow-on after #25585.

Pure style change. Move consts up to top of file.
 2026-08-18 12:45:34 +00:00
overlookmotel 6803a46b5e refactor(packages/codegen): add line breaks (#25849) 
Follow-on after #25585.

Pure style change. Similar to #25848 - just remove the worst excesses of LLM coding style where it produces dense blocks of code with no line breaks, which makes code hard to read.

No substantive changes, just adds line breaks.
 2026-08-18 12:45:34 +00:00
overlookmotel 58f7ab9c1d docs(packages/codegen): reformat docs and comments (#25848) 
Follow-on after #25585.

Pure style change. I hate the way LLMs put line breaks in weird places like after "the" - it makes comments hard to read. And what's with all the colons and semicolons?

No substantive changes, just reformatting comments and docs to avoid this unreadable formatting.
 2026-08-18 12:24:56 +00:00
camc314 300763f328 feat(packages/codegen): add full source map support (#25585) 
## Summary

Align `oxc-codegen` source maps with Rust `oxc_codegen`, using the Rust implementation as the conformance oracle.

This PR:

- changes `printSync` to return `{ code, map? }`;
- returns a standard Source Map v3 object;
- exposes Rust’s generated source map through `codegen_conformance`;
- compares the complete JS and Rust source maps;
- supports Oxc `start`/`end` offsets and TS-ESTree `range` offsets;
- uses `sourceText` to convert UTF-16 offsets and recover original identifier spellings;
- validates both maps and no-maps builds in both `preserveParens` modes;
- aligns mapping placement across JavaScript, TypeScript, JSX, templates, imports, destructuring, delimiters, and postfix chains;
- handles astral characters and all ECMAScript line terminators.
`loc`-only ASTs are not supported for source maps.

## API change

```ts
const { code, map } = printSync(program, {
  sourcemap: true,
  sourceFileName: "input.js",
  sourceText,
});
```

`map` is a Source Map v3 object:

```ts
interface SourceMap {
  version: 3;
  mappings: string;
  names: string[];
  sources: string[];
  sourcesContent?: string[];
}
```

## Implementation

Mapped writes collect only:

- the generated UTF-16 offset;
- the original source offset;
- the optional original identifier name.

After printing, the generator builds line tables for the generated output and `sourceText`, converts the offsets to line/column positions, suppresses duplicate source positions, and encodes the result as base64 VLQ.

The no-maps builds continue to remove mapping-specific writes at build time.

## Validation flow

```mermaid
flowchart TB
    fixture["Fixture<br/>filename + sourceText + parse options"]

    fixture --> rustParse
    fixture --> jsParse

    subgraph rust["Rust oracle"]
        direction TB
        rustParse["Parse with oxc_parser"]
        normalize["Normalize Rust-only AST information"]
        rustPrint["Print with oxc_codegen<br/>source maps enabled"]
        rustResult["Expected result<br/>{ code, map }"]

        rustParse --> normalize
        normalize --> rustPrint
        rustPrint --> rustResult
    end

    subgraph js["JavaScript candidate"]
        direction TB
        jsParse["Parse ESTree with oxc-parser<br/>start/end or range offsets"]

        plainPrint["Print with no-maps build"]
        plainResult["Actual code"]

        mappedPrint["Print with maps-enabled build"]
        deferred["Record generated offsets,<br/>source offsets, and names"]
        encode["Convert UTF-16 offsets to positions<br/>deduplicate and encode VLQ"]
        mappedResult["Actual result<br/>{ code, map }"]

        jsParse --> plainPrint
        plainPrint --> plainResult

        jsParse --> mappedPrint
        mappedPrint --> deferred
        deferred --> encode
        encode --> mappedResult
    end

    rustParse --> parseGate
    jsParse --> parseGate

    rustResult --> codeGate
    plainResult --> codeGate
    mappedResult --> codeGate

    rustResult --> mapGate
    mappedResult --> mapGate

    parseGate["Parse agreement<br/>Rust success requires no JS errors"]
    codeGate["Code equality<br/>both JS builds equal Rust"]
    mapGate["Source Map v3 equality<br/>mappings + names + sources + content"]

    parseGate --> pass["Fixture passes"]
    codeGate --> pass
    mapGate --> pass
```

Each fixture runs with both `preserveParens: false` and `preserveParens: true`.

## AST normalization

Rust’s AST contains information that ESTree cannot represent. The Rust AST is normalized before printing so the comparison measures code generation rather than representation differences.

This includes:

- parenthesized function and arrow-expression flags;
- `import "m"` versus `import {} from "m"`;
- absent versus empty import-attribute clauses;
- `assert` versus `with` attribute keywords;
- the unavailable ESTree span for a complete `with` clause.
 2026-08-18 11:41:34 +00:00
overlookmotel a4478e960a feat(codegen): add oxc-codegen package (#25488) 
Add `oxc-codegen` package - a printer written in TS.

### Features

- Prints JS, JSX, TS, TSX.
- Matches `oxc_codegen` exactly on all Test262, TypeScript, Acorn-JSX fixtures.
- Very fast.

### Deficiencies

- Pretty-printing only (no compact/minified output).
- No support for printing comments.
- Source map support is semi-there, but largely untested, and poor API (design doc has some ideas on how to improve it).

### This PR adds

- The printer code.
- Conformance testing infra.
- Further tests for edge cases.
- README.
- Design doc - how it differs from `oxc_codegen` and why, what produces the perf.

### This PR doesn't add

- Not integrated into CI or release pipeline.

### Notes

This started as a slop port of `oxc_codegen`. I've iterated on that base, cleaned it up a lot, and at least skimmed through all of it. I think it's in fairly decent shape.

The design doc is I think worth a read. It explains the rationale for the choices, in particular the "1 codebase, 4 builds" approach.

I've put it in a new top-level directory called `packages`. It's not an app, so didn't seem to me to fit alongside `apps/oxlint` and `apps/oxfmt`. It's pure JS, so not like e.g. `napi/parser`. And it felt too heavyweight to put into `npm` dir. But maybe there's a better place for it.

Co-authored-by: Cameron <cameron.clark@hey.com>
 2026-08-12 13:41:31 +00:00