mirror of
https://github.com/oxc-project/oxc.git
synced 2026-09-14 19:36:11 +08:00
07a07933c7
Continuation of #26108. Previously we built the whole output as a single rope, and then flattened it in one go at the end. This is fine for smaller files, but for larger files it has 2 problems: 1. Each `output += segment` append allocates a 32-byte cons string cell. When printing a large AST, so many of these are allocated that they can fill "new space" and trigger garbage collection. When that happens, all these little cons strings have to be copied into the other half of new space, which is slow. Even worse, if they survive 2 rounds of GC, they get promoted into "old space" where (a) they may get scattered across memory, ruining cache coherence, and (b) they sit there, taking up memory, until the next major GC round. 2. Flattening a rope string which contains any segments which are 2-byte strings (non-Latin1 characters) is way more expensive than flattening a rope which is pure Latin1/ASCII. If a single segment contains Unicode, the entire flatten operation deopts and becomes several times slower. This PR attacks both these problems by flattening a large output in chunks. - When output reaches 16 KiB, flatten that as a chunk and store it in an array. - This leaves all the little cons string cells unreferenced, so garbage collection will discard them all, instead of copying them across to keep them alive. - If file contains some non-Latin1 characters, that does still heavily impact the performance of flattening the chunk they're in, but only that chunk - other chunks which are pure ASCII still take the fast path. The 16 KiB limit is a heuristic, chosen after experimentation with different values. ### Benchmarks The effect of this change is minimal on small files, but dramatic on large ones (negative is faster): | Fixture | Bytes | Change | |:---| ---:| ---:| | `tiny.js` | 26 | \+3.2% | | `RadixUIAdoptionSection.jsx` | 2,424 | \-0.1% | | `react.development.js` | 50,496 | \+2.0% | | `binder.ts` | 126,212 | \-1.9% | | `lodash.js` | 182,262 | \-32.5% | | `App.tsx` | 298,130 | \-26.4% | | `kitchen-sink.tsx` | 662,560 | \-14.2% | | `antd.js` | 5,100,647 | \-28.4% | ### Total cost of flattening After this PR, compared to where we were prior to #26106 (positive is slower): | Fixture | Bytes | Change | |:---| ---:| ---:| | `tiny.js` | 26 | \+51.5% | | `RadixUIAdoptionSection.jsx` | 2,424 | \+134.2% | | `react.development.js` | 50,496 | \+35.5% | | `binder.ts` | 126,212 | \+39.0% | | `lodash.js` | 182,262 | \+45.3% | | `App.tsx` | 298,130 | \+55.0% | | `kitchen-sink.tsx` | 662,560 | \+51.3% | | `antd.js` | 5,100,647 | \+2.2% | The effect of flattening is still very negative, but (as mentioned in #26106) it was always a cost, just one that we were ignoring. At least we've now managed to mitigate it. In the case of really large files (`antd.js`), benchmark is pretty much unchanged from before flattening came into the picture. But now our measure _includes_ the large flattening cost which was previously _on top of_ the cost of printing - so the real cost for the user is much reduced from where it was previously. ### Sourcemap builds Printing with sourcemaps on large files actually gets faster after this change. It was already paying the cost of string flattening, now it pays less. Printing `lodash.js` with sourcemaps is about 15% faster now.
206 lines
7.8 KiB
TypeScript
206 lines
7.8 KiB
TypeScript
// Printer state.
|
|
//
|
|
// One object, created here and handed to whichever printer build `index.ts` selects.
|
|
// It is deliberately outside `print/`, so that all 4 builds share this one class and therefore one
|
|
// object shape - a build-time flag must never add or remove a field, or the printers would each
|
|
// see a different hidden class for the object they thread through every function.
|
|
//
|
|
// The printers only ever receive it, so they import it as a type.
|
|
// Nothing in `print/` constructs one, and none of the 4 printer builds bundles this file.
|
|
|
|
import { CAT_OTHER } from "./print/categories.ts";
|
|
import { debugAssert } from "./asserts.ts";
|
|
|
|
import type { Category } from "./print/categories.ts";
|
|
import type { Options } from "./print/options.ts";
|
|
|
|
/**
|
|
* The string one level of indentation prints as. Set from the `indent` option, a tab by default.
|
|
*/
|
|
let indentString = "\t";
|
|
|
|
/**
|
|
* Indentation cache, holding the string for each level - `indents[2]` is two levels' worth.
|
|
*
|
|
* There is one per process, and every `State` carries it, so all 4 printer builds grow and read
|
|
* the same array - the cache a build fills is there for the next build to use.
|
|
*
|
|
* It survives across prints and is discarded only when `indentString` changes.
|
|
*/
|
|
const indents = [""];
|
|
|
|
/** The `indent` option must be a non-empty string made up of only spaces and tabs. */
|
|
const INDENT_REGEX = /^[ \t]+$/;
|
|
|
|
/** Upper bound for the process-wide indentation cache. */
|
|
const MAX_STARTING_INDENT_LEVEL = 1_000;
|
|
|
|
/**
|
|
* Initial capacity of the process-wide mapping-positions buffer.
|
|
* Must be even - positions are recorded in pairs, and the buffer only grows between pairs.
|
|
*/
|
|
const MIN_MAP_POSITIONS_LENGTH = 1024;
|
|
|
|
debugAssert(
|
|
MIN_MAP_POSITIONS_LENGTH % 2 === 0,
|
|
"`MIN_MAP_POSITIONS_LENGTH` must be an even number",
|
|
);
|
|
|
|
/**
|
|
* Deferred source mapping positions - pairs of output offset and source offset.
|
|
*
|
|
* Like `indents`, there is one buffer per process, and every maps-enabled `State` carries it,
|
|
* so both sourcemap builds record into the same buffer.
|
|
*
|
|
* `growMapPositions` doubles it when a print fills it.
|
|
*/
|
|
let mapPositions = new Int32Array(MIN_MAP_POSITIONS_LENGTH);
|
|
|
|
export class State {
|
|
// Current output.
|
|
// A string which is appended to as the printing process proceeds.
|
|
declare output: string;
|
|
|
|
// Flattened chunks of `output`, in order, once it has grown past `OUTPUT_CHUNK_LENGTH` (see `print/flatten.ts`),
|
|
// or `null` while it has not - a small print never allocates the array. Sourcemap builds also keep the total
|
|
// length spilled so far - the true output offset the recorders need is `spilledOutputLength + output.length`.
|
|
declare outputChunks: string[] | null;
|
|
declare spilledOutputLength: number;
|
|
|
|
// Current indentation level.
|
|
declare indentLevel: number;
|
|
|
|
// The process-wide indent cache and the string one level of it prints as.
|
|
// `printIndent` reads the cache through here, and `growIndents` extends it.
|
|
declare indents: string[];
|
|
declare indentString: string;
|
|
|
|
// `true` if printing a JSX / TSX file. Only affects printing of TSX.
|
|
declare isJsx: boolean;
|
|
|
|
// Operator/token glue tracking.
|
|
declare pendingIndentAsSpace: boolean;
|
|
|
|
// Category of the last thing written - one of the `CAT_*` constants, never a character.
|
|
//
|
|
// It also carries the three `CAT_START_OF_*` markers, which say that an expression statement,
|
|
// a concise arrow body or an `export default` expression is about to be printed, and nothing
|
|
// has been written since.
|
|
//
|
|
// A node seeing one of those is the leftmost token of that construct, however deeply nested it is,
|
|
// which is how an object literal, a function or a class knows to parenthesize itself.
|
|
//
|
|
// They work because the next write replaces `last`, so "nothing written since" needs nothing else recorded -
|
|
// and because every position they mark follows whitespace, which is what the `CAT_OTHER` they displace
|
|
// means for spacing.
|
|
declare last: Category;
|
|
|
|
// `true` between a `writeNoLast` and the write which follows it,
|
|
// i.e. while `last` describes something other than what was written last.
|
|
// Only used in debug builds. See `debugAssertLastFresh`.
|
|
declare lastIsStale: boolean;
|
|
|
|
// The character the output currently ends with.
|
|
// Only used in debug builds. See `debugAssertCategoryMatches`.
|
|
declare lastCharWritten: string;
|
|
|
|
// Deferred source mappings. When source maps are enabled, `mapPositions` is the process-wide
|
|
// buffer above, holding generated/source offset pairs, and `mapPositionsLen` is how much of it
|
|
// this print has filled.
|
|
// Names are sparse, so their index/name pairs exist only if a mapping carries an original name.
|
|
declare mapPositions: Int32Array | null;
|
|
declare mapPositionsLen: number;
|
|
declare mapNames: (number | string)[] | null;
|
|
|
|
// Original source text, used to preserve names in source maps when the caller provides it.
|
|
declare sourceText: string | null;
|
|
|
|
constructor(options: Options) {
|
|
this.output = "";
|
|
this.outputChunks = null;
|
|
this.spilledOutputLength = 0;
|
|
|
|
let { startingIndentLevel: indentLevel } = options;
|
|
if (indentLevel === undefined) {
|
|
indentLevel = 0;
|
|
} else if (
|
|
!Number.isSafeInteger(indentLevel)
|
|
|| indentLevel < 0
|
|
|| indentLevel > MAX_STARTING_INDENT_LEVEL
|
|
) {
|
|
throw new RangeError(
|
|
"`startingIndentLevel` must be a non-negative safe integer no greater than 1000",
|
|
);
|
|
}
|
|
this.indentLevel = indentLevel;
|
|
|
|
// The `indent` option is validated here, not in the printer, and changing of it discards
|
|
// the cache grown for the old `indentString`.
|
|
// That should be rare - most users have an indent style they prefer, and use it consistently.
|
|
const { indent: indentOption } = options;
|
|
let indent = indentOption;
|
|
if (indent === undefined) {
|
|
indent = "\t";
|
|
} else if (typeof indent !== "string" || !INDENT_REGEX.test(indent)) {
|
|
throw new TypeError("`indent` must be a non-empty string containing only spaces and tabs");
|
|
}
|
|
if (indent !== indentString) {
|
|
indentString = indent;
|
|
indents.length = 1;
|
|
}
|
|
this.indents = indents;
|
|
this.indentString = indent;
|
|
|
|
// `.tsx` mode: lone type parameters print as `<T,>`
|
|
this.isJsx = options.jsx === true;
|
|
|
|
// Operator/token glue tracking
|
|
this.pendingIndentAsSpace = false;
|
|
|
|
// Start of output behaves like after whitespace. Reading the last character from `output`
|
|
// directly would flatten V8's rope representation on every append-then-read (quadratic),
|
|
// which is why the category is tracked rather than derived.
|
|
this.last = CAT_OTHER;
|
|
|
|
// Debug-only fields for checking `last` is correct on both writes and reads
|
|
if (DEBUG) {
|
|
this.lastIsStale = false;
|
|
this.lastCharWritten = "";
|
|
}
|
|
|
|
// `writeWithMap*` functions record the output offset and original position of every mapped node,
|
|
// and `generateSourceMap` encodes them in one pass at the end
|
|
if (options.sourcemap !== true) {
|
|
this.sourceText = null;
|
|
this.mapPositions = null;
|
|
this.mapNames = null;
|
|
} else {
|
|
debugAssert(options.sourceText != null);
|
|
this.sourceText = options.sourceText;
|
|
this.mapPositions = mapPositions;
|
|
this.mapNames = [];
|
|
}
|
|
this.mapPositionsLen = 0;
|
|
}
|
|
|
|
/**
|
|
* Replace the full mapping-positions buffer with one twice the size, copying its contents over.
|
|
*
|
|
* The new buffer also replaces the process-wide one, so later prints start at the new size.
|
|
*
|
|
* @returns The new buffer
|
|
*/
|
|
growMapPositions(): Int32Array {
|
|
const oldBuffer = this.mapPositions;
|
|
debugAssert(
|
|
oldBuffer === mapPositions,
|
|
"`growMapPositions` should only be called on a `State` with source maps enabled",
|
|
);
|
|
const newBuffer = new Int32Array(oldBuffer.length * 2);
|
|
newBuffer.set(oldBuffer);
|
|
mapPositions = newBuffer;
|
|
this.mapPositions = newBuffer;
|
|
return newBuffer;
|
|
}
|
|
}
|