mirror of
https://github.com/oxc-project/oxc.git
synced 2026-09-14 19:36:11 +08:00
overlookmotel
99eef72e2d
Large change to how raw transfer stores metadata about the allocations it uses.
Previously the two metadata structures `RawTransferMetadata` and `FixedSizeAllocator` after the chunk's `ChunkFooter`.
This had a few problems:
1. It was confusing and unwieldy - a recipe for bugs.
2. `ChunkFooter`'s memory was exposed on JS side - a bit unsafe as altering bytes in this region could easily trigger UB.
3. It entwined `Arena` (which is just the thing that allocates) with the details of exactly _what_ raw transfer allocates.
4. Imposed annoying alignment requirements, because `ChunkFooter` must be aligned on 16, and so anything after it must ensure it doesn't break that invariant.
Old layout:
```
WHOLE BLOCK - aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ALLOCATOR
<-----------------------------------------> `Allocator` chunk (`CHUNK_SIZE` bytes)
<----> `ChunkFooter` (aligned on 16)
<-----------------------------------> `Allocator` chunk data storage (for AST)
(`ACTIVE_SIZE` bytes)
METADATA
<----> `RawTransferMetadata`
<----> `FixedSizeAllocatorMetadata`
BUFFER SENT TO JS
<-----------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
This PR moves `RawTransferMetadata` and `FixedSizeAllocatorMetadata` into the chunk itself. New layout:
```
WHOLE BLOCK - size 2 GiB - 16, aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ARENA
<-----------------------------------------------------> Chunk (fills whole block)
<--------------------------------------> Allocatable region for AST (`ACTIVE_SIZE` bytes)
<---> `RawTransferMetadata`
<---> `FixedSizeAllocatorMetadata`
<---> `ChunkFooter` (aligned on 16, last in block)
BUFFER SENT TO JS
<-------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
`FixedSizeAllocatorMetadata` and `ChunkFooter` are no longer in the region which is shared with JS side. As far as `Arena` is concerned, they're now just some data (like any other data) which is allocated in the arena.
Also:
- Introduce more consistency to the naming of constants which specify the size and position of these various data structures in the arena.
- Add more const assertions to ensure everything is laid out and aligned as it should be.
106 lines
2.8 KiB
JavaScript
106 lines
2.8 KiB
JavaScript
// Auto-generated code, DO NOT EDIT DIRECTLY!
|
|
// To edit this generated file you have to edit `tasks/ast_tools/src/generators/raw_transfer.rs`.
|
|
|
|
// See `crates/oxc_allocator/src/pool/fixed_size.rs` for a diagram showing
|
|
// how the constituent parts of the arena fit together.
|
|
|
|
/**
|
|
* Total size of the allocator block (including metadata and allocator `ChunkFooter`).
|
|
*/
|
|
export const BLOCK_SIZE = 2147483632;
|
|
|
|
/**
|
|
* Required alignment of the allocator block (4 GiB).
|
|
*/
|
|
export const BLOCK_ALIGN = 4294967296;
|
|
|
|
/**
|
|
* Total size of the transfer buffer used on JS side, in bytes
|
|
* (`BLOCK_SIZE` minus `FixedSizeAllocatorMetadata` and `ChunkFooter`).
|
|
*/
|
|
export const BUFFER_SIZE = 2147483576;
|
|
|
|
/**
|
|
* Size of the active data region in bytes - the region where source text and AST live
|
|
* (`BUFFER_SIZE` minus `RawTransferMetadata`).
|
|
*/
|
|
export const ACTIVE_SIZE = 2147483560;
|
|
|
|
/**
|
|
* Byte offset of the data pointer within the buffer, divided by 4 (for `Int32Array` indexing).
|
|
*/
|
|
export const DATA_POINTER_POS_32 = 536870890;
|
|
|
|
/**
|
|
* Byte offset of the `is_ts` flag within the buffer.
|
|
*/
|
|
export const IS_TS_FLAG_POS = 2147483572;
|
|
|
|
/**
|
|
* Byte offset of the `is_jsx` flag within the buffer.
|
|
*/
|
|
export const IS_JSX_FLAG_POS = 2147483573;
|
|
|
|
/**
|
|
* Byte offset of the `has_bom` flag within the buffer.
|
|
*/
|
|
export const HAS_BOM_FLAG_POS = 2147483574;
|
|
|
|
/**
|
|
* Byte offset of the tokens offset within the buffer, divided by 4 (for `Int32Array` indexing).
|
|
*/
|
|
export const TOKENS_OFFSET_POS_32 = 536870891;
|
|
|
|
/**
|
|
* Byte offset of the tokens length within the buffer, divided by 4 (for `Int32Array` indexing).
|
|
*/
|
|
export const TOKENS_LEN_POS_32 = 536870892;
|
|
|
|
/**
|
|
* Byte offset of the `program` field, relative to start of `RawTransferData`.
|
|
*/
|
|
export const PROGRAM_OFFSET = 0;
|
|
|
|
/**
|
|
* Byte offset of pointer to start of source text, relative to start of `Program`.
|
|
*/
|
|
export const SOURCE_START_OFFSET = 16;
|
|
|
|
/**
|
|
* Byte offset of length of source text, relative to start of `Program`.
|
|
*/
|
|
export const SOURCE_LEN_OFFSET = 24;
|
|
|
|
/**
|
|
* Byte offset of comments `Vec` pointer, relative to start of `Program`.
|
|
*/
|
|
export const COMMENTS_OFFSET = 32;
|
|
|
|
/**
|
|
* Byte offset of comments `Vec` length, relative to start of `Program`.
|
|
*/
|
|
export const COMMENTS_LEN_OFFSET = 40;
|
|
|
|
/**
|
|
* Size of `Comment` struct in bytes.
|
|
*/
|
|
export const COMMENT_SIZE = 16;
|
|
|
|
/**
|
|
* Byte offset of `kind` field, relative to start of `Comment` struct.
|
|
*/
|
|
export const COMMENT_KIND_OFFSET = 12;
|
|
|
|
/**
|
|
* Byte offset of the deserialized flag within each token/comment entry.
|
|
*
|
|
* Corresponds to `content` field of `Comment` struct, and unused bytes in `Token`.
|
|
* Initialized to 0 by Rust. JS side sets to 1 after deserialization.
|
|
*/
|
|
export const DESERIALIZED_FLAG_OFFSET = 15;
|
|
|
|
/**
|
|
* Discriminant value for `CommentKind::Line`.
|
|
*/
|
|
export const COMMENT_LINE_KIND = 0;
|