Large change to how raw transfer stores metadata about the allocations it uses.
Previously the two metadata structures `RawTransferMetadata` and `FixedSizeAllocator` after the chunk's `ChunkFooter`.
This had a few problems:
1. It was confusing and unwieldy - a recipe for bugs.
2. `ChunkFooter`'s memory was exposed on JS side - a bit unsafe as altering bytes in this region could easily trigger UB.
3. It entwined `Arena` (which is just the thing that allocates) with the details of exactly _what_ raw transfer allocates.
4. Imposed annoying alignment requirements, because `ChunkFooter` must be aligned on 16, and so anything after it must ensure it doesn't break that invariant.
Old layout:
```
WHOLE BLOCK - aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ALLOCATOR
<-----------------------------------------> `Allocator` chunk (`CHUNK_SIZE` bytes)
<----> `ChunkFooter` (aligned on 16)
<-----------------------------------> `Allocator` chunk data storage (for AST)
(`ACTIVE_SIZE` bytes)
METADATA
<----> `RawTransferMetadata`
<----> `FixedSizeAllocatorMetadata`
BUFFER SENT TO JS
<-----------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
This PR moves `RawTransferMetadata` and `FixedSizeAllocatorMetadata` into the chunk itself. New layout:
```
WHOLE BLOCK - size 2 GiB - 16, aligned on 4 GiB
<-----------------------------------------------------> Allocated block (`BLOCK_SIZE` bytes)
ARENA
<-----------------------------------------------------> Chunk (fills whole block)
<--------------------------------------> Allocatable region for AST (`ACTIVE_SIZE` bytes)
<---> `RawTransferMetadata`
<---> `FixedSizeAllocatorMetadata`
<---> `ChunkFooter` (aligned on 16, last in block)
BUFFER SENT TO JS
<-------------------------------------------> Buffer sent to JS (`BUFFER_SIZE` bytes)
```
`FixedSizeAllocatorMetadata` and `ChunkFooter` are no longer in the region which is shared with JS side. As far as `Arena` is concerned, they're now just some data (like any other data) which is allocated in the arena.
Also:
- Introduce more consistency to the naming of constants which specify the size and position of these various data structures in the arena.
- Add more const assertions to ensure everything is laid out and aligned as it should be.
Continuation of #21132. All uses of the `Uint32Array` view of the buffer in raw transfer code have been removed in preceding PRs. So we can now remove it entirely.
Use `int32` (`Int32Array`) instead of `uint32`(`Uint32Array`) for getting data from buffer in raw transfer deserializer. The buffer is 2 GiB in size, so all offsets within the buffer can be represented as a `u31` which can be stored in an `i32` without any loss, and with no risk of being interpreted as negative numbers.
Same as in #21129, the advantage of fetching offsets from an `Int32Array` is that V8 statically knows the value can be stored in an SMI, its native integer type, without any range checks.
Previously we had to store source text at start of buffers sent to JS via raw transfer.
#18376 made changes to how raw transfer deserializer handles strings, in order to support files containing a BOM. Building on that, we're now able to remove the requirement that source text be at start of the buffer entirely.
This PR changes the deserializer used in Oxlint JS plugins to accept source text being anywhere in the buffer, *as long as no other strings are after it*. In practice this just means that the source text must be allocated before anything else, which is easy to satisfy.
Now the source text can be allocated with just the usual safe `allocator.alloc_str(source_text)` method.
This change removes a ton of dodgy workarounds and unsafe code we used previously to get source text at the start of buffer. It makes the code less labyrinthine and far less likely a slip up can inadvertently introduce UB.
Note: In `napi/parser`, source text still *is* at start of the buffer, as that's simpler and more efficient when the source text is written into the buffer on JS side. This change only affects Oxlint.
Add error messages to all `debugAssert()` calls, for easier debugging.
When running Oxlint normally, we can easily figure out where errors are coming from stack traces, but in conformance test snapshot, line and column are removed from errors in the snapshot, so it's less easy.
Add options to the parser used in `RuleTester`. Currently `RuleTester` doesn't actually use these options, but this prepares the way for other changes to `RuleTester` in future PRs.
The extra types for `Comment` etc in `bindings.d.ts` are just the result of using `oxc_napi` crate for its `get_source_type` function. We don't use those types, but they're also not exposed externally, so it doesn't matter.
Boshen