Commit Graph

7 Commits

Author SHA1 Message Date
Jimmy Miller e928f2395f Migrate napi-rs bindings from v2 to v3 (#95412)
Draft PR for testing

<!-- NEXT_JS_LLM_PR -->
2026-08-13 11:35:52 -04:00
Sebastian "Sebbie" Silbermann 6ab0dd6fcd [ci] Install Node.js from the official tarball in the native-builder image (#96298)
Fixes e.g.
https://github.com/vercel/next.js/actions/runs/30314671467/job/90137708228#step:12:603

The `native-builder` image (`scripts/native-builder.Dockerfile`), used
by the `build-native` job of `build_and_deploy.yml` to cross-compile
`next-swc`, installed Node.js 20 via the NodeSource setup script. That
script downloads and imports a signing key before configuring the apt
repository. When the key download returns an HTTP 403 from CI runners
the script still exits successfully, so `apt-get install nodejs` falls
back to Ubuntu's stock `nodejs` package, which does **not** bundle
`npm`. The failure only surfaced two layers later when the image ran
`npm i -g @napi-rs/cli` (`npm: not found`, exit 127), breaking the
native build.

This change installs Node.js 20 from the official `nodejs.org` static
tarball instead. The tarball bundles `npm` and `corepack`, is
glibc-linked (matching the previous intent), and does not depend on
NodeSource's apt repository or key server, removing the flaky external
dependency. The install step also asserts `npm --version`, so a future
regression fails at the install layer rather than several layers later.


## test plan

- [x] [full build_and_deploy on this
branch](https://github.com/vercel/next.js/actions/runs/30372888336)
2026-07-28 15:21:20 -04:00
Zack Tanner cabf0a4197 [ci]: pin rust-musl-cross (#93176)
Pins these docker dependencies. Follow-up from:
https://github.com/vercel/next.js/pull/93164#discussion_r3135035714

x-ref: 

https://github.com/rust-cross/rust-musl-cross/pkgs/container/rust-musl-cross/741104753?tag=x86_64-unknown-linux-musl

https://github.com/rust-cross/rust-musl-cross/pkgs/container/rust-musl-cross/741104508?tag=aarch64-unknown-linux-musl
2026-04-24 02:36:18 +00:00
Zack Tanner 82b320addf [ci]: switch to GitHub runners (#93164)
## Summary

Switch CI from self-hosted runners to GitHub-hosted runners and fix the
follow-up issues that showed up once the hosted jobs were exercised.

## Changes

- Move Linux workflows onto `ubuntu-latest-16-core-oss`, Windows
workflows onto `windows-latest-8-core-oss`, and the native mac release
lane onto `macos-15-intel`.
- Update the reusable build workflow to work in hosted environments by
relying on `runner.os` instead of label string matching, and by removing
the hardcoded `/home/runner` `fnm` path.
- Make Turbo/sccache configuration explicit for hosted runners:
  - unify on `vtest314-next-adapter-e2e-tests`
- pass `TURBO_TOKEN` via workflow env/secrets instead of assuming
runner-level env
- switch Turbo cache mode to `local:rw,remote:rw` so jobs still have
local cache behavior when remote cache is unavailable
- Update the `sccache` action defaults/docs to use the hosted-runner
secret setup (`TURBO_TOKEN`) instead of the old self-hosted token
wiring.
- Make Datadog reporting fail open so missing `DATA_DOG_API_KEY` does
not block CI.
- Fix the CLI reserved-port test so it asserts Next’s own `-p 1`
validation instead of failing earlier on hosted Linux due to
privileged-port binding.
- Replace the live `musl.cc` dependency in `native-builder.Dockerfile`
with musl sysroots imported from GHCR-hosted `rust-musl-cross` images,
keeping the existing `/opt/*-cross` layout used by the Linux native
build scripts.
- Fix the macOS native build workaround s so it applies to all
*-apple-darwin targets based on the Cargo target OS rather than the host
architecture, which avoids napi-build injecting the unsupported -Wl
linker arg when @next/swc is linked with rust-lld on hosted mac runners.

Test Plan:
- This PR's CI
- This
[build_and_deploy](https://github.com/vercel/next.js/actions/runs/24865715983/job/72801225722)
job running on every arch
2026-04-23 18:53:07 -07:00
Matt Mastracci 3e847dfe0b Switch to cargo-binstall and pre-built sccache binaries (#92911)
## What

- Use `cargo binstall` instead of `cargo install` for Rust tools in the
Docker image
- Download pre-built vercel/sccache from GitHub releases instead of
compiling from git

## Why

`cargo install` compiles from source which is slow on cache miss. `cargo
binstall` downloads pre-built binaries in seconds. The vercel/sccache
fork now publishes release binaries, eliminating the need for the
`build-sccache` turbo task (~2-3 min compile).

## Changes

**Dockerfile:**
- Install cargo-binstall at pinned version (1.18.1), use it for
cargo-rustflags
- Remove sccache from Docker image (not needed inside container)

**CI sccache:**
- Download pre-built binary from `vercel/sccache` GitHub releases
(~11MB)
- Cache in `~/.cache/sccache-<version>` on self-hosted runners
- Delete `scripts/build-sccache.js`, remove turbo task and package.json
entry
- `scripts/sccache-version` now contains just the release tag

<!-- NEXT_JS_LLM_PR -->
2026-04-17 17:09:11 +00:00
Matt Mastracci f98a3f1101 [turbopack] Rust dependency caching via sccache (#91873)
## What

Use sccache, backed by the Vercel artifact cache, when compiling Rust
code.

## Why

Each CI runner compiles ~960 external Rust crates from scratch (~5 min)
on binary cache misses, even though `Cargo.lock` rarely changes between
commits. Some workflows don't cache and always build from scratch.

## Notes

Builds will use a local cache on each runner, but we've got a lot of
these runners and they'll take some time to populate that local, fastest
level of cache.

Windows builds are currently hitting ~68% of cached compilations for a
reason that isn't totally clear. Linux builds are caching correctly,
however. A follow-up pass at this work will use an incremental turborepo
caching strategy to improve this, and another pass will experiment with
cross-compiling Windows from Linux which will likely also help.

Linux builds that don't change Rust files at all are nearly
instantaneous (as before), and Linux builds that change Rust files will
complete in approximately 1/2 the time. `check` builds are ~1min faster
(~25% faster).

LTO-heavy runs like the benchmark builders are currently still slow
(approx unchanged), needs further investigation (may be related to git
hash embedding?).
2026-04-09 18:42:03 +00:00
Matt Mastracci b163a8bf66 [turbopack] Rebuild the docker build scripts (#91799)
## Why

The native binary build system uses 3 different napi-rs Docker images
that are opaque, can't be customized, and require runtime toolchain
downloads. Each CI run downloads compilers and sysroots from scratch,
wasting ~5 minutes per build. There's no way to customize the build
environment or share toolchains across the 4 Linux targets.

Some smoke tests are added to ensure that node.js can actually load the
binaries we build - we don't actually perform much beyond that, just
making sure it survives the dynamic linker and correctly exposes the
node APIs.

We also are able to enable `--icf=all`. This appears to (consistently)
chop off about ~1-2% of the binary size.

## What

Replaces the 3 napi-rs Docker images with a single custom
`next-swc-builder` Dockerfile based on Ubuntu 20.04 that can
cross-compile all 4 Linux targets (x86_64/aarch64 × gnu/musl) from
either an x86_64 or aarch64 host. All toolchains are baked into the
image.

Key changes:
- **`docker/native-builder.Dockerfile`**: Ubuntu 20.04 with clang/lld,
GNU cross-sysroots via `crossbuild-essential`, musl sysroots from
musl.cc, Node.js 20, pinned Rust nightly, and `cargo-rustflags` for flag
resolution
- **`scripts/docker-native-build.sh`**: Inner build script that
constructs RUSTFLAGS via `cargo rustflags --config` (merges
`.cargo/config.toml` with cross-compilation overrides), sets up CC/CXX
per target, and runs the napi build
- **`scripts/docker-native-build.js`**: Node.js local build orchestrator
replacing the bash wrapper, with `--quick`, `--rebuild`, `--test` flags
and smoke testing via `getTargetTriple()`
- **`scripts/docker-image-cache.js`**: Turbo-cached Docker image
(save/load with `--load` for cache hit handling)
- **`.cargo/config.toml`**: Added musl `crt-static` flag, removed
cross-compilation flags (now in build script)
- **CI workflows**: Docker image cached via turbo remote cache,
`build-docker-image` turbo task, simplified `build_and_deploy.yml`

## How

All 4 Linux targets use clang as the C compiler + linker driver with
`gnu-lld-cc` flavor (rust-lld does actual linking). Cross-compilation
uses `--target` + `--sysroot`:
- **GNU targets**: Ubuntu multiarch sysroots from
`crossbuild-essential-{amd64,arm64}`
- **Musl targets**: musl.cc cross-toolchain sysroots with GCC crt files
copied into sysroot lib (works around clang 10's `--gcc-toolchain`
limitation)

RUSTFLAGS are resolved inside the container via `cargo rustflags` which
merges `.cargo/config.toml` (base flags like `-Zunstable-options`, musl
`crt-static`) with per-target cross flags passed as `--config` inline
TOML.

The Docker image is cached via turbo's remote cache (~885 MB tar). On
cache hit, `docker load` restores it in seconds. On miss, the full image
build takes ~2 minutes.

### glibc compatibility

Building on Ubuntu 20.04 (glibc 2.31) ensures broad compatibility. The
table below shows the minimum glibc version required by major Linux
distributions ([source](https://repology.org/project/glibc/versions)):

| Distribution | glibc | Compatible? |
|---|---|---|
| Ubuntu 20.04 (Focal) | 2.31 |  baseline |
| Ubuntu 22.04 (Jammy) | 2.35 |  |
| Ubuntu 24.04 (Noble) | 2.39 |  |
| Debian 11 (Bullseye) | 2.31 |  |
| Debian 12 (Bookworm) | 2.36 |  |
| RHEL / AlmaLinux 8 | 2.28 |  EOL May 2029 (security only) |
| RHEL / AlmaLinux 9 | 2.34 |  |
| Amazon Linux 2 | 2.26 |  EOL June 2026 |
| Amazon Linux 2023 | 2.34 |  |
| Fedora 40+ | 2.39+ |  |
| Arch Linux | 2.41+ |  |
2026-03-24 13:07:37 -06:00