### What?
Adds a CI job that checks Turbopack compiles for
`wasm32-wasip1-threads`, so wasm portability
regressions are caught rather than rediscovered.
### Why?
Everything under `#[cfg(target_family = "wasm")]` is invisible to host
builds **and** to host clippy —
it is only checked when you deliberately build for the target. Four
separate defects in this stack were
caught only that way (a wrong build-script condition, two bad imports,
and a value that compiled but was
wrong). Without a gate, the next one lands unnoticed.
This also re-enables coverage that had been off for ~2 years: the old
`test-next-napi-bindings-wasi` job
was disabled with `if: false` pending napi-rs/napi-rs#2009, which closed
in April 2024.
### How?
Modelled on `rust-check` via `build_reusable.yml` (`needsRust`,
`skipInstallBuild`, `skipNativeBuild`),
so it does not pay for a JS build. Beyond `rustup target add` it needs
two things:
- **a WASI C toolchain**, because `lzzzz` (LZ4, via `turbo-persistence`)
and `zstd-sys` have C build
scripts. The SDK build is selected from `$RUNNER_ARCH` —
`build_reusable.yml` defaults to an arm64
runner, and an x86_64 clang fails there with `Exec format error` — with
a pinned sha256 per arch,
unpacked under `$RUNNER_TEMP` so the workspace stays clean.
- **emnapi**, because `next-napi-bindings`' build script calls
`napi_build::setup()`, whose wasi path
panics without `EMNAPI_LINK_DIR` — so it is required even for `cargo
check`. It is installed into a
scratch directory rather than the root `package.json`, because this job
runs with `skipInstallBuild`
and therefore never runs `pnpm install`.
Two things worth recording, both of which cost a CI round trip to find:
- pnpm is invoked from the repo root with `--dir`, not by `cd`-ing into
the scratch directory: corepack
resolves the pnpm version from the nearest `package.json`, and outside
the repo it picks the latest
pnpm (11.x), which cannot run on the pinned Node 20
(`ERR_VM_DYNAMIC_IMPORT_CALLBACK_MISSING`).
- the job is added to `tests-pass`, whose `needs:` list is what actually
blocks a PR — a job that runs
but is absent from that list looks like coverage while blocking nothing.
The `emnapi@2.0.0-alpha.4` pin is deliberate and is the one fragility
here: the archive must define
`emnapi_create_env` / `emnapi_delete_env`, which exists only in emnapi
v2, still a prerelease. Move to
the stable release once it ships.
<!-- fleet b6d0486f-97c7-42a7-bdaf-3490774cdec3 -->
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Deploy tests now use the checkout's built JavaScript packages when
`NEXT_TEST_VERSION` is unset. The harness reuses the isolated-test
packer, includes the tarballs in the deployment source, and installs
them through relative `file:` references.
The previous published-build workflow remains available through
`NEXT_TEST_VERSION`. For example, `NEXT_TEST_VERSION=canary` skips local
packing and tests the published canary. Exact versions and CI preview
tarball URLs remain supported. Existing CI deploy jobs already set
`NEXT_TEST_VERSION`, so their package selection does not change. The
harness skips local package preparation when it reuses an existing
deployment or invokes a custom deploy script, preserving those
workflows.
Staged monorepo peer dependencies use matching local versions so npm
accepts prerelease builds without conflicting direct-package overrides.
Fixture overrides retain their nested rules, including qualified
identity overrides with or without an explicit `.` entry. Configuration
updates preserve linked source files.
Deployments of local JavaScript packages use published SWC binaries
matching the local Next.js version. They do not include locally built
native binaries because those may target a different platform.
**Verification**
```
VERCEL_TURBOPACK_TEST_TEAM=vtest314-next-turbo-e2e-tests \
pnpm test-deploy-turbo test/e2e/app-dir/third-parties/basic.test.ts
```
```
VERCEL_TEST_TEAM=vtest314-next-turbo-e2e-tests \
pnpm test-deploy-webpack test/e2e/app-dir/third-parties/basic.test.ts
```
Both deploy modes passed. Additional temporary checks, removed
afterward, verified that:
- Deployments install the local JavaScript packages and use matching
local versions for monorepo peers.
- Qualified npm overrides preserve a nested `sharp@0.34.5` pin, both
with and without an explicit `.` entry. The check resolved `sharp` from
Next's own dependency context.
- Archive preparation leaves checkout manifests and tarballs unchanged.
Repacked archives preserve other package files, permissions, links, and
external peer declarations.
In #80281 we moved the README from
`../../packages/next/src/client/components/react-dev-overlay/README.md`
to `packages/next/src/next-devtools/README.md`, but did not update the
link in the contributing guide.
Fixed two typos in `contributing/core/testing.md`:
- `sett` → `setting` (`Consider also setting NEXT_E2E_TEST_TIMEOUT=0`)
- `again` → `against` (`run a test against both Turbopack and Webpack`)
### What?
Adds a `--deployable-tar` option to `pack-next` that writes generated
tarballs into the target project and patches package references to use
relative `file:` paths.
### Why?
Projects that need to deploy with locally packed Next.js packages should
be able to include those tarballs inside the project directory instead
of referencing tarballs from the Next.js checkout.
### How?
Resolves the target project package.json location, writes tarballs to a
sibling `tarballs/` directory, and maps package override paths to
project-relative references when patching package.json. Existing `--tar`
behavior continues to write to the repository tarballs directory.
### Verification
- `git diff --check`
- `git diff --cached --check`
- `pnpm prettier --with-node-modules --ignore-path .prettierignore
--write scripts/pack-next.ts scripts/pack-utils/patch-package-json.ts
contributing/core/testing.md contributing/core/developing.md`
- `npx eslint --config eslint.config.mjs --fix scripts/pack-next.ts
scripts/pack-utils/patch-package-json.ts contributing/core/testing.md
contributing/core/developing.md` (Markdown files were ignored because no
matching ESLint config was supplied)
- `pnpm pack-next --project ../sandbox/nextjs-duplicated-html-repro/
--deployable-tar`
<!-- NEXT_JS_LLM_PR -->
@wbinnssmith added these scripts a long time ago at my suggestion/urging, but I think they ended up bad for a couple reasons that were hard to foresee at the time:
- afsctool operates in-place and doesn't atomically write the compressed file, so if the process gets interrupted, your `target` directory is corrupted. https://github.com/Dr-Emann/applesauce is better for this reason.
- This doesn't acquire the cargo lock, and modifying files while cargo is running is a good way to get corruption of the `target` directory.
https://github.com/bgw/cargo-apfs-compress is my latest attempt at solving this, though I don't have a `LaunchAgents` config for it.
Regarding `node_modules`: pnpm creates reflinks from a shared global store on apfs. Trying to compress these reflinked files is just going to hurt you because it'll break the data deduplication that would've otherwise happened.
When running `next build --debug-prerender`, React owner stacks are now
captured and displayed in prerender error output. This makes it much
easier to diagnose which component triggered uncached I/O or accessed
request data without Suspense. Previously, `--debug-prerender` only
enabled source maps and disabled minification. Now it also auto-enables
`allowDevelopmentBuild` and sets `NODE_ENV=development`, which loads
React development builds where `captureOwnerStack()` is available.
The main challenge is that with `NODE_ENV=development`, both server and
client bundles include dev-only code paths (HMR, WebSocket connections,
dev overlay, debug channel, etc.) that expect a running dev server. We
don't want these when using `next start`. To solve this, we introduce
`process.env.__NEXT_DEV_SERVER`, an internal env var that is truthy only
during `next dev`. In client bundles, it's inlined at build time (`'1'`
for `next dev`, `''` for `next build`). In production server runtime
bundles, it's inlined as `''` for dead-code elimination. In development
server runtime bundles, it's left as a runtime check because those
bundles are shared between `next dev` (where it's set) and `next build
--debug-prerender` (where it's not). Meanwhile, `NODE_ENV` continues to
control React's dev/prod mode and error formatting, which is exactly
what we want for `--debug-prerender`.
This also replaces the previous `renderOpts.dev` / `workStore.dev`
pattern, which was unreliable because `RouteModule.isDev` was derived
from `NODE_ENV` at compile time. When `allowDevelopmentBuild` set
`NODE_ENV=development`, `isDev` would be compiled as `true` and
incorrectly activate all dev guards during `next start`.
Key changes:
- `config.ts` auto-enables `allowDevelopmentBuild` and sets
`NODE_ENV=development` when `--debug-prerender` is active
- `define-env.ts` inlines `__NEXT_DEV_SERVER` into all bundles (truthy
for dev, falsy for build) so dev-server features are dead-code
eliminated in production and `--debug-prerender` builds
- `next-dev.ts` and `next.ts` set `__NEXT_DEV_SERVER` in the process
environment for externalized server-side code
- `renderOpts.dev` and `workStore.dev` are removed — all consumers now
use `__NEXT_DEV_SERVER` (for dev-server features) or `NODE_ENV` (for
error formatting that should work in both dev and `--debug-prerender`
builds)
- `patch-error-inspect.ts` devirtualizes React server URLs in source map
URLs so they display as readable file paths
When digging into failing deployment tests, I often will re-run the test
itself without wanting to change anything about Next.js or the test
application, especially when I suspect a logic bug in the test itself or
need to figure out what's causing a flake.
This adds an environment variable that can be used to skip the
build/deploy step and will use an existing deployment URL to run the
tests against.
I suggest reviewing this [without
whitespace](https://github.com/vercel/next.js/pull/88829/changes?w=1).
This makes certain aggregations trivial since we are are guaranteed to only traverse relevant edges.
Use it to fix a bug in the async module identification logic. Previously we would aggregate async cycles after propagating 'asyncness' through the DFS post order traversal, but this could cause us to fail to propagate async to all reverse dependencies depending on which part of a cycle a node would happen to visit first.
By traversing starting from the async modules we just need to mark everything we find and not worry about cycles at all since the underlying DFS mechanism will ensure we don't loop. In addition to only requiring a single pass, this is guaranteed to visit fewer nodes in that single pass.
One caveat is that reverse traversals only work within a single `SingleModuleGraph` so a debug assert was added to prevent misuse
Closes#85988Closes#86391
Closes PACK-5806
Written with @bgw.
- Transitions to using `tsx` for script execution
- Replaces `patch-package` with TypeScript implementation
- Implements argument parsing and help message with `yargs`
- Uses `execa` for some command execution
Test Plan: Run `pnpm pack-next`, `pnpm swc-build-wasm`, `pnpm unpack-next`, and `pnpm sweep` without errors.
This reverts the bundler change part while we investigate source map
issue but keeps the externals optimize preventing the runtime bundle
from inflating back to over `4MB`.
This reverts commit 98b0700e97
This reverts commit 04c3f0b29c
### Improving Documentation
Hello, I’ve added missing punctuation and an inline code block.
The `package.json` file has an inline code block on line 42, but the one
in the heading was missing.
Co-authored-by: JJ Kasper <jj@jjsweb.site>
I commonly add a `debug` script that allows me to attach a debugger when running Next.js in the isolated test dirs.
Now we always add it and make it work just like `debug` from the monorepo. So after the test you can `cd` into the created test directory and run `pnpm debug dev`. `debug` is just an alias for `next` with some additional configurations:
- tracing deprecations
- no telemetry
- no canary checks
- debugger attachable
- source maps
`debug` also ensures the SWC version that was used during the test is used. Previously it used the cached version which isn't necessarily the one built in this specific worktree.
Noticed this after rebuilding on top of #71688
Most Linux distributions default to `gcc` for their default `cc`
implementation (e.g. via Debian's `build-essential` meta-package, which
we can probably implicitly assume is installed), which means that they
won't typically have `clang` installed.
`librocksdb-sys` seems to always depend on `clang` on Linux. It looks
like this might be a consequence of using
[rust-bindgen](https://github.com/rust-lang/rust-bindgen) (though I'm
confused why this didn't come up earlier), as it looks like rocksdb can
otherwise build with gcc.
This shouldn't be an issue for macos, since `clang` is the default `cc`
on that platform.
## What is `pack-next`?
This is a script we use for developing turbopack. It builds tarballs for
all the next.js packages, including `next-swc`, and then provides
instructions about how to modify a project's `package.json` to install
from that tarball.
## What does this PR do?
If you provide a `--project ~/path/to/directory` argument to `pnpm
pack-next`, we'll auto-update the `package.json` there for you, instead
of telling you how to do it manually!
This also understands the React 19 RC peer dependencies, which is
something that was missing from the generated instructions.
## Supported Package Managers
I've tested this with `npm` and `pnpm`.
- `bun` fails with `ENOTDIR` for unknown reasons (maybe it's trying to
handle the local tarball as a directory instead of a tarball?).
- `yarn` fails due to an upstream issue:
https://github.com/yarnpkg/yarn/issues/6339
This does generate the entries for yarn's `resolutions` feature,
assuming the upstream issue is eventually fixed.
## Why Rust and `cargo xtask`?
https://github.com/matklad/cargo-xtask
I'd love for these Turbopack-centric scripts to be written in Rust with
xtask, since they're mostly touched by the turbopack team.
However, it's also hard to justify putting too much effort into these
development scripts, so I don't want to rewrite the whole thing right
now.
I figured I'd write the new functionality in Rust and call it from the
old JS script.
## Test Plan
Perform a fresh checkout of shadcn/ui, and then:
```
pnpm pack-next --project ~/shadcn-ui/apps/www/
```
```
Found workspace! Patching "/home/bgw.linux/shadcn-ui/package.json" with overrides.
Successfully patched "/home/bgw.linux/shadcn-ui/package.json" to use local tarball files!
Run `pnpm i` or `npm i` to install the overrides
```
Check the resulting changes to `~/shadcn-ui`:
```diff
diff --git a/package.json b/package.json
index be47ad7..262c13b 100644
--- a/package.json
+++ b/package.json
@@ -63,12 +63,31 @@
"tsx": "^4.1.4",
"turbo": "^1.9.9",
"vite-tsconfig-paths": "^4.2.0",
- "vitest": "^0.31.1"
+ "vitest": "^0.31.1",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar"
},
"devDependencies": {
"@types/node": "^20.11.27",
"@types/react": "^18.2.65",
"@types/react-dom": "^18.2.22",
"typescript": "^5.5.3"
+ },
+ "overrides": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
+ },
+ "resolutions": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
}
}
```
### What?
Ports a few more useful scripts from nextpack, namely:
- `patch-next`
- `sweep`
- `build-native` (to clean up incremental artifacts on compiler panics)
- the macOS compression agent
Copies changes from https://github.com/vercel/turbo/pull/8166, and
updates contributing documentation to include the installation of lld.
> **What's wrong with `ld`?** It's very slow and uses a lot of memory.
>
> **Why `lld`?** It's fast, mature, and well-supported. Meta and Google
use it for all their linking workloads. We're already using it for macos
and x86-64 Windows. There is [ongoing work to make it the default for
rustc](https://github.com/rust-lang/rust/issues/71515), and it already
is default on a few platforms.
>
> **Why not `mold`?** Mold is generally faster, but the margin is slim
enough for our workloads that it doesn't really matter. Mold only
recently got support for LTO, doesn't support v0 rust symbol demanging,
doesn't support BOLT (though we don't use that yet), etc. Mold is
maturing quickly, but `lld` still seems like the "safer" choice.
This brings over the utility scripts `pnpm pack-next` and `pnpm
unpack-next path/to/app` from Nextpack, along with the documentation,
which has been added to `contributing/core/developing.md`.
~Due to the entry preloading that was enabled by default in #65289, VS
Code is no longer able to map symbols correctly for production builds.
As a temporary solution, we can disable the preloading when a debugger
is started until a proper fix is found.~
By optionally building the Next.js source code with
`NEXT_SERVER_EVAL_SOURCE_MAPS=1` we can ensure, even with the bundled
next server in the node runtime, that the original names are shown in
the debug "Variables" section.
In addition, the `sourceMapPathOverrides` are improved to cover the
different variations of source mapping URLs. We can now also set
breakpoints in the `example/...` and `test/e2e/...` server components.
~Supersedes #66229~
#### Before
<img width="1276" alt="Screenshot 2024-05-27 at 21 36 52"
src="https://github.com/vercel/next.js/assets/761683/f2840c35-f683-445d-bc95-cac9f719d8e7">
#### After
<img width="1276" alt="Screenshot 2024-05-27 at 21 38 42"
src="https://github.com/vercel/next.js/assets/761683/c801c123-3163-46c3-b442-5b72c0a5d51d">
When developing in Next.js repo, the maintainers / contributors
sometimes need to build swc native files.
Added a script `swc-build-native` to run the command `pnpm
--filter=@next/swc build-native` which was verbose to run.
GitHub recommends blobless clones over shallow clones:
https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/
> For these reasons we do not recommend shallow clones except for builds
that delete the repository immediately afterwards. Fetching from shallow
clones can cause more harm than good!
I've been using blobless clones for development for the last couple
weeks. The blobless clone has the benefit of including the full
repository history (for the cloned branch). Tools like `git blame` will
be slower as git fetches the related blobs on-demand.
Benchmarks (using all the flags in the docs):
- The blobless clone is faster on my machine, taking 11.1 seconds versus
13.1 seconds for the shallow clone.
- The blobless clone takes up 256M on disk, versus 244M for the shallow
clone. It's worse, but not by much.
## What?
Adds instructions for how to run tests with Turbopack and how to run
tests while recording using Replay.io.
I've also updated the instructions to mention `test-dev` and
`test-start` which are the commands we generally use.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-2031
This PR removes the wasm target for the next-swc build for the next major version. The main motivation is that Turbopack does not support targeting wasm yet, and it would be a significant amount of work to add. We plan to make Turbopack the default zero-config experience in a minor version, possibly before we are able to support a wasm target, so we need to make this breaking change now. We also plan to make more improvements to the webpack experience with shared Rust code, which we have so far been blocked from implementing because of the current wasm restrictions. We would like to support a wasm target again in the future, but cannot say at this time when that would be.
Closes WEB-1865
## What?
- Rendering `next info` output as code in the bug template
- Removed the "Feature Request" issue template, in favor of a more
granular discussion template
- Added a minimal template for Help discussions
## Why?
The main part of this PR is the new Feature Request template.
Anyone can propose a change to Next.js. However adding new features
often requires community discussions - whether to align expectations,
understand the consequences (eg.: backward compatibility/lifetime of a
feature), to look at and understand historical reasons for the current
behavior or lack of the feature -, which [GitHub
discussions](https://github.com/vercel/next.js/discussions) is more
suited for than issues.
## How?
- Utilizing the new discussion templates
https://docs.github.com/en/discussions/managing-discussions-for-your-community/creating-discussion-category-forms
- Adding a link to the [new
issue](https://github.com/vercel/next.js/issues/new/choose) view to make
it easier to discover
[Slack
thread](https://vercel.slack.com/archives/C04DUD7EB1B/p1670874727319809)
Closes NEXT-742
---------
Tobias Koppers
