### What?
Migrate the five local isolated test installs that explicitly used npm
to pnpm. The Nx fixture now has pnpm workspace metadata, while
filesystem-layout-sensitive fixtures use pnpm's hoisted linker with
copied package files.
### Why?
The npm-based Nx install bypassed the repository's centralized
supply-chain protections and could select a package immediately after
publication, including temporarily incomplete multi-package releases.
Using pnpm makes isolated installs inherit the repository's
`minimumReleaseAge`, exclusions, and exotic-subdependency policy.
The other npm installs depended on npm-style real package directories.
On Node versions affected by nodejs/node#65113, hoisted/copy mode
preserves that layout without leaving these fixtures outside the shared
pnpm security configuration; fixed Node releases use normal pnpm
linking.
### How?
- Use normal pnpm workspace resolution for the Nx fixture.
- Use `node-linker=hoisted` and `package-import-method=copy` for
filesystem tests only on affected Node releases; Node 24.21+ and 26.8+
use normal linking. Node 20 CI keeps the workaround because no fixed
Node 20 release exists.
- Validate local `@next/env` tarballs through the lockfile when hoisted
installs do not expose pnpm's virtual-store path marker.
- Keep the deployment-environment npm install unchanged.
### Verification
- `pnpm build-all`
- `pnpm types`
- A 9-version throwaway assertion verified the affected/fixed Node
release matrix
- `pnpm test-dev-turbo test/e2e/app-dir/nx-handling/nx-handling.test.ts
test/e2e/handle-non-hoisted-swc-helpers/index.test.ts
test/e2e/filesystem-cache/filesystem-cache.test.ts
test/e2e/filesystem-cache/warm-restart-task-stats.test.ts
test/e2e/filesystem-cache/evict-after-snapshot.test.ts` — all 25 tests
passed after installing the sandbox's missing Playwright browser
- Production Turbopack: Nx, non-hoisted SWC helper, build-cache-default,
and warm restart passed (9/9)
- `filesystem-cache.test.ts` production baseline: 15/17 passed; the same
two cache-growth bounds fail under both the unchanged npm fixture and
the pnpm fixture at nearly identical percentages, so they are
pre-existing sandbox-specific failures
- Generated-layout inspection: no package symlinks outside expected
`.bin` command shims; package files are copied; `node_modules/.pnpm` is
metadata-only
<!-- NEXT_JS_LLM -->
<!-- fleet 81cd457d-6956-4cf9-b6f6-9ebf9d95f285 -->
---------
Co-authored-by: vercel-fleet-prod[bot] <318278635+vercel-fleet-prod[bot]@users.noreply.github.com>
Co-authored-by: Tobias Koppers <1365881+sokra@users.noreply.github.com>
Became orphaned in https://github.com/vercel/next.js/pull/97792. Another
script and the test harness needed some utils from `next-stats-action`
so I moved that into `test/lib/link-packed-packages`.
Deploy tests now use the checkout's built JavaScript packages when
`NEXT_TEST_VERSION` is unset. The harness reuses the isolated-test
packer, includes the tarballs in the deployment source, and installs
them through relative `file:` references.
The previous published-build workflow remains available through
`NEXT_TEST_VERSION`. For example, `NEXT_TEST_VERSION=canary` skips local
packing and tests the published canary. Exact versions and CI preview
tarball URLs remain supported. Existing CI deploy jobs already set
`NEXT_TEST_VERSION`, so their package selection does not change. The
harness skips local package preparation when it reuses an existing
deployment or invokes a custom deploy script, preserving those
workflows.
Staged monorepo peer dependencies use matching local versions so npm
accepts prerelease builds without conflicting direct-package overrides.
Fixture overrides retain their nested rules, including qualified
identity overrides with or without an explicit `.` entry. Configuration
updates preserve linked source files.
Deployments of local JavaScript packages use published SWC binaries
matching the local Next.js version. They do not include locally built
native binaries because those may target a different platform.
**Verification**
```
VERCEL_TURBOPACK_TEST_TEAM=vtest314-next-turbo-e2e-tests \
pnpm test-deploy-turbo test/e2e/app-dir/third-parties/basic.test.ts
```
```
VERCEL_TEST_TEAM=vtest314-next-turbo-e2e-tests \
pnpm test-deploy-webpack test/e2e/app-dir/third-parties/basic.test.ts
```
Both deploy modes passed. Additional temporary checks, removed
afterward, verified that:
- Deployments install the local JavaScript packages and use matching
local versions for monorepo peers.
- Qualified npm overrides preserve a nested `sharp@0.34.5` pin, both
with and without an explicit `.` entry. The check resolved `sharp` from
Next's own dependency context.
- Archive preparation leaves checkout manifests and tarballs unchanged.
Repacked archives preserve other package files, permissions, links, and
external peer declarations.
Test fixtures that depend on monorepo packages (for example
`@next/third-parties` and `@next/mdx`) declared them at the `canary`
dist-tag, which installs the published npm canary instead of the build
under test. In deploy tests this also broke the remote install entirely:
the published canary's `peerDependencies` ranges (for example
`^16.0.0-beta.0`) reject the prerelease preview versions that
`NEXT_TEST_VERSION` installs for `next` (for example
`16.4.0-preview-<sha>-<date>`), so `npm install` failed with ERESOLVE.
Test dependencies can now be declared as `workspace:*`, which resolves
to the build from the current checkout in every test mode:
- dev/start: the locally packed tarball from `pack-for-isolated-tests`,
with a descriptive error when the package has no pack task or is not
part of the repository.
- deploy: the preview tarball of the tested commit when
`NEXT_TEST_VERSION` points at a preview build (preview tarballs already
rewrite their monorepo peer dependencies to the same preview URLs, so
peer resolution succeeds), falling back to the version in the worktree
otherwise. Private packages without published preview tarballs fail with
a descriptive error.
All existing tests that used `canary` for monorepo packages are migrated
to `workspace:*`.
- `detectContentType` test was too aggressive about asserting completion
in 1ms, let's give it several tries.
- We weren't correctly setting up pnpm overrides in tests, which breaks
things whenever canary comes out.
We also mitigate the security impact of e2e tests by running everything
in VMs, and by not reading caches in release jobs, but we should still
be using `minimumReleaseAge` in e2e tests.
Ideally we'd also use lockfiles here too, and I'll work on that next,
but that is likely a lot more complicated because we probably need some
way to merge lockfiles, since we can't pin the next.js and react
versions in every test, and we need a reasonable/efficient way to
bulk-update all the lockfiles for tests that need it.
Depends on https://github.com/vercel/next.js/pull/95668 for a bunch of
windows path representation fixes that this ends up exposing in CI,
because adding workspace files causes pnpm to change the absolute path
representation used in the Windows junction points that it creates.
### What?
Converts every test under `test/integration/` to an isolated test
running through `nextTestSetup` (under `test/e2e/`, `test/production/`,
`test/development/`, or `test/unit/`), then deletes `test/integration/`
along with the legacy CI orchestration that was specific to it.
- `test/integration/` removed entirely (~327 test suites)
- New isolated suites added across the existing folders:
- `test/e2e/` — 175
- `test/production/` — 130
- `test/development/` — 43
- `test/unit/` — 1
- `.github/workflows/build_and_test.yml` and `run-tests.js` no longer
have any `integration` branches
- `nextTestSetup` gained a `baseUrl` option on `next.browser()` so a
small number of tests that drive their own proxy/static-export server
can keep using `next.browser(...)` instead of importing `next-webdriver`
directly
### Why?
`test/integration/` predated `nextTestSetup` and ran tests directly
against the source checkout via custom helpers (`launchApp`,
`nextBuild`, `nextStart`, `runNextCommand`, `webdriver`, `fetchViaHTTP`,
…). Each suite hand-rolled its own dev/start/build orchestration,
fixture mutation, and process management.
The isolated test model used by the rest of the repo gives each suite an
isolated working directory containing a packed `next.tgz` install, a
uniform `next.start()` / `next.build()` / `next.fetch()` /
`next.browser()` API, and the same lifecycle for dev, start, and deploy
modes — so a single set of assertions covers all three. Deploy-mode
skips and per-feature gates are expressed declaratively
(`skipDeployment`, `disableAutoSkewProtection`, `if (skipped) return`)
instead of branching on `process.env`.
Removing `test/integration/` lets us:
- Delete the bespoke orchestration code in the CI workflow and
`run-tests.js`
- Run every converted suite consistently in dev, start, and deploy modes
(where applicable)
- Reproduce every test locally with the same `pnpm
test-{dev,start}-{turbo,webpack}` commands; no separate `integration`
path
- Open the door to running `test/production` against deployments in the
future (the converted suites already declare `skipDeployment` so they
can be flipped on)
### How?
Mechanical conversion per suite, with targeted clean-ups:
1. **Per-suite conversion.** Each
`test/integration/<name>/test/index.test.{js,ts}` was rewritten into a
single `<name>.test.ts` under the right folder based on what the
original exercised:
- `launchApp` / dev-only assertions → `test/development/`
- `nextBuild` + `nextStart` / start-only assertions → `test/production/`
- Both → `test/e2e/`
- The one pure jsdom render check (`link-without-router`) → `test/unit/`
2. **API mapping.** Custom helpers were replaced by `nextTestSetup`
equivalents: `launchApp` → `next.start()`, `nextBuild` → `next.build()`,
`runNextCommand` → `next.runCommand`, `fetchViaHTTP` → `next.fetch`,
`webdriver(...)` → `next.browser(...)`. Fixture mutations switched from
raw `fs.writeFile`/`fs.rename` to `next.patchFile` (with the 3-arg
`runWithTempContent` callback when the change has a defined scope) and
`next.deleteFile`.
3. **Deploy-mode handling.** Suites that can't run in deploy mode (use
`patchFile` / `next.build()` / depend on local CLI output) declare
`skipDeployment: true` and early-return on the `skipped` boolean. Suites
where Vercel's edge mutates URLs (`&dpl=`, immutable assets) declare
`disableAutoSkewProtection: true`.
4. **`next.browser({ baseUrl })`.** A handful of tests
(`prerender-export`, `cdn-cache-busting`, `preload-viewport`, both
`react-virtualized` suites) need to drive a separate server (a
static-export server or an `http-proxy` instance) rather than the
Next.js process. Instead of importing `next-webdriver` directly, those
tests now pass `{ baseUrl: <port|url> }` to `next.browser()`. For the
proxy cases, the proxy was moved into `server.js` inside the fixture and
`http-proxy` declared via the `dependencies` option of `nextTestSetup`,
so the test runs with a fully isolated dependency graph.
5. **CI clean-up.** With `test/integration` gone, the `test
integration*` jobs and `integration-tests-manifest`-related logic in
`.github/workflows/build_and_test.yml` were removed, and `run-tests.js`
no longer has the `integration` test-folder branch.
6. **Validation.** The PR was iterated against multiple full CI runs;
the remaining failures on the latest run are pre-existing flakes
(segment-cache 60s `act` timeouts in turbopack-prod) or transient
infrastructure issues unrelated to the conversion.
Nesting TUIs is hard but also a bit excessive since Turborepo's TUI
isn't interactive if `stdin` is ignored.
Co-authored-by: Copilot <copilot@github.com>
### What?
Eliminates the expensive temporary repo directory (`tmpRepoDir`) copy
during isolated test setup by leveraging a Turborepo `pack` task with
caching.
### Why?
The previous test isolation flow copied the entire `packages/` directory
to a temp location, mutated every `package.json` to rewrite workspace
dependency references, and ran `pnpm pack` sequentially for each
package. This added ~10s+ of overhead per test suite. With Turborepo
caching, repeated runs are 500ms (and can still be improved).
### How?
- Adds a `pack-for-isolated-tests` task to `turbo.json` (depends on
`build`, outputs `packed.tgz`)
- Adds a `pack-for-isolated-tests` script (`pnpm pack --out
./packed.tgz`) to every workspace package
- Simplifies `linkPackages` in `repo-setup.js` to scan for pre-built
tarballs instead of copying/rewriting/packing
- Updates `create-next-install.js` to run `pnpm turbo run pack`
(benefits from caching) and use package manager `overrides`
(pnpm/npm/yarn) to resolve transitive workspace deps from local tarballs
- Removes `tmpRepoDir` handling from `base.ts` and `run-tests.js`
**Before:** `createNextInstall` ~16s (copy + sequential pack)
**After:** `createNextInstall` ~5s (first run) / ~4s (cached)
<!-- NEXT_JS_LLM_PR -->
The file already had the `@ts-check` directive, so the IDE did show errors in that file. But since it was not included in the root `tsconfig.json`, those errors were not reported during CI runs.
One issue found this way was that the `related` flag was defined but never used. The referenced script was removed in #67644.
- Follow the naming conventions of other environment variables and prefix `TEST_WASM` with `NEXT_TEST_WASM` so that there's less chance of conflicts with user code
- Remove the logic in `setup-wasm.mjs` that modifies the repository (not great for running tests locally) and instead try to match the environment-variable-override behavior used for native (non-wasm) bindings via `NEXT_TEST_WASM_DIR`.
- Hard-fail if `NEXT_TEST_WASM` is set and we fail to load wasm bindings instead of silently using fallback logic that loads native bindings instead.
- We don't need to install `wasm-pack` with `curl | sh` for tests: `pnpm` already installs it (and pins a specific version).
- The whole dance of renaming the directory from `pkg` to `pkg-nextjs` is a hack we use when publishing the package, as we're building both "web" and "nextjs" targets. We don't need to do that for tests.
- Fix handling of `undefined` values in SWC options objects.
- Remove hacks for Next v12.2
This opts us out of using the native built-in plugins for Rspack for now.
The existing swc plugin we use for webpack does additional work beyond just swc transformation, including validating RSC components to make sure they don’t use React APIs unavailable in that context.
In total, the native plugins result in a less than 20% performance gain which, while significant, isn’t necessary as we focus on correctness for now.
If we decide the maintenance cost of maintaining multiple plugins in parity is worth the performance gain in the future, we can selectively re-enable some or all of these plugins.
Test Plan: `pnpm test-dev-rspack test/development/acceptance/server-component-compiler-errors-in-pages.test.ts` no longer fails by missing the invalid React api messages.
This adds a new package allowing users to opt-into using Rspack in place of webpack in Next.js. While there is some logic to handle rspack in core, the actual third-party dependencies are not included there, only conditionally loaded. This package:
- Includes the actual npm dependencies on `@rspack/core` and `@rspack/plugin-react-refresh`
- Sets the `NEXT_RSPACK` environment variable, which Next.js itself uses to load the dependencies
- It exists as a kind of no-op `withRspack` decorator. @timneutkens is this the best approach to this?
### What?
Improved trace source maps to return relative paths and always relative to process.cwd.
`turbopack://` urls in SourceMaps are always relative to the project root.
`file://` urls do no longer contain duplicate path segments.
Add a test case
Also:
* improve error when sourcemap lookup throws
I commonly add a `debug` script that allows me to attach a debugger when running Next.js in the isolated test dirs.
Now we always add it and make it work just like `debug` from the monorepo. So after the test you can `cd` into the created test directory and run `pnpm debug dev`. `debug` is just an alias for `next` with some additional configurations:
- tracing deprecations
- no telemetry
- no canary checks
- debugger attachable
- source maps
`debug` also ensures the SWC version that was used during the test is used. Previously it used the cached version which isn't necessarily the one built in this specific worktree.
Noticed while testing turbopack version of tests that we weren't loading
the correct swc binary since this path wasn't absolute and was just the
relative path in `node_modules`.
---------
Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
## What?
Before: 25.71s
After: 11.05s (-57%)
## How?
Currently the system for isolation looks like this:
- Copy `packages` folder to an isolated directory
- Run `pnpm pack` for all folders in `packages`
- Collect the pack files, add them as `dependencies` in package.json of
the isolated application
- Run `pnpm install`
Because the `next-swc` (Turbopack + SWC, yes we still need to rename the
package) binary file is quite large in development (900MB+) it means we
have to copy, then zip (pnpm pack), then unzip (pnpm install) that
binary, which takes about 3+ seconds in each step.
The change in this PR is to skip the copy/zip/unzip completely by
providing the folder path for the binary directly to Next.js, as it's a
binary we don't need the special isolation for this, it's already
standalone so running it directly allows us to skip 14,6 seconds of work
that is required for each isolated test in development.
This will likely have little effect on CI times as we already do some
tricks like only running the packing at the start of the CI process.
Only thing that could be better and is probably worth doing is adopting
this change for the time it saves for unzipping, that's almost 4 seconds
per test still.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-3200
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
## What?
In the past folks on the team have reported that anytime the SWC /
Turbopack binary changes they had to suddenly run `pnpm build-native` in
`packages/next-swc`, even though they didn't build from source before.
While investigating optimizing packing for tests I found that it's
copying the native binary into the original repository not the temporary
repository, which then keeps the file there, even after the tests
finish. This fixes the path to use the tmp repository instead.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-3157
Doesn't make any functional changes. Going through the current setup for isolated tests to figure out a better way to cache the repository setup, so that we don't have to wait ~30s+ when running tests locally.
Some tests may depend on `latest` to help us catch problems earlier. This adds support for the `resolutions` field for tests to allow pinning of deps that may be problematic upstream.
## Vendoring
Updates all module resolvers (node, webpack, nft for entrypoints, and nft for next-server) to consider whether vendored packages are suitable for a given resolve request and resolves them in an import semantics preserving way.
### Problem
Prior to the proposed change, vendoring has been accomplished but aliasing module requests from one specifier to a different specifier. For instance if we are using the built-in react packages for a build/runtime we might replace `require('react')` with `require('next/dist/compiled/react')`.
However this aliasing introduces a subtle bug. The React package has an export map that considers the condition `react-server` and when you require/import `'react'` the conditions should be considered and the underlying implementation of react may differ from one environment to another. In particular if you are resolving with the `react-server` condition you will be resolving the `react.shared-subset.js` implementation of React. This aliasing however breaks these semantics because it turns a bare specifier resolution of `react` with path `'.'` into a resolution with bare specifier `next` with path `'/dist/compiled/react'`. Module resolvers consider export maps of the package being imported from but in the case of `next` there is no consideration for the condition `react-server` and this resolution ends up pulling in the `index.js` implementation inside the React package by doing a simple path resolution to that package folder.
To work around this bug there is a prevalence of encoding the "right" resolution into the import itself. We for instance directly alias `react` to `next/dist/compiled/react/react.shared-subset.js` in certain cases. Other times we directly specify the runtime variant for instance `react-server-dom-webpack/server.edge` rather than `react-server-dom-wegbpack/server`, bypassing the export map altogether by selecting the runtime specific variant. However some code is meant to run in more than one runtime, for instance anything that is part of the client bundle which executes on the server during SSR and in the browser. There are workaround like using `require` conditionally or `import(...)` dynamically but these all have consequences for bundling and treeshaking and they still require careful consideration of the environment you are running in and which variant needs to load.
The result is that there is a large amount of manual pinning of aliases and additional complexity in the code and an inability to trust the package to specify the right resolution potentially causing conflicts in future versions as packages are updated.
It should be noted that aliasing is not in and of itself problematic when we are trying to implement a sort of lightweight forking based on build or runtime conditions. We have good examples of this for instance with the `next/head` package which within App Router should export a noop function. The problem is when we are trying to vendor an entire package and have the package behave semantically the same as if you had installed it yourself via node_modules
### Solution
The fix is seemingly straight forward. We need to stop aliasing these module specifiers and instead customize the resolution process to resolve from a location that will contain the desired vendored packages. We can then start simplifying our imports to use top level package resources and generally and let import conditions control the process of providing the right variant in the right context.
It should be said that vendoring is conditional. Currently we only vendor react pacakges for App Router runtimes. The implementation needs to be able to conditionally determine where a package resolves based on whether we're in an App Router context vs a Page Router one.
Additionally the implementation needs to support alternate packages such as supporting the experimental channel for React when using features that require this version.
### Implementation
The first step is to put the vendored packages inside a node_modules folder. This is essential to the correct resolving of packages by most tools that implement module resolution. For packages that are meant to be vendored, meaning whole package substitution we move the from `next/(src|dist)/compiled/...` to `next/(src|dist)/vendored/node_modules`. The purpose of this move is to clarify that vendored packages operate with a different implementation. This initial PR moves the react dependencies for App Router and `client-only` and `server-only` packages into this folder. In the future we can decide which other precompiled dependencies are best implemented as vendored packages and move them over.
It should be noted that because of our use of `JestWorker` we can get warnings for duplicate package names so we modify the vendored pacakges for react adding either `-vendored` or `-experimental-vendored` depending on which release channel the package came from. While this will require us to alter the request string for a module specifier it will still be treating the react package as the bare specifier and thus use the export map as required.
#### module resolvers
The next thing we need to do is have all systems that do module resolution implement an custom module resolution step. There are five different resolvers that need to be considered
##### node runtime
Updated the require-hook to resolve from the vendored directory without rewriting the request string to alter which package is identified in the bare specifier. For react packages we only do this vendoring if the `process.env.__NEXT_PRIVATE_PREBUNDLED_REACT` envvar is set indicating the runtime is server App Router builds. If we need a single node runtime to be able to conditionally resolve to both vendored and non vendored versions we will need to combine this with aliasing and encode whether the request is for the vendored version in the request string. Our current architecture does not require this though so we will just rely on the envvar for now
##### webpack runtime
Removed all aliases configured for react packages. Rely on the node-runtime to properly alias external react dependencies. Add a resolver plugin `NextAppResolverPlugin` to preempt perform resolution from the context of the vendored directory when encountering a vendored eligible package.
##### turbopack runtime
updated the aliasing rules for react packages to resolve from the vendored directory when in an App Router context. This implementation is all essentially config b/c the capability of doing the resolve from any position (i.e. the vendored directory) already exists
##### nft entrypoints runtime
track chunks to trace for App Router separate from Pages Router. For the trace for App Router chunks use a custom resolve hook in nft which performs the resolution from the vendored directory when appropriate.
##### nft next-server runtime
The current implementation for next-server traces both node_modules and vendored version of packages so all versions are included. This is necessary because the next server can run in either context (App vs Page router) and may depend on any possible variants. We could in theory make two traces rather than a combined one but this will require additional downstream changes so for now it is the most conservative thing to do and is correct
Once we have the correct resolution semantics for all resolvers we can start to remove instances targeting our precompiled instances for instance making `import ... from "next/dist/compiled/react-server-dom-webpack/client"` and replacing with `import ... from "react-server-dom-webpack/client"`
We can also stop requiring runtime specific variants like `import ... from "react-server-dom-webpack/client.edge"` replacing it with the generic export `"react-server-dom-webpack/client"`
There are still two special case aliases related to react
1. In profiling mode (browser only) we rewrite `react-dom` to `react-dom/profiling` and `scheduler/tracing` to `scheduler/tracing-profiling`. This can be moved to using export maps and conditions once react publishses updates that implement this on the package side.
2. When resolving `react-dom` on the server we rewrite this to `react-dom/server-rendering-stub`. This is to avoid loading the entire react-dom client bundle on the server when most of it goes unused. In the next major react will update this top level export to only contain the parts that are usable in any runtime and this alias can be dropped entirely
There are two non-react packages currently be vendored that I have maintained but think we ought to discuss the validity of vendoring. The `client-only` and `server-only` packages are vendored so you can use them without having to remember to install them into your project. This is convenient but does perhaps become surprising if you don't realize what is happening. We should consider not doing this but we can make that decision in another discussion/PR.
#### Webpack Layers
One of the things our webpack config implements for App Router is layers which allow us to have separate instances of packages for the server components graph and the client (ssr) graph. The way we were managing layer selection was a but arbitrary so in addition to the other webpack changes the way you cause a file to always end up in a specific layer is to end it with `.serverlayer`, `.clientlayer` or `.sharedlayer`. These act as layer portals so something in the server layer can import `foo.clientlayer` and that module will in fact be bundled in the client layer.
#### Packaging Changes
Most package managers are fine with this resolution redirect however yarn berry (yarn 2+ with PnP) will not resolve packages that are not defined in a package.json as a dependency. This was not a problem with the prior strategy because it was never resolving these vendored packages it was always resolving the next package and then just pointing to a file within it that happened to be from react or a related package.
To get around this issue vendored packages are both committed in src and packed as a tgz file. Then in the next package.json we define these vendored packages as `optionalDependency` pointing to these tarballs. For yarn PnP these packed versions will get used and resolved rather than the locally commited src files. For other package managers the optional dependencies may or may not get installed but the resolution will still resolve to the checked in src files. This isn't a particularly satisfying implemenation and if pnpm were to be updated to have consistent behavior installing from tarballs we could actually move the vendoring entirely to dependencies and simplify our resolvers a fair bit. But this will require an upstream chagne in pnpm and would take time to propogate in the community since many use older versions
#### Upstream Changes
As part of this work I landed some other changes upstream that were necessary. One was to make our packing use `npm` to match our publishing step. This also allows us to pack `node_modules` folders which is normally not supported but is possible if you define the folder in the package.json's files property.
See: #52563
Additionally nft did not provide a way to use the internal resolver if you were going to use the resolve hook so that is now exposed
See: https://github.com/vercel/nft/pull/354
#### additional PR trivia
* When we prepare to make an isolated next install for integration tests we exclude node_modules by default so we have a special case to allow `/vendored/node_modules`
* The webpack module rules were refactored to be a little easier to reason about and while they do work as is it would be better for some of them to be wrapped in a `oneOf` rule however there is a bug in our css loader implementation that causes these oneOf rules to get deleted. We should fix this up in a followup to make the rules a little more robuts.
## Edits
* I removed `.sharedlayer` since this concept is leaky (not really related to the client/server boundary split) and it is getting refactored anyway soon into a precompiled runtime.
This ensures we prefix the dynamic route params in the query so that
they can be kept separate from actual query params from the initial
request.
Fixes: https://github.com/vercel/next.js/issues/43139
This re-enables running the CNA tests for all non-docs changes as it's now ensuring we don't regress on turbopack or our default templates.
Also fixes the main repo's package.json files from being modified during tests.
- Move span wrapping rendering closer to the user code and don't add
span when we have cache-hit
- Add `getStaticProps` span
- Add spans around API handlers (pages and app)
- Add `generateMetadata` span
- Clarify naming that we use `page` for entrypoints like
`/path/[param]/page` or `/path/[param]/layout`. And `route` for
`/path/[param]`
fix NEXT-857 ([link](https://linear.app/vercel/issue/NEXT-857))
Noticed while testing across CI environments that the `test-pack` handling is not-concurrent or cache safe so this removes it from being used by default and moves it behind a flag to allow further investigating later.
In CI if `test-pack` is called at the same time due to multiple concurrency it can cause the resulted archive to be corrupted so may require a form of lock to resolve in CI although locally re-using the same archive name/path isn't safe with pnpm as it won't bust the store cache and continue to leverage previous cache.
This reverts commit fd9ec646ac.
It seems that there is an issue with caching of inputs which uses stale
files.
We also had to disable caching of `pnpm-lock.yaml` because it requires
stable package locations chich we removed.
Tobias Koppers