This work introduces the new concept of **Partial Fallback Prerendering
(PFPR)**.
Traditionally, when a dynamic page needed to be routed to that wasn't
pregenerated, it required a render to generate even the first few bytes
of the static page itself. This resulted in slow page loads for pages
not frequently visited and a reduced Time to First Byte (TTFB) score on
Core Web Vitals (CWV).
PFPR takes advantage of the new systems of Partial Prerendering (PPR)
that allows the application to suspend at different points mid-render,
and resume it later. We mark any unknown parameter access as dynamic
access, and suspend the rendering up to the next suspense boundaries at
those points. Under ideal conditions (correctly placed `<Suspense />`
boundaries or `loading.jsx` files) this generates a static shell that
can be served to users as soon as the request hits Next.js, right out of
the static cache. This minimizes the TTFB for all requests, dynamic or
not for those pages that enable PPR. For example, the following page
would create a usable shell:
```jsx
// /app/users/[userID]/page.jsx
import { Suspense } from 'react'
function Profile({ params }) {
const { userID } = params
return <div>Hello {userID}!</div>
}
export default function ProfilePage({ params }) {
return (
<div>
<h1>User Profile</h1>
<Suspense fallback="Loading...">
<Profile params={params} />
</Suspense>
</div>
)
}
```
Due to the way that suspense works within React components, access of
params within the root page component would cause the whole page to
suspend. Thankfully, that's where the `loading.jsx` comes in handy.
Adding a `loading.jsx` at a segment will automatically wrap the
`page.jsx` with a suspense boundary, setting the contents of the root
`loading.jsx` as the fallback component to use for it. This lets you
maintain your existing style of accessing parameters at the root of the
components while also taking advantage of PFPR.
To enable this feature, you first need to enable both PPR and PFPR:
```js
module.exports = {
experimental: {
ppr: true,
pprFallbacks: true,
}
}
```
Once PFPR has stabilized with hosting providers, the experimental flag
will go away and it will become the default with the PPR flag.
...with `assertHasRedbox` and `assertNoRedbox`.
`hasRedbox()` has a hardcoded timeout of 5s that is only required for
the negative assertion.
Instead, we now have dedicated assertions for the positive
(`assertHasRedbox`) and negative case (`assertNoRedbox`).
The negative assertion still has the hardcoded timeout.
But the positive assertion just retries until we find the Redbox.
This speeds up tests using the positive assertion.
Removing `hasRedbox` also uncovered some unused expressions e.g. `await
hasRedbox(browser)`.
These expressions probably wanted to use `expect(await
hasRedbox(browser)).toBe(true)
### What
Inherit the client boudaries from the found matched target in load
barrel
### Why
The root cause with the barrel transform, we missed the client boundary
directive during the transform.
Since the new version of mui's case looks like this:
Import path
page.js (server module) -> `<module>/index.js` (shared module) ->
`<module/subpath>/index.js` (client module) ->
`<module/subpath/sub-module.js> (client module)
After our transform, we lost the `"use client"` which causes the
mismatch of the transform:
In `rsc` layer: the file pointing to the sub module entry
(`<module/subpath>/index.js`), but without the client boundary.
Fixes#64369
Closes NEXT-3101
### What?
Pass the names of side-effect-free packages specified in `experimental.optimizePackageImports`.
Turbopack counterpart: https://github.com/vercel/turbo/pull/7731
### Why?
Some packages like `@tremor/react` causes a problem without `optimizePackageImports`.
### How?
Closes PACK-2527
We switched to `pnpm` for testing instead of `yarn` for e2e tests
containing customized node_modules packages, it works with
`node_modules` folder before. Rename the existing `node_modules_bak`
hack to make it easy to test with.
Previously we use `yarn` but it will clean `node_modules` folder so we
have to use another script to copy packages, it's not a required thing
now so we can test package from node_modules directly without renaming
folders locally
Closes NEXT-2496
As Barrel Optimization might split one file into multiple different
modules, i.e. when you import different values from it, the target file
might be transformed differently, we can no longer rely on the file path
as the identifier of the client reference.
This fix adds a suffix (`'@' + this._module.matchResource`) to the
identifier so it looks like
`/filepath/file.js@__barrel_optimize__?names=Foo`.
Here's also a quick diagram to explain:
Closes#59804.
Closes NEXT-2108.
---------
Co-authored-by: Jiachi Liu <inbox@huozhi.im>
Fixes#57624. The recent issue was an unexpected side effect caused by
https://github.com/vercel/next.js/commit/305bb015060e82be3e6ae59a3d063c11a3e207f9,
which only affects specific packages like `@mui/material`.
The problem was that the entry file of `@mui/material` has `"use
client"` at top, which affects the compilation result to output
reference info only (when on the RSC layer), instead of keeping the
original export statements. And the fix here is to ignore all layer info
and React specific transforms here, as barrel optimization isn't related
to all these framework features at all. To keep all directives
unchanged, the SWC transform needs to parse and pass that info to the
Webpack loader.
This PR adds a test to ensure that `@mui/material` is working as
expected (less than 1500 modules compiled). Without this feature it'll
be ~2400 modules.
Closes NEXT-1793, closes NEXT-1762.
With Barrel Optimization, we are trying to "shortcut" the import to be
directly pointing to the final target, as barrel files shouldn't have
any side effects. However, there could be `"use client"` and `"use
server"` directives which we can't ignore. Hence we must apply the
`serverComponents` transform of SWC to these barrel files too.
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
This PR flattens the recursive optimization logic of our barrel optimization loader. So now if there're any recursive `export * from ...`, they won't be created as multiple individual Webpack modules, but optimized in one module.
With this change, we are running SWC transform to get the export map directly inside the barrel loader (instead of a separate loader rule). And that map is recursively calculated and cached in memory.
I also published https://unpkg.com/browse/recursive-barrel@1.0.0/ to give this a test. It contains 4 levels of 10 `export *` expressions, which is 10,000 modules in total. Without this change, it takes ~30s to compile and with this change, it goes down to ~7s.
The barrel optimization loader creates a virtual module to re-export
from the original file, which causes the situation that now there are 2
modules with the same resource but only one of them is the actual code.
When the code contains `"use client"`, the Flight plugin has to collect
its `buildInfo` and generate the manifest and client entry module.
However, we currently deduplicate module by resource in the traversal
logic to avoid unnecessary loops. To make it work together with the
virtual barrel module, we'll need to prefix the resource path to make it
different from the original module.
Closes#54967.
Closes#55609.
Closes#55566.
This fix ensures the case that if you `import { foo } from './index'`,
where:
index.js:
```js
export * from './subpkg'
```
subpkg.js:
```js
const unrelatedExpressions = ...
export { foo } from './foo'
export { bar } from './bar'
```
Previously we'll transform the proxy to the second module to `export {
foo } from './subpkg'`. With this fix it will be correctly optimized as
`export { foo } from './foo'` which is a shorter path.
## Initial Pass
The initial pass applies to all user code. If an import (`foo`) matches one of the packages from our list, say:
```js
// user code
import { a } from 'foo'
import { otherThings } from './code'
```
The initial pass transforms it into:
```js
// user code
import { a } from '__barrel_optimize__?names=a!=!foo'
import { otherThings } from './code'
```
## Barrel Optimizations
This `__barrel_optimize__` loader is used to optimize the imports of "barrel" files that have many
re-exports. Currently, both Node.js and Webpack have to enter all of these
submodules even if we only need a few of them.
For example, say a file `foo.js` with the following contents:
```js
export { a } from './a'
export { b } from './b'
export { c } from './c'
...
```
If the user imports `a` only, this loader will accept the `names` option to
be `['a']`. Then, it request the `__barrel_transform__` SWC loader to load
`foo.js` (via `this.loadModule` Webpack API) and receive the following info:
```js
export const __next_private_export_map__ = '[["./a","a","a"],["./b","b","b"],["./c","c","c"],...]'
```
The export map, generated by SWC, is a JSON that represents the exports of
that module, their original file, and their original name (since you can do
`export { a as b }`).
Then, this loader can safely remove all the exports that are not needed and
re-export the ones from `names`:
```js
export { a } from './a'
```
That's the basic situation and also the happy path.
## Wildcard Exports
For wildcard exports (e.g. `export * from './a'`), it becomes a bit more complicated.
Say `foo.js` with the following contents:
```js
export * from './a'
export * from './b'
export * from './c'
...
```
If the user imports `bar` from it, SWC can never know which files are going to be
exporting `bar`. So, we have to keep all the wildcard exports and do the same
process recursively. This loader will return the following output:
```js
export * from '__barrel_optimize__?names=bar&wildcard!=!./a'
export * from '__barrel_optimize__?names=bar&wildcard!=!./b'
export * from '__barrel_optimize__?names=bar&wildcard!=!./c'
...
```
The "!=!" tells Webpack to use the same loader to process './a', './b', and './c'.
After the recursive process, the "inner loaders" will either return an empty string
or:
```js
export * from './target'
```
Where `target` is the file that exports `bar`.
## Non-Barrel Files
If the file is not a barrel, we can't apply any optimizations. That's because
we can't easily remove things from the file. For example, say `foo.js` with:
```js
const v = 1
export function b () {
return v
}
```
If the user imports `b` only, we can't remove the `const v = 1` even though
the file is side-effect free. In these caes, this loader will simply re-export
`foo.js`:
```js
export * from './foo'
```
Besides these cases, this loader also carefully handles the module cache so
SWC won't analyze the same file twice, and no instance of the same file will
be accidentally created as different instances.
---
- [x] Disable this loader for Jest
- [x] Add tests for SWC loader caching
- [x] Add tests for external modules
- [x] Add tests for pages
- [x] Add tests for recursive `export *`s
---
Closes#54038, closes#54286.
Sebastian "Sebbie" Silbermann