The hoisting of the proxy expressions only needs to be done for the
server layer. We can skip this part completely when we know we're
compiling a module for the client layer.
This fixes the server reference ID generation in client modules so that
they can be import and called from client components. In addition,
support for cached functions with decoupled export declarations is
added.
This PR fixes the transform of handling `export default async function()
{}` and `export const foo = async function() {}` cases (both are
exported `fnExpr`, instead of `fnDecl`) in a `"use cache"` annotated
file.
This adds support of collecting and binding encrypted closure variables
to references of `"use cache"` functions. These closures can be mixed
with `"use server"` in any order, too.
Check the tests for more information.
Adds the following missing pieces to Turbopack:
- apply server action transforms to route handler modules
- generate client reference manifests for route handler app endpoints
- make `experimental.cacheLife` config serialisable, to be injected into
the edge route handler modules
- the `nextConfig` injection was already implemented in:
#71258
I pulled out the core change from
https://github.com/vercel/next.js/pull/71155
- Refactor resolve functions to accept a `is_optional` boolean instead
of an explicit severity
- Add `ResolveOptions::loose_errors` to only emit warnings instead of
errors
Not sure if `loose_errors` is the best name. `warn_instead_of_errors`,
`emit_warnings_only` ?
### What?
This PR removes special exports from tree shaking to apply it to more
modules
### Why?
Previously tree shaking pass simply gave up if there's a special export.
### How?
This PR adds support for showing the original server sources of server
components and server actions in the browser's developer tools.
To accomplish that, we're building on top of #69190, #70563, and #70564,
in which we prepared the proper generation of the source maps.
This PR completes the feature, by wiring up `findSourceMapURL` (see
https://github.com/facebook/react/pull/30741) with a new dev server
middleware that serves those source map files to the browser.
Another change we had to make is disabling the dead-code elimination of
server actions in client bundles (see #70103), but only in development
mode. This optimization is primarily intended for production bundles, so
the change should not pose a problem.
With that, it's now possible to jump directly into the server sources
from the browser's dev tools, e.g. for component stacks of server logs
that are replayed in the browser, or when inspecting server action props
in the React DevTools.
### What?
Before we did invalidate all filesystem tasks when an incremental build starts. But that has a performance linear to the number of filesystem tasks. We like to avoid that.
This PR refactors that and introduces a new dirty state, which means "task is dirty in general, but considered as clean for a certain session". All filesystem tasks are marked this way. When an incremental build starts we only increment the session id, which makes all filesystem tasks considered as dirty. No looping over all filesystem tasks to make them dirty anymore.
### Why?
Most of the cost before was spend in updating the task aggregation when task where marked dirty. This requires to restore a lot of aggregated tasks. This cost has moved to the point when the filesystem task is recomputed. So it's per page. This has benefits for next dev which doesn't build all pages.
This PR fixes the cache function transform on the client layer, which
should be transpiled into a Server Reference with proper sourcemap
parameters, similar to Server Actions.
This allows bundle analyzer tools to visualize and explore the output of
Next.js apps built with Turbopack.
Like Turbopack’s support for manifest files, this creates individual
stats files per entry point and merges them into a single stats file for
the app.
To do:
- [x] Make opt-in and build-only
- [x] Apply to pages router
Test Plan: `TURBOPACK_BUILD=1 TURBOPACK=1 pnpm build && pnpm dlx
webpack-bundle-analyzer .next/server/webpack-stats.
json .`
After this change, `cargo check` on a branch that deletes vc generics
support compiles successfully.
## Why?
Rather than extended support for `Vc` generics to `ResolvedVc`, I plan
to remove support for them entirely. That means fixing all the callsites
(there's not many).
Removing this is needed to get us to a point where 100% of structs can
be `ResolvedValue`, because structs using these fields cannot otherwise
be ported over to `ResolvedVc`.
## Okay, but Why?
Explained here:
https://github.com/vercel/turborepo/pull/8843#issuecomment-2253158412
I expect removing support for this will decrease the overall size of the
codebase, as the logic for supporting generics is rather complicated.
Removes the uses of generic vc collections in
`next-core/src/app_structure.rs` and it's callsites.
## Why?
Rather than extended support for `Vc` generics to `ResolvedVc`, I plan
to remove support for them entirely. That means fixing all the callsites
(there's not many).
Removing this is needed to get us to a point where 100% of structs can
be `ResolvedValue`, because structs using these fields cannot otherwise
be ported over to `ResolvedVc`.
## Okay, but Why?
Explained here:
https://github.com/vercel/turborepo/pull/8843#issuecomment-2253158412
I expect removing support for this will decrease the overall size of the
codebase, as the logic for supporting generics is rather complicated.
This:
- Updates the toolchain file
- Adds `#![feature(arbitrary_self_types_pointers)]` as necessary
- Removes opt-ins for now-stable features like
`option_get_or_insert_default`
- Removes now-detectable dead code
`TaskInput` isn't needed/used for a bare `ResolvedVc`, as we'll expose
`ResolvedVc` arguments as `Vc`, but it is useful for structs that
contain `ResolvedVc` and want to derive `TaskInput`.
This PR also ports `next_core::app_structure::Entrypoint` to use
`ResolvedVc` as a way of testing this.
In the same vein as #69190, where we already unwrapped `createServerReference`, we now also unwrap `registerServerReference`, which is required for React to select the right call stack frame when generating source locations for server actions (see facebook/react#30741).
Whereas unwrapping of `createServerReference` was required for server actions that are imported into client components, unwrapping `registerServerReference` is needed for server actions that are passed from server components to client components.
This does not fully enable the source mapping just yet. For this to work end-to-end, the next step is to generate proper spans in the SWC transform, which will be done in the next PR.
### What?
Add a new turbo-tasks backend that stores data normalized and is
prepared for persistent caching.
No longer keeps state in task locals, but all state is stored
immediately. This is required for persistent caching since it could
store and resume at any point and we always need to be in a valid state.
Tasks stay dirty until recomputed. (Previously dirty was changed to
scheduled)
### Why?
* Reduced memory usage
* preparation for persistent caching
This is a handmade refactor of `turbo-tasks-fs` to switch every struct's
field from `Vc<T>` to `ResolvedVc<T>`.
I chose `turbo-tasks-fs` because it's reasonably small, but very widely
used, and it has a representative mix of code patterns.
In the process of creating this PR, I wrote up a guide of the steps
required to refactor this, in the hope that we can automate most of
these steps for most code:
https://vercel.notion.site/Codemod-Notes-for-Vc-T-ResolvedVc-T-10ee06b059c480688ebffe5eb983db25
### What?
* round chunk item sizes to cause less changes to chunking
* pick individual values from compile time info to avoid invalidating on
change
* pick next config keys for more granular caching
* split completions into a tree
---------
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
### What?
The normal file watcher doesn't work inside a docker container, the
workaround is to enable a polling file watcher, which was supported in
webpack but not turbopack so far.
This PR adds support for a polling file watcher to turbopack and a
`next.config.js` option to configure the polling file watcher for both
webpack and turbopack.
Unfortunately, the rust file watcher seems to be a lot slower than the
poll interval.
Closes PACK-3206
Fixes#68255
Closes PACK-3244
Closes https://github.com/vercel/next.js/issues/69562
Await the module in both environments (SSR/browser), if it's async in at
least one of them.
We can't and don't want to make the browser module async (artifically)
if the SSR module is async (due to ESM externals).
The only concern regarding `async` not always being true in the manifest
was that we don't want to await in non TLA cases (e.g. CJS exporting a
promise, etc...).
But the SSR module being ESM with TLA and the browser module being CJS
with a Promise export is exceedingly unlikely.
Initially the non-PPR `dynamicIO` implementation used some experimental
React APIs. This change updates the technique to only depend on APIs in
the canary channel and updates the packaging rules to only not opt into
experimental react when dynamicIO is enabled. Other experimental flags
can still opt you into experimental react such as PPR.
This PR extends the cache directive to accept an additional type
parameter which defaults to `"default"`.
Several follow-ups:
- Accept different syntaxes (like places of spaces) and typo detection;
- Have a limited set of valid options with better errors.
For https://github.com/facebook/react/pull/30741
This PR adds several additional parameters to the
`createServerReference` calls generated by the Server Reference SWC
transform, to later enable React to map server actions that are imported
into client components back to their server locations (dev mode only).
Currently the inner `findSourceMapURL` function is not yet implemented.
In a follow-up we will unwrap `registerServerReference` as well, which
is needed for server actions in the react server layer.
---------
Co-authored-by: Hendrik Liebau <mail@hendrik-liebau.de>
The `#[turbo_tasks::function]` macro always exposes a `impl
Future<Output = Result<ReadRef<T>>>`, regardless of the function's
actual return type.
We only need to use a `Result<...>` return type when the function can
fail. If it's infallible, this just adds noise.
I left a bunch of these behind in #70412, plus I think we had a lot of
these independent of that PR. This cleans them up.
```yaml
language: rust
id: remove_unused_result_return_type
utils:
last-expression-inside-block:
any:
- inside: # single-line blocks just have the expression as the only child
kind: block
nthChild:
position: 1
reverse: true
- inside: # multi-line blocks wrap their final expression in an expression_statement
kind: expression_statement
inside:
kind: block
nthChild:
position: 1
reverse: true
ok-expression:
kind: call_expression
any:
- pattern: Ok($_ARG)
- pattern: $$$::Ok($_ARG)
ok-expression-capturing:
kind: call_expression
any:
- pattern: Ok($ARG)
- pattern: $$$::Ok($ARG)
# ast-grep does not appear to allow utils to be recursive, split out "simple blocks", and limit supported nesting
simple-block-with-implicit-ok-return:
kind: block
has:
nthChild:
position: 1
reverse: true
matches: ok-expression
simple-expression-ok-value:
any:
- matches: simple-block-with-implicit-ok-return
- kind: if_expression
all:
- has:
field: consequence
matches: simple-block-with-implicit-ok-return
- has:
field: alternative
has:
matches: simple-block-with-implicit-ok-return
- kind: match_expression
has:
field: body
not:
has:
kind: match_arm
has:
field: value
not:
any:
- matches: ok-expression
- matches: simple-block-with-implicit-ok-return
block-with-implicit-ok-return:
any:
- matches: simple-block-with-implicit-ok-return
- kind: block
has:
nthChild:
position: 1
reverse: true
any:
- kind: expression_statement
has:
matches: simple-expression-ok-value
- matches: simple-expression-ok-value # single-line blocks don't
result-return-type:
pattern:
context: fn func() -> Result<$INNER_RET_TY> {}
selector: generic_type
infallible-fn: # this function only appears to return `Ok(...)`, it does not use try_expression (`?`) or `anyhow::bail!(...)`
kind: function_item
not:
has:
field: body
any:
- not:
matches: block-with-implicit-ok-return
- has:
stopBy:
kind: function_item
any:
- kind: try_expression
- pattern: "?"
inside:
kind: macro_invocation
stopBy: end
- pattern: bail!($$$)
- pattern: $$$::bail!($$$)
- kind: return_expression
not:
has:
matches: ok-expression
rule:
all:
- pattern: $FUNC
- kind: function_item
has:
field: return_type
matches: result-return-type
follows:
pattern:
context: |
#[turbo_tasks::function]
selector: attribute_item
stopBy:
not:
kind: attribute_item
- matches: infallible-fn
rewriters: # this rewriter is far from perfect, and might rewrite too much
- id: rewrite-return-type
rule:
matches: result-return-type
inside:
kind: function_item
field: return_type
fix: $INNER_RET_TY
- id: unwrap-ok-values
rule:
all:
- matches: ok-expression-capturing
- any:
- matches: last-expression-inside-block
- inside:
kind: return_expression
- inside:
kind: match_arm
fix: $ARG
transform:
NEW_FUNC:
rewrite:
rewriters:
- rewrite-return-type
- unwrap-ok-values
source: $FUNC
fix: $NEW_FUNC
ignores:
- "**/turbo-tasks-testing/**"
- "**/turbo-tasks-memory/tests/**"
```
```
sg scan -U -r ../codemod_remove_unused_result_return_type.yml && cargo fix --lib --allow-dirty && cargo fmt
```
I used `cargo fix` in this case to auto-remove the now-unused
`anyhow::Result` imports.
I manually fixed clippy lints that now violated the `let_and_return`
lint rule (using a separate commit inside this PR), as `cargo clippy
--fix` seemed to hang when processing our `build.rs` files?
Tobias Koppers
