### What?
Make `#[turbo_tasks::function(invalidator)]` a thing. It's a preparation for the automatic (runtime) detection of statically-analyzable immutable turbo task.
### Why?
We need it for an advanced optimization
### What?
Configure HMR benchmark. I'm not sure if this benchmark would be stable enough, but let's try.
I'll remove this from the automated benchmark later this week if it turns out to be flaky.
### Why?
x-ref: https://vercel.slack.com/archives/C06PPGZ0FD3/p1748943158878159
### How?
Closes PACK-4758
### What?
Implement a simple tree-shaking. Simple tree shaking is a tree-shaking implementation that only has two options for each export: used/unused. This information is shared among the whole application, so there's no fine-grained control for each export.
### Why?
Because advanced tree shaking requires some more time to implement.
On its own this is just adding pure overhead. The next step will be to make it _optional_ for a value_trait item to be a turbo-tasks function.
Part of PACK-4560
This is intended as a band-aid "fix" for persistent caching where issues stick around after restarts of the development server. It builds on top of the same codepath we're using for the explicit cache clearing button in the devtools.
As part of this work, this adds a centralized registry for dynamically-registered panic hooks.
## Notes & Next Steps
- I'll disable this behavior for Vercel employees in front, so that we continue to get valuable bug reports.
- I'll follow up with some warning message in the terminal so that the subsequent restart of `next dev` will warn the user that the persistent cache was cleared and that's why build may be slow.
- We've got a some in-progress work handed to me from @wbinnssmith for better logging of anonymized (uncaught) panics to telemetry, so that should be useful in cases like this.
- Yes, this will still invalidate the cache on caught panics. I'm not 100% sure that's the behavior we want, but I think it is.
## Test Plan
Built with:
```
pnpm pack-next -p ~/front/apps/vercel-site
cd ~/front
pnpm i
pnpm turbo --filter=vercel-site dev
```
Restarted a few times and looked at the persistent cache files and saw that they keep accumulating.
Then, deleted a random `.sst` file, restarted, and observed a bunch of panics. Saw that the invalidation file was written. Restarted the server again and saw that it recovered itself by clearing the cache.
# Remove `auto_for_input` serialization mode from turbo-tasks
This PR removes the `auto_for_input` and `custom_for_input` serialization modes from turbo-tasks, along with the `TypedForInput` marker trait.
These were used for values passed via the `Value<>` type, but that is dead.
(#80134)
Replace uses of `Value<Pattern>` in the resolve crate with `Pattern`
This required a manual implementation of `TaskInput` which ended up being trivial since there are no captured `Vcs`. However, the `derive` macro failed because it is a recursive type.
### Why?
The Value<> type is confusing and error prone, lets destroy it!
As discussed over slack, turbo_tasks::Value type always implements is_resolved as true and is_transient as false which is incorrect and can enable smuggling transient vcs into turbo tasks.
• https://vercel.slack.com/archives/C03EWR7LGEN/p1743456121241529
• https://vercel.slack.com/archives/C04NGV98TPS/p1743455453764879
When turbopack reports an issue, compute a set of import-traces for that file so the user can better understand the context of the issue and trivially answer questions like:
* Why is this file included at all?
* Why does next think this is a client-component?
To do this we leverage the `SingleModuleGraph` and use the `astar` algorithm from `petgraph` to compute the shortest path to a root module. This isn't the most optimal approach but should be sufficient since we don't anticipate this being a performance issue.
A complex part of this is that the module-graph tracks the relationships between _modules_ but Issues are associated with _files_. While modules are also associated with files this is a many-to-one relationship. This is why we might report _multiple_ traces for a single issue and also why a single file might appear multiple times in a trace.
## Open formatting questions
* how should we represent paths from other 'filesystems'?
- for disk filesystems i could compute relative paths to the root of the current directory? the `[project]` filesystem? For now i just use the filesystem name as a hypothetically cromulent root.
## Alternatives
The main alternative investigated was associating Issues with `Modules` by collecting them during graph construction. This unfortunately proved to be a non-trivial performance regression and so it was abandoned. The core problem is that we would need to introduce additional `OperationVc` and task roots to simply `collect` the issues. This also wouldn't eliminate the duplicate traces issue, and instead we might end up reporting duplicate issues instead.
## Performance
TODO
Closes PACK-4105
## What?
`export const dynamic` was not being re-exported when creating the route
handler wrapper so it wasn't picked up during builds, causing the tests
to fail. Added the re-exports.
Adopt the `rcstr!` macro in turbopack
Instead of `"foo".into()` call `rcstr!("foo")` while being slightly more annoying to type the macro approach is more readabie and faster as it can leverage constant folding for small strings and amortize construction of larger strings. In general these performance improvement are very small but consistent.
At the same time, eliminate uses of turbo-tasks where they were amortizing construction of RcStr instances since the `rcstr!` macro can do the same thing. Generally speaking we should prefer `RcStr` over `ResolvedVc<RcStr>` as a task input since it has much more reasonable equality semantics and observationally many of these values were constructed via `Vc::cell("foo".into())`.
## Why?
Generally this should be a small consistent memory and performance improvement. My understanding is that many uses of `ResolvedVc<RcStr>` actually predate `RcStr` when we were using `String` directly. In that world, turbotasks are fulfilling a vital function, but now that we can rely on RcStr this is much less needed.
## How?
Mostly this was a bunch of regex replacements and manual fixups. An AstGrep rule would have been ideal but beyond matching `RcStr::from("...")` this doesn't work since it was mostly calls to `into()` which rely on type inference. Still by seeding the codebase with the pattern we can probably prevent regressions.
## Performance
From measuring vercel-site this is a latency progression of a few percent. codspeed agrees!
## Fail Google Fonts in Production Builds When Fetch Fails
### What?
This PR changes the behavior of Google Fonts in Next.js to properly fail
production builds when font fetching fails, while still allowing
development to continue with fallback fonts.
### Why?
Previously, font fetch failures were only logged as warnings but didn't
stop production builds. This could lead to production deployments with
missing fonts, creating a poor user experience. Development mode should
be more forgiving to allow work to continue offline.
### How?
- Added `is_dev_mode()` method to the `ChunkingContext` trait to
determine the current environment
- Modified the Google font fetching logic to:
- In development: Show a warning and use fallback fonts when fetching
fails
- In production: Throw an error that fails the build when fetching fails
- Passed the execution context to the font file replacer to access the
chunking context
This ensures that production builds will fail fast when font resources
can't be fetched, while development builds can continue with appropriate
fallbacks.
Fixes #PACK-4667
### Future Hopes
I would hope to eventually expose an enum for the build mode rather than
just a boolean flag like I have here. I started going down that path for
this PR, but it was expanding the scope significantly of what would have
to be done, so I have left it as a bool to fix this problem.
This function does a few more `Vc` reads than just looking up the source map, so it's better to move the entire thing inside one strongly-consistent operation.
This was an attempt to fix a loop (tracing an error -> react fast refresh -> triggers the same error again -> ...) https://vercel.slack.com/archives/C046HAU4H7F/p1747837763588429, though it doesn't seem like this fixes it.
Replace the `ResolvedVc<RcStr>` uses inside of `AssetIdent` with `RcStr` directly
## Why
This simplifies construction and usage and provides more reasonably equality semantics. In principle, storing `ResolvedVc` would enable finer grained invalidations. e.g. if only the `query` changed and a task only depended upon the `modifiers` then that task wouldn't get invalidated. However, the entire `Assetident` is often consumed monolithically already. For example, it is when producing output chunks and constructing the modulegraph (via the `to_string` function). So modeling it more monolitically seems appropriate.
Additionally, by accepting `RcStr` values as _values_ instead of `cells` we clarify equality semantics. So we should expect to get _more_ cache hits from `AssetIdent::new`. In principle this may make equality slower, but it is very rare that any of the RcStr instances involved are more than a handful of bytes, so this shouldn't be much of a concern.
This makes the type slightly smaller, now it is `120 bytes` instead of `144 bytes`, which should help with performance a bit since there are many thousands of these in a build.
## What
This was mostly done by just replacing the types and chasing build errors backwards to remove the `ResolvedVc` values at the source. This allowed me to replace many tiny turbotask functions with the `rcstr!` macro and remove many calls to `cell`.
## Performance
Testing `vercel-site` reveals a small improvement in MaxRSS which is expected due to 1. eliminating a bunch of turbotask cells, and 2 reducing the size of `AssetIdent`
## What?
Implements handling for `%5F` to `_` conversion to handle pathnames in
App Router that want to bypass the underscore ignore folder.
Implementing this was a bit of a hassle not on the Turbopack side but
there's an additional layer of route handling before the compiler that
turns the filesystem paths into routable routes. Turns out there was
some additional places where the normalization needs to apply.
For now implemented it so that it special cases Turbopack handling,
might change the webpack implementation in the future to output the
files into the same path that Turbopack outputs it at. This is only
relevant for dev, production is handled the same already.
Creates and passes through an option to turbopack to invalidate the cache. The dev overlay can then (not implemented) call this endpoint.
Invalidation is performed by writing a marker file to disk, and then actually deleting the cache upon the next startup because:
- Getting the database to drop all file handles is tricky.
- You can't delete open files on Windows.
- Writing an invalidation file is atomic, recursively deleting a directory is not.
### Testing
Set up a small app with a large dependency (three.js). Ran the build, restarted the dev server, saw that the cached run was fast.
Ran
```
curl -v --request POST --header "Content-Type: application/json" --data '{}' http://localhost:3000/__nextjs_restart_dev\?invalidatePersistentCache\=
```
And saw that the server restarted and the next page load was slow.
### Remaining Work (will come in subsequent PRs)
In rough order of priority:
- A separate endpoint to poll to see when the server is back up.
- A webpack+rspack implementation. It looks like I can do a hack to forcibly change the webpack cache version, which should trigger similar behavior upon the next restart.
- Passing a boolean to the dev overlay telling it if persistent caching is enabled or not.
- The devtools UI implementation.
- Telemetry.
- A CLI subcommand that leverages this, so that you can do it from the terminal.
Fix URL fragment and query handling in Turbopack
## What?
This PR improves how URL fragments and query strings are handled in Turbopack by:
1. Standardizing the representation of fragments and queries in `AssetIdent`
2. Ensuring fragments and queries are either empty or include their prefix characters (`#` and `?`)
3. Fixing the URL parsing logic to correctly handle fragments that contain `?` character
4. Removing some ad-hoc workarounds for this issue in the `resolver` and `Assetident::to_string`
## Why?
The previous implementation had inconsistencies in how fragments and queries were represented which lead to a variety of confusing issues e.g. does `AssetIdent::to_string` need to add a `?` ?, well sometimes depending on how it was constructed.
## Refactor module graph to reduce memory usage
### What?
This PR refactors the module graph implementation to reduce memory usage by:
1. Changing `graph_idx` from `usize` to `u32` to reduce memory footprint
2. Removing unused fields from `SingleModuleGraphModuleNode` (layer and issues)
3. Moving layer resolution to the point of use in dynamic imports and server actions
This is a small progression in runtime:
### Why?
These changes optimize memory usage in the module graph, which is particularly important for large applications. By removing unused fields and using more compact types, we can reduce the memory footprint of the graph.
Notably, SingleModuleGraphNode is now 24 bytes instead of 48. To make it smaller still we would need to reduce the size of visited moduel, perhaps by storing a reference to the graph that did the original visit? This would be tricky from a lifetimes perspective but much more optimal.
## Refactor Turbopack Compilation Events and Timing Reporting
### What?
This PR refactors how compilation events are created, displayed, and
formatted in Turbopack. It introduces a dedicated `TimingEvent` type for
timing-related messages and improves the formatting of duration
displays.
Additionally we now display all compilation events now that it has been
shown to be stable with no breakages.
### Why?
To provide more consistent and user-friendly output for compilation
events, especially for timing information. The new approach makes
duration formatting more readable by automatically selecting appropriate
units (ms, s, min) based on the duration length.
### How?
- Added a new `TimingEvent` struct that properly formats durations based
on their duration
- Updated the `DiagnosticEvent` constructor to take severity first, then
message
- Improved event handling in the build process to properly categorize
and display different severity levels
- Enhanced file I/O benchmarking to use the new timing event format
- Fixed the formatting of compilation events in the UI
---------
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
When a server function passes a closure to a method on a closed-over
value (e.g. an array), we must correctly track the value itself so that
it's included in the bound arguments.
For example, in this snippet, `list` must be bound to the server action:
```js
export function Component({ list }) {
return (
<form
action={async () => {
'use server'
console.log(list.find((x) => !!x))
}}
>
<button>submit</button>
</form>
)
}
```
This use case used to work but was accidentally broken by #73189.
fixes#79608fixes#77247
closes NAR-8
## What?
Implements the missing part of the error to make the test pass.
Small difference between webpack/Turbopack is that Turbopack will spot
the conflict earlier and because of that errors earlier too, making the
path still correct but shorter, not a problem in my opinion.
---------
Co-authored-by: Niklas Mischkulnig <4586894+mischnic@users.noreply.github.com>
## What?
Currently there's a specific case when you set `runtime:
'experimental-edge'` for `_document.js` or `_app.js` where we
incorrectly write a bundle that is wrapped with the route handling code
to disk. This in turn causes `_document.js` to break during
prerendering. Seems no users ran into this one yet since it was
experimental. This change ensures that we always write the module code
for _app / _document instead. This has no negative effect on edge
runtime modules because those include the direct imports for _app /
_document already in the single route module.
I'm deleting all of @sokra's hard work in favor of @sokra's hard work.
RIP memory backend, you served us well 🫡
The new `turbo-tasks-backend` supports both persistent (via LDMB or `turbo-persistence`) and in-memory databases (via the no-op storage implementation), and tends to be faster than `turbo-tasks-memory` when used with the no-op storage. Maintaining both has increasingly become a headache.
Donny/강동윤
