Avoid spawning a worker thread to run turbopack
The typical arguments for worker threads are that they isolate the JS
heap and enable parallelism, but turbopack does both these things by
default (native heap is isolated, the js deps are minor and have a lot
of overlap with existing next server deps).
There is also no substantial parallelism opportunity from worker threads
that aren't immediately delivered by turbopacks execution model
## Benchmarks
Measured with a paired A/B harness that alternates `head` and `base`
builds so thermal drift and background load cancel out, discarding a
warmup build per variant and clearing `.next` before each run. Both
variants are the same commit except for this change. 35 paired
iterations on a default `create-next-app` (App Router, TS), Node
20.11.1, Apple silicon.
| Metric | this PR | canary | Diff | 95% CI | Wins |
| --- | --- | --- | --- | --- | --- |
| `next-build` total | 2478 ms | 2595 ms | **-117 ms (4.5%)** | [50,
183] | 30/35 |
| `run-turbopack` | 1228 ms | 1335 ms | -107 ms (8.0%) | [62, 152] |
31/35 |
| overhead (total - turbopack) | 1250 ms | 1260 ms | -10 ms (0.8%) |
[-27, 46] | 20/35 |
| wall clock | 2775 ms | 2888 ms | -112 ms (3.9%) | [45, 180] | 30/35 |
| peak RSS | 1175 MiB | 1195 MiB | **-19 MiB (1.6%)** | [3.4, 35.3] |
22/35 |
Significant by both a paired t-test and a Wilcoxon signed-rank test (p =
0.0001 for total build time, p = 0.025 for RSS). The Wilcoxon is
included because a couple of slow builds per run would otherwise
dominate the variance.
Nearly all of the saving lands inside the `run-turbopack` span rather
than around it, which is expected: that span wraps the worker spawn, the
IPC to marshal the build across, and the teardown, so deleting the
worker removes work from inside it.
The win is proportionally smaller on larger apps, where compilation
dominates. On `bench/basic-app` (~400 source modules, ~19s builds) the
same ~115 ms is well inside the noise -- 12 paired iterations there
showed no significant difference on any metric. So this is a real but
small improvement that is only user-visible on small builds:
| Metric | this PR | canary | Diff | 95% CI |
| --- | --- | --- | --- | --- |
| `next-build` total | 19313 ms | 19436 ms | -123 ms (0.6%) | [-256,
502] |
| overhead (total - turbopack) | 1424 ms | 1415 ms | +9 ms (0.6% slower)
| [-52, 70] |
Caveats: single machine and two fixture shapes. Peak RSS is the build
process's own footprint (`/usr/bin/time -l`), which includes the worker
thread's isolate but not the separately spawned static-generation
workers. A CI runner with fewer cores would plausibly show a larger
effect.
`experimental.workerThreads` decides whether static generation runs in
real worker threads or forked child processes, and a native addon
declared with `NODE_MODULE` can only be loaded once per process. That is
why the flag defaults to false (#9199) and why the static export worker
was fixed to respect it rather than hardcoding threads on (#25063).
Nothing tested it: the only suite that tried, `firebase-grpc`, had its
assertion skipped since 2019, and modern `firebase` ships no native
module at all, so its remaining test asserted only that a build
succeeds.
This adds a `single-context-addon` fixture, deliberately declared with
`NODE_MODULE`, and a production suite asserting both directions: the
build succeeds by default and fails with "Module did not self-register"
once worker threads are enabled. The fixture loads the addon from
`next.config.js` as well as from the page, because that failure only
happens on a second `dlopen` within one process, so an addon loaded only
inside the worker would register there and the build would pass.
A third case documents a bug rather than intended behaviour. `next
build` runs Turbopack in a worker thread and that worker re-evaluates
`next.config.js`, so requiring a non-context-aware addon from the config
breaks the build even with `experimental.workerThreads` off. Webpack and
rspack are unaffected, since their build workers are forked child
processes. Isolated with an unguarded `require` and default flags,
Turbopack exits 1 with "Module did not self-register" where webpack
exits 0. It is the same class of failure #9199 and #25063 fixed, in a
worker those PRs did not touch. The assertion is branched on the bundler
and carries a note to drop the branch once Turbopack stops evaluating
the config on a worker thread. The `isMainThread` guard in the fixture's
config keeps the first two cases pointed at the static generation worker
instead.
A development suite covers the other direction, asserting that
evaluating a route does not put such an addon on one of the threads
`next dev` uses regardless of the flag, so it would catch a change that
moved route evaluation onto the dev validation pool. The expectation is
the same with and without Cache Components; both were checked by logging
the thread id from the page's module scope, and the route is evaluated
on the main thread either way.
`firebase-grpc` is removed, since it covered the same flag with a
skipped assertion and a vacuous one.
Luke Sandberg