The idea here is that we waste a bunch of CPU setting up each test suite
by spawning an entirely new Chrome process, but playwright can share the
same browser across many tests. Tests are still isolated at the browser
level, just not at the OS process level.
Hoping to see some performance improvement in e2e tests. Compare to:
https://github.com/vercel/next.js/pull/95617
Somewhat unscientific (single sample) results:
```
Comparing the head-commit build-and-test runs — PR #95589 run [28981702219 ](https://github.com/vercel/next.js/actions/runs/28981702219)vs PR #95617 run [28981686692](https://github.com/vercel/next.js/actions/runs/28981686692). Both ran 101 jobs and both succeeded.
┌─────────────────────────┬────────────────┬───────────────────────┬───────────────────┐
│ Metric │ #95617 control │ #95589 shared browser │ Delta │
├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
│ Raw wall-time sum │ 616.2 min │ 571.2 min │ −45.0 min (−7.3%) │
├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
│ Billable (ceil per job) │ 658 min │ 617 min │ −41 min (−6.2%) │
└─────────────────────────┴────────────────┴───────────────────────┴───────────────────┘
The shared-browser experiment is cheaper, and the savings land almost exactly where you'd expect — the browser-driven test suites — while non-browser jobs (rust, lint, unit, windows) are flat within noise:
┌─────────────────────────────────┬─────────┬────────┬───────┐
│ Job category │ control │ shared │ delta │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ test prod │ 130.7 │ 110.5 │ −20.2 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ cache components dev │ 51.5 │ 42.9 │ −8.7 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ turbopack dev │ 75.1 │ 69.5 │ −5.5 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ turbopack production │ 86.3 │ 81.1 │ −5.2 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ test dev │ 110.4 │ 105.5 │ −4.9 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ cache components prod │ 48.7 │ 49.0 │ +0.3 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ firefox and safari │ 5.3 │ 7.3 │ +2.0 │
├─────────────────────────────────┼─────────┼────────┼───────┤
│ flake-detection jobs (combined) │ ~15 │ ~21 │ +~6 │
└─────────────────────────────────┴─────────┴────────┴───────┘
Takeaways:
- Net savings of ~41 billable minutes per run (~6%), concentrated in the Chromium-driven test prod/dev, turbopack, and cache components suites — consistent with reusing one browser process instead of spawning per-suite.
- The small regressions are in firefox and safari and the "new/changed tests for flakes" jobs (+~8 min combined). Worth a glance, though they may just be run-to-run variance.
Caveat: this is a single run per PR, so there's real variance run-to-run. That said, the fact that the deltas track the browser-heavy jobs specifically — and not the Rust/lint/unit jobs — is a good signal the effect is genuine rather than noise. If you want a firmer number, re-running each PR 2–3× and averaging would tighten it up.
```
During `next dev`, certain errors (such as Cache Components validation
errors) are logged directly to the terminal by the dev server and are
also sent to the browser so they can be shown in the dev overlay. The
browser logs them to its own console, and the browser-to-terminal log
forwarding would replay that copy back to the CLI, so the same error
appeared in the terminal twice.
This change marks those errors when the browser receives them, in the
`ERRORS_TO_SHOW_IN_BROWSER` handler, using a non-enumerable symbol.
`forwardErrorLog`, the single function all forwarded error logs pass
through, then skips any error carrying that marker, since it already
originated on the server and was logged there. The marker has to be set
browser-side because it wouldn't survive RSC serialization, which is the
same reason the error code is sent through a side-channel map rather
than on the error itself.
The error still shows in the browser console and the dev overlay; only
the redundant terminal echo is suppressed. This applies to every error
delivered over the errors RSC stream, all of which are logged on the
server before being sent, so the forwarded copy is always a duplicate.
> [!TIP]
> Review the two commits individually to see the before/after change.
When Cache Components is enabled, `next dev` treats a `'use cache'`
value as a miss and renders as if the cache were empty whenever the read
does not resolve right away. Two development configurations triggered
that even for values that were already cached, so warm reloads streamed
slowly instead of serving the cached value: `cacheMaxMemorySize: 0`
replaced the built-in default handler with a no-op stub, so nothing was
cached at all, and custom cache handlers with a slow or remote `get` did
not return in time.
For the size-0 case, development now uses a real in-memory handler
instead of the no-op stub, and the `'use cache'` wrapper forces a
dynamic cache life (`revalidate: 0`, a 5-minute `expire`) for it, the
same treatment private caches already receive, so every read serves the
stale entry and re-warms a fresh one in the background. This also fixes
the dev private handler, which was sized from `cacheMaxMemorySize` and
so degraded to the no-op stub whenever `cacheMaxMemorySize: 0` was set.
Custom cache handlers keep their configured cache life, since their
backing owns it. Instead we put a fast built-in in-memory front handler
in front of the configured one through the new `TieredCacheHandler`,
which serves warm reads from the front, writes through to both tiers,
and reconciles the front against the backing in the background, evicting
the front entry when the backing no longer has it (the handler interface
has no per-key delete, so it overwrites the entry with an
already-expired copy). These dev-only handlers are kept out of the
registered handler set and merged in only where tag operations iterate,
so `revalidateTag` still reaches them.
Everything is gated on `process.env.__NEXT_DEV_SERVER`, so production is
unchanged: `cacheMaxMemorySize: 0` still caches nothing, private entries
are still never persisted, and configured handlers are used directly.
New development test suites cover the size-0 and custom-handler
behavior.
When the browser's HTTP cache entry for a back-navigation target has
been evicted between forward visit and back-press — for example with a
long-lived tab, storage pressure, or a manual cache clear — the browser
re-fetches the document fresh from the server. Previously we relied on
`type === 'back_forward'` alone to decide that the document came from
cache, which meant we treated those re-fetches as cache restores too.
The persisted-chunk lookup would miss on the fresh response, and we'd
recover with an unnecessary `location.reload()`.
The same edge case is exposed by the Playwright/WebKit combination in
the upstack PR: bfcache isn't utilized there, so back-navigations always
come over the network, and the old single-signal check classified those
re-fetches as served-from-cache and triggered the same unnecessary
reload.
With this change we split the cache-restore detection into two phases.
At script-execution time we look at `deliveryType` (Chrome ≥109, Safari
≥17) and the navigation entry's `transferSize`/`encodedBodySize` to
decide cache-restore vs fresh-response when those fields are populated.
When the body bytes haven't been measured yet — the common case for
streaming Next.js dev RSC responses on every browser, and also WebKit
leaving both size fields at zero at exec time — we suspend the readable
until `pageshow` and re-check there. On a fresh re-fetch we route
through the live WebSocket-backed channel, which already has the debug
data for the new response, and skip the reload entirely.
The new `bfcache-regression` test exercises this edge case on Chromium
by clearing the browser cache via CDP between the forward and back
navigations, using a new `clearBrowserCache()` helper on the test
browser wrapper. The same exec-time code path is naturally hit by Safari
whenever its navigation entry's size fields are still zero at
script-execution time, but the harness can't force the eviction
deterministically there.
This PR fixes some HTML/RSC stream interleaving logic that, under some
circumstances, resulted in hydration of a resumed page being delayed
unnecessarily.
Fixes NAR-284
---
When SSRing a page, we're mixing two streams -- the HTML stream, and the
RSC stream (containing scripts with RSC data, used for hydration). In a
regular (non-PPR) render, we need to wait for react to flush the first
HTML chunk (containing `<html><body>...`) before we can send the first
RSC script. This was implemented in `createMergedTransformStream`.
The problem was that `createMergedTransformStream` was also re-used when
rendering a PPR `resume()`. In that case, we're only outputting the
dynamic portion of HTML, and the shell is sent separately (either by the
next server, outside of `app-render`, or by infrastructure), meaning
that is _not_ part of the stream that `createMergedTransformStream`
receives. But the "wait for the first html chunk" logic was still there,
and thus, we were accidentally delaying sending any RSC scripts until
the first piece of dynamic HTML was rendered even if they had no
connection to each other. In particular, this also prevented us from
sending hydration data for the static shell.
There's one edge case here -- if we didn't produce a static shell (e.g.
for suspense-above-body), we should still apply the html-waiting logic,
otherwise we'd end up sending scripts before the body is rendered. But
if we have a shell, we shouldn't wait for the HTML at all.
This is now solved as follows:
1. During the prerender, we track whether or not a shell (aka "prelude")
was produced, and store that information in the postponed state object
2. When resuming, we check whether the prerender had a shell, and if it
does, we don't wait for HTML (under the assumption that it was already
sent separately, outside the dynamic render)
To test this, I've had to extend our testing setup a bit -- with the
default settings, playwright would wait until `load` is fired, which
seems to fire after all the HTML finished streaming and thus doesn't let
us inspect whether partial hydration is working correctly. We're also
waiting for `load` in `elementByCss` (apparently, for compatibility with
tests written before playwright) so i've had to work around that as
well.
When a request URL is rewritten, the resulting data must be keyed by its
rewritten search params — the ones that were used by the server to
render the page — not the original search params. This works in most
places by encoding the rewritten search params into page's segment key
in the server response. However, the Segment Cache implementation did
not handle this correctly.
The fix is to read the rewritten search params from the
x-nextjs-rewritten-query header in the server response.
In upcoming PRs, we will use a similar approach for regular route
params, too, so that we can lift the params out of the body of the
server response.
---------
Co-authored-by: Janka Uryga <lolzatu2@gmail.com>
rip. you served a purpose once, but now you're just getting in the way. this improves typesafety quite a bit, because `BrowserInterface` had a whole bunch of random `any`s everywhere
also
- removes `evalAsync`. no idea why that was needed, but we're happily using promises in normal eval, so it can be dropped
- adds more safety to `chain`
Adjusts the default timeouts:
- elementByCss: 5s
- waitForElementByCss: 10s
Previously, we didn't pass an explicit value, so they both defaulted to
playwright's default of 60s. This is _very_ slow, and if something is
slow enough to require anything in that range, i'd say it deserves to be
wrapped in `retry()` or something similar anyway, so it's fine to fail
here.
The upside of this is that errors because of bad selectors (or missing
elements) happen faster, which is a nice thing, and might also save CI
time.
i could also see an argument for making these even shorter -- e.g. the
default timeout for `retry()` is 3s, so maybe `retry` should be faster
than that.
https://github.com/vercel/next.js/blob/717e54b0cd914a17314c470cd10d6af2b2d75787/test/lib/next-test-utils.ts#L796-L799
but then again, `retry` doesn't factor in the time `fn` took, so these
don't really conflict.
### Why?
When users want to hide the dev indicator, they might be frustrated if
it reappears every reload. Hence, make a default to hide it for the
current server session or a day.
Closes NDX-890
---------
Co-authored-by: Zack Tanner <1939140+ztanner@users.noreply.github.com>
Console logs in server components are replayed in the browser. For
example, when you run `console.log('foo')`, the log will be replayed in
the browser as `[Server] foo`. When the component is inside of a `"use
cache"` scope, it's replayed as `[Cache] foo`.
However, when logging directly in the function body of a `"use cache"`
function, we are currently not replaying the log in the browser.
The reason for that is that the function is called outside of React's
rendering, before handing the result promise over to React for
serialization. Since the function is called without React's request
storage, no console chunks are emitted.
We can work around this by invoking the function lazily when React calls
`.then()` on the promise. This ensures that the function is run inside
of React's request storage and console chunks can be emitted.
In addition, this also unlocks that `React.cache` can be used in a `"use
cache"` function to dedupe other function calls.
closes NAR-83
i.e. the stack will now include the actual test util in the test that failed not just a single frame from util internals.
New stack for e.g. a failed `waitForElementByCSS`:
```diff
● app dir - navigation with Suspense in nested layout › resolves data after client navigation to a nested layout with Suspense
page.waitForSelector: Timeout 1000ms exceeded.
Call log:
- waiting for locator('[data-testid="nested-resolved"]')
423 | return this.chain(() => {
424 | return page
> 425 | .waitForSelector(selector, { timeout, state: 'attached' })
| ^
426 | .then(async (el) => {
427 | // it seems selenium waits longer and tests rely on this behavior
428 | // so we wait for the load event fire before returning
at waitForSelector (lib/browsers/playwright.ts:425:10)
+ at BrowserInterface.chain (lib/browsers/base.ts:17:23)
+ at BrowserInterface.chain [as waitForElementByCss] (lib/browsers/playwright.ts:423:17)
+ at Object.waitForElementByCss (e2e/app-dir/navigation-layout-suspense/navigation-layout-suspense.test.ts:17:19)
```
Ideally we'd just display the stack starting by `at Object.waitForElementByCss` but that would mean passing a sync error from each `this.chain` callsite and this may actually hide internal errors and I'm not sure of the interaction with actual chaining e.g. `browser.waitForElementByCSS().click().waitForElementByCSS()`.
This is improves readability of CI errors by a lot with minimal changes.
When performing a redirect() with an absolute path, action-handler
attempts to detect whether the resource is hosted by NextJS. If we
believe it is, we then attempt to stream it.
Previously we were not accounting for basePath which caused absolute
redirects to resources on the same host, but not underneath the
basePath, to be resolved by NextJS. Since the resource is outside the
basePath we resolve a 404 page which returns back as `text/x-component`
and is thus streamed back to the client within the original POST
request.
This PR adds a check for the presence of the basePath within absolute
redirect URLs. This fixes the above problem.
fixes#64413fixes#64557
---------
Signed-off-by: Chris Frank <chris@cfrank.org>
Co-authored-by: JJ Kasper <jj@jjsweb.site>
This allows us to set breakpoints and debug e2e tests without
encountering a timeout after 60 seconds. For example, using:
```
NEXT_E2E_TEST_TIMEOUT=1000000 NODE_OPTIONS=--inspect-brk pnpm test-dev test/e2e/app-dir/metadata/metadata.test.ts
```
Notably, this change also affects the turbopack dev tests in the CI,
where `NEXT_E2E_TEST_TIMEOUT` is currently set to 240 seconds. Since the
same env variable is also used for `jest.setTimeout()`, a test timeout
will now most likely occur due to jest timing out, and not a specific
playwright check (i.e. `waitForElementByCss` or `waitForCondition`)
timing out. This sounds acceptable to me.
Co-authored-by: JJ Kasper <jj@jjsweb.site>
### Why?
I really dislike the way `.chain` works right now, it shouldn't mutate
the `BrowserInterface`, this PR changes it so it's just a pure chain
without weird side effects.
One example with the current version (before this PR):
```
const el = browser.elementByCss('#version-2')
await el.text()
// throws
await el.text()
```
### Additional Changes
- removes selenium (which is completely unused)
- updates playwright
- makes the playwright tracing not error all the time
This reapplies the `experimental.missingSuspenseWithCSRBailout` option
to bail out during build if there was a missing suspense boundary when
using something that bails out to client side rendering (like
`useSearchParams()`). See #57642
Closes [NEXT-1770](https://linear.app/vercel/issue/NEXT-1770)
### What?
Adds the name of the test that's running when the browser is started to
the recording.
Also makes `RECORD_REPLAY=1` work without `run-tests.js`
Closes PACK-2206
### What?
While scrolled on a page, and when following a link to a new page and
clicking the browser back button or using `router.back()`, the scroll
position would sometimes restore scroll to the incorrect spot (in the
case of the test added in this PR, it'd scroll you back to the top of
the list)
### Why?
The refactor in #56497 changed the way router actions are processed:
specifically, all actions were assumed to be async, even if they could
be handled synchronously. For most actions this is fine, as most are
currently async. However, `ACTION_RESTORE` (triggered when the
`popstate` event occurs) isn't async, and introducing a small amount of
delay in the handling of this action can cause the browser to not
properly restore the scroll position
### How?
This special-cases `ACTION_RESTORE` to synchronously process the action
and call `setState` when it's received, rather than creating a promise.
To consistently reproduce this behavior, I added an option to our
browser interface that'll allow us to programmatically trigger a CPU
slowdown.
h/t to @alvarlagerlof for isolating the offending commit and sharing a
minimal reproduction.
Closes NEXT-1819
Likely addresses #58899 but the reproduction was too complex to verify.
Inferring the protocol from the request meta is not reliable when the next server is running over `http` but sitting behind an https proxy. This instead plumbs the experimental https flag through to the optimizer so we can more reliably determine the protocol
Fixes#55971
While investigating the HMR event types I noticed `pong` is not being used in Pages Router nor in App Router.
This PR removes the code that sends it as it's essentially dead code.
Follow up to https://github.com/vercel/next.js/pull/54081 -- this was
restoring the router tree improperly causing an error on bfcache hits
Had to override some default behaviors to prevent `forward` / `back` in
playwright from hanging indefinitely since no load event is firing in
these cases
Fixes#54184
Closes NEXT-1528
When an mpa navigation takes place, we currently push the user to the new route and suspend the page indefinitely (x-ref: #49058). When navigating back, if the browser opts into using the [bfcache](https://web.dev/bfcache/), it will remain suspended and `pushRef.mpaNavigation` will be true. This means that anything that would cause the component to re-render will trigger the mpa navigation again (such as hovering over another `Link`, as reported in #53347)
This PR checks to see if bfcache is being used by observing `PageTransitionEvent.persisted` and if so, resets the router state to clear out `pushRef`.
Closes NEXT-1511
Fixes#53347
This uses the new built-in progressive enhancement features of React.
These always use `multipart/form-data` atm. When one comes in that's not
a fetch, we can use `decodeAction` to get a resolved function.
This also ensures that we can test this by passing disableJavaScript to
tests. This disables JS for the context.
This PR implements new cache semantics for the app router on the client.
## Context
Currently, on the App Router, every Link navigation is prefetched and
kept forever in the cache. This means that once you visit it, you will
always see the same version of the page for the duration of your
navigation.
## This PR
This PR introduces new semantics for how the App Router will cache
during navigations. Here's a TL;DR of the changes:
- all navigations (prefetched/unprefetched) are cached for a maximum of
30s from the time it was last accessed or created (in this order).
- in addition to this, the App Router will cache differently depending
on the `prefetch` prop passed to a `<Link>` component:
- `prefetch={undefined}`/default behaviour:
- the router will prefetch the full page for static pages/partially for
dynamic pages
- if accessed within 30s, it will use the cache
- after that, if accessed within 5 mins, it will re-fetch and suspend
below the nearest loading.js
- after those 5 mins, it will re-fetch the full content (with a new
loading.js boundary)
- `prefetch={false}`:
- the router will not prefetch anything
- if accessed within 30s again, it will re-use the page
- after that, it will re-fetch fully
- `prefetch={true}`
- this will prefetch the full content of your page, dynamic or static
- if accessed within 5 mins, it will re-use the page
## Follow ups
- we may add another API to control the cache TTL at the page level
- a way to opt-in for prefetch on hover even with prefetch={false}
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation or adding/fixing Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
link NEXT-1011
Hendrik Liebau