Note that https://github.com/advisories/GHSA-6g55-p6wh-862q does not
affect Next.js users unless they build from untrusted source code (which
would have more severe security implications). We're merging this to
reduce noise from security scanners.
- 8.5.21 is 50h+ old so it can be installed
- 8.5.22 is still under 48h - valid at 2026-07-24 08:48:15 UTC
Closes: https://github.com/vercel/next.js/issues/96349
[`httpxy`](https://github.com/unjs/httpxy) is a faster and actively
maintained variant of `http-proxy`. It also doesn't need any manual
patches to avoid security vulnerabilities in Next.js.
`httpxy` is used by Nuxt and maintained by the Nitro team.
This adds `bench/dev-validation/`, wired as `pnpm bench:dev-validation`,
which measures how much dev-mode Cache Components validation contends
for the dev server's event loop during rapid navigation, and how much
running it on a worker thread relieves that. It toggles
`experimental.devValidationWorker` (added in the previous commit) to A/B
the two configurations on the same build. Until the worker
implementation lands the flag is inert and the A/B shows no delta.
The fixture generates one route per family (`client`, `server`,
`sprite`), each nested several layout segments deep under a `(routes)`
route group. Validation renders a combined payload at every URL depth,
so a deeper route means more validation work per navigation, which
mirrors a realistically deep app rather than a single flat segment. The
runner clicks a family's `<Link>` repeatedly, since navigating to the
current route re-renders and re-validates it on every click. The routes
carry no `instant` config because dev validation applies to page
segments by default at the warning level. The three families isolate the
client prerender, the Flight re-encode plus owner-stack work, and the
Flight payload size, respectively.
The signal is browser-observed TTFB taken from Playwright's own network
timing, because it includes the time a request waits for the event loop
while validation monopolizes it. We deliberately do not use the CLI's
logged request durations: the dev server starts that clock inside the
request handler, after the loop has already yielded to the request, so
the queue wait is invisible to it.
The runner prints each configuration's absolute TTFB (p50/p95/max) side
by side rather than a ratio. The time the worker frees is the validation
render's CPU, which is bounded, route-dependent, and does no IO, so a
ratio would overstate a win that does not scale with total request time.
Because the clicks are back-to-back the numbers are a worst case —
navigations that land inside the validation window — and the `max` tail
is the honest headline: it is the main-thread stall the worker removes.
Added a bunch of stuff to the bench.
---
**New metrics in the HTTP benchmark**
- Report TTFB per route (time to first body byte), next to total
latency.
- Report each route's document size, how many bytes are inline Flight
payload, and the Flight share.
**New script: `pnpm bench:render-pipeline:client`**
- Loads each route in Chrome with tracing and 4x CPU throttling, and
breaks down where client time goes: evaluating chunks, evaluating inline
Flight scripts, compiling, background parsing, GC, time to hydration,
and blocking time before hydration.
- Also prints FCP/LCP/DOMContentLoaded/load from the same trace, and JS
transferred vs parsed.
- Off by default, separate from the timing benchmark, since tracing
perturbs timing.
- Hydration time comes from a small client component added to the
fixture root layout that calls `performance.mark`.
**Bug fixes**
- The benchmark was replacing the fixture's `next.config.js` with an
empty one during runs.
- If the port was already taken, the benchmark could silently measure
whatever server was already running there. Both scripts now refuse to
start if something is already on the port.
- A server that died on startup used to look like a slow server; now it
errors immediately.
- One failed request used to abort the whole run and throw away all
results. Now it costs one sample and gets counted in `errors`.
- Killing an already-dead server used to hang the script.
- Bad flags now error upfront instead of crashing at the end.
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Fixes false-negative regex assertions — observed rejecting correct
solutions in recent eval runs — by grading those specific checks with
`@vercel/agent-eval`'s agentic LLM judge instead. Only the assertions
with demonstrated false negatives change; every deterministic check
stays.
- `agent-034-async-cookies`: `/await\s+cookies\s*\(\s*\)/` only matched
the naive `await cookies()` and rejected the correct (arguably better)
`await Promise.all([cookies(), headers()])` — observed failing most runs
of a strong model — and the no-sync-call lookbehind wrongly flagged the
bare `cookies()` inside the array. The four await-mechanics tests
collapse into one judge criterion: the promises must actually be awaited
before use, in any correct form. The theme-cookie and Accept-Language
content checks stay regex.
- `agent-030-app-router-migration-hard`: the home-page test rejected
solutions that didn't match one exact shape (e.g. data fetching
extracted to a helper means no literal `fetch(` in `page.tsx`). Its body
becomes one judge criterion (async Server Component, fetches during
server render, no `getServerSideProps`); the file-existence check and
the other seven tests stay deterministic.
The judge is pinned to **`claude-haiku-4-5`** in the generated
experiment configs — a cheap fixed grader, identical for every run
regardless of the model under test. Each criterion states the
requirement semantically, includes one reference solution to ground the
small grader, and fixes the standard of judgment (runtime correctness,
not style) — no enumerated failure modes, so the judge stays flexible
about equivalent forms.
Supporting changes: bumps `@vercel/agent-eval` `0.9.5` → `1.3.0` (ships
the judge runtime) and adapts `run-evals.js` to the 1.3 CLI (`run
baseline agents-md --force`; `--dry` maps to `status`); generated
experiments now run through the Vercel AI Gateway so `vc env pull` is
the only credential setup; converted fixtures exclude `EVAL.ts` from
tsconfig (the `@vercel/agent-eval/eval` import has no type declarations
— vercel-labs/agent-eval#166 — and `next build` type-checks the fixture,
so experiments running `scripts: ['build']` would otherwise fail on it,
same reason `EVAL.tsx` was already excluded). Not converted: `agent-040`
(already redesigned upstream in #94578) and `agent-041` (no diagnosed
false negative in the report).
Verified end-to-end from this repo: converted `agent-034` with codegen
`claude-sonnet-4-5` via the AI Gateway in a Vercel sandbox, judge pinned
to `claude-haiku-4-5` — passes, with the judge clause executing
in-sandbox.
**Running it:**
```bash
pnpm install # picks up @vercel/agent-eval 1.3.0
pnpm --filter=next build # pnpm eval packs the locally built next
vc env pull .env.local # the only credential: sandbox + AI Gateway (codegen and judge)
pnpm eval agent-034-async-cookies # baseline + AGENTS.md variants
pnpm eval agent-034-async-cookies --dry # preview only
```
<!-- NEXT_JS_LLM_PR -->
Instead of hardcoding the list, we now automatically detect any
`.browser` sibling and create a compiler alias for that module.
Together with a linting rule, we'll ensure no future regressions are
handled automatically (detection of conditional require and missing
.browser sibling).
This isn't the smoothest DX yet. Ideally we'd generate the list at
Next.js compile time (i.e. before publishing) to avoid having to rerun a
script (I'll follow-up). Generating the list during build time (next dev
or next build) is just another tiny cost users would have to pay.
we were invoking
```
xargs --null -n1 -I'{}' ...
```
to tell xargs that we want to:
- use 1 argument per invocation (`-n1`)
- `{}` is the placeholder (`-I'{}'`)
but it seems like `-I'{}'` already implies `-n1`. BSD xargs doesn't
care, but GNU xargs warns about it, so we get this in CI (e.g.
[here](https://github.com/vercel/next.js/actions/runs/25789455488/job/75751231855?pr=93801#step:36:54)):
```
xargs: warning: options --max-args and --replace/-I/-i are mutually exclusive, ignoring previous --max-args value
```
seems like both GNU and BSD xargs are happy if we just drop the `-n1`
and only use `-I{}`.
### What?
Converts every test under `test/integration/` to an isolated test
running through `nextTestSetup` (under `test/e2e/`, `test/production/`,
`test/development/`, or `test/unit/`), then deletes `test/integration/`
along with the legacy CI orchestration that was specific to it.
- `test/integration/` removed entirely (~327 test suites)
- New isolated suites added across the existing folders:
- `test/e2e/` — 175
- `test/production/` — 130
- `test/development/` — 43
- `test/unit/` — 1
- `.github/workflows/build_and_test.yml` and `run-tests.js` no longer
have any `integration` branches
- `nextTestSetup` gained a `baseUrl` option on `next.browser()` so a
small number of tests that drive their own proxy/static-export server
can keep using `next.browser(...)` instead of importing `next-webdriver`
directly
### Why?
`test/integration/` predated `nextTestSetup` and ran tests directly
against the source checkout via custom helpers (`launchApp`,
`nextBuild`, `nextStart`, `runNextCommand`, `webdriver`, `fetchViaHTTP`,
…). Each suite hand-rolled its own dev/start/build orchestration,
fixture mutation, and process management.
The isolated test model used by the rest of the repo gives each suite an
isolated working directory containing a packed `next.tgz` install, a
uniform `next.start()` / `next.build()` / `next.fetch()` /
`next.browser()` API, and the same lifecycle for dev, start, and deploy
modes — so a single set of assertions covers all three. Deploy-mode
skips and per-feature gates are expressed declaratively
(`skipDeployment`, `disableAutoSkewProtection`, `if (skipped) return`)
instead of branching on `process.env`.
Removing `test/integration/` lets us:
- Delete the bespoke orchestration code in the CI workflow and
`run-tests.js`
- Run every converted suite consistently in dev, start, and deploy modes
(where applicable)
- Reproduce every test locally with the same `pnpm
test-{dev,start}-{turbo,webpack}` commands; no separate `integration`
path
- Open the door to running `test/production` against deployments in the
future (the converted suites already declare `skipDeployment` so they
can be flipped on)
### How?
Mechanical conversion per suite, with targeted clean-ups:
1. **Per-suite conversion.** Each
`test/integration/<name>/test/index.test.{js,ts}` was rewritten into a
single `<name>.test.ts` under the right folder based on what the
original exercised:
- `launchApp` / dev-only assertions → `test/development/`
- `nextBuild` + `nextStart` / start-only assertions → `test/production/`
- Both → `test/e2e/`
- The one pure jsdom render check (`link-without-router`) → `test/unit/`
2. **API mapping.** Custom helpers were replaced by `nextTestSetup`
equivalents: `launchApp` → `next.start()`, `nextBuild` → `next.build()`,
`runNextCommand` → `next.runCommand`, `fetchViaHTTP` → `next.fetch`,
`webdriver(...)` → `next.browser(...)`. Fixture mutations switched from
raw `fs.writeFile`/`fs.rename` to `next.patchFile` (with the 3-arg
`runWithTempContent` callback when the change has a defined scope) and
`next.deleteFile`.
3. **Deploy-mode handling.** Suites that can't run in deploy mode (use
`patchFile` / `next.build()` / depend on local CLI output) declare
`skipDeployment: true` and early-return on the `skipped` boolean. Suites
where Vercel's edge mutates URLs (`&dpl=`, immutable assets) declare
`disableAutoSkewProtection: true`.
4. **`next.browser({ baseUrl })`.** A handful of tests
(`prerender-export`, `cdn-cache-busting`, `preload-viewport`, both
`react-virtualized` suites) need to drive a separate server (a
static-export server or an `http-proxy` instance) rather than the
Next.js process. Instead of importing `next-webdriver` directly, those
tests now pass `{ baseUrl: <port|url> }` to `next.browser()`. For the
proxy cases, the proxy was moved into `server.js` inside the fixture and
`http-proxy` declared via the `dependencies` option of `nextTestSetup`,
so the test runs with a fully isolated dependency graph.
5. **CI clean-up.** With `test/integration` gone, the `test
integration*` jobs and `integration-tests-manifest`-related logic in
`.github/workflows/build_and_test.yml` were removed, and `run-tests.js`
no longer has the `integration` test-folder branch.
6. **Validation.** The PR was iterated against multiple full CI runs;
the remaining failures on the latest run are pre-existing flakes
(segment-cache 60s `act` timeouts in turbopack-prod) or transient
infrastructure issues unrelated to the conversion.
Niklas Mischkulnig