Test fixtures that depend on monorepo packages (for example
`@next/third-parties` and `@next/mdx`) declared them at the `canary`
dist-tag, which installs the published npm canary instead of the build
under test. In deploy tests this also broke the remote install entirely:
the published canary's `peerDependencies` ranges (for example
`^16.0.0-beta.0`) reject the prerelease preview versions that
`NEXT_TEST_VERSION` installs for `next` (for example
`16.4.0-preview-<sha>-<date>`), so `npm install` failed with ERESOLVE.
Test dependencies can now be declared as `workspace:*`, which resolves
to the build from the current checkout in every test mode:
- dev/start: the locally packed tarball from `pack-for-isolated-tests`,
with a descriptive error when the package has no pack task or is not
part of the repository.
- deploy: the preview tarball of the tested commit when
`NEXT_TEST_VERSION` points at a preview build (preview tarballs already
rewrite their monorepo peer dependencies to the same preview URLs, so
peer resolution succeeds), falling back to the version in the worktree
otherwise. Private packages without published preview tarballs fail with
a descriptive error.
All existing tests that used `canary` for monorepo packages are migrated
to `workspace:*`.
Replaces the shared, static `PREVIEW_BUILDS_READ_TOKEN` secret with OIDC
tokens for reading auth-protected preview builds from vercel-packages
(see https://github.com/vercel/vercel-packages/pull/96):
- CI jobs mint a GH oidc token for the
`https://vercel-packages.vercel.app` audience
- deploys have a `.npmrc` written that uses the Vercel OIDC token
Tarball polling now also fails fast on 401/403 (retrying can't change
the authorization outcome) and failure messages include response headers
so request IDs are available for debugging.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
The "Test new and changed tests when deployed" jobs install Next.js from
the preview tarballs built for the commit under test, but nothing made
them wait for those tarballs to exist. Until #96438 the job was ordered
behind `test-prod` and the other new-test jobs, which delayed it long
enough that the tarballs had usually landed by the time it looked for
them. Now that it starts immediately it frequently fails with "Artifacts
not found for commit ...".
The ordering cannot be expressed as a `needs:` entry, because the
tarballs are published by a different workflow and, for pull requests
from branches in this repository, a different event than the
`build-and-test` run that consumes them. We therefore add a
`wait-for-preview-tarball` job that polls for the tarball instead, and
make both deploy test jobs depend on it. Doing the waiting in one cheap
job keeps the deploy test matrices off their 16-core runners until there
is something for them to install.
The new job is also listed in `tests-pass`. A dependency failure skips
the jobs that need it, and a skipped job does not count as a failure, so
without that entry a tarball that never arrives would have let the
required check pass with the deploy tests never having run.
`test-new-tests-deploy-cache-components` gets the same treatment,
dropping the incidental ordering it still inherited from
`test-cache-components-prod` and picking up the docs-only guard that
`test-new-tests-deploy` already had.
## What
Cache each passing test file's result in turbo's remote cache during CI runs. On retry attempts (`GITHUB_RUN_ATTEMPT > 1`), skip tests that already passed.
## Why
When `retry_test.yml` calls `rerun-failed-jobs`, the entire test shard re-runs from scratch — 20+ minutes to retry one flaky test. With this change, only the failed tests actually re-run on retry.
## How
- Cache key: `sha256(commit + test file + env fingerprint)` where env fingerprint includes `NEXT_TEST_MODE`, `IS_WEBPACK_TEST`, `IS_TURBOPACK_TEST`, etc.
- Cache value: `{"passed": true, "time": N}` (~50 bytes per test)
- On first attempt: all tests run normally, passes are cached
- On retry: cached passes are skipped, failures re-run. Newly passing tests get cached for further retries
- All caching gated on `CI` + `TURBO_TOKEN` + `GITHUB_SHA` — zero impact on local runs
- Uses existing `scripts/turbo-cache.mjs` client, all cache operations are fire-and-forget with try/catch
Fork the set of deployment tests so they run with both turbopack and
webpack. Now that turbopack is the default bundler we need this
coverage.
Because we are now running twice as many tests, decrease concurrency,
otherwise we hit rate limits on vercel. There were attempts to improve
rate limit recovery in the vercel CLI
(https://github.com/vercel/vercel/pull/14443 and
https://github.com/vercel/vercel/pull/14407) which helped but did not
solve the issue. Also there was an investigation into the API service
where we discovered some suspicious but ultimately correct code
(https://github.com/vercel/api/pull/55967).
The basic issue is that the vercel CLI polls `api-deployments-get`
fairly aggressively at the beginning of a deploymnet, and since we start
so many deployments in parallel we will always hit the rate limits. The
recovery logic in the CLI is good but has (reasonably) a fixed set of
retries it will attempt, inevitably some tasks get unlucky. So retrying
the test does work, but really we should slow down which is what we do
here. 😥https://github.com/vercel/next.js/actions/runs/20290161194/job/58272593085
Fixes PACK-5613
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Flake detection runs changed test(s) 6 times: 3 in Webpack, 3 in
Turbopack. for the purposes of detecting flakes, and to reduce the
burden on CI, this runs flake detection only with Turbopack (esp. now
that it's enabled by default).
Reverts vercel/next.js#84389
Attempt number 3, #84374 fixed propagation of bundler environment variables to vercel cli operations, to ensure the test configuration is respected.
[Deployment Tests Run 1](https://github.com/vercel/next.js/actions/runs/18146684793/job/51649631635). A fair number of failures.
[Deployment Tests Run 2](https://github.com/vercel/next.js/actions/runs/18154875270). After #84395. These still have a set of failures, but i confirmed that the deployment builds are running `webpack`.
The tests that are failing are related to 'prefetches' (test/e2e/app-dir/segment-cache/prefetch-runtime/prefetch-runtime.test.ts and test/e2e/app-dir/segment-cache/prefetch-layout-sharing/prefetch-layout-sharing.test.ts) and generally the error is a timeout. Recent deployment runs for canary releases are also failing, but on different tests.
Trying again after #[84419](https://github.com/vercel/next.js/pull/84419). [Deployment Tests Run 3](https://github.com/vercel/next.js/actions/runs/18173375427). Failures appeared flaky, rerunning failures...
Reverts vercel/next.js#84348 which itself reverted #84216.
In the 2nd attempt to make --turbopack a default behavior.
* adjust the Generate Pull Request States action to specify `--webpack`
* set the `IS_WEBPACK_TEST` flag in the deployment e2e tests (which are now [passing](https://github.com/vercel/next.js/actions/runs/18113062122))
* make the warning about having a webpack config without a turbopack config more verbose and consistently an error
Make Turbopack the default bundler for Next 🚀
## What
Add a `--webpack` flag so users can select webpack
Change behavior so if neither `--turbopack` or `--webpack` is set we default to Turbopack.
If we have defaulted the build to turbopack and we observe that the user has a `webpack` config in their next config but no turbopack config, then we issue a warning about this being a potential problem, if this happens during `next build` the warning becomes an error and fails the build. The solution for users is just to explicitly set `--webpack` or `--turbopack`
There were a number of subtle issues
* some users directly set the `TURBOPACK` environment variable, this PR adds support for that though users should really be passing `--turbopack` so it will not be documented
* rspack is enabled via a plugin that sets an environment variable when loading next config, which means it happens way too late!
* For builds this isn't too bad, we just have to recompute the `Bundler` after loading the config. For `dev` the parent process can get out of sync with the child, but this is only really relevant for telemetry and for that we already load the config in the parent so just defer computing `isTurboSession`.
Most of this is about fixing package.json scripts and CI builds configs.
* For package.json
* i added aliases e.g. `test-dev` == `test-dev-webpack` but preserved the old names. In the long run we will want to remove the unsuffixed aliases
* this required some modifications to existing configs to ensure everything was setting the correct env variables
* For ci, i added a new `IS_WEBPACK_TEST` variable to a number of tests but preserved all names.
* Again, in the future it would make sense to rename ci jobs but that is deferred for right now.
* This also makes it clear that a set of tests scenarios (e.g. ppr, experimental, test-new-tests-*) never run with turbopack. This is not addressed right now but should be in the future.
## Why
Today, the default bundler for next is webpack but with turbopack becoming stable it is time to just ship it. Turbopack is already recommended for dev and builds. Create Next App also steers users towards Turbopack. According to telemetry we already have about 50% of all dev sessions and build adoption is growing. So fundamentally why are we even asking users to make a decision?
### What?
This PR fixes critical URL mutation issues in Next.js routing that were
causing state pollution between requests and breaking compatibility
between App Router and Pages Router during rewrite handling.
### Why?
Several interconnected problems were discovered in the routing system:
1. **URL Object Mutation**: The original `parsedUrl` objects were being
mutated during rewrite processing, causing state to leak between
requests and potentially affecting subsequent routing decisions.
2. **Router Type Confusion**: The routing system lacked explicit
tracking of whether a route belonged to the App Router or Pages Router,
leading to incorrect query parameter handling and normalization logic
being applied inconsistently.
3. **Query Parameter Pollution**: App Router RSC payloads were being
polluted with rewrite query parameters that should only be applied to
Pages Router routes, breaking the separation of concerns between the two
routing systems.
4. **Catch-all Route Normalization**: Pages Router catch-all routes
require specific array normalization (e.g., `["hello", "world"]`) but
this was being applied inconsistently or to the wrong router type.
5. **Middleware Filtering Bypass**: The x-matched-path header handling
was bypassing proper middleware query filtering in deployment scenarios.
These issues manifested as test failures where Pages Router catch-all
routes weren't receiving proper array values, App Router routes were
getting contaminated with rewrite parameters, and the routing behavior
was inconsistent between development and deployment environments.
### Dependant PR's
https://github.com/vercel/vercel/pull/13927
[NAR-398](https://linear.app/vercel/issue/NAR-398)
This updates our workflow for testing new tests against deploy mode to
allow running against forks as we don't want to introduce flakey tests
in this mode even if opened from forks.
As of #77894, there's a new way to indicate Turbopack should run, which breaks flaky tests detection with Turbopack. This swaps out the flag to unblock my downstack PR.
We run flake detection on PRs to help catch flaky tests before they hit
canary, but we currently only run flaky test detection in Webpack and
not Turbopack. This is problematic because there's sufficient forking
logic in tests for the different bundlers & we've seen a high likelihood
of a test to flake more frequently in one bundler vs another due to
things like differing compilation speeds and differing implementation of
core features.
This will add some additional time to this CI job to account for needing
to run the tests 6 times, but will hopefully catch things before they
become a problem later. I think we could also consider reducing the
number of times per bundler to be 2 if it turns out to be adding too
much time.
This updates the flake detection job to run the tests multiple times in
Turbopack as well.
CI run:
https://github.com/vercel/next.js/actions/runs/11822662639/job/32940264160?pr=72773
`x-middleware-cache: "no-store"` in Pages router is a way to signal to
the client that it should not store the response in the cache. However
in certain circumstances, namely when `unstable_skipClientCache` is
true, the data request would be awaited and then stored in the
`inflightCache` regardless of the header.
The original implementation of this in the router has logic to delete
the response from the inflight cache after the request has fulfilled
because `inflightCache` stores the unresolved promise. But in this
optimistic prefetch case, when we're only storing it in the cache once
the request is fulfilled, we can prevent a race condition where the
ignored prefetch is erroneously re-added to the cache by ensuring it's
never added to the cache to begin with if the response says not to.
Fixes#66881
Closes NEXT-3550
Leverages the work from https://github.com/vercel/next.js/pull/66445 to
download artifacts for a particular commit SHA, so that it can be passed
to the `test-deploy` script.
This will let us run deploy tests on PRs, ensuring that they're run from
the Next.js on the target branch rather than `canary`.
This waits until `test-new-tests-dev` and `test-new-tests-start`
complete for 2 reasons:
- No reason to waste deploy resources if the changed/added tests don't
even work in dev/start
- It gives time for deploy-tarball to finish
Sample run:
https://github.com/vercel/next.js/actions/runs/9865244781/job/27241944533?pr=67612#step:28:54
This splits out the existing logic for detecting changed/added tests
into a separate util, so it can be leveraged by #67612.
No changes in functionality, aside from replacing informational logs
with `console.log` rather than `console.error` and a tweak to arg
parsing.
To help detect when newly added/changed assertions are flakey this adds
a job to our build and test workflow to re-run them 3 times. If the
changed test is an E2E test it will re-run in both development and
production mode to ensure it's not flakey specifically in one of those
modes.
Test run with changed test can be seen here
https://github.com/vercel/next.js/actions/runs/8511797725/job/23312158523?pr=63943
Closes NEXT-2973
Sebastian "Sebbie" Silbermann