Reverts vercel/next.js#84389
Attempt number 3, #84374 fixed propagation of bundler environment variables to vercel cli operations, to ensure the test configuration is respected.
[Deployment Tests Run 1](https://github.com/vercel/next.js/actions/runs/18146684793/job/51649631635). A fair number of failures.
[Deployment Tests Run 2](https://github.com/vercel/next.js/actions/runs/18154875270). After #84395. These still have a set of failures, but i confirmed that the deployment builds are running `webpack`.
The tests that are failing are related to 'prefetches' (test/e2e/app-dir/segment-cache/prefetch-runtime/prefetch-runtime.test.ts and test/e2e/app-dir/segment-cache/prefetch-layout-sharing/prefetch-layout-sharing.test.ts) and generally the error is a timeout. Recent deployment runs for canary releases are also failing, but on different tests.
Trying again after #[84419](https://github.com/vercel/next.js/pull/84419). [Deployment Tests Run 3](https://github.com/vercel/next.js/actions/runs/18173375427). Failures appeared flaky, rerunning failures...
Reverts vercel/next.js#84348 which itself reverted #84216.
In the 2nd attempt to make --turbopack a default behavior.
* adjust the Generate Pull Request States action to specify `--webpack`
* set the `IS_WEBPACK_TEST` flag in the deployment e2e tests (which are now [passing](https://github.com/vercel/next.js/actions/runs/18113062122))
* make the warning about having a webpack config without a turbopack config more verbose and consistently an error
### What?
Move persistent caching for `next build` into a separate config option: `turbopackPersistentCachingForBuild`.
And `next dev` to `turbopackPersistentCachingForDev`.
Make Turbopack the default bundler for Next 🚀
## What
Add a `--webpack` flag so users can select webpack
Change behavior so if neither `--turbopack` or `--webpack` is set we default to Turbopack.
If we have defaulted the build to turbopack and we observe that the user has a `webpack` config in their next config but no turbopack config, then we issue a warning about this being a potential problem, if this happens during `next build` the warning becomes an error and fails the build. The solution for users is just to explicitly set `--webpack` or `--turbopack`
There were a number of subtle issues
* some users directly set the `TURBOPACK` environment variable, this PR adds support for that though users should really be passing `--turbopack` so it will not be documented
* rspack is enabled via a plugin that sets an environment variable when loading next config, which means it happens way too late!
* For builds this isn't too bad, we just have to recompute the `Bundler` after loading the config. For `dev` the parent process can get out of sync with the child, but this is only really relevant for telemetry and for that we already load the config in the parent so just defer computing `isTurboSession`.
Most of this is about fixing package.json scripts and CI builds configs.
* For package.json
* i added aliases e.g. `test-dev` == `test-dev-webpack` but preserved the old names. In the long run we will want to remove the unsuffixed aliases
* this required some modifications to existing configs to ensure everything was setting the correct env variables
* For ci, i added a new `IS_WEBPACK_TEST` variable to a number of tests but preserved all names.
* Again, in the future it would make sense to rename ci jobs but that is deferred for right now.
* This also makes it clear that a set of tests scenarios (e.g. ppr, experimental, test-new-tests-*) never run with turbopack. This is not addressed right now but should be in the future.
## Why
Today, the default bundler for next is webpack but with turbopack becoming stable it is time to just ship it. Turbopack is already recommended for dev and builds. Create Next App also steers users towards Turbopack. According to telemetry we already have about 50% of all dev sessions and build adoption is growing. So fundamentally why are we even asking users to make a decision?
# Add automated benchmark runner for module-cost
## What?
This PR adds an automated benchmark runner for the module-cost benchmark, allowing for consistent measurement of module loading and execution times.
## Why?
It is to tedious and error prone to run the benchmark.
## Usage
Build `next` and turbopack however you like
```
pnpm i
pnpm prepare-bench
pnpm build-webpack (or build-turbopack)
pnpm benchmark
```
## How?
v0 mostly
### What?
Add a benchmark that requires/imports nearly 10k of empty modules.
Measures load and execution time for these modules.
Might also be useful to measure the compile overhead of modules.
Closes PACK-4263
This:
- Adds support for using `config.turbopack` for Turbopack. The accepted value for this option is identical to the previous experimental one without the deprecated field `loaders`.
- Deprecates `config.experimental.turbo`. Warns on its use, but will continue to accept it, merging it with `config.turbopack` while preferring fields on `config.turbopack`.
- also see https://github.com/vercel/next.js/pull/77850#pullrequestreview-2749738760
Test Plan:
- [x] Convert use of `config.experimental.turbo` in tests to `config.turbopack`
- [x] Automated test for accepting the experimental option for compatibility, as well as the config merging
Title: Fix typo in .gitignore file: change "commiting" to "committing"
(single "t" to double "t")
Description:
This pull request addresses a minor typo in the .gitignore file, where
the word "commiting" was corrected to "committing" by changing a single
"t" to a double "t". This change ensures that the spelling is consistent
with standard English usage.
Checklist:
I have run pnpm prettier-fix to ensure the code is properly formatted.
I have reviewed the [Docs Contribution
Guide](https://nextjs.org/docs/community/contribution-guide) to ensure
my changes adhere to the guidelines.
Related Issues: None
Notes:
This change does not affect functionality or require additional testing.
No additional documentation is required for this typo fix.
- Fixes https://github.com/vercel/next.js/issues/72378
Co-authored-by: Sam Ko <sam@vercel.com>
Changes `--turbo` -> `--turbopack` to avoid confusion.
- Updated docs
- Updates create-next-app -- For create-next-app it's a rename as
otherwise there is an ordering problem with the prompts
- For the CLI `next dev --turbo` is still supported, will eventually be
a warning in a future version to swap with `--turbopack` but is not a
requirement today, can be handled automatically by the upgrade codemod
- New CLI flag: `next dev --turbopack`
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: Will Binns-Smith <wbinnssmith@gmail.com>
Fairly basic PR that just fixes the `basic-app` benchmark application.
The api route was throwing an error since it wasn't returning a
`Response`, and renamed some `pages/` directory routes so they become
available.
Since we reset the test project on every e2e CI run, deployment protection is automatically enabled by default.
This adds an option to the reset project workflow to disable deployment protection. Our test runners need to be able to hit these pages from an unauthenticated browser in order for the tests to work.
Verified tests are running properly in [this run](https://github.com/vercel/next.js/actions/runs/6971348806/job/18971225559) (fixing any failing tests themselves are out of scope for this PR; will evaluate once the run finishes)
Closes NEXT-1732
We have identical `resetProject` code used in `bench/vercel` and our e2e workflow action -- this updates the `resetProject` script to side-effects free (hence removing the env var) and shared between bench & e2e
Closes NEXT-1731
## What?
In Next, rendering a route involves 3 layers:
- the routing layer, which will direct the request to the correct route to render
- the rendering layer, which will take a route and render it appropriately
- the user layer, which contains the user code
In #51831, in order to optimise the boot time of Next.js, I introduced a change that allowed the routing layer to be bundled. In this PR, I'm doing the same for the rendering layer. This is building up on @wyattjoh's work that initially split the routing and the rendering layer into separate entry-points.
The benefits of having this approach is that this allows us to compartmentalise the different part of Next, optimise them individually and making sure that serving a request is as efficient as possible, e.g. rendering a `pages` route should not need code from the `app router` to be used.
There are now 4 different rendering runtimes, depending on the route type:
- app pages: for App Router pages
- app routes: for App Router route handlers
- pages: for legacy pages
- pages api: for legacy API routes
This change should be transparent to the end user, beside faster cold boots.
## Notable changes
Doing this change required a lot of changes for Next.js under the hood in order to make the different layers play well together.
### New conventions for externals/shared modules
The big issue of bundling the rendering runtimes is that the user code needs to be able to reference an instance of a module/value created in Next during the render. This is the case when the user wants to access the router context during SSR via `next/link` for example; when you call `useContext(value)` the value needs to be the exact same reference to one as the one created by `createContext` earlier.
Previously, we were handling this case by making all files from Next that were affected by this `externals`, meaning that we were marking them not to be bundled.
**Why not keep it this way?**
The goal of this PR as stated previously was to make the rendering process as efficient as possible, so I really wanted to avoid extraneous fs reads to unoptimised code.
In order to "fix" it, I introduced two new conventions to the codebase:
- all files that explicitly need to be shared between a rendering runtime and the user code must be suffixed by `.shared-runtime` and exposed via adding a reference in the relevant `externals` file. At compilation time, a reference to a file ending with this will get re-written to the appropriate runtime.
- all files that need to be truly externals need to be suffixed by `.external`. At compilation time, a reference to it will stay as-is. This special case is needed mostly only for the async local storages that need to be shared with all three layers of Next.
As a side effect, we should be bundling more of the Next code in the user bundles, so it should be slightly more efficient.
### App route handlers are compiled on their own layer
App route handlers should be compiled in their own layer, this allows us to separate more cleanly the compilation logic here (we don't need to run the RSC logic for example).
### New rendering bundles
We now generate a prod and a dev bundle for:
- the routing server
- the app/pages SSR rendering process
- the API routes process
The development bundle is needed because:
- there is code in Next that relies on NODE_ENV
- because we opt out of the logic referencing the correct rendering runtime in dev for a `shared-runtime` file. This is because we don't need to and that Turbopack does not support rewriting an external to something that looks like this `require('foo').bar.baz` yet. We will need to fix that when Turbopack build ships.
### New development pipeline
Bundling Next is now required when developing on the repo so I extended the taskfile setup to account for that. The webpack config for Next itself lives in `webpack.config.js` and contains the logic for all the new bundles generated.
### Misc changes
There are some misc reshuffling in the code to better use the tree shaking abilities that we can now use.
fixes NEXT-1573
Co-authored-by: Alex Kirszenberg <1621758+alexkirsz@users.noreply.github.com>
This replaces the existing recursive directory reading beheviour with a more efficient implementation. Rather than previously doing actual recursion with the function calls to go deeper into each directory, this has been rewritten to instead use a while loop and a stack. This should improve memory usage for projects with very deep directory structures.
The updated design that performs directory scanning on all directories found at each layer, allowing more filesystem calls to be sent to the OS instead of waiting like the previous implementation did.
**Currently the new implementation is about 3x faster.**
I don't believe these benches were intended to run as part of the monorepo `dev` task. This renames the `dev` task to `dev-application` similar to `build-application` (which I assume was aliased for a similar reason)
Fixes#54592
## What
Adds a reworked version of webpack's [built-in memory cache garbage
collection
plugin](https://github.com/webpack/webpack/blob/853bfda35a0080605c09e1bdeb0103bcb9367a10/lib/cache/MemoryWithGcCachePlugin.js#L15)
that is more aggressive in cleaning up unused modules than the default.
The default marks 1/5th of the modules as "up for potentially being
garbage collected". The new plugin always checks all modules. In my
testing this does not cause much overhead compared to the current
approach which leverages writing to two separate maps. The change also
makes the memory cache eviction more predictable: when an item has not
been accessed for 5 compilations it is evicted from the memory cache, it
could still be in the disk cache.
In order to test this change I had to spin up the benchmarks but these
were a bit outdated so I've cleaned up the benchmark applications.
---------
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
<!--
Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change that you're making:
-->
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see
[`contributing.md`](https://github.com/vercel/next.js/blob/canary/contributing.md)
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the
feature request has been accepted for implementation before opening a
PR.
- [ ] Related issues linked using `fixes #number`
- [ ]
[e2e](https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see
[`contributing.md`](https://github.com/vercel/next.js/blob/canary/contributing.md)
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm build && pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing
doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
Fixing + doing some maintenance as I was using it for some things.
Changes:
- fixed a bug related to how React is injected
- added a progress tracker for the numbers of requests
- retry on 500
- added a param for crashing the lambda manually (useful for cold boots)
- added a param to skip build, useful if you want to retest the same variation
- don't crash when there's no data to compare
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see [`contributing.md`](https://github.com/vercel/next.js/blob/canary/contributing.md)
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] [e2e](https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs) tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see [`contributing.md`](https://github.com/vercel/next.js/blob/canary/contributing.md)
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm build && pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
Next.js 13 will require React 18.
In this PR I've only updated the peerDependency and removed the test runs in GH actions. Further cleanup will follow later, this allows us to remove the code supporting it later.
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
Co-authored-by: Jiachi Liu <4800338+huozhi@users.noreply.github.com>
This PR adds a script to run a minimal Next server in order to reproduce more easily the cold boot conditions in production.
You can use it by using `pnpm start` after having built your app with `pnpm next-react-exp build bench/minimal-server/benchmar-app` from the root of this repo.
<img width="338" alt="image" src="https://user-images.githubusercontent.com/11064311/195102966-6997b957-1e6b-433f-b611-e0eaec9462fa.png">
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the feature request has been accepted for implementation before opening a PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see `contributing.md`
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
This PR adds the benchmarking script I've been using for #40251 to
measure the performance improvements that we make to the Edge SSR
runtime.
This tool:
- uploads two version of the benchmarking project to Vercel, one with
the latest canary and the one with your current local changes in dist
(don't forget to build!)
- runs some tests against the published url to measure TTFB
- displays a nice chart and table
What this doesn't do (yet):
- allow you to choose which URL to compare
- allow you to change the measured metric
- run a battery of differnet test
## Bug
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Errors have a helpful link attached, see `contributing.md`
## Feature
- [ ] Implements an existing feature request or RFC. Make sure the
feature request has been accepted for implementation before opening a
PR.
- [ ] Related issues linked using `fixes #number`
- [ ] Integration tests added
- [ ] Documentation added
- [ ] Telemetry added. In case of a feature if it's used or not.
- [ ] Errors have a helpful link attached, see `contributing.md`
https://user-images.githubusercontent.com/11064311/191270204-04447e20-5a40-43a9-bcda-b7eaeb3d270a.mov
## Documentation / Examples
- [ ] Make sure the linting passes by running `pnpm lint`
- [ ] The "examples guidelines" are followed from [our contributing
doc](https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md)
Co-authored-by: kodiakhq[bot] <49736102+kodiakhq[bot]@users.noreply.github.com>
Sebastian "Sebbie" Silbermann
