@wbinnssmith added these scripts a long time ago at my suggestion/urging, but I think they ended up bad for a couple reasons that were hard to foresee at the time:
- afsctool operates in-place and doesn't atomically write the compressed file, so if the process gets interrupted, your `target` directory is corrupted. https://github.com/Dr-Emann/applesauce is better for this reason.
- This doesn't acquire the cargo lock, and modifying files while cargo is running is a good way to get corruption of the `target` directory.
https://github.com/bgw/cargo-apfs-compress is my latest attempt at solving this, though I don't have a `LaunchAgents` config for it.
Regarding `node_modules`: pnpm creates reflinks from a shared global store on apfs. Trying to compress these reflinked files is just going to hurt you because it'll break the data deduplication that would've otherwise happened.
When running `next build --debug-prerender`, React owner stacks are now
captured and displayed in prerender error output. This makes it much
easier to diagnose which component triggered uncached I/O or accessed
request data without Suspense. Previously, `--debug-prerender` only
enabled source maps and disabled minification. Now it also auto-enables
`allowDevelopmentBuild` and sets `NODE_ENV=development`, which loads
React development builds where `captureOwnerStack()` is available.
The main challenge is that with `NODE_ENV=development`, both server and
client bundles include dev-only code paths (HMR, WebSocket connections,
dev overlay, debug channel, etc.) that expect a running dev server. We
don't want these when using `next start`. To solve this, we introduce
`process.env.__NEXT_DEV_SERVER`, an internal env var that is truthy only
during `next dev`. In client bundles, it's inlined at build time (`'1'`
for `next dev`, `''` for `next build`). In production server runtime
bundles, it's inlined as `''` for dead-code elimination. In development
server runtime bundles, it's left as a runtime check because those
bundles are shared between `next dev` (where it's set) and `next build
--debug-prerender` (where it's not). Meanwhile, `NODE_ENV` continues to
control React's dev/prod mode and error formatting, which is exactly
what we want for `--debug-prerender`.
This also replaces the previous `renderOpts.dev` / `workStore.dev`
pattern, which was unreliable because `RouteModule.isDev` was derived
from `NODE_ENV` at compile time. When `allowDevelopmentBuild` set
`NODE_ENV=development`, `isDev` would be compiled as `true` and
incorrectly activate all dev guards during `next start`.
Key changes:
- `config.ts` auto-enables `allowDevelopmentBuild` and sets
`NODE_ENV=development` when `--debug-prerender` is active
- `define-env.ts` inlines `__NEXT_DEV_SERVER` into all bundles (truthy
for dev, falsy for build) so dev-server features are dead-code
eliminated in production and `--debug-prerender` builds
- `next-dev.ts` and `next.ts` set `__NEXT_DEV_SERVER` in the process
environment for externalized server-side code
- `renderOpts.dev` and `workStore.dev` are removed — all consumers now
use `__NEXT_DEV_SERVER` (for dev-server features) or `NODE_ENV` (for
error formatting that should work in both dev and `--debug-prerender`
builds)
- `patch-error-inspect.ts` devirtualizes React server URLs in source map
URLs so they display as readable file paths
When digging into failing deployment tests, I often will re-run the test
itself without wanting to change anything about Next.js or the test
application, especially when I suspect a logic bug in the test itself or
need to figure out what's causing a flake.
This adds an environment variable that can be used to skip the
build/deploy step and will use an existing deployment URL to run the
tests against.
I suggest reviewing this [without
whitespace](https://github.com/vercel/next.js/pull/88829/changes?w=1).
This makes certain aggregations trivial since we are are guaranteed to only traverse relevant edges.
Use it to fix a bug in the async module identification logic. Previously we would aggregate async cycles after propagating 'asyncness' through the DFS post order traversal, but this could cause us to fail to propagate async to all reverse dependencies depending on which part of a cycle a node would happen to visit first.
By traversing starting from the async modules we just need to mark everything we find and not worry about cycles at all since the underlying DFS mechanism will ensure we don't loop. In addition to only requiring a single pass, this is guaranteed to visit fewer nodes in that single pass.
One caveat is that reverse traversals only work within a single `SingleModuleGraph` so a debug assert was added to prevent misuse
Closes#85988Closes#86391
Closes PACK-5806
Written with @bgw.
- Transitions to using `tsx` for script execution
- Replaces `patch-package` with TypeScript implementation
- Implements argument parsing and help message with `yargs`
- Uses `execa` for some command execution
Test Plan: Run `pnpm pack-next`, `pnpm swc-build-wasm`, `pnpm unpack-next`, and `pnpm sweep` without errors.
This reverts the bundler change part while we investigate source map
issue but keeps the externals optimize preventing the runtime bundle
from inflating back to over `4MB`.
This reverts commit 98b0700e97
This reverts commit 04c3f0b29c
### Improving Documentation
Hello, I’ve added missing punctuation and an inline code block.
The `package.json` file has an inline code block on line 42, but the one
in the heading was missing.
Co-authored-by: JJ Kasper <jj@jjsweb.site>
I commonly add a `debug` script that allows me to attach a debugger when running Next.js in the isolated test dirs.
Now we always add it and make it work just like `debug` from the monorepo. So after the test you can `cd` into the created test directory and run `pnpm debug dev`. `debug` is just an alias for `next` with some additional configurations:
- tracing deprecations
- no telemetry
- no canary checks
- debugger attachable
- source maps
`debug` also ensures the SWC version that was used during the test is used. Previously it used the cached version which isn't necessarily the one built in this specific worktree.
Noticed this after rebuilding on top of #71688
Most Linux distributions default to `gcc` for their default `cc`
implementation (e.g. via Debian's `build-essential` meta-package, which
we can probably implicitly assume is installed), which means that they
won't typically have `clang` installed.
`librocksdb-sys` seems to always depend on `clang` on Linux. It looks
like this might be a consequence of using
[rust-bindgen](https://github.com/rust-lang/rust-bindgen) (though I'm
confused why this didn't come up earlier), as it looks like rocksdb can
otherwise build with gcc.
This shouldn't be an issue for macos, since `clang` is the default `cc`
on that platform.
## Description
Some links navigate pages docs.
Then update app router ones.
### Improving Documentation
- [x] Run `pnpm prettier-fix` to fix formatting issues before opening
the PR.
- [x] Read the Docs Contribution Guide to ensure your contribution
follows the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
Co-authored-by: JJ Kasper <jj@jjsweb.site>
# Summary
Add missing comments path to which #61416 changed.
### Improving Documentation
- [x] Run `pnpm prettier-fix` to fix formatting issues before opening
the PR.
- [x] Read the Docs Contribution Guide to ensure your contribution
follows the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
Co-authored-by: JJ Kasper <jj@jjsweb.site>
## What is `pack-next`?
This is a script we use for developing turbopack. It builds tarballs for
all the next.js packages, including `next-swc`, and then provides
instructions about how to modify a project's `package.json` to install
from that tarball.
## What does this PR do?
If you provide a `--project ~/path/to/directory` argument to `pnpm
pack-next`, we'll auto-update the `package.json` there for you, instead
of telling you how to do it manually!
This also understands the React 19 RC peer dependencies, which is
something that was missing from the generated instructions.
## Supported Package Managers
I've tested this with `npm` and `pnpm`.
- `bun` fails with `ENOTDIR` for unknown reasons (maybe it's trying to
handle the local tarball as a directory instead of a tarball?).
- `yarn` fails due to an upstream issue:
https://github.com/yarnpkg/yarn/issues/6339
This does generate the entries for yarn's `resolutions` feature,
assuming the upstream issue is eventually fixed.
## Why Rust and `cargo xtask`?
https://github.com/matklad/cargo-xtask
I'd love for these Turbopack-centric scripts to be written in Rust with
xtask, since they're mostly touched by the turbopack team.
However, it's also hard to justify putting too much effort into these
development scripts, so I don't want to rewrite the whole thing right
now.
I figured I'd write the new functionality in Rust and call it from the
old JS script.
## Test Plan
Perform a fresh checkout of shadcn/ui, and then:
```
pnpm pack-next --project ~/shadcn-ui/apps/www/
```
```
Found workspace! Patching "/home/bgw.linux/shadcn-ui/package.json" with overrides.
Successfully patched "/home/bgw.linux/shadcn-ui/package.json" to use local tarball files!
Run `pnpm i` or `npm i` to install the overrides
```
Check the resulting changes to `~/shadcn-ui`:
```diff
diff --git a/package.json b/package.json
index be47ad7..262c13b 100644
--- a/package.json
+++ b/package.json
@@ -63,12 +63,31 @@
"tsx": "^4.1.4",
"turbo": "^1.9.9",
"vite-tsconfig-paths": "^4.2.0",
- "vitest": "^0.31.1"
+ "vitest": "^0.31.1",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar"
},
"devDependencies": {
"@types/node": "^20.11.27",
"@types/react": "^18.2.65",
"@types/react-dom": "^18.2.22",
"typescript": "^5.5.3"
+ },
+ "overrides": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
+ },
+ "resolutions": {
+ "next": "file:/home/bgw.linux/next.js/tarballs/next.tar",
+ "@next/mdx": "file:/home/bgw.linux/next.js/tarballs/next-mdx.tar",
+ "@next/env": "file:/home/bgw.linux/next.js/tarballs/next-env.tar",
+ "@next/bundle-analyzer": "file:/home/bgw.linux/next.js/tarballs/next-bundle-analyzer.tar",
+ "@next/swc": "file:/home/bgw.linux/next.js/tarballs/next-swc.tar",
+ "react": "19.0.0-rc-7771d3a7-20240827",
+ "react-dom": "19.0.0-rc-7771d3a7-20240827"
}
}
```
### What?
Ports a few more useful scripts from nextpack, namely:
- `patch-next`
- `sweep`
- `build-native` (to clean up incremental artifacts on compiler panics)
- the macOS compression agent
Copies changes from https://github.com/vercel/turbo/pull/8166, and
updates contributing documentation to include the installation of lld.
> **What's wrong with `ld`?** It's very slow and uses a lot of memory.
>
> **Why `lld`?** It's fast, mature, and well-supported. Meta and Google
use it for all their linking workloads. We're already using it for macos
and x86-64 Windows. There is [ongoing work to make it the default for
rustc](https://github.com/rust-lang/rust/issues/71515), and it already
is default on a few platforms.
>
> **Why not `mold`?** Mold is generally faster, but the margin is slim
enough for our workloads that it doesn't really matter. Mold only
recently got support for LTO, doesn't support v0 rust symbol demanging,
doesn't support BOLT (though we don't use that yet), etc. Mold is
maturing quickly, but `lld` still seems like the "safer" choice.
This brings over the utility scripts `pnpm pack-next` and `pnpm
unpack-next path/to/app` from Nextpack, along with the documentation,
which has been added to `contributing/core/developing.md`.
~Due to the entry preloading that was enabled by default in #65289, VS
Code is no longer able to map symbols correctly for production builds.
As a temporary solution, we can disable the preloading when a debugger
is started until a proper fix is found.~
By optionally building the Next.js source code with
`NEXT_SERVER_EVAL_SOURCE_MAPS=1` we can ensure, even with the bundled
next server in the node runtime, that the original names are shown in
the debug "Variables" section.
In addition, the `sourceMapPathOverrides` are improved to cover the
different variations of source mapping URLs. We can now also set
breakpoints in the `example/...` and `test/e2e/...` server components.
~Supersedes #66229~
#### Before
<img width="1276" alt="Screenshot 2024-05-27 at 21 36 52"
src="https://github.com/vercel/next.js/assets/761683/f2840c35-f683-445d-bc95-cac9f719d8e7">
#### After
<img width="1276" alt="Screenshot 2024-05-27 at 21 38 42"
src="https://github.com/vercel/next.js/assets/761683/c801c123-3163-46c3-b442-5b72c0a5d51d">
When developing in Next.js repo, the maintainers / contributors
sometimes need to build swc native files.
Added a script `swc-build-native` to run the command `pnpm
--filter=@next/swc build-native` which was verbose to run.
GitHub recommends blobless clones over shallow clones:
https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/
> For these reasons we do not recommend shallow clones except for builds
that delete the repository immediately afterwards. Fetching from shallow
clones can cause more harm than good!
I've been using blobless clones for development for the last couple
weeks. The blobless clone has the benefit of including the full
repository history (for the cloned branch). Tools like `git blame` will
be slower as git fetches the related blobs on-demand.
Benchmarks (using all the flags in the docs):
- The blobless clone is faster on my machine, taking 11.1 seconds versus
13.1 seconds for the shallow clone.
- The blobless clone takes up 256M on disk, versus 244M for the shallow
clone. It's worse, but not by much.
## What?
Adds instructions for how to run tests with Turbopack and how to run
tests while recording using Replay.io.
I've also updated the instructions to mention `test-dev` and
`test-start` which are the commands we generally use.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
Closes NEXT-2031
This PR removes the wasm target for the next-swc build for the next major version. The main motivation is that Turbopack does not support targeting wasm yet, and it would be a significant amount of work to add. We plan to make Turbopack the default zero-config experience in a minor version, possibly before we are able to support a wasm target, so we need to make this breaking change now. We also plan to make more improvements to the webpack experience with shared Rust code, which we have so far been blocked from implementing because of the current wasm restrictions. We would like to support a wasm target again in the future, but cannot say at this time when that would be.
Closes WEB-1865
### What?
Moving maintenance to a separate repository.
### Why?
I want to make these actions reusable in other projects as they seem to work well in the Next.js repository.
### How?
The code is moved to https://github.com/balazsorban44/nissuer
I tested it on my fork, and all the following functionality is preserved: https://github.com/balazsorban44/next.js/issues/56
- [x] Close/lock/comment without a valid reproduction link
- [x] Minimize "+1"
- [x] Add labels based on user selection
- [x] Add comment based on the maintainer's label (eg.: "please add a complete reproduction")
## For Contributors
### Improving Documentation
- [x] Run `pnpm prettier-fix` to fix formatting issues before opening the PR.
- [x] Read the Docs Contribution Guide to ensure your contribution follows the docs guidelines: https://nextjs.org/docs/community/contribution-guide
### What?
Add instructions for using `bun/bunx` where relevant. I only added mentions where npm/yarn/pnpm were all already listed.
### Why
Bun can be used as a runtime-agnostic [package manager](https://bun.sh/package-manager) and script runner in any project with a `package.json`.
(Sorry, I probably should have consolidated this with https://github.com/vercel/next.js/pull/53467)
Co-authored-by: Steven <229881+styfle@users.noreply.github.com>
Benjamin Woodruff
