Both `shadow-rs` and `vergen-gitcl` call out to the git CLI, which takes optional locks to update the index: https://git-scm.com/docs/git-status#_background_refresh
This is why we sometimes find ourselves with stale `.git/index.lock` files.
Using a lightly patched version of https://github.com/martinpitt/fatrace (to look up parent process names) shows this:
`shadow-rs` has a `git2` feature (enabled by default), but that doesn't prevent it from running the git CLI. Instead it runs *both* and overwrites some values.
`vergen-git2` only uses `libgit2` and does not invoke the CLI. This fixes things because `libgit2` disables `GIT_STATUS_OPT_UPDATE_INDEX` by default.
Additionally, after looking through both codebases, I've decided I like `vergen`'s implementation a bit more than `shadow-rs`'s.
Supposedly, `git`'s command-line should support a `GIT_OPTIONAL_LOCKS=0` environment variable to disable this behavior, but I'm still finding the lock written to when trying to use that. I'm still investigating if I can use something like this to contribute a proper fix upstream to these crates.
Work in progress for PACK-4071 (implementing `config.deploymentId`)
- [x] Implement `?dpl={config.deploymentId}` in client references
manifest
- [x] Implement `?dpl={config.deploymentId}` in Turbopack chunk loading
- [x] Add `__turbopack_load_by_url__`
- [ ] Change react-dom-turbopack to use __turbopack_load_by_url__
Last item will be a follow-up PR to this one, given it's already a large
enough change in chunk loading right now.
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
### What?
Compute module batches and refactor chunking to work on the batches graph instead of the modules graph
With many pages we run into the problem that each page has a lot of modules, even if the total amount of modules is not super high. This multiples the number of edges needed in our graph.
To reduce that, group modules to module batches and module batches to batch groups and query information on these batches resp. batch groups to reduce the work needed.
A module batch will be formed from modules that are always used in a specific execution order.
A batch group will be formed from module batches that are in the same chunk groups.
Usually we work with module batches in a certain order and a unordered list of batch groups. We query information from the batch group and sort the results based on the order of module batches.
### What
Static metadata manifest file should be prefixed with `basePath` when it's specified, as all the requests are sent to the routes under `/<basePath>/*`, it should also show the correct url in the html
Fixes#56687
Closes NEXT-2707
The Form component should not need to check at runtime whether it's
running in Pages or App Router — this is a build-time condition.
The implementation is different enough that we might as well fork the
implementations at the module level.
I used #73019 as a reference, since that PR did the same thing for Link.
I have not changed anything in implementation itself; I'll leave that
for subsequent PRs. All I've done is duplicate the existing module, fix
the imports, and update the bundler configuration to resolve to the
appropriate module. There should be no observable change in behavior.
This PR changes the server action generated code a bit to work around a
typescript bug and remove some false positives we got while
typechecking: https://github.com/microsoft/TypeScript/issues/61165
The trick is that typescript seems to look for *exactly*
`Object.defineProperty(obj, 'literal', { value: ... })`, and changing
any part of the expression bypasses the bug, so we can just use
`Object['defineProperty']` instead.
This PR attempts to add typechecking to the server actions tests
fixtures in order to add an extra layer of validation against compiler
bugs. (there's a couple!). Unfortunately the tests were pretty
loosey-goosey around undefined variables, so there's a lot of noise in
adding missing imports for `Button` etc. i tried to call out the
interesting changes via PR comments.
Note that this is a bit limited, because we often have to rely on
typescript inferring some sensible types for the outputs -- see e.g. the
issue with `24/output.js` mentioned below. In case typescript can't
infer sensible types for something, we can just exclude that file from
checking. this seems to be pretty rare, so i think it's okay.
We should also definitely add typechecking to the other fixture folders,
but this PR is already big, so i want to do that piecemeal.
### TODO
- [x] invoke the typechecker in CI
- added a `check-compiler-fixtures` script that runs as part of
`types-and-precompiled` (open to feedback, maybe this isn't the right
place?)
- [x] figure out how to make `tsc` error on
`crates/next-custom-transforms/tests/fixture/server-actions/server-graph/51/output.js`,
where we're referencing a non-existent variable `$$RSC_SERVER_ACTION_0`.
right now, it emits no errors
- this is a compiler bug, and i want to catch bugs like this in the
future
- solution: https://github.com/vercel/next.js/pull/75944
- [x] figure out why `tsc` is complaining about a type mismatch in
`crates/next-custom-transforms/tests/fixture/server-actions/server-graph/24/output.js`
- we can work around this by loosening the types for
`registerServerReference`, but i'd rather we didn't have to do that
- this seems to be a bizarre TS behavior where `fn.bind(...)` makes all
the params optional in JS files. ignoring this for now
### What?
* move iter_neighbors into shared file, rename to iter_neighbors_rev
* fix async referenced modules by filtering by chunking type
* rename reverse topological to postorder topological
This PR removed the old overlay including the deprecated `buildActivity`
and `appIsrStatus` options.
Closes NDX-785
Closes NDX-855
Closes NDX-865
Closes NDX-866
Closes NDX-867
Closes NDX-868
Just quickly propagate this boolean in the graph, as opposed to chunking. Enables us to eventually get rid of `AvailableModules` and also to reimplement chunking generally.
Closes PACK-3667
TODO:
- [x] Per-layout segment computation of the async info
- [x] Fix computation of the async info to handle cycles
```
testing against 0293c96cf32
canary 42ab0048ad
12,8gb
533.86s user 82.91s system 889% cpu 1:09.36 total
534.25s user 79.95s system 889% cpu 1:09.05 total
mischnic/optimizations 2774fc5a3
12,7
520.29s user 81.71s system 881% cpu 1:08.31 total
512.83s user 78.11s system 887% cpu 1:06.58 total
```
@bgw 😉
- We never actually perform any lookup operations on these maps
- While I'm at it, also make that Vec a boxed slice
No perf difference though, do we still want this?
```
testing against 0293c96cf32
canary 0153bc2d61
13,1 GB
538.20s user 78.03s system 843% cpu 1:13.04 total
553.22s user 84.42s system 893% cpu 1:11.37 total
548.03s user 80.97s system 901% cpu 1:09.79 total
mischnic/resolve-result-repr e5618d72ed
13,1 GB
540.10s user 81.18s system 897% cpu 1:09.23 total
542.19s user 80.08s system 896% cpu 1:09.39 total
552.28s user 84.43s system 888% cpu 1:11.63 total
```
- Every `EcmascriptChunkItem` already had a `chunking_context()` getter via the `ChunkItem` supertrait.
- Produce less turbo tasks for `chunk_item.id()`
- In the future, we really want to move `chunking_context.chunk_item_id()` to the module graph, (or split it out and instead pass a `Vc<ModuleIdStrategy>`). That is however a refactoring we don't need to do right now.
This showed up in the traces, but overall it doesn't seem to have an effect on perf.
Before:
After:
### Why?
Since the dev indicator is visible by default, there could be a case where a user might want to disable it.
However, the indicator needs to be brought back if there is an error since it's tied to the error overlay.
### What?
Added an option `devIndicators: false` to disable the dev indicator.
### Success Criteria
- [x] Does it hide the indicator when there is no error?
- [x] Does it show the indicator when there is an error?
- [x] Does it show the error overlay when there is an error?
Closes NDX-840
### Why?
Since the `buildActivity` is deprecated at #76067, `buildActivityPosition` may also be deprecated. However, we are planning to preserve the option to position the dev indicator. Therefore, this PR adds a new option `position` and deprecates and aliases the `buildActivityPosition` to `position`.
### Success Criteria
Deprecation:
- [x] Does using `buildActivityPosition` show a deprecation warning?
- [x] Does `buildActivityPosition` have a deprecation in JSDoc?
Alias:
> Is it backward compatible with the old overlay?
- [x] Does setting `buildActivityPosition` correctly set the build activity indicator position?
- [x] Does it warn if `buildActivityPosition` is set instead of `position`?
- [x] Does `buildActivityPosition` supersede `position` value to prevent breaking change?
- [x] Does setting the `position` correctly set the build activity indicator position?
### Follow Up
In the following PRs, we will add the behavior for `position` to adjust the dev indicator's position. After removing the old overlay's sources, we can remove any logic that `buildActivityPosition` was holding for the old overlay and make it fully an alias for backwards compatibility.
Closes NDX-837
This updates the Rust toolchain to nightly-2025-02-12. It required:
- Removing `#![feature(async_closure)]`
- Removing `#![feature(map_many_mut)]` and renaming `map.get_many_mut()`
to `map.get_disjoint_mut()`
- Adding a range argument to `vec.extract_if()`
Test Plan: CI
Most functions in `next/cache` don't work on the client, and produce
confusing error messages when called there. It's better to just ban them
at compile time.
NOTE: For legacy/compat reasons, we're allowing `unstable_cache` and
`unstable_noStore`, since they don't currently throw when called.
### What?
in build mode we don't need to process pools anymore after the module graph has finished.
Note: I'm not super happy with this design, but it will work for now... In future we need something more general...
The `--no-mangling` CLI option for `next build` was originally
introduced in #42633. However, we lost this feature during our migration
from Terser to SWC. As part of broader improvements for debugging
dynamic accesses with Dynamic I/O enabled, I needed this functionality
back, so I restored it for Webpack and also added it to Turbopack.
fixes#67037fixes#50208
---------
Co-authored-by: Benjamin Woodruff <benjamin.woodruff@vercel.com>
Store sourcemaps JSON-serialized (as opposed to structs containing arrays of decoded mappings), which is more memory efficient (and the main operation we perform is concatenation anyway, lookups are only used for error message locations)
This will also deduplicate sourcemaps (because cloning a `Rope` is just a reference-counted pointer).
```
testing against 0293c96cf32
canary e5fc495e3d
19,1 GB
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 526.06s user 87.43s system 853% cpu 1:11.91 total
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 510.69s user 82.93s system 835% cpu 1:11.06 total
sourcemap-rope 2dc238215a
16.94 GB
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 535.61s user 85.04s system 855% cpu 1:12.55 total
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 531.07s user 82.25s system 861% cpu 1:11.22 total
```
No real signficant perf difference, but seems still worthwhile. Especially since the `sourcemap` library and `swc` both return u32 anyway. And you likely don't have files with more than 4.294.967.296 bytes, lines, or column width.
```
snapshot in with_modules c2c661b1e9 (previous PR in stack)
13,3 GB
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 491.50s user 79.78s system 856% cpu 1:06.69 total
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 495.62s user 78.61s system 852% cpu 1:07.36 total
locations as u32 b422f84b24
13,25 GB
TURBO_ENGINE_READ_ONLY=1 NEXT_TURBOPACK_TRACING= TURBOPACK=1 TURBOPACK_BUILD= 494.85s user 77.34s system 855% cpu 1:06.87 total
```
Closes PACK-3940
Benjamin Woodruff
