`pnpm dlx` isn't using a lockfile. Using the self-contained binary from
GitHub and validate the checksum lets us effectively pin `datadog-ci`
and all of its possible transitive dependencies.
This is a follow-up for
https://vercel.slack.com/archives/C0APPN2LC83/p1775073127894859
(cherry picked from commit 631fae951d)
The previous method of using the PR endpoint under the hood calls the
GitHub API to get the commit SHA, which is a waste of resources, so just
use the commit SHA directly in the tarball URL.
## Summary
- fix flaky `pnpm install` failures in the stats action caused by
cross-device hardlinking (`ERR_PNPM_EXDEV`)
- force `pnpm` to use `--package-import-method=copy` for installs run in
temp repos
- apply the same setting in both install paths used by the stats
workflow
x-ref:
https://github.com/vercel/next.js/actions/runs/22110883672/job/63906643545
## What?
Measures the time to `Ready in X ms` in the PR stats action so that
improvements/regressions on it are highlighted on each PR.
---------
Co-authored-by: Vercel <vercel[bot]@users.noreply.github.com>
## Summary
Add per-metric significance thresholds that account for variance.
**Time metrics** (high variance from CI):
- Dev boot: <100ms AND <15%, OR <3%
- Build times: <500ms AND <5%, OR <2%
**Size metrics** (deterministic):
- node_modules: <10KB AND <1%, OR <0.01%
- Bundle sizes: <2KB AND <1%, OR <0.1%
## Problem
Current thresholds flag insignificant changes as regressions:
- 3.45 KB change on 457 MB node_modules (0.0008%) was flagged as a
regression
## Solution
Use per-metric thresholds with OR logic for very small percentage
changes:
- Time metrics need more generous thresholds due to CI variance
- Size metrics can be tighter since they're deterministic
## Test plan
- Verify that 3.45 KB on 457 MB (0%) is now marked as insignificant
- Verify that real regressions (>0.1% for sizes, >3% for times) are
still flagged
---
Stack:
- #88157
- **→ This PR**
## Summary
Comprehensive improvements to the PR stats action for more reliable benchmarks and reduced CI noise.
### Vercel KV Integration
- Add `@vercel/kv` for historical data persistence
- Track metrics over time with `loadHistory()` / `saveToHistory()`
- Display trend sparklines in comments when history is available
### Comment Generation Refactor
- New `METRIC_LABELS` and `METRIC_GROUPS` configuration system
- Organized metrics by category (Dev Server, Production Builds, Production Runtime)
- Better formatting utilities (`prettifyTime`, `formatChange`, `generateTrendBar`)
- Bundle group totals for KV persistence
### Noise Reduction
- Significance thresholds: `50ms AND 10%` for time, `1KB AND 1%` for size
- Additional `<2%` filter for long-running operations (builds)
- Filters typical CI variance while catching real regressions
### Stats Config Updates
- Add `measureDevBoot: true` to enable dev boot benchmarks
- Fix `appDevCommand` to include `dev` subcommand
- Add `--webpack` flag to `appBuildCommand`
- Rename "Client Bundles (main, webpack)" to "Client Bundles (main)"
- Add `turbopack: {}` to test configs
### GitHub Actions Updates
- New `action.yml` with bundler input parameter
- Workflow improvements in `pull_request_stats.yml`
### New Files
- `scripts/test-stats-benchmark.sh` - Local testing script
- `.github/actions/next-stats-action/src/util/stats.js` - Shared stats utilities
- `.github/actions/next-stats-action/test-local.js` - Local comment testing
## Test Plan
```bash
cd .github/actions/next-stats-action
node test-local.js # Basic test
node test-local.js --with-history # Test with trend sparklines
```
Update zod v3 to latest. Noticed this while playing with ai sdk as it's requires new versions of zod. Hence realized we haven't updated for a while. Bump to newer versions of `zod@3`
Cleans up unused workflows & scripts
- bankrupt workflow is superseded by the [stale
workflow](https://github.com/vercel/next.js/blob/canary/.github/workflows/issue_stale.yml)
- filtering issues by version hasn't been used in 9 months and can be
accomplished with a GH search
- the AI triaging tool has largely been more noise than signal. the
existing alerts that have been surfaced in the triage channel have been
sufficient
- Add the domain to the `upload-turboyet-data` action so that cache invalidation can happen faster (instead of waiting 15 minutes for the TTL).
- No longer worry about putting "rspack" in PR titles.
🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀
This builds on top of work that @timneutkens did in the
`wbinnssmith/try-ci-test` branch to rebuild the integration test CI on
top of `build_reusable.yml` and work around issues with the
`next-integration-stat`, but takes it a few steps further:
- As much of the integration test logic as possible is moved into a
single re-usable workflow so that we won't need to duplicate logic
between 6 workflows going forward (turbopack dev, turbopack build,
turbopack examples, rspack dev, rspack build, rspack examples). Right
now this just ports turbopack dev and build.
- Cleaned up a ton of legacy dead/broken code in
`next-integration-stat`. We might want to revive the comment-on-PR
functionality, so I didn't completely remove it, just the very obviously
broken parts (e.g. references to a >2-year-old stale branch of the
turborepo repository).
---
Manually triggered the workflows:
Dev tests: https://github.com/vercel/next.js/actions/runs/13466611352
Prod tests: https://github.com/vercel/next.js/actions/runs/13466609857
Tested by patching the `test/build-turbopack-*-tests-manifest.js`
scripts like so:
```diff
diff --git a/test/build-turbopack-dev-tests-manifest.js b/test/build-turbopack-dev-tests-manifest.js
index bc1928cf2d..e6546e0018 100644
--- a/test/build-turbopack-dev-tests-manifest.js
+++ b/test/build-turbopack-dev-tests-manifest.js
@@ -91,7 +91,11 @@ async function fetchLatestTestArtifact() {
const res = JSON.parse(stdout)
for (const artifact of res.artifacts) {
- if (artifact.expired || artifact.workflow_run.head_branch !== 'canary') {
+ if (
+ artifact.expired ||
+ artifact.workflow_run.head_branch !==
+ 'bgw/reusable-integration-test-workflow'
+ ) {
continue
}
```
And then diffing the generated json manifests:
https://gist.github.com/bgw/08778b5f63861811fbe88e3f466a8679
Note that some differences are expected: These jobs have been broken for
at least 2-3 weeks.
Tested by commenting out the turbopack upload bits (since I don't want to risk messing up that dataset) and running:
```
TURBOYET_KV_REST_API_TOKEN="..." TURBOYET_KV_REST_API_URL="..." node .github/actions/upload-turboyet-data/dist/index.js
```
I got the API token from the areweturboyet deployment in the vercel dashboard.
I think it's fine and easiest to just use the same KV database for both bundlers with different key prefixes.
# Summary
Update `popular-feature-requests.mjs` and `popular-issues.mjs` path on
text.
## Description
At #66383, rename `.github/actions/next-repo-info` →
`.github/actions/next-repo-actions`
Furthermore, update current file name.
### Improving Documentation
- [x] Run `pnpm prettier-fix` to fix formatting issues before opening
the PR.
- [x] Read the Docs Contribution Guide to ensure your contribution
follows the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Changes `--turbo` -> `--turbopack` to avoid confusion.
- Updated docs
- Updates create-next-app -- For create-next-app it's a rename as
otherwise there is an ordering problem with the prompts
- For the CLI `next dev --turbo` is still supported, will eventually be
a warning in a future version to swap with `--turbopack` but is not a
requirement today, can be handled automatically by the upgrade codemod
- New CLI flag: `next dev --turbopack`
<!-- Thanks for opening a PR! Your contribution is much appreciated.
To make sure your PR is handled as smoothly as possible we request that
you follow the checklist sections below.
Choose the right checklist for the change(s) that you're making:
## For Contributors
### Improving Documentation
- Run `pnpm prettier-fix` to fix formatting issues before opening the
PR.
- Read the Docs Contribution Guide to ensure your contribution follows
the docs guidelines:
https://nextjs.org/docs/community/contribution-guide
### Adding or Updating Examples
- The "examples guidelines" are followed from our contributing doc
https://github.com/vercel/next.js/blob/canary/contributing/examples/adding-examples.md
- Make sure the linting passes by running `pnpm build && pnpm lint`. See
https://github.com/vercel/next.js/blob/canary/contributing/repository/linting.md
### Fixing a bug
- Related issues linked using `fixes #number`
- Tests added. See:
https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
### Adding a feature
- Implements an existing feature request or RFC. Make sure the feature
request has been accepted for implementation before opening a PR. (A
discussion must be opened, see
https://github.com/vercel/next.js/discussions/new?category=ideas)
- Related issues/discussions are linked using `fixes #number`
- e2e tests added
(https://github.com/vercel/next.js/blob/canary/contributing/core/testing.md#writing-tests-for-nextjs)
- Documentation added
- Telemetry added. In case of a feature if it's used or not.
- Errors have a helpful link attached, see
https://github.com/vercel/next.js/blob/canary/contributing.md
## For Maintainers
- Minimal description (aim for explaining to someone not on the team to
understand the PR)
- When linking to a Slack thread, you might want to share details of the
conclusion
- Link both the Linear (Fixes NEXT-xxx) and the GitHub issues
- Add review comments if necessary to explain to the reviewer the logic
behind a change
### What?
### Why?
### How?
Closes NEXT-
Fixes #
-->
---------
Co-authored-by: Will Binns-Smith <wbinnssmith@gmail.com>
## Why?
Currently, `url` would just use the GitHub API url (JSON response of
issue), not the issue's html url, which is the correct html link to the
issue. We want the html link instead.
## Why?
We're running into the GitHub rate limit when querying for 50
`per_page`, so reducing to 25.
Also adding logic throw a `info(...)` message when no issues by <xyz>
version are found.
## Why?
Because there was no `execute` properly in the Tool Call, the Tool Call
was not immediately executed. Hence, the AI was ending with "_will_
report to Slack."
## How?
Re-add the execute property.
## Why?
It's possible that the `maxAutomaticToolTrips` line is affecting
`generateText()` from having a `results.roundtrips.length > 1` (how to
identify when a Tool Call is made), which is how the Slack message gets
sent.
- `maxAutomaticToolTrips` → `maxToolRoundtrips`
-
https://github.com/vercel/next.js/actions/runs/10525787598/job/29271187944
## Why?
When the top 15 issues workflow runs, we should automatically sync an
issue (sync with Linear) with the label `Turbopack` with `linear:
turbopack` instead of just all with `linear: next`.
## Why?
We need to update the triage guidelines, so random issues are not
flagged to the team.
## How?
- Moved `triage_guidelines.txt` to an external source, so we don't have
to constantly create a PR
- Added a `VERCEL_PROTECTION_BYPASS` secret to to Next.js repo
- Refactor `generateText()` by moving some logic to `system`
- Minor refactors to `main()`
Copies changes from https://github.com/vercel/turbo/pull/8166, and
updates contributing documentation to include the installation of lld.
> **What's wrong with `ld`?** It's very slow and uses a lot of memory.
>
> **Why `lld`?** It's fast, mature, and well-supported. Meta and Google
use it for all their linking workloads. We're already using it for macos
and x86-64 Windows. There is [ongoing work to make it the default for
rustc](https://github.com/rust-lang/rust/issues/71515), and it already
is default on a few platforms.
>
> **Why not `mold`?** Mold is generally faster, but the margin is slim
enough for our workloads that it doesn't really matter. Mold only
recently got support for LTO, doesn't support v0 rust symbol demanging,
doesn't support BOLT (though we don't use that yet), etc. Mold is
maturing quickly, but `lld` still seems like the "safer" choice.
## Why?
There is no way to test opening an issue without clever hacks. So need
to initialize again to see the payload object, which will be necessary
when I set up a Tool Call → Zod Schema with the :ai-stonks: SDK.
---------
Co-authored-by: JJ Kasper <jj@jjsweb.site>
Benjamin Woodruff
