This is mostly a copy-paste of the turbopack tests in `build_and_test.yml`.
- Only runs tests on PRs with the `Rspack` label.
- Only runs the tests listed as passing in the manifest files. We want to identify regressions, but shouldn't report failures on known broken tests.
- Don't bother with the code for running tests against React 18. The cost of running those tests is high, and the value is pretty low. Most users should be on React 19.
- Rspack test failures are not yet blocking. We need to run continuously against canary to have confidence in blocking on this.
Current problems (I don't think these are blocking for this PR):
- The manifests are out of date. I'm working on this! The tests aren't blocking though, so it shouldn't be an issue for this PR.
- If the label is added *after* the PR is created, the rspack tests won't run until you push a new revision of the PR. I discussed this with @ijjk and the solution will likely involve moving these tests into a separate workflow that can subscribe to label changes. I'll experiment with this in a follow-up PR.
The following jobs do not depend on `build-native`, but don't specify `skipNativeBuild`:
- `lint`
- `test-devlow`
- `test-next-swc-wasm`
- `test-next-swc-napi-wasi` *(currently disabled)*
- `test-unit` *(does actually appear to depend on native binaries)*
That means that they still build the native binary, but they don't wait for `build-native` to finish first, so they don't use the cached version (they run before the cache is populated), and instead each spend 5 minutes of CPU time rebuilding the native binary.
**The default timeout is 1 minute.** This PR reduces the timeouts for these tests from 4 minutes to 1 minute.
Maybe these overrides made sense at some point, but turbopack shouldn't ever be significantly slower than webpack in any test (if it is, that's a bug), so we shouldn't need larger timeouts for these.
This long 4 minute timeout has contributed to github action job timeouts we've previously experienced for the arewerspack/areweturboyet nightly jobs, so IMO it's hurting more than it's helping.
areweturboyet development tests: https://github.com/vercel/next.js/actions/runs/14893493964
areweturboyet production tests: https://github.com/vercel/next.js/actions/runs/14893499248
See the comment added to the code.
IMO: Having these environment variables duplicated with `build_reusable.yml` is at best misleading, and at worst dangerous as you could modify the wrong file.
This updates our workflow for testing new tests against deploy mode to
allow running against forks as we don't want to introduce flakey tests
in this mode even if opened from forks.
It seems too easy to accidentally not pass through `TEST_CONCURRENCY` in `build_and_test.yml`, so IMO `run-tests.js` should use it by default, still allowing overriding with `-c` or `--concurrency`.
I tested by spot-checking CI jobs for log lines like this:
```
Running tests with concurrency: 4 in test mode undefined
```
or
```
Running tests with concurrency: 1 in test mode undefined
```
Apparently there's a chicken-and-egg problem between `pnpm install` and `pnpm build` where some packages depend on `pnpm build` being run first, but `pnpm build` can't run until after `pnpm install`.
Remove this one-off hack in `afterBuild` for `devlow-bench` and document it in `build_reusable.yml`.
As part of #76251 I rewrote the `afterBuild` job from:
```yaml
- run: /bin/bash -c "${{ inputs.afterBuild }}"
```
to
```yaml
- run: ${{ inputs.afterBuild }}
# defaults.run.shell sets a stronger options (`-leo pipefail`)
# Set this back to github action's weaker defaults:
# https://docs.github.com/en/actions/writing-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsshell
#
# We must use a login shell: fnm installation may modify the `.profile`
shell: bash -le {0}
```
So it should handle escaping and line wrapping without problems now. Because we run `bash` with `-e`, failed commands should abort the entire script, so `&&` shouldn't be needed everywhere.
This fixes an issue where deploy tests were intermittently passing when
they shouldn't have been due to jobs not being sorted consistently
across retries.
### What?
We use [Nissuer](https://github.com/balazsorban44/nissuer) to assist in
triaging new GitHub Issues. This PR updates our Nissuer config to send
alerts to a webhook endpoint if Nissuer detects a possible security
report/responsible disclosure via a public GitHub Issue.
### Why?
So we can be alerted in a timely manner to possible public disclosures
of unresolved security issues in Next.js
### How?
Nissuer performs keyword detection and sends an alert to a webhook if
any keywords are detected in a newly-raised GitHub Issue.
Note: we must also set the appropriate `NISSUER_WEBHOOK_URL` and
`NISSUER_WEBHOOK_SECRET` environmental variables on the repository for
this to be effective.
Co-authored-by: JJ Kasper <jj@jjsweb.site>
- Add the domain to the `upload-turboyet-data` action so that cache invalidation can happen faster (instead of waiting 15 minutes for the TTL).
- No longer worry about putting "rspack" in PR titles.
🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀🦀
`build-native` and `build-native-windows` were skipped for docs only
changes in #77743, removing the native from the `needs:[]` dependencies
of lint job to ensure lint will still be proceed for docs PR
## What?
Moved the basepath e2e tests into a dedicated directory structure and
simplified the test setup:
1. Relocated `basepath-trailing-slash.test.ts` and `basepath.test.ts`
into a dedicated `basepath` directory
2. Simplified the test file setup by using `files: __dirname` instead of
explicitly referencing individual directories
3. Updated the regex patterns in the prefetch test to be more flexible
with optional characters, matching the Turbopack pattern.
4. Updated the test manifest files to reflect the new file paths
5. Fixed a previously failing test: "basePath should prefetch pages
correctly in viewport with <Link>"
If not supplied, this defaults to a fetch-depth of 1, which likely
explains some of the tagless version strings I've seen...
Use a fetch-depth of 100 so that we're always likely to find a previous
tag.
These scripts were 95% identical, so they can just be one script that
accepts a few arguments.
```
$ node test/update-bundler-manifest.js --help
Options:
--help Show help [boolean]
--version Show version number [boolean]
--bundler The JavaScript bundler the tests were run against
[required] [choices: "turbopack", "rspack"]
--test-suite Test group to update [required] [choices: "dev", "build"]
--branch the git branch to filter CI artifacts to
[string] [default: "canary"]
--override Don't merge with existing test results, allowing tests to
transition to a failed state [boolean]
```
Tested by running:
```
node test/update-bundler-manifest.js --bundler rspack --test-suite dev --override
node test/update-bundler-manifest.js --bundler rspack --test-suite build --override
```
Reverts vercel/next.js#76606
Causes the nightly jobs to fail with:
```
Error: Failed to CreateArtifact: Received non-retryable error: Failed request: (409) Conflict: an artifact with this name already exists on the workflow run
```
because we're running two `build_reusable` jobs with the same stepName.
While I could fix this, we'd still have problems with either duplicating
tests on areweturboyet, or we'd need some logic to merge test results
across the matrix.
I like the idea of running the full matrix, but this type of thing is
rare and maybe not worth solving for (aside from catching it when the PR
goes up, which we now have logic for, thanks to @unstubbable).
This opts us out of using the native built-in plugins for Rspack for now.
The existing swc plugin we use for webpack does additional work beyond just swc transformation, including validating RSC components to make sure they don’t use React APIs unavailable in that context.
In total, the native plugins result in a less than 20% performance gain which, while significant, isn’t necessary as we focus on correctness for now.
If we decide the maintenance cost of maintaining multiple plugins in parity is worth the performance gain in the future, we can selectively re-enable some or all of these plugins.
Test Plan: `pnpm test-dev-rspack test/development/acceptance/server-component-compiler-errors-in-pages.test.ts` no longer fails by missing the invalid React api messages.
While #76544 doesn't hurt, this is the more important fix to ensure that
tests that were previously marked as `failed` are only promoted to
`passed` if they also pass when the test is run with React 18.
This builds on top of work that @timneutkens did in the
`wbinnssmith/try-ci-test` branch to rebuild the integration test CI on
top of `build_reusable.yml` and work around issues with the
`next-integration-stat`, but takes it a few steps further:
- As much of the integration test logic as possible is moved into a
single re-usable workflow so that we won't need to duplicate logic
between 6 workflows going forward (turbopack dev, turbopack build,
turbopack examples, rspack dev, rspack build, rspack examples). Right
now this just ports turbopack dev and build.
- Cleaned up a ton of legacy dead/broken code in
`next-integration-stat`. We might want to revive the comment-on-PR
functionality, so I didn't completely remove it, just the very obviously
broken parts (e.g. references to a >2-year-old stale branch of the
turborepo repository).
---
Manually triggered the workflows:
Dev tests: https://github.com/vercel/next.js/actions/runs/13466611352
Prod tests: https://github.com/vercel/next.js/actions/runs/13466609857
Tested by patching the `test/build-turbopack-*-tests-manifest.js`
scripts like so:
```diff
diff --git a/test/build-turbopack-dev-tests-manifest.js b/test/build-turbopack-dev-tests-manifest.js
index bc1928cf2d..e6546e0018 100644
--- a/test/build-turbopack-dev-tests-manifest.js
+++ b/test/build-turbopack-dev-tests-manifest.js
@@ -91,7 +91,11 @@ async function fetchLatestTestArtifact() {
const res = JSON.parse(stdout)
for (const artifact of res.artifacts) {
- if (artifact.expired || artifact.workflow_run.head_branch !== 'canary') {
+ if (
+ artifact.expired ||
+ artifact.workflow_run.head_branch !==
+ 'bgw/reusable-integration-test-workflow'
+ ) {
continue
}
```
And then diffing the generated json manifests:
https://gist.github.com/bgw/08778b5f63861811fbe88e3f466a8679
Note that some differences are expected: These jobs have been broken for
at least 2-3 weeks.
This PR removed the old overlay including the deprecated `buildActivity`
and `appIsrStatus` options.
Closes NDX-785
Closes NDX-855
Closes NDX-865
Closes NDX-866
Closes NDX-867
Closes NDX-868
- `newDevOverlay: true` by default (enables experimental React builds on
canary until owner stacks progress further)
- `run-tests` now sets the env var for tests that were relying on it for
forking behavior
- PPR runners now run with the flag disabled to help catch regressions
in the old overlay until we remove it
- Fixed a number of tests that had outdated snapshots or missed forking
behavior because they weren't running in CI
- Disabled a test that was failing in Turbopack + Experimental React
that is unrelated to the overlay (see:
https://github.com/vercel/next.js/pull/75989)
---------
Co-authored-by: devjiwonchoi <devjiwonchoi@gmail.com>
## Why?
Issues with the `Documentation` **Type**, which is issues opened via the `Documentation` issue template, should not be prematurely closed with **Invalid Link** from [nissuer](https://github.com/balazsorban44/nissuer).
Donny/강동윤
