Files
vercel__next.js/test/lib/next-modes/next-dev.ts
Andrew Clark 94327e8ade Port React's @gate test directive to the e2e harness (#96228)
The test suite has accumulated a bunch of patterns for disabling tests
that are known to fail under some configuration: `it.skip`, `if
(isNextDev) { test('skipped in dev mode', () => {}); return }`, whole
describes toggled off by checking `process.env.__NEXT_CACHE_COMPONENTS`.
These all have the same flaw: nothing tells you when the thing you
skipped starts working. The test stays disabled forever, and the
workaround it was guarding rots along with it.

React solves this with the `@gate` pragma, and this PR ports it to the
Next.js e2e harness:

```ts
// Blocked on the optimization that marks a route as fully static when
// no dynamic params are referenced in Server Components.
// @gate !cacheComponents
it('navigates to a page with a lazily-generated static param', async () => {
  // body unchanged
})
```

The test still runs. If the condition is false and the test fails, the
failure is absorbed and the suite stays green. If it _passes_, the suite
fails: the gate is stale, delete it. So instead of a skip that hides a
fixed bug indefinitely, you get a CI failure the day the fix lands.

When the condition is static, the inversion is Jest's own `test.failing`
under the hood. A lazy condition isn't known until the fixture's
resolved config is read inside the body, so those tests invert at
runtime instead.

`// @force-gate <condition>` skips for real — for tests that can't even
be attempted (prefetching is disabled in dev, deploy has no local build
output, the fixture won't build under the condition), and for tests of a
new API, where the disabled state can only throw and running it proves
nothing:

```ts
// Prefetching is disabled in dev, so this suite has nothing to test.
// @force-gate prefetching
describe('segment cache prefetch scheduling', () => {
  // ...
})
```

There's no staleness check in that case, so this is a judgment call:
prefer `@gate` when the off state fails for a meaningful reason — the
flag changes behavior that already exists — and `@force-gate` when the
body can only throw because the API doesn't exist. A static condition
(mode, bundler) resolves at collection time into a normal Jest skip. A
lazy condition resolves at runtime, and when a lazy force-gate on a
describe is false, we skip the fixture build entirely — that's what
makes it usable for suites whose fixtures are build-incompatible with
the condition. (One caveat: Jest has no way to skip a test that's
already running, so these report as passing with a warning in the log,
not as skipped.)

Conditions live in a hand-written registry. I considered deriving the
lazy ones from the config schema automatically, but a gate is a claim
about which dimension of the test matrix explains a failure, and I'd
rather each of those claims be spelled out with a description.
Referencing an undeclared name fails the suite at collection time, so a
typo can't silently disable a gate.

The important design decision for lazy conditions is that they read the
fixture's _resolved_ config, never `process.env`. The env var isn't the
truth: `__NEXT_CACHE_COMPONENTS=true` only applies when the fixture
doesn't set `cacheComponents` itself, and config resolution implies
flags the fixture never mentions (`cacheComponents: true` alone turns on
`experimental.ppr`). Resolution happens in a child process, because
in-process `loadConfig` would leak the fixture's `.env` files into the
Jest worker. Suites with no lazy gate never pay for any of this.

The condition expression is parsed using a small grammar (also ported
from the React repo). An expression that doesn't parse fails the suite:

```ts
// @gate mode === 'start' && !cacheComponents
// @gate !(turbopack || rspack)
```

There's also a runtime version, mirroring React's `gate(flags =>
flags.enableFoo)`, for tests that run under both states but assert
differently (and for `it.each`, where the pragma can't attach):

```ts
import { gate } from 'next-test-utils'

it('renders the fallback', async () => {
  if (await gate((conditions) => conditions.cacheComponents)) {
    // PPR: the fallback is part of the static shell
  } else {
    // fully dynamic: the fallback streams in
  }
})
```

It also accepts the pragma expression language as a string: `await
gate('cacheComponents && !dev')`.

Docs are in `test/lib/gate/README.md`; `test/unit/gate/` covers the
transform, the expression language, and the runtime.
2026-08-26 10:22:11 -04:00

336 lines
9.8 KiB
TypeScript

import spawn from 'cross-spawn'
import { Span } from 'next/dist/trace'
import { NextInstance } from './base'
import { PHASE_DEVELOPMENT_SERVER } from 'next/constants'
import { retry, waitFor } from 'next-test-utils'
import stripAnsi from 'strip-ansi'
import { quote as shellQuote } from 'shell-quote'
export class NextDevInstance extends NextInstance {
private _cliOutput: string = ''
public get buildId() {
return 'development'
}
protected get configPhase() {
return PHASE_DEVELOPMENT_SERVER
}
public async setup(parentSpan: Span) {
super.setup(parentSpan)
await super.createTestDir({ parentSpan })
}
public get cliOutput() {
return this._cliOutput || ''
}
private handleStdio = (childProcess) => {
childProcess.stdout.on('data', (chunk) => {
const msg = chunk.toString()
process.stdout.write(chunk)
this._cliOutput += msg
this.emit('stdout', [msg])
})
childProcess.stderr.on('data', (chunk) => {
const msg = chunk.toString()
process.stderr.write(chunk)
this._cliOutput += msg
this.emit('stderr', [msg])
})
}
private getBuildArgs(args?: string[]) {
let buildArgs = ['pnpm', 'next', 'build']
if (this.buildCommand) {
buildArgs = this.buildCommand.split(' ')
}
if (this.buildArgs) {
buildArgs.push(...this.buildArgs)
}
if (args) {
buildArgs.push(...args)
}
if (process.env.NEXT_SKIP_ISOLATE) {
// without isolation yarn can't be used and pnpm must be used instead
if (buildArgs[0] === 'yarn') {
buildArgs[0] = 'pnpm'
}
}
return buildArgs
}
public async build(
options: { env?: Record<string, string>; args?: string[] } = {}
) {
if (this.childProcess) {
throw new Error(
`can not run build while server is running, use next.stop() first`
)
}
return new Promise<{
exitCode: NodeJS.Signals | number | null
cliOutput: string
}>((resolve) => {
const curOutput = this._cliOutput.length
const spawnOpts = this.getSpawnOpts(options.env)
const buildArgs = this.getBuildArgs(options.args)
console.log('running', shellQuote(buildArgs))
this.childProcess = spawn(buildArgs[0], buildArgs.slice(1), spawnOpts)
this.handleStdio(this.childProcess)
this.childProcess.on('error', (error) => {
this.childProcess = undefined
resolve({
exitCode: 1,
cliOutput:
this.cliOutput.slice(curOutput) + '\nSpawn error: ' + error.message,
})
})
this.childProcess.on('exit', (code, signal) => {
this.childProcess = undefined
resolve({
exitCode: signal || code,
cliOutput: this.cliOutput.slice(curOutput),
})
})
})
}
public async start(options: { env?: Record<string, string> } = {}) {
if (this.childProcess) {
throw new Error('next already started')
}
const useTurbo =
!process.env.NEXT_TEST_WASM &&
!process.env.NEXT_TEST_WASM_AFTER_JEST &&
((this as any).turbo || (this as any).experimentalTurbo)
let startArgs = [
'pnpm',
'next',
useTurbo ? '--turbopack' : undefined,
].filter(Boolean) as string[]
if (this.startCommand) {
startArgs = this.startCommand.split(' ')
}
if (this.startArgs) {
startArgs.push(...this.startArgs)
}
if (process.env.NEXT_SKIP_ISOLATE) {
// without isolation yarn can't be used and pnpm must be used instead
if (startArgs[0] === 'yarn') {
startArgs[0] = 'pnpm'
}
}
require('console').log('running', shellQuote(startArgs))
await new Promise<void>((resolve, reject) => {
try {
this.childProcess = spawn(startArgs[0], startArgs.slice(1), {
cwd: this.testDir,
stdio: ['ignore', 'pipe', 'pipe'],
shell: false,
env: {
...process.env,
...this.env,
...options.env,
NODE_ENV: this.env.NODE_ENV || ('' as any),
PORT: this.forcedPort || '0',
__NEXT_TEST_MODE: 'e2e',
},
})
this._cliOutput = ''
this.childProcess.stdout!.on('data', (chunk) => {
const msg = chunk.toString()
process.stdout.write(chunk)
this._cliOutput += msg
this.emit('stdout', [msg])
})
this.childProcess.stderr!.on('data', (chunk) => {
const msg = chunk.toString()
process.stderr.write(chunk)
this._cliOutput += msg
this.emit('stderr', [msg])
})
const serverReadyTimeoutId = this.setServerReadyTimeout(
reject,
this.startServerTimeout
)
this.childProcess.on('close', (code, signal) => {
if (this.isStopping) return
if (code || signal) {
this.childProcess = undefined
const error = new Error(
`next dev exited unexpectedly with code/signal ${code || signal}`
)
clearTimeout(serverReadyTimeoutId)
require('console').error(error)
reject(error)
}
})
const readyCb = (msg) => {
const resolveServer = () => {
clearTimeout(serverReadyTimeoutId)
try {
this._parsedUrl = new URL(this._url)
} catch (err) {
reject({
err,
msg,
})
}
// server might reload so we keep listening
resolve()
}
const colorStrippedMsg = stripAnsi(msg)
if (colorStrippedMsg.includes('- Local:')) {
this._url = msg
.split('\n')
.find((line) => line.includes('- Local:'))
.split(/\s*- Local:/)
.pop()
.trim()
}
if (this.serverReadyPattern.test(colorStrippedMsg)) {
resolveServer()
}
}
this.on('stdout', readyCb)
} catch (err) {
require('console').error(`Failed to run ${shellQuote(startArgs)}`, err)
setTimeout(() => process.exit(1), 0)
}
})
}
private async handleDevWatchDelayBeforeChange(filename: string) {
// This is a temporary workaround for turbopack starting watching too late.
// So we delay file changes by 500ms to give it some time
// to connect the WebSocket and start watching.
if (process.env.IS_TURBOPACK_TEST) {
require('console').log('fs dev delay before', filename)
await waitFor(500)
}
}
private async handleDevWatchDelayAfterChange(filename: string) {
// to help alleviate flakiness with tests that create
// dynamic routes // and then request it we give a buffer
// of 500ms to allow WatchPack to detect the changed files
// TODO: replace this with an event directly from WatchPack inside
// router-server for better accuracy
if (filename.startsWith('app/') || filename.startsWith('pages/')) {
require('console').log('fs dev delay', filename)
await new Promise((resolve) => setTimeout(resolve, 500))
}
}
public override async patchFile(
filename: string,
content: string | ((content: string) => string),
runWithTempContent?: (context: { newFile: boolean }) => Promise<void>
) {
await this.handleDevWatchDelayBeforeChange(filename)
try {
let cliOutputLength = this.cliOutput.length
const isServerRunning = this.childProcess && !this.isStopping
const detectServerRestart = async () => {
await retry(async () => {
const isServerReady = this.serverReadyPattern.test(
this.cliOutput.slice(cliOutputLength)
)
if (isServerRunning && !isServerReady) {
throw new Error('Server has not finished restarting.')
}
}, 5000)
}
const waitServerToBeReadyAfterPatchFile = async () => {
if (!isServerRunning) {
return
}
// If the patch file is a next.config.js, we ignore the delay and wait server restart
if (filename.startsWith('next.config')) {
await detectServerRestart()
return
}
if (this.patchFileDelay > 0) {
require('console').warn(
`Applying patch delay of ${this.patchFileDelay}ms. Note: Introducing artificial delays is generally discouraged, as it may affect test reliability. However, this delay is configurable on a per-test basis.`
)
await waitFor(this.patchFileDelay)
return
}
}
try {
return await super.patchFile(
filename,
content,
runWithTempContent
? async (...args) => {
await waitServerToBeReadyAfterPatchFile()
cliOutputLength = this.cliOutput.length
return runWithTempContent(...args)
}
: undefined
)
} finally {
// It's intentional: when runWithTempContent is defined, we wait twice: once for the patch,
// and once for the restore of the original file
await waitServerToBeReadyAfterPatchFile()
}
} finally {
await this.handleDevWatchDelayAfterChange(filename)
}
}
public override async renameFile(filename: string, newFilename: string) {
await this.handleDevWatchDelayBeforeChange(filename)
await super.renameFile(filename, newFilename)
await this.handleDevWatchDelayAfterChange(filename)
}
public override async renameFolder(
foldername: string,
newFoldername: string
) {
await this.handleDevWatchDelayBeforeChange(foldername)
await super.renameFolder(foldername, newFoldername)
await this.handleDevWatchDelayAfterChange(foldername)
}
public override async deleteFile(filename: string) {
await this.handleDevWatchDelayBeforeChange(filename)
await super.deleteFile(filename)
await this.handleDevWatchDelayAfterChange(filename)
}
}