2 Commits

Author SHA1 Message Date
Hendrik Liebau 0d5d2fb142 [test] Redirect the deployment host in-process for proxy deploy tests (#97260) 2026-08-12 20:57:57 +00:00
Benjamin Woodruff 390eff3b86 [ci] Share a single browser instance across all test suites in a single job (#95589)
The idea here is that we waste a bunch of CPU setting up each test suite
by spawning an entirely new Chrome process, but playwright can share the
same browser across many tests. Tests are still isolated at the browser
level, just not at the OS process level.

Hoping to see some performance improvement in e2e tests. Compare to:
https://github.com/vercel/next.js/pull/95617

Somewhat unscientific (single sample) results:

```
Comparing the head-commit build-and-test runs — PR #95589 run [28981702219 ](https://github.com/vercel/next.js/actions/runs/28981702219)vs PR #95617 run [28981686692](https://github.com/vercel/next.js/actions/runs/28981686692). Both ran 101 jobs and both succeeded.

  ┌─────────────────────────┬────────────────┬───────────────────────┬───────────────────┐
  │         Metric          │ #95617 control │ #95589 shared browser │       Delta       │
  ├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
  │ Raw wall-time sum       │ 616.2 min      │ 571.2 min             │ −45.0 min (−7.3%) │
  ├─────────────────────────┼────────────────┼───────────────────────┼───────────────────┤
  │ Billable (ceil per job) │ 658 min        │ 617 min               │ −41 min (−6.2%)   │
  └─────────────────────────┴────────────────┴───────────────────────┴───────────────────┘

  The shared-browser experiment is cheaper, and the savings land almost exactly where you'd expect — the browser-driven test suites — while non-browser jobs (rust, lint, unit, windows) are flat within noise:

  ┌─────────────────────────────────┬─────────┬────────┬───────┐
  │          Job category           │ control │ shared │ delta │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ test prod                       │   130.7 │  110.5 │ −20.2 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ cache components dev            │    51.5 │   42.9 │  −8.7 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ turbopack dev                   │    75.1 │   69.5 │  −5.5 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ turbopack production            │    86.3 │   81.1 │  −5.2 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ test dev                        │   110.4 │  105.5 │  −4.9 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ cache components prod           │    48.7 │   49.0 │  +0.3 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ firefox and safari              │     5.3 │    7.3 │  +2.0 │
  ├─────────────────────────────────┼─────────┼────────┼───────┤
  │ flake-detection jobs (combined) │     ~15 │    ~21 │   +~6 │
  └─────────────────────────────────┴─────────┴────────┴───────┘

  Takeaways:
  - Net savings of ~41 billable minutes per run (~6%), concentrated in the Chromium-driven test prod/dev, turbopack, and cache components suites — consistent with reusing one browser process instead of spawning per-suite.
  - The small regressions are in firefox and safari and the "new/changed tests for flakes" jobs (+~8 min combined). Worth a glance, though they may just be run-to-run variance.

  Caveat: this is a single run per PR, so there's real variance run-to-run. That said, the fact that the deltas track the browser-heavy jobs specifically — and not the Rust/lint/unit jobs — is a good signal the effect is genuine rather than noise. If you want a firmer number, re-running each PR 2–3× and averaging would tighten it up.
```
2026-07-21 01:36:10 +00:00