lefarcen a36d6fb6c1 perf(bake): render plugin previews concurrently (bounded worker pool) (#5429) ...

The plugin-preview bake renders one preview at a time — a headless-Chrome
screencast + ffmpeg encode per plugin, ~16s each, fully serial. When a PR
touches the deck renderer (as #5083 did), every deck example's served preview
HTML changes, so the content-hash skip re-bakes them all: 80 clips × ~16s ≈
19 min of pure serial render, dwarfing the ~3 min of setup. The bake is the
single long pole on those PRs (24m vs ~5m for everything else).

Parallelize the render with a bounded worker pool (PREVIEW_CONCURRENCY, default
4 = the standard GitHub runner's vCPUs). Local A/B on 6 forced re-bakes: 83s
serial -> 32s at concurrency 3 (~2.6x); the 80-item CI sweep amortizes launch
cost across ~20 clips/worker, so it should land closer to ~4x — dropping the
render from ~19 min to ~5 min. Shared by the pre-merge, post-merge, and
release-cut bake workflows, so all three get faster from this one change.

Three things had to hold for concurrency to be correct, each verified locally:

- One browser PER worker, not a shared browser with N pages. CDP screencast
  only follows the ACTIVE tab, so a page captured while another is foreground
  is starved of frames — I reproduced both failure modes (a frames-0 skip and
  a 5KB near-empty clip). Browser-per-worker keeps every captured page the sole
  foreground tab; clip sizes then match the serial baseline exactly. Launch is
  lazy so an all-cached sweep (the common post-merge case) pays no browser.
- ffmpeg/ffprobe moved off execFileSync onto promisified execFile: a
  synchronous encode blocks the event loop and stalls every other page's
  screencastFrameAck (Chrome only sends the next frame after the ack).
- Clamp inter-frame durations to >=0. CDP frame timestamps are occasionally
  non-monotonic, and the resulting negative concat `duration` makes ffmpeg
  reject the whole clip ("Invalid data found when processing input") — which
  fails the strict pre-merge job. Pre-existing, but CPU contention from
  concurrent renders makes the blips more frequent. Reproduced serially
  (example-deck-swiss-international) and confirmed the clamp turns it green.

Also exit explicitly at the end: puppeteer leaves a lingering handle after
browser.close(), and with a browser per worker that keeps the CLI alive after
all work is done — on CI the step would hang to the 90-min job timeout. Drain
stdout/stderr first so the piped CI summary is never truncated.

Verified against a real daemon (127.0.0.1 /api/plugins): 6/6 baked, 0 skipped,
clean exit (0 lingering processes), clips byte-comparable to serial output.

2026-07-10 10:40:53 +00:00

39 KiB

Raw Permalink Blame History

View Raw