EXPERIMENTAL spike. Adds a Node-side WebGPU compositor backed by the `webgpu` npm package (Dawn) and wires it into the existing shader-transition worker as an opt-in alternative to the CPU blend. Default OFF — set `HF_DAWN_WEBGPU=1` or pass `--gpu-shader-blend` to the CLI to engage. The hf#677 chain (PRs #756-#760) gets us 1.95x on Mac via a CPU shader worker pool, ring-buffered pipelining, and a hybrid layered/parallel capture path. The remaining ceiling is the per-pixel JS blend itself — scalar f64 math in v8. On any host with a real GPU (Mac/Metal, Linux/Vulkan, Windows/D3D), moving the blend to the GPU via Dawn should drop blend wall time on a 854x480 rgb48le frame from ~150-910 ms (depending on shader complexity) to a few ms, projecting 3-5x end-to-end on top of the existing cascade — see the `reference_5x_shader_perf_alternatives.md` memo (option B). beginFrame is structurally Mac-unavailable (crbug.com/40656275); native WebGPU is the next-best Mac-viable lever. The Dawn binding (`webgpu@0.4.0`, dawn-gpu/node-webgpu, maintained by Dawn upstream — Corentin Wallez, Kai Ninomiya) ships a `darwin-universal` prebuilt that targets Apple Metal directly, plus Linux x64/arm64 Vulkan and Windows x64 D3D. Verified locally that the package loads on Linux x64; adapter init returns null in the sandbox (no Vulkan driver) and the worker transparently falls back to the existing CPU path. Mac numbers must be measured on Vance's laptop — there's no GPU host in CI. - `packages/producer/src/services/shaderTransitionGpu.ts` (NEW) Node-side compositor: dynamic-imports `webgpu`, requests an adapter + device, compiles a WGSL compute shader per supported transition, manages per-size GPU resources, dispatches blends, reads back to `rgb48le`. Returns structured init failures rather than throwing — the caller can always fall back to CPU. `HF_DAWN_FORCE_FAIL=1` short-circuits init for testability of the fallback path. - `packages/producer/src/services/shaderTransitionWorker.ts` Augmented: on the first message, if `HF_DAWN_WEBGPU=1`, dynamic-imports the GPU module and probes once. On success, supported shaders run on the GPU. Unsupported shaders (`glitch`, `domain-warp`, `swirl-vortex`, the rest) and any host without a GPU adapter fall through to the existing `TRANSITIONS[shader] ?? crossfade` CPU path with zero behavior change. Mid-render GPU failure disables the GPU path for the rest of the worker's life and falls back — the frame still completes. - `packages/cli/src/commands/render.ts` New `--gpu-shader-blend` boolean flag (default false). When set, the CLI exports `HF_DAWN_WEBGPU=1` into `process.env` before any worker pool spawns. Env-var plumbing chosen over threading the flag through the orchestrator -> stage -> pool -> worker chain because env vars cross the `worker_threads` boundary unchanged. - `packages/producer/build.mjs` + `packages/cli/tsup.config.ts` `webgpu` added to the `external` list in both bundlers. The Dawn binding ships a 70+ MB native `.dawn.node` binary per platform — it must be loaded from the user's node_modules at runtime, not inlined into the bundle. - `packages/producer/package.json` + `packages/cli/package.json` `webgpu@^0.4.0` added to `optionalDependencies` on both. Optional because (a) it's a native binary that may fail to install on some hosts and (b) the feature is opt-in. - `packages/producer/src/services/shaderTransitionGpu.test.ts` (NEW) 3 vitest tests: `HF_DAWN_FORCE_FAIL` short-circuit, init never throws, and PSNR-vs-CPU >= 50 dB on hosts where the adapter is available (auto-skipped on Linux sandbox with a log line). One representative shader (`crossfade`) is ported to WGSL end-to-end as proof of correctness. The harness is shader-agnostic; porting more is a mechanical add to `SHADERS_WGSL` in `shaderTransitionGpu.ts`. Unsupported shaders transparently fall through to CPU even when the flag is on. GPU compute uses f32 + u16 storage; CPU canonical uses f64. Bit-exact equality is not realistic. The default-off path keeps CI byte-equality pins intact (the existing fixtures hit the CPU path unchanged). Fixtures that exercise the GPU path must use a PSNR pin (>= 50 dB documented in the new test). The fallback CPU path is bit-exact with the canonical CPU implementation. Bundled CLI worker smoke (854x480 single crossfade blend, see the investigation log for the harness): | config | wall | notes | |---|---:|---| | CPU baseline (`HF_DAWN_WEBGPU` unset) | 67 ms | reference | | `HF_DAWN_WEBGPU=1` (fell back, no GPU) | 70 ms | +3 ms init+probe, then CPU | | `HF_DAWN_WEBGPU=1` + `HF_DAWN_FORCE_FAIL=1` | 66 ms | fallback verified | Both fallback runs produced byte-identical output to the CPU baseline checksum — fallback fidelity confirmed. **Mac numbers: Vance, please fill in.** Pull the branch, run a shader-transition fixture both with and without `--gpu-shader-blend`, and drop the wall-time pair in a comment. Anywhere the GPU path can't run, the existing CPU path runs unchanged: - `webgpu` package not installed (optional dep skipped on install) -> CPU - `webgpu` loads but `requestAdapter()` returns null (no GPU) -> CPU - WGSL pipeline compile fails -> CPU - `--gpu-shader-blend` set but shader not in `SHADERS_WGSL` -> CPU - Mid-render GPU failure -> log + disable GPU + finish on CPU Failure reason is logged once per worker (not per frame). No crash path. - Port the remaining 12 shaders to WGSL (mechanical: `flashThroughWhite`, `chromaticSplit`, `sdfIris`, `glitch`, `lightLeak`, `crossWarpMorph`, `whipPan`, `cinematicZoom`, `gravitationalLens`, `rippleWaves`, `swirlVortex`, `thermalDistortion`, `domainWarp`, `ridgedBurn`). - Once Mac wall-time confirms the lever, decide on flip-default-on or keep opt-in. - Update fixture pins for any fixture that needs to exercise the GPU path under CI (PSNR-only). - A persistent device + pipeline cache shared across workers (currently per-worker init). _PR drafted by Vai_ Co-Authored-By: Vai <vai@heygen.com>
@hyperframes/producer
Full HTML-to-video rendering pipeline: capture frames with Chrome's BeginFrame API, encode with FFmpeg, mix audio — all in one call.
Install
npm install @hyperframes/producer
Requirements: Node.js >= 22, Chrome/Chromium (auto-downloaded), FFmpeg
Usage
Render a video
import { createRenderJob, executeRenderJob } from "@hyperframes/producer";
const job = createRenderJob({
inputPath: "./my-composition.html",
outputPath: "./output.mp4",
width: 1920,
height: 1080,
fps: 30,
});
const result = await executeRenderJob(job, (progress) => {
console.log(`${Math.round(progress.percent * 100)}%`);
});
console.log(result.outputPath); // ./output.mp4
Run as an HTTP server
The producer can also run as a render server, accepting render requests over HTTP:
import { startServer } from "@hyperframes/producer";
await startServer({ port: 8080 });
// POST /render with a RenderConfig body
Configuration
RenderConfig controls the render pipeline:
| Option | Default | Description |
|---|---|---|
inputPath |
— | Path to the HTML composition |
outputPath |
— | Output video file path (or directory, for format: "png-sequence") |
width |
1920 | Frame width in pixels |
height |
1080 | Frame height in pixels |
fps |
30 | Frames per second (24, 30, or 60) |
quality |
"standard" |
Encoder preset ("draft", "standard", "high") |
format |
"mp4" |
Output container — "mp4", "webm", "mov", or "png-sequence". See Transparent Video Output below. |
Transparent Video Output
The producer can render HTML compositions to formats that carry a true alpha channel — not chroma key. The same composition that renders an opaque MP4 renders a layerable overlay when you set format.
format |
Codec / pixel format | Alpha | Audio | Use case |
|---|---|---|---|---|
"mp4" (default) |
H.264 (yuv420p) or H.265 + HDR10 | No | AAC | Streaming, social, default deliverable |
"webm" |
VP9 + yuva420p | True alpha | Opus | Web playback as overlay (<video> over background); supported in Chrome, Edge, Firefox |
"mov" |
ProRes 4444 + yuva444p10le | True alpha + 10-bit | AAC | Editor ingest (Premiere, Final Cut Pro, DaVinci Resolve) |
"png-sequence" |
Numbered RGBA PNGs in a directory | Lossless alpha | Sidecar audio.aac |
After Effects / Nuke / Fusion, or pipelines that post-process frames before encoding |
Example
import { createRenderJob, executeRenderJob } from "@hyperframes/producer";
const job = createRenderJob({
inputPath: "./my-composition.html",
outputPath: "./output.webm", // or a directory for "png-sequence"
width: 1080,
height: 1920,
fps: 30,
format: "webm", // "mp4" | "webm" | "mov" | "png-sequence"
});
await executeRenderJob(job);
What "transparent background" means here
The producer captures Chrome screenshots with the page background forced transparent (html, body, [data-composition-id] { background: transparent !important }) and the CDP default background override set to RGBA 0,0,0,0. The captured PNGs carry a real alpha channel and that channel is preserved end-to-end:
- VP9 (
webm) is encoded with-pix_fmt yuva420p,-auto-alt-ref 0, andalpha_mode=1metadata. - ProRes 4444 (
mov) is encoded with-pix_fmt yuva444p10le. - PNG sequences are written without re-encoding (zero-padded
frame_NNNNNN.png).
This is not chroma keying. There is no green/blue background to remove and no "key" tolerance to tune — pixels that were transparent in the browser are transparent in the output.
Caveats
- Linux + alpha forces screenshot capture. Chrome's BeginFrame compositor (the default deterministic capture path on Linux headless-shell) does not preserve alpha; the orchestrator falls back to
Page.captureScreenshot, which is slower per frame. macOS and Windows already use screenshot mode by default, so they are unaffected. - HDR + alpha is not supported. Setting
hdr: truetogether with an alpha-capable format logs a warning and falls back to SDR. Useformat: "mp4"for HDR10 output. png-sequencedoes not produce a single muxed file. When the composition contains audio elements, anaudio.aacsidecar is written alongside the PNGs inoutputPath.- Safari + WebM alpha is incomplete. For broad browser playback of an alpha video, ship
format: "mov"to your editor and re-encode for the codec your distribution target supports.
Authoring transparent compositions
Don't paint a fullscreen background in your HTML. The default body background is overridden to transparent automatically — any body { background: ... }, #root { background: ... }, or [data-composition-id] { background: ... } rule is force-overridden during alpha rendering. Backgrounds on inner elements (cards, scenes, components) are kept.
How it works
- Serve — spins up a local file server for the HTML composition
- Capture — opens the page in headless Chrome, seeks frame-by-frame via
HeadlessExperimental.beginFrame(orPage.captureScreenshotfor transparent / non-Linux renders), captures screenshots - Encode — pipes frames through FFmpeg (with GPU encoder detection and chunked concat). Skipped for
format: "png-sequence". - Mix — extracts
<audio>elements and mixes them into the final video. Forpng-sequence, audio is written as anaudio.aacsidecar. - Finalize — applies faststart for streaming-friendly MP4 (no-op for WebM, MOV, and
png-sequence)
Documentation
Full documentation: hyperframes.heygen.com/packages/producer
Related packages
@hyperframes/core— types, parsers, frame adapters@hyperframes/engine— lower-level capture and encode primitiveshyperframes— CLI