mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-14 18:01:20 +08:00
Vance Ingalls
450bb01018
EXPERIMENTAL spike. Adds a Node-side WebGPU compositor backed by the `webgpu` npm package (Dawn) and wires it into the existing shader-transition worker as an opt-in alternative to the CPU blend. Default OFF — set `HF_DAWN_WEBGPU=1` or pass `--gpu-shader-blend` to the CLI to engage. The hf#677 chain (PRs #756-#760) gets us 1.95x on Mac via a CPU shader worker pool, ring-buffered pipelining, and a hybrid layered/parallel capture path. The remaining ceiling is the per-pixel JS blend itself — scalar f64 math in v8. On any host with a real GPU (Mac/Metal, Linux/Vulkan, Windows/D3D), moving the blend to the GPU via Dawn should drop blend wall time on a 854x480 rgb48le frame from ~150-910 ms (depending on shader complexity) to a few ms, projecting 3-5x end-to-end on top of the existing cascade — see the `reference_5x_shader_perf_alternatives.md` memo (option B). beginFrame is structurally Mac-unavailable (crbug.com/40656275); native WebGPU is the next-best Mac-viable lever. The Dawn binding (`webgpu@0.4.0`, dawn-gpu/node-webgpu, maintained by Dawn upstream — Corentin Wallez, Kai Ninomiya) ships a `darwin-universal` prebuilt that targets Apple Metal directly, plus Linux x64/arm64 Vulkan and Windows x64 D3D. Verified locally that the package loads on Linux x64; adapter init returns null in the sandbox (no Vulkan driver) and the worker transparently falls back to the existing CPU path. Mac numbers must be measured on Vance's laptop — there's no GPU host in CI. - `packages/producer/src/services/shaderTransitionGpu.ts` (NEW) Node-side compositor: dynamic-imports `webgpu`, requests an adapter + device, compiles a WGSL compute shader per supported transition, manages per-size GPU resources, dispatches blends, reads back to `rgb48le`. Returns structured init failures rather than throwing — the caller can always fall back to CPU. `HF_DAWN_FORCE_FAIL=1` short-circuits init for testability of the fallback path. - `packages/producer/src/services/shaderTransitionWorker.ts` Augmented: on the first message, if `HF_DAWN_WEBGPU=1`, dynamic-imports the GPU module and probes once. On success, supported shaders run on the GPU. Unsupported shaders (`glitch`, `domain-warp`, `swirl-vortex`, the rest) and any host without a GPU adapter fall through to the existing `TRANSITIONS[shader] ?? crossfade` CPU path with zero behavior change. Mid-render GPU failure disables the GPU path for the rest of the worker's life and falls back — the frame still completes. - `packages/cli/src/commands/render.ts` New `--gpu-shader-blend` boolean flag (default false). When set, the CLI exports `HF_DAWN_WEBGPU=1` into `process.env` before any worker pool spawns. Env-var plumbing chosen over threading the flag through the orchestrator -> stage -> pool -> worker chain because env vars cross the `worker_threads` boundary unchanged. - `packages/producer/build.mjs` + `packages/cli/tsup.config.ts` `webgpu` added to the `external` list in both bundlers. The Dawn binding ships a 70+ MB native `.dawn.node` binary per platform — it must be loaded from the user's node_modules at runtime, not inlined into the bundle. - `packages/producer/package.json` + `packages/cli/package.json` `webgpu@^0.4.0` added to `optionalDependencies` on both. Optional because (a) it's a native binary that may fail to install on some hosts and (b) the feature is opt-in. - `packages/producer/src/services/shaderTransitionGpu.test.ts` (NEW) 3 vitest tests: `HF_DAWN_FORCE_FAIL` short-circuit, init never throws, and PSNR-vs-CPU >= 50 dB on hosts where the adapter is available (auto-skipped on Linux sandbox with a log line). One representative shader (`crossfade`) is ported to WGSL end-to-end as proof of correctness. The harness is shader-agnostic; porting more is a mechanical add to `SHADERS_WGSL` in `shaderTransitionGpu.ts`. Unsupported shaders transparently fall through to CPU even when the flag is on. GPU compute uses f32 + u16 storage; CPU canonical uses f64. Bit-exact equality is not realistic. The default-off path keeps CI byte-equality pins intact (the existing fixtures hit the CPU path unchanged). Fixtures that exercise the GPU path must use a PSNR pin (>= 50 dB documented in the new test). The fallback CPU path is bit-exact with the canonical CPU implementation. Bundled CLI worker smoke (854x480 single crossfade blend, see the investigation log for the harness): | config | wall | notes | |---|---:|---| | CPU baseline (`HF_DAWN_WEBGPU` unset) | 67 ms | reference | | `HF_DAWN_WEBGPU=1` (fell back, no GPU) | 70 ms | +3 ms init+probe, then CPU | | `HF_DAWN_WEBGPU=1` + `HF_DAWN_FORCE_FAIL=1` | 66 ms | fallback verified | Both fallback runs produced byte-identical output to the CPU baseline checksum — fallback fidelity confirmed. **Mac numbers: Vance, please fill in.** Pull the branch, run a shader-transition fixture both with and without `--gpu-shader-blend`, and drop the wall-time pair in a comment. Anywhere the GPU path can't run, the existing CPU path runs unchanged: - `webgpu` package not installed (optional dep skipped on install) -> CPU - `webgpu` loads but `requestAdapter()` returns null (no GPU) -> CPU - WGSL pipeline compile fails -> CPU - `--gpu-shader-blend` set but shader not in `SHADERS_WGSL` -> CPU - Mid-render GPU failure -> log + disable GPU + finish on CPU Failure reason is logged once per worker (not per frame). No crash path. - Port the remaining 12 shaders to WGSL (mechanical: `flashThroughWhite`, `chromaticSplit`, `sdfIris`, `glitch`, `lightLeak`, `crossWarpMorph`, `whipPan`, `cinematicZoom`, `gravitationalLens`, `rippleWaves`, `swirlVortex`, `thermalDistortion`, `domainWarp`, `ridgedBurn`). - Once Mac wall-time confirms the lever, decide on flip-default-on or keep opt-in. - Update fixture pins for any fixture that needs to exercise the GPU path under CI (PSNR-only). - A persistent device + pipeline cache shared across workers (currently per-worker init). _PR drafted by Vai_ Co-Authored-By: Vai <vai@heygen.com>
91 lines
3.7 KiB
JSON
91 lines
3.7 KiB
JSON
{
|
|
"name": "@hyperframes/producer",
|
|
"version": "0.6.6",
|
|
"description": "HTML-to-video rendering engine using Chrome's BeginFrame API",
|
|
"repository": {
|
|
"type": "git",
|
|
"url": "https://github.com/heygen-com/hyperframes",
|
|
"directory": "packages/producer"
|
|
},
|
|
"files": [
|
|
"dist/"
|
|
],
|
|
"type": "module",
|
|
"main": "dist/index.js",
|
|
"types": "dist/index.d.ts",
|
|
"exports": {
|
|
".": {
|
|
"import": "./dist/index.js",
|
|
"types": "./dist/index.d.ts"
|
|
},
|
|
"./server": {
|
|
"import": "./dist/public-server.js"
|
|
}
|
|
},
|
|
"publishConfig": {
|
|
"access": "public",
|
|
"registry": "https://registry.npmjs.org/"
|
|
},
|
|
"scripts": {
|
|
"build": "bun run build:fonts && bun run --cwd ../.. build:hyperframes-runtime:modular && node build.mjs",
|
|
"build:fonts": "node -e \"const fs=require('fs');if(!fs.existsSync('src/services/fontData.generated.ts')){const {execSync}=require('child_process');execSync('node --experimental-strip-types scripts/generate-font-data.ts',{stdio:'inherit'})}else{console.log('[build:fonts] skipped — fontData.generated.ts already exists')}\"",
|
|
"typecheck": "tsc --noEmit",
|
|
"parity:check": "tsx src/parity-harness.ts",
|
|
"parity:fixtures": "tsx src/parity-fixtures.ts",
|
|
"parity:fixtures:ci": "tsx src/parity-fixtures.ts",
|
|
"parity:check:ci": "tsx src/parity-harness.ts --preview-url \"http://127.0.0.1:4173/minimal-wysiwyg.html\" --producer-url \"http://127.0.0.1:4173/minimal-wysiwyg.html?mode=producer\" --checkpoints \"0,0.5,1,1.5\" --allow-mismatch-ratio 0 --emulate-producer-swap true --artifacts-dir \".debug/parity-harness-ci\"",
|
|
"perf:gate": "tsx src/perf-gate.ts",
|
|
"check:runtime-conformance": "tsx src/runtime-conformance.ts",
|
|
"benchmark": "tsx src/benchmark.ts",
|
|
"bench:hdr": "tsx src/benchmark.ts --tags hdr",
|
|
"test": "tsx src/regression-harness.ts --exclude-tags transparency",
|
|
"test:update": "tsx src/regression-harness.ts --update --exclude-tags transparency",
|
|
"test:transparency": "tsx src/transparency-test.ts",
|
|
"docker:build:test": "docker build -f ../../Dockerfile.test -t hyperframes-producer:test ../..",
|
|
"docker:test": "docker run --rm --security-opt seccomp=unconfined --shm-size=2g -v ./tests:/app/packages/producer/tests hyperframes-producer:test",
|
|
"docker:test:update": "docker run --rm --security-opt seccomp=unconfined --shm-size=2g -v ./tests:/app/packages/producer/tests hyperframes-producer:test --update",
|
|
"prepublishOnly": "echo skip"
|
|
},
|
|
"dependencies": {
|
|
"@fontsource/archivo-black": "^5.2.8",
|
|
"@fontsource/eb-garamond": "^5.2.7",
|
|
"@fontsource/ibm-plex-mono": "^5.2.7",
|
|
"@fontsource/inter": "^5.2.8",
|
|
"@fontsource/jetbrains-mono": "^5.2.8",
|
|
"@fontsource/league-gothic": "^5.2.8",
|
|
"@fontsource/montserrat": "^5.2.8",
|
|
"@fontsource/nunito": "^5.2.7",
|
|
"@fontsource/oswald": "^5.2.8",
|
|
"@fontsource/outfit": "^5.2.8",
|
|
"@fontsource/space-mono": "^5.2.9",
|
|
"@hono/node-server": "^1.13.0",
|
|
"@hyperframes/core": "workspace:^",
|
|
"@hyperframes/engine": "workspace:^",
|
|
"hono": "^4.6.0",
|
|
"linkedom": "^0.18.12",
|
|
"postcss": "^8.4.0",
|
|
"puppeteer": "^24.0.0",
|
|
"puppeteer-core": "^24.39.1"
|
|
},
|
|
"devDependencies": {
|
|
"@fontsource/lato": "^5.2.7",
|
|
"@fontsource/noto-sans-jp": "^5.2.9",
|
|
"@fontsource/open-sans": "^5.2.7",
|
|
"@fontsource/playfair-display": "^5.2.8",
|
|
"@fontsource/poppins": "^5.2.7",
|
|
"@fontsource/roboto": "^5.2.10",
|
|
"@fontsource/source-code-pro": "^5.2.7",
|
|
"@types/node": "^25.0.10",
|
|
"@webgpu/types": "^0.1.69",
|
|
"esbuild": "^0.25.12",
|
|
"tsx": "^4.21.0",
|
|
"typescript": "^5.7.2"
|
|
},
|
|
"optionalDependencies": {
|
|
"webgpu": "^0.4.0"
|
|
},
|
|
"engines": {
|
|
"node": ">=22"
|
|
}
|
|
}
|