mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-14 18:01:20 +08:00
Vance Ingalls
450bb01018
EXPERIMENTAL spike. Adds a Node-side WebGPU compositor backed by the `webgpu` npm package (Dawn) and wires it into the existing shader-transition worker as an opt-in alternative to the CPU blend. Default OFF — set `HF_DAWN_WEBGPU=1` or pass `--gpu-shader-blend` to the CLI to engage. The hf#677 chain (PRs #756-#760) gets us 1.95x on Mac via a CPU shader worker pool, ring-buffered pipelining, and a hybrid layered/parallel capture path. The remaining ceiling is the per-pixel JS blend itself — scalar f64 math in v8. On any host with a real GPU (Mac/Metal, Linux/Vulkan, Windows/D3D), moving the blend to the GPU via Dawn should drop blend wall time on a 854x480 rgb48le frame from ~150-910 ms (depending on shader complexity) to a few ms, projecting 3-5x end-to-end on top of the existing cascade — see the `reference_5x_shader_perf_alternatives.md` memo (option B). beginFrame is structurally Mac-unavailable (crbug.com/40656275); native WebGPU is the next-best Mac-viable lever. The Dawn binding (`webgpu@0.4.0`, dawn-gpu/node-webgpu, maintained by Dawn upstream — Corentin Wallez, Kai Ninomiya) ships a `darwin-universal` prebuilt that targets Apple Metal directly, plus Linux x64/arm64 Vulkan and Windows x64 D3D. Verified locally that the package loads on Linux x64; adapter init returns null in the sandbox (no Vulkan driver) and the worker transparently falls back to the existing CPU path. Mac numbers must be measured on Vance's laptop — there's no GPU host in CI. - `packages/producer/src/services/shaderTransitionGpu.ts` (NEW) Node-side compositor: dynamic-imports `webgpu`, requests an adapter + device, compiles a WGSL compute shader per supported transition, manages per-size GPU resources, dispatches blends, reads back to `rgb48le`. Returns structured init failures rather than throwing — the caller can always fall back to CPU. `HF_DAWN_FORCE_FAIL=1` short-circuits init for testability of the fallback path. - `packages/producer/src/services/shaderTransitionWorker.ts` Augmented: on the first message, if `HF_DAWN_WEBGPU=1`, dynamic-imports the GPU module and probes once. On success, supported shaders run on the GPU. Unsupported shaders (`glitch`, `domain-warp`, `swirl-vortex`, the rest) and any host without a GPU adapter fall through to the existing `TRANSITIONS[shader] ?? crossfade` CPU path with zero behavior change. Mid-render GPU failure disables the GPU path for the rest of the worker's life and falls back — the frame still completes. - `packages/cli/src/commands/render.ts` New `--gpu-shader-blend` boolean flag (default false). When set, the CLI exports `HF_DAWN_WEBGPU=1` into `process.env` before any worker pool spawns. Env-var plumbing chosen over threading the flag through the orchestrator -> stage -> pool -> worker chain because env vars cross the `worker_threads` boundary unchanged. - `packages/producer/build.mjs` + `packages/cli/tsup.config.ts` `webgpu` added to the `external` list in both bundlers. The Dawn binding ships a 70+ MB native `.dawn.node` binary per platform — it must be loaded from the user's node_modules at runtime, not inlined into the bundle. - `packages/producer/package.json` + `packages/cli/package.json` `webgpu@^0.4.0` added to `optionalDependencies` on both. Optional because (a) it's a native binary that may fail to install on some hosts and (b) the feature is opt-in. - `packages/producer/src/services/shaderTransitionGpu.test.ts` (NEW) 3 vitest tests: `HF_DAWN_FORCE_FAIL` short-circuit, init never throws, and PSNR-vs-CPU >= 50 dB on hosts where the adapter is available (auto-skipped on Linux sandbox with a log line). One representative shader (`crossfade`) is ported to WGSL end-to-end as proof of correctness. The harness is shader-agnostic; porting more is a mechanical add to `SHADERS_WGSL` in `shaderTransitionGpu.ts`. Unsupported shaders transparently fall through to CPU even when the flag is on. GPU compute uses f32 + u16 storage; CPU canonical uses f64. Bit-exact equality is not realistic. The default-off path keeps CI byte-equality pins intact (the existing fixtures hit the CPU path unchanged). Fixtures that exercise the GPU path must use a PSNR pin (>= 50 dB documented in the new test). The fallback CPU path is bit-exact with the canonical CPU implementation. Bundled CLI worker smoke (854x480 single crossfade blend, see the investigation log for the harness): | config | wall | notes | |---|---:|---| | CPU baseline (`HF_DAWN_WEBGPU` unset) | 67 ms | reference | | `HF_DAWN_WEBGPU=1` (fell back, no GPU) | 70 ms | +3 ms init+probe, then CPU | | `HF_DAWN_WEBGPU=1` + `HF_DAWN_FORCE_FAIL=1` | 66 ms | fallback verified | Both fallback runs produced byte-identical output to the CPU baseline checksum — fallback fidelity confirmed. **Mac numbers: Vance, please fill in.** Pull the branch, run a shader-transition fixture both with and without `--gpu-shader-blend`, and drop the wall-time pair in a comment. Anywhere the GPU path can't run, the existing CPU path runs unchanged: - `webgpu` package not installed (optional dep skipped on install) -> CPU - `webgpu` loads but `requestAdapter()` returns null (no GPU) -> CPU - WGSL pipeline compile fails -> CPU - `--gpu-shader-blend` set but shader not in `SHADERS_WGSL` -> CPU - Mid-render GPU failure -> log + disable GPU + finish on CPU Failure reason is logged once per worker (not per frame). No crash path. - Port the remaining 12 shaders to WGSL (mechanical: `flashThroughWhite`, `chromaticSplit`, `sdfIris`, `glitch`, `lightLeak`, `crossWarpMorph`, `whipPan`, `cinematicZoom`, `gravitationalLens`, `rippleWaves`, `swirlVortex`, `thermalDistortion`, `domainWarp`, `ridgedBurn`). - Once Mac wall-time confirms the lever, decide on flip-default-on or keep opt-in. - Update fixture pins for any fixture that needs to exercise the GPU path under CI (PSNR-only). - A persistent device + pipeline cache shared across workers (currently per-worker init). _PR drafted by Vai_ Co-Authored-By: Vai <vai@heygen.com>
67 lines
2.0 KiB
JSON
67 lines
2.0 KiB
JSON
{
|
|
"name": "@hyperframes/cli",
|
|
"version": "0.6.6",
|
|
"description": "HyperFrames CLI — create, preview, and render HTML video compositions",
|
|
"repository": {
|
|
"type": "git",
|
|
"url": "https://github.com/heygen-com/hyperframes",
|
|
"directory": "packages/cli"
|
|
},
|
|
"bin": {
|
|
"hyperframes": "./dist/cli.js"
|
|
},
|
|
"files": [
|
|
"dist"
|
|
],
|
|
"type": "module",
|
|
"scripts": {
|
|
"test": "vitest run",
|
|
"dev": "tsx src/cli.ts",
|
|
"build": "bun run build:fonts && tsup && bun run build:runtime && bun run build:copy",
|
|
"build:fonts": "node -e \"const fs=require('fs');if(!fs.existsSync('../producer/src/services/fontData.generated.ts')){const {execSync}=require('child_process');execSync('cd ../producer && tsx scripts/generate-font-data.ts',{stdio:'inherit'})}else{console.log('[build:fonts] skipped — fontData.generated.ts already exists')}\"",
|
|
"build:runtime": "tsx scripts/build-runtime.ts",
|
|
"build:copy": "node scripts/build-copy.mjs",
|
|
"typecheck": "tsc --noEmit"
|
|
},
|
|
"dependencies": {
|
|
"@hono/node-server": "^1.8.0",
|
|
"@puppeteer/browsers": "^2.13.0",
|
|
"adm-zip": "^0.5.16",
|
|
"citty": "^0.2.1",
|
|
"compare-versions": "^6.1.1",
|
|
"esbuild": "^0.25.12",
|
|
"giget": "^3.2.0",
|
|
"hono": "^4.0.0",
|
|
"mime-types": "^3.0.2",
|
|
"onnxruntime-node": "^1.20.0",
|
|
"open": "^10.0.0",
|
|
"postcss": "^8.5.8",
|
|
"prettier": "^3.8.1",
|
|
"puppeteer-core": "^24.39.1",
|
|
"sharp": "^0.34.0"
|
|
},
|
|
"devDependencies": {
|
|
"@clack/prompts": "^1.1.0",
|
|
"@hyperframes/core": "workspace:*",
|
|
"@hyperframes/engine": "workspace:*",
|
|
"@hyperframes/producer": "workspace:*",
|
|
"@hyperframes/studio": "workspace:*",
|
|
"@types/adm-zip": "^0.5.7",
|
|
"@types/mime-types": "^3.0.1",
|
|
"@types/node": "^25.0.10",
|
|
"linkedom": "^0.18.12",
|
|
"picocolors": "^1.1.1",
|
|
"tsup": "^8.0.0",
|
|
"tsx": "^4.0.0",
|
|
"typescript": "^5.0.0",
|
|
"vitest": "^3.2.4"
|
|
},
|
|
"optionalDependencies": {
|
|
"@google/genai": "^1.50.1",
|
|
"webgpu": "^0.4.0"
|
|
},
|
|
"engines": {
|
|
"node": ">=22"
|
|
}
|
|
}
|