mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-14 18:01:20 +08:00
Miguel Ángel
ac671bdf5c
* feat(core): add TypeGPU/WebGPU runtime adapter
Adds a deterministic seek adapter for compositions that render with
TypeGPU or raw WebGPU. Follows the same push+poll pattern as the
Three.js adapter:
- Sets `window.__hfTypegpuTime` on every seek so render loops can
poll it instead of `performance.now()`.
- Dispatches a `"hf-seek"` CustomEvent on `window` so compositions
can imperatively re-render a single frame at the new seek position.
Compositions listen for the event and update their time uniform:
```js
window.addEventListener("hf-seek", (e) => render(e.detail.time));
```
Works with TypeGPU (docs.swmansion.com/TypeGPU) and raw WebGPU alike.
No assumptions are made about pipeline construction — multiple canvases
or renderers are supported by sharing the same event.
- 9 unit tests, all pass
- wired in init.ts adapter array
- `__hfTypegpuTime` declared in window.d.ts
* fix(core): deduplicate hf-seek dispatch across GPU adapters
Both three and typegpu adapters previously dispatched the same
"hf-seek" CustomEvent independently, causing any composition that
registered a listener to receive two events per seek tick — doubling
per-scrub GPU work even though the renders are idempotent.
Fix: extract a shared `dispatchSeekEvent` helper (seek-dispatch.ts)
that deduplicates by exact float equality within the same synchronous
call stack. Both adapters now call this helper instead of dispatching
directly.
Also adds:
- `resetSeekDispatchState()` export for test isolation
- `beforeEach` reset in three.test.ts and typegpu.test.ts
- New typegpu test: "duplicate seek to same time fires event only once"
- Docstring additions to typegpu.ts: render-mode determinism contract
(await device.queue.onSubmittedWorkDone()) and navigator.gpu feature
detection guidance for composition authors
* feat(core): video-texture render compat + TypeGPU skill
Adds the missing pieces for video-backed WebGPU effects in render mode:
- `video-texture-compat.ts`: monkey-patches `GPUQueue.copyExternalImageToTexture`
to detect the engine's injected `<img class="__render_frame__">` siblings and
transparently substitute them for `<video>` sources. Headless Chrome can't
supply decoded video frames to WebGPU, but the engine's pre-extracted frame
images work. Falls through to the original path in preview mode.
- `patchVideoTextureCompat()` wired in init.ts after adapter array creation.
- `skills/typegpu/SKILL.md`: full authoring guide for TypeGPU/WebGPU compositions
covering contract, timeline registration, video-backed effects, frosted blur
via downsample pass, WGSL patterns, and deterministic rendering.
* test(producer): add typegpu-adapter regression test
Self-contained WebGPU composition with:
- Procedural gradient background (no video dependency)
- Animated ring driven by hf-seek time uniform
- Pulsing center glow
- Two GSAP-driven captions testing adapter sync
Verifies the TypeGPU adapter's hf-seek → WebGPU render pipeline
produces deterministic frames. workers: 1 for consistency.
Note: output.mp4 baseline needs to be generated in CI — the local
Docker image can't launch Chrome (ARM/x86 mismatch on Mac).
Hyperframe Runtime Engine
This folder owns the runtime that powers preview and producer parity.
Current Direction
- Runtime source of truth is converging on
hyperframe.ts. - Build produces:
dist/hyperframe.runtime.iife.js(browser bootstrap)dist/hyperframe.runtime.mjs(tooling/tests)dist/hyperframe.manifest.json(version + sha256 + artifact map)
- FE owns iframe runtime injection.
- BE persists raw generated HTML without injecting runtime scripts.
- Producer validates pinned runtime checksum from manifest before render.
Runtime Contract (Stable Surface)
Globals:
window.__playerwindow.__playerReadywindow.__renderReadywindow.__timelineswindow.__clipManifest
postMessage:
- parent -> runtime control:
source: "hf-parent"type: "control"- actions:
play,pause,seek,set-muted,set-playback-rate,enable-pick-mode,disable-pick-mode
- runtime -> parent events:
source: "hf-preview"type: "state"andtype: "timeline"
Determinism baseline:
renderSeekis the producer-canonical seek path.- 30fps quantization and readiness gates are correctness requirements.
Build
bun run --filter @hyperframes/core build:hyperframes-runtime
Security Expectations
- Runtime bootstrap URL must be version-pinned and host-allowlisted.
- Iframe bridge payloads must be schema-validated.
- Unsafe URL schemes (
javascript:and unapproveddata:) are rejected. - Fail closed if runtime bootstrap/handshake is not healthy.
Product Editing Model
- Primary mode: prompt + element picking.
- Secondary mode: manual precision controls.
- Avoid timeline-first manual workflows as default product path.