Commit Graph

364 Commits

Author SHA1 Message Date
Vance Ingalls 088a98a19c fix(producer): preserve window.__hf set by page scripts
The render bridge previously overwrote `window.__hf` wholesale at end-of-body,
clobbering anything written during user `<script>` execution. Libraries such as
`@hyperframes/shader-transitions` populate `window.__hf.transitions` inside
their `init()` call, so the engine never saw a transition map and HDR
compositing fell back to plain DOM layers.

Two changes restore the contract:

1. Inject HF_EARLY_STUB at the very start of `<head>` so `window.__hf` exists
   before any other script runs. User libraries can now safely write to it
   during initialisation.
2. HF_BRIDGE_SCRIPT now patches the existing `__hf` object (defining `duration`
   as a getter and assigning `seek`) instead of replacing it, preserving any
   fields the page already populated.

Adds a regression test that simulates the real injection order — early stub,
authored page script writing `__hf.transitions`, bridge script — and asserts
both `transitions` and the bridge-installed `seek`/`duration` are present
afterwards.

Made-with: Cursor
backup/pre-sync-phase1-1776638866
2026-04-19 14:20:05 -07:00
Vance Ingalls 2c1d362172 fix(hdr): address deferred items from PR #288 review
- blitRgb48leRegion: take canvasHeight as an explicit parameter so callers
  with non-square canvases or other byte-stride conventions cannot silently
  produce a wrong height. Updates all alphaBlit.test.ts call sites.
- renderOrchestrator: cache max frame index per pre-extracted HDR frame
  directory and bounds-check videoFrameIndex before existsSync. Avoids
  redundant filesystem syscalls when the requested time falls past the
  last extracted frame for a given clip.

Made-with: Cursor
backup/2026-04-18/hdr-phase-1
2026-04-18 19:31:56 -07:00
Vance Ingalls 4c005e31cb test(engine): expand PNG decoder coverage + α=254 boundary
- decodePng: filter coverage (None/Sub/Up/Average/Paeth), multi-IDAT
  payload, Adam7/missing-IHDR rejection
- decodePngToRgb48le: BE→LE swap, full 16-bit precision, row-major
  layout, 8-bit rejection
- blitRgba8OverRgb48le: α=254 must blend (no fast-path overwrite at
  the opaque boundary)
- producer: replace silent catches in renderOrchestrator with log.warn,
  drop unused convertHdrFrameToRgb48le import, tighten GSAP comment

Made-with: Cursor
2026-04-18 19:31:56 -07:00
Vance Ingalls 4669a152a3 fix(hdr): address review feedback across stack
- Assert PNG interlace === 0 (reject Adam7) and IHDR presence.
- Validate parseTransformMatrix against NaN/Infinity.
- Wrap captureScreenshotWithAlpha restore in finally.
- Document sRGB→HDR LUT convention (scene-light, no OOTF, 203-nit PQ).
- Note optimizeForSpeed empirical behavior for alpha preservation.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 19:31:56 -07:00
Vance Ingalls e5a95ef228 fix(producer): preserve window.__hf properties in bridge script
Use Object.assign to merge runtime bridge (seek/duration) with
existing window.__hf properties instead of overwriting them.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 19:31:56 -07:00
Vance Ingalls 54f6163897 refactor: consolidate PNG decoder, fix assertions in alphaBlit
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 19:31:56 -07:00
Vance Ingalls 7d8c5d20ef feat(engine): add HDR two-pass compositing with DOM alpha overlay
Per-frame in-memory alpha compositing: Chrome screenshots DOM with
transparent background (PNG alpha), FFmpeg extracts native HLG/PQ
frames as 16-bit PNG. DOM pixels (sRGB RGBA8) composited over HDR
(rgb48le) via 256-entry sRGB-to-HLG/PQ LUT. Pure Node.js PNG decoder,
single-pass FFmpeg extraction, native HDR detection before SDR
conversion.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 19:31:55 -07:00
Vance Ingalls fb11145e47 fix(hdr): emit HDR10 mastering display + max-cll SEI in x265 params
Without `master-display=...:max-cll=...` in -x265-params the encoder
writes a perfectly valid HEVC stream tagged BT.2020/PQ, but with no
HDR10 metadata SEI messages. Apple, YouTube, and HDR TVs see the file
as "unknown HDR" and tone-map it incorrectly (washed-out highlights,
clipped specular detail).

Changes:
- Add HdrMasteringMetadata + DEFAULT_HDR10_MASTERING (D65, P3-D65 mastering
  primaries, 1000-nit max, 400-nit MaxFALL — the standard Hollywood HDR10
  envelope).
- Extend getHdrEncoderColorParams to compose the full x265-params string
  including range=full + master-display + max-cll for both PQ and HLG.
- Refactor streamingEncoder to call getHdrEncoderColorParams instead of
  duplicating the colorprim/transfer/colormatrix string literal.
- Export buildStreamingArgs so the new streamingEncoder.test.ts can pin
  down the exact CLI flags as a regression net.

Verified end-to-end with hdr-smoke.ts on hdr-pq + mixed-sdr-hdr fixtures:
ffprobe -show_frames now reports "Mastering display metadata" and
"Content light level metadata" SEI messages on both outputs.
2026-04-18 19:31:54 -07:00
Vance Ingalls f670339edc refactor(hdr): use analyzeCompositionHdr helper, drop duplicate exports
- Replace inline HDR auto-detection in renderOrchestrator with the
  analyzeCompositionHdr helper from @hyperframes/engine
- Remove the backward-compat re-export of isHdrColorSpace from
  videoFrameExtractor; callers should import from @hyperframes/engine
- Use isHdrColorSpaceUtil consistently inside videoFrameExtractor
- Remove duplicate isHdrColorSpace tests from videoFrameExtractor.test.ts
  (canonical tests live in utils/hdr.test.ts)

Addresses reviewer feedback on PR #265.

Made-with: Cursor
2026-04-18 17:06:42 -07:00
Vance Ingalls 1ed6f27df9 fix(deps): declare @webgpu/types as devDependency
The engine package's hdrCapture.ts uses WebGPU APIs and references
@webgpu/types via a triple-slash directive, and both engine and
producer tsconfigs include @webgpu/types in compilerOptions.types.
However the package was never declared as a devDependency, so it
worked locally (transitive resolution) but failed in CI with
TS2688: Cannot find type definition file for '@webgpu/types'.

Add @webgpu/types ^0.1.69 as a devDependency to both packages and
refresh bun.lock so the CI Build and Typecheck jobs pass.

Made-with: Cursor
2026-04-18 17:06:42 -07:00
Vance Ingalls 40e4e61a25 fix(hdr): restore .gitignore and revert package.json changes
Restore .gitignore entries removed during branch creation (docs/images/,
examples/, .playwright-mcp/, skills/, captures/, etc.) and revert the
@webgpu/types devDep additions — the WebGPU path is experimental and
skipLibCheck:true means the reference directive is a no-op when the
types aren't installed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls 58d09ee7d3 fix(hdr): address review feedback across stack
- Distinguish ERR_MODULE_NOT_FOUND from other errors in puppeteer dynamic
  import so broken installs surface instead of silently falling back.
- Document detectTransfer's isHdrColorSpace precondition.
- Rename misleading chunkEncoder test name.
- Clarify streamingEncoder Buffer.from as HDR-only by code path.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls 8fa94d0c44 fix(producer): add crf and videoBitrate to RenderConfig
These fields were added to main after this branch was created. Without
them, the CLI's render command fails typecheck when passing crf/bitrate
options to createRenderJob.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls a410d5835d fix(engine): copy buffer before pipe write in streaming encoder
Node streams hold a reference to the provided buffer and drain it
asynchronously. Without copying, callers that reuse buffers get
partially-overwritten data in the pipe.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls c849f09f9c refactor: remove redundant variables in hdr.ts and hdrCapture.ts
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls 28bce52c9d feat(engine): add HDR video output pipeline
Auto-detect HDR sources via ffprobe, switch to H.265 10-bit with
BT.2020/HLG metadata. HLG pass-through (no conversion), hvc1 codec
tag for Apple playback, 1-hour streaming timeout for HDR. WebGPU
float16 readback pipeline implemented and tested but gated behind
headed Chrome. HDR utilities with 15 unit tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:42 -07:00
Vance Ingalls 7f44d5d248 fix(engine): normalize mixed HDR/SDR video inputs to prevent washed-out colors
When a composition contains both HDR (bt2020) and SDR (bt709) video elements,
the SDR content appears washed out because Chrome clamps HDR frames to sRGB
during rendering. This adds automatic color space detection and normalization
during the video frame extraction phase — if any video is HDR, all SDR videos
are converted to HLG/BT.2020 (npl=600) via zscale before frame extraction.

- Add colorSpace (transfer, primaries, matrix) to VideoMetadata via ffprobe
- Add isHdrColorSpace() to detect bt2020/PQ/HLG content
- Add convertSdrToHdr() using zscale with 600 nit nominal peak luminance
- Restructure extractAllVideoFrames into 3 phases: resolve, normalize, extract
- 6 new unit tests for HDR detection across all color space variants

Same approach as HeyGen Rio's pipeline (heygen_rio.py#L568-585).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-18 17:06:41 -07:00
James 38b7cb1c66 chore: release v0.4.6 v0.4.6 2026-04-18 22:36:20 +00:00
James Russo ad11de698c feat(producer): auto-fallback screenshot capture for raf and iframes (#331)
* fix(core): drive adapter seeks when composition has no GSAP timeline

renderSeek returned early when deps.getTimeline() was null, skipping the
onDeterministicSeek call that drives all frame adapters (CSS, WAAPI,
Lottie, Three.js). That meant compositions using any non-GSAP animation
primitive froze on their initial frame during capture.

Now we still quantize the seek time and fire onDeterministicSeek even
without a timeline, so each adapter gets a chance to advance.

GSAP compositions are unaffected — timeline-driven seek still takes the
same path it did before.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(producer): auto-fallback screenshot capture for raf and iframes

Co-Authored-By: Codex <codex@openai.com>

* test(producer): add render compatibility regression fixtures

Co-Authored-By: Codex <codex@openai.com>

* fix(core): scrub CSS animations via WAAPI currentTime

Co-Authored-By: Codex <codex@openai.com>

* test(producer): cover css keyframe renders

Co-Authored-By: Codex <codex@openai.com>

* fix(producer): propagate virtual time into iframe documents

Co-Authored-By: Codex <codex@openai.com>

* test(producer): refresh iframe docker golden

Co-Authored-By: Codex <codex@openai.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Codex <codex@openai.com>
2026-04-18 15:08:52 -07:00
Miguel Ángel 59aa2c9ec9 chore: release v0.4.5 2026-04-18 20:24:03 +01:00
James Russo f8906e8385 docs(guides): add Performance guide and preview-stutter troubleshooting (#327)
* docs(guides): add performance guide and preview-stutter troubleshooting

Adds a dedicated Performance guide covering preview-vs-render cost model,
expensive CSS patterns (backdrop-filter, filter, shadows), image sizing,
and how to diagnose slow compositions with Chrome DevTools.

Cross-links from troubleshooting (new "Preview stutters" accordion) and
common-mistakes (new "Oversized source images" and "Heavy backdrop-filter
stacks" accordions). Wires the new page into docs.json nav.

Also fixes a pre-commit format hook edge case: oxfmt would exit 2 when
the only staged files matching the format glob were all covered by
.prettierignore (e.g. docs-only changes). Add --no-error-on-unmatched-pattern
to the lefthook oxfmt invocation so docs-only commits are not blocked.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs: call out preview performance limits at the entry points

The preview command, studio package, and determinism concept pages all
frame preview as visually equivalent to render — correct for fidelity,
misleading for playback smoothness. A user who reads those pages and
then hits a paint-heavy composition has no way to know why preview
stutters, short of drilling into troubleshooting.

Adds short notes at each entry point linking out to the new Performance
guide, so users hit the "preview is hardware-bound, render isn't"
explanation wherever they land first.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 12:22:42 -07:00
Miguel Ángel 37370e1e7d fix(cli): set GIT_CLONE_PROTECTION_ACTIVE=0 for skills (GH #316) (#328)
## Summary

Fixes #316 — `hyperframes skills` (and `npx skills add heygen-com/hyperframes`) fails with:

```
■  Failed to clone repository
fatal: active \`post-checkout\` hook found during \`git clone\`
└  Installation failed
```

## Root cause

Two layers stacked:

1. **Git 2.45+ refuses to execute hooks during `git clone` by default.** The opt-in is `GIT_CLONE_PROTECTION_ACTIVE=0` — the env-var name is intentionally explicit about the trade-off.
2. **Users who ran `git lfs install` globally have a post-checkout hook registered at `core.hooksPath`.** When the upstream `skills` CLI shells out to `git clone` to fetch a repo's `skills/` directory, git detects the user's LFS hook and aborts.

The check fires for **any repo**, regardless of whether the cloned repo uses LFS itself — it's protection against the user's own hooks, not the repo's content. Users who have git-lfs installed (very common) hit this for every clone the `skills` CLI does.

## The fix

`hyperframes skills` wraps `npx skills add`. The wrapper now sets `GIT_CLONE_PROTECTION_ACTIVE=0` on the spawned child's env via a single helper (`gitCloneFriendlyEnv`) with a docstring at the call site explaining exactly why. The rest of `process.env` is preserved — proxy settings, extra CA certs, locale, etc. stay untouched.

## What this fix doesn't do (deliberately)

This is the **code-path-we-own** fix. The deeper root cause is that the upstream `skills` CLI (vercel-labs/skills) should set this env var when it shells out to `git clone`. That would fix the bug for every user invoking `skills` directly — not just those who route through our wrapper. An upstream issue should be opened separately; not landing it as part of this PR.

## Users who call `npx skills add` directly

Documented in the new troubleshooting subsection: set the env var manually.

```bash
GIT_CLONE_PROTECTION_ACTIVE=0 npx skills add heygen-com/hyperframes
```

## Tests

`packages/cli/src/commands/skills.test.ts` — 2 cases:
- Every spawned child has `GIT_CLONE_PROTECTION_ACTIVE=0`
- The rest of `process.env` is preserved (not a wiped env)

Uses `vi.mock` on `node:child_process` because ESM doesn't allow live-module `vi.spyOn` on re-exported bindings.

## Docs

`docs/packages/cli.mdx` — new **Troubleshooting** subsection under the `skills` command. Explains both the automatic fix (`hyperframes skills` users are already covered) and the manual workaround (`npx skills add …` users).

## Closes

- #316
2026-04-18 21:17:26 +02:00
Miguel Ángel e8a48a62d0 fix(producer): external assets work on Windows (GH #321) (#324)
* fix(producer): external assets work on Windows (GH #321)

Two Unix-only assumptions in the external-asset pipeline caused every
absolute path on Windows to be rejected as "unsafe" at render time:

1. Containment checks used `child.startsWith(parent + "/")`. On Windows
   the separator is `\`, so the predicate is always false unless the
   paths are equal — every external asset tripped the safety guard in
   `renderOrchestrator.ts`. The reporter saw:

     [Render] Skipping external asset with unsafe path:
       hf-ext/D:\coder\reactGin\hyperframes\reading\assets\segment_001.wav

   Fix: use `path.relative()` through a shared helper
   `isPathInside(child, parent)` that normalises separators per-platform
   and correctly rejects siblings whose names start with the parent
   (e.g. `/foo/bar-sibling` is NOT inside `/foo/bar`).

2. The external-asset key was built as `"hf-ext/" + absPath.replace(/^\//, "")`.
   A Windows absolute path (`D:\coder\...`) became
   `"hf-ext/D:\\coder\\..."` — and because Node's `path.join` treats a
   drive-letter prefix as absolute, `join(compileDir, key)` silently
   escaped `compileDir`. Fix: `toExternalAssetKey()` strips the drive
   colon and normalises to forward slashes, producing
   `hf-ext/D/coder/...` — a pure relative path that `path.join` cannot
   promote to absolute on any OS.

Both helpers live in `packages/producer/src/utils/paths.ts` and are
exercised by 14 unit tests covering Unix paths, Windows drive-letter
paths, mixed separators, sibling-prefix confusion, and `..` traversal.

Docs: new "External assets" section in `docs/packages/producer.mdx`
describes detection, sanitised keys, and the cross-platform containment
invariant.

Closes #321.

* fix(producer): address review on #324 — UNC + integration test

Addresses the non-blocking observations from the PR #324 staff review
(https://github.com/heygen-com/hyperframes/pull/324#issuecomment):

1. UNC and extended-length Windows paths.
   `toExternalAssetKey` now handles:
   - `\\?\D:\very\long\path\clip.mp4` (extended-length)      → `hf-ext/D/very/long/path/clip.mp4`
   - `\\server\share\file.wav` (plain UNC)                    → `hf-ext/unc/server/share/file.wav`
   - `\\?\UNC\server\share\file.wav` (extended-length UNC)   → `hf-ext/unc/server/share/file.wav`
   The UNC-collapsed form keeps the server boundary so two different
   servers exposing the same share/file name cannot collide under one
   relative key. Previously both edge cases silently produced keys with
   stray `?` or `:` characters that downstream `isPathInside` rejected —
   not a security hole, but a silent drop of user assets.

2. Short-circuit on already-sanitised input.
   `toExternalAssetKey("hf-ext/…")` now returns its input unchanged
   instead of prepending `hf-ext/` a second time. Makes the helper
   genuinely idempotent, which is what the unit test claimed all along.
   Renamed the test accordingly.

3. JSDoc caller contract.
   `toExternalAssetKey` now documents that it expects canonicalised
   input (`path.resolve`'d upstream) and does not strip `..`
   components. `isPathInside` at copy time is still the defensive
   backstop — called out explicitly in the doc so future callers read
   the contract before the code.

4. End-to-end integration test.
   `renderOrchestrator.test.ts` gains two seam tests that run the full
   external-asset pipeline — build the sanitised key, populate an
   `externalAssets` map, invoke `writeCompiledArtifacts`, and assert
   both the success path (the file lands under `<compileDir>/hf-ext/…`)
   and the escape-rejection path (a malicious `hf-ext/../../etc/passwd`
   key does NOT materialise above `compileDir`). `writeCompiledArtifacts`
   is exported for the test seam with a clear JSDoc disclaimer that
   it's not part of the public API.

22 tests pass across `paths.test.ts` (17) and `renderOrchestrator.test.ts` (5).

Out of scope for this follow-up (tracked as follow-ups):
- Centralising every `startsWith("/")` absolute-path check into a
  shared helper across htmlCompiler / audioExtractor / audioMixer /
  videoFrameExtractor. Mentioned in the review; touches 5 files and
  deserves its own PR.
- Windows CI runner.
2026-04-18 20:59:51 +02:00
Dylanwoo 64e3735100 fix(cli): doctor shows platform-correct install hints (#319)
The "FFmpeg not found" hint was hardcoded to `sudo apt install ffmpeg`
for any non-macOS platform — Windows users would see an apt command that
doesn't exist on their system, and Red Hat / Arch users got the wrong
package manager too.

`getFFmpegInstallHint()` already exists in browser/ffmpeg.ts (and is
already used by render.ts) and handles darwin / linux / win32 correctly.
Use it here too.

Also rewrite checkFFprobe:
- it previously used `which ffprobe` which is not available on Windows
  (cmd uses `where`), so on Windows the check always reported "Not
  found" even when ffprobe was on PATH
- run `ffprobe -version` directly instead, which works cross-platform
  whenever ffprobe is resolvable on PATH, and surfaces the version
  string in the same style as the FFmpeg check
2026-04-18 11:59:49 -07:00
James Russo 03bf1e69f4 chore: add linguist overrides so TypeScript is the dominant language (#326)
HTML files in this repo are compositions (user-facing content and
registry templates), not the framework source. The framework itself is
TypeScript. Hide HTML from Linguist's detection so the language bar on
GitHub reflects what the repo actually implements.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-18 11:54:58 -07:00
James 0f4fcbeed8 chore: release v0.4.4 v0.4.4 2026-04-18 03:37:09 +00:00
James Russo 686e45dac0 fix(engine): suppress font-load 404s by checking console location URL (#313)
Chrome's "Failed to load resource" message text does not include the failing
URL — it's only on msg.location().url. The previous filter in frameCapture.ts
only checked msg.text(), so every font 404 (e.g. Google Fonts <link> tags
in sandboxed render environments) fell through to the "[non-blocking]"
prefix instead of being suppressed.

Extract the classifier into isFontResourceError() and match against both
text and location.url, and extend the extension match to .ttf/.otf. Adds
a unit test covering the URL-in-location, URL-in-text, and non-font cases.

This is a targeted fix for the render-output noise that PR #311 attempted
to address by adding a ~120-entry SYSTEM_FONTS skip list. That approach
silently shadowed existing FONT_ALIASES (arial→inter, helvetica→inter,
courier new→jetbrains-mono, segoe ui→roboto, etc.) and changed render
output on Linux fleets that don't have those fonts installed. Fixing the
console-log filter here suppresses the noise without changing any font
resolution behavior.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 17:38:46 -07:00
Miguel Ángel 42d39866ff fix(studio): iPhone Safari layout + touch-drag scrubber (#308)
Stacked on top of #307. Fixes three mobile UX bugs that made the studio unusable on iPhone Safari — discovered while testing the audio-ownership work from #307 on a physical device.

## Bugs fixed

### 1. Untappable Play button / bottom controls

`#root` was set to `100vh`. iOS Safari reports `100vh` as the **largest** viewport (toolbar hidden) and never shrinks it — so with the toolbar visible, the bottom of the layout sits under it. The Play button + timecode were fully occluded.

### 2. Scrubber not draggable by touch

The seek bar had only `onMouseDown`. Mouse events don't fire for touches on iOS Safari, so nothing responded. You could tap to jump but not drag.

### 3. Safari's horizontal swipe hijacked scrubber drags

Even when the seek bar caught `pointerdown`, `touch-action: manipulation` still let Safari consume horizontal edge-swipes for back-navigation — dragging the scrubber left was impossible.

## What changed

| File | Fix |
|---|---|
| `packages/studio/src/styles/studio.css` | `#root` → `height: 100dvh` with `100vh` fallback. Dynamic viewport height shrinks when the iOS toolbar is visible, so the bottom of `#root` lines up with the visible area. |
| `packages/studio/src/App.tsx` | Two `h-screen` containers → `h-full` so nested children fill the now-dynamic parent instead of asserting `100vh` and overflowing. |
| `packages/studio/index.html` | Added `viewport-fit=cover` so iOS exposes real `env(safe-area-inset-bottom)` values. |
| `packages/studio/src/player/components/PlayerControls.tsx` | Controls row gets `padding-bottom: calc(0.5rem + env(safe-area-inset-bottom))` so it clears the landscape home indicator. Scrubber replaced `onMouseDown` with `onPointerDown` + `setPointerCapture`, plus `touch-action: none` so Safari doesn't hijack horizontal swipes. Added `pointercancel` + window-level `pointerup` fallbacks. |

All desktop code paths are unchanged: `100dvh` falls back to `100vh`, `env(safe-area-inset-bottom)` is `0` off-iOS, Pointer Events subsume Mouse Events on desktop.

## Verified live

Via the `cloudflared` tunnel I ran during review on the factory-series-c-video project:

- iPhone Safari, portrait: Play button now fully visible and tappable. Bottom controls sit just above the URL bar.
- iPhone Safari, landscape: controls clear the home indicator.
- Finger-drag the scrubber left and right: tracks the touch smoothly, finger can leave the 6 px bar height without losing the drag.
- Desktop click-to-seek and click-drag: still work.
- Arrow-key seeking: still works.

## Stacked dependency

Base is `fix/player-audio-ownership-review` (PR #307). Once #307 merges, rebase this branch onto `main` — the changes are fully independent; the stacking is just to avoid waiting on the review for #307 before shipping pure UX wins.

## Test plan

- [x] `tsc --noEmit` on `packages/studio` — clean
- [x] `bun run --filter @hyperframes/studio build` — clean
- [x] Live repro on iPhone Safari via cloudflared tunnel: Play button tappable, scrubber drags with touch
- [ ] Android Chrome sanity pass before release (same Pointer Events code path, but worth eye-balling)
2026-04-18 01:44:46 +02:00
Miguel Ángel c49181f1fa fix(player): address #298 review — tighter drift, dynamic proxies, ownership event (#307)
Follow-up to PR #298 addressing @jrusso1020's review. Each item below maps to a point in his comment.

## Significant

### 1\. Drift threshold 150 ms → 50 ms

_mirrorParentMediaTime_ was too loose for lip-synced talking-head content. ITU-R BT.1359 puts A/V perceptibility at ±45 ms; 150 ms sat well inside the "unacceptable" zone. Dropped to 50 ms, extracted as a static constant for clarity.

**Verified live on factory-series-c-video (agent-browser):** steady-state offset under parent ownership sampled five times over 400 ms = `[35.7, 33.5, 31.2, 27.2, 36.9]` ms — below the perceptibility floor. Before this PR the same measurement could drift up to 150 ms before correction.

### 2\. Dynamic sub-composition media proxies

Under parent ownership, a sub-composition that attaches a new `<audio data-start>` mid-playback was correctly silenced in the iframe (sticky `outputMuted`) but had no parent-frame counterpart to play → silent hole in the audio track.

Added a `MutationObserver` on the iframe body watching for `audio[data-start]` / `video[data-start]` additions. New elements are adopted through the same `_adoptIframeMedia` helper the initial scan uses, and if parent ownership is already active the new proxy gets its `currentTime` mirrored and `play()` called immediately (gated on `!this._paused`). Observer disconnects on iframe reload + component disconnect.

### 3\. `bridgeMuted` sticky in `syncRuntimeMedia`

The asymmetry James flagged: `outputMuted` was sticky per-tick, `bridgeMuted` was one-shot via `onSetMuted`. A sub-composition activating after a user mute would briefly play at author volume before the next bridge message.

`syncRuntimeMedia` now accepts `userMuted` and the per-clip loop uses a single combined `shouldMute` gate. One invariant, two inputs.

### 4\. Reset `_audioOwner` on iframe reload

The latch never cleared. On composition switch the player would stay in `parent` ownership against a fresh runtime that hadn't received `set-media-output-muted` and whose autoplay-blocked latch was clean — a brief double-audio window until the next `NotAllowedError` re-promoted (idempotently).

`_onIframeLoad` now resets `_audioOwner = "runtime"`, pauses any parent proxies, and disconnects the old MutationObserver before a fresh one attaches to the new document. If the player had been in `parent` ownership, a corresponding `audioownershipchange` event fires with `reason: "iframe-reload"`.

## Worth addressing

### 5\. Promotion → observable event + reason

Promotion was invisible. Added `CustomEvent("audioownershipchange", { detail: { owner, reason } })` fired on every owner transition. `reason` is either `"autoplay-blocked"` (promote → parent) or `"iframe-reload"` (reset → runtime). Gives host apps an SLO-ready signal for "% of sessions in parent ownership" without exposing internal state.

**Verified live:** dispatching a synthetic `media-autoplay-blocked` in the live studio produced `{ owner: "parent", reason: "autoplay-blocked" }` on the web component exactly once.

### 6\. Parent proxy play() rejection → `playbackerror` event

Previously swallowed silently. Now re-emitted as `CustomEvent("playbackerror", { detail: { source: "parent-proxy", error } })` so embedding apps can recover or fall back.

### 7\. Mobile verification on real hardware

Tested with a tunnel in a real iOS device.

## Test gaps (from review)

- `userMuted` stickiness (mirror of the existing `outputMuted` test).
- **OR invariant** between `outputMuted` and `userMuted` — explicit test that setting one false while the other is true keeps `el.muted === true`.
- **Contract pin:** `syncRuntimeMedia` fires `onAutoplayBlocked` on **every** rejection (no internal dedupe) — so a future refactor can't quietly move the latch and break the caller's posting logic.
- **Caller-side latch pattern:** a 5-rejection simulation with the init.ts-style wrapper posts exactly once.
- **`audioownershipchange`** **dispatch** on promotion + once per transition (no duplicate on idempotent re-promote).
- **Mid-playback promotion:** `_paused = false` at flip time fires `_playParentMedia` immediately.
- **`playbackerror`** **surface** on parent proxy rejection with the right `source` tag.

## Minor

- One-line comment on `_promoteToParentProxy` explaining the `postMessage` async race (the mute lands after ~one message-loop tick; the autoplay gate that triggered promotion keeps the iframe rejecting `play()` during that window, so the double-play bug doesn't reappear).

## What's good (from the review)

Kept as-is — noted for posterity:

- `muted` vs `volume` framing (orthogonal channels).
- Probing reality via `NotAllowedError` instead of `matchMedia('(pointer: coarse)')` / UA sniffing.
- Two orthogonal mute channels.
- Backwards compat (new actions / messages safely ignored by either side).

## Test results

- `packages/core/src/runtime/media.test.ts` — **42 tests pass** (+4 new: `userMuted` sticky, OR invariant, fires-every-rejection, caller-latch dedupe)
- `packages/core/src/runtime/bridge.test.ts` — **15 tests pass**
- `packages/player/src/hyperframes-player.test.ts` — **26 tests pass** (+3 new: `audioownershipchange` dispatch, mid-playback promotion, `playbackerror` surface)
- Typecheck green on `core` + `player`
- `tsup` build green on `core` / `player` / `cli`
- Live factory-series-c-video repro via agent-browser: runtime ownership still zero `volumechange` thrash, zero `PARENT.play()` calls; parent ownership measures 27–37 ms steady-state drift, well inside the 50 ms threshold.

## Test plan

- [x] Unit tests (83 total across touched files)
- [x] Typecheck clean
- [x] Build clean
- [x] Live studio repro on factory-series-c-video: runtime path unchanged, parent path drift tightened
- [x] `audioownershipchange` event fires with correct detail on synthetic autoplay block
- [x] Physical iOS / Android device verification (unchanged since #298)
2026-04-18 00:49:57 +02:00
Miguel Ángel e4cfcd3f61 fix(cli): serialize port-availability probes (#309) (#310)
Closes #309. Full credit to @gigadeniga for the diagnosis — the root cause + proposed fix in that issue are exactly what landed here.

## The bug

\`npx hyperframes preview\` failed deterministically on Crostini (ChromeOS Linux) with \`Ports 3002–3101 are all in use\`, even when nothing was actually listening on any of them.

## Why

\`testPortOnAllHosts\` ran four probes in parallel:

\`\`\`ts
const hosts = ["127.0.0.1", "0.0.0.0", "::1", "::"];
const results = await Promise.all(hosts.map((h) => isPortAvailableOnHost(port, h)));
\`\`\`

Each probe binds a socket and then calls \`server.close()\`. Close is async — the socket stays open until its callback fires on the next event-loop tick. While it's open, the wildcard binds (\`0.0.0.0\`, \`::\`) that include the loopback address race the still-open loopback socket and return \`EADDRINUSE\` spuriously. On Crostini this happens 100% of the time; other Linux configs hit it intermittently; macOS is less predictable. Net effect: every port in the 100-port scan range appears busy and the preview refuses to start.

Reproduces on any Linux box with the standalone snippet from the issue:

\`\`\`
127.0.0.1: OK
0.0.0.0: EADDRINUSE   ← false positive
::1: OK
::: EADDRINUSE        ← false positive
\`\`\`

## Fix

Serialize the probes. Each socket is fully closed before the next opens, eliminating the race window entirely.

\`\`\`ts
for (const host of hosts) {
  const available = await isPortAvailableOnHost(port, host);
  if (!available) return false;
}
return true;
\`\`\`

Kept the four-host check rather than collapsing to just \`0.0.0.0\` + \`::\` — the multi-host coverage is load-bearing for the devbox / SSH-forwarding case where a port is free on loopback but held on the wildcard. Sequentializing is the smaller, less-behaviourally-affecting fix.

## Regression tests

\`packages/cli/src/server/portUtils.test.ts\` — three cases binding real sockets, no mocks:

- **Returns true for a genuinely free port** — directly reproduces the Crostini bug; would fail on Linux against the parallel implementation.
- **Returns false when the port is occupied on \`0.0.0.0\`** — confirms the multi-host check still catches the devbox scenario.
- **Releases each probe socket before the next run** — two back-to-back calls for the same free port both return true, pinning the sequential contract against future refactors that might try to reparallelize for perf.

## Test plan

- [x] \`bunx vitest run packages/cli/src/server/portUtils.test.ts\` — 3/3 pass
- [x] Full CLI suite — 109/109 pass
- [x] \`tsc --noEmit\` clean

## Notes

- Independent of any version bump; ship whenever.
- Probing 4 hosts serially adds at most ~tens of milliseconds per port on the scan (binds are very fast on loopback). The worst-case cost shows up when the first port in the range is free — previously 1 parallel round-trip, now 4 sequential — and it's imperceptible (\`preview\` bind is a one-time startup cost, not a hot path).
2026-04-18 00:44:58 +02:00
Ular Kimsanov a95539a0fd Merge pull request #299 from heygen-com/feat/capture-improvements-v2
fix: double-audio scaffold, lint rules, docs guide, Gemini 3.1
2026-04-17 14:49:05 -04:00
Miguel Ángel 9ef864d1f2 fix(docs): serve hyperframes.json / registry JSON schemas (#304) (#305)
Closes #304.

## Summary

The three `/schema/*.json` URLs baked into every Hyperframes project as `\$schema` references are 404ing on the live docs site — blocking editor autocomplete and validation.

- \`https://hyperframes.heygen.com/schema/hyperframes.json\` — **404** (missing entirely)
- \`https://hyperframes.heygen.com/schema/registry.json\` — **404** (only in npm package)
- \`https://hyperframes.heygen.com/schema/registry-item.json\` — **404** (only in npm package)

Mintlify serves top-level non-MDX dirs in \`docs/\` at \`/\<dir>/*\` (confirmed by \`docs/logo/*.svg\` → \`/logo/*.svg\`). This PR drops the three schemas into \`docs/schema/\` so the URLs resolve.

## What changed

| File | Role |
|---|---|
| \`docs/schema/hyperframes.json\` | **New.** Authored from the \`ProjectConfig\` type in \`packages/cli/src/utils/projectConfig.ts\`. |
| \`docs/schema/registry.json\` | Mirror of \`packages/core/schemas/registry.json\`. |
| \`docs/schema/registry-item.json\` | Mirror of \`packages/core/schemas/registry-item.json\`. |
| \`scripts/sync-schemas.ts\` | Keeps the registry mirrors in lockstep with their authoritative copies in \`packages/core/schemas/\`. \`--check\` mode fails the Docs workflow on drift. |
| \`.github/workflows/docs.yml\` | Runs \`tsx scripts/sync-schemas.ts --check\` on every PR touching docs or core schemas. |
| \`package.json\` | \`sync-schemas\` / \`sync-schemas:check\` npm scripts. |

## Why not make \`packages/core/schemas/\` authoritative for \`hyperframes.json\` too?

\`hyperframes.json\` is CLI config, not a core type. Keeping the schema in \`docs/\` avoids an artificial dependency between \`@hyperframes/core\` and \`@hyperframes/cli\`. If the two ever need to align, we can flip the direction then.

## Verification

- \`bun run sync-schemas:check\` → \`2/2 in sync\`.
- Ajv (draft 2020-12, in-process) validation against 9 cases:
  - ✓ real factory-series-c-video config
  - ✓ default shape from \`hyperframes init\`
  - ✓ \`\$schema\` is optional
  - ✓ missing registry → rejected
  - ✓ missing paths.assets → rejected
  - ✓ extra top-level key → rejected
  - ✓ empty registry string → rejected
  - ✓ empty block path → rejected
  - ✓ missing paths entirely → rejected

## Test plan

- [x] \`tsx scripts/sync-schemas.ts --check\` passes locally
- [x] Schemas parse as valid JSON and validate real/default project configs
- [x] After merge: \`curl -sI https://hyperframes.heygen.com/schema/hyperframes.json\` returns 200 once Mintlify redeploys
- [x] Same check for \`/schema/registry.json\` and \`/schema/registry-item.json\`
- [x] VS Code autocomplete and error-highlighting work on \`hyperframes.json\` without extra config

## Notes

- The Docs workflow now triggers on \`packages/core/schemas/**\` and \`scripts/sync-schemas.ts\` in addition to \`docs/**\`, so a core-schemas change that forgets to run \`sync-schemas\` will fail CI instead of silently publishing stale docs.
- No runtime / API changes to any package; ship independent of a version bump.
2026-04-17 17:43:43 +02:00
ukimsanov 274db7a5ef fix: address PR #299 review — lint correctness, docs, Gemini benchmark
- lintMultipleRootCompositions: scan filesystem for HTML files with
  data-composition-id (was filtering results array — always 1 entry)
- lintDuplicateAudioTracks: order-independent attribute extraction,
  dedup by (src,start,duration,trackIndex), Infinity fallback for
  missing data-duration (matches runtime behavior)
- 10 new tests for both lint rules
- docs: explicit skill invocation, remove gsap-skills, fix indentation
- Gemini: env override (HYPERFRAMES_GEMINI_MODEL), benchmark data in
  code comment (49 imgs: 3.1-lite ~507ms/img, 2.5-lite ~230ms/img)
- cli.mdx: version-agnostic "Gemini vision" reference

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-17 10:15:26 -04:00
Miguel Ángel 46afe2ea5d chore: release v0.4.3 v0.4.3 2026-04-17 12:15:37 +02:00
James Russo 4ae5c0340f chore(docs): migrate docs/images/ media to static.heygen.ai CDN (#301)
Move all preview mp4/png/gif assets under docs/images/ out of the repo
and serve them from https://static.heygen.ai/hyperframes-oss/docs/images/
(backed by s3://heygen-public/hyperframes-oss/docs/images/, CloudFront).

Drops ~49MB from the working tree and, more importantly, ~49MB from every
future Mintlify build checkout. Combined with the (already-LFS-tracked)
producer snapshots, the remaining bloat in 'npx skills add heygen-com/
hyperframes' (see #300) is LFS smudge during clone — separate fix needed
in the skills CLI to pass GIT_LFS_SKIP_SMUDGE=1.

Changes:
- Delete docs/images/** (103 files, ~49MB). Files are uploaded to S3 already.
- Rewrite /images/* references in 44 MDX files, TemplateCard.jsx, and
  catalog-index.json to absolute CDN URLs.
- Update README.md img src to CDN URL (renders correctly on GitHub).
- Add docs/images/ to .gitignore so regenerated previews aren't committed.
- Add scripts/upload-docs-images.sh to sync docs/images/ → S3 after running
  the preview generators.
- Wire up bun run upload:docs-images and bun run generate:catalog-previews
  scripts in package.json.
- Update generator script docstrings to point at the upload step.

External contributors can still regenerate previews locally (mintlify dev
reads the CDN URLs, so broken previews appear only for newly added items
pending a maintainer upload). Maintainers run:
  bun run generate:catalog-previews --only <name>
  bun run upload:docs-images

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-16 23:13:43 -07:00
ularkim de5b53c08b feat(capture): switch Gemini 2.5 Flash → 3.1 Flash Lite
Gemini 3.1 Flash Lite Preview: 2.5x faster TTFT, 45% faster output,
slightly cheaper ($0.25/M vs $0.30/M input), near-2.5-Flash quality.
Descriptions are actually more detailed in testing.
2026-04-16 22:58:50 -04:00
ularkim a77a6cbbf7 fix: double-audio bug + lint rules + docs guide + capture improvements
Double-audio bug fix:
- scaffolding.ts: stop writing index.html in captures/ (root cause —
  runtime discovered scaffold + real index.html as two compositions)
- New lint rule: multiple_root_compositions — errors if >1 root HTML
- New lint rule: duplicate_audio_track — warns on overlapping audio

Capture improvements (from testing 30+ websites):
- Catalog runs BEFORE extractHtml (which mutates DOM — converts img src
  to data URLs). HeyKuba: 2 images → 78.
- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load image wait, CSS background-image cataloging
- SVG naming from class/id/parent (not just aria-label)
- Gemini batch 5→20, pause 12s→2s, maxOutputTokens 300→500
- Asset descriptions sorted: captioned first

Docs:
- New guide: guides/website-to-video.mdx (full tutorial)
- CLI docs: added capture and snapshot commands
- docs.json: website-to-video in Guides nav

C
2026-04-16 22:58:50 -04:00
Miguel Ángel 3256551a5e fix(player): single-owner audio to prevent double voice in preview (#298)
## Summary

Fixes the double-voice issue in studio preview where narration plays twice with a drifting offset (measured 23ms → 80ms over a 28s clip).

## Root cause

Two audio pipelines were playing the same source in parallel:

1. The iframe runtime played `<audio data-start>` elements via `syncRuntimeMedia` — the intended path.
2. `<hyperframes-player>` also created parent-frame `<audio>` copies on iframe load and auto-played them in response to every runtime `state` message.

The existing `_muteIframeMedia` tried to silence the iframe copies via `el.volume = 0`, but `syncRuntimeMedia` re-asserts `el.volume` from `data-volume` every tick, so the mute never held. Studio seeks went through `__player.seek()`, which only updated the iframe timeline; parent copies kept their stale `currentTime` and drift compounded across seeks.

Confirmed via agent-browser instrumentation on `factory-series-c-video`:
- 6 `volumechange` events per play cycle (mute-fight signature)
- Both copies audible at `volume=1`, offset growing 23ms → 80ms
- Every seek widened the drift further

PR #295 (v0.4.2) actually **made it audible** — before that, parent copies 404'd on the wrong URL and played silently. Fixing the URL exposed the latent double-playback.

## Fix

Explicit single-owner audio ownership between `<hyperframes-player>` and the runtime.

- **Default ownership is `runtime`**: iframe drives audible playback; parent proxies stay paused and inert. Matches every desktop / studio code path. No parent `play()`, no `volumechange` thrash.
- **On `NotAllowedError`** from the runtime's `play()` attempt (autoplay-gated iframes), the runtime posts `media-autoplay-blocked` once. The player promotes to `parent` ownership: sends `set-media-output-muted: true` to the runtime, starts parent proxies, mirrors `currentTime` from state messages with a 150ms correction threshold.

Two orthogonal mute channels replace the volume fight:

| Channel | Purpose |
|---|---|
| `set-muted` | User's mute preference (existing, unchanged) |
| `set-media-output-muted` | Internal ownership handoff (new) |

`syncRuntimeMedia` now accepts `outputMuted` and asserts `el.muted = true` per active tick — sticky against sub-composition media that arrives mid-playback. Uses native `muted` (orthogonal to `volume`) so no other code path can clobber it.

## Why this shape

- **Single owner, explicit transition.** No races, no tug-of-war.
- **Probes reality, not device class.** We flip on an actual `NotAllowedError`, not on `matchMedia('(pointer: coarse)')` or user-agent sniffing.
- **Uses `muted` instead of abusing `volume`.** `muted` is orthogonal to `volume`; `syncRuntimeMedia` doesn't write to it; author / user settings stay intact.
- **Parent proxies become a thin mirror.** Under parent ownership, their `currentTime` is slaved to the iframe timeline via state messages — no independent drift.
- **Backwards compatible.** Old runtimes without the new bridge action ignore the message; old players without the new message just get the previous behavior.
- **Capture engine unaffected** — it bypasses both DOM pipelines and muxes audio from source files.

## Files changed

- `packages/core/src/runtime/types.ts` — `set-media-output-muted` action + `media-autoplay-blocked` outbound message types.
- `packages/core/src/runtime/state.ts` — `mediaOutputMuted` + `mediaAutoplayBlockedPosted` fields.
- `packages/core/src/runtime/bridge.ts` — route new action to `onSetMediaOutputMuted`.
- `packages/core/src/runtime/media.ts` — `outputMuted` param asserts `el.muted = true` per tick; `NotAllowedError` detection fires `onAutoplayBlocked`.
- `packages/core/src/runtime/init.ts` — wire new bridge handler; coordinate with `set-muted`; post `media-autoplay-blocked` once per session.
- `packages/player/src/hyperframes-player.ts` — `_audioOwner` state; delete `_muteIframeMedia`; `_promoteToParentProxy`; mirror parent `currentTime`; gate all parent play/pause/seek on ownership.

## Verified end-to-end with agent-browser on `factory-series-c-video`

**Runtime ownership (default — desktop studio):**

| | Before | After |
|---|---|---|
| `PARENT.play()` calls per play cycle | 1 | **0** |
| iframe `volumechange` events | 6 | **0** |
| Audible streams | 2 (drifting) | **1 (iframe)** |

**Parent ownership (simulated autoplay block — direct message):**

| | Value |
|---|---|
| iframe audio | `muted=true`, `volume=1` (untouched) |
| parent audio | `muted=false`, `volume=1`, audible |
| Parent ↔ iframe `currentTime` offset | ~6 ms steady state |
| Offset > 150 ms | corrected by mirror sync |

**Mobile path simulated with iPhone 14 emulation + injected `NotAllowedError` from iframe `<audio>.play()`:**

Event timeline captured via agent-browser instrumentation:

```
t=0.0 ms   IFRAME.play() called                       ← runtime attempts playback
t=0.4 ms   IFRAME.play() REJECTED: NotAllowedError    ← simulated mobile gate
t=0.4 ms   →IFRAME bridge set-media-output-muted=true ← player promotes
t=0.6 ms   PARENT.play() called                       ← parent proxy starts
t=0.8 ms   ←IFRAME msg media-autoplay-blocked         ← runtime signal
t=1.0 ms   PARENT.play() resolved                     ← audible
t=1.3 ms   IFRAME muted=true, volume=1                ← iframe silenced via native muted
```

Steady state at t=4 s under promoted parent ownership:

| Element | currentTime | paused | volume | muted |
|---|---|---|---|---|
| Parent audio | 4.060 s | false | 1.0 | **false** (audible) |
| Iframe audio | 4.068 s | false | 1.0 | **true** (silent) |

**Offset: 8 ms**, single audible stream, orthogonal mute channel respected.

## Test plan

- [x] `bunx vitest run` under `packages/core` — **467 / 467 pass** (incl. 4 new `media.test.ts` + 2 new `bridge.test.ts`)
- [x] `bunx vitest run` under `packages/player` — **23 / 23 pass** (3 rewrites for new contract, 2 new for promotion flow)
- [x] `bun run build` — all packages green
- [x] Fresh preview + browser repro on `factory-series-c-video`:
  - [x] Runtime ownership: single audio stream, no drift
  - [x] Parent ownership promotion via direct `media-autoplay-blocked` message: iframe muted, parent audible
  - [x] iPhone 14 emulation + injected `NotAllowedError`: full promotion chain verified in ~1 s, 8 ms steady-state offset
  - [x] No `volumechange` thrash in either ownership mode
- [x] One round of QA on a physical iOS / Android device before release — exercises real `NotAllowedError` path (expected behavior identical to simulation above)
2026-04-17 04:46:22 +02:00
Miguel Ángel d291358cbc chore: release v0.4.2 v0.4.2 2026-04-17 01:16:46 +02:00
Miguel Ángel 5e52e27872 fix(engine): auto-fall back to screenshot mode when chrome-headless-shell drops HeadlessExperimental.beginFrame (#296)
Closes #294.

## Summary

Recent `chrome-headless-shell` builds (observed on 147) no longer expose `HeadlessExperimental.beginFrame`. The domain's `enable`/`disable` methods are deprecated upstream and appear to have been dropped alongside `beginFrame` in these builds, so on Linux with chrome-headless-shell the engine aborts with

\`\`\`
Protocol error (HeadlessExperimental.beginFrame):
'HeadlessExperimental.beginFrame' wasn't found
\`\`\`

and — because the browser was launched with `--enable-begin-frame-control` — the compositor waits for beginFrames the engine can no longer deliver, so every subsequent screenshot also comes back blank. Today users have to discover `PRODUCER_FORCE_SCREENSHOT=true` themselves (openclaw did exactly that — see the issue body).

## Fix

One-time probe, right after the browser launches in beginframe mode:

1. Create a disposable CDP session.
2. `await client.send("HeadlessExperimental.enable")`.
3. Send one no-op `HeadlessExperimental.beginFrame` raced against a 2s timeout.
4. If anything throws / times out — missing method, protocol error, stuck call — close the browser, strip beginframe-only chrome flags, relaunch in screenshot mode, and set \`captureMode = "screenshot"\` for the returned session.

Probing `beginFrame` directly rather than `enable` alone is important because some builds keep the domain registered (so `.enable()` succeeds) while dropping the method itself — that's exactly the failure shape in #294.

Cost on happy path: one extra CDP round-trip per browser acquisition (≈ a few ms, since in beginframe-control mode the command returns as soon as the compositor acks). Cost on broken path: one extra launch, which is what the env-var escape hatch already forces manually.

The beginframe-only flag set is enumerated in-module and matched by the stripper, so adding/removing flags stays in one place with `buildChromeArgs`.

## Test plan

- [x] `bun run --filter=@hyperframes/engine test` — all 42 tests pass
- [x] `bun run --filter=@hyperframes/engine build` — typechecks
- [x] `bunx oxlint` + `bunx oxfmt --check` clean
- [x] Manual: standalone test on Linux x86_64 with chrome-headless-shell 146 — probe returns `supported=true`, no fallback (happy path)
- [x] Manual: same test with `--force-fail` simulating openclaw's missing-method condition — fallback triggers, flags stripped, relaunch succeeds, 6.8 KB PNG captured (broken path)
- [ ] Verify on openclaw / real chrome-headless-shell 147 build that the fallback triggers automatically without `PRODUCER_FORCE_SCREENSHOT`

## Notes

- `probeBeginFrameSupport` catches any failure generically; we trust that a working browser answers the no-op beginFrame in well under 2s.
- Warning is logged once per browser acquisition, not per frame.
- Browser pool interaction: pooled browsers cache the resolved `captureMode`, so subsequent acquires in the same process reuse the post-fallback mode without re-probing.
2026-04-17 01:10:08 +02:00
Miguel Ángel 96376c9be0 chore: release v0.4.1 v0.4.1 2026-04-17 00:25:34 +02:00
Miguel Ángel e70687b66c fix(player): resolve iframe media src against iframe baseURI (#295)
## Summary

`_setupParentMedia` scans the iframe for `audio[data-start]` / `video[data-start]` and creates parallel media elements in the host document (so the studio can scrub audio at sub-frame precision without iframe cross-origin restrictions). It was reading the raw `src` attribute string and assigning it directly to the host-document element, which then resolved relative URLs against the **studio root** instead of the **iframe**.

Result: a composition like

\`\`\`html
<audio id="narration" data-start="0" data-duration="53" src="assets/narration.wav"></audio>
\`\`\`

played fine in rendered MP4 output but 404'd silently in the studio preview (parent audio got `src = http://localhost:PORT/assets/narration.wav` instead of `http://localhost:PORT/api/projects/<name>/preview/assets/narration.wav`).

## Fix

Resolve the src against \`iframeEl.ownerDocument.baseURI\` before passing it to \`_createParentMedia\`. Also read the raw \`src\` attribute on \`<source>\` fallbacks so both paths go through the same resolution.

Diff is 2 lines of meaningful change (9 total once you include the comment).

## Reproduction

1. Create a project with a narration at \`assets/narration.wav\`
2. Reference it in \`index.html\` with \`<audio data-start="0" data-duration="53" src="assets/narration.wav">\`
3. \`npx hyperframes preview\` → open, click play
4. Before: silent (parent audio's \`error.code === 4\` / \`MEDIA_ERR_SRC_NOT_SUPPORTED\`)
5. After: narration plays, scrubbing syncs

## Test plan

- [x] Existing 21 player tests pass (`bun run --filter=@hyperframes/player test`)
- [x] oxlint + oxfmt clean on changed file
- [x] Manual: verified in-studio playback of a narration sourced via relative URL
- [x] Reviewer: confirm render pipeline unaffected (render doesn't go through `_setupParentMedia`)

## Notes

No tests added for this path because the existing harness covers only the `audio-src` attribute codepath — `_setupParentMedia` is triggered by an internal probe interval against a live iframe, which the current fixture doesn't build. Happy to add one in a follow-up if reviewers want that coverage before merge.
2026-04-17 00:23:07 +02:00
James 60780774bb chore: release v0.4.0 v0.4.0 2026-04-16 19:58:39 +00:00
Miguel Ángel f1a37400b4 fix(runtime): silent-first-play + loading overlay for preview (#293)
## Summary

Fixes three audio-sync defects in the studio preview plus a small UX improvement. All four land in one commit so the PR stays aligned with one bug fix per commit.

### 1\. Silent / very-late first play on slow-loading audio (`packages/core/src/runtime/media.ts`)

`syncRuntimeMedia`'s old flow — when it hit `readyState < HAVE_FUTURE_DATA` — called `el.load()` and attached a `canplay` listener to retry `play()`. Two real problems, neither of which is "lost user activation" (the sync runs from a 50 ms `setInterval`, well outside any gesture window):

- `bindMediaMetadataListeners` already sets `preload="auto"` and calls `el.load()` at runtime init. The sync's duplicate `el.load()` aborts that in-flight fetch and restarts from zero — on slow networks this delayed playback by seconds, which users perceived as "silent until a second click."
- The `canplay` listener was racy: the event can fire between `load()` and `addEventListener`, leaving the element wedged.

`HTMLMediaElement.play()` is already spec'd to queue playback until data arrives, so we can unconditionally call it. Drop the `readyState` gate, the redundant `load()`, and the `canplay` listener. Also dedup in-flight `play()` calls with a `WeakSet` (cleared on `playing`/`pause`/`error`) — without it the 50 ms poll fires 20–40 spurious calls per element during buffer, each silencing real `AbortError`/`NotAllowedError` diagnostics in the `.catch`.

### 2\. Audible stutter on rapid pause/play (`packages/core/src/runtime/media.ts`)

The 0.3 s drift-seek threshold fired on nearly every toggle because pause/play ordering between timeline and media produces 0.1–0.4 s of transient drift. Each forced `el.currentTime = relTime` drops `readyState` and surfaces as a `waiting` event the user hears as a stutter. Threshold raised to 0.5 s.

### 3\. Skipped words on cold first play (`packages/core/src/runtime/media.ts`)

Even with 0.5 s, drift grew past 0.5 s during initial buffering while the audio element was stuck at `currentTime = 0`. The old logic would then force-seek audio forward and the user missed the opening of the narration.

Fix distinguishes drift that grows _gradually_ (buffer catch-up, ~16 ms/tick) from drift that _jumps_ in one tick (a scrub). Only jumps, first-tick clip activation, or catastrophic drift (>3 s) trigger a resync. Inline tradeoff note in code: strictly lip-synced dialogue would want a tighter threshold (~0.15 s) outside a 500 ms toggle window — deferred to a future PR.

### 4\. "Loading assets…" overlay in the studio preview (`packages/studio/src/player/components/Player.tsx`)

Spinner while every timed `<audio>`/`<video>` has enough buffered data and every Lottie animation is loaded. Preserves the previous overlay state on cross-origin / transient-DOM catches so a brief access failure doesn't flicker, and logs `console.debug` when the 10 s safety cap trips so a stuck asset is diagnosable. Lottie readiness handles both `lottie-web` (`isLoaded`) and `@dotlottie/player-component` (`totalFrames > 0`), with an inline `@see` pointing to `packages/core/src/runtime/adapters/lottie.ts` so the two sites stay in sync.

## Verification

- 456 core tests pass; 34 in `media.test.ts` cover synchronous play, preload nudge, play-request dedup, offset-jump vs gradual drift, first-tick hard-sync, catastrophic-drift safety valve, and inactive-clip baseline reset.
- Full monorepo build green (`bun run build`), typecheck clean, lint/format clean.
- End-to-end with agent-browser against a composition that uses a 50 s voiceover plus multiple sub-composition video clips. Four scenarios, all pass:

| Scenario | Metric | Result |
| --- | --- | --- |
| Normal first play | Audio plays from click, smooth progression | ✅ |
| Cold play (forced unbuffered) | First `play` event fires at `ct: 0` — no word-skip | ✅ |
| Rapid pause/play (12 toggles) | `waiting` events: 1 (was 40+ bursts) | ✅ |
| Scrub mid-playback | Lands exactly at target frame | ✅ |

## Files changed

- `packages/core/src/runtime/media.ts` — unconditional synchronous `play()`; play-request dedup WeakSet; offset-jump-only drift correction; 0.5 s threshold; first-tick hard-sync; catastrophic-drift safety valve.
- `packages/core/src/runtime/media.test.ts` — coverage for the above plus the gradual-drift cold-play case, scrub offset-jump, in-flight dedup, and inactive-clip baseline reset.
- `packages/core/src/runtime/adapters/lottie.ts` — exported `isLottieAnimationLoaded` helper documenting the two supported player shapes.
- `packages/studio/src/player/components/Player.tsx` — loading-assets overlay with cached-return catch, timeout debug log, and the Lottie readiness check.

## Follow-ups (deferred)

- Tight-threshold short-window drift correction for lip-synced dialogue.
- A perf-regression test that fails on `waiting`\-event resurgence.

## Test plan

- [x] `hyperframes preview` a composition with audio, `Cmd+Shift+R`, click play immediately — audio starts from the very beginning, no skipped words.
- [x] Rapidly pause/play the preview — audio stays smooth (no stutter, no `waiting` events).
- [x] Cold-load a composition — "Loading assets…" overlay appears and disappears once media buffers.
- [ ] Scrub the timeline mid-playback — audio follows the scrub, lands on frame.
2026-04-16 21:51:06 +02:00
Ular Kimsanov 87f4c77e2f feat: website capture pipeline + 7-step video production skill (#284)
* feat(cli): add website capture with AI-powered DESIGN.md generation

Adds `hyperframes capture <url>` command that extracts a complete design
system from any website, producing AI-agent-ready output:

- Full-page screenshot (lazy-load aware, nav at top)
- AI-generated DESIGN.md via Claude API (colors, typography, elevation,
  components, do's/don'ts) with programmatic asset catalog (136+ assets
  with HTML context annotations like img[src], css url(), link[rel=preload])
- CSS-purged compositions (87% size reduction via PurgeCSS)
- HTML-prettified compositions (one-tag-per-line for AI readability)
- CLAUDE.md + .cursorrules auto-generated for AI agent instructions
- Asset deduplication (srcset variants) and tracking pixel filtering

* feat(cli): add gemini 3.1 pro, playwright screenshots, replica refinement

- switch to gemini 3.1 pro (gemini-3.1-pro-preview) with claude fallback
- playwright for full-page screenshots (fixes puppeteer gradient/fixed bugs)
- replica refinement loop: generate, screenshot, compare, fix
- extract inline svgs (50 max, 10kb each) to assets/svgs/
- extract visible text in dom order for content accuracy
- detect js libraries (gsap, three.js, scrolltrigger) via globals
- improved asset catalog grouping and naming
- reverse-engineered aura system prompt documentation
- comprehensive session handoff doc

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: update session handoff with slack research findings

- key finding: team already wants DESIGN.md integration (James, Bin, Vance)
- skills quality matters enormously - must invoke /hyperframes-compose
- eval infrastructure exists (Abhay's dashboards, Teodora's 78-criteria guide)
- templates at templates/ need study before finalizing skill
- session handoff updated with critical next steps

* refactor(cli): simplify capture pipeline, remove replica generator

* feat(capture): add Lottie detection and WebGL shader extraction

Captures Lottie animations via network interception and WebGL shader
source via gl.shaderSource hooking during site crawl. Updates
website-to-hyperframes skill with asset planning guidance, Lottie/shader
reading instructions, and stronger creative direction for scene planning.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* refactor(capture): clean pipeline + shader-first creative workflow

Capture pipeline:
- Remove dead deps (puppeteer-extra, stealth plugin, duplicate devDeps)
- Remove duplicate generateAgentPrompt() call (first lied about DESIGN.md)
- Remove dead canvas-to-image code in htmlExtractor (post canvas removal)
- Parallelize image downloads (batches of 5 via Promise.allSettled)
- Fix pre-existing TS error (match[1] guard in font downloader)
- Default capture output to captures/<hostname>

Skill creative overhaul:
- Add shader transition selection to creative director step (Step 4)
- Add shader wiring instructions to engineer step (Step 5)
- Replace 4-line energy modifiers with visual vocabulary table
- Strip rigid scene-by-scene templates from video-recipes.md
- Strip example fill data from scene plan tables
- Add "read transition refs before planning" instruction
- Add creative ambition language ("how the hell did they make this")

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add skill architecture redesign spec

Comprehensive redesign of website-to-hyperframes skill and capture
pipeline based on code review findings and Claude Code architecture
research. Key changes: remove AI auto-generation, restructure skill
into phases, embed shader boilerplate in scaffold, fix color format.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add implementation plan for skill architecture redesign

13-task plan covering: capture pipeline cleanup (remove AI generation,
fix colors to HEX, add asset descriptions, shader-ready scaffold),
skill restructuring (4 phases with artifact gates), and compose skill
Visual Identity Gate upgrade.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(capture): remove AI auto-generation and SDK dependencies

* fix(capture): convert extracted colors to HEX format

* refactor(capture): remove AI key path, add asset descriptions generator

* refactor(capture): update agent prompt, remove hasDesignMd, add asset descriptions

* feat(capture): pre-wire shader transitions in index.html scaffold

* chore: remove duplicate visual-styles.md (canonical is in hyperframes/)

* refactor(skill): rewrite website-to-hyperframes as phase-based orchestrator

* feat(skill): add Phase 1 understand reference

* feat(skill): add Phase 2 design reference with full DESIGN.md schema

* feat(skill): add Phase 3 creative direction reference

* feat(skill): add Phase 4 build reference with inline shader example

* feat(skill): upgrade Visual Identity Gate to produce full DESIGN.md

* docs: update CLAUDE.md skill references for phase-based workflow

* fix: address code review findings

- Remove orphaned `false` argument in generateAgentPrompt call (critical:
  was shifting hasLottie, hasShaders, catalogedAssets parameters)
- Add HSL color handling in rgbToHex via temp element resolution
- Remove build artifact commit section from phase-4-build.md
- Fix __GSAP_TIMELINE reference to __timelines

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(capture): regex double-escape + simplify scaffold + fix asset descriptions

- Double-escape regex in tokenExtractor template literal (\s→\\s, \d→\\d, \(→\\()
  so browser receives valid regex patterns via page.evaluate()
- Simplify index.html scaffold: scene slots + audio + timeline + comment pointing
  to shader-setup.md reference (no broken inline shader boilerplate)
- Fix asset descriptions: use CatalogedAsset.contexts/notes instead of
  nonexistent htmlContext field

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: code review — 16 bugs, 7-step skill rewrite, cleanup

Code fixes:
- snapshot.ts: path traversal guard, browser leak (try/finally), div-by-zero
  for --frames 1, port bind error handling, rAF-based render settle
- index.ts: remove invalid thinkingConfig for gemini-2.5-flash, fix Gemini
  batch/rate-limit comments, fix video preview viewport y-coordinate
- tokenExtractor.ts: remove dead seen[si] dedup code
- gsap.ts: index ALL classes for inline-style transform conflict detection

Skill architecture rewrite (4-phase → 7-step):
- Replace phase-1 through phase-4 with step-1 through step-7
- Add techniques.md (10 visual techniques with code patterns)
- Fix /hyperframes-compose → /hyperframes (skill doesn't exist)
- Fix captures/arc-browser reference → shader-setup.md (file doesn't exist)
- Fix step-7 hardcoded captures/stripe path
- Document Gemini API free/paid rate limits in step-1

Cleanup:
- CLAUDE.md: restore from Stripe-capture overwrite, update 4-phase → 7-step
- .gitignore: add PR #267 skills (hyperframes-animation-map, hyperframes-contrast)
- Delete old phase-*.md, animation-recreation.md, tts-integration.md

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove dev artifacts, research docs, wrong lockfiles

Remove files that shouldn't ship in this PR:
- docs/research/ (aura analysis, prompt catalogs)
- docs/session-*.md, docs/SESSION-HANDOFF.md (dev notes)
- docs/superpowers/ planning and spec docs
- pnpm-lock.yaml at root and cli (repo uses bun, not pnpm)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(CLAUDE.md): align with main — slim format, add website-to-hyperframes mention

Main PR #283 removed the full skills table from CLAUDE.md and moved it
to AGENTS.md. Align with that decision: use main's slim dev-focused
format, fix pnpm→bun references, add one-line /website-to-hyperframes
pointer.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(cli): add capture command to help groups

The capture command was registered in cli.ts but missing from
the help groups, so it wouldn't appear in `hyperframes --help`.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* style: format skill reference files (oxfmt)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate bun.lock after rebase

The lockfile was stale after rebasing onto main — bun install
--frozen-lockfile failed in CI because new dependencies (google/genai,
patchright, purgecss) weren't reflected in the lockfile.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address PR review comments + improve capture quality

Review fixes (16 comments from jrusso1020 + vanceingalls):
- screenshotCapture: remove Playwright dep, use Puppeteer for all screenshots
- screenshotCapture: dynamic screenshot count based on page height (30% overlap)
- snapshot.ts: fix duration() function-vs-property bug, cross-platform path guard
- htmlExtractor: fix code injection via parameterized evaluate
- index.ts: video preview re-measures position after scroll, .env file loading
- capture.ts: BLOCKED.md on timeout failures
- gsap.ts: 5 inline-style lint tests added (all pass)
- Remove Playwright, patchright deps; @google/genai to optionalDependencies
- Gitignore: generic patterns instead of 20 hardcoded directories
- Remove asset-sourcing.md, video-recipes.md (unused, duplicated guidance)

Capture quality improvements (tested on 10+ websites):
- Color extraction: canvas-based oklch/lab resolver, pixel sampling via
  elementFromPoint, broad sweep for accent colors, gradient/shadow extraction
- Section detection: broadened selectors for div-based layouts, height cap
  to skip page-level wrappers, parent bg walkup for dark sites
- Font downloads: cap 6 per family / 30 total (Cal.com: 306→30)
- CTA detection: text pattern matching + nav context filtering
- Heading text: innerText with whitespace normalization
- Gemini captioning: maxOutputTokens 100→300, .env auto-loading
- .env.example updated with GEMINI_API_KEY docs
- TTS ranking: Kokoro first with Python 3.10+ note

* fix: address PR review comments + improve capture quality

Review round 2 fixes (jrusso1020 + vanceingalls):
- verify/index.ts: add path traversal guard (relative + isAbsolute)
- verify/index.ts: fix sections[i] undefined typecheck error (CI green)
- index.ts: escape Lottie JSON with \u003c to prevent </script> breakout
- step-4-storyboard: fix technique count contradiction (2-3 per beat, not
  across whole video)
- step-6-build: perspective tilt uses gsap.set() instead of CSS transform
  (avoids GSAP overwrite conflict)
- step-1-capture: reorder — command first, Gemini note after (zero-config
  is the default path, API key is optional enhancement)
- step-7-validate: add tsx fallback for snapshot command
- step-3-script: vary hook patterns, don't default to number every time
- assetDownloader: exempt SVGs from 10KB minimum filter (company logos
  like Hubspot/Intel/DHL are 2-6KB; HeyGen capture: 13→75 assets)

Note: adm-zip was NOT removed (reviewer #3) — it's still in
packages/cli/package.json:30. The root package.json had patchright
and purgecss removed, not adm-zip.

Note: ANTHROPIC_API_KEY not restored in .env.example — grep confirms
zero references in the entire codebase. The @anthropic-ai/sdk dependency
was removed earlier in this branch.

* refactor(capture): split index.ts (1175 to 566 lines) into modules

Mechanical extraction, zero logic changes.

New files:
- mediaCapture.ts (345 lines): Lottie preview, video manifest/screenshots
- contentExtractor.ts (314 lines): library detection, text, Gemini, asset descriptions
- scaffolding.ts (135 lines): .env loading, project scaffold generation

Also fixes false-positive BLOCKED.md with structural Cloudflare detection.
Tested on 20 websites, pre/post output identical.

* chore(capture): remove --split flow (splitter, verify, cssPurger, purgecss)

The --split feature auto-generates compositions from captured HTML — a
different approach from the /website-to-hyperframes skill workflow where
agents build compositions from scratch using the storyboard.

No skill file, no step reference, and no test session ever used --split.
Removes 923 lines of unused code + purgecss dependency.

Backed up to ~/Desktop/capture-split-backup/ for reference.

* fix(security): add ssrf protection, lottie injection fix, oom guard

- assetDownloader: add isPrivateUrl() guard blocking private IP ranges
  (127.x, 10.x, 172.16-31.x, 192.168.x, 169.254.x), cloud metadata
  endpoints, localhost, and non-HTTP schemes
- mediaCapture: fix Lottie JSON injection by loading shell HTML first
  then passing animation data via parameterized page.evaluate()
- index.ts: check Content-Length header before response.buffer() in
  Lottie network interception to avoid OOM on multi-GB responses

* fix(capture): security fixes, timeout, sub-agent dispatch instructions

Security (from miguel-heygen review):
- assetDownloader: export isPrivateUrl() SSRF guard
- htmlExtractor: add isPrivateUrl check before CSS fetch
- mediaCapture: add isPrivateUrl check before Lottie fetch
- mediaCapture: fix previewPage leak (try/finally)
- mediaCapture: skip Lottie files > 2MB for preview (CDP limit)
- contentExtractor: skip images > 4MB for Gemini captioning
- index.ts: check Content-Length before response.buffer() (OOM guard)
- snapshot.ts: register error handler before server.listen()

Capture improvements:
- Default timeout 30s to 120s (Shopify needs ~90s for Cloudflare)
- step-6-build: sub-agent dispatch template with explicit rules:
  pass file PATHS not contents, use local fonts not Google Fonts,
  verify ../assets/ references after each beat

* fix(capture): catalog before DOM mutation, networkidle2, faster Gemini

Critical: asset cataloger now runs BEFORE extractHtml which converts img
src to data URLs. Framer sites like heykuba.com went from 2 to 78 images.

- networkidle2 instead of networkidle0 (unblocks SPAs with WebSockets)
- Lazy-load wait: scroll to bottom, wait for img.complete
- CSS background-image cataloging for Framer/Webflow
- SVG naming: checks class, id, parent, inner text (not just aria-label)
- Gemini batch 5->20, pause 12s->2s (paid tier: 2000 RPM, ~0.001/img)
- maxOutputTokens 300->500, descriptions sorted captioned-first
- Remove tsx fallback from step-1 (reviewer nit, published CLI has it)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:03:16 -07:00
James Russo ebc12f7dc9 feat(render): add CRF/bitrate controls and improve default quality (#292)
Raise default encoding quality to visually lossless at 1080p (CRF 18)
and expose fine-grained encoding controls for power users.

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-16 12:02:24 -07:00
James 2718de8776 chore: release v0.3.2 v0.3.2 2026-04-16 06:16:31 +00:00
James Russo 237847e5c6 docs: add prompt cookbook + prompting guide for AI agents (#286)
* docs: add prompt cookbook + prompting guide for AI agents

Addresses user feedback that there's no guidance on how to actually
prompt Claude Code (or other agents) once the hyperframes skills are
installed. Adds copy-pasteable example prompts in the README and
quickstart, a new prompting guide page, and a starter-prompt nudge in
the `hyperframes init` output.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): add vocabulary tables, rules, and TTS voice guide

Merges the best content from the internal prompt guide into
prompting.mdx: easing vocabulary, caption tone table, transition
energy matrix, audio-reactive frequency mapping, marker highlight
modes, TTS voice recommendations, rendering quality presets, and
framework rules (technical requirements vs best practices).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): rename page title to "Prompt Guide"

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove greensock/gsap-skills dependency, fix Math.random nuance

The bundled skills/gsap/ already covers the GSAP surface needed for
HyperFrames compositions. Installing greensock/gsap-skills on top adds
a competing full-ecosystem skill that's mostly irrelevant (ScrollTrigger,
Draggable, SplitText, etc.) and can confuse agents about which GSAP
context to load.

Also adds seeded-PRNG nuance to the Math.random() rule in the prompt
guide (matching the skill's actual guidance).

Removed from: skills.ts, README, AGENTS.md, shared AGENTS.md/CLAUDE.md,
and prompting.mdx.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: require minimal reproduction link in bug report template

Adds a required "Link to reproduction" input field asking users to push
a minimal repro to a public GitHub repo (scaffolded via
`hyperframes init repro --non-interactive --example blank`).

Also consolidates the OS/Node/FFmpeg/version fields into a single
"Environment" field using `npx hyperframes info` output — fewer fields
to fill, more consistent data.

Follows the same pattern as Next.js and Gatsby issue templates.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(issue-template): use hyperframes doctor for environment info

`hyperframes info` only prints project metadata (resolution, duration,
elements). `hyperframes doctor` prints the full environment: version,
Node.js, FFmpeg, Chrome, memory, disk, Docker — everything needed to
diagnose bugs.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(prompting): mention validate alongside lint in anti-patterns

Per Vance's review comment — validate catches runtime errors (JS
exceptions, missing assets, contrast) that lint doesn't.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: replace libretto example URL with hyperframes repo

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-15 22:22:46 -07:00
Miguel Ángel a6ff9e2d9f fix(player): preserve iframe media attributes for runtime sync (#291)
## Summary

- `_setupParentMedia()` (added in #266) was stripping `data-start`, `data-duration`, and `src` from audio/video elements inside the composition iframe
- The runtime's `syncRuntimeMedia` queries `audio[data-start]` to find media clips — removing these attributes made the runtime unable to find, sync, or play audio
- Result: silent audio in studio preview and any context where `__player.play()` is called directly (not through the web component)

## Fix

- Keep all iframe media attributes intact so the runtime can track time position and manage playback
- When parent-frame media `play()` succeeds (mobile use case), mute the iframe copies via `volume = 0` to prevent double audio
- On desktop and in the studio (which calls `__player.play()` directly), the runtime's own media sync handles playback normally

## Test plan

- [x] 21 player unit tests pass
- [x] Verified with John Wu's slideshow project: audio element preserves `data-start`, `data-duration`, `src` after runtime init
- [x] Verified runtime `syncRuntimeMedia` finds and plays audio (currentTime advances in sync with timeline)
- [x] Build passes (lint, format, typecheck)
2026-04-16 06:20:28 +02:00
James Russo 87ce26de8a fix(docs): namespace custom CSS variables to prevent Mintlify collision (#285)
The `Copy page` dropdown panel rendered with a transparent background in
light mode because `docs/custom.css` defined `--background-light: #ffffff`
on `:root`. Mintlify's Maple theme owns that variable as a Tailwind color
(space-separated RGB used via `rgb(var(--background-light)/<alpha>)`), so
the hex override produced invalid CSS like `rgb(#ffffff/1)` and the
dropdown's `bg-background-light` class fell back to transparent. Dark
mode was unaffected because the dropdown panel uses `bg-background-dark`,
which custom.css didn't redefine.

Namespaced every custom variable with `--hf-` to make collisions
impossible, and updated the two consumers (`pre`, `::selection`, link
color in custom.css; `.tpl-card:hover` border in template-gallery.css).
2026-04-15 14:07:34 -07:00