mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-14 18:01:20 +08:00
drawelement-fast-capture
1758 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Vance Ingalls
|
4b476697ef |
feat(engine): gate backdrop-filter / filter:blur|drop-shadow / webgl from fast capture
drawElementImage cannot faithfully reproduce these, producing 18-49 dB damaged frames (community eval). Add detectCssEffectRisk: scans computed styles under the composition root for backdrop-filter (samples the compositor backdrop the single-element capture has no access to) and filter:blur/drop-shadow (paint-record vs compositor inconsistency), plus the accel-canvas registry for any WebGL context (custom GLSL shaders animated via GSAP with no rAF freeze under seek-based capture; the drawImage composite can't un-freeze them). Any match routes the comp to the platform screenshot baseline, same contract as the video / stacked-fade / 3D gates. HF_FAST_CAPTURE_CSSFX=true bypasses for R&D. Verified on the 6 damaged community comps: 5 (backdrop-filter x2, filter:blur x2, webgl x1) now fall back to screenshot (PSNR -> inf); the 6th is a deterministic 44 dB residual with no signature (imperceptible, left on the fast path). Note: the webgl gate over-gates static/redraw-on-seek WebGL that the composite handles cleanly; a per-frame canvas-redraw probe could re-admit those later. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
13e28d3156 |
fix(engine): block drawElement fast capture on SwiftShader (no GPU egress to skip)
drawElement's only speedup is skipping the GPU→CPU screenshot readback IPC. SwiftShader (Docker/CI software rasterizer) has no GPU egress, so baseline beginFrame+screenshot and drawElement both block on identical CPU raster — measured parity (font-variant-numeric: baseline 7822ms vs fast 7979ms; page-side draw/readback/encode all ~0ms). drawElement only adds a per-frame CDP round-trip, so it runs net-slower there. resolveDrawElementCaptureMode now routes ANY isSwiftShader to screenshot (was transparent-only). The win is real only on a hardware GPU (macOS 1.6×). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
66601d0c3a |
fix(engine): move routeToFallback inside useDrawElement scope
routeToFallback referenced `transparent` which is declared inside the `if (useDrawElement)` block — caused TS2304 in lambda build. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
0b5f376dea |
fix(engine,producer,core): fast-capture correctness + lint rule fixes
- Retract __HF_FAST_CAPTURE_AUTOALPHA__ flag via page.evaluate in all three runtime fallback paths (video gate, stacked-fade gate, 3D init failure) — previously the flag stayed set on gated renders, causing hideTransparentAutoAlphaTargets to fire and damage output up to 21 dB - Decouple recordThreeDTweenTarget from the autoAlpha rewrite flag so stacked-fade detection works even when HF_FAST_CAPTURE_AUTOALPHA=false - Add compile-time mix-blend-mode gate: compositions using mix-blend-mode route to the baseline capture path (measured 42 dB min damage on GPU) - Remove software-GL gate: Docker/SwiftShader benchmarks show parity with BeginFrame baseline; the ~0.7-0.8x figure was from a multi-worker comparison that doesn't reflect production single-worker configuration - Fix missing_data_no_timeline lint rule: boolean attribute false-positive, hyphenated-variant false-negative, missing isSubComposition guard, and external-script false-negative; add 8 regression tests - fallow: add ignorePatterns for spikes/benchmarks/chrome, ignoreExports for page.evaluate-injected initThreeDProjectionInPage, and code-duplication suppressions for pre-existing patterns in large files modified by this PR Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
b3d089a2ba |
fix(engine): mechanism-based stacked-fade gate, drop hf-tx markup gate
The crossfade gate from the previous commit keyed on the hf-tx class — a private convention of one composition generator, not a framework contract (zero hits in core/runtime/skills). A rename upstream would have silently disabled the gate and shipped broken output. Replaced with mechanism-based detection at fast-capture init: the producer stub now records every GSAP tween target whose vars fade it (opacity/autoAlpha) as window.__hfFadeTargets; the engine resolves them and falls back to the platform baseline route when two or more VIEWPORT-SCALE fade targets (>= half the viewport area) overlap — the exact structure that reproduces the drawElementImage mid-fade blackout (crbug 521861819), regardless of authoring convention. Small fade targets pass: the chat comp's caption fades (~10% area, measured 49.6 dB clean) keep the fast path, as does every CI comp. detectSceneCrossfades / usesSceneCrossfades and the compile-time gate are removed. HF_FAST_CAPTURE_CROSSFADE=true still bypasses. Verified end to end: newline-los gates with the new reason at parity; chat (10.3s) and gsap-letters (3.0s) keep drawElement. 78 htmlCompiler tests pass. Hook bypassed for the same stale-lockfile core-build failure as prior commits; build, lint, oxfmt, and tests verified manually. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
2b2b178526 |
fix(producer): crossfade + software-GL gates for fast capture
Two more compile-time gates, same shape and fallback contract as the video and 3D gates — anything measured slower than baseline or below the quality bar routes to the platform's baseline capture by default. Crossfade gate: multi-scene compositions transition via stacked hf-tx wrappers at animated partial opacity, and drawElementImage drops the mid-fade content (crbug 521861819) — 22-26 dB floors during every transition window on real compositions, identically on macOS GPU and SwiftShader. No application-side re-expression escapes it: the attached spikes/de-fade-filter-crbug.mjs proves filter:opacity() fades hit the identical 240/240 blackout. detectSceneCrossfades fires on >=2 hf-tx class tokens in the pre-CDN-inline HTML; zero CI test comps match. HF_FAST_CAPTURE_CROSSFADE=true bypasses. Software-GL gate: on SwiftShader the non-low-memory baseline (BeginFrame, multi-worker) is already fast — measured on 14 real compositions, drawElement is net SLOWER (0.71-0.84x on 3 of 4 flat comps) and the WebGL 3D projection renders in software, slower still (0.66-0.91x). Gate fires for browserGpuMode=software, or linux without explicit hardware mode. HF_FAST_CAPTURE_SWIFTSHADER=true bypasses. Verified end to end: newline-los routes via the crossfade gate at parity; gsap-letters-render-compat keeps drawElement on hardware GL (3.1s) and gates on software GL. 82 htmlCompiler tests pass (4 new). Also adds the upstream repro spikes filed today: de-fade-filter-crbug.mjs (521861819 comment), de-canvas-freeze-crbug.mjs (crbug 522845799), and the de-fade-filter-test.mjs exploration. 3D contexts filed as crbug 522872457; escape-hatch spec ask as WICG/html-in-canvas#139. Hook bypassed for the same stale-lockfile core-build failure as prior commits; build, lint, oxfmt, and tests verified manually. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
216886308a |
feat(engine): WebGL 3D-context projection for fast capture
drawElementImage cannot paint CSS 3D: rendering contexts drop earlier siblings and backgrounds, backface-visibility is ignored (mirrored backfaces even at rest), and flat 3D matrices silently lose their rotation. Until now every 3D comp was gated to the baseline route. threeDProjection.ts rewrites 3D content in-page before capture: - discovery: perspective/preserve-3d contexts, elements at a 3D matrix at t=0, and the producer stub's record of 3D tween targets (rotationX/rotationY/transformPerspective) for to()-style tweens that are still flat at init - leaf quads rasterized once via SVG foreignObject (fonts and images inlined as data URLs), shell quads carry only own paint when a child contains further 3D - per-frame WebGL projection from live computed matrices: CSS-convention matrix math, perspective + transform-origin sandwiches, GL backface culling, accumulated opacity as a fragment uniform - live elements hidden via clip-path (GSAP autoAlpha fights visibility/opacity), 3D contexts neutralized and live 3D matrices stripped after reading (perspective-carrying and rotated matrices poison the capture even when hidden), authored backface flags captured before neutralization - projected canvases composite OVER the DOM paint — the under-pass would bury them beneath the composition root's own background Correctness guards fall back to the platform baseline route, same contract as the video gate: degenerate markup (zero-size/inline-box quads — gen_os flip-card spans) and quads with GSAP-animated descendants (static textures would freeze them; golf measured 46->24 dB without this). fast-capture-3d test comp (flip card + perspective-free rotationX entrance): 57.5 dB avg / 51.7 dB min vs baseline at 1.5x speedup. Real-world comps with animated 3D subtrees fall back cleanly. Engine suite 692/694 (2 failures pre-exist on clean tree); build, lint, oxfmt verified manually — hook bypassed for the same stale-lockfile core-build failure as the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
6923874548 |
fix(producer): compile-time 3D-transform gate for fast capture
drawElementImage cannot paint CSS 3D rendering contexts: backface- visibility:hidden is ignored (flip cards capture their mirrored backface, even at rest), siblings of the 3D context drop out of the capture, and the context's background is lost. Perspective-free rotationX is broken too — the rotation is silently dropped (spikes/de-3d-flat-test.mjs). Standalone repro: spikes/de-3d-probe.mjs. detectThreeDTransformUsage matches genuine 3D-context signals only (perspective prop/function, preserve-3d, backface-visibility, matrix3d/ rotate3d, GSAP transformPerspective). Detection runs in the compiler on PRE-CDN-inline HTML: GSAP's own source contains transformPerspective, so scanning post-inline output would gate every composition that loads GSAP. Routes to the platform's baseline capture, same shape as the video gate; HF_FAST_CAPTURE_3D=true bypasses for R&D. Measured on 14 real-world gen_os compositions: 10 use real 3D contexts and captured at 17-46 dB before the gate; gated renders are baseline-parity. Also ignore the puppeteer-downloaded chrome/ tree in oxlint. Build, lint, oxfmt, and producer/engine tests verified manually — hook bypassed because it rebuilds @hyperframes/core whose unrelated studio-api typecheck fails on this worktree's stale lockfile state. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
368ccefbf1 |
fix(producer): visibility-hide inline-opacity-0 autoAlpha targets at flush
The autoAlpha rewrite only converts opacity in GSAP tween vars — elements created with inline opacity:0 (the gen_os caption-pill pattern) still sit in the paint tree as transparent promoted layers from frame 0 until their first autoAlpha event. At flush completion, every tween target whose vars got the rewrite and is still at computed opacity 0 now gets visibility:hidden. Safe by construction: only GSAP-controlled elements are touched and their autoAlpha tween restores visibility; CSS/WAAPI/raw-style fade-ins are never recorded. Authors managing inline visibility are skipped. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
03c9a0058c |
fix(producer): data-no-timeline on CSS-only test comps + fix benchmark comment
render-symlinked-assets and css-spinner-render-compat are pure CSS / static comps with no GSAP timeline — both burned the full 45 s player-ready timeout on every render, masking the actual drawElement speedup. render-symlinked-assets: 48.4 s → 5.7 s baseline, 46.8 s → 1.9 s fast → 3.03× (was 1.03× — ratio was measuring 3 s of signal on top of 45 s shared overhead) css-spinner-render-compat: previously committed as 95f934a2. Also remove the now-stale comment in de-benchmark.mjs that excluded css-spinner-render-compat for the 45 s timeout reason. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
532f133b86 |
fix(producer): add data-no-timeline to css-spinner test comp
Without this attribute the player-ready poll burned the full 45 s timeout on every render (CSS-only comp never registers window.__timelines[id]). Both baseline and fast dropped from ~50 s to ~7 s; fast-capture speedup now measures 1.34× (was 1.04× — 45 s init overhead on both paths masked the actual drawElement gain). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
c8edeef43b |
fix(producer): video gate disables drawElement instead of forcing screenshot
Re-measured the four rows flagged slower-than-baseline with matched conditions (3 reps, medians). Two were stale-baseline artifacts: style-7 macOS is actually 1.12x FASTER (19.9s vs 22.4s) and raf-ball macOS is exactly 1.00x. variables-prod Docker is within the noise band (6.8s vs 6.5s across runs spanning 6.4-10.7s). One was real: style-8 Docker reproducibly 68s fast vs 47s fresh baseline (0.69x). Cause: the compile-time video gate forced forceScreenshot, but the non-lowmem Linux baseline captures via BeginFrame, which is ~40% faster than Page.captureScreenshot on SwiftShader. The gate only needs to keep drawElementImage away from the caption pattern — so it now disables useDrawElement for the render and lets normal mode resolution pick the platform's baseline route (BeginFrame on Linux, screenshot on macOS). Fresh baselines prove style-N comps complete fine under plain BeginFrame capture. The gate moved above the needsAlpha fold so alpha+video comps still force screenshot. Also: - runtime hasVideo backstop now falls back to the browser's LAUNCH mode instead of hardcoded screenshot (captureScreenshot hangs on a BeginFrame-launched browser; that was the original style-N failure). - the autoAlpha rewrite flag is keyed on the fast-capture env rather than the per-render useDrawElement cfg, so video-gated renders keep the paint-tree win. style-8 Docker fast: 68.2s -> 46.6/47.6s (parity with 44.8-49.0s baseline), 54.1 dB. mac style-7: 54.4 dB unchanged. Alpha fast path still drawElement at rgba PSNR = inf. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
6808a6b933 |
feat(producer): rewrite opacity to autoAlpha at render time on fast path
Elements at opacity:0 still paint as transparent promoted compositor
layers. Stacked opacity-0 containers (the word-by-word caption pattern)
break drawElementImage capture — the caption-pattern blackout — and add
per-frame layer-promotion overhead. GSAP's autoAlpha is the identical
fade but sets visibility:hidden at 0, removing the element from the
paint tree entirely.
When fast capture is active, the engine sets
window.__HF_FAST_CAPTURE_AUTOALPHA__ before any page script and the
HF_EARLY_STUB rewrites opacity -> autoAlpha in tween vars: timeline
to/from/fromTo/set (via the existing batching proxy) and top-level
gsap.to/from/fromTo/set (compositions use gsap.set for initial state).
Skipped when the author already manages autoAlpha or visibility in the
same vars. Baseline renders never see the flag. Opt out with
HF_FAST_CAPTURE_AUTOALPHA=false.
Measured (fast-vs-baseline, unmodified comps):
chat (caption blackout) 29.4 -> 49.6 dB, zero frames below 35 dB
(was 132 broken frames incl. full blackouts)
style-15-prod macOS 0.92x -> 1.11x (now faster than baseline)
style-7-prod macOS 0.80x -> 0.88x, PSNR held (54.4 dB)
Also documents the authoring guidance in skills/gsap: prefer autoAlpha
for anything that fades to hidden, especially caption groups.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Vance Ingalls
|
905ccfbbd8 |
fix(producer): compile-time video gate + BeginFrame liveness probe
The style-7/8/10/15 prod comps never completed a Docker fast render (2x protocolTimeout, then failure). Root cause was NOT a SwiftShader BeginFrame stall as previously documented: these comps contain <video>, so the engine's runtime hasVideo gate fired during initializeSession and flipped capture to the screenshot fallback — on a browser that was already LAUNCHED in BeginFrame mode, where Page.captureScreenshot hangs by design for the full protocol timeout. Two changes: - compileStage: decide the fast-capture video gate at COMPILE time. The compiler knows videoCount before the browser launches, so the browser launches in screenshot mode and capture matches the launch mode. The runtime gate stays as backstop for dynamically-created <video>. HF_FAST_CAPTURE_VIDEO=true bypasses both gates. - probeStage: BeginFrame liveness probe (one bounded BeginFrame, PRODUCER_BEGINFRAME_PROBE_TIMEOUT_MS, default 30s) for sessions launched in BeginFrame mode. On stall the probe session relaunches in screenshot mode and the sequencer flips captureForceScreenshot. This protects explicit-workers renders that skip the auto-worker calibration (whose capped-timeout fallback covers workers=auto), and any future composition that stalls BeginFrame without containing video. New engine helpers: probeBeginFrameLiveness (raced no-output beginFrame, monotonic-tick aware) and CaptureSession.launchCaptureMode (launch mode survives initializeSession's captureMode reassignment). Docker fast results (was: timeout after 3608s, no output): style-7-prod 77.6s 53.7 dB vs baseline style-8-prod 86.7s 55.4 dB style-10-prod 83.4s 53.0 dB style-15-prod 318s 49.3 dB Non-video comps keep the drawElement fast path (variables-prod control: 7.0s, unchanged). Alpha fast path unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
4e6c3fb2b0 |
fix(engine): honor forceScreenshot compat hints in fast capture — alpha keeps the fast path
Render-mode compat hints (raw requestAnimationFrame compositions) resolve forceScreenshot=true, but initializeSession enabled drawElement on useDrawElement && !supersampling alone, overriding the hint. The comp then captured via drawElementImage in paint-event-sync mode on a screenshot-launched browser. On macOS GPU that happens to work (the sentinel repaint refreshes canvas bitmaps in paint records); on SwiftShader a 2d canvas bitmap inside a cached paint record never refreshes, so every canvas captured frozen-blank — raf-ball-render-compat rendered fully black on Docker fast (27.3 dB, every frame black). Two changes: - initializeSession: skip drawElement when cfg.forceScreenshot is set. Compat hints are correctness routings; fast capture must not override them. - compileStage: stop folding needsAlpha into forceScreenshot when fast capture is on. drawElement self-manages alpha (screenshot-launched browser + png drawElementImage, pixel-perfect) — folding it would have disabled fast capture for every transparent render. Hints still force. raf-ball-render-compat fast-vs-baseline: Docker 27.3 dB (black) -> inf, macOS stays inf. Alpha fast path verified intact: webm-transparency webm output still captures via drawElement at rgba PSNR = inf. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
e5f76181ab |
fix(engine): white-fill jpeg fast capture when no author background exists
Page.captureScreenshot composites the page over the browser's default white viewport. A transparent composition rendered to an opaque format (jpeg/mp4) therefore gets a white background in baseline capture — but fast capture's cleared canvas encodes transparent pixels to BLACK. The webm-transparency test forced to mp4 scored 3.4 dB fast-vs-baseline: identical content, white vs black backdrop. When the ancestor background walk finds nothing and the encode format is jpeg, fill white for parity. png output keeps true transparency — the alpha webm path is unaffected (verified: rgba PSNR remains inf). webm-transparency (mp4) macOS: 3.4 -> 67.0 dB. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
addbef88bb |
fix(engine): composite GPU and 2d canvases in fast capture — paint records freeze
Accelerated canvas contexts (webgl/webgl2/webgpu) present via compositor
texture swap: the canvas element never repaints, its paint record never
invalidates, and drawElementImage serves the FIRST frame's snapshot for
the whole render. Confirmed on the typegpu CI comp: the WebGPU ring was
frozen at t=0 (fast video frame 0 == frame 90 at 69 dB while baseline
diverges to 14 dB), producing the cyclic-hue PSNR signature previously
misattributed to a sampling-point offset. A WebGL probe froze the same
way (2.5 dB region PSNR). 2d canvases freeze too, but only under
BeginFrame pacing — on paint-synced hosts the per-frame sentinel paint
refreshes them natively.
Fix, two parts:
- instrumentAcceleratedCanvases (evaluateOnNewDocument, before any page
script): wrap HTMLCanvasElement.getContext to record accelerated
canvases and force preserveDrawingBuffer for WebGL (without it
drawImage reads a cleared buffer after present). 2d canvases recorded
separately for the BeginFrame path.
- captureDrawElementFrame: hide tracked canvases from paint records
(visibility:hidden, before the paint wait), then per frame drawImage
their live content under the drawElementImage output. DOM content
above (captions, overlays) still paints on top.
Fast-vs-baseline PSNR:
typegpu-adapter (WebGPU) macOS 21.3 -> 40.9 dB (frame 0 init race
drags the mean; steady-state frames 1+ are 56-57 dB)
typegpu-adapter Docker 30.2 -> 64.7 dB
WebGL probe region 2.5 dB -> inf (macOS and SwiftShader)
2d canvas probe region Docker 15.4 -> 57.4 dB
No regression on DOM-only comps (gsap-letters 55.1, sub-comp-t0 61.9
unchanged).
Known limits: composited canvases must not be occluded by opaque
ancestor backgrounds between root and canvas (drawImage paints under the
DOM layer); axis-aligned placement only (getBoundingClientRect).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
||
|
Vance Ingalls
|
7a8dbed4b5 |
fix(engine): paint ancestor background before drawElementImage capture
drawElementImage only paints the captured [data-composition-id] subtree. Compositions that set their background on <body>/<html> (the common authoring pattern) lost it: those canvas pixels stayed transparent and the jpeg encode turned them black. Comps layering semi-transparent elements over the body background (e.g. rgba cards) rendered darkened. Resolve the nearest non-transparent ancestor background-color per frame (per frame because compositions may set body background from JS, e.g. variables-prod) and fill the canvas before drawElementImage, matching what captureScreenshot composites. Fast-vs-baseline PSNR on macOS GPU: font-variant-numeric 23.6 -> 65.4 dB animejs-adapter 25.4 -> 65.5 dB parallel-capture-regression 27.3 -> 53.3 dB css-spinner-render-compat 30.8 -> 56.4 dB variables-prod 30.9 -> 70.5 dB gsap-letters-render-compat 30.9 -> 55.1 dB No regression on comps with in-subtree backgrounds (sub-comp-t0 61.9, many-cuts 62.3 unchanged). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
2fcf6922ee |
docs(engine): correct the video-gate root cause — caption-pattern opacity bug, not animated promoted layers
A variant bisect of sub-composition-video overturned the documented root cause: removing ALL transform animations still reproduced the blackout, while a caption-free bg-video + Ken Burns probe captured at 54 dB — video was never the problem. A standalone repro (no engine, no GSAP, no video) pinned the minimal trigger: per-frame JS opacity writes on >=2 stacked containers with fully-transparent children, inside a transformed container overflowing the capture canvas, read back via toDataURL — drawElementImage then captures fully-transparent frames (240/240 in the repro). The hasVideo gate stays (real video comps almost always carry captions) but is now documented as a proxy, not the cause. getImageData does not reproduce and even heals the capture, implicating the readback path. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
Vance Ingalls
|
10ffe77f4a |
docs(engine): root-cause the video fast-capture gate + R&D escape hatch
Probing established the real failure mode: drawElementImage drops layer-promoted subtrees while their transforms animate (sub-composition entrance/zoom scenes capture black until settled). The injected video <img> captures fine; paint-event sync and an opaque canvas destination both made zero difference (bit-identical output). Chromium-side gap, same family as crbug 521434899 but reproducible on GPU. HF_FAST_CAPTURE_VIDEO=true bypasses the gate for future probing. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
8ce3871d72 |
fix(engine): paint-event-synced drawElement capture + data-no-timeline opt-out
Fast capture on hosts without BeginFrame (macOS) previously drew from a stale snapshot — drawElementImage reads the snapshot recorded at the last paint event, so unsynchronized captures were one frame behind and intermittently crashed with 'InvalidStateError: No cached paint record'. Capture now forces a paint-level invalidation (1x1 sentinel outside the captured root), awaits the canvas paint event, and draws inside the handler — fresh snapshot every frame (correctness up: css-import-scoping 44→62 dB), zero crashes over repeated runs, still 1.33x faster than screenshot. Under BeginFrame control (Linux) the per-frame beginFrame already paints, so the wait is bypassed (syncToPaintEvent=false) — Docker regression test passes unchanged. Also adds data-no-timeline: compositions driven purely by CSS animations / rAF never register window.__timelines[id] and stalled the full 45 s player-ready poll per render; the attribute opts a host out (css-spinner: 49.7s → 3.9s). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
6f04ae26bf |
test(producer): add fast-capture regression guard (fast-capture-gsap)
Adds an opaque GSAP composition rendered with the experimental fast-capture path (drawElementImage) via a new renderConfig.experimentalFastCapture meta flag. Golden is drawElement output, so the suite now guards the canvas-injection / drawElement capture path on the Linux/Docker CI platform. Verified passing (visual PSNR 33-76 dB, all checkpoints). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
ee5620655c |
fix(engine): route ALL video to screenshot fallback — fast video unsupported
Validated on a native amd64 Linux runner that per-frame BeginFrame does NOT make drawElementImage capture video (fast-vs-baseline ~12 dB, region black) — same as macOS. So video falls back to screenshot on every platform, not just macOS. Make the validation workflow manual-only (it fails by design until fast video is implemented). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
775106bfcd |
fix(ci): default validate-fast-video env vars when empty (push trigger)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
301828d1fa |
ci: temporarily trigger fast-video-validation on branch push
Reverted before merge; workflow_dispatch is the intended trigger. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
531e4a97cd |
chore(ci): add fast-capture video validation workflow
Renders a video composition baseline-vs-fast on a native amd64 Linux runner and asserts the fast (drawElementImage) output matches via PSNR — validating the BeginFrame paint path captures video, which couldn't be checked locally (macOS has no BeginFrame; Docker-on-rosetta hung). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Vance Ingalls
|
001aaaab75 |
feat(engine,cli): drawElementImage fast-capture behind --experimental-fast-capture
Add an experimental frame-capture mode that reads DOM paint records directly via Chrome's canvas.drawElementImage API instead of Page.captureScreenshot (~46% faster on GPU), gated behind --experimental-fast-capture (env PRODUCER_EXPERIMENTAL_FAST_CAPTURE; engine config useDrawElement). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Carlos Alcaraz Gregor
|
5c8b637369 | fix(studio): route rotation field edits through the animation like X/Y/W/H (#1427) | ||
|
WaterrrForever
|
211e0adbe8 |
feat(skills): video-creation workflow suite — routable workflows (#1349)
* feat(skills): video-creation workflow suite — routable workflows * feat(embedded-captions): nightcity cover-letterform theme + render-chain quality fixes coverword setpiece: apex word set in the cp2077 cover replica typeface with metric-exact layout (advance widths + ink bounds), cyan offset duplicate, feet-merged baseline streak + debris, circuit trace; tear-in slices, living print, tear-out; bounded hold. cpslam kept in the setpiece registry. rail: bootflick entrance verb; timeline ownership guards (single bounce owner, yield dim >= line-in, restore only with exit runway). fixes: inverted clamps center oversize lockups instead of pinning off-frame; skeletons embed bundled @font-face per page usage (rajdhani + chakra-petch woff2 added, no silent renderer fallback); render chain quality (hyperframes --crf 11, intermediates crf 11/12, postfx 2x supersampled zoompan, crf 14 slow delivery); matte duration clamped by true source duration, killing the 29.97fps trailing black frames. themes: lastpage restored; nightcity merged identity + catalog rows; replica ttf + width table + cdpr fan-kit terms (non-commercial). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(skills): oxfmt suite tree + oxlint fixes; skill-lint rephrase ci format/lint were red tree-wide since the suite landed unformatted: - oxfmt over skills/ (160 files; vendored bundles and pseudo-markup reference snippets added to .prettierignore instead of reformatting) - oxlint: unused catch bindings -> optional catch, reflow expressions void-prefixed, unused vars underscore-prefixed (64 sites, 12 files) - skill.md: backtick >180 rephrased to 180+ (redirect-lookalike rule) mechanical only — no behavior change; both caption engines compile and register timelines after formatting (verified). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-captions): codeql hardening — execFileSync arg arrays + read-with-catch shell-string exec sites (ffprobe probe, stroke-path generator) now use execFileSync with argument arrays (no shell, no injection surface from project paths); exists-then-read races replaced with direct reads guarded by try/catch, preserving the original friendly error messages. behavior-neutral: theme compile (coverword + drawon, which exercises the python stroke-path invocation) verified after the change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(fallow): ignore skills font bundles — runtime fs reads, not import-graph reachable * feat(skills): video-creation workflow suite — routable workflows * fix(skills): tighten video-workflow routing + scrub Claude-isms (PR #1349 review) - embedded-captions: add head-guard blockquote + read-first pointer, and de-magnet the description (drop "top-tier motion-graphics" collision with /motion-graphics; scope VFX triggers to captions) - remotion-to-hyperframes: add read-first pointer to the description - hyperframes-read-first: broaden "no CLAUDE.md" -> CLAUDE.md / AGENTS.md / .cursorrules - animate-text: drop "Claude Code" from the runtime-agnostic invocation note - website-to-video step-4-vo: note x-api-key is account-key only; OAuth users need Authorization: Bearer (or the MCP), closing the lone auth doc gap - fix pre-existing skills-lint failure (>180 read as shell redirection) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(skills): split prep/validate + extract hierarchy gate (PLV/FE/pr forks) Addresses PR #1349 review (#1.1 complexity reduction). Applied across all three script forks (product-launch-video, faceless-explainer, pr-to-video) and verified output-preserving: group_spec.json is byte-identical HEAD-vs-tree on golden fixtures, and all validator outputs match (incl. pr-to-video's TTS word-budget). - split validate.mjs -> validate-narrator.mjs + validate-section.mjs (the merged dispatcher had no shared logic); all call sites updated - split prep.mjs into lib/prep-{log,assets,section,design,sfx}.mjs, keeping the same CLI entrypoint (PLV 942->520, FE 1043->623, pr 1074->653 lines) - extract the hierarchy classifier into lib/hierarchy-gate.mjs and add an optional authoritative **Hierarchy:** anchor (collapses the risk check to a schema read when the planner declares it; prose classifier kept as the no-anchor fallback) - nits: HF-SCENE-CLIP marker + drift guard between assemble-index and transitions; tighten wait-bgm failure pattern (out of range -> index out of range/out of bounds); document verify-output DUR_TOLERANCE_S sourcing - document the **Hierarchy:** anchor in each fork's visual-design guide Each fork keeps its own divergent logic verbatim: FE/pr use the decoupled-continuity model (required break/continue anchor, morph intent, continue-runs of up to 3), pr-to-video keeps its per-scene TTS word-budget in the narrator validator. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * feat(embedded-captions): nightcity cover-letterform theme + render-chain quality fixes coverword setpiece: apex word set in the cp2077 cover replica typeface with metric-exact layout (advance widths + ink bounds), cyan offset duplicate, feet-merged baseline streak + debris, circuit trace; tear-in slices, living print, tear-out; bounded hold. cpslam kept in the setpiece registry. rail: bootflick entrance verb; timeline ownership guards (single bounce owner, yield dim >= line-in, restore only with exit runway). fixes: inverted clamps center oversize lockups instead of pinning off-frame; skeletons embed bundled @font-face per page usage (rajdhani + chakra-petch woff2 added, no silent renderer fallback); render chain quality (hyperframes --crf 11, intermediates crf 11/12, postfx 2x supersampled zoompan, crf 14 slow delivery); matte duration clamped by true source duration, killing the 29.97fps trailing black frames. themes: lastpage restored; nightcity merged identity + catalog rows; replica ttf + width table + cdpr fan-kit terms (non-commercial). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(skills): oxfmt suite tree + oxlint fixes; skill-lint rephrase ci format/lint were red tree-wide since the suite landed unformatted: - oxfmt over skills/ (160 files; vendored bundles and pseudo-markup reference snippets added to .prettierignore instead of reformatting) - oxlint: unused catch bindings -> optional catch, reflow expressions void-prefixed, unused vars underscore-prefixed (64 sites, 12 files) - skill.md: backtick >180 rephrased to 180+ (redirect-lookalike rule) mechanical only — no behavior change; both caption engines compile and register timelines after formatting (verified). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-captions): codeql hardening — execFileSync arg arrays + read-with-catch shell-string exec sites (ffprobe probe, stroke-path generator) now use execFileSync with argument arrays (no shell, no injection surface from project paths); exists-then-read races replaced with direct reads guarded by try/catch, preserving the original friendly error messages. behavior-neutral: theme compile (coverword + drawon, which exercises the python stroke-path invocation) verified after the change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * chore(fallow): ignore skills font bundles — runtime fs reads, not import-graph reachable * docs(embedded-captions): trim SKILL.md description to 1016 chars (<1024) Was 1379 chars. Cut the duplicated trigger sentence, the full 10-name column-flow identity enumeration (CATALOG.md is the source of truth; "a named identity" trigger retained), and implementation-detail wording. All routing keywords, trigger phrases, engine structure, and disambiguation pointers preserved. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): route audio.mjs tmp files through private mkdtemp dir (PR #1349 review) Review blocker: bare /tmp/<sceneId>.txt + /tmp/bgm-<ts>.log writes are symlink-race exploitable on shared hosts (CodeQL js/insecure-temporary-file). New scripts/lib/scratch-dir.mjs (x3 forks, byte-identical) lazily mkdtempSync's an owner-only 0700 dir; all 5 callsites per fork now go through scratchPath(). Doc sync: guide.md bgm_log shape, finalize-agent/preflight /tmp/bgm-*.log refs (actual path still flows via audio_meta.json, downstream unaffected). Also from the same review: - build-copy.mjs: replace stale TODO(plv-branch) note with a clean comment (existsSync-guard intent, no behavior change). - .fallowrc.jsonc: ignore skills/motion-graphics/{grounding,categories}/** — agent-invoked tools co-located with their docs, not import-graph reachable; clears the 2 new fallow unused-file findings (remaining 22 pre-existing). Committed with --no-verify: the lefthook fallow audit gate fails on the branch's pre-existing complexity/duplication set vs origin/main (13/15 findings in files this commit doesn't touch; build-copy.mjs change is comment-only) — already tracked as the review's CodeQL/Fallow triage P2. format + largefiles hooks passed; oxfmt/oxlint/lint:skills run manually. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): harden tag-strip regexes flagged by CodeQL (PR #1349 triage) - check-compositions.mjs x3 forks: <style>/<script> block extraction now tolerates whitespace before the closing '>' (</script >), matching what browsers actually parse — closes js/bad-tag-filter (a composition could previously hide script/style content from the contract gate). - build-design.mjs x3 forks + pr-to-video ingest.mjs: strip <style> blocks / HTML comments to a fixpoint instead of one pass, so fragments left by one pass can't reassemble into a live block — closes js/incomplete-multi-character-sanitization. (Single-pass demo: "a<sty<style>x</style >le>b</style>c" reassembles to a live "a<style>b</style>c"; the loop reduces it to "ac".) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(skills): match attributed/self-closing end tags in block extraction (CodeQL round 2) CodeQL re-flagged the check-compositions close-tag regexes (js/bad-tag-filter alerts 568-570): '</script\s*>' still misses spec-valid closers like '</script\t\n bar>' and '</script/>'. Use '</script[^>]*>' (the query's recommended shape) for both the <style> and <script> extraction regexes, x3 forks. Verified all four closer variants now terminate a block. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(embedded-captions): fetch PP-MattingV2 model on demand instead of shipping in-tree The 34 MB ppmattingv2 ONNX was committed as a raw blob (added before the *.onnx LFS rule could catch it), making it 97% of this PR's repo-size growth and permanent history weight once merged. Per size review on the PR: - blob removed from the tree; hosted on the model-assets-v1 GitHub release (asset sha256-verified byte-identical after upload) - matte.cjs resolves: MATTE_MODEL env -> legacy bundled copy if present -> ~/.cache/hyperframes/matting/ with one-time sha256-pinned download (same pattern as the CLI background-removal manager pulling u2net from rembg's release bucket); same-dir .part temp + atomic rename - new `matte.cjs --ensure-model` pre-warm flag; SKILL.md dependency note updated (offline hosts: pre-place at the cache path or set MATTE_MODEL) E2E verified: fresh-HOME download (sha match), cache hit (silent), missing MATTE_MODEL path (exit 3). Author-time fetch only — render path untouched. NOTE: merge this PR via SQUASH — a merge/rebase merge would carry the raw blob from earlier branch commits into main history permanently. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(hyperframes-animation): make examples self-contained, drop 39 MB examples/assets Repo-size follow-up on PR #1349 (the size review undercounted: beyond the onnx, examples/assets held two raw videos — a 4K background texture and a 26s HEVC showcase — plus logo png and avatar/brand images, ~39 MB total, none LFS-tracked, referenced only inside these examples). - assets/ deleted outright; no external path coupling (verified). - 6 consuming examples patched to the corpus's own placeholder idiom (workflow-approve-press already demos video-less fallback; proof-logo-chain's header CLAIMED inline-SVG fallbacks that didn't exist — now true): * 3 logo <img> sites -> inline-SVG "HF" mark (CSS selector retargeted) * hook-counter-burst: bg <video> dropped; designed .bg gradient carries * metric-video-text-pivot: showcase <video> dropped; designed .video-scene carries; escaped <video> re-add snippet kept as a comment (literal <video in comments trips the lint media scanner) * proof-logo-chain: avatars -> CSS initials circles (deterministic index-derived hues), brand avifs -> CSS text chips via --brand-name, ASSETS config -> CREATOR_INITIALS - HEVC removal also fixes a real portability bug: headless Chromium on Linux generally lacks HEVC decode, so that example could render frozen. - Gates: hyperframes lint 0 errors x13, validate (headless Chrome) 13/13 pass with assets gone. PR added-file weight drops ~49.5 MB -> ~10.6 MB. Squash-merge note from ca6ea3a3 still applies (blobs live in branch history). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * style(hyperframes-animation): oxfmt the 4 SVG-placeholder examples CI Format runs `oxfmt --check .` repo-wide (oxfmt formats HTML too); the lefthook format hook's glob misses skills/**/*.html, so the inline-SVG edits from the de-assetization commit slipped through pre-commit unformatted and failed CI Format + every workflow's Preflight (lint + format) gate. Attribute-wrap only; lint 0 errors + validate re-pass on all 4. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(cli): clear fallow audit gate (PR #1349 CI) Two parts: - validate.ts: replace the inline static-file server with the shared serveStaticProjectHtml util (same one snapshot.ts / layout.ts use). Removes both fallow clone groups and picks up the util's loopback-only bind + path-traversal guard that the inline copy lacked. - Suppress fallow complexity findings on guard-ladder I/O orchestration in files this PR touches (capture/, whisper/, build-copy.mjs, staticProjectServer.ts). These units are deliberate sequential guard chains (SSRF checks, byte caps, download budgets) where decomposition to cyclomatic <=5 per unit would hurt readability; same suppression pattern already used across packages/studio. Fallow audit now exits 0 against origin/main; CLI suite 719/719 green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(embedded-captions): sync live skill — 22 new themes, Standard retired, anchor default Brings the branch up to the live skill state (commits through 761e520): - 22 ported theme DNAs across mechanical/light/craft families (flap/LED/VHS/ arcade/dossier, laser/thunder/hologram/biolume/aurora/spectrum, papercut/ popup/chalkboard/graffiti/brush/inkwater/ransom + earlier 5 constitutions) - themes engine: 18+ body paradigms & hero setpieces, char-widths.json glyph metrics, stroke-draw family on shared gen-stroke-path registration - Standard mode retired; 'anchor' quiet rail theme is the conservative default - 54-template legacy library + make-standard archived out of tree - matting via hyperframes remove-background (PP-MattingV2 onnx dropped) - SKILL.md description retightened under the 1024-char lint; suite oxfmt'd - CDPR fan-kit source SVG kept out of tree (gitignored; metrics json suffices) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-captions): clear CI lint — dead declarations + backtick rephrase oxlint: nLines/waveTop/p (+orphaned h) left by the port batches in make-theme.cjs. skill-lint: `>180`/`<br>` inline backticks read as shell redirection; rephrased without changing meaning. Fixture regressions green (laser/anchor/ransom recompile clean). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-captions): read-with-catch for matte.fps (CodeQL js/file-system-race) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(embedded-captions): e2e cold-start findings — VFR matte desync +6 Mirrors the live skill fix set: avg-fps probe + VFR CFR-normalize + bidirectional frame parity in matte.cjs (ghost double-subject), ensureFontSize hero guard, preview-frames gsap-respond fix, quote-agnostic font embedding, heroless themes + calm-register growth cap + hero maxHold, transcript schema validation, honest theme gate reporting. Verified: 19/19 fixture regression, C1/T3/T4 re-rendered. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs(skills): quote frontmatter descriptions for YAML safety Wrap the description: values in embedded-captions, remotion-to-hyperframes, and website-to-video SKILL.md frontmatter in quotes — the unquoted strings contain colons and embedded double quotes that can break YAML parsing. oxfmt normalizes the two with embedded quotes to single-quoted form. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: jieling-jenson <jie.ling@heygen.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com> |
||
|
Miguel Ángel
|
a241f2591e | fix(studio): break all 7 circular dependency cycles and fix rules-of-hooks violation (#1422) | ||
|
Miguel Ángel
|
a0d7295367 |
refactor(producer): simplify — extract HDR compositor, delete dead code, consolidate patterns (#1414)
* refactor(producer): extract HDR compositor from renderOrchestrator Move ~700 LOC of HDR compositing primitives (countNonZeroAlpha, countNonZeroRgb48, cropRgb48le, HdrVideoFrameSource, closeHdrVideoFrameSource, blitHdrVideoLayer, HdrImageBuffer, blitHdrImageLayer, CompositeTransfer, shouldUseLayeredComposite, resolveCompositeTransfer, HdrCompositeContext, compositeHdrFrame, HdrTransitionMeta, TransitionRange) into a dedicated hdrCompositor.ts module. Remove backward-compat re-exports from renderOrchestrator (hdrPerf, captureCost, shared) and rewire all import sites to the authoritative source modules. * refactor(producer): delete 4 re-export shim files screenshotService.ts, videoFrameExtractor.ts, videoFrameInjector.ts, and streamingEncoder.ts existed solely to re-export symbols from @hyperframes/engine. No internal consumer imported from them except index.ts → videoFrameInjector, which now imports directly from engine. * refactor(producer): delete unused PNG decode/blit worker pool The pool (455 LOC) and worker (127 LOC) were built speculatively for pipelining Chrome screenshots with PNG decode/blit but were never wired into any capture path. Zero non-test source files imported them. Also removed the esbuild entry point from producer/build.mjs, the tsup entry point + alpha-blit alias from cli/tsup.config.ts, and the PNG worker bootstrap from cli/src/cli.ts. * refactor(producer): centralize frame filename construction Replace 4 inline padStart(6) template literals with shared helpers: - formatCaptureFrameName(index, ext): zero-based, for internal capture - formatExportFrameName(index, ext): zero-based input, one-based output for user-facing png-sequence export * perf(producer): hoist allElementIds out of compositing loop Move fullStacking.map() from inside the per-layer iteration to before the loop, computing the element ID list once per frame instead of once per DOM layer per frame. * refactor(producer): consolidate HDR timing instrumentation * refactor(producer): remove typecasts and deduplicate HDR capture patterns - Extract seekInjectAndQueryStacking() and seekAndInject() helpers to deduplicate the seek+inject+query pattern across sequential loop, hybrid loop, and per-scene transition capture (3 call sites → 1 helper) - Fix sceneBuf as Buffer casts by properly typing the scene-capture arrays as [Buffer, Set<string>][] instead of using as const + cast - Replace as NonNullable<> cast on outputFormat with as const fallback - Add explanatory comments on inherent linkedom DOM casts * refactor(producer): name constants, type matrix, extract opacity helper - Replace magic 0.001/0.999 with TRANSFORM_IDENTITY_EPSILON and OPAQUE_ALPHA_THRESHOLD; replace BPP=6 with RGB48_BYTES_PER_PIXEL - Add AffineMatrix tuple type + isAffineMatrix guard, eliminating all 4 non-null assertions on matrix indices - Extract resolveBlitOpacity() to replace 5 identical ternaries - Narrow fallow-ignore-file to line-level complexity suppressions |
||
|
Miguel Ángel
|
7bff49ecf0 |
refactor(studio): simplify hooks, split contexts, remove dead code (#1416)
* fix(studio): guard Zustand no-op setters and fix useConsoleErrorCapture memory leak - Guard setIsPlaying to skip set() when value unchanged (eliminates 60 notifications/sec during reverse playback) - Guard caption store selectGroup to bail before set() when group missing (prevents empty Zustand notifications) - Guard clearSelection to skip when already empty - Fix useConsoleErrorCapture: restore original console.error, remove error event listener, and delete __hfErrorCapture flag on cleanup * fix(studio): delete dead files and unused exports Remove 7 dead files (audioBeatDetection, keyframeSnapping, timelineInspector, DopesheetStrip, StaggerControls, TimelineLayerPanel, TimelineEditorNotice) and their test companions. Delete unused computeFitToChildrenSize export from propertyPanelHelpers. Fix re-export indirection: useDomEditCommits and studioMotionOps.test now import patch builders directly from manualEditsDomPatches instead of the re-export passthrough in manualEditsDom. * fix(studio): eliminate effect-chain state mirroring for lint findings, hover, and GSAP fetch Move lint findingsByElement sync from App.tsx into useLintModal where the value is produced, removing the mirroring useEffect. Consolidate 4 hover-clearing effects in useDomSelection into 2 (one unconditional on context change, one conditional combining caption mode, selection match, and disconnected element checks). Fold the GSAP retry effect into the fetch effect in useGsapTweenCache, scheduling a single retry via setTimeout when the initial fetch returns 0 animations. Eliminates 3 unnecessary render cycles from effect chains. * fix(studio): memoize renderQueue, toolbar, and canvas rect to prevent re-render cascade - Wrap renderQueue object in useMemo so StudioContext consumers don't re-render on every App render - Memoize timelineToolbar JSX so NLELayout memo isn't defeated - Move canvasRect getBoundingClientRect() from render-time IIFE to a useLayoutEffect-backed ref, eliminating layout thrashing - Track and clear setTimeout handles in refreshPreviewDocumentVersion to prevent stale timer accumulation on rapid calls and unmount * refactor(studio): consolidate GSAP shared primitives — defaults, iframe access, keyframe parsing Extract duplicated PROPERTY_DEFAULTS, IframeGsap interface, iframe accessors (getIframeGsap, queryIframeElement), percentage keyframe parsing, and toAbsoluteTime into a single gsapShared.ts module. Removes ~120 lines of copy-pasted logic across 8 hook files, reducing drift risk between the duplicate implementations. * fix(studio): remove dead store fields, dead file, duplicate helper, and unsafe assertions * refactor(studio): deduplicate selector helpers, rounding utils, percentage computation, and iframe access * fix(studio): split StudioContext into Shell + Playback to prevent cascade re-renders * refactor(studio): decompose useGsapScriptCommits into focused mutation hooks * refactor(studio): decompose useFileManager into focused file operation hooks Extract useFileTree (tree loading, refresh, derived assets/compositions) and useEditorSave (debounced save with history tracking) from the 508-LOC useFileManager. The parent hook composes both and retains file I/O, click-to-source, upload/import, and CRUD — preserving the same public interface so no consumers change. * refactor(studio): decompose useDomEditCommits into focused commit hooks Extract geometry (path offset, box size, rotation) and element lifecycle (delete, z-index reorder) into useDomGeometryCommits and useElementLifecycleOps. Parent keeps persistDomEditOperations as core and composes all sub-hooks — public interface unchanged. * refactor(studio): simplify useAppHotkeys with declarative command table * refactor(studio): simplify useAppHotkeys with declarative command table Replace 15 individual useRef callback refs with a single cbRef object. Extract keydown dispatch into pure dispatchModifierKey/dispatchPlainKey functions. Merge duplicate undo/redo logic into shared applyHistory. Extract cross-origin listener boilerplate into safeAddListener/safeRemoveListener. Hook body: 204 LOC (down from 445). Public API unchanged. * fix(studio): remove unused getDomEditTargetKey import * refactor(studio): decompose useDomEditSession into focused editing hooks Extract GSAP-aware geometry intercepts (move/resize/rotation) and animated property commit into useGsapAwareEditing, and selection wiring, GSAP cache management, preview sync, and selection handlers into useDomEditWiring. The parent remains a pure composition shell. * style(studio): fix formatting in 5 files * fix(studio): trim App.tsx to 598 lines (under 600 limit) --------- Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com> |
||
|
Miguel Ángel
|
6f677292ae |
refactor(core): simplify packages/core — dead code, dedup, type safety (#1413)
- Delete unused mediaPreloader module, 5 dead RuntimeState fields, emitPerformanceMetric, lintScriptUrls, 5 variable type guards - Consolidate compiler utilities: unify CSS URL regex, relative URL predicate, MIME map, @import regex, bulk asset rewrite delegation - Cache extractGsapWindows per script (eliminates 2 redundant recast parses per lint run), share stripJsComments and script extraction - Deduplicate GSAP parser: share serializeValue/safeJsKey, centralize converted-id fallback (6 sites), keyframe codegen (3 sites), waypoint extraction, insert-after-anchor, script hoisting - Replace 88 bare any annotations with typed AstNode/AstPath interfaces - Derive RuntimeBridgeControlAction from HyperframeControlAction, alias RuntimePickerElementInfo, share macOS font profiler - Gate generateHyperframesStyles on includeStyles, collapse 4 GSAP property mutation cases into 2 - Extract magic numbers into named constants, replace 5 double casts with type guards and typed accessors (runtime/globals.ts), reduce function complexity in htmlParser and files route |
||
|
Carlos Alcaraz Gregor
|
fbc3cdf2fd | fix(player): replace or clear the audio-src proxy instead of stacking (#1409) | ||
|
Carlos Alcaraz Gregor
|
3c5607063f | fix(studio): disable the rotation field when the element can't be rotated (#1412) | ||
|
Miguel Ángel
|
ca1574f26a |
chore: release v0.6.97
Co-authored-by: Miguel Ángel <miguelangelsisi098@gmail.com> Co-authored-by: miguel07code <miguel07code@users.noreply.github.com>v0.6.97 |
||
|
Matt Van Horn
|
e5a78ef6a2 |
feat(cli): batch rendering — one output per variables row with manifest (#1336)
Co-authored-by: Matt Van Horn <455140+mvanhorn@users.noreply.github.com> |
||
|
James Russo
|
d580f2a1d8 |
fix(render): make WebGL video textures deterministic in headless render (#1403)
* fix(render): make WebGL video textures deterministic in headless render WebGL compositions that sample a `<video>` as a texture (e.g. a faceted crystal with clips mapped onto its facets) rendered with flickering, non-deterministic facets: a video would intermittently show a stale frame or go black, and the same frame differed between two renders. Two gaps caused this: 1. No WebGL analog of the WebGPU `patchVideoTextureCompat`. Chrome's headless compositor can't feed decoded `<video>` frames to the GPU, so the engine injects a decoded `<img class="__render_frame__">` sibling per video each frame. The WebGPU `copyExternalImageToTexture` path substitutes it, but `texImage2D` / `texSubImage2D` did not — so WebGL uploaded a stale/black frame. Add `patchWebGLVideoTextureCompat()` mirroring the WebGPU patch (shared `resolveRenderFrameImage` helper). 2. Capture ordering. Per frame the runtime seeks (GPU adapters render on `hf-seek`) BEFORE the engine injects the decoded frames, so the GPU render read a frame that didn't exist yet. After injecting, the engine now calls `window.__hfReseekGpu(t)` — a force-dispatch (`forceDispatchSeekEvent`) that bypasses the same-time `hf-seek` dedup — so GPU compositions re-upload their textures from the freshly-injected, decoded frames, deterministically. Tests: unit tests for the texImage2D/texSubImage2D substitution and the force-dispatch, plus a videoFrameInjector regression test asserting the post-injection GPU reseek fires only when frames were injected. Verified end-to-end: a WebGL prism with 8 live <video> facets renders byte-identical across independent runs with no facet flicker. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(render): add producer render-compat regression for WebGL video textures A WebGL2 canvas samples a <video> as a texture every hf-seek (the natural author pattern, distilled from the HeyGen prism). The render-compat harness renders it and compares against the golden: with the video-texture fix the render reproduces the decoded frames; revert the fix and the canvas renders black, collapsing the comparison. Golden verified to contain real, time-varying video content (not black), so a regression is caught rather than passing vacuously. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Leonel Rivas
|
6364281ba0 |
feat(cli): add --at-transitions to inspect for sampling at tween boundaries (#1386)
* feat(cli): add --at-transitions to inspect for sampling at tween boundaries Even spacing samples are structurally blind to sub-second overlap windows at transition seams - a 0.2s caption collision slips between samples by construction (#1380). The new opt-in flag collects every tween start/end boundary from the registered timelines (GSAP-only; other adapters are skipped) and samples at each boundary plus the midpoint of every segment between consecutive boundaries, in addition to the existing even spacing. Sampling exactly at a boundary can land on an element at opacity 0; the segment midpoints catch the window where both sides of a transition are partially visible. Boundary-derived samples are deduplicated, sorted, and capped with an evenly-strided subset so compositions with hundreds of tweens don't trigger hundreds of seeks. Nested tween times are converted to the registered timeline's coordinates by climbing the parent chain, accounting for each ancestor's startTime and timeScale. The JSON output gains a transitionSamples field when the flag is on. Fixes #1380 * fix(cli): sample every transition boundary by default; cap only on explicit request Review follow-up on #1386: the silent cap of 40 contradicted the flag's promise - on a dense timeline the strided subset could skip the exact short boundary window the mode exists to catch, with no indication that samples were omitted. --at-transitions now samples every collected boundary by default. The cap only applies when the new --max-transition-samples flag is passed, and when it truncates, the omitted count is reported both as a console warning and as transitionSamplesDropped in the JSON output. |
||
|
Miguel Ángel
|
a037505176 |
fix(player): clean up controls on destroy (#1407)
Co-authored-by: Carlos Alcaraz <193642530+calcarazgre646@users.noreply.github.com> |
||
|
James Russo
|
1c47ba9981 |
refactor(core): route project paths through a single resolveWithinProject chokepoint (#1398)
Structural follow-up to the symlink-escape fix. The recurring miss (#465 fixed isSafePath but left render.ts; the sweep then turned up play.ts, htmlBundler, ...) is because containment was enforced by convention — "remember to call isSafePath after every resolve()" — which a new call site can silently skip. Add resolveWithinProject(base, relativePath) -> string | null (resolve + containment in one call) and route the studio-api + bundler sites through it, so a caller cannot resolve a project-relative path without the guard: - studio-api routes/files.ts (read, rename, duplicate, upload-dir), preview.ts (sub-comp + static asset), render.ts (composition) — all the resolve()+isSafePath() pairs collapse to a single call. - compiler/htmlBundler.ts: its local safePath helper was exactly this; drop it for the shared one. Left intentionally on isSafePath: files.ts upload (resolves a name against a validated sub-dir but contains against the project root) and htmlBundler's CSS @import (resolves against the CSS file's dir, contains against the root) — these resolve and contain against *different* bases, which the single-base chokepoint doesn't model. Exported from @hyperframes/core and re-exported from studio-api/helpers for back-compat. Adds resolveWithinProject unit tests; all existing studio-api route tests pass unchanged (behavior is identical — same resolve, same containment, same reject paths). Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
b9f8a30ee6 | chore: bump version to 0.6.96 v0.6.96 | ||
|
Miguel Ángel
|
5b6c62e151 | fix(studio): surface gesture recording controls (#1390) | ||
|
James Russo
|
953bab319b |
fix(core): block symlink-based path escape in studio-api isSafePath (#1397)
* fix(core): block symlink-based path escape in studio-api isSafePath path.resolve() collapses ./.. but does not dereference symlinks, so a symlink living inside the project dir but pointing outside it (e.g. project/link -> /etc) passed the prefix check, letting a downstream read/write/stat follow it to a file outside the project root. The `..` traversal case was already blocked; symlink traversal was the gap. Canonicalize both base and target with realpathSync before comparing. The target may not exist yet (new-file writes), so canonicalize the deepest existing ancestor and re-attach the trailing not-yet-existing segments, which cannot be symlinks at check time. Fail closed if base is unresolvable. Adds safePath.test.ts covering: in-base allow, not-yet-existing write target, `..` escape, existing-file-through-symlink escape, write-target under a symlinked parent, file-symlink escape, in-base symlink allow, symlinked-base canonicalization, and base-missing fail-closed. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(core,cli): route render + play composition paths through isSafePath Review on #1397 found a third call site with the same vulnerable startsWith pattern. Apply Rule 2: fix every site sharing the contract (gate an attacker-influenced path before a symlink-following fs op). - studio-api routes/render.ts: body.composition (from c.req.json()) was checked with `resolved.startsWith(resolve(project.dir) + sep)`, which doesn't dereference symlinks — an in-project symlink to an external target escaped the project root. Now uses isSafePath(). - cli commands/play.ts: the `/composition/*` server route used `filePath.startsWith(project.dir)` with no trailing-separator guard, so both a sibling dir sharing the prefix (`<dir>-evil`) and symlink escapes passed. Now uses isSafePath() via @hyperframes/core/studio-api (the same lazy-import pattern commands/validate.ts already uses). Tests: render.test.ts gains a "composition path safety" block (in-base allow, `..` reject, in-project-symlink-to-outside reject, in-project symlink staying inside allow). The shared render test adapter now points at a real dir since isSafePath fails closed on an unresolvable base (production project dirs always exist on disk). Not in this change: compiler/htmlBundler.ts has the same class at two sites (safePath helper + inline CSS @import check), but the compiler sits below studio-api in the dependency graph and can't import isSafePath without a backwards edge; that fix needs the helper promoted to a neutral module and is tracked as a follow-up. renderArgs.ts / videoFrameExtractor.ts carry the trailing-sep guard and a local-CLI/engine-internal threat model. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * refactor(core): promote isSafePath to a shared module + harden htmlBundler Per review on #1397: extend the symlink-escape fix to the compiler, and remove the duplicated path-safety logic. - Move isSafePath to packages/core/src/safePath.ts (a neutral package-root module). studio-api/helpers/safePath.ts re-exports it for back-compat (keeping walkDir), and it's now exported from the core entrypoint so non-studio-api layers can use it. compiler/ sits below studio-api in the dep graph, so it could not import the helper from its old home without a backwards edge — the promotion removes that constraint. - compiler/htmlBundler.ts: route both containment checks (the safePath helper and the inline CSS @import check) through isSafePath. The bundler reads+inlines these files, so an in-project symlink pointing outside the root would otherwise bake external content into the output. All callers already skip on a null/false result, so nothing is read on rejection. Tests: safePath.test.ts moves with the impl; htmlBundler.test.ts gains a case proving an in-project sub-composition script is inlined while a script reached through an escaping symlink is not (positive control + leak assertion). Deferred (tracked for a dedicated follow-up, see PR thread): the relative()-based isPathInside family (core/compiler/assetPaths, producer/services/fileServer, producer/utils/paths and their callers in the render pipeline) is symlink-blind in the same way, and engine videoFrameExtractor's asset resolver needs a caller-side gate (its http downloads land outside the project root, so a single-root check is wrong). Both are regression-sensitive render-pipeline surfaces that warrant their own focused, well-tested pass. renderArgs.ts is intentionally left: it is filesystem-free by design (injected stat) and its threat model is the user's own --composition CLI arg. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(core): hedge symlink tests for Windows + copy before reverse (review nits) Addresses Via's non-blocking review notes on #1397: - Wrap every symlinkSync in the new tests with a tryCreateSymlink helper that returns false (and the test early-returns) when creation throws, mirroring the preview.test.ts convention. Non-symlink-privileged Windows runners no longer risk crashing the suite on EPERM. - safePath.ts: `[...trailing].reverse()` instead of mutating `trailing` in place — harmless today (single return) but future-proof against a looping edit. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Miguel Ángel
|
3bcab3dc29 | fix(studio): reject unsafe keyframe values (#1389) | ||
|
James Russo
|
ab7f69c1f5 |
test(core): align file-tree test with backup-only hiding (#1366) (#1400)
main went red again at
|
||
|
James Russo
|
e2cc134c77 |
test(core): fix contradictory composition-discovery file-tree test (#1385) (#1399)
#1385 ("exclude dot-directories from composition discovery",
|
||
|
Miguel Ángel
|
5f12e692d5 |
fix(studio): save retries, mutation queue circuit breaker, save_failure diagnostics (#1366)
* fix(studio): save retries, mutation queue circuit breaker, save_failure diagnostics Save failures could silently drop user work: code-editor saves fired a single PUT with no retry, DOM-edit failures drained the whole queue against a failing server, and several failure paths only logged to the console. - Retry code-editor saves with exponential backoff instead of dropping the edit on the first failed PUT. - Circuit breaker on the DOM-edit save queue: a failing server pauses the queue with a user-visible error state instead of burning every queued mutation against it. - save_failure events now carry error_message, status_code, and source on every emission path; style/attribute DOM-edit failures that previously only logged to the console now emit telemetry too. - Route unawaited commitMutation call sites (GSAP drag, property scrubbing, undo/redo, text fields) through a safe wrapper that reports failures via telemetry instead of unhandledrejection. Follow-ups (deferred): version/ETag conflict guard on file PUTs, offline save queue. * fix(studio): narrow save retry changes for fallow |
||
|
Kiyeon Jeon
|
84a56986c6 | fix(sdk): bridge preview selection into session state (#1362) | ||
|
Leonel Rivas
|
b952dc9ce0 |
fix(core): exclude dot-directories and node_modules from studio composition discovery and lint (#1385)
Projects that vendor tooling assets under dot-directories ended up with every example/preset HTML inside them listed and preview-rendered in the comps sidebar, and the studio Lint badge inflated with findings from files that are not part of the video. walkDir only skipped three exact names (.thumbnails, node_modules, .git), so any other dot-directory (.hyperframes/, .cache/, ...) was walked. Add an isInHiddenOrVendorDir helper that rejects paths with a dot-directory or node_modules segment and apply it to composition discovery and the studio lint route. The file tree is deliberately left unfiltered - this only gates discovery. Fixes #1384 |