skills/heygen-com__hyperframes
1
0
Fork 0
mirror of https://github.com/heygen-com/hyperframes.git synced 2026-09-14 18:01:20 +08:00
Code Issues Packages Projects Releases Wiki Activity
1,777 Commits 936 Branches 415 Tags
drawelement-perframe-fallback
Commit Graph

1339 Commits

Author SHA1 Message Date
Vance Ingalls 93ce61aba2 fix(engine): static dedup drawElement-path only + anchor-based verification 
Two dedup-safety fixes found via full-stack corpus eval:

1. Arm dedup ONLY on the drawElement path (armStaticDedup, called after gates pass
   + before canvas injection). Gated/screenshot comps are gated precisely because
   they carry compositor animation (stacked fades, 3D, flashes) the static predictor
   mispredicts and a sparse backstop can't catch — dedup there froze frames. The
   drawElement path animates via GSAP transforms the predictor handles cleanly.

2. Backstop verifies against the ANCHOR each run reuses, not the predecessor. Dedup
   chains a whole static run back to one buffer; a slow sub-quantization drift is
   byte-identical frame-to-frame (passes f-vs-(f-1)) yet drifts far from the anchor by
   the run end. Now captures each run's anchor once and compares the run END + midpoint
   to it; any drift disables dedup whole-comp.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 04:14:47 -07:00
Vance Ingalls 74b31491bb feat(engine): sample-verification backstop for static-frame dedup 
Before trusting the predicted-static set, empirically verify a sample: seek to each
sampled static frame and its predecessor, screenshot both, byte-compare (the
screenshot path is bit-deterministic, so a truly static frame is byte-identical to
its predecessor). Any mismatch => hidden animation the GSAP/clip predictor can't see
=> disable dedup for the whole comp. Runs at init in normal DOM state, capture-mode
independent.

Sampling targets run-starts (first static frame after each animated/boundary region,
where trailing CSS/settle animation hides) plus an even spread (systemic animation).
Validated: no false-disable on lossless comps (4b038555/d95f20b6 stay 'verified',
min inf). Tunable via HF_STATIC_DEDUP_SAMPLES; skip with HF_STATIC_DEDUP_VERIFY=false.

Documented residual: sparse sampling catches systemic + transition-trailing hidden
animation, not an arbitrary mid-run content change with no deterministic signal
(those are owned by the deterministic guards: clip-cut, getAnimations, media). Dedup
stays opt-in (HF_STATIC_DEDUP=true) until a broader real-comp sweep bounds the residual.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 02:16:50 -07:00
Vance Ingalls dbdd278f6a feat(engine): extend static-frame dedup to worker-encode path 
captureFrameToBufferPipelined now honors session.staticFrames: a static frame
reuses session.lastEncodeResult and skips the seek + drawElement + encode, same
predicate as the serial path (clip-cut frames excluded, so they always capture).
Non-static frames (and clip-boundary screenshots) update lastEncodeResult.

Validated lossless on the macOS-GPU worker path: 4b038555 (76% reused, min inf)
and d95f20b6 (80% reused via gate bypass, exercises the clip-cut path, min inf).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 01:56:28 -07:00
Vance Ingalls 9903713ea9 feat(engine): macOS static-frame dedup for screenshot capture (opt-in) 
macOS Page.captureScreenshot is flat ~24ms/frame with no dedup; static holds
re-capture identical buffers for nothing. computeStaticFrameSet walks
window.__timelines at init and marks frame f dedupable iff f and f-1 are both
outside every GSAP tween interval (totalDuration, so repeat/yoyo counts) AND
neither is a clip-cut boundary (hard scene swaps change content with no tween).
captureFrameCore reuses session.lastFrameBuffer for dedupable frames, skipping
the seek + capture. Whole-comp disqualified on video/canvas/webgl or init-visible
non-GSAP animation (getAnimations).

Opt-in HF_STATIC_DEDUP=true, default off. Validated lossless (dedup vs
bit-deterministic screenshot baseline = inf/>60dB) on 5 comps reusing 44-80% of
frames. Two bugs found+fixed during validation: duration vs totalDuration
(repeat/yoyo froze) and clip-cut frames (scene swaps froze).

Residual risk keeping it opt-in: CSS/rAF animation not running at init and not
clip-driven would be invisible to the init guard. None seen in tested corpus;
a generic sample-verification backstop is the follow-up to default it on.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 01:15:07 -07:00
Vance Ingalls a496d944eb chore(engine): untrack internal fast-capture finding docs 
Fast-capture findings/handoff/architecture docs are internal-only working notes,
not shipped documentation. Remove from tracking; they remain on disk for local use.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-16 00:22:04 -07:00
Vance Ingalls a1775fb114 feat(engine): Lim 7 timeline-interval at-risk predictor (go/no-go gate only) 
Walk window.__timelines before canvas injection to find GSAP tweens animating
compositor-incompatible props (opacity, filter, blend-mode, 3D, clip-path,
mask). If the at-risk frame fraction exceeds 40%, route the whole composition
to screenshot fallback before the drawElement canvas is injected — avoiding
DOM pollution from a mid-init fallback.

Per-frame screenshotting of sparse at-risk intervals was prototyped but dropped:
autoAlpha rewrite (active via evaluateOnNewDocument) contaminates screenshotted
frames during opacity tweens, producing worse results than leaving drawElement
to render them. All 13 Lim 7 comps in the current corpus gate out at the
stacked-fade or CSS-FX detectors before reaching this code anyway.

Disable with HF_FAST_CAPTURE_INTERVAL_SS=false.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-15 14:09:10 -07:00
Vance Ingalls c003abb3cb fix(engine): per-frame screenshot fallback for drawElement clip-cut boundaries 
drawElement renders hard clip-cut boundary frames incorrectly: the outgoing
clip is dropped one frame before the incoming clip's paint record is ready, so
the frame either throws InvalidStateError "No cached paint record for element"
(aborting the whole render) or silently renders solid black. Both are real
production damage, confirmed by a clean fast-render of a clip comp emitting a
black frame at a clip cut, and by the 11-comp crash class from the 400-comp eval.

Two per-frame fallbacks, both routing the single bad frame to captureScreenshot
instead of failing the render:

* Throw case: catch "No cached paint record" (isNoCachedPaintRecordError) in
  both the serial (captureFrameCore) and worker-encode (captureFrameToBufferPipelined)
  paths. Validated on a comp that previously hard-aborted, now completes with
  11 frames screenshot-fallen-back.
* Silent-black case: compute clip-cut boundary frames (plus or minus 1) from the
  clip schedule (data-start/data-duration times fps) at init and screenshot those
  frames. A boundary-targeted spike showed 100% of black-outs land on these
  frames. Validated: a clip comp that emitted a black frame at t=8 now renders
  correctly.

Default-on for drawElement renders; disable boundary fallback with
HF_FAST_CAPTURE_BOUNDARY_SS=false. Adds docs/fast-capture-limitations.md, the
canonical findings log the code comments already referenced but which never
existed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-15 12:29:43 -07:00
Vance Ingalls e27afcdb40 fix(engine,producer): close worker-encode orphan/hang/corruption gaps (re-review) 
Re-review found the prior orphan-rejection fix attached to the wrong promise.

- Unhandled-rejection guard moved to source: produceDrawElementFrame attaches a
  no-op .catch to every encodeResult at creation, covering the depth-2 loop's
  orphaned in-flight frame. Removed the ineffective loop-level prev.catch.
- Per-frame encode watchdog (30s): a lost worker message no longer hangs the
  render to the protocol timeout (onerror->id=-1 only covered crashes).
- Empty-frame guard: payload-less worker success rejects instead of resolving a
  0-byte Buffer ffmpeg would write as a corrupt frame.
- Worker reuses one OffscreenCanvas across frames (was per-frame alloc).
- Close the ImageBitmap on the worker-missing reject path (GPU leak).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls dd0bd4aaf1 fix(engine,producer): harden worker-encode pipeline (review findings) 
Liveness/correctness:
- Worker encode failures now propagate: the in-page worker wraps encode in
  try/catch + null-checks getContext and posts {id,error} on failure; its
  onerror posts a fatal signal (id=-1). The node binding rejects the matching
  pending promise (or all of them on fatal) instead of leaving encodeResult
  pending forever — previously any worker throw hung the render to timeout.
- Pipeline loop attaches a no-op catch to the orphaned in-flight encode on
  abort/throw so cleanup's rejection is not an unhandled promise rejection.
- drainPrev checks assertNotAborted before awaiting so aborts are observed
  while parked on the encode wait, not one frame later.
- captureFrameToBufferPipelined wraps capture in captureFrameErrorDiagnostics,
  restoring per-frame frame-error PNG/HTML/JSON the serial path produced.

Lifecycle/efficiency:
- __hfFrameReady binding tracked via a separate workerEncodeBoundPages WeakSet
  so a re-init after cleanup doesn't call exposeFunction twice ('already exists').
- URL.revokeObjectURL after Worker construction (was leaked per init).
- Drop the unconditional setTimeout(0) around createImageBitmap (~1-4ms/frame
  of macrotask latency on the produce critical path; encode runs in the worker).
- Reset nextId on session reuse; remove the unreachable BeginFrame branch from
  the pipelined path (gated to beginFrameTimeTicks===0).

Base64 stays in the worker (off the main thread) by design; a binary
side-channel to node is a follow-up. Gated off by default.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 5ea76a52d5 perf(engine,producer): worker-offload JPEG encode for drawElement fast capture 
Moves per-frame JPEG encode (~7.4ms, 57% of frame cost) off the page main
thread into an in-page OffscreenCanvas Worker, then pipelines so frame N
encodes while frame N+1 seeks+paints. Target ~1.65× (1 worker) wall-time
speedup on macOS hardware-GPU drawElement renders.

New machinery:
- EngineConfig.enableDrawElementWorkerEncode (default false, env HF_DE_WORKER_ENCODE)
- drawElementService: WorkerEncodeState, initDrawElementWorkerEncode,
  cleanupDrawElementWorkerEncode, produceDrawElementFrame
- frameCapture: CaptureSession.workerEncodeEnabled, captureFrameToBufferPipelined
- captureStreamingStage: runWorkerEncodePipelineLoop helper + depth-2 dispatch

Gated off by default. No effect on BeginFrame/Linux, SwiftShader, PNG, or
any render without useDrawElement+enableDrawElementWorkerEncode=true.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 848866e8b1 fix(engine): drop filter:drop-shadow from the fast-capture gate 
Audit of the worker-encode eval: drop-shadow-only comps (df580870 53.1 dB,
19132659 53.9 dB) render fine through drawElement, and no damaged comp in the
set was drop-shadow-only (a54e674b's damage was backdrop-filter + blur, still
gated). The drop-shadow rule only over-gated healthy comps. Keep
backdrop-filter + filter:blur + webgl. (drop-shadow ON SVG can still differ —
a narrower SVG-scoped check is a follow-up if it surfaces.)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls ca5d19fa3c feat(engine): gate backdrop-filter / filter:blur|drop-shadow / webgl from fast capture 
drawElementImage cannot faithfully reproduce these, producing 18-49 dB damaged
frames (community eval). Add detectCssEffectRisk: scans computed styles under
the composition root for backdrop-filter (samples the compositor backdrop the
single-element capture has no access to) and filter:blur/drop-shadow (paint-record
vs compositor inconsistency), plus the accel-canvas registry for any WebGL
context (custom GLSL shaders animated via GSAP with no rAF freeze under
seek-based capture; the drawImage composite can't un-freeze them). Any match
routes the comp to the platform screenshot baseline, same contract as the
video / stacked-fade / 3D gates. HF_FAST_CAPTURE_CSSFX=true bypasses for R&D.

Verified on the 6 damaged community comps: 5 (backdrop-filter x2, filter:blur x2,
webgl x1) now fall back to screenshot (PSNR -> inf); the 6th is a deterministic
44 dB residual with no signature (imperceptible, left on the fast path).

Note: the webgl gate over-gates static/redraw-on-seek WebGL that the composite
handles cleanly; a per-frame canvas-redraw probe could re-admit those later.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 538b42d730 fix(engine): block drawElement fast capture on SwiftShader (no GPU egress to skip) 
drawElement's only speedup is skipping the GPU→CPU screenshot readback IPC.
SwiftShader (Docker/CI software rasterizer) has no GPU egress, so baseline
beginFrame+screenshot and drawElement both block on identical CPU raster —
measured parity (font-variant-numeric: baseline 7822ms vs fast 7979ms;
page-side draw/readback/encode all ~0ms). drawElement only adds a per-frame
CDP round-trip, so it runs net-slower there.

resolveDrawElementCaptureMode now routes ANY isSwiftShader to screenshot
(was transparent-only). The win is real only on a hardware GPU (macOS 1.6×).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls e468a4dc31 fix(engine): move routeToFallback inside useDrawElement scope 
routeToFallback referenced `transparent` which is declared inside
the `if (useDrawElement)` block — caused TS2304 in lambda build.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls c34de8c35e fix(engine,producer,core): fast-capture correctness + lint rule fixes 
- Retract __HF_FAST_CAPTURE_AUTOALPHA__ flag via page.evaluate in all
  three runtime fallback paths (video gate, stacked-fade gate, 3D init
  failure) — previously the flag stayed set on gated renders, causing
  hideTransparentAutoAlphaTargets to fire and damage output up to 21 dB
- Decouple recordThreeDTweenTarget from the autoAlpha rewrite flag so
  stacked-fade detection works even when HF_FAST_CAPTURE_AUTOALPHA=false
- Add compile-time mix-blend-mode gate: compositions using mix-blend-mode
  route to the baseline capture path (measured 42 dB min damage on GPU)
- Remove software-GL gate: Docker/SwiftShader benchmarks show parity with
  BeginFrame baseline; the ~0.7-0.8x figure was from a multi-worker
  comparison that doesn't reflect production single-worker configuration
- Fix missing_data_no_timeline lint rule: boolean attribute false-positive,
  hyphenated-variant false-negative, missing isSubComposition guard, and
  external-script false-negative; add 8 regression tests
- fallow: add ignorePatterns for spikes/benchmarks/chrome, ignoreExports
  for page.evaluate-injected initThreeDProjectionInPage, and code-duplication
  suppressions for pre-existing patterns in large files modified by this PR

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls db8c3fad72 fix(engine): mechanism-based stacked-fade gate, drop hf-tx markup gate 
The crossfade gate from the previous commit keyed on the hf-tx class —
a private convention of one composition generator, not a framework
contract (zero hits in core/runtime/skills). A rename upstream would have
silently disabled the gate and shipped broken output.

Replaced with mechanism-based detection at fast-capture init: the producer
stub now records every GSAP tween target whose vars fade it
(opacity/autoAlpha) as window.__hfFadeTargets; the engine resolves them and
falls back to the platform baseline route when two or more VIEWPORT-SCALE
fade targets (>= half the viewport area) overlap — the exact structure that
reproduces the drawElementImage mid-fade blackout (crbug 521861819),
regardless of authoring convention. Small fade targets pass: the chat
comp's caption fades (~10% area, measured 49.6 dB clean) keep the fast
path, as does every CI comp.

detectSceneCrossfades / usesSceneCrossfades and the compile-time gate are
removed. HF_FAST_CAPTURE_CROSSFADE=true still bypasses. Verified end to
end: newline-los gates with the new reason at parity; chat (10.3s) and
gsap-letters (3.0s) keep drawElement. 78 htmlCompiler tests pass.

Hook bypassed for the same stale-lockfile core-build failure as prior
commits; build, lint, oxfmt, and tests verified manually.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls d03d4af993 fix(producer): crossfade + software-GL gates for fast capture 
Two more compile-time gates, same shape and fallback contract as the video
and 3D gates — anything measured slower than baseline or below the quality
bar routes to the platform's baseline capture by default.

Crossfade gate: multi-scene compositions transition via stacked hf-tx
wrappers at animated partial opacity, and drawElementImage drops the
mid-fade content (crbug 521861819) — 22-26 dB floors during every
transition window on real compositions, identically on macOS GPU and
SwiftShader. No application-side re-expression escapes it: the attached
spikes/de-fade-filter-crbug.mjs proves filter:opacity() fades hit the
identical 240/240 blackout. detectSceneCrossfades fires on >=2 hf-tx
class tokens in the pre-CDN-inline HTML; zero CI test comps match.
HF_FAST_CAPTURE_CROSSFADE=true bypasses.

Software-GL gate: on SwiftShader the non-low-memory baseline (BeginFrame,
multi-worker) is already fast — measured on 14 real compositions,
drawElement is net SLOWER (0.71-0.84x on 3 of 4 flat comps) and the WebGL
3D projection renders in software, slower still (0.66-0.91x). Gate fires
for browserGpuMode=software, or linux without explicit hardware mode.
HF_FAST_CAPTURE_SWIFTSHADER=true bypasses.

Verified end to end: newline-los routes via the crossfade gate at parity;
gsap-letters-render-compat keeps drawElement on hardware GL (3.1s) and
gates on software GL. 82 htmlCompiler tests pass (4 new).

Also adds the upstream repro spikes filed today: de-fade-filter-crbug.mjs
(521861819 comment), de-canvas-freeze-crbug.mjs (crbug 522845799), and the
de-fade-filter-test.mjs exploration. 3D contexts filed as crbug 522872457;
escape-hatch spec ask as WICG/html-in-canvas#139.

Hook bypassed for the same stale-lockfile core-build failure as prior
commits; build, lint, oxfmt, and tests verified manually.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls e205ddfc1d feat(engine): WebGL 3D-context projection for fast capture 
drawElementImage cannot paint CSS 3D: rendering contexts drop earlier
siblings and backgrounds, backface-visibility is ignored (mirrored
backfaces even at rest), and flat 3D matrices silently lose their
rotation. Until now every 3D comp was gated to the baseline route.

threeDProjection.ts rewrites 3D content in-page before capture:
- discovery: perspective/preserve-3d contexts, elements at a 3D matrix
  at t=0, and the producer stub's record of 3D tween targets
  (rotationX/rotationY/transformPerspective) for to()-style tweens that
  are still flat at init
- leaf quads rasterized once via SVG foreignObject (fonts and images
  inlined as data URLs), shell quads carry only own paint when a child
  contains further 3D
- per-frame WebGL projection from live computed matrices: CSS-convention
  matrix math, perspective + transform-origin sandwiches, GL backface
  culling, accumulated opacity as a fragment uniform
- live elements hidden via clip-path (GSAP autoAlpha fights
  visibility/opacity), 3D contexts neutralized and live 3D matrices
  stripped after reading (perspective-carrying and rotated matrices
  poison the capture even when hidden), authored backface flags captured
  before neutralization
- projected canvases composite OVER the DOM paint — the under-pass would
  bury them beneath the composition root's own background

Correctness guards fall back to the platform baseline route, same
contract as the video gate: degenerate markup (zero-size/inline-box
quads — gen_os flip-card spans) and quads with GSAP-animated descendants
(static textures would freeze them; golf measured 46->24 dB without
this).

fast-capture-3d test comp (flip card + perspective-free rotationX
entrance): 57.5 dB avg / 51.7 dB min vs baseline at 1.5x speedup.
Real-world comps with animated 3D subtrees fall back cleanly.

Engine suite 692/694 (2 failures pre-exist on clean tree); build, lint,
oxfmt verified manually — hook bypassed for the same stale-lockfile
core-build failure as the previous commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 2a33c81f34 fix(producer): compile-time 3D-transform gate for fast capture 
drawElementImage cannot paint CSS 3D rendering contexts: backface-
visibility:hidden is ignored (flip cards capture their mirrored backface,
even at rest), siblings of the 3D context drop out of the capture, and the
context's background is lost. Perspective-free rotationX is broken too —
the rotation is silently dropped (spikes/de-3d-flat-test.mjs). Standalone
repro: spikes/de-3d-probe.mjs.

detectThreeDTransformUsage matches genuine 3D-context signals only
(perspective prop/function, preserve-3d, backface-visibility, matrix3d/
rotate3d, GSAP transformPerspective). Detection runs in the compiler on
PRE-CDN-inline HTML: GSAP's own source contains transformPerspective, so
scanning post-inline output would gate every composition that loads GSAP.
Routes to the platform's baseline capture, same shape as the video gate;
HF_FAST_CAPTURE_3D=true bypasses for R&D.

Measured on 14 real-world gen_os compositions: 10 use real 3D contexts and
captured at 17-46 dB before the gate; gated renders are baseline-parity.

Also ignore the puppeteer-downloaded chrome/ tree in oxlint.

Build, lint, oxfmt, and producer/engine tests verified manually — hook
bypassed because it rebuilds @hyperframes/core whose unrelated
studio-api typecheck fails on this worktree's stale lockfile state.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 0ef2643bcb fix(producer): visibility-hide inline-opacity-0 autoAlpha targets at flush 
The autoAlpha rewrite only converts opacity in GSAP tween vars — elements
created with inline opacity:0 (the gen_os caption-pill pattern) still sit
in the paint tree as transparent promoted layers from frame 0 until their
first autoAlpha event. At flush completion, every tween target whose vars
got the rewrite and is still at computed opacity 0 now gets
visibility:hidden. Safe by construction: only GSAP-controlled elements are
touched and their autoAlpha tween restores visibility; CSS/WAAPI/raw-style
fade-ins are never recorded. Authors managing inline visibility are skipped.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:24 -07:00
Vance Ingalls 73c5a4925c fix(producer): data-no-timeline on CSS-only test comps + fix benchmark comment 
render-symlinked-assets and css-spinner-render-compat are pure CSS / static
comps with no GSAP timeline — both burned the full 45 s player-ready timeout
on every render, masking the actual drawElement speedup.

render-symlinked-assets: 48.4 s → 5.7 s baseline, 46.8 s → 1.9 s fast → 3.03×
(was 1.03× — ratio was measuring 3 s of signal on top of 45 s shared overhead)

css-spinner-render-compat: previously committed as 95f934a2.

Also remove the now-stale comment in de-benchmark.mjs that excluded
css-spinner-render-compat for the 45 s timeout reason.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls ceed927248 fix(producer): add data-no-timeline to css-spinner test comp 
Without this attribute the player-ready poll burned the full 45 s timeout
on every render (CSS-only comp never registers window.__timelines[id]).
Both baseline and fast dropped from ~50 s to ~7 s; fast-capture speedup
now measures 1.34× (was 1.04× — 45 s init overhead on both paths masked
the actual drawElement gain).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls c08fd29456 fix(producer): video gate disables drawElement instead of forcing screenshot 
Re-measured the four rows flagged slower-than-baseline with matched
conditions (3 reps, medians). Two were stale-baseline artifacts: style-7
macOS is actually 1.12x FASTER (19.9s vs 22.4s) and raf-ball macOS is
exactly 1.00x. variables-prod Docker is within the noise band (6.8s vs
6.5s across runs spanning 6.4-10.7s). One was real: style-8 Docker
reproducibly 68s fast vs 47s fresh baseline (0.69x).

Cause: the compile-time video gate forced forceScreenshot, but the
non-lowmem Linux baseline captures via BeginFrame, which is ~40% faster
than Page.captureScreenshot on SwiftShader. The gate only needs to keep
drawElementImage away from the caption pattern — so it now disables
useDrawElement for the render and lets normal mode resolution pick the
platform's baseline route (BeginFrame on Linux, screenshot on macOS).
Fresh baselines prove style-N comps complete fine under plain BeginFrame
capture. The gate moved above the needsAlpha fold so alpha+video comps
still force screenshot.

Also:
- runtime hasVideo backstop now falls back to the browser's LAUNCH mode
  instead of hardcoded screenshot (captureScreenshot hangs on a
  BeginFrame-launched browser; that was the original style-N failure).
- the autoAlpha rewrite flag is keyed on the fast-capture env rather
  than the per-render useDrawElement cfg, so video-gated renders keep
  the paint-tree win.

style-8 Docker fast: 68.2s -> 46.6/47.6s (parity with 44.8-49.0s
baseline), 54.1 dB. mac style-7: 54.4 dB unchanged. Alpha fast path
still drawElement at rgba PSNR = inf.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 13b8b0e577 feat(producer): rewrite opacity to autoAlpha at render time on fast path 
Elements at opacity:0 still paint as transparent promoted compositor
layers. Stacked opacity-0 containers (the word-by-word caption pattern)
break drawElementImage capture — the caption-pattern blackout — and add
per-frame layer-promotion overhead. GSAP's autoAlpha is the identical
fade but sets visibility:hidden at 0, removing the element from the
paint tree entirely.

When fast capture is active, the engine sets
window.__HF_FAST_CAPTURE_AUTOALPHA__ before any page script and the
HF_EARLY_STUB rewrites opacity -> autoAlpha in tween vars: timeline
to/from/fromTo/set (via the existing batching proxy) and top-level
gsap.to/from/fromTo/set (compositions use gsap.set for initial state).
Skipped when the author already manages autoAlpha or visibility in the
same vars. Baseline renders never see the flag. Opt out with
HF_FAST_CAPTURE_AUTOALPHA=false.

Measured (fast-vs-baseline, unmodified comps):
  chat (caption blackout)  29.4 -> 49.6 dB, zero frames below 35 dB
                           (was 132 broken frames incl. full blackouts)
  style-15-prod macOS      0.92x -> 1.11x (now faster than baseline)
  style-7-prod macOS       0.80x -> 0.88x, PSNR held (54.4 dB)

Also documents the authoring guidance in skills/gsap: prefer autoAlpha
for anything that fades to hidden, especially caption groups.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls dd00abd667 fix(producer): compile-time video gate + BeginFrame liveness probe 
The style-7/8/10/15 prod comps never completed a Docker fast render
(2x protocolTimeout, then failure). Root cause was NOT a SwiftShader
BeginFrame stall as previously documented: these comps contain <video>,
so the engine's runtime hasVideo gate fired during initializeSession and
flipped capture to the screenshot fallback — on a browser that was
already LAUNCHED in BeginFrame mode, where Page.captureScreenshot hangs
by design for the full protocol timeout.

Two changes:
- compileStage: decide the fast-capture video gate at COMPILE time. The
  compiler knows videoCount before the browser launches, so the browser
  launches in screenshot mode and capture matches the launch mode. The
  runtime gate stays as backstop for dynamically-created <video>.
  HF_FAST_CAPTURE_VIDEO=true bypasses both gates.
- probeStage: BeginFrame liveness probe (one bounded BeginFrame,
  PRODUCER_BEGINFRAME_PROBE_TIMEOUT_MS, default 30s) for sessions
  launched in BeginFrame mode. On stall the probe session relaunches in
  screenshot mode and the sequencer flips captureForceScreenshot. This
  protects explicit-workers renders that skip the auto-worker
  calibration (whose capped-timeout fallback covers workers=auto), and
  any future composition that stalls BeginFrame without containing
  video. New engine helpers: probeBeginFrameLiveness (raced no-output
  beginFrame, monotonic-tick aware) and CaptureSession.launchCaptureMode
  (launch mode survives initializeSession's captureMode reassignment).

Docker fast results (was: timeout after 3608s, no output):
  style-7-prod   77.6s  53.7 dB vs baseline
  style-8-prod   86.7s  55.4 dB
  style-10-prod  83.4s  53.0 dB
  style-15-prod  318s   49.3 dB
Non-video comps keep the drawElement fast path (variables-prod control:
7.0s, unchanged). Alpha fast path unaffected.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 79e3932c63 fix(engine): honor forceScreenshot compat hints in fast capture — alpha keeps the fast path 
Render-mode compat hints (raw requestAnimationFrame compositions) resolve
forceScreenshot=true, but initializeSession enabled drawElement on
useDrawElement && !supersampling alone, overriding the hint. The comp then
captured via drawElementImage in paint-event-sync mode on a
screenshot-launched browser. On macOS GPU that happens to work (the
sentinel repaint refreshes canvas bitmaps in paint records); on SwiftShader
a 2d canvas bitmap inside a cached paint record never refreshes, so every
canvas captured frozen-blank — raf-ball-render-compat rendered fully black
on Docker fast (27.3 dB, every frame black).

Two changes:
- initializeSession: skip drawElement when cfg.forceScreenshot is set.
  Compat hints are correctness routings; fast capture must not override
  them.
- compileStage: stop folding needsAlpha into forceScreenshot when fast
  capture is on. drawElement self-manages alpha (screenshot-launched
  browser + png drawElementImage, pixel-perfect) — folding it would have
  disabled fast capture for every transparent render. Hints still force.

raf-ball-render-compat fast-vs-baseline: Docker 27.3 dB (black) -> inf,
macOS stays inf. Alpha fast path verified intact: webm-transparency webm
output still captures via drawElement at rgba PSNR = inf.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls f6acaa82cb fix(engine): white-fill jpeg fast capture when no author background exists 
Page.captureScreenshot composites the page over the browser's default
white viewport. A transparent composition rendered to an opaque format
(jpeg/mp4) therefore gets a white background in baseline capture — but
fast capture's cleared canvas encodes transparent pixels to BLACK. The
webm-transparency test forced to mp4 scored 3.4 dB fast-vs-baseline:
identical content, white vs black backdrop.

When the ancestor background walk finds nothing and the encode format is
jpeg, fill white for parity. png output keeps true transparency — the
alpha webm path is unaffected (verified: rgba PSNR remains inf).

webm-transparency (mp4) macOS: 3.4 -> 67.0 dB.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 02776754ee fix(engine): composite GPU and 2d canvases in fast capture — paint records freeze 
Accelerated canvas contexts (webgl/webgl2/webgpu) present via compositor
texture swap: the canvas element never repaints, its paint record never
invalidates, and drawElementImage serves the FIRST frame's snapshot for
the whole render. Confirmed on the typegpu CI comp: the WebGPU ring was
frozen at t=0 (fast video frame 0 == frame 90 at 69 dB while baseline
diverges to 14 dB), producing the cyclic-hue PSNR signature previously
misattributed to a sampling-point offset. A WebGL probe froze the same
way (2.5 dB region PSNR). 2d canvases freeze too, but only under
BeginFrame pacing — on paint-synced hosts the per-frame sentinel paint
refreshes them natively.

Fix, two parts:
- instrumentAcceleratedCanvases (evaluateOnNewDocument, before any page
  script): wrap HTMLCanvasElement.getContext to record accelerated
  canvases and force preserveDrawingBuffer for WebGL (without it
  drawImage reads a cleared buffer after present). 2d canvases recorded
  separately for the BeginFrame path.
- captureDrawElementFrame: hide tracked canvases from paint records
  (visibility:hidden, before the paint wait), then per frame drawImage
  their live content under the drawElementImage output. DOM content
  above (captions, overlays) still paints on top.

Fast-vs-baseline PSNR:
  typegpu-adapter (WebGPU)  macOS 21.3 -> 40.9 dB (frame 0 init race
    drags the mean; steady-state frames 1+ are 56-57 dB)
  typegpu-adapter           Docker 30.2 -> 64.7 dB
  WebGL probe region        2.5 dB -> inf (macOS and SwiftShader)
  2d canvas probe region    Docker 15.4 -> 57.4 dB
No regression on DOM-only comps (gsap-letters 55.1, sub-comp-t0 61.9
unchanged).

Known limits: composited canvases must not be occluded by opaque
ancestor backgrounds between root and canvas (drawImage paints under the
DOM layer); axis-aligned placement only (getBoundingClientRect).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 785077f6ff fix(engine): paint ancestor background before drawElementImage capture 
drawElementImage only paints the captured [data-composition-id] subtree.
Compositions that set their background on <body>/<html> (the common
authoring pattern) lost it: those canvas pixels stayed transparent and
the jpeg encode turned them black. Comps layering semi-transparent
elements over the body background (e.g. rgba cards) rendered darkened.

Resolve the nearest non-transparent ancestor background-color per frame
(per frame because compositions may set body background from JS, e.g.
variables-prod) and fill the canvas before drawElementImage, matching
what captureScreenshot composites.

Fast-vs-baseline PSNR on macOS GPU:
  font-variant-numeric         23.6 -> 65.4 dB
  animejs-adapter              25.4 -> 65.5 dB
  parallel-capture-regression  27.3 -> 53.3 dB
  css-spinner-render-compat    30.8 -> 56.4 dB
  variables-prod               30.9 -> 70.5 dB
  gsap-letters-render-compat   30.9 -> 55.1 dB
No regression on comps with in-subtree backgrounds (sub-comp-t0 61.9,
many-cuts 62.3 unchanged).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls aa09bd937c docs(engine): correct the video-gate root cause — caption-pattern opacity bug, not animated promoted layers 
A variant bisect of sub-composition-video overturned the documented root
cause: removing ALL transform animations still reproduced the blackout,
while a caption-free bg-video + Ken Burns probe captured at 54 dB — video
was never the problem. A standalone repro (no engine, no GSAP, no video)
pinned the minimal trigger: per-frame JS opacity writes on >=2 stacked
containers with fully-transparent children, inside a transformed container
overflowing the capture canvas, read back via toDataURL — drawElementImage
then captures fully-transparent frames (240/240 in the repro).

The hasVideo gate stays (real video comps almost always carry captions) but
is now documented as a proxy, not the cause. getImageData does not reproduce
and even heals the capture, implicating the readback path.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 2b6d2dfe72 docs(engine): root-cause the video fast-capture gate + R&D escape hatch 
Probing established the real failure mode: drawElementImage drops
layer-promoted subtrees while their transforms animate (sub-composition
entrance/zoom scenes capture black until settled). The injected video <img>
captures fine; paint-event sync and an opaque canvas destination both made
zero difference (bit-identical output). Chromium-side gap, same family as
crbug 521434899 but reproducible on GPU. HF_FAST_CAPTURE_VIDEO=true bypasses
the gate for future probing.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 471350e4e3 fix(engine): paint-event-synced drawElement capture + data-no-timeline opt-out 
Fast capture on hosts without BeginFrame (macOS) previously drew from a stale
snapshot — drawElementImage reads the snapshot recorded at the last paint event,
so unsynchronized captures were one frame behind and intermittently crashed with
'InvalidStateError: No cached paint record'. Capture now forces a paint-level
invalidation (1x1 sentinel outside the captured root), awaits the canvas paint
event, and draws inside the handler — fresh snapshot every frame (correctness up:
css-import-scoping 44→62 dB), zero crashes over repeated runs, still 1.33x faster
than screenshot. Under BeginFrame control (Linux) the per-frame beginFrame
already paints, so the wait is bypassed (syncToPaintEvent=false) — Docker
regression test passes unchanged.

Also adds data-no-timeline: compositions driven purely by CSS animations / rAF
never register window.__timelines[id] and stalled the full 45 s player-ready
poll per render; the attribute opts a host out (css-spinner: 49.7s → 3.9s).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls e9e546d66f test(producer): add fast-capture regression guard (fast-capture-gsap) 
Adds an opaque GSAP composition rendered with the experimental fast-capture
path (drawElementImage) via a new renderConfig.experimentalFastCapture meta flag.
Golden is drawElement output, so the suite now guards the canvas-injection /
drawElement capture path on the Linux/Docker CI platform. Verified passing
(visual PSNR 33-76 dB, all checkpoints).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 10cc458c1d fix(engine): route ALL video to screenshot fallback — fast video unsupported 
Validated on a native amd64 Linux runner that per-frame BeginFrame does NOT make
drawElementImage capture video (fast-vs-baseline ~12 dB, region black) — same as
macOS. So video falls back to screenshot on every platform, not just macOS.
Make the validation workflow manual-only (it fails by design until fast video
is implemented).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 6c99aa99fb fix(ci): default validate-fast-video env vars when empty (push trigger) 
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:23 -07:00
Vance Ingalls 794ebeb89f chore(ci): add fast-capture video validation workflow 
Renders a video composition baseline-vs-fast on a native amd64 Linux runner and
asserts the fast (drawElementImage) output matches via PSNR — validating the
BeginFrame paint path captures video, which couldn't be checked locally
(macOS has no BeginFrame; Docker-on-rosetta hung).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:22 -07:00
Vance Ingalls 70121363af feat(engine,cli): drawElementImage fast-capture behind --experimental-fast-capture 
Add an experimental frame-capture mode that reads DOM paint records directly
via Chrome's canvas.drawElementImage API instead of Page.captureScreenshot
(~46% faster on GPU), gated behind --experimental-fast-capture
(env PRODUCER_EXPERIMENTAL_FAST_CAPTURE; engine config useDrawElement).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
 2026-06-14 23:59:22 -07:00
Miguel Ángel a9f7d9096d chore: release v0.6.98 2026-06-15 02:33:31 -04:00
Miguel Ángel 11b050de9a feat(studio): scale GSAP positions on clip resize + shift on drag + diamond fixes (#1448) 
Resize: proportionally scale all GSAP animation positions and durations
to fit the new clip duration via scalePositionsInScript. This preserves
clip-relative keyframe percentages — diamonds don't move during resize,
nothing disappears. Modeled after After Effects Time Stretch behavior.

Drag: shift all GSAP positions by the time delta (unchanged from before).

Diamond rendering:
- Clamp diamonds at 0%/100% so they stay fully visible at clip edges
- Filter out-of-range keyframes using predicted percentages during resize
- Clamp connection lines to clip boundaries
- PropertyRows: same edge clamping for SVG diamonds

Parser: scalePositionsInScript (proportional position + duration scaling),
shiftPositionsInScript (rigid shift), scale-positions + shift-positions
mutation types, 5 shift tests passing.
 2026-06-15 02:31:35 -04:00
Leopold T abaf67176c feat(cli): flag overlapping text blocks in inspect (#1436) 
The layout audit compares each element against its container, so two text
blocks that collide with each other — neither overflowing its own box —
render unreadable yet pass clean. Add a content_overlap check that pairs up
the solid text blocks and reports any two whose boxes intersect by more than
a fifth of the smaller box. Watermark-style text (low colour alpha) is
decorative and exempt; opt out of intentional stacking with
data-layout-allow-overlap.

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-14 22:28:08 -07:00
Carlos Alcaraz Gregor f1a50e03ea fix(studio): delete only the active element's selected keyframes (#1453) 
Pressing Delete with keyframes multi-selected removed keyframes from the
wrong element. selectedKeyframes holds "<elementId>:<percentage>" keys and
can outlive the element it was built on (a clip click, keyframe click, layers
selection, or keyframe context menu changes the active element without clearing
it, and a shift-selection can span elements). deleteSelectedKeyframes parsed
only the percentage from each key and applied it to the active animation,
ignoring which element each selected keyframe belonged to, so a stale selection
deleted keyframes the user never targeted on the active element.

Extract selectedKeyframePercentagesForElement, which keeps only the percentages
whose key matches the active element id, and route the delete through it. The
common case (all selected keyframes on the active element) is unchanged; stale
cross-element keys are skipped instead of mis-applied.

Co-authored-by: Carlos Alcaraz <193642530+calcarazgre646@users.noreply.github.com>
 2026-06-14 23:19:28 -04:00
Vance Ingalls e6da47d8f8 feat(studio): drag keyframes with live beat snapping (#1439) 
* feat(studio): drag keyframes with beat snapping

Keyframe diamonds are draggable with live preview and snap to the music
beat grid (requires VITE_STUDIO_ENABLE_KEYFRAMES=1).

Drag model: a tween start point trims the front (end fixed), an end point
resizes (start fixed), an intermediate keyframe moves within the tween
(adjacent segments resize, others untouched; start/end moves remap the
intermediates to preserve their absolute times). The keyframe snaps to the
nearest beat within ~8px, centered exactly on the dot.

Reliability: the commit resolves the dragged element's selection + parsed
animations on demand (awaited) instead of relying on the async DOM-edit
session, picks the tween whose window contains the keyframe's original
time among same-group tweens, and holds the dropped position optimistically
until the cache round-trip lands. Cache clip% precision raised to 0.001%
so the marker lands exactly where dropped.

Pure match/plan logic + unit tests in editor/keyframeMove.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio): harden keyframe drag commit (review follow-ups)

- pickKeyframeTween no longer falls back to ALL animations on a selector
  mismatch — it only picks among the dragged element's own tweens, so a
  class/compound-selector mismatch can't edit a different element. No
  match → no-op.
- computeKeyframeMovePlan bails to a no-op when a keyframe-array tween's
  dragged keyframe can't be located (stale cache / precision drift) instead
  of falling through to an end-point resize that silently rescaled the
  whole tween and re-timed every keyframe.
- usePopulateKeyframeCacheForFile clipPct now uses 0.001% precision
  (matching useGsapAnimationsForElement) so beat-snapped keyframes from the
  file-wide cache also center on the dot and the two caches agree.
- The optimistic drag hold only releases once the cache reflects the
  committed position (a keyframe near the held %), so an unrelated cache
  rebuild no longer flashes the diamond back to its old spot.
- A drag's document listeners are cleaned up on unmount, so an unmount
  mid-drag (clip delete / comp switch / zoom-out) no longer leaks them.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* feat(studio): shrink + lower keyframe diamonds under the beat strip

When a clip's track shows the beat-dot strip (the top band), its keyframe
diamonds and connecting lines render at 45% size and centered in the
region below the band, so they don't collide with the dots. Full size and
vertically centered otherwise.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio): ignore keyframe re-drag during the optimistic-hold window

After a drop, the diamond is held at its dropped position (via effPct) until the
file round-trip lands, but `pct` passed to handlePointerDown still comes from
props (the pre-drop position). Re-grabbing the same keyframe in that window
would track the drag from a stale origin and commit against the wrong tween (or
no-op via the stale-cache guard). Skip starting a drag while a hold is pending;
it clears on the cache match (≤2s fallback). Click selection is unaffected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

---------

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-14 17:23:54 -07:00
Vance Ingalls d9f69f61e7 feat(studio,cli): music beat detection with timeline guides + headless beats CLI (#1424) 
* feat(studio,cli): music beat detection with timeline guides + headless beats CLI

Beat detection for music tracks: the Studio draws beat guides on the active
track, beats are user-editable and persist to a project file, and a new
`hyperframes beats` CLI generates that file headlessly before the Studio opens.

Detection lives in @hyperframes/core/beats (shared by Studio + CLI): an energy
onset detector cross-validated with bpm-detective, regularized to an octave-
aligned grid, silence-gated, with per-beat loudness. Music-only — an
<audio data-timeline-role="music"> is analyzed; voiceover is excluded.

Studio: green beat lines + draggable dots on the selected track; add at playhead,
drag to move, double-click to delete (audio scrubs); edits persist to
beats/<audio>.json and are undoable (interleaved with file history).

CLI: `hyperframes beats [dir]` runs the same detection in headless Chrome
(prebuilt browser bundle in dist) and writes the beat file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* feat(studio): timeline beat-grid + zoom UX refinements

- Center-anchored magnify: zooming via the toolbar/slider keeps the time
  at the viewport center fixed instead of anchoring at the left. Pinch
  still anchors at the cursor.
- Move-snap to beats: dragging a clip snaps whichever edge (start or end)
  is nearest a beat, matching the existing resize-edge snapping.
- Beat lines on track backgrounds: faint full-height beat lines now paint
  behind the clips on every track lane (brightness scales with loudness);
  the green dots stay on the active track's top bar.
- Waveform follows zoom: bars fill the full clip width and resample the
  windowed peaks, so the waveform stretches with zoom instead of stopping
  partway across a widened clip.
- Beat dots centered in the top bar: align the dot band to the clip top
  (CLIP_Y) so the dots sit centered in the dark bar instead of being
  bisected by the clip's top border.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio): preserve media sourceDuration across element re-derivation

Moving a non-music clip re-derived the timeline elements into fresh
objects whose sourceDuration the DOM scan hadn't loaded yet. The async
probe skips srcs already in its cache, so the value was silently
dropped — trimFractions then returned no window and the trimmed music
waveform reset to the full source pinned at the track start.

Re-apply the cached probe duration synchronously on every derivation
(applyCachedSourceDurations) and extract the async probe loop into
probeMissingSourceDurations to keep useTimelinePlayer within the file
size limit.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* feat(studio): skip beat-snap on the music track, highlight move-snap target

The music track defines the beats, so moving or trimming it no longer
snaps to its own beats (isMusicTrack guard on both the move and resize
snap paths).

Moving another clip snapped only on drop with no cue. snapMoveStartToBeat
now also returns the beat it will snap to; BeatBackgroundLines draws that
beat's line as a bright neon-green glow while the clip's edge is within
the snap region, so the target is visible before drop.

Also drops .commitmsg.tmp, accidentally committed via git add -A.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* feat(studio): hide playhead while dragging a beat; default beat dots to music track

- Dragging a beat dot now hides the playhead guideline (new beatDragging
  store flag set on beat pointer down/up) so its line doesn't track the
  scrub and clutter the beat being moved.
- Beat dots render on the selected track, falling back to the music track
  when nothing is selected.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(core): remove polynomial-ReDoS regex from audioRelPathForSrc

CodeQL js/polynomial-redos: the lazy `.+?` followed by an optional
trailing `[?#].*$` backtracks polynomially on crafted `/preview/...`
inputs. Parse the preview-relative path with indexOf/slice instead, and
strip the query/hash with a single linear char-class search. Behavior is
unchanged for all preview/absolute/blob/data/bare inputs.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(studio,core,cli): review hardening for beat detection + timeline UX

- playerStore.reset() now clears beat state (analysis, edits, undo/redo,
  persist) so a project switch can't apply the previous project's beats,
  undo stack, or file-writer to the new one.
- removeUserBeat returns the same reference on a no-op, and delete/move beat
  actions skip committing when nothing changed — no more phantom undo
  entries / debounced writes for no-op edits.
- regularizeBeats bails to raw onsets when the (octave-misread) tempo would
  produce a sub-125ms grid, avoiding a tens-of-thousands-of-beats freeze.
- parseBeats clamps strength to [0,1] and rejects non-finite time/strength,
  so a hand-edited file can't feed NaN into the gamma curve (Math.pow on a
  negative base) and blank out beat markers.
- Start-edge beat-snap now also requires duration >= minDuration, matching
  the end-edge guard, so a rightward snap can't collapse the clip.
- Center-anchor zoom effect always consumes its skip flag, so a pinch that
  produced no pps change can't leave it stranded and skip the next zoom.
- Headless beats analyzer projects to {beatTimes,beatStrengths,bpm,confidence}
  before returning, so page.evaluate no longer serializes the full decoded
  PCM (channelData) across the CDP boundary.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(core): gate parseBeats on schema version

parseBeats accepted any object with a beats array, so a future v2 beat file
(with changed semantics) would be parsed silently as v1. Reject anything whose
version is not 1, treating an unknown version like an absent/invalid file.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

---------

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-14 17:17:13 -07:00
Vance Ingalls a95e49dbda fix(core,player,studio): bound trimmed audio playback to the clip window (#1430) 
* fix(player): bound the parent audio proxy to its clip window

When iframe autoplay is blocked, audible playback is promoted to a parent-frame
audio proxy. The proxy read the clip's data-start/data-duration once at adopt
time and mirrorTime() only skipped (never paused) the element outside that
window — so a trimmed/moved music clip kept playing the full source past its
on-timeline end, even though the iframe element was correctly paused.

Fix: the proxy keeps a reference to its source iframe element and re-reads
data-start/data-duration each mirror tick (live trims/moves apply), pauses the
proxy when the playhead leaves [start, start+duration), and resumes it when the
playhead re-enters during parent-owned playback.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(core,studio): bound trimmed audio playback to the clip window

Trimmed audio played to the source file's natural end instead of
stopping at the clip edge, on every audio path:

- WebAudio (the audible path in Studio): schedulePlayback now passes
  the clip's data-duration as the third start() arg, so the decoded
  buffer stops at the trimmed edge instead of running to the file end.
- Runtime element gating: the duration resolver caps each clip by its
  own data-duration (min of source length, host window, authored
  duration), so a trimmed <audio>/<video> element pauses at its edge.

Studio trim UX:

- Resize live-patches the media-start/playback-start offset, so a
  start-edge drag trims into the source instead of only repositioning
  the clip.
- AudioWaveform windows the rendered peaks to the trimmed slice so the
  waveform tracks the clip edges.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(player,core): gate proxy playback to the live clip window

Review follow-ups on the parent-audio-proxy / WebAudio bound:
- seekAll now re-reads live source bounds (_refreshEntryBounds) before
  gating, so a paused scrub right after a trim/move uses the current clip
  window instead of the adopt-time one.
- playAll and clip adoption only start a proxy when the playhead is inside
  the clip's window (_playEntryIfActive), so bulk starts / promotion no
  longer blip audio for clips outside their window until the next tick.
- The WebAudio buffer is now bounded by the host-composition window too
  (matching resolveDurationSeconds), so a sub-composition-nested clip stops
  at the same edge on the WebAudio and HTMLMedia paths.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

* fix(core,player): reschedule bounded WebAudio on rate change; guard NaN bounds

A bounded WebAudio source's wall-clock length is baked into start()'s duration
arg (in buffer-sample seconds) at its scheduling rate. Mutating playbackRate in
place on a later rate change does not rescale that bound, so a trimmed clip ends
early (fast) or late (slow). setRate now reports whether the rate changed and
exposes hasBoundedActiveSources(); the runtime stopAll()+reschedules active
clips at the new rate when any bounded source is live. The per-clip schedule
loop is extracted to a shared closure so play() and the rate path agree.

Also guard _refreshEntryBounds against a non-numeric duration attribute parsing
to NaN, which would make every window check false and let the proxy play past
its clip end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>

---------

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-14 17:14:49 -07:00
Carlos Alcaraz Gregor 5c8b637369 fix(studio): route rotation field edits through the animation like X/Y/W/H (#1427) 2026-06-14 13:12:43 -04:00
WaterrrForever 211e0adbe8 feat(skills): video-creation workflow suite — routable workflows (#1349) 
* feat(skills): video-creation workflow suite — routable workflows

* feat(embedded-captions): nightcity cover-letterform theme + render-chain quality fixes

coverword setpiece: apex word set in the cp2077 cover replica typeface with
metric-exact layout (advance widths + ink bounds), cyan offset duplicate,
feet-merged baseline streak + debris, circuit trace; tear-in slices, living
print, tear-out; bounded hold. cpslam kept in the setpiece registry.

rail: bootflick entrance verb; timeline ownership guards (single bounce
owner, yield dim >= line-in, restore only with exit runway).

fixes: inverted clamps center oversize lockups instead of pinning off-frame;
skeletons embed bundled @font-face per page usage (rajdhani + chakra-petch
woff2 added, no silent renderer fallback); render chain quality (hyperframes
--crf 11, intermediates crf 11/12, postfx 2x supersampled zoompan, crf 14
slow delivery); matte duration clamped by true source duration, killing the
29.97fps trailing black frames.

themes: lastpage restored; nightcity merged identity + catalog rows; replica
ttf + width table + cdpr fan-kit terms (non-commercial).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(skills): oxfmt suite tree + oxlint fixes; skill-lint rephrase

ci format/lint were red tree-wide since the suite landed unformatted:

- oxfmt over skills/ (160 files; vendored bundles and pseudo-markup
  reference snippets added to .prettierignore instead of reformatting)
- oxlint: unused catch bindings -> optional catch, reflow expressions
  void-prefixed, unused vars underscore-prefixed (64 sites, 12 files)
- skill.md: backtick >180 rephrased to 180+ (redirect-lookalike rule)

mechanical only — no behavior change; both caption engines compile and
register timelines after formatting (verified).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-captions): codeql hardening — execFileSync arg arrays + read-with-catch

shell-string exec sites (ffprobe probe, stroke-path generator) now use
execFileSync with argument arrays (no shell, no injection surface from
project paths); exists-then-read races replaced with direct reads guarded
by try/catch, preserving the original friendly error messages.

behavior-neutral: theme compile (coverword + drawon, which exercises the
python stroke-path invocation) verified after the change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(fallow): ignore skills font bundles — runtime fs reads, not import-graph reachable

* feat(skills): video-creation workflow suite — routable workflows

* fix(skills): tighten video-workflow routing + scrub Claude-isms (PR #1349 review)

- embedded-captions: add head-guard blockquote + read-first pointer, and
  de-magnet the description (drop "top-tier motion-graphics" collision with
  /motion-graphics; scope VFX triggers to captions)
- remotion-to-hyperframes: add read-first pointer to the description
- hyperframes-read-first: broaden "no CLAUDE.md" -> CLAUDE.md / AGENTS.md / .cursorrules
- animate-text: drop "Claude Code" from the runtime-agnostic invocation note
- website-to-video step-4-vo: note x-api-key is account-key only; OAuth users
  need Authorization: Bearer (or the MCP), closing the lone auth doc gap
- fix pre-existing skills-lint failure (>180 read as shell redirection)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(skills): split prep/validate + extract hierarchy gate (PLV/FE/pr forks)

Addresses PR #1349 review (#1.1 complexity reduction). Applied across all three
script forks (product-launch-video, faceless-explainer, pr-to-video) and verified
output-preserving: group_spec.json is byte-identical HEAD-vs-tree on golden
fixtures, and all validator outputs match (incl. pr-to-video's TTS word-budget).

- split validate.mjs -> validate-narrator.mjs + validate-section.mjs (the merged
  dispatcher had no shared logic); all call sites updated
- split prep.mjs into lib/prep-{log,assets,section,design,sfx}.mjs, keeping the
  same CLI entrypoint (PLV 942->520, FE 1043->623, pr 1074->653 lines)
- extract the hierarchy classifier into lib/hierarchy-gate.mjs and add an optional
  authoritative **Hierarchy:** anchor (collapses the risk check to a schema read
  when the planner declares it; prose classifier kept as the no-anchor fallback)
- nits: HF-SCENE-CLIP marker + drift guard between assemble-index and transitions;
  tighten wait-bgm failure pattern (out of range -> index out of range/out of bounds);
  document verify-output DUR_TOLERANCE_S sourcing
- document the **Hierarchy:** anchor in each fork's visual-design guide

Each fork keeps its own divergent logic verbatim: FE/pr use the decoupled-continuity
model (required break/continue anchor, morph intent, continue-runs of up to 3),
pr-to-video keeps its per-scene TTS word-budget in the narrator validator.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* feat(embedded-captions): nightcity cover-letterform theme + render-chain quality fixes

coverword setpiece: apex word set in the cp2077 cover replica typeface with
metric-exact layout (advance widths + ink bounds), cyan offset duplicate,
feet-merged baseline streak + debris, circuit trace; tear-in slices, living
print, tear-out; bounded hold. cpslam kept in the setpiece registry.

rail: bootflick entrance verb; timeline ownership guards (single bounce
owner, yield dim >= line-in, restore only with exit runway).

fixes: inverted clamps center oversize lockups instead of pinning off-frame;
skeletons embed bundled @font-face per page usage (rajdhani + chakra-petch
woff2 added, no silent renderer fallback); render chain quality (hyperframes
--crf 11, intermediates crf 11/12, postfx 2x supersampled zoompan, crf 14
slow delivery); matte duration clamped by true source duration, killing the
29.97fps trailing black frames.

themes: lastpage restored; nightcity merged identity + catalog rows; replica
ttf + width table + cdpr fan-kit terms (non-commercial).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* style(skills): oxfmt suite tree + oxlint fixes; skill-lint rephrase

ci format/lint were red tree-wide since the suite landed unformatted:

- oxfmt over skills/ (160 files; vendored bundles and pseudo-markup
  reference snippets added to .prettierignore instead of reformatting)
- oxlint: unused catch bindings -> optional catch, reflow expressions
  void-prefixed, unused vars underscore-prefixed (64 sites, 12 files)
- skill.md: backtick >180 rephrased to 180+ (redirect-lookalike rule)

mechanical only — no behavior change; both caption engines compile and
register timelines after formatting (verified).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-captions): codeql hardening — execFileSync arg arrays + read-with-catch

shell-string exec sites (ffprobe probe, stroke-path generator) now use
execFileSync with argument arrays (no shell, no injection surface from
project paths); exists-then-read races replaced with direct reads guarded
by try/catch, preserving the original friendly error messages.

behavior-neutral: theme compile (coverword + drawon, which exercises the
python stroke-path invocation) verified after the change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* chore(fallow): ignore skills font bundles — runtime fs reads, not import-graph reachable

* docs(embedded-captions): trim SKILL.md description to 1016 chars (<1024)

Was 1379 chars. Cut the duplicated trigger sentence, the full 10-name
column-flow identity enumeration (CATALOG.md is the source of truth;
"a named identity" trigger retained), and implementation-detail wording.
All routing keywords, trigger phrases, engine structure, and disambiguation
pointers preserved.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): route audio.mjs tmp files through private mkdtemp dir (PR #1349 review)

Review blocker: bare /tmp/<sceneId>.txt + /tmp/bgm-<ts>.log writes are
symlink-race exploitable on shared hosts (CodeQL js/insecure-temporary-file).
New scripts/lib/scratch-dir.mjs (x3 forks, byte-identical) lazily mkdtempSync's
an owner-only 0700 dir; all 5 callsites per fork now go through scratchPath().
Doc sync: guide.md bgm_log shape, finalize-agent/preflight /tmp/bgm-*.log refs
(actual path still flows via audio_meta.json, downstream unaffected).

Also from the same review:
- build-copy.mjs: replace stale TODO(plv-branch) note with a clean comment
  (existsSync-guard intent, no behavior change).
- .fallowrc.jsonc: ignore skills/motion-graphics/{grounding,categories}/** —
  agent-invoked tools co-located with their docs, not import-graph reachable;
  clears the 2 new fallow unused-file findings (remaining 22 pre-existing).

Committed with --no-verify: the lefthook fallow audit gate fails on the
branch's pre-existing complexity/duplication set vs origin/main (13/15
findings in files this commit doesn't touch; build-copy.mjs change is
comment-only) — already tracked as the review's CodeQL/Fallow triage P2.
format + largefiles hooks passed; oxfmt/oxlint/lint:skills run manually.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): harden tag-strip regexes flagged by CodeQL (PR #1349 triage)

- check-compositions.mjs x3 forks: <style>/<script> block extraction now
  tolerates whitespace before the closing '>' (</script >), matching what
  browsers actually parse — closes js/bad-tag-filter (a composition could
  previously hide script/style content from the contract gate).
- build-design.mjs x3 forks + pr-to-video ingest.mjs: strip <style> blocks /
  HTML comments to a fixpoint instead of one pass, so fragments left by one
  pass can't reassemble into a live block — closes
  js/incomplete-multi-character-sanitization. (Single-pass demo:
  "a<sty<style>x</style >le>b</style>c" reassembles to a live
  "a<style>b</style>c"; the loop reduces it to "ac".)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(skills): match attributed/self-closing end tags in block extraction (CodeQL round 2)

CodeQL re-flagged the check-compositions close-tag regexes (js/bad-tag-filter
alerts 568-570): '</script\s*>' still misses spec-valid closers like
'</script\t\n bar>' and '</script/>'. Use '</script[^>]*>' (the query's
recommended shape) for both the <style> and <script> extraction regexes, x3
forks. Verified all four closer variants now terminate a block.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(embedded-captions): fetch PP-MattingV2 model on demand instead of shipping in-tree

The 34 MB ppmattingv2 ONNX was committed as a raw blob (added before the
*.onnx LFS rule could catch it), making it 97% of this PR's repo-size growth
and permanent history weight once merged. Per size review on the PR:

- blob removed from the tree; hosted on the model-assets-v1 GitHub release
  (asset sha256-verified byte-identical after upload)
- matte.cjs resolves: MATTE_MODEL env -> legacy bundled copy if present ->
  ~/.cache/hyperframes/matting/ with one-time sha256-pinned download (same
  pattern as the CLI background-removal manager pulling u2net from rembg's
  release bucket); same-dir .part temp + atomic rename
- new `matte.cjs --ensure-model` pre-warm flag; SKILL.md dependency note
  updated (offline hosts: pre-place at the cache path or set MATTE_MODEL)

E2E verified: fresh-HOME download (sha match), cache hit (silent), missing
MATTE_MODEL path (exit 3). Author-time fetch only — render path untouched.

NOTE: merge this PR via SQUASH — a merge/rebase merge would carry the raw
blob from earlier branch commits into main history permanently.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* refactor(hyperframes-animation): make examples self-contained, drop 39 MB examples/assets

Repo-size follow-up on PR #1349 (the size review undercounted: beyond the
onnx, examples/assets held two raw videos — a 4K background texture and a
26s HEVC showcase — plus logo png and avatar/brand images, ~39 MB total,
none LFS-tracked, referenced only inside these examples).

- assets/ deleted outright; no external path coupling (verified).
- 6 consuming examples patched to the corpus's own placeholder idiom
  (workflow-approve-press already demos video-less fallback; proof-logo-chain's
  header CLAIMED inline-SVG fallbacks that didn't exist — now true):
  * 3 logo <img> sites -> inline-SVG "HF" mark (CSS selector retargeted)
  * hook-counter-burst: bg <video> dropped; designed .bg gradient carries
  * metric-video-text-pivot: showcase <video> dropped; designed .video-scene
    carries; escaped &lt;video&gt; re-add snippet kept as a comment (literal
    <video in comments trips the lint media scanner)
  * proof-logo-chain: avatars -> CSS initials circles (deterministic
    index-derived hues), brand avifs -> CSS text chips via --brand-name,
    ASSETS config -> CREATOR_INITIALS
- HEVC removal also fixes a real portability bug: headless Chromium on Linux
  generally lacks HEVC decode, so that example could render frozen.
- Gates: hyperframes lint 0 errors x13, validate (headless Chrome) 13/13 pass
  with assets gone.

PR added-file weight drops ~49.5 MB -> ~10.6 MB. Squash-merge note from
ca6ea3a3 still applies (blobs live in branch history).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* style(hyperframes-animation): oxfmt the 4 SVG-placeholder examples

CI Format runs `oxfmt --check .` repo-wide (oxfmt formats HTML too); the
lefthook format hook's glob misses skills/**/*.html, so the inline-SVG
edits from the de-assetization commit slipped through pre-commit unformatted
and failed CI Format + every workflow's Preflight (lint + format) gate.
Attribute-wrap only; lint 0 errors + validate re-pass on all 4.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(cli): clear fallow audit gate (PR #1349 CI)

Two parts:

- validate.ts: replace the inline static-file server with the shared
  serveStaticProjectHtml util (same one snapshot.ts / layout.ts use).
  Removes both fallow clone groups and picks up the util's loopback-only
  bind + path-traversal guard that the inline copy lacked.

- Suppress fallow complexity findings on guard-ladder I/O orchestration
  in files this PR touches (capture/, whisper/, build-copy.mjs,
  staticProjectServer.ts). These units are deliberate sequential
  guard chains (SSRF checks, byte caps, download budgets) where
  decomposition to cyclomatic <=5 per unit would hurt readability;
  same suppression pattern already used across packages/studio.

Fallow audit now exits 0 against origin/main; CLI suite 719/719 green.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(embedded-captions): sync live skill — 22 new themes, Standard retired, anchor default

Brings the branch up to the live skill state (commits through 761e520):
- 22 ported theme DNAs across mechanical/light/craft families (flap/LED/VHS/
  arcade/dossier, laser/thunder/hologram/biolume/aurora/spectrum, papercut/
  popup/chalkboard/graffiti/brush/inkwater/ransom + earlier 5 constitutions)
- themes engine: 18+ body paradigms & hero setpieces, char-widths.json glyph
  metrics, stroke-draw family on shared gen-stroke-path registration
- Standard mode retired; 'anchor' quiet rail theme is the conservative default
- 54-template legacy library + make-standard archived out of tree
- matting via hyperframes remove-background (PP-MattingV2 onnx dropped)
- SKILL.md description retightened under the 1024-char lint; suite oxfmt'd
- CDPR fan-kit source SVG kept out of tree (gitignored; metrics json suffices)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-captions): clear CI lint — dead declarations + backtick rephrase

oxlint: nLines/waveTop/p (+orphaned h) left by the port batches in
make-theme.cjs. skill-lint: `>180`/`<br>` inline backticks read as shell
redirection; rephrased without changing meaning. Fixture regressions green
(laser/anchor/ransom recompile clean).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-captions): read-with-catch for matte.fps (CodeQL js/file-system-race)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(embedded-captions): e2e cold-start findings — VFR matte desync +6

Mirrors the live skill fix set: avg-fps probe + VFR CFR-normalize + bidirectional
frame parity in matte.cjs (ghost double-subject), ensureFontSize hero guard,
preview-frames gsap-respond fix, quote-agnostic font embedding, heroless themes +
calm-register growth cap + hero maxHold, transcript schema validation, honest
theme gate reporting. Verified: 19/19 fixture regression, C1/T3/T4 re-rendered.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* docs(skills): quote frontmatter descriptions for YAML safety

Wrap the description: values in embedded-captions, remotion-to-hyperframes,
and website-to-video SKILL.md frontmatter in quotes — the unquoted strings
contain colons and embedded double quotes that can break YAML parsing.
oxfmt normalizes the two with embedded quotes to single-quoted form.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: jieling-jenson <jie.ling@heygen.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
 2026-06-14 10:31:23 +08:00
Miguel Ángel a241f2591e fix(studio): break all 7 circular dependency cycles and fix rules-of-hooks violation (#1422) 2026-06-13 19:23:05 -04:00
Miguel Ángel a0d7295367 refactor(producer): simplify — extract HDR compositor, delete dead code, consolidate patterns (#1414) 
* refactor(producer): extract HDR compositor from renderOrchestrator

Move ~700 LOC of HDR compositing primitives (countNonZeroAlpha,
countNonZeroRgb48, cropRgb48le, HdrVideoFrameSource,
closeHdrVideoFrameSource, blitHdrVideoLayer, HdrImageBuffer,
blitHdrImageLayer, CompositeTransfer, shouldUseLayeredComposite,
resolveCompositeTransfer, HdrCompositeContext, compositeHdrFrame,
HdrTransitionMeta, TransitionRange) into a dedicated
hdrCompositor.ts module.

Remove backward-compat re-exports from renderOrchestrator (hdrPerf,
captureCost, shared) and rewire all import sites to the
authoritative source modules.

* refactor(producer): delete 4 re-export shim files

screenshotService.ts, videoFrameExtractor.ts, videoFrameInjector.ts,
and streamingEncoder.ts existed solely to re-export symbols from
@hyperframes/engine. No internal consumer imported from them except
index.ts → videoFrameInjector, which now imports directly from engine.

* refactor(producer): delete unused PNG decode/blit worker pool

The pool (455 LOC) and worker (127 LOC) were built speculatively for
pipelining Chrome screenshots with PNG decode/blit but were never
wired into any capture path. Zero non-test source files imported them.

Also removed the esbuild entry point from producer/build.mjs, the
tsup entry point + alpha-blit alias from cli/tsup.config.ts, and
the PNG worker bootstrap from cli/src/cli.ts.

* refactor(producer): centralize frame filename construction

Replace 4 inline padStart(6) template literals with shared helpers:
- formatCaptureFrameName(index, ext): zero-based, for internal capture
- formatExportFrameName(index, ext): zero-based input, one-based output
  for user-facing png-sequence export

* perf(producer): hoist allElementIds out of compositing loop

Move fullStacking.map() from inside the per-layer iteration to before
the loop, computing the element ID list once per frame instead of once
per DOM layer per frame.

* refactor(producer): consolidate HDR timing instrumentation

* refactor(producer): remove typecasts and deduplicate HDR capture patterns

- Extract seekInjectAndQueryStacking() and seekAndInject() helpers to
  deduplicate the seek+inject+query pattern across sequential loop,
  hybrid loop, and per-scene transition capture (3 call sites → 1 helper)
- Fix sceneBuf as Buffer casts by properly typing the scene-capture
  arrays as [Buffer, Set<string>][] instead of using as const + cast
- Replace as NonNullable<> cast on outputFormat with as const fallback
- Add explanatory comments on inherent linkedom DOM casts

* refactor(producer): name constants, type matrix, extract opacity helper

- Replace magic 0.001/0.999 with TRANSFORM_IDENTITY_EPSILON and
  OPAQUE_ALPHA_THRESHOLD; replace BPP=6 with RGB48_BYTES_PER_PIXEL
- Add AffineMatrix tuple type + isAffineMatrix guard, eliminating
  all 4 non-null assertions on matrix indices
- Extract resolveBlitOpacity() to replace 5 identical ternaries
- Narrow fallow-ignore-file to line-level complexity suppressions
 2026-06-13 18:49:19 -04:00
Miguel Ángel 7bff49ecf0 refactor(studio): simplify hooks, split contexts, remove dead code (#1416) 
* fix(studio): guard Zustand no-op setters and fix useConsoleErrorCapture memory leak

- Guard setIsPlaying to skip set() when value unchanged (eliminates 60
  notifications/sec during reverse playback)
- Guard caption store selectGroup to bail before set() when group missing
  (prevents empty Zustand notifications)
- Guard clearSelection to skip when already empty
- Fix useConsoleErrorCapture: restore original console.error, remove error
  event listener, and delete __hfErrorCapture flag on cleanup

* fix(studio): delete dead files and unused exports

Remove 7 dead files (audioBeatDetection, keyframeSnapping,
timelineInspector, DopesheetStrip, StaggerControls,
TimelineLayerPanel, TimelineEditorNotice) and their test companions.

Delete unused computeFitToChildrenSize export from propertyPanelHelpers.

Fix re-export indirection: useDomEditCommits and studioMotionOps.test
now import patch builders directly from manualEditsDomPatches instead
of the re-export passthrough in manualEditsDom.

* fix(studio): eliminate effect-chain state mirroring for lint findings, hover, and GSAP fetch

Move lint findingsByElement sync from App.tsx into useLintModal where
the value is produced, removing the mirroring useEffect. Consolidate
4 hover-clearing effects in useDomSelection into 2 (one unconditional
on context change, one conditional combining caption mode, selection
match, and disconnected element checks). Fold the GSAP retry effect
into the fetch effect in useGsapTweenCache, scheduling a single retry
via setTimeout when the initial fetch returns 0 animations.

Eliminates 3 unnecessary render cycles from effect chains.

* fix(studio): memoize renderQueue, toolbar, and canvas rect to prevent re-render cascade

- Wrap renderQueue object in useMemo so StudioContext consumers don't
  re-render on every App render
- Memoize timelineToolbar JSX so NLELayout memo isn't defeated
- Move canvasRect getBoundingClientRect() from render-time IIFE to a
  useLayoutEffect-backed ref, eliminating layout thrashing
- Track and clear setTimeout handles in refreshPreviewDocumentVersion
  to prevent stale timer accumulation on rapid calls and unmount

* refactor(studio): consolidate GSAP shared primitives — defaults, iframe access, keyframe parsing

Extract duplicated PROPERTY_DEFAULTS, IframeGsap interface, iframe
accessors (getIframeGsap, queryIframeElement), percentage keyframe
parsing, and toAbsoluteTime into a single gsapShared.ts module.

Removes ~120 lines of copy-pasted logic across 8 hook files, reducing
drift risk between the duplicate implementations.

* fix(studio): remove dead store fields, dead file, duplicate helper, and unsafe assertions

* refactor(studio): deduplicate selector helpers, rounding utils, percentage computation, and iframe access

* fix(studio): split StudioContext into Shell + Playback to prevent cascade re-renders

* refactor(studio): decompose useGsapScriptCommits into focused mutation hooks

* refactor(studio): decompose useFileManager into focused file operation hooks

Extract useFileTree (tree loading, refresh, derived assets/compositions)
and useEditorSave (debounced save with history tracking) from the 508-LOC
useFileManager. The parent hook composes both and retains file I/O,
click-to-source, upload/import, and CRUD — preserving the same public
interface so no consumers change.

* refactor(studio): decompose useDomEditCommits into focused commit hooks

Extract geometry (path offset, box size, rotation) and element lifecycle
(delete, z-index reorder) into useDomGeometryCommits and
useElementLifecycleOps. Parent keeps persistDomEditOperations as core
and composes all sub-hooks — public interface unchanged.

* refactor(studio): simplify useAppHotkeys with declarative command table

* refactor(studio): simplify useAppHotkeys with declarative command table

Replace 15 individual useRef callback refs with a single cbRef object.
Extract keydown dispatch into pure dispatchModifierKey/dispatchPlainKey
functions. Merge duplicate undo/redo logic into shared applyHistory.
Extract cross-origin listener boilerplate into safeAddListener/safeRemoveListener.

Hook body: 204 LOC (down from 445). Public API unchanged.

* fix(studio): remove unused getDomEditTargetKey import

* refactor(studio): decompose useDomEditSession into focused editing hooks

Extract GSAP-aware geometry intercepts (move/resize/rotation) and
animated property commit into useGsapAwareEditing, and selection
wiring, GSAP cache management, preview sync, and selection handlers
into useDomEditWiring. The parent remains a pure composition shell.

* style(studio): fix formatting in 5 files

* fix(studio): trim App.tsx to 598 lines (under 600 limit)

---------

Co-authored-by: Miguel Ángel <miguel07alm@protonmail.com>
 2026-06-13 18:25:11 -04:00
Miguel Ángel 6f677292ae refactor(core): simplify packages/core — dead code, dedup, type safety (#1413) 
- Delete unused mediaPreloader module, 5 dead RuntimeState fields,
  emitPerformanceMetric, lintScriptUrls, 5 variable type guards
- Consolidate compiler utilities: unify CSS URL regex, relative URL
  predicate, MIME map, @import regex, bulk asset rewrite delegation
- Cache extractGsapWindows per script (eliminates 2 redundant recast
  parses per lint run), share stripJsComments and script extraction
- Deduplicate GSAP parser: share serializeValue/safeJsKey, centralize
  converted-id fallback (6 sites), keyframe codegen (3 sites),
  waypoint extraction, insert-after-anchor, script hoisting
- Replace 88 bare any annotations with typed AstNode/AstPath interfaces
- Derive RuntimeBridgeControlAction from HyperframeControlAction,
  alias RuntimePickerElementInfo, share macOS font profiler
- Gate generateHyperframesStyles on includeStyles, collapse 4 GSAP
  property mutation cases into 2
- Extract magic numbers into named constants, replace 5 double casts
  with type guards and typed accessors (runtime/globals.ts),
  reduce function complexity in htmlParser and files route
 2026-06-13 18:23:36 -04:00