mirror of
https://github.com/heygen-com/hyperframes.git
synced 2026-09-14 18:01:20 +08:00
9b7531b908
Surgical bugfixes accumulated across a series of real-AI-test runs
(heygen.com, huly.io, heygen-showcase). Each fix targets a specific
observed defect; happy paths are untouched.
packages/cli/src/capture/
• assetCataloger.ts: surface three structural logo signals on every
cataloged asset (inBanner / inHomeLink / matchesTitleBrand). The
prior class-substring-only isLogo detector caught 0/32 SVGs on
heygen.com and 0/19 on huly.io — modern React/Tailwind builds
don't put "logo" or "brand" in any className. The new signals
catch the universal "site header logo" pattern. Boolean merge
semantics: any positive sample wins through context-merge +
srcset dedup.
• tokenExtractor.ts: broaden inline-SVG isLogo via the same three
structural signals (header/nav/role=banner ancestor, root-href
anchor parent, document.title brand-segment match in aria-label).
No change to the existing class-substring detector — runs first,
new heuristics only fire when it misses.
• assetDownloader.ts: content-hash SVG slugs. SVG filenames are now
`svg-<8char-sha1>.svg` (or `logo-<hash>.svg` when isLogo flags
fire), replacing the previous label-derived slugging that
mis-attributed brand carousels. Verified by rasterizing real
captured SVGs: heygen-logo.svg actually contained the Google
wordmark, hubspot-logo.svg contained Trivago, huly-logo.svg
contained "Kube", heygen-logo.svg → "oogo". Catalog → URL label
inference (aria-label / nearest-heading / sectionClasses) is too
drift-prone across partner-logo carousels; content-hash names are
invariant by construction.
• contentExtractor.ts: SVG→PNG rasterization via sharp before
sending to Gemini Vision. Previous path sent raw SVG markup as
text and hit pure-hallucination output on wordmarks (VIVIENNE
for HubSpot, "wrestling" for Workday). Vision models can read
PNG pixels reliably; they cannot mental-render path commands.
Adds polarity detection (white-glyph vs dark-glyph) so an SVG
that flattens to a blank PNG against the wrong background gets
inverted automatically before captioning.
• contentExtractor.ts: LOGO tag in asset-descriptions.md lines
when the structural signals fire (independent of Gemini). The
no-Gemini-key fallback still emits an ⚠ banner + the LOGO-tagged
lines so agents can grep for logos via filename pattern even
without Vision.
• index.ts: asset-descriptions.md header branches on Gemini-key
presence with an explicit "Vision was OFF, descriptions are
catalog-derived" warning + a fallback recipe ("open LOGO-tagged
SVGs in a previewer before referencing"). Progress message also
reports catalog-fallback mode.
• capture/assetCataloger.ts + capture/tokenExtractor.ts regex
escape: `/^https?:\\/\\/[^/]+\\/?$/` inside the page.evaluate
template literal. The original `/^https?:\/\/[^/]+\/?$/` was
collapsing `\/` to `/` inside the template (because backslash
before a non-escape char is consumed), producing a parse error
on every capture. Capture against heygen.com and huly.io both
100% blocked on this until the escape was fixed.
packages/core/src/lint/utils.ts
• findRootTag masks <!-- ... -->, <style>...</style>, and
<script>...</script> ranges before tag extraction. A literal
<video> token inside a CSS comment (`/* The card uses <video>
as the surface */`) inside a <style> block was being picked as
the composition root, producing two cascading false errors
(root_missing_composition_id + root_missing_dimensions).
Verified against a synthetic repro plus the real beat that hit
this. Existing stripJsComments / extractScriptTextsAndSrcs
exports preserved — earlier work-in-progress commits had
accidentally removed them; they're consumers of these helpers
in lint/rules/{adapters,composition,core,gsap}.ts.
packages/cli/src/utils/lintProject.ts
• New lintMissingLocalAsset rule: scans <video>/<img>/<source>
src attributes for local files that don't exist in the project.
Uses resolveExistingLocalAsset (same helper stylesheet lint
uses) so the existence check matches the bundler's notion of
"resolves" — handles root-absolute "/assets/foo.png" relative
to projectDir, rejects "../outside.png" that escapes the
project. The renderer otherwise 404s these silently and ships
a video with missing visuals. Empirically the most common
sub-agent mistake across multi-URL runs (~5+ per run).
packages/producer/build.mjs
• ESM banner shims __dirname / __filename from import.meta.url
alongside the existing createRequire shim. Bundled CJS deps —
notably the ffmpeg/Emscripten wasm glue, which does
`scriptDirectory = __dirname + "/"` — were crashing at render
time with "__dirname is not defined in ES module scope".
Verified by re-running a producer render after rebuild;
ffmpeg pipeline completes cleanly.
All targeted tests pass (255/255 across capture / lint / lintProject
/ sfx unit tests). Typecheck clean for @hyperframes/core and
@hyperframes/cli. No happy-path behavior change; each fix targets
a specific observed failure.
@hyperframes/core
Types, parsers, generators, compiler, linter, runtime, and frame adapters for the Hyperframes video framework.
Install
npm install @hyperframes/core
Most users don't need to install core directly — the CLI, producer, and studio packages depend on it internally.
What's inside
| Module | Description |
|---|---|
| Types | TimelineElement, CompositionSpec, Asset, canvas dimensions, defaults |
| Parsers | parseHtml — extract timeline elements from HTML; parseGsapScript — parse GSAP animations |
| Generators | generateHyperframesHtml — produce valid Hyperframes HTML from a composition spec |
| Compiler | compileTimingAttrs — resolve data-start / data-duration into absolute times |
| Linter | lintHyperframeHtml — validate Hyperframes HTML (missing attributes, overlapping tracks, etc.) |
| Runtime | IIFE script injected into the browser — manages seek, media playback, and the window.__hf protocol |
| Frame Adapters | Pluggable animation drivers (GSAP, Lottie, CSS, or custom) |
Frame Adapters
A frame adapter tells the engine how to seek your animation to a specific frame:
import { createGSAPFrameAdapter } from "@hyperframes/core";
const adapter = createGSAPFrameAdapter({
getTimeline: () => gsap.timeline(),
compositionId: "my-video",
});
Implement FrameAdapter for custom animation runtimes:
import type { FrameAdapter } from "@hyperframes/core";
const myAdapter: FrameAdapter = {
id: "my-adapter",
getDurationFrames: () => 300,
seekFrame: (frame) => {
/* seek your animation */
},
};
Parsing and generating HTML
import { parseHtml, generateHyperframesHtml } from "@hyperframes/core";
const { elements, metadata } = parseHtml(htmlString);
const html = generateHyperframesHtml(spec);
Linting
import { lintHyperframeHtml } from "@hyperframes/core/lint";
const result = lintHyperframeHtml(htmlString);
// result.findings: { severity, message, elementId }[]
Documentation
Full documentation: hyperframes.heygen.com/packages/core
Related packages
@hyperframes/engine— rendering engine that drives the browser@hyperframes/producer— full render pipeline (capture + encode)hyperframes— CLI