- .env.example: replace removed speech-1.x mention with the actual
supported backends (s1 / s2-pro / s2.1-pro)
- docs/PROVIDERS.md: add fish.audio section (setup, backend models,
per-byte pricing incl. free s2.1-pro-free tier) plus entries in the
env var summary, Provider-to-Tool Mapping, and Capability Coverage
- skills/INDEX.md: list fish-audio-tts in the TTS & Audio Layer 3 row
Add an Azure AI Speech transcription tool. It is opt-in: when
AZURE_SPEECH_KEY is configured the agent may prefer it for cloud STT,
while the local faster-whisper `transcriber` stays the default offline
path. Shared pipeline manifests are intentionally left unchanged, so no
default provider selection is altered for existing users.
- tools/analysis/azure_stt.py: new `azure_stt` tool (capability=analysis,
provider=azure) calling the Fast Transcription REST API. The local file
is uploaded via multipart and transcribed synchronously with word-level
timestamps and optional diarization — no Blob storage or async polling.
Output schema mirrors `transcriber` exactly, so it is a drop-in for
`subtitle_gen` and other transcript consumers. Follows the existing
provider-tool conventions (env-var status check, `_transcribe` helper,
cost_usd/model on the result, fallback="transcriber").
- Auto-discovered by the registry; no registry or selector changes.
- tests/tools/test_azure_stt.py: contract, discovery, status, response
mapping, execute guardrails, and a mocked-network success path (no live
API calls).
- .agents/skills + .claude/skills: azure-speech-to-text Layer-3 skill.
- docs/PROVIDERS.md: Azure AI Speech setup, API notes, and pricing.
- .env.example, skills/INDEX.md, AGENT_GUIDE.md: document the optional
cloud STT path alongside the default whisper transcriber.
Fixes surfaced by the "How Salt Changed History" run:
- Decisions rail showed a stale choice. Contract now requires a superseding
decision_log entry when a logged choice changes mid-run; the board collapses
by category+subject and renders the latest as current, tagged "revised"
(AGENT_GUIDE Decision Communication Contract; board.js renderDecisions).
- Assets gate jumped by a draft render. The review surface is the filmstrip:
render per-scene stills to snapshots/<scene_id>.png (scripts/atelier_snapshots.py)
and STOP; the full/draft render is the compose stage, post-approval
(checkpoint-protocol, bespoke-composition).
- Broken thumbnails / missing scenes. A .tsx "animation" asset is no longer a
renderable visual — resolve snapshots/<scene_id>.* or a shot-spec placeholder;
add <img onerror> fallback and a BESPOKE placeholder (state.py, board.js).
- Clicking a card video did nothing. .thumb video had no sizing, so the frame
and click box drifted; add object-fit:cover. Click-to-play on the render-hero
player; narration is clickable -> modal with the full text (board.css/js).
- Type too small everywhere. Add one --fs-scale (1.16) knob; every font-size is
calc(px * var(--fs-scale)) so text scales proportionally and reversibly,
verified for zero overflow at desktop + tablet widths.
Licensing (finding 1):
- Mocap library is now 100% CMU-sourced (free for any use). Replaced the 3
Meta/FAIR clips with CMU equivalents: wave=141_16, shuffle=77_29 (creep);
dropped the un-CMU "dab", added twist=141_12. Deleted the 4 committed FAIR
BVHs (dab/jumping/wave_hello/zombie).
- Ship Patrick Hand's SIL OFL license (assets/OFL.txt) beside the bundled font
— OFL permits embedding/redistribution; this is the required attribution.
- Add THIRD_PARTY_NOTICES.md (Patrick Hand OFL + CMU) and rewrite mocap/NOTE.md
to drop the "verify before commercial use" caveat (no longer applicable).
Stale clip names (finding 2):
- Update the /ink-art command files (.claude/.github/.cursor/.codex),
character-design-director, and the creative skills to the current catalog
(wave/twist/…), and point to add-motion.mjs.
- ink-puppet.js: console.warn on an unknown clip name instead of silent dead-time.
Examples (finding 3):
- Remove broken standalone reel.html/momentum.html (they referenced a missing
ink-theater.js and momentum shipped the subset font the fix warns against).
- Keep mocap-figure/ as the canonical self-contained, lintable example; refresh
its bundled clips.js/ink-puppet.js; add examples/README.md with the lint path.
Adds an explicit 'choose a character source' step so the agent never silently
reuses a bundled character (the 'same mascot every video' problem). Options:
user uploads a drawing; user uploads a photo -> doodle-ify (img2img) -> rig;
generate a fresh doodle (FLUX/Imagen, recommended default); or stock
(pixabay/pexels, hit-or-miss). Bundled chars are demo-only.
Ships Phases 1-3 so the feature isn't left hanging:
- Phase 1: 12-action mocap library (walk/run/jump/climb/march/kick/sit/wave/dab/
shuffle/dance_spin/dance_glide) from CMU + fair1, with mocap/catalog.json.
Fixes the '4 repeated moves' repetition problem.
- Phase 2: bvh2clip.mjs is skeleton-agnostic (fair1/CMU/Mixamo joint aliases) +
frame cap + --name; add-motion.mjs fetches/converts/rebundles any BVH by CMU
id / URL / path — self-extending, no code changes.
- Phase 3: skill guidance to read catalog.json + choreograph variedly (never
loop a clip), add-motion, and the balloon primitive.
- Engine: InkTheater.balloon() speech-balloon primitive (both audit videos
hand-built it).
- Audit fixes: /animated-drawing skill gains a motion->retarget mapping table
(2 motions crashed on the wrong config) + a 'Compositing into HyperFrames'
section (transparent output, GIF-freezes->VP9-alpha, the linter video
contract) + character-variety guidance (animate the user's/a generated
drawing, not a bundled char) + pipeline-exempt notes.
Example refreshed to the new library + full Patrick Hand TTF.
Audit finding from video production: handwriting rendered as serif everywhere
because the woff2 we downloaded was a Google-Fonts css2 subset MISSING basic-latin
(ASCII) glyphs — not because 'SVG text can't use webfonts' (the old README claim
was a misdiagnosis; prior demos were silently serif). Fix: embed the FULL font.
Bundles ink-theater/assets/patrickhand.ttf and corrects README + skill guidance
(HTML overlay divs + full TTF).
Makes the doodle-animation capability findable instead of hidden.
Routing (OpenMontage-internal):
- AGENT_GUIDE: Style Playbooks gets an ink-sketch row + a 'hand-drawn doodle'
routing note (it's a style + engine on the animation / character-animation
pipelines, NOT a new pipeline).
- animation/idea-director + character-animation/character-design-director point
to the Ink Theater / Ink Puppet skills for hand-drawn character briefs.
Cross-agent slash commands (/ink-art, /animated-drawing) for Claude Code,
Copilot, Cursor, and Codex — thin pointers to the canonical skills so they stay
in sync. /ink-art = create a vector doodle from scratch (Ink Theater + Ink
Puppet mocap); /animated-drawing = animate a SUPPLIED drawing via Meta
AnimatedDrawings (raster). New skills/creative/animated-drawing.md documents the
Path A setup/run/limits. Codex prompts are user-home only — .codex/prompts/README
explains the copy/symlink step.
.claude/commands/*.md (Claude Code)
.github/prompts/*.prompt.md (Copilot)
.cursor/commands/*.md (Cursor)
.codex/prompts/*.md + README (Codex; copy to ~/.codex/prompts)
Reusable engine (ink-theater/ink-theater.js, global InkTheater) for hand-drawn
ink-on-white animated 'moving art', authored for HyperFrames (atelier). Five
seek-safe primitives: variable-width brush strokes (inkPath/inkRibbon),
timeline-driven boil, closed-form damped-spring eases, FABRIK 2D IK + riggable
mascot, and a parametric contraption grammar (crank/gauge/hopper/slot/lever/box).
Adds skills/creative/ink-theater.md (metaphor method + mined archetypes + color
grammar) and README with the HyperFrames font gotcha (webfonts apply to HTML
divs, not SVG <text>). Two reference builds under ink-theater/examples/.
Grounded in deep-research on vector/physics/metaphor foundations (FABRIK,
bounded-biharmonic/ARAP, closed-form springs, LLM metaphor+layout planning).
Step 6 still showed a legacy publish_log entry with 'metadata' and 'video_path'
fields, which the publish_log schema rejects (entries set additionalProperties:
false). Replace it with the actual schema-valid shape export_bundle returns in
data['publish_log'] (status 'exported', export_path, metadata_used) and note
that it should be persisted directly without extra entry fields.
Make the new PUBLISH-tier tool usable rather than dead code: add export_bundle
to the animated-explainer publish stage's tools_available, and update the
explainer publish-director skill Step 5 to call export_bundle for packaging
(passing the prepared metadata) instead of hand-creating the export tree. The
documented export layout and the no-upload caveat are kept in sync with the
tool. Other pipelines' publish stages can adopt it the same way.
Building the in-a-hurry showcase video surfaced a real workflow gap:
bespoke-composition.md was Remotion-flavored throughout. A fresh agent
reading it for a music-driven hero piece would be routed straight to
Remotion atelier — missing that HyperFrames has no cut-schema and is
already atelier by its nature (every HF composition is a hand-authored
index.html with data-* timing and a GSAP timeline you write).
This commit reframes the doctrine to be runtime-aware:
- New "Atelier and the two runtimes" subsection explicitly separates
Remotion (atelier as escape hatch from cut-schema registry) from
HyperFrames (always atelier; composition_mode "atelier" is implicit).
Same principles apply in both cases; the rendering path differs.
- Step 3 (vocabulary) now routes to /hyperframes-animation for HF
projects — 36+ atomic rules (kinetic-beat-slam, 3d-text-depth-layers,
motion-blur-streak, ...), 15+ scene blueprints, 16 transition families,
7 runtime adapters (GSAP / Lottie / Three.js / Anime.js / CSS / WAAPI /
TypeGPU), and the html-in-canvas-patterns headline capability for 1–3
cinematic hero beats per video. Also points at /hyperframes-creative,
/hyperframes-media, /media-use, /hyperframes-cli.
- Step 4 (engine mechanics) adds HF-side gotchas alongside Remotion:
the data-* contract, mandatory class="clip", paused-timeline + seek
rule, lint/validate/snapshot verification primitives, stable-id
selectors over nth-of-type.
- Step 5 (render) split into per-runtime paths:
* Remotion atelier — auto-stage copy + bespoke contract (unchanged).
* HyperFrames — npx hyperframes init scaffold, beats workflow,
lint+validate+snapshot verification, npx hyperframes render.
Flags the known gap (F13): hyperframes_compose.render still requires
cuts[] from the templated path; call npx directly until the tool
grows a bespoke branch parallel to _render_via_atelier.
- Worked precedents section now lists both runtimes — Phantom Reach /
Compound Snowball / Library of Alexandria for Remotion atelier, plus
in-a-hurry for HyperFrames (kinetic-beat-slam + 3d-text-depth-layers
+ motion-blur-streak + css-distortion + html-in-canvas + Three.js
bloom).
## Remotion bump (mechanical, semver-safe)
remotion-composer: Remotion 4.0.441 → 4.0.484 (43 patch versions, all
within 4.0.x). Includes the seven core packages: remotion + @remotion/cli,
captions, google-fonts, media, player, transitions.
Smoke test: re-rendered the compound-snowball atelier composition through
the unchanged tool path; final_review status=pass, atelier checks clean.
Note: package.json also carries d3-geo@^3.1.1 forward — this line was
already in the working tree from prior unrelated WIP and is not part of
this bump. Removing it would leave package-lock mismatched, so it's
preserved as-is here; clean up separately when its consumer lands.
## HyperFrames skills re-vendor (0.4.2 → 0.7.17)
The runtime invoked by hyperframes_compose (`npx hyperframes`) was
already pulling 0.7.17 on every render, but the vendored skill docs the
agent reads were frozen at 0.4.2-era. This commit closes that gap.
Re-vendored from upstream commit 3351fb1a (tag v0.7.17, 2026-06-27):
Re-vendored core 4 (restructured upstream):
- hyperframes (slim entry; deep content moved to focused skills)
- hyperframes-cli (1 → 7 files; covers validate/inspect/snapshot/
benchmark/lambda natively, dropping the obsolete
OM-local validate patch)
- hyperframes-registry
- website-to-video (renamed upstream from website-to-hyperframes)
Newly vendored (8 strategic additions in 0.5–0.7):
- hyperframes-core composition contract (data-*/tracks/sub-comps)
- hyperframes-creative palette, type, narration, beat planning
- hyperframes-media TTS, BGM, SFX, transcription, captions, bg-remove
- hyperframes-animation all motion knowledge (rules, blueprints,
transitions, 7 runtime adapters)
- media-use agent Media OS (one `resolve` verb for
BGM/SFX/image/icon; project + global cache)
- motion-graphics short design-led motion patterns
- remotion-to-hyperframes migration guidance (directly relevant since
OpenMontage runs both runtimes)
- music-to-video beat-synced video using `hyperframes beats`
Intentionally NOT vendored (HF-workflow-specific; would compete with
OpenMontage pipeline routing): embedded-captions, faceless-explainer,
general-video, pr-to-video, product-launch-video, slideshow,
talking-head-recut. Re-evaluate per pipeline need.
PROVENANCE.md refreshed with the new vendor point + re-sync instructions.
## GSAP CDN pin
.agents/skills/hyperframes/SKILL.md: gsap@3.14.2 → gsap@3 (auto-latest 3.x
on jsdelivr; avoids future drift without breaking the API surface).
## Doctrine updates routing to new skill structure
- skills/INDEX.md — HyperFrames row expanded to enumerate the 12 vendored
skills and their roles.
- skills/meta/animation-runtime-selector.md — runtime decision matrix
updated for the rename (website-to-hyperframes → website-to-video) and
three new rows added: beat-synced music videos, Remotion→HF porting,
and the media-use resolve verb. The HyperFrames composition row in the
animation-library matrix split into four (core/creative/media/animation)
per the upstream skill structure.
- skills/core/hyperframes.md — Layer-2 routing skill rewritten to point at
the new focused skills and all website-to-hyperframes references renamed.
F12 — Hero-component spine is a templating anti-pattern. The first
atelier proof (alexandria-fire) leaned on a single candle as scaffolding
under every scene with different text below — same hero, different
caption — which is exactly the "branded slides" pattern atelier was
supposed to break. The signature device is meant to appear in ONE or
at most two beats (typically the climactic moment), not as the visual
scaffolding of every scene. Each scene must earn its own composition:
a different primary visual subject, a different scale, a different motion
register. Doctrine added as step 1.5 in skills/meta/bespoke-composition.md
("Plan each scene as its own composition") plus a reviewer
scene_distinctness gate enforced at compose stage — the reviewer samples
one representative frame per scene and refuses to pass if two or more
scenes share their primary visual subject.
F9 — Captions/on-screen-text dedup. When a SerifLine reads the script
verbatim AND auto-captions emit the same text, the piece looks amateurish
even when the rest is beautiful. Rule added under engine mechanics: decide
once per piece whether captions add meaning (numbers, names, translations,
attributions) or are accessibility subtitles echoing narration — never
both for the same content. Reviewer gate at compose stage compares active
caption text against on-screen text in the same time window.
Both gates are CRITICAL; the publish-stage check was updated to expect
six (was four) resolved atelier findings.
Self-audit of alexandria-fire under the new rules: 4 of 5 scenes share
the candle as their primary visual subject — would now flag CRITICAL
scene_distinctness and require re-planning. (Re-author the piece in a
follow-up; the doctrine is the deliverable here.)
Closes the two biggest gaps from the atelier audit: hero bespoke renders
were skipping every automated quality gate, and the templated→atelier
inversion was advisory rather than enforced.
F6 — _render_via_atelier now runs the full _run_final_review (technical
probe + visual spotcheck + audio spotcheck + duration drift + transcript
comparison) plus two atelier-specific doctrine checks:
- stock_reuse_check: rglobs the bespoke project tree for imports from
the stock creative registry (src/components, src/Explainer, etc.).
Any hit fails the render with status="fail" and names the offending
file — the doctrine bypass is now mechanically caught.
- art_direction_check: requires edit_decisions.bespoke.art_direction.
Missing → warning surfaced in final_review.issues_found; forcing
function for step 1 of skills/meta/bespoke-composition.md.
Verified end-to-end on the compound-snowball proof render: positive case
status=pass, atelier checks clean; negative case (deliberate stock import
+ missing art_direction) returns status=fail with both findings named.
F7 — skills/meta/reviewer.md gets a "Composition Authoring Mode Review"
section that enforces the inversion at every stage:
- proposal: composition_mode must be a presented decision with both
options; hero briefs default to atelier; templated for hero requires
an explicit sanctioned reason.
- scene_plan/edit: any stock cut.type in an atelier piece is CRITICAL;
bespoke.{entry, composition_id, art_direction} required.
- compose: consumes final_review.checks.atelier from F6; mandates a
recorded distinctness review (the human taste call the tool can't
automate — its absence is itself CRITICAL).
- publish: blocks ship on any unresolved atelier finding.
Engine knowledge stays reusable; creative components don't.
Introduce a hand-authored, project-local Remotion render path that bypasses
the cut-schema and the stock scene-type registry, for hero/bespoke videos
that must look distinct from one another.
- video_compose: composition_mode="atelier" (or renderer_family="bespoke")
routes to _render_via_atelier, which renders a project-local entry under
remotion-composer/projects/<slug>/ with an optional per-project public_dir
(skips copying the bloated shared public/). No cut-schema, no stock registry.
- skills/meta/bespoke-composition.md: routing skill — art direction
(visual-style) -> motion principles (Disney 12) -> engine mechanics
(remotion-best-practices + stock components read only as a mechanics codex)
-> atelier render. Doctrine: reuse engine knowledge, never creative components.
- AGENT_GUIDE: "Composition Authoring Mode" (templated vs atelier); default
atelier for hero work; scene-type catalog reframed as a mechanics codex.
- animation-runtime-selector + INDEX: authoring-mode-first pointers.
- base_tool.run_command: decode subprocess output as UTF-8/replace (Windows
cp1252 crashed the reader thread on Remotion's Unicode progress output).
- .gitignore: remotion-composer/projects/ (throwaway bespoke compositions).
The video-reference-analyst capability audit pre-locked Remotion as the
default composition engine ('Remotion is the default ... Never default to
FFmpeg when Remotion is available'). Because the analyst skill runs first
in any reference-led flow, this silently locked the runtime before the user
was ever offered a choice — violating the 'Present Both Composition Runtimes
(HARD RULE)' in AGENT_GUIDE.md, which forbids silently picking a default and
treats Remotion and HyperFrames as parallel, non-ranked runtimes.
It also omitted HyperFrames from the engine list entirely, so it was never
surfaced as an option in the reference-led path.
Update Step 2 (Capability Audit) to:
- add HyperFrames to the audited runtime list
- remove the 'Remotion is the default / preferred' framing
- defer engine selection to the AGENT_GUIDE 'Present Both' gate
- keep FFmpeg scoped to standalone ops, not composition
Closes#60
Adds a transcript_comparison check to VideoCompose._run_final_review
that word-diffs the whisper/whisperx transcript against script.txt and
fails loudly when the TTS engine literally voiced punctuation tokens
(dot, dots, ellipsis, comma, dash, hyphen, period). Chirp3-HD did this
to ellipses in a production run and it slipped past review; now it
cannot.
Also corrects the "tiny background video" gotcha in
skills/core/hyperframes.md. After six renders of blaming HyperFrames
CSS, the real root cause was 640x360 Pexels sources combined with a
fit-and-pad pre-transform — HyperFrames was rendering the letterboxed
input faithfully. Gotcha now walks through the ffprobe diagnostic and
the scale-to-cover fix, and keeps the wrapper-div pattern for the
right reasons (aspect mismatch handling, not framework bug workaround).
Four regression tests cover the new check: punctuation-leak detection,
clean-audio false-positive guard, graceful skip when inputs missing,
and always-present transcript_comparison section.
Surfaces six real traps an agent learns the expensive way: full-frame
videos need BOTH HTML width/height attrs AND `!important` CSS because
the runtime applies inline size from intrinsic dimensions; preview-scrub
matters more than a 60-minute render for visual bugs; legibility comes
from scrims + text-shadow, not from lowering video opacity; downloaded
stock needs dense keyframes (-g 30) before render or parallel capture
times out; --workers 1 is mandatory for video-heavy compositions; the
deterministic-font compiler silently replaces unmapped families like
Space Grotesk, use Outfit/Inter/JetBrains Mono/Poppins instead. Each
gotcha documented with the fix-at-author-time pattern so the next agent
ships a clean first render.
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
Seedance 2.0 is now routed as the top-ranked video generator whenever a
premium gateway is available. Touches the tool layer, scoring engine,
cinematic pipeline, and both skill layers so discovery works from every
entry point.
- tools/video/seedance_video: BETA stability, quality_score=0.95, add
reference_to_video operation plus 9 img + 3 vid + 3 audio ceilings,
fix pre-existing upload_image_fal import
- tools/base_tool: surface optional quality_score / success_rate /
latency fields in get_info so the scorer can read them
- lib/scoring: fix reliability enum-vs-string bug that was pinning every
available tool to 0.0, switch to overlap coefficient so rich best_for
descriptions aren't penalized, add premium-cinematic feature bonus
- pipeline_defs/cinematic + cinematic asset-director: add pixabay_music
and freesound_music, restore pixabay-first music default
- cinematic compose-director: mandatory Remotion preflight at stage entry
- New Layer 3 .agents/skills/seedance-2-0/SKILL.md (8-part prompt
structure, multi-shot, lip-sync, reference-to-video, provider landscape)
- New Layer 2 skills/creative/prompting/seedance-prompting.md
- Update ai-video-gen, video-gen-prompting, AGENT_GUIDE, INDEX to flag
Seedance 2.0 as the preferred premium default and make the skill
discoverable from every routing path
Mirror the official GSAP AI skills (greensock/gsap-skills, MIT) into
.agents/skills/ and add the Layer 2 wiring that makes them discoverable
from a fresh context.
GSAP covers animation needs that Remotion primitives strain at:
per-character text reveals (SplitText), SVG shape morphs (MorphSVG),
curved camera paths (MotionPath), stroke-reveal line drawing (DrawSVG),
layout-to-layout flight (Flip), and custom bezier easings (CustomEase).
Also becomes mandatory day-1 knowledge if we wire in HyperFrames later
(HF uses GSAP timelines as its native animation runtime).
Layer 3 adds (.agents/skills/):
- gsap-core, gsap-timeline, gsap-plugins, gsap-utils
- gsap-react, gsap-performance
- gsap-scrolltrigger, gsap-frameworks (situational)
- gsap/README.md — OpenMontage-specific framing and Remotion-safe usage
Layer 2 wiring (the discovery triggers):
- skills/meta/animation-runtime-selector.md — NEW routing meta-skill.
Decision matrix covering Remotion primitives, GSAP plugins, framer-
motion, Lottie, Manim, D3, TerminalScene. Enforces the "keep it
simple" bias: reach for GSAP only when the plugin genuinely earns
its bundle weight.
- skills/pipelines/explainer/asset-director.md — references GSAP for
kinetic typography, multi-step choreography, SVG line draws.
- skills/pipelines/animation/asset-director.md — references GSAP for
logo morphs, motion paths, FLIP transitions, custom easings.
- skills/pipelines/cinematic/asset-director.md — references GSAP for
cinematic camera moves, per-char title reveals, prestige easings.
- AGENT_GUIDE.md — adds a categorized Layer 3 skills table so a fresh-
context agent can find the right skill by what they're trying to do,
plus a pointer to animation-runtime-selector.md for routing.
Determinism: every GSAP use inside Remotion must drive timeline progress
from useCurrentFrame(), never requestAnimationFrame. Three Remotion-safe
patterns are documented in both the gsap/README and the selector skill.
Attribution: https://github.com/greensock/gsap-skills (MIT).
