17 Commits

Author SHA1 Message Date
calesthio
2702362c24 feat: add production 3D world pipeline 2026-08-13 07:48:38 -07:00
xucailiang
7c5dfdd31a Add official Kling API providers 2026-07-07 14:56:40 +08:00
Remotely Human
fabc0e7088 Replace retired OpenAI image models with gpt-image-2 (DALL-E shut down 2026-05-12)
- openai_image: enum narrowed to ["gpt-image-2"]; DALL-E code path, sizes
  (1024x1792/1792x1024) and qualities (standard/hd) removed; estimate_cost
  updated to real GPT Image 2 pricing ($0.006/$0.053/$0.211 at 1024x1024)
- image_gen (deprecated but callable): default dall-e-3 -> gpt-image-2; drop
  response_format="b64_json", which GPT image models reject; update pricing
- docs/skills sweep: scrub retired-model references from README (en/zh),
  .env.example, docs/PROVIDERS.md (incl. pricing table + deprecation note),
  docs/ARCHITECTURE.md, AGENT_GUIDE.md, skills/creative/*, 14 pipeline
  director skills, and tests/qa/QA_PLAN.md

Source: https://developers.openai.com/api/docs/deprecations

proofread by fable 5 + codex
2026-07-02 11:36:31 +08:00
calesthio
7837bfee4a ink-theater: clear code-review findings — license-clean mocap + font, fix stale clip refs
Licensing (finding 1):
- Mocap library is now 100% CMU-sourced (free for any use). Replaced the 3
  Meta/FAIR clips with CMU equivalents: wave=141_16, shuffle=77_29 (creep);
  dropped the un-CMU "dab", added twist=141_12. Deleted the 4 committed FAIR
  BVHs (dab/jumping/wave_hello/zombie).
- Ship Patrick Hand's SIL OFL license (assets/OFL.txt) beside the bundled font
  — OFL permits embedding/redistribution; this is the required attribution.
- Add THIRD_PARTY_NOTICES.md (Patrick Hand OFL + CMU) and rewrite mocap/NOTE.md
  to drop the "verify before commercial use" caveat (no longer applicable).

Stale clip names (finding 2):
- Update the /ink-art command files (.claude/.github/.cursor/.codex),
  character-design-director, and the creative skills to the current catalog
  (wave/twist/…), and point to add-motion.mjs.
- ink-puppet.js: console.warn on an unknown clip name instead of silent dead-time.

Examples (finding 3):
- Remove broken standalone reel.html/momentum.html (they referenced a missing
  ink-theater.js and momentum shipped the subset font the fix warns against).
- Keep mocap-figure/ as the canonical self-contained, lintable example; refresh
  its bundled clips.js/ink-puppet.js; add examples/README.md with the lint path.
2026-07-01 04:03:19 -07:00
calesthio
a80bb6e7e1 feat: /animated-drawing character-source chooser (ask first; generate/upload/stock/doodle-ify)
Adds an explicit 'choose a character source' step so the agent never silently
reuses a bundled character (the 'same mascot every video' problem). Options:
user uploads a drawing; user uploads a photo -> doodle-ify (img2img) -> rig;
generate a fresh doodle (FLUX/Imagen, recommended default); or stock
(pixabay/pexels, hit-or-miss). Bundled chars are demo-only.
2026-07-01 02:52:44 -07:00
calesthio
0b9cde8458 feat: complete Ink Theater motion system — action library + self-extending converter + balloon primitive + audit doc fixes
Ships Phases 1-3 so the feature isn't left hanging:
- Phase 1: 12-action mocap library (walk/run/jump/climb/march/kick/sit/wave/dab/
  shuffle/dance_spin/dance_glide) from CMU + fair1, with mocap/catalog.json.
  Fixes the '4 repeated moves' repetition problem.
- Phase 2: bvh2clip.mjs is skeleton-agnostic (fair1/CMU/Mixamo joint aliases) +
  frame cap + --name; add-motion.mjs fetches/converts/rebundles any BVH by CMU
  id / URL / path — self-extending, no code changes.
- Phase 3: skill guidance to read catalog.json + choreograph variedly (never
  loop a clip), add-motion, and the balloon primitive.
- Engine: InkTheater.balloon() speech-balloon primitive (both audit videos
  hand-built it).
- Audit fixes: /animated-drawing skill gains a motion->retarget mapping table
  (2 motions crashed on the wrong config) + a 'Compositing into HyperFrames'
  section (transparent output, GIF-freezes->VP9-alpha, the linter video
  contract) + character-variety guidance (animate the user's/a generated
  drawing, not a bundled char) + pipeline-exempt notes.
Example refreshed to the new library + full Patrick Hand TTF.
2026-06-30 21:13:38 -07:00
calesthio
2a7f130bee fix: correct the font gotcha — subset trap, not SVG-vs-HTML; bundle Patrick Hand TTF
Audit finding from video production: handwriting rendered as serif everywhere
because the woff2 we downloaded was a Google-Fonts css2 subset MISSING basic-latin
(ASCII) glyphs — not because 'SVG text can't use webfonts' (the old README claim
was a misdiagnosis; prior demos were silently serif). Fix: embed the FULL font.
Bundles ink-theater/assets/patrickhand.ttf and corrects README + skill guidance
(HTML overlay divs + full TTF).
2026-06-30 20:40:12 -07:00
calesthio
ccbce8bb1e feat: discoverability + cross-agent slash commands for Ink Theater / Animated Drawing
Makes the doodle-animation capability findable instead of hidden.

Routing (OpenMontage-internal):
- AGENT_GUIDE: Style Playbooks gets an ink-sketch row + a 'hand-drawn doodle'
  routing note (it's a style + engine on the animation / character-animation
  pipelines, NOT a new pipeline).
- animation/idea-director + character-animation/character-design-director point
  to the Ink Theater / Ink Puppet skills for hand-drawn character briefs.

Cross-agent slash commands (/ink-art, /animated-drawing) for Claude Code,
Copilot, Cursor, and Codex — thin pointers to the canonical skills so they stay
in sync. /ink-art = create a vector doodle from scratch (Ink Theater + Ink
Puppet mocap); /animated-drawing = animate a SUPPLIED drawing via Meta
AnimatedDrawings (raster). New skills/creative/animated-drawing.md documents the
Path A setup/run/limits. Codex prompts are user-home only — .codex/prompts/README
explains the copy/symlink step.

  .claude/commands/*.md            (Claude Code)
  .github/prompts/*.prompt.md      (Copilot)
  .cursor/commands/*.md            (Cursor)
  .codex/prompts/*.md + README     (Codex; copy to ~/.codex/prompts)
2026-06-30 20:13:48 -07:00
calesthio
0aa986e61b feat: Ink Theater — deterministic hand-drawn moving-art engine + creative skill
Reusable engine (ink-theater/ink-theater.js, global InkTheater) for hand-drawn
ink-on-white animated 'moving art', authored for HyperFrames (atelier). Five
seek-safe primitives: variable-width brush strokes (inkPath/inkRibbon),
timeline-driven boil, closed-form damped-spring eases, FABRIK 2D IK + riggable
mascot, and a parametric contraption grammar (crank/gauge/hopper/slot/lever/box).

Adds skills/creative/ink-theater.md (metaphor method + mined archetypes + color
grammar) and README with the HyperFrames font gotcha (webfonts apply to HTML
divs, not SVG <text>). Two reference builds under ink-theater/examples/.

Grounded in deep-research on vector/physics/metaphor foundations (FABRIK,
bounded-biharmonic/ARAP, closed-form springs, LLM metaphor+layout planning).
2026-06-30 17:03:20 -07:00
calesthio
fdd6457fed docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise
Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2).
The paper proves prompts structured around five aspects (Subject /
Subject Motion / Scene / Spatial Framing / Camera) unlock controllable
cinematography in fine-tuned video generation models. Off-the-shelf VLMs
already nail subject and scene; the gains live in motion, spatial, and
camera, which prompts routinely omit.

Universal layer (skills/creative/video-gen-prompting.md, +125 lines):
- 5-aspect prompt skeleton replaces flat formula
- Camera movements regrouped (translation / rotation / lens-only) with
  dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations
- New primitive tables: camera height, camera angle, POV, lens
  distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking),
  playback speed (6 modes), subject transitions
- Order-matters and self-contained-prompt rules
- Identity anchoring rule for multi-shot
- Strict static-shot rule, anti-subjective callout, overlays-not-depth
- Per-model word-count guidance

Per-model guides (sora, veo, hunyuan, ltx, seedance):
- Add the primitives each model honors literally
- Word-count sweet spots per model
- Strengthen seedance verbatim-identity and subject-transition guidance

Pipeline directors (cinematic / explainer / animation scene-director,
cinematic / explainer asset-director):
- 5-aspect scene-plan checklist (per-pipeline adapted)
- Overlays-not-depth callout
- Pre / critique / post self-review loop for generation prompts

Reviewer (skills/meta/reviewer.md):
- CHAI critique-quality rules: accurate / complete / constructive
- Critical findings now require a proposed_fix

Storytelling, cinematic, broll, video-reference-analyst:
- Anti-subjective rule (replace mood adjectives with visual causes)
- Camera-intent-per-beat for script writers
- POV column in stock-footage query templates
- 5-aspect structured output mandatory for reference-video analysis

skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
2026-04-28 08:11:31 -07:00
calesthio
b4f7ec4eee hyperframes: add HTML/CSS/GSAP as a parallel composition runtime
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
2026-04-18 18:37:00 -07:00
calesthio
16791a3a80 video-gen: adopt Seedance 2.0 as preferred premium default
Seedance 2.0 is now routed as the top-ranked video generator whenever a
premium gateway is available. Touches the tool layer, scoring engine,
cinematic pipeline, and both skill layers so discovery works from every
entry point.

- tools/video/seedance_video: BETA stability, quality_score=0.95, add
  reference_to_video operation plus 9 img + 3 vid + 3 audio ceilings,
  fix pre-existing upload_image_fal import
- tools/base_tool: surface optional quality_score / success_rate /
  latency fields in get_info so the scorer can read them
- lib/scoring: fix reliability enum-vs-string bug that was pinning every
  available tool to 0.0, switch to overlap coefficient so rich best_for
  descriptions aren't penalized, add premium-cinematic feature bonus
- pipeline_defs/cinematic + cinematic asset-director: add pixabay_music
  and freesound_music, restore pixabay-first music default
- cinematic compose-director: mandatory Remotion preflight at stage entry
- New Layer 3 .agents/skills/seedance-2-0/SKILL.md (8-part prompt
  structure, multi-shot, lip-sync, reference-to-video, provider landscape)
- New Layer 2 skills/creative/prompting/seedance-prompting.md
- Update ai-video-gen, video-gen-prompting, AGENT_GUIDE, INDEX to flag
  Seedance 2.0 as the preferred premium default and make the skill
  discoverable from every routing path
2026-04-17 21:51:54 -07:00
calesthio
1b7e13d24b Fix Remotion-first rendering docs and post-render verification gaps
Compose-director had contradictory instructions: Step 2 described Remotion
captions/audio, but Steps 5/5b gave detailed FFmpeg code that agents followed
instead. This caused three failures in production: FFmpeg subtitles instead of
Remotion CaptionOverlay, missing audio (mixed externally but never embedded in
Remotion props), and skipped audio verification in post-render review.

Changes:
- compose-director: Remotion is now DEFAULT for audio, captions, text overlays;
  FFmpeg is labeled FALLBACK only. Post-render review has mandatory ffprobe gate
  and audio transcription with explicit stop conditions.
- remotion.md: routing table updated (captions/audio → Remotion), added universal
  Post-Render Verification Protocol for all pipelines (only 2/10 had one).
- scene-director, asset-director: added pitfall for AI-generated text in CTA
  screens — must use Remotion text_card for verbatim text.
- image-provider-usage: added Recraft V4 caveat (style param causes 422 on fal.ai).
- recraft_image.py: documented the style parameter 422 issue inline.
2026-04-06 08:21:48 -07:00
calesthio
7ca04e66d8 Add Grok media providers and improve selector routing 2026-04-05 15:31:37 -07:00
calesthio
fd4c58f30b Refine visual identity guidance across pipelines 2026-04-03 10:13:39 -07:00
calesthio
2cd36fa8e0 Implementation spec: governance, decision intelligence, theme system, and E2E bug fixes
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.

Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage

Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
  dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
  custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions

Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
  options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
2026-04-03 09:35:09 -07:00
calesthio
a3e735cc7a Initial release — OpenMontage: the first open-source agentic video production system
11 production pipelines, 47 tools, 124 agent skills.
Supports cloud APIs (fal.ai, OpenAI, ElevenLabs, Suno, HeyGen, Runway) and
free local providers (diffusers, Piper TTS, WAN 2.1, Hunyuan, CogVideo).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-29 08:25:17 -07:00