mirror of
https://github.com/calesthio/OpenMontage.git
synced 2026-08-17 05:56:29 +08:00
Merge remote-tracking branch 'origin/main' into codex/add-gemini-seedance-minimax-models
This commit is contained in:
calesthio
45
README.md
45
README.md
@@ -87,22 +87,22 @@ Turn your AI coding assistant into a full video production studio. Describe what
|
||||
> **"THE LAST BANANA"** — a 60-second Pixar-style animated short about a lonely banana who finds friendship with a kiwi. 6 Kling v3-generated motion clips (via fal.ai), Google Chirp3-HD narration, royalty-free piano music, TikTok-style word-level captions, and Remotion composition. Total cost: **$1.33**.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/e03b5d1f-1199-4093-9f31-a43aa9da2c68" width="100%" controls></video>
|
||||
<video src="https://github.com/user-attachments/assets/88962725-97a0-4aac-a08e-34aaa9d8bb92" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"The Library at Alexandria"** — a 70-second history elegy on what humanity lost in a single night. Five hand-authored scenes — an illuminated manuscript page, cascading scroll-tags, a Burning Counter ticking 700,000 → 0 inside a candle's flame, a charred vellum fragment with surviving Greek, and an empty void — set to OpenAI 'ash' narration and a free Pixabay strings score. Total cost: **$0.02**. Built through OpenMontage's atelier (bespoke) composition mode — every scene crafted from scratch, no shared components.
|
||||
> **"Reimagine Your Universe"** — a 50-second vertical transformation film in which one visual idea moves across objects, eras, materials, and scale. Five generated motion scenes, sparse Google Chirp narration, a Pixabay score, and a bespoke HyperFrames composition turn separate clips into one authored cinematic journey. Total cost: **about $4**.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/8a6d2cc3-7ad2-46f5-922f-a8e3e5848d9f" width="100%" controls></video>
|
||||
<video src="https://github.com/user-attachments/assets/c947070c-95ee-4d73-8d76-0bd3dc4826eb" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"VOID — Neural Interface"** — a product ad produced with just one API key (OpenAI). 4 AI-generated images (gpt-image-1), TTS narration, auto-sourced royalty-free music, word-level subtitles via WhisperX, and Remotion data visualizations. Total cost: **$0.69**. Zero manual asset work.
|
||||
> **"Products Come to Life"** — a 60-second product film built from approved hero stills. Five hard-surface products separate into their own engineering and reassemble, with each still pinned as the first and last frame so the model invents motion without losing product identity. Image-to-video generation, bespoke sound, narration, and a custom composition complete the film.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/3c5d7122-7198-43e2-a97d-ed27558dd324" width="100%" controls></video>
|
||||
<video src="https://github.com/user-attachments/assets/6815c2d2-17a3-4057-b9a0-893fc9c05bef" width="100%" controls></video>
|
||||
</div>
|
||||
|
||||
> **"Afternoon in Candyland"** — a Ghibli-style anime animation. A little girl's whimsical afternoon adventure through candy gates, gumdrop rivers, and lollipop gardens. 12 FLUX-generated images with multi-image crossfade, cinematic camera motion (zoom, pan, Ken Burns), sparkle/petal/firefly particle overlays, and ambient music with auto-detected energy offset. Total cost: **$0.15**. No video generation, no manual editing.
|
||||
> **"Imagine the Possibilities with OpenMontage"** — seven generated worlds collected into one music-only showcase. Three image models supply campaign, fashion, and miniature-world artwork; four video models expand the journey through architecture, material transformation, a living greenhouse, and a creature encounter. OpenMontage animates the stills, edits the motion, unifies the soundtrack, and closes with Monty the Clapper. Source generation cost: **about $5**.
|
||||
|
||||
<div align="center">
|
||||
<video src="https://github.com/user-attachments/assets/e8dc5e32-5c70-46de-bd52-eef887719d13" width="100%" controls></video>
|
||||
@@ -385,8 +385,9 @@ Most "free AI video" stacks quietly mean "animate still images." OpenMontage can
|
||||
|
||||
Edit your own talking-head footage. Generate a fully animated explainer from scratch. Cut a 2-hour podcast into a dozen social clips. Translate and dub your content into 10 languages. Build a cinematic brand teaser from stock footage and AI-generated scenes. **If a production team can make it, OpenMontage can orchestrate it.**
|
||||
|
||||
- **12 production pipelines** — explainers, talking heads, screen demos, cinematic trailers, animations, podcasts, localization, documentary montages, and more
|
||||
- **10+ production pipelines** — explainers, talking heads, screen demos, cinematic trailers, animations, podcasts, localization, documentary montages, character animation, and more
|
||||
- **100+ production tools** — spanning video generation, image creation, text-to-speech, music, audio mixing, subtitles, enhancement, and analysis
|
||||
- **60+ provider integrations** — cloud APIs, local models, stock libraries, open archives, and production runtimes behind one scored selection layer
|
||||
- **700+ agent skill and production-knowledge files** — pipeline directors, creative techniques, quality checklists, and deep technology knowledge packs that teach the agent how to use every tool like an expert
|
||||
- **Reference-driven creation** — paste a video you like and the agent turns it into a grounded, differentiated production plan instead of forcing you to invent the perfect prompt from scratch
|
||||
- **Real-footage documentary creation without paid video models** — build actual edited videos from free/open motion footage and archival sources, not just Ken Burns over images
|
||||
@@ -444,10 +445,10 @@ Final video output -- only if self-review passes
|
||||
|
||||
```
|
||||
OpenMontage/
|
||||
├── tools/ # 100+ Python tools (the agent's hands)
|
||||
│ ├── video/ # 13 video gen tools + compose, stitch, trim
|
||||
│ ├── audio/ # 4 TTS providers + Suno/ElevenLabs music, mixing, enhancement
|
||||
│ ├── graphics/ # 9 image/graphics generation tools + diagrams, code snippets, math
|
||||
├── tools/ # 100+ registered production tools (the agent's hands)
|
||||
│ ├── video/ # 20+ generation providers + compose, stitch, trim
|
||||
│ ├── audio/ # 10+ speech providers + music, mixing, enhancement
|
||||
│ ├── graphics/ # 15+ image providers + diagrams, code snippets, math
|
||||
│ ├── enhancement/ # Upscale, bg remove, face enhance, color grade
|
||||
│ ├── analysis/ # Transcription, scene detect, frame sampling
|
||||
│ ├── avatar/ # Talking head, lip sync
|
||||
@@ -460,7 +461,7 @@ OpenMontage/
|
||||
│ ├── core/ # Core tool skills
|
||||
│ └── meta/ # Reviewer, checkpoint protocol
|
||||
│
|
||||
├── schemas/ # 15 JSON Schemas (contract validation)
|
||||
├── schemas/ # 20+ JSON Schemas (contract validation)
|
||||
├── styles/ # Visual style playbooks (YAML)
|
||||
├── remotion-composer/ # React/Remotion video composition engine
|
||||
├── lib/ # Core infrastructure (config, checkpoints, pipeline loader)
|
||||
@@ -484,20 +485,23 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
|
||||
> **Full setup guide with pricing and free tiers:** [`docs/PROVIDERS.md`](docs/PROVIDERS.md)
|
||||
|
||||
<details>
|
||||
<summary><strong>Video Generation — 16 providers</strong></summary>
|
||||
<summary><strong>Video Generation — 20+ providers</strong></summary>
|
||||
|
||||
| Provider | Type | Notes |
|
||||
|----------|------|-------|
|
||||
| **Kling (fal.ai)** | Cloud API | High quality, fast via fal.ai gateway |
|
||||
| **Kling Official** | Cloud API | Official direct API with separate `kling_official` provider |
|
||||
| **Atlas Cloud** | Cloud API | Unified gateway for Seedance, MiniMax, Hunyuan, and other multimodal models |
|
||||
| **Seedance 2.0 (Volcengine Ark)** | Cloud API | Official direct API with separate `seedance_ark` provider |
|
||||
| **Seedance 2.5 / 2.0** | Cloud API | Text, image, and reference-driven video workflows through supported gateways |
|
||||
| **Gemini Omni Flash** | Cloud API | Conversational multimodal video generation and editing |
|
||||
| **Runway Gen-4** | Cloud API | Cinematic quality, Gen-3 Alpha Turbo / Gen-4 Turbo / Gen-4 Aleph |
|
||||
| **Google Veo 3** | Cloud API | Long-form, cinematic. Via fal.ai or HeyGen. |
|
||||
| **Google Veo 3.1** | Cloud API | Premium cinematic video via Google GenAI or fal.ai |
|
||||
| **Grok Imagine Video** | Cloud API | Strong reference-image video and xAI-native short-form generation |
|
||||
| **Higgsfield** | Cloud API | Multi-model orchestrator with Soul ID for character consistency |
|
||||
| **MiniMax** | Cloud API | Cost-effective |
|
||||
| **MiniMax / H3** | Cloud API | Cost-effective generation, including text, image, and reference-driven H3 workflows |
|
||||
| **HeyGen** | Cloud API | Multi-model gateway |
|
||||
| **WAN 2.1** | Local GPU | Free, 1.3B and 14B variants |
|
||||
| **WAN 2.1 / 2.2** | Local GPU | Free local variants plus accelerated ComfyUI workflows |
|
||||
| **Hunyuan** | Local GPU | Free, high quality |
|
||||
| **CogVideo** | Local GPU | Free, 2B and 5B variants |
|
||||
| **LTX-Video** | Local GPU / Modal | Free locally, or self-hosted cloud |
|
||||
@@ -508,7 +512,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Image Generation — 11 tools/providers</strong></summary>
|
||||
<summary><strong>Image Generation — 15+ providers</strong></summary>
|
||||
|
||||
| Provider | Type | Notes |
|
||||
|----------|------|-------|
|
||||
@@ -516,6 +520,9 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
|
||||
| **Google Imagen** | Cloud API | Imagen 4 — high-quality, multiple aspect ratios |
|
||||
| **Grok Imagine Image** | Cloud API | Strong image edits, style transfer, and multi-image compositing |
|
||||
| **GPT Image 2** | Cloud API | OpenAI's image model |
|
||||
| **Seedream 5.0** | Cloud API | High-fidelity text-to-image and image editing through supported gateways |
|
||||
| **Nano Banana 2** | Cloud API | Multimodal image generation and editing |
|
||||
| **Atlas Cloud** | Cloud API | Unified access to multiple image-generation model families |
|
||||
| **Recraft** | Cloud API | Design-focused generation |
|
||||
| **Kling Official** | Cloud API | Official direct API for Kling image generation and reference workflows |
|
||||
| **Local Diffusion** | Local GPU | Stable Diffusion, free |
|
||||
@@ -527,7 +534,7 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
|
||||
</details>
|
||||
|
||||
<details>
|
||||
<summary><strong>Text-to-Speech — 5 providers</strong></summary>
|
||||
<summary><strong>Text-to-Speech — 10+ providers</strong></summary>
|
||||
|
||||
| Provider | Type | Notes |
|
||||
|----------|------|-------|
|
||||
@@ -536,6 +543,8 @@ Each tool declares which Layer 3 skills it relies on. The agent reads Layer 1 to
|
||||
| **Kling Official TTS** | Cloud API | Official Kling narration when a `voice_id` is known |
|
||||
| **OpenAI TTS** | Cloud API | Fast, affordable |
|
||||
| **Piper** | Local | Completely free, offline |
|
||||
| **Azure Speech** | Cloud API | Fast multilingual speech services |
|
||||
| **DashScope / Doubao / Fish Audio** | Cloud API | Additional multilingual and expressive voice options |
|
||||
|
||||
</details>
|
||||
|
||||
|
||||
Reference in New Issue
Block a user