Start here. This is the complete operating guide and agent contract for OpenMontage.
For architecture, key files, and conventions see [`PROJECT_CONTEXT.md`](PROJECT_CONTEXT.md).
## What OpenMontage Is
OpenMontage is an instruction-driven video production system. The AI agent IS the intelligence — it reads instructions (pipeline manifests + stage director skills + meta skills) and drives the pipeline using tools.
-> checkpoints (Python utility) -> presents to human for approval
```
**Python = tools + persistence.** No orchestration logic, creative decisions, review logic, or checkpoint policy in Python code. The agent makes those decisions guided by instructions.
Core loop:
1. Select a pipeline.
2. Run preflight.
3. Discover real tools from the registry.
4. Present the user with concepts, tool plan, production plan, and cost.
5. Execute stage by stage with checkpoints.
## Orchestrator
The agent itself orchestrates the production state machine:
This returns every capability grouped by status — how many providers the user has configured vs. how many exist. **Present this to the user as a capability menu**, not as a flat tool list.
You can produce videos now with images + TTS + FFmpeg.
Quick upgrades available — see below.
```
For EACH capability with unavailable providers, read the `install_instructions` field from the menu output and present setup options grouped by effort:
```
QUICK SETUP OPTIONS (1-minute each — set an env var in .env)
Video Generation (0/12 -> unlock the biggest upgrade):
Each unavailable provider lists its own install_instructions.
Read them from the provider_menu output and present grouped by env var.
Example: if 3 tools need FAL_KEY, group them: "FAL_KEY unlocks 3 providers"
Image Generation (1/7 -> more style options):
Same pattern — read install_instructions from each unavailable tool.
Text-to-Speech (1/3):
Same pattern.
LOCAL OPTIONS (free, needs hardware):
Tools with runtime=LOCAL or runtime=LOCAL_GPU — read from the menu.
Already Available:
List what's working. The user should feel good about what they have.
```
**Rules:**
- Do NOT hardcode provider names, API key names, or setup URLs in your prompts.
Read them from the registry's `install_instructions` field on each tool.
- Always show the ratio: "X of Y configured" — this makes breadth visible.
- Group by capability, not by individual tool.
- Show what they CAN do now, then what they COULD unlock.
- If the user declines setup, proceed with the best available path — no nagging.
- If a tool shares an env var with others, group them (read from `dependencies` field).
### Setup Offer Protocol
When tools are `UNAVAILABLE` but can be fixed with simple configuration, **offer the user setup help instead of silently working around the limitation.** Many tools are one env var away from working.
| Fix Complexity | Action |
|----------------|--------|
| **1-minute fix** (env var) | Offer to help configure now — read `install_instructions` from the tool |
| **5-minute fix** (install) | Explain what to install and why — read `install_instructions` from the tool |
| **Complex fix** (GPU, model download) | Note the limitation, explain what it would unlock, move on |
**Rules:**
- Always tell the user what they're missing AND what they'd gain
- Show the cost difference (free local vs. paid API)
- If the user declines setup, proceed with the best available path — no nagging
- Group related fixes (tools sharing the same env var dependency)
### Remotion Rendering (Inside video_compose)
`video_compose` has two render engines. Check which are available:
```bash
python -c "
from tools.tool_registry import registry
registry.discover()
info = registry._tools['video_compose'].get_info()
| **Remotion** | Still images -> animated video, text cards, stat cards, charts, callouts, comparisons, transitions with spring physics | Node.js (`npx`) + `remotion-composer/` project |
**When Remotion is available**, the agent should design production plans around it:
- Explainer videos with `flat-motion-graphics` playbook -> Remotion animated scenes, not Ken Burns
- Data-driven videos -> Remotion stat cards and charts, not static image screenshots
- Any pipeline using still images -> Remotion spring animations, not FFmpeg pan-and-zoom
**When Remotion is NOT available**, `video_compose` falls back to FFmpeg Ken Burns motion on still images. This still works but produces less engaging visuals. Mention this tradeoff in the proposal.
The routing is automatic — the `render` operation in `video_compose` calls `_needs_remotion()` and routes accordingly. But the **agent must know Remotion exists at proposal time** so it can design the visual approach to take advantage of it (animated text cards, component scenes, spring transitions) rather than designing around static images.
## Capability Discovery
OpenMontage uses two layers for capability choice:
- selector tools: capability-level routing such as `tts_selector` and `video_selector`
- provider tools: concrete tools discovered via the registry that call a specific backend
- **analysis** — Transcription, scene detection, frame sampling.
- **avatar** — Talking head and lip sync generation.
- **enhancement** — Upscale, background removal, face enhance, color grading.
Each tool in the registry declares `best_for`, `install_instructions`, `runtime` (LOCAL, API, LOCAL_GPU, HYBRID), and `status`. Read these fields — do not assume tool strengths from memory.
### Selector Pattern
Three selector tools abstract multi-provider capabilities. **Selectors auto-discover providers from the registry.** Adding a new provider tool automatically makes it available through the selector — no selector code changes needed.
| `video_selector` | All tools with `capability="video_generation"` | `registry.get_by_capability("video_generation")` |
Selectors route based on: user preference > availability > discovery order. They adapt input schemas between providers transparently.
## User-Facing Planning Protocol
Before committing to execution, present:
1.`4-5` concept directions when the brief is still open.
2. Recommended pipeline.
3. Recommended tool path.
4. Alternative tool paths that are actually available.
5. Cost estimate and quality tradeoffs.
6. Production plan by stage.
7. Approval gate before asset generation.
If a user prefers a specific vendor and that tool is available, surface it directly. Do not hide provider choice.
## Pipeline Asset Expectations
Each pipeline manifest's `tools_available` field declares what tools a stage can use. Use selectors for multi-provider capabilities — the selector handles routing to whatever is available. Read the pipeline manifest for the authoritative list per stage.
## Stage Agents
Each stage produces one canonical artifact that becomes the contract for the next stage. The stage director skill teaches the agent HOW to produce it.
| Stage | Director Skill | Canonical output | Core quality bar |
