Resolves the "multi-server" open question from the adapter plan.
ComfyUIClient(capability="image"|"video") now resolves its server URL
from COMFYUI_IMAGE_SERVER_URL / COMFYUI_VIDEO_SERVER_URL first, falling
back to the shared COMFYUI_SERVER_URL and then the localhost default —
so comfyui_image and comfyui_video can point at separate ComfyUI
instances (different GPUs, different model sets) with zero extra config
for single-server setups. is_default_url/unavailable_reason() and the
setup_offer metadata account for the override.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the "async generation" open question from the adapter plan.
generate() now watches ComfyUI's websocket events (executing/progress/
execution_error) and reacts immediately instead of sleeping between REST
polls, with an optional on_progress callback that comfyui_video uses to
print step progress on long renders. websocket-client is an optional
import; _wait() falls back to the original poll() loop (with the
remaining time budget, not a fresh one) when it's unavailable or the
connection drops, so resume_prompt_id recovery is unaffected either way.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Non-accelerated local-GPU workflows (e.g. Wan 1.3B at 832x480/81-97
frames) routinely took ~1360-1630s, so the old 900s default false-failed
real renders that were still completing server-side. Timeout is now a
configurable timeout_seconds input (default 3600s), and ComfyUIError
carries the prompt_id on error/timeout so a timed-out-but-still-running
job can be resumed via resume_prompt_id instead of resubmitted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
The env var added complexity without meaningful benefit — explicit model
selection via the "model" input parameter is sufficient for both image and
video TokenHub tools.
- tools/video/hunyuan_cloud_video.py: drop env var fallback from
_resolve_model(), remove mention from install_instructions
- tests/contracts/test_hunyuan_cloud_video.py: remove
test_resolve_model_from_env and test_resolve_model_input_overrides_env
- .env.example: move TENCENT_TOKENHUB_API_KEY to a dedicated "Tencent
Hunyuan TokenHub API" section, drop TENCENT_TOKENHUB_MODEL comment,
broaden description from "video generation" to generic "Tencent Hunyuan
via TokenHub API"
Introduce a new video generation provider backed by the Tencent TokenHub
API (tokenhub.tencentmaas.com), an OpenAI-compatible gateway for Tencent
Hunyuan video models with simple Bearer-token auth.
- Add hunyuan_cloud_video tool (submit → poll → download) supporting
both text-to-video (hy-video-1.5) and image-to-video (yt-video-2.0)
- Add env vars: TENCENT_TOKENHUB_API_KEY, TENCENT_TOKENHUB_MODEL
- Add contract tests for the new tool
- Document setup, API flow, model pricing, and schema constraints in
PROVIDERS.md
- Update provider tables and capability matrix throughout docs
The CI Linux FFmpeg build carried the keyed frame forward without an alpha
plane, so overlay drew opaque green over the background (corner stayed green)
instead of compositing — the E2E test failed there even though it passed on
macOS/Windows.
Force `format=yuva420p` immediately after chromakey so the keyed transparency
always has an explicit alpha plane, and size the background to the frame up
front (color=...:size=WxH, passing the probed width/height into
_process_chromakey) instead of scaling a 1x1 source with scale2ref — dropping
scale2ref also removes the format negotiation that discarded the alpha on some
builds. Output is flattened to yuv420p after the overlay.
Include every request field that can alter Kling video, image, avatar, or lip-sync media in the public idempotency contract. Add a shared regression matrix that detects future cache-key collisions while excluding transport-only controls.
minimax_tokenplan_video does not exist in main branch (added in PR #297,
not yet merged). Use minimax_video which is the existing tool.
Fixes Copilot review comments on fallback_tools reference and
contract test assertion.
Implements the Jimeng/Volcengine part of issue #249, as agreed with
@xucailiang (who is handling the Kling provider separately).
Adds a first-class Jimeng video provider that calls the Volcengine
visual API directly (visual.volcengineapi.com) using HMAC-SHA256 V4
request signing with IAM AK/SK credentials. This is the first provider
in OpenMontage to use V4 signing (all others use Bearer token auth).
API flow: POST CVSync2AsyncSubmitTask -> poll CVSync2AsyncGetResult ->
download video_url.
Features:
- Text-to-video and image-to-video (Jimeng 3.0 Pro)
- Configurable frame count (121=5s, 241=10s at 24fps)
- Aspect ratio selection (16:9, 9:16, 1:1, etc.)
- Seed for reproducibility
- Full V4 HMAC-SHA256 request signing (not Bearer token)
- Error handling with Jimeng code 10000 success convention
- API key redaction in error messages (both env vars, no empty-string bug)
Env vars: VOLC_ACCESSKEY + VOLC_SECRETKEY (IAM AK/SK pair).
Idempotency keys include all output-affecting fields.
Files:
- tools/video/jimeng_video.py — new tool (V4 signing + submit/poll/download)
- tests/contracts/test_jimeng_video.py — 46 contract tests (no AK/SK needed)
- .env.example — VOLC_ACCESSKEY + VOLC_SECRETKEY
- docs/PROVIDERS.md — Volcengine Jimeng provider section
End-to-end tested with real Volcengine IAM credentials: generated a
1920x1088 H.264 5.04s video, ffprobe verified.
Test results:
python -m pytest tests/contracts/test_jimeng_video.py -q # 46 passed
_process_chromakey built the composite background from a 1x1 lavfi color source
and tried to size it with `[0:v]scale=iw:ih`. That scale is a no-op — iw/ih are
the 1x1 source's own dimensions, and there is no cross-reference to the frame.
FFmpeg's overlay then takes the size of its first input (the 1x1 background), so
every processed frame is clipped to a single pixel. The exception fallback never
runs because the primary command exits 0 (a valid 1x1 PNG), and
_reconstruct_video upscales those 1x1 frames — producing a solid-color video
with the keyed subject entirely gone. Total data loss for method="chromakey"
(and method="auto" when it selects chromakey).
Use scale2ref to resize the background to the actual frame dimensions before
overlaying, so the keyed subject is composited at full resolution.
Verified with ffmpeg: a 320x240 green frame with a red subject now produces a
320x240 output with the subject preserved and green replaced by the background,
instead of a 1x1 (then upscaled solid-color) frame.
Review findings from PR #333:
P1: _download_via_uri assumed output_video.uri is always files/<id>.
The API can return a full resource URI or a ready-made
.../files/<id>:download?alt=media download URL, which produced an
invalid poll path with a second :download appended. New
_file_id_from_uri() extracts the bare id from every documented shape;
regression tests cover the full-URL form plus a parametrized matrix of
URI shapes.
P2: docs/PROVIDERS.md still described the Google key as TTS + Imagen
only. The shared-key section now covers gemini_omni_video (model id,
~$0.10/sec pricing table, paid-tier-only, edit-turn billing note), and
the env snippet, provider-to-tool mapping, and capability coverage
tables include the new provider.
Add gemini_omni_video, a native Gemini API provider wrapping
gemini-omni-flash-preview via the Interactions API. Text-to-video,
image/reference-to-video with <FIRST_FRAME>/<IMAGE_REF_N> prompt tags,
and stateful edit_video turns via previous_interaction_id — the only
provider in the fleet that can refine a clip without regenerating it.
Reuses the existing GOOGLE_API_KEY / GEMINI_API_KEY, so one Google key
now unlocks images, TTS, and video.
- New Layer 3 skill .agents/skills/gemini-omni (prompting, edit-loop
rules, tag/timecode syntax, preview limits) sourced from official
Google docs; linked via agent_skills and the AGENT_GUIDE Layer 3 map
- ai-video-gen gains the Gemini API gateway row + editing pointer
- veo_video/sora_video fallback lists and video_selector agent_skills
reference the new provider; quality_score 0.85 with rationale
- Contract tests: registry discovery, selector routing, status from
env keys, uri + inline delivery, edit turns, typed image parts,
store=false editability, cost clamp
COGVIDEO_VARIANTS declares cogvideo-2b i2v=False (it is t2v-only), but
cogvideo_video advertised image_to_video + reference_image unconditionally
and the variant flag was never consulted. An image_to_video brief against
the 2B variant reached the diffusion pipeline and failed opaquely.
- Add is_operation_available(operation) that derives capability from the
variant table (the selector calls it without inputs, so it reports the
DEFAULT variant cogvideo-5b: t2v + i2v both True). This replaces an
implicit unconditional-True.
- Add an execute()-time guard that consults the CALLER's chosen variant
and fails fast with a clear error when it lacks the requested mode
(2B + image_to_video), instead of dropping into generate_local_video.
- Add _variant_for(inputs) helper shared by estimate_runtime / the guard.
Tests pin: the 2B premise (i2v=False), default-variant capability
reporting, fast-fail for 2B+i2v (generation never runs), and that 5B+i2v
still routes through to generate_local_video.
Refs: docs/REVIEW-image-to-video-voice.md §8 #4
Co-Authored-By: Claude <noreply@anthropic.com>
Three routing defects in video_selector, none previously covered by
tests (REVIEW §8 #3, #5, #7); plus the routing-test coverage itself (#10).
#3 Seedance dedup race
tool_by_provider keyed by provider STRING, so two tools legitimately
sharing provider="seedance" (seedance_video=fal, seedance_replicate)
collided — only the first-registered was ever selectable; the other
was invisible to the selector regardless of rank. Key selectable tools
by NAME instead; ranking picks the best of the shared-provider backends.
#5 preferred_provider had no score-gap gate
The selector returned the preferred provider on the first ranking match
no matter how far below the top it scored (the comment claimed "unless
drastically worse" but nothing enforced it). Add a configurable
preferred_provider_gap (default 0.15): honor the preference only when
its best ranked tool is within the gap of the overall top, else yield
to the top-ranked provider.
#7 fallback_tools appended image_selector unconditionally
The motion-required prohibition lived only in director skills, so a
direct caller could silently fall back to an image-only tool for an
image_to_video / reference_to_video brief. Add input-aware
fallback_tools_for(inputs) that drops image_selector for
motion-required operations; keep the static fallback_tools property
(with image_selector) for external consumers / contracts.
#10 routing coverage
First routing tests for video_selector: dedup reachability, the gap
gate (honored / ignored / configurable), motion-aware fallback, and
estimate_cost / estimate_runtime delegation. 13 tests, scoring patched
for determinism so they test routing logic, not the scorer.
Full tools + contracts suite green (638 passed, 6 skipped).
Refs: docs/REVIEW-image-to-video-voice.md §8 #3, #5, #7, #10
Co-Authored-By: Claude <noreply@anthropic.com>
Every premium video provider sets quality_score (seedance 0.95, runway /
higgsfield 0.9) so the scorer ranks them above stock/local options.
grok_video had none, so it was scored only on supports/stability flags
despite shipping native synchronized audio (lip-sync + dialogue + SFX
in a single generation pass) — likely under-ranked.
Set quality_score=0.9, on par with the other native-audio premium
providers. Add a regression pinning the field and its get_info() surface.
Refs: docs/REVIEW-image-to-video-voice.md §8 #6
Co-Authored-By: Claude <noreply@anthropic.com>
The bundled wan22-t2v-4step.json workflow loads the 14B FP8 diffusion
pair (wan2.2_t2v_high/low_noise_14B_fp8_scaled.safetensors), which
produce 16-channel latents, but its VAELoader referenced
wan2.2_vae.safetensors — the WAN 2.2 5B model's VAE, which expects
48-channel latents. Every T2V run therefore failed at VAEDecode with:
Expected tensor to have size 48 at dimension 1, but got size 16
Switch the workflow to wan_2.1_vae.safetensors, matching the 14B
models and the sibling wan22-i2v-4step.json, and update the T2V
required-models list in tools/video/comfyui_video.py to match so
preflight checks for the VAE that is actually used.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
video_compose.get_info() reported render_engines.ffmpeg as always
available, unlike the real availability checks used for remotion and
hyperframes. On a machine without ffmpeg on PATH, preflight would
falsely report ffmpeg as usable, letting render_runtime="ffmpeg" get
locked at proposal time only to fail at compose.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
