ACE-Step v1's node-pack fragmentation turns out to be moot: ComfyUI ships
TextEncodeAceStepAudio/EmptyAceStepLatentAudio as native core nodes
(comfy_extras/nodes_ace.py), not a third-party pack, and Comfy-Org's own
workflow_templates repo has an official ACE-Step-v1 template built from
those plus long-stable core nodes. tools/_comfyui/workflows/ace-step-1-t2a.json
was built by cross-checking every node's class_type and input names against
ComfyUI's own source (nodes_ace.py, nodes_audio.py, nodes_latent.py,
nodes.py) rather than trusting the UI-format export directly.
comfyui_music now defaults to this bundled workflow: prompt maps to
ACE-Step's tags field (matching suno_music's "prompt = music description"
convention), lyrics/duration_seconds/steps/cfg/lyrics_strength/seed are all
patchable, and missing ace_step_v1_3.5b.safetensors surfaces through the
same missing_models contract as image/video. workflow_json/workflow_path +
output_node remains available for ACE-Step 1.5, other node packs, or
different audio models entirely.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the "music generation" open question from the adapter plan.
Unlike comfyui_image/comfyui_video there is no bundled workflow: ACE-Step's
ComfyUI node interface isn't standardized across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so instead of
picking one pack and breaking for everyone else, comfyui_music always
requires a caller-supplied workflow_json/workflow_path + output_node --
the same override contract image/video offer as an alternative, just
mandatory here. prompt is provenance-only, never injected into the graph.
Routed through the existing registry.get_by_capability("music_generation")
path alongside suno_music/music_gen -- no dedicated selector needed.
ComfyUIClient.generate() now also reads the "audio" output key (what
ComfyUI's native SaveAudio node writes), and gets timeout/resume/websocket-
wait/multi-server support for free via the shared client. Duration is a
best-effort ffprobe probe of the downloaded file since a custom workflow
gives no other way to know it ahead of time.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the "multi-server" open question from the adapter plan.
ComfyUIClient(capability="image"|"video") now resolves its server URL
from COMFYUI_IMAGE_SERVER_URL / COMFYUI_VIDEO_SERVER_URL first, falling
back to the shared COMFYUI_SERVER_URL and then the localhost default —
so comfyui_image and comfyui_video can point at separate ComfyUI
instances (different GPUs, different model sets) with zero extra config
for single-server setups. is_default_url/unavailable_reason() and the
setup_offer metadata account for the override.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the "async generation" open question from the adapter plan.
generate() now watches ComfyUI's websocket events (executing/progress/
execution_error) and reacts immediately instead of sleeping between REST
polls, with an optional on_progress callback that comfyui_video uses to
print step progress on long renders. websocket-client is an optional
import; _wait() falls back to the original poll() loop (with the
remaining time budget, not a fresh one) when it's unavailable or the
connection drops, so resume_prompt_id recovery is unaffected either way.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Non-accelerated local-GPU workflows (e.g. Wan 1.3B at 832x480/81-97
frames) routinely took ~1360-1630s, so the old 900s default false-failed
real renders that were still completing server-side. Timeout is now a
configurable timeout_seconds input (default 3600s), and ComfyUIError
carries the prompt_id on error/timeout so a timed-out-but-still-running
job can be resumed via resume_prompt_id instead of resubmitted.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Require 'model' in the input schema and accept 'model_id' as a
selector-compatible alias (tts_selector exposes model_id); add a
selector-routing regression test
- Document s2.1-pro-free as promotional (free through end of July 2026,
Fair Use, no SLA, possible request retention, commercial-use
restrictions) in PROVIDERS.md and the Layer 3 skill; estimate_cost()
falls back to the paid s2.1-pro rate after the promo window
- Normalize voice_id/reference_id and model_id/model aliases before
computing the idempotency key, and include all output-affecting inputs
(bitrate, sample_rate, temperature, top_p, repetition_penalty, latency,
prosody, normalize, chunk_length) with API defaults applied
- Declare env:FISH_AUDIO_API_KEY in dependencies so registry metadata
reports the requirement
- Add fish_audio to the TTS provider set in the phase3 registry contract
test
- .env.example: replace removed speech-1.x mention with the actual
supported backends (s1 / s2-pro / s2.1-pro)
- docs/PROVIDERS.md: add fish.audio section (setup, backend models,
per-byte pricing incl. free s2.1-pro-free tier) plus entries in the
env var summary, Provider-to-Tool Mapping, and Capability Coverage
- skills/INDEX.md: list fish-audio-tts in the TTS & Audio Layer 3 row
Add an Azure AI Speech transcription tool. It is opt-in: when
AZURE_SPEECH_KEY is configured the agent may prefer it for cloud STT,
while the local faster-whisper `transcriber` stays the default offline
path. Shared pipeline manifests are intentionally left unchanged, so no
default provider selection is altered for existing users.
- tools/analysis/azure_stt.py: new `azure_stt` tool (capability=analysis,
provider=azure) calling the Fast Transcription REST API. The local file
is uploaded via multipart and transcribed synchronously with word-level
timestamps and optional diarization — no Blob storage or async polling.
Output schema mirrors `transcriber` exactly, so it is a drop-in for
`subtitle_gen` and other transcript consumers. Follows the existing
provider-tool conventions (env-var status check, `_transcribe` helper,
cost_usd/model on the result, fallback="transcriber").
- Auto-discovered by the registry; no registry or selector changes.
- tests/tools/test_azure_stt.py: contract, discovery, status, response
mapping, execute guardrails, and a mocked-network success path (no live
API calls).
- .agents/skills + .claude/skills: azure-speech-to-text Layer-3 skill.
- docs/PROVIDERS.md: Azure AI Speech setup, API notes, and pricing.
- .env.example, skills/INDEX.md, AGENT_GUIDE.md: document the optional
cloud STT path alongside the default whisper transcriber.
Implements the Jimeng/Volcengine part of issue #249, as agreed with
@xucailiang (who is handling the Kling provider separately).
Adds a first-class Jimeng video provider that calls the Volcengine
visual API directly (visual.volcengineapi.com) using HMAC-SHA256 V4
request signing with IAM AK/SK credentials. This is the first provider
in OpenMontage to use V4 signing (all others use Bearer token auth).
API flow: POST CVSync2AsyncSubmitTask -> poll CVSync2AsyncGetResult ->
download video_url.
Features:
- Text-to-video and image-to-video (Jimeng 3.0 Pro)
- Configurable frame count (121=5s, 241=10s at 24fps)
- Aspect ratio selection (16:9, 9:16, 1:1, etc.)
- Seed for reproducibility
- Full V4 HMAC-SHA256 request signing (not Bearer token)
- Error handling with Jimeng code 10000 success convention
- API key redaction in error messages (both env vars, no empty-string bug)
Env vars: VOLC_ACCESSKEY + VOLC_SECRETKEY (IAM AK/SK pair).
Idempotency keys include all output-affecting fields.
Files:
- tools/video/jimeng_video.py — new tool (V4 signing + submit/poll/download)
- tests/contracts/test_jimeng_video.py — 46 contract tests (no AK/SK needed)
- .env.example — VOLC_ACCESSKEY + VOLC_SECRETKEY
- docs/PROVIDERS.md — Volcengine Jimeng provider section
End-to-end tested with real Volcengine IAM credentials: generated a
1920x1088 H.264 5.04s video, ffprobe verified.
Test results:
python -m pytest tests/contracts/test_jimeng_video.py -q # 46 passed
Review findings from PR #333:
P1: _download_via_uri assumed output_video.uri is always files/<id>.
The API can return a full resource URI or a ready-made
.../files/<id>:download?alt=media download URL, which produced an
invalid poll path with a second :download appended. New
_file_id_from_uri() extracts the bare id from every documented shape;
regression tests cover the full-URL form plus a parametrized matrix of
URI shapes.
P2: docs/PROVIDERS.md still described the Google key as TTS + Imagen
only. The shared-key section now covers gemini_omni_video (model id,
~$0.10/sec pricing table, paid-tier-only, edit-turn billing note), and
the env snippet, provider-to-tool mapping, and capability coverage
tables include the new provider.
- README 'Watch It Happen' section with four staged screenshots (fictional
demo productions, generated placeholder art — no private project content)
+ governance section notes enforced approval gates
- scripts/backlot_screenshot_stage.py: stages 4 demo projects in 4 pipeline
states against OPENMONTAGE_PROJECTS_DIR and captures the shots with
Playwright — reproducible README imagery
- lib/paths.py: OPENMONTAGE_PROJECTS_DIR env override (checkpoints, events,
and the board all follow one root); backlot/state.py now imports it
(was still defining its own copy — the exact drift the review warned of)
- undeclared-stage checkpoints (e.g. a legacy 'idea' run under a cinematic
manifest) now slot into their canonical rail position instead of dangling
after publish, drawn dashed + italic with an 'unlisted' note
Removed comfyui_music and its workflow. The ACE-Step model runs in
ComfyUI but the node class names differ across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so a
bundled workflow would break for most users.
Documented the reasoning in the plan doc and listed it as an open
question for future work. Users with ACE-Step working can still use
the workflow_json override on any tool.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Adds three new BaseTool providers that delegate GPU work to a running
ComfyUI server via its REST API. This avoids the need to install
PyTorch/diffusers directly, which is critical on hardware where the
ecosystem hasn't caught up (e.g. NVIDIA Blackwell / DGX Spark, aarch64
+ CUDA 13.0).
New files:
- tools/_comfyui/client.py — shared REST client (submit/poll/download)
- tools/_comfyui/workflows/ — 4 bundled workflow templates
- tools/graphics/comfyui_image.py — FLUX 2 Dev NVFP4 text-to-image
- tools/video/comfyui_video.py — WAN 2.2 14B t2v + i2v (4-step LightX2V)
- tools/audio/comfyui_music.py — ACE-Step 3.5B music generation
- tests/contracts/test_comfyui_tools.py — 41 contract tests
- docs/comfyui-adapter-plan.md — design document
Zero changes to existing tools, selectors, registry, or pipelines.
Tools are auto-discovered and selectors pick them up via capability match.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
