ACE-Step v1's node-pack fragmentation turns out to be moot: ComfyUI ships
TextEncodeAceStepAudio/EmptyAceStepLatentAudio as native core nodes
(comfy_extras/nodes_ace.py), not a third-party pack, and Comfy-Org's own
workflow_templates repo has an official ACE-Step-v1 template built from
those plus long-stable core nodes. tools/_comfyui/workflows/ace-step-1-t2a.json
was built by cross-checking every node's class_type and input names against
ComfyUI's own source (nodes_ace.py, nodes_audio.py, nodes_latent.py,
nodes.py) rather than trusting the UI-format export directly.
comfyui_music now defaults to this bundled workflow: prompt maps to
ACE-Step's tags field (matching suno_music's "prompt = music description"
convention), lyrics/duration_seconds/steps/cfg/lyrics_strength/seed are all
patchable, and missing ace_step_v1_3.5b.safetensors surfaces through the
same missing_models contract as image/video. workflow_json/workflow_path +
output_node remains available for ACE-Step 1.5, other node packs, or
different audio models entirely.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Resolves the "music generation" open question from the adapter plan.
Unlike comfyui_image/comfyui_video there is no bundled workflow: ACE-Step's
ComfyUI node interface isn't standardized across custom node packs
(AceStepModelLoader vs native TextEncodeAceStepAudio, etc.), so instead of
picking one pack and breaking for everyone else, comfyui_music always
requires a caller-supplied workflow_json/workflow_path + output_node --
the same override contract image/video offer as an alternative, just
mandatory here. prompt is provenance-only, never injected into the graph.
Routed through the existing registry.get_by_capability("music_generation")
path alongside suno_music/music_gen -- no dedicated selector needed.
ComfyUIClient.generate() now also reads the "audio" output key (what
ComfyUI's native SaveAudio node writes), and gets timeout/resume/websocket-
wait/multi-server support for free via the shared client. Duration is a
best-effort ffprobe probe of the downloaded file since a custom workflow
gives no other way to know it ahead of time.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- Require 'model' in the input schema and accept 'model_id' as a
selector-compatible alias (tts_selector exposes model_id); add a
selector-routing regression test
- Document s2.1-pro-free as promotional (free through end of July 2026,
Fair Use, no SLA, possible request retention, commercial-use
restrictions) in PROVIDERS.md and the Layer 3 skill; estimate_cost()
falls back to the paid s2.1-pro rate after the promo window
- Normalize voice_id/reference_id and model_id/model aliases before
computing the idempotency key, and include all output-affecting inputs
(bitrate, sample_rate, temperature, top_p, repetition_penalty, latency,
prosody, normalize, chunk_length) with API defaults applied
- Declare env:FISH_AUDIO_API_KEY in dependencies so registry metadata
reports the requirement
- Add fish_audio to the TTS provider set in the phase3 registry contract
test
Add FishAudioTTS (capability=tts) so tts_selector auto-discovers a new
high-quality, voice-clone-capable provider. Backend model is required per
call: s1 (previous flagship, kept for compatibility), s2-pro (first S2
generation), s2.1-pro (latest flagship — inline emotion tags, 80+
languages), s2.1-pro-free (free tier for drafts). s1-mini and the
speech-1.x tier have been removed from the current fish.audio API and are
no longer supported. Voice cloning via reference_id with voice_id as a
selector-compatible alias. Adds temperature/top_p/repetition_penalty
sampling controls, optional sample_rate, opus output format, and a "low"
latency tier. Cost is estimated per UTF-8 byte to match fish.audio
billing. Includes a Layer 3 skill, .env.example entry, and unit tests.
Verified end-to-end with s2.1-pro + reference_id: generated a 7-segment
Japanese narration successfully.
skills/creative/music-gen-usage.md mandates 'Always set
force_instrumental=true for video background', but music_gen.py never
sent the kwarg, so ElevenLabs could return vocal tracks that collide
with narration/dialogue.
- Add force_instrumental to input_schema (default True) so the mandate
holds by default; callers may opt out only with an explicit False.
- Include force_instrumental in the /v1/music payload.
- Add tests pinning: kwarg sent True by default, explicit opt-out
honored, and the schema default.
Refs: docs/REVIEW-image-to-video-voice.md §8 #8
Co-Authored-By: Claude <noreply@anthropic.com>
audio_mixer hard-coded loudnorm I=-16 (Apple Podcasts) in both _mix
and _full_mix. sound-design.md targets -14 for YouTube/TikTok/IG, and
edit_decisions.metadata.loudnorm_target is the declarative form — but
the mixer never read it, so the executed loudness silently defaulted
to podcast levels regardless of the target platform.
- Add loudnorm_target to input_schema (default -16, clamped to [-40, 0]).
- Extract _loudnorm_filter() helper and use it in _mix and _full_mix so
a director can forward edit_decisions.metadata.loudnorm_target (or a
caller can pass it directly) to hit the right platform target.
- Add tests pinning: default -16, -14 honored, out-of-range clamped,
non-numeric fallback, and the schema default.
Refs: docs/REVIEW-image-to-video-voice.md §8 #1
Co-Authored-By: Claude <noreply@anthropic.com>
`_segmented_music` mixed the video's audio with the shaped music via
`amix=inputs=2`, whose default `normalize=1` scales every input by 1/inputs
(x0.5, -6 dB). Unlike `_mix` and `_full_mix`, this path has no `loudnorm` stage
afterward to re-normalize, so the narration was permanently attenuated across
the entire timeline — including the stretches where the music volume expression
evaluates to 0. A one-second music segment quietly dropped the narration by
~6 dB for the whole video.
Add `normalize=0` to the amix: the music is already scaled to `music_volume`
by the `volume` expression, so speech passes at unity. Verified with ffmpeg —
narration in a no-music region tracks the stereo/aac conversion baseline
instead of sitting 6 dB below it.
The prior fix removed the dangling pad but still reused the speech filter
output for two consumers (sidechain key + final mix). FFmpeg auto-splits a
reused *input* label on some builds (macOS) but the Linux ffmpeg on CI rejects
it, so both full_mix ducking tests failed there.
Build a single [speech_all] stream and asplit it into [speech_key] (sidechain
key) and [speech_out] (final mix) so every filter label is produced once and
consumed once. Verified the generated graph for the single- and multi-narration
cases: no label is consumed more than once.
Refs #265
full_mix with ducking enabled (the default) failed for a single narration
track + one music bed — the most common shape — because the ducking branch
appended an acopy[speech_dup] filter whose output pad was never consumed,
leaving the filtergraph with a dangling output that ffmpeg rejects.
For a single speech track speech_out is '[a0]' (starts with '[a'), so the
guarded append fired; the compensating pop() only removes the empty-string
case from the multi-speech branch, so the dead pad survived exactly in the
single-narration case. The speech stream is already re-derived for the final
mix via [speech_out], and ffmpeg auto-splits the reused input label, so the
duplicate is unnecessary. Multi-speech and SFX paths are unaffected.
Adds regression tests for single- and multi-narration full_mix with ducking.
Closes#265
Address PR #240 review feedback from @calesthio:
1. dashscope_image: save EVERY returned image URL, not just the first.
The tool advertised multiple_outputs and accepted n>1 but only read
content[0], silently dropping paid outputs. Now collects all image
URLs across choices/content and downloads each to a distinct indexed
path (foo.png -> foo_1.png, foo_2.png, ...). images_generated now
reflects the actual count downloaded.
Per Qwen Cloud docs, a multi-output task is SUCCEEDED if at least one
image is generated; choices with finish_reason != "stop" are skipped
to avoid downloading partial/failed results.
2. Complete idempotency_key_fields so different requests no longer
collide and reuse stale artifacts:
- image: + negative_prompt, seed, prompt_extend, watermark
- tts: + instructions
- asr: + enable_words, language_hints
Adds 19 regression tests (114 total, all pass, no API keys needed):
- TestDashscopeImageMultiOutput: URL extraction across choices / within
one choice / failed-choice skipping, path resolution for
single/multi/no-extension, end-to-end multi-image download with a
mocked 3-URL DashScope response verifying all 3 files land on disk,
single-image legacy path behavior
- TestDashscopeIdempotencyKeys: field presence + key-differs-on-value
for every newly added field across all three tools
AGENT_GUIDE.md requires the music decision to be made at the proposal stage,
but the only check for the user's music_library/ folder lived in the
asset-director skills, which run later. A user could approve a creative
direction without ever being told a free, intentional music option was sitting
on disk (issue #168).
music_library/ was already referenced as a source_tool in asset artifacts but
had no backing tool. Add a small read-only tool that scans the library folder
(default <project root>/music_library, override via MUSIC_LIBRARY_DIR or a
library_dir input) and lists the audio tracks it finds, with best-effort
durations via ffprobe when present.
Because it inherits BaseTool, the registry auto-discovers it and it appears in
the preflight provider menu alongside music_gen and the stock music sources:
- AVAILABLE when the folder holds at least one audio track
- UNAVAILABLE otherwise, with install_instructions telling the user how to add
tracks
So the user sees their music options before approving creative direction, with
no orchestration code changes. Read-only: no side effects, no cost.
Closes#168
Google's TTS and Imagen tools advertised service-account auth
(GOOGLE_APPLICATION_CREDENTIALS) but only ever authenticated with an API
key string, so users with a service-account JSON could not use either tool.
google_tts.get_status() also over-reported availability when the JSON was
set, then failed at execute() — a silent-availability bug.
Separately, both hand-rolled _load_dotenv parsers kept inline comments as
values, so after `cp .env.example .env` every keyed tool falsely reported
"available" with no real credentials.
Changes:
- Add tools/google_credentials.py: lazy google-auth Bearer-token helper.
- google_tts: authenticate via Cloud TTS Bearer token when only a service
account is configured; make get_status() honest.
- google_imagen: route service-account auth to Vertex AI
({location}-aiplatform.googleapis.com) with project/location resolution,
alongside the existing AI Studio API-key path.
- Fix both _load_dotenv parsers to strip inline comments (quote-aware).
- Add google-auth to requirements; document the new env vars in .env.example.
- .gitignore: never commit GCP service-account key files.
Verified locally with a real service account: TTS produced a valid MP3 and
Imagen produced a valid 1408x768 PNG via Vertex AI. Existing test suite
passes (2 unrelated pre-existing failures only).
Closes#131
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
End-tag now composited over final body footage (ProRes 4444 with alpha)
instead of concatenated as a black card. Updated idea-director,
edit-director, compose-director, and pipeline manifest with overlay-first
instructions. Default Google TTS voice changed from Neural2-D to
Chirp3-HD-Orus across tool code and docs.
Route Chirp and Journey voice families to the v1beta1 API endpoint
automatically. Adds beta voice detection, dynamic API version
selection, and Chirp3-HD cost estimation ($30/1M chars).
The instructions parameter was passed as None when not provided,
causing a 400 error. Now only included when explicitly set. Also
adds speed passthrough support.
Implements the 2026-04-02 transformation spec (Phases 1-8) and fixes all
critical bugs found during 5-pipeline E2E testing.
Governance & Decision Intelligence:
- Pipeline-specific stage order in checkpoint (replaces global STAGES list)
- Provider scoring engine (lib/scoring.py) with 7-dimension weighted ranking
- Decision log artifact enforced at proposal/idea stage across all 10 pipelines
- Delivery promise classifier prevents silent motion-to-still downgrades
- Structured shot language in scene_plan schema (camera, lens, lighting, DOF)
- Variation checker and slideshow risk scorer block samey output before render
- Creative intake, capability extension, and creative-intake meta skills
- Final self-review artifact with 5 mandatory checks before presenting output
- Source media review contract for user-supplied footage
Render & Theme System:
- Remotion AnimatedBackground now derives colors from playbook (no more hardcoded
dark blue fintech gradient on every video)
- video_compose builds custom ThemeConfig from playbook YAML colors/fonts —
custom playbooks flow through to Remotion automatically
- Explainer component wires theme to all child components (charts, cards, etc.)
- resolveAsset() handles absolute paths on Windows/Unix via file:// URIs
- RENDERER_FAMILY_MAP synced with actual Remotion compositions
Critical Bug Fixes:
- Windows npx subprocess: run_command() resolves .cmd wrappers via shutil.which()
- Silent renderer downgrade: Remotion failure now returns explicit error with
options instead of silently falling back to FFmpeg
- .env inline comment parsing strips trailing # comments from API keys
- concat_path UnboundLocalError in video_compose finally block
- audio_mixer and showcase_card capture=True kwarg bug
- Selector estimate_cost() calls fixed (_select_tool -> _select_best_tool)
- asset_manifest schema expanded with provider, license, subtype fields
- screen-demo subtitle_gen moved from required to optional tools
- Duration drift detection in post-render final review (>25% warns)
