zlei6/OpenMontage
1
0
Fork 0
mirror of https://github.com/calesthio/OpenMontage.git synced 2026-08-16 21:51:24 +08:00
Code Issues Packages Projects Releases Wiki Activity

feat: bundle a native-node ACE-Step v1 workflow for comfyui_music

Browse Source 
ACE-Step v1's node-pack fragmentation turns out to be moot: ComfyUI ships
TextEncodeAceStepAudio/EmptyAceStepLatentAudio as native core nodes
(comfy_extras/nodes_ace.py), not a third-party pack, and Comfy-Org's own
workflow_templates repo has an official ACE-Step-v1 template built from
those plus long-stable core nodes. tools/_comfyui/workflows/ace-step-1-t2a.json
was built by cross-checking every node's class_type and input names against
ComfyUI's own source (nodes_ace.py, nodes_audio.py, nodes_latent.py,
nodes.py) rather than trusting the UI-format export directly.

comfyui_music now defaults to this bundled workflow: prompt maps to
ACE-Step's tags field (matching suno_music's "prompt = music description"
convention), lyrics/duration_seconds/steps/cfg/lyrics_strength/seed are all
patchable, and missing ace_step_v1_3.5b.safetensors surfaces through the
same missing_models contract as image/video. workflow_json/workflow_path +
output_node remains available for ACE-Step 1.5, other node packs, or
different audio models entirely.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
Ntsako
2026-08-06 15:10:48 +02:00
parent 172acca6ea
commit cf1a13a722
6 changed files with 316 additions and 90 deletions
Download Patch File Download Diff File Expand all files Collapse all files

10
.agents/skills/comfyui/SKILL.md
View File

@@ -58,7 +58,9 @@ Use this skill before calling `comfyui_image`, `comfyui_video`, or `comfyui_musi
## Music (`comfyui_music`)
- Unlike `comfyui_image`/`comfyui_video`, there is **no bundled workflow**. ACE-Step's ComfyUI node interface isn't standardized across custom node packs (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.), so `workflow_json`/`workflow_path` + `output_node` are always required, not optional.
- `prompt` is provenance/logging only -- it is never injected into the workflow. Bake the actual tags/lyrics into the workflow JSON yourself before calling, the same way you would patch a custom image/video workflow.
- `output_node` should be the node that writes the final audio, typically ComfyUI's native `SaveAudio`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
- Provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow -- there's no bundled model stack to fall back on here.
- Bundled default is ACE-Step v1 (3.5B) text-to-audio, built from ComfyUI's *native* `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` nodes (core, not a third-party pack) -- unlike ACE-Step 1.5 or other custom node packs, v1's interface is standardized enough to bundle safely.
- `prompt` maps to the bundled workflow's `tags` field (style/genre/mood, e.g. `"upbeat electronic pop, female vocals"`), matching the same "prompt = music description" convention `suno_music` uses. `lyrics` is a separate optional field -- leave empty for instrumental, or use `[verse]`/`[chorus]`/`[bridge]` structure tags and `[zh]`/`[ja]`/`[ko]`-style language-code prefixes for non-English lines.
- `duration_seconds`, `steps`, `cfg`, `lyrics_strength`, and `seed` are patchable on the bundled workflow. Missing `ace_step_v1_3.5b.safetensors` surfaces through the same `data.missing_models[]` contract as image/video.
- Need ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model? Fall back to `workflow_json`/`workflow_path` + `output_node`, exactly like a custom image/video workflow -- in that mode `prompt` becomes provenance/logging only again and must already be baked into the graph.
- `output_node` (bundled or custom) should be the node that writes the final audio -- the bundled workflow's is `SaveAudioMP3`. The client reads artifacts from that node's `"audio"` output key (parallel to `"images"` for image/video savers).
- For custom workflows, provide `workflow_name`/`workflow_model`/`workflow_model_stack` for provenance exactly as you would for a custom image/video workflow.

69
docs/comfyui-adapter-plan.md
View File

@@ -296,24 +296,33 @@ not promote ComfyUI for an operation whose bundled models are missing.
---
### `comfyui_music` -- Music Generation (shipped, custom-workflow-only)
### `comfyui_music` -- Music Generation (shipped, with a native-node bundled workflow)
`tools/audio/comfyui_music.py`. `capability="music_generation"`, `provider="comfyui"`.
Ships with **no bundled workflow** -- the ACE-Step node-pack fragmentation
described below is real and unsolved, so instead of picking one pack and
breaking for everyone else, the tool always requires a caller-supplied
`workflow_json`/`workflow_path` + `output_node`, exactly like the image/video
tools' *optional* override path, just mandatory here. `prompt` is accepted
for provenance/logging only and is never injected into the workflow --
tags/lyrics must already be baked into the graph before calling, same
convention as image/video custom workflows.
Originally not shipped because: the ComfyUI node interface for ACE-Step is
not standardized -- there are multiple custom node packs with different
class names (`AceStepModelLoader` vs native `TextEncodeAceStepAudio`, etc.).
Shipping a workflow that only works with one specific custom node pack would
break for most users. The custom-workflow-only design sidesteps this
entirely: whichever node pack is installed, the caller exports it themselves.
**Bundled default:** ACE-Step v1 (3.5B) text-to-audio, via `tools/_comfyui/workflows/ace-step-1-t2a.json`.
The node-pack fragmentation that originally blocked this tool (`AceStepModelLoader`
vs native `TextEncodeAceStepAudio`, etc.) turned out to be moot for ACE-Step v1:
ComfyUI ships `TextEncodeAceStepAudio`/`EmptyAceStepLatentAudio` as **native core
nodes** (`comfy_extras/nodes_ace.py`), not a third-party pack, and Comfy-Org's own
[`workflow_templates`](https://github.com/Comfy-Org/workflow_templates) repo bundles
an official ACE-Step-v1 template built entirely from those native nodes plus
long-stable core nodes (`CheckpointLoaderSimple`, `KSampler`, `ModelSamplingSD3`,
`VAEDecodeAudio`, `SaveAudioMP3`). Every node's `class_type` and input names in
`ace-step-1-t2a.json` were cross-checked against ComfyUI's own source
(`comfy_extras/nodes_ace.py`, `nodes_audio.py`, `nodes_latent.py`, `nodes.py`) --
not guessed from the UI export -- since the UI-format template Comfy-Org ships
isn't directly usable as the API-format JSON this client submits.
`prompt` maps to ACE-Step's `tags` field (style/genre/mood description, matching
the "prompt = description of desired music" convention `suno_music` already uses).
`lyrics` is a separate optional field (empty for instrumental). `duration_seconds`,
`steps`, `cfg`, `lyrics_strength`, and `seed` are all patchable; `shift` and the
tonemap `multiplier` stay at the official template's defaults.
Newer/different setups aren't locked out: `workflow_json`/`workflow_path` +
`output_node` still works exactly like the image/video tools' override path --
for ACE-Step 1.5, a different node pack, or a non-ACE-Step audio model entirely.
**Selector integration:** no dedicated `music_selector` exists in OpenMontage
(unlike `tts_selector`/`image_selector`/`video_selector`) -- music tools are
@@ -323,17 +332,16 @@ do. `fallback_tools = ["suno_music", "music_gen"]`.
**Audio artifact schema:** `ToolResult.data` follows the same shape as the
image/video tools (`provider`, `model`, `output`, `format`, `workflow_provenance`),
plus `duration_seconds` -- a best-effort `ffprobe` probe of the downloaded
file (`None` if `ffprobe` isn't on PATH), since a custom workflow gives no
other reliable way to know actual output duration ahead of time.
plus `lyrics` and `duration_seconds` -- the latter a best-effort `ffprobe` probe
of the downloaded file (`None` if `ffprobe` isn't on PATH), since even the bundled
workflow doesn't report actual rendered duration back through `/history`.
**Workflow/output-node contract:** identical to image/video -- `output_node`
must be the ID of the node that writes the final artifact (typically ComfyUI's
native `SaveAudio` node). `ComfyUIClient.generate()`'s artifact extraction now
also checks the `"audio"` output key (previously only `"images"`/`"gifs"`),
which is what `SaveAudio` writes to in ComfyUI's `/history` response --
this is the one part of the contract that *is* standardized regardless of
which ACE-Step loader pack sits upstream of it.
must be the ID of the node that writes the final artifact (the bundled workflow's
is `SaveAudioMP3`, ComfyUI's native audio saver). `ComfyUIClient.generate()`'s
artifact extraction now also checks the `"audio"` output key (previously only
`"images"`/`"gifs"`), which is what `SaveAudioMP3`/`SaveAudio` write to in
ComfyUI's `/history` response.
---
@@ -535,11 +543,12 @@ pipeline definition, or any schema.
override, and `COMFYUI_SETUP_OFFER.per_capability_env_var_overrides` documents
it for the setup-offer surfacing in `provider_menu()`.
4. ~~**Music generation:**~~ **Resolved -- shipped as custom-workflow-only.**
4. ~~**Music generation:**~~ **Resolved -- shipped with a bundled ACE-Step v1 workflow.**
`comfyui_music` is a real tool now (not a hidden image/video override), routed
through the existing `registry.get_by_capability("music_generation")` path
like `suno_music`/`music_gen`. It has no bundled workflow -- the node-pack
fragmentation that originally blocked this is real, so the tool always
requires caller-supplied `workflow_json`/`workflow_path` + `output_node`
rather than betting on one pack. See the `comfyui_music` section above for
the artifact schema and workflow/output-node contract.
like `suno_music`/`music_gen`. The node-pack fragmentation that originally
blocked this turned out not to apply to ACE-Step v1: its ComfyUI nodes are
native core nodes, not a third-party pack, so `ace-step-1-t2a.json` ships as
the default, verified node-by-node against ComfyUI's own source. Custom
`workflow_json`/`workflow_path` + `output_node` remains available for other
versions/packs. See the `comfyui_music` section above for the full contract.

69
tests/contracts/test_comfyui_tools.py
View File

@@ -137,6 +137,7 @@ EXPECTED_WORKFLOWS = [
"flux2-txt2img.json",
"wan22-i2v-4step.json",
"wan22-t2v-4step.json",
"ace-step-1-t2a.json",
]
@@ -651,6 +652,11 @@ class TestModelRequirements:
assert len(_REQUIRED_MODELS_T2V) > 0
assert any("t2v" in m.lower() for m in _REQUIRED_MODELS_T2V)
def test_music_tool_has_required_models(self):
from tools.audio.comfyui_music import _REQUIRED_MODELS
assert len(_REQUIRED_MODELS) > 0
assert any("ace_step" in m.lower() for m in _REQUIRED_MODELS)
# ------------------------------------------------------------------
# Custom workflow contract and provenance
@@ -863,16 +869,23 @@ class TestComfyUIMusic:
assert tool.capability == "music_generation"
assert tool.provider == "comfyui"
def test_requires_workflow_json_or_path(self):
def test_bundled_path_requires_no_workflow_json_or_output_node(self, tmp_path):
"""Without workflow_json/workflow_path it should attempt the bundled
ACE-Step workflow, not demand a custom one."""
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
tool._client.check_models = lambda required: (list(required), [])
tool._client.generate = lambda workflow, output_node, dest, **kwargs: [Path(dest)]
result = tool.execute({"prompt": "ambient pad", "output_node": "9"})
result = tool.execute({
"prompt": "ambient pad",
"output_path": str(tmp_path / "music.mp3"),
})
assert result.success is False
assert "workflow_json" in result.error or "workflow_path" in result.error
assert result.success is True
assert result.data["workflow_provenance"]["source"] == "bundled"
def test_requires_output_node(self):
def test_custom_workflow_without_output_node_errors(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
@@ -884,6 +897,52 @@ class TestComfyUIMusic:
assert result.success is False
assert "output_node" in result.error
def test_bundled_missing_models_returns_structured_payload(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
tool._client.check_models = lambda required: ([], list(required))
result = tool.execute({"prompt": "ambient pad"})
assert result.success is False
assert result.data["missing_models"][0]["name"] == "ace_step_v1_3.5b.safetensors"
assert result.data["missing_models"][0]["download_url"]
def test_bundled_generation_patches_tags_lyrics_and_seed(self, tmp_path):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
tool._client.check_models = lambda required: (list(required), [])
seen = {}
def fake_generate(workflow, output_node, dest, **kwargs):
seen["workflow"] = workflow
seen["output_node"] = output_node
return [Path(dest)]
tool._client.generate = fake_generate
result = tool.execute({
"prompt": "lofi hip hop, chill, rain sounds",
"lyrics": "[verse]\nquiet streets",
"duration_seconds": 45,
"seed": 777,
"output_path": str(tmp_path / "music.mp3"),
})
assert result.success is True
assert seen["output_node"] == "10"
assert seen["workflow"]["2"]["inputs"]["tags"] == "lofi hip hop, chill, rain sounds"
assert seen["workflow"]["2"]["inputs"]["lyrics"] == "[verse]\nquiet streets"
assert seen["workflow"]["4"]["inputs"]["seconds"] == 45
assert seen["workflow"]["8"]["inputs"]["seed"] == 777
assert result.data["model"] == "ace-step-v1-3.5b"
def test_get_status_degraded_when_model_missing(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: True
tool._client.check_models = lambda required: ([], list(required))
assert tool.get_status() == ToolStatus.DEGRADED
def test_unavailable_server_reports_unavailable_reason(self):
tool = ComfyUIMusic()
tool._client.is_available = lambda: False

11
tools/_comfyui/metadata.py
View File

@@ -184,6 +184,17 @@ BUNDLED_MODEL_STACKS: dict[str, list[dict[str, Any]]] = {
),
},
],
"ace-step-1-t2a": [
{
"role": "checkpoint",
"name": "ace_step_v1_3.5b.safetensors",
"destination_hint": "ComfyUI/models/checkpoints/",
"download_url": (
"https://huggingface.co/Comfy-Org/ACE-Step_ComfyUI_repackaged/"
"blob/main/all_in_one/ace_step_v1_3.5b.safetensors"
),
},
],
}

80
tools/_comfyui/workflows/ace-step-1-t2a.json Normal file
View File

@@ -0,0 +1,80 @@
{
"1": {
"class_type": "CheckpointLoaderSimple",
"inputs": {
"ckpt_name": "ace_step_v1_3.5b.safetensors"
}
},
"2": {
"class_type": "TextEncodeAceStepAudio",
"inputs": {
"clip": ["1", 1],
"tags": "",
"lyrics": "",
"lyrics_strength": 0.99
}
},
"3": {
"class_type": "ConditioningZeroOut",
"inputs": {
"conditioning": ["2", 0]
}
},
"4": {
"class_type": "EmptyAceStepLatentAudio",
"inputs": {
"seconds": 120,
"batch_size": 1
}
},
"5": {
"class_type": "ModelSamplingSD3",
"inputs": {
"model": ["1", 0],
"shift": 5.0
}
},
"6": {
"class_type": "LatentOperationTonemapReinhard",
"inputs": {
"multiplier": 1.0
}
},
"7": {
"class_type": "LatentApplyOperationCFG",
"inputs": {
"model": ["5", 0],
"operation": ["6", 0]
}
},
"8": {
"class_type": "KSampler",
"inputs": {
"model": ["7", 0],
"positive": ["2", 0],
"negative": ["3", 0],
"latent_image": ["4", 0],
"seed": 0,
"steps": 50,
"cfg": 5.0,
"sampler_name": "euler",
"scheduler": "simple",
"denoise": 1.0
}
},
"9": {
"class_type": "VAEDecodeAudio",
"inputs": {
"samples": ["8", 0],
"vae": ["1", 2]
}
},
"10": {
"class_type": "SaveAudioMP3",
"inputs": {
"audio": ["9", 0],
"filename_prefix": "openmontage",
"quality": "V0"
}
}
}

167
tools/audio/comfyui_music.py
View File

@@ -1,13 +1,11 @@
"""ComfyUI music generation via a local or remote ComfyUI server.
No bundled workflow: ACE-Step's ComfyUI node interface is not standardized
across custom node packs (``AceStepModelLoader`` vs native
``TextEncodeAceStepAudio``, etc.), so a hardcoded template would break for
most installs. This tool always runs a caller-supplied ``workflow_json`` or
``workflow_path`` -- the same override contract ``comfyui_image``/
``comfyui_video`` offer as an alternative to their bundled workflow, just
mandatory here instead of optional. See the ``comfyui`` skill for how to
convert a community ACE-Step workflow into a call.
Default workflow: ACE-Step v1 (3.5B) text-to-audio using ComfyUI's native
``TextEncodeAceStepAudio``/``EmptyAceStepLatentAudio`` nodes (built into
ComfyUI core, not a third-party pack). Custom workflows are still accepted
via ``workflow_json``/``workflow_path`` for other ACE-Step node packs, other
versions (e.g. ACE-Step 1.5), or entirely different audio models -- the same
override contract ``comfyui_image``/``comfyui_video`` offer.
"""
from __future__ import annotations
@@ -32,12 +30,23 @@ from tools.base_tool import (
ToolTier,
)
from tools._comfyui.client import ComfyUIClient, ComfyUIError
from tools._comfyui.metadata import COMFYUI_SETUP_OFFER, workflow_hash
from tools._comfyui.metadata import (
BUNDLED_MODEL_STACKS,
COMFYUI_SETUP_OFFER,
missing_models_payload,
model_stack,
workflow_hash,
)
_WORKFLOWS = Path(__file__).resolve().parent.parent / "_comfyui" / "workflows"
# Model required by the bundled ACE-Step v1 workflow
_REQUIRED_MODELS = ["ace_step_v1_3.5b.safetensors"]
class ComfyUIMusic(BaseTool):
name = "comfyui_music"
version = "0.1.0"
version = "0.2.0"
tier = ToolTier.GENERATE
capability = "music_generation"
provider = "comfyui"
@@ -49,10 +58,10 @@ class ComfyUIMusic(BaseTool):
dependencies = [] # checked at runtime via server health
setup_offer = COMFYUI_SETUP_OFFER
install_instructions = (
"Start a ComfyUI server with ACE-Step installed (any node pack) and "
"set COMFYUI_SERVER_URL (default http://localhost:8188).\n"
"There is no bundled workflow for this tool -- export your ACE-Step "
"graph in API format and pass it as workflow_json/workflow_path.\n"
"Start a ComfyUI server and set COMFYUI_SERVER_URL "
"(default http://localhost:8188).\n"
"Requires ace_step_v1_3.5b.safetensors in ComfyUI's checkpoints "
"directory for the bundled workflow.\n"
"Running a separate ComfyUI instance for music? Set "
"COMFYUI_MUSIC_SERVER_URL instead -- it takes priority over "
"COMFYUI_SERVER_URL for this tool only."
@@ -62,59 +71,73 @@ class ComfyUIMusic(BaseTool):
capabilities = ["generate_background_music", "generate_song", "generate_instrumental"]
supports = {
"seed": True,
"lyrics": True,
"custom_workflow": True,
"custom_output_node": True,
"offline": True,
}
best_for = [
"local GPU music generation without API costs, using whatever ACE-Step node pack is installed",
"full control over sampling via custom ComfyUI workflows",
"local GPU music generation without API costs",
"instrumentals and songs with lyrics via the bundled ACE-Step v1 workflow",
"full control over sampling or other ACE-Step versions/node packs via custom ComfyUI workflows",
]
not_good_for = [
"setups without a running ComfyUI server",
"quick generation without first exporting/adapting an ACE-Step workflow",
"CPU-only machines",
]
fallback_tools = ["suno_music", "music_gen"]
input_schema = {
"type": "object",
"required": ["prompt", "output_node"],
"required": ["prompt"],
"properties": {
"prompt": {
"type": "string",
"description": (
"Description of the desired music, for provenance/logging only. "
"Not injected into the workflow -- bake the actual tags/lyrics "
"into workflow_json/workflow_path before calling."
"Style/mood/genre description (ACE-Step 'tags'), e.g. "
"'upbeat electronic pop, female vocals, driving bassline'. "
"Comma-separated tags work best. Not injected for custom workflows."
),
},
"lyrics": {
"type": "string",
"default": "",
"description": (
"Optional lyrics. Leave empty for instrumental. Supports structure "
"tags like [verse]/[chorus]/[bridge] and language-code prefixes "
"(e.g. [zh], [ja]) for non-English lines."
),
},
"duration_seconds": {"type": "number", "default": 120.0},
"steps": {"type": "integer", "default": 50},
"cfg": {"type": "number", "default": 5.0},
"lyrics_strength": {"type": "number", "default": 0.99},
"seed": {"type": "integer", "description": "Random if omitted"},
"output_path": {"type": "string", "description": "Where to save the audio"},
"workflow_json": {
"type": "string",
"description": "Full ComfyUI ACE-Step workflow JSON (API format). Required if workflow_path is omitted.",
"description": "Optional full ComfyUI workflow JSON. Requires output_node.",
},
"workflow_path": {
"type": "string",
"description": "Path to a ComfyUI ACE-Step workflow JSON file. Required if workflow_json is omitted.",
"description": "Optional path to a ComfyUI workflow JSON file. Requires output_node.",
},
"output_node": {
"type": "string",
"description": "ComfyUI output node ID (e.g. the SaveAudio node) to download the artifact from.",
"description": "ComfyUI output node ID for custom workflow_json/workflow_path.",
},
"workflow_name": {
"type": "string",
"description": "Optional human-readable provenance label for the workflow.",
"description": "Optional human-readable provenance label for a custom workflow.",
},
"workflow_model": {
"type": "string",
"description": "Optional model/provenance label (e.g. 'ace-step-v1-3.5b').",
"description": "Optional model/provenance label for a custom workflow.",
},
"workflow_model_stack": {
"type": "array",
"description": (
"Optional provenance metadata for workflow dependencies. "
"Optional provenance metadata for custom workflow dependencies. "
"Items should include name, role, and node-pack origin when known."
),
"items": {"type": "object"},
@@ -134,7 +157,7 @@ class ComfyUIMusic(BaseTool):
cpu_cores=2, ram_mb=8000, vram_mb=8000, disk_mb=500, network_required=False,
)
retry_policy = RetryPolicy(max_retries=1, retryable_errors=["timeout"])
idempotency_key_fields = ["prompt", "seed", "workflow_json", "workflow_path", "output_node"]
idempotency_key_fields = ["prompt", "lyrics", "duration_seconds", "seed"]
side_effects = ["writes audio file to output_path"]
user_visible_verification = ["Listen to generated audio for mood, genre accuracy, and quality"]
@@ -145,21 +168,21 @@ class ComfyUIMusic(BaseTool):
def get_status(self) -> ToolStatus:
if not self._client.is_available():
return ToolStatus.UNAVAILABLE
_, missing = self._client.check_models(_REQUIRED_MODELS)
if missing:
return ToolStatus.DEGRADED
return ToolStatus.AVAILABLE
def estimate_cost(self, inputs: dict[str, Any]) -> float:
return 0.0
def estimate_runtime(self, inputs: dict[str, Any]) -> float:
# Actual runtime depends entirely on the caller's custom workflow
# (steps, duration, sampler); this is a conservative flat estimate.
return 180.0
return float(inputs.get("steps", 50)) * 2.0
def get_info(self) -> dict[str, Any]:
info = super().get_info()
info["setup_offer"] = self.setup_offer
info["bundled_workflow"] = None
info["custom_workflow_required"] = True
info["bundled_model_stack"] = BUNDLED_MODEL_STACKS["ace-step-1-t2a"]
return info
def _log_progress(self, data: dict) -> None:
@@ -173,35 +196,63 @@ class ComfyUIMusic(BaseTool):
print(f"[comfyui_music] step {value}/{max_value}")
def execute(self, inputs: dict[str, Any]) -> ToolResult:
if not (inputs.get("workflow_json") or inputs.get("workflow_path")):
custom_workflow = bool(inputs.get("workflow_json") or inputs.get("workflow_path"))
if custom_workflow and not inputs.get("output_node"):
return ToolResult(
success=False,
error=(
"comfyui_music requires workflow_json or workflow_path -- there "
"is no bundled default. ACE-Step's ComfyUI node interface isn't "
"standardized across custom node packs, so a hardcoded template "
"would break for most installs. Export the ACE-Step workflow "
"you actually have installed (API format) and pass it in."
"Custom ComfyUI workflows require output_node so OpenMontage "
"knows which ComfyUI node to download artifacts from."
),
)
if not inputs.get("output_node"):
return ToolResult(
success=False,
error="output_node is required so OpenMontage knows which ComfyUI node to download the audio from.",
)
if not self._client.is_available():
return ToolResult(success=False, error=self._client.unavailable_reason())
if not custom_workflow:
_, missing = self._client.check_models(_REQUIRED_MODELS)
if missing:
return ToolResult(
success=False,
data=missing_models_payload(
missing,
workflow_key="ace-step-1-t2a",
workflow_name="ace-step-1-t2a.json",
),
error=(
f"ComfyUI server is running but missing required models: "
f"{', '.join(missing)}.\n"
f"See data.missing_models for destination hints and download URLs."
),
)
start = time.time()
seed = inputs.get("seed") or ComfyUIClient.random_seed()
output_path = Path(inputs.get("output_path", f"comfyui_music_{seed}.mp3"))
output_node = str(inputs["output_node"])
try:
workflow = self._load_custom_workflow(inputs)
provenance = self._workflow_provenance(inputs, output_node, workflow)
if custom_workflow:
workflow = self._load_custom_workflow(inputs)
output_node = str(inputs["output_node"])
else:
workflow = ComfyUIClient.load_workflow(_WORKFLOWS / "ace-step-1-t2a.json")
workflow = ComfyUIClient.patch_workflow(workflow, {
"2": {
"tags": inputs["prompt"],
"lyrics": inputs.get("lyrics", ""),
"lyrics_strength": inputs.get("lyrics_strength", 0.99),
},
"4": {"seconds": inputs.get("duration_seconds", 120.0)},
"8": {
"seed": seed,
"steps": inputs.get("steps", 50),
"cfg": inputs.get("cfg", 5.0),
},
"10": {"filename_prefix": output_path.stem},
})
output_node = "10"
provenance = self._workflow_provenance(inputs, custom_workflow, output_node, workflow)
paths = self._client.generate(
workflow,
output_node=output_node,
@@ -229,13 +280,14 @@ class ComfyUIMusic(BaseTool):
return ToolResult(success=False, error=f"ComfyUI music generation failed: {exc}")
duration = self._probe_duration(paths[0])
model_name = self._model_name(inputs)
model_name = self._model_name(inputs, custom_workflow)
return ToolResult(
success=True,
data={
"provider": "comfyui",
"model": model_name,
"prompt": inputs["prompt"],
"lyrics": inputs.get("lyrics", ""),
"duration_seconds": duration,
"output": str(paths[0]),
"format": paths[0].suffix.lstrip("."),
@@ -255,7 +307,9 @@ class ComfyUIMusic(BaseTool):
return ComfyUIClient.load_workflow(Path(inputs["workflow_path"]))
@staticmethod
def _model_name(inputs: dict[str, Any]) -> str:
def _model_name(inputs: dict[str, Any], custom_workflow: bool) -> str:
if not custom_workflow:
return "ace-step-v1-3.5b"
return (
inputs.get("workflow_model")
or inputs.get("model")
@@ -265,8 +319,19 @@ class ComfyUIMusic(BaseTool):
@staticmethod
def _workflow_provenance(
inputs: dict[str, Any], output_node: str, workflow: dict[str, Any]
inputs: dict[str, Any],
custom_workflow: bool,
output_node: str,
workflow: dict[str, Any],
) -> dict[str, Any]:
if not custom_workflow:
return {
"source": "bundled",
"workflow": "ace-step-1-t2a.json",
"workflow_hash_sha256": workflow_hash(workflow),
"model_stack": model_stack("ace-step-1-t2a", inputs),
"output_node": output_node,
}
stack = inputs.get("workflow_model_stack")
return {
"source": "user_supplied",