mirror of
https://github.com/calesthio/OpenMontage.git
synced 2026-08-16 21:51:24 +08:00
Merge remote-tracking branch 'origin/main' into codex/repair-pr-442
This commit is contained in:
@@ -5,7 +5,22 @@ description: Generate AI voiceovers, sound effects, and music using ElevenLabs A
|
||||
|
||||
# ElevenLabs Audio Generation
|
||||
|
||||
Requires `ELEVENLABS_API_KEY` in `.env`.
|
||||
## OpenMontage provider routing
|
||||
|
||||
Inspect the OpenMontage registry before choosing an authentication path.
|
||||
|
||||
- Prefer `fal_elevenlabs_tts` when it is available. It provides Eleven v3,
|
||||
Multilingual v2, and Turbo v2.5 through the centrally managed fal.ai
|
||||
connection; no separate ElevenLabs credential is needed.
|
||||
- Use `elevenlabs_tts` only when that direct provider is already reported as
|
||||
available by the registry.
|
||||
- In a shared installation, never tell the user to create a `.env`, export a
|
||||
key, or paste a credential. Report missing direct-provider access as an
|
||||
administrator setup request.
|
||||
|
||||
The direct API examples below require a centrally configured
|
||||
`ELEVENLABS_API_KEY`; they are not the default path when the fal.ai provider is
|
||||
available.
|
||||
|
||||
## Text-to-Speech
|
||||
|
||||
|
||||
@@ -68,6 +68,10 @@ DOUBAO_SPEECH_VOICE_TYPE=
|
||||
# Get one at https://dashscope.aliyun.com/
|
||||
DASHSCOPE_API_KEY=
|
||||
|
||||
# --- Tencent Hunyuan TokenHub ---
|
||||
# Hunyuan Image 3.0 via Tencent TokenHub.
|
||||
TENCENT_TOKENHUB_API_KEY=
|
||||
|
||||
# --- Music ---
|
||||
# Suno AI music generation (full songs, instrumentals, any genre).
|
||||
SUNO_API_KEY=
|
||||
|
||||
@@ -711,6 +711,8 @@ The `.agents/skills/` directory is large. When you're not coming in through a to
|
||||
- **Do not skip stage director skills.** Before executing any pipeline stage, read its director skill. The skill contains the quality bar, the workflow, and the review criteria.
|
||||
- Do not use deleted legacy names such as `tts_cloud`, `tts_engine`, or `video_gen`.
|
||||
- Do not hardcode provider names, API key names, or setup URLs. Read them from the registry's `install_instructions` and `dependencies` fields.
|
||||
- Do not tell a restricted shared-installation user to add credentials manually, create a `.env`, or export a key. Surface the unavailable provider as an administrator setup request and continue only with centrally provisioned capabilities.
|
||||
- Before offering a direct vendor credential, check the registry for an available wrapper through an already configured provider (for example, a partner model hosted by fal.ai).
|
||||
- Do not begin asset generation before user approval on the production plan.
|
||||
- Do not hide degraded paths. Record substitutions and blocked options explicitly.
|
||||
- Do not present a single unavailable tool in isolation. Always show the full capability picture: "X of Y providers configured for this capability."
|
||||
|
||||
@@ -251,11 +251,29 @@ The ASR tool (`qwen3-asr-flash-filetrans`) uses an async submit-poll pattern. Au
|
||||
|
||||
---
|
||||
|
||||
### Tencent Hunyuan Cloud — Image Generation
|
||||
|
||||
> **Chinese-friendly first-party image generation.** `hunyuan_image` accesses
|
||||
> Hunyuan Image 3.0 through Tencent TokenHub with Bearer-token authentication.
|
||||
> It supports seeded text-to-image, up to three reference images, custom
|
||||
> resolutions, prompt rewriting, and watermark controls.
|
||||
|
||||
**Tool unlocked:** `hunyuan_image`
|
||||
|
||||
**Env var:** `TENCENT_TOKENHUB_API_KEY`
|
||||
|
||||
Generate an API key in the Tencent Cloud TokenHub console and add it to
|
||||
`.env`. The tool reports approximately $0.08 per generated image based on
|
||||
TokenHub's credit price. It is available through `image_selector`; shared
|
||||
reference-image inputs are normalized to the provider's `images` array.
|
||||
|
||||
---
|
||||
|
||||
### fal.ai — Multi-Model Gateway
|
||||
|
||||
> **Broad single-key coverage.** One API key unlocks image and video providers across multiple models.
|
||||
|
||||
**Tools unlocked:** `flux_image`, `recraft_image`, `seedream_image`, `kling_video`, `veo_video`, `minimax_video`
|
||||
**Tools unlocked:** `flux_image`, `recraft_image`, `seedream_image`, `kling_video`, `veo_video`, `minimax_video`, `fal_elevenlabs_tts`, `fal_elevenlabs_music`
|
||||
**Env var:** `FAL_KEY`
|
||||
|
||||
#### Setup
|
||||
@@ -290,6 +308,10 @@ No subscription — pure pay-as-you-go, no minimum spend.
|
||||
|
||||
**Free tier:** None — but $0 to start, you only pay for what you use.
|
||||
|
||||
The same key can also access ElevenLabs speech and music through fal.ai. Use
|
||||
`fal_elevenlabs_tts` when direct ElevenLabs credentials are unavailable, or
|
||||
select it through `tts_selector` with `preferred_provider: "fal.ai"`.
|
||||
|
||||
---
|
||||
|
||||
### MiniMax — Official Direct Image API
|
||||
@@ -492,7 +514,7 @@ allowance). OpenMontage estimates cost from the transcribed audio duration. See
|
||||
|
||||
### Google — TTS + Imagen + Music + Video (Shared Key)
|
||||
|
||||
> **One key, five tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. Imagen 4 generates high-quality images. Google Lyria generates high-quality background music. Gemini Omni Flash supports conversational video editing, and direct Veo generation covers premium short video clips.
|
||||
> **One key, five tools.** Google Cloud TTS has 700+ voices in 50+ languages — the strongest localization option. `google_imagen` supports both Imagen 4 and Gemini 2.5 Flash Image, including projects without Imagen catalog access. Google Lyria generates high-quality background music. Gemini Omni Flash supports conversational video editing, and direct Veo generation covers premium short video clips.
|
||||
|
||||
**Tools unlocked:** `google_tts`, `google_imagen`, `google_music`, `gemini_omni_video`, `veo_video`
|
||||
**Env var:** `GOOGLE_API_KEY` (or `GEMINI_API_KEY` — either works; `GEMINI_API_KEY` takes precedence)
|
||||
@@ -533,9 +555,15 @@ The free tiers apply *independently* — you get 1M Standard AND 1M WaveNet AND
|
||||
| Imagen 4 Fast | $0.02 |
|
||||
| Imagen 4 Standard | $0.04 |
|
||||
| Imagen 4 Ultra | $0.06 |
|
||||
| Gemini 2.5 Flash Image (`gemini-2.5-flash-image`) | $0.039 |
|
||||
|
||||
**Free tier for Imagen:** None. Paid tier only.
|
||||
|
||||
To select the Gemini backend through the governed `image_selector`, pass
|
||||
`preferred_provider: "google_imagen"` and
|
||||
`model_name: "gemini-2.5-flash-image"`. The selector maps its neutral
|
||||
`model_name` field to the provider's `model` input.
|
||||
|
||||
#### Gemini Omni Video Pricing
|
||||
|
||||
| Model | Price | Notes |
|
||||
|
||||
@@ -188,6 +188,7 @@ stages:
|
||||
- fal_3d
|
||||
- blender_world
|
||||
- music_gen
|
||||
- fal_elevenlabs_music
|
||||
tools_available:
|
||||
- tts_selector
|
||||
- image_selector
|
||||
@@ -201,6 +202,7 @@ stages:
|
||||
- fal_3d
|
||||
- blender_world
|
||||
- music_gen
|
||||
- fal_elevenlabs_music
|
||||
checkpoint_required: true
|
||||
human_approval_default: true
|
||||
review_focus:
|
||||
|
||||
@@ -31,7 +31,7 @@ Quick routing for common animation-pipeline needs:
|
||||
|-------|----------|---------|
|
||||
| Schema | `schemas/artifacts/asset_manifest.schema.json` | Artifact validation |
|
||||
| Prior artifacts | `state.artifacts["scene_plan"]["scene_plan"]`, `state.artifacts["script"]["script"]`, `state.artifacts["proposal"]["proposal_packet"]` | Tool path and beat map |
|
||||
| Tools | `tts_selector`, `image_selector`, `video_selector`, `math_animate`, `diagram_gen`, `code_snippet`, `threejs_world`, `music_gen` — selectors auto-discover all available providers from the registry | Asset production options |
|
||||
| Tools | `tts_selector`, `image_selector`, `video_selector`, `math_animate`, `diagram_gen`, `code_snippet`, `threejs_world`, `music_gen`, `fal_elevenlabs_music` — selectors auto-discover all available providers from the registry | Asset production options |
|
||||
| Playbook | Active style playbook | Visual consistency |
|
||||
|
||||
## Process
|
||||
|
||||
@@ -694,6 +694,7 @@ class TestCapabilityMetadata:
|
||||
"dashscope",
|
||||
"doubao",
|
||||
"elevenlabs",
|
||||
"fal.ai",
|
||||
"google_tts",
|
||||
"kling_official",
|
||||
"openai",
|
||||
|
||||
103
tests/tools/test_fal_elevenlabs_music.py
Normal file
103
tests/tools/test_fal_elevenlabs_music.py
Normal file
@@ -0,0 +1,103 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from tools.audio.fal_elevenlabs_music import FalElevenLabsMusic
|
||||
from tools.base_tool import ToolStatus
|
||||
from tools.tool_registry import ToolRegistry
|
||||
|
||||
|
||||
def _response(*, json_data=None, content=b""):
|
||||
response = MagicMock()
|
||||
response.json.return_value = json_data
|
||||
response.content = content
|
||||
response.raise_for_status.return_value = None
|
||||
return response
|
||||
|
||||
|
||||
def test_contract_and_rounded_cost(monkeypatch):
|
||||
tool = FalElevenLabsMusic()
|
||||
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
assert tool.get_status() == ToolStatus.AVAILABLE
|
||||
assert tool.estimate_cost({"duration_seconds": 20}) == 0.80
|
||||
assert tool.estimate_cost({"duration_seconds": 61}) == 1.60
|
||||
assert tool.get_info()["capability"] == "music_generation"
|
||||
assert tool.get_info()["provider"] == "fal.ai"
|
||||
|
||||
|
||||
def test_registry_discovers_provider(monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
registry = ToolRegistry()
|
||||
registry.discover()
|
||||
|
||||
tool = registry.get("fal_elevenlabs_music")
|
||||
assert tool is not None
|
||||
assert tool.get_status() == ToolStatus.AVAILABLE
|
||||
|
||||
|
||||
def test_execute_submits_once_and_downloads_audio(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
output_path = tmp_path / "music.mp3"
|
||||
tool = FalElevenLabsMusic()
|
||||
tool._POLL_INTERVAL_SECONDS = 0
|
||||
|
||||
post_response = _response(
|
||||
json_data={
|
||||
"status_url": "https://queue.example/status",
|
||||
"response_url": "https://queue.example/result",
|
||||
}
|
||||
)
|
||||
status_response = _response(json_data={"status": "COMPLETED"})
|
||||
result_response = _response(
|
||||
json_data={"audio": {"url": "https://media.example/music.mp3"}}
|
||||
)
|
||||
audio_response = _response(content=b"fake-mp3")
|
||||
|
||||
with (
|
||||
patch("requests.post", return_value=post_response) as mock_post,
|
||||
patch(
|
||||
"requests.get",
|
||||
side_effect=[status_response, result_response, audio_response],
|
||||
) as mock_get,
|
||||
):
|
||||
result = tool.execute(
|
||||
{
|
||||
"prompt": "gentle felt piano",
|
||||
"duration_seconds": 20,
|
||||
"force_instrumental": True,
|
||||
"output_path": str(output_path),
|
||||
}
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert result.cost_usd == 0.80
|
||||
assert result.model == "fal-ai/elevenlabs/music"
|
||||
assert output_path.read_bytes() == b"fake-mp3"
|
||||
assert mock_post.call_count == 1
|
||||
assert mock_post.call_args.kwargs["json"]["music_length_ms"] == 20000
|
||||
assert mock_post.call_args.kwargs["json"]["force_instrumental"] is True
|
||||
assert mock_get.call_count == 3
|
||||
|
||||
|
||||
def test_execute_rejects_missing_duration_without_request(monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
with patch("requests.post") as mock_post:
|
||||
result = FalElevenLabsMusic().execute({"prompt": "gentle piano"})
|
||||
|
||||
assert result.success is False
|
||||
assert result.error == "duration_seconds is required"
|
||||
mock_post.assert_not_called()
|
||||
|
||||
|
||||
def test_error_redacts_fal_key(monkeypatch):
|
||||
secret = "test-production-shaped-fal-key"
|
||||
monkeypatch.setenv("FAL_KEY", secret)
|
||||
with patch("requests.post", side_effect=RuntimeError(f"request used {secret}")):
|
||||
result = FalElevenLabsMusic().execute(
|
||||
{"prompt": "gentle piano", "duration_seconds": 20}
|
||||
)
|
||||
|
||||
assert result.success is False
|
||||
assert secret not in result.error
|
||||
assert "[REDACTED]" in result.error
|
||||
132
tests/tools/test_fal_elevenlabs_tts.py
Normal file
132
tests/tools/test_fal_elevenlabs_tts.py
Normal file
@@ -0,0 +1,132 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
from tools.audio.fal_elevenlabs_tts import FalElevenLabsTTS
|
||||
from tools.base_tool import ToolStatus
|
||||
from tools.tool_registry import ToolRegistry
|
||||
|
||||
|
||||
def _response(*, json_data=None, content=b""):
|
||||
response = MagicMock()
|
||||
response.json.return_value = json_data
|
||||
response.content = content
|
||||
response.raise_for_status.return_value = None
|
||||
return response
|
||||
|
||||
|
||||
def test_contract_models_and_cost(monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
tool = FalElevenLabsTTS()
|
||||
|
||||
assert tool.get_status() == ToolStatus.AVAILABLE
|
||||
assert tool.get_info()["capability"] == "tts"
|
||||
assert tool.get_info()["provider"] == "fal.ai"
|
||||
assert tool.estimate_cost({"text": "a" * 1000, "model_id": "eleven-v3"}) == 0.1
|
||||
assert tool.estimate_cost({"text": "a" * 1000, "model_id": "multilingual-v2"}) == 0.1
|
||||
assert tool.estimate_cost({"text": "a" * 1000, "model_id": "turbo-v2.5"}) == 0.05
|
||||
|
||||
|
||||
def test_registry_discovers_fal_tts(monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
registry = ToolRegistry()
|
||||
registry.discover()
|
||||
|
||||
tool = registry.get("fal_elevenlabs_tts")
|
||||
assert tool is not None
|
||||
assert tool.get_status() == ToolStatus.AVAILABLE
|
||||
|
||||
|
||||
def test_tts_selector_routes_to_fal_provider(monkeypatch):
|
||||
from tools.audio.tts_selector import TTSSelector
|
||||
from tools.base_tool import ToolResult
|
||||
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
tool = FalElevenLabsTTS()
|
||||
selector = TTSSelector()
|
||||
monkeypatch.setattr(selector, "_providers", lambda: [tool])
|
||||
monkeypatch.setattr(
|
||||
selector,
|
||||
"_select_best_tool",
|
||||
lambda _inputs, _candidates, _context: (tool, None),
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
tool,
|
||||
"execute",
|
||||
lambda inputs: ToolResult(success=True, data={"received": inputs}),
|
||||
)
|
||||
result = selector.execute(
|
||||
{"text": "hello", "preferred_provider": "fal.ai", "voice_id": "Rachel"}
|
||||
)
|
||||
assert result.success
|
||||
assert result.data["selected_tool"] == "fal_elevenlabs_tts"
|
||||
assert result.data["selected_provider"] == "fal.ai"
|
||||
|
||||
|
||||
def test_execute_submits_once_and_downloads_audio(tmp_path, monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
output_path = tmp_path / "speech.mp3"
|
||||
tool = FalElevenLabsTTS()
|
||||
tool._POLL_INTERVAL_SECONDS = 0
|
||||
|
||||
post_response = _response(
|
||||
json_data={
|
||||
"status_url": "https://queue.example/status",
|
||||
"response_url": "https://queue.example/result",
|
||||
}
|
||||
)
|
||||
status_response = _response(json_data={"status": "COMPLETED"})
|
||||
result_response = _response(
|
||||
json_data={"audio": {"url": "https://media.example/speech.mp3"}}
|
||||
)
|
||||
audio_response = _response(content=b"fake-mp3")
|
||||
|
||||
with (
|
||||
patch("requests.post", return_value=post_response) as mock_post,
|
||||
patch(
|
||||
"requests.get",
|
||||
side_effect=[status_response, result_response, audio_response],
|
||||
) as mock_get,
|
||||
):
|
||||
result = tool.execute(
|
||||
{
|
||||
"text": "A calm, measured test.",
|
||||
"voice_id": "Rachel",
|
||||
"model_id": "eleven-v3",
|
||||
"stability": 0.65,
|
||||
"language_code": "en",
|
||||
"output_path": str(output_path),
|
||||
}
|
||||
)
|
||||
|
||||
assert result.success is True
|
||||
assert result.model == "fal-ai/elevenlabs/tts/eleven-v3"
|
||||
assert output_path.read_bytes() == b"fake-mp3"
|
||||
assert mock_post.call_count == 1
|
||||
assert mock_post.call_args.args[0].endswith("/fal-ai/elevenlabs/tts/eleven-v3")
|
||||
assert mock_post.call_args.kwargs["json"]["voice"] == "Rachel"
|
||||
assert mock_post.call_args.kwargs["json"]["stability"] == 0.65
|
||||
assert mock_get.call_count == 3
|
||||
|
||||
|
||||
def test_invalid_model_does_not_submit(monkeypatch):
|
||||
monkeypatch.setenv("FAL_KEY", "test-key")
|
||||
with patch("requests.post") as mock_post:
|
||||
result = FalElevenLabsTTS().execute(
|
||||
{"text": "hello", "model_id": "not-a-model"}
|
||||
)
|
||||
|
||||
assert result.success is False
|
||||
assert "model_id must be one of" in result.error
|
||||
mock_post.assert_not_called()
|
||||
|
||||
|
||||
def test_error_redacts_fal_key(monkeypatch):
|
||||
secret = "test-production-shaped-fal-key"
|
||||
monkeypatch.setenv("FAL_KEY", secret)
|
||||
with patch("requests.post", side_effect=RuntimeError(f"request used {secret}")):
|
||||
result = FalElevenLabsTTS().execute({"text": "hello"})
|
||||
|
||||
assert result.success is False
|
||||
assert secret not in result.error
|
||||
assert "[REDACTED]" in result.error
|
||||
156
tests/tools/test_google_imagen_gemini_backend.py
Normal file
156
tests/tools/test_google_imagen_gemini_backend.py
Normal file
@@ -0,0 +1,156 @@
|
||||
"""Tests for the Gemini image backend in google_imagen.
|
||||
|
||||
Models named `gemini-*` (e.g. gemini-2.5-flash-image) are not served by the
|
||||
Imagen `:predict` endpoint — they generate images through generate_content
|
||||
with an image_config. This backend matters on Vertex projects that have no
|
||||
Imagen catalog access, where it is the only working Google image path.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import types as pytypes
|
||||
from pathlib import Path
|
||||
|
||||
import pytest
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
|
||||
class _FakeInline:
|
||||
def __init__(self, data: bytes):
|
||||
self.data = data
|
||||
|
||||
|
||||
class _FakePart:
|
||||
def __init__(self, data: bytes):
|
||||
self.inline_data = _FakeInline(data)
|
||||
|
||||
|
||||
class _FakeContent:
|
||||
def __init__(self, parts):
|
||||
self.parts = parts
|
||||
|
||||
|
||||
class _FakeCandidate:
|
||||
def __init__(self, parts):
|
||||
self.content = _FakeContent(parts)
|
||||
|
||||
|
||||
class _FakeResponse:
|
||||
def __init__(self, parts):
|
||||
self.candidates = [_FakeCandidate(parts)]
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def imagen_tool(monkeypatch):
|
||||
monkeypatch.setenv("GOOGLE_API_KEY", "test-key")
|
||||
|
||||
calls: list[dict] = []
|
||||
|
||||
class _FakeModels:
|
||||
def generate_content(self, model=None, contents=None, config=None):
|
||||
calls.append({"model": model, "contents": contents, "config": config})
|
||||
return _FakeResponse([_FakePart(b"GEMINI_IMG")])
|
||||
|
||||
class _FakeClient:
|
||||
models = _FakeModels()
|
||||
|
||||
import tools.google_credentials as gc
|
||||
|
||||
monkeypatch.setattr(
|
||||
gc, "get_genai_client", lambda http_options=None, location=None: _FakeClient()
|
||||
)
|
||||
|
||||
from tools.graphics.google_imagen import GoogleImagen
|
||||
|
||||
return GoogleImagen(), calls
|
||||
|
||||
|
||||
def test_gemini_model_routes_to_generate_content(imagen_tool, tmp_path):
|
||||
tool, calls = imagen_tool
|
||||
out = tmp_path / "img.png"
|
||||
|
||||
result = tool.execute(
|
||||
{
|
||||
"prompt": "a flower",
|
||||
"model": "gemini-2.5-flash-image",
|
||||
"aspect_ratio": "16:9",
|
||||
"output_path": str(out),
|
||||
}
|
||||
)
|
||||
|
||||
assert result.success
|
||||
assert result.data["model"] == "gemini-2.5-flash-image"
|
||||
assert out.read_bytes() == b"GEMINI_IMG"
|
||||
|
||||
assert len(calls) == 1
|
||||
assert calls[0]["model"] == "gemini-2.5-flash-image"
|
||||
# Aspect ratio must reach the API through image_config, not be dropped.
|
||||
assert calls[0]["config"].image_config.aspect_ratio == "16:9"
|
||||
|
||||
|
||||
def test_image_selector_maps_model_name_to_google_model(
|
||||
imagen_tool, monkeypatch, tmp_path
|
||||
):
|
||||
"""The governed selector must be able to reach the Gemini backend."""
|
||||
from tools.graphics.image_selector import ImageSelector
|
||||
|
||||
tool, calls = imagen_tool
|
||||
selector = ImageSelector()
|
||||
monkeypatch.setattr(selector, "_providers", lambda: [tool])
|
||||
|
||||
result = selector.execute(
|
||||
{
|
||||
"prompt": "a flower",
|
||||
"preferred_provider": "google_imagen",
|
||||
"model_name": "gemini-2.5-flash-image",
|
||||
"output_path": str(tmp_path / "selected.png"),
|
||||
}
|
||||
)
|
||||
|
||||
assert result.success, result.error
|
||||
assert calls[0]["model"] == "gemini-2.5-flash-image"
|
||||
assert result.data["selected_tool"] == "google_imagen"
|
||||
assert result.data["model"] == "gemini-2.5-flash-image"
|
||||
|
||||
|
||||
def test_gemini_cost_estimate_is_per_image():
|
||||
from tools.graphics.google_imagen import GoogleImagen
|
||||
|
||||
tool = GoogleImagen()
|
||||
assert tool.estimate_cost(
|
||||
{"model": "gemini-2.5-flash-image", "number_of_images": 2}
|
||||
) == pytest.approx(0.039 * 2)
|
||||
|
||||
|
||||
def test_text_only_response_is_a_clear_error(monkeypatch, tmp_path):
|
||||
monkeypatch.setenv("GOOGLE_API_KEY", "test-key")
|
||||
|
||||
class _TextPart:
|
||||
inline_data = None
|
||||
|
||||
class _FakeModels:
|
||||
def generate_content(self, model=None, contents=None, config=None):
|
||||
return _FakeResponse([_TextPart()])
|
||||
|
||||
class _FakeClient:
|
||||
models = _FakeModels()
|
||||
|
||||
import tools.google_credentials as gc
|
||||
|
||||
monkeypatch.setattr(
|
||||
gc, "get_genai_client", lambda http_options=None, location=None: _FakeClient()
|
||||
)
|
||||
|
||||
from tools.graphics.google_imagen import GoogleImagen
|
||||
|
||||
result = GoogleImagen().execute(
|
||||
{
|
||||
"prompt": "a flower",
|
||||
"model": "gemini-2.5-flash-image",
|
||||
"output_path": str(tmp_path / "img.png"),
|
||||
}
|
||||
)
|
||||
|
||||
assert not result.success
|
||||
assert "No image data" in result.error
|
||||
36
tests/tools/test_google_tts_scoped_key.py
Normal file
36
tests/tools/test_google_tts_scoped_key.py
Normal file
@@ -0,0 +1,36 @@
|
||||
from tools.audio.google_tts import GoogleTTS
|
||||
from tools.google_credentials import has_google_credentials
|
||||
|
||||
|
||||
def test_tts_only_key_does_not_enable_shared_google_providers(monkeypatch):
|
||||
monkeypatch.setenv("GOOGLE_TTS_API_KEY", "test-tts-only-key")
|
||||
monkeypatch.delenv("GOOGLE_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GOOGLE_APPLICATION_CREDENTIALS", raising=False)
|
||||
|
||||
assert GoogleTTS().get_status().value == "available"
|
||||
assert has_google_credentials() is False
|
||||
|
||||
|
||||
def test_tts_key_uses_header_and_is_redacted_from_errors(monkeypatch, tmp_path):
|
||||
import requests
|
||||
|
||||
secret = "test-production-shaped-tts-key"
|
||||
monkeypatch.setenv("GOOGLE_TTS_API_KEY", secret)
|
||||
monkeypatch.delenv("GOOGLE_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GEMINI_API_KEY", raising=False)
|
||||
monkeypatch.delenv("GOOGLE_APPLICATION_CREDENTIALS", raising=False)
|
||||
|
||||
def fail_request(url, **kwargs):
|
||||
assert kwargs["headers"]["x-goog-api-key"] == secret
|
||||
assert "params" not in kwargs
|
||||
raise requests.HTTPError(f"403 for {url}?key={secret}")
|
||||
|
||||
monkeypatch.setattr(requests, "post", fail_request)
|
||||
result = GoogleTTS().execute(
|
||||
{"text": "safe test sentence", "output_path": str(tmp_path / "speech.mp3")}
|
||||
)
|
||||
|
||||
assert result.success is False
|
||||
assert secret not in result.error
|
||||
assert "[REDACTED]" in result.error
|
||||
534
tests/tools/test_hunyuan_image.py
Normal file
534
tests/tools/test_hunyuan_image.py
Normal file
@@ -0,0 +1,534 @@
|
||||
"""Unit tests for hunyuan_image — TokenHub 混元生图 3.0 tool."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
PROJECT_ROOT = Path(__file__).resolve().parent.parent.parent
|
||||
sys.path.insert(0, str(PROJECT_ROOT))
|
||||
|
||||
from tools.base_tool import ToolStatus
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Tool discovery & metadata
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_hunyuan_image_is_discovered_by_registry():
|
||||
from tools.tool_registry import ToolRegistry
|
||||
|
||||
registry = ToolRegistry()
|
||||
registry.discover()
|
||||
|
||||
tool = registry.get("hunyuan_image")
|
||||
assert tool is not None
|
||||
assert tool.provider == "hunyuan_cloud"
|
||||
assert tool.capability == "image_generation"
|
||||
assert tool.name == "hunyuan_image"
|
||||
|
||||
|
||||
def test_hunyuan_image_metadata():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
info = tool.get_info()
|
||||
|
||||
assert info["tier"] == "generate"
|
||||
assert info["stability"] == "experimental"
|
||||
assert info["runtime"] == "api"
|
||||
assert "text_to_image" in info["capabilities"]
|
||||
assert info["supports"]["seed"] is True
|
||||
assert info["supports"]["reference_image"] is True
|
||||
assert info["supports"]["prompt_rewrite"] is True
|
||||
assert info["supports"]["negative_prompt"] is False
|
||||
assert "env:TENCENT_TOKENHUB_API_KEY" in info["dependencies"]
|
||||
assert "visual-style" in info["agent_skills"]
|
||||
|
||||
|
||||
def test_idempotency_includes_custom_watermark():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
base = {"prompt": "x"}
|
||||
assert tool.idempotency_key(base) != tool.idempotency_key(
|
||||
{**base, "logo_param": {"logo_url": "https://example.com/logo.png"}}
|
||||
)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Status reporting
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_status_unavailable_when_no_api_key(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.delenv("TENCENT_TOKENHUB_API_KEY", raising=False)
|
||||
assert HunyuanImage().get_status() == ToolStatus.UNAVAILABLE
|
||||
|
||||
|
||||
def test_status_available_when_api_key_set(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "test-key-123")
|
||||
assert HunyuanImage().get_status() == ToolStatus.AVAILABLE
|
||||
|
||||
|
||||
def test_api_key_filters_comment_like_values(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "# this-is-a-comment")
|
||||
assert HunyuanImage()._api_key() is None
|
||||
assert HunyuanImage().get_status() == ToolStatus.UNAVAILABLE
|
||||
|
||||
|
||||
def test_api_key_strips_whitespace(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", " my-key ")
|
||||
assert HunyuanImage()._api_key() == "my-key"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Cost & runtime estimation
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_estimate_cost():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
cost = tool.estimate_cost({})
|
||||
assert cost > 0
|
||||
assert cost == pytest.approx(0.08, rel=0.1)
|
||||
|
||||
|
||||
def test_estimate_runtime():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
runtime = tool.estimate_runtime({})
|
||||
assert runtime == 120.0
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Payload construction (_build_payload)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_build_payload_minimal():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({"prompt": "a cat"})
|
||||
assert payload == {"prompt": "a cat"}
|
||||
|
||||
|
||||
def test_build_payload_all_params():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({
|
||||
"prompt": "a cat",
|
||||
"resolution": "768:1024",
|
||||
"seed": 42,
|
||||
"revise": 0,
|
||||
"logo_add": 1,
|
||||
"images": ["https://example.com/ref.jpg"],
|
||||
})
|
||||
assert payload["prompt"] == "a cat"
|
||||
assert payload["resolution"] == "768:1024"
|
||||
assert payload["seed"] == 42
|
||||
assert payload["revise"] == 0
|
||||
assert payload["logo_add"] == 1
|
||||
assert payload["images"] == ["https://example.com/ref.jpg"]
|
||||
|
||||
|
||||
def test_build_payload_with_logo_param():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({
|
||||
"prompt": "a cat",
|
||||
"logo_param": {"logo_url": "https://example.com/wm.png"},
|
||||
})
|
||||
assert payload["logo_param"] == {"logo_url": "https://example.com/wm.png"}
|
||||
|
||||
|
||||
def test_build_payload_logo_param_skips_empty():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({
|
||||
"prompt": "a cat",
|
||||
"logo_param": {},
|
||||
})
|
||||
assert "logo_param" not in payload
|
||||
|
||||
|
||||
def test_build_payload_omits_none_seed():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({"prompt": "a cat", "seed": None})
|
||||
assert "seed" not in payload
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Image resolution (_resolve_images)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_images_passes_urls_through():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
refs = [
|
||||
"https://example.com/a.jpg",
|
||||
"data:image/png;base64,abc123",
|
||||
]
|
||||
resolved = HunyuanImage._resolve_images(refs)
|
||||
assert resolved == refs
|
||||
|
||||
|
||||
def test_resolve_images_encodes_local_file(tmp_path):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
img = tmp_path / "test.png"
|
||||
img.write_bytes(b"fake-png-data")
|
||||
|
||||
resolved = HunyuanImage._resolve_images([str(img)])
|
||||
assert len(resolved) == 1
|
||||
assert resolved[0].startswith("data:image/png;base64,")
|
||||
|
||||
|
||||
def test_resolve_images_raises_on_missing_file():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
with pytest.raises(FileNotFoundError):
|
||||
HunyuanImage._resolve_images(["/nonexistent/path.jpg"])
|
||||
|
||||
|
||||
def test_resolve_images_raises_on_oversized_file(tmp_path):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
big = tmp_path / "big.jpg"
|
||||
big.write_bytes(b"x" * (7 * 1024 * 1024)) # 7MB > 6MB limit
|
||||
|
||||
with pytest.raises(ValueError, match="too large"):
|
||||
HunyuanImage._resolve_images([str(big)])
|
||||
|
||||
|
||||
def test_resolve_images_detects_mime_from_extension(tmp_path):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
cases = [
|
||||
("ref.jpg", "image/jpeg"),
|
||||
("ref.jpeg", "image/jpeg"),
|
||||
("ref.png", "image/png"),
|
||||
("ref.bmp", "image/bmp"),
|
||||
("ref.tiff", "image/tiff"),
|
||||
("ref.tif", "image/tiff"),
|
||||
("ref.webp", "image/webp"),
|
||||
("ref.unknown", "image/png"), # fallback
|
||||
]
|
||||
for filename, expected_mime in cases:
|
||||
f = tmp_path / filename
|
||||
f.write_bytes(b"data")
|
||||
resolved = HunyuanImage._resolve_images([str(f)])
|
||||
assert resolved[0].startswith(f"data:{expected_mime};base64,")
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Output path resolution (_resolve_output_paths)
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_resolve_output_paths_single():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
paths = HunyuanImage._resolve_output_paths("/out/img.png", 1)
|
||||
assert len(paths) == 1
|
||||
assert paths[0] == Path("/out/img.png")
|
||||
|
||||
|
||||
def test_resolve_output_paths_multi():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
paths = HunyuanImage._resolve_output_paths("/out/img.png", 3)
|
||||
assert len(paths) == 3
|
||||
assert [p.name for p in paths] == ["img_1.png", "img_2.png", "img_3.png"]
|
||||
assert len(set(paths)) == 3
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Auth headers
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_auth_headers():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
headers = HunyuanImage._auth_headers("my-api-key")
|
||||
assert headers["Authorization"] == "Bearer my-api-key"
|
||||
assert headers["Content-Type"] == "application/json"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JSON error handling
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_json_or_raise_parses_valid_json():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
class FakeResp:
|
||||
status_code = 200
|
||||
|
||||
def json(self):
|
||||
return {"status": "ok"}
|
||||
|
||||
assert HunyuanImage._json_or_raise(FakeResp()) == {"status": "ok"}
|
||||
|
||||
|
||||
def test_json_or_raise_raises_on_invalid_json():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
class FakeResp:
|
||||
status_code = 500
|
||||
|
||||
def json(self):
|
||||
raise ValueError("not json")
|
||||
|
||||
with pytest.raises(RuntimeError, match="Non-JSON response"):
|
||||
HunyuanImage._json_or_raise(FakeResp())
|
||||
|
||||
|
||||
def test_check_response_passes_clean_payload():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
HunyuanImage._check_response({"status": "completed"}) # no error -> no raise
|
||||
|
||||
|
||||
def test_check_response_raises_on_error_field():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
with pytest.raises(RuntimeError, match="TokenHub API error"):
|
||||
HunyuanImage._check_response({
|
||||
"error": {"code": "AUTH_FAILED", "message": "invalid key"},
|
||||
})
|
||||
|
||||
|
||||
def test_check_response_raises_on_error_without_code():
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
with pytest.raises(RuntimeError, match="TokenHub API error"):
|
||||
HunyuanImage._check_response({"error": {"message": "something broke"}})
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Safe error redaction
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_safe_error_redacts_api_key(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "secret-key-abc")
|
||||
msg = HunyuanImage._safe_error(Exception("failed with secret-key-abc"))
|
||||
assert "secret-key-abc" not in msg
|
||||
assert "[redacted]" in msg
|
||||
|
||||
|
||||
def test_safe_error_preserves_other_text(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "sk-123")
|
||||
msg = HunyuanImage._safe_error(Exception("network timeout: connection refused"))
|
||||
assert "network timeout" in msg
|
||||
assert "sk-123" not in msg
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Execute guards
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_execute_returns_error_without_api_key(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.delenv("TENCENT_TOKENHUB_API_KEY", raising=False)
|
||||
result = HunyuanImage().execute({"prompt": "a cat"})
|
||||
assert not result.success
|
||||
assert "TENCENT_TOKENHUB_API_KEY" in result.error
|
||||
|
||||
|
||||
def test_image_selector_maps_shared_reference_input(monkeypatch, tmp_path):
|
||||
from tools.base_tool import ToolResult
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
from tools.graphics.image_selector import ImageSelector
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "test-key")
|
||||
tool = HunyuanImage()
|
||||
selector = ImageSelector()
|
||||
monkeypatch.setattr(selector, "_providers", lambda: [tool])
|
||||
monkeypatch.setattr(
|
||||
selector,
|
||||
"_select_best_tool",
|
||||
lambda _inputs, _candidates, _context: (tool, None),
|
||||
)
|
||||
observed = {}
|
||||
|
||||
def fake_execute(inputs):
|
||||
observed.update(inputs)
|
||||
return ToolResult(success=True, data={}, artifacts=[inputs["output_path"]])
|
||||
|
||||
monkeypatch.setattr(tool, "execute", fake_execute)
|
||||
result = selector.execute(
|
||||
{
|
||||
"prompt": "adapt this frame",
|
||||
"preferred_provider": "hunyuan_cloud",
|
||||
"image_path": str(tmp_path / "reference.png"),
|
||||
"output_path": str(tmp_path / "out.png"),
|
||||
}
|
||||
)
|
||||
assert result.success
|
||||
assert observed["images"] == [str(tmp_path / "reference.png")]
|
||||
assert result.data["selected_tool"] == "hunyuan_image"
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Dry run
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_dry_run_no_side_effects(monkeypatch):
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "test-key")
|
||||
tool = HunyuanImage()
|
||||
info = tool.dry_run({"prompt": "a cat"})
|
||||
assert info["tool"] == "hunyuan_image"
|
||||
assert info["estimated_cost_usd"] > 0
|
||||
assert info["would_execute"] is True
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# End-to-end with mocked API
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
def test_execute_full_flow_with_mocked_api(monkeypatch, tmp_path):
|
||||
"""Simulate the full submit → poll → download flow."""
|
||||
from tools.graphics.hunyuan_image import HunyuanImage, _MODEL, _HOST
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "test-key")
|
||||
|
||||
# Track calls across all mocked endpoints
|
||||
api_calls = []
|
||||
poll_count = [0] # mutable counter for poll iteration
|
||||
|
||||
class _FakeResp:
|
||||
status_code = 200
|
||||
def __init__(self, data, content=None):
|
||||
self._data = data
|
||||
self.content = content
|
||||
def json(self):
|
||||
return self._data
|
||||
def raise_for_status(self):
|
||||
pass
|
||||
|
||||
submit_url = f"https://{_HOST}/v1/api/image/submit"
|
||||
query_url = f"https://{_HOST}/v1/api/image/query"
|
||||
|
||||
# Safety sentinel: if a real network call slips through the mock,
|
||||
# this flag is flipped and we fail-fast instead of leaking files.
|
||||
mock_active = [False]
|
||||
|
||||
def fake_post(url, *, json, headers, timeout):
|
||||
mock_active[0] = True
|
||||
api_calls.append(("post", url, json))
|
||||
if "submit" in url:
|
||||
return _FakeResp({"id": "job-001", "status": "queued"})
|
||||
elif "query" in url:
|
||||
poll_count[0] += 1
|
||||
if poll_count[0] == 1:
|
||||
return _FakeResp({"status": "running"})
|
||||
return _FakeResp({
|
||||
"status": "completed",
|
||||
"data": [{"url": "https://example.com/result.png"}],
|
||||
})
|
||||
raise RuntimeError(f"Unexpected URL: {url}")
|
||||
|
||||
def fake_get(url, timeout):
|
||||
mock_active[0] = True
|
||||
api_calls.append(("get", url))
|
||||
return _FakeResp({}, content=b"fake-image-data")
|
||||
|
||||
with (
|
||||
patch("requests.post", side_effect=fake_post),
|
||||
patch("requests.get", side_effect=fake_get),
|
||||
):
|
||||
out = tmp_path / "gen.png"
|
||||
result = HunyuanImage().execute({
|
||||
"prompt": "a programmer coding",
|
||||
"resolution": "1024:1024",
|
||||
"seed": 12345,
|
||||
"revise": 1,
|
||||
"logo_add": 0,
|
||||
"output_path": str(out),
|
||||
})
|
||||
|
||||
# Guard: mock must have been exercised — if not, a real API call leaked
|
||||
assert mock_active[0], (
|
||||
"Mock was never triggered — a real API call may have leaked. "
|
||||
"Check that requests.post / requests.get patching is effective."
|
||||
)
|
||||
|
||||
assert result.success, result.error
|
||||
assert result.data["provider"] == "hunyuan_cloud"
|
||||
assert result.data["model"] == _MODEL
|
||||
assert result.data["task_id"] == "job-001"
|
||||
assert result.data["resolution"] == "1024:1024"
|
||||
assert result.data["images_generated"] == 1
|
||||
assert result.artifacts == [str(out)]
|
||||
assert out.read_bytes() == b"fake-image-data"
|
||||
|
||||
# Verify submit payload was correct
|
||||
submit_calls = [c for c in api_calls if "submit" in c[1]]
|
||||
assert len(submit_calls) == 1
|
||||
_, _, submit_body = submit_calls[0]
|
||||
assert submit_body["prompt"] == "a programmer coding"
|
||||
assert submit_body["resolution"] == "1024:1024"
|
||||
assert submit_body["seed"] == 12345
|
||||
assert submit_body["revise"] == 1
|
||||
assert submit_body["logo_add"] == 0
|
||||
assert submit_body["model"] == _MODEL
|
||||
|
||||
# Verify polling happened (submit + at least 1 query + download)
|
||||
assert any("query" in c[1] for c in api_calls)
|
||||
assert any(c[0] == "get" for c in api_calls)
|
||||
|
||||
|
||||
def test_execute_with_local_reference_images(monkeypatch, tmp_path):
|
||||
"""Reference images from local paths should be base64-encoded in payload."""
|
||||
from tools.graphics.hunyuan_image import HunyuanImage
|
||||
|
||||
monkeypatch.setenv("TENCENT_TOKENHUB_API_KEY", "test-key")
|
||||
|
||||
ref_img = tmp_path / "ref.png"
|
||||
ref_img.write_bytes(b"reference-data")
|
||||
|
||||
tool = HunyuanImage()
|
||||
payload = tool._build_payload({
|
||||
"prompt": "enhance this",
|
||||
"images": [str(ref_img)],
|
||||
})
|
||||
|
||||
assert "images" in payload
|
||||
assert len(payload["images"]) == 1
|
||||
assert payload["images"][0].startswith("data:image/png;base64,")
|
||||
@@ -36,7 +36,9 @@ class ElevenLabsTTS(BaseTool):
|
||||
install_instructions = (
|
||||
"Set the ELEVENLABS_API_KEY environment variable:\n"
|
||||
" export ELEVENLABS_API_KEY=your_key_here\n"
|
||||
"Get a key at https://elevenlabs.io"
|
||||
"Get a key at https://elevenlabs.io\n"
|
||||
"If fal_elevenlabs_tts is available, use it instead to access ElevenLabs "
|
||||
"speech through fal.ai without a separate ElevenLabs key."
|
||||
)
|
||||
fallback = "openai_tts"
|
||||
fallback_tools = ["openai_tts", "piper_tts"]
|
||||
|
||||
231
tools/audio/fal_elevenlabs_music.py
Normal file
231
tools/audio/fal_elevenlabs_music.py
Normal file
@@ -0,0 +1,231 @@
|
||||
"""Generate music with ElevenLabs Music through fal.ai.
|
||||
|
||||
This provider uses OpenMontage's shared ``FAL_KEY`` credential and the fal.ai
|
||||
queue API, then downloads the generated MP3 to a project-local path.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from tools.base_tool import (
|
||||
BaseTool,
|
||||
Determinism,
|
||||
ExecutionMode,
|
||||
ResourceProfile,
|
||||
RetryPolicy,
|
||||
ToolResult,
|
||||
ToolRuntime,
|
||||
ToolStability,
|
||||
ToolStatus,
|
||||
ToolTier,
|
||||
)
|
||||
|
||||
|
||||
class FalElevenLabsMusic(BaseTool):
|
||||
"""Generate a precisely timed music track through fal.ai."""
|
||||
|
||||
name = "fal_elevenlabs_music"
|
||||
version = "0.1.0"
|
||||
tier = ToolTier.GENERATE
|
||||
capability = "music_generation"
|
||||
provider = "fal.ai"
|
||||
stability = ToolStability.BETA
|
||||
execution_mode = ExecutionMode.ASYNC
|
||||
determinism = Determinism.STOCHASTIC
|
||||
runtime = ToolRuntime.API
|
||||
|
||||
dependencies = ["env:FAL_KEY"]
|
||||
install_instructions = (
|
||||
"Set FAL_KEY to a fal.ai API key. "
|
||||
"Get one at https://fal.ai/dashboard/keys"
|
||||
)
|
||||
fallback_tools = ["music_gen", "google_music"]
|
||||
agent_skills = ["music", "elevenlabs"]
|
||||
|
||||
capabilities = [
|
||||
"generate_background_music",
|
||||
"generate_instrumental",
|
||||
]
|
||||
supports = {
|
||||
"instrumental": True,
|
||||
"style_control": True,
|
||||
"exact_duration": True,
|
||||
}
|
||||
best_for = [
|
||||
"precisely timed instrumental background music",
|
||||
"high-quality ElevenLabs Music through a shared fal.ai account",
|
||||
"short-form social video soundtracks",
|
||||
]
|
||||
not_good_for = [
|
||||
"offline generation",
|
||||
"free music sourcing",
|
||||
"sub-3-second sound effects",
|
||||
]
|
||||
|
||||
input_schema = {
|
||||
"type": "object",
|
||||
"required": ["prompt", "duration_seconds"],
|
||||
"properties": {
|
||||
"prompt": {
|
||||
"type": "string",
|
||||
"description": "Music description including mood, style, and instruments",
|
||||
},
|
||||
"duration_seconds": {
|
||||
"type": "number",
|
||||
"minimum": 3,
|
||||
"maximum": 600,
|
||||
"description": "Exact target length in seconds",
|
||||
},
|
||||
"force_instrumental": {
|
||||
"type": "boolean",
|
||||
"default": True,
|
||||
"description": "Generate music without vocals",
|
||||
},
|
||||
"output_path": {
|
||||
"type": "string",
|
||||
"default": "fal_music_output.mp3",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resource_profile = ResourceProfile(
|
||||
cpu_cores=1,
|
||||
ram_mb=256,
|
||||
vram_mb=0,
|
||||
disk_mb=50,
|
||||
network_required=True,
|
||||
)
|
||||
retry_policy = RetryPolicy(
|
||||
max_retries=0,
|
||||
retryable_errors=["rate_limit", "timeout"],
|
||||
)
|
||||
idempotency_key_fields = ["prompt", "duration_seconds", "force_instrumental"]
|
||||
side_effects = [
|
||||
"writes an MP3 file to output_path",
|
||||
"submits one paid fal.ai generation request",
|
||||
]
|
||||
user_visible_verification = [
|
||||
"Listen to the generated music for mood, mix, and duration",
|
||||
]
|
||||
|
||||
_MODEL = "fal-ai/elevenlabs/music"
|
||||
_QUEUE_URL = f"https://queue.fal.run/{_MODEL}"
|
||||
_POLL_INTERVAL_SECONDS = 5
|
||||
_MAX_WAIT_SECONDS = 600
|
||||
|
||||
def _get_api_key(self) -> str | None:
|
||||
return os.environ.get("FAL_KEY") or os.environ.get("FAL_AI_API_KEY")
|
||||
|
||||
def get_status(self) -> ToolStatus:
|
||||
return ToolStatus.AVAILABLE if self._get_api_key() else ToolStatus.UNAVAILABLE
|
||||
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
"""fal bills $0.80 per output minute, rounded up to a full minute."""
|
||||
duration = inputs.get("duration_seconds")
|
||||
if duration is None:
|
||||
raise ValueError("duration_seconds is required for cost estimation")
|
||||
return round(math.ceil(float(duration) / 60.0) * 0.80, 2)
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
api_key = self._get_api_key()
|
||||
if not api_key:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="No fal.ai API key found. " + self.install_instructions,
|
||||
)
|
||||
|
||||
duration = inputs.get("duration_seconds")
|
||||
if duration is None:
|
||||
return ToolResult(success=False, error="duration_seconds is required")
|
||||
duration = float(duration)
|
||||
if not 3 <= duration <= 600:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="duration_seconds must be between 3 and 600",
|
||||
)
|
||||
|
||||
import requests
|
||||
|
||||
started = time.time()
|
||||
headers = {
|
||||
"Authorization": f"Key {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
payload = {
|
||||
"prompt": inputs["prompt"],
|
||||
"music_length_ms": round(duration * 1000),
|
||||
"force_instrumental": bool(inputs.get("force_instrumental", True)),
|
||||
}
|
||||
|
||||
try:
|
||||
submit_response = requests.post(
|
||||
self._QUEUE_URL,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
submit_response.raise_for_status()
|
||||
queue_data = submit_response.json()
|
||||
status_url = queue_data["status_url"]
|
||||
response_url = queue_data["response_url"]
|
||||
|
||||
deadline = time.monotonic() + self._MAX_WAIT_SECONDS
|
||||
while True:
|
||||
if time.monotonic() >= deadline:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="fal.ai music generation timed out while waiting in the queue",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
time.sleep(self._POLL_INTERVAL_SECONDS)
|
||||
status_response = requests.get(status_url, headers=headers, timeout=20)
|
||||
status_response.raise_for_status()
|
||||
status = status_response.json().get("status", "UNKNOWN")
|
||||
if status == "COMPLETED":
|
||||
break
|
||||
if status in {"FAILED", "CANCELLED"}:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"fal.ai music generation {status.lower()}",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
|
||||
result_response = requests.get(response_url, headers=headers, timeout=30)
|
||||
result_response.raise_for_status()
|
||||
result_data = result_response.json()
|
||||
audio_url = result_data["audio"]["url"]
|
||||
|
||||
audio_response = requests.get(audio_url, timeout=120)
|
||||
audio_response.raise_for_status()
|
||||
output_path = Path(inputs.get("output_path", "fal_music_output.mp3"))
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(audio_response.content)
|
||||
except Exception as exc:
|
||||
safe_error = str(exc).replace(api_key, "[REDACTED]")
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"fal.ai ElevenLabs Music generation failed: {safe_error}",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
data={
|
||||
"provider": "fal.ai",
|
||||
"model": self._MODEL,
|
||||
"prompt": inputs["prompt"],
|
||||
"duration_seconds": duration,
|
||||
"force_instrumental": payload["force_instrumental"],
|
||||
"output": str(output_path),
|
||||
"format": "mp3",
|
||||
},
|
||||
artifacts=[str(output_path)],
|
||||
cost_usd=self.estimate_cost(inputs),
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
model=self._MODEL,
|
||||
)
|
||||
359
tools/audio/fal_elevenlabs_tts.py
Normal file
359
tools/audio/fal_elevenlabs_tts.py
Normal file
@@ -0,0 +1,359 @@
|
||||
"""Generate ElevenLabs speech through fal.ai using the shared FAL credential."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from tools.base_tool import (
|
||||
BaseTool,
|
||||
Determinism,
|
||||
ExecutionMode,
|
||||
ResourceProfile,
|
||||
RetryPolicy,
|
||||
ToolResult,
|
||||
ToolRuntime,
|
||||
ToolStability,
|
||||
ToolStatus,
|
||||
ToolTier,
|
||||
)
|
||||
|
||||
|
||||
class FalElevenLabsTTS(BaseTool):
|
||||
"""Generate expressive narration with ElevenLabs models hosted by fal.ai."""
|
||||
|
||||
name = "fal_elevenlabs_tts"
|
||||
version = "0.1.0"
|
||||
tier = ToolTier.VOICE
|
||||
capability = "tts"
|
||||
provider = "fal.ai"
|
||||
stability = ToolStability.BETA
|
||||
execution_mode = ExecutionMode.ASYNC
|
||||
determinism = Determinism.STOCHASTIC
|
||||
runtime = ToolRuntime.API
|
||||
|
||||
dependencies = ["env:FAL_KEY"]
|
||||
install_instructions = (
|
||||
"Set FAL_KEY to a fal.ai API key. No separate ElevenLabs key is needed. "
|
||||
"Get a fal.ai key at https://fal.ai/dashboard/keys"
|
||||
)
|
||||
fallback_tools = ["google_tts", "piper_tts", "elevenlabs_tts"]
|
||||
agent_skills = ["elevenlabs"]
|
||||
|
||||
capabilities = [
|
||||
"text_to_speech",
|
||||
"voice_selection",
|
||||
"expressive_delivery",
|
||||
"multilingual",
|
||||
"word_timestamps",
|
||||
]
|
||||
supports = {
|
||||
"voice_cloning": False,
|
||||
"multilingual": True,
|
||||
"offline": False,
|
||||
"native_audio": True,
|
||||
"inline_audio_tags": True,
|
||||
"word_timestamps": True,
|
||||
}
|
||||
best_for = [
|
||||
"expressive ElevenLabs narration through an existing fal.ai connection",
|
||||
"emotionally directed voiceover with Eleven v3 audio tags",
|
||||
"multilingual narration without a separate ElevenLabs credential",
|
||||
]
|
||||
not_good_for = [
|
||||
"offline generation",
|
||||
"voice cloning or private custom ElevenLabs voices",
|
||||
]
|
||||
|
||||
_MODELS = {
|
||||
"eleven-v3": "fal-ai/elevenlabs/tts/eleven-v3",
|
||||
"multilingual-v2": "fal-ai/elevenlabs/tts/multilingual-v2",
|
||||
"turbo-v2.5": "fal-ai/elevenlabs/tts/turbo-v2.5",
|
||||
}
|
||||
_MODEL_ALIASES = {
|
||||
"eleven_v3": "eleven-v3",
|
||||
"eleven_multilingual_v2": "multilingual-v2",
|
||||
"multilingual_v2": "multilingual-v2",
|
||||
"eleven_turbo_v2_5": "turbo-v2.5",
|
||||
"turbo_v2_5": "turbo-v2.5",
|
||||
**{value: key for key, value in _MODELS.items()},
|
||||
}
|
||||
_PRICE_PER_CHARACTER = {
|
||||
"eleven-v3": 0.0001,
|
||||
"multilingual-v2": 0.0001,
|
||||
"turbo-v2.5": 0.00005,
|
||||
}
|
||||
_POLL_INTERVAL_SECONDS = 2
|
||||
_MAX_WAIT_SECONDS = 300
|
||||
|
||||
input_schema = {
|
||||
"type": "object",
|
||||
"required": ["text"],
|
||||
"properties": {
|
||||
"text": {
|
||||
"type": "string",
|
||||
"description": "Text to speak. Eleven v3 supports inline tags such as [whispers].",
|
||||
},
|
||||
"voice": {
|
||||
"type": "string",
|
||||
"default": "Rachel",
|
||||
"description": "fal.ai ElevenLabs voice name or ID",
|
||||
},
|
||||
"voice_id": {
|
||||
"type": "string",
|
||||
"description": "Alias for voice, for compatibility with tts_selector",
|
||||
},
|
||||
"model_id": {
|
||||
"type": "string",
|
||||
"default": "eleven-v3",
|
||||
"description": "eleven-v3, multilingual-v2, or turbo-v2.5",
|
||||
},
|
||||
"stability": {
|
||||
"type": "number",
|
||||
"default": 0.5,
|
||||
"minimum": 0,
|
||||
"maximum": 1,
|
||||
},
|
||||
"similarity_boost": {
|
||||
"type": "number",
|
||||
"default": 0.75,
|
||||
"minimum": 0,
|
||||
"maximum": 1,
|
||||
},
|
||||
"style": {
|
||||
"type": "number",
|
||||
"minimum": 0,
|
||||
"maximum": 1,
|
||||
},
|
||||
"speed": {
|
||||
"type": "number",
|
||||
"default": 1.0,
|
||||
"minimum": 0.7,
|
||||
"maximum": 1.2,
|
||||
},
|
||||
"language_code": {
|
||||
"type": "string",
|
||||
"description": "Optional ISO 639-1 language code",
|
||||
},
|
||||
"timestamps": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
},
|
||||
"apply_text_normalization": {
|
||||
"type": "string",
|
||||
"default": "auto",
|
||||
"enum": ["auto", "on", "off"],
|
||||
},
|
||||
"output_format": {
|
||||
"type": "string",
|
||||
"default": "mp3_44100_128",
|
||||
"enum": [
|
||||
"mp3_22050_32",
|
||||
"mp3_44100_64",
|
||||
"mp3_44100_96",
|
||||
"mp3_44100_128",
|
||||
"mp3_44100_192",
|
||||
"pcm_16000",
|
||||
"pcm_24000",
|
||||
"pcm_44100",
|
||||
"pcm_48000",
|
||||
"opus_48000_64",
|
||||
"opus_48000_96",
|
||||
"opus_48000_128",
|
||||
"opus_48000_192",
|
||||
],
|
||||
},
|
||||
"seed": {"type": "integer"},
|
||||
"output_path": {"type": "string"},
|
||||
},
|
||||
}
|
||||
|
||||
resource_profile = ResourceProfile(
|
||||
cpu_cores=1,
|
||||
ram_mb=256,
|
||||
vram_mb=0,
|
||||
disk_mb=50,
|
||||
network_required=True,
|
||||
)
|
||||
retry_policy = RetryPolicy(
|
||||
max_retries=0,
|
||||
retryable_errors=["rate_limit", "timeout"],
|
||||
)
|
||||
idempotency_key_fields = [
|
||||
"text",
|
||||
"voice",
|
||||
"voice_id",
|
||||
"model_id",
|
||||
"stability",
|
||||
"similarity_boost",
|
||||
"style",
|
||||
"speed",
|
||||
"language_code",
|
||||
"seed",
|
||||
]
|
||||
side_effects = [
|
||||
"writes an audio file to output_path",
|
||||
"submits one paid fal.ai ElevenLabs speech request",
|
||||
]
|
||||
user_visible_verification = [
|
||||
"Listen to the generated voice sample before approving full narration",
|
||||
]
|
||||
|
||||
def _get_api_key(self) -> str | None:
|
||||
return os.environ.get("FAL_KEY") or os.environ.get("FAL_AI_API_KEY")
|
||||
|
||||
def get_status(self) -> ToolStatus:
|
||||
return ToolStatus.AVAILABLE if self._get_api_key() else ToolStatus.UNAVAILABLE
|
||||
|
||||
def _resolve_model(self, requested: str | None) -> tuple[str, str]:
|
||||
model_name = requested or "eleven-v3"
|
||||
model_name = self._MODEL_ALIASES.get(model_name, model_name)
|
||||
if model_name not in self._MODELS:
|
||||
choices = ", ".join(self._MODELS)
|
||||
raise ValueError(f"model_id must be one of: {choices}")
|
||||
return model_name, self._MODELS[model_name]
|
||||
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
model_name, _ = self._resolve_model(inputs.get("model_id"))
|
||||
return round(
|
||||
len(inputs.get("text", "")) * self._PRICE_PER_CHARACTER[model_name],
|
||||
4,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _output_extension(output_format: str) -> str:
|
||||
return {
|
||||
"mp3": "mp3",
|
||||
"pcm": "pcm",
|
||||
"opus": "opus",
|
||||
}.get(output_format.split("_", 1)[0], "audio")
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
api_key = self._get_api_key()
|
||||
if not api_key:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="No fal.ai API key found. " + self.install_instructions,
|
||||
)
|
||||
|
||||
text = str(inputs.get("text", "")).strip()
|
||||
if not text:
|
||||
return ToolResult(success=False, error="text is required")
|
||||
|
||||
try:
|
||||
model_name, model_id = self._resolve_model(inputs.get("model_id"))
|
||||
except ValueError as exc:
|
||||
return ToolResult(success=False, error=str(exc))
|
||||
|
||||
stability = float(inputs.get("stability", 0.5))
|
||||
similarity_boost = float(inputs.get("similarity_boost", 0.75))
|
||||
speed = float(inputs.get("speed", 1.0))
|
||||
if not 0 <= stability <= 1 or not 0 <= similarity_boost <= 1:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="stability and similarity_boost must be between 0 and 1",
|
||||
)
|
||||
if not 0.7 <= speed <= 1.2:
|
||||
return ToolResult(success=False, error="speed must be between 0.7 and 1.2")
|
||||
|
||||
output_format = inputs.get("output_format", "mp3_44100_128")
|
||||
voice = inputs.get("voice") or inputs.get("voice_id") or "Rachel"
|
||||
payload: dict[str, Any] = {
|
||||
"text": text,
|
||||
"voice": voice,
|
||||
"stability": stability,
|
||||
"similarity_boost": similarity_boost,
|
||||
"speed": speed,
|
||||
"timestamps": bool(inputs.get("timestamps", False)),
|
||||
"apply_text_normalization": inputs.get("apply_text_normalization", "auto"),
|
||||
"output_format": output_format,
|
||||
}
|
||||
for optional in ("language_code", "seed", "style"):
|
||||
if inputs.get(optional) is not None:
|
||||
payload[optional] = inputs[optional]
|
||||
|
||||
import requests
|
||||
|
||||
started = time.time()
|
||||
headers = {
|
||||
"Authorization": f"Key {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
queue_url = f"https://queue.fal.run/{model_id}"
|
||||
|
||||
try:
|
||||
submit_response = requests.post(
|
||||
queue_url,
|
||||
headers=headers,
|
||||
json=payload,
|
||||
timeout=30,
|
||||
)
|
||||
submit_response.raise_for_status()
|
||||
queue_data = submit_response.json()
|
||||
status_url = queue_data["status_url"]
|
||||
response_url = queue_data["response_url"]
|
||||
|
||||
deadline = time.monotonic() + self._MAX_WAIT_SECONDS
|
||||
while True:
|
||||
if time.monotonic() >= deadline:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="fal.ai ElevenLabs speech timed out while waiting in the queue",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
time.sleep(self._POLL_INTERVAL_SECONDS)
|
||||
status_response = requests.get(status_url, headers=headers, timeout=20)
|
||||
status_response.raise_for_status()
|
||||
status = status_response.json().get("status", "UNKNOWN")
|
||||
if status == "COMPLETED":
|
||||
break
|
||||
if status in {"FAILED", "CANCELLED"}:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"fal.ai ElevenLabs speech {status.lower()}",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
|
||||
result_response = requests.get(response_url, headers=headers, timeout=30)
|
||||
result_response.raise_for_status()
|
||||
result_data = result_response.json()
|
||||
audio_url = result_data["audio"]["url"]
|
||||
|
||||
audio_response = requests.get(audio_url, timeout=120)
|
||||
audio_response.raise_for_status()
|
||||
default_output = f"fal_elevenlabs_tts.{self._output_extension(output_format)}"
|
||||
output_path = Path(inputs.get("output_path", default_output))
|
||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
output_path.write_bytes(audio_response.content)
|
||||
except Exception as exc:
|
||||
safe_error = str(exc).replace(api_key, "[REDACTED]")
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"fal.ai ElevenLabs speech failed: {safe_error}",
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
)
|
||||
|
||||
data = {
|
||||
"provider": self.provider,
|
||||
"model": model_id,
|
||||
"voice": voice,
|
||||
"text_length": len(text),
|
||||
"stability": stability,
|
||||
"similarity_boost": similarity_boost,
|
||||
"speed": speed,
|
||||
"output": str(output_path),
|
||||
"format": output_format,
|
||||
}
|
||||
if payload["timestamps"] and "timestamps" in result_data:
|
||||
data["timestamps"] = result_data["timestamps"]
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
data=data,
|
||||
artifacts=[str(output_path)],
|
||||
cost_usd=self.estimate_cost({**inputs, "model_id": model_name, "text": text}),
|
||||
duration_seconds=round(time.time() - started, 2),
|
||||
model=model_id,
|
||||
)
|
||||
@@ -27,7 +27,6 @@ from tools.base_tool import (
|
||||
from tools.google_credentials import (
|
||||
get_access_token,
|
||||
service_account_configured,
|
||||
has_google_credentials,
|
||||
)
|
||||
|
||||
|
||||
@@ -44,7 +43,8 @@ class GoogleTTS(BaseTool):
|
||||
|
||||
dependencies = []
|
||||
install_instructions = (
|
||||
"Auth option A — API key: set GOOGLE_API_KEY (or GEMINI_API_KEY) to a\n"
|
||||
"Auth option A — TTS-only API key: set GOOGLE_TTS_API_KEY.\n"
|
||||
" GOOGLE_API_KEY or GEMINI_API_KEY remain supported for broader Google setups.\n"
|
||||
" Google Cloud API key with Text-to-Speech enabled.\n"
|
||||
" Enable the API at https://console.cloud.google.com/apis/library/texttospeech.googleapis.com\n"
|
||||
"Auth option B — service account: set GOOGLE_APPLICATION_CREDENTIALS to the\n"
|
||||
@@ -149,12 +149,16 @@ class GoogleTTS(BaseTool):
|
||||
}
|
||||
|
||||
def _get_api_key(self) -> str | None:
|
||||
return os.environ.get("GOOGLE_API_KEY") or os.environ.get("GEMINI_API_KEY")
|
||||
return (
|
||||
os.environ.get("GOOGLE_TTS_API_KEY")
|
||||
or os.environ.get("GOOGLE_API_KEY")
|
||||
or os.environ.get("GEMINI_API_KEY")
|
||||
)
|
||||
|
||||
def get_status(self) -> ToolStatus:
|
||||
# Available via either an API key or a service-account JSON. Both paths
|
||||
# are honoured by execute() — so this no longer over-reports.
|
||||
if has_google_credentials():
|
||||
if self._get_api_key() or service_account_configured():
|
||||
return ToolStatus.AVAILABLE
|
||||
return ToolStatus.UNAVAILABLE
|
||||
|
||||
@@ -204,7 +208,11 @@ class GoogleTTS(BaseTool):
|
||||
try:
|
||||
result = self._generate(inputs, api_key=api_key, bearer_token=bearer_token)
|
||||
except Exception as exc:
|
||||
return ToolResult(success=False, error=f"Google TTS failed: {exc}")
|
||||
safe_error = str(exc)
|
||||
for credential in (api_key, bearer_token):
|
||||
if credential:
|
||||
safe_error = safe_error.replace(credential, "[REDACTED]")
|
||||
return ToolResult(success=False, error=f"Google TTS failed: {safe_error}")
|
||||
|
||||
result.duration_seconds = round(time.time() - start, 2)
|
||||
result.cost_usd = self.estimate_cost(inputs)
|
||||
@@ -266,16 +274,14 @@ class GoogleTTS(BaseTool):
|
||||
url = f"https://texttospeech.googleapis.com/{api_version}/text:synthesize"
|
||||
|
||||
headers = {"Content-Type": "application/json"}
|
||||
params: dict[str, str] = {}
|
||||
if bearer_token:
|
||||
headers["Authorization"] = f"Bearer {bearer_token}"
|
||||
elif api_key:
|
||||
params["key"] = api_key
|
||||
headers["x-goog-api-key"] = api_key
|
||||
|
||||
response = requests.post(
|
||||
url,
|
||||
headers=headers,
|
||||
params=params,
|
||||
json=payload,
|
||||
timeout=120,
|
||||
)
|
||||
|
||||
@@ -45,6 +45,10 @@ class TTSSelector(BaseTool):
|
||||
"type": "string",
|
||||
"description": "Provider-specific voice ID. Passed through to the selected TTS provider.",
|
||||
},
|
||||
"voice": {
|
||||
"type": "string",
|
||||
"description": "Provider-specific voice name or ID. fal.ai ElevenLabs accepts names such as Rachel.",
|
||||
},
|
||||
"voice_language": {
|
||||
"type": "string",
|
||||
"enum": ["zh", "en"],
|
||||
@@ -58,7 +62,7 @@ class TTSSelector(BaseTool):
|
||||
},
|
||||
"model_id": {
|
||||
"type": "string",
|
||||
"description": "TTS model to use (e.g. eleven_multilingual_v2). Passed through to provider.",
|
||||
"description": "TTS model to use (e.g. eleven-v3 or eleven_multilingual_v2). Passed through to provider.",
|
||||
},
|
||||
"stability": {
|
||||
"type": "number", "minimum": 0, "maximum": 1,
|
||||
@@ -100,6 +104,24 @@ class TTSSelector(BaseTool):
|
||||
"default": "text",
|
||||
"description": "Use 'ssml' only when the selected provider supports tags such as <break>.",
|
||||
},
|
||||
"language_code": {
|
||||
"type": "string",
|
||||
"description": "Provider-specific language code, such as en-US for Google or en for fal.ai ElevenLabs.",
|
||||
},
|
||||
"timestamps": {
|
||||
"type": "boolean",
|
||||
"default": False,
|
||||
"description": "Request word timestamps when the selected provider supports them.",
|
||||
},
|
||||
"apply_text_normalization": {
|
||||
"type": "string",
|
||||
"enum": ["auto", "on", "off"],
|
||||
"description": "Text normalization mode for providers that support it.",
|
||||
},
|
||||
"seed": {
|
||||
"type": "integer",
|
||||
"description": "Optional generation seed for providers that support reproducible speech.",
|
||||
},
|
||||
"voice_performance": {
|
||||
"type": "object",
|
||||
"description": "Structured voice-performance plan or section delivery cues from the script artifact.",
|
||||
|
||||
@@ -119,9 +119,12 @@ class GoogleImagen(BaseTool):
|
||||
"imagen-4.0-generate-001",
|
||||
"imagen-4.0-fast-generate-001",
|
||||
"imagen-4.0-ultra-generate-001",
|
||||
"gemini-2.5-flash-image",
|
||||
],
|
||||
"default": "imagen-4.0-generate-001",
|
||||
"description": "Imagen model variant",
|
||||
"description": "Imagen model variant, or a Gemini image model "
|
||||
"(gemini-*) routed through generate_content. Use "
|
||||
"gemini-2.5-flash-image when the project has no Imagen access.",
|
||||
},
|
||||
"number_of_images": {
|
||||
"type": "integer",
|
||||
@@ -177,13 +180,111 @@ class GoogleImagen(BaseTool):
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
model = inputs.get("model", "imagen-4.0-generate-001")
|
||||
n = inputs.get("number_of_images", 1)
|
||||
if model.startswith("gemini-"):
|
||||
# ~1290 output tokens per image at $30/1M tokens
|
||||
return 0.039 * n
|
||||
if "ultra" in model:
|
||||
return 0.06 * n
|
||||
if "fast" in model:
|
||||
return 0.02 * n
|
||||
return 0.04 * n
|
||||
|
||||
def _resolve_aspect_ratio(self, inputs: dict[str, Any]) -> str:
|
||||
"""Explicit aspect_ratio > derived from width/height > default 1:1."""
|
||||
if "aspect_ratio" in inputs:
|
||||
return inputs["aspect_ratio"]
|
||||
if "width" in inputs and "height" in inputs:
|
||||
import logging
|
||||
|
||||
aspect_ratio = _dims_to_aspect_ratio(inputs["width"], inputs["height"])
|
||||
logging.getLogger(__name__).info(
|
||||
"google_imagen: remapped %sx%s to nearest supported aspect ratio %s",
|
||||
inputs["width"],
|
||||
inputs["height"],
|
||||
aspect_ratio,
|
||||
)
|
||||
return aspect_ratio
|
||||
return "1:1"
|
||||
|
||||
def _execute_gemini(self, inputs: dict[str, Any], model: str) -> ToolResult:
|
||||
"""Generate via a Gemini image model (e.g. gemini-2.5-flash-image).
|
||||
|
||||
These models use generate_content with an image_config instead of the
|
||||
Imagen :predict endpoint, and work on both auth paths (API key and
|
||||
Vertex service account) through the shared genai client.
|
||||
"""
|
||||
start = time.time()
|
||||
try:
|
||||
from google.genai import types
|
||||
from tools.google_credentials import get_genai_client
|
||||
|
||||
client = get_genai_client()
|
||||
except Exception as e:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"Failed to initialize Google GenAI client: {e}",
|
||||
)
|
||||
|
||||
prompt = inputs["prompt"]
|
||||
aspect_ratio = self._resolve_aspect_ratio(inputs)
|
||||
number_of_images = inputs.get("number_of_images", 1)
|
||||
config = types.GenerateContentConfig(
|
||||
image_config=types.ImageConfig(aspect_ratio=aspect_ratio),
|
||||
)
|
||||
|
||||
image_bytes: list[bytes] = []
|
||||
try:
|
||||
for _ in range(number_of_images):
|
||||
response = client.models.generate_content(
|
||||
model=model, contents=prompt, config=config
|
||||
)
|
||||
for part in response.candidates[0].content.parts or []:
|
||||
inline = getattr(part, "inline_data", None)
|
||||
if inline and inline.data:
|
||||
image_bytes.append(inline.data)
|
||||
break
|
||||
except Exception as e:
|
||||
return ToolResult(
|
||||
success=False, error=f"Gemini image generation failed: {e}"
|
||||
)
|
||||
|
||||
if not image_bytes:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"No image data returned by {model} (text-only response).",
|
||||
)
|
||||
|
||||
output_paths = self._output_paths(inputs.get("output_path"), len(image_bytes))
|
||||
outputs: list[str] = []
|
||||
for data, out_path in zip(image_bytes, output_paths):
|
||||
out_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
out_path.write_bytes(data)
|
||||
outputs.append(str(out_path))
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
data={
|
||||
"provider": "google_imagen",
|
||||
"model": model,
|
||||
"prompt": prompt,
|
||||
"aspect_ratio": aspect_ratio,
|
||||
"output": outputs[0],
|
||||
"outputs": outputs,
|
||||
"images_generated": len(outputs),
|
||||
},
|
||||
artifacts=outputs,
|
||||
cost_usd=self.estimate_cost(inputs),
|
||||
duration_seconds=round(time.time() - start, 2),
|
||||
model=model,
|
||||
)
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
# Gemini image models go through generate_content via the shared genai
|
||||
# client, which resolves auth (API key or Vertex service account) itself.
|
||||
model = inputs.get("model", "imagen-4.0-generate-001")
|
||||
if model.startswith("gemini-"):
|
||||
return self._execute_gemini(inputs, model)
|
||||
|
||||
# Two auth paths: an AI Studio API key, or a service-account JSON that
|
||||
# routes to Vertex AI (the AI Studio endpoint does not accept service
|
||||
# accounts). API key wins when both are present.
|
||||
@@ -213,27 +314,9 @@ class GoogleImagen(BaseTool):
|
||||
import requests
|
||||
|
||||
start = time.time()
|
||||
model = inputs.get("model", "imagen-4.0-generate-001")
|
||||
prompt = inputs["prompt"]
|
||||
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Resolve aspect ratio: explicit > derived from width/height > default
|
||||
if "aspect_ratio" in inputs:
|
||||
aspect_ratio = inputs["aspect_ratio"]
|
||||
elif "width" in inputs and "height" in inputs:
|
||||
requested_ratio = f"{inputs['width']}x{inputs['height']}"
|
||||
aspect_ratio = _dims_to_aspect_ratio(inputs["width"], inputs["height"])
|
||||
logger.info(
|
||||
"google_imagen: remapped %s to nearest supported aspect ratio %s",
|
||||
requested_ratio,
|
||||
aspect_ratio,
|
||||
)
|
||||
else:
|
||||
aspect_ratio = "1:1"
|
||||
|
||||
aspect_ratio = self._resolve_aspect_ratio(inputs)
|
||||
number_of_images = inputs.get("number_of_images", 1)
|
||||
|
||||
parameters: dict[str, Any] = {
|
||||
|
||||
580
tools/graphics/hunyuan_image.py
Normal file
580
tools/graphics/hunyuan_image.py
Normal file
@@ -0,0 +1,580 @@
|
||||
"""Tencent Hunyuan (腾讯混元) cloud image generation (3.0) via TokenHub API.
|
||||
|
||||
Calls the Tencent TokenHub API (tokenhub.tencentmaas.com) using simple Bearer
|
||||
token authentication. This is the OpenAI-compatible API gateway for Tencent
|
||||
Hunyuan image models — no TC3-HMAC-SHA256 signing required.
|
||||
|
||||
API flow: POST /v1/api/image/submit -> poll /v1/api/image/query ->
|
||||
download data[].url.
|
||||
|
||||
Authentication uses a TokenHub API key obtained from the Tencent Cloud
|
||||
TokenHub console (https://console.cloud.tencent.com/tokenhub).
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import time
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from tools.base_tool import (
|
||||
BaseTool,
|
||||
Determinism,
|
||||
ExecutionMode,
|
||||
ResourceProfile,
|
||||
RetryPolicy,
|
||||
ToolResult,
|
||||
ToolRuntime,
|
||||
ToolStability,
|
||||
ToolStatus,
|
||||
ToolTier,
|
||||
)
|
||||
|
||||
_HOST = "tokenhub.tencentmaas.com"
|
||||
_SUBMIT_PATH = "/v1/api/image/submit"
|
||||
_QUERY_PATH = "/v1/api/image/query"
|
||||
|
||||
# TokenHub model identifier for 混元生图 3.0
|
||||
_MODEL = "hy-image-v3.0"
|
||||
|
||||
|
||||
class HunyuanImage(BaseTool):
|
||||
"""Tencent Hunyuan cloud image generation (3.0) via TokenHub API."""
|
||||
|
||||
name = "hunyuan_image"
|
||||
version = "0.1.0"
|
||||
tier = ToolTier.GENERATE
|
||||
capability = "image_generation"
|
||||
provider = "hunyuan_cloud"
|
||||
stability = ToolStability.EXPERIMENTAL
|
||||
execution_mode = ExecutionMode.ASYNC
|
||||
determinism = Determinism.SEEDED
|
||||
runtime = ToolRuntime.API
|
||||
|
||||
dependencies = ["env:TENCENT_TOKENHUB_API_KEY"]
|
||||
install_instructions = (
|
||||
"Set TENCENT_TOKENHUB_API_KEY to your Tencent Cloud TokenHub API key.\n"
|
||||
" Get it at https://console.cloud.tencent.com/tokenhub"
|
||||
)
|
||||
agent_skills = ["visual-style"]
|
||||
|
||||
capabilities = ["generate_image", "text_to_image"]
|
||||
supports = {
|
||||
"negative_prompt": False,
|
||||
"seed": True,
|
||||
"custom_size": True,
|
||||
"reference_image": True,
|
||||
"prompt_rewrite": True,
|
||||
}
|
||||
best_for = [
|
||||
"Hunyuan text-to-image via Tencent TokenHub API",
|
||||
"simple Bearer-token auth (no TC3 signing required)",
|
||||
"direct Tencent Cloud quota usage (not through a third-party gateway)",
|
||||
"Chinese-language prompt understanding",
|
||||
]
|
||||
not_good_for = [
|
||||
"offline generation or air-gapped environments",
|
||||
"users without Tencent Cloud account and real-name verification",
|
||||
]
|
||||
fallback_tools = ["dashscope_image", "flux_image", "openai_image", "recraft_image"]
|
||||
|
||||
input_schema = {
|
||||
"type": "object",
|
||||
"required": ["prompt"],
|
||||
"properties": {
|
||||
"prompt": {
|
||||
"type": "string",
|
||||
"maxLength": 8192,
|
||||
"description": (
|
||||
"Image description. Max 8192 UTF-8 characters. "
|
||||
"Supports Chinese and English. Be specific about subject, "
|
||||
"composition, style, and mood."
|
||||
),
|
||||
},
|
||||
"images": {
|
||||
"type": "array",
|
||||
"items": {"type": "string"},
|
||||
"maxItems": 3,
|
||||
"description": (
|
||||
"Reference images per upstream Images.N param (max 3). "
|
||||
"Each entry is a publicly accessible URL or a local file path "
|
||||
"(auto-encoded to base64 data URI). "
|
||||
"Single image: 50-5000px per side, base64 < 6MB. "
|
||||
"Formats: jpg/png/jpeg/webp/bmp/tiff."
|
||||
),
|
||||
},
|
||||
"resolution": {
|
||||
"type": "string",
|
||||
"default": "1024:1024",
|
||||
"description": (
|
||||
'Image resolution as "W:H" (colon separator, per upstream '
|
||||
"Resolution param). W, H in [512, 2048], product (W*H) <= "
|
||||
'1024x1024 pixels. Examples: "1024:1024", "768:1024", '
|
||||
'"1024:576".'
|
||||
),
|
||||
},
|
||||
"seed": {
|
||||
"type": "integer",
|
||||
"minimum": 1,
|
||||
"maximum": 4294967295,
|
||||
"description": (
|
||||
"Random seed in [1, 4294967295]. "
|
||||
"Note: seed is ignored when revise is enabled (default)."
|
||||
),
|
||||
},
|
||||
"revise": {
|
||||
"type": "integer",
|
||||
"enum": [0, 1],
|
||||
"default": 1,
|
||||
"description": (
|
||||
"Prompt auto-rewrite toggle per upstream Revise param. "
|
||||
"1 = enabled (default, adds ~20s processing), 0 = disabled. "
|
||||
"When disabled, caller should handle prompt rewriting."
|
||||
),
|
||||
},
|
||||
"logo_add": {
|
||||
"type": "integer",
|
||||
"enum": [0, 1],
|
||||
"default": 1,
|
||||
"description": (
|
||||
"Add 'AI-generated' watermark per upstream LogoAdd param. "
|
||||
"1 = add watermark (default), 0 = no watermark. "
|
||||
"Values other than 0 or 1 are treated as 1."
|
||||
),
|
||||
},
|
||||
"logo_param": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"logo_url": {
|
||||
"type": "string",
|
||||
"description": "Custom watermark image URL.",
|
||||
},
|
||||
"logo_image": {
|
||||
"type": "string",
|
||||
"description": "Custom watermark image as base64-encoded string.",
|
||||
},
|
||||
},
|
||||
"description": (
|
||||
"Custom watermark settings per upstream LogoParam. "
|
||||
"Default: \"图片由 AI 生成\" at bottom-right. "
|
||||
"Note: custom watermarks may not be supported by the engine "
|
||||
"(error: InvalidParameterValue.LogoParamErr)."
|
||||
),
|
||||
},
|
||||
"output_path": {
|
||||
"type": "string",
|
||||
"description": "Output file path for the generated image (PNG).",
|
||||
},
|
||||
"poll_interval_seconds": {
|
||||
"type": "number",
|
||||
"minimum": 2,
|
||||
"default": 5.0,
|
||||
"description": "Seconds between status polls.",
|
||||
},
|
||||
"timeout_seconds": {
|
||||
"type": "integer",
|
||||
"minimum": 60,
|
||||
"default": 600,
|
||||
"description": "Maximum seconds to wait for generation.",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
resource_profile = ResourceProfile(
|
||||
cpu_cores=1, ram_mb=512, vram_mb=0, disk_mb=100, network_required=True,
|
||||
)
|
||||
retry_policy = RetryPolicy(
|
||||
max_retries=2,
|
||||
backoff_seconds=2.0,
|
||||
retryable_errors=["rate_limit", "timeout"],
|
||||
)
|
||||
idempotency_key_fields = [
|
||||
"prompt",
|
||||
"resolution",
|
||||
"images",
|
||||
"seed",
|
||||
"revise",
|
||||
"logo_add",
|
||||
"logo_param",
|
||||
]
|
||||
side_effects = [
|
||||
"writes image file to output_path",
|
||||
"calls Tencent TokenHub API (Bearer-token submit + poll + download)",
|
||||
]
|
||||
user_visible_verification = [
|
||||
"Inspect generated image for quality and prompt adherence",
|
||||
"Check for watermark if logo_add=0 was requested",
|
||||
]
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Credential helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _api_key() -> str | None:
|
||||
val = os.environ.get("TENCENT_TOKENHUB_API_KEY", "")
|
||||
if val and not val.strip().startswith("#"):
|
||||
return val.strip()
|
||||
return None
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Tool contract methods
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def get_status(self) -> ToolStatus:
|
||||
if self._api_key():
|
||||
return ToolStatus.AVAILABLE
|
||||
return ToolStatus.UNAVAILABLE
|
||||
|
||||
def estimate_cost(self, inputs: dict[str, Any]) -> float:
|
||||
"""Estimate cost in USD.
|
||||
|
||||
Tencent TokenHub credit-based pricing (1 credit = 1.2 RMB ≈ $0.167 USD):
|
||||
- hy-image-v3.0: ~0.5 credits/image → ~$0.08
|
||||
|
||||
Source: https://cloud.tencent.com.cn/document/product/1823/130054
|
||||
"""
|
||||
_CREDIT_TO_USD = 1.2 / 7.2 # 1 credit = 1.2 RMB, ~7.2 RMB/USD
|
||||
credits = 0.5
|
||||
return round(credits * _CREDIT_TO_USD, 2)
|
||||
|
||||
def estimate_runtime(self, inputs: dict[str, Any]) -> float:
|
||||
"""Estimate wall-clock time in seconds.
|
||||
|
||||
Per upstream docs, prompt rewrite (revise=1) adds ~20s. Including
|
||||
queuing and download, 120s is a safe upper-bound.
|
||||
"""
|
||||
return 120.0
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Main execution
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
api_key = self._api_key()
|
||||
if not api_key:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="TENCENT_TOKENHUB_API_KEY not set. " + self.install_instructions,
|
||||
)
|
||||
|
||||
start = time.time()
|
||||
try:
|
||||
result = self._generate(inputs, api_key=api_key)
|
||||
except Exception as exc:
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error=f"Hunyuan TokenHub image generation failed: {self._safe_error(exc)}",
|
||||
)
|
||||
|
||||
result.duration_seconds = round(time.time() - start, 2)
|
||||
return result
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Generation pipeline
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _generate(
|
||||
self, inputs: dict[str, Any], *, api_key: str,
|
||||
) -> ToolResult:
|
||||
import requests
|
||||
|
||||
# Guard: refuse to make paid API calls without an explicit output_path.
|
||||
# A CWD-relative default would leak files into the project root when
|
||||
# called by selectors or other automated tooling.
|
||||
if not inputs.get("output_path"):
|
||||
return ToolResult(
|
||||
success=False,
|
||||
error="output_path is required for hunyuan_image generation.",
|
||||
)
|
||||
|
||||
payload = self._build_payload(inputs)
|
||||
task_id = self._submit_task(payload, model=_MODEL, api_key=api_key)
|
||||
image_urls = self._poll_task(
|
||||
task_id,
|
||||
model=_MODEL,
|
||||
api_key=api_key,
|
||||
poll_interval=float(inputs.get("poll_interval_seconds", 5.0)),
|
||||
timeout_seconds=int(inputs.get("timeout_seconds", 600)),
|
||||
)
|
||||
|
||||
output_paths = self._resolve_output_paths(
|
||||
inputs["output_path"],
|
||||
count=len(image_urls),
|
||||
)
|
||||
for path, url in zip(output_paths, image_urls):
|
||||
download = requests.get(url, timeout=120)
|
||||
download.raise_for_status()
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
path.write_bytes(download.content)
|
||||
|
||||
return ToolResult(
|
||||
success=True,
|
||||
data={
|
||||
"provider": "hunyuan_cloud",
|
||||
"route": "tokenhub",
|
||||
"model": _MODEL,
|
||||
"prompt": inputs["prompt"],
|
||||
"resolution": payload.get("resolution", "1024:1024"),
|
||||
"revise": payload.get("revise", 1),
|
||||
"logo_add": payload.get("logo_add", 1),
|
||||
"task_id": task_id,
|
||||
"output": str(output_paths[0]),
|
||||
"outputs": [str(p) for p in output_paths],
|
||||
"images_generated": len(output_paths),
|
||||
},
|
||||
artifacts=[str(p) for p in output_paths],
|
||||
cost_usd=self.estimate_cost(inputs),
|
||||
model=_MODEL,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Payload construction
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
def _build_payload(self, inputs: dict[str, Any]) -> dict[str, Any]:
|
||||
"""Build the request body for TokenHub image submit.
|
||||
|
||||
TokenHub async endpoints use snake_case versions of the upstream
|
||||
SubmitTextToImageJob parameter names (e.g. Resolution -> resolution,
|
||||
Images -> images).
|
||||
"""
|
||||
payload: dict[str, Any] = {
|
||||
"prompt": inputs["prompt"],
|
||||
}
|
||||
|
||||
# Optional parameters (snake_case of upstream SubmitTextToImageJob params)
|
||||
if inputs.get("resolution"):
|
||||
payload["resolution"] = inputs["resolution"]
|
||||
if inputs.get("seed") is not None:
|
||||
payload["seed"] = int(inputs["seed"])
|
||||
if "revise" in inputs:
|
||||
payload["revise"] = int(inputs["revise"])
|
||||
if "logo_add" in inputs:
|
||||
payload["logo_add"] = int(inputs["logo_add"])
|
||||
if inputs.get("logo_param"):
|
||||
logo_param: dict[str, str] = {}
|
||||
lp = inputs["logo_param"]
|
||||
if lp.get("logo_url"):
|
||||
logo_param["logo_url"] = lp["logo_url"]
|
||||
if lp.get("logo_image"):
|
||||
logo_param["logo_image"] = lp["logo_image"]
|
||||
if logo_param:
|
||||
payload["logo_param"] = logo_param
|
||||
|
||||
# Reference images — maps to upstream Images.N
|
||||
# TokenHub accepts URLs or base64 data URIs in the images array
|
||||
image_refs = inputs.get("images")
|
||||
if image_refs:
|
||||
payload["images"] = self._resolve_images(image_refs)
|
||||
|
||||
return payload
|
||||
|
||||
@staticmethod
|
||||
def _resolve_images(refs: list[str]) -> list[str]:
|
||||
"""Resolve reference images to strings for the TokenHub API.
|
||||
|
||||
Each entry may be:
|
||||
- An HTTP(S) URL → passed through unchanged
|
||||
- A data URI (``data:...``) → passed through unchanged
|
||||
- A local file path → base64-encoded as a data URI
|
||||
|
||||
Per upstream docs: single image 50-5000px per side, base64 < 6MB.
|
||||
Formats: jpg/jpeg/png/bmp/tiff/webp.
|
||||
"""
|
||||
import base64
|
||||
|
||||
resolved: list[str] = []
|
||||
for ref in refs:
|
||||
if ref.startswith("data:") or ref.startswith("http://") or ref.startswith("https://"):
|
||||
resolved.append(ref)
|
||||
continue
|
||||
|
||||
image_path = Path(ref)
|
||||
if not image_path.is_file():
|
||||
raise FileNotFoundError(f"Reference image not found: {ref}")
|
||||
|
||||
raw = image_path.read_bytes()
|
||||
max_raw = 6 * 1024 * 1024 # 6MB per upstream limit
|
||||
if len(raw) > max_raw:
|
||||
raise ValueError(
|
||||
f"Image too large ({len(raw)} bytes). Max ~6MB raw."
|
||||
)
|
||||
|
||||
suffix = image_path.suffix.lower()
|
||||
mime_map = {
|
||||
".jpg": "image/jpeg",
|
||||
".jpeg": "image/jpeg",
|
||||
".png": "image/png",
|
||||
".bmp": "image/bmp",
|
||||
".tiff": "image/tiff",
|
||||
".tif": "image/tiff",
|
||||
".webp": "image/webp",
|
||||
}
|
||||
mime = mime_map.get(suffix, "image/png")
|
||||
data = base64.b64encode(raw).decode("ascii")
|
||||
resolved.append(f"data:{mime};base64,{data}")
|
||||
return resolved
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# API communication (TokenHub OpenAI-compatible)
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _auth_headers(api_key: str) -> dict[str, str]:
|
||||
"""Build common request headers for TokenHub API calls."""
|
||||
return {
|
||||
"Authorization": f"Bearer {api_key}",
|
||||
"Content-Type": "application/json",
|
||||
}
|
||||
|
||||
def _submit_task(
|
||||
self, payload: dict[str, Any], *, model: str, api_key: str,
|
||||
) -> str:
|
||||
"""Submit an image generation task and return the task ID.
|
||||
|
||||
POST /v1/api/image/submit
|
||||
Request: {"model": "hy-image-v3.0", "prompt": "...", ...}
|
||||
Response: {"id": "...", "status": "queued", ...}
|
||||
"""
|
||||
import requests
|
||||
|
||||
body = {
|
||||
"model": model,
|
||||
**payload,
|
||||
}
|
||||
url = f"https://{_HOST}{_SUBMIT_PATH}"
|
||||
resp = requests.post(
|
||||
url,
|
||||
json=body,
|
||||
headers=self._auth_headers(api_key),
|
||||
timeout=30,
|
||||
)
|
||||
data = self._json_or_raise(resp)
|
||||
self._check_response(data)
|
||||
|
||||
task_id = data.get("id")
|
||||
if not task_id:
|
||||
raise RuntimeError(
|
||||
f"TokenHub submit returned no task id: {data}"
|
||||
)
|
||||
return task_id
|
||||
|
||||
def _poll_task(
|
||||
self,
|
||||
task_id: str,
|
||||
*,
|
||||
model: str,
|
||||
api_key: str,
|
||||
poll_interval: float,
|
||||
timeout_seconds: int,
|
||||
) -> list[str]:
|
||||
"""Poll /v1/api/image/query until completion, return image download URLs.
|
||||
|
||||
Response when completed:
|
||||
{"status": "completed", "data": [{"url": "...", "revised_prompt": "..."}]}
|
||||
|
||||
The data array may contain multiple images. Each URL is valid for ~1 hour.
|
||||
"""
|
||||
import requests
|
||||
|
||||
url = f"https://{_HOST}{_QUERY_PATH}"
|
||||
|
||||
deadline = time.time() + timeout_seconds
|
||||
while time.time() < deadline:
|
||||
time.sleep(poll_interval)
|
||||
|
||||
resp = requests.post(
|
||||
url,
|
||||
json={"model": model, "id": task_id},
|
||||
headers=self._auth_headers(api_key),
|
||||
timeout=30,
|
||||
)
|
||||
data = self._json_or_raise(resp)
|
||||
self._check_response(data)
|
||||
|
||||
status = data.get("status", "")
|
||||
|
||||
if status == "completed":
|
||||
result_data = data.get("data") or []
|
||||
urls = [item.get("url") for item in result_data if item.get("url")]
|
||||
if not urls:
|
||||
raise RuntimeError(
|
||||
f"TokenHub task {task_id} completed but no data[].url: {data}"
|
||||
)
|
||||
return urls
|
||||
|
||||
if status == "failed":
|
||||
error_info = data.get("error") or {}
|
||||
error_msg = error_info.get("message", "unknown error")
|
||||
raise RuntimeError(
|
||||
f"TokenHub task {task_id} failed: {error_msg}"
|
||||
)
|
||||
|
||||
# queued / running / in_progress — continue polling
|
||||
if status not in ("queued", "running", "in_progress"):
|
||||
raise RuntimeError(
|
||||
f"TokenHub task {task_id} returned unknown status: {status}"
|
||||
)
|
||||
|
||||
raise TimeoutError(
|
||||
f"TokenHub task {task_id} did not finish within {timeout_seconds}s"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Error handling helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _safe_error(exc: Exception) -> str:
|
||||
"""Redact secret values from exception messages."""
|
||||
msg = str(exc)
|
||||
for var in ("TENCENT_TOKENHUB_API_KEY",):
|
||||
val = os.environ.get(var, "")
|
||||
if val:
|
||||
msg = msg.replace(val, "[redacted]")
|
||||
return msg
|
||||
|
||||
@staticmethod
|
||||
def _json_or_raise(response: Any) -> dict[str, Any]:
|
||||
"""Parse JSON response body or raise with HTTP status."""
|
||||
try:
|
||||
return response.json()
|
||||
except ValueError as exc:
|
||||
raise RuntimeError(
|
||||
f"Non-JSON response from TokenHub API: HTTP {response.status_code}"
|
||||
) from exc
|
||||
|
||||
@staticmethod
|
||||
def _check_response(payload: dict[str, Any]) -> None:
|
||||
"""Check the TokenHub API response for errors.
|
||||
|
||||
TokenHub returns errors at the top level with an ``error`` field.
|
||||
"""
|
||||
error = payload.get("error")
|
||||
if error:
|
||||
message = error.get("message", "unknown error")
|
||||
code = error.get("code", error.get("type", "unknown"))
|
||||
raise RuntimeError(
|
||||
f"TokenHub API error: code={code}, message={message}"
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------------
|
||||
# Output helpers
|
||||
# ------------------------------------------------------------------
|
||||
|
||||
@staticmethod
|
||||
def _resolve_output_paths(base: str, count: int) -> list[Path]:
|
||||
"""Derive distinct paths for ``count`` images.
|
||||
|
||||
Single image keeps the base path unchanged; multiple images insert an
|
||||
index before the extension (foo.png -> foo_1.png, foo_2.png, ...).
|
||||
"""
|
||||
base_path = Path(base)
|
||||
if count <= 1:
|
||||
return [base_path]
|
||||
stem = base_path.stem
|
||||
suffix = base_path.suffix
|
||||
parent = base_path.parent
|
||||
return [parent / f"{stem}_{i}{suffix}" for i in range(1, count + 1)]
|
||||
@@ -242,6 +242,25 @@ class ImageSelector(BaseTool):
|
||||
props = tool.input_schema.get("properties", {})
|
||||
if "query" in props and "query" not in adapted:
|
||||
adapted["query"] = adapted.get("prompt", "")
|
||||
# Normalize the selector's shared reference-image inputs for
|
||||
# providers whose native contract accepts an ``images`` array.
|
||||
if "images" in props and "images" not in adapted:
|
||||
refs = (
|
||||
adapted.get("image_paths")
|
||||
or adapted.get("image_urls")
|
||||
or ([adapted["image_path"]] if adapted.get("image_path") else None)
|
||||
or ([adapted["image_url"]] if adapted.get("image_url") else None)
|
||||
)
|
||||
if refs:
|
||||
adapted["images"] = refs
|
||||
# The selector exposes a provider-neutral ``model_name`` field,
|
||||
# while several providers call the same input ``model``.
|
||||
if (
|
||||
"model_name" in adapted
|
||||
and "model" in props
|
||||
and "model" not in adapted
|
||||
):
|
||||
adapted["model"] = adapted["model_name"]
|
||||
if "n" in adapted and "num_images" in props and "num_images" not in adapted:
|
||||
adapted["num_images"] = adapted["n"]
|
||||
|
||||
|
||||
Reference in New Issue
Block a user