* Adopt uv for Python dependency management
Replace manual pip/venv setup with `uv sync` (pyproject.toml + uv.lock),
and switch all tool invocations across docs, skills, commands, and
examples to `uv run`. Optional extras: --extra whisper, --extra modal.
tools/requirements.txt is kept as a pip fallback.
* FIX: bring uv migration current with main — post-June deps and docs
Adds websocket-client (60db websocket transport) to core deps and a
'youtube' extra for the google-* packages (tools/youtube_upload.py),
regenerates uv.lock, bumps pyproject version to 0.18.0, and converts the
51 'python3 tools/…' invocations added since the PR was opened (60db,
YouTube, Kiro docs, publish command) to 'uv run'. Verified with
uv sync --frozen --extra youtube; voiceover, sixtydb_tts, youtube_upload
and migrate_to_kiro all run through the project environment.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Asharib Ali <asharibarain4@gmail.com>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* Added 60dB integration
* FIX: make 60db TTS synthesize transport work against the live API
The original integration coded to 60db's documented /tts-synthesize contract
(single JSON object with `audio_base64` in the requested container format).
In production the endpoint instead streams newline-delimited JSON of raw
16-bit mono PCM (Content-Type: application/x-ndjson) with a trailing
`{metadata}` line, so the default voiceover path failed with "Invalid JSON
response".
Changes (tools/sixtydb_tts.py):
- Rewrite _synthesize_rest to consume the NDJSON PCM stream, while still
accepting the documented single-JSON shape if 60db ships it.
- Add _finalize_audio: sniff bytes for an audio container (mp3/wav/ogg/flac)
and write/transcode as-is, else wrap raw PCM as WAV and transcode to the
requested --output-format via ffmpeg.
- Add _derive_pcm_sample_rate: infer the rate from byte-count and the
metadata audio_sec (snap to nearest standard rate) instead of hardcoding.
- Surface 60db metadata warnings; route _synthesize_stream through the same
PCM-aware finalizer and flag that /tts-stream currently 500s upstream.
Verified live: sixtydb_tts.py and `voiceover.py --provider 60db --scene-dir`
both produce valid 48kHz MP3s for English and Hindi.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* FIX: apply pacing QC (wpm/--max-wpm) to the 60db provider path
Rebase integration: pacing QC landed on main after this branch was cut;
elevenlabs and qwen3 paths report wpm/pacing and honor --max-wpm, but the
60db path bypassed it. Wire _apply_pacing_qc into generate_single_audio_60db
and pass max_wpm at both call sites.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MKbS3m7TXrfYjE7gDVEpnV
---------
Co-authored-by: manishEMS47 <manish.prasad@engineermaster.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
modal app list --json emits capitalised keys ('Description', 'State') on
CLI <=1.3.x and lowercase keys on newer CLIs. #51 fixed detection for new
CLIs but broke it for older ones; accept both so the check works on any
Modal version. Verified live against modal 1.3.5 (6 apps detected) and
via mocked lowercase output.
Claude-Session: https://claude.ai/code/session_01MKbS3m7TXrfYjE7gDVEpnV
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Three unrelated gaps found while running /setup end to end on Windows.
1. MODAL_LTX2_ENDPOINT_URL was read by tools/cloud_gpu.py but appeared in
neither .env.example nor verify_setup.py, so LTX-2 could only be configured
by someone who had read the source. Added to both, plus the deploy command
and env var in docs/modal-setup.md.
2. verify_setup.py always reported "no toolkit apps deployed". `modal app list
--json` emits lowercase keys ("description", "state") while the check read
the capitalised column headers of the human-readable table, so the filter
never matched. Verified against a workspace with 6 deployed apps: now
correctly reports "6 deployed". Also made the "N/7 tools" header derive its
total from the tool list instead of hardcoding it, since adding LTX-2 would
otherwise have desynced the count.
3. Documented three Windows blockers that each present as a toolkit bug:
- modal deploy aborts mid-build with a cp1252 'charmap' encode error unless
PYTHONIOENCODING=utf-8 is set
- python3 resolves to the zero-byte WindowsApps App Execution Alias, which
fails with "Permission denied" — this breaks every documented
`python3 tools/...` invocation
- FFmpeg needs a manual install, with a fallback for when winget's source
index is corrupted
Also documented two Modal behaviours that cost real debugging time: AppCreate
is rate-limited so deploys must be serial, and A100-class apps (image-edit,
ltx2) fail at deploy time until a payment method is added. Noted that ltx2
separately needs a `huggingface-token` secret, and that Modal truncates long
endpoint labels with a hash so URLs must be copied rather than reconstructed.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
youtube_upload.py mapped every HTTP 403 to errorType "quota", but accessNotConfigured
(API not enabled) and forbidden (permission denied) are also 403s — the first one bit
us on the very first real upload. Add extract_api_reason() + classify_http_error() to
read YouTube's machine reason from the error body and map to accurate types:
quota | config | forbidden | auth | http, plus a new errorReason field in --json-out.
The config case now prints an actionable "enable the API, wait 1-2 min" hint.
Unit-tested across 9 cases (each 403 reason, 401, 500, malformed body, status fallback).
Docs + /publish error table updated with the new types.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The three google-api-python-client / google-auth-oauthlib / google-auth-httplib2
packages were uncommented, so every `pip install -r tools/requirements.txt` pulled
them in even for users who never publish to YouTube. Comment them out under an
"uncomment to enable" header, mirroring how openai-whisper is handled, keeping base
installs lean. youtube_upload.py already prints an actionable install hint when the
deps are missing, so the opt-in path is self-documenting.
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* feat: add YouTube upload pipeline (tools/youtube_upload.py + /publish)
Adds the toolkit's first publishing capability: upload a rendered video to
YouTube via the Data API v3.
- tools/youtube_upload.py: OAuth 2.0 installed-app flow with silent token
refresh (per-account cache), resumable upload with exponential-backoff
retry, optional thumbnail/captions/playlist, and private/unlisted/public
or scheduled (publishAt) visibility. Flags for --auth, --dry-run, --json-out.
- /publish command: guided workflow that auto-fills title/description/tags
from a project's project.json, dry-runs, uploads, and writes back the URL.
- config.py: get_youtube_client_secrets_file(), get_youtube_token_dir().
- docs/youtube-upload.md: Google Cloud Console walkthrough, quota math, and
the unverified-app private-lock / 7-day token gotchas.
- Wiring: requirements (google-api-python-client/auth-oauthlib/auth-httplib2),
registry entries, CLAUDE.md publishing section, .env.example, and a
.gitignore rule for the _internal/.youtube/ token cache.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* docs: soften the unaudited-project private-lock warning to match reality
The pre-upload warning and docs asserted that unverified apps force every
upload to private. Verified empirically that first-party uploads (own Cloud
project + own channel + own account) publish public/scheduled fine without an
audit — the lock mainly targets third-party apps uploading to other channels.
Rewords the runtime warning, module comment, docs/youtube-upload.md, CLAUDE.md,
and .env.example to say public uploads MAY be locked (chiefly cross-account),
and to treat the tool's actual returned privacyStatus as the source of truth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Make /publish channel-agnostic for public use
Drop the SUPERINTELLIGENCE-specific category default and the
schedule-by-default cadence from the /publish command:
- category: neutral default "22" (People & Blogs) with override examples
- privacy: default to `private` (safe least-surprise default) instead of
auto-scheduling a public go-live
Channel-specific defaults (playlist, public-pairs cadence, Studio
checklist) stay in the local working copy, out of the public toolkit.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Robert Strobl <rs@digitalsamba.com>
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* Add concept-explainer-short template: 9:16 TikTok/Reels explainer shorts (Python/moviepy)
First non-Remotion template. The whole video derives from scenes.json
(per-scene narration text + visual asset); a three-script pipeline turns
it into a finished vertical short:
- gen_vo.py — per-scene TTS via tools/voiceover.py (clone or built-in
speaker, batch mode, --max-wpm pacing clamp), writes the
vo_durations.json timeline anchor; warns past the 3-minute
Shorts/Reels ceiling
- gen_captions.py — burned karaoke caption timing: whisper word
timestamps force-aligned to the ground-truth script text via difflib
(never burns whisper's own transcription, which mishears TTS)
- build.py — audio-anchored moviepy composite: Ken Burns on stills,
boomerang-looped clips (asset extension decides), caption pills with
width cap and platform-safe positioning, ducked looped music; renders
at every stage (gradient placeholder cards before assets exist,
word-count timing estimates before VO, silent before audio)
Pattern proven on a 3:41 hallucinations explainer; tested end-to-end
(out-of-box placeholder render + full Modal clone pipeline with three
takes auto-clamped 181-188→165 wpm). Registered in the toolkit registry,
CLAUDE.md, and /video (with Python-flow notes); openai-whisper documented
as an optional dependency.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add sky-blue-short showcase example + fix Modal qwen3-tts batch custom_voice
examples/sky-blue-short: complete worked example of the
concept-explainer-short template — a 52s vertical "Why is the sky blue?"
short with Qwen3 VO (built-in Ryan, warm tone), two Ideogram diagram/CTA
cards, two LTX sky clips, ACE-Step music bed, and burned aligned captions
(~$0.80 total generation cost). All source assets committed; re-renders
in place with python3 build.py.
Building it surfaced two real bugs, fixed here:
- Modal qwen3-tts handler: batch custom_voice hit the same qwen-tts
tensor-aliasing crash the clone path already works around — now
iterates per text with the shared speaker (redeployed and verified)
- gen_vo.py: failed batch scenes crashed with KeyError instead of
reporting which scenes failed and why
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Add hallucinations-short and sky-blue-short demo videos to README showcase
Both produced with the concept-explainer-short pipeline: sky-blue-short
uses the stock Qwen3-TTS Ryan voice; hallucinations-short uses a Qwen3
voice clone from a Pixabay voice-over sample.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
Cloned voices inherit speaking pace from their reference audio, and
regenerating at lower temperature does not correct it — a short, snappy
reference reliably produces 170-210 wpm narration on every take (observed
across 11 consecutive regens). Measuring pace and correcting it
deterministically beats sampling.
- New tools/pacing.py: wpm measurement (pause-marker/SSML aware),
fast/slow/ok labels, and a pitch-preserving ffmpeg atempo clamp
(floor 0.85x so stretching never becomes audible)
- voiceover.py and qwen3_tts.py results now always include wpm + pacing;
human mode prints per-scene pace notes and warnings
- New --max-wpm flag on both tools (all providers, single/batch/scene-dir)
slows rushed takes in place and reports pace_adjusted in JSON
- /voice-clone now validates reference audio before saving (duration +
word-count gates) and teaches that the clone copies the reference's pace
- CLAUDE.md: pacing row in the TTS drift table; registry: pacingQC option
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* FEAT: add Ideogram 4 text-to-image (best-in-class in-image text rendering)
New `tools/ideogram4.py` generates images via Ideogram's hosted v4 API —
its strength is legible baked-in text (title cards, thumbnails, CTAs, quote
cards, in-scene signage), exact hex color palettes, and bounding-box layout,
where FLUX.2 and LTX-2 both garble text.
- Thin multipart wrapper over POST /v1/ideogram-v4/generate (Api-Key header).
Mutually-exclusive --json (structured json_prompt, the quality path) and
--prompt (text_prompt, server-side magic prompt). --brand injects brand.json
hex colors into the caption palette; --speed FLASH/TURBO/DEFAULT/QUALITY.
- New `ideogram4` skill (.claude/skills/ideogram4/) teaching the structured
JSON caption format — Claude acts as the magic-prompt expander (beats the
weaker free hosted one). SKILL.md + prompting.md + examples.md.
- Hosted paid API chosen over self-hosting: the v4 weights are non-commercial,
the API includes a commercial license, and the API's json_prompt preserves
the full prompting-skill value (~$0.03-0.09/image).
- get_ideogram_api_key() in tools/config.py; registry entries for skill+tool.
- CLAUDE.md: FLUX.2-vs-Ideogram-4 decision guidance (baked-in text vs.
background-for-Remotion-overlay).
Tested live: json_prompt title card and text_prompt thumbnail both render with
crisp legible text; resolution 2048x2048 accepted. Closes#25.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* DOCS: refresh ideogram4 registry notes (tool built + live-tested)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
16 tools use `X | None` union syntax in type annotations, which is
evaluated at runtime on function definition and raises TypeError on
Python 3.9 (PEP 604 unions require 3.10+). This broke every cloud GPU
tool on import, despite CLAUDE.md and verify_setup.py advertising
"Python 3.9+" support.
Add `from __future__ import annotations` to each affected file so all
annotations are treated as lazy strings. Verified no non-annotation
3.10+ usage (no match statements, no runtime isinstance unions), so
this fully restores 3.9 support with zero behavior change.
Co-authored-by: Lainie M (LNL Group) <lainiem@lnlcreatives.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replaces the old "fresh ref upload + per-scene clone" pattern with the
proper Qwen3-TTS workflow: VoiceDesign generates a character voice from
a natural-language brief once, the resulting ref is cached, and all
subsequent scenes share a single voice_clone_prompt extracted from it —
Qwen's recommended pattern for consistent multi-scene narration.
Handler (Modal + RunPod, kept in lockstep):
- New "voice_design" mode (Qwen/Qwen3-TTS-12Hz-1.7B-VoiceDesign baked
into the image alongside CustomVoice + Base; ~19GB total)
- New "outputs": [...] response shape; legacy top-level keys still
populated for single-text calls (back-compat)
- Three modes: custom_voice / clone / voice_design
Clone path: extracts voice_clone_prompt ONCE per request and reuses it
across all utterances. Note: qwen-tts's batch path
(generate_voice_clone(text=[list])) hits a PyTorch tensor-aliasing bug
("written-to tensor refers to a single memory location") — we serialize
the calls per-text instead, while still sharing the pre-computed
prompt, which is what gives us the consistency win.
Tools:
- tools/qwen3_tts.py: new --design-instruct flag; generate_audio()
accepts list[str] for batch; design_instruct kwarg routes to
voice_design mode.
- tools/voiceover.py: --scene-dir auto-batches qwen3 clone mode (one
call, shared prompt); brand loader handles new clone.design block
with self-caching of the designed ref.
- tools/align_captions.py (new): transcribes each scene MP3 via
ElevenLabs Scribe, fuzzy-matches each cap() phrase to the word
stream, and patches sprint-config.ts in --apply mode.
- scratch/update_qwen3_template.py (new): one-shot helper to point the
RunPod template at a new image tag via saveTemplate GraphQL.
Deployment status:
- Modal: deployed and live
- RunPod: image built locally as ghcr.io/conalmullan/video-toolkit-
qwen3-tts:0.2.0-voicedesign (20.4GB); GHCR push was auth-denied
(PAT refresh needed). Push + scratch/update_qwen3_template.py to
promote when ready.
NOT YET REVIEWED OR TESTED ON RUNPOD. Modal smoke-test passed
(designed Lugh in 60s, batch-cloned 9 scenes successfully). Verify
RunPod parity once the image is pushed and the template updated.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Ships the lovis93/crt-animation-terminal LoRA baked into the Modal image.
Pipeline rebuilds only when the requested LoRA differs from the currently
loaded one, so repeat calls skip the ~60s reload. Client CLI gains a
--lora flag with LORA_PRESETS that auto-prepend the trigger token and
switch geometry/negative-prompt defaults to what the LoRA expects.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The two new moviepy examples (quick-spot, data-viz-chart) imported
moviepy, Pillow, and matplotlib but had no declaration anywhere in
the toolkit — anyone cloning the repo and running `python3 build.py`
would hit a bare ModuleNotFoundError with no guidance.
This also fixes a pre-existing latent gap: tools/flux2.py and
tools/image_edit.py both `from PIL import Image` inside a try/except
guard but Pillow was never declared in tools/requirements.txt either.
The "friendly error" message was the only signal users got that they
needed to install Pillow themselves. Adding Pillow to the central
file as a free rider.
Changes
-------
- tools/requirements.txt: adds Pillow>=10.0, moviepy>=2.0, matplotlib>=3.7
with comments explaining which tools/examples use each
- examples/quick-spot/build.py, examples/data-viz-chart/build.py: wrap
the third-party imports in a try/except ImportError matching the
existing flux2.py/image_edit.py pattern, with a precise error
message pointing at `python3 -m pip install -r tools/requirements.txt`
- examples/*/README.md: Quick Start now lists the toolkit-level pip
install as step one, then `python3 build.py`
- README.md: updates the optional Python install callout to use
`python3 -m pip install` (avoids the pip/python3 env mismatch
footgun) and mentions moviepy examples alongside AI voiceover/
image gen/music in the "what this installs" note
- .claude/skills/moviepy/SKILL.md: adds a short Dependencies note so
Claude Code knows where the deps live and what to run on ImportError
Net effect: one install command covers every Python feature in the
toolkit. Both AI agents (via stderr-reactive friendly errors) and
humans (via the README Quick Start) can recover from a missing-dep
state without guesswork.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Add 20-min subprocess timeout (buffer over ltx2's 15-min cloud timeout)
- Switch to capture_output=True with stderr in error messages
- Validate --prompts-file JSON with helpful error on malformed input
- Clean up .frame-*.png temp files after run completes
- Fix glob pattern to require hyphen separator (prevents 05 matching 050)
- Add chain_video to toolkit-registry.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
chain-04-brigid.mp4 wasn't matched by exact path chain-04.mp4.
Now globs for chain-{num}*.mp4 to handle manually-named clips.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Runs all scenes as a single command — solves OpenClaw agent stalling
between per-scene tool calls. Supports per-scene prompts via JSON,
auto-resume (skips existing files), and --progress json.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
ProgressReporter class in cloud_gpu.py with two modes:
- human (default): colored stderr logs, unchanged from today
- json (--progress json): JSON Lines to stderr for bots/agents
Supports heartbeat thread for synchronous providers (acemusic, Modal)
and poll-based events for RunPod. All 8 cloud tools wired up:
music_gen, qwen3_tts, flux2, upscale, sadtalker, image_edit,
dewatermark, ltx2.
OpenClaw skill updated with --progress json in all example commands
and a CRITICAL directive for the bot to always use it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Switch music_gen.py from self-hosted Modal/RunPod to the official ACE-Step
cloud API (acemusic.ai) as the default provider. This gives access to the
XL Turbo 4B model with 5Hz LM thinking mode — significantly better quality
and variety than the self-hosted 2B model, with free API keys and no GPU costs.
New features: --thinking/--no-thinking, --variations, --repaint,
--continuation, --guidance-scale, --infer-method. Modal/RunPod remain as
--cloud modal/runpod fallbacks.
Skill doc updated with vocal gender control and duet patterns discovered
through testing.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Fix hardcoded 1800s Modal container timeout that could kill jobs during
cold starts or moderate-length audio
- Dynamic per-chunk subprocess timeouts scale with audio duration
instead of flat 1800s
- Add pre-flight estimate (chunks, time, cost) before job submission
- Add chunk-level progress logging to Modal app
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All 6 tools (flux2, image_edit, music_gen, qwen3_tts, sadtalker,
voiceover) now default to --cloud modal. RunPod remains available
as --cloud runpod for users who prefer it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Integrates Lightricks LTX-2.3 22B DiT model for AI video generation.
Generates ~5s video clips at up to 1024x1536 with synchronized audio.
- docker/modal-ltx2/app.py: Modal deployment on A100-80GB with baked weights
(~55GB: dev checkpoint + distilled LoRA + spatial upsampler + Gemma 3 12B)
- tools/ltx2.py: CLI tool with --prompt, --input (I2V), --width/--height,
--num-frames, --quality, --seed
- tools/cloud_gpu.py: Wire ltx2 into Modal endpoint routing and cost estimation
- docs/ltx2.md: Setup guide, parameters, prompting tips, troubleshooting
Key implementation notes:
- torch.inference_mode() required around pipeline call (GitHub #152) —
without it, Gemma text encoder retains ~37GB of autograd activations
causing OOM even on 80GB GPU
- Weights are on Lightricks/LTX-2.3 HF repo (not Lightricks/LTX-2)
- transformers pinned to 4.57.6 (5.x breaks Gemma3TextConfig)
- Pipeline returns lazy iterator — inference_mode scope must include
encode_video() call
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New /setup command: interactive 6-phase onboarding wizard for new users
(prerequisites, cloud GPU provider, R2 file transfer, tool deployment,
voice config, verification)
- New verify_setup.py: checks all prerequisites, env vars, deployed apps,
and optionally runs smoke tests (--test flag)
- Migrate dewatermark.py to cloud_gpu.py: replace ~400 lines of inline
RunPod code with shared abstractions, add --cloud runpod|modal flag
- Add dewatermark to Modal env vars and GPU cost tables in cloud_gpu.py
- Rewrite .env.example: all 22 env vars organized by service, Modal
section added, references /setup
- Update README, CLAUDE.md, getting-started.md, video.md to reference
/setup as the primary onboarding path
- Register setup command and verify_setup tool in toolkit-registry.json
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Extract duplicated RunPod submit/poll and R2 file transfer code into
shared modules. All 6 cloud GPU tools now route through cloud_gpu.py
and file_transfer.py, reducing ~1,150 lines of duplication.
Changes:
- Create tools/file_transfer.py with shared R2 upload/download + fallback
- Migrate image_edit, music_gen, upscale, sadtalker to call_cloud_endpoint()
- Add --cloud runpod|modal flag to all cloud GPU tools
- Deprecate --runpod flag in upscale.py (alias for --cloud runpod)
- Add --cloud passthrough to voiceover.py for Qwen3-TTS
- Backport file_transfer.py to qwen3_tts.py and flux2.py
- Remove dead legacy submit/poll code from Phase 1 tools
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
RunPod has been unreliable — Modal provides faster cold starts and
more robust infrastructure. This adds Modal support for qwen3_tts
and flux2 behind a new --cloud runpod|modal flag (default: runpod).
New files:
- tools/cloud_gpu.py: shared provider abstraction with cost estimation
- docker/modal-qwen3-tts/app.py: Modal deployment for Qwen3-TTS
- docker/modal-flux2/app.py: Modal deployment for FLUX.2 Klein 4B
Modified:
- tools/qwen3_tts.py: --cloud flag, routes through cloud_gpu
- tools/flux2.py: --cloud flag, routes through cloud_gpu
- tools/config.py: Modal config getters
- _internal/toolkit-registry.json: Modal provider section
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Integrates ACE-Step 1.5, an open-source music generation model, as a free
alternative to ElevenLabs Music with precise BPM/key/duration control,
vocal support, and stem extraction capabilities.
- tools/music_gen.py: CLI tool with text2music, cover, stem extraction,
8 scene presets, brand integration, and RunPod --setup
- docker/runpod-acestep/: Dockerfile (CUDA 12.8, torch 2.10, baked weights)
and RunPod serverless handler using ACE-Step inference API
- Registry updated with music_gen tool and acestep cloud endpoint
RunPod end-to-end test pending GPU availability.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
All 6 RunPod tools now auto-cancel jobs stuck in queue >5 min and
cancel orphaned jobs on polling timeout, preventing runaway billing
from GPU unavailability. Fixed R2 upload support in flux2 and
image_edit clients. Switched flux2 default GPU to AMPERE_24 with
ADA_24 fallback. Added RunPod API reference to skill docs.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- New tools/flux2.py: text-to-image, image editing, multi-image compositing
- 8 scene presets for video production (title-bg, problem, solution, etc.)
- Brand-aware generation reads brand.json colors into prompts
- Docker image with baked model weights (ghcr.io/conalmullan/video-toolkit-flux2)
- Rebuilt qwen3-tts Docker image with baked models
- "The Space Between" showcase video (all AI-generated)
- Updated README, registry, and changelog
Closes#6
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
New tool: tools/sync_timing.py — syncs scene durationSeconds in Remotion
config with actual audio durations after TTS generation. Auto-detects
template type, matches audio to scenes via 3-pass algorithm (audioFile →
index → name), shows comparison table, and applies updates in-place.
CLAUDE.md reduced from 861 → 480 lines (44%) by removing catalog data
duplicated in toolkit-registry.json (skills, commands, components,
transitions, presets, Docker images) and adding cross-references.
All workflow guidance, timing knowledge, and code patterns retained.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Make it clear that videos render with just Node.js installed. Reorganize
.env.example with commented-out defaults, add "Try It Now" quick start
pointing to hello-world example, and replace terse API key errors in all
Python tools with actionable guidance (add key, use alternative, or skip).
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Pass temperature and top_p kwargs through the full stack (handler.py →
qwen3_tts.py → voiceover.py) to control voice expressiveness and sampling.
Fix docker-build.yml multi-line JSON breaking $GITHUB_OUTPUT by using jq -sc.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Add named tone presets (warm, professional, excited, calm, etc.) with
--tone flag and --list-tones for both qwen3_tts.py and voiceover.py
- Add per-scene instruct overrides via [tone: X] frontmatter in .txt files
- Add /voice-clone command for guided clone setup workflow
- Add brand voice config loading (speaker, tone, clone from voice.json)
- Guard against tone/instruct with cloned voice (clone mode ignores instruct)
- Fix PR #3 code quality issues (unused import, unused variable, empty except)
- Update commands, docs, and CLAUDE.md with tone examples
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds Qwen3-TTS as a free, self-hosted alternative to ElevenLabs for
voiceover generation via RunPod serverless. Supports 9 built-in speakers,
natural-language emotion control, and voice cloning from reference audio.
- Docker image (docker/runpod-qwen3-tts/) with RunPod handler
- CLI tool (tools/qwen3_tts.py) with --setup, --list-voices, generation
- voiceover.py: --provider qwen3 support with lazy ElevenLabs imports
- GitHub Actions workflow for automated Docker image builds
- Config, registry, docs, and generate-voiceover command updated
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- voiceover.py: --scene-dir processes .txt files, --concat for SadTalker
- sprint-review: audioFile field on DemoConfig, info, overview, summary
- SprintReview.tsx: per-scene <Audio> elements with backward compatibility
- /generate-voiceover: detects scene scripts, offers per-scene as default
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add comprehensive docs/sadtalker.md with NarratorPiP workflow
- Document critical patterns: 16:9 image prep, --preprocess full
- Add auto-timeout calculation based on audio duration
- Add --retrieve flag for job recovery after client timeout
- Update NarratorPiP component with objectFit: contain
- Track learnings in BACKLOG.md for future improvements
Key insight: Always use --preprocess full to preserve image
dimensions for video overlays. Let users crop their own images
for best face framing.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add constraints file to enforce numpy<2 across all pip installs
- Patch numpy 2.0 deprecated APIs in SadTalker and dependencies
- Fix trans_params ValueError with dtype=object
- Fix output video detection (check direct files, not just subdirs)
- Add verbose error logging to handler
- Update PROGRESS.md with testing results and resume instructions
Tested with short (15s) and long (2.3min) audio - chunking works.
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Docker image: runpod-sadtalker with model weights baked in
- Handler with automatic chunking for long audio (>45s)
- CLI tool following existing patterns (--setup, R2 integration)
- GFPGAN face enhancement included
Experiment branch - needs Docker build/push and testing.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add AI image editing tools (qwen-edit, upscale)
New features:
- image_edit.py: AI image editing using Qwen-Image-Edit-2511
- Style transfer, background replacement, custom prompts
- Multi-image compositing support
- Presets for common operations
- upscale.py: AI image upscaling using RealESRGAN
- 2x/4x upscaling with general, anime, and photo models
- Optional face enhancement (GFPGAN)
- qwen-edit skill: Prompting patterns and best practices
- RunPod workers for cloud GPU processing
Both tools use RunPod serverless for cloud processing.
See docs/qwen-edit-patterns.md for prompting guidance.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Add pre-built Docker image references for RunPod tools
- Add GHCR image URLs for qwen-edit, realesrgan, propainter
- Consolidate Cloud GPU tools category (image_edit, upscale, dewatermark)
- Document that --setup uses pre-built images (no building required)
Images hosted at ghcr.io/conalmullan/video-toolkit-*
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
* Update RunPod docs for all Cloud GPU tools, bump to v0.8.0
- Restructure runpod-setup.md to cover image_edit, upscale, dewatermark
- Add GPU requirements table for each tool
- Document all three endpoint environment variables
- Update pre-built image references
- Bump version to 0.8.0
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Adds 'sora' preset to both dewatermark.py and locate_watermark.py
for removing the bottom-right "SORA" text watermark from Sora-generated videos.
Region: 1710,965,140,50 at 1920x1080
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
AI-powered watermark removal using ProPainter video inpainting:
dewatermark.py (2400+ lines):
- Cloud processing via RunPod serverless GPUs
- Local processing for NVIDIA GPU users
- Auto resize-ratio based on video duration (1.0 for <30s, 0.75 for <1min)
- Cloudflare R2 for reliable file transfer
- Automated --setup for one-command RunPod configuration
- Chunked processing for long videos
- Preset watermark regions (notebooklm, tiktok, stock variants)
locate_watermark.py:
- Coordinate grid overlay for identifying watermark positions
- Region verification across multiple frames
- Preset support for common watermarks
Infrastructure:
- docker/runpod-propainter/ - Serverless Docker image + handler
- docs/runpod-setup.md - Cloud GPU setup guide
- docs/optional-components.md - ML component documentation
- R2 + RunPod configuration in .env.example
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New tool handles the specific challenge of replacing NotebookLM branding
on redubbed/synced videos where TTS audio extends beyond the visual trim point.
Features:
- Separates video and audio tracks for independent handling
- Trims video to remove NotebookLM visual outro
- Creates freeze frame to bridge gap while audio finishes
- Appends custom branded outro card
- Auto-detects or accepts manual trim point
Usage:
python tools/notebooklm_brand.py \
--input synced.mp4 \
--audio-file tts.mp3 \
--outro-card outro.png \
--output final.mp4
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
New utility tool that adds background music to existing videos:
- Generate music via ElevenLabs prompt (auto-matches video duration)
- Or use existing music file
- Volume controls for both original audio and music
- Fade in/out options for smooth transitions
- Standard flags: --json, --dry-run, --keep-temp
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
AI-native video production workspace for Claude Code featuring:
- 4 skills: remotion, elevenlabs, ffmpeg, playwright-recording
- 3 commands: /new-sprint-video, /record-demo, /generate-voiceover
- Sprint review template with theme system
- Brand profiles for visual identity
- Python tools for audio generation (voiceover, music, sfx)
- 2 example projects: sprint-review-cho-oyu, digital-samba-skill-demo
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Conal Mullan