* Add Modal as alternative cloud GPU provider (Phase 1) RunPod has been unreliable — Modal provides faster cold starts and more robust infrastructure. This adds Modal support for qwen3_tts and flux2 behind a new --cloud runpod|modal flag (default: runpod). New files: - tools/cloud_gpu.py: shared provider abstraction with cost estimation - docker/modal-qwen3-tts/app.py: Modal deployment for Qwen3-TTS - docker/modal-flux2/app.py: Modal deployment for FLUX.2 Klein 4B Modified: - tools/qwen3_tts.py: --cloud flag, routes through cloud_gpu - tools/flux2.py: --cloud flag, routes through cloud_gpu - tools/config.py: Modal config getters - _internal/toolkit-registry.json: Modal provider section Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Migrate all cloud GPU tools to shared abstraction (Phase 2) Extract duplicated RunPod submit/poll and R2 file transfer code into shared modules. All 6 cloud GPU tools now route through cloud_gpu.py and file_transfer.py, reducing ~1,150 lines of duplication. Changes: - Create tools/file_transfer.py with shared R2 upload/download + fallback - Migrate image_edit, music_gen, upscale, sadtalker to call_cloud_endpoint() - Add --cloud runpod|modal flag to all cloud GPU tools - Deprecate --runpod flag in upscale.py (alias for --cloud runpod) - Add --cloud passthrough to voiceover.py for Qwen3-TTS - Backport file_transfer.py to qwen3_tts.py and flux2.py - Remove dead legacy submit/poll code from Phase 1 tools Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add Modal apps for upscale, image-edit, music-gen, and sadtalker (Phase 3) Create Modal deployment files for all remaining cloud GPU tools, giving each tool --cloud modal support alongside RunPod. New Modal apps: - docker/modal-upscale/app.py — RealESRGAN (3 models, face enhance) - docker/modal-image-edit/app.py — Qwen-Image-Edit (A100 for large model) - docker/modal-music-gen/app.py — ACE-Step 1.5 (text2music, cover, extract) - docker/modal-sadtalker/app.py — SadTalker (subprocess, audio chunking) Deploy any app with: modal deploy docker/modal-{tool}/app.py Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix Modal app build issues discovered during deployment - modal-upscale: deployed successfully (no changes needed) - modal-image-edit: deployed successfully (no changes needed) - modal-music-gen: deployed successfully (no changes needed) - modal-sadtalker: fix pip install order (numpy<2 before basicsr), add mkdir for /root/.cache/torch before unzipping hub models All 4 new endpoints deployed and URLs saved to .env. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix sadtalker Modal app: numpy-torch bridge and basicsr compat Root cause: Modal's pip mirror serves generic torch (cu121) which doesn't detect numpy at install time, causing "Numpy is not available" at runtime. Also basicsr imports removed torchvision.transforms.functional_tensor. Fixes: - Use torch 2.4.0 (handles numpy 1.x/2.x correctly) - Patch basicsr's deprecated functional_tensor import - Add debug logging for numpy-torch bridge in @modal.enter() Tested: claude_avatar.png + test_modal.mp3 → 4.4s video in 130s (cold start) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Track Modal cloud GPU integration in ROADMAP.md Add Modal provider section to Phase 3 with checklist of completed and remaining items. Add modal skill to status table as beta. Remaining: docs/modal-setup.md, --setup --cloud modal automation. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add Modal docs, propainter app, and cloud GPU quick start - Create docs/modal-setup.md with motivation, quick start, cold start times, monitoring, and architecture overview - Add propainter (dewatermark) Modal app for video inpainting - Update CLAUDE.md with cloud GPU providers quick start section - Update registry with dewatermark Modal endpoint - Update ROADMAP with docs tracking Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add /setup wizard, verify script, and migrate dewatermark to cloud_gpu - New /setup command: interactive 6-phase onboarding wizard for new users (prerequisites, cloud GPU provider, R2 file transfer, tool deployment, voice config, verification) - New verify_setup.py: checks all prerequisites, env vars, deployed apps, and optionally runs smoke tests (--test flag) - Migrate dewatermark.py to cloud_gpu.py: replace ~400 lines of inline RunPod code with shared abstractions, add --cloud runpod|modal flag - Add dewatermark to Modal env vars and GPU cost tables in cloud_gpu.py - Rewrite .env.example: all 22 env vars organized by service, Modal section added, references /setup - Update README, CLAUDE.md, getting-started.md, video.md to reference /setup as the primary onboarding path - Register setup command and verify_setup tool in toolkit-registry.json Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Restructure README: quick start first, eliminate overlap - Move quick start to top of file, right after title - Consolidate 5 setup sub-sections into tight 3-line flow - Remove "What is this?" section (merged into intro line) - Update author note: recommend Modal over RunPod, point to /setup - Update tool examples to use --cloud modal - Rename "Pre-built Docker Images" to "Cloud GPU" with both providers - Add /setup to commands table - Add Modal setup doc to documentation links - Net -56 lines while adding more useful content Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Rewrite modal-setup.md: Modal as recommended provider, add R2 and /setup refs - Frame Modal as recommended (not "alternative"), explain $30/mo free tier - Add account creation steps with signup link - Add Cloudflare R2 section (was completely missing) - Reference /setup as the automated path throughout - Add verify_setup.py to monitoring section - Update cold start times from actual testing (dewatermark was missing) - Move RunPod to "Alternative" section at bottom - Add tool usage examples with --cloud modal Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix Cloud GPU section: Modal builds its own images, not GHCR Modal apps deploy from docker/modal-*/app.py — Modal builds containers from the Image definition, it doesn't pull GHCR images. GHCR images are only used by RunPod. Restructured table to focus on tools and costs, clarified the two provider paths below. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Reframe 'what's free' around open-source models, not just pricing The toolkit's value prop is deploying capable open-source AI models to your own cloud GPU — not just that some services have free tiers. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Clarify Modal free tier capacity: a few 5-minute videos per month Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix R2 region, SadTalker timeouts, and tool path guidance - file_transfer.py: Add region_name="auto" for R2 client — fixes InvalidRegionName error when boto3 inherits AWS config region - modal-sadtalker/app.py: Bump subprocess timeout 600s→1800s and app timeout 900s→1800s — 60s audio needs ~10min processing - sadtalker.py: Bump client timeout multiplier 4x→12x to match actual processing time observed on Modal A10G - CLAUDE.md: Add guidance to always invoke tools from toolkit root directory (not from project subdirectories) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Fix lib/transitions barrel: remove @remotion/transitions re-exports The barrel re-exported from @remotion/transitions which breaks when projects outside lib/ import from it — webpack can't resolve the package from lib/'s directory. Now the barrel only exports custom transitions (glitch, lightLeak, etc). Projects import TransitionSeries, linearTiming, fade etc directly from @remotion/transitions. Also add webpack resolve.modules override to all 3 template configs as a belt-and-suspenders fix for other lib/ imports. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add OffthreadVideo guidance: never use raw <video> in Remotion Remotion requires <OffthreadVideo> for frame-accurate rendering. Raw <video> tags won't render correctly during export. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add OpenClaw skill for autonomous video creation (alpha) Self-contained SKILL.md with the complete workflow for an OpenClaw bot to create videos autonomously: setup verification, project creation, asset generation (TTS, images, music, talking heads), timing sync, review, and rendering. Includes all the gotchas we discovered during the toolkit-intro video project. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Add full setup flow to OpenClaw skill The skill now includes Modal CLI install, tool deployment, endpoint URL capture, .env configuration, R2 setup, and verification — everything a bot needs to go from zero to ready. Removed the hard env var requirement from metadata since the skill handles setup itself. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * Harden OpenClaw skill for autonomous use - Hardcode toolkit path with cd before every command block - Fix speaker list (was wrong names) and add missing tones - Add quick-test verification step in setup - Add --break-system-packages for Debian/managed Python - Add error recovery table for common failures - Repeat cd to toolkit root before every command (LLM session memory lesson) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
5.4 KiB
Getting Started
This guide will help you create your first video using the claude-code-video-toolkit.
Prerequisites
Minimum (renders videos immediately)
- Node.js 18+ — that's it
Optional: AI Voiceover
| Provider | Cost | Setup |
|---|---|---|
| Qwen3-TTS | Free (self-hosted) | RunPod account + python tools/qwen3_tts.py --setup |
| ElevenLabs | Pay-per-use | API key in .env |
Optional: Full Toolkit
- Python 3.9+ — for audio tools, image editing, upscaling
- FFmpeg — for media conversion and compression
- RunPod account — for cloud GPU processing (TTS, image editing, watermark removal, talking heads)
- ElevenLabs API key — for premium AI voices
Your First Video in 2 Minutes
cd examples/hello-world
npm install
npm run studio # Preview in browser
npm run render # Export MP4
No API keys needed. Edit src/config/sprint-config.ts to customize content.
Full Setup (for AI Tools)
This is optional — you can render videos with just Node.js installed.
-
Clone the repository
git clone https://github.com/digitalsamba/claude-code-video-toolkit.git cd claude-code-video-toolkit -
Install Python dependencies
python -m venv .venv source .venv/bin/activate # On Windows: .venv\Scripts\activate pip install -r tools/requirements.txt -
Start Claude Code and run the setup wizard
claudeThen type
/setup— this walks you through configuring cloud GPU, file transfer, and voice in about 5 minutes. Most features are free:- Cloudflare R2: Free (10GB storage, zero egress)
- Modal: $30/month free compute on Starter plan
- Qwen3-TTS: Free AI voiceovers (runs on your Modal compute)
Or configure manually:
cp .env.example .envand edit with your API keys.
Your First Video
The easiest way to create a video is using the /video command:
/video
This unified command will:
- Scan for existing projects (or start fresh if none found)
- Let you choose a template (Sprint Review or Product Demo)
- Let you choose a brand (or create one with
/brand) - Gather your content (paste notes, provide URLs, or describe what you want)
- Plan scenes interactively with your input
- Create a project in
projects/with all scaffolding ready
Manual Project Creation
If you prefer manual setup:
-
Copy a template
cp -r templates/sprint-review projects/my-video cd projects/my-video npm install -
Edit the config Edit
src/config/sprint-config.tswith your content. -
Add demo videos Place
.mp4files inpublic/demos/ -
Preview
npm run studio -
Render
npm run render
Available Commands
| Command | Description |
|---|---|
/setup |
First-time setup - cloud GPU, file transfer, voice, prerequisites |
/video |
Video projects - list, resume, or create new |
/scene-review |
Scene-by-scene review in Remotion Studio |
/design |
Focused design refinement session for a scene |
/brand |
Brand profiles - list, edit, or create new |
/template |
List available templates or create new ones |
/record-demo |
Record browser interactions with Playwright |
/generate-voiceover |
Generate AI voiceover from script (supports per-scene mode) |
/redub |
Redub existing video with a different voice |
/skills |
List installed skills or create new ones |
/contribute |
Share improvements - issues, PRs, examples |
/versions |
Check dependency versions and toolkit updates |
Project Structure
After creating a project, you'll have:
projects/my-video/
├── project.json # Project state (phase, scenes, assets)
├── CLAUDE.md # Auto-generated status for Claude Code
├── VOICEOVER-SCRIPT.md # Narration script with asset markers
├── src/
│ ├── config/
│ │ ├── sprint-config.ts # Your video content (or demo-config.ts)
│ │ ├── brand.ts # Brand colors/fonts (auto-generated)
│ │ └── theme.ts
│ └── components/
├── public/
│ ├── demos/ # Your demo videos
│ ├── audio/ # Voiceovers, music, SFX
│ └── images/ # Logo, screenshots
└── package.json
Multi-Session Workflow
Projects can span multiple Claude Code sessions. The /video command tracks progress:
/video
When you have existing projects, you'll see:
Found 2 video projects:
1. **my-release-video** (sprint-review)
Phase: assets - 2/5 demos recorded
Last worked: 2 days ago
2. **product-launch** (product-demo)
Phase: audio - voiceover needed
Last worked: 5 days ago
Which project? (or 'new' for a new project)
Project Phases
| Phase | Description |
|---|---|
planning |
Defining scenes, writing script |
assets |
Recording demos, gathering materials |
review |
Scene-by-scene review in Remotion Studio (/scene-review) |
audio |
Generating voiceover, music |
editing |
Adjusting timing, previewing |
rendering |
Final render in progress |
complete |
Done |
Next Steps
- Creating Templates - Build custom video structures
- Creating Brands - Define visual identity