mirror of
https://github.com/calesthio/OpenMontage.git
synced 2026-08-25 17:42:50 +08:00
feat(stt): add Azure AI Speech as an optional cloud speech-to-text provider
Add an Azure AI Speech transcription tool. It is opt-in: when AZURE_SPEECH_KEY is configured the agent may prefer it for cloud STT, while the local faster-whisper `transcriber` stays the default offline path. Shared pipeline manifests are intentionally left unchanged, so no default provider selection is altered for existing users. - tools/analysis/azure_stt.py: new `azure_stt` tool (capability=analysis, provider=azure) calling the Fast Transcription REST API. The local file is uploaded via multipart and transcribed synchronously with word-level timestamps and optional diarization — no Blob storage or async polling. Output schema mirrors `transcriber` exactly, so it is a drop-in for `subtitle_gen` and other transcript consumers. Follows the existing provider-tool conventions (env-var status check, `_transcribe` helper, cost_usd/model on the result, fallback="transcriber"). - Auto-discovered by the registry; no registry or selector changes. - tests/tools/test_azure_stt.py: contract, discovery, status, response mapping, execute guardrails, and a mocked-network success path (no live API calls). - .agents/skills + .claude/skills: azure-speech-to-text Layer-3 skill. - docs/PROVIDERS.md: Azure AI Speech setup, API notes, and pricing. - .env.example, skills/INDEX.md, AGENT_GUIDE.md: document the optional cloud STT path alongside the default whisper transcriber.
This commit is contained in:
@@ -81,7 +81,8 @@ Key capability families to look for in the output:
|
||||
| FFmpeg | `core/ffmpeg.md` | Video encoding, filtering, composition | `ffmpeg`, `video-toolkit` |
|
||||
| Remotion | `core/remotion.md` | React-based composition, Phase 3+ | `remotion-best-practices`, `remotion` |
|
||||
| HyperFrames | `core/hyperframes.md` | HTML/CSS/GSAP composition runtime — kinetic typography, music-to-video, product promos, website capture. Vendored at v0.7.17 (2026-06-27). | `hyperframes` (router) → `hyperframes-core` (contract), `hyperframes-creative` (palette/type/narration), `hyperframes-media` (TTS/BGM/SFX/captions), `hyperframes-animation` (all motion), `hyperframes-cli`, `hyperframes-registry`, `media-use`, `motion-graphics`, `music-to-video` (beats-driven), `website-to-video`, `remotion-to-hyperframes` (migration), `gsap-core`, `gsap-timeline` |
|
||||
| WhisperX | `core/whisperx.md` | Transcription with word-level timestamps | `speech-to-text` |
|
||||
| WhisperX | `core/whisperx.md` | Transcription with word-level timestamps — default STT (offline, free) | `speech-to-text` |
|
||||
| Azure STT | (tool: `azure_stt`) | Optional cloud speech-to-text, word-level timestamps — preferred when `AZURE_SPEECH_KEY` is set | `azure-speech-to-text` |
|
||||
| Subtitle Sync | `core/subtitle-sync.md` | Subtitle timing and alignment | `remotion-best-practices` |
|
||||
| Color Grading | `core/color-grading.md` | FFmpeg color profiles, LUT workflow, accessibility | `ffmpeg` |
|
||||
|
||||
@@ -306,7 +307,7 @@ Claude Code accesses them via symlinks in `.claude/skills/`.
|
||||
|----------|-----------------|--------|
|
||||
| **Video Composition** | `remotion-best-practices`, `remotion`, `hyperframes` (router), `hyperframes-core`, `hyperframes-creative`, `hyperframes-media`, `hyperframes-animation`, `hyperframes-cli`, `hyperframes-registry`, `media-use`, `motion-graphics`, `music-to-video`, `remotion-to-hyperframes`, `website-to-video` | `remotion-dev/skills`, `digitalsamba/claude-code-video-toolkit`, `heygen-com/hyperframes` (vendored v0.7.17, see `.agents/skills/hyperframes/PROVENANCE.md`) |
|
||||
| **Video Processing** | `ffmpeg`, `video-toolkit` | `digitalsamba/claude-code-video-toolkit` |
|
||||
| **TTS & Audio** | `text-to-speech`, `speech-to-text`, `music`, `sound-effects`, `elevenlabs`, `agents`, `setup-api-key` | `elevenlabs/skills`, `digitalsamba/claude-code-video-toolkit` |
|
||||
| **TTS & Audio** | `text-to-speech`, `speech-to-text` (whisper, default STT), `azure-speech-to-text` (optional cloud STT), `music`, `sound-effects`, `elevenlabs`, `agents`, `setup-api-key` | `elevenlabs/skills`, `digitalsamba/claude-code-video-toolkit` |
|
||||
| **Image Generation** | `flux-best-practices`, `bfl-api`, `grok-media` | `black-forest-labs/skills`, local OpenMontage skill |
|
||||
| **Math Animation** | `manimce-best-practices`, `manimgl-best-practices`, `manim-composer` | `adithya-s-k/manim_skill` |
|
||||
| **3D Graphics** | `threejs-animation`, `threejs-fundamentals`, `threejs-geometry`, `threejs-interaction`, `threejs-lighting`, `threejs-loaders`, `threejs-materials`, `threejs-postprocessing`, `threejs-shaders`, `threejs-textures` | `cloudai-x/threejs-skills` |
|
||||
|
||||
Reference in New Issue
Block a user