mirror of
https://github.com/CopilotKit/CopilotKit.git
synced 2026-09-14 16:26:20 +08:00
Alem Tuzlak
e96471bd3d
Ports beautiful-chat, byoc-hashbrown, byoc-json-render, multimodal, open-gen-ui, open-gen-ui-advanced, and voice from the langgraph-python reference into the mastra integration. Each frontend page is reused as-is. Backends use existing weatherAgent for cells where the UI shape is enforced on the frontend (byoc-*, open-gen-ui-*, beautiful-chat). Multimodal gets a dedicated vision-capable Mastra agent (gpt-4o) registered in the Mastra config. Voice wires the V2 runtime directly with TranscriptionServiceOpenAI so OPENAI_API_KEY-missing returns a clean 401. Manifest features and demos lists updated. Frontend/backend agent name aliases aligned with each page's <CopilotKit agent=...> prop.
Voice demo audio
Drop a small (<100KB) WAV file named sample.wav in this directory. The voice
demo (/demos/voice) fetches it client-side and POSTs to the transcription
endpoint so the flow works without mic permissions.
Expected content: an audio clip speaking the phrase "What is the weather in Tokyo?" — the demo caption advertises that phrase to the user, and the bundled QA checklist + E2E spec assert the transcribed text contains "weather" and/or "Tokyo".
Generate locally, for example:
- macOS:
say -o sample.aiff "What is the weather in Tokyo?" && ffmpeg -i sample.aiff -ar 16000 -ac 1 sample.wav - Linux:
espeak-ng -w sample.wav "What is the weather in Tokyo?" - Windows: PowerShell
System.Speech.Synthesis.SpeechSynthesizer→SetOutputToWaveFile
Target: 16kHz mono, 3-5s duration, <100KB.