zlei6/ragflow
1
0
Fork 0
mirror of https://github.com/infiniflow/ragflow.git synced 2026-07-18 21:57:27 +08:00
Code Issues Packages Projects Releases Wiki Activity

Go: implement TTS for fishaudio, openrouter and asr for fishaudio (#14926)

Browse Source 
### What problem does this PR solve?

This PR implement TTS for FishAudio and MiniMax provider and ASR for
FishAudio

**The following functionalities are now supported:**

**FishAudio:**
- [x] Text To Speech
- [x] Stream Text To Speech
- [x] Audio To Text

**OpenRouter:**

- [x] Text To Speech

**Verified examples from the CLI:**
```plaintext

**FishAudio**

RAGFlow(user)> tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> stream tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> asr with 'transcribe-1@test@fishaudio' audio './internal/test.wav' param '{"language": "en", "ignore_timestamps": true}'
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+

```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
This commit is contained in:
Haruko386
2026-05-14 18:58:00 +08:00
committed by GitHub
parent a98994ff91
commit 106f4b777e
9 changed files with 436 additions and 33 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
internal/handler/providers.go
View File

@@ -1049,13 +1049,14 @@ func (h *ProviderHandler) RerankDocument(c *gin.Context) {
}
type TranscribeAudioRequest struct {
ProviderName *string `json:"provider_name"`
InstanceName *string `json:"instance_name"`
ModelName *string `json:"model_name"`
File *string `json:"file"`
Language []string `json:"language"`
Prompt int `json:"prompt"`
Stream bool `json:"stream"`
ProviderName *string `json:"provider_name"`
InstanceName *string `json:"instance_name"`
ModelName *string `json:"model_name"`
File *string `json:"file"`
Language []string `json:"language"`
Prompt int `json:"prompt"`
Stream bool `json:"stream"`
ASRConfig *models.ASRConfig `json:"asr_config"`
}
func (h *ProviderHandler) TranscribeAudio(c *gin.Context) {
@@ -1101,6 +1102,9 @@ func (h *ProviderHandler) TranscribeAudio(c *gin.Context) {
}
asrConfig := models.ASRConfig{}
if req.ASRConfig != nil {
asrConfig = *req.ASRConfig
}
// Check if it's a stream request
if req.Stream {