mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-12 20:03:40 +08:00
feat(stt): add FunASR / SenseVoice provider (#16473)
### Summary Adds FunASR as a self-hosted speech-to-text provider through its OpenAI-compatible `/v1/audio/transcriptions` endpoint. This is a focused replacement for #15526 by @Rene0422 and relates to #15448. The unrelated Markdown parser changes from the previous branch are intentionally removed so this PR contains only the FunASR provider integration. - register FunASR as a `SPEECH2TEXT` factory; - add `FunASRSeq2txt` with `sensevoice` and `http://localhost:8000/v1` defaults, an optional API key, URL normalization, and inherited transcription handling; - wire FunASR into the current local-provider schema with a prefilled local URL and official documentation link; - discover the server's `/v1/models` dynamically and expose every returned model as speech-to-text in the model picker; - use RAGFlow's existing default provider icon fallback instead of referencing a missing `funasr` asset; - list FunASR in the supported-provider documentation; - add focused backend and frontend regression tests. ### Validation - focused backend pytest suite -> `7 passed` - real CPU `funasr-server` + RAGFlow provider smoke test -> discovered `fun-asr-nano`, `sensevoice`, and `paraformer`; transcribed a real WAV as `我现在在录一段测试音频` (`10` tokens, `0.504s`) - `ruff check` and `ruff format --check` on the changed Python files - `python3 -m py_compile` on the provider and its test - JSON parse and a semantic assertion for exactly one enabled FunASR `SPEECH2TEXT` factory - focused frontend Jest test -> `2 passed` - ESLint and Prettier on all changed TypeScript files - `npm run build` -> production build succeeded (`14,181` modules transformed) - `git diff --check` ### Deployment Run FunASR separately and point the RAGFlow provider at it: ```bash pip install funasr funasr-server --device cuda --model sensevoice ``` The API key remains optional because the stock local server does not require authentication. A key can still be supplied when the endpoint is protected by a gateway. --------- Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com> Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
This commit is contained in:
@@ -458,6 +458,29 @@ class OpenAIAPICompatible(Base):
|
||||
return model_list
|
||||
|
||||
|
||||
class FunASR(Base):
|
||||
_FACTORY_NAME = "FunASR"
|
||||
|
||||
def _format_model_list(self, raw_model_list):
|
||||
models = raw_model_list.get("data") if isinstance(raw_model_list, dict) else None
|
||||
if not isinstance(models, list):
|
||||
return []
|
||||
|
||||
model_list = []
|
||||
for model in models:
|
||||
if not isinstance(model, dict) or not model.get("id"):
|
||||
continue
|
||||
model_list.append(
|
||||
{
|
||||
"name": model["id"],
|
||||
"model_types": [LLMType.ASR.value],
|
||||
"features": [],
|
||||
"max_tokens": 8192,
|
||||
}
|
||||
)
|
||||
return model_list
|
||||
|
||||
|
||||
class VLLM(OpenAIAPICompatible):
|
||||
_FACTORY_NAME = "VLLM"
|
||||
|
||||
|
||||
@@ -408,3 +408,16 @@ class NewAPISeq2txt(GPTSeq2txt):
|
||||
raise ValueError("url cannot be None")
|
||||
model_name = model_name.split("___")[0]
|
||||
super().__init__(key, model_name=model_name, base_url=base_url, **kwargs)
|
||||
|
||||
|
||||
class FunASRSeq2txt(GPTSeq2txt):
|
||||
"""FunASR speech-to-text provider for its OpenAI-compatible API."""
|
||||
|
||||
_FACTORY_NAME = "FunASR"
|
||||
|
||||
def __init__(self, key, model_name="sensevoice", base_url="http://localhost:8000/v1", **kwargs):
|
||||
"""Initialize a client for a FunASR OpenAI-compatible endpoint."""
|
||||
if not base_url:
|
||||
base_url = "http://localhost:8000/v1"
|
||||
super().__init__(key=key or "funasr", model_name=model_name, base_url=base_url, **kwargs)
|
||||
logging.info("[FunASR] Speech2Text initialized with model %s at %s", model_name, self.base_url)
|
||||
|
||||
Reference in New Issue
Block a user