feat(stt): add FunASR / SenseVoice provider (#16473)

### Summary

Adds FunASR as a self-hosted speech-to-text provider through its
OpenAI-compatible `/v1/audio/transcriptions` endpoint.

This is a focused replacement for #15526 by @Rene0422 and relates to
#15448. The unrelated Markdown parser changes from the previous branch
are intentionally removed so this PR contains only the FunASR provider
integration.

- register FunASR as a `SPEECH2TEXT` factory;
- add `FunASRSeq2txt` with `sensevoice` and `http://localhost:8000/v1`
defaults, an optional API key, URL normalization, and inherited
transcription handling;
- wire FunASR into the current local-provider schema with a prefilled
local URL and official documentation link;
- discover the server's `/v1/models` dynamically and expose every
returned model as speech-to-text in the model picker;
- use RAGFlow's existing default provider icon fallback instead of
referencing a missing `funasr` asset;
- list FunASR in the supported-provider documentation;
- add focused backend and frontend regression tests.

### Validation

- focused backend pytest suite -> `7 passed`
- real CPU `funasr-server` + RAGFlow provider smoke test -> discovered
`fun-asr-nano`, `sensevoice`, and `paraformer`; transcribed a real WAV
as `我现在在录一段测试音频` (`10` tokens, `0.504s`)
- `ruff check` and `ruff format --check` on the changed Python files
- `python3 -m py_compile` on the provider and its test
- JSON parse and a semantic assertion for exactly one enabled FunASR
`SPEECH2TEXT` factory
- focused frontend Jest test -> `2 passed`
- ESLint and Prettier on all changed TypeScript files
- `npm run build` -> production build succeeded (`14,181` modules
transformed)
- `git diff --check`

### Deployment

Run FunASR separately and point the RAGFlow provider at it:

```bash
pip install funasr
funasr-server --device cuda --model sensevoice
```

The API key remains optional because the stock local server does not
require authentication. A key can still be supplied when the endpoint is
protected by a gateway.

---------

Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com>
Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
This commit is contained in:
zhifu gao
2026-07-15 19:02:05 +08:00
committed by GitHub
parent 2223a514de
commit 06e36d24f4
11 changed files with 233 additions and 0 deletions

View File

@@ -458,6 +458,29 @@ class OpenAIAPICompatible(Base):
return model_list
class FunASR(Base):
_FACTORY_NAME = "FunASR"
def _format_model_list(self, raw_model_list):
models = raw_model_list.get("data") if isinstance(raw_model_list, dict) else None
if not isinstance(models, list):
return []
model_list = []
for model in models:
if not isinstance(model, dict) or not model.get("id"):
continue
model_list.append(
{
"name": model["id"],
"model_types": [LLMType.ASR.value],
"features": [],
"max_tokens": 8192,
}
)
return model_list
class VLLM(OpenAIAPICompatible):
_FACTORY_NAME = "VLLM"

View File

@@ -408,3 +408,16 @@ class NewAPISeq2txt(GPTSeq2txt):
raise ValueError("url cannot be None")
model_name = model_name.split("___")[0]
super().__init__(key, model_name=model_name, base_url=base_url, **kwargs)
class FunASRSeq2txt(GPTSeq2txt):
"""FunASR speech-to-text provider for its OpenAI-compatible API."""
_FACTORY_NAME = "FunASR"
def __init__(self, key, model_name="sensevoice", base_url="http://localhost:8000/v1", **kwargs):
"""Initialize a client for a FunASR OpenAI-compatible endpoint."""
if not base_url:
base_url = "http://localhost:8000/v1"
super().__init__(key=key or "funasr", model_name=model_name, base_url=base_url, **kwargs)
logging.info("[FunASR] Speech2Text initialized with model %s at %s", model_name, self.base_url)