mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-04 06:40:29 +08:00
06e36d24f4
### Summary Adds FunASR as a self-hosted speech-to-text provider through its OpenAI-compatible `/v1/audio/transcriptions` endpoint. This is a focused replacement for #15526 by @Rene0422 and relates to #15448. The unrelated Markdown parser changes from the previous branch are intentionally removed so this PR contains only the FunASR provider integration. - register FunASR as a `SPEECH2TEXT` factory; - add `FunASRSeq2txt` with `sensevoice` and `http://localhost:8000/v1` defaults, an optional API key, URL normalization, and inherited transcription handling; - wire FunASR into the current local-provider schema with a prefilled local URL and official documentation link; - discover the server's `/v1/models` dynamically and expose every returned model as speech-to-text in the model picker; - use RAGFlow's existing default provider icon fallback instead of referencing a missing `funasr` asset; - list FunASR in the supported-provider documentation; - add focused backend and frontend regression tests. ### Validation - focused backend pytest suite -> `7 passed` - real CPU `funasr-server` + RAGFlow provider smoke test -> discovered `fun-asr-nano`, `sensevoice`, and `paraformer`; transcribed a real WAV as `我现在在录一段测试音频` (`10` tokens, `0.504s`) - `ruff check` and `ruff format --check` on the changed Python files - `python3 -m py_compile` on the provider and its test - JSON parse and a semantic assertion for exactly one enabled FunASR `SPEECH2TEXT` factory - focused frontend Jest test -> `2 passed` - ESLint and Prettier on all changed TypeScript files - `npm run build` -> production build succeeded (`14,181` modules transformed) - `git diff --check` ### Deployment Run FunASR separately and point the RAGFlow provider at it: ```bash pip install funasr funasr-server --device cuda --model sensevoice ``` The API key remains optional because the stock local server does not require authentication. A key can still be supplied when the endpoint is protected by a gateway. --------- Signed-off-by: LauraGPT <LauraGPT@users.noreply.github.com> Co-authored-by: LauraGPT <LauraGPT@users.noreply.github.com>
69 lines
2.2 KiB
TypeScript
69 lines
2.2 KiB
TypeScript
/*
|
|
* Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
|
|
*
|
|
* Licensed under the Apache License, Version 2.0 (the "License");
|
|
* you may not use this file except in compliance with the License.
|
|
* You may obtain a copy of the License at
|
|
*
|
|
* http://www.apache.org/licenses/LICENSE-2.0
|
|
*
|
|
* Unless required by applicable law or agreed to in writing, software
|
|
* distributed under the License is distributed on an "AS IS" BASIS,
|
|
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
* See the License for the specific language governing permissions and
|
|
* limitations under the License.
|
|
*/
|
|
|
|
import { LLMFactory } from '@/constants/llm';
|
|
|
|
/**
|
|
* Provider factories that opt into the "List Models" picker UI.
|
|
*
|
|
* For these factories, the modal hides the traditional model_name,
|
|
* model_type, max_tokens, and is_tools form fields and instead shows a
|
|
* "List Models" button that fetches available models from the provider's
|
|
* `/providers/<factory>/models` endpoint. The user can multi-select models
|
|
* from the response; each selected model is converted to an `IModelInfo`
|
|
* entry and submitted as `model_info`.
|
|
*
|
|
* For all other factories the picker is hidden and the form renders the
|
|
* 4 model_* fields directly.
|
|
*/
|
|
export const LIST_MODEL_PROVIDERS = new Set<string>([
|
|
LLMFactory.Ollama,
|
|
LLMFactory.OpenRouter,
|
|
LLMFactory.VLLM,
|
|
LLMFactory.OpenAiAPICompatible,
|
|
LLMFactory.LMStudio,
|
|
LLMFactory.VolcEngine,
|
|
LLMFactory.Xinference,
|
|
LLMFactory.LocalAI,
|
|
LLMFactory.FunASR,
|
|
LLMFactory.BaiduYiYan,
|
|
LLMFactory.NewAPI,
|
|
|
|
// LLMFactory.HuggingFace,
|
|
// LLMFactory.GoogleCloud,
|
|
// LLMFactory.TencentCloud,
|
|
// LLMFactory.XunFeiSpark,
|
|
// LLMFactory.GPUStack,
|
|
// LLMFactory.FishAudio,
|
|
// LLMFactory.MinerU,
|
|
// LLMFactory.PaddleOCR,
|
|
]);
|
|
|
|
/**
|
|
* The set of form-field names that are owned by the list-models picker
|
|
* (not registered in the dynamic form when the picker is active).
|
|
*
|
|
* Doubles as the whitelist of fields that remain editable in viewMode —
|
|
* in viewMode every other field is disabled so only model-related edits
|
|
* are possible.
|
|
*/
|
|
export const LIST_MODEL_FIELD_NAMES = new Set<string>([
|
|
'model_name',
|
|
'model_type',
|
|
'max_tokens',
|
|
'is_tools',
|
|
]);
|