jay77721
2e2d12d262
feat: add batch_size to all embedding model configs ( #17877 )
...
## Summary
Add a `batch_size` field to every embedding model entry in
`conf/models/*.json`. The field represents the maximum number of text
inputs that can be submitted to the embedding API in a single request.
**75 embedding models across 30 config files** now carry a `batch_size`.
Values were verified against each provider's official documentation (see
the verification table at `Desktop/embedding_models_verified.md`).
## Distribution
| batch_size | # models | Provider / Model |
|---|---|---|
| 1 | 2 | AWS Bedrock `amazon.titan-embed-text-v1/v2:0` — Bedrock
`invoke` accepts a single input per call |
| 10 | 3 | Aliyun `text-embedding-v3/v4`, Volcengine
`doubao-embedding-vision-251215` |
| 16 | 5 | BaiChuan `Baichuan-Text-Embedding`, Baidu Qianfan
`embedding-v1`, Mistral `mistral-embed`, Replicate (x2) |
| 32 | 10 | NVIDIA NIM (x3), SILICONFLOW (x2), PPIO (x3), GiteeAI
`bge-m3`, HuaweiCloud `bge-m3` |
| 50 | 4 | Tencent Hunyuan `kinfra` embeddings (x4) — `InputList.N` max
50 |
| 96 | 8 | Cohere embed-v3/v4 (x5), Bedrock Cohere (x3) |
| 100 | 3 | Google Gemini `text-embedding-004`, Upstage (x2) |
| 512 | 5 | Zhipu GLM `embedding-2/3` (x2), Perplexity `pplx-embed`
(x2), Astraflow `text-embedding-3-large` |
| 1000 | 9 | Voyage AI (x9) — API reference max |
| 1024 | 1 | DeepInfra `Qwen/Qwen3-Embedding-4B` |
| 2048 | 15 | OpenAI (x3) + OpenAI-API-compatible proxies (CometAPI,
n1n, Jiekou.AI, GreenPT, TogetherAI, NovitaAI) — OpenAI contract limit |
| 16384 | 10 | Jina (x8), 302.AI, GiteeAI `jina-clip-v2` — no documented
Jina batch limit, safe high cap |
---------
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com >
2026-08-06 10:47:38 +08:00
Haruko386
3f02ca7ba1
Go: implement embed, rerank, tts for AstraFlow ( #15135 )
...
### What problem does this PR solve?
implement embed, rerank, tts for AstraFlow
**Verify from CLI**
```
# Astraflow
RAGFlow(user)> tts with 'IndexTeam/IndexTTS-2@test3@astraflow' text 'hello? show yourself' play format 'wav' param '{"voice": "jack_cheng"}'
SUCCESS
RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'bge-reranker-v2-m3@test3@astraflow' top 3;
+-------+---------------------+
| index | relevance_score |
+-------+---------------------+
| 0 | 0.9837390184402466 |
| 2 | 0.06322699040174484 |
| 1 | 0.04663187265396118 |
+-------+---------------------+
RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'text-embedding-3-large@test3@astraflow' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 3072 | 0 |
| 3072 | 1 |
+-----------+-------+
# Xinference
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-22 18:02:01 +08:00
dripsmvcp
d5ba14a128
feat(go): implement provider Astraflow ( #15062 ) ( #15064 )
...
- Adds an `Astraflow` Go driver so the new API server can route
Astraflow (UCloud ModelVerse) chat instances, matching the existing
Python `AstraflowChat` (`rag/llm/chat_model.py:1237`). Follows the same
SaaS-driver shape used for Avian, Novita, TogetherAI, Replicate,
DeepInfra, Upstage, and LongCat.
Closes #15062
---------
Co-authored-by: Jin Hai <haijin.chn@gmail.com >
2026-05-21 15:32:56 +08:00
