Commit Graph

14 Commits

Author SHA1 Message Date
Jin Hai
405275935f Go: fix unused check (#17922)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-08-06 14:45:10 +08:00
jay77721
2e2d12d262 feat: add batch_size to all embedding model configs (#17877)
## Summary

Add a `batch_size` field to every embedding model entry in
`conf/models/*.json`. The field represents the maximum number of text
inputs that can be submitted to the embedding API in a single request.

**75 embedding models across 30 config files** now carry a `batch_size`.
Values were verified against each provider's official documentation (see
the verification table at `Desktop/embedding_models_verified.md`).

## Distribution

| batch_size | # models | Provider / Model |
|---|---|---|
| 1 | 2 | AWS Bedrock `amazon.titan-embed-text-v1/v2:0` — Bedrock
`invoke` accepts a single input per call |
| 10 | 3 | Aliyun `text-embedding-v3/v4`, Volcengine
`doubao-embedding-vision-251215` |
| 16 | 5 | BaiChuan `Baichuan-Text-Embedding`, Baidu Qianfan
`embedding-v1`, Mistral `mistral-embed`, Replicate (x2) |
| 32 | 10 | NVIDIA NIM (x3), SILICONFLOW (x2), PPIO (x3), GiteeAI
`bge-m3`, HuaweiCloud `bge-m3` |
| 50 | 4 | Tencent Hunyuan `kinfra` embeddings (x4) — `InputList.N` max
50 |
| 96 | 8 | Cohere embed-v3/v4 (x5), Bedrock Cohere (x3) |
| 100 | 3 | Google Gemini `text-embedding-004`, Upstage (x2) |
| 512 | 5 | Zhipu GLM `embedding-2/3` (x2), Perplexity `pplx-embed`
(x2), Astraflow `text-embedding-3-large` |
| 1000 | 9 | Voyage AI (x9) — API reference max |
| 1024 | 1 | DeepInfra `Qwen/Qwen3-Embedding-4B` |
| 2048 | 15 | OpenAI (x3) + OpenAI-API-compatible proxies (CometAPI,
n1n, Jiekou.AI, GreenPT, TogetherAI, NovitaAI) — OpenAI contract limit |
| 16384 | 10 | Jina (x8), 302.AI, GiteeAI `jina-clip-v2` — no documented
Jina batch limit, safe high cap |

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-06 10:47:38 +08:00
jay77721
47ecc2a231 feat: populate content_length and max_output from official docs (#17836)
## Summary

- Verify and populate `content_length` (context window) and `max_output`
(max generation tokens) for all **478 chat/vision models** across **47
provider configs**
- Data sourced from **official API documentation** via 12 parallel
agents + targeted web verification
- Update Go test assertions to match verified values

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-08-05 15:22:57 +08:00
euvre
b4c2431a8b fix: implement Tongyi-Qianwen TTS via DashScope OpenAI-compatible endpoint (#17770) 2026-08-04 16:39:31 +08:00
Haruko386
a75e733b39 fix: qwen can not get model list (#17709)
### Summary

As title
2026-08-03 18:08:16 +08:00
euvre
b095a5511a fix: make qwen3-vl-plus usable as chat/vision model in Go mode (#17254) 2026-07-23 11:14:08 +08:00
Hz_
e87008e26c fix(go-agent): support Aliyun tool calls (#17099)
## Summary

  - Enable synchronous and streaming tool calls for Aliyun models.
- Preserve provider-specific chat endpoints and prevent repeated
qwen-flash tool calls.
  - Restrict retrieval tool inputs to model-provided query parameters.

  ## Testing

- `bash build.sh --test ./internal/entity/models
./internal/agent/component ./internal/agent/tool`
  - Manual frontend UI testing passed.
2026-07-20 15:54:55 +08:00
Haruko386
166ed3c159 Json: add some aliyun models (#16926)
### Summary

As title
2026-07-15 15:42:54 +08:00
Lynn
1430d0e431 Fix: provider name (#16733) 2026-07-09 10:19:10 +08:00
Jin Hai
83d09b16ce Fix Go: list providers order issue. (#16616)
### Summary

As title.

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-03 18:27:32 +08:00
Joseff
2ad854c586 Go: implement Rerank in Aliyun driver (#14676)
### What problem does this PR solve?

The Aliyun Go driver has a stub `Rerank` method that always returns
`"Aliyun, Rerank not implemented"`. DashScope exposes an
OpenAI-compatible rerank endpoint (`compatible-mode/v1/rerank`) and
hosts dedicated bilingual rerankers (`gte-rerank-v2`, `gte-rerank`) that
are a natural pairing with the embedding models already in
`aliyun.json`. Without this, Aliyun users cannot use reranking within
RAGFlow.

Closes #14675

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 20:21:04 +08:00
Panda Dev
a82ae4a991 Go: implement Encode (embeddings) in Aliyun driver (#14647)
### What problem does this PR solve?

The Aliyun Go driver shipped with a stub \`Encode\` method that returned
\`no such method\`, even though \`conf/models/aliyun.json\` already
wires the OpenAI-compatible embeddings URL suffix at
\`compatible-mode/v1/embeddings\`. The same config also did not list any
embedding models, so the picker had nothing to select.

So an Aliyun tenant who wanted to use Tongyi text-embedding-v3 or v4 in
the Go layer could not, even though the upstream endpoint is public and
uses the standard \`POST /v1/embeddings\` shape that the SiliconFlow and
ZhipuAI
drivers already support.

This PR fills the gap.

### What this PR includes

- \`conf/models/aliyun.json\`: add \`text-embedding-v4\` and
\`text-embedding-v3\` to the \`models\` array.
- \`internal/entity/models/aliyun.go\`: replace the \`Encode\` stub with
a real implementation. Adds a small local response type that matches the
OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, validate the model name,
resolve the region with a default fallback, build the
  URL from \`BaseURL[region] + URLSuffix.Embedding\`.
- Send all input texts in one request as the \`input\` array, the same
OpenAI-compatible shape the SiliconFlow \`Encode\`
  uses.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\` so the output order matches the input order
even if the API returns items in a different order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks every input slot got a vector and returns a clear
error if any slot is still nil.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`AliyunModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Encode implementation.

Closes #14646

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 13:58:25 +08:00
Jin Hai
f670913bb4 Refactor model type to model class (#14426)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 16:05:15 +08:00
Jin Hai
c3eac4103a Go: aliyun model provider (#14379)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-27 14:53:33 +08:00