### Summary
The **aimlapi.com** provider added in #17311 does not identify itself on
any of its outgoing requests, so its traffic cannot be attributed to the
integration. This PR adds the two headers AIMLAPI expects —
`X-AIMLAPI-Source` and `X-AIMLAPI-Partner-ID` — to every request the
provider makes.
## Summary
GreenPT is a European AI provider with an OpenAI-compatible API,
optimized infrastructure, and datacenters powered by 100% renewable
energy.
This adds native GreenPT support across RAGFlow’s Go-first provider
system and its Python compatibility layer:
- discovers the current catalog from `GET /v1/models`
- features `glm-5.2` and `kimi-k2.7-code` for chat and coding
- supports `green-embedding` through `/v1/embeddings`
- supports `green-rerank` through `/v1/rerank`
- supports `green-s` and `green-s-pro` speech-to-text through
`/v1/listen`
- adds provider configuration, UI icon, and supported-provider
documentation
…rphaned clusters
### Summary
**Problem**
When deleting a document from the knowledge base, the parent cluster
could never be located and was therefore never cleaned up, leaving
orphaned clusters in Infinity indefinitely.
**Root Cause**
InfinityConnection.field_keyword() treats all _kwd-suffixed columns as
keyword lists and applies .split("###") to them in get_fields().
parent_kwd was missing from the exclusion list. This caused a parent
cluster name like "Toronto Transit & Authority Control 5e8d5eb6" to
become ['Toronto Transit & Authority Control 5e8d5eb6'] (a
single-element list). When that list was fed into _nav_cluster_id(), it
produced a different hash than the actual cluster's hash (computed from
the plain string), so every deletion-time cluster lookup silently
failed.
**Fix**
Added "parent_kwd" to the exclusion list in field_keyword(), alongside
docnm_kwd, important_kwd, question_kwd, etc.
### Summary
This PR adds **aimlapi.com** as a model provider, so a RAGFlow user can
enter one API key in the model settings and use AIMLAPI's models across
the app. AIMLAPI ([aimlapi.com](https://aimlapi.com)) is an
OpenAI-compatible aggregator that serves 700+ models (LLM, embedding,
vision, TTS, ASR) from many providers behind a single API.
The change mirrors the repo's existing "add provider" pattern (e.g.
FuturMix / OpenRouter): provider logic lives in the same files those
providers use, and shared / UI files get only registration entries.
**Backend**
- `conf/llm_factories.json` — the `aimlapi.com` factory entry.
- `rag/llm/__init__.py`, `rag/llm/{chat,embedding,cv}_model.py` —
LiteLLM adapters (chat, embedding, image2text) with a production base
URL, overridable via `AIMLAPI_API_URL`.
- `rag/llm/model_meta.py` — an `AIMLAPI` model-meta so the provider
lists its full `/v1/models` catalog dynamically (classified by the
endpoint `type`), the same way OpenRouter does.
- `api/apps/restful_apis/aimlapi_api.py` — an optional "Get API key"
flow using AIMLAPI's agent-authorization (OAuth 2.0 Device Authorization
Grant, RFC 8628). The device code is kept server-side (Redis); only the
issued key reaches the browser.
**Frontend (`web/`)**
- Provider registration (constant, icon allowlist, brand logo), the
model picker (`LIST_MODEL_PROVIDERS` + a `buildLocalConfig` entry), and
the "Get API key" button in the provider dialog. Locales added to `en`
and `zh`.
**Configuration** — production defaults are compiled in; endpoints and
the partner id are overridable through `AIMLAPI_*` environment
variables, so the same build works across environments.
**Testing** — the `web` build passes; chat, embedding and dynamic model
listing were smoke-tested against the live API.
### What problem does this PR solve?
Adds first-class support for **Mistral OCR** (`POST /v1/ocr`) as a
document parser, and fixes the long-standing bug where selecting
`mistral-ocr-latest` fails with `Can't find model for
<tenant>/image2text/mistral-ocr-latest`.
`mistral-ocr-latest` is Mistral's dedicated document-OCR endpoint, not a
vision-chat (`image2text`) model, but the catalog tagged it `image2text`
— so it resolved to the `CvModel` registry, which has no `Mistral`
entry, and there was no `OcrModel` entry either. This PR registers it
correctly and wires it end to end.
Closes#17056Closes#5782Closes#7075
**What it does**
1. **`MistralParser` + `MistralOcrModel`**
(`deepdoc/parser/mistral_parser.py`, `rag/llm/ocr_model.py`) — a proper
`OcrModel` factory `Mistral OCR`, mirroring the SoMark cloud-OCR
template. Tables stay inline as HTML; the page range maps to Mistral's
native `pages` selector (absolute page indices, billed per selected
page, so multi-task documents do not re-OCR the whole file); documents
over the inline limit go through the `/v1/files` signed-URL flow with
cleanup.
2. **Removes the `image2text` mis-tag** for `mistral-ocr-latest` from
the `Mistral` factory in `conf/llm_factories.json` (it now lives only in
the `Mistral OCR` factory, typed `ocr`). This is what closes the `Can't
find model` path.
3. **`MistralCV`** (`rag/llm/cv_model.py`) — a thin `GptV4` subclass
over Mistral's OpenAI-compatible endpoint, registering a `Mistral` entry
in the `CvModel` registry so Mistral vision models (`pixtral-*`) become
usable as `image2text` at all.
4. **Figure description** — Mistral-OCR-extracted figures are captioned
using the tenant's configured `image2text` model (any provider),
matching MinerU/deepdoc behaviour.
5. **Wires the parser into every chunking method** (`naive`, `paper`,
`book`, `laws`, `manual`, `one`, `presentation`) and the `rag/flow` DAG
path. This also fixes a related latent gap where those chunkers
forwarded only `mineru_llm_name`, so any model-based OCR provider
selected on a non-`naive` method silently fell through.
**Notes on the API contract** (verified against the live Mistral API):
`pages` is a selector (returns absolute `index`, bills only the
requested pages); `include_blocks: true` returns per-block bounding
boxes usable for chunk highlighting and figure cropping; large files use
`POST /v1/files` → signed URL → OCR → `DELETE`.
**Testing**: new unit tests cover the response→sections contract (both
the 2-tuple `naive` path and the typed 3-tuple DAG path), the
position-tag rescale, the HTTP client incl. upload failure/cleanup
paths, `parse_pdf` page-range threading, registry registration, env
config, the suffix normalization, the factory catalog entry, `MistralCV`
registration, and figure-description injection. Verified end to end
against the live Mistral API on real PDFs (table extraction,
page-selector cost avoidance, figure captioning).
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve?
Fix Infinity compatibility issues in knowledge compilation.
This change:
- Stores compilation source ID lists as JSON arrays in Infinity.
- Parses JSON array fields when reading compiled documents.
- Uses `json_contains` for filtering JSON array fields.
- Adds the missing `name` column to the Infinity mapping.
- Updates dataset navigation KNN search to use the unified
`MatchDenseExpr` interface.
- Handles unavailable embeddings without querying an invalid `q_0_vec`
field.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
## Summary
Old `zhipuai==2.0.1` SDK (MetaGLM/zhipuai-sdk-python-v4) requires
`pyjwt~=2.8.0`, which blocks upgrading pyjwt past 2.8.0 to address
active CVEs. The upstream project now recommends the maintained
successor [`zai-sdk`](https://github.com/zai-org/z-ai-sdk-python).
`zai-sdk` relaxes the pyjwt constraint to `>=2.9.0,<3.0.0`, allowing
pyjwt 2.13.0+. The embedding API surface (`client.embeddings.create`) is
identical, no functional changes required.
Related CVE's:
- CVE-2026-48522
- CVE-2026-48524
- CVE-2026-48525
- CVE-2026-48526
- CVE-2026-32597
### Problem
Parsing a Q&A `.csv` can splice unrelated text into the wrong answers
(reported in #16791).
### Root cause
The `.csv` branch of `rag/app/qa.py`'s `chunk()` builds records with
`csv.reader(lines, delimiter=delimiter)` (default `quotechar='"'`), but
then indexes `lines[i]` by the reader's *record* index in `answer +=
"\n" + lines[i]`. When a line's field opens with a `"`, `csv.reader`
treats it as an unclosed quoted field and merges several physical lines
into one record. From there the record index permanently desyncs from
the physical line numbers, so `lines[i]` returns the wrong line and
unrelated Q&A content gets appended to the wrong answer.
### Reproduction (stdlib only)
```python
import csv
lines = 'Q1,A1\n"quoted answer start\ncontinues here,extra\nQ2,A2\n'.split("\n")
list(csv.reader(lines, delimiter=","))
# record 1 swallows 3 physical lines: ['quoted answer startcontinues here,extraQ2,A2']
# -> the reader index no longer matches lines[i]
list(csv.reader(lines, delimiter=",", quoting=csv.QUOTE_NONE))
# one physical line per record; index stays aligned
```
### Fix
Pass `quoting=csv.QUOTE_NONE` so one physical line maps to one record,
keeping the reader index aligned with `lines[i]` (the surrounding code
already relies on that 1:1 mapping).
Fixes#16791.
---------
Signed-off-by: Yash Raj Pandey <yashpn62@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
### What problem does this PR solve?
When a table dataset's `field_map` is missing or stale,
`aggregate_table_doc_metadata` falls back to probing chunk dictionaries
for each column's Elasticsearch field key. It currently performs that
probe only once, against the first dictionary chunk, and caches `(None,
"none")` if the field is absent there.
Sparse table rows commonly omit empty columns. If the first row has no
`notes` field but a later row contains `notes_raw`, the cached miss
causes every later row to be skipped and the document-level `notes`
metadata is silently lost. The result depends only on row order:
```python
chunks = [{}, {"notes_raw": "Handle with care"}]
aggregate_table_doc_metadata(chunks, task) # before: {}
aggregate_table_doc_metadata(list(reversed(chunks)), task)
# before: {"notes": ["Handle with care"]}
```
This was also identified in CodeRabbit's review of the merged
table-metadata implementation in #15780, but remained unfixed after that
PR merged:
https://github.com/infiniflow/ragflow/pull/15780#pullrequestreview-4448490676
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
### Fix
When the initial lookup found no key for a column, retry the existing
`_resolve_es_chunk_field_key` against the current chunk. Cache the first
successful resolution so subsequent rows retain the existing fast path.
Field-map-backed columns and columns found in the first chunk are
unchanged.
### Testing
- Added `test_aggregate_auto_mode_probes_later_sparse_chunks` with an
empty first row and a populated second row.
- Confirmed red→green: before the fix the assertion received `{}`; after
the fix it receives `{"notes": ["Handle with care"]}`.
- Full existing `test_table_metadata_aggregation.py`: **15 passed**.
- `ruff check` and `ruff format --check`: clean.
- `compileall` for both changed files: clean.
The local test environment did not contain the repository's full service
dependency set and had a corrupt pre-existing NLTK `wordnet.zip`. The
test module does not use those services or corpora, so the run stubbed
only `common.settings` engine flags, `json_repair`, and the global
conftest's NLTK resource lookup; the production module and aggregation
tests themselves ran unchanged.
### Duplicate-work check
Checked all currently open PRs (including changed file paths) and found
none touching `rag/utils/table_es_metadata.py` or its aggregation test.
The earlier #15780 review is historical context, not active competing
work.
### Disclosure
AI-assisted (Codex): the candidate came from an AI-assisted review
queue. I independently reproduced the order-dependent data loss against
the real module, checked the historical review and all open PR file
paths, and ran the regression plus full existing test file before
submitting.
Signed-off-by: chuenchen309 <48723787+chuenchen309@users.noreply.github.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
## What
Adds a reranker connector for the **Bedrock** factory, which previously
offered
chat/embedding/CV models but no reranker — selecting a Bedrock rerank
model
raised `Factory not in rerank model`.
## How
`BedrockRerank` calls the `bedrock-agent-runtime` Rerank API. It reuses
the same
JSON key protocol as `BedrockEmbed` (`auth_mode` / `bedrock_region` /
`bedrock_ak` / `bedrock_sk`, with `access_key_secret` / `iam_role` /
`assume_role` modes). Documents are truncated to the model window
(Cohere Rerank
v3.5 ~2k of its shared 4k window, Amazon Rerank v1 8k) on top of
Bedrock's own
internal truncation. Scores are returned in `[0, 1]`, so the shared
`Base.similarity` normalization applies unchanged.
Verified against `amazon.rerank-v1:0` and `cohere.rerank-v3-5:0` in
`eu-central-1`.
> Note: this PR adds the connector only. Bedrock rerank models can be
selected by
> adding the relevant entries to `conf/llm_factories.json` under the
Bedrock
> provider; that catalog change is intentionally left out of this PR.
## Tests
`test/unit_test/rag/llm/test_bedrock_rerank.py` — boto3 is mocked (no
AWS call):
score-by-index mapping, per-model document truncation, model ARN
construction,
auth-mode validation and the empty-input short-circuit. `pytest` green
alongside
the existing reranker normalization suite.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
