feat: add Mistral OCR (/v1/ocr) as a document parser; fix "Can't find model" mis-tag (#5782, #7075) (#17057)

### What problem does this PR solve?

Adds first-class support for **Mistral OCR** (`POST /v1/ocr`) as a
document parser, and fixes the long-standing bug where selecting
`mistral-ocr-latest` fails with `Can't find model for
<tenant>/image2text/mistral-ocr-latest`.

`mistral-ocr-latest` is Mistral's dedicated document-OCR endpoint, not a
vision-chat (`image2text`) model, but the catalog tagged it `image2text`
— so it resolved to the `CvModel` registry, which has no `Mistral`
entry, and there was no `OcrModel` entry either. This PR registers it
correctly and wires it end to end.

Closes #17056
Closes #5782
Closes #7075

**What it does**

1. **`MistralParser` + `MistralOcrModel`**
(`deepdoc/parser/mistral_parser.py`, `rag/llm/ocr_model.py`) — a proper
`OcrModel` factory `Mistral OCR`, mirroring the SoMark cloud-OCR
template. Tables stay inline as HTML; the page range maps to Mistral's
native `pages` selector (absolute page indices, billed per selected
page, so multi-task documents do not re-OCR the whole file); documents
over the inline limit go through the `/v1/files` signed-URL flow with
cleanup.
2. **Removes the `image2text` mis-tag** for `mistral-ocr-latest` from
the `Mistral` factory in `conf/llm_factories.json` (it now lives only in
the `Mistral OCR` factory, typed `ocr`). This is what closes the `Can't
find model` path.
3. **`MistralCV`** (`rag/llm/cv_model.py`) — a thin `GptV4` subclass
over Mistral's OpenAI-compatible endpoint, registering a `Mistral` entry
in the `CvModel` registry so Mistral vision models (`pixtral-*`) become
usable as `image2text` at all.
4. **Figure description** — Mistral-OCR-extracted figures are captioned
using the tenant's configured `image2text` model (any provider),
matching MinerU/deepdoc behaviour.
5. **Wires the parser into every chunking method** (`naive`, `paper`,
`book`, `laws`, `manual`, `one`, `presentation`) and the `rag/flow` DAG
path. This also fixes a related latent gap where those chunkers
forwarded only `mineru_llm_name`, so any model-based OCR provider
selected on a non-`naive` method silently fell through.

**Notes on the API contract** (verified against the live Mistral API):
`pages` is a selector (returns absolute `index`, bills only the
requested pages); `include_blocks: true` returns per-block bounding
boxes usable for chunk highlighting and figure cropping; large files use
`POST /v1/files` → signed URL → OCR → `DELETE`.

**Testing**: new unit tests cover the response→sections contract (both
the 2-tuple `naive` path and the typed 3-tuple DAG path), the
position-tag rescale, the HTTP client incl. upload failure/cleanup
paths, `parse_pdf` page-range threading, registry registration, env
config, the suffix normalization, the factory catalog entry, `MistralCV`
registration, and figure-description injection. Verified end to end
against the live Mistral API on real PDFs (table extraction,
page-selector cost avoidance, figure captioning).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
Xavierando
2026-07-24 15:07:48 +02:00
committed by GitHub
parent bbd0dc5463
commit 08332501a8
22 changed files with 1412 additions and 13 deletions

View File

@@ -0,0 +1,24 @@
def test_env_keys_and_defaults_present():
from common.constants import MISTRAL_OCR_ENV_KEYS, MISTRAL_OCR_DEFAULT_CONFIG
assert "MISTRAL_OCR_API_KEY" in MISTRAL_OCR_ENV_KEYS
assert "MISTRAL_OCR_BASE_URL" in MISTRAL_OCR_ENV_KEYS
assert MISTRAL_OCR_DEFAULT_CONFIG["MISTRAL_OCR_BASE_URL"] == "https://api.mistral.ai/v1"
def test_collect_env_config_returns_none_without_env(monkeypatch):
from common.constants import MISTRAL_OCR_ENV_KEYS, MISTRAL_OCR_DEFAULT_CONFIG
from api.db.joint_services.tenant_model_service import _collect_env_config
for k in MISTRAL_OCR_ENV_KEYS:
monkeypatch.delenv(k, raising=False)
assert _collect_env_config(MISTRAL_OCR_ENV_KEYS, MISTRAL_OCR_DEFAULT_CONFIG) is None
def test_collect_env_config_populated_when_key_set(monkeypatch):
from common.constants import MISTRAL_OCR_ENV_KEYS, MISTRAL_OCR_DEFAULT_CONFIG
from api.db.joint_services.tenant_model_service import _collect_env_config
monkeypatch.setenv("MISTRAL_OCR_API_KEY", "sk-live")
cfg = _collect_env_config(MISTRAL_OCR_ENV_KEYS, MISTRAL_OCR_DEFAULT_CONFIG)
assert cfg["MISTRAL_OCR_API_KEY"] == "sk-live"

View File

@@ -0,0 +1,25 @@
import json
from pathlib import Path
def test_mistral_ocr_factory_present_and_ocr_tagged():
repo_root = Path(__file__).resolve().parents[3]
data = json.loads((repo_root / "conf" / "llm_factories.json").read_text())
factories = {f["name"]: f for f in data["factory_llm_infos"]}
assert "Mistral OCR" in factories, "Mistral OCR factory missing"
fac = factories["Mistral OCR"]
assert "OCR" in fac["tags"]
# ships a default OCR model (like the other OCR factories) so it is usable
# without manually adding one through the model provider page; the llm_name
# is the real Mistral API id because the parser POSTs it verbatim to /v1/ocr.
models = {m["llm_name"]: m for m in fac["llm"]}
assert "mistral-ocr-latest" in models
assert models["mistral-ocr-latest"]["model_type"] == "ocr"
assert "OCR" in models["mistral-ocr-latest"]["tags"]
def test_mistral_ocr_factory_distinct_from_mistral():
repo_root = Path(__file__).resolve().parents[3]
data = json.loads((repo_root / "conf" / "llm_factories.json").read_text())
names = [f["name"] for f in data["factory_llm_infos"]]
assert "Mistral" in names and "Mistral OCR" in names

View File

@@ -0,0 +1,23 @@
from common.parser_config_utils import normalize_layout_recognizer
def test_mistral_ocr_suffix_normalized():
raw = "mistral-ocr-latest@inst@Mistral OCR"
layout, model_name = normalize_layout_recognizer(raw)
assert layout == "Mistral OCR"
assert model_name == raw
def test_mistral_ocr_suffix_case_insensitive():
raw = "mistral-ocr-latest@inst@MISTRAL OCR"
layout, model_name = normalize_layout_recognizer(raw)
assert layout == "Mistral OCR"
assert model_name == raw
def test_plain_mistral_not_captured():
# a pixtral vision model on the multi-type Mistral factory must NOT be
# dirverted to OCR
layout, model_name = normalize_layout_recognizer("pixtral-large-latest@inst@Mistral")
assert layout == "pixtral-large-latest@inst@Mistral"
assert model_name is None