Commit Graph

833 Commits

Author SHA1 Message Date
Wang Qi
4592d06a67 Fix deepcopy the chat model (#17560) 2026-07-30 14:29:24 +08:00
Wang Qi
6e0e495929 Fix: upload document might be hung the whole ragflow (#17537) 2026-07-30 13:54:07 +08:00
Kevin Hu
9bb037271e Refact: Rename "artifact_" to "wiki_". (#17553) 2026-07-30 11:06:16 +08:00
Kevin Hu
48a3280eac Refactor: merge dataset scope graph. (#17526)
### Summary

merge dataset scope graph.

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-07-29 18:23:51 +08:00
balibabu
1140db3b8c Fix: The multi-model comparison page cannot output messages. (#17507) 2026-07-29 15:16:40 +08:00
Wang Qi
3297b5c756 Fix: drop and add index for mysql (#17513)
Drop complile template not used index
2026-07-29 14:54:20 +08:00
Sbaaoui Idriss
e9637c9f94 fix: list model function for nvidia on python/go not returning current models (#17501)
### Summary

fix the list model logic for nvidia models
2026-07-29 13:15:34 +08:00
Lynn
8d1eff6ea2 Fix: resolve model_id to model before judge parse method (#17493) 2026-07-29 09:34:52 +08:00
Jin Hai
7f21a7ba18 Go: add context, part14 (#17446)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-28 19:05:59 +08:00
Wang Qi
619b58faef Add {date} replacement to chat (#17430) 2026-07-28 09:49:17 +08:00
buua436
6e1e540f98 fix: persist pipeline tree graph rows (#17400) 2026-07-27 16:40:29 +08:00
euvre
6cc862fc00 Fix: agent list owner filter not applied (#17410) 2026-07-27 13:58:43 +08:00
Wang Qi
53afc32349 Fix get datasets owner retrieve the whole dataset (#17370) 2026-07-27 10:24:13 +08:00
Xavierando
08332501a8 feat: add Mistral OCR (/v1/ocr) as a document parser; fix "Can't find model" mis-tag (#5782, #7075) (#17057)
### What problem does this PR solve?

Adds first-class support for **Mistral OCR** (`POST /v1/ocr`) as a
document parser, and fixes the long-standing bug where selecting
`mistral-ocr-latest` fails with `Can't find model for
<tenant>/image2text/mistral-ocr-latest`.

`mistral-ocr-latest` is Mistral's dedicated document-OCR endpoint, not a
vision-chat (`image2text`) model, but the catalog tagged it `image2text`
— so it resolved to the `CvModel` registry, which has no `Mistral`
entry, and there was no `OcrModel` entry either. This PR registers it
correctly and wires it end to end.

Closes #17056
Closes #5782
Closes #7075

**What it does**

1. **`MistralParser` + `MistralOcrModel`**
(`deepdoc/parser/mistral_parser.py`, `rag/llm/ocr_model.py`) — a proper
`OcrModel` factory `Mistral OCR`, mirroring the SoMark cloud-OCR
template. Tables stay inline as HTML; the page range maps to Mistral's
native `pages` selector (absolute page indices, billed per selected
page, so multi-task documents do not re-OCR the whole file); documents
over the inline limit go through the `/v1/files` signed-URL flow with
cleanup.
2. **Removes the `image2text` mis-tag** for `mistral-ocr-latest` from
the `Mistral` factory in `conf/llm_factories.json` (it now lives only in
the `Mistral OCR` factory, typed `ocr`). This is what closes the `Can't
find model` path.
3. **`MistralCV`** (`rag/llm/cv_model.py`) — a thin `GptV4` subclass
over Mistral's OpenAI-compatible endpoint, registering a `Mistral` entry
in the `CvModel` registry so Mistral vision models (`pixtral-*`) become
usable as `image2text` at all.
4. **Figure description** — Mistral-OCR-extracted figures are captioned
using the tenant's configured `image2text` model (any provider),
matching MinerU/deepdoc behaviour.
5. **Wires the parser into every chunking method** (`naive`, `paper`,
`book`, `laws`, `manual`, `one`, `presentation`) and the `rag/flow` DAG
path. This also fixes a related latent gap where those chunkers
forwarded only `mineru_llm_name`, so any model-based OCR provider
selected on a non-`naive` method silently fell through.

**Notes on the API contract** (verified against the live Mistral API):
`pages` is a selector (returns absolute `index`, bills only the
requested pages); `include_blocks: true` returns per-block bounding
boxes usable for chunk highlighting and figure cropping; large files use
`POST /v1/files` → signed URL → OCR → `DELETE`.

**Testing**: new unit tests cover the response→sections contract (both
the 2-tuple `naive` path and the typed 3-tuple DAG path), the
position-tag rescale, the HTTP client incl. upload failure/cleanup
paths, `parse_pdf` page-range threading, registry registration, env
config, the suffix normalization, the factory catalog entry, `MistralCV`
registration, and figure-description injection. Verified end to end
against the live Mistral API on real PDFs (table extraction,
page-selector cost avoidance, figure captioning).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-07-24 21:07:48 +08:00
Wang Qi
7180d3024e Add cancel as final state (#918) (#17352) 2026-07-24 18:58:04 +08:00
buua436
2ba7ccecaf fix: stabilize knowledge compilation navigation updates (#17345) 2026-07-24 17:48:27 +08:00
Lynn
ff163764b3 Fix: tencent model in python conf (#17337) 2026-07-24 12:21:42 +08:00
Wang Qi
1b07bf286d Only record and show the log in final state (#908) (#17302) 2026-07-24 10:08:44 +08:00
Kevin Hu
3e4c6dfc0a Feat: refine the tree navigation during compilations (#17140) 2026-07-23 18:39:16 +08:00
buua436
d0a48816bb fix: improve page index descriptions (#17268) 2026-07-23 10:34:45 +08:00
Wang Qi
178b40cb25 Fix SharePoint connector not run after added (#17058) 2026-07-21 10:06:01 +08:00
Jin Hai
7c698f8e4b Python: remove unused index (#17008)
### Summary

Remove below indexes.
```
idx_tenant_langfuse_secret_key
idx_tenant_langfuse_public_key
idx_tenant_langfuse_host
```

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-16 23:00:44 +08:00
buua436
bd485dbdfe fix: scope compilation template names by group (#16998) 2026-07-16 19:37:18 +08:00
Jin Hai
d2e5e480a1 Fix database data types (#16965)
### Summary

Fix two data types definition.

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-16 11:44:00 +08:00
Kevin Hu
454dea686e Feat: agentic search framework (#16859)
### Summary

Agentic search

<img width="1149" height="1575" alt="image"
src="https://github.com/user-attachments/assets/bce9a3e7-0517-4fb2-80a2-5d2a81a4da78"
/>

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-07-15 23:46:23 +08:00
Jin Hai
2a6e210020 Go: align db schema to python (#16935)
As title.

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-07-15 23:05:06 +08:00
Wang Qi
2223a514de Fix cancel ingest task, it will be stilling running and show internal server error (#836) (#16945) 2026-07-15 18:33:51 +08:00
buua436
09abe5f513 fix: ensure database model indexes (#16860) 2026-07-13 17:34:28 +08:00
euvre
3bfad1f00e fix: correct model type mappings and improve system setting persistence (#16501) 2026-07-13 16:42:55 +08:00
buua436
d291e4641d fix: improve pipeline compilation, template updates, and document resets (#16815)
### What problem does this PR solve?

- Clear stale pipeline IDs and generated data when updating documents
without `pipeline_id`.
- Support tree compilation results in pipeline workflows.
- Update compilation templates in place while preserving existing
template IDs.
- Improve duplicate-template validation messages.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-07-10 21:17:19 +08:00
buua436
74bbbba3e0 fix: align model default handling (#16782) 2026-07-10 10:34:19 +08:00
Lynn
cc94639555 Fix: get_by_id (#16765) 2026-07-09 14:52:41 +08:00
buua436
6a77523bf0 refa: resolve tenant model refs consistently (#16744) 2026-07-09 14:02:08 +08:00
qinling0210
ae96e636e9 Handle searching dataset without embedding model (#16742)
### Summary

Handle searching dataset without embedding model

In this PR, Searching datasets with different embedding models or
searching dataset with/without embedding models are not allowed. We will
improve the behavior later.
2026-07-09 11:38:55 +08:00
Lynn
1430d0e431 Fix: provider name (#16733) 2026-07-09 10:19:10 +08:00
euvre
74d3508a37 Fix: prevent auto-incrementing memory name suffix when only permissions change (#16750) 2026-07-08 20:12:45 +08:00
Lynn
0ae5961e1c Feat: v0.27.0 model provider (#16604) 2026-07-08 09:47:29 +08:00
euvre
41801ad2b8 fix: prevent memory name from auto-appending (1) on description update (#16714) 2026-07-07 19:34:51 +08:00
Öndery
28a41ed070 fix(task_executor): fix Langfuse flush/shutdown deadlock that freezes document parsing (#16502) 2026-07-07 19:06:30 +08:00
S
f477d3329d Fix: ValueError: too many values to unpack in list_tenant_added_models for model IDs containing '@' (#16467) (#16468) 2026-07-07 09:40:27 +08:00
Kevin Hu
52f985f43e Refactor: Remove redundant functions. (#16671)
### Summary

Remove redundant functions.
2026-07-06 19:02:25 +08:00
Wang Qi
3a247dbb3c Fix filter to use Chinese (#16673) 2026-07-06 18:20:42 +08:00
Wang Qi
a0e65637eb Delete canvas_app.py and evaluation_service.py (#16614)
Follow on PR #13295
2026-07-03 21:03:54 +08:00
Kevin Hu
cf634b92b4 Feat: Put some wiki templates. (#16617)
### Summary

Add a few of wiki templates.
2026-07-03 20:52:27 +08:00
Yingfeng
706fa4e87a Feat: add gbrain compile template for session/memory data (#16613) 2026-07-03 18:22:29 +08:00
Wang Qi
6a4b9be426 Refactor: reformat all code for lefthook using ruff and gofmt (#16585) 2026-07-03 12:53:39 +08:00
euvre
918229613a fix: prevent duplicate 'skills' and '.knowledgebase' folders caused by race conditions (#16568) 2026-07-03 12:06:45 +08:00
Kevin Hu
62f94cd59b Feat: Add knowledge compilation workflows (#16515)
## Summary
- Add knowledge compilation template APIs, services, and builtin
template seed data
- Add advanced knowledge compile structure/artifact/RAPTOR workflow
support
- Update parsing, dataset/document APIs, and supporting services for
compilation workflows
2026-07-02 23:22:07 +08:00
Wang Qi
93f6d647d4 Fix the sandbox exec image cannot show and download (#16577) 2026-07-02 20:49:51 +08:00
Lynn
bc54903bf6 Fix: display model_id in memory_list (#16567) 2026-07-02 20:28:27 +08:00