Compare commits

...

1267 Commits

Author SHA1 Message Date
Jin Hai
30d1c1dc28 Fix go compilation (#14900)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 20:05:56 +08:00
jony376
7f699d1202 Fix: enforce tenant authorization for tenant_rerank_id in retrieval flows (#14782)
### Related issues

Closes #14781 

### What problem does this PR solve?

Some retrieval endpoints accepted caller-supplied `tenant_rerank_id` and
resolved it through `get_model_config_by_id(...)`. That helper loaded
`TenantLLM` rows by global database id and returned decoded model
configuration without checking whether the model belonged to the
authenticated tenant or the dataset owner tenant.

This meant dataset access was validated, but rerank-model selection was
not. A caller who knew or could guess another tenant's
`tenant_rerank_id` could attempt retrieval with a foreign rerank model
config, creating a cross-tenant authorization gap for model usage.

This PR closes that gap by making `tenant_rerank_id` resolution
tenant-aware across the retrieval paths that accept it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Solution

- Extend `get_model_config_by_id(...)` to accept an optional
`allowed_tenant_ids` set and reject `TenantLLM` rows whose `tenant_id`
is outside that set.
- Pass the allowed tenant scope from retrieval endpoints that accept
`tenant_rerank_id`:
  - `api/apps/sdk/doc.py`
  - `api/apps/sdk/session.py`
  - `api/apps/services/dataset_api_service.py`
- Use the authenticated tenant plus dataset-owner tenant ids already
derived by each retrieval flow as the authorization boundary for rerank
model selection.
- Add focused unit coverage to assert unauthorized `tenant_rerank_id`
values are rejected and that the allowed tenant set is propagated
correctly.

### Testing

- `python -m py_compile` on:
  - `api/db/joint_services/tenant_model_service.py`
  - `api/apps/services/dataset_api_service.py`
  - `api/apps/sdk/doc.py`
  - `api/apps/sdk/session.py`
- Added unit tests in:
-
`test/testcases/test_http_api/test_file_management_within_dataset/test_doc_sdk_routes_unit.py`
-
`test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`

### Notes for reviewers

- This change is intentionally narrow: it affects only the
`tenant_rerank_id` path, not the normal `rerank_id` name-based
resolution path.
- Local lint/syntax checks passed.
- Full pytest execution could not be completed in this environment
because the local test runtime is missing `strenum`, so the route-test
files fail during collection before exercising the updated cases.

---------

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-13 19:53:08 +08:00
writinwaters
bb1a6259d3 Docs: Updated v0.25.3 release notes draft (#14899)
### What problem does this PR solve?

Draft v0.25.3 release notes

### Type of change


- [x] Documentation Update
2026-05-13 19:49:07 +08:00
Jin Hai
87516edadf Bump to infinity v0.7.0-dev7 (#14897)
### What problem does this PR solve?

Upgrade infinity

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:42:50 +08:00
writinwaters
9ed4da74b8 Docs: Draft 0.25.3 release notes (#14898)
### What problem does this PR solve?

A draft 0.25.3 release note.

### Type of change


- [x] Documentation Update

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:35:55 +08:00
tmimmanuel
0a4b733b2a Go: implement Rerank in LocalAI driver (#14813)
### What problem does this PR solve?

The LocalAI Go driver landed in #14809 and Embed landed in #14811.
`Rerank` was left as a stub that returns `"not implemented"`. This PR
fills the gap.

LocalAI exposes a public rerank endpoint at `<tenant-url>/v1/rerank`
with a Cohere-shaped request and response (`{model, query, documents,
top_n}` → `{results: [{index, relevance_score}]}`). The Python side has
had `LocalAIRerank` in `rag/llm/rerank_model.py` for a long time. Until
this PR, a tenant who wanted to use LocalAI for reranking in the Go
layer got `"not implemented"`.

### What this PR includes

- `conf/models/localai.json`: add `"rerank": "rerank"` under
`url_suffix` so the driver can build the URL from config. This matches
the `URLSuffix.Rerank` field already used by aliyun and siliconflow.
- `internal/entity/models/localai.go`: replace the `Rerank` stub with a
real implementation that POSTs to `/v1/rerank`. Adds local
request/response types `localAIRerankRequest` and
`localAIRerankResponse`.

No factory change. No interface change.

### How the implementation works

- Validate the model name and resolve the tenant-supplied base URL with
the existing `resolveBaseURL` helper.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern `ChatWithMessages`,
`ListModels`, and `Embed` already use in this file.
- Only set the `Authorization` header when a non-empty API key was
supplied. LocalAI accepts an empty key by default, so this preserves the
optional-auth contract.
- Default `top_n` to `len(documents)` when `rerankConfig.TopN == 0`,
matching the existing Aliyun and SiliconFlow rerank implementations.
- Validate every `results[].index` against `len(documents)`. If the
upstream returns an out-of-range index, fail clearly instead of silently
writing past the slice.
- An empty `documents` slice returns `&RerankResponse{}` with no HTTP
call.
- Non-200 responses propagate the upstream status line and body.

### Note on stacking

This PR builds on #14809 (LocalAI driver) and #14811 (LocalAI Embed).
Until both merge, this PR's diff on GitHub will include all three
commits. After #14809 and #14811 land on `main`, GitHub will auto-reduce
this PR to only the `Rerank` changes (one commit, ~99 line diff in
`localai.go` plus 1 line in `localai.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `LocalAIModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Aliyun Rerank
(`internal/entity/models/aliyun.go`) and SiliconFlow Rerank
(`internal/entity/models/siliconflow.go`) implementations.

Closes #14812
Depends on #14809, #14811
Tracking: #14736

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:35:19 +08:00
Liu An
3182fd0789 Docs: Update version references to v0.25.3 in READMEs and docs (#14896)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.2 to v0.25.3
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-13 18:42:42 +08:00
Wang Qi
f3b3596c29 Speed up ragflow server (#14894)
### What problem does this PR solve?

Speed up ragflow server

### Type of change

- [ ] Refactoring
2026-05-13 18:01:33 +08:00
Jin Hai
b18640d228 Go: fix OCR command (#14891)
### What problem does this PR solve?

RAGFlow(user)> ocr with 'hunyuanocr@test@gitee' file './picture.png'
+----------------------------------------------------------+
| text                                                     |
+----------------------------------------------------------+
| 生活不是等待风暴过去,而是学会在雨中翩翩起舞。

——佚名                                                       |
+----------------------------------------------------------+

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 17:29:53 +08:00
buua436
8cb2bf04fb Fix: llm add api key overridden (#14885)
### What problem does this PR solve?

llm add api key overridden

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 17:15:32 +08:00
Panda Dev
bf90c8948a Go: implement ListModels in ZhipuAI driver (#14886)
### What problem does this PR solve?

Fixes #14884

The ZhipuAI Go driver in `internal/entity/models/zhipu-ai.go` had a stub
`ListModels` method that always returned `"zhipu-ai, no such method"`.
The DeepSeek, Gitee, NVIDIA, OpenAI, SiliconFlow, and OpenRouter drivers
in the same package already implement `ListModels` against the
OpenAI-compatible `/models` endpoint, and the model picker UI relies on
it. This PR brings ZhipuAI in line with that pattern.

### Changes

- `internal/entity/models/zhipu-ai.go`: implement
`ZhipuAIModel.ListModels`.
  - Resolve region with default fallback.
- GET `${BaseURL[region]}/${URLSuffix.Models}` (resolves to
`https://open.bigmodel.cn/api/paas/v4/models` with the default region).
- Send `Authorization: Bearer <api_key>` when an API key is configured.
Omit the header when the key is empty, so an unauthenticated caller gets
a clear `401` from upstream.
- Surface non-200 responses with the upstream status line and body,
matching the other Go drivers.
- Parse the response via the package-level `DSModelList` / `DSModel`
types already used by DeepSeek, Gitee, and SiliconFlow.
- When the response includes `owned_by`, render the entry as
`id@owned_by`, matching the convention of Gitee and SiliconFlow.
- `conf/models/zhipu-ai.json`: add `"models": "models"` to `url_suffix`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-13 16:39:14 +08:00
chanx
bbb0798c41 Fix: Set embedded models during form initialization. (#14889)
### What problem does this PR solve?

Fix: Set embedded models during form initialization.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 16:34:25 +08:00
tmimmanuel
8b53960819 Go: implement provider: LongCat (#14809)
### What problem does this PR solve?

Add a Go driver for LongCat (Meituan, https://longcat.chat), one of the
unchecked providers on the umbrella tracking issue #14736. LongCat
exposes an OpenAI-compatible REST API at
`https://api.longcat.chat/openai/v1` with three public chat models
including `LongCat-Flash-Thinking`, a reasoning model that returns
chain-of-thought in `reasoning_content` (OpenAI o-series shape).

Until this PR, a tenant who configured `longcat` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/longcat.go` with a `LongCatModel`
implementing the `ModelDriver` interface.
- New `conf/models/longcat.json` with the 3 public chat models
(Flash-Chat, Flash-Lite, Flash-Thinking) and `url_suffix` for `chat` and
`models`.
- `factory.go`: route `"longcat"` to `NewLongCatModel`.

Method coverage:

- `ChatWithMessages`: `POST /openai/v1/chat/completions`, non-streaming
- `ChatStreamlyWithSender`: SSE stream against the same endpoint
- `ListModels` / `CheckConnection`: `GET /openai/v1/models`
- **Reasoning extraction**: `message.reasoning_content` (non-stream) and
`delta.reasoning_content` (stream) flow into
`ChatResponse.ReasonContent` / the sender's second arg. Matches the
OpenAI o-series convention also used by kimi-k2.6 and DeepSeek-R1.
- **`reasoning_effort` propagation**: `ChatConfig.Effort` → request body
`reasoning_effort` (LongCat-Flash-Thinking honors it; non-reasoning
models ignore it).
- `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` return `"no such method"` (LongCat does not expose any of
these surfaces).

No interface change. No new dependencies.

### How was this tested?

**21 unit tests** in `internal/entity/models/longcat_test.go` — all
pass:

```
$ go test -vet=off -run TestLongCat -count=1 -v ./internal/entity/models/...
=== RUN   TestLongCatName
--- PASS: TestLongCatName (0.00s)
=== RUN   TestLongCatChatHappyPath
--- PASS: TestLongCatChatHappyPath (0.00s)
=== RUN   TestLongCatChatExtractsReasoningContent
--- PASS: TestLongCatChatExtractsReasoningContent (0.00s)
=== RUN   TestLongCatChatPropagatesReasoningEffort
--- PASS: TestLongCatChatPropagatesReasoningEffort (0.00s)
=== RUN   TestLongCatChatOmitsReasoningEffortWhenUnset
--- PASS: TestLongCatChatOmitsReasoningEffortWhenUnset (0.00s)
=== RUN   TestLongCatChatRequiresAPIKey
--- PASS: TestLongCatChatRequiresAPIKey (0.00s)
=== RUN   TestLongCatChatRequiresMessages
--- PASS: TestLongCatChatRequiresMessages (0.00s)
=== RUN   TestLongCatChatRejectsHTTPError
--- PASS: TestLongCatChatRejectsHTTPError (0.00s)
=== RUN   TestLongCatStreamHappyPath
--- PASS: TestLongCatStreamHappyPath (0.00s)
=== RUN   TestLongCatStreamExtractsReasoningContent
--- PASS: TestLongCatStreamExtractsReasoningContent (0.00s)
=== RUN   TestLongCatStreamRejectsExplicitFalse
--- PASS: TestLongCatStreamRejectsExplicitFalse (0.00s)
=== RUN   TestLongCatStreamRequiresSender
--- PASS: TestLongCatStreamRequiresSender (0.00s)
=== RUN   TestLongCatStreamFailsWithoutTerminal
--- PASS: TestLongCatStreamFailsWithoutTerminal (0.00s)
=== RUN   TestLongCatListModelsHappyPath
--- PASS: TestLongCatListModelsHappyPath (0.00s)
=== RUN   TestLongCatListModelsRequiresAPIKey
--- PASS: TestLongCatListModelsRequiresAPIKey (0.00s)
=== RUN   TestLongCatCheckConnectionDelegatesToListModels
--- PASS: TestLongCatCheckConnectionDelegatesToListModels (0.00s)
=== RUN   TestLongCatEmbedReturnsNoSuchMethod
--- PASS: TestLongCatEmbedReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatRerankReturnsNoSuchMethod
--- PASS: TestLongCatRerankReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatBalanceReturnsNoSuchMethod
--- PASS: TestLongCatBalanceReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatAudioOCRReturnNoSuchMethod
--- PASS: TestLongCatAudioOCRReturnNoSuchMethod (0.00s)
PASS
ok      ragflow/internal/entity/models  0.020s
```

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.longcat.chat`:

```
=== RUN   TestLongCatLiveSmoke
    [OK] Name() = "longcat"
    [OK] CheckConnection
    [OK] ListModels: 5 models -> [LongCat-Flash-Lite LongCat-Flash-Chat LongCat-Flash-Thinking-2601 LongCat-Flash-Omni-2603 LongCat-2.0-Preview]
    [OK] Chat (Flash-Chat) answer="Got it! Let me know if you" reason=""
    [OK] Chat (Flash-Thinking) answer len=443 head="To find 15 % of 80, follow these steps:\n\n1. **Convert the percentage to a frac..."
                  ReasonContent len=557 head="The user asks: \"15% of 80?\" They want step by step reasoning and final answer in \\boxed{}. So we need to compute 15% of ..."
    [OK] Stream content: 78 chunks, 351 chars
    [OK] Stream reasoning: 107 chunks, 537 chars
    [OK] Balance returns longcat, no such method
    [OK] Embed returns longcat, no such method
    [OK] Rerank returns longcat, no such method

LONGCAT LIVE SMOKE PASSED
--- PASS: TestLongCatLiveSmoke (31.01s)
```

What the live run proves on the wire:

- Auth header (`Bearer <key>`) is accepted by `api.longcat.chat`.
- `/openai/v1/models` parser handles the real 5-model response (note:
live API returns versioned aliases `LongCat-Flash-Thinking-2601`,
`LongCat-Flash-Omni-2603`, `LongCat-2.0-Preview` plus the un-versioned
`LongCat-Flash-Chat` and `LongCat-Flash-Lite`).
- Non-stream chat against `LongCat-Flash-Chat`: visible answer parses
correctly, `ReasonContent` correctly empty.
- Non-stream chat against `LongCat-Flash-Thinking`: 443-char answer
flows into `Answer`, 557-char chain-of-thought flows into
`ReasonContent` via the new `message.reasoning_content` extraction.
- Streaming chat against `LongCat-Flash-Thinking`: 107 reasoning chunks
(537 chars) reach the sender's second arg via `delta.reasoning_content`;
78 content chunks (351 chars) reach the first arg. Before this code, the
reasoning chunks would have been silently dropped.
- All sentinel methods (Balance, Embed, Rerank, audio/OCR) return the
documented `"no such method"` strings.

### Note on PR history

This branch was previously named for LocalAI work which is now
consolidated into PR #14813. The branch was reset to `upstream/main` and
rebuilt for LongCat. The diff against `main` is a clean +969 lines
across 4 files.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 16:27:56 +08:00
Wang Qi
ff685d3131 Delete duplicate route (#14883)
### What problem does this PR solve?

The delete /graph is duplicated of
`/datasets/<dataset_id>/<index_type>`, delete it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 15:57:44 +08:00
Idriss Sbaaoui
09e1fd290a Chore: migrate tests to restful api (#14871)
### What problem does this PR solve?

add new testing suite for the new restful api endpoints meant to replace
http and web api tests

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-13 15:07:23 +08:00
tmimmanuel
d63d3bb7d2 Go: implement provider: Novita.ai (#14850)
### What problem does this PR solve?

Add a Go driver for Novita.ai (https://novita.ai), one of the unchecked
providers on the umbrella tracking issue #14736. Novita exposes an
OpenAI-compatible REST API at `https://api.novita.ai/v3/openai` and
proxies a large catalog of third-party models (DeepSeek, Llama, Qwen3,
Kimi, Gemma, Mistral, MiniMax, GLM, etc.) behind a single OpenAI-shaped
surface — 102 models live at the time of writing.

Until this PR, a tenant who configured `novita` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/novita.go` with a `NovitaModel`
implementing the `ModelDriver` interface (~520 lines).
- New `conf/models/novita.json` with 7 representative chat models
(DeepSeek-V4, Llama-3.3-70B, Qwen3-30B/235B reasoning, Kimi-K2,
Gemma-3-27B, Mistral-Nemo).
- `factory.go`: route `"novita"` to `NewNovitaModel`.
- `internal/entity/models/novita_test.go`: 23 unit tests.

### Notable design point: `<think>...</think>` reasoning extraction

Novita-routed reasoning models like `qwen3-*` and `deepseek-r1-*` embed
their chain-of-thought **inline inside content as `<think>...</think>`
tags**, rather than in a separate `reasoning_content` field. Verified
live by probing `api.novita.ai`:

```
content head 200: <think>
Okay, let's see. I need to find 15% of 80. Hmm, percentages can sometimes be tricky, but I think
content tail 100: h, that works.
Alternatively, 0.15 × 80. If I move the decimal two places to the left for </think>
```

Without handling, a tenant picking qwen3 via Novita would see raw
`<think>` tags in their UI answer — different from every other reasoning
provider in the Go layer.

The driver detects those tags and routes the inner text to
`ChatResponse.ReasonContent` (non-stream) or the sender's second arg
(stream), keeping the visible answer clean of tag clutter:

- **`splitNovitaThink`** — scans a complete content string. Used by the
non-streaming path. Handles multiple `<think>` blocks, unclosed tags
(the model got cut off mid-reasoning), pure-text content with no tags.
- **`novitaThinkExtractor`** — stateful streaming version. Buffers
trailing bytes that might be the start of a tag (e.g. `<thi` held back
when the next chunk completes `nk>`), then emits segments in routing
order so callers can pipe them to a UI. Tested with byte-level chunk
boundaries and tag-spanning scenarios.

### Method coverage

| Method | Behavior |
|---|---|
| `ChatWithMessages` | `POST /v3/openai/chat/completions`, `<think>`
extraction on response |
| `ChatStreamlyWithSender` | SSE stream, stateful `<think>` extraction
across deltas |
| `ListModels` / `CheckConnection` | `GET /v3/openai/models` (102 live)
|
| `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` | `"no such method"` — Novita's OpenAI-compatible surface does
not expose any |

No interface change. No new dependencies.

### How was this tested?

**23 unit tests** in `internal/entity/models/novita_test.go` — all pass:

```
$ go test -vet=off -run "TestNovita|TestSplitNovita" -count=1 ./internal/entity/models/...
ok      ragflow/internal/entity/models  0.020s
```

Coverage:

- `splitNovitaThink` (5 cases: pure text, single block, leading text,
multiple blocks, unclosed tag)
- `novitaThinkExtractor` (6 cases: single-chunk, opening tag span,
closing tag span, byte-level chunking, no tags, lone `<` not as tag
start)
- `ChatWithMessages`: pure text, with `<think>` tags, missing API key,
empty messages, HTTP error
- `ChatStreamlyWithSender`: tag-stripping with spanning deltas, pure
content, sender-required, stream-true-required
- `ListModels` / `CheckConnection` (happy paths)
- All sentinel methods

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.novita.ai/v3/openai`:

```
=== RUN   TestNovitaLiveSmoke
    [OK] Name() = "novita"
    [OK] CheckConnection
    [OK] ListModels: 102 models (showing first 6) [deepseek/deepseek-v4-pro deepseek/deepseek-v4-flash deepseek/deepseek-v3.2 xiaomimimo/mimo-v2.5-pro moonshotai/kimi-k2.6 zai-org/glm-5.1]
    [OK] Chat (llama-3.3) answer="ok" reason=""
    [OK] Chat (qwen3) answer len=0 head=""
                  ReasonContent len=1657 head="Okay, so I need to figure out what 15% of 80 is. Hmm, percentages can sometimes trip me up, but let ..."
    [OK] Stream content: 0 chunks, 0 chars; reasoning: 600 chunks, 1667 chars
    [OK] Embed/Rerank/Balance/TranscribeAudio/AudioSpeech/OCRFile all return "novita, no such method"

NOVITA LIVE SMOKE PASSED
--- PASS: TestNovitaLiveSmoke (26.18s)
```

What the live run proves on the wire:

- Auth (`Bearer <key>`) accepted by `api.novita.ai`.
- `/v3/openai/models` parser handles the real 102-model response.
- Non-stream chat against `meta-llama/llama-3.3-70b-instruct`: clean
string answer, empty ReasonContent (non-reasoning model, pure-text
path).
- Non-stream chat against `qwen/qwen3-30b-a3b-fp8`: 1657-char reasoning
extracted from `<think>...</think>` and routed to
`ChatResponse.ReasonContent`. Visible answer is 0 chars in this run
because qwen3 spent its 600-token budget entirely on reasoning before
reaching the answer phase — that's the model's behavior, not a driver
bug. The important thing: **no `<think>` tags leaked into the visible
Answer field**.
- Streaming against qwen3: 600 reasoning chunks (1667 chars) emitted via
the sender's 2nd arg across SSE deltas; **no `<think>` tag fragments
leaked into the content channel** despite tag boundaries crossing chunk
boundaries on the wire.
- All 6 sentinel methods return the documented `"no such method"`
strings.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736
2026-05-13 14:10:50 +08:00
Jackie
71d327b11c Fix: The text field resizing function in the knowledge block creation… (#14212)
… modal

- Add vertical resizing functionality for the text field

### What problem does this PR solve?

_Fix the issue where the text content of the knowledge base editing
parsing block is too long to scroll._

<img width="701" height="775" alt="image"
src="https://github.com/user-attachments/assets/b258422e-fbc1-466d-abab-062e642c21d5"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: chenyun <chenyun@chenyundemacbook-pro.local>
2026-05-13 13:57:05 +08:00
Wang Qi
45d676bc05 Fix delete graphrag not take effect in UI (#14879)
### What problem does this PR solve?

Fix delete graphrag not take effect in UI

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 13:49:16 +08:00
Joseff
733d75d6a7 Fix(Go): make Baidu Encode fail loudly on malformed responses (#14721)
### What problem does this PR solve?

The Baidu (Qianfan) `Encode` method silently swallowed malformed
responses. If a `data[]` item from the API was missing a field (`index`,
`embedding`, or unexpected shape), the loop did `continue` instead of
returning an error, leaving `nil` entries in the result slice. Callers
got back partial results with no indication anything went wrong, which
then crashes downstream consumers when they try to use a `nil` vector.

Concrete gaps fixed:

- No count-mismatch check between `data` length and input texts (only
checked for empty)
- No duplicate-index detection (a duplicate would silently overwrite)
- No missing-index final scan
- No empty-embedding rejection
- No per-call context timeout
- `EmbeddingConfig.Dimension` (added in #14735) was not propagated

This PR replaces `map[string]interface{}` parsing with a typed
`baiduEmbeddingResponse` struct, applies the standard four-layer
validation (count → out-of-range → duplicate → empty → final
missing-index scan), adds `context.WithTimeout(nonStreamCallTimeout)`,
and forwards `embeddingConfig.Dimension` as the `dimensions` parameter
(Baidu Qianfan v2 uses an OpenAI-compatible API).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 12:54:00 +08:00
shawnxiao105-afk
8b6dd6a5c2 fix: guard whitespace-only chunks before embedding (#13938)
## Problem

When parsing DOCX files with many tables, DeepDOC generates chunks
containing only empty HTML table tags, such as:

```html
<table><tr><td></td></tr><tr><td></td></tr><tr><td></td></tr><tr><td></td></tr></table>
```

After the regex cleanup at `task_executor.py:584`, this becomes `" "`
(whitespace only).

The guard at line 585 (`if not c`) only catches empty strings `""`, but
whitespace strings are truthy in Python and pass through. When sent to
Zhipu `embedding-3` API, it rejects them with error 1213:
`未正常接收到prompt参数`.

## Root Cause

```python
c = re.sub(r"</?(table|td|caption|tr|th)( [^<>]{0,12})?>", " ", c)
if not c:       # ← only catches "", not "   " / "\n" / "\t"
    c = "None"
```

Verified with Zhipu `embedding-3`:
| Input | Result |
|---|---|
| `""` | error 1213 |
| `" "` | error 1213 |
| `"\n"` | error 1213 |
| `"None"` | OK |

## Fix

```diff
- if not c:
+ if not c.strip():
      c = "None"
```

## Testing

Reproduced with a 678KB DOCX file (166 tables, 270 chunks). Chunk #89 is
the empty table above. After fix, `"None"` is sent instead and embedding
succeeds.

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-13 11:47:50 +08:00
Wang Qi
64bd0130d3 Add REST API backward compatibility (#14872)
### What problem does this PR solve?

Add REST API backward compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 11:44:40 +08:00
dale053
5a5e766386 fix(api): authorize owner_ids for list chats and search apps (#14775)
Closes #14768
### What problem does this PR solve?
The `list_chats` and `list_searches` REST API endpoints did not enforce
authorization on the `owner_ids` query parameter. Any authenticated user
could pass arbitrary tenant IDs to `owner_ids` and retrieve chats or
search apps belonging to other tenants they are not a member of.

This PR resolves the issue by:
1. Looking up the current user's authorized tenants via
`TenantService.get_joined_tenants_by_user_id` and rejecting any
`owner_ids` that fall outside that set.
2. When no `owner_ids` are provided, scoping the query to only the
user's authorized tenants instead of returning an unfiltered result.
3. Adding unit tests that verify unauthorized `owner_ids` are rejected
with `OPERATING_ERROR`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 09:43:44 +08:00
Paul Yao
c34c81e8e6 fix: remove duplicate .wav and .aac in audio supported extensions list (#14791)
What problem does this PR solve?

In rag/app/audio.py, the supported audio extensions list contains
duplicate entries: .wav appears twice (positions 3 and 5) and .aac
appears twice (positions 6 and 14). While this does not affect runtime
behavior, it is redundant and makes the code harder to maintain.

This PR removes the duplicate entries to keep the list clean and
consistent.

Type of change

 - [X]  Bug Fix (non-breaking change which fixes an issue)
2026-05-13 09:42:31 +08:00
writinwaters
5e46457c28 Docs: How to add Bitbucket as data source. (#14846)
### What problem does this PR solve?

Added a guide on integrating Bitbucket as an external data source.

### Type of change

- [x] Documentation Update
2026-05-12 20:48:30 +08:00
Jin Hai
ad4717f40a Go: fix model type check when use the model (#14843)
### What problem does this PR solve?
```
RAGFlow(user)> chat with 'glm-ocr@test@zhipu-ai' message 'what is this'
CLI error: expect  model glm-ocr@zhipu-ai is a chat or multimodal model
```
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 19:44:01 +08:00
Wang Qi
76d5240fb5 Fix #14801 to allow search dataset list when add (#14841)
### What problem does this PR solve?

Fix #14801 to allow search dataset list when add, following on #14825

<img width="2172" height="857" alt="image"
src="https://github.com/user-attachments/assets/65ea7647-56f4-4c16-8437-121b834811f0"
/>


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 19:36:23 +08:00
balibabu
3f41f8cfae Feat: When a Wait Node precedes a Message Node within a Loop Node, the outgoing message is split into two separate messages. (#14839)
### What problem does this PR solve?

Feat: When a Wait Node precedes a Message Node within a Loop Node, the
outgoing message is split into two separate messages.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-05-12 18:48:44 +08:00
0xτensor
127aeac4aa fix: expose gpt-5.5 and gpt-5.4 in OpenAI model list (#14828)
### What problem does this PR solve?

OpenAI model catalogs used in provider selection flows were missing the
latest GPT models (`gpt-5.5` and `gpt-5.4`).
Because model availability is driven by seeded catalog data
(`conf/llm_factories.json` → DB seed → API response), these models were
not selectable in the UI or `/llm/list` responses.

This PR updates and synchronizes the OpenAI catalog definitions across
configuration sources and ensures the new models are correctly exposed
through the API layer and validated in tests.

---

### Type of change

* [x] New Feature (non-breaking change which adds functionality)

---

### Changes Made

* Added `gpt-5.5` and `gpt-5.4` to OpenAI catalog definitions in:

  * `conf/llm_factories.json`
  * `conf/models/openai.json` (chat + vision support)
* Ensured consistency between DB-seeded factory config and provider
model configuration
* Updated test coverage in:

  * `test_llm_list_unit.py`

    * seeded OpenAI catalog entries
* added response-level assertion validating `/llm/list` includes both
new model IDs under OpenAI grouping

---

### Root Cause

OpenAI model listings in selection flows are generated from catalog data
seeded via `conf/llm_factories.json`.
The catalog had not been updated to include the latest GPT models,
resulting in missing availability in UI and API responses.

---

### Testing

* Created isolated test environment:

  * `python -m venv .venv-review`
  * installed `pytest`
* Ran targeted and full test suite:

  * `test_list_app_grouping_availability_and_merge`:  passed
  * Full `test_llm_list_unit.py`:  10 passed

---

### Risks / Limitations

* Adding models to the catalog does not guarantee upstream provider
availability or account entitlement.
* Environments with pre-seeded DB catalogs may require reseed or refresh
to reflect updated configuration.

---

### Notes

* Changes are minimal and scoped strictly to catalog configuration and
related test coverage.
* Ensures `/llm/list` API remains aligned with expected latest OpenAI
model availability.
* Closes #14827
2026-05-12 18:03:47 +08:00
Haruko386
45ee5ca9cd Go: implement provider: Jina (#14838)
### What problem does this PR solve?

This PR completes the Jina provider

**The following functionalities are now supported:**

**Jina:**
- [ ] Chat / Stream Chat (Not available for now: [(Jina chat API
docs)](https://api.jina.ai/docs#/Search%20Foundation%20Models/chat_completions_v1_chat_completions_post))
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'jina-embeddings-v2-base-en@test@jina' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 768       | 0     |
| 768       | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'jina-reranker-v2-base-multilingual@test@jina' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.74316794      |
| 2     | 0.18713269      |
| 1     | 0.15817434      |
+-------+-----------------+

RAGFlow(user)> list supported models from 'jina' 'test'
+---------------------------------------------+
| model_name                                  |
+---------------------------------------------+
| Jina AI: Jina VLM                           |
| Jina AI: Jina Reranker v3                   |
| Jina AI: Jina Code Embeddings 0.5b          |
| Jina AI: Jina Code Embeddings 1.5b          |
| Jina AI: Jina Embeddings v4                 |
| Jina AI: Jina Reranker M0                   |
| Jina AI: ReaderLM v2                        |
| Jina AI: Jina Clip v2                       |
| Jina AI: Jina Embeddings v3                 |
| Jina AI: Jina Colbert v2                    |
| Jina AI: Reader LM 0.5b                     |
| Jina AI: Reader LM 1.5b                     |
| Jina AI: Jina Reranker v2 Base Multilingual |
| Jina AI: Jina Clip v1                       |
| Jina AI: Jina Reranker v1 Tiny EN           |
| Jina AI: Jina Reranker v1 Turbo EN          |
| Jina AI: Jina Reranker v1 Base EN           |
| Jina AI: Jina Colbert v1 EN                 |
| Jina AI: Jina Embeddings v2 Base ES         |
| Jina AI: Jina Embeddings v2 Base Code       |
| Jina AI: Jina Embeddings v2 Base DE         |
| Jina AI: Jina Embeddings v2 Base ZH         |
| Jina AI: Jina Embeddings v2 Base EN         |
| Jina AI: Jina Embedding B EN v1             |
| Jina AI: Jina Embeddings v5 Text Small      |
| Jina AI: Jina Embeddings v5 Omni Small      |
| Jina AI: Jina Embeddings v5 Omni Nano       |
| Jina AI: Jina Embeddings v5 Text Nano       |
+---------------------------------------------+

RAGFlow(user)> check instance 'test' from 'jina'
SUCCESS
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-12 18:03:05 +08:00
tmimmanuel
7d3836907a Go: implement Embed (embeddings) in Mistral driver (#14807)
### What problem does this PR solve?

The Mistral Go driver landed in #14805 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

`conf/models/mistral.json` did not list any embedding model out of the
box, so a tenant who wanted to use Mistral end to end (chat +
embeddings) could not run an embedding call. This PR adds
`mistral-embed` to the config and a real `/v1/embeddings`
implementation.

### What this PR includes

- `conf/models/mistral.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, siliconflow,
zhipu-ai), and add a `mistral-embed` entry under `models`
(1024-dimensional vectors, 8192 max input tokens).
- `internal/entity/models/mistral.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/embeddings`. Adds local response
types `mistralEmbeddingData` and `mistralEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Mistral API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14805 (the Mistral driver). Until #14805 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14805 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~111 line diff in `mistral.go` plus 8
lines in `mistral.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `MistralModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing OpenAI Embed implementation
(`internal/entity/models/openai.go`).

Closes #14806
Depends on #14805
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 17:45:48 +08:00
buua436
14332dd75c Go: fix dataset time unit (#14837)
### What problem does this PR solve?

fix dataset time unit

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 17:22:16 +08:00
Jin Hai
d08bf02d9b Go: add ASR, TTS, OCR command (#14836)
### What problem does this PR solve?

```
RAGFlow(user)> asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method
RAGFlow(user)> stream asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method

RAGFlow(user)> tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> stream tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> ocr with 'glm-ocr@test@zhipu-ai' file './test.log';
CLI error: zhipu, no such method
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 17:17:44 +08:00
buua436
9ee481807f GO: implement GET /api/v1/datasets/:dataset_id (#14834)
### What problem does this PR solve?

implement GET /api/v1/datasets/:dataset_id

### Type of change

- [x] Refactoring
2026-05-12 17:16:48 +08:00
Wang Qi
4374e07a29 Speed up start time (#14833)
### What problem does this PR solve?

Speed up start time

### Type of change
- [x] Refactoring
2026-05-12 17:00:45 +08:00
tmimmanuel
eaa2e46b1e Go: implement Embed (embeddings) in Upstage driver (#14819)
### What problem does this PR solve?

The Upstage Go driver landed in #14817 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

Upstage exposes an OpenAI-compatible embeddings endpoint at
`https://api.upstage.ai/v1/solar/embeddings` via the
`solar-embedding-1-large` family (`solar-embedding-1-large-query` for
queries, `solar-embedding-1-large-passage` for passages), and the Python
side has had `UpstageEmbed(OpenAIEmbed)` in `rag/llm/embedding_model.py`
for a long time targeting this same path. The existing
`conf/models/upstage.json` did not list any embedding model out of the
box, so a tenant who wanted to use Upstage end to end could not run an
embedding call. This PR fills the gap.

### What this PR includes

- `conf/models/upstage.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, mistral,
siliconflow, zhipu-ai), and add `solar-embedding-1-large-query` and
`solar-embedding-1-large-passage` entries under `models`.
- `internal/entity/models/upstage.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/solar/embeddings`. Adds local
response types `upstageEmbeddingData` and `upstageEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Upstage API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14817 (the Upstage driver). Until #14817 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14817 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~119 line diff in `upstage.go` plus ~15
lines in `upstage.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `UpstageModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Mistral Embed
(`internal/entity/models/mistral.go`) and OpenAI Embed
(`internal/entity/models/openai.go`) implementations.

Closes #14818
Depends on #14817
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 16:11:06 +08:00
Haruko386
ebab3513c4 Go: implement provider: Baichuan (#14832)
### What problem does this PR solve?

This PR completes the Baichuan provider

**The following functionalities are now supported:**

**Baichuan:**
- [x] Chat / Stream Chat
- [x] Embedding
- [ ] ~~Rerank~~
- [ ] ~~Model listing~~
- [ ] ~~Provider connection checking~~
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
# Baichuan

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'Baichuan-Text-Embedding@test@baichuan' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 1024      | 0     |
| 1024      | 1     |
+-----------+-------+

AGFlow(user)> chat with 'Baichuan-M2@test@baichuan' message 'who r u'
Answer: I'm BaiChuan, a helpful AI assistant created by Baichuan-AI. I'm designed to be a knowledgeable, friendly, and reliable assistant for various tasks like answering questions, explaining concepts, writing content, and more. Feel free to ask me anything! 😊
Time: 1.637975

RAGFlow(user)> stream chat with 'Baichuan-M2@test@baichuan' message 'who r u'
Answer: I'm BaiChuan-m2, an AI assistant developed by Baichuan-AI. My purpose is to help you with a wide range of tasks by providing information, answering questions, solving problems, and assisting with creative projects. Think of me as a helpful digital companion! If you have any questions or need assistance, just let me know.😊
Time: 1.692321
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-12 16:10:32 +08:00
Achieve3318
2cc206ee85 Test : aggregation edge cases for list and scalar values (#14170)
This PR adds focused unit tests for aggregate_by_field in OceanBase
memory utilities to improve behavior coverage for real-world input
shapes.

- Adds test coverage for list-valued aggregation fields, including
whitespace trimming and skipping invalid list entries.
- Adds test coverage for scalar field values to ensure blank/non-string
values are ignored.
- Confirms aggregation output remains correct and stable for
mixed-quality message payloads.

### Why this helps
It strengthens regression protection for aggregation logic used by
memory retrieval flows, with no production code changes and minimal
review risk.
2026-05-12 15:53:35 +08:00
Magicbook1108
f85e18afbc Refact: sandbox quickstart.md & add tutorial for code exec component (#14786)
### What problem does this PR solve?

Refact: sandbox quickstart.md && add tutorial for code exec component

### Type of change

- [x] Refactoring


<img width="700" alt="img_v3_0211j_dcff835b-e3bb-4c77-9bc5-3b31a983229g"
src="https://github.com/user-attachments/assets/7842fc0f-639a-458f-b164-bc81a99ce4a5"
/>

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2026-05-12 14:42:20 +08:00
buua436
e8adc977bd Fix: some agent bug (#14829)
### What problem does this PR solve?

fix: 
update null checks to use 'is None' for better clarity
replace RAGFlowSelect with SelectWithSearch in DebugContent
add max height and overflow to DialogContent in ParameterDialog
 remove unused types from DataOperationsForm

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 14:41:49 +08:00
lif
a02b456720 fix(docs): correct broken knowledge graph construction link (#13838)
Fixes #13817

### What problem does this PR solve?

The "knowledge graph construction" link on line 21 of
`docs/guides/dataset/run_retrieval_test.md` points to
`./construct_knowledge_graph.md`, which doesn't exist. The actual file
is at `./advanced/construct_knowledge_graph.md`.

### Type of change

- [x] Documentation Update

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-05-12 14:27:56 +08:00
tmimmanuel
558ea51a0f Go: implement provider: StepFun (#14815)
### What problem does this PR solve?

Add a Go driver for StepFun (阶跃星辰), one of the unchecked providers on
the umbrella tracking issue #14736.

Until this PR, a tenant who configured `stepfun` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver. Chat, list
models, and check connection all returned `"not implemented"` instead of
reaching the StepFun API.

The Python side has had StepFun registered in `rag/llm/__init__.py` as a
`SupportedLiteLLMProvider` with base URL `https://api.stepfun.com/v1`,
plus `StepFunCV` for vision and `StepFunSeq2txt` for ASR, but no Go
path. StepFun's chat API is OpenAI-compatible, so the implementation
pattern is the same as the merged Moonshot driver (#14433) and OpenAI
driver (#14605).

### What this PR includes

- New file `internal/entity/models/stepfun.go` with a `StepFunModel`
that implements the `ModelDriver` interface.
- `factory.go`: route the `"stepfun"` provider name to
`NewStepFunModel`.
- New `conf/models/stepfun.json` with the public StepFun chat models
(step-2-16k, step-1 family in 8k/32k/128k/256k context lengths,
step-1-flash, and the step-1v / step-1o vision models) and `url_suffix`
entries for `chat` and `models`.

### How the driver works

- StepFun exposes the OpenAI-compatible API at
`https://api.stepfun.com/v1`.
- `ChatWithMessages` and `ChatStreamlyWithSender` post to
`/chat/completions` in the same shape as the merged moonshot,
openrouter, and openai drivers.
- `ListModels` and `CheckConnection` call `/models` to list available
ids and confirm the API key works.
- `Embed` is left as `"not implemented"`. StepFun has not advertised a
public embeddings endpoint in the API reference linked from the umbrella
issue
(`https://platform.stepfun.com/docs/en/api-reference/chat/chat-completion-create`
is the chat endpoint), so any real implementation belongs in a separate
follow-up only after the endpoint is verified.
- `Rerank` and `Balance` return `"no such method"` because StepFun does
not expose either.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 with no errors
on go 1.25 (the `go.mod` minimum).
- Method set of `StepFunModel` matches the `ModelDriver` interface:
`NewInstance`, `Name`, `ChatWithMessages`, `ChatStreamlyWithSender`,
`Embed`, `Rerank`, `ListModels`, `Balance`, `CheckConnection`.
- Pattern parity with the merged moonshot (#14433), openai (#14605),
openrouter (#14652), and xai (#14550) drivers.

Closes #14814
Tracking: #14736
2026-05-12 13:49:35 +08:00
hyl64
02c2587ca4 fix(agent): support iteration item aliases in child nodes (#14146)
## Summary
This PR fixes the iteration variable mismatch reported in #14142.

Changes:
- restore compatibility for `IterationItem@result` by exposing `result`
alongside `item`
- support bare iteration aliases like `{item}`, `{index}`, and
`{result}` inside iteration child-node inputs
- add focused unit/runtime tests covering both alias styles and
multi-item iteration execution

## Validation
```bash
pytest -q --noconftest \
  test/testcases/test_web_api/test_canvas_app/test_iterationitem_unit.py \
  test/testcases/test_web_api/test_canvas_app/test_iteration_runtime_unit.py \
  test/testcases/test_web_api/test_canvas_app/test_invoke_component_unit.py
```

Result: `12 passed`

Closes #14142
2026-05-12 13:05:21 +08:00
Haruko386
128a64eae5 Refactor(Go): remove hardcode in huggingface provider (#14822)
### What problem does this PR solve?

remove hardcode in `huggingface` provider

### Type of change

- [x] Refactoring
2026-05-12 11:35:26 +08:00
dependabot[bot]
139b76d2b1 Chore(deps): Bump urllib3 from 2.6.3 to 2.7.0 in /agent/sandbox (#14824)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/releases">urllib3's
releases</a>.</em></p>
<blockquote>
<h2>2.7.0</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Security</h2>
<p>Addressed high-severity security issues. Impact was limited to
specific use cases detailed in the accompanying advisories; overall user
exposure was estimated to be marginal.</p>
<ul>
<li>
<p>Decompression-bomb safeguards of the streaming API were bypassed:</p>
<ol>
<li>When <code>HTTPResponse.drain_conn()</code> was called after the
response had been read and decompressed partially. (Reported by <a
href="https://github.com/Cycloctane"><code>@​Cycloctane</code></a>)</li>
<li>During the second <code>HTTPResponse.read(amt=N)</code> or
<code>HTTPResponse.stream(amt=N)</code> call when the response was
decompressed using the official <a
href="https://pypi.org/project/brotli/">Brotli</a> library. (Reported by
<a
href="https://github.com/kimkou2024"><code>@​kimkou2024</code></a>)</li>
</ol>
<p>See GHSA-mf9v-mfxr-j63j for details.</p>
</li>
<li>
<p>HTTP pools created using
<code>ProxyManager.connection_from_url</code> did not strip sensitive
headers specified in <code>Retry.remove_headers_on_redirect</code> when
redirecting to a different host. (GHSA-qccp-gfcp-xxvc reported by <a
href="https://github.com/christos-spearbit"><code>@​christos-spearbit</code></a>)</p>
</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Used <code>FutureWarning</code> instead of
<code>DeprecationWarning</code> for better visibility of existing
deprecation notices. Rescheduled the removal of deprecated features to
version 3.0. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3763">urllib3/urllib3#3763</a>)</li>
<li>Removed support for end-of-life Python 3.9. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3720">urllib3/urllib3#3720</a>)</li>
<li>Removed support for end-of-life PyPy3.10. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4979">urllib3/urllib3#4979</a>)</li>
<li>Bumped the minimum supported pyOpenSSL version to 19.0.0. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3777">urllib3/urllib3#3777</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Fixed a bug where <code>HTTPResponse.read(amt=None)</code> was
ignoring decompressed data buffered from previous partial reads. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3636">urllib3/urllib3#3636</a>)</li>
<li>Fixed a bug where <code>HTTPResponse.read()</code> could cache only
part of the response after a partial read when
<code>cache_content=True</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4967">urllib3/urllib3#4967</a>)</li>
<li>Fixed <code>HTTPResponse.stream()</code> and
<code>HTTPResponse.read_chunked()</code> to handle <code>amt=0</code>.
(<a
href="https://redirect.github.com/urllib3/urllib3/issues/3793">urllib3/urllib3#3793</a>)</li>
<li>Updated <code>_TYPE_BODY</code> type alias to include missing
<code>Iterable[str]</code>, matching the documented and runtime behavior
of chunked request bodies. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3798">urllib3/urllib3#3798</a>)</li>
<li>Fixed <code>LocationParseError</code> when paths resembling
schemeless URIs were passed to
<code>HTTPConnectionPool.urlopen()</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3352">urllib3/urllib3#3352</a>)</li>
<li>Fixed <code>BaseHTTPResponse.readinto()</code> type annotation to
accept <code>memoryview</code> in addition to <code>bytearray</code>,
matching the <code>io.RawIOBase.readinto</code> contract and enabling
use with <code>io.BufferedReader</code> without type errors. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3764">urllib3/urllib3#3764</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's
changelog</a>.</em></p>
<blockquote>
<h1>2.7.0 (2026-05-07)</h1>
<h2>Security</h2>
<p>Addressed high-severity security issues.
Impact was limited to specific use cases detailed in the accompanying
advisories; overall user exposure was estimated to be marginal.</p>
<ul>
<li>
<p>Decompression-bomb safeguards of the streaming API were bypassed:</p>
<ol>
<li>When <code>HTTPResponse.drain_conn()</code> was called after the
response had been
read and decompressed partially.</li>
<li>During the second <code>HTTPResponse.read(amt=N)</code> or
<code>HTTPResponse.stream(amt=N)</code> call when the response was
decompressed
using the official <code>Brotli
&lt;https://pypi.org/project/brotli/&gt;</code>__ library.</li>
</ol>
<p>See <code>GHSA-mf9v-mfxr-j63j
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-mf9v-mfxr-j63j&gt;</code>__
for details.</p>
</li>
<li>
<p>HTTP pools created using
<code>ProxyManager.connection_from_url</code> did not strip
sensitive headers specified in
<code>Retry.remove_headers_on_redirect</code> when
redirecting to a different host.
(<code>GHSA-qccp-gfcp-xxvc
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-qccp-gfcp-xxvc&gt;</code>__)</p>
</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Used <code>FutureWarning</code> instead of
<code>DeprecationWarning</code> for better
visibility of existing deprecation notices. Rescheduled the removal of
deprecated features to version 3.0.
(<code>[#3763](https://github.com/urllib3/urllib3/issues/3763)
&lt;https://github.com/urllib3/urllib3/issues/3763&gt;</code>__)</li>
<li>Removed support for end-of-life Python 3.9.
(<code>[#3720](https://github.com/urllib3/urllib3/issues/3720)
&lt;https://github.com/urllib3/urllib3/issues/3720&gt;</code>__)</li>
<li>Removed support for end-of-life PyPy3.10.
(<code>[#4979](https://github.com/urllib3/urllib3/issues/4979)
&lt;https://github.com/urllib3/urllib3/issues/4979&gt;</code>__)</li>
<li>Bumped the minimum supported pyOpenSSL version to 19.0.0.
(<code>[#3777](https://github.com/urllib3/urllib3/issues/3777)
&lt;https://github.com/urllib3/urllib3/issues/3777&gt;</code>__)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Fixed a bug where <code>HTTPResponse.read(amt=None)</code> was
ignoring decompressed
data buffered from previous partial reads.
(<code>[#3636](https://github.com/urllib3/urllib3/issues/3636)
&lt;https://github.com/urllib3/urllib3/issues/3636&gt;</code>__)</li>
<li>Fixed a bug where <code>HTTPResponse.read()</code> could cache only
part of the
response after a partial read when <code>cache_content=True</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9a950b92d9"><code>9a950b9</code></a>
Release 2.7.0</li>
<li><a
href="5ec0de499b"><code>5ec0de4</code></a>
Merge commit from fork</li>
<li><a
href="2bdcc44d1e"><code>2bdcc44</code></a>
Merge commit from fork</li>
<li><a
href="f45b0df09d"><code>f45b0df</code></a>
Fix a misleading example for <code>ProxyManager</code> (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4970">#4970</a>)</li>
<li><a
href="577193ca02"><code>577193c</code></a>
Switch to nightly PyPy3.11 in CI for now (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4984">#4984</a>)</li>
<li><a
href="e90af45bb0"><code>e90af45</code></a>
Avoid infinite loop in <code>HTTPResponse.read_chunked</code> when
<code>amt=0</code> (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4974">#4974</a>)</li>
<li><a
href="67ed74fdae"><code>67ed74f</code></a>
Bump dev dependencies (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4972">#4972</a>)</li>
<li><a
href="3abd481097"><code>3abd481</code></a>
Upgrade mypy to version 1.20.2 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4978">#4978</a>)</li>
<li><a
href="2b8725dfca"><code>2b8725d</code></a>
Drop support for EOL PyPy3.10 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4979">#4979</a>)</li>
<li><a
href="2944b2a0a6"><code>2944b2a</code></a>
Upgrade <code>setup-chrome</code> and <code>setup-firefox</code> to fix
warnings (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4973">#4973</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/urllib3/urllib3/compare/2.6.3...2.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=uv&previous-version=2.6.3&new-version=2.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-12 11:10:15 +08:00
CaptainTimon
2717ee283f feat(raptor): add Psi tree builder with original-space ranking and safe migration (#14679)
### What problem does this PR solve?

Closes #14674.

This PR improves RAPTOR configuration and tree construction while
preserving the existing RAPTOR behavior as the default.

RAPTOR currently builds summary layers with the original UMAP + GMM
clustering path. This PR keeps that default path, and adds:

- A hidden backend tree-builder option:
  - `tree_builder="raptor"`: default, existing RAPTOR behavior.
- `tree_builder="psi"`: rank-aware Psi-style tree builder using original
embedding-space cosine ranking.
- A user-facing clustering method option for the default RAPTOR builder:
  - `clustering_method="gmm"`: existing default.
- `clustering_method="ahc"`: agglomerative hierarchical clustering path.
- A RAPTOR UI setting for `Clustering method` and `Max cluster`.

### What changed

#### Backend

- Added `tree_builder` support for RAPTOR/Psi.
- Added `clustering_method` support for GMM/AHC.
- Kept existing RAPTOR + GMM as the default.
- Added Psi tree building from original-space cosine similarity.
- Added bucketed Psi building controls for large inputs:
  - `raptor.ext.psi_exact_max_leaves`
  - `raptor.ext.psi_bucket_size`
- Added method-aware RAPTOR summary metadata using existing
`extra.raptor_method`.
- Avoided adding a dedicated DB schema field for experimental method
tracking.
- Added cleanup/migration logic to avoid mixing stale RAPTOR summary
trees.
- Added defensive checks for Psi tree construction and summary failures.

#### Frontend/UI

- Added `Clustering method` in RAPTOR settings with `GMM` and `AHC`.
- Added/kept `Max cluster` in RAPTOR settings.
- Enlarged max cluster UI limit to `1024`, matching backend validation.
- Kept AHC editable even when a RAPTOR task has already finished.
- Fixed the UI save payload so `clustering_method` and `tree_builder`
are serialized through `parser_config.raptor.ext`, avoiding backend
validation errors for extra top-level RAPTOR fields.

Example saved RAPTOR config:

```json
{
  "raptor": {
    "max_cluster": 317,
    "ext": {
      "clustering_method": "ahc",
      "tree_builder": "raptor"
    }
  }
}

Co-authored-by: CaptainTimon <CaptainTimon@users.noreply.github.com>
2026-05-12 09:42:31 +08:00
黄圣祺
415169d497 fix(dify): add GET method support to /dify/retrieval for health check (#13837)
## Summary
- Add GET method handler to `/api/v1/dify/retrieval` endpoint for Dify
external knowledge base connectivity verification
- GET requests return a simple success response; POST requests retain
existing retrieval logic unchanged

## Problem
When Dify integrates with RAGFlow as an external knowledge base, it
sends periodic GET requests to the retrieval endpoint for
health/connectivity checks. The endpoint only accepted POST, causing
werkzeug to return `405 Method Not Allowed`. After several successful
POST retrievals, the failing GET health checks trigger Dify's circuit
breaker, causing all subsequent requests to fail.

Traceback from the issue:
```
werkzeug.exceptions.MethodNotAllowed: 405 Method Not Allowed: The method is not allowed for the requested URL.
```

## Changes
- `api/apps/sdk/dify_retrieval.py`: Added a separate GET route handler
(`retrieval_health_check`) that returns `get_json_result(data=True)`

## Test plan
- [ ] Verify `GET /api/v1/dify/retrieval` returns `{"code": 0,
"message": "success", "data": true}`
- [ ] Verify `POST /api/v1/dify/retrieval` with valid API key and body
still works as before
- [ ] Verify Dify external knowledge base integration no longer returns
405 errors

Closes #13788

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-12 09:37:07 +08:00
Ramin M.
765cdc2ec2 [Bug]: REDIS error #12870 (#13875)
Fix for: [Bug]: REDIS error #12870
2026-05-12 09:31:47 +08:00
Jin Hai
2f2d1569e6 Go: fix retrieval test error (#14794)
### What problem does this PR solve?

1. Add region check in zhipu-ai embed method
2. Fix retrieval test

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 20:19:08 +08:00
Haruko386
3e90d303e0 Go: implement provider: CoHere and FishAudio (#14790)
### What problem does this PR solve?

This PR completes the Cohere provider integration (upgrading to the new
Cohere V2 API) and enhances the Fish Audio provider in RAGFlow.

**The following functionalities are now supported:**

**Cohere:**
- [x] Chat / Think Chat / Stream Chat / Stream Think Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

**Fish Audio:**
- [x] Model listing (`ListModels`)
- [x] Balance (`Balance`)

-----

**Verified examples from the CLI:**

```plaintext

# Cohere

RAGFlow(user)> think chat with 'command-a-reasoning-08-2025@test3@cohere' message 'jumperwho'
Thinking: Okay, the user wrote "jumperwho". Let me try to figure out what they might be asking. First, I'll check if it's a misspelling. "Jumper" ...... Hmm. Since the query is unclear, the best approach is to ask the user to provide more context or correct any possible typos.
Answer: It seems there might be a typo or missing context in your query "jumperwho." Could you clarify what you're referring to? For example:
- Are you asking about a **jumper** (a type of sweater, a person who jumps, or a component in electronics)?
- Is this related to a specific context, like a movie (e.g., the 2008 film *Jumper*) or a game?
- Did you mean to ask about a person ("who") associated with jumping (e.g., a parachutist)?

Let me know so I can provide a helpful response! 😊
Time: 6.710331

RAGFlow(user)> stream think chat with 'command-a-reasoning-08-2025@test3@cohere' message 'jumperwho'
Thinking: , the user mentioned "jumperwho". Let me try to figure out what they're referring to. First, I'll check if it's a misspelling. "Jumper" could be a typo for "jumper" or maybe a username. Alternatively, it might be a combination of words like "jumper who",....... the best approach is to inform the user that I don't recognize the term and ask if they can provide more context or clarify what they mean by "jumperwho". That way, I can assist them better once I have more information.
Answer:  seems "jumperwho" isn't a widely recognized term, proper noun, or acronym in common usage. Could you provide more context or clarify what you mean by "jumperwho"? This will help me understand your question or request better!
Time: 4.513596

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'embed-v4.0@test3@cohere' dimension 16;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| embedding                                                                                                                                                                                                                                                        | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| [-0.016643638 -0.001957038 0.0055713872 0.009027058 0.05275187 -0.024542313 -0.044006906 0.024119169 0.0014192933 0.006558722 0.0019129605 -0.021016119 -0.026516981 -0.017489925 0.021298215 0.017772019 0.04569948 0.008886009 0.012059584 -0.0014721862 0.... | 0     |
| [0.018778935 -0.0063459855 -0.0006839742 0.0046623563 0.0067668925 -0.018001877 -0.03963003 0.035744734 -0.014246088 -0.0020721585 -0.006313608 0.025124922 -0.010749322 0.01217393 -0.010231283 -0.025254432 0.021498645 -0.028880708 0.019167464 -0.0058279... | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'rerank-v4.0-pro@test@cohere' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.91744334      |
| 1     | 0.7458429       |
| 2     | 0.68729424      |
+-------+-----------------+

RAGFlow(user)> list supported models from 'cohere' 'test'
+-------------------------------------+
| model_name                          |
+-------------------------------------+
| c4ai-aya-expanse-32b                |
| c4ai-aya-vision-32b                 |
| cohere-transcribe-03-2026           |
| command-a-03-2025                   |
| command-a-reasoning-08-2025         |
| command-a-translate-08-2025         |
| command-a-vision-07-2025            |
| command-r-08-2024                   |
| command-r-plus-08-2024              |
| command-r7b-12-2024                 |
| command-r7b-arabic-02-2025          |
| embed-english-light-v3.0            |
| embed-english-light-v3.0-image      |
| embed-english-v3.0                  |
| embed-english-v3.0-image            |
| embed-multilingual-light-v3.0       |
| embed-multilingual-light-v3.0-image |
| embed-multilingual-v3.0             |
| embed-multilingual-v3.0-image       |
| embed-v4.0                          |
+-------------------------------------+

RAGFlow(user)> check instance 'test' from 'cohere'
SUCCESS


# FishAudio

RAGFlow(user)> list supported models from 'fishaudio' 'test'
+----------------------------------------+
| model_name                             |
+----------------------------------------+
| Valentino Narración Biblica Fer        |
| Super Smash Bros. 4/Ultimate Announcer |
| Farid Dieck                            |
| عصام الشوالي                           |
| ALEX_CHIKNA                            |
| Energetic Male                         |
| voz de locutor k                       |
| يي                                     |
| ELITE                                  |
| Mortal Kombat                          |
+----------------------------------------+

RAGFlow(user)> show balance from 'fishaudio' 'test'
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+
| _id                              | created_at                  | credit | has_free_credit | has_phone_sha256 | updated_at                  | user_id                          |
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+
| 82ffec12cf984d88a30ec504d7909812 | 2026-05-09T07:52:16.119000Z | 0      |                 | false            | 2026-05-09T07:52:16.119000Z | 2578ab1126804d6eaa630552400d7ff3 |
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-11 20:18:38 +08:00
buua436
daf8a58c4b Fix: add codeexec attachments output (#14787)
### What problem does this PR solve?

add codeexec attachments output

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 19:16:33 +08:00
Renzo
39ee2fb120 Go: implement Rerank in NVIDIA driver (#14778)
## Summary

- Replaces the `"no such method"` stub on `NvidiaModel.Rerank`
(`internal/entity/models/nvidia.go`) with a real implementation against
NVIDIA NIM's `/ranking` endpoint.
- Mirrors the existing Python `NvidiaRerank` class at
`rag/llm/rerank_model.py:149-190` for behavior parity: same
`passages`/`query.text`/`logit` payload shape; `top_n` set to
`len(documents)` so every input gets a score returned in original order
(the issue body's spec omitted `top_n`, which would cause silent data
loss).
- Adds the `"rerank": "ranking"` URL suffix and two NIM rerank model
entries (`nvidia/nv-rerankqa-mistral-4b-v3`,
`nvidia/llama-3.2-nv-rerankqa-1b-v2`) to `conf/models/nvidia.json` so
the picker exposes them.
- Follows the same shape as the recently merged Aliyun (#14676), Gitee
(#14656), and ZhipuAI (#14608) Rerank implementations: lowercase
per-driver request/response types, conversion to the project-wide
`RerankResponse{Data: []RerankResult}`, per-call `context.WithTimeout`
of 30s.

Closes #14720

## Test plan

- [x] `gofmt -l internal/entity/models/nvidia.go` — clean
- [x] `go vet ./internal/entity/models/...` — no new errors introduced
(the two pre-existing vet errors in `baidu.go:642` and
`openrouter.go:566` are unrelated to this PR)
- [x] `go build ./internal/entity/models/...` — succeeds
- [x] `python3 -c "import json;
json.load(open('conf/models/nvidia.json'))"` — JSON valid
- [ ] Live smoke test against NVIDIA NIM with a real API key (requires
reviewer with NIM credentials)

## Notes for reviewers

- The issue body suggested omitting `top_n`. The Python reference
includes it (`top_n: len(texts)`), and without it NVIDIA returns only
the default top-K rankings rather than scores for every input. This PR
follows the Python.
- The URL host is `integrate.api.nvidia.com` (kept consistent with the
existing chat/embeddings BaseURL in `nvidia.go`), not the legacy
`ai.api.nvidia.com` host the Python uses. NIM's unified endpoint accepts
the model names as-is, so no per-model URL transform is needed.
2026-05-11 17:21:16 +08:00
Jin Hai
9b3850339b Go: add development guide document (#14785)
### What problem does this PR solve?

As the title suggests.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 17:20:41 +08:00
tmimmanuel
663fc1d42c fix(opensearch): implement doc-meta dispatch surface on OSConnection (#14577)
### What problem does this PR solve?

Fixes #14570. On OpenSearch backends (`DOC_ENGINE=opensearch`) every
document-metadata write failed with `'OSConnection' object has no
attribute 'create_doc_meta_idx'`, so both `PATCH
/api/v1/datasets/{ds}/documents/{doc}` with `meta_fields` and `POST
/api/v1/datasets/{ds}/metadata/update` were unusable while every other
document operation (retrieval, parsing, name update, chunk management)
worked correctly on the same OpenSearch cluster.

The bug runs deeper than the missing method name in the error message
suggests. `DocMetadataService` also reached into
`settings.docStoreConn.es.*` directly for the index refresh, the
scripted partial update, and the count call, which means that even after
adding `create_doc_meta_idx` to `OSConnection` the very next call in the
same metadata flow would still raise `AttributeError` because
`OSConnection` exposes `self.os` rather than `self.es`. Fixing only the
reported symptom would have moved the failure one line down without
restoring the feature.

This PR adds a uniform document-metadata dispatch surface to both
connection classes so they present the same abstract API, and routes the
service layer through that surface via `getattr` guards instead of
poking at backend-specific attributes. The four new methods on
`OSConnection` and `ESConnectionBase` are `create_doc_meta_idx`,
`refresh_idx`, `count_idx`, and `replace_meta_fields`.
`OSConnection.create_doc_meta_idx` reuses the existing
`conf/doc_meta_es_mapping.json` schema in the OpenSearch `body=` form
because OpenSearch and Elasticsearch share the same index-creation
payload, and `replace_meta_fields` emits a full scripted assignment
(`ctx._source.meta_fields = params.meta_fields`) on both backends so
removed keys actually disappear instead of being preserved by deep-merge
semantics.

The `getattr`-guarded dispatch in `DocMetadataService` keeps the
existing fall-through paths intact for Infinity and OceanBase, which
continue to rely on their search-based count fallback and on the
delete-then-insert metadata replacement they used before, so this change
is strictly additive for those two backends.

Verification: `pytest
test/unit_test/rag/utils/test_opensearch_doc_meta.py` runs 16 new unit
tests that pass locally and pin the `OSConnection` dispatch surface, the
`create_doc_meta_idx` short-circuit when the index already exists, the
mapping-file payload routing, the `IndicesClient.create` failure path,
the `refresh_idx` and `count_idx` success and error sentinels, and the
full-assignment script emitted by `replace_meta_fields`. The test module
stubs `common.settings` and `rag.nlp` at import time so the suite runs
without the heavy backend SDKs that the rest of the repository pulls in
transitively.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: tmimmanuel <tmimmanuel@users.noreply.github.com>
2026-05-11 17:04:28 +08:00
box4wangjing
292b0b8bce chore: fix some comments to improve readability (#14756)
### What problem does this PR solve?

fix some comments to improve readability

### Type of change

- [x] Documentation Update

---------

Signed-off-by: box4wangjing <box4wangjing@outlook.com>
2026-05-11 16:48:48 +08:00
Octopus
c58906b69e fix: OCR.detect() returns truthy None-tuple causing NoneType subscript crash (#13951)
Fixes #13851

## Problem

`OCR.detect()` in `deepdoc/vision/ocr.py` returns `None, None,
time_dict` (a truthy 3-tuple) when the text detector fails or receives a
`None` image. However, the caller in `pdf_parser.py:__ocr()` checks:

```python
bxs = self.ocr.detect(np.array(img), device_id)
if not bxs:  # False! (None, None, time_dict) is a non-empty tuple → truthy
    self.boxes.append([])
    return
bxs = [(line[0], line[1][0]) for line in bxs]  # iterates (None, None, time_dict)
# line = None → None[0] → TypeError: 'NoneType' object is not subscriptable
```

This causes the `NoneType object is not subscriptable` error that
appears after "OCR started" in the chunking pipeline when using PDF +
General parser.

## Solution

Simplified `OCR.detect()` to return `None` (falsy) instead of `None,
None, time_dict` on failure. The `time_dict` was unused by the only
caller of this method. The early-return guard `if not bxs:` in
`pdf_parser.py` then correctly catches it.

## Testing

- The method's only caller (`pdf_parser.py:__ocr`) already has a `if not
bxs:` guard that handles the `None` return correctly.
- No other callers of `OCR.detect()` exist in the codebase.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Modified OCR detection function return behavior to streamline output.
The function now returns detection results only, without timing
metadata. Error cases now return `None` instead of empty tuple values.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-11 16:19:28 +08:00
Nie WeiYang
1e80be77a2 fix(web): fix incomplete Docx preview in citation reference (#14122)
This PR fixes a UI issue where the .docx document preview was displayed
incompletely when clicking on a citation/reference link during a
knowledge base conversation.

### What problem does this PR solve?

The Issue:
In the chat interface, when a user clicks the source citation at the end
of an answer, the DocPreviewer opens. However, for .docx files, if the
content exceeded the window height, it was truncated and unscrollable,
preventing users from reading the full referenced text.

Changes:
web/src/components/document-preview/doc-preview.tsx: Added the
overflow-auto Tailwind class to the DocPreviewer root container to
ensure scrollbars appear automatically when content overflows.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: nie.weiyang <nie.weiyang@embedway.com>
2026-05-11 16:17:48 +08:00
as-ondewo
6fb8c31c22 Fix: Document parse status set to DONE before chunks are retrievable (#13352)
### What problem does this PR solve?

The document parse status was set to DONE before the document chunks
were actually retrievable from Elasticsearch/Opensearch because it did
not wait for the index refresh. This meant that it was possible that the
document parse status returned by the API was DONE but when trying to
retrieve chunks there were none. Since the index refreshes every 1
second this was quite likely to happen when wait for document parsing by
polling with a short interval and then immediately trying to retrieve
chunks once the status was DONE.

I fixed this bug and added a test case that would have caught it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 16:04:08 +08:00
Sank
592dba1489 Refact: Added a private helper _visibility_and_status_filter (#13627)
### What problem does this PR solve?

Added a private helper _visibility_and_status_filter(joined_tenant_ids,
user_id) that returns the Peewee condition: visible to user (team or
own) and status is VALID.

### Type of change
- [x] Refactoring

---------

Co-authored-by: Serobabov Aleksandr <40SerobabovAS@region.cbr.ru>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-05-11 15:21:41 +08:00
tmimmanuel
6ce014c23b fix: offload blocking DB/Redis calls to thread pool for high-concurrency support (#13825) (#13941)
### What problem does this PR solve?

Addresses event-loop blocking under high concurrency reported in #13825.
When multiple requests hit the API simultaneously, synchronous DB/Redis
calls block the async event loop, preventing Quart from handling other
requests and causing cascading 502/504 timeouts.

This PR wraps all remaining blocking DB/Redis calls in `canvas_app.py`,
`chat_api.py`, `session.py`, and `canvas_service.py` with `await
thread_pool_exec()`
- Offload all synchronous `Service.*`, `REDIS_CONN.*`, and
`APIToken.query` calls to the thread pool
- Convert sync endpoint handlers (`list_chats`, `get_chat`, `templates`,
`sessions`, etc.) to `async def`
- Convert sync helper functions (`_ensure_owned_chat`,
`_validate_llm_id`, `_validate_dataset_ids`, etc.) to async - no
duplicate sync/async pairs
- Wrap `CanvasReplicaService` Redis IO calls (`bootstrap`,
`replace_for_set`, `commit_after_run`)
- Use `asyncio.gather()` for concurrent file uploads and chat response
building

**Note:** This fixes the code-level event-loop blocking, which is a
prerequisite for handling concurrent requests. For the full "30
concurrent requests without 502/504" goal described in the issue, users
should also tune deployment config:
- `WS=4` or higher (HTTP worker processes, default 1)
- `MAX_CONCURRENT_CHATS=50` (default 10)
- `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` for workflow-heavy workloads

### Performance verification

Reviewer asked for a before-vs-after comparison
([comment](https://github.com/infiniflow/ragflow/pull/13941#issuecomment-4393667231)).
I built a self-contained microbenchmark that reproduces the exact
failure mode this PR targets: an async handler that performs blocking
DB/Redis-style calls (50 ms each, 3 per request, 30 concurrent requests)
is run twice — once with the pre-PR pattern (sync call directly inside
the async handler) and once with the post-PR pattern (`await
thread_pool_exec(...)`). The benchmark imports nothing from RAGFlow
except `thread_pool_exec` itself, so it is hermetic and reproducible
(`THREAD_POOL_MAX_WORKERS=128`, Python 3.13.12).

**Throughput — wall-clock for 30 concurrent requests (lower is better)**

| flavour | wall(s) | p50(s) | p95(s) | max(s) |
|---|---:|---:|---:|---:|
| before | 4.986 | 0.158 | 0.207 | 0.269 |
| after  | 0.248 | 0.181 | 0.230 | 0.231 |

The pre-PR handler serializes the entire load on the event-loop thread,
so 30 × 3 × 50 ms ≈ 4.5 s shows up as the wall time. The post-PR handler
parallelizes the blocking work across the thread pool and finishes the
same load in 248 ms — a **~20× speedup** on this workload.

**Event-loop responsiveness — latency of an unrelated probe coroutine
while the 30 slow requests are running (lower is better)**

| flavour | samples | probe p50 (ms) | probe p95 (ms) | probe max (ms) |
|---|---:|---:|---:|---:|
| before | 1 | 5442.26 | 5442.26 | 5442.26 |
| after  | 28 | 0.88 | 11.53 | 98.02 |

This is the metric that maps directly to "the API still answers other
requests while one is busy". A 5 ms-interval probe was scheduled while
the 30 slow handlers ran. With the pre-PR code the event loop was frozen
for the entire duration of the blocking work, so only one probe sample
was ever picked up and it waited **5,442 ms**. After the PR, 28 probe
samples landed with **p50 0.88 ms / p95 11.53 ms**, meaning unrelated
requests are no longer starved by the slow ones. That is the regression
mode behind the cascading 502/504s reported in #13825.

<details>
<summary>Raw benchmark output</summary>

```
config: 30 concurrent requests, 3 blocking calls of 50ms each per request, THREAD_POOL_MAX_WORKERS=128

=== Throughput (lower wall is better) ===
flavour       wall(s)     p50(s)     p95(s)     max(s)
before          4.986      0.158      0.207      0.269
after           0.248      0.181      0.230      0.231

=== Event-loop responsiveness (lower probe latency is better) ===
flavour       samples    probe p50(ms)    probe p95(ms)    probe max(ms)
before              1          5442.26          5442.26          5442.26
after              28             0.88            11.53            98.02
```

</details>

The benchmark script is included as a comment on the PR for
reproducibility.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Performance Improvement

Closes [#13825](https://github.com/infiniflow/ragflow/issues/13825)

---------

Co-authored-by: tmimmanuel <tmimmanuel@users.noreply.github.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 15:08:55 +08:00
Paul Y Hui
a0efc453f3 Fix: safe argument guard and remove redundant redis call (#14060)
### What problem does this PR solve?

- Moved if not all([email, new_pwd, new_pwd2]) guard to the top, before
any decryption that could crash on None value
- Removed the redundant REDIS_CONN.get() call — one call is sufficient

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-11 15:02:24 +08:00
Jin Hai
c55e23e7e2 Go: refactor embedding interface (#14757)
### What problem does this PR solve?

Provide embedding index according to the input text

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 14:45:30 +08:00
Ricardo-M-L
5ef7f50eef fix: use context manager for ThreadPoolExecutor in file_service.py (#14144)
## Summary
- Wrap 2 `ThreadPoolExecutor` instances in `file_service.py` with `with`
statement
- Ensures threads are properly shut down after all futures complete

## Problem

`parse_docs()` (line 532) and the file processing method (line 694)
create `ThreadPoolExecutor` instances that are never shut down. In a
long-running server process, this leaks thread resources on every
invocation — threads remain alive consuming memory even after all
submitted work is complete.

## Fix

Replace bare `ThreadPoolExecutor()` with `with ThreadPoolExecutor() as
exe:` context manager, which calls `executor.shutdown(wait=True)` on
exit.

## Test plan
- [x] Verified both call sites use `with` statement after fix
- [x] No remaining bare `ThreadPoolExecutor` in `file_service.py`
- [x] `document_service.py:1066` is a module-level executor (different
pattern, not changed in this PR)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 14:02:45 +08:00
buua436
a03b95f8c4 Fix: shared dataset chunk index lookup (#14764)
### What problem does this PR solve?

shared dataset chunk index lookup

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:50:08 +08:00
buua436
024c8cb0b5 Fix: dataset search rerank id type (#14759)
### What problem does this PR solve?
issue: https://github.com/infiniflow/ragflow/issues/14748

change: dataset search rerank id type
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:48:05 +08:00
jony376
46897d6fa4 Fix: bind memory message user_id to authenticated user for JWT auth (#14745)
### Related issues

Closes #14744

### What problem does this PR solve?

The Memory REST endpoint `POST /api/v1/messages` previously persisted
whatever `user_id` the client sent in the JSON body. Memory rows were
therefore attributed to an arbitrary string, even when the caller
authenticated as a normal workspace user via JWT (browser/session-style
bearer token decoded into an access token). That broke attribution and
audit semantics for shared memories (team visibility): any authorized
writer could spoof another subject id.

The Python SDK already sends an optional `user_id` for integrations
using **API keys** (`APIToken`) to tag an external subject distinct from
the tenant owner user.

### Solution

- Record **`g.auth_via_api_token`** in `_load_user`
(`api/apps/__init__.py`): set `True` only when authentication resolves
via `APIToken`, otherwise `False` after JWT-based login succeeds.
- In **`POST /messages`** (`memory_api.add_message`): if the request was
authenticated with an API key, keep accepting optional `user_id` from
the body (default empty string). For JWT-authenticated users, **always**
set stored `user_id` to **`current_user.id`** and ignore the client
field.
- Guard reads of `g` with **`RuntimeError`** handling so isolated
imports or tests without a Quart application context do not fail when
resolving `user_id`.
- Document on **`RAGFlow.add_message`** that `user_id` is only
meaningful for API-key authentication.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- `python -m py_compile` on modified modules (`api/apps/__init__.py`,
`api/apps/restful_apis/memory_api.py`).
- Recommended: run web/SDK memory message tests (`test_add_message`,
`test_message_routes_unit`) against a full environment with `quart` and
configured services.

### Notes for reviewers

- Behavior change **only** for callers using JWT-style authorization on
`POST /messages`; API-key callers keep prior optional `user_id`
semantics.

Co-authored-by: jony376 <jony376@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 13:26:05 +08:00
Achieve3318
16354f4e14 fix(dify): guard retrieval argument error behavior (#14169)
## What problem does this PR solve?

The Dify-compatible `/dify/retrieval` endpoint recently gained stricter
parsing and validation for its request payload, including:
- Normalized `retrieval_setting.top_k` and
`retrieval_setting.score_threshold` types.
- Clear separation between malformed arguments vs missing required
fields.
Previously, there was no unit test explicitly guarding the exact error
code and message contract for these cases.

## What does this PR change?

- **Add guard-style unit test** in `test_dify_retrieval_routes_unit.py`:
  - `test_retrieval_argument_error_messages`:
    - Sends a request with malformed numeric options:
- `retrieval_setting = {"top_k": "not-int", "score_threshold":
"not-float"}`
      - Asserts `code == RetCode.ARGUMENT_ERROR` and message contains  
        `"invalid or malformed arguments:"`.
    - Sends a request with required fields missing:
      - Empty payload (`{}`)
      - Asserts `code == RetCode.ARGUMENT_ERROR` and message contains  
        `"required arguments are missing:"`.

This test encodes the intended behavior of the Dify retrieval API so
future refactors cannot silently regress error handling.

## Type of change

- [x] Tests (add coverage and guardrails for existing behavior)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 13:17:42 +08:00
FPlust
0734fd793a fix: scope pending_cell_images by sheet in excel parser (#14120)
pending_cell_images should be scoped by sheet

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:17:14 +08:00
Wang Qi
3838770e7a GraphRAG feature - Part 1 - add spacy to extract entity and relation (#14670)
### What problem does this PR solve?

GraphRAG feature - Part 1 - add spacy to extract entity and relation

<img width="1621" height="1288" alt="image"
src="https://github.com/user-attachments/assets/aadeddad-94da-46c6-adad-9c3784181f61"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 12:59:59 +08:00
web-dev0521
cc207b5b05 Refactor: tidy up ThreadPoolExecutor lifecycle in file_service and task executor (#14668)
## Summary
- Wrap the `ThreadPoolExecutor` instances in `FileService.parse_docs`
and `FileService.get_files` with `with ... as exe:` blocks for
deterministic cleanup
- Replace the `concurrent.futures.ThreadPoolExecutor` in
`do_handle_task` with `asyncio.create_task(asyncio.to_thread(build_TOC,
...))`, preserving the existing parallelism with chunk insertion while
leveraging the surrounding async context
- Drop the now-unused `import concurrent` and the
`executor.shutdown(wait=False)` call in the `finally` block

Closes #14622.

No behavioral change, no public API change. Net diff: ~19 insertions /
25 deletions across two files.

## Test plan
- [ ] `uv run ruff check api/db/services/file_service.py
rag/svr/task_executor.py` passes
- [ ] Upload a multi-file batch through the chat/file endpoint and
confirm `FileService.parse_docs` still returns combined parsed text
- [ ] Trigger `FileService.get_files` via the chat reference flow with a
mix of image and non-image files; verify both `raw=True` and `raw=False`
paths return correctly
- [ ] Run a `naive`-parser document task with `toc_extraction: true` and
confirm the TOC chunk is generated and inserted exactly as before
- [ ] Run a `naive`-parser document task with `toc_extraction: false`
and confirm the path with `toc_thread = None` is unaffected
- [ ] Cancel a running task to exercise the `finally` block and confirm
cleanup still works without the executor shutdown call

---------

Co-authored-by: web-dev0521 <jasonpette1783@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-05-11 12:59:00 +08:00
Joseff
13e6554901 Fix(Go): make OpenRouter Encode fail loudly on malformed responses (#14717)
### What problem does this PR solve?

The OpenRouter `Encode` method silently swallowed malformed responses.
If a `data[]` item from the API was missing a field (`index`,
`embedding`, or unexpected shape), the loop did `continue` instead of
returning an error — leaving `nil` entries in the result slice. Callers
got back partial results with no indication anything went wrong, which
then crashes downstream consumers when they try to use a `nil` vector.
There were three concrete gaps:

- No count-mismatch check between `data` length and input texts (only
checked for empty)
- No duplicate-index detection (a duplicate would silently overwrite)
- Parse failures on individual items returned partial slices instead of
erroring

This PR replaces `map[string]interface{}` parsing with a typed
`openrouterEmbeddingResponse` struct and applies the same 3-layer
validation used in the other drivers (count mismatch → out-of-range
index → duplicate index), so any malformed response produces a clear
error instead of corrupted data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 12:57:11 +08:00
Panda Dev
530edbac99 Go: implement Encode (embeddings) in LM Studio driver (#14694)
### What problem does this PR solve?

The LM Studio Go driver shipped with a stub \`Encode\` method that
returned \`no such method\`, even though LM Studio is one of the most
common local LLM runners on macOS and Windows and exposes an
OpenAI-compatible embeddings endpoint at \`/v1/embeddings\`.

LM Studio users routinely load local embedding models such as
\`nomic-ai/nomic-embed-text-v1.5\`,
\`mixedbread-ai/mxbai-embed-large-v1\`, or \`BAAI/bge-m3\`. They run on
the same \`/v1\` namespace as chat. The existing \`ListModels\` already
discovers them, but because \`Encode\` was a stub, a tenant who picked
one of these models in the Go layer could not actually run an embedding
call.

This finishes the local-LLM trio: Ollama Encode (#14664) and vLLM Encode
(#14688) are already in flight, both using the
same OpenAI-compatible \`/embeddings\` shape.

### What this PR includes

- \`conf/models/lmstudio.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the driver can build the URL from config.
- \`internal/entity/models/lmstudio.go\`: replace the \`Encode\` stub
with a real implementation. Adds a small local response type that
matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for local LM Studio,
so the Authorization header is only set when both \`apiConfig\` and
\`ApiKey\` are non-nil and non-empty, the same pattern the recently
merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
Aliyun Encode (#14647) and the in-flight Ollama Encode (#14664) and vLLM
Encode (#14688) use.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`LmStudioModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the merged Aliyun Encode (#14647), the in-flight
Ollama Encode (#14664) and vLLM Encode (#14688), and the existing
SiliconFlow Encode.

Closes #14693
2026-05-11 12:55:57 +08:00
Joseff
0580c137fa Perf(Go): batch SiliconFlow Encode requests with 32-item chunking (#14719)
### What problem does this PR solve?

The SiliconFlow `Encode` method sent one HTTP request per text, which is
wasteful and slow when indexing many documents (e.g., 100 docs = 100
round-trips).

SiliconFlow's `/v1/embeddings` is OpenAI-compatible and accepts an array
of strings in `input` (officially documented at
https://docs.siliconflow.cn/en/api-reference/embeddings/create-embeddings,
with a documented max array size of 32). This PR batches the requests up
to that limit, reducing 100 docs to ~4 round-trips, and replaces
`map[string]interface{}` parsing with a typed struct using the same
3-layer validation (count mismatch, out-of-range index, duplicate index)
used in the other drivers.

### Type of change

- [x] Performance Improvement
2026-05-11 12:55:27 +08:00
BitToby
4b96362092 Go: implement Encode (embeddings) in NVIDIA driver (#14700)
### What problem does this PR solve?

The NVIDIA Go driver in `internal/entity/models/nvidia.go` shipped with
a stub `Encode`
method that returned `no such method`. `conf/models/nvidia.json` already
lists
`nvidia/llama-3.2-nemoretriever-1b-vlm-embed-v1` as an embedding model,
but the conf had
no `embedding` URL suffix, so the picker had nothing wired even if
`Encode` worked.

A tenant who wanted to use NVIDIA NIM for chat (already working) and
embeddings from a
single provider could not, even though the upstream endpoint is public
at
`https://integrate.api.nvidia.com/v1/embeddings` and uses an
OpenAI-compatible request
body extended with the NVIDIA-specific `input_type` and `truncate`
fields. Several other
Go drivers already implement `Encode` (siliconflow, zhipu-ai, aliyun),
so the interface
and the pattern are well-established.

This PR fills the gap.

### What this PR includes

* `conf/models/nvidia.json`: declare the `embedding` URL suffix
alongside the existing
`chat` and `models` entries. The embedding model entry was already
present, so no
  model addition is needed.
* `internal/entity/models/nvidia.go`: replace the `Encode` stub with a
real
implementation. Adds a small local response type that matches the
OpenAI-compatible
  shape NVIDIA NIM returns.

No factory change. No interface change.

### How the driver works

* Validates `apiConfig` and the API key, validates the model name,
resolves the region
with a default fallback (matching the pattern the merged `ListModels`
and
`CheckConnection` paths in this driver already use), and builds the URL
from
  `BaseURL[region] + URLSuffix.Embedding`.
* Sends all input texts in one request as the `input` array, with the
NVIDIA-specific `input_type: "query"`, `encoding_format: "float"`, and
`truncate: "END"`
  fields, mirroring the Python `NvidiaEmbed` reference.
* Parses `data[*].embedding` and copies each slice into `[][]float64`
indexed by
`data[*].index` so the output order matches the input order even if the
API returns
  items in a different order.
* Handles both `float64` and `float32` element types.
* Empty input returns `[][]float64{}` with no HTTP call.
* Non-200 responses propagate the upstream status line and body.
* A final pass checks every input slot got a vector and returns a clear
error if any
  slot is still nil.
* Per-call 30s context deadline so a slow call cannot block forever.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

* `go build ./internal/entity/models/...` returns exit 0.
* `go vet ./internal/entity/models/...` is clean.
* `gofmt -l internal/entity/models/nvidia.go` is clean.
* The full method set on `NvidiaModel` still matches the `ModelDriver`
interface.
* Pattern parity with the just-merged Aliyun `Encode` (#14647).

Closes #14699
2026-05-11 12:50:50 +08:00
Jack Storment
8ff623fbc4 Go: implement Encode (embeddings) in Ollama driver (#14664)
### What problem does this PR solve?

The Ollama Go driver shipped with a stub \`Encode\` method that returned
\`no such method\`, even though Ollama is one of the most common local
LLM runners and exposes an OpenAI-compatible embeddings endpoint at
\`/v1/embeddings\`.

Ollama users routinely run local embedding models such as
\`nomic-embed-text\`, \`mxbai-embed-large\`, or \`bge-m3\`.
Pulled with \`ollama pull <model>\` and served on the same \`/v1\`
namespace as chat. The existing \`ListModels\` already
discovers them, but because \`Encode\` was a stub, a tenant who picked
one of these models in the Go layer could not
actually run an embedding call.

### What this PR includes

- \`conf/models/ollama.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the
  driver can build the URL from config.
- \`internal/entity/models/ollama.go\`: replace the \`Encode\` stub with
a real implementation. Adds a small local response
  type that matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for local Ollama, so
the Authorization header is only set when both
\`apiConfig\` and \`ApiKey\` are non-nil and non-empty, the same pattern
the recently merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
  Aliyun Encode (#14647) uses.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`OllamaModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the merged Aliyun Encode (#14647) and the existing
SiliconFlow Encode.

Closes #14662
2026-05-11 12:50:15 +08:00
hyl64
77ce88dfcc fix(prompt): reserve system budget in message_fit_in (#14164)
## Summary
This PR fixes the `message_fit_in()` truncation bug reported in #13607.

Changes:
- fix the user-message truncation branch to reserve room for the system
prompt token budget
- guard the zero-token edge case to avoid dividing by zero in the
truncation ratio check
- add focused regression tests covering both the user-dominant
truncation path and the zero-token boundary case

## Validation
```bash
pytest -q --noconftest test/unit_test/rag/prompts/test_generator_message_fit_in.py
```

Result: `2 passed`

Closes #13607
2026-05-11 12:44:27 +08:00
07heco
e46989832e fix: complete robustness fixes for rerank module addressing all review comments (#14265)
## Summary
This PR fully addresses all CodeRabbit review feedback and enhances the
robustness of the reranking module with 100% backward compatibility.

## Key Fixes
1. Fixed JinaRerank hardcoded base_url to support subclass endpoint
overrides
2. Corrected GPUStackRerank exception handling to use proper requests
exceptions and preserve stack traces
3. Added 30s timeout to all API calls to prevent service hanging
4. Added empty input validation for all rerank providers
5. Replaced direct dict key access with .get() to eliminate KeyError
crashes
6. Fixed _normalize_rank edge case for empty arrays
7. Implemented missing functionality for Ai302Rerank
8. Standardized type hints and fixed typo issues

## Compatibility
- No breaking changes to any existing functionality
- All rerank providers work as originally intended
- Fully compatible with existing configurations and workflows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 12:40:41 +08:00
Panda Dev
fa53b93dd5 Go: implement Encode (embeddings) in vLLM driver (#14688)
### What problem does this PR solve?

The vLLM Go driver shipped with a stub \`Encode\` method that returned
\`not implemented\`, even though vLLM is one of the most common
production-grade self-hosted inference servers and exposes an
OpenAI-compatible embeddings endpoint at \`/v1/embeddings\`.

Users who self-host \`BAAI/bge-m3\`, \`Qwen3-Embedding-*\`,
\`NV-Embed-v2\`, or similar models on vLLM could not run an embedding
call through the Go layer. The existing \`ListModels\` already discovers
the loaded models, but the embedding path failed because \`Encode\` was
a stub.

### What this PR includes

- \`conf/models/vllm.json\`: add \`\"embedding\": \"embeddings\"\` under
\`url_suffix\` so the driver can build the URL from config.
- \`internal/entity/models/vllm.go\`: replace the \`Encode\` stub with a
real implementation. Adds a small local response
  type that matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for self-hosted vLLM,
so the Authorization header is only set when both \`apiConfig\` and
\`ApiKey\` are non-nil and non-empty, the same pattern the recently
merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
  Aliyun Encode (#14647) and in-flight Ollama Encode (#14664) use.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`VllmModel\` still matches the \`ModelDriver\`
interface.
- Pattern parity with the merged Aliyun Encode (#14647), the in-flight
Ollama Encode (#14664), and the existing
  SiliconFlow Encode.

Closes #14687
2026-05-11 12:09:17 +08:00
Qinsanz
d6660cf156 fix(keyword_extraction): accept Chinese commas/semicolons/newlines as keyword delimiters (#14540)
## What
Widen the keyword delimiter in `rag/svr/task_executor.py`:
both `build_chunks` (LLM `keyword_extraction` cache parsing) and
`run_dataflow` (chunk-level `keywords` ingestion) now split on
`, , ; ; 、 \r \n` instead of only ASCII comma.

## Why
`rag/prompts/keyword_prompt.md` instructs the LLM:

> The keywords are delimited by ENGLISH COMMA.

In practice, Chinese-leaning models (Qwen / Tongyi-Qianwen, GLM,
etc.) frequently ignore this instruction when the source content is
Chinese and emit Chinese commas (`,`) instead. Result:
`cached.split(",")` sees the full LLM output as a *single* keyword.

Repro: `auto_keywords>=4` + Chinese docs + `qwen-plus@Tongyi-Qianwen`.
We observed entries in `important_kwd` like
`"功能介绍,配置说明,参数详解,问题排查"` — one bucket instead of four.

## Impact
- Silent data-quality bug; no exception thrown.
- BM25 `important_kwd^30` boost effectively stops firing — the
  indexed term is the whole list, never matches user query tokens.
- Any downstream aggregating `important_kwd` (tagging, analytics,
  candidate-keyword review UIs) sees garbage.

## Compatibility
- Pure widening of the splitter; ASCII-comma-only outputs continue
  to work identically.
- No schema / API change.

## Test plan
Manually verified against `qwen-plus@Tongyi-Qianwen` with
`auto_keywords=10` on Chinese .txt files:

- Before: `important_kwd` contains one element per chunk that is the
  full LLM string with `,`-separated phrases inside.
- After: `important_kwd` contains N elements, one per phrase, as the
  LLM intended.
2026-05-11 12:05:24 +08:00
BitToby
bfb4a0eea2 Go: implement Encode (embeddings) in Gitee AI driver (#14698)
### What problem does this PR solve?

The Gitee AI Go driver in `internal/entity/models/gitee.go` shipped with
a stub `Encode` method that returned `gitee, no such method`, even
though `conf/models/gitee.json` already wires the `embedding` URL
suffix. The conf also listed no embedding models, so the picker had
nothing to select.

This blocked any tenant who wanted to use Gitee AI for chat, rerank
(already working, see #14656), and embeddings from a single provider.

This PR fills the gap, mirroring the just-merged Aliyun `Encode`
(#14647):

- `internal/entity/models/gitee.go`: replace the `Encode` stub with a
real implementation.
Validates inputs, resolves the region with a default fallback, POSTs the
standard OpenAI-compatible `{"model", "input": [...]}` body to
`BaseURL[region] + URLSuffix.Embedding`, parses `data[*].embedding`
indexed by `data[*].index` so output order matches input order, handles
both `float64` and `float32` element types, and uses a 30s per-call
context deadline matching the merged `Rerank`.
- `conf/models/gitee.json`: add `BAAI/bge-m3` so the embedding picker
has something to select.

No factory change. No interface change. No URL suffix change.

Verified with `go build`, `go vet`, and `gofmt -l` : all clean.

Closes #14697

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 11:56:46 +08:00
VincentLambert
b83e2ae5a2 fix: handle missing parent chunk in retrieval_by_children (#14556)
### What problem does this PR solve?

`retrieval_by_children()` in `rag/nlp/search.py` crashes with a
`TypeError: 'NoneType' object is not subscriptable` when a parent
("mom") chunk referenced by child chunks is missing from the index.

This happens when the index is in an inconsistent state — for example
after a partial re-index, a document deletion that didn't clean up all
children, or a race condition during ingestion. `dataStore.get()`
returns `None` for the missing parent, and the subsequent access to
`chunk["content_with_weight"]` raises a `TypeError`.

**Stack trace:**
```
TypeError: 'NoneType' object is not subscriptable
  File "rag/nlp/search.py", line 792, in retrieval_by_children
    "content_with_weight": chunk["content_with_weight"],
```

### Type of change

- [x] Bug Fix

### Fix

When `dataStore.get()` returns `None` for a parent chunk, fall back to
using the child chunks directly and continue processing the remaining
parents. This preserves retrieval results for all other chunks rather
than aborting the entire query with an exception.

```python
chunk = self.dataStore.get(id, idx_nms[0], [ck["kb_id"] for ck in cks])
if chunk is None:
    chunks.extend(cks)
    continue
```

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:55:44 +08:00
Sp1kyss
e6cb9faace fix: close two security analyzer bypass paths in sandbox executor (#14690)
## Summary

Two bypass vectors in the sandbox code security analyzer allowed
malicious code to pass the safety check undetected and reach the Docker
executor.

### 1. JavaScript: template-literal bypass of `require()` block

The `SecureJavaScriptAnalyzer` regex patterns used `['"]` to match
module names, covering only single and double quotes. An attacker could
use ES6 template literals to bypass all three `require` checks:

`javascript
const cp = require(`child_process`);
async function main() {
  return cp.execSync('cat /etc/passwd').toString();
}
`

The same bypass applied to `fs` and `worker_threads`.

**Fix:** Updated all three `require` patterns from `['"]` to `['"\]` to
also match backtick template literals.

### 2. Python: `builtins` not blocked + attribute-call blind spot in
`visit_Call`

`visit_Call` only checked `ast.Name` nodes, so attribute-style calls
like `module.func()` were invisible to the analyzer. Additionally,
`builtins` was absent from `DANGEROUS_IMPORTS`. Combined, this allowed:

`python
import builtins
def main():
    builtins.exec('import os; os.system("id")')
`

Neither the import nor the exec call triggered any flag.

**Fix:** Added `builtins` to `DANGEROUS_IMPORTS` and added an
`ast.Attribute` branch to `visit_Call` so that `module.dangerous_func()`
style calls are caught alongside bare `dangerous_func()` calls.

## Tests

Added four regression tests covering each new bypass vector:
- `test_javascript_child_process_template_literal_is_rejected`
- `test_javascript_fs_template_literal_is_rejected`
- `test_python_builtins_import_is_rejected`
- `test_python_attribute_eval_call_is_rejected`

---------

Co-authored-by: bounty-hunter <bounty@hunter.local>
2026-05-11 11:46:27 +08:00
Joseff
827cceccba Fix(Go): correct Name() and region URL fallback in Aliyun driver (#14673)
### What problem does this PR solve?

Two bugs in the Aliyun Go driver:

1. **`Name()` returns `"siliconflow"`** — a copy-paste bug from when the
driver was created. `Name()` is used in error messages and log output,
so every Aliyun error incorrectly attributed itself to SiliconFlow.

2. **Silent empty URL for unknown regions in `ChatWithMessages`,
`ChatStreamlyWithSender`, and `ListModels`** — all three methods
construct the request URL as `z.BaseURL[region]` without checking
whether the key exists. For an unrecognised region this returns `""`,
producing a malformed URL like `"/chat/completions"` that the HTTP
transport rejects with a confusing error. `Encode` and `Rerank` (already
merged) correctly fall back to `"default"` and return a clear error.
This PR applies the same pattern to the remaining three methods.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 11:26:24 +08:00
Carmen Fernández Ruiz
f852a7524e fix(go): wire Google CheckConnection to ListModels (#14660)
### What problem does this PR solve?

Closes #14703

`GoogleModel.CheckConnection` currently returns a hardcoded `no such
method` error even though the Google Go driver already supports
`ListModels`. This makes provider connection checks fail regardless of
whether the configured API key can list Google models.

This PR makes `CheckConnection` call `ListModels`, adds a small API-key
guard for nil, empty, and whitespace-only keys, and keeps `ListModels`
useful by following paginated Google model responses.

### What stays unchanged

* Google model listing still uses the Google GenAI SDK with
`genai.BackendGeminiAPI`.
* Model names still come from `models.Items[*].Name`.
* `Balance`, `Encode`, chat, streaming, provider config, and factory
wiring are unchanged.

### Tests and validation

Added focused unit coverage for:

* `CheckConnection` delegating to `ListModels` and returning its error
* nil, missing, empty, and whitespace-only API key validation
* model-name passthrough from the list-models adapter
* paginated model listing, empty-result preservation, and next-page
error propagation

Validated current PR head `17ceef43515ba8c46c254dd349b9085bf26dcbea`
locally with Go 1.25.0:

* `go test ./internal/entity/models -run
'TestGoogleModel|TestCollectGoogleModelNames' -count=1 -v` - PASS
* `go test ./internal/entity/models -count=1` - PASS
* `go test -race ./internal/entity/models -count=1` - PASS
* `gofmt -w internal/entity/models/google.go
internal/entity/models/google_test.go` - PASS, no diff
* `git diff --check` - PASS

### Type of change

* [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 11:25:17 +08:00
Joseff
f4f8bed9f7 Go: implement Encode (embeddings) in Google Gemini driver (#14682)
### What problem does this PR solve?

- Implements the `Encode` method in the Google Gemini driver, which was
previously a stub returning `not implemented`
- Uses the `google.golang.org/genai` SDK's `EmbedContent` API, which
routes to the `batchEmbedContents` endpoint internally — all texts are
sent in a single request
- Adds `text-embedding-004` (max 2048 tokens) to
`conf/models/google.json`
- Response values are `[]float32` from the SDK and are cast to
`[]float64` to satisfy the `ModelDriver` interface

## Files changed

- `internal/entity/models/google.go` — full `Encode` implementation
- `conf/models/google.json` — adds `text-embedding-004` embedding model

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 11:24:21 +08:00
Ricardo-M-L
13922209e6 fix(llm): add timeout to HTTP requests in LLM integration layer (#14313)
### What problem does this PR solve?

Multiple `requests.post()` calls across the LLM integration layer lack a
`timeout` parameter. Without a timeout, a single unresponsive upstream
service can block the calling thread **indefinitely**, eventually
exhausting the thread pool and degrading the entire system.

This is a well-known issue — Python's `requests` library defaults to
`timeout=None` (infinite wait), and [the library docs explicitly
recommend](https://requests.readthedocs.io/en/latest/user/advanced/#timeouts)
always setting a timeout.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Change

Added `timeout` to all `requests.post()` calls missing it:

| File | Calls fixed | Timeout |
|------|-------------|---------|
| `rag/llm/rerank_model.py` | 9 | 30s |
| `rag/llm/embedding_model.py` | 8 | 30s |
| `rag/llm/cv_model.py` | 3 | 60s |
| `rag/llm/tts_model.py` | 2 | 60s |
| `rag/llm/sequence2txt_model.py` | 2 | 60s |

Embedding/rerank calls use 30s (lightweight API calls). Vision, TTS, and
audio transcription use 60s (heavier workloads with file uploads).

Note: other files in the codebase (e.g. `check_minio_alive`,
`check_ragflow_server_alive`) already use `timeout=10`, so this PR
brings the LLM layer in line with existing practice.

Signed-off-by: Ricardo-M-L <Sibyl_Hartmanbnb@webname.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 11:19:07 +08:00
Paras Sondhi
51b73850e1 feat: make sandbox Dockerfile mirrors optional with ARG (#14553)
### What problem does this PR solve?

Resolves #14447. *(Note: This supersedes stalled PR #14448 and
implements the requested CodeRabbitAI fixes).*

Currently, the Dockerfiles inside `agent/sandbox/sandbox_base_image`
(both Python and Node.js) have hardcoded Chinese package mirrors. This
forces the mirrors on all users globally, which causes build network
timeouts for contributors outside of China.

This PR introduces an enhancement to fix the issue by:
1. Implementing the `NEED_MIRROR` build argument in the sandbox
Dockerfiles.
2. Replacing static `ENV` instructions with conditional shell logic
inside `RUN` blocks to dynamically set the package registries.
3. Allowing the build to cleanly fall back to default global registries
(`pypi.org` and `npmjs.org`) when `--build-arg NEED_MIRROR=0` is passed.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 11:01:43 +08:00
BitToby
39a1773f7f Go: implement ListModels in Volcengine driver (#14702)
### What problem does this PR solve?

The VolcEngine Go driver in `internal/entity/models/volcengine.go`
shipped with a
`ListModels` stub that returned `volcengine, no such method`.
`conf/models/volcengine.json`
also did not declare a `models` URL suffix, so the model picker had
nothing to call even
if the method body were filled in.

A tenant who configured Volcengine (Doubao / Ark) as a provider could
not see the list of
available endpoints from the RAGFlow UI. Several other Go drivers
already implement
`ListModels` against the OpenAI-compatible `/models` endpoint (deepseek,
gitee, nvidia,
openai, siliconflow), so the interface and pattern are well-established.

This PR fills the gap.

### What this PR includes

* `conf/models/volcengine.json`: declare the `models` URL suffix
alongside the existing
  `chat`, `files`, and `embedding` entries. The Ark v3 API exposes
`https://ark.cn-beijing.volces.com/api/v3/models`, so the suffix is just
`models`.
* `internal/entity/models/volcengine.go`: replace the `ListModels` stub
with a real
implementation. Reuses the package-level `DSModelList` / `DSModel` types
that
DeepSeek, Gitee, and SiliconFlow already use to parse the
OpenAI-compatible models
  response shape.

No factory change. No interface change.

### How the driver works

* Resolves the region with a default fallback, the same way the other
VolcEngine methods
  in this driver already do.
* Builds the URL from `BaseURL[region] + URLSuffix.Models`, with
`strings.TrimSuffix` on
  the base to keep the join robust.
* Issues a `GET` with optional `Authorization: Bearer <api_key>` (the
header is omitted
when no key is configured, mirroring the existing NVIDIA `ListModels`).
* Reads the response body once, surfaces a non-200 with the upstream
status line plus
  body, and parses the JSON via the shared `DSModelList` type.
* Returns the model id list in input order. When the response includes
an `owned_by`
field, the entry is rendered as `id@owned_by`, matching the convention
used by the
  other Go drivers.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


### How was this tested?

* `go build ./internal/entity/models/...` returns exit 0.
* `go vet ./internal/entity/models/...` is clean.
* `gofmt -l internal/entity/models/volcengine.go` is clean.
* The full method set on `VolcEngine` still matches the `ModelDriver`
interface.
* Endpoint reachability check: `GET
https://ark.cn-beijing.volces.com/api/v3/models`
returns `401 Unauthorized` without an API key, confirming the path
exists and accepts
  Bearer authentication.
* Pattern parity with DeepSeek, Gitee, NVIDIA, and SiliconFlow
`ListModels`.

Fixes #14701

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 10:59:18 +08:00
VincentLambert
08bb53bbb1 Feat: add BedrockCV for vision/image2text inference via LiteLLM (#14705)
## Summary

- `CvModel["Bedrock"]` was absent from `rag/llm/cv_model.py`, causing
`model_instance()` to return `None` when a Bedrock model was used as a
PDF parser — even after correct model resolution.
- This PR adds `BedrockCV`, enabling Bedrock vision models (e.g.
`amazon.nova-pro-v1:0`, `anthropic.claude-3-5-sonnet`) to be used as PDF
parsers.

## What problem does this PR solve?

When a Bedrock model is selected as the PDF parser in a knowledge base,
ingestion failed with:

```
'LiteLLMBase' object has no attribute 'describe_with_prompt'
```

The root cause: `LiteLLMBase` (the Bedrock chat implementation) was the
only registered handler for the Bedrock factory. It does not implement
`describe_with_prompt`. `CvModel` had no Bedrock entry, so
`model_instance()` returned `None` for `image2text` requests.

## Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Changes

**`rag/llm/cv_model.py`**

Adds `BedrockCV(Base)` with `_FACTORY_NAME = "Bedrock"`:

- Uses `litellm.completion` with the `bedrock/` prefix (consistent with
`LiteLLMBase`)
- Parses AWS credentials from the JSON key assembled by `add_llm`
(`auth_mode`, `bedrock_ak`, `bedrock_sk`, `bedrock_region`,
`aws_role_arn`)
- Supports three auth modes: `access_key_secret`, `iam_role` (via STS
`assume_role`), and default credential chain (IRSA, instance profile)
- Implements `describe_with_prompt` and `describe`

## Test plan

- [ ] Configure a Bedrock vision model (e.g. `amazon.nova-pro-v1:0`)
with valid AWS credentials
- [ ] Select it as PDF parser in a knowledge base
- [ ] Verify ingestion of a PDF document completes without errors
- [ ] Verify `CvModel["Bedrock"]` resolves to `BedrockCV`

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 10:29:58 +08:00
Ahmad Intisar
3c4d1da98f Feature/table parser column roles (#13710)
### What problem does this PR solve?

The table file parser (CSV/Excel) currently treats all columns
identically — every column is both vectorized (embedded in chunk text)
and stored as filterable metadata. There's no way for users to control
which columns should be searchable by semantic meaning versus which
should only be filterable attributes.

For example, when ingesting a news articles CSV with columns like title,
content, country, category, source, etc., the embedding includes
metadata fields like country: Brazil and source: Reuters in the chunk
text, which dilutes the semantic quality of the embedding without adding
retrieval value.

The RDBMS connector (MySQL/PostgreSQL) already supports content_columns
/ metadata_columns, but this capability was missing for file-based table
ingestion.

This PR adds column-level control (vectorize / metadata / both) for the
table file parser, following RAGFlow's existing patterns.

Backward compatible: Datasets without table_column_roles or with
table_column_mode: auto behave exactly as before (all columns = both).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 10:06:04 +08:00
Igor Ilinskii
889aba6a32 fix base_url handling in HuggingfaceRerank (#14555)
### What problem does this PR solve?

HuggingfaceRerank.post() unconditionally prepends `http://` to base_url,
which already contains a protocol. This creates invalid URLs like
http://http://127.0.0.1:8080/rerank, breaking all requests. The fix
normalizes URL handling to match the rest of the codebase, removing
redunant `http://`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Related Issues
- #7318 
- #7796

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-11 10:04:40 +08:00
Tim Wang
ed01ac9994 Fix: resolve template strings in tool component parameters (#14601)
## Summary

- Tool-type components (Email, Invoke, etc.) fail to resolve template
strings that mix variable references with literal text in their
parameters.
- This adds template string resolution to `get_input()` in
`ComponentBase`, reusing existing `get_input_elements_from_text()` and
`string_format()` methods.

## Problem

`get_input()` in `ComponentBase` handles two cases:
1. **Pure reference** (`{Component:ID@field}`) — resolved via
`is_reff()` + `get_variable_value()`
2. **Literal value** — passed through as-is

But template strings like `{UserFillUp:X@name}@duke.edu` or `Question
from {Agent:Y@topic}` fall through to the literal branch because
`is_reff()` returns `False` (it expects the entire string to be a single
reference). The unresolved template is passed directly to the tool.

This affects **all** tool components (Email, Invoke, etc.) that need
mixed reference + text parameters — for example, constructing email
addresses or subjects dynamically.

## Fix

```python
# In get_input(), between is_reff check and literal fallback:
elif isinstance(v, str) and re.search(self.variable_ref_patt, v):
    elements = self.get_input_elements_from_text(v)
    kv = {k: e.get('value', '') for k, e in elements.items()}
    self.set_input_value(var, self.string_format(v, kv))
```

This reuses `get_input_elements_from_text()` and `string_format()` which
are already used by `Message` components for the same purpose. The fix
only activates when the string contains at least one variable reference
pattern but is not a pure reference.

## Test plan

- [x] Pure references (`{Component:ID@field}`) still resolve correctly
via `is_reff()` path
- [x] Literal values without references pass through unchanged
- [x] Template strings like `{ref}@duke.edu` resolve the reference and
keep the literal suffix
- [x] Template strings like `Question from {ref}` resolve correctly
- [x] Multiple references in one string (`{ref1} and {ref2}`) both
resolve
- [x] Message components unaffected (they use their own template
resolution in `_run`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 10:01:41 +08:00
Mehmet Karakose
7ec87f7cb7 fix(auth): fall back to session-based auth in _load_user (#14569)
## Summary

Closes #13663.

OAuth / OIDC callbacks call `login_user(user)` which writes `_user_id`
into the session cookie, but `_load_user()` in `api/apps/__init__.py`
only ever looked at the `Authorization` header. The SPA's response
interceptor wipes the Authorization value from `localStorage` on the
first 401 it sees — meaning that during the post-redirect window after
an OAuth login, a single transient 401 sends every subsequent request
back to the login page even though `login_user()` had already
established a perfectly good server-side session.

The reporter's analysis traces this all the way through the redirect →
`navigate('/')` → first request → empty header → 401 → `removeAll()` →
infinite-redirect-to-login chain.

## What changed

- New `_load_user_from_session()` helper that reads
`session["_user_id"]`, looks up the user in `UserService` (with the same
`StatusEnum.VALID` and `access_token` checks already used elsewhere),
and assigns `g.user`.
- Every `return None` path in `_load_user()` now routes through that
helper before giving up:
  - missing `Authorization` header
  - malformed `bearer ` prefix
  - empty / too-short JWT payload
  - JWT signature failure
  - JWT-resolved user not found / has no `access_token`
  - `APIToken.query()` fallback exhausted

The JWT and API-token paths still take precedence — the session is only
consulted when those can't authenticate the request. So existing
local-login and SDK callers see no behaviour change; only OAuth / OIDC
users that hit the original race now stay logged in.

The Bearer-prefix issue called out in #13663 (lines 103-110) is already
handled in the current code, so this PR only addresses the second half
of the report.

## Test plan

- [ ] Configure OIDC under `oauth` in `service_conf.yaml`
- [ ] Click the OIDC login button, complete auth at the IdP
- [ ] Confirm that navigating between pages no longer bounces back to
`/login`
- [ ] Confirm local email/password login still issues + accepts JWTs
- [ ] Confirm SDK/API key callers still authenticate via `Authorization:
Bearer <api-token>`

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 09:59:52 +08:00
很拉风的James
6cb4bc2947 Fix: Radio.Group cloneElement crashes on non-element children (#14407)
### What problem does this PR solve?

`Radio.Group` in `web/src/components/ui/radio.tsx` injects the parent's
`disabled` prop into each child via `React.cloneElement` with
`as React.ReactElement` and no validation.

This throws at runtime when a consumer passes strings, numbers, `null`,
`false`, or other non-element nodes, while the cast hides the unsafe
access from TypeScript.

Use `React.isValidElement<RadioProps>(child)` as a type guard before
calling `cloneElement`. Non-element children pass through unchanged,
and `child.props` access becomes type-checked without an `as` cast.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 09:54:42 +08:00
Panda Dev
6bfe0f9a10 Go: implement Encode (embeddings) in OpenAI driver (#14630)
### What problem does this PR solve?

The OpenAI Go driver landed in #14605 with chat, list models, and check
connection. Encode was left as a stub that returns \`not implemented\`.

\`conf/models/openai.json\` already lists three embedding models out of
the box:

- text-embedding-ada-002
- text-embedding-3-small
- text-embedding-3-large

So a tenant who picked one of these in the Go layer could not actually
run an embedding call. This PR fills the gap.

### What this PR includes

- \`conf/models/openai.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the driver can build the URL from config. This
matches the \`URLSuffix.Embedding\` field used by other drivers
(siliconflow, zhipu-ai).
- \`internal/entity/models/openai.go\`: replace the Encode stub with a
real implementation that POSTs to \`/v1/embeddings\`. Adds a small local
response type \`openaiEmbeddingResponse\`.

No factory change. No interface change.

### How the implementation works

- Validate \`apiConfig\` and the API key, validate the model name. Use
the existing \`baseURLForRegion\` helper so an unknown region fails fast
with a clear error.
- Wrap the request with \`context.WithTimeout(nonStreamCallTimeout)\` so
the call has a clear deadline. Same pattern as \`ChatWithMessages\` and
\`ListModels\` already use in this file.
- Send all input texts in one request. The OpenAI API accepts the
\`input\` field as an array.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\` so the output order matches the input order
even if the API returns items in a different order.
- Handle both \`float64\` and \`float32\` element types, the way the
SiliconFlow driver does.
- An empty input slice returns \`[][]float64{}\` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still nil, return a clear error so the caller does not silently use a
zero vector.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`OpenAIModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Encode implementation
(\`internal/entity/models/siliconflow.go\`).

Closes #14629

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-10 10:31:37 +08:00
Jin Hai
048ec2fc5c Go: fix siliconflow rerank issue (#14743)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 20:45:53 +08:00
Jin Hai
779cd83862 Go: fix Baidu rerank issue (#14742)
### What problem does this PR solve?

top_n is missing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 20:05:57 +08:00
Hunnyboy1217
782084780e feat(connectors): ETag-based bypass for incremental S3 ingestion (#14628) (#14677)
### What problem does this PR solve?

S3-family connector syncs currently re-download every in-window object
just so we can compute `xxhash128(blob)` and compare against
`Document.content_hash`. Anything that bumps `LastModified` without
changing bytes (`aws s3 cp` touches, bucket re-encryption, etc.) pays
full bandwidth and re-parses files that didn't actually change. #14628
covers the broader incremental-ingestion redesign; this PR is the first
slice.

The fix is a pre-listing short-circuit. `BlobStorageConnector` (S3 / R2
/ GCS / OCI / S3-compat) now implements a new `FingerprintConnector`
interface: `list_keys()` paginates `list_objects_v2` and yields
`KeyRecord(key, fingerprint)` where `fingerprint = xxhash128(ETag)`. The
orchestrator joins those against the connector's existing `{doc_id:
content_hash}` map and only calls `get_value(key)` when the fingerprint
differs. Unchanged keys are skipped entirely — no `GetObject`, no
re-parse.

No DDL. xxhash128(ETag) is 32 hex chars and reuses the existing
`Document.content_hash` column per @yingfeng's suggestion; the connector
decides at listing time whether to populate it. Local uploads and
connectors that don't opt in fall through to the existing post-download
`xxhash128(blob)` path with no behavior change.

This is PR-1 of a 4-PR series — full design lives on #14628. Subsequent
PRs extend tier 1 to local FS / WebDAV / Dropbox / Seafile / RDBMS
(PR-2), wire up tier 2 cursor connectors with `SyncLogs.next_checkpoint`
(PR-3), and unify deletion via `KeyRecord(deleted=True)` reconciliation
(PR-4). Holding those back keeps this PR additive and reviewable on its
own.

#### Files touched

- `common/data_source/models.py` — new `KeyRecord`; optional
`fingerprint` on `Document`
- `common/data_source/interfaces.py` — `IncrementalCapability` enum,
`FingerprintConnector` ABC
- `common/data_source/blob_connector.py` — `BlobStorageConnector`
implements `FingerprintConnector`; per-object download factored into
`_build_document_from_obj()` so `_yield_blob_objects`, `list_keys`,
`get_value` all share it
- `rag/svr/sync_data_source.py` —
`_BlobLikeBase._fingerprint_filtered_generator` does the bypass loop;
`_run_task_logic` plumbs `doc.fingerprint` into the upload dict
- `api/db/services/document_service.py` —
`list_id_content_hash_map_by_kb_and_source_type()` helper
- `api/db/services/connector_service.py` + `file_service.py` —
fingerprint flows through `duplicate_and_parse → upload_document` and
lands in `content_hash`
- `test/unit_test/common/test_blob_connector_fingerprint.py` — 14 tests
covering ETag normalization (single-part, multipart, quoted, empty),
`list_keys()` not calling `GetObject`, `get_value()` materializing with
fingerprint, deterministic/stable fingerprints, and the bypass loop
asserting `GetObject` is *not* called on a match

#### Worth flagging for review

Old `_BlobLikeBase._generate` called `poll_source(start, now)` with a
`LastModified` window when `poll_range_start` was set. New code uses
`_fingerprint_filtered_generator` (full bucket listing + fingerprint
compare) outside of explicit `reindex=1`. Strictly better for
unchanged-bucket cases since it skips `GetObject`, but it does mean
every sync now does a full `list_objects_v2` paginate. Should still be
cheap for most buckets — flagging in case anyone has a very large bucket
where the time-window filter was meaningful.

On migration: existing rows have `content_hash = xxhash128(blob)` from
the old code. The first sync after this lands sees ETag-derived
fingerprints that don't match, re-fetches every object once, and writes
the new fingerprint. From the second sync onward the bypass works as
expected. "Slow day one, fast every day after." A `fingerprint_backfill:
trust` opt-out is sketched in the design doc but not in this PR.

#### Test plan

- [x] `uv run ruff check` — clean on all 8 touched files
- [x] `uv run pytest
test/unit_test/common/test_blob_connector_fingerprint.py -v` — 14 passed
- [x] Broader unit-test suite — no regressions in anything I touched
- [ ] Manual smoke against a real S3 bucket — configure a connector, run
sync twice, expect the second sync to log `bypassed=N, fetched=0` and no
`GetObject` calls in CloudTrail / bucket access logs
- [ ] Manual smoke with `reindex=1` — confirm the full re-download path
still works

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-05-09 20:03:56 +08:00
Haruko386
7931b693dc Go: implement provider: Baidu (#14741)
### What problem does this PR solve?

This PR completes the Baidu Qianfan provider integration in RAGFlow.

**The following functionalities are now supported:**

- [x] Chat / Think Chat / Stream Chat / Stream Think Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

-----

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'embedding-3@test@zhipu-ai' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 16        | 0     |
| 16        | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'qwen3-reranker-4b@test@baidu' top 2;
+-------+---------------------+
| index | relevance_score     |
+-------+---------------------+
| 0     | 0.974821150302887   |
| 1     | 0.14223189651966095 |
| 2     | 0.08632347732782364 |
+-------+---------------------+

RAGFlow(user)> think chat with 'deepseek-v3.2@test@baidu' message 'who r u'
Thinking: Hmm, the user is asking for a simple introduction. This is straightforward – no need for overcomplication. 

I should give a clear, friendly response that covers my basic identity as an AI assistant, my purpose, and my capabilities. Keeping it concise but informative is key here. 

Mentioning my creator Anthropic adds credibility, and ending with an offer to help invites further interaction. No need for technical details unless the user asks later.
Answer: Hello! I'm an AI assistant created by Anthropic, designed to help with a wide variety of tasks. You can think of me as a helpful digital companion—I can answer questions, assist with writing, help solve problems, provide explanations, and engage in conversation on many topics. I'm here to help with whatever you need! How can I assist you today?
Time: 8.103902

RAGFlow(user)> stream think chat with 'deepseek-v3.2@test@baidu' message 'who r u'
Thinking: mm, the user is asking "who r u" with casual spelling. This is a straightforward identity question. should give a clear, friendly introduction without overcomplicating it. Can start with my core function as an AI assistant, mention my creator, and briefly state my key capabilities. response should be welcoming and invite further interaction since this seems like an introductory question. Keeping it concise but covering the essentials: who I am, what I do, and how I can help.
Answer: ! I am DeepSeek, an AI assistant created by DeepSeek Company. I'm designed to help answer questions, provide information, assist with various tasks, and engage in conversations on a wide range of topics. I'm here to assist you with whatever you need - whether it's answering questions, helping with analysis, writing, coding, or just having a friendly chat!Is there anything specific I can help you with today? 😊
Time: 7.219703

RAGFlow(user)> list supported models from 'baidu' 'test'
+--------------------------------------+
| model_name                           |
+--------------------------------------+
| ernie-3.5-8k-preview                 |
| ernie-4.0-8k                         |
| ernie-4.0-turbo-8k-latest            |
| ernie-4.0-turbo-8k-preview           |
| ernie-4.0-8k-preview                 |
| ernie-speed-pro-128k                 |
| ernie-char-fiction-8k                |
| ernie-3.5-8k                         |
| ernie-3.5-128k                       |
| ernie-lite-pro-128k                  |
| ernie-novel-8k                       |
| ernie-4.0-turbo-8k                   |
| ernie-4.0-turbo-128k                 |
| ernie-4.0-8k-latest                  |
| irag-1.0                             |
| ...........                          |
| glm-5.1                              |
| ernie-image-turbo                    |
| deepseek-v4-pro                      |
| deepseek-v4-flash                    |
| ernie-5.1                            |
+--------------------------------------+

RAGFlow(user)> check instance 'test' from 'baidu'
SUCCESS
```

Additionally, this PR fixes an incorrect error message typo:

Before:

```go
fmt.Errorf("API requestssss failed with status %d: %s : %s", ...)
```

After:

```go
fmt.Errorf("API request failed with status %d: %s", ...)
```

This PR mainly improves provider compatibility, API completeness, and
runtime stability.

### Type of change

* [x] Bug Fix (non-breaking change which fixes an issue)
* [x] New Feature (non-breaking change which adds functionality)
* [x] Refactoring
2026-05-09 19:21:13 +08:00
Liu An
57b24be6d6 Docs: Update version references to v0.25.2 in READMEs and docs (#14731)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.1 to v0.25.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-09 19:06:05 +08:00
writinwaters
a3de873617 Docs: Updated release date (#14740)
### What problem does this PR solve?

Updated v0.25.2 release date.

### Type of change


- [x] Documentation Update
2026-05-09 18:49:33 +08:00
euvre
f4b8f53b6d Fix: restore embedding model switching for datasets with existing chunks (#14732)
### What problem does this PR solve?

## Problem

During the REST API refactoring (#13690), the
`/api/v2/kb/check_embedding` endpoint was removed and never migrated to
the new RESTful structure. The frontend was pointed to the
`/api/v1/datasets/{id}/embedding` endpoint (which is `run_embedding` — a
completely different function). Additionally, a hard guard was
introduced that rejects any `embd_id` change when `chunk_num > 0`,
making it impossible to switch embedding models on datasets with
existing chunks.

## Root Cause

1. **Missing endpoint**: The old `check_embedding` logic (sample random
chunks, re-embed with the new model, compare cosine similarity) was not
carried over to the new REST API service layer.
2. **Wrong frontend URL**: `checkEmbedding` in `api.ts` pointed to
`/datasets/{id}/embedding` (`run_embedding`) instead of a dedicated
check endpoint.
3. **Overly restrictive guard**: `dataset_api_service.py` line 310
blocked all `embd_id` updates when `chunk_num > 0`. This check did not
exist in the pre-refactor code — it was incorrectly introduced during
the refactor.

## Changes

### Backend

- **`api/apps/services/dataset_api_service.py`**
  - Remove the `chunk_num > 0` hard guard on `embd_id` updates
- Add `check_embedding()` service function: samples random chunks,
re-embeds them with the candidate model, computes cosine similarity,
returns compatibility result (avg ≥ 0.9 = compatible)
  - Add `import re` for the `_clean()` helper

- **`api/apps/restful_apis/dataset_api.py`**
- Add `POST /datasets/<dataset_id>/embedding/check` endpoint following
the new REST API conventions
  - Clean up unused top-level imports (`random`, `re`, `numpy`)

### Frontend

- **`web/src/utils/api.ts`**
- Fix `checkEmbedding` URL from `/datasets/${datasetId}/embedding` →
`/datasets/${datasetId}/embedding/check`

### Tests

-
**`test/testcases/test_http_api/test_dataset_management/test_update_dataset.py`**
- Update `test_embedding_model_with_existing_chunks` to assert success
(`code == 0`) instead of expecting the old `102` error

-
**`test/testcases/test_web_api/test_dataset_management/test_dataset_sdk_routes_unit.py`**
- Update `test_update_route_branch_matrix_unit` to assert
`RetCode.SUCCESS` when updating `embd_id` on a chunked dataset,
replacing the old `chunk_num` error assertion

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-05-09 18:48:57 +08:00
buua436
330257b611 Fix: Add legacy system healthz route (#14738)
### What problem does this PR solve?

Add legacy system healthz route

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:49:26 +08:00
Jin Hai
17d71e5d79 Go CLI: embed and rerank (#14735)
### What problem does this PR solve?

```
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'embedding-3@test@zhipu-ai' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 16        | 0     |
| 16        | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'rerank@test@zhipu-ai' top 2;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 1               |
| 2     | 0.99999976      |
+-------+-----------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 17:41:54 +08:00
Lynn
efe6d23d61 Fix: handle id as keyword (#14729)
### What problem does this PR solve?

Update mapping.json to treat id as a keyword.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:41:08 +08:00
chanx
8ac14b597f Fix: Some bugs (#14734)
### What problem does this PR solve?

Fix: Some bugs
- Error during batch modification of metadata in the Knowledge Base
- Manually configured metadata is not displayed in search settings

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:40:22 +08:00
akie
c11650bb4c Fix IDOR: Add permission checks to file ancestry endpoints (#14725)
Close #14292

## Issue

File ancestry endpoints return folder metadata without validating tenant
permissions, allowing any authenticated user to query arbitrary
`file_id` values across tenant boundaries.

## Affected Endpoints
- `GET /v1/file/parent_folder?file_id={file_id}`
- `GET /v1/file/all_parent_folder?file_id={file_id}`  
- `GET /api/v1/files/{id}/ancestors`

## Root Cause

These endpoints **skip the permission check** that other file operations
(Delete, Download, Move) perform.

## Expected Permission Check

All file operations should follow this 3-step validation:

- Check file.tenant_id
- Check if user_id belongs to this tenant (via user_tenant join table)
- Check KB permission type (team permission)


**Code reference:** This is implemented in `checkFileTeamPermission()`
and used by Delete/Download/Move, but **missing** from
GetParentFolder/GetAllParentFolders.

## Reproduction

```bash
# User B (tenant: BBB) accessing User A's file (tenant: AAA)
curl -H "Authorization: Bearer USER_B_TOKEN" \
  "http://localhost:9384/v1/file/parent_folder?file_id=AAA_FILE_123"

# Result: Returns User A's folder metadata 
# Expected: "No authorization." 
Fix
Pass userID from handler to service and call checkFileTeamPermission() — same as Download/Delete/Move handlers.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-09 16:03:23 +08:00
writinwaters
6465753968 Docs: Added v0.25.2 release notes (#14727)
### What problem does this PR solve?

Added v0.25.2 release notes.

### Type of change

- [x] Documentation Update
2026-05-09 15:13:01 +08:00
Magicbook1108
f7e8c39dcc Fix: filter api in dataset document (#14728)
### What problem does this PR solve?

Fix: filter api in dataset document

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 14:45:40 +08:00
buua436
de2abe9ed8 Fix: tag parser id (#14724)
### What problem does this PR solve?
tag parser id
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 14:29:09 +08:00
Haruko386
ee0de58204 Go: implement provider: HuggingFace (#14722)
### What problem does this PR solve?

Implement `HuggingFace` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-09 13:36:03 +08:00
jony376
3b6eeabb09 Fix: private dataset authorization bypass in shared dataset access checks (#14645)
### Related issues
Closes #14644

### What problem does this PR solve?

This PR fixes an authorization bug where datasets marked with
`permission = me` could still be accessed by other members of the same
tenant through APIs that relied on `KnowledgebaseService.accessible()`
or `DocumentService.accessible()`.

Before this change, those shared access helpers only checked tenant
membership and did not enforce the dataset's permission mode. As a
result, a non-owner who knew a private `dataset_id` could still reach
downstream document and chunk operations even though the dataset was
intended to be owner-only.

This change updates the central access checks so that:

- dataset owners always retain access
- joined tenant members only get access when the dataset permission is
`TEAM`
- private datasets with `permission = me` remain inaccessible to
non-owners
- document-level access follows the same dataset permission rules

The PR also adds regression coverage for private-vs-team dataset access
behavior.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- Added
`test/unit_test/api/db/services/test_dataset_access_permissions.py`
- Attempted to run: `python -m pytest
test\\unit_test\\api\\db\\services\\test_dataset_access_permissions.py
-q`
- Local execution in this workspace is currently blocked during test
collection because the environment is missing the `strenum` dependency

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: jony376 <jony376@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
Co-authored-by: d 🔹 <liusway405@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Magicbook1108 <newyorkupperbay@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: sxxtony <166789813+sxxtony@users.noreply.github.com>
Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
Co-authored-by: Baki Burak Öğün <63836730+bakiburakogun@users.noreply.github.com>
Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
Co-authored-by: Panda Dev <56657208+pandadev66@users.noreply.github.com>
Co-authored-by: Haruko386 <tryeverypossible@163.com>
Co-authored-by: D2758695161 <13510221939@163.com>
Co-authored-by: Hunter <hunter@yitong.ai>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: buua436 <sz_buua@foxmail.com>
Co-authored-by: web-dev0521 <jasonpette1783@gmail.com>
Co-authored-by: Tim Wang <38489718+wanghualoong@users.noreply.github.com>
Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: qinling0210 <88864212+qinling0210@users.noreply.github.com>
Co-authored-by: dale053 <star05223@outlook.com>
2026-05-09 13:30:14 +08:00
Ricardo-M-L
1046042e01 fix(llm): replace mutable default gen_conf={} with None + defensive copy (#14566)
### What

19 methods across `rag/llm/chat_model.py` and `rag/llm/cv_model.py`
declare `gen_conf={}` (or `gen_conf: dict = {}`) as a parameter default
and then mutate `gen_conf` in place — typically `del
gen_conf["max_tokens"]`, `gen_conf["penalty_score"] = ...`, or
`gen_conf.pop(...)` as part of provider-specific normalization.

### The two bugs in this pattern

**1. Mutable default argument (Python footgun).** Python evaluates
default values **once** at function-definition time, so the single `{}`
dict is *shared* across every caller that doesn't pass `gen_conf`. The
first such call's mutations leak into the default seen by every
subsequent call.

```python
# Before
def chat_streamly(self, system, history, gen_conf={}, **kwargs):
    if "max_tokens" in gen_conf:
        del gen_conf["max_tokens"]   # mutates the SHARED default dict
    ...
```

After call N with `max_tokens` set, call N+1 that omits `gen_conf` no
longer sees `max_tokens` — even though the caller never touched it.

**2. Caller-dict pollution.** When the caller *does* pass a `gen_conf`
dict, the same in-place mutations modify the caller's dict. A reused
`gen_conf` (very common for chat-loop callers that build the config once
and pass it on every turn) silently loses `max_tokens`,
`presence_penalty`, etc. after the first round.

### The fix

In every affected method:

- Change `gen_conf={}` (or `gen_conf: dict = {}`) → `gen_conf=None`.
- Add `gen_conf = dict(gen_conf or {})` as the first statement of the
body so all subsequent mutations operate on a fresh local copy.

```python
# After
def chat_streamly(self, system, history, gen_conf=None, **kwargs):
    gen_conf = dict(gen_conf or {})
    if "max_tokens" in gen_conf:
        del gen_conf["max_tokens"]   # local copy — safe
    ...
```

This is byte-for-byte identical provider-side behavior for callers that
already pass a fresh `gen_conf` per call. The new `dict(...)` copy is
O(small constant) per call.

### Files changed

- `rag/llm/chat_model.py` — 17 methods
- `rag/llm/cv_model.py` — 2 methods

### Tests

Adds `test/unit_test/rag/llm/test_gen_conf_no_mutable_default.py` — an
`ast`-based regression guard that walks both modules and asserts no
parameter named `gen_conf` ever has a mutable literal (`{}` or `[]`) as
its default. The test caught **five additional `gen_conf: dict = {}`
sites** that an initial `gen_conf={}` text grep had missed (annotated
parameters with whitespace), and would fail again if the pattern is ever
reintroduced.

```
$ pytest test/unit_test/rag/llm/test_gen_conf_no_mutable_default.py -v
============================== 3 passed in 0.04s ===============================
```

`ruff check` passes on all touched files.

### Notes

- This PR is intentionally focused on **just** the `gen_conf` default +
copy fix. There's a related (but separate) `history.insert(0, ...)`
pattern in the same files that mutates the caller's history list in 12
places — left for a follow-up so this PR stays mechanical and easy to
review.

### Latest revision (`700bb54a7`) — addresses CodeRabbit review

- Type annotation: `gen_conf: dict = None` → `gen_conf: dict | None =
None` (5 occurrences in `chat_model.py`). The old annotation was a
static-checker mismatch since `None` isn't a `dict`.
- Regression test: the AST check accessed `default.keys` directly.
`ast.List` has no `.keys` attribute — a future `gen_conf=[]` would crash
with `AttributeError` instead of being caught. Use `getattr` for both
`.keys` (Dict) and `.elts` (List). Manually verified the updated check
correctly catches both `gen_conf={}` and `gen_conf=[]` while ignoring
`gen_conf=None` and non-empty literals.

---------

Co-authored-by: Ricardo <ricardo@example.com>
2026-05-09 13:11:44 +08:00
Wang Qi
42504fa18c Bugfix: keep document api backward compatible (#14726)
### What problem does this PR solve?

Bugfix: keep document api backward compatible

Fix 1: https://github.com/infiniflow/ragflow/issues/14634 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 13:03:09 +08:00
Yingfeng
3234a0ef35 Update README (#14723)
### Type of change

- [x] Documentation Update
2026-05-09 11:28:44 +08:00
VincentLambert
4f3711d37f fix: handle missing 'total' key causing KeyError in deep research retrieval (#13942)
## Summary

- When KB retrieval fails (e.g. ES `AssertionError` on empty
`index_names`), `kbinfos` falls back to a dict without a `total` key
- `_async_update_chunk_info` then iterates over `chunk_info.keys()`
(which includes `total`) and tries `kbinfos['total']`, raising a
`KeyError`
- This error surfaces when using Tavily web retrieval in a chat with no
knowledge base attached

## Changes

- Add `'total': 0` to all default `kbinfos` dicts in
`_retrieve_information`
- Add `setdefault('total', 0)` guard after successful KB retrieval to
handle cases where the retrieval result omits the key
- Accumulate `total` correctly in the merge branch of
`_async_update_chunk_info`

## Test plan

- [ ] Start a chat with Tavily configured and no knowledge base
- [ ] Verify no `KeyError: 'total'` is raised
- [ ] Verify Tavily results are returned correctly

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:57:51 +08:00
VincentLambert
870bc59365 Fix: Bedrock api_key overridden by existing-key fallback in add_llm (#14707)
## Summary

- Adding a Bedrock model from the frontend fails with `Fail to access
model(Bedrock/<model>).Expecting value: line 1 column 1 (char 0)`.
- The assembled Bedrock JSON credentials are silently replaced by `"x"`
before the connection test, causing `json.loads("x")` to raise a
`JSONDecodeError`.

## What problem does this PR solve?

Commit `050113482` introduced a fallback in `add_llm()` that reuses the
existing DB key when `req.get("api_key") is None`:

```python
if req.get("api_key") is None:
    api_key = existing_api_key if existing_api_key is not None else "x"
```

For Bedrock, credentials are sent as separate fields (`auth_mode`,
`bedrock_ak`, `bedrock_sk`, `bedrock_region`, `aws_role_arn`) — the
frontend does not send an `api_key` field. The function correctly
assembles the JSON key:

```python
api_key = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])
```

But since `req.get("api_key")` is `None`, the override immediately
replaces `api_key` with `"x"` (or a stale DB value). `LiteLLMBase` then
calls `json.loads("x")` for Bedrock auth → `JSONDecodeError`.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

## Changes

**`api/apps/llm_app.py`**

Write the assembled key into `req["api_key"]` so the `None` check
evaluates to `False` and the override is skipped — consistent with how
`Tencent Cloud` is already handled.

```python
# Before
api_key = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])

# After
req["api_key"] = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])
api_key = req["api_key"]
```

## Test plan

- [ ] Configure a Bedrock provider in Model Providers with valid AWS
credentials
- [ ] Add a Bedrock chat model — verify no `Expecting value` error
- [ ] Update the same model — verify the existing key is reused
correctly when credentials fields are left empty

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:54:58 +08:00
Xing Hong
c428187350 Fix: validate kb_ids as UUIDs before SQL interpolation in use_sql (#14087)
### What problem does this PR solve?

The use_sql() function in dialog_service.py constructed SQL WHERE
clauses and Infinity table names by directly interpolating kb_id values
using Python f-strings, with no validation of the input values. A
malformed or maliciously crafted kb_id (introduced via a compromised
admin account or a separate injection vector) could alter the structure
of the generated SQL query, potentially leading to unauthorized data
access or data manipulation.

This PR adds strict UUID format validation for all kb_id values before
they are interpolated into any SQL string, causing requests with invalid
IDs to fail fast with a ValueError rather than executing a tampered
query.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-09 10:52:06 +08:00
VincentLambert
c44dc85143 Fix: IMAGE2TEXT→CHAT fallback with model_type normalization in tenant_model_service (#14704)
## Summary

- When a model is registered as `chat` in `tenant_llm` but has the
`IMAGE2TEXT` tag in `llm_factories.json`, requesting it as `image2text`
(e.g. PDF parser) fails with `Tenant Model with name <model> and type
image2text not found`.
- After resolution via the new fallback, the returned
`config_dict["model_type"]` was still `"chat"`, causing
`tenant_llm_service.model_instance()` to instantiate `ChatModel` instead
of `CvModel` — breaking `describe_with_prompt` at ingestion time.

## What problem does this PR solve?

RAGFlow already has a `CHAT→IMAGE2TEXT` fallback: when a chat model is
not found, it retries with `image2text`. The symmetric fallback
(`IMAGE2TEXT→CHAT`) was missing.

This matters for multimodal models declared as `model_type: "chat"` with
an `IMAGE2TEXT` tag in `llm_factories.json` (e.g. models added after
tenant creation, or providers where a single model serves both
purposes). The frontend PDF parser selector correctly surfaces these
models via the `IMAGE2TEXT` tag, but the backend fails to resolve them
at runtime.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

## Changes

**`api/db/joint_services/tenant_model_service.py`**

1. Add `IMAGE2TEXT→CHAT` fallback in
`get_model_config_by_type_and_name`: when an `image2text` model is not
found in `tenant_llm`, retry with `chat` — but only if the `llm` table
confirms `IMAGE2TEXT` capability via the `tags` field. This mirrors the
philosophy of the existing `CHAT→IMAGE2TEXT` fallback: substitution is
only allowed when the model has declared the required capability.

2. Normalize `config_dict["model_type"]` to `image2text` after the
fallback, so the caller (`model_instance`) correctly routes to `CvModel`
instead of `ChatModel`.

3. Extend the type validation guard to allow `(requested=image2text,
found=chat)` alongside the existing `(requested=chat, found=image2text)`
exception.

## Test plan

- [ ] Add a model with `model_type=chat` and `tags` containing
`IMAGE2TEXT` to a tenant
- [ ] Select it as PDF parser in a knowledge base
- [ ] Verify ingestion succeeds without `image2text not found` or
`describe_with_prompt` errors
- [ ] Verify the same model still works correctly in chat context

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:40:58 +08:00
Octopus
653b00b94c fix(sync): scope document IDs per connector to prevent cross-KB collisions (#14378)
Fixes #14360

## Problem

When the same blob storage bucket is connected to multiple knowledge
bases (each through a different data source connector), the sync
pipeline hashes only the blob path
(`bucket_type:bucket_name:object_key`) to derive the document ID. Every
connector pointing at the same bucket therefore produces **identical
IDs** for the same object. The collision guard in
`FileService.upload_document` then fires for the second knowledge base:

```
Existing document id collision with another knowledge base; skipping update.
```

This makes it impossible to index the same bucket into more than one KB
simultaneously.

## Solution

Include `connector_id` in the hash input so that each connector produces
a distinct document ID even when the underlying blob path is identical:

```python
# Before
"id": hash128(doc.id),

# After
"id": hash128(f"{task['connector_id']}:{doc.id}"),
```

Because each KB connection uses its own connector (with a unique
`connector_id`), documents are now namespaced per connector and no
collision occurs.

**Note:** This is a breaking change for existing synced data sources.
After upgrading, a re-sync will create new documents with the updated ID
format. Old documents (indexed under the previous format) will remain in
the database but can be manually deleted or cleaned up via a re-sync
with reindex enabled.

## Testing

- Verified that the one-line change produces unique IDs for two
connectors pointing at the same S3 path.
- Existing unit test
`test_upload_document_skips_cross_kb_document_id_collision` continues to
pass — the collision guard in `FileService` is still valid for genuinely
colliding IDs from other sources.

---------

Co-authored-by: octo-patch <octo-patch@github.com>
2026-05-09 10:33:54 +08:00
writinwaters
d487a7f190 Docs: Added a guide on configuring SSL certificates (#14696)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2026-05-09 10:08:14 +08:00
Jin Hai
b6abce50b1 Go: Admin list ingestion tasks (#14695)
### What problem does this PR solve?

```
RAGFlow(admin)> list tasks;
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
| chunk_count | digest           | document_id                      | duration    | from_page | id                               | priority | progress             | retry_count | task_type | to_page |
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
| 16          | 8a0016a0dc3cbdbb | f6aa38bb4ad111f1ba6338a74640adcc | 1511.156966 | 0         | f91e4f104ad111f1aaaf38a74640adcc | 0        | 1                    | 1           |           | 12      |
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 10:03:23 +08:00
Jin Hai
5e96c5cae6 Fix go cli: search on datasets (#14692)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 20:25:14 +08:00
Joseff
2ad854c586 Go: implement Rerank in Aliyun driver (#14676)
### What problem does this PR solve?

The Aliyun Go driver has a stub `Rerank` method that always returns
`"Aliyun, Rerank not implemented"`. DashScope exposes an
OpenAI-compatible rerank endpoint (`compatible-mode/v1/rerank`) and
hosts dedicated bilingual rerankers (`gte-rerank-v2`, `gte-rerank`) that
are a natural pairing with the embedding models already in
`aliyun.json`. Without this, Aliyun users cannot use reranking within
RAGFlow.

Closes #14675

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 20:21:04 +08:00
Wang Qi
0552b1695a Fix UI search multiple datasets (#14689)
### What problem does this PR solve?

Fix UI search multiple datasets

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 20:20:09 +08:00
chanx
cacb7f2c18 Fix: Route error in dataset files page (#14691)
### What problem does this PR solve?

Fix: Route error in dataset  files page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 20:19:26 +08:00
Wang Qi
7d35e40c7b Refactor : Allow search multiple datasets (#14685)
### What problem does this PR solve?

Refactor : Allow search multiple datasets
1. support /datasets/search
2. get rid of /graph/search, use /graph

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-08 19:01:35 +08:00
dale053
26d70189b6 fix: enforce tenant-scoped authorization for chatbot SDK endpoints (#14592)
Closes #14590

## Self Checks

- [x] I have searched for existing issues [search for existing
issues](https://github.com/infiniflow/ragflow/issues), including closed
ones.
- [x] I confirm that I am using English to submit this report ([Language
Policy](https://github.com/infiniflow/ragflow/issues/5910)).
- [x] Non-english title submitions will be closed directly (
非英文标题的提交将会被直接关闭 ) ([Language
Policy](https://github.com/infiniflow/ragflow/issues/5910)).
- [x] Please do not modify this template :) and fill in all the required
fields.

## RAGFlow workspace code commit ID

`a1b2c3d4e5f67890123456789abcdef12345678`

## RAGFlow image version

`0.13.1`

## Other environment information

- Hardware parameters: N/A
- OS type: Linux 6.17.0-22-generic
- Others: API key authentication via `Authorization: Bearer <token>`

## Actual behavior

The chatbot API endpoints:

- `POST /chatbots/<dialog_id>/completions`
- `GET /chatbots/<dialog_id>/info`

validate only that the bearer token exists in `APIToken`, but do not
verify that `dialog_id` belongs to the same tenant as that token.

Current flow (simplified):

1. Route extracts bearer token and checks `APIToken.query(beta=token)`.
2. If token exists, request is accepted.
3. Downstream service resolves dialog globally by ID
(`DialogService.get_by_id(dialog_id)` in `conversation_service.py`).
4. No tenant ownership check is enforced for `dialog_id`.

Impact: Any user with a valid API key can attempt arbitrary `dialog_id`
values and access/invoke chatbots outside their own tenant boundary if
IDs are known/guessed/leaked.

Security classification:

- Vulnerability class: Broken Access Control (IDOR, OWASP Top 10 A01)
- Severity recommendation: Critical
- Exploit prerequisite: any valid API key + discoverable target
`dialog_id`

## Expected behavior

Requests to `/chatbots/<dialog_id>/completions` and
`/chatbots/<dialog_id>/info` must be authorized only when:

1. bearer token is valid, and
2. `dialog_id` belongs to the same `tenant_id` as the token.

Otherwise, reject with authorization failure (e.g., 403 or
404-equivalent policy).

## Steps to reproduce

1. Prepare two tenants:
   - Tenant A with API key `TOKEN_A`
   - Tenant B with chatbot `dialog_id = DIALOG_B`
2. Send request from Tenant A to Tenant B chatbot completion endpoint:

```bash
curl -X POST "https://<host>/chatbots/DIALOG_B/completions" \
  -H "Authorization: Bearer TOKEN_A" \
  -H "Content-Type: application/json" \
  -d '{"question":"hello","stream":false}'
```

3. Observe request is processed (or reaches dialog resolution) without
tenant ownership rejection.
4. Repeat against info endpoint:

```bash
curl -X GET "https://<host>/chatbots/DIALOG_B/info" \
  -H "Authorization: Bearer TOKEN_A"
```

5. Observe the same missing ownership enforcement.

## Additional information

Affected code paths:

- `api/apps/sdk/session.py`
  - `chatbot_completions(dialog_id)`
  - `chatbots_inputs(dialog_id)`
- `api/db/services/conversation_service.py`
  - `async_iframe_completion(...)` uses global dialog lookup

Suggested fix:

1. In both chatbot endpoints:
   - Resolve `tenant_id = objs[0].tenant_id` from validated token.
- Fetch dialog with tenant-scoped query
(`DialogService.query(id=dialog_id, tenant_id=tenant_id)`).
   - Reject if dialog is not found/owned by tenant.
2. Defense in depth:
- Require and enforce `tenant_id` in service-layer dialog resolution for
external flows.
- Avoid global `get_by_id(dialog_id)` where user-controlled dialog IDs
are reachable.
3. Add regression tests:
   - Positive: same-tenant token + dialog succeeds.
   - Negative: cross-tenant token + dialog fails for both endpoints.
2026-05-08 18:00:18 +08:00
Lynn
ada6d47880 Fix: move file check (#14681)
### What problem does this PR solve?

Restrict file move operations: prevent moving a folder to itself or to
one of its own subfolders.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 17:58:37 +08:00
qinling0210
4d6e8dffac Do not bypass threshold for rerank when metadata filter is enabled (#14684)
### What problem does this PR solve?

Do not bypass threshold for rerank when metadata filter is enabled

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 17:48:30 +08:00
web-dev0521
a32ebf32bd Fix: handle null document_metadata in kb_prompt to prevent citation crash (#14651) (#14666)
### What problem does this PR solve?

Fixes #14651.

`kb_prompt()` in `rag/prompts/generator.py` crashes with
`AttributeError: 'NoneType' object has no attribute 'items'` during
agent citation generation when a retrieved chunk carries
`document_metadata: null`.

**Root cause.** The crash happens at `rag/prompts/generator.py:132-133`:

```python
meta = ck.get("document_metadata", {})
for k, v in meta.items():
```

`dict.get(key, default)` only returns the default when the key is
*missing*. When the key is present with an explicit `None` value,
`.get()` returns `None`, and `.items()` crashes.

**How the chunk gets `None`.** It's a round-trip inside RAGFlow itself,
not bad input from retrieval:

1. The agent stores retrieved chunks via `agent/canvas.py:814`, which
routes them through `chunks_format()`.
2. `rag/prompts/generator.py:61` canonicalizes the field with
`chunk.get("document_metadata")` (no default), so chunks without
metadata become `{"document_metadata": None, ...}`.
3. `agent/component/agent_with_tools.py:314` feeds those canonicalized
chunks back into `kb_prompt()` for citation generation, and
`.get("document_metadata", {})` no longer protects us.

**Fix.** One-line change at `rag/prompts/generator.py:132`: use
`ck.get("document_metadata") or {}` so an explicit `None` is also
coerced to `{}`.

The line-61 `None` is intentionally part of the API/UI contract — the
frontend handles it via optional chaining
(`web/src/components/markdown-content/index.tsx:184`,
`web/src/pages/next-search/search-view.tsx:217`) — so the fix belongs at
the consumer, not the producer.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-08 16:54:33 +08:00
Jin Hai
ce2ec86b5e Go: fix CLI logout command (#14672)
### What problem does this PR solve?

```
RAGFlow(user)> logout;
SUCCESS
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 16:47:25 +08:00
Haruko386
94f82acd03 Fix(Go): prevent global state pollution in local model connection check (#14669)
### What problem does this PR solve?

1. **Fix Global State Pollution in Local Providers (Critical Bug):** -
Resolved a severe concurrency and architecture issue in
`model_service.go`. Previously, `ListSupportedModels` would permanently
overwrite the global provider singleton with a localized URL instance
(`driver.NewInstance`). This caused cross-request contamination in
multi-tenant environments.
- Fixed `CheckProviderConnection` for local models (LM Studio, vLLM,
Ollama). It now properly creates a localized driver copy and injects the
`base_url` before testing the connection, entirely eliminating the
false-positive `missing base URL` error without polluting the global
state.
2. **Implement `VolcEngine` Embeddings:** - Fully implemented the
`Encode` method for the `volcengine` provider, enabling text embedding
capabilities for VolcEngine models.
3. **Enhance Region Validation in `SiliconFlow`:** - Added a strict
empty string check (`*apiConfig.Region != ""`) alongside the existing
`nil` check when parsing regions. This ensures that if an empty string
is passed, the system safely falls back to the `"default"` region,
preventing malformed URL requests and `unsupported protocol scheme`
errors.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 15:54:27 +08:00
Jin Hai
ee5ae6f1a4 Go CLI: fix register user (#14665)
### What problem does this PR solve?

1. Update API URL
2. Add password encryption

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 15:53:06 +08:00
Lynn
69197d4a8f Fix: type of tenant_rerank_id (#14667)
### What problem does this PR solve?

Update the type of tenant_rerank_id in validation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 15:32:34 +08:00
Tim Wang
decb5dcb6f Fix: path-aware reset in canvas.run() to preserve cross-run outputs (#14600)
## Summary

- When an agent workflow has multiple `UserFillUp` pause points,
`canvas.run()` calls `reset(True)` on **all** components at the start of
each run. This clears outputs from components that completed in prior
runs, so downstream references like `{Agent:XXX@content}` resolve to
`None`.
- This fix only resets components on the **current execution path**
(`self.path`), preserving outputs from previously completed components.

## Problem

In a multi-step agent (e.g. draft email → user confirms → send email):

1. First `run()`: Agent drafts content, UserFillUp pauses for user input
→ Agent output is saved
2. Second `run()`: User submits input, but `reset(True)` clears **all**
components including the Agent that already completed
3. Email component references `{Agent:XXX@content}` → gets `None`
instead of the draft

This affects **all** agents that reference upstream component outputs
after a UserFillUp pause point.

## Fix

```python
# Before: reset ALL components
for k, cpn in self.components.items():
    self.components[k]["obj"].reset(True)

# After: only reset components on current execution path
path_set = set(self.path)
for k, cpn in self.components.items():
    if k in path_set:
        self.components[k]["obj"].reset(True)
```

`self.path` already tracks the current execution path. For agents
without UserFillUp (single run), `path` contains all components, so
behavior is unchanged.

## Test plan

- [x] Agent with single UserFillUp: outputs from prior components are
preserved after resume
- [x] Agent with multiple UserFillUp: each resume preserves all
previously completed outputs
- [x] Agent without UserFillUp: behavior unchanged (all components in
path, all reset)
- [x] Webhook-triggered agents: unaffected (path includes all components
on first run)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-08 15:10:15 +08:00
buua436
d843035c8b Fix: add compatibility route for document download under /v1 (#14663)
### What problem does this PR solve?

add compatibility route for document download under /v1

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 14:44:02 +08:00
Tim Wang
1bcb6deb6f Fix: collapsible thinking display and separate deep research retrieval tag (#14613)
## Summary

- **Collapsible thinking**: Replace `<section>` with `<details>` for
`<think>` content, so model thinking output is collapsed by default
(click to expand). Works for all models that output `<think>` tags
(Qwen3, DeepSeek, Gemini, Claude, etc.).
- **Fix double thinking tags**: When reasoning/deep research mode is
enabled in knowledge base chat, both the retrieval progress and model
thinking were wrapped in `<think>` tags, producing two "Thinking..."
blocks. Now retrieval progress uses a dedicated `<retrieving>` tag
rendered as a separate "Retrieving..." collapsible with a distinct green
accent.

### Before
- Thinking content displayed as flat gray-bordered `<section>`,
occupying significant screen space
- Deep research + model thinking both use `<think>` → two identical
"Thinking..." blocks

### After
- Thinking content collapsed by default in a `<details>` element, click
"Thinking..." to expand
- Deep research shows "Retrieving..." (green border), model thinking
shows "Thinking..." (gray border)

## Changes

**Backend (`api/db/services/dialog_service.py`)**
- Deep research callback: replace `start_to_think`/`end_to_think` marker
flags with direct `<retrieving>`/`</retrieving>` answer text

**Frontend**
- `web/src/utils/chat.ts`: `replaceThinkToSection()` now uses
`<details>` instead of `<section>`; add new
`replaceRetrievingToSection()`
- 4 tsx files: import and pipe `replaceRetrievingToSection`, whitelist
`details`, `summary`, `retrieving` in DOMPurify `ADD_TAGS`
- 4 less files: `section.think` → `details.think` with `<summary>`
styles; add `details.retrieving` with green accent; dark mode and RTL
variants

## Test plan
- [ ] Open a chat WITHOUT knowledge base, ask a question to a model with
thinking (e.g. Qwen3) → thinking content should be collapsed by default,
click "Thinking..." to expand
- [ ] Open a chat WITH knowledge base and reasoning enabled, ask a
question → "Retrieving..." (green) shows retrieval progress,
"Thinking..." (gray) shows model thinking, each independently
collapsible
- [ ] Verify dark mode renders correctly for both collapsible blocks
- [ ] Verify RTL layout renders correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-08 14:40:00 +08:00
web-dev0521
d51fb88573 Fix: enforce tenant authorization on document download endpoint (#14618) (#14625)
### What problem does this PR solve?

Closes #14618.

The `GET /v1/document/get/<doc_id>` endpoint in
`api/apps/document_app.py` was protected only by `@login_required` and
called `DocumentService.get_by_id(doc_id)` without verifying that the
document's knowledge base belonged to the requesting user's tenant. Any
authenticated user who knew (or guessed) a document ID could download
files belonging to any other tenant — a cross-tenant IDOR.

This PR adds a `DocumentService.accessible(doc_id, current_user.id)`
check before serving the file. The helper already exists and joins
`Document` → `Knowledgebase` → `UserTenant` to verify the requesting
user belongs to the tenant that owns the document's KB. The same pattern
is already used by `api/apps/restful_apis/document_api.py` and mirrors
the tenant scoping in the SDK route at `api/apps/sdk/doc.py`.

The check returns the existing `"Document not found!"` error for both
non-existent and inaccessible documents, so attackers cannot use the
response to enumerate valid doc IDs across tenants.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Other (please describe): Security fix (cross-tenant IDOR /
authorization bypass)
2026-05-08 14:24:03 +08:00
Panda Dev
a82ae4a991 Go: implement Encode (embeddings) in Aliyun driver (#14647)
### What problem does this PR solve?

The Aliyun Go driver shipped with a stub \`Encode\` method that returned
\`no such method\`, even though \`conf/models/aliyun.json\` already
wires the OpenAI-compatible embeddings URL suffix at
\`compatible-mode/v1/embeddings\`. The same config also did not list any
embedding models, so the picker had nothing to select.

So an Aliyun tenant who wanted to use Tongyi text-embedding-v3 or v4 in
the Go layer could not, even though the upstream endpoint is public and
uses the standard \`POST /v1/embeddings\` shape that the SiliconFlow and
ZhipuAI
drivers already support.

This PR fills the gap.

### What this PR includes

- \`conf/models/aliyun.json\`: add \`text-embedding-v4\` and
\`text-embedding-v3\` to the \`models\` array.
- \`internal/entity/models/aliyun.go\`: replace the \`Encode\` stub with
a real implementation. Adds a small local response type that matches the
OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, validate the model name,
resolve the region with a default fallback, build the
  URL from \`BaseURL[region] + URLSuffix.Embedding\`.
- Send all input texts in one request as the \`input\` array, the same
OpenAI-compatible shape the SiliconFlow \`Encode\`
  uses.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\` so the output order matches the input order
even if the API returns items in a different order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks every input slot got a vector and returns a clear
error if any slot is still nil.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`AliyunModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Encode implementation.

Closes #14646

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 13:58:25 +08:00
Haruko386
d13a240dc0 Go: implement remaining interface for OpenRouter (#14657)
### What problem does this PR solve?

1. implement `rerank`, `embedding`, `balance`, `checkConnet` method for
`OpenRouter`
2. delete `chat` method in `internal/entity/models/volcengine.go`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-08 13:56:45 +08:00
Jin Hai
731c887ba0 Fix cli login (#14658)
### What problem does this PR solve?

Since API is updated, CLI login failed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 13:56:19 +08:00
jony376
6547751936 Fix: missing authorization checks in /files/link-to-datasets (#14649)
### Related issues
Closes #14648

### What problem does this PR solve?

This PR fixes an authorization flaw in `POST /files/link-to-datasets`.

Before this change, the endpoint only checked whether the supplied
`file_ids` and `kb_ids` existed. It did not verify whether the
authenticated user was actually allowed to access those files or target
datasets. As a result, an authenticated user who knew valid IDs could
relink another user's files to arbitrary datasets.

This was especially risky because the relinking flow is state-changing:
the background worker removes existing file-document mappings and then
recreates documents under the attacker-supplied dataset IDs.

This change makes the route enforce the same permission model already
used by nearby file and document operations:

- each resolved file must pass `check_file_team_permission(...)`
- each target dataset must pass `check_kb_team_permission(...)`
- authorization is enforced before scheduling background relinking work

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- Added regression coverage in
`test/testcases/test_web_api/test_file_app/test_file2document_routes_unit.py`
- Covered:
  - unauthorized file access is rejected
  - unauthorized dataset access is rejected
- existing success path still returns immediately after scheduling
background work
- Attempted to run:
- `python -m pytest
test\\testcases\\test_web_api\\test_file_app\\test_file2document_routes_unit.py
-q`
- Local execution in this workspace is currently blocked by missing test
dependencies during bootstrap, including `ragflow_sdk`

---------

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-08 13:49:23 +08:00
buua436
f703169117 Refa: migrate document preview/download to RESTful API (#14633)
### What problem does this PR solve?

migrate document preview/download to RESTful API

### Type of change
- [x] Refactoring
2026-05-08 13:26:13 +08:00
Lynn
412fae7ac2 Fix: display error (#14654)
### What problem does this PR solve?

Use right key in error text.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 13:11:59 +08:00
Panda Dev
d8d49df35e Go: implement Rerank in Gitee AI driver (#14656)
### What problem does this PR solve?

The Gitee AI Go driver shipped with a stub \`Rerank\` method that
returned \`Rerank not implemented\`, even though
\`conf/models/gitee.json\` already wires the rerank URL suffix at
\`\"rerank\": \"rerank\"\`. The same config did not list any
rerank model, so the picker had nothing to select.

So a Gitee tenant could not use BAAI/bge-reranker-v2-m3 as a reranker
through the Go layer today, even though the
infrastructure was one config entry and one method body away.

### What this PR includes

- \`conf/models/gitee.json\`: add \`BAAI/bge-reranker-v2-m3\` to the
\`models\` array.
- \`internal/entity/models/gitee.go\`: replace the \`Rerank\` stub with
a real implementation. Adds two small local types
that match the OpenAI-compatible \`/rerank\` shape already used by the
SiliconFlow and ZhipuAI drivers.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, validate the model name,
resolve the region with a default fallback, build the
  URL from \`BaseURL[region] + URLSuffix.Rerank\`.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, matching the pattern the
  recently merged Aliyun Encode and the OpenAI driver already use.
- Send \`{model, query, documents, top_n, return_documents:false}\` in
the body.
- Parse \`results[*].relevance_score\` and copy each score into the
output slice indexed by \`results[*].index\`, so the
output order matches the input order even if the API returns items in a
different order.
- Empty input returns \`[]float64{}\` with no HTTP call.
- An out-of-range result index returns a clear error rather than
silently skipping the entry.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`GiteeModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Rerank and the recently
merged ZhipuAI Rerank (#14608).

Closes #14655
2026-05-08 13:08:22 +08:00
D2758695161
f063e03a24 fix: add bucket prefix to Azure Blob SPN and SAS storage operations (#14347)
## Summary

Fixes file collision between different datasets when using Azure Blob
storage (SPN or SAS authentication).

## Bug

azure_spn_conn.py and zure_sas_conn.py ignored the ucket parameter
entirely, storing all files flat with just the filename. This caused
files with the same name from different datasets (knowledge bases) to
overwrite each other.

## Fix

Prepend bucket/ as a path prefix in all methods (put, 
m, get, obj_exist, get_presigned_url, health) to match the behavior of
MinIO and S3 implementations.

## Changes

- **rag/utils/azure_spn_conn.py**: Added {bucket}/ prefix to file paths
in all operations
- **rag/utils/azure_sas_conn.py**: Same fix applied for consistency
(also noted in the original issue)

## Testing

Manual verification: files from different datasets now stored under
distinct bucket/ prefixes, preventing collisions.

Fixes #14159

Co-authored-by: Hunter <hunter@yitong.ai>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 12:06:28 +08:00
Panda Dev
c7ddc8c039 fix(go): implement ListModels and CheckConnection in NVIDIA driver (#14636)
### What problem does this PR solve?

The NVIDIA Go driver added in #14623 has a real chat path, but
\`ListModels\` and \`CheckConnection\` are stubs that always return \`no
such method\`. So:

- The model picker cannot auto-populate available NVIDIA NIM model ids.
Users have to type the full id by hand (e.g.
  \`abacusai/dracarys-llama-3.1-70b-instruct\`).
- The "Check connection" button always fails for NVIDIA, even when the
base URL is reachable and the API key is accepted.

NVIDIA NIM is OpenAI-compatible. \`/v1/models\` works with the same
Bearer token used for chat. The
\`conf/models/nvidia.json\` file already wires the \`models\`
url_suffix, so no config change is needed.

### What this PR includes

- \`internal/entity/models/nvidia.go\`:
  - \`ListModels\` now calls
    \`GET ${BaseURL}/${URLSuffix.Models}\`, parses
    \`response.data[*].id\`, and returns the list. Same shape
    as the moonshot, xai, and openai drivers.
  - \`CheckConnection\` now calls \`ListModels\` and returns its
    error. Same pattern xai, moonshot, deepseek, aliyun, and
    gitee already use.

\`Balance\`, \`Encode\`, and \`Rerank\` are still stubs in this PR and
can be added in follow-ups.

No JSON change. No factory change. No interface change.

### How the implementation works

- Region resolution falls back to \`default\` when the supplied region
is unknown, so a stray region value does not break a valid request.
- The Authorization header is only set when \`apiConfig\` and \`ApiKey\`
are non-nil and non-empty. This avoids a nil-pointer dereference and
lets self-hosted NIM deployments without a key still work.
- Non-200 responses propagate the upstream status line and body so the
user sees a real error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`NvidiaModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing xai, moonshot, deepseek, aliyun,
gitee, and openai drivers.

Closes #14635
2026-05-08 12:04:28 +08:00
Panda Dev
e729eced45 Go: implement Balance in DeepSeek driver (#14632)
Closes #14631 

### What problem does this PR solve?

The DeepSeek Go driver shipped with a stub \`Balance\` method that
returned \`no such method\`, even though DeepSeek exposes a public \`GET
/user/balance\` endpoint that works with the same Bearer token used for
chat.

So the "Balance" panel in the model provider UI always shows an error
for DeepSeek tenants, while it already works for Moonshot and Gitee.
This PR fills the gap.

### What this PR includes

- \`conf/models/deepseek.json\`: add \`\"balance\": \"user/balance\"\`
under \`url_suffix\` so the driver can build the URL from config the
same way the other endpoints do.
- \`internal/entity/models/deepseek.go\`: replace the \`Balance\` stub
with a real implementation. Adds a small local response type
\`deepseekBalanceResponse\` that matches the upstream shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, resolve the region (with a
\`default\` fallback), and build the URL from \`BaseURL[region] +
URLSuffix.Balance\`.
- GET the URL with \`Authorization: Bearer <api_key>\`.
- Parse the upstream response:

  \`\`\`json
  {
    \"is_available\": true,
    \"balance_infos\": [
      {\"currency\": \"USD\", \"total_balance\": \"10.00\", ...},
      {\"currency\": \"CNY\", \"total_balance\": \"70.00\", ...}
    ]
  }
  \`\`\`

\`total_balance\` is a string in the upstream API, so the driver parses
it with \`strconv.ParseFloat\`.
- Return the first balance entry as \`{\"balance\": <float>,
\"currency\": <string>}\`, the same shape the Moonshot driver returns.
The UI can render it with no provider-specific code.

### Edge cases

- Missing or empty API key returns a clear local error before any HTTP
call.
- Empty \`balance_infos\` returns a clear \"no balance info in
response\" error rather than a zero-value silent success.
- Non-numeric \`total_balance\` returns a clear parse error.
- Non-200 responses propagate the upstream status line and body so the
user can see why the call failed.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`DeepSeekModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing Moonshot and Gitee Balance
implementations.
2026-05-08 12:03:39 +08:00
Haruko386
a377512110 Go: implement provider: OpenRouter (#14652)
### What problem does this PR solve?

1. **Implement `OpenRouter` Provider:** Fully support OpenRouter AI
models (e.g., `gemma`, `minimax`). Includes robust handling of
Server-Sent Events (SSE) streams, error event interception, and proper
parsing of both `reasoning_content` and standard `content`.
2. **Fix BaseURL Resolution Bug:** Fixed a critical edge case in region
configuration parsing. Added a strict empty string check
(`*apiConfig.Region != ""`) alongside the `nil` check. This ensures that
if the UI passes an empty string, the system correctly falls back to the
`"default"` region, preventing `unsupported protocol scheme ""` errors
during HTTP requests.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 12:02:37 +08:00
Panda Dev
a86e0ca0ca Go: implement Balance in SiliconFlow driver (#14643)
### What problem does this PR solve?

The SiliconFlow Go driver shipped with a stub \`Balance\` method that
returned \`no such method\`, even though SiliconFlow exposes a public
\`GET /v1/user/info\` endpoint that returns the account balance per
currency.

So the "Balance" panel in the model provider UI always shows an error
for SiliconFlow tenants, while it already works for
Moonshot and Gitee. This PR fills the gap.

### What this PR includes

- \`conf/models/siliconflow.json\`: add \`\"balance\": \"user/info\"\`
under \`url_suffix\` so the driver builds the URL from config.
- \`internal/entity/models/siliconflow.go\`: replace the \`Balance\`
stub with a real implementation. Adds a small local response type that
matches the upstream shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, resolve the region with a
default fallback, and build the URL from \`BaseURL[region] +
URLSuffix.Balance\`.
- GET the URL with \`Authorization: Bearer <api_key>\`.
- Parse the upstream response. SiliconFlow returns balance fields as
strings, so the driver parses them with \`strconv.ParseFloat\`. It
prefers \`totalBalance\` over \`balance\` when both are present.
- Return \`{\"balance\": <float>, \"currency\": \"CNY\"}\`, the same
shape the Moonshot driver returns. The UI can render it
  with no provider-specific code.

### Edge cases

- Missing or empty API key returns a clear local error before any HTTP
call.
- An unknown region falls back to the default base URL.
- Empty \`balance\` and \`totalBalance\` returns a clear "no balance
info in response" error rather than a zero-value silent success.
- Non-numeric balance string returns a clear parse error.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`SiliconflowModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing Moonshot and Gitee Balance
implementations.

Closes #14642
2026-05-08 12:01:10 +08:00
Panda Dev
2fd8cdc3cc fix(go): wire CheckConnection to ListModels in ollama, lm-studio, and vllm (#14614)
### What problem does this PR solve?

Three Go drivers had `CheckConnection` returning a hardcoded `no such
method` error, even though each one already has a working `ListModels`
that hits the configured base URL with the configured API key. So the
"Check connection" button in the model provider UI always failed for
these three providers, even when the underlying setup was fine.

Affected drivers:

- `internal/entity/models/ollama.go`
- `internal/entity/models/lmstudio.go`
- `internal/entity/models/vllm.go`

This is a real user-facing gap because Ollama and LM Studio are two of
the most popular local LLM runners, and vLLM is widely used for
self-hosted deployments.

### What this PR includes

For each of the three drivers, replace the stub with a small
implementation that calls `ListModels` and returns its error:

```go
func (o *OllamaModel) CheckConnection(apiConfig *APIConfig) error {
    _, err := o.ListModels(apiConfig)
    return err
}
```

This is the exact pattern that xai, moonshot, deepseek, aliyun, and
gitee already use for the same method.

No JSON change. No factory change. No interface change.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- `go build ./internal/entity/models/...` in a clean go 1.25 image (the
go.mod minimum) returns exit 0.
- The full ModelDriver interface still resolves on each driver
(NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender, Encode,
Rerank, ListModels, Balance, CheckConnection).
- Pattern parity with the existing xai, moonshot, deepseek, aliyun, and
gitee CheckConnection methods.

Closes #14609
2026-05-08 12:00:10 +08:00
Baki Burak Öğün
5d28bb0701 feat: update Turkish localization strings (#14650)
## Summary
Update the Turkish locale file to match the latest English locale keys.

## Changes
- Add missing Turkish translations for the new Skills and Skill Search
sections
- Add newly introduced common, header, dataset, settings, and agent
workflow strings
- Align renamed flow keys such as file format options and list
operations with the English source
- Add empty-state strings for skill spaces

## Validation
- Compared web/src/locales/en.ts and web/src/locales/tr.ts: 0 missing
keys, 0 extra keys
- Checked jsonjoy-builder locale: Turkish is already complete
- Checked translated README variants: no new Turkish-specific
documentation gap found
- VS Code diagnostics: no errors in web/src/locales/tr.ts

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
2026-05-08 09:56:04 +08:00
sxxtony
59c35100c5 Perf: push metadata filters down to Elasticsearch (#14576)
### What problem does this PR solve?

Fixes #14412.

`common.metadata_utils.meta_filter` evaluates user-defined metadata
conditions in Python after `DocMetadataService.get_flatted_meta_by_kbs`
loads the entire `meta_fields` table into memory. Past a few thousand
documents per knowledge base this becomes a memory bottleneck and a
wasted ES round-trip — every filter request currently fetches up to
10000 metadata rows even when the resulting `doc_ids` list is tiny.

This PR adds an ES push-down path that translates the same filter
language into a `bool` query and returns just the matching document IDs.

**Changes**

- `common/metadata_es_filter.py` *(new)*: pure-Python translator from
the RAGflow filter list to ES DSL. Covers every operator the in-memory
path supports (`=`, `≠`, `>`, `<`, `≥`, `≤`, `in`, `not in`, `contains`,
`not contains`, `start with`, `end with`, `empty`, `not empty`) with
`case_insensitive: true` on `prefix` and `wildcard` for parity with the
existing lower-cased Python comparisons. User wildcard metacharacters
are escaped before being injected into `wildcard` patterns. Negative
operators (`≠`, `not in`, `not contains`, ranges) are wrapped with an
`exists` guard so they do not accidentally match documents missing the
key, matching the legacy `if k not in metas` behaviour.
- `api/db/services/doc_metadata_service.py`: new
`DocMetadataService.filter_doc_ids_by_meta_pushdown(kb_ids, filters,
logic)` that returns the doc IDs ES matched, or `None` to signal the
caller should fall back to the in-memory path. Returns `None` when the
active doc store is Infinity (`meta_fields` is a JSON column, not a
dotted-object mapping), when any filter cannot be expressed in DSL
(`UnsupportedMetaFilter`), or when the ES request or metadata index
lookup errors.
- `common/metadata_utils.py`: `apply_meta_data_filter` accepts an
optional `kb_ids` argument. When supplied, conditions go through
push-down first via a new `_try_meta_pushdown` helper; on `None` the
function falls back to the original `meta_filter` call. Default
behaviour is unchanged for callers that don't pass `kb_ids`.
- Updated all four callers (`agent/tools/retrieval.py`,
`api/db/services/dialog_service.py` ×2,
`api/apps/services/dataset_api_service.py`, `api/apps/sdk/session.py`)
to forward `kb_ids` so the push-down path is exercised in production.
- `test/unit_test/common/test_metadata_es_filter.py` *(new)*: 35 unit
tests covering every operator's DSL shape, value coercion
(`ast.literal_eval`, lowercasing, ISO-date pass-through), wildcard
escaping, OR-logic wrapping that protects negative clauses, and the
doc-ID extractor.

**Behaviour preserved**

- The in-memory `meta_filter` is untouched and still services every
fallback case (Infinity backend, unknown operators, ES outages).
- The eligibility / credibility / issue-multiplier semantics described
in the LLM-driven `auto` and `semi_auto` modes still hand the LLM the
full in-memory `metas` dict to choose conditions from. Only the
*evaluation* of those generated conditions is pushed down.
- Existing tests in
`test/unit_test/common/test_metadata_filter_operators.py` continue to
pass (14/14).

**Test plan**

- `pytest test/unit_test/common/test_metadata_es_filter.py` — 35 passed.
- `pytest test/unit_test/common/test_metadata_filter_operators.py` — 14
passed.
- `ruff check` clean on every modified file.
- Reviewer please validate the ES query shapes against a live cluster —
particularly `case_insensitive` on `wildcard` and `prefix` (requires ES
7.10+) and the `exists` + `must_not` pairing for `≠`.

**Notes**

- The first cut caps each push-down request at 10000 results, matching
the existing `get_flatted_meta_by_kbs` limit, and logs a warning when
the cap is hit. A `search_after` follow-up would let us drop the cap
entirely once the push-down path is validated.
- Operator parity with the in-memory path is exact for the canonical
unicode operators (`≥`, `≤`, `≠`) used internally; the ASCII aliases
(`>=`, `<=`, `!=`) are normalised by `convert_conditions` before they
reach the translator.

### Type of change

- [x] Performance Improvement

---------

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
2026-05-07 21:23:43 +08:00
chanx
805a2daac2 Fix: Change route name (#14639)
### What problem does this PR solve?

Fix: Change route name

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 21:23:29 +08:00
Magicbook1108
c29335cbff Feat: support local provider for code exec component & remove some outdated models (#14637)
### What problem does this PR solve?

Feat: support local provider for code exec component & remove some
outdated models

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 21:23:13 +08:00
d 🔹
057806d7f1 fix: prepend bucket prefix to Azure SPN and SAS storage paths (#14185)
## Summary

Fixes #14159 — files from different datasets can overwrite each other in
Azure Blob storage.

## Problem

Both `azure_spn_conn.py` and `azure_sas_conn.py` ignore the `bucket`
parameter in all storage operations (`put`, `get`, `rm`, `obj_exist`,
`get_presigned_url`). Files are stored flat using only the filename, so
two datasets containing a file with the same name will overwrite each
other.

The MinIO and S3 implementations correctly use the bucket (typically the
knowledge base ID) as a path prefix to create logical folder isolation:
- MinIO: uses `use_prefix_path` decorator → `{orig_bucket}/{fnm}`
- S3: uses `use_prefix_path` decorator → `{prefix_path}/{bucket}/{fnm}`

## Fix

Prepend `{bucket}/` to the file path in all 5 operations across both
Azure connector files:

| File | Methods fixed |
|------|---------------|
| `azure_spn_conn.py` | `put`, `get`, `rm`, `obj_exist`,
`get_presigned_url` |
| `azure_sas_conn.py` | `put`, `get`, `rm`, `obj_exist`,
`get_presigned_url` |

This matches the existing convention where `bucket` is the knowledge
base ID used as a directory prefix.

## ⚠️ Migration Note

Existing Azure SPN/SAS deployments have files stored without the bucket
prefix. After this fix, new files will be stored under
`{bucket}/{filename}` while existing files remain at `{filename}`. A
one-time migration script or manual file move may be needed for existing
deployments. New deployments are unaffected.

## Testing

- Verified the fix is consistent across all 5 methods in both files
- The `health()` method is intentionally left unchanged as it uses a
hardcoded test filename without bucket semantics

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 20:48:32 +08:00
Panda Dev
bb10b83e61 Go: implement Rerank in ZhipuAI driver (#14608)
### What problem does this PR solve?

The ZhipuAI Go driver had a stub Rerank method that returned "not
implemented", even though conf/models/zhipu-ai.json already ships
glm-rerank as a rerank model and the rerank URL suffix is already wired
in url_suffix:

```json
"url_suffix": {
  ...
  "rerank": "rerank"
},
"models": [
  {"name": "glm-rerank", "model_types": ["rerank"]},
  ...
]
```

So the config was ready but the driver was not. A tenant who picked
glm-rerank in the Go layer could not actually run a rerank call. This PR
fills the gap so the listed model works end to end.

### What this PR includes

- `internal/entity/models/zhipu-ai.go`: real implementation of
`ZhipuAIModel.Rerank`, plus two small local types (`zhipuRerankRequest`,
`zhipuRerankResponse`) that mirror the standard OpenAI-compatible rerank
shape used by SiliconFlow.

No factory change. No JSON change. No interface change.

### How the driver works

- POST to `${BaseURL}/${URLSuffix.Rerank}` (resolves to
`https://open.bigmodel.cn/api/paas/v4/rerank` with the default config),
reusing the existing httpClient on the driver.
- Validate apiConfig and the API key, validate the model name, and
resolve the region. Return a clear local error before any HTTP call when
something is missing.
- Send `{model, query, documents, top_n, return_documents: false}` in
the body, the same shape the SiliconFlow driver already uses.
- Walk `results[*].relevance_score` and copy each score into the output
slice indexed by `results[*].index`, so the output order matches the
input order even if the API returns results in a different order.
- Empty `texts` input returns an empty `[]float64` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` in a clean go 1.25 image (the
go.mod minimum) returns exit 0.
- The full method set on `ZhipuAIModel` still matches the `ModelDriver`
interface (NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender,
Encode, ListModels, Balance, CheckConnection, Rerank).
- Pattern parity with the existing SiliconFlow Rerank implementation
(`internal/entity/models/siliconflow.go`).

Closes #14607
2026-05-07 17:56:30 +08:00
Jack Storment
59bb184e63 feat(moodle): support deleted-file sync (#14548)
Fixes #14551 

### What problem does this PR solve?

The Moodle connector did not let the sync runner clean up indexed
documents that were deleted from the source. Other connectors such as
dropbox, seafile, webdav, and rss already do this through a slim
snapshot pass. This PR adds the same support for Moodle.

When `sync_deleted_files` is on, the runner now asks the Moodle
connector for a lightweight list of every module id that could be
indexed. The runner then compares this list with the index and removes
any indexed document whose id is not in the list.

The slim pass does not download files. It only goes through courses and
modules and yields ids. The id format matches the ids that the loader
produces, so the match is exact.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Notes

- `MoodleConnector` now also implements `SlimConnectorWithPermSync`.
- New `retrieve_all_slim_docs_perm_sync` yields slim docs with the same
ids the loader uses (`moodle_resource_<id>`, `moodle_forum_<id>`,
`moodle_page_<id>`, `moodle_book_<id>`, `moodle_assign_<id>`,
`moodle_quiz_<id>`).
- The `Moodle` sync class now returns `(document_generator, file_list)`
so the runner can do the cleanup. If the slim snapshot fails,
`file_list` is set back to `None` and the run continues without cleanup.
- The web data source map exposes `syncDeletedFiles` for Moodle so the
option shows up in the UI.

### How was this tested?

- `ruff check` passes on the changed Python files.
- Manual review of the produced slim ids against the ids the loader
builds in `_process_resource`, `_process_forum`, `_process_page`,
`_process_book`, and `_process_activity`.
- Behavior parity with the merged dropbox (#14476), seafile (#14499),
webdav (#14491), and rss (#14493) PRs.
2026-05-07 17:44:46 +08:00
Jin Hai
94324afee9 Go: fix auth issue in hybrid mode (#14611)
### What problem does this PR solve?

Since secret key get and set logic is updated, the go server also need
to update.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 17:14:22 +08:00
Octopus
5c9124c3ef fix: prepend bucket prefix in Azure Blob (SAS/SPN) to prevent cross-dataset file overwrites (#14174)
Fixes #14159

## Problem

The `put()`, `get()`, `rm()`, and `obj_exist()` methods in both
`azure_spn_conn.py` and `azure_sas_conn.py` ignore the `bucket`
parameter entirely, storing all files flat using only the filename. This
causes files from different datasets to overwrite each other when they
share the same filename.

By contrast, the MinIO and S3 implementations correctly use the bucket
(typically the knowledge base ID) as a path prefix, creating logical
folder isolation like `{kb_id}/{filename}`.

## Solution

Prepend the `bucket` parameter as a path prefix to all file operations
in both Azure storage implementations:

- `azure_spn_conn.py`: `create_file`, `delete_file`, `get_file_client`
now use `f"{bucket}/{fnm}"`
- `azure_sas_conn.py`: `upload_blob`, `delete_blob`, `download_blob`,
`get_blob_client` now use `f"{bucket}/{fnm}"`

This matches the behavior of all other storage backends (MinIO, S3) and
prevents filename collisions across knowledge bases.

## Testing

- Verified the fix aligns with how MinIO/S3 connectors handle the bucket
parameter
- The `health()` method is left unchanged as it uses a fixed test path
for connectivity checks only

Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 17:13:43 +08:00
buua436
0501134820 Fix: support tool call config (#14616)
### What problem does this PR solve?
support tool call config

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:54:57 +08:00
buua436
5b162a0c46 Fix: preserve doc generator download metadata in message (#14626)
### What problem does this PR solve?

preserve doc generator download metadata

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:48:36 +08:00
Wang Qi
c50028b1f3 Fix team member cannot edit agent (#14612)
### What problem does this PR solve?

Follow on PR: https://github.com/infiniflow/ragflow/pull/14602
to fix: team member cannot edit agent.
new behavior: beside delete, everything is allowed for team member.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:09:13 +08:00
Wang Qi
1d114f034b Allow more task logs for #14617 (#14624)
### What problem does this PR solve?

Allow more task logs for #14617

### Type of change

- [x] Refactoring
2026-05-07 15:03:08 +08:00
Haruko386
078ea3bf4a Go: implement provider: Nvidia (#14623)
### What problem does this PR solve?

1. **Implement `Nvidia` Provider:** Fully support NVIDIA NIM APIs with
robust parameter handling (including the `thinking` parameter) and safe
URL merging in `NewInstance`.
2. **Fix Misleading CLI Errors:** Corrected a bug in `common_command.go`
where failed chat requests inaccurately reported `failed to list
instance models`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 14:17:57 +08:00
Magicbook1108
911671cef0 Feat: enable sync deleted files for RDBMS & fix remove last file issue (#14615)
### What problem does this PR solve?

Feat: enable sync deleted files for RDBMS & fix remove last file issue

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 13:31:05 +08:00
Panda Dev
b8b741555f Go: implement provider: OpenAI (#14605)
### What problem does this PR solve?

Add a Go driver for OpenAI (GPT models).

The config file conf/models/openai.json has been in the repo for a while
with the full GPT-5 model list, but
internal/entity/models/factory.go had no case for "openai". So any
tenant that configured OpenAI as a model provider in the Go layer fell
through to the default branch and got the dummy driver. Chat, list
models, and check connection all returned dummy responses instead of
reaching the API.

OpenAI is the most commonly requested provider and the JSON config
already ships with the repo, so this gap is high impact even though the
JSON has been there for some time.

### What this PR includes

- New file internal/entity/models/openai.go with an OpenAIModel that
implements the ModelDriver interface.
- factory.go: route the "openai" provider name to NewOpenAIModel.
- conf/models/openai.json: add "models": "models" under url_suffix so
ListModels can hit /v1/models with no hardcoded fallback.

### How the driver works

- OpenAI exposes the canonical OpenAI-compatible API at
https://api.openai.com/v1.
- ChatWithMessages and ChatStreamlyWithSender post to /chat/completions
in the same shape the moonshot, vllm, and xai drivers use.
- ListModels and CheckConnection call /models to list available ids and
confirm the API key works.
- reasoning_content is passed through for the o-series and other
reasoning models, in both the non-stream and stream paths.
- Encode (embeddings) is left as "not implemented" for now, the same way
the other recent provider drivers do it. Rerank and Balance are not part
of OpenAI's public API surface in this layer and return a clear "not
implemented" or "no such method" error.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- go build ./internal/entity/models/... in a clean go 1.25 image (the
go.mod minimum) returns exit 0 with no errors.
- Method set of OpenAIModel matches the ModelDriver interface:
NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender, Encode,
Rerank, ListModels, Balance, CheckConnection.
- Pattern parity with the merged moonshot (#14433), volcengine (#14460),
minimax (#14478), vllm (#14532), xai (#14550), and lm-studio (#14586)
PRs.

Closes #14604
2026-05-07 13:09:51 +08:00
Zhichang Yu
86fe78c73f feat(llm): add MiniMax GroupId header support (#14610)
## Summary
- Add MiniMax provider GroupId query parameter support in `LiteLLMBase`
- Extract `group_id` from key configuration in `__init__`
- Append `GroupId` as query parameter to `api_base` in
`_construct_complete_args`

## Why this change is needed

MiniMax provides an OpenAI-compatible API endpoint
(`/v1/chat/completions`), but `GroupId` is a MiniMax-specific account
identifier required for billing and rate limiting - it is not part of
the OpenAI standard.

Looking at LiteLLM's `MinimaxChatConfig`:
- `get_complete_url()` only constructs the base URL (e.g.,
`https://api.minimaxi.com/v1/chat/completions`)
- LiteLLM does **not** automatically inject `GroupId` into requests
- This must be handled by the caller (ragflow's chat_model.py)

The implementation appends `GroupId` as a query parameter to `api_base`:
```python
api_base = completion_args.get("api_base", self.base_url)
separator = "&" if "?" in api_base else "?"
completion_args["api_base"] = f"{api_base}{separator}GroupId={self.group_id}"
```

This matches MiniMax's official API format (as documented by
LlamaFactory):
```bash
curl --location 'https://api.minimaxi.chat/v1/text/chatcompletion?GroupId=你的GroupId' \
  --header 'Authorization: Bearer 你的API_Key'
```

## Test plan
- [ ] Verify MiniMax API calls work with GroupId query parameter
- [ ] Verify backward compatibility for other providers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 11:54:49 +08:00
qinling0210
12f80f170c Bump to infinity v0.7.0-dev6 (#14606)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev6

(uv lock --upgrade-package infinity-sdk)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 10:51:17 +08:00
Stephen Hu
53a4edfded refactor: use warp to improve canvas access check logic (#14587)
### What problem does this PR solve?

use warp to improve canvas access check logic

### Type of change

- [x] Refactoring
2026-05-07 10:46:43 +08:00
Jin Hai
1d0519d025 Fix secret key inconsistency cross the RAGFlow servers (#14591)
### What problem does this PR solve?

A and B, two API servers and a REDIS server.
If A and REDIS restart, B will hold the obsolete secret key and will
lead to error.

TODO:
app.config['SECRET_KEY'] and app.secret_key still hold obsolete secret
key.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 10:10:02 +08:00
Wang Qi
15dcdd7b5b Revert "Fix agent permission issue" (#14602)
Reverts infiniflow/ragflow#14597
2026-05-06 20:52:54 +08:00
buua436
3e396c0a72 Fix: add base64 to doc generator output (#14599)
### What problem does this PR solve?
add base64 to doc generator output
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:33:08 +08:00
buua436
faae91d34f Fix: support non-stream runtime agent completion (#14596)
### What problem does this PR solve?

support non-stream runtime agent completion

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:29:15 +08:00
Wang Qi
67e1de50ab Fix agent permission issue (#14597)
### What problem does this PR solve?

Fix agent permission issue.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:17:36 +08:00
Wang Qi
04c5f1b3b6 Bug fix: Support question and custom_header (#14594)
### What problem does this PR solve?

Support question and custom_header

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 19:26:29 +08:00
Haruko386
dd7a0ce1d3 Go: implement provider: lm-studio (#14586)
### What problem does this PR solve?

implement `lm-studio` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-06 19:23:11 +08:00
Vivek Dubey
33d8320ce8 fix: normalize double-escaped LaTeX backslashes and HTML entities (#14564)
Fixes #14562

## Problem
LLMs like DeepSeek V4 Flash and Qwen3-MAX return \\( and \\[ 
(double backslash) in LaTeX output. The preprocessLaTeX() function 
only handled single backslash delimiters, so equations showed as raw
text.
HTML entities like &lt; and &gt; were also not decoded.

## Solution
Added normalization step before existing delimiter conversion:
- \\( → \(  and  \\[ → \[
- &lt; → <  and  &gt; → >  and  &amp; → &

---------

Co-authored-by: Vivek <viveksantoshkumardubey@email.com>
2026-05-06 19:14:34 +08:00
buua436
c9513e5ecb Fix: bootstrap agent replica on demand (#14588)
### What problem does this PR solve?

bootstrap agent replica on demand

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 19:07:50 +08:00
Wang Qi
f32034e83e Refactor: completion -> completions (#14584)
### What problem does this PR solve?

Keep only /completions, deprecated /completion

### Type of change

- [x] Refactoring
2026-05-06 17:19:22 +08:00
buua436
a190a6d67f Fix: add file convert backward compatibility (#14583)
### What problem does this PR solve?

add file convert backward compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 15:19:38 +08:00
Preston Percival
e8f19aa338 feat(graphrag): fix merge concurrency and add resume-from-checkpoint (#14238)
This PR addresses three related GraphRAG reliability issues that
together allow long-running GraphRAG tasks (10+ hours of LLM extraction)
to be resumed after a crash or pause without re-doing completed work. It
builds on #14096 (per-doc subgraph cache) and extends the same idea to
the resolution and community-detection phases.

Fixes #14236.

## 1. Fix concurrent merge crash

Long GraphRAG runs would crash near the end of entity resolution with:
```
RuntimeError: dictionary keys changed during iteration
```
in `Extractor._merge_graph_nodes`. Two changes:

- `rag/graphrag/general/extractor.py`: snapshot `graph.neighbors(node1)`
via `list(...)` before iterating, so concurrent `add_edge` /
`remove_node` mutations on the shared `nx.Graph` cannot invalidate the
iterator. Also tracks each redirected neighbour in `node0_neighbors` so
a later merged node sharing the same external neighbour takes the
edge-merge branch instead of overwriting via `add_edge`.
- `rag/graphrag/entity_resolution.py`: serialize the merge step with a
dedicated `asyncio.Semaphore(1)`. `nx.Graph` is not thread-safe and
concurrent merges on overlapping neighbourhoods can produce incorrect
results even with the snapshot fix.

## 2. Don't wipe partial graph on pause

Previously the pause / cancel UI path called
`settings.docStoreConn.delete({"knowledge_graph_kwd": [...]}, ...)`,
destroying every subgraph, entity, relation, and graph row.
Re-triggering then started GraphRAG from scratch even though #14096 had
already added `load_subgraph_from_store`.

After main was merged in (which deleted `api/apps/kb_app.py` per
#14394), the pause path now lives on the new REST surface `DELETE
/v1/datasets/<id>/<index_type>`:

- `api/apps/services/dataset_api_service.py`: `delete_index` accepts a
`wipe: bool = True` parameter. When `False` the doc-store rows and
GraphRAG phase markers are left intact and only the running task is
cancelled. Default preserves historical behaviour.
- `api/apps/restful_apis/dataset_api.py`: parses `?wipe=false|0|no|off`
from the query string and forwards it.
- `web/src/utils/api.ts` + `web/src/services/knowledge-service.ts`:
`unbindPipelineTask` appends `?wipe=false` when explicitly false.
- The GraphRAG pause action in
`web/src/pages/dataset/dataset/generate-button/hook.ts` passes `wipe:
false` for `KnowledgeGraph`; raptor is unchanged.

**UX impact:** the pause icon next to a running GraphRAG task no longer
wipes graph data. The only path that still wipes is the explicit Delete
action in `GenerateLogButton` (trash icon behind a confirmation modal).

## 3. Phase-completion markers (`rag/graphrag/phase_markers.py`)

A small Redis-backed marker layer at
`graphrag:phase:{kb_id}:{resolution_done|community_done}` (7-day TTL).
`run_graphrag_for_kb` consults the markers on entry and skips phases
that already completed in a prior run. Markers are cleared automatically
when:
- new docs are merged into the graph (which invalidates prior resolution
and community results),
- `delete_index` wipes the graph, or
- `delete_knowledge_graph` is called.

Redis failures never block a run -- markers are an optimization, not a
gate.

## 4. Idempotent community detection

`extract_community` previously did `delete-then-insert` on
`community_report` rows; a crash mid-insert left the dataset with no
reports. Now report IDs are derived deterministically from `(kb_id,
community.title)`, the existing report IDs are snapshotted before
insert, new rows are written, then only stale rows are pruned. A failure
at any step leaves either the prior or the new report set intact --
never a partial mix.

## 5. Tunable doc-store insert pipeline

The GraphRAG insert loop in `rag/graphrag/utils.py` and the
`community_report` insert in `rag/graphrag/general/index.py` were both
hardcoded to `es_bulk_size = 4` and ran strictly sequentially. On a real
KB this meant 1077 chunks took ~21 minutes for a 100-chunk slice -- pure
round-trip overhead.

- New `insert_chunks_bounded()` helper in `rag/graphrag/utils.py`
batches inserts via a bounded `asyncio.Semaphore`. Same retry / timeout
semantics as the prior loop.
- Defaults: 64 docs per batch, 4 batches in flight (matches the regular
ingest pipeline in `document_service.py`). Tunable per-deployment via
`GRAPHRAG_INSERT_BULK_SIZE` and `GRAPHRAG_INSERT_CONCURRENCY`.
- Both `set_graph` and `extract_community` now use the helper.

This dropped the same 1077-chunk insert from minutes to seconds in local
testing without measurable extra pressure on Infinity (total in-flight
docs ≤ `BULK_SIZE × CONCURRENCY` = 256 by default).

## Tests

- `test/unit_test/rag/graphrag/test_merge_graph_nodes.py` (3 tests):
dense neighbourhood merge, neighbour-snapshot regression, concurrent
serialized merges.
- `test/unit_test/rag/graphrag/test_phase_markers.py` (4 tests): set/has
round-trip, kb-scoped clear, no-op on empty input, graceful Redis
failure.
-
`test/testcases/test_web_api/test_dataset_management/test_dataset_sdk_routes_unit.py`:
new `test_delete_index_wipe_flag_unit` covers `wipe=false` for both
GraphRAG and raptor on the new REST route, and confirms the default
still wipes and clears phase markers.

## Compatibility

- Backward compatible: tasks queued before this change behave
identically (default `wipe=true`, no markers expected).
- No schema/migration changes; all new state lives in Redis.
- New optional REST query param `wipe` on `DELETE
/v1/datasets/<id>/<index_type>`.
- New optional env vars `GRAPHRAG_INSERT_BULK_SIZE` and
`GRAPHRAG_INSERT_CONCURRENCY`; defaults preserve safe behaviour.

## Example of resume

Screenshot below shows a test resuming knowledge graph generation after
applying the concurrency fix and re-deploying.

<img width="521" height="677" alt="image"
src="https://github.com/user-attachments/assets/9ef0d405-cbb3-420d-a1a1-e51f3e7e9b7a"
/>

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-06 15:01:01 +08:00
Idriss Sbaaoui
38f6484e98 Fix OpenDataLoader naive parsing by normalizing @OpenDataLoader and filtering unsupported parser kwargs (#14581)
### What problem does this PR solve?
This PR fixes a bug where `layout_recognize="<name>@OpenDataLoader"` was
misrouted and then failed during parsing in the naive parser path. It
now routes correctly to OpenDataLoader and avoids passing unsupported
arguments that caused runtime errors. fixes #14572

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 15:00:55 +08:00
Sebastion
7e83c5f421 fix: authorize beta document downloads by tenant (#14496)
## Summary

This fixes a missing authorization check in the beta API document
download endpoint:

- **CWE:** CWE-862 (Missing Authorization)
- **Severity:** Medium
- **Affected route/file:** `GET /api/v1/documents/<document_id>` in
`api/apps/sdk/doc.py`
- **Data flow:** the route reads a bearer beta API token, resolves the
token with `APIToken.query(beta=token)`, accepts `document_id` directly
from the URL, loads the document with
`DocumentService.query(id=document_id)`, and then fetches the backing
object through `File2DocumentService.get_storage_address()` /
`settings.STORAGE_IMPL.get()`.

Before this change, that flow verified that the API token was valid, but
it did not verify that the token's tenant owned the document's knowledge
base. A caller with any valid beta API token and a known document ID
could therefore reach storage for a document belonging to another
tenant.

## Fix

The endpoint now takes the tenant ID from the resolved API token and
checks the document's knowledge base with:

```python
KnowledgebaseService.query(id=doc[0].kb_id, tenant_id=tenant_id)
```

If the knowledge base is not owned by the token tenant, the request
returns an access error before any storage lookup occurs. This mirrors
the tenant-scoped ownership checks used by the dataset-scoped document
download path and keeps the patch small.

## Tests

Added unit coverage for `download_doc()` to assert that:

- the beta token tenant ID is used in the knowledge-base ownership
lookup;
- cross-tenant access returns `You do not have access to this
document.`;
- storage resolution is not called before tenant authorization succeeds;
- the existing same-tenant empty-file and successful-download paths
still run after the authorization gate passes.

I also verified the final patch is limited to `api/apps/sdk/doc.py` and
the related document SDK route unit test. A local `pytest` invocation
could not complete in this checkout because the shared test fixture
attempts to log in to a RAGFlow server at `127.0.0.1:9380`, which was
not running in the local environment.

## Security analysis

This is exploitable when an attacker has a valid beta API token for
their own tenant and obtains or guesses a document ID from another
tenant. The token alone should not grant access to other tenants' files,
but the direct document route previously authorized only the token
itself and not the requested resource. The new tenant-scoped
knowledge-base check binds the requested document back to the token
tenant before storage is accessed, preventing cross-tenant document
downloads through this endpoint.

Before submitting, we attempted to disprove this by checking whether
existing dataset-scoped routes, token validation, or framework
protections already enforced ownership. They do not apply to this direct
document-ID route: it bypassed the dataset path parameter and used only
`DocumentService.query(id=document_id)` before reading storage.

cc @lewiswigmore
2026-05-06 14:55:41 +08:00
alfaadriel
5e01feb755 fix(connector_service): add TIMEZONE setting and correct interval log… (#14446)
### What problem does this PR solve?



### Type of change

- [v] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wiratama <dafa.wiratama@bankraya.co.id>
2026-05-06 14:40:35 +08:00
euvre
8269fa01b4 Fix AttributeError when appending non-streaming tool calls to chat history in Agentic Agent (#14456)
### What problem does this PR solve?

Fix #14340 

## Problem Description

When using an **Agentic Agent** (not Workflow) with one or more
Retrieval tools (e.g., Dataset Retrieval + Memory Retrieval), the agent
silently returns an empty response (`agent_response: ""`) after hanging
for several minutes. The server logs show:

```
AttributeError: 'ChatCompletionMessageToolCall' object has no attribute 'index'
```

This error propagates as a `GENERIC_ERROR`, causing the canvas to return
an empty response. The subsequent Memory save task then receives the
empty `agent_response` and logs:

```
Document for referred_document_id XXXX not found
```

## Reproduction Steps

1. Set `DOC_ENGINE=infinity` (or `elasticsearch` — the engine itself is
not the root cause).
2. Create a blank **Agentic Agent** (not a Workflow).
3. Add **two Retrieval tools** to the Agent node:
   - `Retrieval_DS` → Dataset (Knowledge Base)
   - `Retrieval_Mem` → Memory component
4. Add a **Message** node with **Save to Memory** enabled.
5. Launch the agent and send any message (e.g., "hola").
6. The agent hangs and returns an empty response.

## Root Cause Analysis

The crash occurs in `_append_history` and `_append_history_batch` inside
`rag/llm/chat_model.py`. These methods directly access `.index` on tool
call objects:

```python
# _append_history_batch
{
    "index": tc.index,   # <-- crashes here
    ...
}
```

However, **non-streaming** LLM responses (`stream=False`) return
`ChatCompletionMessageToolCall` objects, which **do not have an `index`
field** according to the OpenAI API specification. The `index` field
only exists on `ChoiceDeltaToolCall` objects returned in **streaming**
responses (`stream=True`).

When the agentic agent triggers an internal `full_question` call (used
to compress multi-turn conversation history), the request is incorrectly
routed through `async_chat_with_tools` because `is_tools=True` is set at
the `LLMBundle` level. If the LLM decides to emit `tool_calls` during
this auxiliary request, the code enters the non-streaming tool loop and
crashes when trying to append history.

## Fix

Replaced all direct `.index` accesses with `getattr(..., "index", None)`
for safe, backward-compatible access:

| Method | File | Line | Change |
|--------|------|------|--------|
| `_append_history` | `rag/llm/chat_model.py` | ~L304 |
`tool_call.index` → `getattr(tool_call, "index", None)` |
| `_append_history_batch` | `rag/llm/chat_model.py` | ~L332 | `tc.index`
→ `getattr(tc, "index", None)` |
| `_append_history` | `rag/llm/chat_model.py` | ~L1467 |
`tool_call.index` → `getattr(tool_call, "index", None)` |
| `_append_history_batch` | `rag/llm/chat_model.py` | ~L1496 |
`tc.index` → `getattr(tc, "index", None)` |

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-05-06 14:39:40 +08:00
Shiyao Huang
406b36a452 fix(#14389): normalize list metadata values for in filters (#14410)
## Summary
- normalize string items for list-valued metadata filters in
`meta_filter`
- fix `in` / `not in` case asymmetry when document metadata is
lowercased but filter list values are not
- add regression tests that cover the original issue scenario using
uppercase list values

## Validation
- `PYTHONPATH=external/ragflow pytest
external/ragflow/test/unit_test/common/test_metadata_filter_operators.py
-q`

## Notes
- I commented on #14389 before opening this PR to claim the issue.
- The new tests use `value=["F2", "F11"]` so they fail on the old
implementation and pass with this fix.
- This also benefits other non-comparison operators that flow through
the same normalization path.

Co-authored-by: copizza <copizza@users.noreply.github.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-05-06 14:28:25 +08:00
buua436
e4aee25b4b Fix: add legacy agent completion API compatibility (#14582)
### What problem does this PR solve?

add legacy agent completion API compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 14:22:48 +08:00
jony376
94f8779a00 Memory API: enforce tenant permissions on memory and message endpoints (#14535)
### What problem does this PR solve?

This PR fixes missing authorization checks in the Memory API.
Previously, several authenticated endpoints accepted caller-supplied
`tenant_id`, `owner_ids`, or `memory_id` values and used them directly
to list, read, update, delete, or search Memory data.

That could allow an authenticated user to access or mutate another
tenant's Memory records if they knew a tenant ID or memory ID. The fix
centralizes Memory access checks and applies them consistently across
Memory and Memory-message operations.

The change:

- Adds helper logic to parse list filters and compute tenant IDs
accessible to `current_user`.
- Requires direct `memory_id` operations to pass Memory access checks
before reading, updating, deleting, or changing message state.
- Filters list/search/recent-message requests to accessible memories
only.
- Applies Memory visibility filtering before count and pagination in
`MemoryService.get_by_filter`.
- Accepts `owner_ids` in the Memory list route, matching the frontend
owner filter while still intersecting values with the caller's
accessible tenants.
- 

### Related issues
Closes #14534 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-06 14:10:47 +08:00
buua436
5672be0652 Feat: add IMAP deleted document sync (#14539)
### What problem does this PR solve?

add IMAP deleted document sync

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-06 14:06:46 +08:00
NeedmeFordev
89961962c0 feat(dingtalk-ai-table): support deleted-file sync via slim snapshot (#14525)
### What problem does this PR solve?

Incremental DingTalk AI Table (Notable) sync did not reconcile rows
removed on the remote side with documents already in the knowledge base.
This follows the coordinated datasource work in #14362 (“sync deleted
files”).

This PR adds a **full slim snapshot**
(`retrieve_all_slim_docs_perm_sync`) that lists **current record IDs for
all sheets** without building document blobs, using the same logical
document IDs as full ingest
(`dingtalk_ai_table:{table_id}:{sheet_id}:{record_id}`). When
**`sync_deleted_files`** is enabled on incremental runs,
`DingTalkAITable._generate` returns **`(document_generator,
file_list)`** so **`SyncBase`** can run
**`cleanup_stale_documents_for_task`** and remove KB rows that no longer
exist remotely.

Design notes:

- **`_document_id`** centralizes the ID string so slim snapshots and
**`_convert_record_to_document`** stay aligned with
**`hash128(doc.id)`** semantics used during ingestion/cleanup.
- **`end_ts`** is captured before building **`file_list`**, then
**`poll_source`** uses the same upper bound (consistent with other
Dropbox-style connectors).
- **`batch_size`** from connector config is coerced to a positive
**`int`** before constructing the connector.
- Slim snapshot failures are caught in **`_generate`**; **`file_list`**
is set to **`None`** so cleanup is skipped rather than running on
partial/error state.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Files changed (summary)

| Area | Change |
|------|--------|
| `common/data_source/dingtalk_ai_table_connector.py` |
`SlimConnectorWithPermSync`, `retrieve_all_slim_docs_perm_sync`,
`_document_id` shared with document conversion |
| `rag/svr/sync_data_source.py` | `DingTalkAITable._generate`: slim
snapshot + tuple return; `batch_size` validation; shared `end_ts` with
`poll_source` |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`syncDeletedFiles` for DingTalk AI Table in
`DataSourceFeatureVisibilityMap` |

Closes / relates to: #14362
2026-05-06 14:06:23 +08:00
Idriss Sbaaoui
c502001d9e Fix MinerU output fallback and NameError regression (#14538)
### What problem does this PR solve?

This fixes a MinerU parsing failure where output JSON was not found in
nested v0.24.0 layouts, and also fixes a `content_names` NameError in
`_read_output()`. As a result, successful MinerU API runs no longer end
with false “MinerU not found” parsing failures.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 14:03:57 +08:00
sapienza yoan
c0fc8b32f2 Fix: retry RocksDB metadata contention on concurrent CREATE/DROP (#14563)
Concurrent CREATE TABLE / CREATE INDEX / DROP TABLE on the same Infinity
instance can race on the catalog counter (e.g. db|1|next_table_id) and
fail with error 9003 "Resource busy" instead of waiting on a lock. Two
users creating a knowledge base at the same instant, or any deployment
with multiple backend workers behind one Infinity, can hit it.

Wrap the metadata paths in create_idx, create_doc_meta_idx, and
delete_idx with exponential backoff + jitter (5 attempts, 50ms base).
The wrapped operations already use ConflictType.Ignore, so retrying is
idempotent — worst case the second attempt is a no-op against an
already-created table. Tunable via INFINITY_META_RETRY_MAX /
INFINITY_META_RETRY_BASE_DELAY_MS.

Repro: stress 30 concurrent POST /api/v1/datasets against a 4-worker
backend → ~50% of requests fail without the patch (Resource busy from
the second worker that hits the counter), 100% succeed with it. At 100
concurrent requests, all 100 succeed in ~1.2s; the retry budget never
exhausted in our tests.

Scope is limited to metadata paths only — data-path operations (INSERT
chunks, SELECT for retrieval) go through per-table code paths and don't
share the contended counter.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: yoan sapienza <Yoan Sapienza yoan.sapienza@orange.fr Yoan Sapienza zappy@macbookpro.home>
2026-05-06 13:32:20 +08:00
Jack Storment
c2ad672c09 Go: implement provider: xAI (#14550)
Closes #14552 

### What problem does this PR solve?

Add a Go driver for xAI (Grok models).

The config file conf/models/xai.json has been in the repo since the
early Go provider work, but internal/entity/models/factory.go had no
case for "xai". So any xAI request fell through to the dummy driver
and never reached the API. This PR adds the missing driver and wires it
up.

### What this PR includes

- New file internal/entity/models/xai.go with an XAIModel that
implements the ModelDriver interface.
- factory.go: route the "xai" provider name to NewXAIModel.

### How the driver works

- xAI exposes an OpenAI-compatible API at https://api.x.ai/v1.
- ChatWithMessages and ChatStreamlyWithSender post to /chat/completions
in the same shape the moonshot and deepseek drivers use.
- ListModels and CheckConnection call /models to confirm the API key
works and to list available model ids.
- reasoning_content is passed through for grok-3-mini and other xAI
reasoning models, both in the non-stream and stream paths.
- Encode, Rerank, and Balance are not part of the public xAI API at the
moment, so they return a clear "not implemented" or "no such method"
error.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- go build ./internal/entity/models/... in a clean go 1.25 image (the
  go.mod minimum) returns exit 0 with no errors.
- Method set of XAIModel matches the ModelDriver interface: NewInstance,
  Name, ChatWithMessages, ChatStreamlyWithSender, Encode, Rerank,
  ListModels, Balance, CheckConnection.
- Pattern parity with the merged moonshot (#14433), volcengine (#14460),
  minimax (#14478), and vllm (#14532) PRs.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 12:16:37 +08:00
Haruko386
cd54c08e84 Go: implement provider: Ollama (#14580)
### What problem does this PR solve?

implement `Ollama` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 12:03:58 +08:00
Yingfeng
28388993a4 Update README (#14547)
### Type of change

- [x] Documentation Update
2026-05-06 11:57:29 +08:00
qinling0210
7335916868 Use GetChatModel, remove duplicate functions in model_service.go (#14546)
### What problem does this PR solve?

Use GetChatModel, remove duplicate functions in model_service.go

### Type of change

- [x] Refactoring

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 11:33:32 +08:00
dependabot[bot]
9e4f3614de Chore(deps-dev): Bump pillow from 12.1.1 to 12.2.0 (#14578)
As title
2026-05-06 11:08:38 +08:00
Jin Hai
aa57b5bd8b Go: move logger to common module (#14545)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 10:41:58 +08:00
Jin Hai
3a51c27a75 Go: CLI chat with text, image, video (#14573)
### What problem does this PR solve?

```
RAGFlow(user)> chat with 'glm-4.6v-flash@test@zhipu-ai' message 'What are the pics talk about?' image 'https://cdn.bigmodel.cn/static/logo/register.png' 'https://cdn.bigmodel.cn/static/logo/api-key.png'
Answer: The first picture shows a login/register modal with options for phone number login, account login, and WeChat QR code login, along with a prompt for new users to get a 20 million tokens experience package. The second picture displays the API keys management page of a platform, including a warning about API key security and a table listing existing API keys with details like creation time and usage history.
Time: 31.600545
RAGFlow(user)> chat with 'glm-4.6v-flash@test@zhipu-ai' message 'What are the video talk about?' video 'https://cdn.bigmodel.cn/agent-demos/lark/113123.mov'
Answer: Based on the sequence of frames provided, the video is a demonstration of a web search and navigation process.

1.  The video starts with a blank Google search page.
2.  The user types "智谱" (which is the Chinese name for the company Zhipu AI) into the search box.
3.  The search is initiated and the page shows "About 0 results".
4.  The search results load, showing information about Zhipu AI, including its website.
5.  The user clicks on the main website link (www.zhipuai.cn).
6.  The video ends by showing the homepage of Zhipu AI's website, titled "Z.ai GLM Large Model Open Platform".

In summary, the video is about searching for the company "智谱" (Zhipu AI) on Google and then navigating to its official website.
Time: 76.582520
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-05 18:14:39 +08:00
Attili-sys
24af0875e5 Feat/configurable metadata display (#13464)
### What problem does this PR solve?

Currently, RAGFlow's Search and Chat interfaces display only raw
vectorized text chunks during retrieval, without contextual information
about their source documents. Users cannot see document titles, page
numbers, upload dates, or custom metadata fields that would help them
understand and trust the retrieved results.

This PR introduces an **optional metadata display feature** that
enriches retrieved chunks with document-level metadata in both the
Search tab and Chatbot interface.

**Key improvements:**
- **Search results**: Display document metadata as styled badges beneath
chunk snippets
- **Chat citations**: Show metadata in citation popovers and reference
lists for better source context
- **LLM context**: Metadata is injected into the LLM prompt to enable
more accurate, citation-aware responses
- **External API support**: Applications using RAGFlow's SDK retrieval
endpoints (`/v1/retrieval`, `/v1/searchbots/retrieval_test`) can opt-in
via request parameters
- **User control**: Multi-select dropdown UI allows users to choose
which metadata fields to display

**Implementation approach:**
-  Reuses existing `DocMetadataService` infrastructure (no new database
tables or indices)
-  Settings stored in existing JSON configuration fields
(`search_config.reference_metadata`, `prompt_config.reference_metadata`)
-  No database migrations required
-  Disabled by default (fully opt-in and backward-compatible)
-  Dynamic metadata field selection populated from actual document
metadata keys
-  Fixed critical bug where Python's builtin `set()` was shadowed by a
route handler function

**Modified endpoints (all backward-compatible):**
- `POST /v1/retrieval` (Public SDK)
- `POST /v1/searchbots/retrieval_test` (Searchbots)
- `POST /v1/chunk/retrieval_test` (UI/Internal)
- Chat completions endpoints (via `extra_body.reference_metadata` or
`prompt_config`)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


###Images
-
<img width="879" height="1275" alt="image"
src="https://github.com/user-attachments/assets/95b2d731-31ae-45a1-b081-bf5893f52aeb"
/>
<br><br>
<br><br>

<img width="1532" height="362" alt="image"
src="https://github.com/user-attachments/assets/9cebc65b-b7a7-459f-b25e-3b13fa9b638e"
/>
<br><br>
<br><br>

<img width="2586" height="1320" alt="image"
src="https://github.com/user-attachments/assets/2153d493-d899-461f-a7a9-041391e07776"
/>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Attili-sys <Attili-sys@users.noreply.github.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-04-30 23:13:27 +08:00
writinwaters
d38d6e7931 Doc: RAGFlow now supports DeepSeek v4 (#14544)
### What problem does this PR solve?

RAGFlow now supports DeepSeek v4.

### Type of change

- [x] Documentation Update
2026-04-30 20:12:29 +08:00
writinwaters
f14abf858e Doc: Minor editorial updates (#14543)
### What problem does this PR solve?

Minor editorial updates.

### Type of change


- [x] Documentation Update
2026-04-30 20:06:28 +08:00
qinling0210
12af73f2ca Support stream for multimodal chat (#14537)
### What problem does this PR solve?

Support stream for multimodal chat

### Type of change

- [x] Refactoring
2026-04-30 19:33:57 +08:00
Magicbook1108
5fd4579a2f Fix: sync data source empty list (#14530)
### What problem does this PR solve?

Fix: sync data source empty list

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 18:56:43 +08:00
buua436
05ee7f8bb6 Fix: remove delete_documents uuid validation (#14533)
### What problem does this PR solve?

remove delete_documents uuid validation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 18:56:33 +08:00
bitloi
a69e0c73c7 feat(rss): support deleted-file sync (#14493)
### What problem does this PR solve?

Partially addresses #14362.

This PR enables syncing deleted files for RSS data sources.

Previously, RSS incremental sync only returned feed entries whose
timestamps were inside the poll window. If an entry was removed from the
RSS feed, RAGFlow had no full current RSS snapshot to pass into the
shared stale-document cleanup path, so the deleted remote entry could
remain in the knowledge base.

This PR:
- adds `retrieve_all_slim_docs_perm_sync()` to `RSSConnector`
- reuses the same `rss:<md5(stable_key)>` document ID derivation used by
normal RSS ingest
- returns `(document_generator, file_list)` for incremental RSS sync
when `sync_deleted_files` is enabled
- captures the poll end timestamp before snapshot/poll so cleanup does
not race against the same sync window
- adds start/end logs around RSS slim snapshot collection
- exposes the deleted-file sync toggle for RSS in the data source UI

Per maintainer request on related datasource PRs, this PR contains no
test-case changes. Local verification was run with an external script.

Validation:
- `uv run ruff check common/data_source/rss_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`
- `git diff --check`
- `uv run python /tmp/verify_rss_deleted_sync.py --repo
/root/74/ragflow`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 18:56:13 +08:00
NeedmeFordev
bedf9592ef feat(webdav): support deleted-file sync via slim snapshot (#14491)
## What problem does this PR solve?

Incremental WebDAV sync only ingested files whose modification time fell
inside the poll window; documents removed on the WebDAV server were
never removed from the knowledge base. This aligns with
[#14362](https://github.com/infiniflow/ragflow/issues/14362)
(coordinated datasource “sync deleted files” work).

This PR adds a **full-tree slim snapshot**
(`retrieve_all_slim_docs_perm_sync`) that enumerates current remote
paths **without downloading file contents**, using the same logical
document IDs as full ingest (`webdav:{base_url}:{file_path}`). When
**`sync_deleted_files`** is enabled on incremental runs, sync returns
**`(document_generator, file_list)`** so **`SyncBase`** runs
**`cleanup_stale_documents_for_task`** and removes KB rows no longer
present remotely.

Design notes:

- **`_list_files_recursive`** gains **`filter_by_mtime`**: snapshot
passes **`filter_by_mtime=False`** (full tree under **`remote_path`**);
**`poll_source`** keeps mtime-window filtering as before.
- Slim snapshot applies the same **extension** and **`size_threshold`**
rules as **`_yield_webdav_documents`** so retain IDs match what would be
indexed.
- **`end_ts`** is captured before building **`file_list`**, then
**`poll_source`** uses the same upper bound (consistent with
Dropbox-style connectors).

## Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Files changed

| Area | Change |
|------|--------|
| `common/data_source/webdav_connector.py` |
`SlimConnectorWithPermSync`, `retrieve_all_slim_docs_perm_sync`,
`filter_by_mtime` on `_list_files_recursive` |
| `rag/svr/sync_data_source.py` | WebDAV `_generate`: `file_list` +
tuple return; pass **`batch_size`** from connector config |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`syncDeletedFiles` for WebDAV in `DataSourceFeatureVisibilityMap` |
2026-04-30 17:26:27 +08:00
Wang Qi
c363e43664 Fix #14443 (#14536)
### What problem does this PR solve?

Use variable from os.env

### Type of change

- [ ] Documentation Update
2026-04-30 17:21:28 +08:00
Haruko386
93f3b90121 Go: implement provider: Vllm (#14532)
### What problem does this PR solve?

Implement the vLLM model provider for RAGFlow to fully support local and
self-hosted open-source models (e.g., Qwen, GLM, Llama) via the vLLM
framework, and fix several critical bugs related to model instance
management and API requests.

**Key changes and fixes:**
1. **Added Standard vLLM Provider (`vllm.go`, `vllm.json`):**
- Implemented `VllmModel` driver strictly adhering to the OpenAI API
specification.
- Removed hardcoded and dangerous routing logic (e.g., forcing
`AsyncChat` for Qwen/GLM prefixes), ensuring standard
`/v1/chat/completions` compatibility.
- Refactored `ListModels` to use safe JSON parsing (resolving nil
pointer panics) and standard `GET` requests without bodies.
- Added `APIConfig.Region` fallback logic to prevent empty `base_url`
fetching when checking models.

2. **Fixed `ChatToModelStreamWithSender` Bug (`model_service.go`):**
- Resolved the `model is disabled` error when streaming chat with local
database-saved models.
- Added the missing `if modelInfo.Status == "active"` block to correctly
invoke `NewInstance` and inject the dynamic `base_url` into the provider
driver before starting the SSE stream.

3. **Fixed `ListSupportedModels` Bug (`model_service.go`):**
- Added dynamic `NewInstance` injection for `base_url`. Previously, the
list models function used the static JSON config without injecting
user-configured dynamic URLs from the database, resulting in an
`unsupported protocol scheme ""` error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 16:30:14 +08:00
balibabu
00e03a1945 Fix: LaTeX formulas cannot be displayed on the chat page. (#14531)
### What problem does this PR solve?

Fix: LaTeX formulas cannot be displayed on the chat page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 16:12:13 +08:00
qinling0210
265f92c83e Simplify chat and support multimodal chat (#14523)
### What problem does this PR solve?

Simplify chat and support multimodal chat

### Type of change

- [x] Refactoring
2026-04-30 15:25:01 +08:00
Wang Qi
f45ce00347 Not allow to sort by id (#14526)
### What problem does this PR solve?

id as "text", not a "keyword", order by it will cause error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 14:52:43 +08:00
Jin Hai
45d77dc778 Fix version info (#14529)
### What problem does this PR solve?

Fix docker image version info in comment

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-30 14:47:28 +08:00
bitloi
17eda04b8d feat(zendesk): support deleted-file sync (#14487)
### What problem does this PR solve?

Refs #14362.

This PR enables syncing deleted files for Zendesk data sources.

Previously, Zendesk incremental sync never returned a slim remote
snapshot to the shared stale-document cleanup path, so deleted remote
Zendesk records could remain in RAGFlow. The existing Zendesk slim
snapshot also included records that ingestion intentionally skips, such
as draft articles, articles without bodies, skipped-label articles,
empty-body articles, and tickets with `status == "deleted"`.

This PR:
- exposes the deleted-file sync option for Zendesk in the data source UI
- returns Zendesk slim snapshots during incremental sync when
`sync_deleted_files` is enabled
- reuses Zendesk indexability rules so cleanup compares against the same
records ingestion can materialize
- adds start/end logs around Zendesk slim snapshot collection for
operational visibility

Per maintainer request, this PR contains no test-case changes. Manual
verification recording will be provided separately.

Validation:
- `uv run ruff check common/data_source/zendesk_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-30 14:44:05 +08:00
bitloi
8f75e52bbf feat(asana): support deleted-file sync (#14468)
### What problem does this PR solve?

Partially addresses #14362.

Adds deleted-file sync support for the Asana data source. Asana already
indexes task attachments as documents, but it did not provide the slim
document snapshot required by stale-document reconciliation, and the
sync wrapper never returned a `file_list` for cleanup.

This PR:
- adds `retrieve_all_slim_docs_perm_sync()` to `AsanaConnector`
- builds slim IDs with the same `asana:{task_id}:{attachment_gid}`
format used by indexed documents
- avoids downloading attachment blobs during the snapshot
- aborts the snapshot if Asana API errors occur, preventing partial
snapshots from deleting valid local docs
- captures the incremental poll end time before snapshotting and makes
`poll_source()` respect that boundary
- exposes the deleted-file sync toggle for Asana in the data source UI

Per maintainer request, this PR contains no test-case changes. Manual
verification recording will be provided separately.

Validation:
- `uv run ruff check common/data_source/asana_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`
- `git diff --check`

### Type of change

- [x] New Feature
2026-04-30 14:41:36 +08:00
orbisai0security
e992fe39b2 fix: the oceanbase database connector constructs sql... in ob_conn.py (#14470)
## Summary
Fix critical severity security issue in `rag/utils/ob_conn.py`.

## Vulnerability
| Field | Value |
|-------|-------|
| **ID** | V-003 |
| **Severity** | CRITICAL |
| **Scanner** | multi_agent_ai |
| **Rule** | `V-003` |
| **File** | `rag/utils/ob_conn.py:691` |

**Description**: The OceanBase database connector constructs SQL WHERE
clauses by directly embedding user-controlled filter expressions using
Python f-strings at lines 726, 777, 781, 787, 793, 821, and 827. No
parameterization or allowlist validation is applied before the
expressions are incorporated into live SQL queries. This is the most
critical vulnerability in the codebase because it directly exposes the
RAG knowledge base — the platform's core business asset — to complete
compromise.

## Changes
- `rag/utils/ob_conn.py`

## Verification
- [x] Build passes
- [x] Scanner re-scan confirms fix
- [x] LLM code review passed

---
*Automated security fix by [OrbisAI Security](https://orbisappsec.com)*
2026-04-30 14:25:17 +08:00
Yingfeng
4ee0702aed Feat: add skills space to context engine (#13908)
### What problem does this PR solve?

issue #13714

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:36:03 +08:00
Magicbook1108
bb3b99f0a5 Feat: add button for remove header & footer in pipeline (#14486)
### What problem does this PR solve?

Feat: add button for remove header & footer in pipeline

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:30:41 +08:00
NeedmeFordev
2932b65da6 feat(seafile): support deleted-file sync via slim snapshot (#14499)
### What problem does this PR solve?

Incremental Seafile sync only ingests files whose modification time
falls in the poll window; documents removed in Seafile were never
removed from the knowledge base. This contributes to
[#14362](https://github.com/infiniflow/ragflow/issues/14362) (datasource
“sync deleted files” coordination).

This PR adds a **slim snapshot** (`retrieve_all_slim_docs_perm_sync`)
that enumerates current remote file IDs **without downloading content**,
using the same logical IDs as full ingest
(`seafile:{repo_id}:{file_id}`). When **`sync_deleted_files`** is
enabled on incremental runs, **`SeaFile._generate`** returns
**`(document_generator, file_list)`** so **`SyncBase`** can run
**`cleanup_stale_documents_for_task`** and remove stale KB documents.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
### What changed

- **`common/data_source/seafile_connector.py`**: `SeaFileConnector`
implements **`SlimConnectorWithPermSync`**;
**`_list_files_recursive(..., filter_by_mtime=...)`** supports full-tree
listing for snapshots; **`retrieve_all_slim_docs_perm_sync()`** reuses
the same library/root scan as ingest and applies the same **size**
ceiling; logging for snapshot start/end and counts.
- **`rag/svr/sync_data_source.py`**: **`SeaFile._generate`** validates
**`batch_size`**, captures **`end_ts`** before snapshot +
**`poll_source`**, wraps slim retrieval in **`try`/`except`** (
**`file_list = None`** on failure so ingest continues), returns
**`(generator, file_list)`**.
- **`web/src/pages/user-setting/data-source/constant/index.tsx`**:
**`syncDeletedFiles`** for Seafile in
**`DataSourceFeatureVisibilityMap`**.
2026-04-30 12:05:12 +08:00
writinwaters
8aaf0942b1 Doc: Updated v0.25.1 release notes (#14519)
### What problem does this PR solve?

Updated v0.25.1 release notes.

### Type of change


- [x] Documentation Update
2026-04-30 11:59:53 +08:00
Idriss Sbaaoui
9075872435 Fix: Manual/Naive outline tuple unpack crash (#14518)
### What problem does this PR solve?

This fixes a crash in Manual and Naive parsing when PDF outlines include
page numbers as a third tuple value. It makes outline unpacking accept
extra values so parsing no longer fails. fixes #14411

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:55:02 +08:00
dependabot[bot]
71952b6b58 Chore(deps): Bump go.opentelemetry.io/otel from 1.39.0 to 1.41.0 (#14516)
Bumps
[go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go)
from 1.39.0 to 1.41.0.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 11:43:24 +08:00
buua436
06c6da5d94 Fix: add document delete permission check (#14472)
### What problem does this PR solve?

add document delete permission check

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:01:09 +08:00
buua436
47129fdd08 Fix: optimize file batch delete (#14473)
### What problem does this PR solve?

optimize file batch delete

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:00:39 +08:00
sapienza yoan
811e9826d0 perf: avoid O(n²) array growth in embedding accumulation (#14369)
### What problem does this PR solve?

Both tokenizer (`rag/flow/tokenizer/tokenizer.py`) and
`BuiltinEmbed.encode`
(`rag/llm/embedding_model.py`) currently accumulate embedding batches
via
`np.concatenate` inside the per-batch loop. `np.concatenate` allocates a
new
array and copies all existing data on every call, so accumulating N
batches
is O(N²) in both time and peak memory.

Replacing the incremental concatenate with a list-of-batches + a single
`np.vstack` at the end gives O(N) total work.

For tokenizer the title-vector broadcast `np.concatenate([vts[0]] * N)`
is
also replaced by `np.tile`, which does the same job with a single
contiguous
allocation instead of building a Python list of references.

This is purely a CPU/memory optimisation — output shape and dtype are
unchanged. Measured impact grows with document size:
  -   1k chunks (batch 512, 2 iters):    ~negligible
  -  10k chunks (20 iters):              ~10× speedup on this stage
  - 100k chunks (195 iters):             ~100× speedup, and peak RAM
drops from O(N) extra to near-zero

### Type of change

- [x] Performance Improvement

Co-authored-by: yoan sapienza <Yoan Sapienza yoan.sapienza@orange.fr Yoan Sapienza zappy@macbookpro.home>
2026-04-30 11:00:10 +08:00
FuturMix
2548c28d65 feat: add FuturMix as model provider (#14419)
## Summary

Add [FuturMix](https://futurmix.ai) as a new model provider. FuturMix is
an OpenAI-compatible unified AI gateway that provides access to 22+
models (GPT, Claude, Gemini, DeepSeek, and more) through a single API
endpoint and key.

- **API Base**: `https://futurmix.ai/v1` (OpenAI-compatible)
- **Supported capabilities**: Chat, Embedding, Image2Text, TTS,
Speech2Text, Rerank

### Changes

| File | Change |
|------|--------|
| `rag/llm/__init__.py` | Add `FuturMix` to `SupportedLiteLLMProvider`
enum, `FACTORY_DEFAULT_BASE_URL`, and `LITELLM_PROVIDER_PREFIX` |
| `rag/llm/chat_model.py` | Add `FuturMixChat(Base)` — follows
Astraflow/Avian pattern |
| `rag/llm/embedding_model.py` | Add `FuturMixEmbed(OpenAIEmbed)` —
follows Astraflow pattern |
| `rag/llm/cv_model.py` | Add `FuturMixCV(GptV4)` — follows
SILICONFLOW/OpenRouter pattern |
| `rag/llm/tts_model.py` | Add `FuturMixTTS(OpenAITTS)` — follows
CometAPI/DeerAPI pattern |
| `rag/llm/sequence2txt_model.py` | Add `FuturMixSeq2txt(GPTSeq2txt)` —
follows StepFun pattern |
| `rag/llm/rerank_model.py` | Add `FuturMixRerank(OpenAI_APIRerank)` |
| `conf/llm_factories.json` | Add factory config with 8 chat, 2
embedding, 1 image2text, 2 TTS, 1 speech2text models |
| `docs/guides/models/supported_models.mdx` | Add FuturMix to supported
models table |

### Models included

- **Chat**: claude-sonnet-4-20250514, claude-3.5-haiku, gpt-4o,
gpt-4o-mini, gemini-2.5-flash, gemini-2.0-flash, deepseek-chat,
deepseek-reasoner
- **Embedding**: text-embedding-3-small, text-embedding-3-large
- **Image2Text**: gpt-4o
- **TTS**: tts-1, tts-1-hd
- **Speech2Text**: whisper-1

## Test plan

- [ ] Verify FuturMix appears in the model provider list in RAGFlow UI
- [ ] Configure FuturMix with API key and test chat completion
- [ ] Test embedding model with document indexing
- [ ] Test image2text with a sample image

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-30 10:59:37 +08:00
Liu An
ce4c782fd7 Docs: Update version references to v0.25.1 in READMEs and docs (#14488)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.0 to v0.25.1
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-04-30 10:49:26 +08:00
balibabu
7c0584a2b7 Fix: The GraphRAG icon is not displaying. (#14514)
### What problem does this PR solve?

Fix: The GraphRAG icon is not displaying.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 10:44:05 +08:00
dependabot[bot]
0fa2bd539d Chore(deps): Bump google.golang.org/grpc from 1.66.2 to 1.79.3 (#14513)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from
1.66.2 to 1.79.3.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 10:35:03 +08:00
euvre
6dd38eca6a fix: file logs not displayed in dataset ingestion page (#14479)
### What problem does this PR solve?

## Summary

Fixed a bug where the **File Logs** tab in the dataset ingestion page
always showed "No logs" even after files were parsed successfully.

## Root Cause

Both the **File Logs** and **Dataset Logs** tabs on the frontend called
the same backend endpoint `/datasets/{dataset_id}/ingestions`. However,
the backend only queried `get_dataset_logs_by_kb_id`, which
hard-filtered records by `document_id == GRAPH_RAPTOR_FAKE_DOC_ID`
(dataset-level logs). As a result, real file-level logs were never
returned, causing the table to appear empty.

## Changes

### Backend

- **`api/apps/restful_apis/dataset_api.py`**
  - Added two new query parameters to `list_ingestion_logs`:
    - `log_type` — `"file"` or `"dataset"` (default: `"dataset"`)
    - `keywords` — search keyword for filtering by document / task name

- **`api/apps/services/dataset_api_service.py`**
- Updated `list_ingestion_logs` signature to accept `log_type` and
`keywords`.
  - Added conditional routing:
- When `log_type == "file"`, call
`PipelineOperationLogService.get_file_logs_by_kb_id`
- Otherwise, call
`PipelineOperationLogService.get_dataset_logs_by_kb_id`

- **`api/db/services/pipeline_operation_log_service.py`**
- Extended `get_dataset_logs_by_kb_id` with an optional `keywords`
parameter so dataset logs can also be searched.

### Frontend

- **`web/src/pages/dataset/dataset-overview/hook.ts`**
- Removed the separate API function switching (`listPipelineDatasetLogs`
vs `listDataPipelineLogDocument`).
- Unified both tabs to call `listDataPipelineLogDocument` with the new
`log_type` query parameter (`"file"` or `"dataset"`).
  - Ensured `keywords` and filter values are passed through correctly.

## Behavior After Fix

| Tab | `log_type` | Returned Records | Searchable Field |
|---|---|---|---|
| File Logs | `file` | Real document-level logs | `document_name` (file
name) |
| Dataset Logs | `dataset` | GraphRAG / RAPTOR / MindMap logs |
`document_name` (task type) |
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-04-29 22:10:24 +08:00
Wang Qi
5018459112 Fix metadata config (#14480)
### What problem does this PR solve?

Fix metadata config

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 21:09:54 +08:00
writinwaters
d4147efc66 Docs: (#14492)
### What problem does this PR solve?

Added v0.25.1 release notes

### Type of change


- [x] Documentation Update
2026-04-29 20:29:58 +08:00
Wang Qi
c4d0b0ebcf Fix visit dataset error (#14490)
### What problem does this PR solve?

Fix visit dataset error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 20:17:00 +08:00
balibabu
1692f0928f Fix: The pipeline column header in the FileLogsTable is displaying incorrectly. (#14489)
### What problem does this PR solve?
Fix: The pipeline column header in the FileLogsTable is displaying
incorrectly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 19:52:28 +08:00
writinwaters
9280c64518 Docs: Updated Title chunker references (#14483)
### What problem does this PR solve?

Updated Title chunker references

### Type of change

- [x] Documentation Update
2026-04-29 19:37:24 +08:00
Jin Hai
261be81127 Go: add drop instance models (#14485)
### What problem does this PR solve?

1. drop instance model
2. Fix issue of drop instance but not drop models.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 19:18:49 +08:00
Haruko386
0e1477eb23 Go: implement provider: MiniMax (#14478)
### What problem does this PR solve?

implement MiniMax provider

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:06:40 +08:00
Magicbook1108
de8c6ad0f3 Feat: enable sync deleted file for Discord (#14451)
### What problem does this PR solve?

Feat: enable sync deleted file for Discord

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:05:40 +08:00
bitloi
2bc8c6d35e feat(dropbox): support deleted-file sync (#14476)
### What problem does this PR solve?

Partially addresses #14362 by adding deleted-file sync support for the
Dropbox data source.

Dropbox previously did not provide the slim current-file snapshot
required by stale document reconciliation, and its sync runner returned
only document batches. As a result, enabling deleted-file sync could not
remove local documents that had been deleted from Dropbox.

This PR:
- Adds `retrieve_all_slim_docs_perm_sync()` to `DropboxConnector`.
- Reuses Dropbox metadata traversal to collect current remote file IDs
without downloading file contents.
- Wires incremental Dropbox sync to return `(document_generator,
file_list)` when `sync_deleted_files` is enabled.
- Enables the deleted-file sync toggle for Dropbox in the data source
settings UI.
- Adds regression coverage for slim snapshots, nested folders, paginated
listings, duplicate filenames, and full reindex behavior.

Tests:
- `uv run pytest test/unit_test/common/test_dropbox_connector.py -q`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `uv run pytest test/unit_test/common/test_dropbox_connector.py
test/unit_test/rag/test_sync_data_source.py -q`
- `uv run ruff check common/data_source/dropbox_connector.py
rag/svr/sync_data_source.py
test/unit_test/common/test_dropbox_connector.py
test/unit_test/rag/test_sync_data_source.py`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:05:11 +08:00
Magicbook1108
db1a73b255 Feat: enable sync deleted files in gitlab (#14481)
### What problem does this PR solve?

Feat: enable sync deleted files in gitlab

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:04:10 +08:00
euvre
a0f9ae16d2 Fix: RAPTOR "Generation scope" reset to "Single file" when selecting "Dataset" (#14477)
## Problem
In the Dataset Configuration page, changing the RAPTOR **Generation
scope** from "Single file" to "Dataset" and clicking **Save** did not
persist the change. After refreshing or re-entering the page, the scope
always reverted to "Single file".

## Root Cause
1. **Backend**: The `RaptorConfig` Pydantic model in
`api/utils/validation_utils.py` was configured with `extra="forbid"` but
did not declare a `scope` field. When the frontend sent `"scope":
"dataset"`, Pydantic rejected the request.
2. **Frontend**: The `extractRaptorConfigExt` utility in
`web/src/hooks/parser-config-utils.ts` treated `scope` as an unknown
field and moved it into the nested `ext` object. Consequently, the
backend could not read `raptor_config.get("scope", "file")` correctly,
so the default `"file"` was always used.

## Changes
- Added `scope: Literal["file", "dataset"]` to the backend
`RaptorConfig` model with a default of `"file"`.
- Added `scope` to the known-field whitelist in the frontend
`extractRaptorConfigExt` helper so it is transmitted as a top-level
raptor field instead of being buried in `ext`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-29 18:46:28 +08:00
Wang Qi
1b84892e3a Fix delete graph (#14484)
### What problem does this PR solve?

Fix delete graph

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 18:09:10 +08:00
Wang Qi
3991bdfaf5 Fix graph task type (#14475)
### What problem does this PR solve?
Fix graph task type

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:05:56 +08:00
Jin Hai
bb05a8bd7e Update create model instance command (#14441)
### What problem does this PR solve?

1. support command:

```
RAGFlow(user)> create provider 'vllm' instance 'test' key 'test-key' url 'base-url' region 'abc';
SUCCESS
RAGFlow(user)> list instances from 'vllm';
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| apiKey   | extra                                  | id                               | instanceName | providerID                       | status |
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| test-key | {"base_url":"base-url","region":"abc"} | 40213c89430311f1a7cf38a74640adcc | test         | b4d40e6142d311f1a4f938a74640adcc | enable |
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
```
2. support add vllm model
```
RAGFlow(user)> add model 'Qwen/Qwen2-0.5B' to provider 'vllm' instance 'test' with tokens 131072 chat;
SUCCESS
```
3. add vllm chat

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 17:05:08 +08:00
qinling0210
486ca463aa Port PR14454 to GO (PruneDeletedChunks) (#14463)
### What problem does this PR solve?

Port PR14454 to GO (PruneDeletedChunks)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:04:22 +08:00
Magicbook1108
e0b3070012 Feat: enable sync deleted files for Gmail && fix google drive issues (#14462)
### What problem does this PR solve?

Feat: enable sync deleted files for Gmail && fix google drive issues

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: bill <yibie_jingnian@163.com>
Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-29 17:03:56 +08:00
balibabu
a736948493 Fix: Clicking the button in the bottom-right corner of the /chats/widget page fails to display the dialog box. (#14465)
### What problem does this PR solve?

Fix: Clicking the button in the bottom-right corner of the
`/chats/widget` page fails to display the dialog box.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:03:33 +08:00
Wang Qi
6afb1957d8 Fix query param type (#14471)
### What problem does this PR solve?

Fix query param type

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 16:53:28 +08:00
Wang Qi
9690923516 Fix delete graphrag raptor (#14469)
### What problem does this PR solve?

Fix delete graphrag raptor

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 16:47:42 +08:00
Haruko386
decf673049 Go: implement provider: volcengine (#14460)
### What problem does this PR solve?

implement `volcengine` provider 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 15:45:08 +08:00
Wang Qi
b684c89950 Add backward compat APIs (#14427)
### What problem does this PR solve?

Add backward compat APIs:

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 15:15:49 +08:00
buua436
c08ced09a7 Fix: add retrieval fallback comments (#14457)
### What problem does this PR solve?

add retrieval fallback comments

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 14:44:31 +08:00
qinling0210
f3c232cf47 Remove model_bundle.go, modify chat_session.go (#14458)
### What problem does this PR solve?

Remove model_bundle.go, modify chat_session.go

### Type of change

- [x] Refactoring
2026-04-29 14:44:12 +08:00
balibabu
ce933357c6 Fix: Dataset: When configuring the "general chunk method," options such as chunk size and parent-child slicing are unavailable. (#14459)
### What problem does this PR solve?

Fix: Dataset: When configuring the "general chunk method," options such
as chunk size and parent-child slicing are unavailable.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-29 14:37:48 +08:00
buua436
a7ce1b1677 Fix: prune deleted doc chunks from retrieval (#14454)
### What problem does this PR solve?

prune deleted doc chunks from retrieval

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 13:03:09 +08:00
Jin Hai
b493a33316 Go: update chat URL (#14453)
### What problem does this PR solve?

Update the URL to: /api/v1/chat/completions

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 11:45:06 +08:00
Magicbook1108
3b7a6eaa6c Feat: sync deleted files in Bitbucket (#14450)
### What problem does this PR solve?

Feat: sync deleted files in Bitbucket

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 11:29:17 +08:00
Paras Sondhi
74fa54f122 feat(google-drive): optimize memory payload and enable sync deletion (#14372)
**Addresses the Google Drive integration for #14362**

This PR completely overhauls the Google Drive sync logic to accurately
detect remote deletions, while drastically reducing the memory footprint
during the snapshot phase.

### What changed under the hood:

* **Killed the memory bloat:** Swapped out the massive document
dictionary objects for a lightweight `collections.namedtuple` (`SlimDoc
= namedtuple('SlimDoc', ['id'])`). This prevents RAM spikes during
`retrieve_all_slim_docs_perm_sync` on massive enterprise drives.
* **Flawless downstream integration:** The `SlimDoc` object relies on
simple duck typing. It perfectly delivers the `.id` attribute required
by `ConnectorService.cleanup_stale_documents_for_task`, meaning your
core `hash128` vector cleanup logic runs natively without modification.
* **Fixed the Shared Drive blindspot:** The standard API query was
missing team folders. Injected the `corpora="allDrives"` and
`includeItemsFromAllDrives=True` override flags so the connector now
accurately maps state across both personal workspaces and organizational
Shared Drives.

### Testing:
Isolated the Google API retrieval logic locally to prove the `SlimDoc`
mapping works and correctly registers state drops when a file is trashed
remotely.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Performance Improvement
2026-04-29 10:04:36 +08:00
Stephen Hu
345bec812d refactor: improve QwenRerank logic (#14388)
### What problem does this PR solve?

improve QwenRerank logic

### Type of change

- [x] Refactoring
2026-04-28 20:17:34 +08:00
Magicbook1108
0d18b293f5 Fix: enable sync deleted file in airtable (#14438)
### What problem does this PR solve?

Fix: enable sync deleted file in airtable

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 20:09:08 +08:00
Magicbook1108
926efbd29b Fix: update based on #14436 (#14440)
### What problem does this PR solve?

Fix: update based on #14436

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 20:08:42 +08:00
euvre
35f6d81b73 Refactor: migrate chunk retrieval_test and knowledge_graph to REST API endpoints (#14402)
### What problem does this PR solve?

## Summary

Migrate two web API endpoints to REST-style HTTP API endpoints,
following the pattern established in #14222:

| Old Endpoint | New Endpoint |
|---|---|
| `POST /v1/chunk/retrieval_test` | `POST
/api/v1/datasets/<dataset_id>/search` |
| `GET /v1/chunk/knowledge_graph` | `GET
/api/v1/datasets/<dataset_id>/graph` |
2026-04-28 20:00:26 +08:00
Magicbook1108
85575259ac Fix: google authentication - gmail && google-drive (#14422)
### What problem does this PR solve?

Fix: google authentication - gmail && google-drive

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 18:09:02 +08:00
qinling0210
dcce864d4c Simplify Encode (#14437)
### What problem does this PR solve?

Simplify Encode

### Type of change

- [x] Refactoring
2026-04-28 18:07:42 +08:00
Magicbook1108
d532151be0 Feat: more model for paddle (#14436)
### What problem does this PR solve?

Feat: more model for paddle
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 18:07:00 +08:00
Haruko386
4e5a093ac5 Go: implement provider: Moonshot (#14433)
### What problem does this PR solve?

implement `Moonshot` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 18:06:25 +08:00
Jack
c330005659 Fix: document level auto metadata config missing after save (#14421)
### What problem does this PR solve?

Steps to re-produce (existing bug before API migration):

create a new dataset
upload a file 
click on "General" in "Parse" column and then click on "switch or
configure ingestion pipeline"
click on "Settings" (at right of "Auto metadata")
click "Add" to add new metadata
click on "Save"
re-open "Settings" and the newly added metadata is not there

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 17:09:23 +08:00
buua436
e6e80041f5 Fix: agent toolcall null response & schema validation & DeepSeek think history (#14425)
### What problem does this PR solve?
agent toolcall null response & schema validation & DeepSeek think
history

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 17:09:08 +08:00
Jin Hai
f670913bb4 Refactor model type to model class (#14426)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 16:05:15 +08:00
Jin Hai
7c25870923 Go: update db model (#14423)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 16:04:55 +08:00
Magicbook1108
18fbfafca6 Feat: enable sync deleted files for more connectors (#14353)
### What problem does this PR solve?

Feat: enable sync delted files for connectors

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 15:07:14 +08:00
NeedmeFordev
0df65d358a Fix case-insensitive matching for manual meta_data_filter in / not in list values (#14397)
## Summary

Fixes case-asymmetric matching for manual `meta_data_filter` when using
**`in`** / **`not in`** with a **list** `value`. Document metadata
strings were lowercased, but list elements were not, so values like
`"F2"` failed to match `["F2", "F11"]` even though **`=`** behaved
correctly.

Closes #14389

## Changes

- **`common/metadata_utils.py`**: For **`in`** / **`not in`**, normalize
string elements when `value` and/or `input` is a list, consistent with
scalar string lowercasing.
- **`test/unit_test/common/test_metadata_filter_operators.py`**:
Regression tests for list `value` case-insensitivity and **`not in`**.

## Type of change

- [x] Bug fix (non-breaking)
2026-04-28 14:51:48 +08:00
Idriss Sbaaoui
2a37562791 Fix manual naive parser position extraction fallback (#14420)
### What problem does this PR solve?
This PR fixes a regression where Manual pipeline + Naive (Plain Text)
PDF parsing crashed with `AttributeError: 'PlainParser' object has no
attribute 'extract_positions'` in `rag/app/manual.py`.
fixes #14411 
### Type of change:
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 14:21:30 +08:00
Jin Hai
ae420f6358 Go: fix compilation (#14418)
### What problem does this PR solve?

Add methods to volcengine

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 13:21:05 +08:00
qinling0210
effc84a042 Refactor model in GO (#14398)
### What problem does this PR solve?

Refactor model in GO

### Type of change

- [x] Refactoring
2026-04-28 12:59:01 +08:00
Wang Qi
5885691c68 Always return success if no such task id (#14417)
### What problem does this PR solve?

Always return success if no such task id to follow existing code logic.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:55:24 +08:00
buua436
444e564329 Fix: align chat recommendation and thumbup APIs (#14413)
### What problem does this PR solve?
align chat recommendation and thumbup APIs
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:55:16 +08:00
buua436
7a70a0fd85 Fix: preserve infinity available_int zero filter (#14416)
### What problem does this PR solve?

preserve infinity available_int zero filter

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:54:32 +08:00
Jin Hai
819257f257 Go: add volcengine (#14409)
### What problem does this PR solve?

1. Refactor server_main
2. Add volcengine

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 12:12:58 +08:00
Jack
2d522ccb36 Fix: thumbnails issue in chat (#14415)
[Uploading part_4-13.pdf…]()
### What problem does this PR solve?

In chat, the thumbnails didn't display correctly

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)

Steps to reproduce:
1. create dataset and upload a file (see attached)
2. parse the document
3. once parsing completed, create a chat and associate it with the
dataset
4. ask a question (DAP VS DAPE comparison)
5. check result
2026-04-28 11:39:29 +08:00
writinwaters
0cf105da8d Doc: Added a database schema and migration guide. (#14404)
### What problem does this PR solve?

Added a database schema and migration guide.

### Type of change


- [x] Documentation Update
2026-04-28 09:54:33 +08:00
Jack
c81081f8ef Refactor: Doc change parser (#14327)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/change_parser
HTTP API: PATCH /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API
PATCH /api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-27 23:42:57 +08:00
Jack
872ff08304 Fix: add executor.shutdown (#14403)
### What problem does this PR solve?

Add executor shutdown in finally clause to free resources.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 22:38:43 +08:00
Jack
c5116b90e5 Refactor: migrate document thumbnails API (#14344)
### What problem does this PR solve?

Before migration: GET /v1/document/thumbnails
After migration:  GET /api/v1/thumbnails

### Type of change

- [x] Refactoring
2026-04-27 21:29:09 +08:00
Jack
49912a156e Refactor: migrate document run api (#14351)
### What problem does this PR solve?

Before migration: POST /v1/document/run
After migration: POST /api/v1/documents/ingest/

### Type of change

- [x] Refactoring
2026-04-27 21:25:58 +08:00
Jin Hai
965717c4fb Go: add new provider: google (#14395)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-27 20:35:47 +08:00
Jack
343bda1119 Refactor: deco document upload_and_parse API (#14366)
### What problem does this PR solve?

remove unused "POST /v1/document/upload_and_parse"

### Type of change

- [x] Refactoring
2026-04-27 20:35:00 +08:00
euvre
d78013964a tests: add missing HTTP API tests for dataset management endpoints removed in #14222 (#14390)
### What problem does this PR solve?

### Summary

PR #14222 consolidated KB (web) API endpoints into RESTful Dataset
(HTTP) API endpoints and deleted the web API test suite under
`test_web_api/test_kb_app/` and `test_web_api/test_document_app/`. While
most test coverage was migrated to the HTTP API test suite, some tests
were not ported over. This PR adds back the missing coverage.

### Route migration reference

| Old Web API | New HTTP API | Missing tests |
|---|---|---|
| `POST /v1/kb/update_metadata_setting` | `PUT
/api/v1/datasets/<id>/metadata/config` | auth & error paths |
| `GET /api/v1/datasets/<id>/auto_metadata` | `GET
/api/v1/datasets/<id>/metadata/config` | auth & CRUD |
| `PUT /api/v1/datasets/<id>/auto_metadata` | `PUT
/api/v1/datasets/<id>/metadata/config` | auth & CRUD |
| `GET /v1/kb/<kb_id>/basic_info` | `GET
/api/v1/datasets/<id>/ingestions/summary` | covered |
| `POST /v1/kb/list_pipeline_logs` | `GET
/api/v1/datasets/<id>/ingestions` | edge cases missing |

### Changes

#### `test_file_management_within_dataset/test_metadata_config.py` (new,
10 tests)

Covers `GET/PUT /datasets/<id>/metadata/config` (migrated from
`test_kb_tags_meta.py`'s `test_update_metadata_setting` and
`test_document_metadata.py`'s negative tests):
- Authorization for dataset metadata config GET/PUT
- Authorization for document metadata config PUT
- Success, invalid dataset, missing payload, not found scenarios

#### `test_dataset_management/test_ingestion_logs.py` (extended, +2
tests)

Covers `GET /datasets/<id>/ingestions` edge cases (migrated from
`test_kb_pipeline_tasks.py`):
- Missing dataset ID
- Abnormal date filter

### Type of change

- [x] Other: Test coverage improvement

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 20:01:28 +08:00
Jack
a536980e22 Refactor: Doc batch change status (#14337)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/change_status

After consolidation, Restful API
POST /api/v1/datasets/<dataset_id>/documents/batch-update-status 

### Type of change

- [x] Refactoring
2026-04-27 20:00:23 +08:00
buua436
c949096db0 Refactor: optimize agent reset conversation variable defaults (#14401)
### What problem does this PR solve?
optimize agent reset conversation variable defaults
### Type of change
- [x] Refactoring
2026-04-27 19:57:56 +08:00
Wang Qi
488c3ef6a3 Add task API (#14393)
### What problem does this PR solve?

Add task API

### Type of change

- [x] Refactor
2026-04-27 19:16:37 +08:00
buua436
82313020c7 Refa: align list operations and strict mode (#14387)
### What problem does this PR solve?

align list operations and strict mode

### Type of change
- [x] Refactoring
2026-04-27 19:13:00 +08:00
Jack
c1941fd503 Refactor: deco doc-parse API that is not used any more (#14367)
### What problem does this PR solve?

Delete un-used API "POST /v1/document/parse"

### Type of change

- [x] Refactoring
2026-04-27 18:54:49 +08:00
buua436
4f6651968a Fix: prioritize explore session ID and reset default conversation variables (#14399)
### What problem does this PR solve?

 prioritize explore session ID and reset default conversation variables

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 18:52:40 +08:00
mginfn
10e28e5c5f Helm template ragflow.yaml: fix nginx-config-volume mountPath according to Dockerfile v0.25.0 (#14361)
### What problem does this PR solve?

Dockerfile v0.25.0 expects nginx conf at path
/etc/nginx/ragflow.conf.python, see
[Dockerfile#L200](ca01c7a745/Dockerfile (L200))
However current helm template mount the conf at path
/etc/nginx/ragflow.conf causing runtime error at startup time.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Mauro Gattari <mauro.gattari@infn.it>
2026-04-27 18:51:55 +08:00
buua436
0f2778efe7 Fix: support release in agent update api (#14396)
### What problem does this PR solve?

support release in agent update api

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 17:35:35 +08:00
Jack
61a24a2c14 Refactor: migrate doc upload info used in chat (#14359)
### What problem does this PR solve?

Before migration: POST /v1/document/upload_info/
After migration: POST /api/v1/documentss/upload/

### Type of change

- [x] Refactoring
2026-04-27 16:58:42 +08:00
Zhichang Yu
c446c403de perf: lazy img_np loading and chunked parse_into_bboxes for large PDFs (#14385)
## Summary

- **Lazy img_np loading**: `np.array(img)` is now deferred until the
first OCR text extraction is actually needed, avoiding unnecessary
memory allocation for pages that already have text.
- **Chunked parse_into_bboxes**: Large PDFs (>50 pages, configurable via
`PDF_PARSER_PAGE_BATCH_SIZE`) are processed in batches. Each chunk's
boxes are normalized with `_to_global_boxes` to produce globally
consistent page numbers and position tags.
- **DLA early init**: Move remote-client initialization before model
loading in `LayoutRecognizer.__init__` so `DEEPDOC_URL` (or legacy
`TENSORRT_DLA_SVR`) short-circuits unnecessary model download for parser
containers relying on remote inference.
- **Fix outline regression**: Restore `self.outlines =
extract_pdf_outlines(fnm)` in `parse_into_bboxes`; this was dropped
during refactoring and is required by downstream `remove_toc` and
metadata handling in `rag/flow/parser/parser.py`.

## Test plan

- [ ] Small PDF (<=50 pages): verify parse succeeds and `self.outlines`
is populated
- [ ] Large PDF (>50 pages): verify chunked processing produces globally
consistent page numbers
- [ ] With `DEEPDOC_URL` set: verify remote DLA client is used and local
model is not downloaded
- [ ] With legacy `TENSORRT_DLA_SVR` set: verify backward compatibility

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 16:52:43 +08:00
Idriss Sbaaoui
4303be223f Fix metadata parsing regression for upgraded v0.24 datasets (#14383)
### What problem does this PR solve?

This PR fixes issue #14371 where file parsing failed after upgrading
from v0.24.0 to v0.25.0, because metadata config could be a JSON Schema
object but was handled like a list and later caused `KeyError:
'properties'`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 16:18:06 +08:00
Wang Qi
d88f7ac8d2 Remove evaluation_app.py and kb_app.py (#14394)
### What problem does this PR solve?

Delete not used APIs

### Type of change

- [x] Refactoring
2026-04-27 16:08:54 +08:00
Jack
290f0294d6 Refactor: migrate artifact API (#14348)
### What problem does this PR solve?

Before migration: GET /v1/document/artifact/<filename>
After migration:  GET /api/v1/documents/artifact/<filename>

### Type of change

- [x] Refactoring
2026-04-27 15:19:41 +08:00
euvre
2846a93998 Fix: Remove hardcoded page limits causing parsing failures on large PDFs (>300 pages) (#14382)
### What problem does this PR solve?

Fixes #14196

## Problem

When using DeepDOC to parse large PDFs (over 1000 pages), the parser
silently truncated processing at 300 pages due to a hardcoded default
`page_to=299` in `RAGFlowPdfParser.__images__()`. This caused:

- **Errors** on pages beyond the limit
- **Poor image quality** as the parser attempted to compensate with
missing page data
- **Inconsistent chunk splitting** between full PDF imports and partial
imports

Additionally, the codebase scattered magic numbers (`299`, `600`,
`10000`, `100000`, `100000000`, `10000000000`, `10**9`) across 22 files
as sentinel values for "parse all pages", making future maintenance
error-prone.

## Root Cause

```python
# deepdoc/parser/pdf_parser.py (before)
def __images__(self, fnm, zoomin=3, page_from=0, page_to=299, callback=None):
    # Only the first 300 pages were rendered; everything beyond was silently dropped
```

While most callers in `rag/app/*.py` correctly passed `to_page=100000`,
the base class `RAGFlowPdfParser.__call__()` and `parse_into_bboxes()`
invoked `__images__` **without** forwarding `page_from`/`page_to`,
falling back to the restrictive default of 299.

## Solution

### 1. Define constants in `common/constants.py`

```python
MAXIMUM_PAGE_NUMBER = 100000                        # Used by the parsing layer
MAXIMUM_TASK_PAGE_NUMBER = MAXIMUM_PAGE_NUMBER * 1000  # Used by the task/DB layer
```

### 2. Replace all hardcoded sentinel values

| Layer | Files Changed | Old Values | New Value |
|---|---|---|---|
| **Deepdoc parsers** | `pdf_parser.py`, `mineru_parser.py`,
`docling_parser.py`, `opendataloader_parser.py`, `paddleocr_parser.py`,
`docx_parser.py` | `299`, `600`, `10**9`, `100000000` |
`MAXIMUM_PAGE_NUMBER` |
| **Chunk parsers** | `naive.py`, `book.py`, `qa.py`, `one.py`,
`manual.py`, `paper.py`, `presentation.py`, `laws.py`, `resume.py`,
`email.py`, `table.py` | `100000`, `10000`, `10000000000` |
`MAXIMUM_PAGE_NUMBER` |
| **Task/DB layer** | `db_models.py`, `task_service.py`,
`document_service.py`, `file_service.py` | `100000000` |
`MAXIMUM_TASK_PAGE_NUMBER` |

### 3. Fix `parse_into_bboxes()` missing parameters

Added `from_page`/`to_page` parameters to `parse_into_bboxes()` so that
the `rag/flow/parser/parser.py` DeepDOC path no longer falls back to the
restrictive default.

## Files Changed (22)

- `common/constants.py`
- `deepdoc/parser/pdf_parser.py`
- `deepdoc/parser/mineru_parser.py`
- `deepdoc/parser/docling_parser.py`
- `deepdoc/parser/opendataloader_parser.py`
- `deepdoc/parser/paddleocr_parser.py`
- `deepdoc/parser/docx_parser.py`
- `rag/app/naive.py`
- `rag/app/book.py`
- `rag/app/qa.py`
- `rag/app/one.py`
- `rag/app/manual.py`
- `rag/app/paper.py`
- `rag/app/presentation.py`
- `rag/app/laws.py`
- `rag/app/resume.py`
- `rag/app/email.py`
- `rag/app/table.py`
- `api/db/db_models.py`
- `api/db/services/task_service.py`
- `api/db/services/document_service.py`
- `api/db/services/file_service.py`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 14:57:20 +08:00
Jin Hai
c3eac4103a Go: aliyun model provider (#14379)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-27 14:53:33 +08:00
buua436
0b46ab07c5 Refa: restore openai-compatible chat completions api (#14380)
### What problem does this PR solve?
restore openai-compatible chat completions api
### Type of change

- [x] Refactoring
2026-04-27 14:02:19 +08:00
LeonTung
6a23dfeec1 chore(CLAUDE.md): add shared UI component lock convention to CLAUDE.md (#14381)
### What problem does this PR solve?

AI coding agents (Claude, Copilot, etc.) tend to directly edit files in
`src/components/ui/` when asked to tweak styles or add props, treating
them like ordinary feature code. This silently breaks the shared
component library that both shadcn primitives and project-authored
common components live in.

This PR adds a `Shared UI Component Lock` convention to `web/CLAUDE.md`
to instruct AI agents to treat the entire `src/components/ui/` directory
as read-only. Any customization must be done via wrappers or composition
outside the directory; exceptions require explicit user approval.

### Type of change
- [x] Other (please describe): Update `CLAUDE.md`
2026-04-27 12:03:32 +08:00
yuch85
0d87cecae2 feat: persist PDF bookmark outline as document metadata (#13287)
## Summary

PDF files often contain a bookmark/outline tree (table of contents built
into the file by the authoring tool). RAGFlow's `pdf_parser.outlines`
already extracts these `(title, depth)` tuples via pypdf, but they are
used ephemerally during chunking (`manual` parser uses them for
hierarchy detection) and then discarded.

This PR persists the outline as `doc.meta_fields["outline"]` — a JSON
array of `{"title": str, "depth": int}` objects — so downstream features
can use the structural information.

### Why this matters

- **Complementary to `toc_extraction`** — the existing `toc_extraction`
feature uses LLM calls to generate a TOC and only works for the `naive`
parser. The raw PDF outline is free (already extracted by pypdf), works
for all parsers, and captures the author's original document structure.
- **Document navigation** — frontends can render a clickable TOC from
the outline
- **Entity extraction** — the outline provides a structural map for
identifying document sections and key topics
- **Search result context** — knowing which section a chunk belongs to
helps users evaluate relevance

### Changes

| File | Change | LOC |
|------|--------|-----|
| `rag/app/naive.py` | Attach `pdf_parser.outlines` as `__outline__` on
first chunk dict | ~7 |
| `rag/app/manual.py` | Same for the manual parser | ~5 |
| `rag/svr/task_executor.py` | Extract `__outline__`, persist via
`DocMetadataService.update_document_metadata()` | ~12 |

### Design decisions

- **Transient key pattern**: The outline is passed from parser →
task_executor via `__outline__` on the first chunk dict, then removed
before indexing. This follows the same pattern as `metadata_obj` for
LLM-generated metadata.
- **No schema changes**: Uses the existing `meta_fields` JSON column on
the document table.
- **Graceful degradation**: If a PDF has no outline (common for scanned
docs), nothing is stored. If persistence fails, it logs a warning and
continues — parsing is not interrupted.

### Backward compatibility

- **Fully backward compatible** — no existing fields, behavior, or
schemas changed
- PDFs without outlines are unaffected
- Existing `meta_fields` data is preserved (merged, not overwritten)

## Test plan

- [ ] Parse a PDF with bookmarks (e.g. any multi-chapter document),
verify `meta_fields["outline"]` is populated
- [ ] Parse a PDF without bookmarks, verify no errors and no outline key
in meta_fields
- [ ] Verify existing `meta_fields` data is preserved (not overwritten)
when outline is added
- [ ] Verify `manual` parser also persists outlines
- [ ] Verify outline JSON structure: `[{"title": "Chapter 1", "depth":
0}, ...]`

Related: #9921 (Deterministic Document Access Layer)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuch85 <yuch85.1@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-04-27 11:57:06 +08:00
euvre
f3b7d55a1e fix: handle Infinity table-not-exist error (3022) in update() methods (#14153)
### What problem does this PR solve?

## Summary

Closes #6102

When using Infinity as the document store engine (GPU version), calling
`update()` on a non-existent table throws an unhandled
`InfinityException` with error code 3022 (`TABLE_NOT_EXIST`). This
causes users to see a raw "3022" error when clicking on a parsed
document.

## Root Cause

The `update()` methods in both `rag/utils/infinity_conn.py` and
`memory/utils/infinity_conn.py` call `db_instance.get_table(table_name)`
without catching `InfinityException`. In contrast, other CRUD methods
(`insert`, `delete`, `search`) all handle this exception gracefully:

| Method   | Handles table-not-exist? | Behavior |
|----------|--------------------------|----------|
| `insert`  |  Yes | Auto-creates the table |
| `search`  |  Yes | Skips the table |
| `delete`  |  Yes | Returns 0 |
| `update`  |  **No** | Crashes with 3022 |

Additionally, `api/apps/document_app.py` worked around this with a
fragile string match (`"3022" in msg`) to detect the error.

## Changes

- **`rag/utils/infinity_conn.py`**: Catch `InfinityException` in
`update()`. When `TABLE_NOT_EXIST` is detected, log a warning and return
`False` — consistent with `delete()`.
- **`memory/utils/infinity_conn.py`**: Apply the same fix to its
`update()` method.
- **`api/apps/document_app.py`**: Remove the fragile `"3022"`
string-matching workaround. Table-not-exist is now handled by the `if
not ok` path with an improved error message.

### Type of change

- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 11:52:22 +08:00
euvre
33bb464ce3 fix: skip canvas SSE fetch in chat shared page to eliminate spurious 103 error (#14190)
## What does this PR do?

Fixes the `hint : 103 Only owner of canvas authorized for this
operation` error that appears when opening a **Chat** shared link
(`/chats/share?shared_id=...&from=chat`).

## Root Cause

The Chat shared page (`web/src/pages/next-chats/share/index.tsx`)
unconditionally calls `useFetchFlowSSE()`, which requests
`/api/canvas/getsse/{sharedId}`. This is an Agent Canvas endpoint that
validates canvas ownership. When sharing a **Chat** dialog (not an
Agent):

1. `sharedId` is a `dialog_id`, not a `canvas_id`
2. The API token's `tenant_id` doesn't match any canvas owner
3. The backend returns `code: 103, message: "Only owner of canvas
authorized for this operation."`
4. The global error interceptor in `request.ts` displays it as a
notification: `hint : 103 Only owner of canvas authorized for this
operation.`

## Changes

- **`web/src/hooks/use-agent-request.ts`**: Added an `enabled` parameter
to `useFetchFlowSSE` so callers can conditionally skip the query.
- **`web/src/pages/next-chats/share/index.tsx`**: Only enable
`useFetchFlowSSE` when `from === SharedFrom.Agent`. For Chat shares, the
hook is disabled, avoiding the unnecessary canvas API call entirely.

## Related Issue

Closes #14115

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 11:27:39 +08:00
yuch85
3ad3241ae0 feat: persist RAPTOR layer metadata on summary chunks (#13286)
## Summary

RAPTOR's recursive clustering builds a `layers` list tracking
`(start_idx, end_idx)` boundaries per level, but currently discards this
information — only the flat `chunks` list is returned. This makes it
impossible to distinguish leaf-level summaries from top-level ones.

This PR:
- Returns `(chunks, layers)` tuple from `raptor.py`'s `__call__`
- Annotates each RAPTOR summary chunk with `raptor_layer_int` (1 = first
summary level, 2 = summary-of-summaries, etc.)
- Adds `raptor_layer_int` to `infinity_mapping.json` (Elasticsearch
handles it via existing `*_int` dynamic template)

### Why this matters

Downstream features need to know which RAPTOR layer a summary belongs
to:
- **Retrieving the top-level document summary** for entity extraction,
search snippets, or document comparison
- **Filtering by abstraction level** — users may want only high-level
summaries or only leaf-level cluster summaries
- **RAPTOR recall quality** — #10951 reports summaries not being
recalled for definition queries; layer metadata enables targeted
retrieval

### Changes

| File | Change | LOC |
|------|--------|-----|
| `rag/raptor.py` | Return `(chunks, layers)` tuple | ~3 |
| `rag/svr/task_executor.py` | Build `chunk_layer` mapping, set
`raptor_layer_int` | ~12 |
| `conf/infinity_mapping.json` | Add `raptor_layer_int` integer field |
~1 |

### Backward compatibility

- **Additive only** — no existing fields or behavior changed
- Existing RAPTOR chunks continue to work (they'll have
`raptor_layer_int = 0` by default)
- New RAPTOR chunks get layer metadata automatically

## Test plan

- [ ] Parse a document with RAPTOR enabled, verify `raptor_layer_int` is
set on indexed chunks
- [ ] Verify `raptor_layer_int` values increase with abstraction level
(layer 1 < layer 2 < ...)
- [ ] Verify existing RAPTOR deletion (`delete by raptor_kwd`) still
works
- [ ] Verify Infinity backend accepts the new field

Fixes #7488
Related: #4104, #11191, #10951

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuch85 <yuch85.1@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-04-27 10:20:46 +08:00
buua436
a9e5724b46 Refa: unify document create flows under REST documents API (#14345)
### What problem does this PR solve?

unify document create flows under REST documents API

### Type of change

- [x] Refactoring
2026-04-27 10:18:16 +08:00
euvre
4dcc42e0e1 feat(api): add unified index API and dataset management endpoints (#14222)
### What problem does this PR solve?

## Summary

Refactor the dataset API layer into a clean service/REST separation
pattern, add a unified `/index` API for graph/raptor/mindmap operations,
and introduce several new dataset management endpoints with full test
coverage.

## Changes

### Service Layer (`dataset_api_service.py`)

- Added `trace_index(dataset_id, tenant_id, index_type)` — unified trace
function for all index types
- Added `run_index`, `delete_index` service functions
- Added `get_dataset`, `get_ingestion_summary`, `list_ingestion_logs`,
`get_ingestion_log`
- Added `run_embedding`, `list_tags`, `aggregate_tags`, `delete_tags`,
`rename_tag`
- Added `get_flattened_metadata`, `get_auto_metadata`,
`update_auto_metadata`

### REST API Layer (`dataset_api.py`)

**New unified routes:**

| Method | Route | Description |
|--------|-------|-------------|
| POST | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Run index
task |
| GET | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Trace index
task |
| DELETE | `/datasets/<id>/<index_type>` | Delete index |
| GET | `/datasets/<id>` | Get dataset details |
| GET | `/datasets/<id>/ingestions/summary` | Ingestion summary |
| GET | `/datasets/<id>/ingestions` | List ingestion logs |
| GET | `/datasets/<id>/ingestions/<log_id>` | Get single ingestion log
|
| POST | `/datasets/<id>/embedding` | Run embedding |
| GET | `/datasets/<id>/tags` | List tags |
| GET | `/datasets/tags/aggregation` | Aggregate tags across datasets |
| DELETE | `/datasets/<id>/tags` | Delete tags |
| PUT | `/datasets/<id>/tags` | Rename tag |
| GET | `/datasets/metadata/flattened` | Get flattened metadata |
| GET/PUT | `/datasets/<id>/metadata/config` | New metadata config path
|

**Removed routes (replaced by unified `/index`):**

- `POST /datasets/<id>/mindmap`
- `GET /datasets/<id>/mindmap`

**Preserved legacy routes (backward compatibility):**

- `/run_graphrag`, `/trace_graphrag`, `/run_raptor`, `/trace_raptor`
- `/auto_metadata` GET/PUT

### Test Suite

- Updated `common.py` helpers: added `trace_index`, removed
`run_mindmap`/`trace_mindmap`
- Added 7 new test files with 39 test cases total:

| Test File | Cases |
|-----------|-------|
| `test_get_dataset.py` | 4 |
| `test_ingestion_summary.py` | 2 |
| `test_ingestion_logs.py` | 5 |
| `test_index_api.py` | 14 |
| `test_embedding.py` | 2 |
| `test_tags.py` | 8 |
| `test_flattened_metadata.py` | 4 |

- Deleted `test_mindmap_tasks.py` (covered by unified index tests)

## Design Decisions

1. **Unified `/index?type=...`** — single endpoint replaces 3 separate
route pairs for graph/raptor/mindmap
2. **Backward compatibility** — old routes (`/run_graphrag`,
`/run_raptor`, `/auto_metadata`) preserved alongside new paths
3. **`_VALID_INDEX_TYPES = {"graph", "raptor", "mindmap"}`** — input
validation via constant set
4. **`_INDEX_TYPE_TO_TASK_ID_FIELD`** — maps index type to KB model task
ID field for clean dispatch

## Files Changed

- `api/apps/restful_apis/dataset_api.py`
- `api/apps/services/dataset_api_service.py`
- `sdk/python/ragflow_sdk/modules/dataset.py`
- `test/testcases/test_http_api/common.py`
- `test/testcases/test_http_api/test_dataset_management/` (7 new files)
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 09:38:01 +08:00
Xing Hong
fb95136f39 Fix: validate URL scheme and resolved IP before crawling to prevent SSRF (#14090)
### What problem does this PR solve?

The POST /upload_info?url=<url> endpoint accepted a user-supplied URL
and passed it directly to AsyncWebCrawler without any validation. There
were no restrictions on URL scheme, destination hostname, or resolved IP
address. This allowed any authenticated user to instruct the server to
make outbound HTTP requests to internal infrastructure — including RFC
1918 private networks, loopback addresses, and cloud metadata services
such as http://169.254.169.254 — effectively using the server as a proxy
for internal network reconnaissance or credential theft.

This PR adds an SSRF guard (_validate_url_for_crawl) that runs before
any crawl is initiated. It enforces an allowlist of safe schemes
(http/https), resolves the hostname at validation time, and rejects any
URL whose resolved IP falls within a private or reserved network range.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-25 14:30:15 +08:00
wdeveloper16
78188ce9e9 Feat: add OpenDataLoader PDF parser backend (#14058) (#14097)
### What problem does this PR solve?

Closes #14058.

RAGFlow supports multiple PDF parsing backends (DeepDOC, MinerU,
Docling, TCADP, PaddleOCR). This PR adds **OpenDataLoader**
([opendataloader-project/opendataloader-pdf](https://github.com/opendataloader-project/opendataloader-pdf))
as a new optional backend, giving users a deterministic, local-first
alternative with competitive table extraction accuracy.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---

### Changes

#### Backend
- `deepdoc/parser/opendataloader_parser.py` — new `OpenDataLoaderParser`
class inheriting `RAGFlowPdfParser`. Implements `check_installation()`
(guards Python package + Java 11+ runtime), `parse_pdf()` with
JSON-first extraction (heading/paragraph/table/list/image/formula) and
Markdown fallback, position-tag generation compatible with the shared
`@@page\tx0\tx1\ty0\ty1##` format, and temp-dir lifecycle with cleanup.
- `rag/app/naive.py` — new `by_opendataloader()` wrapper, registered in
`PARSERS` dict, added to `chunk_token_num=0` override list.
- `rag/flow/parser/parser.py` — `"opendataloader"` branch in the
pipeline PDF handler + check validation list.

#### Infrastructure
- `docker/entrypoint.sh` — `ensure_opendataloader()` function: opt-in
via `USE_OPENDATALOADER=true`, skips gracefully if Java is not on PATH.

#### Frontend
- `web/src/components/layout-recognize-form-field.tsx` —
`OpenDataLoader` added to `ParseDocumentType` enum and parser dropdown.
Cascades automatically to the pipeline editor's Parser component.

#### Docs
- `docs/guides/dataset/select_pdf_parser.md` — added OpenDataLoader
entry and full env-var reference.

---

### Environment variables

| Variable | Default | Description |
|---|---|---|
| `USE_OPENDATALOADER` | `false` | Set `true` to install
`opendataloader-pdf` on container startup |
| `OPENDATALOADER_VERSION` | latest | Pin the PyPI release (e.g.
`==2.2.1`) |
| `OPENDATALOADER_HYBRID` | _(unset)_ | Enable hybrid AI mode (e.g.
`docling-fast`) |
| `OPENDATALOADER_IMAGE_OUTPUT` | _(unset)_ | `off` / `embedded` /
`external` |
| `OPENDATALOADER_OUTPUT_DIR` | _(tmp)_ | Persistent output dir; temp
dir used + cleaned if unset |
| `OPENDATALOADER_DELETE_OUTPUT` | `1` | `0` to retain intermediate
files for debugging |
| `OPENDATALOADER_SANITIZE` | _(unset)_ | `1` to filter prompt-injection
patterns from output |

---

### Dependencies

- **Runtime**: `opendataloader-pdf` (PyPI, Apache 2.0) — opt-in, not
added to `pyproject.toml` core deps. Installed by
`ensure_opendataloader()` at container startup when
`USE_OPENDATALOADER=true`.
- **System**: Java 11+ on PATH (JVM is the underlying engine). The
installer skips with a warning if `java` is not found.

---

### How to test

**Standalone parser:**
```bash
source .venv/bin/activate
uv pip install opendataloader-pdf
python3 -c "
import sys; sys.path.insert(0, '.')
from deepdoc.parser.opendataloader_parser import OpenDataLoaderParser
p = OpenDataLoaderParser()
print('available:', p.check_installation())
s, t = p.parse_pdf('path/to/test.pdf', parse_method='pipeline')
print(f'sections={len(s)} tables={len(t)}')
"

```
### Benchmark vs Docling
```
file                      parser            secs  sections  tables
----------------------------------------------------------------------
text-heavy.pdf            docling           45.29       148      10
text-heavy.pdf            opendataloader     3.14       559       0
table-heavy.pdf           docling           7.05        76       3
table-heavy.pdf           opendataloader     3.71        90       0
complex.pdf               docling            42.67       114       8
complex.pdf               opendataloader     3.51       180       0
```
2026-04-25 00:33:02 +08:00
Lynn
e22cf333ed Fix: allow search id or _id (#14356)
### What problem does this PR solve?

Allow search id or _id when using es as doc_engine.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 21:38:19 +08:00
Magicbook1108
25089600d0 Feat: introduce minimum type check for pipeline (#14354)
### What problem does this PR solve?

Feat: introduce minimum type check for pipeline

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 21:12:50 +08:00
Jin Hai
1c244df90d Go: add gitee and siliconflow as model provider (#14336)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-24 20:59:30 +08:00
writinwaters
e5cfe7fb8f Doc: Updated a 0.25-specific faq (#14365)
### What problem does this PR solve?

Updated a 0.25 faq.

### Type of change


- [x] Documentation Update
2026-04-24 20:57:32 +08:00
Wang Qi
7fb6a12067 Update API document (#14364)
### What problem does this PR solve?

Update API document

### Type of change

- [ ] Documentation Update
2026-04-24 20:36:47 +08:00
balibabu
3ccd58f28c Fix: The button styles in the PaddleOCR dialog are not applying correctly. (#14350)
### What problem does this PR solve?

Fix: The button styles in the PaddleOCR dialog are not applying
correctly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Copilot <copilot@github.com>
2026-04-24 20:17:01 +08:00
writinwaters
1870c934c6 Refact: Updated rootAsHeadingTip (#14363)
### What problem does this PR solve?

Updated rootASHeadingTip.

### Type of change

- [x] Documentation Update
2026-04-24 20:08:44 +08:00
Idriss Sbaaoui
ca01c7a745 Fix blob sync: skip unsupported files before download (#14357)
### What problem does this PR solve?

Blob storage sync was downloading unsupported files first and rejecting
them later, which wasted bandwidth and made sync slower. This PR skips
unsupported extensions before download and applies `allow_images` in
blob sync. fixes #14338

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 19:22:32 +08:00
Cocoon-Break
620088be2f fix: check isinstance before len in VariableAssigner _remove_first/_remove_last (#14281)
fix: check isinstance before len in VariableAssigner _remove_first/_remove_last
2026-04-24 19:09:44 +08:00
Paras Sondhi
eeb89d604e feat: route docling parsing through native chunking endpoints (#14218)
Resolves #14211

**Background:** Currently, RAGFlow routes all Docling parsing through
the standard `/convert/source` endpoint. For large documents, this
returns massive, unchunked text that exceeds RAGFlow's internal
embedding model context limits, causing pipeline failures.

**Solution:**
This PR updates the `_parse_pdf_remote` ingestion logic in
`docling_parser.py` to prioritize `docling-serve`'s native chunking
endpoints (`/v1/chunk/source` and `/v1alpha/chunk/source`).
- By receiving pre-sliced chunk objects directly from Docling, RAGFlow
natively bypasses token limit overflows.
- Included a graceful fallback mechanism to the standard
`/convert/source` endpoints to maintain backwards compatibility for
users running older versions of the Docling server that return 404s on
the new routes.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 19:03:19 +08:00
Lynn
beb2406b86 Fix: allow use image2text as chat model (#14331)
### What problem does this PR solve?

Allow image2text models (multimodal) to be used as chat models.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 17:58:25 +08:00
buua436
9ad752f497 Refa:migrate agent webhook routes to REST APIs (#14330)
### What problem does this PR solve?

migrate agent webhook routes to REST APIs

### Type of change
- [x] Refactoring
2026-04-24 17:55:53 +08:00
Wang Qi
b8d831c1c3 Fix api user patch verb does not work (#14358)
### What problem does this PR solve?

Fix api user patch verb does not work

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 17:27:41 +08:00
Mukunda Rao Katta
8a2f63e77d docs: fix API key guide typo (#14352)
Fixes a small typo in the RAGFlow API key guide: `This documents
provides` -> `This document provides`.
2026-04-24 16:59:25 +08:00
qinling0210
1473000135 Implement retrieval_test in GO (#14231)
### What problem does this PR solve?

Implement retrieval_test in GO

### Type of change

- [x] Refactoring
2026-04-24 15:30:14 +08:00
Magicbook1108
aadd9a333f Feat: deepseek v4 (#14346)
### What problem does this PR solve?

Feat: deepseek v4
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 13:07:59 +08:00
Wang Qi
199fbceb72 Refactor user REST API (#14334)
### What problem does this PR solve?
Refactor user REST API

### Type of change
- [x] Refactoring
2026-04-24 10:25:15 +08:00
RazmikGevorgyan
c41b5e8a5d fix: migrate Langfuse integration from start_generation to start_obse… (#14205)
The Langfuse Python SDK v3+ removed `start_generation()` method.
RagFlow's code called this non-existent method, causing AttributeError
when Langfuse tracing is enabled.

Replace all `start_generation()` calls with
`start_observation(as_type="generation")` which is the correct v4 SDK
API.

Affected files:
- api/db/services/llm_service.py (12 occurrences)
- api/db/services/dialog_service.py (1 occurrence)

Fixes #14204
Related to #9243

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 10:03:57 +08:00
Magicbook1108
c74aece63c Feat: Agent api (#14157)
### What problem does this PR solve?

1. **List agents**  
   **Prev API**:  
   - `/v1/canvas/list GET`  
   - `/api/v1/agents GET`  
   **Current API**: `/api/v2/agents GET`

2. **Get canvas template**  
   **Prev API**: `/v1/canvas/templates GET`  
   **Current API**: `/api/v2/agents/templates GET`

3. **Delete an agent**  
   **Prev API**: 
    - `/v1/canvas/rm POST`  
    - `/api/v1/agents/<agent_id> DELETE`
   **Current API**: `/api/v2/agents/<agent_id> DELETE`

4. **Update an agent**  
   **Prev API**: 
    - `/api/v1/agents/<agent_id> PUT`   
    - `/v1/canvas/setting POST `
   **Current API**: `/api/v2/agents/<agent_id> PATCH`


5. **Create an agent**  
   **Prev API**: 
    - `/v1/canvas/set POST`  
    - `/api/v1/agents POST`
   **Current API**: `/api/v2/agents POST`


6. **Get an agent**  
   **Prev API**: 
    - `/v1/canvas/get/<canvas_id> GET `  
   **Current API**: `/api/v2/agents/<agent_id> GET`


7. **Reset an agent**  
   **Prev API**: 
    - `/v1/canvas/reset POST`  
   **Current API**: `/api/v2/agents/<agent_id>/reset POST`


8. **Upload a file to an agent**  
   **Prev API**: 
    - `/v1/canvas/upload/<canvas_id> POST`  
   **Current API**: `/api/v2/agents/<agent_id>/upload POST`


9. **Input form**  
   **Prev API**: 
    - `/v1/canvas/input_form GET`  
**Current API**:
`/api/v2/agents/<agent_id>/components/<component_id>/input-form GET`


10. **Debug an agent**  
   **Prev API**: 
    - `/v1/canvas/debug POST`  
**Current API**:
`/api/v2/agents/<agent_id>/components/<component_id>/debug POST`


11. **Trace an agent**  
   **Prev API**: 
    - `/v1/canvas/trace GET`  
   **Current API**: `/api/v2/agents/<agent_id>/logs/<message_id> GET`


12. **Get an agent version list**  
   **Prev API**: 
    - `/v1/canvas/getlistversion/<canvas_id>`  
   **Current API**: `/api/v2/agents/<agent_id>/versions GET`


13. **Get a version of agent**  
   **Prev API**: 
    - `/v1/canvas/getversion/<version_id>`  
**Current API**: `/api/v2/agents/<agent_id>/versions/<version_id> GET`


14. **Test db connection**  
   **Prev API**: 
    - `/v1/canvas/test_db_connect POST`  
   **Current API**: `/api/v2/agents/test_db_connection`


15. **Rerun the agent**  
   **Prev API**: 
    - `/v1/canvas/rerun POST`  
   **Current API**: `/api/v2/agents/rerun POST`


16. **Get prompts**  
   **Prev API**: 
    - `/v1/canvas/prompts GET`  
   **Current API**: `/api/v2/agents/prompts GET`

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chanx <1243304602@qq.com>
2026-04-24 10:02:22 +08:00
newyangyang
d84438fd53 fix azure blob put method param (#14329)
### What problem does this PR solve?

when use azure blob as the file container, when click parse file, it
calls:

```python
partial(settings.STORAGE_IMPL.put, tenant_id=task["tenant_id"])
```
So any storage backend used there must accept tenant_id as a kwarg. 
RAGFlowAzureSasBlob.put() did not, causing:
```
TypeError: ... got an unexpected keyword argument 'tenant_id'
```
Now it does, so parsing should proceed past this point.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-23 20:40:54 +08:00
buua436
d4fa57311c Refa: remove legacy MCP server web API (#14322)
### What problem does this PR solve?

remove legacy MCP server web API

### Type of change

- [x] Refactoring
2026-04-23 19:01:22 +08:00
Magicbook1108
75a5548b85 Feat: optimize title chunk (#14325)
### What problem does this PR solve?

Feat: optimize title chunk
1. Add a new button to enable "Use root chunk as H0 heading", so that
the first chunk is carried on to all remaining chunks.
2. Update resume agent template

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<img width="700" alt="img_v3_02111_63b04951-b3d7-4001-a08b-539db6d5298g"
src="https://github.com/user-attachments/assets/4179ac4d-90e7-4353-9b93-d649a455e634"
/>

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/c0ba0f3c-05aa-4f2c-b418-e808ca1a2641"
/>
2026-04-23 18:55:55 +08:00
Wang Qi
ba47c13eb5 Fix commit override from #14298 of api-key to api_key (#14328)
### What problem does this PR solve?

Fix commit override from
https://github.com/infiniflow/ragflow/pull/14298/ of `api-key` to `api_key`

### Type of change

- [x] Refactoring
2026-04-23 17:16:32 +08:00
Wang Qi
4458763a93 API refactor: stats_api and plugin_api (#14324)
### What problem does this PR solve?

API refactor: stats_api and plugin_api

### Type of change

- [x] Refactoring
2026-04-23 17:16:04 +08:00
buua436
7817b0d779 Refa: migrate chunk APIs to RESTful routes (#14291)
### What problem does this PR solve?

migrate chunk APIs to RESTful routes

### Type of change
- [x] Refactoring
2026-04-23 14:17:23 +08:00
Magicbook1108
76b017ca32 Refact: system apis (#14298)
### What problem does this PR solve?
Refact: system apis

### Type of change

- [x] Refactoring
2026-04-23 14:09:42 +08:00
Ricardo-M-L
57f527eb02 Add missing timeout to ragflow server health check (#14311)
### What problem does this PR solve?

`check_ragflow_server_alive()` in `api/utils/health_utils.py` calls
`requests.get(url)` without a `timeout` parameter. Unlike
`check_minio_alive()` which correctly specifies `timeout=10`, this
health check can hang indefinitely if the server is unresponsive.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

Added `timeout=10` to the `requests.get()` call, consistent with
`check_minio_alive()`.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 14:08:52 +08:00
dependabot[bot]
8901c18cb8 Build(deps): Bump lxml from 6.0.2 to 6.1.0 in /sdk/python (#14318)
Bumps [lxml](https://github.com/lxml/lxml) from 6.0.2 to 6.1.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/lxml/lxml/blob/master/CHANGES.txt">lxml's
changelog</a>.</em></p>
<blockquote>
<h1>6.1.0 (2026-04-17)</h1>
<p>This release fixes a possible external entity injection (XXE)
vulnerability in
<code>iterparse()</code> and the <code>ETCompatXMLParser</code>.</p>
<h2>Features added</h2>
<ul>
<li>
<p>GH#486: The HTML ARIA accessibility attributes were added to the set
of safe attributes
in <code>lxml.html.defs</code>. This allows <code>lxml_html_clean</code>
to pass them through.
Patch by oomsveta.</p>
</li>
<li>
<p>The default chunk size for reading from file-likes in
<code>iterparse()</code> is now configurable
with a new <code>chunk_size</code> argument.</p>
</li>
</ul>
<h2>Bugs fixed</h2>
<ul>
<li>LP#2146291: The <code>resolve_entities</code> option was still set
to <code>True</code> for
<code>iterparse</code> and <code>ETCompatXMLParser</code>, allowing for
external entity injection (XXE)
when using these parsers without setting this option explicitly.
The default was now changed to <code>'internal'</code> only (as for the
normal XML and HTML parsers
since lxml 5.0).
Issue found by Sihao Qiu as CVE-2026-41066.</li>
</ul>
<h1>6.0.4 (2026-04-12)</h1>
<h2>Bugs fixed</h2>
<ul>
<li>LP#2148019: Spurious MemoryError during namespace cleanup.</li>
</ul>
<h1>6.0.3 (2026-04-09)</h1>
<h2>Bugs fixed</h2>
<ul>
<li>
<p>Several out of memory error cases now raise <code>MemoryError</code>
that were not handled before.</p>
</li>
<li>
<p>Slicing with large step values (outside of <code>+/-
sys.maxsize</code>) could trigger undefined C behaviour.</p>
</li>
<li>
<p>LP#2125399: Some failing tests were fixed or disabled in PyPy.</p>
</li>
<li>
<p>LP#2138421: Memory leak in error cases when setting the
<code>public_id</code> or <code>system_url</code> of a document.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="43722f4402"><code>43722f4</code></a>
Update changelog.</li>
<li><a
href="87470409b1"><code>8747040</code></a>
Name version of option change in docstring.</li>
<li><a
href="6c36e6cef7"><code>6c36e6c</code></a>
Fix pypistats URL in download statistics script.</li>
<li><a
href="c7d76d6cb8"><code>c7d76d6</code></a>
Change security policy to point to Github security advisories.</li>
<li><a
href="378ccf82db"><code>378ccf8</code></a>
Update project income report.</li>
<li><a
href="315270b810"><code>315270b</code></a>
Docs: Reduce TOC depth of package pages and move module contents
first.</li>
<li><a
href="6dbba7f3c7"><code>6dbba7f</code></a>
Docs: Show current year in copyright line.</li>
<li><a
href="e4385bfa5d"><code>e4385bf</code></a>
Update project income report.</li>
<li><a
href="5bed1e1a22"><code>5bed1e1</code></a>
Validate file hashes in release download script.</li>
<li><a
href="c13ee10a42"><code>c13ee10</code></a>
Prepare release of 6.1.0.</li>
<li>Additional commits viewable in <a
href="https://github.com/lxml/lxml/compare/lxml-6.0.2...lxml-6.1.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lxml&package-manager=uv&previous-version=6.0.2&new-version=6.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-23 13:23:12 +08:00
Wang Qi
224574831c Add REDIS zcard (#14316)
### What problem does this PR solve?

As description.

### Type of change

- [x] Refactoring
2026-04-23 12:51:55 +08:00
buua436
aa4526266f Refa: migrate MCP APIs to RESTful api (#14317)
### What problem does this PR solve?

migrate MCP APIs to RESTful api

### Type of change

- [x] Refactoring
2026-04-23 12:51:27 +08:00
Jack
dbf8c6ed90 Refactor: Doc metadata update (#14289)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/metadata/update

After migration, Restful API
PATCH /api/v2/datasets/<dataset_id>/documents/metadatas 

### Type of change

- [x] Refactoring
2026-04-23 12:04:34 +08:00
Wang Qi
aae45b959b Refactor: API file2document (#14306)
Refactor: API file2document
2026-04-23 11:40:45 +08:00
Wang Qi
e79b896637 Refactor: REST API langfuse api-key (#14315)
REST API langfuse api-key
2026-04-23 11:36:16 +08:00
balibabu
f98597a19e Fix: Recall Test Page Metadata Not Displaying. (#14297)
### What problem does this PR solve?

Fix: Recall Test Page Metadata Not Displaying.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-23 10:57:05 +08:00
Jin Hai
2b029882d7 Go: add new provider minimax (#14296)
### What problem does this PR solve?

1. Add new provider minimax
2. Add new command: CHECK INSTANCE 'instance_name' FROM 'provider_name';
```
RAGFlow(user)> check instance 'test' from 'minimax';
SUCCESS
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-23 10:16:20 +08:00
chanx
387e2903d3 Fix: Some bugs (#14287)
### What problem does this PR solve?

Fix: Some bugs

- Pipeline runtime log files could not be viewed
- Corrected TOC terminology errors in the English translation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-23 10:15:26 +08:00
balibabu
ffa8738a78 Fix: Remove duplicate text output from the thought model on the chat page. (#14301)
### What problem does this PR solve?

Fix: Remove duplicate text output from the thought model on the chat
page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 23:22:51 +08:00
Wang Qi
01753b8f31 Refactor: API connectors (#14228)
### What problem does this PR solve?

Refactor /api/v1/connectors to be more RESTful.

### Type of change
- [x] Refactoring
2026-04-22 20:42:41 +08:00
Jack
c08cd8e090 Refactor: Migrate document metadata config update API (#14286)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/update_metadata_setting

After consolidation, Restful API
PUT
/api/v1/datasets/<dataset_id>/documents/<document_id>/metadata/config

### Type of change

- [x] Refactoring
2026-04-22 20:01:31 +08:00
Magicbook1108
d1c62fc19d Refact: Tenant api (#14288)
### What problem does this PR solve?

Refact: Tenant api

### Type of change

- [x] Refactoring
2026-04-22 20:00:32 +08:00
writinwaters
1434f8ade8 Doc: two PDF parser optimizers are supported as of v0.25.0. (#14261)
### What problem does this PR solve?

Multi-column layout detection is supported in v0.25.0

### Type of change


- [x] Documentation Update
2026-04-22 20:00:06 +08:00
Wang Qi
b52c518ec9 Set image tag v0.25.0 (#14299)
### What problem does this PR solve?

AD

### Type of change
2026-04-22 19:12:21 +08:00
NeedmeFordev
38e45a1117 Fix: serialize GraphRAG entity resolution merges to avoid graph mutation races (#14237)
### What problem does this PR solve?

This PR fixes the merge-phase crash reported in #14236 during GraphRAG
entity resolution.

The issue happens after candidate pair resolution completes, when
multiple merge coroutines mutate the same shared `networkx` graph
concurrently. In `_merge_graph_nodes`, the code iterates over
`graph.neighbors(node1)` and also awaits during edge/description
merging. That allows another coroutine to modify the graph adjacency
structure in between, which can trigger `RuntimeError: dictionary keys
changed during iteration` and can also lead to unsafe shared-graph
mutation.

This change keeps the PR scoped to that single issue by:
- serializing merge-time graph mutations with a dedicated merge lock
- snapshotting `graph.neighbors(node1)` with `list(...)` before
iteration

Together, these changes prevent concurrent mutation of the shared graph
during the merge phase and make the merge loop safe against live-view
invalidation.

Fixes #14236

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 16:42:53 +08:00
bohdansolovie
e0f0eb277d Fix upload stream handling to prevent truncated files (#14267)
## Summary
- Replace single `Read()` call in Go upload service with `io.ReadAll()`.
- Prevent potential truncated/corrupted file content during multipart
upload.
- Keep existing API behavior unchanged while fixing data integrity risk.

## Root Cause
`io.Reader.Read()` may return fewer bytes than requested without an
error. The previous implementation read once into a full buffer and
assumed all bytes were populated.

## Test plan
- Upload files of multiple sizes and verify uploaded content integrity.
- Confirm upload endpoint still returns successful responses.
- Verify downstream document parsing works on uploaded files.

## Issues
Closes #14266
2026-04-22 16:32:38 +08:00
Jin Hai
b8660b9919 Add deepseek and moonshot model json (#14290)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 15:59:41 +08:00
ucloudnb666
f853a39b40 feat: Add Astraflow provider support (global + China endpoints) (#14270)
## Add Astraflow Provider Support

This PR integrates [Astraflow](https://astraflow.ucloud.cn/) (by UCloud
/ 优刻得) as a new AI model provider in RAGFlow, with support for both
global and China endpoints.

### About Astraflow
Astraflow is an OpenAI-compatible AI model aggregation platform
supporting 200+ models from major providers including DeepSeek, Qwen,
GPT, Claude, Gemini, Llama, Mistral, and more.

| Variant | Factory Name | Endpoint | Env Var |
|---------|-------------|----------|---------|
| Global | `Astraflow` | `https://api-us-ca.umodelverse.ai/v1` |
`ASTRAFLOW_API_KEY` |
| China | `Astraflow-CN` | `https://api.modelverse.cn/v1` |
`ASTRAFLOW_CN_API_KEY` |

- **API key signup**: https://astraflow.ucloud.cn/

---

### Files Changed

| File | Change |
|------|--------|
| `rag/llm/__init__.py` | Register `Astraflow` and `Astraflow-CN` in
`SupportedLiteLLMProvider` enum, `FACTORY_DEFAULT_BASE_URL`, and
`LITELLM_PROVIDER_PREFIX` |
| `rag/llm/chat_model.py` | Add `AstraflowChat` and `AstraflowCNChat`
(OpenAI-compatible `Base` subclass) |
| `rag/llm/embedding_model.py` | Add `AstraflowEmbed` and
`AstraflowCNEmbed` (subclasses of `OpenAIEmbed`) |
| `rag/llm/rerank_model.py` | Add `AstraflowRerank` and
`AstraflowCNRerank` (subclasses of `OpenAI_APIRerank`) |
| `rag/llm/cv_model.py` | Add `AstraflowCV` and `AstraflowCNCV`
(subclasses of `GptV4`) |
| `rag/llm/tts_model.py` | Add `AstraflowTTS` and `AstraflowCNTTS`
(subclasses of `OpenAITTS`) |
| `rag/llm/sequence2txt_model.py` | Add `AstraflowSeq2txt` and
`AstraflowCNSeq2txt` (subclasses of `GPTSeq2txt`) |
| `conf/llm_factories.json` | Register `Astraflow` and `Astraflow-CN`
factories with a curated list of popular models |

---

### Supported Model Types
-  **Chat / LLM** — DeepSeek-V3/R1, Qwen3, GPT-4o/4.1, Claude 3.5/3.7,
Gemini 2.0/2.5 Flash, Llama 3.3/4, Mistral, and 200+ more
-  **Text Embedding** — text-embedding-3-small/large
-  **Image / Vision (IMAGE2TEXT)** — GPT-4o, GPT-4.1, Claude, Gemini,
Llama-4, etc.
-  **Text Re-Rank**
-  **TTS** — tts-1
-  **Speech-to-Text (SPEECH2TEXT)** — whisper-1

### Implementation Notes
- Uses the `openai/` LiteLLM prefix — consistent with other
OpenAI-compatible aggregation platforms (SILICONFLOW, DeerAPI, CometAPI,
OpenRouter, n1n, Avian, etc.)
- `Astraflow` (global, rank 250) and `Astraflow-CN` (China, rank 249)
are separate factory entries, allowing users to choose the optimal
endpoint based on their region.
- All model classes cleanly subclass existing base classes (`Base`,
`OpenAIEmbed`, `OpenAI_APIRerank`, `GptV4`, `OpenAITTS`, `GPTSeq2txt`)
with no custom logic needed — the provider is fully OpenAI-compatible.

---------

Co-authored-by: user <user@xzaaaMacBook-Air.local>
2026-04-22 15:38:34 +08:00
Idriss Sbaaoui
d5d162b374 Fix: MinerU 3.x output discovery and API contract (#14282)
### What problem does this PR solve?

update MinerU parser to most recent minerU v3 logic

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 14:44:41 +08:00
Lynn
3ce1e44b2d Fix: document and sdk support of searching message with user_id (#14283)
### What problem does this PR solve?

Add document of search message with user_id, add sdk support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-04-22 14:43:38 +08:00
Jin Hai
01c5437fdf Fix uv.lock (#14285)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 13:09:21 +08:00
Wang Qi
61d756e1b5 Fix #14213 create folder does not accept FOLDER (#14276)
### What problem does this PR solve?

As description.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:55:10 +08:00
writinwaters
69d8aed792 Doc: v0.25.0 release notes. (#14284)
### What problem does this PR solve?

Added v0.25.0 release notes

### Type of change


- [x] Documentation Update
2026-04-22 11:48:28 +08:00
Idriss Sbaaoui
77a843503d Fix: switch MinerU API endpoint to /pdf_parse (#14272)
### What problem does this PR solve?

update MinerU endpoint to /pdf_parse which has been exposed since v3.x.
fixes #14263

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:15:46 +08:00
buua436
ff29484d42 fix: normalize think tags in final chat answer (#14271)
### What problem does this PR solve?

normalize think tags in final chat answer

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:15:08 +08:00
Jack
3d8a82c0aa Refactor: Consolidation WEB API & HTTP API for document delete api (#14254)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/rm
Http API - DELETE /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- DELETE
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-22 10:49:52 +08:00
buua436
6baf74afc1 Refa: align chat and search restful APIs (#14229)
### What problem does this PR solve?

Refactor /api/v1/chats to be more RESTful.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 10:49:11 +08:00
Jin Hai
bfac0195df Update release note (#14275)
### What problem does this PR solve?

As title.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 10:47:43 +08:00
Jin Hai
74b44e1aa3 Go: add balance command (#14262)
### What problem does this PR solve?

```
RAGFlow(user)> list supported models from 'moonshot' 'test';
+---------------------------------+
| model_name                      |
+---------------------------------+
| moonshot-v1-32k-vision-preview  |
| kimi-k2.6                       |
| moonshot-v1-8k                  |
| moonshot-v1-auto                |
| moonshot-v1-128k                |
| moonshot-v1-32k                 |
| kimi-k2.5                       |
| moonshot-v1-8k-vision-preview   |
| moonshot-v1-128k-vision-preview |
+---------------------------------+
RAGFlow(user)> show balance from 'moonshot' 'test';
+---------+----------+
| balance | currency |
+---------+----------+
| 0       | CNY      |
+---------+----------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-21 21:31:50 +08:00
Jack
2d05475693 Refactor: Consolidation WEB API & HTTP API for document infos (#14239)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/infos
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents?ids=id1&ids=id2

### Type of change

- [ ] Refactoring
2026-04-21 19:35:11 +08:00
writinwaters
779deadf76 Docs: User-level memory is supported in v0.25.0 (#14259)
### What problem does this PR solve?

v0.25.0 supports linking User ID with conversations.

### Type of change


- [x] Documentation Update
2026-04-21 18:59:00 +08:00
hyl64
b439f8a74d docs: add DeepWiki developer guide page (#14244)
Closes #14165

Add a short documentation page under Developer Guides introducing
DeepWiki as a resource for developers doing secondary development or
exploring RAGFlow's codebase internals.

---------

Co-authored-by: hyl64 <hyl64@users.noreply.github.com>
2026-04-21 18:57:20 +08:00
Jack
009e538a4e Refactor: Consolidation WEB API & HTTP API for document get_filter (#14248)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/filter
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents?type=filter
### Type of change

- [x] Refactoring
2026-04-21 18:55:30 +08:00
Liu An
a33d0737cd Docs: Update version references to v0.25.0 in READMEs and docs (#14257)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.24.0 to v0.25.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-04-21 17:26:50 +08:00
Lynn
afdf0814d7 Fix: get metadata conf (#14250)
### What problem does this PR solve?

Get metadata configuration from union of custom metadata and
built_in_metadata.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 17:22:42 +08:00
writinwaters
0db2d544a9 Docs: 0.25.0 agent apps can be published. (#14252)
### What problem does this PR solve?

Agent apps can be published.

### Type of change

- [x] Documentation Update
2026-04-21 16:56:11 +08:00
balibabu
4841ce4239 Fix: Component definition is missing display name. (#14255)
### What problem does this PR solve?

Fix: Component definition is missing display name.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 16:53:08 +08:00
Jin Hai
e48d75987c Go: add stream / think chat (#14242)
### What problem does this PR solve?

1. Supports stream and non-stream chat
2. Supports think and non-think chat
3. List supported models from DeepSeek service. (This command can be
used to verify the API validity)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-21 16:52:32 +08:00
balibabu
a2bea30749 Fix: Editing an empty response in the retrieval operator will cause the focus to shift to the metadata input box. (#14253)
### What problem does this PR solve?

Fix: Editing an empty response in the retrieval operator will cause the
focus to shift to the metadata input box.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 16:19:55 +08:00
chanx
05c39b90a8 Fix: pipeline parser log not display (#14251)
### What problem does this PR solve?

Fix: pipeline parser log not display

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 15:24:13 +08:00
Liu An
6e33d8722f Revert "Fix: forwarding highlight param" (#14249)
Reverts infiniflow/ragflow#14112
2026-04-21 15:23:18 +08:00
balibabu
78b800e685 Fix: Fix: The minimum value for the "Suggested text block size" input box is set to 1. (#14246)
### What problem does this PR solve?

Fix: The minimum value for the "Suggested text block size" input box is
set to 1.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 14:06:36 +08:00
Magicbook1108
b3891ba6a4 Fix audio/video in pipeline (#14241)
### What problem does this PR solve?

Fix audio/video in pipeline

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 12:17:57 +08:00
Wang Qi
8aab158942 OpenSource Resume is supported only with Elasticsearch. (#14233)
### What problem does this PR solve?

OpenSource Resume is supported only with Elasticsearch.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 10:05:47 +08:00
Jin Hai
f269ee9739 Go: add thinking features to zhipu-ai (#14234)
### What problem does this PR solve?

```
RAGFlow(user)> list models from 'zhipu-ai';
+------------+------------+---------------+----------------+
| features   | max_tokens | model_types   | name           |
+------------+------------+---------------+----------------+
| [thinking] | 128000     | [chat]        | glm-4.7        |
| [thinking] | 128000     | [chat]        | glm-4.5        |
| [thinking] | 128000     | [chat vision] | glm-4.6v-Flash |
| [thinking] | 128000     | [chat]        | glm-4.5-x      |
| [thinking] | 128000     | [chat]        | glm-4.5-air    |
| [thinking] | 128000     | [chat]        | glm-4.5-airx   |
| [thinking] | 128000     | [chat]        | glm-4.5-flash  |
| [thinking] | 64000      | [vision]      | glm-4.5v       |
|            | 128000     | [chat]        | glm-4-plus     |
|            | 128000     | [chat]        | glm-4-0520     |
|            | 128000     | [chat]        | glm-4          |
|            | 8000       | [chat]        | glm-4-airx     |
|            | 128000     | [chat]        | glm-4-air      |
|            | 128000     | [chat]        | glm-4-flash    |
|            | 128000     | [chat]        | glm-4-flashx   |
|            | 1000000    | [chat]        | glm-4-long     |
|            | 128000     | [chat]        | glm-3-turbo    |
|            | 2000       | [vision]      | glm-4v         |
|            | 8192       | [chat]        | glm-4-9b       |
|            | 512        | [embedding]   | embedding-2    |
|            | 512        | [embedding]   | embedding-3    |
|            | 4096       | [asr]         | glm-asr        |
|            | 0          | [tts]         | glm-tts        |
|            | 0          | [ocr]         | glm-ocr        |
|            | 0          | [rerank]      | glm-rerank     |
+------------+------------+---------------+----------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-20 21:53:27 +08:00
balibabu
c43367eca3 Fix: The number of chunks in the file list is not displayed. (#14232)
### What problem does this PR solve?

Fix: The number of chunks in the file list is not displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 19:24:20 +08:00
balibabu
5265def967 Fix: The mind map on the search page does not display completely upon initial loading. (#14226)
### What problem does this PR solve?

Fix: The mind map on the search page does not display completely upon
initial loading.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 19:24:13 +08:00
Julian
ba7d3f6c31 Add debugpy dependency to pyproject.toml (#14225)
In order to attach the debugger to a running docker container it has to
be inside the docker image

### What problem does this PR solve?

[#14224](https://github.com/infiniflow/ragflow/issues/14224)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 18:05:17 +08:00
NeedmeFordev
78c3583964 Fix memory resolution regression for multimodal Gemini models (#14209)
### What problem does this PR solve?

Fixes #14206.

This issue is a regression. PR #9520 previously changed Gemini models
from `image2text` to `chat` to fix chat-side resolution, but PR #13073
later restored those Gemini entries to `image2text` during model-list
updates, which reintroduced the bug.

The underlying problem is that Gemini models are multimodal and
advertise both `CHAT` and `IMAGE2TEXT`, while tenant model resolution
still depends on a single stored `model_type`. That makes chat-only
flows such as memory extraction fragile when a compatible model is
stored as `image2text`.

This PR fixes the issue at the model resolution layer instead of
changing `llm_factories.json` again:
- keep the stored tenant model type unchanged
- try exact `model_type` lookup first
- if no exact match is found, fall back only when the model metadata
shows the requested capability is supported
- coerce the runtime config to the requested type for chat callers
- fail fast in memory creation instead of silently persisting
`tenant_llm_id=0`

This preserves existing multimodal and `image2text` behavior while
restoring chat compatibility for memory-related flows.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Testing

- Re-checked the current memory creation and memory message extraction
paths against the updated resolution logic
- Verified locally that a Gemini-style tenant model stored as
`image2text` but tagged with `CHAT` can still be resolved for `chat`
- Verified `get_model_config_by_type_and_name(..., CHAT, ...)` returns a
chat-compatible runtime config
- Verified `get_model_config_by_id(..., CHAT)` also returns a
chat-compatible runtime config
- Verified strict resolution still fails when the model metadata does
not advertise chat capability
2026-04-20 16:37:36 +08:00
Magicbook1108
9c7c105007 Fix: Doc generator (#14223)
### What problem does this PR solve?

Doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 16:37:33 +08:00
Jin Hai
af2ed416a7 Add extra field to model instance (#14203)
### What problem does this PR solve?

Now each model support region with different URL

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-20 15:31:12 +08:00
Jack
939933649a Refactor: Consolidation WEB API & HTTP API for document list_docs (#14176)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/list
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-20 14:54:40 +08:00
Magicbook1108
d053317c4d Fix: variable in doc generator (#14180)
### What problem does this PR solve?

Fix: variable in doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 14:19:42 +08:00
Magicbook1108
19eedeec61 Fix: accept empty value as 0 chunk (#14220)
### What problem does this PR solve?

Fix: accept empty value as 0 chunk
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 12:53:47 +08:00
LeonTung
f554f6ae85 chore(docs): tips for installing CN fonts (#14189)
### What problem does this PR solve?
Add tips for installing Chinse fonts under code sandbox. Otherwise,
`matplotlib `won't render Chinese correctly.

<img width="2082" height="1186" alt="sales_analysis"
src="https://github.com/user-attachments/assets/57e675ab-1e92-4662-9aeb-ad72a6121eb5"
/>



### Type of change

- [x] Documentation Update
2026-04-20 12:11:23 +08:00
Lynn
0f806dc3ca Feat: mysql sync (#14200)
### What problem does this PR solve?

Add a script to sync db schema with peewee_migrate.

### Type of change

- [x] Other (please describe): tool script
2026-04-20 11:40:01 +08:00
rhinoceros.xn
4e992de91f Add tongyi gte-rerank-v2 (#14215)
https://bailian.console.aliyun.com/cn-beijing?tab=api#/api/?type=model&url=2780056

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change
- [x] Other (please describe): add gte-rerank-v2、qwen3-rerank
2026-04-20 11:39:17 +08:00
Liu An
d5c306de30 Fix: remove unit test checkpoint resume (#14216)
### What problem does this PR solve?

remove unit test checkpoint resume

### Type of change

- [x] Performance Improvement
2026-04-20 11:27:40 +08:00
euvre
84b6069ec7 fix: escape single quotes in Infinity SQL filter conditions (#14186)
### What problem does this PR solve?

## Summary

Fixes #5939

Entity names containing single quotes (e.g., `投影直线L'`) caused SQL syntax
errors when building filter conditions for Infinity queries, due to
unescaped string interpolation in `equivalent_condition_to_str`.

## Changes

In `common/doc_store/infinity_conn_base.py`, added `.replace("'", "''")`
escaping for string values in two branches of
`equivalent_condition_to_str` where it was missing:

1. **`field_keyword` branch with non-list value** (line 190): The list
branch already escaped single quotes on line 183, but the single-string
branch did not.
2. **Plain string value branch** (line 209): Direct f-string
interpolation `{k}='{v}'` was vulnerable to unescaped quotes.

Both fixes use the same SQL-standard escape pattern (`'` → `''`) already
applied elsewhere in this method.

## How to Test

1. Upload a document containing entity names with single quotes.
2. Enable Knowledge Graph (GraphRAG) in the parsing configuration.
3. Initiate document parsing — it should complete without SQL syntax
errors.

## Note

The original issue also reported a typo (`dge_graph_kwd` instead of
`knowledge_graph_kwd`), which has already been fixed in the current
codebase.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-20 10:04:07 +08:00
balibabu
6712b504e6 Fix: Clicking on the empty dialog box on the agent exploration page will result in an error. (#14198)
### What problem does this PR solve?

Fix: Clicking on the empty dialog box on the agent exploration page will
result in an error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 23:52:13 +08:00
Lynn
c3387cd5b8 Fix: parent child config (#14199)
### What problem does this PR solve?

Correctly set and display parent-child config in parser_config, and
allow to pass `tenant_id` in PATCH `/api/v1/chats`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 23:02:42 +08:00
balibabu
09622c6353 Fix: Spaces cannot be entered in the code editor of the code operator. (#14183)
### What problem does this PR solve?

Fix: Spaces cannot be entered in the code editor of the code operator.

[Monaco Editor with XYFlow fails to accept most space bar keypresses,
who is at fault?
#5204](https://github.com/microsoft/monaco-editor/discussions/5204)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:41:39 +08:00
balibabu
fa644c5a15 Fix: The embedded page for search is inaccessible. (#14194)
### What problem does this PR solve?

Fix: The embedded page for search is inaccessible.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:37:34 +08:00
chanx
60506ef7a5 fix: Add internationalization configurations related to text segmentation identifiers. (#14201)
### What problem does this PR solve?

fix: Add internationalization configurations related to text
segmentation identifiers.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:37:14 +08:00
balibabu
3a4d17cb0d Fix: The placeholder in PromptEditor is obscured. (#14179)
### What problem does this PR solve?

Fix: The placeholder in PromptEditor is obscured.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:02:41 +08:00
Daniil Sivak
22c6648348 Fix: forwarding highlight param (#14112)
Closes #9078

### What problem does this PR solve?

The `retrieval_test` endpoint in `chunk_app.py` never forwarded the
`highlight` request parameter to `retriever.retrieval()`, so the search
engine never produced highlight snippets. Additionally, the frontend
always rendered `content_with_weight` instead of preferring the
`highlight` field, and the CSS rule color `var(--accent-primary)` didn't
work because the variable stores an RGB triplet `(45,212,191)` requiring
the `rgb()` wrapper.

### Before

- Search page: displayed raw content_with_weight as a wall of plain
white text with no term highlighting, including markdown headings
rendered as literal text
- Retrieval testing page: showed `content_with_weight` in a plain `<p>`
tag, no `<em>` tags rendered, no highlight coloring
- Children chunks: when child chunks were consolidated into a parent via
`retrieval_by_children`, any highlight data from children was discarded
- TOC chunks: chunks fetched via `retrieval_by_toc` had no `highlight`
field, appearing as plain text while other chunks had highlights

**Retrieval testing**:
<img width="1449" height="1178"
alt="before-retrieval-no-highlight-cropped"
src="https://github.com/user-attachments/assets/5c6f5a5e-6c11-461a-bdb4-049d7dfb7a33"
/>

**Search**:
<img width="1378" height="711" alt="before-search-no-highlight-cropped"
src="https://github.com/user-attachments/assets/be7b5152-72ef-40da-a8fd-921e997ae7d3"
/>

### After

- Search page: displays the highlight field with search terms rendered
in teal/cyan color (`rgb(var(--accent-primary))`)
- Retrieval testing page: sends highlight: true in the request, uses
`HighLightMarkdown` component to render `<em>` tags with proper coloring
- Children chunks: highlights from child chunks are joined and preserved
on the parent
- TOC chunks: when other chunks have highlights, TOC-fetched chunks use
`content_with_weight` as a highlight fallback

**Retrieval testing**:
<img width="1410" height="1015" alt="05-retrieval-testing-results"
src="https://github.com/user-attachments/assets/f0cff8cf-0962-4320-b559-cd5037f622d2"
/>

**Search**:
<img width="1294" height="455" alt="03-search-highlight-results"
src="https://github.com/user-attachments/assets/a90e0e3e-3837-46be-8ddd-2412ff7cbc19"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 20:59:20 +08:00
Yongteng Lei
fac46ef67f Refa: change Minimax base url to mainland by default to align with UI (#14195)
### What problem does this PR solve?

Change Minimax base url to mainland by default to align with UI.

### Type of change

- [x] Refactoring
2026-04-17 19:08:57 +08:00
dependabot[bot]
b34a726acd Build(deps): Bump pypdf from 6.9.2 to 6.10.2 (#14184)
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.9.2 to 6.10.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/releases">pypdf's
releases</a>.</em></p>
<blockquote>
<h2>Version 6.10.2, 2026-04-15</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.1...6.10.2">Full
Changelog</a></p>
<h2>Version 6.10.1, 2026-04-14</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use new parameter names for compress_identical_objects by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.0...6.10.1">Full
Changelog</a></p>
<h2>Version 6.10.0, 2026-04-10</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Disallow custom XML entity declarations for XMP metadata (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3724">#3724</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>New Features (ENH)</h3>
<ul>
<li>Skip MD5 key derivation for AES-256 encrypted PDFs (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3694">#3694</a>)
by <a href="https://github.com/Ygnas"><code>@​Ygnas</code></a></li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use remove_orphans in compress_identical_objects (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3310">#3310</a>)
by <a href="https://github.com/j-t-1"><code>@​j-t-1</code></a></li>
<li>Fix PdfReadError when xref table contains comments before trailer
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3710">#3710</a>)
by <a href="https://github.com/rassie"><code>@​rassie</code></a></li>
<li>Correctly verify AES padding during decryption (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3699">#3699</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Fix stale object cache from non-authoritative object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3698">#3698</a>)
by <a
href="https://github.com/astahlman"><code>@​astahlman</code></a></li>
<li>Fix extract_links pairing when annotations include non-links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3687">#3687</a>)
by <a
href="https://github.com/ReinerBRO"><code>@​ReinerBRO</code></a></li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Add AI policy (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3717">#3717</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.0">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md">pypdf's
changelog</a>.</em></p>
<blockquote>
<h2>Version 6.10.2, 2026-04-15</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)</li>
<li>Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.1...6.10.2">Full
Changelog</a></p>
<h2>Version 6.10.1, 2026-04-14</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)</li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)</li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use new parameter names for compress_identical_objects</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.0...6.10.1">Full
Changelog</a></p>
<h2>Version 6.10.0, 2026-04-10</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Disallow custom XML entity declarations for XMP metadata (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3724">#3724</a>)</li>
</ul>
<h3>New Features (ENH)</h3>
<ul>
<li>Skip MD5 key derivation for AES-256 encrypted PDFs (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3694">#3694</a>)</li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use remove_orphans in compress_identical_objects (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3310">#3310</a>)</li>
<li>Fix PdfReadError when xref table contains comments before trailer
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3710">#3710</a>)</li>
<li>Correctly verify AES padding during decryption (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3699">#3699</a>)</li>
<li>Fix stale object cache from non-authoritative object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3698">#3698</a>)</li>
<li>Fix extract_links pairing when annotations include non-links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3687">#3687</a>)</li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Add AI policy (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3717">#3717</a>)</li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.0">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c476b4f293"><code>c476b4f</code></a>
REL: 6.10.2</li>
<li><a
href="c50a0104cf"><code>c50a010</code></a>
SEC: Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)</li>
<li><a
href="ac734dab4e"><code>ac734da</code></a>
SEC: Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)</li>
<li><a
href="b49e7eb454"><code>b49e7eb</code></a>
REL: 6.10.1</li>
<li><a
href="62338e9d36"><code>62338e9</code></a>
SEC: Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)</li>
<li><a
href="5dcc0aebaa"><code>5dcc0ae</code></a>
DEV: Update pytest-benchmark to 5.2.3</li>
<li><a
href="b42e4aa98a"><code>b42e4aa</code></a>
DEV: Update pinned pillow and pytest where possible (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3732">#3732</a>)</li>
<li><a
href="717446b121"><code>717446b</code></a>
ROB: Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)</li>
<li><a
href="9e461d361b"><code>9e461d3</code></a>
DEV: Bump softprops/action-gh-release from 2 to 3 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3730">#3730</a>)</li>
<li><a
href="500d09d92f"><code>500d09d</code></a>
TST: Update <code>test_embedded_file__basic</code> to use
<code>tmp_path</code> fixture (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3726">#3726</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypdf&package-manager=uv&previous-version=6.9.2&new-version=6.10.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-17 18:43:19 +08:00
Jin Hai
94106646e7 Go: set and list default models (#14191)
### What problem does this PR solve?

```
RAGFlow(user)> set default vlm "zhipu-ai" "ccc" "glm-4.6v-flash";
SUCCESS
RAGFlow(user)> list default models;
+--------+----------------+----------------+----------------+------------+
| enable | model_instance | model_name     | model_provider | model_type |
+--------+----------------+----------------+----------------+------------+
| true   | ccc            | glm-4.6v-flash | zhipu-ai       | llm        |
| true   | ccc            | glm-4.6v-flash | zhipu-ai       | image2text |
+--------+----------------+----------------+----------------+------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-17 18:05:33 +08:00
Wang Qi
28d8b1c883 [Fix] trivial fix log creation (#14181)
### What problem does this PR solve?

Trivial fix log creation, follow on PR:
https://github.com/infiniflow/ragflow/pull/14136

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 13:13:41 +08:00
Magicbook1108
797aa6076a Fix: keyword extraction (#14177)
### What problem does this PR solve?

Fix: keyword extraction

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 11:32:48 +08:00
LeonTung
c3bf8d9d60 feat(templates): add a data analysis agent template (#14130)
### What problem does this PR solve?

Add a new agent template that demonstrates how to leverage the
`CodeExec` component to do the data analysis.

### Type of change

- [x] Other (please describe): Agent template
2026-04-17 11:32:04 +08:00
writinwaters
0df5d830d4 Refact: Updated agent template descriptions. (#14175)
### What problem does this PR solve?

Updated ingestion pipeline template descriptions for better technical
accuracy and readability.

### Type of change

- [x] Refactoring
2026-04-17 10:46:06 +08:00
Lynn
f194a09cd6 Fix: dataset update parent child (#14167)
### What problem does this PR solve?

Correctly set parent child config in parser_config.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 10:41:50 +08:00
Jin Hai
e03212fd7a Fix go cli models command and api (#14166)
### What problem does this PR solve?

```
RAGFlow(user)> list providers;
+--------------------------------------+----------+-------------------------------------------+--------------+
| base_url                             | name     | tags                                      | total_models |
+--------------------------------------+----------+-------------------------------------------+--------------+
| https://open.bigmodel.cn/api/paas/v4 | ZHIPU-AI | LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION | 21           |
| https://api.x.ai/v1                  | xAI      | LLM                                       | 6            |
+--------------------------------------+----------+-------------------------------------------+--------------+
RAGFlow(user)> show provider 'zhipu-ai';
+--------------------------------------+----------+-------------------------------------------+--------------+
| base_url                             | name     | tags                                      | total_models |
+--------------------------------------+----------+-------------------------------------------+--------------+
| https://open.bigmodel.cn/api/paas/v4 | ZHIPU-AI | LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION | 21           |
+--------------------------------------+----------+-------------------------------------------+--------------+
RAGFlow(user)> delete provider 'zhipu-ai';
SUCCESS
RAGFlow(user)> add provider 'zhipu-ai';
SUCCESS
RAGFlow(user)> create provider 'zhipu-ai' instance 'ccc' 'ccxxccxx';
SUCCESS
RAGFlow(user)> list instances from 'zhipu-ai';
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| apiKey                                            | id                               | instanceName | providerID                       | status |
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| ccxxccxx | 640dd7ee398711f1bdd838a74640adcc | ccc          | d1d59de5398411f1bdd838a74640adcc | active |
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
RAGFlow(user)> list models from 'zhipu-ai';
+----------+------------+---------------+---------------+
| features | max_tokens | model_types   | name          |
+----------+------------+---------------+---------------+
| map[]    | 128000     | [chat]        | glm-4.7       |
| map[]    | 128000     | [chat]        | glm-4.5       |
| map[]    | 128000     | [chat]        | glm-4.5-x     |
| map[]    | 128000     | [chat]        | glm-4.5-air   |
| map[]    | 128000     | [chat]        | glm-4.5-airx  |
| map[]    | 128000     | [chat]        | glm-4.5-flash |
| map[]    | 64000      | [image2text]  | glm-4.5v      |
| map[]    | 128000     | [chat]        | glm-4-plus    |
| map[]    | 128000     | [chat]        | glm-4-0520    |
| map[]    | 128000     | [chat]        | glm-4         |
| map[]    | 8000       | [chat]        | glm-4-airx    |
| map[]    | 128000     | [chat]        | glm-4-air     |
| map[]    | 128000     | [chat]        | glm-4-flash   |
| map[]    | 128000     | [chat]        | glm-4-flashx  |
| map[]    | 1000000    | [chat]        | glm-4-long    |
| map[]    | 128000     | [chat]        | glm-3-turbo   |
| map[]    | 2000       | [image2text]  | glm-4v        |
| map[]    | 8192       | [chat]        | glm-4-9b      |
| map[]    | 512        | [embedding]   | embedding-2   |
| map[]    | 512        | [embedding]   | embedding-3   |
| map[]    | 4096       | [speech2text] | glm-asr       |
+----------+------------+---------------+---------------+
RAGFlow(user)> disable model 'glm-4.5-flash' from 'zhipu-ai' 'ccc';
SUCCESS
RAGFlow(user)> drop instance 'ccc' from 'zhipu-ai';
SUCCESS
RAGFlow(user)> list instances from 'zhipu-ai';
No data to print
```

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-17 09:55:25 +08:00
Wang Qi
96a23d2fd0 [Bug fix] fix bug found in regression when view chunks for document that not parsed in infinity, it would fail in UI (#14168)
### What problem does this PR solve?
See title, the fail image:
<img width="2667" height="915" alt="20260416-205718"
src="https://github.com/user-attachments/assets/0c564237-5ed0-49af-bf4c-d3b5519abc6e"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 09:51:23 +08:00
Magicbook1108
f906a203bb Fix doc generator (#14160)
### What problem does this PR solve?

Fix doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:38 +08:00
balibabu
4a9bfd18bc Fix: The PromptEditor's placeholder is only half displayed. (#14161)
### What problem does this PR solve?

Fix: The PromptEditor's placeholder is only half displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:16 +08:00
Magicbook1108
ea8de1bb47 Fix: different llm in chat (#14162)
### What problem does this PR solve?

Fix: different llm in chat

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:01 +08:00
writinwaters
8a874c7a09 Doc: Added Ingetrating Notion connector (#14163)
### What problem does this PR solve?

Added How to integrate Notion to RAGFlow.

### Type of change

- [x] Documentation Update
2026-04-16 20:06:02 +08:00
Lynn
655dd2f8c6 Fix: simplify _load_user (#14154)
### What problem does this PR solve?

Simplify _load_user, remove unused fallback.

### Type of change

- [x] Refactoring
2026-04-16 18:47:43 +08:00
balibabu
4cf4d444d2 Fix: Login page type error. (#14156)
### What problem does this PR solve?

Fix: Login page type error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 18:46:52 +08:00
Magicbook1108
901023a80a Fix: literal eval http request input (#14145)
### What problem does this PR solve?

Fix: literal eval http request input

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

<img width="700" alt="img_v3_0210q_f4b49ff7-e670-4054-ab0e-9443a09215fg"
src="https://github.com/user-attachments/assets/089300be-06f9-4bb6-97af-61bf5f4a5e8c"
/>


<img width="700" alt="img_v3_0210q_398cd52a-2ad9-42be-8d5b-4e6e68a7d22g"
src="https://github.com/user-attachments/assets/239b43cd-a2a5-49d8-9200-991bb26336c8"
/>
2026-04-16 16:52:34 +08:00
euvre
9a785b26bd fix: change file size column from IntegerField to BigIntegerField to support files > 2GB (#14148)
### What problem does this PR solve?

Fixes #6034

Changes the `size` field in both `Document` and `File` models from
`IntegerField` (32-bit, max ~2GB) to `BigIntegerField` (64-bit, max
~9.2EB), and adds corresponding database migrations.

## Problem

When uploading a file larger than 2GB, the `size` value overflows a
32-bit signed integer (max 2,147,483,647). This causes:

- The stored `size` wraps around to an incorrect value (e.g., a 3GB file
shows as 2,097,152 KB in File Management).
- Subsequent file operations (e.g., download) fail because the corrupted
size leads to invalid storage lookups.

## Changes

- `Document.size`: `IntegerField` → `BigIntegerField`
- `File.size`: `IntegerField` → `BigIntegerField`
- Added `alter_db_column_type` migrations in `migrate_db()` for both
`document.size` and `file.size` columns to ensure existing deployments
are upgraded automatically.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-16 15:43:29 +08:00
euvre
0cd49e14dd fix: make Infinity connection pool size configurable and add retry logic for GraphRAG write bursts (#14143)
### What problem does this PR solve?

Resolve #14137 .

### Problem

Graph resolution succeeds (nodes/edges merged, pagerank updated), but
the subsequent burst of Infinity write operations in `set_graph`
exhausts the connection pool with `TOO_MANY_CONNECTIONS` errors. Root
causes:

1. **Hardcoded pool size** — `infinity_conn_pool.py` hardcoded
`ConnectionPool(max_size=4)` on initial creation and `max_size=32` on
refresh. Operators cannot tune this without patching code.
2. **No retry on transient failures** — a single `TOO_MANY_CONNECTIONS`
on edge deletes or chunk inserts kills the entire resolution+community
pipeline with no retry.

### Changes

#### `common/doc_store/infinity_conn_pool.py`

- Read `ConnectionPool` `max_size` from the `INFINITY_POOL_MAX_SIZE`
environment variable (default: `4`), applied consistently to both
initial creation and refresh paths.
- Log the actual pool size on startup for easier debugging.

#### `rag/graphrag/utils.py` — `set_graph()`

- **Edge deletes**: add exponential-backoff retry (3 attempts, 1s/2s/4s
delays) so transient `TOO_MANY_CONNECTIONS` errors are retried instead
of failing the entire job. Concurrency continues to be gated by the
existing `chat_limiter`.
- **Batch inserts**: add exponential-backoff retry (3 attempts, 1s/2s/4s
delays) for the same reason.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-16 15:40:54 +08:00
Qi Wang
969ce3a79f [Bug fix #14133] fix graph rag, raptor, mindmap log cannot show correctly in UI (#14136)
### What problem does this PR solve?
Fix #14133, knowledge graph, raptor, mindmap log cannot show correctly
in UI
<img width="1930" height="982" alt="Image"
src="https://github.com/user-attachments/assets/d2f8e6c1-d82d-4b00-a377-949aada545ca"
/>
After Fix:
<img width="2108" height="805" alt="image"
src="https://github.com/user-attachments/assets/b37426c1-83d3-4a32-a83c-9d340d69e0e6"
/>
<img width="2173" height="1067" alt="image"
src="https://github.com/user-attachments/assets/30105222-3310-43a0-9f83-1e320d05e413"
/>

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 13:08:36 +08:00
Yongteng Lei
356ba5650a Fix: sandbox don't attach attachment metadata (#14135)
### What problem does this PR solve?

Sandbox don't attach attachment metadata

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 12:08:54 +08:00
balibabu
53154b2cc3 Feat: Add a title prefix to the testid on the login page. (#14129)
### What problem does this PR solve?

Feat: Add a title prefix to the testid on the login page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-16 12:08:44 +08:00
Magicbook1108
944a90d645 Feat: add button to turn off vlm parsing (#14125)
### What problem does this PR solve?

Feat: add button to turn off vlm parsing

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chanx <1243304602@qq.com>
2026-04-15 19:06:00 +08:00
chanx
dce0b1c030 Fix: Pipeline page style optimizations (#14128)
### What problem does this PR solve?

Fix: Pipeline page style optimizations

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-15 19:05:54 +08:00
Daniil Sivak
c93ec0a1f3 Fix: reject empty/space-only content in update_chunk API (#14082)
Closes #6541

### What problem does this PR solve?

Add content validation to `update_chunk` (SDK and non-SDK) to reject
empty or whitespace-only content before it reaches the embedding model.

**Before:** Calling `update_chunk` with space-only content (like `" "`,
`""`, `"\n"`) bypassed validation and was sent directly to the embedding
model, which returned an error. This was the same bug previously fixed
for `add_chunk` in #6390, but `update_chunk` was missed.

**After:** Empty/whitespace-only content is caught by validation and
returns an error: `` `content` is required ``

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-15 18:43:53 +08:00
Magicbook1108
d51789e2be Feat: update templates && add resume template (#14124)
### What problem does this PR solve?

Feat: update templates  && add resume template

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-15 18:42:29 +08:00
balibabu
c56a7f99d1 Fix: The pop-up menu of the PromptEditor will be blocked. #14126 (#14127)
### What problem does this PR solve?

Fix: The pop-up menu of the PromptEditor will be blocked.  #14126

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-15 18:42:02 +08:00
writinwaters
2520065c5a Doc: Added Integrate Confluence (#14131)
### What problem does this PR solve?

Added a guide on integrating Confluence as connector.

### Type of change

- [x] Documentation Update
2026-04-15 18:38:36 +08:00
Minal Mahala
f930389311 Refact: improve task resume mechanism for graphrag (#14096)
### What problem does this PR solve?

Addresses review feedback on #14074 (Checkpoint mechanism for
long-running workflow jobs, issue #12494).

**Changes based on @yuzhichang's review:**

1. **Renamed `checkpoint_service.py` → `task_checkpoint.py`** as
suggested.
2. **Replaced Redis with direct docEngine queries** as suggested — the
subgraph already gets persisted to the doc store by
`generate_subgraph()`, so we just query for it instead of maintaining a
separate checkpoint in Redis. This is simpler, has no extra dependency,
and uses a single source of truth.

**Changes based on CodeRabbit review:**

3. **Fixed `source_id` query format mismatch** — subgraphs are stored
with `source_id: [doc_id]` (list), but the original query used
`source_id: doc_id` (string). Now follows the same pattern as
`does_graph_contains()` in `rag/graphrag/utils.py`: filter by
`knowledge_graph_kwd` only, then match `source_id` in Python. This
avoids ambiguity across Elasticsearch / Infinity / OceanBase backends.

### Changes

| File | Change |
|---|---|
| `api/db/services/task_checkpoint.py` (new) |
`load_subgraph_from_store()` and `has_raptor_chunks()` — docEngine-based
checkpoint queries |
| `rag/graphrag/general/index.py` | `build_one()` calls
`load_subgraph_from_store()` before running LLM extraction |
| `rag/svr/task_executor.py` | RAPTOR per-doc loop calls
`has_raptor_chunks()` before processing |
| `test/unit_test/rag/graphrag/test_checkpoint_resume.py` (new) | 10
unit tests covering subgraph loading, source_id filtering, edge cases |

### How it works

- **GraphRAG:** Before running expensive LLM entity/relation extraction
for a doc, checks the doc store for an existing subgraph (saved by a
previous interrupted run). If found, loads it directly and skips LLM
calls.
- **RAPTOR:** Before processing a doc, checks if RAPTOR chunks
(`raptor_kwd="raptor"`) already exist for it. If yes, skips.

### Testing

- 10 new unit tests — all passing
- Full existing suite: 617 passed

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-04-15 17:37:28 +08:00
euvre
3364d86e6b Auto-inject knowledge parameter in async_chat when prompt_config is missing it (#14121)
### What problem does this PR solve?

Resolve #14115 .

## Problem

On the shared chat link page (`/chats/share?shared_id=...`), querying
the knowledge base returns "no relevant information was found", while
the same query works correctly on the editor chat page.

## Root Cause

Knowledge base retrieval in `async_chat()` is gated by the check `if
"knowledge" in param_keys` (line 598), where `param_keys` is derived
from `prompt_config["parameters"]`. If `parameters` is empty or missing
the `{"key": "knowledge", "optional": false}` entry, retrieval is
entirely skipped.

This can happen because `_apply_prompt_defaults()` — which ensures
`parameters` contains the `knowledge` entry — is only called in the
`create` (POST) and `update_chat` (PUT) handlers, but **not** in
`patch_chat` (PATCH). If a chat's `prompt_config` was updated via PATCH
without including `parameters`, the `knowledge` entry would be absent.
Additionally, `prompt_config["parameters"]` would raise a `KeyError` if
the key was missing entirely.

## Fix

Added a defensive safety net in `async_chat()`
(`api/db/services/dialog_service.py`) that auto-injects the `knowledge`
parameter when:
- `dialog.kb_ids` is set (knowledge bases are configured)
- `"knowledge"` is not already in `param_keys`
- `{knowledge}` placeholder exists in the system prompt

Also changed `prompt_config["parameters"]` to
`prompt_config.get("parameters", [])` to prevent `KeyError` when the key
is absent.

## Files Changed

- `api/db/services/dialog_service.py` — added auto-injection of
`knowledge` parameter and safe `.get()` access for `parameters`


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-15 17:31:31 +08:00
Ea001
38cefd88e2 Fix tag_feas code injection in retrieval ranking (#13923)
## Summary
- remove eval-based parsing from retrieval rank feature scoring
- validate `tag_feas` at write time in chunk APIs and SDK routes
- add regression tests for safe parsing and malicious payload rejection

## Details
`tag_feas` is intended to be structured rank-feature data, but the
retrieval ranking path was evaluating stored values as Python
expressions. This change treats `tag_feas` strictly as data.

### What changed
- replace `eval()` in `rag/nlp/search.py` with safe parsing via
`json.loads()` and optional `ast.literal_eval()` compatibility for
legacy Python-dict strings
- strictly filter parsed values down to `dict[str, finite number]`
- reject invalid `tag_feas` payloads at write time in web chunk routes
and SDK document chunk routes
- add focused regression tests to prove executable strings are ignored
and invalid payloads are rejected

## Validation
- `python -m pytest test/unit_test/common/test_tag_feature_utils.py
test/unit_test/rag/test_rank_feature_scores.py -q`

---------

Co-authored-by: unknown <zhenglinkai@CCN.Local>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-04-15 16:31:11 +08:00
Eden
1f33ca1099 fix(dialog): restore decorated answer in async_ask final SSE event (#13917)
## What's the problem

Both `async_chat()` and `async_ask()` call `decorate_answer()` to build
the final SSE payload — it inserts citation markers (`##N$$`) into the
answer text and prunes `doc_aggs` to only the cited documents.
Immediately after, both functions overwrite `final["answer"]` with `""`:

```python
# async_chat(), line ~774  (issue #13828)
final = decorate_answer(thought + full_answer)
final["final"] = True
final["audio_binary"] = None
final["answer"] = ""   # discards decorated text
yield final

# async_ask(), line ~1444  (same bug, different path)
final = decorate_answer(full_answer)
final["final"] = True
final["answer"] = ""   # discards decorated text
yield final
```

The client receives filtered references (built for a citation-decorated
answer it never sees) while displaying the raw, undecorated streaming
text. Citations can never match.

## Root cause

`final["answer"] = ""` was left over from an earlier design where
clients were meant to reconstruct the full answer purely from delta
events. Once `decorate_answer()` started placing citation markers, this
blank-out broke the contract: the final event is where the decorated
answer should land.

## Fix

Remove the two blank-override lines — one in `async_chat()`, one in
`async_ask()`:

```diff
-    final["answer"] = ""
```

`decorate_answer()` already sets `final["answer"]` to the correct
decorated string; there is nothing to override.

## Relation to #13828

Issue #13828 and PR #13835 identify the bug in `async_chat()`. This PR
absorbs that fix and also corrects the identical pattern in
`async_ask()` (used by the `/retrieval` route in `chat_api.py`), which
PR #13835 does not touch.

## Regression test

Added
`test/unit_test/api/db/services/test_dialog_service_final_answer.py`
with three tests:

| Test | Purpose |
|------|---------|
| `test_buggy_pattern_drops_answer` | Documents the old behaviour:
blank-override empties the final answer |
| `test_fixed_pattern_preserves_decorated_answer` | Core invariant:
final event carries the decorated text from `decorate_answer()` |
| `test_final_event_reference_matches_decorated_result` | Citation
markers in the answer must match the pruned `doc_aggs` in the same event
|

Local run result:

```
test_dialog_service_final_answer.py::test_buggy_pattern_drops_answer         PASSED
test_dialog_service_final_answer.py::test_fixed_pattern_preserves_decorated_answer PASSED
test_dialog_service_final_answer.py::test_final_event_reference_matches_decorated_result PASSED

3 passed in 0.04s
```

`ruff check` passes with no issues on all changed files.

---------

Co-authored-by: edenfunf <edenfunf@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-15 14:10:36 +08:00
balibabu
f08d13287a Feat: Edit the code of the code operator from a broad perspective. (#14116)
### What problem does this PR solve?

Feat: Edit the code of the code operator from a broad perspective.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-15 11:51:17 +08:00
chanx
2d291cd841 fix(flow): Fix text descriptions for multi-column layout options. (#14107)
### What problem does this PR solve?

fix(flow): Fix text descriptions for multi-column layout options.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-15 11:50:58 +08:00
Jin Hai
a0a4029f01 Fix document (#14118)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-15 11:35:16 +08:00
Jack
bc5f78996b Consolidateion of document upload API (#14106)
### What problem does this PR solve?

Consolidation WEB API & HTTP API for document upload

Before consolidation
Web API: POST /v1/document/upload
Http API - POST /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- POST
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-15 11:27:43 +08:00
xinmotlanthua
e1dede1366 fix(web): replace hardcoded English strings with i18n in floating chat widget (#14095)
## Summary
- Replace 3 hardcoded English strings in `floating-chat-widget.tsx` with
`react-i18next` `t()` calls so the widget respects the `locale` query
parameter
- Add `useTranslation` hook to the component
- Add translation keys (`chat.chatSupport`, `chat.replyInstantly`,
`chat.typeYourMessage`) to all 14 locale files

## Strings replaced
| Original | i18n key |
|---|---|
| `'Chat Support'` | `t('chat.chatSupport')` |
| `'We typically reply instantly'` | `t('chat.replyInstantly')` |
| `'Type your message...'` | `t('chat.typeYourMessage')` |

Closes #14072

Co-authored-by: khanhkhanhlele <namkhanh2172@gmail.com>
2026-04-14 20:12:56 +08:00
akie
a98b64326c Add warning log when metadata query hits 10000 result limit (#14109)
## What problem does this PR solve?

Add a warning log when `get_flatted_meta_by_kbs` returns 10,000 results,
which indicates the query limit has been reached and metadata may be
silently truncated.


## Type of change
- [x] Improvement (non-breaking change which improves observability)
2026-04-14 20:04:32 +08:00
NeedmeFordev
1a1b5aa53e Fix: respect the internet toggle before running Tavily web search (#14051) (#14052)
### What problem does this PR solve?

Fixes #14051.

The chat UI already sends an `internet` flag with each request, but the
backend previously triggered Tavily web retrieval whenever
`prompt_config.tavily_api_key` was configured. As a result, web search
could still run even when the internet toggle was off.

This PR makes web search an explicit opt-in at request time:
- `tavily_api_key` only indicates that web search is available
- Tavily retrieval runs only when `internet` is explicitly enabled
- the same behavior now applies to both the normal retrieval path and
the deep-research / reasoning path

This also fixes the no-KB fallback case so chats without KBs fall back
to normal solo chat when `internet` is off.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 19:55:20 +08:00
Jin Hai
8e9cef3687 Remove unused API (#14046)
### What problem does this PR solve?

1. Remove unused token related API
2. Fix typo

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-14 19:32:16 +08:00
chanx
912fedc9b9 Fix: metadata bug (#14105)
### What problem does this PR solve?

Fix: metadata bug

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 18:45:09 +08:00
writinwaters
1c0c1f27ef Doc: Updated FAQ (#14108)
### What problem does this PR solve?

Updated frequently asked questions.

### Type of change


- [x] Documentation Update
2026-04-14 18:42:16 +08:00
balibabu
1bc4868abe Fix: The file count in the file header did not change after uploading or deleting files. (#14034)
### What problem does this PR solve?
Fix: The file count in the file header did not change after uploading or
deleting files.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-14 18:07:32 +08:00
Jack
576431de99 Refactor: Change update doc from PUT to patch (#14067)
### What problem does this PR solve?

Before change, update_document in api/apps/restful_apis/document_api.py
is using "PUT".
After change, it will use "PATCH" which is more suitable.

### Type of change

- [x] Refactoring
2026-04-14 17:12:23 +08:00
Qi Wang
57aec2e65d Fix bug: run Knowledge graph or RAPTOR, it will update an existing task (#14102)
### What problem does this PR solve?

It fixed the bug: https://github.com/infiniflow/ragflow/issues/14101
When run Knowledge graph or RAPTOR, the last document running status
will be wrongly set, see below:
It should never touch existing document result.

![Image](https://github.com/user-attachments/assets/14fe1f9e-0541-4093-8111-ed0bd25b87ba)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 16:37:41 +08:00
balibabu
27ebc64ec0 Feat: Adapted for the upgraded knowledge graph of @antv/g6. (#14103)
### What problem does this PR solve?

Feat: Adapted for the upgraded knowledge graph of @antv/g6.

### Type of change

- [x] Refactoring
2026-04-14 16:33:52 +08:00
Magicbook1108
1376c004a9 Fix: update docs generator (#14070)
### What problem does this PR solve?

Refactor: update docs generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

1. Support multiple document generator components and correctly display
messages in the message component. The document generator will not
overwrite other messages.

<img width="700" alt="Screenshot from 2026-04-13 13-56-17"
src="https://github.com/user-attachments/assets/3f3e06e8-33ce-4df1-8b05-510c86af70a4"
/>

2. Support Chinese content and ensure correct Markdown rendering in PDF
and DOCX
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/69bf1f7b-261d-48e5-a9f3-8e94462b90ed"
/>

3. Simplify configuration page and support more output format
 
<img height="700" alt="image"
src="https://github.com/user-attachments/assets/8647374c-c055-4daa-ad71-cd9052eb138e"
/>

4. Hide download from other components except for message 
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/a723dfcb-b60d-4eb5-b2f6-d41ca5955eb4"
/>

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/a8762ac4-807b-4f0b-9287-65f82f7c9c98"
/>

5. Sanitize filename
 
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/df49509f-37c0-40f9-b03d-bd6ce7fdefa8"
/>


6. And more changes on usability
2026-04-14 15:24:43 +08:00
chanx
1031aebc8f feat(file): Add file ancestor directory lookup feature by go (#14037)
### What problem does this PR solve?

feat(file): Add file ancestor directory lookup feature by go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-14 15:22:03 +08:00
chanx
6aec8058bb refactor: Remove knowledge base-related API handlers that are already included in the dataset. (#14094)
### What problem does this PR solve?

refactor: Remove knowledge base-related API handlers that are already
included in the dataset.

### Type of change

- [x] Refactoring
2026-04-14 15:19:31 +08:00
Jin Hai
2b6c50734f Sync code from EE (#14080)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-14 15:03:46 +08:00
Ricardo-M-L
c22811f096 fix: close file handles in json.load() calls in resume parser (#14061)
## Summary
- Replace `json.load(open(...))` with `with open(...) as f:
json.load(f)` in 2 resume parser files
- Fixes 4 leaked file descriptors in `corporations.py` (3) and
`schools.py` (1)

## Why
In a long-running server process like RAGFlow, leaked file handles can
accumulate and hit the OS file descriptor limit (`OSError: [Errno 24]
Too many open files`). The other instances mentioned in the issue
(`infinity_conn_base.py` and `init_data.py`) have already been fixed.

## Test plan
- [x] Verified affected files use `with` statement after fix
- [x] Grep confirms no remaining `json.load(open(` patterns in codebase

Fixes #13996

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:43:58 +08:00
Idriss Sbaaoui
de6a8e789a Fix: rerank overflow by enforcing top_k and 64 cap (#14084)
### What problem does this PR solve?

This fixes rerank overflow where retrieval could send more documents
than allowed (for example 66 when `page_size=6`), causing provider 400
errors and bypassing the user’s `top_k` intent in rerank-enabled paths.
this pr fixes #14081

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 10:47:25 +08:00
Idriss Sbaaoui
d6987b4d8f Fix p3 ci fails (#14069)
### What problem does this PR solve?

fix issue with stale tests on p3 level

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 10:47:07 +08:00
balibabu
d2b744facd Fix: The indented tree text generated on the search page overlaps. #14077 (#14078)
### What problem does this PR solve?

Fix: The indented tree text generated on the search page overlaps.
#14077

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-14 10:02:00 +08:00
Magicbook1108
8723c3aa86 Feat: more templates (#14075)
### What problem does this PR solve?

Feat: more templates
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/533e88f1-fc56-4337-a026-6623fc978893"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2026-04-14 10:00:55 +08:00
chanx
6ffa566ec3 Refactor: Standardize naming convention to camelCase (#14079)
### What problem does this PR solve?

Refactor: Standardize naming convention to camelCase

### Type of change

- [x] Refactoring
2026-04-13 21:07:07 +08:00
balibabu
9a38af7cbf Feat: Hide the download button embedded in the agent page. (#14083)
### What problem does this PR solve?

Feat: Hide the download button embedded in the agent page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-13 21:06:41 +08:00
Syed Shahmeer Ali
c7ce062ea8 Fix: model_type not passed in ensure_tenant_model_id_for_params causing wrong tenant model lookup (#13782)
Summary

When setting a default model for an OpenAI-API-Compatible provider,
ensure_tenant_model_id_for_params called get_api_key
without a model_type filter. If the same model name was registered under
multiple types (e.g., both chat and embedding),
it could return the wrong tenant_llm_id, leading to Model(@None) not
authorized errors during chat.

This applies the same type-scoped fix that PR #13569 introduced in
get_model_config_by_type_and_name — now consistently
  in tenant_utils.py as well.

  Changes

  - Added _KEY_TO_MODEL_TYPE mapping in tenant_utils.py
- Each model key (llm_id, embd_id, etc.) now passes its correct LLMType
to get_api_key

  Fixes #13775
2026-04-13 20:57:28 +08:00
天海蒼灆
356d45fda1 Feat: add cell type coercion for Excel export (#13808)
### What problem does this PR solve?

- Implemented a helper function to convert markdown cell text to native
numeric types for Excel output.
- Ensured that leading zeros are preserved and handled various numeric
formats, including those with thousand separators and scientific
notation.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-13 20:54:57 +08:00
Lynn
47d3741dcc Feat: migrate script (#14076)
### What problem does this PR solve?

Add command line arguments for mysql config.

### Type of change

- [x] Other (please describe): tool scripts.
2026-04-13 20:45:11 +08:00
bitloi
853021ff2a feat: support multiple canvas_types for agent templates and remove duplicate files (#14030)
### What problem does this PR solve?

Closes #13907

The template catalog had duplicate files (e.g. `*_r.json`) only to place
the same template into multiple sidebar groups.
This increases maintenance cost and makes template updates error-prone.

This PR adds first-class support for multiple template categories in a
single file via `canvas_types`, then removes duplicate template files.

What changed:
- Added `canvas_types` to `CanvasTemplate` model and DB migration.
- Added normalization logic when loading templates:
  - accepts legacy `canvas_type`
  - accepts new `canvas_types`
  - merges/deduplicates values
- preserves backward compatibility by keeping `canvas_type` as first
normalized value.
- Updated template import flow to load only `.json` files and in stable
sorted order.
- Updated frontend template filtering to match on `canvas_types` first,
with fallback to legacy `canvas_type`.
- Consolidated duplicated template pairs into single files and removed:
  - `deep_search_r.json`
  - `reflective_academic_paper_generator_r.json`
  - `seo_article_writer_r.json`
- Added regression/edge-case tests for category normalization and route
serialization expectations.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-13 20:26:30 +08:00
writinwaters
ef07faea80 Doc: Updated frequently asked questions and answers. (#14085)
### What problem does this PR solve?

Updated frequently asked questions. 

### Type of change

- [x] Documentation Update
2026-04-13 20:26:16 +08:00
Tong Liu
6fdca2d212 [Security] Fix jinja2 SSTI vulnerability using SandboxedEnvironment (#14068) 2026-04-13 19:24:13 +08:00
balibabu
a023305b96 Fix: The chat page is not displaying the meta tags. (#14071)
### What problem does this PR solve?

Fix: The chat page is not displaying the meta tags.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-13 16:18:25 +08:00
Krishna Chaitanya
5ece2d8aa8 Fix: upgrade Apache Tika from 3.2.3 to 3.3.0 to address GHSA-72hv-8253-57qq (#13769)
### What problem does this PR solve?

Upgrades Apache Tika from 3.2.3 to 3.3.0 to address the security
vulnerability GHSA-72hv-8253-57qq (TIKA-4687).

Closes #13601

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- `Dockerfile`: Updated tika JAR filename and `TIKA_SERVER_JAR` env var
from 3.2.3 to 3.3.0
- `Dockerfile.deps`: Updated tika JAR filename in COPY instruction from
3.2.3 to 3.3.0
- `download_deps.py`: Updated both Maven Central and Huawei Cloud mirror
download URLs from 3.2.3 to 3.3.0

### References

- Apache Tika 3.3.0 release:
https://www.apache.org/dyn/closer.lua/tika/3.3.0/tika-app-3.3.0.jar
- TIKA-4687: https://issues.apache.org/jira/browse/TIKA-4687
- GHSA-72hv-8253-57qq
2026-04-13 16:01:08 +08:00
Jin Hai
3e787b3b09 Go: update search (#14023)
### What problem does this PR solve?

Update search

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-13 15:07:04 +08:00
Yongteng Lei
1638083e18 Fix: sandbox cannot accept large args list (#14063)
### What problem does this PR solve?

Sandbox cannot accept large args list.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-13 14:14:08 +08:00
Jack
51ce6aab01 Consolidate set_meta into update_document (#14045)
### What problem does this PR solve?

Consolidate "set_meta" API into "update_document" .

Before consolidation
Web API: POST /api/v1/document/set_meta
Http API - PUT /v1/datasets/<dataset_id>/document/<document_id>

After consolidation, Restful API -- PUT
/v1/datasets/<dataset_id>/document/<document_id>

### Type of change

- [x] Refactoring
2026-04-13 12:47:17 +08:00
akie
3911d90993 Fix: agent application can not show Cite (#14047)
Close #14018

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Problem
In Agent applications, even with the cite option enabled, only inline
[ID: x] citation markers are visible (showing chunk content on hover).
The Agent does not display the referenced file cards below the response,
unlike Chat applications.

### Root Cause
The Agent's Retrieval tool (agent/tools/retrieval.py) calls
retriever.retrieval() with aggs=False, which means the retrieval results
do not include doc_aggs (document aggregation) data. Without doc_aggs,
the frontend ReferenceDocumentList component has no data to render the
file cards.

In contrast, the Chat application (api/db/services/dialog_service.py)
calls the same retriever.retrieval() method with aggs=True.

### Fix
Changed aggs=False to aggs=True in agent/tools/retrieval.py so that
document aggregation data is returned along with the retrieved chunks.
2026-04-13 11:06:14 +08:00
writinwaters
52442c8eb5 Docs: Added a guide on adding Github repo as data source (#14048)
### What problem does this PR solve?

Added a guide on adding Github repo as data source

### Type of change


- [x] Documentation Update
2026-04-10 21:32:26 +08:00
balibabu
462be53b76 Fix: When creating a dataset, if no chunk_method is selected, there is no indication that this is a required field. (#14039)
### What problem does this PR solve?
Fix: When creating a dataset, if no `chunk_method` is selected, there is
no indication that this is a required field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 19:05:14 +08:00
Magicbook1108
82d74fd276 Refact: update pipeline template (#14036)
### What problem does this PR solve?

Refact: update pipeline template

### Type of change

- [x] Refactoring
2026-04-10 19:04:52 +08:00
Jack
4046a4cfb6 Consolidateion metadata summary API (#14031)
### What problem does this PR solve?

Consolidation WEB API & HTTP API for document metadata summary

Before consolidation
Web API: POST /api/v1/document/metadata/summary
Http API - GET /v1/datasets/<dataset_id>/metadata/summary

After consolidation, Restful API -- GET
/v1/datasets/<dataset_id>/metadata/summary

### Type of change

- [x] Refactoring
2026-04-10 18:41:30 +08:00
balibabu
11c89d87da Fix: The dataset on the search page is not displaying the required field error message. (#14041)
### What problem does this PR solve?

Fix: The dataset on the search page is not displaying the required field
error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 18:20:50 +08:00
Zhichang Yu
a9ca4ea1a1 Disable flask and quart debug (#14042)
### What problem does this PR solve?

Visit
`http://127.0.0.1:9381/?__debugger__=yes&cmd=resource&f=debugger.js`
will expose the flask code:
```
docReady(() => {
  if (!EVALEX_TRUSTED) {
    initPinBox();
  }
  // if we are in console mode, show the console.
  if (CONSOLE_MODE && EVALEX) {
    createInteractiveConsole();
  }

  const frames = document.querySelectorAll("div.traceback div.frame");
  if (EVALEX) {
    addConsoleIconToFrames(frames);
  }
  addEventListenersToElements(document.querySelectorAll("div.detail"), "click", () =>
    document.querySelector("div.traceback").scrollIntoView(false)
  );
  addToggleFrameTraceback(frames);
  addToggleTraceTypesOnClick(document.querySelectorAll("h2.traceback"));
  addInfoPrompt(document.querySelectorAll("span.nojavascript"));
  wrapPlainTraceback();
});

function addToggleFrameTraceback(frames) {
  frames.forEach((frame) => {
    frame.addEventListener("click", () => {
      frame.getElementsByTagName("pre")[0].parentElement.classList.toggle("expanded");
    });
  })
}

```

### Type of change

- [x] Other (please describe): Fix security risk
2026-04-10 18:01:49 +08:00
Jin Hai
cfc2928de2 Go: remove unused API route (#14028)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 18:00:41 +08:00
Jin Hai
3d59448b0d Go: add parameter parsing of list chats (#14026)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 14:33:32 +08:00
Magicbook1108
18cafff790 Fix: markdown parser in pipeline (#14032)
### What problem does this PR solve?

Fix: markdown parser in pipeline

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 14:11:14 +08:00
Magicbook1108
9ce293a736 Refact: update exesql notification (#14027)
### What problem does this PR solve?

Refact: update exesql notification

### Type of change


- [x] Refactoring
2026-04-10 13:42:57 +08:00
Magicbook1108
87a87a7122 Feat: pipeline support ONE chunking method (#14024)
### What problem does this PR solve?

Feat: pipeline support ONE chunking method

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-10 13:11:22 +08:00
Jin Hai
a37605cbd2 Go: add get chat (#14025)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 13:06:51 +08:00
eason
aa92abe73c fix: close file handles properly in json.load() calls (#13997)
## Summary

Fixes #13996

Replace `json.load(open(...))` with `with open(...) as f: json.load(f)`
in two files to ensure file descriptors are properly closed.

**Affected files:**
- `common/doc_store/infinity_conn_base.py` — schema loading for Infinity
doc store
- `api/db/init_data.py` — agent template loading at startup

## Why this matters

In a long-running server process like RAGFlow, leaked file descriptors
from `json.load(open(...))` can accumulate over time. While CPython's
refcounting usually cleans these up, it's not guaranteed (especially
under memory pressure or with alternative Python runtimes), and can lead
to `OSError: [Errno 24] Too many open files`.

## Test plan

- [ ] Verify Infinity doc store schema loading still works correctly
- [ ] Verify agent templates load correctly on startup

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Improved file handling in internal data processing to ensure proper
resource cleanup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: easonysliu <easonysliu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 12:16:49 +08:00
chanx
4538910b52 feat: Implement file-related functionality (#14011)
### What problem does this PR solve?

feat: Implement file-related functionality

- Implement file deletion API and business logic
- Add context support for file deletion operations and prevent root
folder deletion
-  Implement file move functionality
-  Add File Download API Endpoints and Utility Functions

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-10 12:15:27 +08:00
corevibe555
e7d044413f Fix: Google Drive connector missing new files after initial sync (#13943)
Closes https://github.com/infiniflow/ragflow/issues/13939

## What problem does this PR solve?

The Google Drive connector fails to detect new files after the initial
sync (#13939). The root cause is that `generate_time_range_filter()`
applies a strict `modifiedTime > poll_range_start` cutoff when querying
the Google Drive API. Files uploaded to Google Drive that retain their
original `modifiedTime` (common behavior) get silently excluded if their
timestamp predates the last sync's cutoff.

Unlike the Confluence and Jira connectors which use a configurable time
buffer (`CONFLUENCE_SYNC_TIME_BUFFER_SECONDS`) to offset
`poll_range_start` backward, the Google Drive connector had no such
mechanism — resulting in a razor-sharp timestamp boundary with zero
tolerance for overlap.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


## Summary

* **New Features**
* Added a configurable time buffer for Google Drive synchronization to
address timing delays and improve sync reliability.
* Improved file detection logic to include recently created files
alongside modified ones, reducing missed synchronizations.
2026-04-10 11:39:19 +08:00
NeedmeFordev
7315d25cbc Fix retrieval API handling for omitted dataset IDs (#13990)
### What problem does this PR solve?

This PR fixes a mismatch between the MCP retrieval contract and the
backend retrieval API.

`ragflow_retrieval` already describes `dataset_ids` as optional, but
`/api/v1/retrieval` still rejected omitted or empty `dataset_ids` with
`` `dataset_ids` is required. ``. That made MCP retrieval fail even
though the tool schema promised that the request could search across all
available datasets.

This change updates `/api/v1/retrieval` to accept missing or empty
`dataset_ids`, resolve all accessible datasets for the authenticated
user, and keep the route schema aligned with the new runtime behavior.
It also adds focused unit coverage for the fallback resolution path and
the no-accessible-datasets case.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Fixes: #13981

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved dataset resolution to reliably discover all accessible
datasets through proper pagination, replacing the previous parsing
method.
* Enhanced error handling with clearer messaging when no datasets are
available for retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-10 11:34:15 +08:00
Magicbook1108
27329b40ed Refact: refact on parser structure (#14012)
### What problem does this PR solve?

Refact: refact on parser structure

### Type of change

- [x] Refactoring
2026-04-10 10:03:44 +08:00
Jin Hai
cd04467b9b Go: add delete search (#14014)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 09:42:37 +08:00
balibabu
56810ec5a3 Fix: The knowledge base selected by the retrieval node is not displayed. (#14013)
### What problem does this PR solve?

Fix: The knowledge base selected by the retrieval node is not displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 09:40:35 +08:00
Magicbook1108
52f5880d21 Fix: support vlm fall back in pipeline (#14007)
### What problem does this PR solve?

Fix: support vlm fall back in pipeline for img/table parsing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-09 20:20:11 +08:00
Jin Hai
5951e2b564 Go: Add create search (#13998)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 20:04:06 +08:00
Yongteng Lei
b33d2fdea5 Refa: GraphRAG to use async chat methods instead of thread pool execution (#14002)
### What problem does this PR solve?

GraphRAG _async_chat.

### Type of change

- [x] Refactoring
- [x] Performance Improvement


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Unified chat calls to an async invocation across extractors, improving
timeout handling and ensuring task IDs propagate reliably.
* **Tests**
* Added and expanded unit tests and mocks to cover extractor behavior,
timeout scenarios, and safe test-package imports, reducing regression
risk.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-09 19:57:35 +08:00
Octopus
c2ce49e037 fix: strip single quotes from synonym terms to prevent Infinity TokenError (#13969)
Fixes #13823

## Problem

When querying with words like `cat`, RAGFlow's query expansion system
looks up synonyms via WordNet, which can return terms containing single
quotes (e.g., `cat-o'-nine-tails`). When using Infinity as the document
store, these unescaped single quotes in the query string cause a
`TokenError` because Infinity's lexer treats `'` as a string delimiter.

```
TokenError: Error tokenizing ' OR "big cat" OR "computerized tomography")^0.7)': Missing ' from 1:531
```

## Solution

Strip single quotes from synonym terms before they are inserted into
query expressions, consistent with how single quotes are already
stripped from the input query text (line 51 of `query.py`):

- **`common/query_base.py`**: In `sub_special_char()`, strip `'` before
escaping other special characters. This fixes the Chinese text
processing path and the `paragraph()` method.
- **`rag/nlp/query.py`**: In the English text path, strip `'` from
tokenized synonym terms.
- **`memory/services/query.py`**: Same fix for the memory query English
text path.

## Testing

The fix can be verified by:
1. Using Infinity as the document store (`DOC_ENGINE=infinity`)
2. Creating a dataset and running a retrieval test with the keyword
`cat`
3. Confirming no `TokenError` is raised and results are returned
normally

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced special character handling in query processing and synonym
expansion by properly sanitizing single quotes before text processing.
* Simplified OCR detection output by removing timing metadata while
preserving core detection accuracy.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ximi <octo-patch@github.com>
2026-04-09 19:10:34 +08:00
Jin Hai
e2b879b258 Fix tiny issues (#14006)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Improved authentication error logging to better distinguish between
JWT and API token failures.
* Enhanced code documentation with clarifying comments for better
maintainability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 19:01:36 +08:00
balibabu
3c5a3e5fb4 Feat: Integrate the name, avatar, and description of chat and search into a single component. (#14008)
### What problem does this PR solve?

Feat: Integrate the name, avatar, and description of chat and search
into a single component.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Inline-editable avatar, name, and description fields
  * Expandable content blocks in search results
  * New RAGFlow heading/logo component

* **Refactor**
* Replaced scattered form fields with a composed Avatar/Name/Description
component
  * Mindmap drawer converted to a sheet-based drawer and layout cleanup
* Simplified search page controls and layout; improved scroll viewport
handling

* **Chores**
* Added/updated English and Chinese localization keys (placeholders,
view more/less)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-09 18:51:45 +08:00
eviaaaaa
1e83c8c051 Fix: align MCP tool call timeout and handle empty content (#13899)
### What problem does this PR solve?
Resolves #12105
This PR fixes two MCP tool call issues in
`common/mcp_tool_call_conn.py`.
First, the timeout passed to `tool_call(..., timeout=...)` was only
applied to the outer `future.result(...)` wait, but was not forwarded to
the internal MCP request. As a result, callers could pass a longer
timeout while the actual MCP request still failed after the default
internal timeout.
Second, the MCP tool call result handling assumed `result.content[0]`
always existed. If an MCP server returned an empty content list, this
could raise an exception unexpectedly.
This PR fixes both issues by:
- forwarding the external `timeout` value to the internal MCP request
timeout
- returning a clear message when the MCP server returns empty content
instead of indexing into an empty list

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update                                             
- [ ] Refactoring                               
- [ ] Performance Improvement
- [ ] Other (please describe)
2026-04-09 18:44:04 +08:00
Zhichang Yu
b7744e053e fix: support dense_vector from ES fields response (ES 9.x compatibility) (#13972)
fix: support dense_vector from ES fields response (ES 9.x compatibility)

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Configuration Chore (non-breaking change which updates
configuration)


## Summary by CodeRabbit

* **Bug Fixes**
* More accurate handling and unwrapping of dense-vector fields so
returned values have correct shapes.
* Field selection reliably limits returned data and falls back to
alternate result locations when needed.
* Use of consistent result IDs and tolerant handling when score values
are missing.

* **Chores / Configuration**
* Increased build memory and adjusted build-time flags for the frontend
build.
* Simplified runtime model/GPU checks and removed an automated runtime
GPU-install attempt.

* **Build Fixes**
* `web/vite.config.ts`: make `build.minify` and `build.sourcemap`
respect `VITE_MINIFY` and `VITE_BUILD_SOURCEMAP` env vars from
Dockerfile instead of hardcoding `terser` and `true`.

* **Environment**
* Allow stack version override and default the runtime image tag to
"latest".

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Correct unwrapping of dense-vector fields and reliable field selection
with fallback locations.
* Consistent use of hit-level IDs and tolerant handling when score
values are missing.

* **Chores / Configuration**
* Increased frontend build memory and added build-time minify/sourcemap
flags; build minification and sourcemap now configurable.
* Removed runtime GPU detection for model initialization; force CPU
initialization.

* **Environment**
* Allow stack version override and default runtime image tag to
"latest".

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 17:44:13 +08:00
Magicbook1108
107fe6cf90 Feat: support doc for pipeline parser in word (#14005)
### What problem does this PR solve?

Feat: support doc for pipeline parser in word

### Type of change

- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for processing legacy Word `.doc` file formats,
extending document compatibility.

* **Bug Fixes**
* Enhanced error handling during document parsing to improve reliability
and prevent processing failures.
2026-04-09 16:40:42 +08:00
Magicbook1108
8d52ef2893 Feat: enable sync deleted files for connector (#14000)
### What problem does this PR solve?

Feat: enable sync deleted files for connector
1. first comes with github

### Type of change

- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added "sync deleted files" feature for data sources, enabling
automatic removal of files deleted from the source system.
* Added multilingual support for the new sync deleted files setting
across multiple languages.

* **UI Improvements**
  * Improved checkbox form field rendering and layout.
  * Enhanced full-width display for authentication token input fields.
2026-04-09 16:40:14 +08:00
Jack
577c96bf2a Refactor: Merge document update API (#13962)
### What problem does this PR solve?

Refactor: merge document.rename into document.update_document

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a unified document update API (PUT) supporting name, metadata,
parser/chunk settings, and status changes.

* **Breaking Changes**
* Legacy single-parameter rename endpoint removed; renames now require
dataset + document identifiers.
  * `/list` now reads dataset id from a different query parameter.

* **Validation / Bug Fixes**
* Stricter meta_fields and parser-config validation; unauthenticated
requests return 401.

* **Frontend**
  * UI now sends dataset id when saving document names.

* **Tests**
* Numerous unit and HTTP tests adjusted or removed to match new API and
validations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: MkDev11 <94194147+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
Co-authored-by: Qi Wang <wangq8@outlook.com>
Co-authored-by: dataCenter430 <161712630+dataCenter430@users.noreply.github.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
2026-04-09 11:17:38 +08:00
Ricardo-M-L
c13f8856a1 fix: correct typos in agent component filename and templates (#13930)
## Summary
- Rename misspelled file `varaiable_aggregator.py` →
`variable_aggregator.py`
- Fix `unkown` → `unknown` in template and frontend constant (3
instances)
- Fix `Finale` → `Final` in customer feedback template (2 instances)

## Test plan
- [ ] Verify variable aggregator component loads correctly
- [ ] Verify agent templates render properly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 11:06:01 +08:00
Lynn
dbfb439239 Feat: migrate script (#13976)
### What problem does this PR solve?

Add stage for migrate tenant_llm data into table tenant_model_instance
and tenant_model.

### Type of change

- [x] Other (please describe): tool script


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Added two new migration stages to move tenant model and instance
records into new target tables, with dry-run, full-execute, and "create
table only" modes; migration skips already-migrated rows to avoid
duplicates.
* **Bug Fixes**
  * Cleaned up migration header logging for clearer output.
* **Documentation**
* Added usage guide describing stages, options, modes, config format,
examples, and expected logs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-09 11:03:39 +08:00
Magicbook1108
c5871c1078 Fix: dsl import/export (#13992)
### What problem does this PR solve?

Fix: dsl import/export
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Enhanced JSON import functionality for agents to automatically
populate components from imported graph structures.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 10:55:22 +08:00
qinling0210
82fa85c837 Implement Delete in GO and refactor functions (#13974)
### What problem does this PR solve?

Implement Delete in GO and refactor functions

### Type of change

- [x] Refactoring

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a remove_chunks command to delete specific or all chunks from a
document.
  * Added new endpoints for chunk removal and chunk update.

* **Refactor**
* Renamed index commands to dataset/metadata table terminology and
updated REST routes accordingly.
* Updated chunk update flow to a JSON POST style and improved metadata
error messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-04-09 09:52:31 +08:00
Jack
3b7723855c Fix: revert xgboost version to 1.6.0 (#13984)
### What problem does this PR solve?

Revert xgboost version to 1.6.0

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated xgboost dependency from version 3.2.0 to 1.6.0

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 19:53:47 +08:00
Jin Hai
5fe6f7c9ac Go CLI: Add list configs and set log level command (#13983)
### What problem does this PR solve?

1. list configs
2. set log level debug/info/warn/error/fatal/panic

```

RAGFlow(user)> list configs;
+--------------------+-----------------------+
| key                | value                 |
+--------------------+-----------------------+
| redis_host         | localhost:6379        |
| doc_engine         | elasticsearch         |
| elasticsearch_host | http://localhost:1200 |
| log_level          | info                  |
| database           | mysql                 |
| database_host      | localhost:3306        |
| admin              | 0.0.0.0:9383          |
| storage_engine     | minio                 |
| minio_host         | localhost:9000        |
+--------------------+-----------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added `LIST CONFIGS` command to view system configuration details
(Redis, database, log level, storage engine, and host settings).
* Added `SET LOG LEVEL` command to adjust logging verbosity at runtime.

* **Improvements**
* Enhanced log level configuration defaults and runtime state
management.
* Reorganized token management and system endpoints under `/system/`
routes for better API organization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 19:32:53 +08:00
balibabu
86900dca99 Refactor: Remove unused API code (#13978)
### What problem does this PR solve?

Refactor: Remove unused API code

### Type of change


- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Style**
* Updated table header styling in dataset settings by removing a
hard-coded background color class, allowing the header to use default or
inherited component styling instead.

* **Refactor**
* Removed token management endpoints from the API service. Token
creation, listing, and removal functions are no longer available.
  * Removed the statistics data endpoint from available API routes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 18:46:08 +08:00
balibabu
c0c3287af4 Fix: Error message: Use 'const' instead. (#13982)
### What problem does this PR solve?

Fix: Linter error message: Use 'const' instead.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Updated variable declarations across form components, agent utilities,
memory management hooks, and data handling functions to enhance code
consistency and maintainability throughout the application codebase.

* **Style**
* Added ESLint suppressions to document intentional constant-condition
patterns in asynchronous event streaming operations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 18:13:14 +08:00
Yongteng Lei
3064895bbb Fix: import error in sandbox provider (#13971)
### What problem does this PR solve?

Fix import error in sandbox provider.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated internal configuration import mechanism for sandbox provider
initialization. No end-user impact.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 15:35:30 +08:00
Jin Hai
fa75aee3b9 Refactor system API (#13958)
### What problem does this PR solve?

- ping
- token
- log level

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* System endpoints consolidated under /api/v1/system: ping, health
check, and token management moved to the centralized API surface.
* Token management unified at /api/v1/system/tokens with
list/create/delete behavior.

* **Documentation**
  * API reference updated to reflect the new /api/v1/system paths.

* **Tests**
* Client fixtures and test utilities updated to use
/api/v1/system/tokens; one unit test for health/oceanbase status
removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 15:26:18 +08:00
Jin Hai
ad789f5c43 Fix list files (#13960)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Standardized the query parameter used when listing documents so
listings behave consistently across the web and client interfaces.
* Clarified the error message shown when a required dataset ID is
missing to give clearer guidance to users.

* **Tests**
* Updated test coverage to reflect the standardized dataset identifier
usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 13:38:30 +08:00
balibabu
b8764cfa11 Fix: The document management table cannot be displayed. (#13967)
### What problem does this PR solve?

Fix: The document management table cannot be displayed.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved table layout and overflow behavior in the files view to
ensure proper scrolling and display.

* **Chores**
* Removed unused system status functionality and cleaned up service
methods.
  * Updated TypeScript configuration for compatibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 11:37:27 +08:00
dataCenter430
62a1333cf2 Feat: expose parent-child chunking configuration via HTTP API and Python SDK (#13940)
…
### What problem does this PR solve?

Closes #13857

Parent-child chunking was introduced in v0.23.0 but is only configurable
through the web UI. Users managing datasets programmatically cannot
enable it via the HTTP API or Python SDK because `ParserConfig` uses
`extra="forbid"`, rejecting the `children_delimiter` field at
validation.

### What does this PR change?

Adds a `parent_child` nested config to `ParserConfig`, following the
same pattern as `raptor` and `graphrag`:

```json
"parser_config": {
  "parent_child": {
    "use_parent_child": true,
    "children_delimiter": "\n"
  }
}
```

- api/utils/validation_utils.py — new ParentChildConfig model, added to
ParserConfig
- api/utils/api_utils.py — naive defaults + flatten to
children_delimiter for the execution layer
- api/apps/services/dataset_api_service.py — flatten on the update path
- test/testcases/configs.py — updated DEFAULT_PARSER_CONFIG
-
test/testcases/test_http_api/test_dataset_management/test_create_dataset.py
— 4 valid + 2 invalid test cases

No changes to the execution layer (rag/app/naive.py, rag/nlp/search.py).
Existing UI flow via ext is unaffected.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added parent-child chunking configuration for dataset creation and
updates with new `use_parent_child` toggle and customizable
`children_delimiter` setting to specify how parent chunks are split into
child chunks.

* **Documentation**
* Updated HTTP and Python API references with parent-child chunking
configuration details and examples.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 11:36:57 +08:00
Qi Wang
0ced071a0b Use uv run python3 x.py instead of uv run x.py (#13966)
### Use uv run python3 x.py instead of uv run x.py

When directly call `uv run x.py` it will use the python in shebang, it
does not work if the default python lack of some packages, so change it
to best practices `uv run python3 x.py`

### Type of change

- [x] Documentation Update

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **Documentation**
* Updated development setup instructions across all README files
(English and multiple language translations) to use explicit Python
interpreter invocation for the dependency download command.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 10:33:46 +08:00
MkDev11
cfee2bc9db feat: Auto-adjust chunk recall weights based on user feedback (#12689)
### What problem does this PR solve?

Implements automatic adjustment of knowledge base chunk recall weights
based on user feedback (upvotes/downvotes). When users upvote or
downvote a response, the system locates the corresponding knowledge
snippets and adjusts their recall weight to improve future retrieval
quality.

**Closes #12670**

**How it works:**
1. User upvotes/downvotes a response via `POST /thumbup`
2. System extracts chunk IDs from the conversation reference
3. For each referenced chunk:
   - Reads current `pagerank_fea` value from document store
   - Increments (+1) for upvote or decrements (-1) for downvote
   - Clamps weight to [0, 100] range
   - Updates chunk in ES/Infinity/OceanBase
4. Future retrievals score these chunks higher/lower based on
accumulated feedback

**Files changed:**
- `api/db/services/chunk_feedback_service.py` - New service for updating
chunk pagerank weights
- `api/apps/conversation_app.py` - Integrated feedback service into
thumbup endpoint
- `test/testcases/test_web_api/test_chunk_feedback/` - Unit tests

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Chat message feedback now updates per-chunk relevance weights
(feature-flag gated), with configurable weighting and atomic updates
across storage backends.

* **Bug Fixes**
* Stricter validation for message feedback inputs and more robust
handling of feedback transitions.

* **Tests**
* Expanded test coverage for chunk-feedback behavior, weighting
strategies, storage backends, and thumb-flip scenarios.

* **Chores**
  * CI workflow extended to run the new chunk-feedback web API tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
2026-04-08 09:52:18 +08:00
Jin Hai
4a2a17c27a Fix typos (#13961)
### What problem does this PR solve?

as title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Internal code quality improvements with no user-facing changes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 23:16:52 +08:00
Jin Hai
931021875a Refactor system/version API to RESTful style (#13956)
### What problem does this PR solve?

Refactor version API to RESTful style. Python and go server API also
updated.
### Type of change

- [x] Refactoring



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **Refactor**
* Migrated core API endpoints to the `/api/v1/` namespace for improved
consistency and organization.
* Standardized system version, search, and chat list endpoints under the
new API versioning structure.

* **New Features**
* Added MinIO region configuration support, allowing specification of
storage engine regional settings via environment variables or
configuration files.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 19:07:47 +08:00
Yang_Ming
bc8d67ce78 feat: add region parameter support to MinIO connection (#13954)
## Summary
- Add optional `region` parameter to `Minio()` client constructor in
`rag/utils/minio_conn.py`
- Reads from `MINIO.region` in settings, defaults to `None` when not
configured
- Required by some S3-compatible storage services (e.g., AWS S3, Tencent
COS) for proper bucket access

## Motivation
When using RAGFlow with S3-compatible storage that requires a region
(such as AWS S3 or Tencent Cloud COS), the MinIO client fails to access
buckets because the `region` parameter is not passed through.

The `Minio()` Python client already supports the `region` parameter
natively — this PR simply wires it up from the RAGFlow configuration.

## Changes
- `rag/utils/minio_conn.py`: Pass `region=settings.MINIO.get("region",
None) or None` to `Minio()` constructor

## Backward Compatibility
- No breaking changes. When `region` is not configured, it defaults to
`None`, preserving the existing behavior exactly.

## Test Plan
- [ ] Verified with MinIO (no region set) — works as before
- [x] Verified with S3-compatible storage requiring region — bucket
access succeeds

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced MinIO client initialization with regional configuration
support for improved compatibility with region-specific deployments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Jarry Wang <code-better-life@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 16:38:23 +08:00
Jin Hai
68f665be7a CLI: Add float parsing (#13955)
### What problem does this PR solve?

Add float parsing

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 15:09:45 +08:00
Jin Hai
393efa9b7c Refactor variable of front end (#13953)
### What problem does this PR solve?

api_host -> webAPI
ExternalApi -> restAPIv1

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Updated internal API endpoint configuration to use consolidated base
URL constants for improved maintainability and consistency across the
application.

* **Chores**
* Updated server-side protocol validation for admin connectivity checks.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 15:08:11 +08:00
balibabu
38acf34724 Fix: The agent selected a knowledge base, but the API returned the error: "No dataset is selected". (#13950)
### What problem does this PR solve?

Fix: The agent selected a knowledge base, but the API returned the
error: "No dataset is selected".

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-07 14:16:37 +08:00
auyua9
fa08fa2a17 docs: fix broken internal links in guides (#13935)
### What problem does this PR solve?

This fixes two broken internal documentation links in the guides:

- `docs/develop/mcp/launch_mcp_server.md` linked
`./acquire_ragflow_api_key.md`, but the target page lives one level up
as `../acquire_ragflow_api_key.md`.
- `docs/guides/dataset/run_retrieval_test.md` linked
`./construct_knowledge_graph.md`, but the actual page lives under
`./advanced/construct_knowledge_graph.md`.

These broken links make it harder to follow the MCP and retrieval-test
docs from the local docs tree.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-07 14:01:12 +08:00
Jin Hai
9ac5d28f06 Refactor context command (#13952)
### What problem does this PR solve?

Refactor context search command

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 13:59:27 +08:00
Ricardo-M-L
424aee5bec fix: correct typos in code comments, docstrings and docs (#13931)
## Summary
- Fix `a image` → `an image` in README and log message
- Fix `colomn` → `column` in table structure recognizer comment
- Fix `formated` → `formatted` in confluence connector docstring
- Fix `tabel of content` → `table of contents` in TOC prompt

## Test plan
- [ ] Documentation and comment changes, no functional impact

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 13:05:39 +08:00
Ricardo-M-L
29cf8aba48 fix: correct typos in locale files and search hooks (#13932)
## Summary
- Fix `Refrence` → `Reference` in zh, id, zh-traditional locale files
(en.ts already correct)
- Fix `from from` → `from` and `this files` → `this file` in en.ts
- Fix variable name `reponse` → `response` in search hooks

## Test plan
- [ ] Verify UI strings display correctly
- [ ] Verify search functionality works with renamed variable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 12:26:25 +08:00
Yongteng Lei
112007243d Refa: refine code_exec component (#13925)
### What problem does this PR solve?

Refine code_exec component.

### Type of change

- [x] Refactoring
2026-04-07 11:48:29 +08:00
Jack
c4b0aaa874 Fix: #6098 - Add validation logic for parser_config when update document (#13911)
### What problem does this PR solve?

Add validation logic for parser_config.
Refactor the processing flow. Before change, validation logics and
update logics are mixed up - some validation logis executes followed by
some update logic executes and then another such
"validation-and-then-update" which is not good. After change, all
validation logic executes firstly. Update logic will be executed after
ALL validation logic executed.
Validation logic for parameters (that come from front end) will be
checked using Pydantic. For validation logic that depends on data from
DB, they will be in separate methods.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-04-07 11:33:05 +08:00
Jin Hai
5673245134 Refactor context command (#13948)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 11:30:09 +08:00
Idriss Sbaaoui
ff27ce86d6 fix: gpt-5 name-based config clearing from base chat path (#13949)
### What problem does this PR solve?

fix #13944 where OpenAI-compatible custom endpoints failed verification
when model names contained `gpt-5` becauser of incorrect name-based
handling in the Base/backend=`base` path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-07 11:24:47 +08:00
buildearth
a0be7c7ca7 Fix(connector): expose id_column, timestamp_column, metadata_columns for MySQL/PostgreSQL incremental sync (#13849)
### What problem does this PR solve?
The MySQL and PostgreSQL sync classes in `sync_data_source.py` were not
passing `id_column`, `timestamp_column`, and `metadata_columns` to
`RDBMSConnector`,
making incremental sync and document update impossible even when
configured.
   
- Without `id_column`: updated records generate new documents instead of
overwriting existing ones (doc ID is derived from content hash, so any
change produces a new ID).
- Without `timestamp_column`: `poll_source` always falls back to full
sync,
ignoring the configured time range.
- The three fields existed in the frontend default values but had no
form
inputs, so users had no way to fill them in.
### Type of change
  - [x] Bug Fix (non-breaking change which fixes an issue)        
  - [x] New Feature (non-breaking change which adds functionality)

### Changes
   
- **Backend** (`rag/svr/sync_data_source.py`): pass `id_column`,
    `timestamp_column`, and `metadata_columns` from `self.conf` to
`RDBMSConnector` for both `MySQL` and `PostgreSQL` sync classes.
- **Frontend**
(`web/src/pages/user-setting/data-source/constant/index.tsx`):
add `ID Column`, `Timestamp Column`, and `Metadata Columns` form fields
    to MySQL and PostgreSQL data source configuration UI with tooltips.

Signed-off-by: lixintao <lixintao@uniontech.com>
Co-authored-by: lixintao <lixintao@uniontech.com>
2026-04-07 10:24:30 +08:00
qinling0210
49386bc1b5 Implement UpdateDataset and UpdateMetadata in GO (#13928)
### What problem does this PR solve?

Implement UpdateDataset and UpdateMetadata in GO

Add cli:
UPDATE CHUNK <chunk_id> OF DATASET <dataset_name> SET <update_fields>
REMOVE TAGS 'tag1', 'tag2' from DATASET 'dataset_name';
SET METADATA OF DOCUMENT <doc_id> TO <meta>


### Type of change

- [ ] Refactoring
2026-04-07 09:44:51 +08:00
Lynn
60ec5880e5 Feat: mysql data migrate script (#13927)
### What problem does this PR solve?

Add a script to migrate data in tenant_llm into tenant_model_provider.

### Type of change

- [x] Other (please describe): tool script.
2026-04-03 20:01:37 +08:00
Magicbook1108
69264b3a70 Feat: Refact pipeline (#13826)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 19:26:45 +08:00
Jin Hai
6d9430a125 Add think chat to CLI (#13922)
### What problem does this PR solve?

Now user can use 'think mode' to chat with LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-03 18:11:23 +08:00
Yingfeng
e518c20736 Update README (#13924)
### Type of change

- [x] Documentation Update
2026-04-03 17:29:48 +08:00
akie
35b2a714f9 Fix: tag datasets not visible in tag sets dropdown (#13921)
## Problem Description

When a user creates Dataset A using the **Tag parser** (for CSV/Excel
files with tag definitions), and then creates Dataset B, the Tag Sets
dropdown in Dataset B's Configuration page cannot display Dataset A.

### Steps to Reproduce
1. Create Dataset A with **Tag** as the chunking method
2. Upload a CSV file to Dataset A to generate tags
3. Create Dataset B
4. Navigate to Dataset B → Configuration → Tag Sets
5. **Expected**: Dataset A should appear in the dropdown
6. **Actual**: The dropdown is empty, Dataset A is not visible

---

## Root Cause Analysis

After thorough code review, **the original code logic is correct**. The
`chunk_method` field flows properly through the system:

### Data Flow

```mermaid
sequenceDiagram
    participant Frontend
    participant Pydantic
    participant API
    participant Database

    Note over Frontend,Database: Creating a Tag Dataset
    Frontend->>Pydantic: POST {chunk_method: "tag"}
    Pydantic->>API: serialization_alias converts<br/>chunk_method → parser_id
    API->>Database: INSERT {parser_id: "tag"}

    Note over Frontend,Database: Querying Datasets
    Frontend->>API: GET /api/v1/datasets
    API->>Database: SELECT parser_id, ...
    Database-->>API: Returns {parser_id: "tag"}
    API->>API: remap_dictionary_keys()<br/>parser_id → chunk_method
    API-->>Frontend: {chunk_method: "tag"}

    Note over Frontend: Filter: x.chunk_method === 'tag'
    Note over Frontend:  Match found!
```

### Field Mapping

**Location**: `api/utils/api_utils.py:657-662`
```python
DEFAULT_KEY_MAP = {
    "chunk_num": "chunk_count",
    "doc_num": "document_count",
    "parser_id": "chunk_method",  # Maps DB field to API response
    "embd_id": "embedding_model",
}
```

### Frontend Filtering (Already Correct)

**Location**:
`web/src/pages/dataset/dataset-setting/components/tag-item.tsx:24`
```typescript
const knowledgeOptions = knowledgeList
  .filter((x) => x.chunk_method === 'tag')  //  Correct field
  .map((x) => ({...}));
```

---

## Actual Issue

The most likely causes for the "bug" are:

1. **Browser Cache**: Old data cached before proper deployment
2. **Stale Data**: Datasets created before the code was fully deployed
3. **Container Not Restarted**: Changes not applied to running container

---

## Resolution

**No code changes are needed.** The existing code correctly:

1. Accepts `chunk_method` from frontend
2. Converts to `parser_id` via Pydantic serialization_alias
3. Stores in database as `parser_id`
4. Maps back to `chunk_method` in API response
5. Frontend filters by `chunk_method === 'tag'`
2026-04-03 17:29:10 +08:00
LeonTung
0b724be521 chore(templates): Update the customer feedback dispatcher template (#13919)
### What problem does this PR solve?
Update the customer feedback dispatcher template and introduce a new
operator `Variable Aggregator`.

### Type of change

- [x] Other (please describe): Template change

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-03 16:51:39 +08:00
balibabu
5b43c7cf16 Feat: Place the language configuration in web/.env for easy user configuration. (#13920)
### What problem does this PR solve?

Feat: Place the language configuration in web/.env for easy user
configuration.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-03 16:50:18 +08:00
Ricardo-M-L
354108922b fix: use f-string with separator in switch operator error message (#13915)
\`switch.py\` line 137 concatenates the operator directly after the text
without separator:
\`'Not supported operator' + operator\` → produces \`"Not supported
operatorXXX"\`

Changed to: \`f'Not supported operator: {operator}'\`
2026-04-03 16:49:28 +08:00
chanx
21af67f6f9 feat(File Management): Refactor File List API and Add Knowledge Base Document Initialization (#13914)
### What problem does this PR solve?

feat(File Management): Refactor File List API and Add Knowledge Base
Document Initialization

- Migrate the file list API endpoint from `/v1/file/list` to
`/api/v1/files` to align with the Python implementation.
- Add logic for initializing knowledge base documents; automatically
create the `.knowledgebase` folder and associated documents when
retrieving the root directory.
- Enhance parameter validation and error handling, including the
introduction of a new `CodeParamError` error code.
- Optimize the file list response structure to match the implementation
on the Python side.
- Update the Vite configuration to support proxying the new
`/api/v1/files` endpoint.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-03 15:08:43 +08:00
writinwaters
6263857c1e Agent templates regrouped and renamed (#13873)
### What problem does this PR solve?

Regrouped and renamed agent templates to increase user engagement.

### Type of change


- [x] Refactoring
2026-04-03 13:43:25 +08:00
Zhichang Yu
ab358fe949 feat: make Azure cloud authority configurable for SPN auth (#13898)
## Summary
- The Azure SPN storage handler hardcoded
`AzureAuthorityHosts.AZURE_CHINA`, preventing users in Azure Public
Cloud regions (UK-South, EU, US, etc.) from authenticating
- Add a `cloud` config option (env: `AZURE_CLOUD`) supporting all four
Azure sovereignties: `public`, `china`, `government`, `germany`
- Defaults to `public` (global Azure) — the most common international
use case

Closes #13259

## Test plan
- [ ] Verify default (`cloud: public`) connects to Azure Public Cloud
endpoints
- [ ] Verify `cloud: china` retains existing behavior for Azure China
users
- [ ] Verify `AZURE_CLOUD` env var overrides the config file value

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 12:51:26 +08:00
Zhichang Yu
384fa6fc6e Replace MinIO official image with pgsty/minio fork (#13896)
## Summary

- Replace `quay.io/minio/minio` with `pgsty/minio` community fork in
`docker/docker-compose-base.yml`

MinIO stopped distributing pre-built Docker images and changed its
license. The pgsty/minio fork provides drop-in compatible images under
AGPLv3.

Closes #13840

## Test plan

- [x] Verify `docker compose -f docker/docker-compose-base.yml up -d`
pulls the pgsty/minio image successfully
- [ ] Verify MinIO console accessible on port 9001
- [ ] Verify RAGFlow backend can connect to MinIO and perform file
operations normally

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 22:03:02 +08:00
Yongteng Lei
b7daf6285b Refa: Chat conversations /convsersation API to RESTFul (#13893)
### What problem does this PR solve?

Chat conversations /convsersation API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-02 20:49:23 +08:00
chanx
bbb9b1df85 feat: Implement file upload and folder creation features by GO (#13903)
### What problem does this PR solve?

feat: Implement file upload and folder creation features

- Add file upload route in router.go
- Add file operation methods in dao/file.go
- Add util/file.go for file type detection and filename handling
- Implement file upload and folder creation endpoints in handler/file.go
- Implement file upload and folder creation logic in service/file.go
- Modify response message format in memory.go
- Add document count method in dao/document.go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-02 20:21:04 +08:00
Jin Hai
6c29128de1 Refactor model provider and command (#13887)
### What problem does this PR solve?

Introduce 5 new tables, including model groups and provider instance.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-02 20:20:35 +08:00
qinling0210
f02f5fa435 Get ROW_ID from search() in Infinity (#13901)
### What problem does this PR solve?

1. Search() in Infinity can return row_id now

2.  To Get ROW_ID from search(), refer to handling of retrieval_test.

example
```
$ curl -s -X POST "http://localhost:$PORT/v1/chunk/retrieval_test" -H "Authorization: $TOKEN" -H "Content-Type: application/json" -d '{"kb_id": "4fcd01582ca911f1954184ba59049aa3", "question": "曹操"}'
```


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 18:56:43 +08:00
Idriss Sbaaoui
ee1bb8a8b5 Fix: overlapping document parse race that can clear chunks (#13900)
### What problem does this PR solve?

This PR fixes a race in batch document parsing where overlapping parse
requests for the same document could clear/rewrite chunk state and make
previously parsed content appear lost. It adds an atomic per-document
parse guard so only one parse can run at a time for that document (Fixes
#13864 ).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 18:50:56 +08:00
writinwaters
3b96cedece Docs: Updated chat-specific APIs (#13888)
### What problem does this PR solve?

Chat-specific API descriptions updated.

### Type of change

- [x] Documentation Update
2026-04-02 14:15:09 +08:00
NeedmeFordev
6b7989b4b4 Add file type validation (#13802)
### What problem does this PR solve?

This PR fixes WebDAV sync behavior for unsupported file types
([#13795](https://github.com/infiniflow/ragflow/issues/13795)).

Previously, the WebDAV connector selected files primarily by modified
time (and size threshold) and could still pass unsupported extensions
into the download/document-generation path. This caused unnecessary
processing and inconsistent behavior compared with connectors that
validate file type earlier.

This change adds extension validation in two places:

1. **Early filter during recursive listing** to skip unsupported files
before they enter the download flow.
2. **Defensive filter before download/document creation** to prevent
unsupported files from being processed if any listing edge case slips
through.

It also wires `allow_images` into the WebDAV sync path so image
extension handling follows connector policy.

Scope is intentionally limited to WebDAV for a focused bug-fix PR.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- Manual verification with mixed file types under the configured WebDAV
path:
  - supported: `.pdf`, `.txt`, `.md`
  - unsupported: `.exe`, `.bin`, `.dat`
- Triggered full sync and polling sync.
- Confirmed unsupported files are skipped before download.
- Confirmed supported files are still indexed normally.
- Confirmed image handling follows `allow_images` setting.

Fixes: #13795
2026-04-02 14:12:27 +08:00
Idriss Sbaaoui
dd529137eb Fix: markdown table double extraction in parser (#13892)
### What problem does this PR solve?

Fixes markdown tables being parsed twice (once as markdown and again as
generated HTML), which caused duplicate table chunks in the chunk list
UI.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 13:31:56 +08:00
Sank
1c2c4b337e [RU] Add schema synchronization and translate (#13891)
### What problem does this PR solve?

Add schema synchronization and translate

### Type of change

- [x] Translation into Russian
2026-04-02 11:18:27 +08:00
Ricardo-M-L
09a09a5b20 fix: correct typo in IterationItem name check and incomplete error message (#13890)
Two small fixes:

1. **iterationitem.py line 72**: Typo "interationitem" → "iterationitem"
(missing 't'). The component name check never matched IterationItem
components.

2. **raptor.py line 94**: Error message "Embedding error: " had a
trailing colon with no details. Changed to "Embedding error: empty
embeddings returned".
2026-04-02 10:35:28 +08:00
balibabu
af40be68c3 Fix: The dataset on the list page cannot be renamed. (#13886)
### What problem does this PR solve?

Fix: The dataset on the list page cannot be renamed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-01 20:53:05 +08:00
Yongteng Lei
b622c47ed6 Refa: Chats /chat API to RESTFul (#13881)
### What problem does this PR solve?

 Refactor Chats /chat API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-01 20:10:37 +08:00
qinling0210
bb4a06f759 Implement InsertDataset and InsertMetadata in GO (#13883)
### What problem does this PR solve?

Implement InsertDataset and InsertMetadata in GO

new internal cli for go:

INSERT DATASET FROM FILE "file_name"
INSERT METADATA FROM FILE "file_name"

### Type of change

- [x] Refactoring
2026-04-01 16:16:25 +08:00
Liu An
b1d28b5898 Revert "Refa: Chats /chat API to RESTFul (#13871)" (#13877)
### What problem does this PR solve?

This reverts commit 1a608ac411.

### Type of change

- [x] Other (please describe):
2026-04-01 11:05:29 +08:00
Yongteng Lei
1a608ac411 Refa: Chats /chat API to RESTFul (#13871)
### What problem does this PR solve?

Chats /chat API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-01 10:50:22 +08:00
balibabu
00b62dd587 Feat: If a model configured in the agent is deleted from the user center, a notification will be displayed on the canvas with a red border. (#13872)
### What problem does this PR solve?

Feat: If a model configured in the agent is deleted from the user
center, a notification will be displayed on the canvas with a red
border.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-31 18:43:24 +08:00
Jin Hai
efd6ecc3e5 New provider and models API and CLI (#13865)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-31 18:42:12 +08:00
Sank
68b4287892 Add translate [RU] for MinerU (#13832)
add translate for MinerU in knowledgeConfiguration

### Type of change

- [X] Other (please describe):
2026-03-31 17:03:31 +08:00
balibabu
36513313f8 Fix: The agent form sheet will be obscured by the message log sheet. (#13870)
### What problem does this PR solve?

Fix: The agent form sheet will be obscured by the message log sheet.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-31 16:18:43 +08:00
Paul Y Hui
3e702c6265 fix: guard against missing/malformed Authorization header in apikey_required (#13860)
### What problem does this PR solve?

Previously, `apikey_required` called
`request.headers.get('Authorization').split()[1]` without checking for
None or insufficient parts, causing an unhandled AttributeError or
IndexError (500) instead of a proper 403 JSON response.

This applies the same guarding pattern already used by `token_required`
in the same file.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-03-31 15:25:00 +08:00
balibabu
4f27090289 Fix: Unable to reconnect after deleting the connection between begin and parser #13868 (#13869)
### What problem does this PR solve?

Fix: Unable to reconnect after deleting the connection between begin and
parser #13868
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-31 14:44:47 +08:00
writinwaters
db5ab7bbe8 Docs: Image2text is supported by GPUStack. (#13856)
### What problem does this PR solve?

 Image2text is supported by GPUStack. #9515 

### Type of change

- [x] Documentation Update
2026-03-30 20:39:02 +08:00
balibabu
3a4f0d1a83 Fix: The chat settings are not displayed correctly on the first page load. (#13855)
### What problem does this PR solve?
Fix: The chat settings are not displayed correctly on the first page
load.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-30 20:16:52 +08:00
qinling0210
620fe215a4 Fix python metadata search (#13727)
### What problem does this PR solve?

Fix python metadata search

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-30 19:37:19 +08:00
qinling0210
0462c20113 Fix special characters in matching text of search() (#13852)
### What problem does this PR solve?

Fix special characters in matching text of search(). We should escape
some special characters(such as ?, *,:) before passing to matching_text
of search()

Fix https://github.com/infiniflow/ragflow/issues/13729

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-30 18:47:10 +08:00
Zhichang Yu
0d85a8e7aa feat: add dynamic log level adjustment APIs (#13850)
Add REST APIs to dynamically query and modify log levels at runtime for
both Python (Flask) and Go servers.

Changes:
- common/log_utils.py: add set_log_level() and get_log_levels()
functions
- admin/server/routes.py: add GET/PUT /api/v1/admin/log_levels endpoints
- api/apps/system_app.py: add GET/PUT /api/{version}/system/log_levels
endpoints
- internal/logger/logger.go: add GetLevel() and SetLevel() with atomic
level support
- internal/handler/system.go: add GetLogLevel, SetLogLevel, Health
handlers
- internal/router/router.go: route /health to systemHandler
- internal/admin/handler.go: add GetLogLevel, SetLogLevel handlers
- internal/admin/router.go: add /api/v1/admin/log_level routes

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:40:58 +08:00
黄圣祺
534729546e fix(html-parser): correct h4 heading mapping from ##### to #### (#13833)
## Summary

- Fix incorrect Markdown heading mapping for `h4` in `TITLE_TAGS`
dictionary
- `h4` was mapped to `"#####"` (h5 level) instead of `"####"` (correct
h4 level)

Closes #13819

## Details

In `deepdoc/parser/html_parser.py`, the `TITLE_TAGS` dictionary had a
typo where `h4` was assigned 5 `#` characters instead of 4, causing h4
headings to be converted to h5-level Markdown headings during HTML
parsing.

## Test plan

- [ ] Parse an HTML document containing `<h4>` tags and verify the
output uses `####` (4 hashes)
- [ ] Verify other heading levels remain correct

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 13:17:32 +08:00
Jin Hai
2faaa9f9ce Update docker container start printout (#13847)
### What problem does this PR solve?

Printout RAGFlow version

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-30 12:59:42 +08:00
Jin Hai
e20cf39735 Refactor Go server model provider reading and access (#13831)
### What problem does this PR solve?

1. Refactor model provider json file format
2. Use memory data structure to replace database
3. Add CLI command to access

```
RAGFlow(user)> list pool models from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| features                                                                            | max_tokens | model_types | name                  |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| map[]                                                                               | 256000     | [llm]       | grok-4                |
| map[]                                                                               | 131072     | [llm]       | grok-3                |
| map[]                                                                               | 131072     | [llm]       | grok-3-fast           |
| map[]                                                                               | 131072     | [llm]       | grok-3-mini           |
| map[]                                                                               | 131072     | [llm]       | grok-3-mini-mini-fast |
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768      | [vlm]       | grok-2-vision         |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
RAGFlow(user)> show pool model 'grok-2-vision' from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| features                                                                            | max_tokens | model_types | name          |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768      | [vlm]       | grok-2-vision |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
RAGFlow(user)> list pool providers;
+--------+------------------------------------------------------------+---------------------------+
| name   | tags                                                       | url                       |
+--------+------------------------------------------------------------+---------------------------+
| OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | https://api.openai.com/v1 |
| xAI    | LLM                                                        | https://api.x.ai/v1       |
+--------+------------------------------------------------------------+---------------------------+
RAGFlow(user)> show pool provider 'openai';
+---------------------------+--------+------------------------------------------------------------+--------------+
| base_url                  | name   | tags                                                       | total_models |
+---------------------------+--------+------------------------------------------------------------+--------------+
| https://api.openai.com/v1 | OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | 27           |
+---------------------------+--------+------------------------------------------------------------+--------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-30 12:00:49 +08:00
qinling0210
a8bbe167a9 Bump to infinity v0.7.0-dev5 (#13846)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev5

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-30 10:19:06 +08:00
Heyang Wang
641b319647 feat: support reading tags via API (#12891) (#13732)
### What problem does this PR solve?

Enable reading Tag Set tags via API (expose tag_kwd field). The result
of the queried list chunks is as shown below:

<img width="1422" height="818" alt="image"
src="https://github.com/user-attachments/assets/abd1960a-fe34-489e-9d72-525f8e574938"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: heyang.why <heyang.why@alibaba-inc.com>
2026-03-29 20:17:01 +08:00
KeJun
cb78ce0a7b feat: support rss datasource (#13721)
### What problem does this PR solve?

Supporting public RSS/Atom feed URLs as data sources for RagFlow.

link https://github.com/infiniflow/ragflow/issues/12313

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-27 22:58:44 +08:00
Jin Hai
f32a832f92 Add rename model directory to entity to avoid name misunderstanding (#13829)
### What problem does this PR solve?

Model-> entity

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-27 19:25:18 +08:00
balibabu
308b3a1299 Feat: Remove antd-related code and upgrade lucide-react to the latest version. (#13830)
### What problem does this PR solve?

Feat: Remove antd-related code and upgrade lucide-react to the latest
version.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-27 19:24:52 +08:00
Jin Hai
1fff48b656 Add minio go test (#13800)
### What problem does this PR solve?

1. Add go test
2. Update CI process

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-27 18:12:56 +08:00
Liu An
2240fc778c Fix: add missing "mom" field to infinity_mapping.json for parent-child chunker (#13821)
### What problem does this PR solve?

When using Infinity as DOC_ENGINE with parent-child chunker enabled,
vector insertion fails because the "mom" field is missing from the index
mapping. This fix adds the required field to resolve the issue.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-27 13:06:18 +08:00
Krishna Chaitanya
cdbbd2620c Fix: upgrade pyasn1 from 0.6.2 to 0.6.3 to address CVE-2026-30922 (#13773)
## Summary

- Adds `pyasn1>=0.6.3` as a `[tool.uv.constraint-dependencies]` entry to
mitigate **CVE-2026-30922** (CVSS 7.5 HIGH)
- Regenerates `uv.lock` so the resolved pyasn1 version moves from
**0.6.2 to 0.6.3**

## Details

**CVE-2026-30922** is a Denial of Service vulnerability in pyasn1 caused
by unbounded recursion when decoding ASN.1 data with deeply nested
structures. An attacker can send crafted payloads with thousands of
nested SEQUENCE or SET tags to trigger a `RecursionError` crash or
memory exhaustion.

- **Severity:** HIGH (CVSS 7.5)
- **Affected versions:** pyasn1 < 0.6.3
- **Fixed in:** pyasn1 >= 0.6.3
- **NVD:** https://nvd.nist.gov/vuln/detail/CVE-2026-25769

`pyasn1` is not a direct dependency of RAGFlow but is pulled in
transitively via `google-auth` -> `rsa` -> `pyasn1-modules` -> `pyasn1`.
The `constraint-dependencies` mechanism in uv is the correct way to
enforce a minimum version for transitive dependencies without polluting
the direct dependency list.

## Test plan

- [x] `pyproject.toml` passes TOML validation
- [x] `uv lock` resolves successfully with the new constraint
- [x] pyasn1 version in `uv.lock` is now 0.6.3
- [ ] Existing CI/CD tests continue to pass

Closes #13686
2026-03-27 10:37:34 +08:00
chanx
8a9bbf3d6d Feat: add memory function by go (#13754)
### What problem does this PR solve?

Feat: Add Memory function by go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-27 09:49:50 +08:00
黄圣祺
406339af1f Fix(paddleocr): load all PDF pages for image cropping instead of first 100 (#13811)
## Summary

Closes #13803

The `__images__` method in `paddleocr_parser.py` defaulted to
`page_to=100`, only loading the first 100 pages for image cropping.
However, the PaddleOCR API processes **all** pages of the PDF. For PDFs
with more than 100 pages, page indices beyond 99 were rejected as out of
range during crop validation, causing content loss.

## Root Cause

```
__images__(page_to=100) → loads pages 0-99 → page_images has 100 entries
PaddleOCR API → processes all 226 pages → tags reference pages 1-226
extract_positions() → converts tag "101" to index 100
crop() validation → 0 <= 100 < 100 → False → "All page indices [100] out of range"
```

## Fix

Changed `page_to` default from `100` to `10**9`, so all PDF pages are
loaded for cropping. Python's list slicing safely handles oversized
indices.

## Test plan

- [ ] Parse a PDF with >100 pages using PaddleOCR — no more "out of
range" warnings
- [ ] Parse a PDF with <100 pages — behavior unchanged
- [ ] Verify cropped images are generated correctly for all pages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 09:33:11 +08:00
Sank
992a15146d Add translate autoMetadata (#13807)
### What problem does this PR solve?

add translate autoMetadata in Russia

### Type of change

- [x] Other
2026-03-27 09:31:20 +08:00
Renzo
f3aa3381a2 Fix username line break in SharedBadge component (#13794)
## Summary
- Added Tailwind truncation classes (`inline-block max-w-[120px]
truncate align-middle`) to the username `<span>` in `SharedBadge` to
prevent long usernames from wrapping onto multiple lines
- Added `title` attribute to show the full username on hover when
truncated


![ragflow](https://github.com/user-attachments/assets/8b3d8c03-d605-4957-bcf0-8b4d81fc4e70)


## Test plan
- [x] Verify long usernames display truncated with ellipsis (`...`)
- [x] Verify hovering over a truncated username shows the full name as a
tooltip
- [x] Verify short usernames display normally without truncation

Closes #13748
2026-03-27 09:31:08 +08:00
Yingfeng
6e309f9d0a Feat: Initialize context engine CLI (#13776)
### What problem does this PR solve?

- Add multiple output format to ragflow_cli
- Initialize contextengine to Go module
  - ls datasets/ls files
  - cat file
  - search -d dir -q query

issue: #13714

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-26 21:07:06 +08:00
Idriss Sbaaoui
3b1e77a6d4 Fix: shared KB embedding authorization for team members (#13809)
### What problem does this PR solve?

fixes issue #13799 where team members get model not authorized when
running RAG on an admin-shared knowledge base after the admin changes
the KB embedding model (for example to bge-m3).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 21:01:07 +08:00
Lynn
8d4a3d0dfe Fix: create dataset with chunk_method or pipeline (#13814)
### What problem does this PR solve?

Allow create datasets with parse_type == 1/None and chunk_method, or
parse_type == 2 and pipeline_id.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 20:43:53 +08:00
Lynn
6a4a9debd2 Fix: allow create dataset with resume chunk_method (#13798)
### What problem does this PR solve?

Allow create dataset with resume chunk_method.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 19:06:51 +08:00
balibabu
8402fcac6b Fix: The chunk method of the knowledge base cannot be saved. (#13813)
### What problem does this PR solve?

Fix: The chunk method of the knowledge base cannot be saved.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 19:05:49 +08:00
Syed Shahmeer Ali
ff92b5575b Fix: /file2document/convert blocks event loop on large folders causing 504 timeout (#13784)
Problem

The /file2document/convert endpoint ran all file lookups, document
deletions, and insertions synchronously inside the
request cycle. Linking a large folder (~1.7GB with many files) caused
504 Gateway Timeout because the blocking DB loop
  held the HTTP connection open for too long.

  Fix

- Extracted the heavy DB work into a plain sync function _convert_files
- Inputs are validated and folder file IDs expanded upfront (fast path)
- The blocking work is dispatched to a thread pool via
get_running_loop().run_in_executor() and the endpoint returns 200
  immediately
- Frontend only checks data.code === 0 so the response change
(file2documents list → True) has no impact

  Fixes #13781

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 16:45:10 +08:00
Jin Hai
e705ac6643 Add logout (#13796)
### What problem does this PR solve?

Add command: logout

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-26 11:54:23 +08:00
qinling0210
ebf36950e4 Implement Create/Drop Index/Metadata index in GO (#13791)
### What problem does this PR solve?

Implement Create/Drop Index/Metadata index in GO

New API handling in GO:
POST/kb/index 
DELETE /kb/index
POST /tenant/doc_meta_index
DELETE /tenant/doc_meta_index

CREATE INDEX FOR DATASET 'dataset_name' VECTOR_SIZE 1024;
DROP INDEX FOR DATASET 'dataset_name';
CREATE INDEX DOC_META;
DROP INDEX DOC_META;

### Type of change

- [x] Refactoring
2026-03-26 11:54:10 +08:00
Yongteng Lei
d19ca71b43 Refa: Searches /search API to RESTFul (#13770)
### What problem does this PR solve?

Searches /search API to RESTFul

### Type of change

- [x] Documentation Update
- [x] Refactoring

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-26 01:07:41 +08:00
Yongteng Lei
ea1430bec5 Security: do not use litellm 1.82.7 and 1.82.8 (#13768)
### What problem does this PR solve?

See [issue](https://github.com/BerriAI/litellm/issues/24518) from
Litellm.

Upgraded from `1.81.15` to `1.82.6`, so RAGFlow is safe as always. 

### Type of change

- [x] Security

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-25 22:39:33 +08:00
Jin Hai
61cc5ffef2 Add api tokens commands of go admin cli (#13765)
### What problem does this PR solve?

- GENERATE TOKENS OF USER 'xxx@xxx.com'
- DROP KEY 'ragflow-yyyyy' OF 'xxx@xxx.com'
- LIST KEYS OF 'xxx@xxx.com'

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 21:39:14 +08:00
balibabu
33948b9dd8 Fix: Fix the issue of errors when creating datasets. (#13787)
### What problem does this PR solve?

Fix: Fix the issue of errors when creating datasets.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-25 21:37:58 +08:00
balibabu
8f45398422 Fix: Using AvatarUpload in a dialog and pressing Enter will cause a file selection pop-up to appear. #13779 (#13780)
### What problem does this PR solve?
Fix: Using AvatarUpload in a dialog and pressing Enter will cause a file
selection pop-up to appear. #13779

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 19:02:51 +08:00
Jin Hai
24fcd6bbc7 Update CI (#13774)
### What problem does this PR solve?

CI isn't stable, try to fix it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 18:17:52 +08:00
Idriss Sbaaoui
f3b4d6ab0e Fix: ci fails (#13778)
### What problem does this PR solve?

fix tests failing at p2 and p3

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 17:56:13 +08:00
Liu An
543d164e9b Fix: add build-essential for Python C extension packages (#13772)
### What problem does this PR solve?

The removal of cargo in commit f59d96f87 also removed build-essential
which was needed to compile C extension packages like datrie.

Use aliyun mirror for coverage pip install

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 17:14:48 +08:00
chanx
a2e6daa8d6 Fix: Metadata,chunk,dataset Related bugs (#13760)
### What problem does this PR solve?

Fix: Metadata,chunk,dataset Related bugs
- metadata not show add button #13731
- chunk edit question style
- dataset modified chunk method bug
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 10:47:34 +08:00
Yongteng Lei
1b29522279 Fix: migrate_add_unique_email silently skips unique constraint (#13744)
### What problem does this PR solve?

Fix
migrate_add_unique_email-silently-skips-unique-constraint-when-non-unique-user_email-index-exists.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 20:24:24 +08:00
NeedmeFordev
840cc8fbe9 fix(asana): use project memberships endpoint for project IDs in connector (#13746)
### What problem does this PR solve?

Fixes a bug in the Asana connector where providing `Project IDs` caused
sync to fail with:

`project_membership: Not a recognized ID: <PROJECT_GID>`

Root cause: the connector called `get_project_membership(project_gid)`,
but that API expects a **project membership gid**, not a **project
gid**.
This PR switches to the correct project-scoped API and adds regression
tests.

Fixes: [#13669](https://github.com/infiniflow/ragflow/issues/13669)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes made

- Updated `common/data_source/asana_connector.py`:
- Replaced `get_project_membership(pid, ...)` with
`get_project_memberships_for_project(pid, ...)`
- Trimmed and filtered `asana_project_ids` parsing to avoid
empty/whitespace IDs
  - Normalized `asana_team_id` by trimming whitespace
  - Used safer access for membership email extraction (`m.get("user")`)
- Added `test/unit_test/common/test_asana_connector.py`:
  - Verifies the correct project-membership API method is called
  - Verifies empty `project_ids` path returns workspace emails
  - Verifies project/team input normalization behavior

### Compatibility / risk

- Non-breaking bug fix
- No API contract changes
- Existing behavior for empty `Project IDs` remains unchanged
2026-03-24 20:21:31 +08:00
qinling0210
7c8927c4fb Implement GetChunk() in Infinity in GO (#13758)
### What problem does this PR solve?

Implement GetChunk() in Infinity in GO

Add cli:
GET CHUNK 'XXX';
LIST CHUNKS OF DOCUMENT 'XXX';

### Type of change

- [x] Refactoring
2026-03-24 20:10:21 +08:00
Jin Hai
b308cd3a02 Update go cli (#13717)
### What problem does this PR solve?

Go cli

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 20:08:36 +08:00
balibabu
d84b688b91 Fix: This resolves the issue where selecting a knowledge base in chat could not differentiate between different users. (#13764)
### What problem does this PR solve?

Fix: This resolves the issue where selecting a knowledge base in chat
could not differentiate between different users.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 20:07:06 +08:00
Yongteng Lei
3d10e2075c Refa: files /file API to RESTFul style (#13741)
### What problem does this PR solve?

Files /file API to RESTFul style.

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 19:24:41 +08:00
Idriss Sbaaoui
10a36d6443 Tests : add tests for dataset settings (#13747)
### What problem does this PR solve?

add tests

### Type of change

- [x] Other (please describe): test

Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 19:04:04 +08:00
Yongteng Lei
1b1f1bc69f Fix: minor fix of refacotr excel parser use lazy image loader (#13752)
### What problem does this PR solve?

Minor fix.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Hu Di <812791840@qq.com>
2026-03-24 19:03:54 +08:00
Baki Burak Öğün
8a4da41406 docs: add Turkish README translation (README_tr.md) (#13750)
## Summary
Add a complete Turkish translation of the README and include a Turkish
language badge across all existing README files.

## Changes
- **New file**: `README_tr.md` - Full Turkish translation of README.md,
covering all sections (What is RAGFlow, Demo, Latest Updates, Key
Features, System Architecture, Get Started, Configurations, Docker
Image, Development from Source, Documentation, Roadmap, Community,
Contributing)
- **Updated 9 existing README files** (README.md, README_zh.md,
README_tzh.md, README_ja.md, README_ko.md, README_id.md,
README_pt_br.md, README_fr.md, README_ar.md) to include the Turkish
language badge in the language selector

## Impact
- 10 files changed, 417 insertions
- Follows the same structure and conventions as other language-specific
README files (README_ja.md, README_ko.md, etc.)
- Turkish badge uses the same styling pattern (highlighted with DBEDFA
in README_tr.md, standard DFE0E5 in others)

---------

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
2026-03-24 19:00:48 +08:00
Baki Burak Öğün
1319a25416 feat: complete Turkish localization (#13749)
## Summary
Complete and improve the existing Turkish (tr.ts) localization to fully
match the English (en.ts) reference file.

## Changes
- **Translate 6 English model tips** in the setting section
(chatModelTip, embeddingModelTip, img2txtModelTip, sequence2txtModelTip,
rerankModelTip, ttsModelTip) to Turkish
- **Expand all 13 truncated parser HTML descriptions** (book, laws,
manual, naive, paper, presentation, qa, resume, table, picture, one,
knowledgeGraph, tag) to match the full en.ts structure
- **Expand shortened tooltips** across knowledgeDetails,
knowledgeConfiguration, chat, and setting sections (~40+ tooltips
expanded)
- **Add missing translation details** for data source connectors
(SeaFile, Jira, Gmail, Moodle, Dropbox, Google Drive, etc.)

## Impact
- 182 insertions, 71 deletions in web/src/locales/tr.ts
- No structural changes, only translation content improvements
- All application terminology maintained consistently

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 18:58:58 +08:00
Jin Hai
f59d96f879 Remove rust/cargo install in docker (#13739)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 17:04:57 +08:00
balibabu
48c60b8ce5 Fix: Fixed the issue where agent log time could not be selected. (#13756)
### What problem does this PR solve?
Fix: Fixed the issue where agent log time could not be selected.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 16:02:26 +08:00
Jin Hai
9eb11bf65d Fix ping response (#13757)
### What problem does this PR solve?

As title to be compatible with go server

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 15:15:21 +08:00
Stephen Hu
d32967eda8 refactor: let excel use lazy image loader (#13558)
### What problem does this PR solve?

let excel use lazy image loader

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-23 21:24:40 +08:00
Magicbook1108
f991cd362e Fix: type check in resume parsing method (#13740)
### What problem does this PR solve?

Fix: type check in resume parsing method
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 21:19:09 +08:00
Idriss Sbaaoui
df2cc32f51 Fix: dataset settings save (#13745)
### What problem does this PR solve?

Saving dataset settings failed with validation error 101 (Extra inputs
are not permitted)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 17:46:41 +08:00
qinling0210
ac542da505 Fix tokenizer in cpp (#13735)
### What problem does this PR solve?

Tokenzier in Infinity is modified in
https://github.com/infiniflow/infinity/pull/3330, sync the code change
to cpp files in ragflow

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 15:40:35 +08:00
qinling0210
7b86f577be Implement metadata search in Infinity in GO (#13706)
### What problem does this PR solve?

Add cli

LIST DOCUMENTS OF DATASET quoted_string ";"
LIST METADATA OF DATASETS quoted_string ("," quoted_string)* ";"
LIST METADATA SUMMARY OF DATASET quoted_string (DOCUMENTS quoted_string
("," quoted_string)*)? ";"

### Type of change

- [x] Refactoring
2026-03-21 18:10:00 +08:00
Lynn
db57155b30 Fix: get user_id from variables (#13716)
### What problem does this PR solve?

Get user_id from canvas variable when input a {} pattern value.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-20 23:39:34 +08:00
Yongteng Lei
dd839f30e8 Fix: code supports matplotlib (#13724)
### What problem does this PR solve?

Code as "final" node: 

![img_v3_02vs_aece4caf-8403-4939-9e68-9845a22c2cfg](https://github.com/user-attachments/assets/9d87b8df-da6b-401c-bf6d-8b807fe92c22)

Code as "mid" node:

![img_v3_02vv_f74f331f-d755-44ab-a18c-96fff8cbd34g](https://github.com/user-attachments/assets/c94ef3f9-2a6c-47cb-9d2b-19703d2752e4)


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-20 20:32:00 +08:00
balibabu
0507463f4e Fix: The retrieval_test interface is continuously requested when the user enters a question. #13719 (#13720)
### What problem does this PR solve?

Fix: The retrieval_test interface is continuously requested when the
user enters a question. #13719

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-20 15:46:41 +08:00
Jin Hai
9ce766192f Init storage engine (#13707)
### What problem does this PR solve?

1. Init Minio / S3 / OSS
2. Fix minio / s3 / oss config

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-20 13:15:41 +08:00
Jin Hai
04a60a41e0 Allow default admin user login ragflow user of go server (#13715)
### What problem does this PR solve?

1. Allow admin@ragflow.io login go ragflow server
2. Fix go server start error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-20 12:02:44 +08:00
tmimmanuel
13d0df1562 feat: add Perplexity contextualized embeddings API as a new model provider (#13709)
### What problem does this PR solve?

Adds Perplexity contextualized embeddings API as a new model provider,
as requested in #13610.

- `PerplexityEmbed` provider in `rag/llm/embedding_model.py` supporting
both standard (`/v1/embeddings`) and contextualized
(`/v1/contextualizedembeddings`) endpoints
- All 4 Perplexity embedding models registered in
`conf/llm_factories.json`: `pplx-embed-v1-0.6b`, `pplx-embed-v1-4b`,
`pplx-embed-context-v1-0.6b`, `pplx-embed-context-v1-4b`
- Frontend entries (enum, icon mapping, API key URL) in
`web/src/constants/llm.ts`
- Updated `docs/guides/models/supported_models.mdx`
- 22 unit tests in `test/unit_test/rag/llm/test_perplexity_embed.py`

Perplexity's API returns `base64_int8` encoded embeddings (not
OpenAI-compatible), so this uses a custom `requests`-based
implementation. Contextualized vs standard model is auto-detected from
the model name.

Closes #13610

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-03-20 10:47:48 +08:00
Zhicheng Wu
456b1bbf66 fix: row selection leaks across pages in dataset and file list tables (#13668)
### What problem does this PR solve?

When using pagination in the Dataset file list or File Manager,
selecting row N on page 1 would incorrectly cause row N on page 2 (and
subsequent pages) to also appear selected. This is a state pollution
bug.

### Root Cause

TanStack React Table defaults to using array indices (0, 1, 2...) as
`rowSelection` keys. With server-side (manual) pagination, each page's
rows start from index 0, so a selection like `{2: true}` on page 1 also
matches index 2 on every other page.

### Fix

- Added `getRowId: (row) => row.id` to `useReactTable` in both
`DatasetTable` and `FilesTable`, so selection state is keyed by unique
document/file IDs instead of positional indices.
- Updated the `useSelectedIds` helper to support ID-based selection keys
while maintaining backward compatibility with index-based keys.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Files Changed

| File | Change |
|------|--------|
| `web/src/pages/dataset/dataset/dataset-table.tsx` | Added `getRowId`
to table config |
| `web/src/pages/files/files-table.tsx` | Added `getRowId` to table
config |
| `web/src/hooks/logic-hooks/use-row-selection.ts` | Updated
`useSelectedIds` to handle ID-based selection |
2026-03-19 21:08:09 +08:00
chanx
e1dbfb8a9c fix(dao): Remove unnecessary status filter conditions in user queries (#13698)
### What problem does this PR solve?

Fix: Enhanced the user deletion function to return detailed deletion
information.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 21:05:15 +08:00
Magicbook1108
cfe6ea6f56 Feat: CREATE / DELETE / LIST dataset api in Go (#13695)
### What problem does this PR solve?

Feat: CREATE / DELETE / LIST dataset api in Go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 20:48:32 +08:00
Lynn
f06e332c44 Fix: allow on (#13704)
### What problem does this PR solve?

Allow input on/ON as status.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 20:41:02 +08:00
writinwaters
b5e0b37d69 Refact: Renamed 'Agent flow' to 'Workflow' (#13705)
### What problem does this PR solve?

'Agent flow' rebranded.

### Type of change

- [x] Refactoring
2026-03-19 20:17:25 +08:00
Jin Hai
8d50ee632d Add environments reading (#13701)
### What problem does this PR solve?

environment variable > config file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-19 18:50:28 +08:00
yH
757d8d42dd Fix: use configured OrderByExpr in _community_retrieval_ (#13683)
The `odr` variable was configured with `desc("weight_flt")` but a new
empty `OrderByExpr()` was passed to `dataStore.search()` instead,
causing the descending sort to have no effect.

### What problem does this PR solve?

In `_community_retrieval_`, the configured `OrderByExpr` with
`desc("weight_flt")` was discarded — a new empty `OrderByExpr()` was
passed to `dataStore.search()` instead, so community reports were never
sorted by weight.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 17:55:40 +08:00
Lynn
e12147f5b9 Fix: admin client (#13699)
### What problem does this PR solve?

Define a crypt function in admin directory, remove import from
api.utils. And move requests-toolbelt to dependency.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 17:06:54 +08:00
Lynn
4bb1acaa5b Refactor: dataset / kb API to RESTFul style (#13690)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 14:41:36 +08:00
Idriss Sbaaoui
7827f0fce5 fix : empty mind map (#13693)
### What problem does this PR solve?

Fix graphrag extractor chat response parsing and skip truncated cache
values

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 13:53:06 +08:00
Jin Hai
7ebe1d2722 Fix docker building (#13681)
### What problem does this PR solve?

1. Refactor go server log
2. Update docker building, since nginx config should be set according to
the deployment.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-19 10:25:35 +08:00
NeedmeFordev
c3f79dbcb0 fix(jira): prevent missed incremental updates after issue edits (#13674)
### What problem does this PR solve?

Fixes [#13505](https://github.com/infiniflow/ragflow/issues/13505): Jira
incremental sync could miss updated issues after initial sync,
especially near time boundaries.

Root cause:
- Jira JQL uses minute-level precision for `updated` filters.
- Incremental windows had no overlap buffer, so boundary updates could
be skipped.
- Sync log cursor tracking used a backward-facing update for
`poll_range_start`.
- Existing-doc updates in `upload_document` lacked a KB ownership guard
for doc-id collisions.

What changed:
- Added Jira incremental overlap buffer (`time_buffer_seconds`,
defaulting to `JIRA_SYNC_TIME_BUFFER_SECONDS`) when building JQL
lower-bound time.
- Preserved second-level post-filtering to avoid duplicate reprocessing
while still catching boundary updates.
- Improved Jira sync logging to include start/end window and overlap
configuration.
- Updated sync cursor tracking in `increase_docs` to keep
`poll_range_start` moving forward with max update time.
- Added KB ID safety check before updating existing document records in
`upload_document`.

Verification performed:
- Python syntax compile checks passed for modified files.
- Manual verification flow:
  1. Run full Jira sync.
  2. Edit an already-indexed Jira issue.
  3. Run next incremental sync.
  4. Confirm updated content is re-ingested into KB.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 23:31:05 +08:00
Daniil Sivak
dee68c571b Feat: support variable interpolation in headers (#13680)
Closes #13277

### What problem does this PR solve?

Adds `{variable_name}` (and `{component@variable}`) interpolation
support to HTTP header values in the `Invoke` component, matching the
existing URL interpolation behavior.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

<img width="1280" height="867" alt="image"
src="https://github.com/user-attachments/assets/8ab7b4e9-7cc0-4a7f-8a5f-f838a15a5fda"
/>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 22:38:20 +08:00
Mustafa YILDIZ
e4d8cdaff3 feat: add Turkish language support (#13670)
### What problem does this PR solve?
RAGFlow had no Turkish language support. This PR adds Turkish (tr)
locale translations to the UI.

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?

Co-authored-by: Mustafa YILDIZ <mustafa.yildiz@cilek.com>
2026-03-18 21:09:32 +08:00
writinwaters
bbd0cd80e4 Docs: Updated Add Google Drive as data source (#13684)
### What problem does this PR solve?

Gave an editorial pass to the Add Google Drive document.

### Type of change

- [x] Documentation Update
2026-03-18 21:05:25 +08:00
Octopus
f171554c0a feat: upgrade MiniMax default model to M2.7 (#13676)
## Summary
Upgrade MiniMax model configuration to include the latest M2.7 model.

## Changes
- Add `MiniMax-M2.7` and `MiniMax-M2.7-highspeed` to the model selection
list in `conf/llm_factories.json`
- Place M2.7 models at the top of the list as the recommended default
- Retain all previous models (M2.5, M2.5-highspeed, M2.1, M2) as
available alternatives

## Why
MiniMax-M2.7 is the latest flagship model with enhanced reasoning and
coding capabilities. This update ensures RAGFlow users can access the
newest model while maintaining backward compatibility with existing
configurations.

## Testing
- JSON config validated (well-formed)
- No existing MiniMax-specific unit tests affected
- Model entries follow the same structure as existing entries

Co-authored-by: PR Bot <pr-bot@minimaxi.com>
2026-03-18 19:20:10 +08:00
Idriss Sbaaoui
9070408b04 Fix : model-specific handling (#13675)
### What problem does this PR solve?

add a handler for gpt 5 models that do not accept parameters by dropping
them, and centralize all models with specific paramter handling function
into a single helper.
solves issue #13639 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-03-18 17:28:20 +08:00
Yongteng Lei
53e395ca2e Fix: cannot debug invoke component (#13649)
### What problem does this PR solve?

Cannot debug invoke component.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-18 14:22:13 +08:00
Jin Hai
74866371ef Fix compatiblity issue (#13667)
### What problem does this PR solve?

1. Change go admin server port from 9385 to 9383 to avoid conflicts
2. Start go server after python servers are started completely, in
entrypoint.sh
3. Fix some database migration issue
4. Add more API routes in web to compliant with EE.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-18 11:51:03 +08:00
Daniil Sivak
60ad32a0c2 Feat: support epub parsing (#13650)
Closes #1398

### What problem does this PR solve?

Adds native support for EPUB files. EPUB content is extracted in spine
(reading) order and parsed using the existing HTML parser. No new
dependencies required.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

To check this parser manually:

```python
uv run --python 3.12 python -c "
from deepdoc.parser import EpubParser

with open('$HOME/some_epub_book.epub', 'rb') as f:
  data = f.read()

sections = EpubParser()(None, binary=data, chunk_token_num=512)
print(f'Got {len(sections)} sections')
for i, s in enumerate(sections[:5]):
  print(f'\n--- Section {i} ---')
  print(s[:200])
"
```
2026-03-17 20:14:06 +08:00
Idriss Sbaaoui
1399c60164 fix builtin model fail when parsing (#13657)
### What problem does this PR solve?

using builtin model when parsing gave an error because it expects
fid==builtin. split_model_name_and_factory returns id=None. pr allows
the model to be accepted wheter with or without @Builtin

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 19:38:54 +08:00
balibabu
6cae364ac2 Feat: Export Agent Logs. (#13658)
### What problem does this PR solve?
Feat: Export Agent Logs.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-17 18:51:26 +08:00
balibabu
fc4f1e2488 Fix: The dataset description should not be a required field. (#13655)
### What problem does this PR solve?

Fix: The dataset description should not be a required field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 18:51:18 +08:00
Idriss Sbaaoui
ad6bdb5bfe Fix: left preview containment regression for file previews (#13652)
### What problem does this PR solve?

Fix left preview containment regression for file previews

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 17:21:13 +08:00
Yongteng Lei
ca6c3218c3 Refa: follow-up expose agent structured outputs in non-stream completions (#13524)
### What problem does this PR solve?

Follow-up expose agent structured outputs in non-stream completions
#13389.

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-17 17:11:27 +08:00
qinling0210
ca182dc188 Implement Search() in Infinity in GO (#13645)
### What problem does this PR solve?

Implement Search() in Infinity in GO.

The function can handle the following request. 
"search '曹操' on datasets 'infinity'" 
"search '常胜将军' on datasets 'infinity'"
"search '卓越儒雅' on datasets 'infinity'"
"search '辅佐刘禅北伐中原' on datasets 'infinity'"

The output is exactly the same as  request to python Search()

### Type of change

- [ ] New Feature (non-breaking change which adds functionality)
2026-03-17 16:45:45 +08:00
balibabu
549833b8a4 Fix: Fixed an issue where agent template titles were not displayed in Chinese mode. (#13647)
### What problem does this PR solve?

Fix: Fixed an issue where agent template titles were not displayed in
Chinese mode.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 15:56:57 +08:00
Stephen Hu
77483b1e58 refactor: remove useless variable in raptor (#13648)
### What problem does this PR solve?

remove useless variable in raptor

### Type of change


- [x] Refactoring
2026-03-17 15:56:51 +08:00
Jin Hai
986dcf1cc8 Revert "Refactor: dataset / kb API to RESTFul style" (#13646)
Reverts infiniflow/ragflow#13619
2026-03-17 12:09:48 +08:00
balibabu
fdf2d84ffc Fix: Fixed an issue where the agent could not publish. (#13644)
### What problem does this PR solve?

Fix: Fixed an issue where the agent could not publish.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 11:44:01 +08:00
Lynn
1db5409d82 Refactor: dataset / kb API to RESTFul style (#13619)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-16 22:51:34 +08:00
Yingfeng
73bc9b91de Limit max recursion depth for rag analyzer#3318 (#13637)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 22:49:56 +08:00
chanx
5403f142ae Feat: Add chunk also supports uploading image. (#13628)
### What problem does this PR solve?

Feat: Add chunk also supports uploading image.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-16 20:15:49 +08:00
Yongteng Lei
af7e24ba8c Feat: add_chunk supports add image (#13629)
### What problem does this PR solve?

Add_chunk supports add image.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 20:15:36 +08:00
Magicbook1108
09ff1bc2b0 Fix: paddle ocr coordinate lower > upper (#13630)
### What problem does this PR solve?

Fix: paddle ocr coordinate lower > upper #13618 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 20:15:26 +08:00
Jin Hai
0545801251 Update CI process (#13632)
### What problem does this PR solve?

This pull request updates the GitHub Actions workflow for testing,
primarily to simplify Docker Compose usage and environment file
management. The main changes focus on removing unnecessary subdirectory
references, updating environment file handling, and streamlining the
workflow steps.


### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-16 19:00:28 +08:00
Idriss Sbaaoui
d5ed179d15 Playwright : add test ids and chat test (#13432)
### What problem does this PR solve?


### Type of change

- [x] Other
2026-03-16 16:39:05 +08:00
balibabu
f4d126acb0 Fix: Shared chat link triggers infinite POST loop with empty question, input disabled #13606 (#13625)
### What problem does this PR solve?

Fix: Shared chat link triggers infinite POST loop with empty question,
input disabled #13606

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 15:43:18 +08:00
balibabu
fa48ffe5de Feat: Translate embedded dialog text. (#13623)
### What problem does this PR solve?

Feat: Translate embedded dialog text.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 15:43:13 +08:00
Idriss Sbaaoui
c98ad2f0d8 fix multimodel input bars slowly being pushed offscreen (#13620)
### What problem does this PR solve?

when the conversation starts to get long on multimodel chat, the
conversation pushes the input bar offscreem

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 15:03:50 +08:00
Idriss Sbaaoui
bb962e67b0 fix : build error (#13622)
### What problem does this PR solve?

add timeout to fix fail at build during uvsync step

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 15:03:25 +08:00
Yingfeng
b686a60713 Switch from demo.ragflow.io to cloud.ragflow.io (#13624)
### What problem does this PR solve?

Switch from demo.ragflow.io to cloud.ragflow.io

### Type of change

- [x] Documentation Update
2026-03-16 14:44:39 +08:00
Liu An
5b3bb25010 Fix: switch Python package mirror from Tsinghua to Aliyun (#13617)
### What problem does this PR solve?

Replace pypi.tuna.tsinghua.edu.cn with mirrors.aliyun.com to resolve
issues with missing packages on the Tsinghua mirror.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 12:12:25 +08:00
Jin Hai
a2d72202cf Revert "Refactor dataset / kb API to RESTFul style" (#13614)
Reverts infiniflow/ragflow#13263
2026-03-16 10:44:38 +08:00
Ram Mourya
ae9b1c7f6a Docs : Fixed the links for user and developer guide in readme files (#13609)
Fixed the links for user and developer guide in readme files.
2026-03-16 10:23:52 +08:00
Yongteng Lei
287637162c Revert "fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416" (#13613)
Reverts infiniflow/ragflow#13583 which cause uv sync fails.
2026-03-16 10:19:29 +08:00
Lynn
7c32e206be Refactor dataset / kb API to RESTFul style (#13263)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-13 20:02:35 +08:00
apps-lycusinc
8b984c9d5f Fixing WordNetCorpusReader object has no attribute _LazyCorpusLoader_… (#13600)
### What problem does this PR solve?

Forces NLTK to load the corpus synchronously once, preventing concurrent
tasks from triggering the lazy-loading race condition that cause Fixing
WordNetCorpusReader object has no attribute _LazyCorpusLoader_… #13590


### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: shakeel <shakeel@lollylaw.com>
2026-03-13 19:55:01 +08:00
Magicbook1108
161659becc Fix: model selecton rule in get_model_config_by_type_and_name (#13569)
### What problem does this PR solve?

Fix: model selecton rule in get_model_config_by_type_and_name

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 19:46:13 +08:00
balibabu
cb49cd30c4 Feat: Add the user_id field to the agent log table and the embedded page. (#13596)
### What problem does this PR solve?

Feat: Add the `user_id` field to the agent log table and the embedded
page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-13 19:06:18 +08:00
Jin Hai
cc7e94ffb6 Use different API route according the ENV (#13597)
### What problem does this PR solve?

1. Fix go server date precision
2. Use API_SCHEME_PROXY to control the web API route

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-13 19:05:30 +08:00
balibabu
1569ed82f8 Refactor: Delete flow.ts (#13498)
### What problem does this PR solve?

Feat: Delete flow.ts

### Type of change


- [x] Refactoring
2026-03-13 18:04:54 +08:00
chanx
a3e6c2e84a Fix: Enhanced user management functionality and cascading data deletion. (#13594)
### What problem does this PR solve?
Fix: Enhanced user management functionality and cascading data deletion.

Added tenant and related data initialization functionality during user
creation, including tenants, user-tenant relationships, LLM
configuration, and root folder.
Added cascading deletion logic for user deletion, ensuring that all
associated data is cleaned up simultaneously when a user is deleted.
Implemented a Werkzeug-compatible password hash algorithm (scrypt) and
verification functionality.
Added multiple DAO methods to support batch data operations and
cascading deletion.
Improved user login processing and added token signing functionality.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 16:53:54 +08:00
Sank
a67fa03584 fix CVE-2026-28804 CVE-2026-31826 (#13592)
What problem does this PR solve?

fix CVE-2026-28804 CVE-2026-31826

 Bug Fix (non-breaking change which fixes an issue)

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 16:34:28 +08:00
balibabu
717f1f1362 Feat: Modify the style of the release confirmation box. (#13542)
### What problem does this PR solve?

Feat: Modify the style of the release confirmation box.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: balibabu <assassin_cike@163.com>
Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
2026-03-13 16:31:17 +08:00
Lynn
02070bab2a Feat: record user_id in memory (#13585)
### What problem does this PR solve?

Get user_id from canvas and record it.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-13 15:38:35 +08:00
Jin Hai
5c955a31cc Update go server (#13589)
### What problem does this PR solve?

1. Add more CLI command
2. Add some license hooks

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-13 14:41:02 +08:00
Idriss Sbaaoui
ef94a9c291 Fix : remove min value for description field (#13587)
### What problem does this PR solve?

min value and message force users to input a descript in datasets. Also
had a wrong error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 13:11:54 +08:00
Ethan T.
71804bf5bc fix(db_models): guard MySQL-specific SQL in migration with DB_TYPE check (fixes #13544) (#13582)
## Summary

Fixes #13544: PostgreSQL startup crash because
`update_tenant_llm_to_id_primary_key()` unconditionally uses
MySQL-specific SQL.

- Split `update_tenant_llm_to_id_primary_key()` into
`_update_tenant_llm_to_id_primary_key_mysql()` and
`_update_tenant_llm_to_id_primary_key_postgres()`, dispatching on
`settings.DATABASE_TYPE`
- MySQL path: unchanged (existing `DATABASE()`, `SET @row = 0`,
`AUTO_INCREMENT`, `DROP PRIMARY KEY` logic)
- PostgreSQL path: uses `current_database()`, `ROW_NUMBER() OVER (ORDER
BY ...)` for sequential IDs, `CREATE SEQUENCE` + `nextval()` for
auto-increment, and `information_schema.table_constraints` to find the
PK constraint name
- Also fix `migrate_add_unique_email()`: MySQL-only
`information_schema.statistics` is replaced with `pg_indexes` on
PostgreSQL

## Test plan

- [ ] Start RAGFlow with `DB_TYPE=postgres` — startup should complete
without `function database() does not exist` error
- [ ] Start RAGFlow with `DB_TYPE=mysql` (default) — existing behaviour
unchanged, migration runs as before
- [ ] Fresh PostgreSQL install: verify `tenant_llm.id` column is created
as a serial primary key after migration
- [ ] Idempotency: running migration twice on PostgreSQL should be a
no-op (column already exists check passes)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: gambletan <gambletan@github>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 11:53:01 +08:00
Sank
e90f0e8910 fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416 (#13583)
### What problem does this PR solve?

fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 11:17:39 +08:00
Liu An
667f9c1c3a fix: remove duplicate "arabic" key in French translations (#13529)
### What problem does this PR solve?

Removed duplicate key that caused build warning during Vite build.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 10:57:19 +08:00
Idriss Sbaaoui
810692dfa3 fix: restore cross_languages default chat-model fallback for retrieval (#13471)
### What problem does this PR solve?

issue #13465 
POST /api/v1/retrieval failed with
{"code":100,...,"message":"Exception('Model Name is required')"} when
cross_languages was provided and no explicit llm_id was passed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 10:52:37 +08:00
Jimmy Ben Klieve
1a4dee4313 refactor(ui): unify top level pages structure, use standard language codes and time zones (#13573)
### What problem does this PR solve?

- Unify top level pages structure
- Standardize locale language codes (BCP 47) and time zones (IANA tz)


> **Note:** 
> Newly created user info brings non-standard default values `timezone:
"UTC+8\tAsia/Shanghai"` and `language: "English"`.


### Type of change

- [x] Refactoring
2026-03-12 21:01:09 +08:00
Ethan Clarke
35cd56f990 feat: add MiniMax-M2.5 and M2.5-highspeed models (#13557)
## Summary

Add MiniMax's latest M2.5 model family to the model registry and update
the default API base URL to the international endpoint for broader
accessibility.

## Changes

- **Add MiniMax-M2.5 models** to `conf/llm_factories.json`:
- `MiniMax-M2.5` — Peak Performance. Ultimate Value. Master the Complex.
  - `MiniMax-M2.5-highspeed` — Same performance, faster and more agile.
- Both support 204,800 token context window and tool calling (`is_tools:
true`).
- **Update default MiniMax API base URL** in `rag/llm/__init__.py`:
- From `https://api.minimaxi.com/v1` (domestic) to
`https://api.minimax.io/v1` (international).
- Chinese users can still override via the Base URL field in the UI
settings (as documented in existing i18n strings).

## Supported Models

| Model | Context Window | Tool Calling | Description |
|-------|---------------|-------------|-------------|
| `MiniMax-M2.5` | 204,800 tokens | Yes | Peak Performance. Ultimate
Value. |
| `MiniMax-M2.5-highspeed` | 204,800 tokens | Yes | Same performance,
faster and more agile. |

## API Documentation

- OpenAI Compatible API:
https://platform.minimax.io/docs/api-reference/text-openai-api

## Testing

- [x] JSON validation passes
- [x] Python syntax validation passes
- [x] Ruff lint passes
- [x] MiniMax-M2.5 API call verified (returns valid response)
- [x] MiniMax-M2.5-highspeed API call verified (returns valid response)

Co-authored-by: PR Bot <pr-bot@minimaxi.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 20:41:46 +08:00
Jin Hai
3fbf8bc3d4 Expose go version server and admin server port out of docker in CI (#13572)
### What problem does this PR solve?

- Print Go version log when start server
- Expose the server port in CI docker container

### Type of change

- [x] Other (please describe): For CI

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:39:57 +08:00
Jin Hai
d688b72dff Go: Add admin server status checking (#13571)
### What problem does this PR solve?

RAGFlow server isn't available when admin server isn't connected.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:02:50 +08:00
chanx
1df804a14a Feature (System Settings): Implemented system settings management functionality (#13556)
### What problem does this PR solve?

Feature (System Settings): Implemented system settings management
functionality

- Added a new SystemSettings model, including creation and update time
fields.

- Implemented SystemSettingsDAO, providing CRUD operations and
transaction support.

- Implemented management interfaces for variables, configurations, and
environment variables in the admin service.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 19:06:20 +08:00
guptas6est
7c79602c77 fix(web): upgrade lodash to 4.17.23 and dompurify to 3.3.2 to fix CVE-2026-0540 and CVE-2025-13465 (#13488)
### What problem does this PR solve?

This PR fixes two security vulnerabilities in web dependencies
identified by Trivy:

1. CVE-2025-13465 (lodash): Prototype pollution vulnerability in _.unset
and _.omit functions
2. CVE-2026-0540 (dompurify): Cross-site scripting (XSS) vulnerability

**Changes:**
- Upgraded lodash from 4.17.21 to 4.17.23
- Upgraded dompurify from 3.3.1 to 3.3.2
- Added npm override to force monaco-editor's transitive dependency on
dompurify to use 3.3.2 (monaco-editor still depends on vulnerable 3.2.7)

Both upgrades are backward-compatible patch versions. Build verified
successfully with no breaking changes.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 19:04:26 +08:00
Ray Zhang
375f62a6c3 docs(migration): add project name (-p) usage to backup & migration guide (#13565)
## Summary

- Add documentation for the `-p project_name` flag in the migration
script, covering all steps (stop, backup, restore, start)
- Add a note explaining how Docker volume name prefixes relate to the
Compose project name
- Update `docker-compose` to `docker compose` (Compose V2 syntax) for
consistency
- Fix `sh` to `bash` to match the script's shebang line

This is the documentation follow-up to #12187 which added `-p` project
name support to `docker/migration.sh`.

## Test plan

- [ ] Verify the documentation renders correctly on the docs site
- [ ] Confirm all example commands are accurate against the current
`migration.sh`
2026-03-12 19:01:25 +08:00
qinling0210
1be07a0a34 Fix "Result window is too large" during meta data search (#13521)
### What problem does this PR solve?

Fix
https://github.com/infiniflow/ragflow/issues/13210#issuecomment-3982878498

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 18:59:56 +08:00
Jin Hai
cebf5892ec Create go version storage component, but not used (#13561)
### What problem does this PR solve?

Implement: minio, s3, oss, azure_sas, azure_spn, gcs, opendal

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 18:58:25 +08:00
Jinghan Xu
f6b06fab72 Fix: allow document parsing status recovery after transient errors (#13341)
### What problem does this PR solve?

Fixes #13285

When an LLM returns a transient error (e.g. overloaded) during parsing,
the task progress is set to -1. Previously, the progress could never be
updated again, leaving the document permanently stuck in FAIL status
even after the task successfully recovered and completed.

Three coordinated changes address this:

1. task_service.update_progress: relax the progress update guard to
accept prog >= 1 even when current progress is -1, so a task that
recovers from a transient failure can report completion.

2. document_service.get_unfinished_docs: include documents that are
marked FAIL (progress == -1) but still have at least one non-failed task
(task.progress >= 0) in the polling set, so their status can be
re-synced once a task recovers. Documents where all tasks have
permanently failed are excluded to avoid unnecessary polling.

3. document_service.update_progress: explicitly set document status to
RUNNING when not all tasks have finished, instead of preserving whatever
stale status (potentially FAIL) the document previously had.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 18:02:12 +08:00
Yongteng Lei
13a34d7689 Feat: inject sys.date into canvas (#13567)
### What problem does this PR solve?

Inject sys.date into canvas.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-12 17:49:13 +08:00
Magicbook1108
eda7835d47 Fix: image pdf in ingestion pipeline (#13563)
### What problem does this PR solve?

Fix: image pdf in ingestion pipeline #13550


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 17:49:02 +08:00
NeedmeFordev
387b0b27c4 feat(parser): support external Docling server via DOCLING_SERVER_URL (#13527)
### What problem does this PR solve?

This PR adds support for parsing PDFs through an external Docling
server, so RAGFlow can connect to remote `docling serve` deployments
instead of relying only on local in-process Docling.

It addresses the feature request in
[#13426](https://github.com/infiniflow/ragflow/issues/13426) and aligns
with the external-server usage pattern already used by MinerU.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### What is changed?

- Add external Docling server support in `DoclingParser`:
  - Use `DOCLING_SERVER_URL` to enable remote parsing mode.
- Try `POST /v1/convert/source` first, and fallback to
`/v1alpha/convert/source`.
- Keep existing local Docling behavior when `DOCLING_SERVER_URL` is not
set.
- Wire Docling env settings into parser invocation paths:
  - `rag/app/naive.py`
  - `rag/flow/parser/parser.py`
- Add Docling env hints in constants and update docs:
  - `docs/guides/dataset/select_pdf_parser.md`
  - `docs/guides/agent/agent_component_reference/parser.md`
  - `docs/faq.mdx`

### Why this approach?

This keeps the change focused on one issue and one capability (external
Docling connectivity), without introducing unrelated provider-model
plumbing.

### Validation

- Static checks:
  - `python -m py_compile` on changed Python files
  - `python -m ruff check` on changed Python files
- Functional checks:
  - Remote v1 endpoint path works
  - v1alpha fallback works
  - Local Docling path remains available when server URL is unset

### Related links

- Feature request: [Support external Docling server (issue
#13426)](https://github.com/infiniflow/ragflow/issues/13426)
- Compare view for this branch:
[main...feat/docling-server](https://github.com/infiniflow/ragflow/compare/main...spider-yamet:ragflow:feat/docling-server?expand=1)

##### Fixes [#13426](https://github.com/infiniflow/ragflow/issues/13426)
2026-03-12 17:09:03 +08:00
Josh
a353c7bdd7 Fix: avoid empty doc filter in knowledge retrieval (#13484)
## Summary
Fix knowledge-base chat retrieval when no individual document IDs are
selected.

## Root Cause
`async_chat()` initialized `doc_ids` as an empty list when the request
did not explicitly select documents. That empty list was then forwarded
into retrieval as an active `doc_id` filter, effectively becoming
`doc_id IN []` and suppressing all chunk matches.

## Changes
- treat missing selected document IDs as `None` instead of `[]`
- keep explicit document filtering when IDs are actually provided
- add regression coverage for the shared chat retrieval path

## Validation
- `python3 -m py_compile api/db/services/dialog_service.py
test/unit_test/api/db/services/test_dialog_service_use_sql_source_columns.py`
- `.venv/bin/python -m pytest
test/unit_test/api/db/services/test_dialog_service_use_sql_source_columns.py`
- manually verified that chat completions again inject retrieved
knowledge into the prompt

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 16:03:30 +08:00
cambrianlee
227c852e67 Fix typo: documnet_keyword -> document_keyword in Chunk class (#13531)
### What problem does this PR solve?
The Chunk class had a typo in the attribute name 'documnet_keyword',
which caused the document_name field to remain empty when retrieving
chunks via the SDK. This fix corrects the spelling to
'document_keyword'.

Changes:
- Line 36: Changed self.documnet_keyword to self.document_keyword
- Line 52: Updated backward compatibility code to use
self.document_keyword


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 15:23:55 +08:00
Jin Hai
e78938c72c Update go admin server default port to 9383 (#13559)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 13:41:08 +08:00
Jimmy Ben Klieve
31a8184f63 refactor(ui): update ui for user settings, etc. (#13532)
### What problem does this PR solve?

Update UI styles:
- **User settings**
- Component styles: 
   - `ui/button.tsx`
   - `ui/checkbox.tsx`
   - `avatar-upload.tsx`
   - `file-uploader.tsx`
   - `icon-font.tsx`

### Type of change

- [x] Refactoring
2026-03-12 13:33:36 +08:00
chanx
0da9c4618d feat(cli): Enhance CLI functionality and add administrator mode support (#13539)
### What problem does this PR solve?

feat(cli): Enhance CLI functionality and add administrator mode support

- Modify `parseActivateUser` in `parser.go` to support 'on'/'off' states
- Add administrator mode switching and host port settings functionality
to `cli.go`
- Implement user management API calls in `client.go`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-12 13:33:13 +08:00
chanx
4bd5bb141d Fix: data-source-detail page style (#13507)
### What problem does this PR solve?

Fix: data-source-detail page style

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 13:32:39 +08:00
Jin Hai
5cbdfc5f17 Fix Gitee embedding model URL error (#13553)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 13:13:06 +08:00
Yongteng Lei
375a910bcf Fix: add deadlock retry (#13552)
### What problem does this PR solve?

 Add deadlock retry.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 12:39:01 +08:00
Jin Hai
90afce192c Add license and fingerprint API hook (#13548)
### What problem does this PR solve?

For EE

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 11:52:39 +08:00
Jin Hai
2fb1360d9d Add command line parameter and fix error message (#13526)
### What problem does this PR solve?

`./server_main -p 9380`

`./server_main -h`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 09:50:57 +08:00
Yongteng Lei
e1b632a7bb Feat: add delete all support for delete operations (#13530)
### What problem does this PR solve?

Add delete all support for delete operations.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-12 09:47:42 +08:00
qinling0210
d201a81db7 Add command history in ragflow cli (#13538)
### What problem does this PR solve?

In ragflow cli,  use Up/Down arrows to navigate command history,

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-11 19:14:18 +08:00
Liu An
852393c114 Test: Lower priority of chat assistant and chunk list API tests (#13540)
### What problem does this PR solve?

Mark test cases as lower priority (p3) for:
- Creating chat assistants
- Deleting chat assistants
- Listing chat assistants
- Listing chunks within datasets

### Type of change

- [x] Update testcases
2026-03-11 19:00:18 +08:00
foyou
f75dc6a452 Docs: Fix normalization of case and some code blocks (#13520)
### What problem does this PR solve?

Standardize term capitalization in `deploy_local_llm.mdx` and improve
code block formatting.

### Type of change

- [x] Documentation Update
2026-03-11 17:51:13 +08:00
Ethan T.
1cee8b1a7b fix: use context managers for file handles to prevent resource leaks (#13514)
## Summary
- Convert bare `open()` calls to `with` context managers or
`Path.read_text()`
- File handles leak if not properly closed, especially on exceptions
- Fixes in crypt.py, sequence2txt_model.py, term_weight.py,
deepdoc/vision/__init__.py

## Test plan
- [x] File operations work correctly with context managers
- [x] Resources properly cleaned up on exceptions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:47:06 +08:00
Attili-sys
6afd13ff29 Feat/arabic language support (#13516)
### What problem does this PR solve?

This PR implements comprehensive Arabic language support for the RAGFlow
application. The changes include:
- Complete Arabic translation of all UI text elements in the web
interface
- RTL (right-to-left) layout support for Arabic content
- Localization updates for all supported languages (ar, bg, de, en, es,
fr, id, it, ja, pt-br, ru, vi, zh-traditional, zh)
- UI component adjustments to properly display Arabic text and support
RTL layout

The implementation ensures that Arabic-speaking users can fully interact
with the application in their native language with proper text rendering
and layout direction.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

<img width="2866" height="1617" alt="image"
src="https://github.com/user-attachments/assets/f2751b34-1b65-4867-b81d-a1068c17b9b7"
/>

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-11 15:06:07 +08:00
chanx
9ca2bac984 Feat: Implement user creation, deletion, and permission management functionality. (#13519)
### What problem does this PR solve?

Feat: Implement user creation, deletion, and permission management
functionality.

- Added the `ListByEmail` method to `user.go` to query users by email
address.

- Updated the user activation status handling logic in `handler.go`,
adding input validation.

- Added RSA password decryption functionality to `password.go`.

- Implemented complete user management functionality in `service.go`,
including user creation, deletion, password modification, activation
status, and permission management.

- Added input validation and error handling logic.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-11 14:04:00 +08:00
Jin Hai
2028e895fd Add license and time record DAO (#13522)
### What problem does this PR solve?

1. Change go server default port to 9382
2. Compatible with EE data model.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-11 14:02:24 +08:00
qinling0210
1815f5950b Call get_flatted_meta_by_kbs in dify retrieval (#13509)
### What problem does this PR solve?

Fix https://github.com/infiniflow/ragflow/issues/13388

Call get_flatted_meta_by_kbs in dify retrieval. Remove get_meta_by_kbs.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-11 13:42:24 +08:00
Josh
2d2d3cdbcf Fix document metadata loading for paged listings (#13515)
## Summary
- scope normal document-list metadata lookups to the current page's
document IDs
- keep the `return_empty_metadata=True` path dataset-wide because it
needs full knowledge of docs that already have metadata
- add unit tests for both paged listing paths and the unchanged
empty-metadata behavior

## Why
`DocumentService.get_list()` and the normal `get_by_kb_id()` path were
calling `DocMetadataService.get_metadata_for_documents(None, kb_id)`,
which loads metadata for the entire dataset on every page request.

That becomes especially problematic on large datasets. The metadata scan
path paginates through the full metadata index without an explicit sort,
while the ES helper only switches to `search_after` beyond `10000`
results when a sort is present. In practice this can lead to unnecessary
full-dataset metadata work, slower document-list loading, and unreliable
`meta_fields` in list responses for large KBs.

This change keeps the existing empty-metadata filter behavior intact,
but scopes normal list responses to metadata for the current page only.
2026-03-11 13:42:16 +08:00
Jimmy Ben Klieve
507ba4ea20 refactor(ui): update knowledge graph, chunk, metadata, agent log styles (#13518)
### What problem does this PR solve?

Update UI styles:
- **Dataset** > **Knowledge graph** tooltip
- **Dataset** > **Files** > **Manage metadata** modal
- **Dataset** > **Files** > **Modify Chunking Method** > **Auto
metadata** > **Manage generation settings** modal
- **Agent** > **Canvas (Ingestion pipeline)** > **Dataflow result**

### Type of change

- [x] Refactoring
2026-03-11 11:27:20 +08:00
Jin Hai
2133fd76a8 Add auth middleware (#13506)
### What problem does this PR solve?

Use auth middle-ware to check authorization.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-11 11:23:13 +08:00
eviaaaaa
d0ca388bec Refa: implement unified lazy image loading for Docx parsers (qa/manual) (#13329)
## Summary
This PR is the direct successor to the previous `docx` lazy-loading
implementation. It addresses the technical debt intentionally left out
in the last PR by fully migrating the `qa` and `manual` parsing
strategies to the new lazy-loading model.

Additionally, this PR comprehensively refactors the underlying `docx`
parsing pipeline to eliminate significant code redundancy and introduces
robust fallback mechanisms to handle completely corrupted image streams
safely.


## What's Changed

* **Centralized Abstraction (`docx_parser.py`)**: Moved the
`get_picture` extraction logic up to the `RAGFlowDocxParser` base class.
Previously, `naive`, `qa`, and `manual` parsers maintained separate,
redundant copies of this method. All downstream strategies now natively
gather raw blobs and return `LazyDocxImage` objects automatically.
* **Robust Corrupted Image Fallback (`docx_parser.py`)**: Handled edge
cases where `python-docx` encounters critically malformed magic headers.
Implemented an explicit `try-except` structure that safely intercepts
`UnrecognizedImageError` (and similar exceptions) and seamlessly falls
back to retrieving the raw binary via `getattr(related_part, "blob",
None)`, preventing parser crashes on damaged documents.

* **Legacy Code & Redundancy Purge**:
* Removed the duplicate `get_picture` methods from `naive.py`, `qa.py`,
and `manual.py`.
* Removed the standalone, immediate-decoding `concat_img` method in
`manual.py`. It has been completely replaced by the globally unified,
lazy-loading-compatible `rag.nlp.concat_img`.
* Cleaned up unused legacy imports (e.g., `PIL.Image`, docx exception
packages) across all updated strategy files.

## Scope
To keep this PR focused, I have restricted these changes strictly to the
unification of `docx` extraction logic and the lazy-load migration of
`qa` and `manual`.

## Validation & Testing
I've tested this to ensure no regressions and validated the fallback
logic:

* **Output Consistency**: Compared identical `.docx` inputs using `qa`
and `manual` strategies before and after this branch: chunk counts,
extracted text, table HTML, and attached images match perfectly.
* **Memory Footprint Drop**: Confirmed a noticeable drop in peak memory
usage when processing image-dense documents through the `qa` and
`manual` pipelines, bringing them up to parity with the `naive`
strategy's performance gains.

## Breaking Changes
* None.
2026-03-11 10:00:07 +08:00
balibabu
d36e3c97d1 Feat: Add a user_id field to the message and retrieval operators. (#13508)
### What problem does this PR solve?

Feat: Add a user_id field to the message and retrieval operators.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-10 22:18:27 +08:00
Yongteng Lei
3c80a0ae09 Fix: support vLLM's new reasoning field (#13493)
### What problem does this PR solve?

Support vLLM's new reasoning field

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 21:13:14 +08:00
yzy
07c9cf6cbe Fix: return structured JSON output for non-streaming agent API (#13389)
### What problem does this PR solve?

Previously, when an Agent component was configured with structured
output, the non-streaming /agents/{agent_id}/completions API never
returned the structured field in its response.

The root cause: the non-streaming code path only collected message
events to build full_content, then returned the workflow_finished
payload — which only contains the output of the last component in the
execution path (typically a Message component).
Any structured output set by upstream components (e.g., Agent or LLM)
was silently discarded.

This PR fixes the non-streaming handler to iterate node_finished events
and collect structured output from intermediate components.
If any component produced a non-empty structured value, it is included
in the final response under data.structured. The streaming path is
unaffected, as it already exposes node_finished events to the caller.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 19:22:04 +08:00
Heyang Wang
08f83ff331 Feat: Support get aggregated parsing status to dataset via the API (#13481)
### What problem does this PR solve?

Support getting aggregated parsing status to dataset via the API

Issue: #12810

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: heyang.why <heyang.why@alibaba-inc.com>
2026-03-10 18:05:45 +08:00
Liu An
68a623154a Fix: bin directory cannot be copied to docker image introduced by #13444 (#13502)
### What problem does this PR solve?

bin directory cannot be copied to docker image introduced by

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 17:31:20 +08:00
chanx
f14b53c764 feat(admin): Implemented default administrator initialization and login functionality. (#13504)
### What problem does this PR solve?

feat(admin): Implemented default administrator initialization and login
functionality.

Added support for default administrator configuration, including super
user nickname, email, and password.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 17:30:21 +08:00
balibabu
81461b4505 Fix: The number of deleted session prompts is displayed incorrectly. #13499 (#13500)
### What problem does this PR solve?

Fix: The number of deleted session prompts is displayed incorrectly.
#13499
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 16:01:31 +08:00
Magicbook1108
675810e0cf Refact: optimize confluence performance (#13497)
### What problem does this PR solve?

Refact: optimize confluence performance #13494

### Type of change

- [x] Refactoring
2026-03-10 15:02:24 +08:00
Alexander Vostres
9ba43ae4ee Fix "Coordinate lower is less than upper" error with MinerU (#13483)
### What problem does this PR solve?

Fixes #6004 #7142 #11959

Unlike #9207 we actually normalize the coordinates here

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 15:02:01 +08:00
balibabu
aaf900cf16 Feat: Display release status in agent version history. (#13479)
### What problem does this PR solve?
Feat: Display release status in agent version history.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-10 14:25:27 +08:00
Idriss Sbaaoui
249b78561b Fix missmatch docnm_kwd in raptor chunks (#13451)
### What problem does this PR solve?

issue #13393 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 14:24:33 +08:00
qinling0210
185ab0d4ef Fix delete_document_metadata (#13496)
### What problem does this PR solve?

Avoid getting doc in function delete_document_metadata as the doc might
have been removed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 13:44:24 +08:00
Magicbook1108
7143954b48 Fix: chats_openai in none stream condition (#13495)
### What problem does this PR solve?

Fix: chats_openai in none stream condition #13453

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 13:44:17 +08:00
qinling0210
7c92f51133 Fix retrieval function when metadata_condtion is specified in retrieval API (#13473)
### What problem does this PR solve?

Fix https://github.com/infiniflow/ragflow/issues/13388

The following command returns empty when there is doc with the meta data
```
curl --request POST \
     --url http://localhost:9222/api/v1/retrieval \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer ragflow-fO3mPFePfLgUYg8-9gjBVVXbvHqrvMPLGaW0P86PvAk' \
     --data '{
          "question": "any question",
          "dataset_ids": ["9bb4f0591b8811f18a4a84ba59049aa3"],
           "metadata_condition": {
            "logic": "and",
            "conditions": [
              {
                "name": "character",
                "comparison_operator": "is",
                "value": "刘备"
              }
            ]
          }
     }'
```

When metadata_condtion is specified in the retrieval API, it is
converted to doc_ids and doc_ids is passed to retrieval function.
In retrieval funciton, when doc_ids is explicitly provided , we should
bypass threshold.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 11:57:32 +08:00
tunsuy
292a1a8566 fix: detect and fallback garbled PDF text to OCR (#13366) (#13404)
## Problem

When PDF fonts lack ToUnicode/CMap mappings, pdfplumber (pdfminer)
cannot map CIDs to correct Unicode characters, outputting PUA characters
(U+E000~U+F8FF) or `(cid:xxx)` placeholders. The original code fully
trusted pdfplumber text without any garbled detection, causing garbled
output in the final parsed result.

Relates to #13366

## Solution

### 1. Garbled text detection functions
- `_is_garbled_char(ch)`: Detects PUA characters (BMP/Plane 15/16),
replacement character U+FFFD, control characters, and
unassigned/surrogate codepoints
- `_is_garbled_text(text, threshold)`: Calculates garbled ratio and
detects `(cid:xxx)` patterns

### 2. Box-level fallback (in `__ocr()`)
When a text box has ≥50% garbled characters, discard pdfplumber text and
fallback to OCR recognition.

### 3. Page-level detection (in `__images__()`)
Sample characters from each page; if garbled rate ≥30%, clear all
pdfplumber characters for that page, forcing full OCR.

### 4. Layout recognizer CID filtering
Filter out `(cid:xxx)` patterns in `layout_recognizer.py` text
processing to prevent them from polluting layout analysis.

## Testing
- 29 unit tests covering: normal CJK/English text, PUA characters, CID
patterns, mixed text, boundary thresholds, edge cases
- All 85 existing project unit tests pass without regression
2026-03-10 11:20:31 +08:00
Jin Hai
7f6a9e8ee9 Update ext field type of heartbeat message (#13490)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 10:49:39 +08:00
chanx
02108772d8 refactor: Moves the LLM factory initialization logic to the dao package. (#13476)
### What problem does this PR solve?

refactor: Moves the LLM factory initialization logic to the `dao`
package.

Removes the `init_data` package and integrates the LLM factory
initialization functionality into the `dao` package.
Adds a `utility` package to provide general utility functions.
Updates `server_main.go` to use the new initialization path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 10:35:55 +08:00
atian8179
88a40b95a2 fix: include missing modules in ragflow-cli PyPI package (#13457)
## Problem

The `ragflow-cli` PyPI package (v0.24.0) is missing `http_client.py`,
`ragflow_client.py`, and `user.py`, causing import errors when installed
from PyPI.

## Root Cause

`pyproject.toml` only lists `ragflow_cli` and `parser` in
`[tool.setuptools] py-modules`.

## Fix

Add the three missing modules to `py-modules`.

Fixes #13456

Co-authored-by: atian8179 <atian8179@users.noreply.github.com>
2026-03-10 10:02:21 +08:00
Jin Hai
4fe706876c Service list and minio status (#13480)
### What problem does this PR solve?

1. Resolve standard user can access admin service
2. Get RAGFlow service status
3. Fix minio status fetching

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 09:56:43 +08:00
writinwaters
4f507c0058 Docs: Updated Switch chunk availability (#13482)
### What problem does this PR solve?

A quick editorial pass.

### Type of change

- [x] Documentation Update
2026-03-09 21:14:45 +08:00
Yongteng Lei
7484298c82 Refa: convert download_img to async (#13477)
### What problem does this PR solve?

Convert download_img to async.

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-03-09 19:00:17 +08:00
Jin Hai
52bcd98d29 Add scheduled tasks (#13470)
### What problem does this PR solve?

1. RAGFlow server will send heartbeat periodically.
2. This PR will including:
- Scheduled task
- API server message sending
- Admin server API to receive the message.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 17:48:29 +08:00
Jin Hai
c732a1c8e0 Refactor the go_binding to binding (#13469)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 15:52:31 +08:00
chanx
25ace613b0 feat: Added LLM factory initialization functionality and knowledge base related API interfaces (#13472)
### What problem does this PR solve?

feat: Added LLM factory initialization functionality and knowledge base
related API interfaces

refactor(dao): Refactored the TenantLLMDAO query method
feat(handler): Implemented knowledge base related API endpoints
feat(service): Added LLM API key setting functionality
feat(model): Extended the knowledge base model definition
feat(config): Added default user LLM configuration

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-09 15:52:14 +08:00
Stephen Hu
d0465ba909 refactor: improve paddle ocr logic (#13467)
### What problem does this PR solve?

improve paddle ocr logic

### Type of change
- [x] Refactoring
2026-03-09 14:16:57 +08:00
天海蒼灆
3ce236c4e3 Feat: add switch_chunks endpoint to manage chunk availability (#13435)
### What problem does this commit solve?

This commit introduces a new API endpoint
`/datasets/<dataset_id>/documents/<document_id>/chunks/switch` that
allows users to switch the availability status of specified chunks in a
document as same as chunk_app.py

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-09 12:36:45 +08:00
guptas6est
32d31284cc Fix: upgrade pypdf to 6.7.5 and migrate from deprecated pypdf2 to fix CVE-2026-28804 and CVE-2023-36464 (#13454)
### What problem does this PR solve?

This PR addresses security vulnerabilities in PDF processing
dependencies identified by Trivy security scan:

1. CVE-2026-28804 (MEDIUM): pypdf 6.7.4 vulnerable to inefficient
decoding of ASCIIHexDecode streams
2. CVE-2023-36464 (MEDIUM): pypdf2 3.0.1 susceptible to infinite loop
when parsing malformed comments

Since pypdf2 is deprecated with no available fixes, this PR migrates all
pypdf2 usage to the actively maintained pypdf library (version 6.7.5),
which resolves
both vulnerabilities.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-09 12:06:00 +08:00
JiangNan
2634cfc06f Fix: undefined variable and wrong method name in agent components (#13462)
## Summary

This PR fixes two runtime bugs in agent components:

**Bug 1: `agent/component/invoke.py` — `NameError` in POST +
`clean_html` path**

The POST method's `clean_html` branch uses the variable `sections`
without ever defining it. Both the GET and PUT branches correctly call
`sections = HtmlParser()(None, response.content)` before referencing
`sections`, but this line was missing from the POST branch (copy-paste
omission). This causes a `NameError` whenever a user configures an
Invoke component with `method="post"` and `clean_html=True`.

**Bug 2: `agent/component/data_operations.py` — `AttributeError` in
`_recursive_eval`**

The `_recursive_eval` method recursively calls `self.recursive_eval()`
(without the leading underscore) instead of `self._recursive_eval()`.
Since the method is defined as `_recursive_eval`, this causes an
`AttributeError` at runtime when the `literal_eval` operation processes
nested dicts or lists.

## Test plan

- [ ] Configure an Invoke node with `method=post` and `clean_html=True`,
verify HTML is parsed correctly without `NameError`
- [ ] Configure a DataOperations node with `operations=literal_eval` on
nested data, verify no `AttributeError`

---------

Signed-off-by: JiangNan <1394485448@qq.com>
2026-03-09 11:09:47 +08:00
Jin Hai
610c1b507d Add more API of admin server of go (#13403)
### What problem does this PR solve?

Add APIs to admin server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 10:44:53 +08:00
Eden
ab6ca75245 fix(agent): ensure database connections are properly closed in ExeSQL tool (#13427)
## Summary

Fix a database connection and cursor resource leak in the ExeSQL agent
tool.

When SQL execution raises an exception (for example syntax error or
missing table),
the existing code path skips `cursor.close()` and `db.close()`, causing
database
connections to accumulate over time.

This can eventually lead to connection exhaustion in long-running agent
workflows.

## Root Cause

The cleanup logic for database cursors and connections is placed after
the SQL
execution loop without `try/finally` protection. If an exception occurs
during
`cursor.execute()`, `fetchmany()`, or result processing, the cleanup
code is not
reached and the connection remains open.

The same issue also exists in the IBM DB2 execution path where
`ibm_db.close(conn)`
may be skipped when exceptions occur.

## Fix

- Wrap SQL execution logic in `try/finally` blocks to guarantee resource
cleanup.
- Ensure `cursor.close()` and `db.close()` are always executed.
- Add explicit `db.close()` when `db.cursor()` creation fails.
- Remove redundant close calls in early-return branches since `finally`
now handles cleanup.

## Impact

- No change to normal execution behavior.
- Ensures database resources are always released when errors occur.
- Prevents connection leaks in long-running workflows.
- Only affects `agent/tools/exesql.py`.

## Testing

Manual test scenarios:

1. Valid SQL execution
2. SQL syntax error
3. Query against a non-existing table
4. Execution cancellation during query

In all scenarios the database cursor and connection are properly closed.

Code quality checks:

- `ruff check` passed
- No new warnings introduced
2026-03-09 10:36:02 +08:00
Liu An
89e495e1bc Chore: update release workflow configuration (#13466)
### What problem does this PR solve?

update release workflow configuration

### Type of change

- [x] Update CI
2026-03-09 10:32:51 +08:00
Heyang Wang
c217b8f3d8 Feat: add DingTalk AI Table connector and integration for data synch… (#13413)
### What problem does this PR solve?

Add DingTalk AI Table connector and integration for data synchronization

Issue #13400

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: wangheyang <wangheyang@corp.netease.com>
2026-03-06 21:13:23 +08:00
Jimmy Ben Klieve
094eae3cf5 refactor(ui): adjust dataset page styles (#13452)
### What problem does this PR solve?

- Adjust UI styles in **Dataset** pages.
- Adjust several shared components styles
- Modify files and directory structure in `src/layouts`

### Type of change

- [x] Refactoring
2026-03-06 21:13:14 +08:00
Liu An
7166a7e50e Test: adjust test priority markers for API tests (#13450)
### What problem does this PR solve?

Changed test priority markers from p1/p2 to p3 in three test files:
- test_table_parser_dataset_chat.py: Adjusted priority for table parser
dataset chat test
- test_delete_chunks.py: Updated priority for chunk deletion test with
invalid IDs
- test_retrieval_chunks.py: Modified priority for chunks retrieval
pagination test

These changes demote the priority of specific test cases to p3,
indicating they are lower priority tests that can run later in the test
suite execution.

### Type of change

- [x] Test update
2026-03-06 20:17:39 +08:00
chanx
ae4645e01b Fix: Add folder upload #9743 (#13448)
### What problem does this PR solve?

Fix: Add folder upload  #9743

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 20:17:29 +08:00
balibabu
82a616589b Feat: Add PublishConfirmDialog (#13447)
### What problem does this PR solve?

Feat: Add PublishConfirmDialog

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 20:17:21 +08:00
Achieve3318
45cf24cd2f feat(memory): implement get_highlight for OceanBase memory (#13449)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 20:17:11 +08:00
Jin Hai
01a100bb29 Fix data models (#13444)
### What problem does this PR solve?

Since database model is updated in python version, go server also need
to update

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-06 20:05:10 +08:00
OliverW
3ed91345aa fix(auth): return HTTP 401 for token-auth failures (#13420)
Follow-up to #12488 #13386

### What problem does this PR solve?

Previously, token authentication failures returned HTTP 200 with an
error code in the response body.

This PR updates `token_required` to raise `Unauthorized` and relies on
the global error handler to return a structured JSON response with HTTP
401 status.

The response body structure (`code`, `message`, `data`) remains
unchanged to preserve compatibility with the official SDK.

Frontend logic has been updated to handle HTTP 401 responses in addition
to checking `data.code`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 18:18:14 +08:00
Yongteng Lei
51be1f1442 Refa: empty ids means no-op operation (#13439)
### What problem does this PR solve?

Empty ids means no-op operation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-06 18:16:42 +08:00
Zhichang Yu
7781c51a21 Revert aliyun registry to registry.cn-hangzhou.aliyuncs.com (#13445)
## Summary
- Revert aliyun registry from
`infiniflow-registry.cn-shanghai.cr.aliyuncs.com` back to
`registry.cn-hangzhou.aliyuncs.com`

## Test plan
- [ ] Verify the docker/.env file contains the correct registry URL

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:03:35 +08:00
Magicbook1108
826af383b4 Fix: paddle ocr missing outlines (#13441)
### What problem does this PR solve?

Fix: paddle ocr missing outlines #13422

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 17:19:51 +08:00
Jin Hai
2504c3adde Fix docker file (#13438)
### What problem does this PR solve?

To copy infinity/resource into docker images

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-06 16:56:12 +08:00
chanx
81fd1811b8 Feat:Using Go to implement user registration logic (#13431)
### What problem does this PR solve?

Feat:Using Go to implement user registration logic

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 16:42:49 +08:00
Achieve3318
37eb533fea Feat(memory): implement get_aggregation for OceanBase memory (#13428)
### What problem does this PR solve?

- Add aggregation_utils.aggregate_by_field for pure aggregation logic
- Wire OBConnection.get_aggregation to use it (unwrap tuple, pass
messages)
- Add unit tests for aggregate_by_field (no DB/heavy deps)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 12:51:22 +08:00
BitToby
383986dc5f fix: re-chunk documents when data source content is updated (#12918)
Closes: #12889 

### What problem does this PR solve?

When syncing external data sources (e.g., Jira, Confluence, Google
Drive), updated documents were not being re-chunked. The raw content was
correctly updated in blob storage, but the vector database retained
stale chunks, causing search results to return outdated information.

**Root cause:** The task digest used for chunk reuse optimization was
calculated only from parser configuration fields (`parser_id`,
`parser_config`, `kb_id`, etc.), without any content-dependent fields.
When a document's content changed but the parser configuration remained
the same, the system incorrectly reused old chunks instead of
regenerating new ones.

**Example scenario:**
1. User syncs a Jira issue: "Meeting scheduled for Monday"
2. User updates the Jira issue to: "Meeting rescheduled to Friday"
3. User triggers sync again
4. Raw content panel shows updated text ✓
5. Chunk panel still shows old text "Monday" ✗

**Solution:**
1. Include `update_time` and `size` in the chunking config, so the task
digest changes when document content is updated
2. Track updated documents separately in `upload_document()` and return
them for processing
3. Process updated documents through the re-parsing pipeline to
regenerate chunks


[1.webm](https://github.com/user-attachments/assets/d21d4dcd-e189-4d39-8700-053bae0ca5a0)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 12:48:47 +08:00
Lynn
0214257886 Fix: init func (#13430)
### What problem does this PR solve?

Fix update_cnt add error in init_data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 11:42:31 +08:00
balibabu
6849d35bf5 Feat: Optimize the style of the chat page. (#13429)
### What problem does this PR solve?

Feat: Optimize the style of the chat page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 11:42:25 +08:00
Jonah Hartmann
6023eb27ac feat: add Ragcon provider (#13425)
### What problem does this PR solve?

This PR aims to extend the list of possible providers. Adds new Provider
"RAGcon" within the Ollama Modal. It provides all model types except OCR
via Openai-compatible endpoints.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jakob <16180662+hauberj@users.noreply.github.com>
2026-03-06 09:37:27 +08:00
guptas6est
c35b210c3a fix(security): upgrade requests to 2.32.5 in agent/sandbox to fix CVE-2024-47081 (#13424)
### What problem does this PR solve?

This PR remediates CVE-2024-47081 (MEDIUM severity) in the agent/sandbox
component by upgrading the requests library from version 2.32.3 to
2.32.5. The vulnerability allows .netrc credentials to leak via
malicious URLs.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 09:29:18 +08:00
guptas6est
aa57bcf92a fix: upgrade urllib3 to 2.6.3 to resolve CVE-2025-66418, CVE-2025-66471, CVE-2026-21441 (#13423)
### What problem does this PR solve?

This PR remediates three HIGH severity vulnerabilities in urllib3
affecting the admin client and Python SDK:
- **CVE-2025-66418**: Unbounded decompression chain leads to resource
exhaustion
- **CVE-2025-66471**: Streaming API improperly handles highly compressed
data
- **CVE-2026-21441**: Decompression-bomb safeguard bypass when following
HTTP redirects
Trivy security scan identified urllib3 v2.5.0 as vulnerable in both
`admin/client/uv.lock` and `sdk/python/uv.lock`. This PR updates urllib3
to v2.6.3 to eliminate these security risks.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 09:29:10 +08:00
Jimmy Ben Klieve
ef4cbe72a3 refactor(ui): adjust global navigation bar style (#13419)
### What problem does this PR solve?

Renovate global navigation bar, align styles to the design.
(May causes minor layout issues in sub-pages, will check and fix soon)

### Type of change

- [x] Refactoring
2026-03-05 20:47:29 +08:00
leonardlin
9e0e128ce5 Add checksum/values annotation to ragflow.yaml (#13409)
Add checksum annotation for values in ragflow.yaml

### What problem does this PR solve?

This PR is about this ticket: #13408

Ragflow helm charts do not include the Values.yaml in the list of
watched changes.
If you update the Values.yaml for an existing deployment, helm will not
detect it and not update the deployment.

This PR fixes that.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 20:27:38 +08:00
writinwaters
963e31e9b5 Refact: Updated the doc structure. (#13414)
### What problem does this PR solve?

Updated the doc structure.

### Type of change


- [x] Documentation Update
2026-03-05 19:04:56 +08:00
Idriss Sbaaoui
d90d6026af Playwright : new chat multi model test (#13402)
### What problem does this PR solve?

new test for chat multiple model and other chat parameters under
playwright

### Type of change

- [x] Other (please describe): new test/ data-testid
2026-03-05 18:51:57 +08:00
Yongteng Lei
d9785ea2ce Fix: Alibaba cloud OSS config issue (#13406)
### What problem does this PR solve?

 Alibaba Could OSS config issue #13390.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 18:13:45 +08:00
chanx
8b534c895e Fix: UI Placeholder and Hint Optimization (#13416)
### What problem does this PR solve?

Fix: UI Placeholder and Hint Optimization

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 18:13:19 +08:00
chanx
35fc5edc93 feat: Adds the tenant model ID field to the interface definition. (#13274)
### What problem does this PR solve?

feat: Adds the tenant model ID field to the interface definition

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 17:27:34 +08:00
Lynn
62cb292635 Feat/tenant model (#13072)
### What problem does this PR solve?

Add id for table tenant_llm and apply in LLMBundle.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-05 17:27:17 +08:00
Magicbook1108
47540a4147 Feat: published agent version control (#13410)
### What problem does this PR solve?

Feat: published agent version control

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 17:26:39 +08:00
guptas6est
8c9b080499 fix: update axios to 1.13.5+ to remediate CVE-2026-25639 DoS vulnerability (#13380)
### What problem does this PR solve?

This PR remediates CVE-2026-25639, a HIGH severity Denial of Service
vulnerability in axios caused by __proto__ pollution in the mergeConfig
function. The vulnerability affects both the web frontend and the
sandbox nodejs environment.

Trivy security scan identified axios versions below 1.13.5 as
vulnerable. This PR updates axios to secure versions (1.13.6 in web,
1.13.5 in sandbox) to eliminate the security risk.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 17:26:04 +08:00
Yongteng Lei
f13a1fb007 Refa: improve model verification ux (#13392)
### What problem does this PR solve?

Improve model verification UX. #13395 

### Type of change

- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-05 17:23:47 +08:00
Liu An
3124fa955e chore: add bin and internal dirs to .gitignore for Go server build output (#13407)
### What problem does this PR solve?

add bin and internal dirs to .gitignore for Go server build output
2026-03-05 15:52:01 +08:00
tunsuy
e1f1184b01 test: add unit tests for graphrag/utils.py (87 test cases) (#13328)
Add comprehensive unit tests for `graphrag/utils.py`, covering 15
functions/classes with 87 test cases.

Tested functions:
- clean_str, dict_has_keys_with_types, perform_variable_replacements
- get_from_to, compute_args_hash, is_float_regex
- GraphChange dataclass
- handle_single_entity_extraction, handle_single_relationship_extraction
- graph_merge, tidy_graph
- split_string_by_multi_markers, pack_user_ass_to_openai_messages
- is_continuous_subsequence, merge_tuples, flat_uniq_list

All 327 existing + new tests pass with no regressions.
2026-03-05 15:30:43 +08:00
Jin Hai
3e3b665b89 RAGFlow admin server go version (#13394)
### What problem does this PR solve?

1. init go admin server
2. refactor api server router
3. add benchmark CI to 450s time limit
4. remove docker builder container after building

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-05 15:18:40 +08:00
Liu An
6f5bd4d2e9 feat: add bin and internal dirs to .gitignore for Go server build output (#13391)
### What problem does this PR solve?

add bin and internal dirs to .gitignore for Go server build output
2026-03-05 14:26:40 +08:00
天海蒼灆
118f737b3a Feat:Enhance chunk management by adding support for 'available', 'tag_kwd' and 'tag_feas' (#13383)
### What problem does this PR solve?

Enhance chunk management by adding support for 'available', 'tag_kwd'
and 'tag_feas' fields in list, add, and update chunk functions just like
chunk_app.py.This improves data handling and flexibility in chunk
processing.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 13:45:39 +08:00
orbcom-pedroferreira
61209ff3bf Feat: File uploads for future conversations on SDK API (#13378)
### What problem does this PR solve?

This PR aims to:

1. Enable file uploads for the public API, similarly to what
/document/upload_info accomplishes for the frontend;
2. Enable files sent to the /chat/:chat_id/completions endpoint to be
used within the conversation.
We classify the first item as a new future, while classifying the second
one as a bug fix.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

*The work related to this PR was co-authored by*

[Bruno Ferreira](https://github.com/brunopferreira): Custom Solutions
Manager @ [Orbcom](https://orbcom.pt/)
[Pedro Ferreira](https://github.com/sirj0k3r): Lead Software Developer @
[Orbcom](https://orbcom.pt/)
[Pedro Cardoso](https://github.com/pedromiguel4560): Associate Software
Developer @ [Orbcom](https://orbcom.pt/)

*This PR replaces #13248*

---------

Co-authored-by: Pedro Cardoso <pedrocardoso@orbcom.pt>
Co-authored-by: Pedro Ferreira <pedroferreira@orbcom.pt>
2026-03-04 22:26:58 +08:00
tunsuy
020068dd16 Fix: preserve field boundaries in chunked documents from MySQL… (#13369)
### What problem does this PR solve?

When multiple columns are used as content columns in RDBMS connector,
the generated document text gets chunked by TxtParser which strips
newline delimiters during merge. This causes field names and values from
different columns to be concatenated without any separator, making the
content unreadable.

Changes:
- txt_parser.py: restore newline separator when merging adjacent text
segments within a chunk, so that split sections are not directly
concatenated
- rdbms_connector.py: use double newline between fields and place field
value on a new line after the field name bracket, giving TxtParser
clearer boundaries to work with

Closes #13001

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: tunsuytang <tunsuytang@tencent.com>
2026-03-04 21:42:02 +08:00
writinwaters
9deb3a6249 Refact: Fine tweaks to the doc structure. (#13379)
### What problem does this PR solve?

Fine tweaks to the doc structure.

### Type of change


- [x] Documentation Update
2026-03-04 21:30:28 +08:00
balibabu
be231faec0 Feat: Write the row and column numbers into the element's data attribute for easy code location. (#13368)
### What problem does this PR solve?

Feat: Write the row and column numbers into the element's data attribute
for easy code location.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 20:50:58 +08:00
Idriss Sbaaoui
b3a7332c08 playwright : add data-testids for new test (#13364)
### What problem does this PR solve?

add data-testids for new test

### Type of change

- [x] Other (please describe): add data-testids for new test
2026-03-04 19:28:36 +08:00
Yao Wei
c99b53064d fix: remove company info from resume_summary to prevent over-retrieval (#13358)
### What problem does this PR solve?

Problem: When searching for a specific company name like(Daofeng
Technology), the search would incorrectly return unrelated resumes
containing generic terms like (Technology) in their company names

Root Cause: The `corporation_name_tks` field was included in the
identity fields that are redundantly written to every chunk. This caused
common words like "科技" to match across all chunks, leading to
over-retrieval of irrelevant resumes.

Solution: Remove `corporation_name_tks` from the `_IDENTITY_FIELDS`
list. Company information is still preserved in the "Work Overview"
chunk where it belongs, allowing proper company-based searches while
preventing false positives from generic terms.

---------

Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 19:24:49 +08:00
Jin Hai
70e9743ef1 RAGFlow go API server (#13240)
# RAGFlow Go Implementation Plan 🚀

This repository tracks the progress of porting RAGFlow to Go. We'll
implement core features and provide performance comparisons between
Python and Go versions.

## Implementation Checklist

- [x] User Management APIs
- [x] Dataset Management Operations
- [x] Retrieval Test
- [x] Chat Management Operations
- [x] Infinity Go SDK

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-03-04 19:17:16 +08:00
Idriss Sbaaoui
2508c46c8f Playwright : add new test for configuration tab in datasets (#13365)
### What problem does this PR solve?

this pr adds new tests, for the full configuration tab in datasests

### Type of change

- [x] Other (please describe): new tests
2026-03-04 19:10:06 +08:00
Idriss Sbaaoui
88e8509159 benchmark fail in ci (#13377)
### What problem does this PR solve?
ci fails in elastic search because of benchmark

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 19:01:41 +08:00
Stephen Hu
c7d17c84b2 Refa:improve excel parser logic (#13372)
### What problem does this PR solve?

improve excel parser logic

### Type of change
- [x] Refactoring
2026-03-04 18:00:17 +08:00
Jin Hai
6bb00e2762 Update graspologic to gitee (#13362)
### What problem does this PR solve?

Accelerate python module downloading

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 17:48:47 +08:00
Good0987
8a7272f423 Test: add scenario for embedding_model update when chunk_count > 0 (#13351)
### What problem does this PR solve?

Guard embedding_model change when dataset has existing chunks. API must
return code 102 with message 'When chunk_num (N) > 0, embedding_model
must remain <current_model>' to prevent silent embedding drift.

### Type of change

- [x] Add Testcases

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 17:41:35 +08:00
Jin Hai
f47c47df99 Disable benchmark (#13370)
### What problem does this PR solve?

benchmark always failed in new CI machine. please enable it after the
issue is fixed.

### Type of change

- [x] Other (please describe): disable benchmark

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 16:36:42 +08:00
yiminghub2024
5eb602166c Enhance local model deployment documentation support gpustack guide (#13339)
### Type of change

- [X] Documentation Update:Enhance local model deployment documentation
support gpustack guide
2026-03-04 13:54:20 +08:00
少卿
54ae5b4a27 Fix Dify external retrieval by providing metadata.document_id (#13337)
### What problem does this PR solve?

## Summary                                                           
  Dify’s external retrieval expects `records[].metadata.document_id` to
  be a non-empty string.                                               
  RAGFlow currently only sets `metadata.doc_id`, which causes Dify     
  validation to fail.                                                  
                                                                       
  This PR adds `metadata.document_id` (mapped from `doc_id`) in the    
  Dify-compatible retrieval response.                                  
                                                                       
  ## Changes                                                           
- Add `meta["document_id"] = c["doc_id"]` in
`api/apps/sdk/dify_retrieval.py`
                                                                       
  ## Testing                                                           
  - Not run (logic-only change).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 13:23:37 +08:00
Jin Hai
b9ad014f63 Supports login cross multiple RAGFlow servers (#13322)
### What problem does this PR solve?

1. Use redis to store the secret key.
2. During startup API server will read the secret from redis. If no such
secret key, generate one and store it into redis, atomically.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 13:07:45 +08:00
balibabu
5f8966608d Fix: The dropdown menu for large models does not automatically focus on the search box. #13313 (#13360)
### What problem does this PR solve?

Fix: The dropdown menu for large models does not automatically focus on
the search box. #13313

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 12:48:35 +08:00
Magicbook1108
93d621a666 Fix: Correct PDF chunking parameter name in naive (#13357)
### What problem does this PR solve?

Fix: Correct PDF chunking parameter name in naive #13325

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 11:51:10 +08:00
balibabu
733a64f0d6 Fix: Change the background color of the message notification button. (#13344)
### What problem does this PR solve?

Fix: Change the background color of the message notification button.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 11:10:05 +08:00
statxc
839b603768 feat: Add PDF parser selection to Agent Begin and Await Response comp… (#13325)
### Issue: #12756

### What problem does this PR solve?

When users upload files through Agent's Begin or Await Response
components, the parsing is hardcoded to "Plain Text", ignoring all other
available parsers (DeepDOC, TCADP, Docling, MinerU, PaddleOCR). This PR
adds a PDF parser dropdown to these components so users can select the
appropriate parser for their file inputs.


### Changes

**Backend**
- `agent/component/fillup.py` - Added `layout_recognize` param to
`UserFillUpParam`, forwarded to `FileService.get_files()`
- `agent/component/begin.py` - Same forwarding in `Begin._invoke()`
- `agent/canvas.py` - Extract Begin's `layout_recognize` for `sys.files`
parsing, added param to `get_files_async()` / `get_files()`
- `api/db/services/file_service.py` - Added `layout_recognize` param to
`parse()` and `get_files()`, replacing hardcoded `"Plain Text"`
- `rag/app/naive.py` - Added `"plain text"` and `"tcadp parser"` aliases
to PARSERS dict to match dropdown values after `.lower()`

**Frontend**
- `web/src/pages/agent/form/begin-form/index.tsx` - Show
`LayoutRecognizeFormField` dropdown when file inputs exist
- `web/src/pages/agent/form/begin-form/schema.ts` - Added
`layout_recognize` to Zod schema
- `web/src/pages/agent/form/user-fill-up-form/index.tsx` - Same dropdown
for Await Response component


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-04 11:09:33 +08:00
Liu An
7715bad04e refactor: reorganize unit test files into appropriate directories (#13343)
### What problem does this PR solve?

Move test files from utils/ to their corresponding functional
directories:
- api/db/ for database related tests
- api/utils/ for API utility tests
- rag/utils/ for RAG utility tests

### Type of change

- [x] Refactoring
2026-03-04 11:02:56 +08:00
Copilot
33ba955b02 Translate Chinese text to English in agent/sandbox (#13356)
Chinese text remained in generated code comments, log messages, field
descriptions, and documentation files under `agent/sandbox/`.

### Changes

- **`tests/MIGRATION_GUIDE.md`** — Full EN translation (migration guide
from OpenSandbox → Code Interpreter)
- **`tests/QUICKSTART.md`** — Full EN translation (quick test guide for
Aliyun sandbox provider)
- **`providers/aliyun_codeinterpreter.py`** — Removed `(主账号ID)` from
docstring, error log, and config field description
- **`sandbox_spec.md`** — Removed `(主账号ID)` from `account_id` field
description
- **`tests/test_aliyun_codeinterpreter_integration.py`** — Removed
`(主账号ID)` from inline comment

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yuzhichang <153784+yuzhichang@users.noreply.github.com>
2026-03-04 10:49:38 +08:00
wyou
0a4c0c38c7 Feat: expose admin service in helm configuration (#13345)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
For helm deployment, there is also requirement to enable the Admin
Service for administrative operations.
So expose the ability of enable/disable this function by helm
configuration.
        When it's enabled (by default),
<img width="486" height="190" alt="image"
src="https://github.com/user-attachments/assets/4db0dc3d-bd94-4ad9-bb5d-a240aac5e1c5"
/>
        Admin access and operations would be feasible like below,
<img width="2530" height="876" alt="image"
src="https://github.com/user-attachments/assets/3e948e1b-7522-4f8d-8dc0-c80a22242022"
/>
Something like 'user management' is very much important for Ragflow
User/Owner to control their clients.
2026-03-04 10:26:10 +08:00
Idriss Sbaaoui
2f4ca38adf Fix : make playwright tests idempotent (#13332)
### What problem does this PR solve?

Playwright tests previously depended on cross-file execution order
(`auth -> provider -> dataset -> chat`).
This change makes setup explicit and idempotent via fixtures so tests
can run independently.

- Added/standardized prerequisite fixtures in
`test/playwright/conftest.py`:
- `ensure_auth_context`, `ensure_model_provider_configured`,
`ensure_dataset_ready`, `ensure_chat_ready`
- Made provisioning reusable/idempotent with `RUN_ID`-based resource
naming.
- Synced auth envs (`E2E_ADMIN_EMAIL`, `E2E_ADMIN_PASSWORD`) into seeded
creds.
- Fixed provider cache freshness (`auth_header`/`page` refresh on cache
hit).

Also included minimal stability fixes:
- dataset create stale-element click handling,
- search wait logic for results/empty-state,
- agent create-menu handling,
- agent run-step retry when run UI doesn’t open first click.

### Type of change

- [x] Test fix
- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 10:07:14 +08:00
writinwaters
1c87f97dde Docs: Minor document structure tweak. (#13346)
### What problem does this PR solve?

Refactored the document architecture.

### Type of change

- [x] Documentation Update
2026-03-03 20:09:34 +08:00
writinwaters
f7c808383f Docs: Refactored documentation (#13340)
### What problem does this PR solve?

Refactored documentation. 

### Type of change

- [x] Documentation Update
2026-03-03 17:48:48 +08:00
Yao Wei
48755a3352 Fix: (resume) Cross-verify project experience and work experience, and remove duplicate text (#13323)
Cross-verify project experience and work experience, and remove
duplicate text

---------

Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
2026-03-03 14:53:46 +08:00
balibabu
eca60208e3 Fix: The document generation node cannot generate the output content of a large model to a file. #13321 (#13326)
### What problem does this PR solve?

Fix: The document generation node cannot generate the output content of
a large model to a file. #13321
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-03 11:05:24 +08:00
Magicbook1108
4f09b3e2a4 Fix: pipeline canvas category (#13319)
### What problem does this PR solve?

Fix: pipeline canvas category

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 20:27:36 +08:00
Yongteng Lei
707de2461a Fix: use async_chat with sync wrapper in resume parser (#13320)
### What problem does this PR solve?

Fix AttributeError when calling llm.chat() in resume parser. LLMBundle
only has async_chat method, not chat method. Use `_run_coroutine_sync`
wrapper to call async_chat synchronously.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 19:51:06 +08:00
chanx
ef264b52c7 Fix: Fixed some errors in the console (#13317)
### What problem does this PR solve?

Fix: Fixed some errors in the console
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 19:19:15 +08:00
Yingfeng
a806f7b707 Potential fix for code scanning alert no. 71: Incomplete URL substring sanitization (#13318)
Potential fix for
[https://github.com/infiniflow/ragflow/security/code-scanning/71](https://github.com/infiniflow/ragflow/security/code-scanning/71)

In general, instead of using `String.prototype.includes` on the entire
URL string, parse the URL and make decisions based on its `host` (or
`hostname`) field. This avoids cases where the trusted domain appears in
the path, query, or as part of a different hostname.

Here, `payload.source_fid` is set to `'siliconflow_intl'` if
`postBody.base_url` “contains” `api.siliconflow.com`. To keep behavior
for correct inputs but close the hole, we should:

1. Safely parse `postBody.base_url` using the standard `URL` class.
2. Extract the hostname (`url.hostname`).
3. Compare it appropriately:
- If we only want the exact host `api.siliconflow.com`, use strict
equality.
- If international endpoints may include subdomains like
`foo.api.siliconflow.com`, allow those via suffix check on the hostname.
4. Fall back to `LLMFactory.SILICONFLOW` if parsing fails or the host
does not match.

Concretely, in `web/src/pages/user-setting/setting-model/hooks.tsx`, in
the `onApiKeySavingOk` callback where `payload.source_fid` is set,
replace the `toLowerCase().includes('api.siliconflow.com')` logic with a
small block that:

- Initializes a local `let sourceFid = LLMFactory.SILICONFLOW;`
- If `postBody.base_url` is present, attempts `new
URL(postBody.base_url)` inside a `try/catch`, lowercases `url.hostname`,
and checks whether it equals `api.siliconflow.com` or ends with
`.api.siliconflow.com`.
- Assigns `payload.source_fid = sourceFid`.

No new external dependencies are required; `URL` is available in modern
browsers and Node, and TypeScript understands it.


_Suggested fixes powered by Copilot Autofix. Review carefully before
merging._

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-02 19:11:52 +08:00
Idriss Sbaaoui
b0ace2c5d0 feat: enable Arabic in production UI and add complete Arabic documentation (#13315)
### What problem does this PR solve?

This PR adds end-to-end Arabic support in production. It also adds a
full Arabic README

### Type of change

 - [x] New Feature (non-breaking change which adds functionality)
 - [x] Documentation Update
2026-03-02 19:10:11 +08:00
Yao Wei
f8c91e8854 Refa: Resume parsing module (architectural optimizations based on SmartResume Pipeline) (#13255)
Core optimizations (refer to arXiv:2510.09722):

1. PDF text fusion: Metadata + OCR dual-path extraction and fusion

2. Page-aware reconstruction: YOLOv10 page segmentation + hierarchical
sorting + line number indexing

3. Parallel task decomposition: Basic information/work
experience/educational background three-way parallel LLM extraction

4. Index pointer mechanism: LLM returns a range of line numbers instead
of generating the full text, reducing the illusion of full text.

---------

Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 19:05:50 +08:00
balibabu
7d6f20585f Feat: Modify the style of the classification operator and fix some console errors. (#13314)
### What problem does this PR solve?

Feat: Modify the style of the classification operator and fix some
console errors.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 16:53:24 +08:00
Magicbook1108
5fc3bd38b0 Feat: Support siliconflow.com (#13308)
### What problem does this PR solve?

Feat: Support siliconflow.com

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 15:37:42 +08:00
Magicbook1108
1db221f19e Feat: add more models for siliconflow and tongyi-qwen (#13311)
### What problem does this PR solve?

Feat: add more models for siliconflow and tongyi-qwen

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 15:37:08 +08:00
liuxiaoyusky
8ba66dd62a Fix: respect user-configured chunk_token_num for MinerU/docling/paddleocr parsers (#13234)
## Summary

When using MinerU, docling, TCADP, or paddleocr as the PDF parser with
the General (naive) chunk method, the user-configured `chunk_token_num`
is **unconditionally overwritten to 0** at
[rag/app/naive.py#L858-L859](https://github.com/infiniflow/ragflow/blob/main/rag/app/naive.py#L858-L859),
effectively disabling chunk merging regardless of what the user sets in
the UI.

### Problem

A user sets `chunk_token_num = 2048` in the dataset configuration UI,
expecting small parser blocks to be merged into larger chunks. However,
this line:

```python
if name in ["tcadp", "docling", "mineru", "paddleocr"]:
    parser_config["chunk_token_num"] = 0
```

silently overrides the user's setting. As a result, every MinerU output
block becomes its own chunk. For short documents (e.g. a 3-page PDF fund
factsheet parsed by MinerU), this produces **47 tiny chunks** — some as
small as 11 characters (`"July 2025"`) or 15 characters (`"CIES
Eligible"`).

This severely degrades retrieval quality: vector embeddings of such
short fragments have minimal semantic value, and keyword search produces
excessive noise.

### Fix

Only apply the `chunk_token_num = 0` override when the user has **not**
explicitly configured a positive value:

```python
if name in ["tcadp", "docling", "mineru", "paddleocr"]:
    if int(parser_config.get("chunk_token_num", 0)) <= 0:
        parser_config["chunk_token_num"] = 0
```

This preserves the original default behavior (no merging) while
respecting the user's explicit configuration.

### Before / After (MinerU, 3-page PDF, chunk_token_num=2048)

| | Before | After |
|---|---|---|
| Chunks produced | 47 | ~8 (merged by token limit) |
| Smallest chunk | 11 chars | ~500 chars |
| User setting respected | No | Yes |

## Test plan

- [ ] Parse a PDF with MinerU and `chunk_token_num = 2048` → verify
chunks are merged up to token limit
- [ ] Parse a PDF with MinerU and `chunk_token_num = 0` (or default) →
verify original behavior (no merging)
- [ ] Parse a PDF with DeepDOC parser → verify no change in behavior
(not affected by this code path)
- [ ] Repeat with docling/paddleocr if available
2026-03-02 15:31:40 +08:00
少卿
d430446e69 fix:absolute page index mix-up in DeepDoc PDF parser (#12848)
### What problem does this PR solve?

Summary:
This PR addresses critical indexing issues in
deepdoc/parser/pdf_parser.py that occur when parsing long PDFs with
chunk-based pagination:

Normalize rotated table page numbering: Rotated-table re-OCR now writes
page_number in chunk-local 1-based form, eliminating double-addition of
page_from offset that caused misalignment between table positions and
document boxes.
Convert absolute positions to chunk-local coordinates: When inserting
tables/figures extracted via _extract_table_figure, positions are now
converted from absolute (0-based) to chunk-local indices before distance
matching and box insertion. This prevents IndexError and out-of-range
accesses during paged parsing of long documents.

Root Cause:
The parser mixed absolute (0-based, document-global) and relative
(1-based, chunk-local) page numbering systems. Table/figure positions
from layout extraction carried absolute page numbers, but insertion
logic expected chunk-local coordinates aligned with self.boxes and
page_cum_height.


Testing(I do):

Manual verification: Parse a 200+ page PDF with from_page > 0 and table
rotation enabled. Confirm that:

Tables and figures appear on correct pages
No IndexError or position mismatches occur
Page numbers in output match expected chunk-local offsets


Automated testing: 我没做


## Separate Discussion: Memory Optimization Strategy(from codex-5.2-max
and claude 4.5 opus and me)

### Context

The current implementation loads entire page ranges into memory
(`__images__`, `page_chars`, intermediates), which can cause RAM
exhaustion on large documents. While the page numbering fix resolves
correctness issues, scalability remains a concern.

### Proposed Architecture

**Pipeline-Driven Chunking with Explicit Resource Management:**

1. **Authoritative chunk planning**: Accept page-range specifications
from upstream pipeline as the single source of truth. The parser should
be a stateless worker that processes assigned chunks without making
independent pagination decisions.

2. **Granular memory lifecycle**:
   ```python
   for chunk_spec in chunk_plan:
       # Load only chunk_spec.pages into __images__
       page_images = load_page_range(chunk_spec.start, chunk_spec.end)
       
       # Process with offset tracking
       results = process_chunk(page_images, offset=chunk_spec.start)
       
       # Explicit cleanup before next iteration
       del page_images, page_chars, layout_intermediates
       gc.collect()  # Force collection of large objects
   ```

3. **Persistent lightweight state**: Keep model instances (layout
detector, OCR engine), document metadata (outlines, PDF structure), and
configuration across chunks to avoid reinitialization overhead (~2-5s
per chunk for model loading).

4. **Adaptive fallback**: Provide `max_pages_per_chunk` (default: 50)
only when pipeline doesn't supply a plan. Never exceed
pipeline-specified ranges to maintain predictable memory bounds.

5. **Optional: Dynamic budgeting**: Expose a memory budget parameter
that adjusts chunk size based on observed image dimensions and format
(e.g., reduce chunk size for high-DPI scanned documents).

### Benefits

- **Predictable memory footprint**: RAM usage bounded by `chunk_size ×
avg_page_size` rather than total document size
- **Horizontal scalability**: Enables parallel chunk processing across
workers
- **Failure isolation**: Page extraction errors affect only current
chunk, not entire document
- **Cloud-friendly**: Works within container memory limits (e.g., 2-4GB
per worker)

### Trade-offs

- **Increased I/O**: Re-opening PDF for each chunk vs. keeping file
handle (mitigated by page-range seeks)
- **Complexity**: Requires careful offset tracking and stateful
coordination between pipeline and parser
- **Warmup cost**: Model initialization overhead amortized across chunks
(acceptable for documents >100 pages)

### Implementation Priority

This optimization should be **deferred to a separate PR** after the
current correctness fix is merged, as:
1. It requires broader architectural changes across the pipeline
2. Current fix is critical for correctness and can be backported
3. Memory optimization needs comprehensive benchmarking on
representative document corpus


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 14:58:37 +08:00
Ahmad Intisar
184388879d feat: Add disable_password_login configuration to support SSO-only authentication (#13151)
### What problem does this PR solve?

Enterprise deployments that use an external Identity Provider (e.g.,
Microsoft Entra ID, Okta, Keycloak) need the ability to enforce SSO-only
authentication by hiding the email/password login form. Currently, the
login page always shows the password form alongside OAuth buttons, with
no way to disable it.

This PR adds a `disable_password_login` configuration option under the
existing `authentication` section in `service_conf.yaml`. When set to
`true`, the login page only displays configured OAuth/SSO buttons and
hides the email/password form, "Remember me" checkbox, and "Sign up"
link.

The flag can be set via:
- `service_conf.yaml` (`authentication.disable_password_login: true`)
- Environment variable (`DISABLE_PASSWORD_LOGIN=true`)

Default behavior is unchanged (`false`).

### Behavior

| `disable_password_login` | OAuth configured | Result |
|---|---|---|
| `false` (default) | No | Standard email/password form |
| `false` | Yes | Email/password form + SSO buttons below |
| `true` | Yes | **SSO buttons only** (no form, no sign up link) |
| `true` | No | Empty card (admin should configure OAuth first) |

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Files changed (5)

1. `docker/service_conf.yaml.template` — added `disable_password_login:
false` under authentication
2. `common/settings.py` — added `DISABLE_PASSWORD_LOGIN` global variable
and loader in `init_settings()`
3. `common/config_utils.py` — fixed `TypeError` in `show_configs()` when
authentication section contains non-dict values (e.g., booleans)
4. `api/apps/system_app.py` — exposed `disablePasswordLogin` flag in
`/config` endpoint
5. `web/src/pages/login/index.tsx` — conditionally render password form
based on config flag; OAuth buttons always render when channels exist

---------

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-03-02 14:06:03 +08:00
Magicbook1108
daec36e935 Fix: add soft limit for graph rag size (#13252)
### What problem does this PR solve?

Fix: add soft limit for graph rag size #13258 Q2

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 14:02:36 +08:00
huber
8a6b5ced6b fix: add missing chunk_data column to OceanBase schema migration (#13306)
### What problem does this PR solve?

When using OceanBase as the document storage engine, parsing and
inserting chunks with chunk_data (e.g., table parser row data) fails
with the following error:
```
[ERROR][Exception]: Insert chunk error: ['Unconsumed column names: chunk_data']
This happens because the chunk_data column was recently introduced but was omitted from the EXTRA_COLUMNS list in 
rag/utils/ob_conn.py
```
As a result, the automatic schema migration for existing OceanBase
tables does not append the missing chunk_data column, causing the
underlying pyobvector or SQLAlchemy to raise an unconsumed column names
error during data insertion.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### What is the solution?
Added column_chunk_data to the EXTRA_COLUMNS list in 
```
rag/utils/ob_conn.py
```
This ensures that the OceanBase connection wrapper can correctly detect
the missing column and automatically alter existing chunk tables to
include the chunk_data field during initialization.
2026-03-02 13:25:11 +08:00
Magicbook1108
f0dd12289c Feat: add preprocess parameters for ingestion pipeline (#13300)
### What problem does this PR solve?
Feat: add preprocess parameters for ingestion pipeline

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 13:18:57 +08:00
Yihang Wang
7fc97da610 security: Adopt Jinja2 SandboxedEnvironment for template rendering. (#13305) 2026-03-02 13:17:29 +08:00
Idriss Sbaaoui
860c4bd0bb Feat: UI testing automation with playwright (#12749)
### What problem does this PR solve?

This PR helps automate the testing of the ui interface using pytest
Playwright

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test automation infrastructure

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-02 13:04:08 +08:00
Attili-sys
21bc1ab7ec Feature rtl support (#13118)
### What problem does this PR solve?

This PR adds comprehensive **Right-to-Left (RTL) language support**,
primarily targeting Arabic and other RTL scripts (Hebrew, Persian, Urdu,
etc.).

Previously, RTL content had multiple rendering issues:

- Incorrect sentence splitting for Arabic punctuation in citation logic
- Misaligned text in chat messages and markdown components  
- Improper positioning of blockquotes and “think” sections  
- Incorrect table alignment  
- Citation placement ambiguity in RTL prompts  
- UI layout inconsistencies when mixing LTR and RTL text  

This PR introduces backend and frontend improvements to properly detect,
render, and style RTL content while preserving existing LTR behavior.

#### Backend
- Updated sentence boundary regex in `rag/nlp/search.py` to include
Arabic punctuation:
  - `،` (comma)
  - `؛` (semicolon)
  - `؟` (question mark)
  - `۔` (Arabic full stop)
- Ensures citation insertion works correctly in RTL sentences.
- Updated citation prompt instructions to clarify citation placement
rules for RTL languages.

#### Frontend
- Introduced a new utility: `text-direction.ts`
  - Detects text direction based on Unicode ranges.
  - Supports Arabic, Hebrew, Syriac, Thaana, and related scripts.
  - Provides `getDirAttribute()` for automatic `dir` assignment.

- Applied dynamic `dir` attributes across:
  - Markdown rendering
  - Chat messages
  - Search results
  - Tables
  - Hover cards and reference popovers

- Added proper RTL styling in LESS:
  - Text alignment adjustments
  - Blockquote border flipping
  - Section indentation correction
  - Table direction switching
  - Use of `<bdi>` for figure labels to prevent bidirectional conflicts

#### DevOps / Environment
- Added Windows backend launch script with retry handling.
- Updated dependency metadata.
- Adjusted development-only React debugging behavior.

---

### Type of change

- [x] Bug Fix (non-breaking change which fixes RTL rendering and
citation issues)
- [x] New Feature (non-breaking change which adds RTL detection and
dynamic direction handling)

---------

Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Ahmad Intisar <168020872+ahmadintisar@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-02 13:03:44 +08:00
balibabu
a897aedea9 Feat: Modify the form styles for retrieval and conditional operators. (#13299)
### What problem does this PR solve?

Feat: Modify the form styles for retrieval and conditional operators.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 12:05:27 +08:00
chanx
0cdddea59a feat: pipeline add preprocess (#13302)
### What problem does this PR solve?

feat: pipeline add preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:50:48 +08:00
balibabu
cf3d3c7c89 Feat: When exporting the agent DSL, the tailkey, password, and history fields need to be cleared. #13281 (#13282)
### What problem does this PR solve?
Feat: When exporting the agent DSL, the tailkey, password, and history
fields need to be cleared. #13281

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:41:38 +08:00
dependabot[bot]
b956ad180c Build(deps): Bump pypdf from 6.7.3 to 6.7.4 (#13298)
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.7.3 to 6.7.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/releases">pypdf's
releases</a>.</em></p>
<blockquote>
<h2>Version 6.7.4, 2026-02-27</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md">pypdf's
changelog</a>.</em></p>
<blockquote>
<h2>Version 6.7.4, 2026-02-27</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)</li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)</li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1650bc31e8"><code>1650bc3</code></a>
REL: 6.7.4</li>
<li><a
href="f309c60037"><code>f309c60</code></a>
SEC: Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)</li>
<li><a
href="993f052748"><code>993f052</code></a>
DEV: Bump actions/upload-artifact from 6 to 7 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3662">#3662</a>)</li>
<li><a
href="a3c996bffc"><code>a3c996b</code></a>
DEV: Bump actions/download-artifact from 7 to 8 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3663">#3663</a>)</li>
<li><a
href="37de32022e"><code>37de320</code></a>
ROB: Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)</li>
<li>See full diff in <a
href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypdf&package-manager=uv&previous-version=6.7.3&new-version=6.7.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:32:12 +08:00
Idriss Sbaaoui
9d78d3ddb1 Tests: fix failling http in CI (#13301)
### What problem does this PR solve?
test_doc_sdk_routes_unit had two flaky/incorrect branch assumptions:

1. parse/stop_parsing production logic gates on doc.run, but tests used
progress, causing branch mismatch and unintended fallthrough into
mutation/DB paths.
2. stop_parsing invalid-state test asserted an outdated message
fragment, making the contract brittle.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 10:44:33 +08:00
Jimmy Ben Klieve
7e0dd906f2 refactor: update admin ui (#13280)
### What problem does this PR solve?

Update for Admin UI:
- Update file picker input in **Registration whitelist** > **Import from
Excel** modal
- Modify DOM structure of **Sandbox Settings** and move several
hardcoded texts into translation files

### Type of change

- [x] Refactoring
2026-02-28 19:21:51 +08:00
Idriss Sbaaoui
e62552d482 Added some React IDs for playwright e2e tests (#13265)
### What problem does this PR solve?

Necessary ids for implementing the new testing suite with playwright for
UI

### Type of change

- [x] Other (please describe): Testing IDs

Co-authored-by: Liu An <asiro@qq.com>
2026-02-28 15:13:47 +08:00
Magicbook1108
1027916bfe Fix: inconsistent state handling for multi-user single-canvas access (#13267)
### What problem does this PR solve?

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/1db7412e-4554-44bc-84ba-16421949aacc"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-02-28 15:09:21 +08:00
Yongteng Lei
c91e803a38 Fix: close detached PIL image on JPEG save failure in encode_image (#13278)
### What problem does this PR solve?

Properly close detached PIL image on JPEG save failure in encode_image.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 14:43:35 +08:00
天海蒼灆
983150b936 Fix (api): fix the document parsing status check logic (#12504)
### What problem does this PR solve?
When the original code terminates the parsing task halfway, the progress
may not be 0 or 1, which will result in the inability to call the
interface to parse again

-Change the document parsing progress check to task status check, and
use TaskStatus.RUNNING.value to judge
-Update the condition judgment for stopping parsing documents, and check
whether the task is running instead


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 14:38:55 +08:00
Jin Hai
32ec950ca8 Fix create / drop chat session syntax (#13279)
### What problem does this PR solve?

This pull request refactors the chat session creation and deletion logic
in both the parser and client code to use unique session IDs instead of
session names. It also updates the corresponding command syntax and
payloads, ensuring more robust and unambiguous session management.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 14:18:21 +08:00
Jin Hai
d9d4825079 Add chat sessions related command (#13268)
### What problem does this PR solve?

1. Create / Drop / List chat sessions
2. Chat with LLM and datasets

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 12:52:45 +08:00
Jin Hai
54094771a3 Fix streaming chat on web API (#13275)
### What problem does this PR solve?

This pull request makes a small but important fix to how streaming
requests are handled in the `completion` endpoint of
`conversation_app.py`. The main change ensures that the `stream`
argument is not passed twice, which could cause errors.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 12:16:38 +08:00
Yongteng Lei
0110151e12 Fix: document remove race condition (#13242)
### What problem does this PR solve?

Fix document remove race condition.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 11:23:24 +08:00
eviaaaaa
fa71f8d0c7 refactor(word): lazy-load DOCX images to reduce peak memory without changing output (#13233)
**Summary**
This PR tackles a significant memory bottleneck when processing
image-heavy Word documents. Previously, our pipeline eagerly decoded
DOCX images into `PIL.Image` objects, which caused high peak memory
usage. To solve this, I've introduced a **lazy-loading approach**:
images are now stored as raw blobs and only decoded exactly when and
where they are consumed.

This successfully reduces the memory footprint while keeping the parsing
output completely identical to before.

**What's Changed**
Instead of a dry file-by-file list, here is the logical breakdown of the
updates:

* **The Core Abstraction (`lazy_image.py`)**: Introduced `LazyDocxImage`
along with helper APIs to handle lazy decoding, image-type checks, and
NumPy compatibility. It also supports `.close()` and detached PIL access
to ensure safe lifecycle management and prevent memory leaks.
* **Pipeline Integration (`naive.py`, `figure_parser.py`, etc.)**:
Updated the general DOCX picture extraction to return these new lazy
images. Downstream consumers (like the figure/VLM flow and base64
encoding paths) now decode images right at the use site using detached
PIL instances, avoiding shared-instance side effects.
* **Compatibility Hooks (`operators.py`, `book.py`, etc.)**: Added
necessary compatibility conversions so these lazy images flow smoothly
through existing merging, filtering, and presentation steps without
breaking.

**Scope & What is Intentionally Left Out**
To keep this PR focused, I have restricted these changes strictly to the
**general Word pipeline** and its downstream consumers.
The `QA` and `manual` Word parsing pipelines are explicitly **not
modified** in this PR. They can be safely migrated to this new lazy-load
model in a subsequent, standalone PR.

**Design Considerations**
I briefly considered adding image compression during processing, but
decided against it to avoid any potential quality degradation in the
derived outputs. I also held off on a massive pipeline re-architecture
to avoid overly invasive changes right now.

**Validation & Testing**
I've tested this to ensure no regressions:

* Compared identical DOCX inputs before and after this branch: chunk
counts, extracted text, table HTML, and image descriptions match
perfectly.
* **Confirmed a noticeable drop in peak memory usage when processing
image-dense documents.** For a 30MB Word document containing 243 1080p
screenshots, memory consumption is reduced by approximately 1.5GB.

**Breaking Changes**
None.
2026-02-28 11:22:31 +08:00
SFL79
4f0c892b32 feat(ui): add individual model delete buttons across all providers (#13271)
### What problem does this PR solve?

Added the option to delete models individually from providers.
For additional context, see
[issue-13184](https://github.com/infiniflow/ragflow/issues/13184)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Note: when deleting a selected model, it leaves the full model name as
text as seen here:
<img width="676" height="90" alt="image"
src="https://github.com/user-attachments/assets/c11c7c1b-3f2a-4119-b20c-bb8148a8ad16"
/>

If attempting to use ragflow with that deleted model, ragflow will throw
an unauthorized model error as expected.
I left it like that on purpose, so it's easier for the user to
understand what he deleted and that he needs to replace it with another
model.

Co-authored-by: Shahar Flumin <shahar@Shahars-MacBook-Air.local>
2026-02-28 10:51:39 +08:00
Yesid Cano Castro
d1afcc9e71 feat(seafile): add library and directory sync scope support (#13153)
### What problem does this PR solve?

The SeaFile connector currently synchronises the entire account — every
library
visible to the authenticated user. This is impractical for users who
only need
a subset of their data indexed, especially on large SeaFile instances
with many
shared libraries.

This PR introduces granular sync scope support, allowing users to choose
between
syncing their entire account, a single library, or a specific directory
within a
library. It also adds support for SeaFile library-scoped API tokens
(`/api/v2.1/via-repo-token/` endpoints), enabling tighter access control
without
exposing account-level credentials.


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Test

```
from seafile_connector import SeaFileConnector
import logging
import os

logging.basicConfig(level=logging.DEBUG)

URL = os.environ.get("SEAFILE_URL", "https://seafile.example.com")
TOKEN = os.environ.get("SEAFILE_TOKEN", "")
REPO_ID = os.environ.get("SEAFILE_REPO_ID", "")
SYNC_PATH = os.environ.get("SEAFILE_SYNC_PATH", "/Documents")
REPO_TOKEN = os.environ.get("SEAFILE_REPO_TOKEN", "")

def _test_scope(scope, repo_id=None, sync_path=None):
    print(f"\n{'='*50}")
    print(f"Testing scope: {scope}")
    print(f"{'='*50}")

    creds = {"seafile_token": TOKEN} if TOKEN else {}
    if REPO_TOKEN and scope in ("library", "directory"):
        creds["repo_token"] = REPO_TOKEN

    connector = SeaFileConnector(
        seafile_url=URL,
        batch_size=5,
        sync_scope=scope,
        include_shared = False,
        repo_id=repo_id,
        sync_path=sync_path,
    )
    connector.load_credentials(creds)
    connector.validate_connector_settings()

    count = 0
    for batch in connector.load_from_state():
        for doc in batch:
            count += 1
            print(f"  [{count}] {doc.semantic_identifier} "
                  f"({doc.size_bytes} bytes, {doc.extension})")

    print(f"\n-> {scope} scope: {count} document(s) found.\n")

# 1. Account scope
if TOKEN:
    _test_scope("account")
else:
    print("\nSkipping account scope (set SEAFILE_TOKEN)")

# 2. Library scope
if REPO_ID and (TOKEN or REPO_TOKEN):
    _test_scope("library", repo_id=REPO_ID)
else:
    print("\nSkipping library scope (set SEAFILE_REPO_ID + token)")

# 3. Directory scope
if REPO_ID and SYNC_PATH and (TOKEN or REPO_TOKEN):
    _test_scope("directory", repo_id=REPO_ID, sync_path=SYNC_PATH)
else:
    print("\nSkipping directory scope (set SEAFILE_REPO_ID + SEAFILE_SYNC_PATH + token)")
```
2026-02-28 10:24:28 +08:00
Stephen Hu
aec2ef4232 refactor:improve tts model's codes (#13137)
### What problem does this PR solve?

improve tts model's codes

### Type of change

- [x] Refactoring
2026-02-28 10:18:00 +08:00
Stephen Hu
9577753c10 Refactor: improve the logic about docling parser extract box (#13215)
### What problem does this PR solve?
 improve the logic about docling parser extract box

### Type of change
- [x] Refactoring
2026-02-28 10:05:24 +08:00
chanx
510ff89661 Fix: remove unused files (#13232)
### What problem does this PR solve?

Fix: remove unused files

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 23:05:40 +08:00
Jimmy Ben Klieve
c0823e8d6d refactor: update chat ui (#13269)
### What problem does this PR solve?

Update **Chat** UI:
- Align to the design.
- Update `<AudioButton>` visualizer logic.
- Fix keyboard navigation issue.

### Type of change

- [x] Refactoring
2026-02-27 22:26:19 +08:00
Enes Delibalta
4e48aba5c4 fix: update DoclingParser return type hint (#13243)
### What problem does this PR solve?

The _transfer_to_sections method was throwing a type hint violation
because it occasionally returns 3-item tuples instead of 2. Adjusted to
list[tuple[str, ...]] to prevent runtime crashes.

Error: 

20:53:21 Page(1~10): [ERROR]Internal server error while chunking:
Method
deepdoc.parser.docling_parser.DoclingParser._transfer_to_sections()
return [(1. JIRA Nasıl Kullanılır?, text,
@@1\t70.8\t194.9\t70.9\t85.5##), (1.1. Proje O...##)] violates type
hint list[tuple[str, str]], as list index
15 item tuple tuple (Gelen ekran
üzerinden alanları isterlerine göre doldurduğunuz taktirde Create
düğmesi i...##) length 3 != 2.
20:53:21 [ERROR][Exception]: Method
deepdoc.parser.docling_parser.DoclingParser._transfer_to_sections()
return [('1. JIRA Nasıl Kullanılır?', 'text',
'@@1\t70.8\t194.9\t70.9\t85.5##'), ('1.1. Proje O...##')] violates
type hint list[tuple[str, str]], as list index
15 item tuple tuple ('Gelen ekran
üzerinden alanları isterlerine göre doldurduğunuz taktirde Create
düğmesi i...##') length 3 != 2.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Enes Delibalta <enes.delibalta@pentanom.com>
2026-02-27 20:13:50 +08:00
Yuxing Deng
51b180d991 fix: adding GPUStack chat model requires v1 suffix (#13237)
### What problem does this PR solve?

Refer to issue: #13236
The base url for GPUStack chat model requires `/v1` suffix. For the
other model type like `Embedding` or `Rerank`, the `/v1` suffix is not
required and will be appended in code.
So keep the same logic for chat model as other model type.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 20:13:07 +08:00
as-ondewo
194e076e26 Fix: init superuser can create duplicate users (#13221)
### What problem does this PR solve?

This PR fixes 2 bugs related to RAGFlow's init superuser functionality.

#### Bug 1

When the RAGFlow server was started with the `--init-superuser` option
it would always create a new admin user even if it already exists
resulting in duplicate users.

To fix this, I added an additional check before create the superuser and
added the *unique* constraint to the email column of the database, to
mitigate potential TOCTOU race conditions. Since existing databases
could contain duplicate emails I added email de-duplication to the
database migration.

#### Bug 2

When the RAGFlow server was started with the `--init-superuser` option
but without configured default LLM and embedding models it would fail to
start because the `init_superuser` function would always make test
request to the models even if they were not set.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 19:55:51 +08:00
balibabu
6d0100ca67 Fix: The output content of the multi-model comparison will disappear. #13227 (#13241)
### What problem does this PR solve?

Fix: The output content of the multi-model comparison will disappear.
#13227
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 19:18:40 +08:00
balibabu
861ebfc6e1 Feat: Make the embedded page of chat compatible with mobile devices. (#13262)
### What problem does this PR solve?
Feat: Make the embedded page of chat compatible with mobile devices.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-27 19:17:41 +08:00
avianion
5f53fbe0f1 feat: Add Avian as an LLM provider (#13256)
### What problem does this PR solve?

This PR adds [Avian](https://avian.io) as a new LLM provider to RAGFlow.
Avian provides an OpenAI-compatible API with competitive pricing,
offering access to models like DeepSeek V3.2, Kimi K2.5, GLM-5, and
MiniMax M2.5.

**Provider details:**
- API Base URL: `https://api.avian.io/v1`
- Auth: Bearer token via API key
- OpenAI-compatible (chat completions, streaming, function calling)
- Models:
  - `deepseek/deepseek-v3.2` — 164K context, $0.26/$0.38 per 1M tokens
  - `moonshotai/kimi-k2.5` — 131K context, $0.45/$2.20 per 1M tokens
  - `z-ai/glm-5` — 131K context, $0.30/$2.55 per 1M tokens
  - `minimax/minimax-m2.5` — 1M context, $0.30/$1.10 per 1M tokens

**Changes:**
- `rag/llm/chat_model.py` — Add `AvianChat` class extending `Base`
- `rag/llm/__init__.py` — Register in `SupportedLiteLLMProvider`,
`FACTORY_DEFAULT_BASE_URL`, `LITELLM_PROVIDER_PREFIX`
- `conf/llm_factories.json` — Add Avian factory with model definitions
- `web/src/constants/llm.ts` — Add to `LLMFactory` enum, `IconMap`,
`APIMapUrl`
- `web/src/components/svg-icon.tsx` — Register SVG icon
- `web/src/assets/svg/llm/avian.svg` — Provider icon
- `docs/references/supported_models.mdx` — Add to supported models table

This follows the same pattern as other OpenAI-compatible providers
(e.g., n1n #12680, TokenPony).

cc @KevinHuSh @JinHai-CN

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-02-27 17:36:55 +08:00
6ba3i
bb59a27e55 Doc : Add french Readme (#13254)
### What problem does this PR solve?

Add fench Readme

### Type of change

- [x] Documentation Update
2026-02-27 11:34:13 +08:00
qinling0210
8b6d363a98 Use pagination in _search_metadata (#13238)
### What problem does this PR solve?

Fix [#13210](https://github.com/infiniflow/ragflow/issues/13210)

Remove limit in _search_metadata, use pagination in _search_metadata.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 11:24:49 +08:00
Jin Hai
a1549c0fdc Fix UI (#13239)
### What problem does this PR solve?

This pull request makes a minor update to the English locale strings for
the Table of Contents toggle buttons, changing the labels from "Show
TOC"/"Hide TOC" to "Show content"/"Hide content" for improved clarity.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-26 19:21:08 +08:00
Magicbook1108
c03c537bf8 Feat: optimize gmail/google-drive (#13230)
### What problem does this PR solve?

Feat: optimize gmail/google-drive

Now:
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/0c4b6044-7209-4c4f-ac0c-32070b79daf7"
/>
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/406f93d8-9b0f-4f5a-b8bb-3936990f558c"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 19:19:40 +08:00
6ba3i
22c4d72891 tests: improve RAGFlow coverage based on Codecov report (#13219)
### What problem does this PR solve?

Codecov’s coverage report shows that several RAGFlow code paths are
currently untested or under-tested. This makes it easier for regressions
to slip in during refactors and feature work.
This PR adds targeted automated tests to cover the files and branches
highlighted by Codecov, improving confidence in core behavior while
keeping runtime functionality unchanged.

### Type of change

- [x] Other (please describe): Test coverage improvement (adds/extends
unit and integration tests to address Codecov-reported gaps)
2026-02-26 19:03:26 +08:00
Magicbook1108
1aa49a11f0 Feat: support AWS SES smtp (#13195)
### What problem does this PR solve?

Support AWS SES smtp #13179

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 13:49:53 +08:00
writinwaters
74dc43406f Docs: After careful consideration, the RAGFlow team decided to hold o… (#13226)
…ff publishing this guide.

### What problem does this PR solve?

Removed failsure mode checklist per your request. @JinHai-CN 

### Type of change


- [x] Documentation Update
2026-02-26 12:39:58 +08:00
balibabu
d2dd0b7e50 Fix: The agent is embedded in the webpage; interrupting its operation will redirect to the login page. #12697 (#13224)
### What problem does this PR solve?

Fix: The agent is embedded in the webpage; interrupting its operation
will redirect to the login page. #12697

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 12:39:28 +08:00
chanx
8bce212284 Fix: error in retrieval testing page (#13225)
### What problem does this PR solve?

Fix: error in retrieval testing page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 12:39:09 +08:00
Angel98518
024edba1b8 fix(web): prevent duplicate i18n languageChanged listeners (#13218)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring
2026-02-26 10:45:50 +08:00
PandaMan
d43aebe701 Fix/13142 auto metadata (#13217)
### What problem does this PR solve?

Close #13142

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 10:25:48 +08:00
Angel98518
b54260bcd7 fix(web): correct initial chat variable enabled state (#13214)
## Summary

Fixes the initial enabled/disabled state of chat variable checkboxes by
correcting a helper function that previously always returned .

## Problem

 in  had two  statements:



Because of the early , the function always returned , so all chat
variable checkboxes were initially disabled regardless of the field.
This also made the helper inconsistent with , which enables all fields
by default except .

## Fix

Update  to use the same condition as :



This ensures:
- All chat variable checkboxes are enabled by default
-  remains the only field disabled by default
- Behavior is consistent between the helper and the checkbox map
initialization in .

No API or backend changes are involved; this is a small, isolated
frontend bugfix.
2026-02-26 10:25:14 +08:00
Magicbook1108
158503a1aa Feat: optimize ingestion pipeline with preprocess (#13211)
### What problem does this PR solve?

Feat: optimize ingestion pipeline with preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 10:24:13 +08:00
PSBigBig × MiniPS
b7eca981d4 docs: add RAG failure modes checklist guide (refs #13138) (#13204)
### What problem does this PR solve?

This PR adds a new guide: **"RAG failure modes checklist"**.

RAG systems often fail in ways that are not immediately visible from a
single metric like accuracy or latency. In practice, debugging
production RAG applications requires identifying recurring failure
patterns across retrieval, routing, evaluation, and deployment stages.

This guide introduces a structured, pattern-based checklist (P01–P12) to
help users interpret traces, evaluation results, and dataset behavior
within RAGFlow. The goal is to provide a practical way to classify
incidents (e.g., retrieval hallucination, chunking issues, index
staleness, routing misalignment) and reason about minimal structural
fixes rather than ad-hoc prompt changes.

The change is documentation-only and does not modify any code or
configuration.

Refs #13138


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-02-25 19:35:15 +08:00
writinwaters
f9e0eb38ec Refact: Updated ingestion pipeline UI. (#13216)
### What problem does this PR solve?

Updated ingestion pipeline-specific UI tips.

### Type of change

- [x] Refactoring
2026-02-25 19:29:04 +08:00
6ba3i
38011f2c16 tests: improve RAGFlow coverage based on Codecov report (#13200)
### What problem does this PR solve?

Codecov’s coverage report shows that several RAGFlow code paths are
currently untested or under-tested. This makes it easier for regressions
to slip in during refactors and feature work.
This PR adds targeted automated tests to cover the files and branches
highlighted by Codecov, improving confidence in core behavior while
keeping runtime functionality unchanged.

### Type of change

- [x] Other (please describe): Test coverage improvement (adds/extends
unit and integration tests to address Codecov-reported gaps)
2026-02-25 19:12:11 +08:00
balibabu
2a5ddf064d Fix: Note component text area does not resize with component #13065 (#13212)
### What problem does this PR solve?

Fix: Note component text area does not resize with component #13065

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 17:24:07 +08:00
Jimmy Ben Klieve
220e611e33 refactor: ux improvements for variable picker in prompt editor (#13213)
### What problem does this PR solve?

User experience enhancement for variable picker in prompt editor:

- Add case-insensitive string search for variables.
- Add basic keyboard navigation in variable picker:
   - Hit <kbd>UpArrow</kbd> and <kbd>DownArrow</kbd> for navigating.
- Hit <kbd>Tab</kbd> or <kbd>Enter</kbd> for selecting focused item into
editor.
- Fix unexpectedly inserting invalid variable into editor by hitting
<kbd>Tab</kbd>.

_Note: you still need to pick variables inside secondary menu (agent
structured output, etc.) by using your pointing device. May finish these
later._

### Type of change

- [x] Refactoring
2026-02-25 17:22:48 +08:00
He Wang
394ff16b66 fix: OceanBase metadata not returned in document list API (#13209)
### What problem does this PR solve?

Fix #13144.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 15:29:17 +08:00
Phives
4ceb668d40 feat(api/utils): Harden file_utils for robustness and edge cases (#12915)
## Summary
Improves robustness and edge-case handling in `api.utils.file_utils` to
avoid crashes, DoS/OOM risks, and timeouts when processing user-provided
filenames, paths, and file blobs.

## Changes

### Resource limits & timeouts
- **`MAX_BLOB_SIZE_THUMBNAIL`** (50 MiB) and **`MAX_BLOB_SIZE_PDF`**
(100 MiB) to reject oversized inputs before thumbnail/PDF processing.
- **`GHOSTSCRIPT_TIMEOUT_SEC`** (120 s) for
`repair_pdf_with_ghostscript` subprocess to avoid hangs on malicious or
broken PDFs.

### `filename_type`
- Handles `None`, empty string, non-string (e.g. int/list), and
path-only input via new **`_normalize_filename_for_type()`**.
- Uses basename for type detection (e.g. `a/b/c.pdf` → PDF).
- Enforces **`FILE_NAME_LEN_LIMIT`**; invalid input returns
`FileType.OTHER`.

### `thumbnail_img`
- Rejects `None`/empty/oversized blob and invalid filename; returns
`None` instead of raising.
- Wraps PDF, image, and PPT handling in try/except so corrupt or
malformed files return `None`.
- Ensures PDF has pages and PPT has slides before use.
- Normalizes PIL image mode (RGBA/P/LA → RGB) for safe PNG export.

### `repair_pdf_with_ghostscript`
- Handles `None`/empty input; skips repair when input size exceeds
limit.
- Uses `subprocess.run(..., timeout=GHOSTSCRIPT_TIMEOUT_SEC)` and
catches `TimeoutExpired`.
- Returns original bytes when Ghostscript output is empty.

### `read_potential_broken_pdf`
- `None` → `b""`; non–sequence-like (no `len`) → `b""`; empty → return
as-is.
- Oversized blob returned as-is (no repair) to avoid DoS.

### `sanitize_path`
- Explicit `None` and non-string check; strips whitespace before
normalizing.

## Testing
- **`test/unit_test/utils/test_api_file_utils.py`** added with 36 unit
tests covering the above behavior (filename_type, sanitize_path,
read_potential_broken_pdf, thumbnail_img, thumbnail,
repair_pdf_with_ghostscript, constants).
- All tests pass.

---------

Co-authored-by: Gittensor Miner <miner@gittensor.io>
2026-02-25 14:34:47 +08:00
PentaFDevs
8ad47bf242 feat: add 'Open in new tab' button for agents (#13044)
- Add new button in agent management dropdown to open agent in new tab
- Implement token-based authentication for shared agent access
- Add translations for 9 languages (en, zh, zh-tw, de, fr, it, ru,
pt-br, vi)
- Keep existing 'Embed into webpage' functionality intact

### What problem does this PR solve?

This allows users to open agents in a separate tab to work in background
while continuing to use other parts of the application.

<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/ca1719c8-2f00-4570-a730-1321fa0bfd57"
/>
<img width="254" height="222" alt="image"
src="https://github.com/user-attachments/assets/b3dd6d9f-b7e7-46b0-83e7-f0ea86e7b156"
/>
<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/e94e99f9-9039-43f7-b2d9-862b9448630c"
/>

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-25 13:39:02 +08:00
Yao Wei
cf6fd6f115 fix: When using OceanBase as storage, the list_chunk sorting is abnormal. #13198 (#13208)
Actual behavior
When using OceanBase as storage, the list_chunk sorting is abnormal. The
following is the SQL statement.

SELECT id, content_with_weight, important_kwd, question_kwd, img_id,
available_int, position_int, doc_type_kwd, create_timestamp_flt,
create_time, array_to_string(page_num_int, ',') AS page_num_int_sort,
array_to_string(top_int, ',') AS top_int_sort FROM
rag_store_284250730805059584 WHERE doc_id = '' AND kb_id IN ('') ORDER
BY page_num_int_sort ASC, top_int_sort ASC, create_timestamp_flt DESC
LIMIT 0, 20

<img width="1610" height="740" alt="image"
src="https://github.com/user-attachments/assets/84e14c30-a97f-4e8f-8c8c-6ccac915d97d"
/>

Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
2026-02-25 13:36:18 +08:00
Ray Zhang
cbe64402db feat(migration): support docker compose -p project name for backup/restore (#13191)
### What problem does this PR solve?

When users start RAGFlow with `docker compose -p <alias>`, Docker
creates volumes prefixed with the alias (e.g., `myproject_mysql_data`).
The migration script (`docker/migration.sh`) previously hardcoded the
`docker_` prefix in volume names, causing backup/restore to silently
skip all volumes for any non-default project name.

This PR adds a `-p <project_name>` option so the script correctly
targets volumes regardless of the Docker Compose project name used.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes

- Add `-p <project_name>` flag (default: `docker`) for specifying Docker
Compose project name
- Build volume names dynamically: `${project_name}_${base_name}`
- Update help text with new option documentation and examples
- Show project-aware `docker compose` commands in error messages
- Fix deprecated `docker-compose` to `docker compose` in hints
- Use dynamic step count instead of hardcoded `4`
- Fully backward compatible — existing usage without `-p` works
unchanged

### Usage

```bash
# Existing usage (unchanged)
./migration.sh backup
./migration.sh restore my_backup

# New: custom project name
./migration.sh -p myproject backup
./migration.sh -p myproject restore my_backup
```
2026-02-25 13:18:47 +08:00
Yongteng Lei
2bf2abfdbc Fix: authorization bypass (IDOR) in /v1/document/web_crawl (#13203)
### What problem does this PR solve?

Fix authorization bypass (IDOR) in `/v1/document/web_crawl` allows
Cross-Tenant Dataset Modification.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 12:59:41 +08:00
Ahmad Intisar
99d1c9725c Bug mysql connector empty content resolved: Semantic ID Issue (#13206)
The RDBMS (MySQL/PostgreSQL) connector generates document filenames
using the first 100 characters of the content column
(semantic_identifier). When the content contains newline characters
(\n), the resulting filename includes those newlines — for example:
Category: غير صحيح كليًا\nTitle: تفنيد حقائق....txt
RAGFlow's filename_type() function uses re.match(r".*\.txt$", filename)
to detect file types, but .* does not match newline characters by
default in Python regex. This causes the regex to fail, returning
FileType.OTHER, which triggers:
pythonraise RuntimeError("This type of file has not been supported
yet!")
As a result, all documents synced via the MySQL/PostgreSQL connector are
silently discarded. The sync logs report success (e.g., "399 docs
synchronized"), but zero documents actually appear in the dataset. This
is the root cause of issue #13001.
Root cause trace:

rdbms_connector.py → _row_to_document() sets semantic_identifier from
raw content (may contain \n)
connector_service.py → duplicate_and_parse() uses semantic_identifier as
the filename
file_service.py → upload_document() calls filename_type(filename)
file_utils.py → filename_type() regex .*\.txt$ fails on newlines →
returns FileType.OTHER
upload_document() raises "This type of file has not been supported yet!"

Fix: Sanitize the semantic_identifier in _row_to_document() by replacing
newlines and carriage returns with spaces before truncating to 100
characters.
Relates to: #13001, #12817
Type of change

 Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-02-25 12:55:04 +08:00
Yongteng Lei
72b89304c1 Fix: LFI vulnerability in document parsing API (#13196)
### What problem does this PR solve?

Fix LFI vulnerability in document parsing API.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 09:47:39 +08:00
PandaMan
f4cbdc3a3b fix(api): MinIO health check use dynamic scheme and verify (Closes #13159 and #13158) (#13197)
## Summary

Fixes MinIO SSL/TLS support in two places: the MinIO **client**
connection and the **health check** used by the Admin/Service Health
dashboard. Both now respect the `secure` and `verify` settings from the
MinIO configuration.

Closes #13158
Closes #13159

---

## Problem

**#13158 – MinIO client:** The client in `rag/utils/minio_conn.py` was
hardcoded with `secure=False`, so RAGFlow could not connect to MinIO
over HTTPS even when `secure: true` was set in config. There was also no
way to disable certificate verification for self-signed certs.

**#13159 – MinIO health check:** In `api/utils/health_utils.py`, the
MinIO liveness check always used `http://` for the health URL. When
MinIO was configured with SSL, the health check failed and the dashboard
showed "timeout" even though MinIO was reachable over HTTPS.

---

## Solution

### MinIO client (`rag/utils/minio_conn.py`)

- Read `MINIO.secure` (default `false`) and pass it into the `Minio()`
constructor so HTTPS is used when configured.
- Add `_build_minio_http_client()` that reads `MINIO.verify` (default
`true`). When `verify` is false, return an `urllib3.PoolManager` with
`cert_reqs=ssl.CERT_NONE` and pass it as `http_client` to `Minio()` so
self-signed certificates are accepted.
- Support string values for `secure` and `verify` (e.g. `"true"`,
`"false"`).

### MinIO health check (`api/utils/health_utils.py`)

- Add `_minio_scheme_and_verify()` to derive URL scheme (http/https) and
the `verify` flag from `MINIO.secure` and `MINIO.verify`.
- Update `check_minio_alive()` to use the correct scheme, pass `verify`
into `requests.get(..., verify=verify)`, and use `timeout=10`.

### Config template (`docker/service_conf.yaml.template`)

- Add commented optional MinIO keys `secure` and `verify` (and env vars
`MINIO_SECURE`, `MINIO_VERIFY`) so deployers know they can enable HTTPS
and optional cert verification.

### Tests

- **`test/unit_test/utils/test_health_utils_minio.py`** – Tests for
`_minio_scheme_and_verify()` and `check_minio_alive()` (scheme, verify,
status codes, timeout, errors).
- **`test/unit_test/utils/test_minio_conn_ssl.py`** – Tests for
`_build_minio_http_client()` (verify true/false/missing, string values,
`CERT_NONE` when verify is false).

---

## Testing

- Unit tests added/updated as above; run with the project's test runner.
- Manually: configure MinIO with HTTPS and `secure: true` (and
optionally `verify: false` for self-signed); confirm client operations
work and the Service Health dashboard shows MinIO as alive instead of
timeout.
2026-02-25 09:47:12 +08:00
Yongteng Lei
c292d617ca Fix: stored XSS via HTML File upload and inline Rendering in file get (#13202)
### What problem does this PR solve?

Fix stored XSS via HTML file upload and inline rendering in
/v1/file/get/<id>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 09:46:48 +08:00
ksufer
5a8fa7cf31 Fix #13119: Use email.utils to fix IMAP parsing for names with commas (#13120)
## Type of Change
- [x] Bug fix

## Description
Closes #13119

The current IMAP connector uses `split(',')` to parse email headers,
which crashes when a sender's display name contains a comma inside
quotes (e.g., `"Doe, John" <john@example.com>`).

This PR replaces the manual string splitting with Python's standard
`email.utils.getaddresses`. This correctly handles RFC 5322 quoted
strings and prevents the `RuntimeError: Expected a singular address`.

## Checklist
- [x] I have checked the code and it works as expected.

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-24 19:18:55 +08:00
as-ondewo
0a7c520579 Fix: empty response from OpenAI chat completion endpoint (#13166)
### What problem does this PR solve?

When using a chat assistant that has a hardcoded `empty_response`, that
response was not returned correctly in streaming mode when no
information is found in the knowledge base. In this case only one
response with `"content": null` was yielded. If `"references": true`,
then the `empty_response` is still put into the `final_content` so there
is technically some content returned, but when `"references": false` no
content at all is returned.

I update the OpenAI chat completion endpoint to yield an additional
response with the `empty_response` in the content.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:18:12 +08:00
Magicbook1108
5de92e57d3 Fix: 'None None' in log (#13192)
### What problem does this PR solve?

Fix: 'None None' in log

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:15:20 +08:00
Magicbook1108
46dec98f52 Fix: Chat/Agent embedded page (#13199)
### What problem does this PR solve?

Fix: Chat/Agent embedded page #13190

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:14:24 +08:00
tuandang-diag
d89ad8b79d fix: handle null response in LLM and improve JSON parsing in agent (#13187)
Fixes AttributeError in _remove_reasoning_content() when LLM returns
None, and improves JSON parsing regex for markdown code fences in
agent_with_tools.py
2026-02-24 13:15:09 +08:00
Lynn
67befc9119 Fix: add back MCP tool custom header (#13188)
### What problem does this PR solve?

Add back custom header when use MCP.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 13:14:21 +08:00
chanx
7db2fb200c Fix: Metadata mult-selected display error (#13189)
### What problem does this PR solve?

Fix: Metadata mult-selected display error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 13:10:32 +08:00
PandaMan
f462a9d85a fix(web): prevent LaTeX from being cut at \right] or \big) in agent o… (#13155)
### What problem does this PR solve?

- Use negative lookbehind (?<![a-zA-Z]) so \] and \) inside commands
(e.g. \right], \big)) are not treated as block/inline delimiters
- Use greedy matching to capture up to the last valid delimiter, fixing
truncated formulas (e.g. C_{seq}(y|x) = \frac{1}{|y|} ...)
- Add unit tests for preprocessLaTeX

Closes #13134


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 12:30:06 +08:00
Bradley Boveinis
3280772934 fix(helm): exclude password keys from env range loop to prevent duplicate YAML keys (#13136)
## Summary

- Fix duplicate YAML mapping keys in `helm/templates/env.yaml` that
cause deployment failures with strict YAML parsers

## Problem

The `range` loop in `env.yaml` iterates over all `.Values.env` keys and
emits them into a Secret. The exclusion filter skips host/port/user
keys, but does **not** skip password keys (`MYSQL_PASSWORD`,
`REDIS_PASSWORD`, `MINIO_PASSWORD`, `ELASTIC_PASSWORD`,
`OPENSEARCH_PASSWORD`). These same keys are then explicitly defined
again later in the template, producing duplicate YAML mapping keys.

Go's `yaml.v3` (used by Flux's helm-controller for post-rendering)
rejects duplicate keys per the YAML spec:

```
Helm install failed: yaml: unmarshal errors:
  mapping key "MINIO_PASSWORD" already defined
  mapping key "MYSQL_PASSWORD" already defined
  mapping key "REDIS_PASSWORD" already defined
```

Plain `helm install` does not surface this because Helm's internal
parser (`yaml.v2`) silently accepts duplicate keys (last value wins).

## Fix

Add password keys to the exclusion filter on line 12 so they are only
emitted by their explicit definitions later in the template.

Note: `MINIO_ROOT_USER` is intentionally **not** excluded — it is only
emitted by the range loop and has no explicit definition elsewhere.
Excluding it causes MinIO to crash with `Missing credential environment
variable, "MINIO_ROOT_USER"`.

## Test plan

- [ ] Deploy with Flux helm-controller (uses yaml.v3) — no duplicate key
errors
- [ ] Verify all passwords are present in the rendered Secret
- [ ] Verify `MINIO_ROOT_USER` is present in the rendered Secret
- [ ] Test with `DOC_ENGINE=elasticsearch` (ELASTIC_PASSWORD)
- [ ] Test with `DOC_ENGINE=opensearch` (OPENSEARCH_PASSWORD)

Fixes #13135
2026-02-24 11:09:31 +08:00
as-ondewo
91d1a81937 fix: error during admin tenant creation when using Postgres (#13164)
### What problem does this PR solve?

This fixes the bug described in #13130. When starting RAGFlow with
Postgres the admin tenant create failed because the rerank model was not
set.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 10:57:31 +08:00
chanx
59e9e77061 fix: Add admin proxy (#13186)
### What problem does this PR solve?

fix: Add admin proxy

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 10:29:58 +08:00
Magicbook1108
98e1d5aa5c Refact: switch from google-generativeai to google-genai (#13140)
### What problem does this PR solve?

Refact: switch from oogle-generativeai to google-genai  #13132
Refact: commnet out unused pywencai.

### Type of change

- [x] Refactoring
2026-02-24 10:28:33 +08:00
zagnaan
45aa3a0e89 fix(compose): use official opensearch image instead of hub.icert.top mirror (#13131)
### What problem does this PR solve?

The Docker Compose configuration was using hub.icert.top as the registry
for the OpenSearch image. That registry is not reachable in our
environment, which causes podman pull and docker compose pull to fail
with a connection refused error. As a result, the application cannot
start because the OpenSearch image cannot be downloaded.

This PR updates the image reference to use the official Docker Hub image
(opensearchproject/opensearch:2.19.1) instead of the hub.icert.top
mirror. After this change, the image pulls successfully and the services
start as expected.


![photo_2026-02-12_15-11-56](https://github.com/user-attachments/assets/6db736a5-b701-450f-96c1-9c23f092c3ab)


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Shynggys Samarkhanov <shynggys.samarkhanov@nixs.com>
2026-02-24 09:50:02 +08:00
Trifon
ce71d87867 Add Bulgarian language support (#13147)
### What problem does this PR solve?

RAGFlow supports 12 UI languages but does not include Bulgarian. This PR
adds Bulgarian (`bg` / `Български`) as the 13th supported language,
covering the full UI translation (2001 keys across all 26 sections) and
OCR/PDF parser language mapping.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Changes

- **`web/src/constants/common.ts`** — Registered Bulgarian in all 5
language data structures (`LanguageList`, `LanguageMap`,
`LanguageAbbreviation` enum, `LanguageAbbreviationMap`,
`LanguageTranslationMap`)
- **`web/src/locales/config.ts`** — Added lazy-loading dynamic import
for the `bg` locale
- **`web/src/locales/bg.ts`** *(new)* — Full Bulgarian translation file
with all 26 sections, matching the English source (`en.ts`). All
interpolation placeholders, HTML tags, and technical terms are preserved
as-is
- **`deepdoc/parser/mineru_parser.py`** — Mapped `'Bulgarian'` to
`'cyrillic'` in `LANGUAGE_TO_MINERU_MAP` for OCR/PDF parser support

### How it works

The language selector automatically picks up the new entry. When a user
selects "Български", the translation bundle is lazy-loaded on demand.
The preference is persisted to the database and localStorage across
sessions.
2026-02-14 16:51:29 +08:00
chanx
f612d2254d Refactor: i18n language pack for on-demand import (#13139)
### What problem does this PR solve?

Refactor:   i18n language pack for on-demand import

### Type of change

- [x] Refactoring

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-13 18:42:48 +08:00
chanx
f2a1d59c36 Refactor: Remove ant design component (#13143)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring
2026-02-13 18:40:41 +08:00
writinwaters
bc9ed24a85 Docs: Updated v0.24.0 release notes. (#13129)
### What problem does this PR solve?

Added more details to v0.24.0 release notes.

### Type of change

- [x] Documentation Update
2026-02-12 20:14:05 +08:00
Levi
6d6c54db19 fix(metadata): handle unhashable list values in metadata split (#13116)
### What problem does this PR solve?

This PR fixes missing metadata on documents synced from the Moodle
connector, especially for **Book** modules.

Background:
- Moodle Book metadata includes fields like `chapters`, which is a
`list[dict]`.
- During metadata normalization in
`DocMetadataService._split_combined_values`, list deduplication used
`dict.fromkeys(...)`.
- `dict.fromkeys(...)` fails for unhashable values (like `dict`),
causing metadata update to fail.
- Result: documents were imported, but metadata was not saved for
affected module types (notably Books).

What this PR changes:
- Replaces hash-based list deduplication with `dedupe_list(...)`, which
safely handles unhashable list items while preserving order.
- This allows Book metadata (and other complex list metadata) to be
persisted correctly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Contribution during my time at RAGcon GmbH.
2026-02-12 19:48:51 +08:00
chanx
b922a5cbdf Fix: replace session page icons and fix nested list search functionality in filters (#13127)
### What problem does this PR solve?

Fix: replace session page icons and fix nested list search functionality
in filters

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 19:48:35 +08:00
Ahmad Intisar
5885f150ab fix: register WebDAVConnector in data_source __init__.py (#13121)
What problem does this PR solve?
The sync_data_source.py module imports WebDAVConnector from
common.data_source, but WebDAVConnector was never registered in the
package's __init__.py. This causes an ImportError at startup, crashing
the data sync service:
ImportError: cannot import name 'WebDAVConnector' from
'common.data_source'
The webdav_connector.py file already exists in the common/data_source/
directory — it just wasn't exported. This PR adds the import and
registers it in __all__.
Type of change

 Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-02-12 16:05:58 +08:00
Magicbook1108
e89fd686e2 Improve: optimize file name (with path) in box container. (#13124)
### What problem does this PR solve?

Refact: optimize file name (with path) in box container. 

### Type of change

- [x] Performance Improvement

<img width="2357" height="1258" alt="image"
src="https://github.com/user-attachments/assets/f4c5c90b-d885-4514-b7bc-f17ab62b045f"
/>
2026-02-12 15:40:55 +08:00
疯癫
e72291bc9a Fix the bug where the mcp service tools/list does not return knowledge base IDs information. (#13123)
Fix the issue where the server-side parameter validation fails when the
id parameter is None in the asynchronous list_datasets method.

### What problem does this PR solve?

Fix the issue where the server-side parameter validation fails when the
id parameter is None in the asynchronous list_datasets method.

### Type of change

- [√ ] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 15:40:15 +08:00
Lynn
6e7bcf58bc Refactor: split message apis to gateway and service (#13126)
### What problem does this PR solve?

Split message apis to gateway and service

### Type of change

- [x] Refactoring
2026-02-12 14:43:52 +08:00
chanx
7210178620 Fix: Bugs fixed (#13109) (#13122)
### What problem does this PR solve?

Fix: Bugs fixed (#13109)
- chat pdf preview error
- data source add box error
- change route next-chat -> chat , next-search->search ...

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 13:42:12 +08:00
Liu An
65ebc06956 Refa: test file location for better organization (#13107)
### What problem does this PR solve?

Renamed test/unit/test_delete_query_construction.py to
test/unit_test/common/test_delete_query_construction.py to align with
the project's directory structure and improve test categorization.

### Type of change

- [x] Refactoring
2026-02-12 10:15:09 +08:00
Lynn
30d5fc1a07 Refactor: split memory API into gateway and service layers (#13111)
### What problem does this PR solve?

Decouple the memory API into a gateway layer (for routing/param parse)
and a service layer (for business logic).

### Type of change

- [x] Refactoring
2026-02-12 10:11:50 +08:00
Levi
4b50b8c579 Fix: persist SSO auth token on root route loader (#12784)
### What problem does this PR solve?

This PR fixes SSO/OIDC login persistence after the Vite migration
#12568. Because wrappers are ignored by React Router, the OAuth callback
never stored the auth token in localStorage, causing auth to only work
while ?auth= stayed in the URL. We move that logic into a route loader
and remove the Bearer prefix for the signed token so the backend accepts
it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Contribution during my time at RAGcon GmbH.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-02-12 10:09:35 +08:00
TheoG
67937a668e Fix graphrag extraction (#13113)
### What problem does this PR solve?

Fix error when extracting the graph.
A string is expected, but a tuple was provided.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-11 20:11:56 +08:00
writinwaters
57dc25f150 Docs: Updated v0.24.0 release notes (#13115)
### What problem does this PR solve?

Updated v0.24.0 release notes.

### Type of change

- [x] Documentation Update
2026-02-11 18:08:56 +08:00
writinwaters
a9272c26eb Docs: Updated sandbox reference (#13114)
### What problem does this PR solve?

Updated sandbox reference.

### Type of change

- [x] Documentation Update
2026-02-11 17:58:08 +08:00
Kevin Hu
88920d23e6 Refa: Change aliyun repo. (#13103)
### Type of change

- [x] Refactoring
2026-02-11 11:21:03 +08:00
Jim Smith
7029b8ca81 Fix: Make time_utils tests timezone-independent (#13100)
## Summary
- Replace hardcoded CST (UTC+8) expected values in `test_time_utils.py`
with dynamically computed local-time expectations using
`time.localtime()` and `time.mktime()`
- Tests previously failed in any timezone other than UTC+8; they now
pass regardless of the system's local timezone

## Test plan
- [x] `uv run pytest test/unit_test/ -v` — 317 passed, 25 skipped

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Jim Smith <jhsmith0@me.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:51:53 +08:00
Ahmad Intisar
99ed8e759d Fix: Correct Gemini embedding model name in llm_factories.json (#13051)
## Problem
   RAGFlow was using incorrect model names for Google Gemini embeddings:
   - `embedding-001` (missing `gemini-` prefix)
   - `text-embedding-004` (OpenAI model name, not Gemini)
   
   This caused API errors when users tried to use Gemini embeddings.
   
   ## Solution
- Updated `conf/llm_factories.json` to use the correct model name:
`gemini-embedding-001`
   - Removed the incorrect `text-embedding-004` entry
- Added volume mount in `docker-compose.yml` to ensure config changes
persist
   
   ## Testing
Tested with a valid Gemini API key and confirmed embeddings now work
correctly.

## Changes
- Modified `conf/llm_factories.json`
- Modified `docker/docker-compose.yml`

---------

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-11 09:49:48 +08:00
Magicbook1108
109441628b Fix: upload image files (#13071)
### What problem does this PR solve?

Fix: upload image files

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-11 09:47:33 +08:00
writinwaters
630f05b8a1 Docs: Added v0.24.0 release notes (#13096)
### What problem does this PR solve?

Added v0.24.0 release notes.

### Type of change


- [x] Documentation Update
2026-02-10 17:38:27 +08:00
Liu An
392ec99651 Docs: Update version references to v0.24.0 in READMEs and docs (#13095)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.23.1 to v0.24.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-02-10 17:24:03 +08:00
Lynn
d938b47877 Fix: judge table name prefix before migrate (#13094)
### What problem does this PR solve?

Judge table created with current infinity mapping before migrate db.
#13089

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-10 17:05:34 +08:00
akie
6f785e06a4 Fix issue #13084 (#13088)
When match_expressions contains coroutine objects (from GraphRAG's
Dealer.get_vector()), the code cannot identify this type because it only
checks for MatchTextExpr, MatchDenseExpr, or FusionExpr.

As a result:

score_func remains initialized as an empty string ""
This empty string is appended to the output list
The output list is passed to Infinity SDK's table_instance.output()
method
Infinity's SQL parser (via sqlglot) fails to parse the empty string,
throwing a ParseError
2026-02-10 17:04:45 +08:00
writinwaters
4341d81e29 Refact: Updated UI tips. (#13093)
### What problem does this PR solve?

Updated UI tips.

### Type of change


- [x] Refactoring
2026-02-10 16:25:56 +08:00
Yongteng Lei
48591cb1e7 Refa: boost OpenAI-compatible reranker UX (#13087)
### What problem does this PR solve?

boost OpenAI-compatible reranker UX.

### Type of change

- [x] Refactoring
2026-02-10 16:13:21 +08:00
chanx
586a9e05a7 Fix: Memory log style (#13090)
### What problem does this PR solve?

Fix: Memory log style

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-10 16:12:59 +08:00
balibabu
126ec85ef6 Feat: Hide log button (#13085)
### What problem does this PR solve?

Feat: Hide log button
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-10 14:05:17 +08:00
chanx
4186821de8 Fix: Bugs fixed (#13086)
### What problem does this PR solve?

Fix: Bugs fixed
- metadata icon error
- search page's image not display

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-10 14:04:50 +08:00
balibabu
141157f529 Feat: Translation page index. (#13083)
### What problem does this PR solve?

Feat: Translation page index.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-10 10:37:38 +08:00
writinwaters
9c39ac11c6 Docs: Replaced TOC Enhance with Page Index. (#13075)
### What problem does this PR solve?

Replaced TOC Enhance with Page Index.

### Type of change

- [x] Documentation Update
2026-02-09 20:13:34 +08:00
balibabu
db37804f10 Feat: Add Explore page (#13043)
### What problem does this PR solve?

Feat: Add Explore page

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-09 19:53:51 +08:00
chanx
8ad7339448 Fix: Add authentication validation to the document API interface for embedded pages. (#13078)
### What problem does this PR solve?

Fix: Add authentication validation to the document API interface for
embedded pages and modify the document display styles.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 19:53:24 +08:00
Kevin Hu
9bc16d8df2 Fix: agent files issue, (#13067)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 19:52:52 +08:00
Neel Harsola
a2dda8fb70 Fix: enable chat input resizing (#12998)
## Summary
- add resizable support to shared textarea component
- enable vertical resizing for chat inputs in chat and share surfaces
- preserve autosize behavior while honoring manual resize height

## Test plan
- not run (not requested)

Fixes #12803

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-09 19:33:05 +08:00
qinling0210
4bc622b409 Fix parameter of calling self.dataStore.get() and warning info during parser (#13068)
### What problem does this PR solve?

Fix parameter of calling self.dataStore.get() and warning info during
parser

https://github.com/infiniflow/ragflow/issues/13036

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 17:56:59 +08:00
Magicbook1108
25a32c198d Fix: gemini model names (#13073)
### What problem does this PR solve?

Fix: gemini model names #13053


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 17:51:22 +08:00
6ba3i
fabbfcab90 Fix: failing p3 test for SDK/HTTP APIs (#13062)
### What problem does this PR solve?

Adjust highlight parsing, add row-count SQL override, tweak retrieval
thresholding, and update tests with engine-aware skips/utilities.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 14:56:10 +08:00
Yingfeng
ba95167e13 Clean directories (#13061)
### Type of change

- [x] Documentation Update
2026-02-09 12:08:12 +08:00
Stephen Hu
2ee39f64fe Refactor: improve ppt shape order logic (#13054)
### What problem does this PR solve?

improve ppt shape order logic

### Type of change

- [x] Refactoring
2026-02-09 11:59:24 +08:00
eviaaaaa
0b55d1e860 fix: remove 10-item display limit in Agent Canvas configuration tables (#13049)
## Description

This PR fixes an issue where the input and variable configuration tables
in the Agent Canvas (specifically for **Begin**, **UserFillUp**, and
**Invoke** nodes) were truncated at 10 items.

**Root Cause:**
The tables utilized `@tanstack/react-table` with
`getPaginationRowModel()` enabled. Since the default page size is 10 and
no pagination UI controls were implemented, users could not access items
beyond the 10th row.

**Solution:**
Removed `getPaginationRowModel` from the table configurations. These
lists (inputs/variables) are typically short, so rendering all items in
a single scrollable view is the intended behavior.

* Modified `query-table.tsx`
* Modified `variable-table.tsx`


## How to verify

1. Create a **Begin**, **UserFillUp**, or **Invoke** node in the Agent
Canvas.
2. Add more than 10 input items or variables.
3. Verify that all items are visible in the list and not truncated at
the 10th item.

## What kind of change does this PR introduce?

* [x] Bugfix
2026-02-09 10:43:50 +08:00
Kevin Hu
e51a40fdfc Fix: launch an agent. (#13039)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 10:08:36 +08:00
chanx
8217ccced8 Fix: whyDidYouRender error (#13040)
### What problem does this PR solve?

Fix: whyDidYouRender error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-09 09:57:52 +08:00
Clint-chan
38289084a8 Chore/upgrade dashscope to 1.25.11 (#13007)
## Description
  Upgrade dashscope package to support text-embedding-v4 model.

  ## Changes
  - Update dashscope version from 1.20.11 to 1.25.11 in pyproject.toml

  ## Reason
The text-embedding-v4 model requires dashscope >= 1.25.0 to function
properly. This upgrade ensures compatibility with the latest embedding
models.

Co-authored-by: Clint-chan <Clint-chan@users.noreply.github.com>
2026-02-06 19:06:41 +08:00
Yongteng Lei
279b01a028 Feat: MCP host mode supports STREAMABLE-HTTP endpoint (#13037)
### What problem does this PR solve?

MCP host mode supports STREAMABLE-HTTP endpoint

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-06 16:22:43 +08:00
chanx
c130ac0f88 Fix: Lazy loading adds a loading state to the page (#13038)
### What problem does this PR solve?

Fix: Lazy loading adds a loading state to the page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-06 16:20:52 +08:00
Magicbook1108
301ed76aa4 Fix: task cancel (#13034)
### What problem does this PR solve?

Fix: task cancel #11745 
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-06 14:48:24 +08:00
MkDev11
13a6545e48 fix(rdbms): use brackets around field names to preserve distinction after chunking (#13010)
Fix RDBMS field separation after chunking by wrapping field names in
brackets (【field】: value). This ensures fields remain distinguishable
even when TxtParser strips newline delimiters during chunk merging.

Closes  #13001

Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
2026-02-06 14:44:58 +08:00
yH
5333e764fc fix: optimize Excel row counting for files with abnormal max_row (#13018)
### What problem does this PR solve?

Some Excel files have abnormal `max_row` metadata (e.g.,
`max_row=1,048,534` with only 300 actual data rows). This causes:
- `row_number()` returns incorrect count, creating 350+ tasks instead of
1
- `list(ws.rows)` iterates through millions of empty rows, causing
system hang

This PR uses binary search to find the actual last row with data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Performance Improvement

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-06 14:43:52 +08:00
chanx
00c392e633 Fix: dataset page enter key to save (#13035)
### What problem does this PR solve?

Fix dataset page enter key to save 
Fix the warnings and optimize the code.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-06 14:42:16 +08:00
Magicbook1108
4b0d65f089 Fix: correct llm_id for graphrag (#13032)
### What problem does this PR solve?

Fix: correct llm_id for graphrag #13030

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-06 14:05:32 +08:00
Yingfeng
6a17e8cc85 Update basics (#13033)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2026-02-06 13:15:33 +08:00
Clint-chan
a68c56def7 fix: ensure all metadata filters are processed in AND logic (#13019)
### What problem does this PR solve?

Bug: When a filter key doesn't exist in metas or has no matching values,
the filter was skipped entirely, causing AND logic to fail.

Example:
- Filter 1: meeting_series = '宏观早8点' (matches doc1, doc2, doc3)
- Filter 2: date = '2026-03-05' (no matches)
- Expected: [] (AND should return empty)
- Actual: [doc1, doc2, doc3] (Filter 2 was skipped)

Root cause:
Old logic iterated metas.items() first, then filters. If a filter's key
wasn't in metas, it was never processed.

Fix:
Iterate filters first, then look up in metas. If key not found, treat as
no match (empty result), which correctly applies AND logic.

Changes:
- Changed loop order from 'for k in metas: for f in filters' to 'for f
in filters: if f.key in metas'
- Explicitly handle missing keys as empty results


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Clint-chan <Clint-chan@users.noreply.github.com>
2026-02-06 12:57:27 +08:00
LIRUI YU
0586d5148d fixed vulnerabilities CVE-2025-53859 & CVE-2025-23419 (#13016)
### What problem does this PR solve?

Fixed vulnerabilities CVE-2025-53859 & CVE-2025-23419 by updating nginx
to 1.29.5-1~noble

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
<img width="709" height="54" alt="image"
src="https://github.com/user-attachments/assets/d8c3518f-bca4-4314-a85c-1aed1678f72e"
/>
2026-02-06 12:55:06 +08:00
Stephen Hu
11703d957d Refactor: Improve Picture.py resource usage (#13011)
### What problem does this PR solve?

Improve Picture.py resource usage

### Type of change


- [x] Refactoring
2026-02-06 09:50:53 +08:00
Kevin Hu
1262533b74 Feat: support verify to set llm key and boost bigrams. (#12980)
#12863

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-05 19:19:09 +08:00
balibabu
bbd8ba64a1 Feat: Control interface documentation directory display and hiding (#13008)
### What problem does this PR solve?

Feat: Control interface documentation directory display and hiding

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-05 16:59:20 +08:00
Neel Harsola
1a85d2f8de Fix: prevent streaming message width collapse (#12999)
## Summary
- keep assistant message containers stretched to available width
- avoid width collapse during streaming by allowing flex items to shrink

## Test plan
- not run (not requested)

Fixes #12985

Made with [Cursor](https://cursor.com)

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-02-05 15:58:55 +08:00
chanx
2a7dca6fc9 Fix: parser bug (#13014)
…, clicking "Parse" will still ask if you want to clear the chunks of
the already parsed files.

### What problem does this PR solve?

Fix: After selecting all and then unchecking the already parsed files,
clicking "Parse" will still ask if you want to clear the chunks of the
already parsed files.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-05 15:57:38 +08:00
Magicbook1108
0a08fc7b07 Fix: example code in session.py (#13004)
### What problem does this PR solve?

Fix: example code in session.py #12950

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Levi <stupse-tipp0j@icloud.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-02-05 15:56:58 +08:00
Magicbook1108
75b2d482e2 Fix: ingestion pipeline (#13012)
### What problem does this PR solve?

Fix ingestion pipeline
Only 1 file is acceptable for ingestion pipeline.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-05 15:55:41 +08:00
chanx
89fdb1d498 Feat: Add model verify (#13005)
### What problem does this PR solve?

Feat: Add model verify

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-02-05 15:53:20 +08:00
Clint-chan
90b726c988 fix: support date comparison operators (>=, <=, >, <) in metadata filtering (#12982)
## Description

This PR fixes the issue where date metadata conditions with comparison
operators (`>=`, `<=`, `>`, `<`) did not work correctly in the
`/api/v1/retrieval` endpoint.

  ## Problem

  When using metadata conditions like:
  ```json
  {
    "metadata_condition": {
      "conditions": [
        {
          "name": "date",
          "comparison_operator": ">=",
          "value": "2027-01-13"
        }
      ]
    }
  }

  The filtering did not work as expected because:
  1. Operators >= and <= were not mapped to internal symbols ≥ and ≤
2. Date strings like "2027-01-13" failed to parse with
ast.literal_eval()
  3. Non-standard date formats were incorrectly compared as strings

  Solution

  Changes in common/metadata_utils.py:

  1. Added operator mapping in convert_conditions():
    - >= → ≥
    - <= → ≤
    - != → ≠
  2. Implemented strict date format detection in meta_filter():
- Only processes dates in YYYY-MM-DD format (10 characters, properly
formatted)
- When query value is a date, only matches data in the same standard
format
    - Non-standard formats (e.g., "2026年1月13日", "2026-1-22") are skipped
  3. Maintained backward compatibility:
    - Numeric comparisons still work
    - String comparisons still work
    - Only affects date-formatted queries

  Testing

  All test cases pass (8/8):
  -  Date >= comparison
  -  Date > comparison
  -  Date < comparison
  -  Date <= comparison
  -  Date = comparison
  -  Date range queries
  -  Non-date string comparison (backward compatibility)
  -  Numeric comparison (backward compatibility)

  Example Usage

  {
    "dataset_ids": ["xxx"],
    "question": "test",
    "metadata_condition": {
      "conditions": [
        {
          "name": "date",
          "comparison_operator": ">=",
          "value": "2027-01-13"
        }
      ]
    }
  }

  Notes

  - Only supports standard YYYY-MM-DD format
- Non-standard date formats in data are treated as data quality issues
and will not match
  - Users should ensure their date metadata is in the correct format

---------

Co-authored-by: Clint-chan <Clint-chan@users.noreply.github.com>
2026-02-05 13:52:51 +08:00
Magicbook1108
1349e6b7d1 Fix: adressing style without a default value (#13009)
### What problem does this PR solve?

Fix: adressing style without a default value #12396 #11510

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-05 13:52:23 +08:00
Yongteng Lei
6361fc4b33 Feat: update stepfun list (#12991)
### What problem does this PR solve?

Update stepfun list.

Add TTS and Sequence2Text functionalities.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-05 12:47:04 +08:00
Levi
803b480f9c feat: Add optional document metadata in OpenAI-compatible response references (#12950)
### What problem does this PR solve?

This PR adds an opt‑in way to include document‑level metadata in
OpenAI‑compatible reference chunks. Until now, metadata could be used
for filtering but wasn’t returned in responses. The change enables
clients to show richer citations (author/year/source, etc.) while
keeping payload size and privacy under control via an explicit request
flag and optional field allowlist.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Contribution during my time at RAGcon GmbH.
2026-02-05 09:54:33 +08:00
writinwaters
2843570d8e Refact: Updated Agent template description. (#12995)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2026-02-05 09:50:44 +08:00
Yongteng Lei
3a86e7c224 Feat: support doubao-embedding-vision model (#12983)
### What problem does this PR solve?

Add support `doubao-embedding-vision` model.
`doubao-embedding-large-text` is deprecated.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-02-05 09:49:46 +08:00
balibabu
2ff2e72488 Fix: Fixed the issue where deleted images in the agent chat box would still be sent to the backend. (#12992)
### What problem does this PR solve?

Fix: Fixed the issue where deleted images in the agent chat box would
still be sent to the backend.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-05 09:49:01 +08:00
balibabu
2627a7f5a8 Feat: Move the reasoning field to the root of the payload in the completion interface. (#12990)
### What problem does this PR solve?


### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-04 19:21:49 +08:00
BitToby
4d4b5a978d feat: enable multi-file upload for chat and agent workflows (#12977)
### Closes: #12921 

### What problem does this PR solve?

Previously, multi-file upload was not working correctly across the
application:

- **Chat**: UI displayed "Upload max 5 files" but only the first file
was actually uploaded
- **Agent conversational mode**: Frontend sent multiple files but
backend only processed one
- **Agent task-mode file inputs**: Explicitly limited to single file
only

This PR enables proper multi-file upload support for both chat and agent
workflows, allowing users to upload and process multiple files (up to 5)
as the UI originally suggested.

**Changes:**
- `web/src/pages/next-chats/hooks/use-upload-file.ts`: Process all files
instead of only `files[0]`
- `api/apps/canvas_app.py`: Handle multiple files via
`files.getlist("file")`
- `web/src/pages/agent/debug-content/uploader.tsx`: Allow up to 5 files
with `multiple={true}`
- `agent/component/begin.py` & `fillup.py`: Support file arrays while
maintaining backward compatibility

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-04 18:03:21 +08:00
balibabu
ffdf19b27f Fix: Variables within multiple parentheses cannot be displayed correctly. #12987 (#12988)
### What problem does this PR solve?
Fix: Variables within multiple parentheses cannot be displayed
correctly. #12987

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-04 17:54:02 +08:00
Magicbook1108
a37d287fad Fix: pdf chunking / table rotation (#12981)
### What problem does this PR solve?

Fix: PDF chunking issue for single-page documents
Refactor: Change the default refresh frequency to 5
Fix: Add a 0-degree threshold; require other rotation angles to exceed
it by at least 0.2
Fix: Put connector name tips to correct place
Fix: incorrect example response in delete datasets.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-02-04 17:00:25 +08:00
Liu An
0470fc59b1 Docs: Update error message example in HTTP API reference (#12984)
### What problem does this PR solve?

Changed the error message example in the HTTP API reference
documentation from a duplicate dataset name error to a validation error
about string length requirements. This update reflects the current
behavior of the API when validation fails.

### Type of change

- [x] Documentation Update
2026-02-04 15:42:53 +08:00
MkDev11
6f31c5fed2 feat/add MySQL and PostgreSQL data source connectors (#12817)
### What problem does this PR solve?

This PR adds MySQL and PostgreSQL as data source connectors, allowing
users to import data directly from relational databases into RAGFlow for
RAG workflows.

Many users store their knowledge in databases (product catalogs,
documentation, FAQs, etc.) and currently have no way to sync this data
into RAGFlow without exporting to files first. This feature lets them
connect directly to their databases, run SQL queries, and automatically
create documents from the results.

Closes #763
Closes #11560

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### What this PR does

**New capabilities:**
- Connect to MySQL and PostgreSQL databases
- Run custom SQL queries to extract data
- Map database columns to document content (vectorized) and metadata
(searchable)
- Support incremental sync using a timestamp column
- Full frontend UI with connection form and tooltips

**Files changed:**

Backend:
- `common/constants.py` - Added MYSQL/POSTGRESQL to FileSource enum
- `common/data_source/config.py` - Added to DocumentSource enum
- `common/data_source/rdbms_connector.py` - New connector (368 lines)
- `common/data_source/__init__.py` - Exported the connector
- `rag/svr/sync_data_source.py` - Added MySQL and PostgreSQL sync
classes
- `pyproject.toml` - Added mysql-connector-python dependency

Frontend:
- `web/src/pages/user-setting/data-source/constant/index.tsx` - Form
fields
- `web/src/locales/en.ts` - English translations
- `web/src/assets/svg/data-source/mysql.svg` - MySQL icon
- `web/src/assets/svg/data-source/postgresql.svg` - PostgreSQL icon

### Testing done

Tested with MySQL 8.0 and PostgreSQL 16:
- Connection validation works correctly
- Full sync imports all query results as documents
- Incremental sync only fetches rows updated since last sync
- Custom SQL queries filter data as expected
- Invalid credentials show clear error messages
- Lint checks pass (`ruff check` returns no errors)

---------

Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
2026-02-04 10:14:32 +08:00
writinwaters
0ab02854d9 Refact: Updated UI tips (#12976)
### What problem does this PR solve?

Updated UI tips.

### Type of change

- [x] Refactoring
2026-02-04 09:48:59 +08:00
balibabu
414e261eda Fix: If the agent debug sheet contains too much content, some of it will not be displayed. #12974 (#12975)
### What problem does this PR solve?

Fix: If the agent debug sheet contains too much content, some of it will
not be displayed. #12974

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-04 09:48:28 +08:00
Carve_
7b230aadf4 chore(tests): move oceanbase peewee test under test/ and fix enum check (#12969)
### What problem does this PR solve?

This mistake was made by PR #12926 
This PR makes the OceanBase peewee unit test discoverable by the default
unit test runner/CI (by moving it under test/), so it’s included in the
unified unit test suite.
It also fixes `test_database_lock_enum_values` to correctly handle Enum
alias members (DatabaseLock uses the same value for MYSQL and
OCEANBASE).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Screenshots
The original `test_oceanbase_peewee.py` was placed under tests/, which
isn’t included in the default unit test runner’s testpaths, so it wasn’t
picked up by the unit test suite. So we need to move it to correct path.
<img width="670" height="540" alt="image"
src="https://github.com/user-attachments/assets/69d39346-450f-46dc-8965-29c3d7b32bc9"
/>

When using old version in `test_oceanbase_peewee.py`:
```
    def test_database_lock_enum_values(self):
        """Test DatabaseLock enum has all expected values."""
        expected = {'MYSQL', 'OCEANBASE', 'POSTGRES'}
        actual = {e.name for e in DatabaseLock}
        assert expected.issubset(actual), f"Missing: {expected - actual}"
```
The old check iterated Enum members, so alias values were skipped and
only `MYSQL/POSTGRES` were seen, making OCEANBASE appear missing.

<img width="1998" height="931" alt="65e2837f23b7b298980a410c7d5c2f09"
src="https://github.com/user-attachments/assets/d8e98c5a-2cfa-4182-ae35-a3ef03554a27"
/>

and new version uses `DatabaseLock.__members__` and passes:
<img width="2024" height="1170" alt="1aa8c6facb28d24149270fe1bc4a9dd9"
src="https://github.com/user-attachments/assets/d8688936-ccac-4a39-a389-23dc6f0fe276"
/>
2026-02-03 17:28:53 +08:00
qinling0210
205ae769bb Fix "metadata table not exists" (#12949)
### What problem does this PR solve?

Fix "metadata table not exists" when updating a meta data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 17:28:10 +08:00
He Wang
ff7afcbe5f feat: add OceanBase memory store (#12955)
### What problem does this PR solve?

Add OceanBase memory store and extracting base class `OBConnectionBase`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-02-03 16:46:17 +08:00
yun.kou
4d9a3a5739 fix(docs): fix #12963 , rename "total" field to "total_datasets" for clarity (#12967)
### What problem does this PR solve?

Update HTTP API reference to rename "total" field to

### Type of change

- [x] Bug Fix #12963

Co-authored-by: Yun.kou <yunkou@deepglint.com>
2026-02-03 15:40:17 +08:00
sunsui
c3f71e9ef9 Fix:Incorrect ingestion pipeline template (#12961)
### What problem does this PR solve?

This PR fixes an incorrect variable reference in the Advanced Ingestion
Pipeline template, which causes a runtime failure in the Auto Keywords
stage.

When creating a pipeline using the `advanced ingestion pipeline`
template, the **Auto Keywords** stage fails with the following error:
Can't find variable: 'Splitter:NineTiesSin@chunks'

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: sunsui <suisun@trip.com>
2026-02-03 15:39:32 +08:00
Lynn
32f9a87b2e Fix: default admin tenant (#12964)
### What problem does this PR solve?

Add tenant for default admin, and allow login to ragflow server as
default admin.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 15:37:36 +08:00
Magicbook1108
f11ca54e0e Fix: docx parser output consistent (#12965)
### What problem does this PR solve?

Fix: docx parser output consistent

> File "/home/bxy/ragflow/rag/flow/parser/parser.py", line 506, in _word
>     sections, tbls = docx_parser(name, binary=blob)
>     ^^^^^^^^^^^^^^
> ValueError: too many values to unpack (expected 2)
> 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 15:36:58 +08:00
Yesid Cano Castro
deeae8dba4 feat(connector): add Seafile as data source (#12945)
### What problem does this PR solve?
This PR adds **Seafile** as a new data source connector for RAGFlow.

[Seafile](https://www.seafile.com/) is an open-source, self-hosted file
sync and share platform widely used by enterprises, universities, and
organizations that require data sovereignty and privacy. Users who store
documents in Seafile currently have no way to index and search their
content through RAGFlow.

This connector enables RAGFlow users to:
- Connect to self-hosted Seafile servers via API token
- Index documents from personal and shared libraries
- Support incremental polling for updated files
- Seamlessly integrate Seafile-stored documents into their RAG pipelines


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
### Changes included

- `SeaFileConnector` implementing `LoadConnector` and `PollConnector`
interfaces
- Support for API token
- Recursive file traversal across libraries
- Time-based filtering for incremental updates
- Seafile logo (sourced from Simple Icons, CC0)
- Connector configuration and registration

### Testing

- Tested against self-hosted Seafile Community Edition
- Verified authentication (token)
- Verified document ingestion from personal and shared libraries
- Verified incremental polling with time filters
2026-02-03 13:42:05 +08:00
chanx
25bb2e1616 Fix:Optimize metadata and optimize the empty state style of the agent page. (#12960)
### What problem does this PR solve?
Fix:Optimize metadata and optimize the empty state style of the agent
page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 11:43:44 +08:00
Jimmy Ben Klieve
fafaaa26c3 feat: memory status (#12959)
### What problem does this PR solve?

Add memory status indicator and detail message dialog 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-03 11:16:18 +08:00
Philipp Heyken Soares
ad06c042c4 Support operator constraints in semi-automatic metadata filtering (#12956)
### What problem does this PR solve?

#### Summary
This PR enhances the Semi-automatic metadata filtering mode by allowing
users to explicitly pre-define operators (e.g., contains, =, >, etc.)
for selected metadata keys. While the LLM still dynamically extracts the
filter value from the user's query, it is now strictly constrained to
use the operator specified in the UI configuration.

Using this feature is optional. By default the operator selection is set
to "automatic" resulting in the LLM choosing the operator (as
presently).

#### Rationale & Use Case
This enhancement was driven by a concrete challenge I encountered while
working with technical documentation.
In my specific use case, I was trying to filter for software versions
within a technical manual. In this dataset, a single document chunk
often applies to multiple software versions. These versions are stored
as a combined string within the metadata for each chunk.

When using the standard semi-automatic filter, the LLM would
inconsistently choose between the contains and equals operators. When it
chose equals, it would exclude every chunk that applied to more than one
version, even if the version I was searching for was clearly included in
that metadata string. This led to incomplete and frustrating retrieval
results.

By extending the semi-automatic filter to allow pre-defining the
operator for a specific key, I was able to force the use of contains for
the version field. This change immediately led to significantly improved
and more reliable results in my case.

I believe this functionality will be equally useful for others dealing
with "tagged" or multi-value metadata where the relationship between the
query and the field is known, but the specific value needs to remain
dynamic.

#### Key Changes
##### Backend & Core Logic
- `common/metadata_utils.py`: Updated apply_meta_data_filter to support
a mixed data structure for semi_auto (handling both legacy string arrays
and the new object-based format {"key": "...", "op": "..."}).
- `rag/prompts/generator.py`: Extended gen_meta_filter to accept and
pass operator constraints to the LLM.
- `rag/prompts/meta_filter.md`: Updated the system prompt to instruct
the LLM to strictly respect provided operator constraints.

##### Frontend
- `web/src/components/metadata-filter/metadata-semi-auto-fields.tsx`:
Enhanced the UI to include an operator dropdown for each selected
metadata key, utilizing existing operator constants.
- `web/src/components/metadata-filter/index.tsx`: Updated the validation
schema to accommodate the new state structure.

#### Test Plan
- Backward Compatibility: Verified that existing semi-auto filters
stored as simple strings still function correctly.
- Prompt Verification: Confirmed that constraints are correctly rendered
in the LLM system prompt when specified.
- Added unit tests as
`test/unit_test/common/test_apply_semi_auto_meta_data_filter.py`
 - Manual End-to-End:
- Configured a "Semi-automatic" filter for a "Version" key with the
"contains" operator.
   - Asked a version-specific query.
   - Result
   
<img width="1173" height="704" alt="Screenshot 2026-02-02 145359"
src="https://github.com/user-attachments/assets/510a6a61-a231-4dc2-a7fe-cdfc07219132"
/>




### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: Philipp Heyken Soares <philipp.heyken-soares@am.ai>
2026-02-03 11:11:34 +08:00
zhanglei
7cbe8b5b53 feat: Add a custom header to the SDK for chatting with the agent. (#12430)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Liu An <asiro@qq.com>
2026-02-03 11:01:18 +08:00
Josh
aa8d0a36f1 Update default Docling version to 2.71.0 to resolve table parsing issues (#12952)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 10:24:51 +08:00
Stephen Hu
6c9ca45b30 Refactor: improve close for presentation (#12957)
### What problem does this PR solve?

improve close for presentation

### Type of change

- [x] Refactoring
2026-02-03 10:24:27 +08:00
Paul Y Hui
f028f74883 Fixed 12787 with syntax error in generated MySql json path expression (#12929)
### What problem does this PR solve?

Fixed 12787 with syntax error in generated MySql json path expression
https://github.com/infiniflow/ragflow/issues/12787 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

#### What was fixed:
- Changed line 237 in ob_conn.py from value_str = get_value_str(value)
if value else "" to value_str = get_value_str(value)
- This fixes the bug where falsy but valid values (0, False, "", [], {})
were being converted to empty strings, causing invalid SQL syntax
#### What was tested:
- Comprehensive unit tests covering all edge cases
- Regression tests specifically for the bug scenario

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-03 09:50:14 +08:00
writinwaters
59d7f3f456 Sandbox (#12951)
### What problem does this PR solve?

Proofread the Sandbox Specification document and moved it to a dedicated
folder outside of the original docs.

### Type of change


- [x] Documentation Update
2026-02-03 09:43:41 +08:00
Magicbook1108
7be3dacdaa Fix: custom delimeter in docx (#12946)
### What problem does this PR solve?

Fix: custom delimeter in docx

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-03 09:43:18 +08:00
eviaaaaa
2e5a18602b refactor: optimize agent list payload and improve multimodal detection logic (#12942)
## Description
This PR focuses on API performance optimization and refining the model
capability detection logic in the Agent/Canvas module.

### 1. Performance Optimization (Backend)
- **Changes**: Removed `cls.model.dsl` from query fields in
`UserCanvasService.get_by_tenant_ids`.
- **Reasoning**: The `dsl` object is large and unnecessary for the Agent
list view. Excluding it reduces the payload size of the
`/v1/canvas/list` API, leading to faster serialization and reduced
network latency.
- **Consistency**: Full DSL data remains accessible via the individual
`/v1/canvas/get/<id>` endpoint used in the detail view.

### 2. Multimodal Detection Refinement (Frontend)
- **Changes**: Replaced `model_type === LlmModelType.Image2text` with
`tags?.includes('IMAGE2TEXT')`.
- **Reasoning**: In RAGFlow, `model_type` defines the primary role of a
model (e.g., `chat`). However, many advanced Chat models are also
vision-capable. Since `model_type` is a single-value field, it cannot
represent these multiple capabilities.
- **Solution**: Utilizing the `tags` field (which supports multiple
attributes) to check for `IMAGE2TEXT` ensures that models like
`gpt-5.2-pro` correctly display multimodal input options.



## Type of Change
- [x] Bug fix (logic correction for multimodal detection)
- [x] Optimization (performance improvement for list API)

## Main Changes
- `api/db/services/canvas_service.py`: Optimized DB query by excluding
heavy DSL fields.
- `web/src/pages/agent/form/agent-form/index.tsx`: Enhanced capability
detection using the tags system.

## Verification
- [x] Verified Agent list loads faster with reduced response payload.
- [x] Confirmed that `chat` models with the `IMAGE2TEXT` tag now
correctly enable the multimodal input UI.
2026-02-02 17:35:54 +08:00
Magicbook1108
0121866ce4 Fix: pdf page_number error (#12938)
### What problem does this PR solve?
Fix: pdf page_number error #12937

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-02 17:35:00 +08:00
方程
8fc3986f70 fix(llm): Fix Gitee AI links and update the reranker model configuration (#12916)
### What problem does this PR solve?

Fix Gitee AI links and update the reranker model configuration

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: franco <1787003204@q.comq>
2026-02-02 13:43:16 +08:00
eviaaaaa
1a2d69edc4 feat: Implement legacy .ppt parsing via Tika (alternative to Aspose) (#12932)
## What problem does this PR solve?
This PR implements parsing support for legacy PowerPoint files (`.ppt`,
97-2003 format).
Currently, parsing these files fails because `python-pptx` **natively
lacks support** for the legacy OLE2 binary format.

## **Context:**
I originally using `aspose-slides` for this purpose. However, since
`aspose-slides` is **no longer a project dependency**, I implemented a
fallback mechanism using the existing `tika-server` to ensure
compatibility and stability.

## **Key Changes:**
- **Fallback Logic**: Modified `rag/app/presentation.py` to catch
`python-pptx` failures and automatically fall back to Tika parsing.
- **No New Dependencies**: Utilizes the `tika` service that is already
part of the RAGFlow stack.
- **Note**: Since Tika focuses on text extraction, this implementation
extracts text content but does not generate slide thumbnails .
## 🧪 Test / Verification Results

### 1. Before (The Issue)
I have verified the fix using a legacy `.ppt` file (`math(1).ppt`,
~8MB).
<img width="963" height="970" alt="image"
src="https://github.com/user-attachments/assets/468c4ba8-f90b-4d7b-b969-9c5f5e42c474"
/>

### 2. After (The Fix)
With this PR, the system detects the failure in python-pptx and
successfully falls back to Tika. The text is extracted correctly.
<img width="1467" height="1121" alt="image"
src="https://github.com/user-attachments/assets/fa0fed3b-b923-4c86-ba2c-24b3ce6ee7a6"
/>


**Type of change**
- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: evilhero <2278596667@qq.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-02-02 13:40:51 +08:00
akie
51210a1762 Add secondary index to infinity (#12825)
Add secondary index:
1. kb_id
2. available_int

---------

Signed-off-by: zpf121 <1219290549@qq.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-02-02 13:22:29 +08:00
LIRUI YU
01f9b98c3f Fix duplicate POSTGRES enum entry causing backend startup failure (#12936)
### What problem does this PR solve?
Fixes a duplicate POSTGRES entry in the TextFieldType enum that triggers
TypeError: 'POSTGRES' already defined as 'TEXT' on
import, preventing the backend from starting and resulting in 502 errors
on the frontend.
### Type of change
  - [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-02 12:34:48 +08:00
Paul Y Hui
af7acb23cb Fixed regression issue that unable to start the service (#12933)
### What problem does this PR solve?

Fixed the regression issue that unable to start the server as below
details, which was related to this pr
https://github.com/infiniflow/ragflow/pull/12926 looks like.
``` Error Trace
Traceback (most recent call last):
  File "/mnt/c/Workspace/ragflow/api/ragflow_server.py", line 33, in <module>
    from api.apps import app
  File "/mnt/c/Workspace/ragflow/api/apps/__init__.py", line 26, in <module>
Traceback (most recent call last):
  File "/mnt/c/Workspace/ragflow/rag/svr/task_executor.py", line 34, in <module>
    from api.db.db_models import close_connection, APIToken
  File "/mnt/c/Workspace/ragflow/api/db/db_models.py", line 49, in <module>
    from api.db.services.knowledgebase_service import KnowledgebaseService
  File "/mnt/c/Workspace/ragflow/api/db/services/__init__.py", line 19, in <module>
    class TextFieldType(Enum):
  File "/mnt/c/Workspace/ragflow/api/db/db_models.py", line 53, in TextFieldType
    from .user_service import UserService as UserService
  File "/mnt/c/Workspace/ragflow/api/db/services/user_service.py", line 24, in <module>
    POSTGRES = "TEXT"
    ^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 443, in __setitem__
    raise TypeError('%r already defined as %r' % (key, self[key]))
TypeError: 'POSTGRES' already defined as 'TEXT'
    from api.db.db_models import DB, UserTenant
  File "/mnt/c/Workspace/ragflow/api/db/db_models.py", line 49, in <module>
    class TextFieldType(Enum):
  File "/mnt/c/Workspace/ragflow/api/db/db_models.py", line 53, in TextFieldType
    POSTGRES = "TEXT"
    ^^^^^^^^
  File "/usr/lib/python3.12/enum.py", line 443, in __setitem__
    raise TypeError('%r already defined as %r' % (key, self[key]))
TypeError: 'POSTGRES' already defined as 'TEXT'
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-02 12:34:12 +08:00
Carve_
8bc12deb02 Fix:Duplicate enum member causes backend startup failure (#12931)
### What problem does this PR solve?

close #12930 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes
only delete duplicate definition in `api/db/db_models.py`
2026-02-02 12:33:07 +08:00
Liu An
1b587013d8 Fix: remove unused imports and f-string formatting (#12935)
### What problem does this PR solve?

- Remove unused imports (Mock, patch, MagicMock, json, os,
RAGFLOW_COLUMNS, VECTOR_FIELD_PATTERN) from multiple files
- Replace f-string formatting with regular strings for console output
messages in cli.py
- Clean up unnecessary imports that were no longer being used in the
codebase

### Type of change

- [x] Refactoring
2026-02-02 12:11:39 +08:00
Se7en
332b11cf96 feat(tools): add Elasticsearch to OceanBase migration tool (#12927)
### What problem does this PR solve?

fixes https://github.com/infiniflow/ragflow/issues/12774

Add a CLI tool for migrating RAGFlow data from Elasticsearch to
OceanBase, enabling users to switch their document storage backend.

- Automatic discovery and migration of all `ragflow_*` indices
- Schema conversion with vector dimension auto-detection
- Batch processing with progress tracking and resume capability
- Data consistency validation and migration report generation

**Note**: Due to network issues, I was unable to pull the required
Docker images (Elasticsearch, OceanBase) to run the full end-to-end
verification. Unit tests have been verified to pass. I will complete the
e2e verification when network conditions allow, and submit a follow-up
PR if any fixes are needed.

```bash
============================= test session starts ==============================
platform darwin -- Python 3.13.6, pytest-9.0.2, pluggy-1.6.0
rootdir: /Users/sevenc/code/ai/oceanbase/ragflow/tools/es-to-oceanbase-migration
configfile: pyproject.toml
testpaths: tests
plugins: anyio-4.12.1, asyncio-1.3.0, cov-7.0.0
collected 86 items

tests/test_progress.py::TestMigrationProgress::test_create_basic_progress PASSED [  1%]
tests/test_progress.py::TestMigrationProgress::test_create_progress_with_counts PASSED [  2%]
tests/test_progress.py::TestMigrationProgress::test_progress_default_values PASSED [  3%]
tests/test_progress.py::TestMigrationProgress::test_progress_status_values PASSED [  4%]
tests/test_progress.py::TestProgressManager::test_create_progress_manager PASSED [  5%]
tests/test_progress.py::TestProgressManager::test_create_progress_manager_creates_dir PASSED [  6%]
tests/test_progress.py::TestProgressManager::test_create_progress PASSED [  8%]
tests/test_progress.py::TestProgressManager::test_save_and_load_progress PASSED [  9%]
tests/test_progress.py::TestProgressManager::test_load_nonexistent_progress PASSED [ 10%]
tests/test_progress.py::TestProgressManager::test_delete_progress PASSED [ 11%]
tests/test_progress.py::TestProgressManager::test_update_progress PASSED [ 12%]
tests/test_progress.py::TestProgressManager::test_update_progress_multiple_batches PASSED [ 13%]
tests/test_progress.py::TestProgressManager::test_mark_completed PASSED  [ 15%]
tests/test_progress.py::TestProgressManager::test_mark_failed PASSED     [ 16%]
tests/test_progress.py::TestProgressManager::test_mark_paused PASSED     [ 17%]
tests/test_progress.py::TestProgressManager::test_can_resume_running PASSED [ 18%]
tests/test_progress.py::TestProgressManager::test_can_resume_paused PASSED [ 19%]
tests/test_progress.py::TestProgressManager::test_can_resume_completed PASSED [ 20%]
tests/test_progress.py::TestProgressManager::test_can_resume_nonexistent PASSED [ 22%]
tests/test_progress.py::TestProgressManager::test_get_resume_info PASSED [ 23%]
tests/test_progress.py::TestProgressManager::test_get_resume_info_nonexistent PASSED [ 24%]
tests/test_progress.py::TestProgressManager::test_progress_file_path PASSED [ 25%]
tests/test_progress.py::TestProgressManager::test_progress_file_content PASSED [ 26%]
tests/test_schema.py::TestRAGFlowSchemaConverter::test_analyze_ragflow_mapping PASSED [ 27%]
tests/test_schema.py::TestRAGFlowSchemaConverter::test_detect_vector_size PASSED [ 29%]
tests/test_schema.py::TestRAGFlowSchemaConverter::test_unknown_fields PASSED [ 30%]
tests/test_schema.py::TestRAGFlowSchemaConverter::test_get_column_definitions PASSED [ 31%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_basic_document PASSED [ 32%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_with_vector PASSED [ 33%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_array_fields PASSED [ 34%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_json_fields PASSED [ 36%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_unknown_fields_to_extra PASSED [ 37%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_kb_id_list PASSED [ 38%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_content_with_weight_dict PASSED [ 39%]
tests/test_schema.py::TestRAGFlowDataConverter::test_convert_batch PASSED [ 40%]
tests/test_schema.py::TestVectorFieldPattern::test_valid_patterns PASSED [ 41%]
tests/test_schema.py::TestVectorFieldPattern::test_invalid_patterns PASSED [ 43%]
tests/test_schema.py::TestVectorFieldPattern::test_extract_dimension PASSED [ 44%]
tests/test_schema.py::TestConstants::test_array_columns PASSED           [ 45%]
tests/test_schema.py::TestConstants::test_json_columns PASSED            [ 46%]
tests/test_schema.py::TestConstants::test_ragflow_columns_completeness PASSED [ 47%]
tests/test_schema.py::TestConstants::test_fts_columns PASSED             [ 48%]
tests/test_schema.py::TestConstants::test_ragflow_columns_types PASSED   [ 50%]
tests/test_schema.py::TestRAGFlowSchemaConverterEdgeCases::test_empty_mapping PASSED [ 51%]
tests/test_schema.py::TestRAGFlowSchemaConverterEdgeCases::test_mapping_without_properties PASSED [ 52%]
tests/test_schema.py::TestRAGFlowSchemaConverterEdgeCases::test_multiple_vector_fields PASSED [ 53%]
tests/test_schema.py::TestRAGFlowSchemaConverterEdgeCases::test_get_column_definitions_without_analysis PASSED [ 54%]
tests/test_schema.py::TestRAGFlowSchemaConverterEdgeCases::test_get_vector_fields PASSED [ 55%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_empty_document PASSED [ 56%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_document_without_source PASSED [ 58%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_boolean_to_integer PASSED [ 59%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_invalid_integer PASSED [ 60%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_float_field PASSED [ 61%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_array_with_special_characters PASSED [ 62%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_already_json_array PASSED [ 63%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_single_value_to_array PASSED [ 65%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_detect_vector_fields_from_document PASSED [ 66%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_with_default_values PASSED [ 67%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_list_content PASSED [ 68%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_convert_batch_empty PASSED [ 69%]
tests/test_schema.py::TestRAGFlowDataConverterEdgeCases::test_existing_extra_field_merged PASSED [ 70%]
tests/test_verify.py::TestVerificationResult::test_create_basic_result PASSED [ 72%]
tests/test_verify.py::TestVerificationResult::test_result_default_values PASSED [ 73%]
tests/test_verify.py::TestVerificationResult::test_result_with_counts PASSED [ 74%]
tests/test_verify.py::TestMigrationVerifier::test_verify_counts_match PASSED [ 75%]
tests/test_verify.py::TestMigrationVerifier::test_verify_counts_mismatch PASSED [ 76%]
tests/test_verify.py::TestMigrationVerifier::test_verify_samples_all_match PASSED [ 77%]
tests/test_verify.py::TestMigrationVerifier::test_verify_samples_some_missing PASSED [ 79%]
tests/test_verify.py::TestMigrationVerifier::test_verify_samples_data_mismatch PASSED [ 80%]
tests/test_verify.py::TestMigrationVerifier::test_values_equal_none_values PASSED [ 81%]
tests/test_verify.py::TestMigrationVerifier::test_values_equal_array_columns PASSED [ 82%]
tests/test_verify.py::TestMigrationVerifier::test_values_equal_json_columns PASSED [ 83%]
tests/test_verify.py::TestMigrationVerifier::test_values_equal_kb_id_list PASSED [ 84%]
tests/test_verify.py::TestMigrationVerifier::test_values_equal_content_with_weight_dict PASSED [ 86%]
tests/test_verify.py::TestMigrationVerifier::test_determine_result_passed PASSED [ 87%]
tests/test_verify.py::TestMigrationVerifier::test_determine_result_failed_count PASSED [ 88%]
tests/test_verify.py::TestMigrationVerifier::test_determine_result_failed_samples PASSED [ 89%]
tests/test_verify.py::TestMigrationVerifier::test_generate_report PASSED [ 90%]
tests/test_verify.py::TestMigrationVerifier::test_generate_report_with_missing PASSED [ 91%]
tests/test_verify.py::TestMigrationVerifier::test_generate_report_with_mismatches PASSED [ 93%]
tests/test_verify.py::TestValueComparison::test_string_comparison PASSED [ 94%]
tests/test_verify.py::TestValueComparison::test_integer_comparison PASSED [ 95%]
tests/test_verify.py::TestValueComparison::test_float_comparison PASSED  [ 96%]
tests/test_verify.py::TestValueComparison::test_boolean_comparison PASSED [ 97%]
tests/test_verify.py::TestValueComparison::test_empty_array_comparison PASSED [ 98%]
tests/test_verify.py::TestValueComparison::test_nested_json_comparison PASSED [100%]

======================= 86 passed, 88 warnings in 0.66s ========================
```

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-01-31 16:11:27 +08:00
NTLx
c4c3f744c0 feat: add Peewee ORM support for OceanBase as primary database (#12769) (#12926)
## Summary

This PR adds Peewee ORM support for OceanBase as the primary database in
RAGFlow, as requested in issue #12769.

## Changes

### Core Implementation

1. **RetryingPooledOceanBaseDatabase Class**
   - Inherits from `PooledMySQLDatabase` (OceanBase is MySQL-compatible)
   - Implements retry mechanism for connection issues
   - Handles MySQL-specific error codes (2013, 2006 for connection loss)
   - Provides connection pool management

2. **PooledDatabase Enum**
   - Added `OCEANBASE = RetryingPooledOceanBaseDatabase`

3. **DatabaseLock Enum**
   - Added `OCEANBASE = MysqlDatabaseLock`
   - OceanBase uses MySQL-style locking

4. **TextFieldType Enum**
   - Added `OCEANBASE = "LONGTEXT"`
   - OceanBase uses same text field type as MySQL

5. **DatabaseMigrator Enum**
   - Added `OCEANBASE = MySQLMigrator`
   - OceanBase uses MySQL migration tools

### Usage

```bash
# Set environment variable to use OceanBase
export DB_TYPE=oceanbase

# Configure connection (in docker/.env or environment)
OCEANBASE_HOST=localhost
OCEANBASE_PORT=2881
OCEANBASE_USER=root
OCEANBASE_PASSWORD=password
OCEANBASE_DATABASE=ragflow
```

### Technical Details

- **Location**: `api/db/db_models.py`
- **Dependencies**: No new dependencies (uses existing Peewee MySQL
support)
- **Code Size**: ~90 lines
- **Difficulty**: Simple

### Testing

- Added comprehensive unit tests in
`tests/unit/test_oceanbase_peewee.py`
- Tests cover:
  - OceanBase database class existence and inheritance
  - Enum values for PooledDatabase, DatabaseLock, TextFieldType
  - Initialization with custom retry settings
  - Environment variable configuration

### Acceptance Criteria

 Can switch to OceanBase database via `DB_TYPE=oceanbase` environment
variable
 All database operations work normally in OceanBase environment  
 OceanBase uses MySQL compatibility mode (no additional dependencies)  

### Background

This is part of the RAGFlow + OceanBase Hackathon to allow users to
choose OceanBase as RAGFlow's primary database, leveraging OceanBase's
high availability and scalability.

---

## Related Issues
- **Primary**: https://github.com/infiniflow/ragflow/issues/12769
- **Context**: https://github.com/oceanbase/seekdb/issues/123 (OceanBase
Developer Challenge)

---

Closes infiniflow/ragflow#12769
2026-01-31 15:45:20 +08:00
Carve_
23bdf25a1f feature:Add OceanBase Storage Support for Table Parser (#12923)
### What problem does this PR solve?

close #12770 

This PR adds OceanBase as a storage backend for the Table Parser. It
enables dynamic table schema storage via JSON and implements OceanBase
SQL execution for text-to-SQL retrieval.


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes
- Table Parser stores row data into `chunk_data` when doc engine is
OceanBase. (table.py)
- OceanBase table schema adds `chunk_data` JSON column and migrates if
needed.
- Implemented OceanBase `sql()` to execute text-to-SQL results.
(ob_conn.py)
- Add `DOC_ENGINE_OCEANBASE` flag for engine detection (setting.py)

### Test
1. Set `DOC_ENGINE=oceanbase` (e.g. in `docker/.env`)
<img width="1290" height="783" alt="doc_engine_ob"
src="https://github.com/user-attachments/assets/7d1c609f-7bf2-4b2e-b4cc-4243e72ad4f1"
/>

2. Upload an Excel file to Knowledge Base.(for test, we use as below)
<img width="786" height="930" alt="excel"
src="https://github.com/user-attachments/assets/bedf82f2-cd00-426b-8f4d-6978a151231a"
/>

3. Choose **Table** as parsing method.
<img width="2550" height="1134" alt="parse_excel"
src="https://github.com/user-attachments/assets/aba11769-02be-4905-97e1-e24485e24cd0"
/>

4.Ask a natural language query in chat.
<img width="2550" height="1134" alt="query"
src="https://github.com/user-attachments/assets/26a910a6-e503-4ac7-b66a-f5754bbb0e91"
/>
2026-01-31 15:11:54 +08:00
Carve_
ee23b9eb63 feature:Add OceanBase Support to Text-to-SQL Agent (#12919)
### What problem does this PR solve?

Close #12768.

This PR adds OceanBase support to RAGFlow’s Text-to-SQL (ExeSQL)
component.
OceanBase is integrated via MySQL compatibility mode, and the UI
`db_type` options are updated accordingly.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes

**Backend**
- Add `oceanbase` `db_type` validation and connection logic in
`exesql.py` and reuse existing MySQL compatibility mode

**Frontend**
- Add OceanBase option to the ExeSQL `db_type` selector

### How to test
1. Configure OceanBase connection in ExeSQL node
(host/port/user/password/database)
2. Input: “Show 10 rows from test table”
3. Generated SQL: `SELECT * FROM test LIMIT 10;`
4. Query executes successfully and results are returned

### Screenshots
- ExeSQL db_type includes OceanBase
<img width="649" height="1015" alt="2"
src="https://github.com/user-attachments/assets/e0a5f7b9-e282-402a-8639-64c1aef8fce6"
/>

- ExeSQL test OceanBase connection
<img width="2247" height="1140" alt="test_ob"
src="https://github.com/user-attachments/assets/f16ebd93-b48e-4d18-b53f-8496581e755d"
/>



- Query results from OceanBase shown in UI
<img width="2550" height="1351" alt="1"
src="https://github.com/user-attachments/assets/b44163dc-baab-420d-b31e-b644bdcb77a9"
/>
2026-01-31 15:03:40 +08:00
Liu An
c4f60b349d Fix(test): downgrade test priorities (#12913)
### What problem does this PR solve?

Changed test priorities in multiple test files, downgrading from p1 to
p2 and p2 to p3.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-30 20:02:56 +08:00
writinwaters
eb75b1ce82 Docs: Fixed a docusaurus display issue (#12914)
### What problem does this PR solve?

Fixed a docusaurus display issue.

### Type of change


- [x] Documentation Update
2026-01-30 18:04:05 +08:00
Haipeng LI
e385b19d67 Test: Add code coverage reporting to CI (#12874)
### What problem does this PR solve?

Add code coverage reporting to CI

### Type of change

- [x] Test (please describe): coverage report

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-30 14:49:16 +08:00
Phives
87305cb08c fix: close file handles when loading JSON mapping in doc store connectors (#12904)
**What problem does this PR solve?**

When loading JSON mapping/schema files, the code used
json.load(open(path)) without closing the file. The file handle stayed
open until garbage collection, which can leak file descriptors under
load (e.g. repeated reconnects or migrations).

**Type of change**
[x] Bug Fix (non-breaking change which fixes an issue)
**Change**
Replaced json.load(open(...)) with a context manager so the file is
closed after loading:
with open(fp_mapping, "r") as f:    ... = json.load(f)

**Files updated**
rag/utils/opensearch_conn.py – mapping load (1 place)
common/doc_store/es_conn_base.py – mapping load + doc_meta_mapping load
(2 places)
common/doc_store/infinity_conn_base.py – schema loads in _migrate_db,
doc metadata table creation, and SQL field mapping (4 places)
Behavior is unchanged; only resource handling is fixed.

Co-authored-by: Gittensor Miner <miner@gittensor.io>
2026-01-30 14:07:51 +08:00
qinling0210
212d6f3660 Fix metadata in get_list() (#12906)
### What problem does this PR solve?

test_update_document.py failed as metadata is not included in the
response of get_list(), fix the issue.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-30 14:06:49 +08:00
Kevin Hu
f262d416fe Refa: remove aspose dependency. (#12910)
### Type of change

- [x] Refactoring
2026-01-30 14:06:19 +08:00
Kevin Hu
f1c2fac03e Refa: remove ppt image. (#12909)
### What problem does this PR solve?

remove `aspose`

### Type of change

- [x] Refactoring
2026-01-30 13:35:42 +08:00
BitToby
73645e2f78 fix: preserve line breaks in prompt editor and add auto-save on blur (#12887)
Closes #12762 

### What problem does this PR solve?
**Line break issue in Agent prompt editor:**
- Text with blank lines in `system_prompt` or `user_prompt` would have
extra/fewer blank lines after save/reload or paste
- Root cause: Mismatch between Lexical editor's paragraph nodes (`\n\n`
separator) and line break nodes (`\n` separator)

**Auto-save issue:**
- Changes were only saved after 20-second debounce, causing data loss on
page refresh before timer completed

### Solution
1. **Line break fix**: Use `LineBreakNode` consistently for all line
breaks (typing Enter, paste, load)
2. **Auto-save**: Save immediately when prompt editor loses focus


[1.webm](https://github.com/user-attachments/assets/eb2c2428-54a3-4d4e-8037-6cc34a859b83)

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-30 10:29:51 +08:00
Liu An
4947e9473a Fix(test): Update error message assertions for unsupported content type tests (#12901)
### What problem does this PR solve?

This commit updates test cases for create, delete, and update dataset
endpoints to expect consistent error messages when an unsupported
content type is provided.

### Type of change

- [x] Bug Fix (test)
2026-01-30 09:45:04 +08:00
Angel98518
98b6a0e6d1 feat: Add OceanBase Performance Monitoring and Health Check Integration (#12886)
## Description

This PR implements comprehensive OceanBase performance monitoring and
health check functionality as requested in issue #12772. The
implementation follows the existing ES/Infinity health check patterns
and provides detailed metrics for operations teams.

## Problem

Currently, RAGFlow lacks detailed health monitoring for OceanBase when
used as the document engine. Operations teams need visibility into:
- Connection status and latency
- Storage space usage
- Query throughput (QPS)
- Slow query statistics
- Connection pool utilization

## Solution

### 1. Enhanced OBConnection Class (`rag/utils/ob_conn.py`)

Added comprehensive performance monitoring methods:
- `get_performance_metrics()` - Main method returning all performance
metrics
- `_get_storage_info()` - Retrieves database storage usage
- `_get_connection_pool_stats()` - Gets connection pool statistics
- `_get_slow_query_count()` - Counts queries exceeding threshold
- `_estimate_qps()` - Estimates queries per second
- Enhanced `health()` method with connection status

### 2. Health Check Utilities (`api/utils/health_utils.py`)

Added two new functions following ES/Infinity patterns:
- `get_oceanbase_status()` - Returns OceanBase status with health and
performance metrics
- `check_oceanbase_health()` - Comprehensive health check with detailed
metrics

### 3. API Endpoint (`api/apps/system_app.py`)

Added new endpoint:
- `GET /v1/system/oceanbase/status` - Returns OceanBase health status
and performance metrics

### 4. Comprehensive Unit Tests
(`test/unit_test/utils/test_oceanbase_health.py`)

Added 340+ lines of unit tests covering:
- Health check success/failure scenarios
- Performance metrics retrieval
- Error handling and edge cases
- Connection pool statistics
- Storage information retrieval
- QPS estimation
- Slow query detection

## Metrics Provided

- **Connection Status**: connected/disconnected
- **Latency**: Query latency in milliseconds
- **Storage**: Used and total storage space
- **QPS**: Estimated queries per second
- **Slow Queries**: Count of queries exceeding threshold
- **Connection Pool**: Active connections, max connections, pool size

## Testing

- All unit tests pass
- Error handling tested for connection failures
- Edge cases covered (missing tables, connection errors)
- Follows existing code patterns and conventions

## Code Statistics

- **Total Lines Changed**: 665+ lines
- **New Code**: ~600 lines
- **Test Coverage**: 340+ lines of comprehensive tests
- **Files Modified**: 3
- **Files Created**: 1 (test file)

## Acceptance Criteria Met

 `/system/oceanbase/status` API returns OceanBase health status
 Monitoring metrics accurately reflect OceanBase running status
 Clear error messages when health checks fail
 Response time optimized (metrics cached where possible)
 Follows existing ES/Infinity health check patterns
 Comprehensive test coverage

## Related Files

- `rag/utils/ob_conn.py` - OceanBase connection class
- `api/utils/health_utils.py` - Health check utilities
- `api/apps/system_app.py` - System API endpoints
- `test/unit_test/utils/test_oceanbase_health.py` - Unit tests

Fixes #12772

---------

Co-authored-by: Daniel <daniel@example.com>
2026-01-30 09:44:42 +08:00
Yongteng Lei
183803e56b Pref: fix thread pool workers (#12882)
### What problem does this PR solve?

Fixed thread pool workers and improve retrieval component

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-01-30 09:44:23 +08:00
writinwaters
efb136c29c Docs: minor (#12899)
### What problem does this PR solve?

Removed redundant command + "*the* current version" @JinHai-CN 

### Type of change

- [x] Documentation Update
2026-01-29 19:23:18 +08:00
eviaaaaa
c59ae4c7c2 Fix: codeExec return types & error handling; Update Spark model mappings (#12896)
## What problem does this PR solve?

This PR addresses three specific issues to improve agent reliability and
model support:

1. **`codeExec` Output Limitation**: Previously, the `codeExec` tool was
strictly limited to returning `string` types. I updated the output
constraint to `object` to support structured data (Dicts, Lists, etc.)
required for complex downstream tasks.
2. **`codeExec` Error Handling**: Improved the execution logic so that
when runtime errors occur, the tool captures the exception and returns
the error message as the output instead of causing the process to abort
or fail silently.
3. **Spark Model Configuration**:
    - Added support for the `MAX-32k` model variant.
- Fixed the `Spark-Lite` mapping from `general` to `lite` to match the
latest API specifications.

## Type of change

- [x] Bug Fix (fixes execution logic and model mapping)
- [x] New Feature / Enhancement (adds model support and improves tool
flexibility)

## Key Changes

### `agent/tools/code_exec.py`
- Changed the output type definition from `string` to `object`.
- Refactored the execution flow to gracefully catch exceptions and
return error messages as part of the tool output.

### `rag/llm/chat_model.py`
- Added `"Spark-Max-32K": "max-32k"` to the model list.
- Updated `"Spark-Lite"` value from `"general"` to `"lite"`.

## Checklist
- [x] My code follows the style guidelines of this project.
- [x] I have performed a self-review of my own code.

Signed-off-by: evilhero <2278596667@qq.com>
2026-01-29 19:22:35 +08:00
writinwaters
d99f6a611a Refact: Updated UI tips. (#12898)
### What problem does this PR solve?

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-01-29 17:56:07 +08:00
6ba3i
7053d3683c Feat: Add CLI retrieval test to CI workflow (#12881)
### What problem does this PR solve?

Adds a CLI-based retrieval test to CI after the Elasticsearch HTTP API
tests to validate end-to-end admin/user flows and dataset retrieval via
ragflow_cli.py. This helps catch regressions in the CLI path that aren’t
covered by existing API tests.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-29 17:55:32 +08:00
Jimmy Ben Klieve
ec88e17710 fix: task executor bar chart error (#12894)
### What problem does this PR solve?

Fix wrong data rendered in task executor bar chart 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-29 17:34:05 +08:00
Kevin Hu
32c0161ff1 Refa: Clean the folders. (#12890)
### Type of change

- [x] Refactoring
2026-01-29 14:23:26 +08:00
akie
d86b7f9721 Remove filter (kb_id) in infinity (#12853)
Secondary indexes in infinity do not support **IN** expr

---------

Signed-off-by: zpf121 <1219290549@qq.com>
2026-01-29 11:04:25 +08:00
Philipp Heyken Soares
6305c7e411 Fix metadata filter (#12861)
### What problem does this PR solve?

##### Summary
This PR fixes a bug in the metadata filtering logic where the contains
and not contains operators were behaving identically to the in and not
in operators. It also standardizes the syntax for string-based
operators.

##### The Issue
On the main branch, the contains operator was implemented as:
`matched = input in value if not isinstance(input, list) else all(i in
value for i in input)`
This logic is identical to the `in` operator. It checks if the metadata
(`input`) exists within the filter (`value`). For a "contains" search,
the logic should be reversed: _we want to check if the filter value
exists within the metadata input_.

##### Solution Presented Here
The operators have been rewritten using str.find():
Contains: `str(input).find(value) >= 0`
Not Contains: `str(input).find(value) == -1`

##### Advantage
This approach places the metadata (input) on the left side of the
expression. This maintains stylistic consistency with the existing start
with and end with operators in the same file, which also place the input
on the left (e.g., str(input).lower().startswith(...)).

##### Considered Alternative
In a previous PR we considered using the standard Python `in` operator:
`value in str(input)`.
The `in` operator is approximately 15% faster because it uses optimized
Python bytecode (CONTAINS_OP) and avoids an attribute lookup. However
following rejection of this PR we now propose the change presented here.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

---------

Co-authored-by: Philipp Heyken Soares <philipp.heyken-soares@am.ai>
2026-01-29 09:59:48 +08:00
dependabot[bot]
47e55ab324 Chore(deps): Bump starlette from 0.46.2 to 0.49.1 in /agent/sandbox (#12878)
Bumps [starlette](https://github.com/Kludex/starlette) from 0.46.2 to
0.49.1.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/Kludex/starlette/releases">starlette's
releases</a>.</em></p>
<blockquote>
<h2>Version 0.49.1</h2>
<p>This release fixes a security vulnerability in the parsing logic of
the <code>Range</code> header in <code>FileResponse</code>.</p>
<p>You can view the full security advisory: <a
href="https://github.com/Kludex/starlette/security/advisories/GHSA-7f5h-v6xp-fcq8">GHSA-7f5h-v6xp-fcq8</a></p>
<h2>Fixed</h2>
<ul>
<li>Optimize the HTTP ranges parsing logic <a
href="4ea6e22b48">4ea6e22b489ec388d6004cfbca52dd5b147127c5</a></li>
</ul>
<hr />
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Kludex/starlette/compare/0.49.0...0.49.1">https://github.com/Kludex/starlette/compare/0.49.0...0.49.1</a></p>
<h2>Version 0.49.0</h2>
<h2>Added</h2>
<ul>
<li>Add <code>encoding</code> parameter to <code>Config</code> class <a
href="https://redirect.github.com/Kludex/starlette/pull/2996">#2996</a>.</li>
<li>Support multiple cookie headers in <code>Request.cookies</code> <a
href="https://redirect.github.com/Kludex/starlette/pull/3029">#3029</a>.</li>
<li>Use <code>Literal</code> type for <code>WebSocketEndpoint</code>
encoding values <a
href="https://redirect.github.com/Kludex/starlette/pull/3027">#3027</a>.</li>
</ul>
<h2>Changed</h2>
<ul>
<li>Do not pollute exception context in <code>Middleware</code> when
using <code>BaseHTTPMiddleware</code> <a
href="https://redirect.github.com/Kludex/starlette/pull/2976">#2976</a>.</li>
</ul>
<hr />
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/TheWesDias"><code>@​TheWesDias</code></a> made
their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/3017">Kludex/starlette#3017</a></li>
<li><a href="https://github.com/gmos2104"><code>@​gmos2104</code></a>
made their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/3027">Kludex/starlette#3027</a></li>
<li><a
href="https://github.com/secrett2633"><code>@​secrett2633</code></a>
made their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/2996">Kludex/starlette#2996</a></li>
<li><a
href="https://github.com/adam-sikora"><code>@​adam-sikora</code></a>
made their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/2976">Kludex/starlette#2976</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Kludex/starlette/compare/0.48.0...0.49.0">https://github.com/Kludex/starlette/compare/0.48.0...0.49.0</a></p>
<h2>Version 0.48.0</h2>
<h2>Added</h2>
<ul>
<li>Add official Python 3.14 support <a
href="https://redirect.github.com/Kludex/starlette/pull/3013">#3013</a>.</li>
</ul>
<h2>Changed</h2>
<ul>
<li>Implement <a
href="https://www.rfc-editor.org/rfc/rfc9110">RFC9110</a> http status
names <a
href="https://redirect.github.com/Kludex/starlette/pull/2939">#2939</a>.</li>
</ul>
<hr />
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/yakimka"><code>@​yakimka</code></a> made
their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/2943">Kludex/starlette#2943</a></li>
<li><a href="https://github.com/mbeijen"><code>@​mbeijen</code></a> made
their first contribution in <a
href="https://redirect.github.com/Kludex/starlette/pull/2939">Kludex/starlette#2939</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/Kludex/starlette/compare/0.47.3...0.48.0">https://github.com/Kludex/starlette/compare/0.47.3...0.48.0</a></p>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/Kludex/starlette/blob/main/docs/release-notes.md">starlette's
changelog</a>.</em></p>
<blockquote>
<h2>0.49.1 (October 28, 2025)</h2>
<p>This release fixes a security vulnerability in the parsing logic of
the <code>Range</code> header in <code>FileResponse</code>.</p>
<p>You can view the full security advisory: <a
href="https://github.com/Kludex/starlette/security/advisories/GHSA-7f5h-v6xp-fcq8">GHSA-7f5h-v6xp-fcq8</a></p>
<h4>Fixed</h4>
<ul>
<li>Optimize the HTTP ranges parsing logic <a
href="4ea6e22b48">4ea6e22b489ec388d6004cfbca52dd5b147127c5</a></li>
</ul>
<h2>0.49.0 (October 28, 2025)</h2>
<h4>Added</h4>
<ul>
<li>Add <code>encoding</code> parameter to <code>Config</code> class <a
href="https://redirect.github.com/Kludex/starlette/pull/2996">#2996</a>.</li>
<li>Support multiple cookie headers in <code>Request.cookies</code> <a
href="https://redirect.github.com/Kludex/starlette/pull/3029">#3029</a>.</li>
<li>Use <code>Literal</code> type for <code>WebSocketEndpoint</code>
encoding values <a
href="https://redirect.github.com/Kludex/starlette/pull/3027">#3027</a>.</li>
</ul>
<h4>Changed</h4>
<ul>
<li>Do not pollute exception context in <code>Middleware</code> when
using <code>BaseHTTPMiddleware</code> <a
href="https://redirect.github.com/Kludex/starlette/pull/2976">#2976</a>.</li>
</ul>
<h2>0.48.0 (September 13, 2025)</h2>
<h4>Added</h4>
<ul>
<li>Add official Python 3.14 support <a
href="https://redirect.github.com/Kludex/starlette/pull/3013">#3013</a>.</li>
</ul>
<h4>Changed</h4>
<ul>
<li>Implement <a
href="https://www.rfc-editor.org/rfc/rfc9110">RFC9110</a> http status
names <a
href="https://redirect.github.com/Kludex/starlette/pull/2939">#2939</a>.</li>
</ul>
<h2>0.47.3 (August 24, 2025)</h2>
<h4>Fixed</h4>
<ul>
<li>Use <code>asyncio.iscoroutinefunction</code> for Python 3.12 and
older <a
href="https://redirect.github.com/Kludex/starlette/pull/2984">#2984</a>.</li>
</ul>
<h2>0.47.2 (July 20, 2025)</h2>
<h4>Fixed</h4>
<ul>
<li>Make <code>UploadFile</code> check for future rollover <a
href="https://redirect.github.com/Kludex/starlette/pull/2962">#2962</a>.</li>
</ul>
<h2>0.47.1 (June 21, 2025)</h2>
<h4>Fixed</h4>
<ul>
<li>Use <code>Self</code> in <code>TestClient.__enter__</code> <a
href="https://redirect.github.com/Kludex/starlette/pull/2951">#2951</a>.</li>
<li>Allow async exception handlers to type-check <a
href="https://redirect.github.com/Kludex/starlette/pull/2949">#2949</a>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="7e4b7428f2"><code>7e4b742</code></a>
Version 0.49.1 (<a
href="https://redirect.github.com/Kludex/starlette/issues/3047">#3047</a>)</li>
<li><a
href="4ea6e22b48"><code>4ea6e22</code></a>
Merge commit from fork</li>
<li><a
href="7d88ea6f8e"><code>7d88ea6</code></a>
Version 0.49.0 (<a
href="https://redirect.github.com/Kludex/starlette/issues/3046">#3046</a>)</li>
<li><a
href="26d66bbfb0"><code>26d66bb</code></a>
Do not pollute exception context in Middleware (<a
href="https://redirect.github.com/Kludex/starlette/issues/2976">#2976</a>)</li>
<li><a
href="a59397db88"><code>a59397d</code></a>
Set encodings when reading config files (<a
href="https://redirect.github.com/Kludex/starlette/issues/2996">#2996</a>)</li>
<li><a
href="3b7f0cbf59"><code>3b7f0cb</code></a>
test: add test for unknown status (<a
href="https://redirect.github.com/Kludex/starlette/issues/3035">#3035</a>)</li>
<li><a
href="b09ce1a99d"><code>b09ce1a</code></a>
docs: fix legibility issues on sponsorship page (<a
href="https://redirect.github.com/Kludex/starlette/issues/3039">#3039</a>)</li>
<li><a
href="0f0edcf800"><code>0f0edcf</code></a>
Revert &quot;Add Marcelo Trylesinski to the license (<a
href="https://redirect.github.com/Kludex/starlette/issues/3025">#3025</a>)&quot;
(<a
href="https://redirect.github.com/Kludex/starlette/issues/3044">#3044</a>)</li>
<li><a
href="3912d63137"><code>3912d63</code></a>
docs: add social icons (<a
href="https://redirect.github.com/Kludex/starlette/issues/3038">#3038</a>)</li>
<li><a
href="4915a9309f"><code>4915a93</code></a>
Add discord to README/docs (<a
href="https://redirect.github.com/Kludex/starlette/issues/3034">#3034</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/Kludex/starlette/compare/0.46.2...0.49.1">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=starlette&package-manager=uv&previous-version=0.46.2&new-version=0.49.1)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-28 19:04:01 +08:00
dependabot[bot]
82b932dbc7 Chore(deps): Bump urllib3 from 2.4.0 to 2.6.3 in /agent/sandbox (#12877)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.4.0 to 2.6.3.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/releases">urllib3's
releases</a>.</em></p>
<blockquote>
<h2>2.6.3</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Changes</h2>
<ul>
<li>Fixed a security issue where decompression-bomb safeguards of the
streaming API were bypassed when HTTP redirects were followed.
(CVE-2026-21441 reported by <a
href="https://github.com/D47A"><code>@​D47A</code></a>, 8.9 High,
GHSA-38jv-5279-wg99)</li>
<li>Started treating <code>Retry-After</code> times greater than 6 hours
as 6 hours by default. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3743">urllib3/urllib3#3743</a>)</li>
<li>Fixed <code>urllib3.connection.VerifiedHTTPSConnection</code> on
Emscripten. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3752">urllib3/urllib3#3752</a>)</li>
</ul>
<h2>2.6.2</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Changes</h2>
<ul>
<li>Fixed <code>HTTPResponse.read_chunked()</code> to properly handle
leftover data in the decoder's buffer when reading compressed chunked
responses. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3734">urllib3/urllib3#3734</a>)</li>
</ul>
<h2>2.6.1</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Changes</h2>
<ul>
<li>Restore previously removed <code>HTTPResponse.getheaders()</code>
and <code>HTTPResponse.getheader()</code> methods. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3731">#3731</a>)</li>
</ul>
<h2>2.6.0</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Security</h2>
<ul>
<li>Fixed a security issue where streaming API could improperly handle
highly compressed HTTP content (&quot;decompression bombs&quot;) leading
to excessive resource consumption even when a small amount of data was
requested. Reading small chunks of compressed data is safer and much
more efficient now. (CVE-2025-66471 reported by <a
href="https://github.com/Cycloctane"><code>@​Cycloctane</code></a>, 8.9
High, GHSA-2xpw-w6gg-jr37)</li>
<li>Fixed a security issue where an attacker could compose an HTTP
response with virtually unlimited links in the
<code>Content-Encoding</code> header, potentially leading to a denial of
service (DoS) attack by exhausting system resources during decoding. The
number of allowed chained encodings is now limited to 5. (CVE-2025-66418
reported by <a
href="https://github.com/illia-v"><code>@​illia-v</code></a>, 8.9 High,
GHSA-gm62-xv2j-4w53)</li>
</ul>
<blockquote>
<p>[!IMPORTANT]</p>
<ul>
<li>If urllib3 is not installed with the optional
<code>urllib3[brotli]</code> extra, but your environment contains a
Brotli/brotlicffi/brotlipy package anyway, make sure to upgrade it to at
least Brotli 1.2.0 or brotlicffi 1.2.0.0 to benefit from the security
fixes and avoid warnings. Prefer using <code>urllib3[brotli]</code> to
install a compatible Brotli package automatically.</li>
</ul>
</blockquote>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's
changelog</a>.</em></p>
<blockquote>
<h1>2.6.3 (2026-01-07)</h1>
<ul>
<li>Fixed a high-severity security issue where decompression-bomb
safeguards of
the streaming API were bypassed when HTTP redirects were followed.
(<code>GHSA-38jv-5279-wg99
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-38jv-5279-wg99&gt;</code>__)</li>
<li>Started treating <code>Retry-After</code> times greater than 6 hours
as 6 hours by
default. (<code>[#3743](https://github.com/urllib3/urllib3/issues/3743)
&lt;https://github.com/urllib3/urllib3/issues/3743&gt;</code>__)</li>
<li>Fixed <code>urllib3.connection.VerifiedHTTPSConnection</code> on
Emscripten.
(<code>[#3752](https://github.com/urllib3/urllib3/issues/3752)
&lt;https://github.com/urllib3/urllib3/issues/3752&gt;</code>__)</li>
</ul>
<h1>2.6.2 (2025-12-11)</h1>
<ul>
<li>Fixed <code>HTTPResponse.read_chunked()</code> to properly handle
leftover data in
the decoder's buffer when reading compressed chunked responses.
(<code>[#3734](https://github.com/urllib3/urllib3/issues/3734)
&lt;https://github.com/urllib3/urllib3/issues/3734&gt;</code>__)</li>
</ul>
<h1>2.6.1 (2025-12-08)</h1>
<ul>
<li>Restore previously removed <code>HTTPResponse.getheaders()</code>
and
<code>HTTPResponse.getheader()</code> methods.
(<code>[#3731](https://github.com/urllib3/urllib3/issues/3731)
&lt;https://github.com/urllib3/urllib3/issues/3731&gt;</code>__)</li>
</ul>
<h1>2.6.0 (2025-12-05)</h1>
<h2>Security</h2>
<ul>
<li>Fixed a security issue where streaming API could improperly handle
highly
compressed HTTP content (&quot;decompression bombs&quot;) leading to
excessive resource
consumption even when a small amount of data was requested. Reading
small
chunks of compressed data is safer and much more efficient now.
(<code>GHSA-2xpw-w6gg-jr37
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-2xpw-w6gg-jr37&gt;</code>__)</li>
<li>Fixed a security issue where an attacker could compose an HTTP
response with
virtually unlimited links in the <code>Content-Encoding</code> header,
potentially
leading to a denial of service (DoS) attack by exhausting system
resources
during decoding. The number of allowed chained encodings is now limited
to 5.
(<code>GHSA-gm62-xv2j-4w53
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-gm62-xv2j-4w53&gt;</code>__)</li>
</ul>
<p>.. caution::</p>
<ul>
<li>If urllib3 is not installed with the optional
<code>urllib3[brotli]</code> extra, but
your environment contains a Brotli/brotlicffi/brotlipy package anyway,
make
sure to upgrade it to at least Brotli 1.2.0 or brotlicffi 1.2.0.0 to
benefit from the security fixes and avoid warnings. Prefer using</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="0248277dd7"><code>0248277</code></a>
Release 2.6.3</li>
<li><a
href="8864ac407b"><code>8864ac4</code></a>
Merge commit from fork</li>
<li><a
href="70cecb27ca"><code>70cecb2</code></a>
Fix Scorecard issues related to vulnerable dev dependencies (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3755">#3755</a>)</li>
<li><a
href="41f249abe1"><code>41f249a</code></a>
Move &quot;v2.0 Migration Guide&quot; to the end of the table of
contents (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3747">#3747</a>)</li>
<li><a
href="fd4dffd2fc"><code>fd4dffd</code></a>
Patch <code>VerifiedHTTPSConnection</code> for Emscripten (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3752">#3752</a>)</li>
<li><a
href="13f0bfd55e"><code>13f0bfd</code></a>
Handle massive values in Retry-After when calculating time to sleep for
(<a
href="https://redirect.github.com/urllib3/urllib3/issues/3743">#3743</a>)</li>
<li><a
href="8c480bf87b"><code>8c480bf</code></a>
Bump actions/upload-artifact from 5.0.0 to 6.0.0 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3748">#3748</a>)</li>
<li><a
href="4b40616e95"><code>4b40616</code></a>
Bump actions/cache from 4.3.0 to 5.0.1 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3750">#3750</a>)</li>
<li><a
href="82b8479663"><code>82b8479</code></a>
Bump actions/download-artifact from 6.0.0 to 7.0.0 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3749">#3749</a>)</li>
<li><a
href="34284cb017"><code>34284cb</code></a>
Mention experimental features in the security policy (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3746">#3746</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/urllib3/urllib3/compare/2.4.0...2.6.3">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=uv&previous-version=2.4.0&new-version=2.6.3)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after
your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge
and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating
it. You can achieve the same result by closing it manually
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-01-28 19:03:41 +08:00
LIRUI YU
c8bd413e4c Fixed bug: Prevent 400 errors from Image2Text providers by skipping images smaller than 11px on any side during figure enhancement. (#12868)
### What problem does this PR solve?
During figure enhancement, some cropped figure images are extremely
small. Sending these to the Image2Text/VLM provider fails with a 400
invalid_parameter_error because the image width/height must

be >10px. This aborts the enhancement step. This PR adds a minimal size
guard to skip tiny crops and continue processing.
<img width="1084" height="494" alt="image"
src="https://github.com/user-attachments/assets/ad074270-94e6-4571-91c8-37df85212639"
/>

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-28 14:59:02 +08:00
Magicbook1108
2c4499ec45 Fix: key error "content" #12844 (#12847)
### What problem does this PR solve?

Fix: key error "content" #12844

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-28 14:39:34 +08:00
dive2tech
15a534909f fix: avoid ZeroDivisionError when fulltext column weights sum to zero (#12856)
### What problem does this PR solve?

When all fulltext_search_columns use explicit weight 0 (e.g. "col^0"),
weight_sum is 0 and dividing by it raises ZeroDivisionError. Use equal
weights 1/n when weight_sum <= 0 and n > 0; otherwise normalize as
before.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [x] Refactoring
2026-01-28 14:38:03 +08:00
qinling0210
9a5208976c Put document metadata in ES/Infinity (#12826)
### What problem does this PR solve?

Put document metadata in ES/Infinity.

Index name of meta data: ragflow_doc_meta_{tenant_id}

### Type of change

- [x] Refactoring
2026-01-28 13:29:34 +08:00
Zhichang Yu
fd11aca8e5 feat: Implement pluggable multi-provider sandbox architecture (#12820)
## Summary

Implement a flexible sandbox provider system supporting both
self-managed (Docker) and SaaS (Aliyun Code Interpreter) backends for
secure code execution in agent workflows.

**Key Changes:**
-  Aliyun Code Interpreter provider using official
`agentrun-sdk>=0.0.16`
-  Self-managed provider with gVisor (runsc) security
-  Arguments parameter support for dynamic code execution
-  Database-only configuration (removed fallback logic)
-  Configuration scripts for quick setup

Issue #12479

## Features

### 🔌 Provider Abstraction Layer

**1. Self-Managed Provider** (`agent/sandbox/providers/self_managed.py`)
- Wraps existing executor_manager HTTP API
- gVisor (runsc) for secure container isolation
- Configurable pool size, timeout, retry logic
- Languages: Python, Node.js, JavaScript
- ⚠️ **Requires**: gVisor installation, Docker, base images

**2. Aliyun Code Interpreter**
(`agent/sandbox/providers/aliyun_codeinterpreter.py`)
- SaaS integration using official agentrun-sdk
- Serverless microVM execution with auto-authentication
- Hard timeout: 30 seconds max
- Credentials: `AGENTRUN_ACCESS_KEY_ID`, `AGENTRUN_ACCESS_KEY_SECRET`,
`AGENTRUN_ACCOUNT_ID`, `AGENTRUN_REGION`
- Automatically wraps code to call `main()` function

**3. E2B Provider** (`agent/sandbox/providers/e2b.py`)
- Placeholder for future integration

### ⚙️ Configuration System

- `conf/system_settings.json`: Default provider =
`aliyun_codeinterpreter`
- `agent/sandbox/client.py`: Enforces database-only configuration
- Admin UI: `/admin/sandbox-settings`
- Configuration validation via `validate_config()` method
- Health checks for all providers

### 🎯 Key Capabilities

**Arguments Parameter Support:**
All providers support passing arguments to `main()` function:
```python
# User code
def main(name: str, count: int) -> dict:
    return {"message": f"Hello {name}!" * count}

# Executed with: arguments={"name": "World", "count": 3}
# Result: {"message": "Hello World!Hello World!Hello World!"}
```

**Self-Describing Providers:**
Each provider implements `get_config_schema()` returning form
configuration for Admin UI

**Error Handling:**
Structured `ExecutionResult` with stdout, stderr, exit_code,
execution_time

## Configuration Scripts

Two scripts for quick Aliyun sandbox setup:

**Shell Script (requires jq):**
```bash
source scripts/configure_aliyun_sandbox.sh
```

**Python Script (interactive):**
```bash
python3 scripts/configure_aliyun_sandbox.py
```

## Testing

```bash
# Unit tests
uv run pytest agent/sandbox/tests/test_providers.py -v

# Aliyun provider tests
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v

# Integration tests (requires credentials)
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v

# Quick SDK validation
python3 agent/sandbox/tests/verify_sdk.py
```

**Test Coverage:**
- 30 unit tests for provider abstraction
- Provider-specific tests for Aliyun
- Integration tests with real API
- Security tests for executor_manager

## Documentation

- `docs/develop/sandbox_spec.md` - Complete architecture specification
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration from legacy
sandbox
- `agent/sandbox/tests/QUICKSTART.md` - Quick start guide
- `agent/sandbox/tests/README.md` - Testing documentation

## Breaking Changes

⚠️ **Migration Required:**

1. **Directory Move**: `sandbox/` → `agent/sandbox/`
   - Update imports: `from sandbox.` → `from agent.sandbox.`

2. **Mandatory Configuration**: 
   - SystemSettings must have `sandbox.provider_type` configured
   - Removed fallback default values
- Configuration must exist in database (from
`conf/system_settings.json`)

3. **Aliyun Credentials**:
   - Requires `AGENTRUN_*` environment variables (not `ALIYUN_*`)
   - `AGENTRUN_ACCOUNT_ID` is now required (Aliyun primary account ID)

4. **Self-Managed Provider**:
   - gVisor (runsc) must be installed for security
   - Install: `go install gvisor.dev/gvisor/runsc@latest`

## Database Schema Changes

```python
# SystemSettings.value: CharField → TextField
api/db/db_models.py: Changed for unlimited config length

# SystemSettingsService.get_by_name(): Fixed query precision
api/db/services/system_settings_service.py: startswith → exact match
```

## Files Changed

### Backend (Python)
- `agent/sandbox/providers/base.py` - SandboxProvider ABC interface
- `agent/sandbox/providers/manager.py` - ProviderManager
- `agent/sandbox/providers/self_managed.py` - Self-managed provider
- `agent/sandbox/providers/aliyun_codeinterpreter.py` - Aliyun provider
- `agent/sandbox/providers/e2b.py` - E2B provider (placeholder)
- `agent/sandbox/client.py` - Unified client (enforces DB-only config)
- `agent/tools/code_exec.py` - Updated to use provider system
- `admin/server/services.py` - SandboxMgr with registry & validation
- `admin/server/routes.py` - 5 sandbox API endpoints
- `conf/system_settings.json` - Default: aliyun_codeinterpreter
- `api/db/db_models.py` - TextField for SystemSettings.value
- `api/db/services/system_settings_service.py` - Exact match query

### Frontend (TypeScript/React)
- `web/src/pages/admin/sandbox-settings.tsx` - Settings UI
- `web/src/services/admin-service.ts` - Sandbox service functions
- `web/src/services/admin.service.d.ts` - Type definitions
- `web/src/utils/api.ts` - Sandbox API endpoints

### Documentation
- `docs/develop/sandbox_spec.md` - Architecture spec
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration guide
- `agent/sandbox/tests/QUICKSTART.md` - Quick start
- `agent/sandbox/tests/README.md` - Testing guide

### Configuration Scripts
- `scripts/configure_aliyun_sandbox.sh` - Shell script (jq)
- `scripts/configure_aliyun_sandbox.py` - Python script

### Tests
- `agent/sandbox/tests/test_providers.py` - 30 unit tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter.py` - Provider tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py` -
Integration tests
- `agent/sandbox/tests/verify_sdk.py` - SDK validation

## Architecture

```
Admin UI → Admin API → SandboxMgr → ProviderManager → [SelfManaged|Aliyun|E2B]
                                      ↓
                                  SystemSettings
```

## Usage

### 1. Configure Provider

**Via Admin UI:**
1. Navigate to `/admin/sandbox-settings`
2. Select provider (Aliyun Code Interpreter / Self-Managed)
3. Fill in configuration
4. Click "Test Connection" to verify
5. Click "Save" to apply

**Via Configuration Scripts:**
```bash
# Aliyun provider
export AGENTRUN_ACCESS_KEY_ID="xxx"
export AGENTRUN_ACCESS_KEY_SECRET="yyy"
export AGENTRUN_ACCOUNT_ID="zzz"
export AGENTRUN_REGION="cn-shanghai"
source scripts/configure_aliyun_sandbox.sh
```

### 2. Restart Service

```bash
cd docker
docker compose restart ragflow-server
```

### 3. Execute Code in Agent

```python
from agent.sandbox.client import execute_code

result = execute_code(
    code='def main(name: str) -> dict: return {"message": f"Hello {name}!"}',
    language="python",
    timeout=30,
    arguments={"name": "World"}
)

print(result.stdout)  # {"message": "Hello World!"}
```

## Troubleshooting

### "Container pool is busy" (Self-Managed)
- **Cause**: Pool exhausted (default: 1 container in `.env`)
- **Fix**: Increase `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` to 5+

### "Sandbox provider type not configured"
- **Cause**: Database missing configuration
- **Fix**: Run config script or set via Admin UI

### "gVisor not found"
- **Cause**: runsc not installed
- **Fix**: `go install gvisor.dev/gvisor/runsc@latest && sudo cp
~/go/bin/runsc /usr/local/bin/`

### Aliyun authentication errors
- **Cause**: Wrong environment variable names
- **Fix**: Use `AGENTRUN_*` prefix (not `ALIYUN_*`)

## Checklist

- [x] All tests passing (30 unit tests + integration tests)
- [x] Documentation updated (spec, migration guide, quickstart)
- [x] Type definitions added (TypeScript)
- [x] Admin UI implemented
- [x] Configuration validation
- [x] Health checks implemented
- [x] Error handling with structured results
- [x] Breaking changes documented
- [x] Configuration scripts created
- [x] gVisor requirements documented

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-28 13:28:21 +08:00
Yongteng Lei
b57c82b122 Feat: add kimi-k2.5 (#12852)
### What problem does this PR solve?

Add kimi-k2.5

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-28 12:41:20 +08:00
Stephen Hu
3a8c848af5 Fix:OSConnection.create_idx 4 arguments (#12862)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/12858

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-28 12:41:01 +08:00
balibabu
fe99905a2b Refactor: Remove the brute-force deduplication method for agent logs. (#12864)
### What problem does this PR solve?

Refactor: Remove the brute-force deduplication method for agent logs.

### Type of change

- [x] Refactoring
2026-01-28 12:04:30 +08:00
Jin Hai
591870eb6e Update quickstart (#12866)
### What problem does this PR solve?

To notify developer use the correct release.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-28 11:06:17 +08:00
BitToby
df3d044f03 fix: enable auto-resize for chat input textarea (#12836)
Closes #12803

### What problem does this PR solve?
The chat input textarea in the Chat UI (and Embed UI) has a fixed height
and cannot be resized, causing poor UX when users type messages longer
than 2 sentences. The input becomes cramped and difficult to read/edit.
**Root cause:** The `Textarea` component in
[NextMessageInput](cci:1://file:///ragflow/web/src/components/message-input/next.tsx:62:0-290:1)
had `resize-none` and `field-sizing-content` CSS classes that prevented
resizing, and the existing `autoSize` prop was not being utilized.

**Solution:** 
- Removed `resize-none` and `field-sizing-content` classes
- Added `autoSize={{ minRows: 1, maxRows: 8 }}` to enable auto-expand
- Added `max-h-40` class to limit maximum height to 160px
The textarea now auto-expands from 1 to 8 rows as users type longer
messages.

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-28 09:53:02 +08:00
Magicbook1108
ee654f08d2 Refact: update description for max_token in embedding #12792 (#12845)
### What problem does this PR solve?

Refact: update description for max_token in embedding #12792

### Type of change


- [x] Refactoring

Co-authored-by: Liu An <asiro@qq.com>
2026-01-28 09:52:32 +08:00
writinwaters
ceff119f89 Docs: Added build Ecommerce customer support guide (#12832)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2026-01-28 09:48:54 +08:00
Liu An
c2e8f90023 feat(ci): Add Redis service port configuration to test environment (#12855)
### What problem does this PR solve?

Added Redis port calculation and environment variable export to support
Redis service in test environment. The port is dynamically assigned
based on runner number to prevent conflicts during parallel test
execution. Removed by #12685

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-28 09:27:47 +08:00
Jin Hai
702b5b35e8 Fix error handle in RAGFlow CLI (#12829)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-27 17:22:23 +08:00
Yongteng Lei
2a758402ad Fix: Hunyuan cannot work properly (#12843)
### What problem does this PR solve?

Hunyuan cannot work properly

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-27 17:04:53 +08:00
Angel98518
e77168feba Fix: Handle whitespace-only question in /retrieval endpoint (#12831)
## Description

This PR fixes issue #12805 by adding validation to handle
whitespace-only questions in the `/retrieval` endpoint.

## Problem

Sending a single space `" "` as the `question` parameter to `/retrieval`
crashes the request with an `AssertionError`. This happens because:
1. The endpoint doesn't trim or validate the question parameter
2. A whitespace-only string is treated as valid input
3. The retrieval logic only checks for empty strings (which are falsy),
but `" "` is truthy
4. Invalid match expressions are constructed, causing an assertion
failure in the Elasticsearch layer

## Solution

- Trim whitespace from the question parameter before processing
- Return an empty result for whitespace-only or empty questions
- Prevents the AssertionError and provides expected behavior

## Changes

- Added whitespace trimming and validation in `api/apps/sdk/doc.py`
- Returns empty result early if question is empty after trimming

## Testing

- Tested with single space input - now returns empty result instead of
crashing
- Tested with empty string - returns empty result
- Tested with normal questions - works as expected

Fixes #12805

Co-authored-by: Daniel <daniel@example.com>
2026-01-27 15:57:47 +08:00
Stephen Hu
52da81cf9e Fix:Redis configuration template error in v0.22.1 (#12685)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/12674

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-27 12:47:46 +08:00
Mathias Panzenböck
b36d9744ae shortcut metadata_condition if there is none (#12835)
### What problem does this PR solve?

If no `metadata_condition` parameter is given then don't load the
metadata of all documents into memory. Instead just pass `doc_ids` as
`None` to the `retrieval()` method, which means to use all documents of
the given datasets.

This is relevant if you have *a lot* of documents!

### Type of change

- [x] Performance Improvement
2026-01-27 12:45:58 +08:00
Yongteng Lei
c8338dec57 Refa: convert RAGFlow MCP server from sync to async (#12834)
### What problem does this PR solve?

Convert RAGFlow MCP server from sync to async.

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-01-27 12:45:43 +08:00
Yongteng Lei
f096917eeb Fix: overlap cannot be properly applied (#12828)
### What problem does this PR solve?

Overlap cannot be properly applied.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-27 12:43:01 +08:00
Jonah Hartmann
413956e9dd Feat: Add German language support for agent template and various UI elements (#12830)
### What problem does this PR solve?

This PR updates and extends the german language support in the frontend.
Additionally two more elements are handled dynamically now. The
interactive Agent is also titled and described in german now.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Jakob <16180662+hauberj@users.noreply.github.com>
2026-01-27 12:42:44 +08:00
Zhichang Yu
6404af0a91 Bump to infinity v0.7.0-dev2 (#12839)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev2

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-27 11:48:02 +08:00
Lin Manhui
27a36344d4 Feat: Support PaddleOCR-VL-1.5 interface (#12819)
### What problem does this PR solve?

This PR adds support to PaddleOCR-VL-1.5 interface to the PaddleOCR PDF
Parser.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-27 09:49:46 +08:00
Kevin Hu
e20d56a34c Fix: metadata update issue (#12815)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-26 18:02:44 +08:00
chanx
1d93519cb2 Fix: Issues with metadata parameter addition failures and single-file chunk saving failures. (#12818)
### What problem does this PR solve?

Fix: Issues with metadata parameter addition failures and single-file
chunk saving failures.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-26 18:00:40 +08:00
Yongteng Lei
13076bb87b Fix: Parent chunking fails on DOCX files (#12822)
### What problem does this PR solve?

Fixes parent chunking fails on DOCX files.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-26 17:55:09 +08:00
balibabu
e04cd99ae2 Feat: Add the history field to the agent's system variables. #7322 (#12823)
### What problem does this PR solve?

Feat: Add the history field to the agent's system variables. #7322

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-26 17:54:30 +08:00
Jin Hai
41905e2569 Update RAGFlow CLI (#12816)
### What problem does this PR solve?

Improve performance slightly.

### Type of change

- [x] Refactoring
- [x] Performance Improvement

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-26 12:58:04 +08:00
Stephen Hu
0782a7d3c6 Refactor: improve task cancellation checks in RAPTOR (#12813)
### What problem does this PR solve?
Introduced a helper method _check_task_canceled to centralize and
simplify task cancellation checks throughout
RecursiveAbstractiveProcessing4TreeOrganizedRetrieval. This reduces code
duplication and improves maintainability.

### Type of change

- [x] Refactoring
2026-01-26 11:34:54 +08:00
LIRUI YU
4236a62855 Fix: Cancel tasks before document or datasets deletion to prevent queue blocking (#12799)
### What problem does this PR solve?
When deleting the knowledge base, the records in the Document and
Knowledgebase tables are immediately deleted
But there are still a large number of pending task messages in the Redis
queue (asynchronous queue) if you did not click on stopping tasks before
deleting knowledge base.
TaskService.get_task() uses a JOIN query to associate three tables (Task
← Document ← Knowledgebase)
Since Document/Knowledgebase have been deleted, the JOIN returns an
empty result, even though the Task records still exist
task-executor considers the task does not exist ("collect task xxx is
unknown"), can only skip and warn
log:2026-01-23 16:43:21,716 WARNING 1190179 collect task
110fbf70f5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:21,818 WARNING 1190179 collect task
11146bc4f5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:21,918 WARNING 1190179 collect task
111c3336f5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:22,021 WARNING 1190179 collect task
112471b8f5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:26,719 WARNING 1190179 collect task
112e855ef5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:26,734 WARNING 1190179 collect task
1134380af5bd11f0945a23b0930487df is unknown
2026-01-23 16:43:26,834 WARNING 1190179 collect task
1138cb2cf5bd11f0945a23b0930487df is unknown

As a consequence, a large number of such tasks occupy the queue
processing capacity, causing new tasks to queue and wait

<img width="1910" height="947"
alt="9a00f2e0-9112-4dbb-b357-7f66b8eb5acf"
src="https://github.com/user-attachments/assets/0e1227c2-a2df-4ef3-ba8f-e04c3f6ef0e1"
/>


Solution
Add logic to stop all ongoing tasks before deleting the knowledge base
and Tasks


### Type of change
- Bug Fix (non-breaking change which fixes an issue)
2026-01-26 10:45:59 +08:00
Da22wei
9afb5bc136 Add Copilot setting and conventions (#12807)
### What problem does this PR solve?

Added project instructions for setting up and running the application.

### Type of change

- [x] Documentation Update
2026-01-26 10:44:20 +08:00
Kevin Hu
f0fcf8aa9a Fix: reset conversation variables. (#12814)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-26 10:43:57 +08:00
Jin Hai
274fc5ffaa Fix RAGFlow CLI bug (#12811)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-25 23:08:59 +08:00
writinwaters
80a16e71df Docs: Added webhook specific configuration tips (#12802)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2026-01-23 22:09:49 +08:00
balibabu
6220906164 Fix: Fixed the error on the login page. (#12801)
### What problem does this PR solve?

Fix:  Fixed the error on the login page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 18:58:54 +08:00
Jimmy Ben Klieve
fa5284361c feat: support admin assign superuser in admin ui (#12798)
### What problem does this PR solve?

Allow superuser(admin) to grant or revoke other superuser.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-23 18:08:46 +08:00
Lynn
f3923452df Fix: add tokenized content (#12793)
### What problem does this PR solve?

Add tokenized content es field to query zh message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 16:56:03 +08:00
chanx
11470906cf Fix: Metadata time Picker (#12796)
### What problem does this PR solve?

Fix: Metadata time Picker

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 16:55:43 +08:00
Jin Hai
e1df82946e RAGFlow CLI: ping server before input password when login user (#12791)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-23 15:03:05 +08:00
Kevin Hu
08c01b76d5 Fix: missing parent chunk issue. (#12789)
### What problem does this PR solve?

Close #12783

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 12:54:08 +08:00
apps-lycusinc
678392c040 feat(deepdoc): add configurable ONNX thread counts and GPU memory shrinkage (#12777)
### What problem does this PR solve?

This PR addresses critical memory and CPU resource management issues in
high-concurrency environments (multi-worker setups):

GPU Memory Exhaustion (OOM): Currently, onnxruntime-gpu uses an
aggressive memory arena that does not effectively release VRAM back to
the system after a task completes. In multi-process worker setups ($WS >
4), this leads to BFCArena allocation failures and OOM errors as workers
"hoard" VRAM even when idle. This PR introduces an optional GPU Memory
Arena Shrinkage toggle to mitigate this issue.

CPU Oversubscription: ONNX intra_op and inter_op thread counts are
currently hardcoded to 2. When running many workers, this causes
significant CPU context-switching overhead and degrades performance.
This PR makes these values configurable to match the host's actual CPU
core density.

Multi-GPU Support: The memory management logic has been improved to
dynamically target the correct device_id, ensuring stability on systems
with multiple GPUs.

Transparency: Added detailed initialization logs to help administrators
verify and troubleshoot their ONNX session configurations.

 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: shakeel <shakeel@lollylaw.com>
2026-01-23 11:36:28 +08:00
Julien Deveaux
6be197cbb6 Fix: Use tiktoken for proper token counting in OpenAI-compatible endpoint #7850 (#12760)
### What problem does this PR solve?
The OpenAI-compatible chat endpoint
(`/chats_openai/<chat_id>/chat/completions`) was not returning accurate
token
usage in streaming responses. The token counts were either missing or
inaccurate because the underlying LLM API
responses weren't being properly parsed for usage data.
This PR adds proper token counting using tiktoken (cl100k_base encoding)
as a fallback when the LLM API doesn't provide usage data in streaming
chunks. This ensures clients always receive token usage information in
the
response, which is essential for billing and quota management.
**Changes:**
- Add tiktoken-based token counting for streaming responses in
OpenAI-compatible endpoint
- Ensure `usage` field is always populated in the final streaming chunk
- Add unit tests for token usage calculation
  Fixes #7850

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 09:36:21 +08:00
balibabu
8dd4a41bf8 Feat: Add a web search button to the chat box on the chat page. (#12786)
### What problem does this PR solve?

Feat: Add a web search button to the chat box on the chat page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-23 09:33:50 +08:00
chanx
e9453a3971 Fix: Metadata supports precise time selection (#12785)
### What problem does this PR solve?

Fix: Metadata supports precise time selection

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-23 09:33:34 +08:00
balibabu
7c9b6e032b Fix: The minimum size of the historical message window for the classification operator is 1. #12778 (#12779)
### What problem does this PR solve?

Fix: The minimum size of the historical message window for the
classification operator is 1. #12778

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-22 19:45:25 +08:00
Kevin Hu
3beb85efa0 Feat: enhance metadata arranging. (#12745)
### What problem does this PR solve?
#11564

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-22 15:34:08 +08:00
LIRUI YU
bc7b864a6c top_k parameter ignored, always returned page_size results (#12753)
### What problem does this PR solve?
**Backend**
\rag\nlp\search.py
*Before the fix*
The top_k parameter was not applied to limit the total number of chunks,
and the rerank model also uses the exact whole valid_idx rather than
assigning valid_idx = valid_idx[:top] firstly.
*After the fix*
The top_k limit is applied to the total results before pagination, using
a default value of top = 1024 if top_k is not modified.

session.py
*Before the fix:*
When the frontend calls the retrieval API with `search_id`, the backend
only reads `meta_data_filter` from the saved `search_config`. The
`rerank_id`, `top_k`, `similarity_threshold`, and
`vector_similarity_weight` parameters are only taken from the direct
request body. Since the frontend doesn't pass these parameters
explicitly (it only passes `search_id`), they always fall back to
default values:
- `similarity_threshold` = 0.0
- `vector_similarity_weight` = 0.3
- `top_k` = 1024
- `rerank_id` = "" (no rerank)
This means user settings saved in the Search Settings page have no
effect on actual search results.

*After the fix:*
When a `search_id` is provided, the backend now reads all relevant
configuration from the saved `search_config`, including `rerank_id`,
`top_k`, `similarity_threshold`, and `vector_similarity_weight`. Request
parameters can still override these values if explicitly provided,
allowing flexibility. The rerank model is now properly instantiated
using the configured `rerank_id`, making the rerank feature actually
work.



**Frontend** 
\web\src\pages\next-search\search-setting.tsx
*Before the fix*
search-setting.tsx file, the top_k input box is only displayed when
rerank is enabled (wrapped in the rerankModelDisabled condition). If the
rerank switch is turned off, the top_k input field will be hidden, but
the form value will remain unchanged. In other words: - When rerank is
enabled, users can modify top_k (default 1024). - When rerank is
disabled, top_k retains the previous value, but it's not visible on the
interface. Therefore, the backend will always receive the top_k
parameter; it's just that the frontend UI binds this configuration item
to the rerank switch. When rerank is turned off, top_k will not
automatically reset to 1024, but will retain its original value.
*After the fix*
On the contrary, if we switch off the button rerank model, the value
top-k will be reset to 1024. By the way, If we use top-k in an
individual method, rather than put it into the method retrieval, we can
control it separately



Now all methods valid
Using rerank

<img width="2378" height="1565" alt="Screenshot 2026-01-21 190206"
src="https://github.com/user-attachments/assets/fa2b0df0-1334-4ca3-b169-da6c5fd59935"
/>

Not using rerank
<img width="2596" height="1559" alt="Screenshot 2026-01-21 190229"
src="https://github.com/user-attachments/assets/c5a80522-a0e1-40e7-b349-42fe86df3138"
/>




Before fixing they are the same

### Type of change
- Bug Fix (non-breaking change which fixes an issue)
2026-01-22 15:33:42 +08:00
zhanxin.xu
93091f4551 [Feat]Automatic table orientation detection and correction (#12719)
### What problem does this PR solve?
This PR introduces automatic table orientation detection and correction
within the PDF parser. This ensures that tables in PDFs are correctly
oriented before structure recognition, improving overall parsing
accuracy.

### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-01-22 12:47:55 +08:00
会敲代码的喵
2d9e7b4acd Fix: aliyun oss need to use s3 signature_version (#12766)
### What problem does this PR solve?

Aliyun OSS do not support boto s4 signature_version which will lead to
an error:

```
botocore.exceptions.ClientError: An error occurred (InvalidArgument) when calling the PutObject operation: aws-chunked encoding is not supported with the specified x-amz-content-sha256 value
```

According to aliyun oss docs, oss_conn need to use s3 signature_version.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-22 11:43:55 +08:00
天海蒼灆
6f3f69b62e Feat: API adds audio to text and text to speech functions (#12764)
### What problem does this PR solve?

API adds audio to text and text to speech functions

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-22 11:20:26 +08:00
chanx
bfd5435087 Fix: After deleting metadata in batches, the selected items need to be cleared. (#12767)
### What problem does this PR solve?

Fix: After deleting metadata in batches, the selected items need to be
cleared.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-22 11:20:11 +08:00
balibabu
0e9fe68110 Feat: Adjust the icons in the chat page's collapsible panel. (#12755)
### What problem does this PR solve?

Feat: Adjust the icons in the chat page's collapsible panel.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-22 09:48:44 +08:00
Jin Hai
89f438fe45 Add ping command to test ping API (#12757)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-22 00:18:29 +08:00
Jin Hai
2e2c8f6ca9 Add more commands to RAGFlow CLI (#12731)
### What problem does this PR solve?

This PR is going to make RAGFlow CLI to access RAGFlow as normal user,
and work as the a testing tool for RAGFlow server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-21 18:49:52 +08:00
balibabu
6cd4fd91e6 Fix: Allow classification operators to be followed by other classification operators. #9082 (#12744)
### What problem does this PR solve?

Fix: Allow classification operators to be followed by other
classification operators. #9082

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-21 16:24:39 +08:00
chanx
83e17d8c4a Fix: Optimize the metadata code structure to implement metadata list structure functionality. (#12741)
### What problem does this PR solve?

Fix: Optimize the metadata code structure to implement metadata list
structure functionality.

#11564

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-21 16:15:43 +08:00
balibabu
e1143d40bc Feat: Add a think button to the chat box. #12742 (#12743)
### What problem does this PR solve?

Feat: Add a think button to the chat box. #12742
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-21 15:39:18 +08:00
Liu An
f98abf14a8 Refa(test): improve code formatting and remove debug prints (#12739)
### What problem does this PR solve?

- Improving code formatting and consistency
- Removing debug print statements

### Type of change

- [x] Refactoring
2026-01-21 14:53:17 +08:00
Liu An
2a87778e10 Chore(ci): use new Web API test cases in CI (#12738)
### What problem does this PR solve?

- Update pytest commands to use new test directory structure

### Type of change

- [x] chore(ci)
2026-01-21 14:53:05 +08:00
Stephen Hu
5836823187 Refactor:better handle list agent api desc param (#12733)
### What problem does this PR solve?
better handle list agent api desc param

### Type of change

- [x] Refactoring
2026-01-21 13:09:27 +08:00
chanx
5a7026cf55 Feat: Improve metadata logic (#12730)
### What problem does this PR solve?

Feat: Improve metadata logic

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-21 11:31:26 +08:00
LGRY
bc7935d627 feat: add batch delete for conversations in chat(web) (#12584)
Resolves #12572

## What problem does this PR solve?
The conversation list in chat sessions previously only supported
deleting conversations one by one. This was inefficient when users
needed to clean up multiple conversations. This PR adds batch delete
functionality to improve user experience.

## Type of change
 - [x] New Feature (non-breaking change which adds functionality)

## Specific changes
  - Add selection mode with checkboxes for conversation list
  - Add batch delete functionality with custom icons
  - Add internationalization support (en/zh)
  - Use existing removeConversation API which supports batch deletion

## UI modification status
  - Default: Show [+] and [batch delete icon]
  - Selection mode: Show checkboxes, keep [+] and [select all icon]
  - Items selected: Show [return icon] and [red trash icon]"

### Repair Comparison
**1.Before Repair**
<img width="982" height="1221" alt="image"
src="https://github.com/user-attachments/assets/8a80f7c0-7da6-41ec-9d1a-ac887ede96ba"
/>


**2.After Repair**
<img width="1273" height="919" alt="新增批量删除效果图"
src="https://github.com/user-attachments/assets/e179bdf3-3779-4bd5-84b6-8e24780a22ea"
/>

---
Co-authored-by: Gongzi

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-20 19:13:53 +08:00
Haipeng LI
7787085664 Doc: add README for test (#12728)
### What problem does this PR solve?

We added instructions on how to test RAGFlow in test/README.md.

### Type of change

- [x] Documentation Update
2026-01-20 19:12:35 +08:00
6ba3i
960ecd3158 Feat: update and add new tests for web api apps (#12714)
### What problem does this PR solve?

This PR adds missing web API tests (system, search, KB, LLM, plugin,
connector). It also addresses a contract mismatch that was causing test
failures: metadata updates did not persist new keys (update‑only
behavior).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): Test coverage expansion and test helper
instrumentation
2026-01-20 19:12:15 +08:00
6ba3i
aee9860970 Make document change-status idempotent for Infinity doc store (#12717)
### What problem does this PR solve?

This PR makes the document change‑status endpoint idempotent under the
Infinity doc store. If a document already has the requested status, the
handler returns success without touching the engine, preventing
unnecessary updates and avoiding missing‑table errors while keeping
responses consistent.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 19:11:21 +08:00
Jimmy Ben Klieve
9ebbc5a74d chore: redirect to login page if api reports unauthorized in admin page (#12726)
### What problem does this PR solve?

Auto redirect to login page if API reports `401: Unauthroized` in ANY
**Admin** page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 18:58:13 +08:00
Jimmy Ben Klieve
1c65f64bda fix: missing route for user detail page (#12725)
### What problem does this PR solve?

Add missing route for navigating to `/admin/users/:id`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 18:55:44 +08:00
Kevin Hu
32841549c1 Fix: Not within a request context (#12723)
### What problem does this PR solve?

ERROR    1819426 Unhandled exception during request
Traceback (most recent call last):
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/document_app.py](http://github.com/infiniflow/ragflow/api/apps/document_app.py)",
line 639, in run
    return await thread_pool_exec(_run_sync)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/common/misc_utils.py](http://github.com/infiniflow/ragflow/common/misc_utils.py)",
line 132, in thread_pool_exec
return await loop.run_in_executor(_thread_pool_executor(), func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/futures.py", line 287, in __await__
    yield self  # This tells Task to wait for completion.
    ^^^^^^^^^^
  File "/usr/lib/python3.12/asyncio/tasks.py", line 385, in __wakeup
    future.result()
  File "/usr/lib/python3.12/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
File "/usr/lib/python3.12/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/document_app.py](http://github.com/infiniflow/ragflow/api/apps/document_app.py)",
line 593, in _run_sync
if not DocumentService.accessible(doc_id,
[current_user.id](http://current_user.id/)):
                                              ^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 318, in __get__
    obj = instance._get_current_object()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 526, in _get_current_object
    return get_name(local())
                    ^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/api/apps/__init__.py](http://github.com/infiniflow/ragflow/api/apps/__init__.py)",
line 97, in _load_user
    authorization = request.headers.get("Authorization")
                    ^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 318, in __get__
    obj = instance._get_current_object()
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File
"/home/qinling/[github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py](http://github.com/infiniflow/ragflow/.venv/lib/python3.12/site-packages/werkzeug/local.py)",
line 519, in _get_current_object
    raise RuntimeError(unbound_message) from None
RuntimeError: Not within a request context

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 16:56:41 +08:00
writinwaters
046d4ffdef Docs: Updated configuration file name (#12720)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2026-01-20 15:40:03 +08:00
longbingljw
4c4d434bc1 Unify MySQL configuration (#12644)
### What problem does this PR solve?

Align MySQL defaults between docker/.env and
docker/service_conf.yaml.template
close #12645

### Type of change

- [x] Other (please describe):Unify MySQL configuration
2026-01-20 13:42:22 +08:00
balibabu
80612bc992 Refactor: Replace antd with shadcn (#12718)
### What problem does this PR solve?

Refactor: Replace antd with shadcn
### Type of change

- [x] Refactoring
2026-01-20 13:38:54 +08:00
Kevin Hu
927db0b373 Refa: asyncio.to_thread to ThreadPoolExecutor to break thread limitat… (#12716)
### Type of change

- [x] Refactoring
2026-01-20 13:29:37 +08:00
lys1313013
120648ac81 fix: inaccurate error message when uploading multiple files containing an unsupported file type (#12711)
### What problem does this PR solve?

When uploading multiple files at once, if any of the files are of an
unsupported type and the blob is not removed, it triggers a
TypeError('Object of type bytes is not JSON serializable') exception.
This prevents the frontend from responding properly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 12:24:54 +08:00
E.G
f367189703 fix(raptor): handle missing vector fields gracefully (#12713)
## Summary

This PR fixes a `KeyError` crash when running RAPTOR tasks on documents
that don't have the expected vector field.

## Related Issue

Fixes https://github.com/infiniflow/ragflow/issues/12675

## Problem

When running RAPTOR tasks, the code assumes all chunks have the vector
field `q_<size>_vec` (e.g., `q_1024_vec`). However, chunks may not have
this field if:
1. They were indexed with a **different embedding model** (different
vector size)
2. The embedding step **failed silently** during initial parsing
3. The document was parsed before the current embedding model was
configured

This caused a crash:
```
KeyError: 'q_1024_vec'
```

## Solution

Added defensive validation in `run_raptor_for_kb()`:

1. **Check for vector field existence** before accessing it
2. **Skip chunks** that don't have the required vector field instead of
crashing
3. **Log warnings** for skipped chunks with actionable guidance
4. **Provide informative error messages** suggesting users re-parse
documents with the current embedding model
5. **Handle both scopes** (`file` and `kb` modes)

## Changes

- `rag/svr/task_executor.py`: Added validation and error handling in
`run_raptor_for_kb()`

## Testing

1. Create a knowledge base with an embedding model
2. Parse documents
3. Change the embedding model to one with a different vector size
4. Run RAPTOR task
5. **Before**: Crashes with `KeyError`
6. **After**: Gracefully skips incompatible chunks with informative
warnings

---

<!-- Gittensor Contribution Tag: @GlobalStar117 -->

Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
2026-01-20 12:24:20 +08:00
writinwaters
1b1554c563 Docs: Added ingestion pipeline quickstart (#12708)
### What problem does this PR solve?

Added ingestion pipeline quickstart

### Type of change

- [x] Documentation Update
2026-01-20 09:48:32 +08:00
balibabu
59f3da2bdf Fix: The time zone is unable to update properly in the database #12696 (#12704)
### What problem does this PR solve?

Fix: The time zone is unable to update properly in the database #12696

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-20 09:47:16 +08:00
qinling0210
b40d639fdb Add dataset with table parser type for Infinity and answer question in chat using SQL (#12541)
### What problem does this PR solve?

1) Create  dataset using table parser for infinity
2) Answer questions in chat using SQL

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-19 19:35:14 +08:00
balibabu
05da2a5872 Fix: When large models output data rapidly, the scrollbar cannot remain at the bottom. #12701 (#12702)
### What problem does this PR solve?



### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-19 19:09:41 +08:00
qinling0210
4fbaa4aae9 Bump to infinity v0.7.0-dev1 (#12699)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev1

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-19 16:36:03 +08:00
E.G
3188cd2659 fix: Ensure pip is available in venv for runtime installation (#12667)
## Summary

Fixes #12651

The Docker container was failing at startup with:
```
/ragflow/.venv/bin/python3: No module named pip
```

This occurred when `USE_DOCLING=true` because the `entrypoint.sh` tries
to use `uv pip install` to install docling at runtime.

## Root Cause

As explained in the issue:
1. `uv sync` creates a minimal, production-focused environment **without
pip**
2. The production stage copies the venv from builder
3. Runtime commands using `uv pip install` fail because pip is not
present

## Solution

Added `python -m ensurepip --upgrade` after `uv sync` in the Dockerfile
to ensure pip is available in the virtual environment:

```dockerfile
uv sync --python 3.12 --frozen && \
# Ensure pip is available in the venv for runtime package installation (fixes #12651)
.venv/bin/python3 -m ensurepip --upgrade
```

This is a minimal change that:
- Ensures pip is installed during build time
- Doesn't change any other behavior
- Allows runtime package installation via `uv pip install` to work

---
This is a Gittensor contribution.
gittensor:user:GlobalStar117

Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
2026-01-19 16:08:14 +08:00
longbingljw
c4a982e9fa feat: add seekdb which is lite version of oceanbase (#12692)
### What problem does this PR solve?

Add seekdb as doc_engine wich is the lite version of oceanbase.
close #12691
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-19 16:07:43 +08:00
Hwwwww-dev
b27dc26be3 fix: Update answer concatenation logic to handle overlapping values (#12676)
### What problem does this PR solve?

Update answer concatenation logic to handle overlapping values

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-19 16:06:36 +08:00
LIRUI YU
ab1836f216 An issue involving node.js OOM happened (#12690)
### What problem does this PR solve?
The Node.js memory issue occurred due to JavaScript heap exhaustion
during the Vite build process sometimes. Here's what happened:
export NODE_OPTIONS="--max-old-space-size=4096" && \
Root Cause:
The Node.js memory issue occurred due to JavaScript heap exhaustion
during the Vite build process sometimes. Here's what happened:

Root Cause:

When building the web frontend with npm run build, Vite needs to bundle,
transform, and optimize all JavaScript/TypeScript code
Node.js has a default maximum heap size of ~2GB
The RAGFlow web application is large enough that the build process
exceeded this limit
This triggered garbage collection failures ("Ineffective mark-compacts
near heap limit") and eventually crashed with exit code 134 (SIGABRT)

The solution I attempted:
I did not find a simple method to reduce the use of memory for node.js,
so I added NODE_OPTIONS=--max-old-space-size=4096 to allocate 4GB heap
memory for Node.js during the build.

### Type of change
- Bug Fix (non-breaking change which fixes an issue)

=> ERROR [builder 6/8] RUN --mount=type=cache,id=ragflow_npm,target=/ro
53.3s
[builder 6/8] RUN
--mount=type=cache,id=ragflow_npm,target=/root/.npm,sharing=locked cd
web && npm install && npm run build:
4.551
4.551 > prepare
4.551 > cd .. && husky web/.husky
4.551
4.810 .git can't be found
4.833 added 7 packages in 4s
4.833
4.833 499 packages are looking for funding
4.833 run npm fund for details
5.206
5.206 > build
5.206 > vite build --mode production
5.206
5.939 vite v7.3.0 building client environment for production...
6.169 transforming...
6.472
6.472 WARN
6.472
6.472
6.472 WARN warn - As of Tailwind CSS v3.3, the @tailwindcss/line-clamp
plugin is now included by default.
6.472
6.472
6.472 WARN warn - Remove it from the plugins array in your configuration
to eliminate this warning.
6.472
53.14
53.14 <--- Last few GCs --->
53.14
53.14 [41:0x55f82d0] 47673 ms: Scavenge (reduce) 2041.5 (2086.0) ->
2038.7 (2079.7) MB, 6.11 / 0.00 ms (average mu = 0.330, current mu =
0.319) allocation failure;
53.14 [41:0x55f82d0] 47727 ms: Scavenge (reduce) 2039.4 (2079.7) ->
2038.7 (2080.2) MB, 5.34 / 0.00 ms (average mu = 0.330, current mu =
0.319) allocation failure;
53.14 [41:0x55f82d0] 47809 ms: Scavenge (reduce) 2039.6 (2080.2) ->
2038.7 (2080.2) MB, 4.59 / 0.00 ms (average mu = 0.330, current mu =
0.319) allocation failure;
53.14
53.14
53.14 <--- JS stacktrace --->
53.14
53.14 FATAL ERROR: Ineffective mark-compacts near heap limit Allocation
failed - JavaScript heap out of memory
53.14 ----- Native stack trace -----
53.14
53.14 1: 0xb76db1 node::OOMErrorHandler(char const*, v8::OOMDetails
const&) [node]
53.14 2: 0xee62f0 v8::Utils::ReportOOMFailure(v8::internal::Isolate*,
char const*, v8::OOMDetails const&) [node]
53.14 3: 0xee65d7
v8::internal::V8::FatalProcessOutOfMemory(v8::internal::Isolate*, char
const*, v8::OOMDetails const&) [node]
53.14 4: 0x10f82d5 [node]
53.14 5: 0x10f8864
v8::internal::Heap::RecomputeLimits(v8::internal::GarbageCollector)
[node]
53.14 6: 0x110f754
v8::internal::Heap::PerformGarbageCollection(v8::internal::GarbageCollector,
v8::internal::GarbageCollectionReason, char const*) [node]
53.14 7: 0x110ff6c
v8::internal::Heap::CollectGarbage(v8::internal::AllocationSpace,
v8::internal::GarbageCollectionReason, v8::GCCallbackFlags) [node]
53.14 8: 0x11120ca v8::internal::Heap::HandleGCRequest() [node]
53.14 9: 0x107d737 v8::internal::StackGuard::HandleInterrupts() [node]
53.15 10: 0x151fb9a v8::internal::Runtime_StackGuard(int, unsigned
long*, v8::internal::Isolate*) [node]
53.15 11: 0x1959ef6 [node]
53.22 Aborted

[+] up 0/1
⠙ Image docker-ragflow Building 58.0s
Dockerfile:161

160 | COPY docs docs

161 | >>> RUN
--mount=type=cache,id=ragflow_npm,target=/root/.npm,sharing=locked \

162 | >>> cd web && npm install && npm run build

163 |

failed to solve: process "/bin/bash -c cd web && npm install && npm run
build" did not complete successfully: exit code: 134

View build details:
docker-desktop://dashboard/build/default/default/j68n2ke32cd8bte4y8fs471au
2026-01-19 14:28:38 +08:00
Loganaden Velvindron
7a53d2dd97 Fix CVE-2025-59466 (#12679)
### What problem does this PR solve?


https://nodejs.org/en/blog/vulnerability/january-2026-dos-mitigation-async-hooks


### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-01-19 13:15:15 +08:00
n1n.ai
f3d347f55f feat: Add n1n provider (#12680)
This PR adds n1n as an LLM provider to RAGFlow.

Co-authored-by: Qun <qun@ip-10-5-5-38.us-west-2.compute.internal>
2026-01-19 13:12:42 +08:00
E.G
9da48ab0bd fix: Handle NaN/Infinity values in ExeSQL JSON response (#12666)
## Summary

Fixes #12631

When SQL query results contain NaN (Not a Number) or Infinity values
(e.g., from division by zero or other calculations), the JSON
serialization would fail because **NaN and Infinity are not valid JSON
values**.

This caused the agent interface to show 'undefined' error, as described
in the issue where `EXAMINE_TIMES` became `NaN` and broke the JSON
parsing.

## Root Cause

The `convert_decimals` function in `exesql.py` was only handling
`Decimal` types, but not `float` values that could be `NaN` or
`Infinity`.

When these invalid JSON values were serialized:
```json
{"EXAMINE_TIMES": NaN}  // Invalid JSON!
```

The frontend JSON parser would fail, causing the 'undefined' error.

## Solution

Extended `convert_decimals` to detect `float` values and convert
`NaN`/`Infinity` to `null` before JSON serialization:

```python
if isinstance(obj, float):
    if math.isnan(obj) or math.isinf(obj):
        return None
    return obj
```

This ensures all SQL results can be properly serialized to valid JSON.

---
This is a Gittensor contribution.
gittensor:user:GlobalStar117

Co-authored-by: GlobalStar117 <GlobalStar117@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-01-19 12:46:06 +08:00
Stephen Hu
4a7e40630b Refactor:memory delete will re-use super method (#12684)
### What problem does this PR solve?
memory delete will re-use super method

### Type of change

- [x] Refactoring
2026-01-19 12:45:37 +08:00
Jin Hai
d6897b6054 Fix chat error (#12693)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-19 12:45:14 +08:00
qinling0210
828ae1e82f Round float value of minimum_should_match (#12688)
### What problem does this PR solve?

In paragraph() of class FulltextQueryer, "len(keywords) / 10" should be
rounded to integer before set to minimum_should_match.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-19 11:39:33 +08:00
francisye19
57d189b483 fix: Correct gitlab_url access in sync_data_source.py (#12681)
### What problem does this PR solve?

Correct gitlab_url access. See
https://github.com/infiniflow/ragflow/blob/main/web/src/pages/user-setting/data-source/constant/index.tsx#L660-L666

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-19 11:01:34 +08:00
Mohan
0a8eb11c3d fix: Add proper error handling for database reconnection attempts (#12650)
## Problem
When database connection is lost, the reconnection logic had a bug: if
the first reconnect attempt failed, the second attempt was not wrapped
in error handling, causing unhandled exceptions.

## Solution
Added proper try-except blocks around the second reconnect attempt in
both MySQL and PostgreSQL database classes to ensure errors are properly
logged and handled.

## Changes
- Fixed `_handle_connection_loss()` in `RetryingPooledMySQLDatabase`
- Fixed `_handle_connection_loss()` in
`RetryingPooledPostgresqlDatabase`

Fixes #12294

---

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=158349177

Co-authored-by: SID <158349177+0xsid0703@users.noreply.github.com>
2026-01-19 09:48:10 +08:00
Jin Hai
38f0a92da9 Use RAGFlow CLI to replace RAGFlow Admin CLI (#12653)
### What problem does this PR solve?

```
$ python admin/client/ragflow_cli.py -t user -u aaa@aaa.com -p 9380

ragflow> list datasets;
ragflow> list default models;
ragflow> show version;

```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-17 17:52:38 +08:00
writinwaters
067ddcbf23 Docs: Added configure memory (#12665)
### What problem does this PR solve?

As title.

### Type of change

- [x] Documentation Update
2026-01-17 17:49:19 +08:00
Hetavi Shah
46305ef35e Add User API Token Management to Admin API and CLI (#12595)
## Summary

This PR extends the RAGFlow Admin API and CLI with comprehensive user
API token management capabilities. Administrators can now generate,
list, and delete API tokens for users through both the REST API and the
Admin CLI interface.

## Changes

### Backend API (`admin/server/`)

#### New Endpoints
- **POST `/api/v1/admin/users/<username>/new_token`** - Generate a new
API token for a user
- **GET `/api/v1/admin/users/<username>/token_list`** - List all API
tokens for a user
- **DELETE `/api/v1/admin/users/<username>/token/<token>`** - Delete a
specific API token for a user

#### Service Layer Updates (`services.py`)
- Added `get_user_api_key(username)` - Retrieves all API tokens for a
user
- Added `save_api_token(api_token)` - Saves a new API token to the
database
- Added `delete_api_token(username, token)` - Deletes an API token for a
user

### Admin CLI (`admin/client/`)

#### New Commands
- **`GENERATE TOKEN FOR USER <username>;`** - Generate a new API token
for the specified user
- **`LIST TOKENS OF <username>;`** - List all API tokens associated with
a user
- **`DROP TOKEN <token> OF <username>;`** - Delete a specific API token
for a user

### Testing

Added comprehensive test suite in `test/testcases/test_admin_api/`:
- **`test_generate_user_api_key.py`** - Tests for API token generation
- **`test_get_user_api_key.py`** - Tests for listing user API tokens
- **`test_delete_user_api_key.py`** - Tests for deleting API tokens
- **`conftest.py`** - Shared test fixtures and utilities

## Technical Details

### Token Generation
- Tokens are generated using `generate_confirmation_token()` utility
- Each token includes metadata: `tenant_id`, `token`, `beta`,
`create_time`, `create_date`
- Tokens are associated with user tenants automatically

### Security Considerations
- All endpoints require admin authentication (`@check_admin_auth`)
- Tokens are URL-encoded when passed in DELETE requests to handle
special characters
- Proper error handling for unauthorized access and missing resources

### API Response Format
All endpoints follow the standard RAGFlow response format:
```json
{
  "code": 0,
  "data": {...},
  "message": "Success message"
}
```

## Files Changed

- `admin/client/admin_client.py` - CLI token management commands
- `admin/server/routes.py` - New API endpoints
- `admin/server/services.py` - Token management service methods
- `docs/guides/admin/admin_cli.md` - CLI documentation updates
- `test/testcases/test_admin_api/conftest.py` - Test fixtures
- `test/testcases/test_admin_api/test_user_api_key_management/*` - Test
suites

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Alexander Strasser <alexander.strasser@ondewo.com>
Co-authored-by: Hetavi Shah <your.email@example.com>
2026-01-17 15:21:00 +08:00
He Wang
bd9163904a fix(ob_conn): ignore duplicate errors when executing 'create_idx' (#12661)
### What problem does this PR solve?

Skip duplicate errors to avoid 'create_idx' failures caused by slow
metadata refresh or external modifications.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 20:46:37 +08:00
Kevin Hu
b6d7733058 Feat: metadata settings in KB. (#12662)
### What problem does this PR solve?

#11910

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-16 20:14:02 +08:00
6ba3i
4f036a881d Fix: Infinity keyword round-trip, highlight fallback, and KB update guards (#12660)
### What problem does this PR solve?

Fixes Infinity-specific API regressions: preserves ```important_kwd```
round‑trip for ```[""]```, restores required highlight key in retrieval
responses, and enforces Infinity guards for unsupported
```parser_id=tag``` and pagerank in ```/v1/kb/update```. Also removes a
slow/buggy pandas row-wise apply that was throwing ```ValueError``` and
causing flakiness.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 20:03:52 +08:00
6ba3i
59075a0b58 Fix : p3 level sdk test error for update chat (#12654)
### What problem does this PR solve?

fix for update chat failing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 17:47:12 +08:00
PentaFDevs
30bd25716b Fix PDF Generator output variables not appearing in subsequent agent steps (#12619)
This commit fixes multiple issues preventing PDF Generator (Docs
Generator) output variables from being visible in the Output section and
available to downstream nodes.

### What problem does this PR solve?

Issues Fixed:
1. PDF Generator nodes initialized with empty object instead of proper
initial values
2. Output structure mismatch (had 'value' property that system doesn't
expect)
3. Missing 'download' output in form schema
4. Output list computed from static values instead of form state
5. Added null/undefined guard to transferOutputs function

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
Changes:
- web/src/pages/agent/constant/index.tsx: Fixed output structure in
initialPDFGeneratorValues
- web/src/pages/agent/hooks/use-add-node.ts: Initialize PDF Generator
with proper values
- web/src/pages/agent/form/pdf-generator-form/index.tsx: Fixed schema
and use form.watch
- web/src/pages/agent/form/components/output.tsx: Added null guard and
spacing
2026-01-16 16:50:53 +08:00
balibabu
99dae3c64c Fix: In the agent loop, if the await response is selected as the variable, the operator cannot be selected. #12656 (#12657)
### What problem does this PR solve?

Fix: In the agent loop, if the await response is selected as the
variable, the operator cannot be selected. #12656

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 16:49:48 +08:00
Magicbook1108
045314a1aa Fix: duplicate content in chunk (#12655)
### What problem does this PR solve?

Fix: duplicate content in chunk #12336

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 15:32:04 +08:00
6ba3i
2b20d0b3bb Fix : Web API tests by normalizing errors, validation, and uploads (#12620)
### What problem does this PR solve?

Fixes web API behavior mismatches that caused test failures by
normalizing error responses, tightening validations, correcting error
messages, and closing upload file handles.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-16 11:09:22 +08:00
zagnaan
59f4c51222 fix(entrypoint): Preserve $ in passwords during template expansion (#12509)
### What problem does this PR solve?

Fix shell variable expansion to preserve $ in password defaults when 
env vars are unset. Fixes Azure RDS auto-rotated passwords (that contain
$) being
truncated during template processing.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 19:30:33 +08:00
chanx
8c1fbfb130 Fix:Some bugs (#12648)
### What problem does this PR solve?

Fix: Modified and optimized the metadata condition card component.
Fix: Use startOfDay and endOfDay to ensure the date range includes a
full day.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 19:28:22 +08:00
Kevin Hu
cec06bfb5d Fix: empty chunk issue. (#12638)
#12570

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 17:46:21 +08:00
writinwaters
2167e3a3c0 Docs: Added share memory (#12647)
### Type of change

- [x] Documentation Update
2026-01-15 17:21:36 +08:00
liuxiaoyusky
2ea8dddef6 fix(infinity): Use comma separator for important_kwd to preserve mult… (#12618)
## Problem

The \`important_kwd\` field in Infinity connector was using mismatched
separators:
- **Storage**: \`list2str(v)\` uses space as default separator
- **Reading**: \`v.split()\` splits by all whitespace

This causes multi-word keywords like \`\"Senior Fund Manager\"\` to be
incorrectly split into \`[\"Senior\", \"Fund\", \"Manager\"]\`.

## Solution

Use comma \`,\` as separator for both storing and reading, consistent
with:
1. The LLM output format in \`keyword_prompt.md\` (\"delimited by
ENGLISH COMMA\")
2. The \`cached.split(\",\")\` in \`task_executor.py\`

## Changes

- \`insert()\`: \`list2str(v)\` → \`list2str(v, \",\")\`
- \`update()\`: \`list2str(v)\` → \`list2str(v, \",\")\`
- \`get_fields()\`: \`v.split()\` → \`v.split(\",\") if v else []\`

## Impact

This bug affects:
- Python-level reranking weight calculation (\`important_kwd * 5\`)
- API response keyword display
- Search precision due to fragmented keywords
2026-01-15 15:32:40 +08:00
longbingljw
18867daba7 chore: bump pyobvector from 0.2.18 to 0.2.22 (#12640)
### What problem does this PR solve?

Update ob client

### Type of change

- [x] Other (please describe):dependency upgrade
2026-01-15 15:21:34 +08:00
longbingljw
d68176326d feat: add oceanbase mount to gitignore (#12642)
### What problem does this PR solve?

feat: add oceanbase mount to .gitignore

### Type of change

- [x] Refactoring
2026-01-15 15:20:40 +08:00
balibabu
d531bd4f1a Fix: Editing the agent greeting causes the greeting to be continuously added to the message list. #12635 (#12636)
### What problem does this PR solve?

Fix: Editing the agent greeting causes the greeting to be continuously
added to the message list. #12635
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 14:55:19 +08:00
Vedant Madane
ac936005e6 fix: ensure deleted chunks are not returned in retrieval (#12520) (#12546)
## Summary
Fixes #12520 - Deleted chunks should not appear in retrieval/reference
results.

## Changes

### Core Fix
- **api/apps/chunk_app.py**: Include \doc_id\ in delete condition to
properly scope the delete operation

### Improved Error Handling
- **api/db/services/document_service.py**: Better separation of concerns
with individual try-catch blocks and proper logging for each cleanup
operation

### Doc Store Updates
- **rag/utils/es_conn.py**: Updated delete query construction to support
compound conditions
- **rag/utils/opensearch_conn.py**: Same updates for OpenSearch
compatibility

### Tests
- **test/testcases/.../test_retrieval_chunks.py**: Added
\TestDeletedChunksNotRetrievable\ class with regression tests
- **test/unit/test_delete_query_construction.py**: Unit tests for delete
query construction

## Testing
- Added regression tests that verify deleted chunks are not returned by
retrieval API
- Tests cover single chunk deletion and batch deletion scenarios
2026-01-15 14:45:55 +08:00
Pegasus
d8192f8f17 Fix: validate regex pattern in split_with_pattern to prevent crash (#12633)
### What problem does this PR solve?

Fix regex pattern validation in split_with_pattern (#12605)

- Add try-except block to validate user-provided regex patterns before
use
- Gracefully fallback to single chunk when invalid regex is provided
- Prevent server crash during DOCX parsing with malformed delimiters

## Problem

Parsing DOCX files with custom regex delimiters crashes with `re.error:
nothing to repeat at position 9` when users provide invalid regex
patterns.

Closes #12605 

## Solution

Validate and compile regex pattern before use. On invalid pattern, log
warning and return content as single chunk instead of crashing.

## Changes

- `rag/nlp/__init__.py`: Add regex validation in `split_with_pattern()`
function

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=42954461
2026-01-15 14:24:51 +08:00
Kevin Hu
eb35e2b89f Fix: async invocation isssue. (#12634)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 14:22:16 +08:00
MkDev11
97b983fd0b fix: add fallback parser list for empty parser_ids (#12632)
### What problem does this PR solve?

Fixes #12570 - The slicing method dropdown was empty when deploying
RAGFlow v0.23.1 from source code.

The issue occurred because `parser_ids` from the tenant info was empty
or undefined, causing `useSelectParserList` to return an empty array.
This PR adds a fallback to a default parser list when `parser_ids` is
empty, ensuring the dropdown always has options.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---
Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=94194147
2026-01-15 14:05:25 +08:00
Magicbook1108
b40a7b2e7d Feat: Hash doc id to avoid duplicate name. (#12573)
### What problem does this PR solve?

Feat: Hash doc id to avoid duplicate name. 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-15 14:02:15 +08:00
Kevin Hu
9a10558f80 Refa: async retrieval process. (#12629)
### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-01-15 12:28:49 +08:00
SID
f82628c40c Fix: langfuse connection error handling #12621 (#12626)
## Description

Fixes connection error handling when langfuse service is unavailable.
The application now gracefully handles connection failures instead of
crashing.

## Changes

- Wrapped `langfuse.auth_check()` calls in try-except blocks in:
  - `api/db/services/dialog_service.py`
  - `api/db/services/tenant_llm_service.py`

## Problem

When langfuse service is unavailable or connection is refused,
`langfuse.auth_check()` throws `httpx.ConnectError: [Errno 111]
Connection refused`, causing the application to crash during document
parsing or dialog operations.

## Solution

Added try-except blocks around `langfuse.auth_check()` calls to catch
connection errors and gracefully skip langfuse tracing instead of
crashing. The application continues functioning normally even when
langfuse is unavailable.

## Related Issue

Fixes #12621

---

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=158349177
2026-01-15 11:23:15 +08:00
chanx
7af98328f5 Fix: the styles of the multi-select component and the filter pop-up. (#12628)
### What problem does this PR solve?

Fix: Fix the styles of the multi-select component and the filter pop-up.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-15 10:53:18 +08:00
MkDev11
678a4f959c Fix: skip internal bookmark references in DOCX parsing (#12604) (#12611)
### What problem does this PR solve?

Fixes #12604 - DOCX files containing hyperlinks to internal bookmarks
(e.g., `#_文档目录`) cause a `KeyError` during parsing:

```
KeyError: "There is no item named 'word/#_文档目录' in the archive"
```

This happens because python-docx incorrectly tries to read internal
bookmark references as files from the ZIP archive. Internal bookmarks
are relationship targets starting with `#` and are not actual files.

This PR extends the existing `load_from_xml_v2` workaround (which
already handles `NULL` targets) to also skip relationship targets
starting with `#`.

Related upstream issue:
https://github.com/python-openxml/python-docx/issues/902

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---
Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=94194147
2026-01-14 19:08:46 +08:00
Kevin Hu
15a8bb2e9c Fix: chunk list async issue. (#12615)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-14 17:32:07 +08:00
Pegasus
b091ff2730 Fix enable_thinking parameter for Qwen3 models (#12603)
### Issue

When using Qwen3 models (`qwen3-32b`, `qwen3-max`) through the
Tongyi-Qianwen provider for non-streaming calls (e.g., knowledge graph
generation), the API fails with:

Closes #12424

```
parameter.enable_thinking must be set to false for non-streaming calls
```

### Root Cause

In `LiteLLMBase.async_chat()`, the `extra_body={"enable_thinking":
False}` was set in `kwargs` but never forwarded to
`_construct_completion_args()`.

### What problem does this PR solve?

Pass merged kwargs to `_construct_completion_args()` using
`**{**gen_conf, **kwargs}` to safely handle potential duplicate
parameters.

### Changes

- `rag/llm/chat_model.py`: Forward kwargs containing `extra_body` to
`_construct_completion_args()` in `async_chat()`


_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Contribution by Gittensor, see my contribution statistics at
https://gittensor.io/miners/details?githubId=42954461
2026-01-14 16:35:46 +08:00
6ba3i
5b22f94502 Feat: Benchmark CLI additions and documentation (#12536)
### What problem does this PR solve?

This PR adds a dedicated HTTP benchmark CLI for RAGFlow chat and
retrieval endpoints so we can measure latency/QPS.

### Type of change

- [x] Documentation Update
- [x] Other (please describe): Adds a CLI benchmarking tool for
chat/retrieval latency/QPS

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-14 13:49:16 +08:00
Yongteng Lei
a7671583b3 Feat: add CN regions for AWS (#12610)
### What problem does this PR solve?

Add CN regions for AWS.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-14 12:34:55 +08:00
balibabu
d32fa02d97 Fix: Unable to copy category node. #12607 (#12609)
### What problem does this PR solve?

Fix: Unable to copy category node. #12607

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-14 11:45:31 +08:00
lys1313013
f72a35188d refactor: remove debug print statements (#12598)
### What problem does this PR solve?

This PR eliminates unnecessary debug print statements that were left in
hot paths of the codebase.

### Type of change

- [x] Refactoring
2026-01-14 10:05:34 +08:00
6ba3i
ea619dba3b Added to the HTTP API test suite (#12556)
### What problem does this PR solve?

This PR adds missing HTTP API test coverage for dataset
graph/GraphRAG/RAPTOR tasks, metadata summary, chat completions, agent
sessions/completions, and related questions. It also introduces minimal
HTTP test helpers to exercise these endpoints consistently with the
existing suite.

### Type of change

- [x]  Other (please describe): Test coverage (HTTP API tests)

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-14 10:02:30 +08:00
writinwaters
36b0835740 Docs: Use memory (#12599)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2026-01-14 09:40:31 +08:00
6ba3i
0795616b34 Align p3 HTTP/SDK tests with current backend behavior (#12563)
### What problem does this PR solve?

Updates pre-existing HTTP API and SDK tests to align with current
backend behavior (validation errors, 404s, and schema defaults). This
ensures p3 regression coverage is accurate without changing production
code.

### Type of change

- [x] Other (please describe): align p3 HTTP/SDK tests with current
backend behavior

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-13 19:22:47 +08:00
Yongteng Lei
941651a16f Fix: wrong input trace in Category component (#12590)
### What problem does this PR solve?

Wrong input trace in Category component

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-13 17:54:57 +08:00
He Wang
360114ed42 fix(ob_conn): avoid reusing SQLAlchemy Column objects in DDL (#12588)
### What problem does this PR solve?

When there are multiple users, parsing a document for a new user can
trigger the reuse of column objects, leading to the error
`sqlalchemy.exc.ArgumentError: Column object 'id' already assigned to
Table xxx`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-13 17:39:20 +08:00
chanx
ffedb2c6d3 Feat: The MetadataFilterConditions component supports adding values ​​via search. (#12585)
### What problem does this PR solve?

Feat: The MetadataFilterConditions component supports adding values
​​via search.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-13 17:03:25 +08:00
LIRUI YU
947e63ca14 Fixed typos and added pptx preview for frontend (#12577)
### What problem does this PR solve?
Previously, we added support for previewing PPT and PPTX files in the
backend. Now, we are adding it to the frontend, so when the slides in
the chat interface are referenced, they will no longer be blank.
### Type of change

- Bug Fix (non-breaking change which fixes an issue)
2026-01-13 17:02:36 +08:00
He Wang
34d74d9928 fix: add uv-aarch64-unknown-linux-gnu.tar.gz to deps image (#12516)
### What problem does this PR solve?

Add uv-aarch64-unknown-linux-gnu.tar.gz to support building ARM64 Docker
images.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Liu An <asiro@qq.com>
2026-01-13 15:37:32 +08:00
balibabu
accae95126 Feat: Exported Agent JSON Should Include Conversation Variables Configuration #11796 (#12579)
### What problem does this PR solve?

Feat: Exported Agent JSON Should Include Conversation Variables
Configuration #11796

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-13 15:35:45 +08:00
Yongteng Lei
68e5c86e9c Fix: image not displaying thumbnails when using pipeline (#12574)
### What problem does this PR solve?

Fix image not displaying thumbnails when using pipeline.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-13 12:54:13 +08:00
Yongteng Lei
64c75d558e Fix: zip extraction vulnerabilities in MinerU and TCADP (#12527)
### What problem does this PR solve?

Fix zip extraction vulnerabilities:
   - Block symlink entries in zip files.
   - Reject encrypted zip entries.
   - Prevent absolute path attacks (including Windows paths).
   - Block path traversal attempts (../).
   - Stop zip slip exploits (directory escape).
   - Use streaming for memory-safe file handling.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-13 12:24:50 +08:00
LIRUI YU
41c84fd78f Add MIME types for PPT and PPTX files (#12562)
Otherwise, slide files cannot be opened in Chat module

### What problem does this PR solve?

Backend Reason (API): In the api/utils/web_utils.py file of the backend,
the CONTENT_TYPE_MAP dictionary is missing ppt and pptx.
MIME type mapping. This means that when the frontend requests a PPTX
file, the backend cannot correctly inform the browser that it is a PPTX
file, resulting in the file being displayed incorrectly.
Type identification error.

### Type of change

-  Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
2026-01-13 12:17:49 +08:00
LGRY
d76912ab15 Fix: Use uv pip install for Docling installation (#12567)
Fixes #12440
### What problem does this PR solve?
The current implementation uses `python3 -m pip` which can fail in
certain environments. This change leverages `uv pip install` instead,
which aligns with the project's existing tooling.

### Type of change
- Removed the ensurepip line (not needed since uv manages pip)
- Changed python3 to "$PY" for consistency with the rest of the script
- Changed python3 -m pip install to uv pip install

Co-authored-by: Gongzi <gongzi@192.168.0.100>
2026-01-13 11:48:42 +08:00
Lin Manhui
4fe3c24198 feat: PaddleOCR PDF parser supports thumnails and positions (#12565)
### What problem does this PR solve?

1. PaddleOCR PDF parser supports thumnails and positions.
2. Add FAQ documentation for PaddleOCR PDF parser.


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-13 09:51:08 +08:00
Kevin Hu
44bada64c9 Feat: support tree structured deep-research policy. (#12559)
### What problem does this PR solve?

#12558
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-13 09:41:35 +08:00
Jimmy Ben Klieve
867ec94258 revert white-space changes in docs (#12557)
### What problem does this PR solve?

Trailing white-spaces in commit 6814ace1aa
got automatically trimmed by code editor may causes documentation
typesetting broken.

Mostly for double spaces for soft line breaks.  

### Type of change

- [x] Documentation Update
2026-01-13 09:41:02 +08:00
chanx
fd0a1fde6b Feat: Enhanced metadata functionality (#12560)
### What problem does this PR solve?

Feat: Enhanced metadata functionality
- Metadata filtering supports searching.
- Values ​​can be directly modified.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-12 19:05:33 +08:00
Lynn
653001b14f Doc: python sdk document (#12554)
### What problem does this PR solve?

Add python sdk document for memory api.

### Type of change

- [x] Documentation Update
2026-01-12 15:31:02 +08:00
chanx
d4f8c724ed Fix:Automatically enable metadata and optimize parser dialog logic (#12553)
### What problem does this PR solve?

Fix:Automatically enable metadata and optimize parser dialog logic

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-12 15:29:50 +08:00
Jin Hai
a7dd3b7e9e Add time cost when start servers (#12552)
### What problem does this PR solve?

- API server
- Ingestion server
- Data sync server
- Admin server

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-12 12:48:23 +08:00
Stephen Hu
638c510468 refactor: introduce common normalize method in rerank base class (#12550)
### What problem does this PR solve?

introduce common normalize method in rerank base class

### Type of change

- [x] Refactoring
2026-01-12 11:07:11 +08:00
Zhizhou Li
ff11e3171e Feat: SandBox docker CLI error in ARM CPU #12433 (#12434)
### What problem does this PR solve?

Add multi-architecture support for Sandbox

Updated Dockerfile to support multiple architectures for Docker Sandbox
installation.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-12 11:06:33 +08:00
Zhichang Yu
030d6ba004 CI collect ragflow log (#12543)
### What problem does this PR solve?

As title

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [x] Other (please describe): CI
2026-01-10 09:52:32 +08:00
lys1313013
b226e06e2d refactor: remove debug print statements (#12534)
### What problem does this PR solve?

refactor: remove debug print statements

### Type of change

- [x] Refactoring
2026-01-09 19:23:50 +08:00
Lin Manhui
2e09db02f3 feat: add paddleocr parser (#12513)
### What problem does this PR solve?

Add PaddleOCR as a new PDF parser.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-09 17:48:45 +08:00
Haiyang.Pu
6abf55c048 Feat: support openapi (#12521)
### What problem does this PR solve?
Support OpenAPI interface description.

The issue of not supporting the Swagger interface after upgrading the
system framework from Flask to Quart has been resolved.

Resolved https://github.com/infiniflow/ragflow/issues/5264

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: puhaiyang <“761396462@qq.com”>
2026-01-09 17:48:20 +08:00
Lynn
f9d4179bf2 Feat:memory sdk (#12538)
### What problem does this PR solve?

Move memory and message apis to /api, and add sdk support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-09 17:45:58 +08:00
balibabu
64b1e0b4c3 Feat: The translation model type options should be consistent with the model's labels. #1036 (#12537)
### What problem does this PR solve?

Feat: The translation model type options should be consistent with the
model's labels. #1036

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-09 17:39:40 +08:00
chanx
b65daeb945 Fix: Baiduyiyan key invaild (#12531)
### What problem does this PR solve?

Fix: Baiduyiyan key invaild

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-09 17:37:17 +08:00
He Wang
fbe55cef05 fix: keep password in opendal config to fix connection initialization (#12529)
### What problem does this PR solve?

If we delete the password in kwargs, func 'init_db_config' will fail, so
we need to keep this field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-09 14:19:32 +08:00
balibabu
0878526ba8 Refactor: Refactoring OllamaModal using shadcn. #1036 (#12530)
### What problem does this PR solve?

Refactor: Refactoring OllamaModal using shadcn.  #1036

### Type of change

- [x] Refactoring
2026-01-09 13:42:28 +08:00
chanx
a2db3e3292 Fix: Bugs fixed (#12524)
### What problem does this PR solve?

Fix: Bugs fixed
- The issue of filter conditions not being able to be deleted on the
knowledge base file page
- The issue of metadata filter conditions not working.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-09 13:41:24 +08:00
Stephen Hu
f522391d1e Fix: "AttributeError(\"'list' object has no attribute 'get'\")" (#12518)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/12515

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-09 10:19:51 +08:00
lys1313013
9562762af2 docs: fix embedding model switching tooltip (#12517)
### What problem does this PR solve?
After version 0.22.1, the embedding model supports switching; the
corresponding tooltip needs to be updated.

### Type of change

- [x] Documentation Update
2026-01-09 10:19:40 +08:00
balibabu
455fd04050 Refactor: Replace Ant Design with shadcn in SparkModal, TencentCloudModal, HunyuanModal, and GoogleModal. #1036 (#12510)
### What problem does this PR solve?

Refactor: Replace Ant Design with shadcn in SparkModal,
TencentCloudModal, HunyuanModal, and GoogleModal. #1036
### Type of change

- [x] Refactoring
2026-01-08 19:42:45 +08:00
Jin Hai
14c250e3d7 Fix adding column error (#12503)
### What problem does this PR solve?

1. Fix redundant column adding
2. Refactor the code

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-08 16:44:53 +08:00
Magicbook1108
a093e616cf Fix: add multimodel models in chat api (#12496)
### What problem does this PR solve?

Fix: add multimodel models in chat api #11986
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-08 16:12:08 +08:00
Kevin Hu
696397ebba Fix: apply kb setting while re-parsing.... (#12501)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-08 16:11:50 +08:00
Liu An
6f1a555d5f Refa(sdk/python/test): remove unused testcases and utilities (#12505)
### What problem does this PR solve?

Removed the following dir:
- sdk/python/test/libs/
- sdk/python/test/test_http_api/
- sdk/python/test/test_sdk_api/

### Type of change

- [x] Refactoring
2026-01-08 16:11:35 +08:00
buua436
1996aa0dac Refactor: Enhance delta streaming in chat functions for improved reasoning and content handling (#12453)
### What problem does this PR solve?

change:
Enhance delta streaming in chat functions for improved reasoning and
content handling

### Type of change


- [x] Refactoring
2026-01-08 13:34:16 +08:00
Paul Lu
f4e2783eb4 optimize doc id check: do not query db when doc id to validate is empty (#12500)
### What problem does this PR solve?
when a kb contains many documents, say 50000, and the retrieval is only
made against some kb without specifying any doc ids, the query for all
docs from the db is not necessary, and can be omitted to improve
performance.

### Type of change

- [x] Performance Improvement
2026-01-08 13:22:58 +08:00
Lynn
2fd4a3134d Doc: memory http api (#12499)
### What problem does this PR solve?

Use task save function for add_message api, and added http API document.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
2026-01-08 12:54:10 +08:00
Stephen Hu
f1dc2df23c Fix:Bedrock assume_role auth mode fails with LiteLLM "Extra inputs are not permitted" error (#12495)
### What problem does this PR solve?
https://github.com/infiniflow/ragflow/issues/12489

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-08 12:53:41 +08:00
balibabu
de27c006d8 Feat: The chat feature supports streaming output, displaying results one by one. #12490 (#12493)
### What problem does this PR solve?

Feat: The chat feature supports streaming output, displaying results one
by one.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-08 09:43:57 +08:00
Kevin Hu
23a9544b73 Fix: toc async issue. (#12485)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 15:35:30 +08:00
Magicbook1108
011bbe9556 Feat: support context window for docx (#12455)
### What problem does this PR solve?

Feat: support context window for docx

#12303

Done:
- [x] naive.py
- [x] one.py

TODO:
- [ ] book.py
- [ ] manual.py

Fix: incorrect image position
Fix: incorrect chunk type tag

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-01-07 15:08:17 +08:00
balibabu
a442c9cac6 Fix: Fixed an issue where ESLint suggestions were not working in VS Code after upgrading to Vite. #12483 (#12484)
### What problem does this PR solve?

Fix: Fixed an issue where ESLint suggestions were not working in VS Code
after upgrading to Vite. #12483

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 14:32:17 +08:00
chanx
671e719d75 Feat: Memory-message supports categorized display (#12482)
### What problem does this PR solve?

Feat: Memory-message supports categorized display

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 13:48:40 +08:00
Lynn
07845be5bd Fix: display agent name for extract messages (#12480)
### What problem does this PR solve?

Display agent name for extract messages

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 13:19:54 +08:00
OliverW
8d406bd2e6 fix: prevent MinIO health check failure in multi-bucket mode (#12446)
### What problem does this PR solve?

- Fixes the health check failure in multi-bucket MinIO environments.
Previously, health checks would fail because the default
"ragflow-bucket" did not exist. This caused false negatives for system
health.

- Also removes the _health_check write in single-bucket mode to avoid
side effects (minor optimization).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 10:07:18 +08:00
chanx
2a4627d9a0 Fix: Issues and style fixes related to the 'Memory' page (#12469)
### What problem does this PR solve?

Fix:  Some bugs
- Issues and style fixes related to the 'Memory' page
- Data source icon replacement
- Build optimization

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 10:03:54 +08:00
Jimmy Ben Klieve
6814ace1aa docs: update docs icons (#12465)
### What problem does this PR solve?

Update icons for docs.
Trailing spaces are auto truncated by the editor, does not affect real
content.

### Type of change

- [x] Documentation Update
2026-01-07 10:00:09 +08:00
Lynn
ca9645f39b Feat: adapt to , arglist (#12468)
### What problem does this PR solve?

Adapt to ',' joined arg list in get method url.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-07 09:59:08 +08:00
Jimmy Ben Klieve
8e03843145 fix: task executor with status "timeout" corrupts page when checking its details (#12467)
### What problem does this PR solve?

In **Admin UI** > **Service Status**, clicking "Show details" on task
executor with status "Timeout" may corrupts page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-07 09:58:16 +08:00
Jimmy Ben Klieve
51ece37db2 refactor: migrate env prefix to VITE_* (#12466)
### What problem does this PR solve?

`UMI_APP_*` to `VITE_*`

### Type of change

- [x] Refactoring
2026-01-07 09:39:18 +08:00
buua436
45fb2719cf Fix: update uv python installation to version 3.12 in Dockerfile (#12464)
### What problem does this PR solve?
issue:
https://github.com/infiniflow/ragflow/issues/12440
change:
update uv python installation to version 3.12 in Dockerfile

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-06 19:27:46 +08:00
Lynn
bdd9f3d4d1 Fix: try handle authorization as api-token (#12462)
### What problem does this PR solve?

Try handle authorization as api-token when jwt load failed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-06 19:25:42 +08:00
writinwaters
1f60863f60 Docs: Fixed a display issue. (#12463)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2026-01-06 17:40:53 +08:00
Stephen Hu
02e6870755 Refactor: import_test_cases use bulk_create (#12456)
### What problem does this PR solve?

import_test_cases use bulk_create

### Type of change

- [x] Refactoring
2026-01-06 11:39:07 +08:00
chanx
aa08920e51 Fix: The avatar and greeting message no longer appear in the Agent iFrame. [#12410] (#12459)
### What problem does this PR solve?
Fix: The avatar and greeting message no longer appear in the Agent
iFrame. [#12410]

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-06 11:01:16 +08:00
AAAkater
7818644129 Fix: add uv binary archive to ignored files (#12451)
### What problem does this PR solve?

After I ran this command, 

```bash
uv run ./download_deps.py 
```

a file was not ignored.

```bash
❯ git status
On branch feat/ignore-uv
Untracked files:
  (use "git add <file>..." to include in what will be committed)
        uv-x86_64-unknown-linux-gnu.tar.gz

nothing added to commit but untracked files present (use "git add" to track)
```

Add this file name to `.gitignore`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-05 20:22:35 +08:00
He Wang
55c9fc0017 fix: add 'mom_id' column to OBConnection chunk table (#12444)
### What problem does this PR solve?

Fix #12428

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-05 19:31:44 +08:00
balibabu
140dd2c8cc Refactor: Refactor FishAudioModal and BedrockModal using shadcn. #1036 (#12449)
### What problem does this PR solve?

Refactor: Refactor FishAudioModal and BedrockModal using shadcn. #1036

### Type of change

- [x] Refactoring
2026-01-05 19:27:56 +08:00
Lynn
fada223249 Feat: process memory (#12445)
### What problem does this PR solve?

Add task status for raw message, and move extract message as a nested
property under raw message

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-05 17:58:32 +08:00
chanx
00f8a80ca4 Fix: Some bugs (#12441)
### What problem does this PR solve?

Fix: Some bugs
- In a production environment, a second-level page refresh results in a
white screen.
- The knowledge graph cannot be opened.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-05 15:28:57 +08:00
balibabu
4e9407b4ae Refactor: Refactoring AzureOpenAIModal using shadcn. #10427 (#12436)
### What problem does this PR solve?

Refactor: Refactoring AzureOpenAIModal using shadcn. #10427

### Type of change

- [x] Refactoring
2026-01-05 14:09:55 +08:00
Jin Hai
42461bc378 Update admin doc (#12439)
### What problem does this PR solve?

update for 'list configs' and 'list envs'

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-05 13:26:33 +08:00
Jin Hai
92780c486a Add list configs and environments (#12438)
### What problem does this PR solve?

1. list configs;
3. list envs;

```
admin> list configs;
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| extra                                                                                     | host      | id | name          | port  | service_type   |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
| {}                                                                                        | 0.0.0.0   | 0  | ragflow_0     | 9380  | ragflow_server |
| {'meta_type': 'mysql', 'password': 'infini_rag_flow', 'username': 'root'}                 | localhost | 1  | mysql         | 5455  | meta_data      |
| {'password': 'infini_rag_flow', 'store_type': 'minio', 'user': 'rag_flow'}                | localhost | 2  | minio         | 9000  | file_store     |
| {'password': 'infini_rag_flow', 'retrieval_type': 'elasticsearch', 'username': 'elastic'} | localhost | 3  | elasticsearch | 1200  | retrieval      |
| {'db_name': 'default_db', 'retrieval_type': 'infinity'}                                   | localhost | 4  | infinity      | 23817 | retrieval      |
| {'database': 1, 'mq_type': 'redis', 'password': 'infini_rag_flow'}                        | localhost | 5  | redis         | 6379  | message_queue  |
| {'message_queue_type': 'redis'}                                                           |           | 6  | task_executor | 0     | task_executor  |
+-------------------------------------------------------------------------------------------+-----------+----+---------------+-------+----------------+
admin> list envs;
+-------------------------+------------------+
| env                     | value            |
+-------------------------+------------------+
| DOC_ENGINE              | elasticsearch    |
| DEFAULT_SUPERUSER_EMAIL | admin@ragflow.io |
| DB_TYPE                 | mysql            |
| DEVICE                  | cpu              |
| STORAGE_IMPL            | MINIO            |
+-------------------------+------------------+
admin> 
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-05 13:26:22 +08:00
lif
81f9296d79 Fix: handle invalid img_id format in chunk update (#12422)
## Summary
- Fix ValueError when updating chunk with invalid/empty `img_id` format
- Add validation before splitting `img_id` by hyphen
- Use `split("-", 1)` to handle object names containing hyphens

## Test plan
- [x] Verify chunk update works with valid `img_id` (format:
`bucket-objectname`)
- [x] Verify chunk update doesn't crash with empty `img_id`
- [x] Verify chunk update doesn't crash when `img_id` has no hyphen
- [x] Verify ruff check passes

Fixes #12035

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-01-05 11:27:19 +08:00
Liu An
606f4e6c9e Refa: improve TOC building with better error handling (#12427)
### What problem does this PR solve?

Refactor TOC building logic to use enumerate instead of while loop, add
comprehensive error handling for missing/invalid chunk_id values, and
improve logging with more specific error messages. The changes make the
code more robust against malformed TOC data while maintaining the same
functionality for valid inputs.

### Type of change

- [x] Refactoring
2026-01-05 10:02:42 +08:00
Yongteng Lei
4cd4526492 Feat: PDF vision figure parser supports reading context (#12416)
### What problem does this PR solve?

PDF vision figure parser supports reading context.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-01-05 09:55:43 +08:00
balibabu
cc8a10376a Refactor: Refactoring VolcEngine and Yiyan modal using shadcn. #10427 (#12426)
### What problem does this PR solve?

Refactor: Refactoring VolcEngine and Yiyan modal using shadcn. #10427
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-01-05 09:53:47 +08:00
Jin Hai
5ebe334a2f Refactor setting type (#12425)
### What problem does this PR solve?

Refactor setting type

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-04 20:26:12 +08:00
buua436
932496a8ec Fix:bug fix (#12423)
### What problem does this PR solve?
change: 
initialize webhook configuration in webhook function
remove debug print statement from airtable_connector
remove redundant uuid import in imap_connector

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-04 19:16:29 +08:00
chanx
a8a060676a Refactor: UmiJs -> Vite (#12410)
### What problem does this PR solve?

Refactor: UmiJs -> Vite+React

### Type of change

- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-04 19:14:20 +08:00
Liu An
2c10ccd622 Chore(compose): remove unnecessary history_data_agent volume mount (#12418)
### What problem does this PR solve?

Removed the volume mount mapping
../history_data_agent:/ragflow/history_data_agent from
docker-compose.yml as it appears to be no longer in use

### Type of change

- [x] Chore
2026-01-04 16:58:23 +08:00
Lynn
a2211c200d Feat: message write testcase (#12417)
### What problem does this PR solve?

Write testcase for message web apis.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-04 16:52:44 +08:00
Jin Hai
21ba9e6d72 doc: update admin CLI (#12413)
### What problem does this PR solve?

`SHOW VERSION;`
- Display the current RAGFlow version.

`GRANT ADMIN <username>`
- Grant administrator privileges to the specified user.

`REVOKE ADMIN <username>`
- Revoke administrator privileges from the specified user.

`LIST VARS`
- List all system configurations and settings.

`SHOW VAR <var_name>`
- Display the content of a specific system configuration/setting by its
name or name prefix.

`SET VAR <var_name> <var_value>`
- Set the value for a specified configuration item.

related to: #12409 

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-04 15:22:01 +08:00
Jin Hai
ac9113b0ef feature: add system setting service (#12408)
### What problem does this PR solve?

#12409 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-04 14:21:39 +08:00
Lynn
11779697de Test: get message content testcase (#12403)
### What problem does this PR solve?

Testcase for get_message_content api.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-04 11:25:24 +08:00
OliverW
d6e006f086 Improve task executor heartbeat handling and cleanup (#12390)
Improve task executor heartbeat handling and cleanup.

### What problem does this PR solve?

- **Reduce lock contention during executor cleanup**: The cleanup lock
is acquired only when removing expired executors, not during regular
heartbeat reporting, reducing potential lock contention.

- **Optimize own heartbeat cleanup**: Each executor removes its own
expired heartbeat using `zremrangebyscore` instead of `zcount` +
`zpopmin`, reducing Redis operations and improving efficiency.

- **Improve cleanup of other executors' heartbeats**: Expired executors
are detected by checking their latest heartbeat, and stale entries are
removed safely.

- **Other improvements**: IP address and PID are captured once at
startup, and unnecessary global declarations are removed.

### Type of change

- [x] Performance Improvement

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-04 11:24:05 +08:00
chanx
d39fa75d36 Fix: Not able to add MCP Server [#12394](https://github.com/infiniflow/ragflow/issues/12394) (#12406)
### What problem does this PR solve?

Fix: Not able to add MCP Server
[#12394](https://github.com/infiniflow/ragflow/issues/12394)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-04 11:22:34 +08:00
Magicbook1108
f56bceb2a9 Fix: remvoe async wrappers (#12405)
### What problem does this PR solve?

Fix: remvoe async wrappers  #12396

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-01-04 11:19:48 +08:00
Rin
bbaf918d74 security: harden OpenDAL SQL initialization against injection (#12393)
Eliminates SQL injection vectors in the OpenDAL MySQL initialization
logic by implementing strict input validation and explicit type casting.

**Modifications:**
1. **`init_db_config`**: Enforced integer casting for
`max_allowed_packet` before formatting it into the SQL string.
2. **`init_opendal_mysql_table`**: Implemented regex-based validation
for `table_name` to ensure only alphanumeric characters and underscores
are permitted, preventing arbitrary SQL command injection through
configuration parameters.

These changes ensure that even if configuration values are sourced from
untrusted environments, the database initialization remains secure.
2026-01-04 11:19:26 +08:00
KKM
89a97be2c5 Remove duplicated tag_feas assignment in create route (api/apps/chunk_app.py) (#12392)
### What problem does this PR solve?

This PR removes a duplicated assignment of `tag_feas` in the
`@manager.route('/create')` API handler located in
`api/apps/chunk_app.py`.

The same conditional block was unintentionally repeated twice, which had
no
functional impact but reduced code readability and maintainability.
This change eliminates the redundancy while preserving existing
behavior.

### Type of change

- [x] Refactoring

Co-authored-by: 김경만 <kmkim7@humaxit.com>
2026-01-04 10:32:36 +08:00
Stephen Hu
6f2fc2f1cb refactor:re order logics in clean_gen_conf (#12391)
### What problem does this PR solve?

re order logics in clean_gen_conf
#12388

### Type of change
- [x] Refactoring
2026-01-04 10:31:56 +08:00
balibabu
42da080d89 Fix: Fixed the issue where the upload DSL dialog box was too narrow. #10427 (#12384)
### What problem does this PR solve?

Fix: Fixed the issue where the upload DSL dialog box was too narrow.
#10427

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-04 09:40:59 +08:00
Lynn
1f4a17863f Feat: read web api testcases (#12383)
### What problem does this PR solve?

Web API testcase for list_messages, get_recent_message.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-01 12:52:40 +08:00
Jin Hai
4d3a3a97ef Update HELP command of ADMIN CLI (#12387)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-01-01 12:52:13 +08:00
Jin Hai
ff1020ccfb ADMIN CLI: support grant/revoke user admin authorization (#12381)
### What problem does this PR solve?

```
admin> grant admin 'aaa@aaa1.com';
Fail to grant aaa@aaa1.com admin authorization, code: 404, message: User 'aaa@aaa1.com' not found
admin> grant admin 'aaa@aaa.com';
Grant successfully!
admin> revoke admin 'aaa1@aaa.com';
Fail to revoke aaa1@aaa.com admin authorization, code: 404, message: User 'aaa1@aaa.com' not found
admin> revoke admin 'aaa@aaa.com';
Revoke successfully!
admin> revoke admin 'aaa@aaa.com';
aaa@aaa.com isn't superuser, yet!
admin> grant admin 'aaa@aaa.com';
Grant successfully!
admin> grant admin 'aaa@aaa.com';
aaa@aaa.com is already superuser!
admin> revoke admin 'aaa@aaa.com';
Revoke successfully!

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-01-01 12:49:34 +08:00
Yingfeng
ca3bd2cf9f Update README (#12386)
### Type of change

- [x] Documentation Update
2025-12-31 20:07:40 +08:00
Maxine Lai
eb661c028d Fix Tika version mismatch in Dockerfile.deps (3.0.0 → 3.2.3) (#12267)
Fixes #12266 

Dockerfile.deps still referenced `tika-server-standard-3.0.0.jar` even
after
the project moved to Tika 3.2.3 for security reasons.

This caused Docker builds to fail due to a version mismatch and missing
artifact.

Changes:
- Update Dockerfile.deps to consistently use Tika 3.2.3

No functional changes beyond dependency alignment.

Co-authored-by: Liu An <asiro@qq.com>
2025-12-31 19:55:39 +08:00
balibabu
10c28c5ecd Feat: Refactoring the documentation page using shadcn. #10427 (#12376)
### What problem does this PR solve?

Feat: Refactoring the documentation page using shadcn. #10427

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-12-31 19:00:37 +08:00
Magicbook1108
96810b7d97 Fix: webdav connector (#12380)
### What problem does this PR solve?

fix webdav #11422

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-12-31 19:00:00 +08:00
chanx
365f9b01ae Fix: metadata data synchronization issues; add memory tab in home page (#12368)
### What problem does this PR solve?

fix: metadata data synchronization issues; add memory tab in home page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-12-31 17:19:04 +08:00
Magicbook1108
7d4d687dde Feat: Bitbucket connector (#12332)
### What problem does this PR solve?

Feat: Bitbucket connector NOT READY TO MERGE

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-12-31 17:18:30 +08:00
writinwaters
6a664fea3b Docs: Updated v0.23.0 release notes (#12374)
### What problem does this PR solve?


### Type of change


- [x] Documentation Update
2025-12-31 17:10:15 +08:00
Yingfeng
dcdc1b0ec7 Fix urls for basic docs (#12372)
### Type of change

- [x] Documentation Update
2025-12-31 17:02:34 +08:00
writinwaters
4af4c36e60 Docs: Added v0.23.1 release notes (#12371)
### What problem does this PR solve?


### Type of change

- [x] Documentation Update
2025-12-31 16:43:56 +08:00
Jin Hai
05e5244d94 Refactor docs of RAGFlow admin (#12361)
### What problem does this PR solve?

as title

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-12-31 14:42:53 +08:00
buua436
c2ee2bf7fe Feat: add Zendesk data source integration with configuration and sync capabilities (#12344)
### What problem does this PR solve?
issue:
#12313
change:
add Zendesk data source integration with configuration and sync
capabilities

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-12-31 14:40:49 +08:00
Kevin Hu
461c81e14a Fix: KG search issue. (#12364)
### What problem does this PR solve?

Close #12347

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-12-31 14:40:27 +08:00
2316 changed files with 379484 additions and 108971 deletions

192
.agents/rules/named.md Normal file
View File

@@ -0,0 +1,192 @@
# Go Naming Best Practices
## 1. Package Naming
- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`
- **Short and meaningful**: `package http`, `package json`, `package dao`
- **Avoid plurals**: `package user` not `package users`
- **Avoid generic names**: Avoid `package util`, `package common`, `package base`
```go
// Recommended
package user
package handler
package service
// Not recommended
package UserService
package user_service
package utils
```
## 2. File Naming
- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`
- **Test files**: `user_handler_test.go`
- **Platform-specific**: `user_linux.go`, `user_windows.go`
```
user/
├── user_handler.go
├── user_service.go
├── user_dao.go
└── user_test.go
```
## 3. Directory Naming
- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`
- **Short and descriptive**: `handler/`, `service/`, `dao/`
```
project/
├── cmd/ # Main entry point
│ └── server_main.go
├── internal/ # Private code
│ ├── handler/
│ ├── service/
│ ├── dao/
│ ├── model/
│ └── middleware/
├── pkg/ # Public code
└── api/ # API definitions
```
## 4. Interface Naming
- **Single-method interfaces end with "-er"**: `Reader`, `Writer`, `Handler`
- **Verb form**: `Reader`, `Executor`, `Validator`
```go
// Recommended
type Reader interface {
Read(p []byte) (n int, err error)
}
type UserService interface {
Register(req *RegisterRequest) (*User, error)
Login(req *LoginRequest) (*User, error)
}
// Not recommended
type UserInterface interface {}
type IUserService interface {}
```
## 5. Struct Naming
- **CamelCase**: `UserService`, `UserHandler`
- **Avoid redundant prefixes**: `User` not `UserModel`
```go
// Recommended
type UserService struct {}
type UserHandler struct {}
type RegisterRequest struct {}
// Not recommended
type user_service struct {}
type SUserService struct {}
type UserModel struct {}
```
## 6. Method/Function Naming
- **CamelCase**
- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`
- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`
```go
// Recommended
func (s *UserService) Register(req *RegisterRequest) (*User, error)
func (s *UserService) GetUserByID(id uint) (*User, error)
func (s *UserService) IsEmailExists(email string) bool
// Not recommended
func (s *UserService) register_user()
func (s *UserService) get_user_by_id()
func (s *UserService) CheckEmailExists() // Should use Is/Has
```
## 7. Constant Naming
- **CamelCase**: `const MaxRetryCount = 3`
- **Enum constants**: `const StatusActive = "active"`
```go
// Recommended
const (
StatusActive = "1"
StatusInactive = "0"
MaxRetryCount = 3
)
// Not recommended
const (
STATUS_ACTIVE = "1" // Not all uppercase
status_active = "1" // Not all lowercase
)
```
## 8. Error Variable Naming
- **Start with "Err"**: `ErrNotFound`, `ErrInvalidInput`
```go
// Recommended
var (
ErrNotFound = errors.New("not found")
ErrInvalidInput = errors.New("invalid input")
ErrUnauthorized = errors.New("unauthorized")
)
```
## 9. Acronyms Keep Consistent Case
```go
// Recommended
type HTTPHandler struct {}
var URL string
func GetHTTPClient() {}
func ParseJSON() {}
// Not recommended
type HttpHandler struct {}
var Url string
func GetHttpClient() {}
```
## 10. Project Structure Naming
```
project-name/
├── cmd/ # Main programs
│ └── app_name/
│ └── main.go
├── internal/ # Private code
│ ├── handler/ # HTTP handlers
│ ├── service/ # Business logic
│ ├── repository/ # Data access
│ ├── model/ # Data models
│ └── config/ # Configuration
├── pkg/ # Public code
├── api/ # API definitions
├── configs/ # Config files
├── scripts/ # Scripts
├── docs/ # Documentation
├── go.mod
└── go.sum
```
## Summary Table
| Type | Rule | Example |
| -------------- | ----------------------------------- | ------------------- |
| Package | All lowercase, no underscores | `package user` |
| File | All lowercase, underscore separated | `user_service.go` |
| Directory | All lowercase, no separators | `internal/handler/` |
| Struct | CamelCase, capitalized first letter | `UserService` |
| Interface | CamelCase, -er suffix | `Reader`, `Writer` |
| Method | CamelCase, verb prefix | `GetUserByID` |
| Constant | CamelCase | `MaxRetryCount` |
| Error Variable | Err prefix | `ErrNotFound` |

View File

@@ -0,0 +1,6 @@
---
name: go-naming
description: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.
---
Strictly follow the naming conventions in [rules/named.md](rules/named.md)

View File

@@ -1 +1,22 @@
Refer to [AGENTS.MD](../AGENTS.md) for all repo instructions.
# Project instructions for Copilot
## How to run (minimum)
- Install:
- python -m venv .venv && source .venv/bin/activate
- pip install -r requirements.txt
- Run:
- (fill) e.g. uvicorn app.main:app --reload
- Verify:
- (fill) curl http://127.0.0.1:8000/health
## Project layout (what matters)
- app/: API entrypoints + routers
- services/: business logic
- configs/: config loading (.env)
- docs/: documents
- tests/: pytest
## Conventions
- Prefer small, incremental changes.
- Add logging for new flows.
- Add/adjust tests for behavior changes.

View File

@@ -23,7 +23,7 @@ concurrency:
jobs:
release:
runs-on: [ "self-hosted", "ragflow-test" ]
runs-on: [ "self-hosted", "ragflow-release" ]
steps:
- name: Ensure workspace ownership
run: echo "chown -R ${USER} ${GITHUB_WORKSPACE}" && sudo chown -R ${USER} ${GITHUB_WORKSPACE}

View File

@@ -86,6 +86,9 @@ jobs:
mkdir -p ${RUNNER_WORKSPACE_PREFIX}/artifacts/${GITHUB_REPOSITORY}
echo "${PR_SHA} ${GITHUB_RUN_ID}" > ${PR_SHA_FP}
fi
ARTIFACTS_DIR=${RUNNER_WORKSPACE_PREFIX}/artifacts/${GITHUB_REPOSITORY}/${GITHUB_RUN_ID}
echo "ARTIFACTS_DIR=${ARTIFACTS_DIR}" >> ${GITHUB_ENV}
rm -rf ${ARTIFACTS_DIR} && mkdir -p ${ARTIFACTS_DIR}
# https://github.com/astral-sh/ruff-action
- name: Static check with Ruff
@@ -126,20 +129,24 @@ jobs:
fi
fi
- name: Run unit test
- name: Build ragflow go server
run: |
uv sync --python 3.12 --group test --frozen
source .venv/bin/activate
which pytest || echo "pytest not in PATH"
echo "Start to run unit test"
python3 run_tests.py
BUILDER_CONTAINER=ragflow_build_$(od -An -N4 -tx4 /dev/urandom | tr -d ' ')
echo "BUILDER_CONTAINER=${BUILDER_CONTAINER}" >> ${GITHUB_ENV}
TZ=${TZ:-$(readlink -f /etc/localtime | awk -F '/zoneinfo/' '{print $2}')}
sudo docker run --privileged -d --name ${BUILDER_CONTAINER} -e TZ=${TZ} -e UV_INDEX=https://mirrors.aliyun.com/pypi/simple -v ${PWD}:/ragflow -v ${PWD}/internal/cpp/resource:/usr/share/infinity/resource infiniflow/infinity_builder:ubuntu22_clang20
sudo docker exec ${BUILDER_CONTAINER} bash -c "git config --global safe.directory \"*\" && cd /ragflow && ./build.sh --cpp"
./build.sh --go
if [[ -n "${BUILDER_CONTAINER}" ]]; then
sudo docker rm -f -v "${BUILDER_CONTAINER}"
fi
- name: Build ragflow:nightly
run: |
RUNNER_WORKSPACE_PREFIX=${RUNNER_WORKSPACE_PREFIX:-${HOME}}
RAGFLOW_IMAGE=infiniflow/ragflow:${GITHUB_RUN_ID}
echo "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> ${GITHUB_ENV}
sudo docker pull ubuntu:22.04
sudo docker pull ubuntu:24.04
sudo DOCKER_BUILDKIT=1 docker build --build-arg NEED_MIRROR=1 --build-arg HTTPS_PROXY=${HTTPS_PROXY} --build-arg HTTP_PROXY=${HTTP_PROXY} -f Dockerfile -t ${RAGFLOW_IMAGE} .
if [[ ${GITHUB_EVENT_NAME} == "schedule" ]]; then
export HTTP_API_TEST_LEVEL=p3
@@ -149,125 +156,432 @@ jobs:
echo "HTTP_API_TEST_LEVEL=${HTTP_API_TEST_LEVEL}" >> ${GITHUB_ENV}
echo "RAGFLOW_CONTAINER=${GITHUB_RUN_ID}-ragflow-cpu-1" >> ${GITHUB_ENV}
- name: Start ragflow:nightly
- name: Run unit test
run: |
uv sync --python 3.12 --group test --frozen
source .venv/bin/activate
which pytest || echo "pytest not in PATH"
echo "Start to run unit test"
python3 run_tests.py -i
- name: Prepare function test environment
working-directory: docker
run: |
# Determine runner number (default to 1 if not found)
RUNNER_NUM=$(sudo docker inspect $(hostname) --format '{{index .Config.Labels "com.docker.compose.container-number"}}' 2>/dev/null || true)
RUNNER_NUM=${RUNNER_NUM:-1}
RUNNER_NUM=$(sudo docker inspect $(hostname) --format '{{index .Config.Labels "com.docker.compose.container-number"}}' 2>/dev/null || true)
RUNNER_NUM=${RUNNER_NUM:-1}
# Compute port numbers using bash arithmetic
ES_PORT=$((1200 + RUNNER_NUM * 10))
OS_PORT=$((1201 + RUNNER_NUM * 10))
INFINITY_THRIFT_PORT=$((23817 + RUNNER_NUM * 10))
INFINITY_HTTP_PORT=$((23820 + RUNNER_NUM * 10))
INFINITY_PSQL_PORT=$((5432 + RUNNER_NUM * 10))
MYSQL_PORT=$((5455 + RUNNER_NUM * 10))
MINIO_PORT=$((9000 + RUNNER_NUM * 10))
MINIO_CONSOLE_PORT=$((9001 + RUNNER_NUM * 10))
REDIS_PORT=$((6379 + RUNNER_NUM * 10))
TEI_PORT=$((6380 + RUNNER_NUM * 10))
KIBANA_PORT=$((6601 + RUNNER_NUM * 10))
SVR_HTTP_PORT=$((9380 + RUNNER_NUM * 10))
ADMIN_SVR_HTTP_PORT=$((9381 + RUNNER_NUM * 10))
SVR_MCP_PORT=$((9382 + RUNNER_NUM * 10))
SANDBOX_EXECUTOR_MANAGER_PORT=$((9385 + RUNNER_NUM * 10))
SVR_WEB_HTTP_PORT=$((80 + RUNNER_NUM * 10))
SVR_WEB_HTTPS_PORT=$((443 + RUNNER_NUM * 10))
ES_PORT=$((1200 + RUNNER_NUM * 10))
OS_PORT=$((1201 + RUNNER_NUM * 10))
INFINITY_THRIFT_PORT=$((23817 + RUNNER_NUM * 10))
INFINITY_HTTP_PORT=$((23820 + RUNNER_NUM * 10))
INFINITY_PSQL_PORT=$((5432 + RUNNER_NUM * 10))
EXPOSE_MYSQL_PORT=$((5455 + RUNNER_NUM * 10))
MINIO_PORT=$((9000 + RUNNER_NUM * 10))
MINIO_CONSOLE_PORT=$((9001 + RUNNER_NUM * 10))
REDIS_PORT=$((6379 + RUNNER_NUM * 10))
TEI_PORT=$((6380 + RUNNER_NUM * 10))
KIBANA_PORT=$((6601 + RUNNER_NUM * 10))
SVR_HTTP_PORT=$((9380 + RUNNER_NUM * 10))
ADMIN_SVR_HTTP_PORT=$((9381 + RUNNER_NUM * 10))
SVR_MCP_PORT=$((9382 + RUNNER_NUM * 10))
GO_HTTP_PORT=$((9384 + RUNNER_NUM * 10))
GO_ADMIN_PORT=$((9383 + RUNNER_NUM * 10))
SANDBOX_EXECUTOR_MANAGER_PORT=$((9385 + RUNNER_NUM * 10))
SVR_WEB_HTTP_PORT=$((80 + RUNNER_NUM * 10))
SVR_WEB_HTTPS_PORT=$((443 + RUNNER_NUM * 10))
# Persist computed ports into docker/.env so docker-compose uses the correct host bindings
echo "" >> docker/.env
echo -e "ES_PORT=${ES_PORT}" >> docker/.env
echo -e "OS_PORT=${OS_PORT}" >> docker/.env
echo -e "INFINITY_THRIFT_PORT=${INFINITY_THRIFT_PORT}" >> docker/.env
echo -e "INFINITY_HTTP_PORT=${INFINITY_HTTP_PORT}" >> docker/.env
echo -e "INFINITY_PSQL_PORT=${INFINITY_PSQL_PORT}" >> docker/.env
echo -e "MYSQL_PORT=${MYSQL_PORT}" >> docker/.env
echo -e "MINIO_PORT=${MINIO_PORT}" >> docker/.env
echo -e "MINIO_CONSOLE_PORT=${MINIO_CONSOLE_PORT}" >> docker/.env
echo -e "REDIS_PORT=${REDIS_PORT}" >> docker/.env
echo -e "TEI_PORT=${TEI_PORT}" >> docker/.env
echo -e "KIBANA_PORT=${KIBANA_PORT}" >> docker/.env
echo -e "SVR_HTTP_PORT=${SVR_HTTP_PORT}" >> docker/.env
echo -e "ADMIN_SVR_HTTP_PORT=${ADMIN_SVR_HTTP_PORT}" >> docker/.env
echo -e "SVR_MCP_PORT=${SVR_MCP_PORT}" >> docker/.env
echo -e "SANDBOX_EXECUTOR_MANAGER_PORT=${SANDBOX_EXECUTOR_MANAGER_PORT}" >> docker/.env
echo -e "SVR_WEB_HTTP_PORT=${SVR_WEB_HTTP_PORT}" >> docker/.env
echo -e "SVR_WEB_HTTPS_PORT=${SVR_WEB_HTTPS_PORT}" >> docker/.env
# Persist computed ports into .env so docker-compose uses the correct host bindings
echo "" >> .env
echo -e "ES_PORT=${ES_PORT}" >> .env
echo -e "OS_PORT=${OS_PORT}" >> .env
echo -e "INFINITY_THRIFT_PORT=${INFINITY_THRIFT_PORT}" >> .env
echo -e "INFINITY_HTTP_PORT=${INFINITY_HTTP_PORT}" >> .env
echo -e "INFINITY_PSQL_PORT=${INFINITY_PSQL_PORT}" >> .env
echo -e "EXPOSE_MYSQL_PORT=${EXPOSE_MYSQL_PORT}" >> .env
echo -e "MINIO_PORT=${MINIO_PORT}" >> .env
echo -e "MINIO_CONSOLE_PORT=${MINIO_CONSOLE_PORT}" >> .env
echo -e "REDIS_PORT=${REDIS_PORT}" >> .env
echo -e "TEI_PORT=${TEI_PORT}" >> .env
echo -e "KIBANA_PORT=${KIBANA_PORT}" >> .env
echo -e "SVR_HTTP_PORT=${SVR_HTTP_PORT}" >> .env
echo -e "ADMIN_SVR_HTTP_PORT=${ADMIN_SVR_HTTP_PORT}" >> .env
echo -e "SVR_MCP_PORT=${SVR_MCP_PORT}" >> .env
echo -e "GO_HTTP_PORT=${GO_HTTP_PORT}" >> .env
echo -e "GO_ADMIN_PORT=${GO_ADMIN_PORT}" >> .env
echo -e "SANDBOX_EXECUTOR_MANAGER_PORT=${SANDBOX_EXECUTOR_MANAGER_PORT}" >> .env
echo -e "SVR_WEB_HTTP_PORT=${SVR_WEB_HTTP_PORT}" >> .env
echo -e "SVR_WEB_HTTPS_PORT=${SVR_WEB_HTTPS_PORT}" >> .env
echo -e "COMPOSE_PROFILES=\${COMPOSE_PROFILES},tei-cpu" >> .env
echo -e "TEI_MODEL=BAAI/bge-small-en-v1.5" >> .env
echo -e "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> .env
echo "HOST_ADDRESS=http://host.docker.internal:${SVR_HTTP_PORT}" >> ${GITHUB_ENV}
echo -e "COMPOSE_PROFILES=\${COMPOSE_PROFILES},tei-cpu" >> docker/.env
echo -e "TEI_MODEL=BAAI/bge-small-en-v1.5" >> docker/.env
echo -e "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> docker/.env
echo "HOST_ADDRESS=http://host.docker.internal:${SVR_HTTP_PORT}" >> ${GITHUB_ENV}
# Patch entrypoint.sh for coverage
sed -i '/"\$PY" api\/ragflow_server.py \${INIT_SUPERUSER_ARGS} &/c\ echo "Ensuring coverage is installed..."\n "$PY" -m pip install coverage -i https://mirrors.aliyun.com/pypi/simple\n export COVERAGE_FILE=/ragflow/logs/.coverage\n echo "Starting ragflow_server with coverage..."\n "$PY" -m coverage run --source=./api/apps --omit="*/tests/*,*/migrations/*" -a api/ragflow_server.py ${INIT_SUPERUSER_ARGS} &' ./entrypoint.sh
cd ..
uv sync --python 3.12 --group test --frozen && uv pip install -e sdk/python
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
uv sync --python 3.12 --only-group test --no-default-groups --frozen && uv pip install sdk/python --group test
- name: Run sdk tests against Elasticsearch
- name: Start ragflow:nightly for Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_sdk_api 2>&1 | tee es_sdk_test.log
- name: Run frontend api tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short sdk/python/test/test_frontend_api/get_email.py sdk/python/test/test_frontend_api/test_dataset.py 2>&1 | tee es_api_test.log
- name: Run http api tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api 2>&1 | tee es_http_api_test.log
- name: Stop ragflow:nightly
if: always() # always run this step even if previous steps failed
run: |
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
- name: Start ragflow:nightly
run: |
sed -i '1i DOC_ENGINE=infinity' docker/.env
sed -i 's/^DOC_ENGINE=.*$/DOC_ENGINE=infinity/' docker/.env
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
- name: Run sdk tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_sdk_api 2>&1 | tee infinity_sdk_test.log
echo "Start to run test sdk on Infinity"
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} --junitxml=pytest-infinity-sdk.xml --cov=sdk/python/ragflow_sdk --cov-branch --cov-report=xml:coverage-infinity-sdk.xml test/testcases/test_sdk_api 2>&1 | tee infinity_sdk_test.log
- name: Run frontend api tests against Infinity
- name: Run New RESTFUL api tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short sdk/python/test/test_frontend_api/get_email.py sdk/python/test/test_frontend_api/test_dataset.py 2>&1 | tee infinity_api_test.log
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/restful_api 2>&1 | tee infinity_restful_api_test.log
- name: Run http api tests against Infinity
- name: RAGFlow CLI retrieval test Infinity
env:
PYTHONPATH: ${{ github.workspace }}
run: |
set -euo pipefail
source .venv/bin/activate
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
EMAIL="ci-${GITHUB_RUN_ID}@example.com"
PASS="ci-pass-${GITHUB_RUN_ID}"
DATASET="ci_dataset_${GITHUB_RUN_ID}"
CLI="python admin/client/ragflow_cli.py"
LOG_FILE="infinity_cli_test.log"
: > "${LOG_FILE}"
ERROR_RE='Traceback|ModuleNotFoundError|ImportError|Parse error|Bad response|Fail to|code:\\s*[1-9]'
run_cli() {
local logfile="$1"
shift
local allow_re=""
if [[ "${1:-}" == "--allow" ]]; then
allow_re="$2"
shift 2
fi
local cmd_display="$*"
echo "===== $(date -u +\"%Y-%m-%dT%H:%M:%SZ\") CMD: ${cmd_display} =====" | tee -a "${logfile}"
local tmp_log
tmp_log="$(mktemp)"
set +e
timeout 500s "$@" 2>&1 | tee "${tmp_log}"
local status=${PIPESTATUS[0]}
set -e
cat "${tmp_log}" >> "${logfile}"
if grep -qiE "${ERROR_RE}" "${tmp_log}"; then
if [[ -n "${allow_re}" ]] && grep -qiE "${allow_re}" "${tmp_log}"; then
echo "Allowed CLI error markers in ${logfile}"
rm -f "${tmp_log}"
return 0
fi
echo "Detected CLI error markers in ${logfile}"
rm -f "${tmp_log}"
exit 1
fi
rm -f "${tmp_log}"
return ${status}
}
set -a
source docker/.env
set +a
HOST_ADDRESS="http://host.docker.internal:${SVR_HTTP_PORT}"
USER_HOST="$(echo "${HOST_ADDRESS}" | sed -E 's#^https?://([^:/]+).*#\1#')"
USER_PORT="${SVR_HTTP_PORT}"
ADMIN_HOST="${USER_HOST}"
ADMIN_PORT="${ADMIN_SVR_HTTP_PORT}"
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
admin_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "ping"; then
admin_ready=1
break
fi
sleep 1
done
if [[ "${admin_ready}" -ne 1 ]]; then
echo "Admin service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "show version"
ALLOW_USER_EXISTS_RE='already exists|already exist|duplicate|already.*registered|exist(s)?'
run_cli "${LOG_FILE}" --allow "${ALLOW_USER_EXISTS_RE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "create user '$EMAIL' '$PASS'"
user_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "ping"; then
user_ready=1
break
fi
sleep 1
done
if [[ "${user_ready}" -ne 1 ]]; then
echo "User service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "show version"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "create dataset '$DATASET' with embedding 'BAAI/bge-small-en-v1.5@Builtin' parser 'auto'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "import 'test/benchmark/test_docs/Doc1.pdf,test/benchmark/test_docs/Doc2.pdf' into dataset '$DATASET'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "parse dataset '$DATASET' sync"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "Benchmark 16 100 search 'what are these documents about' on datasets '$DATASET'"
- name: Stop ragflow to save coverage Infinity
if: ${{ !cancelled() }}
run: |
# Send SIGINT to ragflow_server.py to trigger coverage save
PID=$(sudo docker exec ${RAGFLOW_CONTAINER} ps aux | grep "ragflow_server.py" | grep -v grep | awk '{print $2}' | head -n 1)
if [ -n "$PID" ]; then
echo "Sending SIGINT to ragflow_server.py (PID: $PID)..."
sudo docker exec ${RAGFLOW_CONTAINER} kill -INT $PID
# Wait for process to exit and coverage file to be written
sleep 10
else
echo "ragflow_server.py not found!"
fi
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} stop
- name: Generate server coverage report Infinity
if: ${{ !cancelled() }}
run: |
# .coverage file should be in docker/ragflow-logs/.coverage
if [ -f docker/ragflow-logs/.coverage ]; then
echo "Found .coverage file"
cp docker/ragflow-logs/.coverage .coverage
source .venv/bin/activate
# Create .coveragerc to map container paths to host paths
echo "[paths]" > .coveragerc
echo "source =" >> .coveragerc
echo " ." >> .coveragerc
echo " /ragflow" >> .coveragerc
coverage xml -o coverage-infinity-server.xml
rm .coveragerc
else
echo ".coverage file not found!"
fi
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
if: ${{ !cancelled() }}
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
- name: Collect ragflow log Infinity
if: ${{ !cancelled() }}
run: |
if [ -d docker/ragflow-logs ]; then
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-infinity
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
else
echo "No docker/ragflow-logs directory found; skipping log collection"
fi
sudo rm -rf docker/ragflow-logs || true
- name: Stop ragflow:nightly for Infinity
if: always() # always run this step even if previous steps failed
run: |
# Sometimes `docker compose down` fail due to hang container, heavy load etc. Need to remove such containers to release resources(for example, listen ports).
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
- name: Start ragflow:nightly for Elasticsearch
run: |
sed -i 's/^DOC_ENGINE=.*$/DOC_ENGINE=elasticsearch/' docker/.env
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
- name: Run sdk tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api 2>&1 | tee infinity_http_api_test.log
echo "Start to run test sdk on Elasticsearch"
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} --junitxml=pytest-infinity-sdk.xml --cov=sdk/python/ragflow_sdk --cov-branch --cov-report=xml:coverage-es-sdk.xml test/testcases/test_sdk_api 2>&1 | tee es_sdk_test.log
- name: Run New RESTFUL api tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/restful_api 2>&1 | tee es_restful_api_test.log
- name: Stop ragflow:nightly
- name: RAGFlow CLI retrieval test Elasticsearch
env:
PYTHONPATH: ${{ github.workspace }}
run: |
set -euo pipefail
source .venv/bin/activate
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
EMAIL="ci-${GITHUB_RUN_ID}@example.com"
PASS="ci-pass-${GITHUB_RUN_ID}"
DATASET="ci_dataset_${GITHUB_RUN_ID}"
CLI="python admin/client/ragflow_cli.py"
LOG_FILE="es_cli_test.log"
: > "${LOG_FILE}"
ERROR_RE='Traceback|ModuleNotFoundError|ImportError|Parse error|Bad response|Fail to|code:\\s*[1-9]'
run_cli() {
local logfile="$1"
shift
local allow_re=""
if [[ "${1:-}" == "--allow" ]]; then
allow_re="$2"
shift 2
fi
local cmd_display="$*"
echo "===== $(date -u +\"%Y-%m-%dT%H:%M:%SZ\") CMD: ${cmd_display} =====" | tee -a "${logfile}"
local tmp_log
tmp_log="$(mktemp)"
set +e
timeout 500s "$@" 2>&1 | tee "${tmp_log}"
local status=${PIPESTATUS[0]}
set -e
cat "${tmp_log}" >> "${logfile}"
if grep -qiE "${ERROR_RE}" "${tmp_log}"; then
if [[ -n "${allow_re}" ]] && grep -qiE "${allow_re}" "${tmp_log}"; then
echo "Allowed CLI error markers in ${logfile}"
rm -f "${tmp_log}"
return 0
fi
echo "Detected CLI error markers in ${logfile}"
rm -f "${tmp_log}"
exit 1
fi
rm -f "${tmp_log}"
return ${status}
}
set -a
source docker/.env
set +a
HOST_ADDRESS="http://host.docker.internal:${SVR_HTTP_PORT}"
USER_HOST="$(echo "${HOST_ADDRESS}" | sed -E 's#^https?://([^:/]+).*#\1#')"
USER_PORT="${SVR_HTTP_PORT}"
ADMIN_HOST="${USER_HOST}"
ADMIN_PORT="${ADMIN_SVR_HTTP_PORT}"
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null 2>&1; do
echo "Waiting for service to be available... (last exit code: $?)"
sleep 5
done
admin_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "ping"; then
admin_ready=1
break
fi
sleep 1
done
if [[ "${admin_ready}" -ne 1 ]]; then
echo "Admin service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "show version"
ALLOW_USER_EXISTS_RE='already exists|already exist|duplicate|already.*registered|exist(s)?'
run_cli "${LOG_FILE}" --allow "${ALLOW_USER_EXISTS_RE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "create user '$EMAIL' '$PASS'"
user_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "ping"; then
user_ready=1
break
fi
sleep 1
done
if [[ "${user_ready}" -ne 1 ]]; then
echo "User service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "show version"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "create dataset '$DATASET' with embedding 'BAAI/bge-small-en-v1.5@Builtin' parser 'auto'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "import 'test/benchmark/test_docs/Doc1.pdf,test/benchmark/test_docs/Doc2.pdf' into dataset '$DATASET'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "parse dataset '$DATASET' sync"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "Benchmark 16 100 search 'what are these documents about' on datasets '$DATASET'"
- name: Stop ragflow to save coverage Elasticsearch
if: ${{ !cancelled() }}
run: |
# Send SIGINT to ragflow_server.py to trigger coverage save
PID=$(sudo docker exec ${RAGFLOW_CONTAINER} ps aux | grep "ragflow_server.py" | grep -v grep | awk '{print $2}' | head -n 1)
if [ -n "$PID" ]; then
echo "Sending SIGINT to ragflow_server.py (PID: $PID)..."
sudo docker exec ${RAGFLOW_CONTAINER} kill -INT $PID
# Wait for process to exit and coverage file to be written
sleep 10
else
echo "ragflow_server.py not found!"
fi
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} stop
- name: Generate server coverage report Elasticsearch
if: ${{ !cancelled() }}
run: |
# .coverage file should be in docker/ragflow-logs/.coverage
if [ -f docker/ragflow-logs/.coverage ]; then
echo "Found .coverage file"
cp docker/ragflow-logs/.coverage .coverage
source .venv/bin/activate
# Create .coveragerc to map container paths to host paths
echo "[paths]" > .coveragerc
echo "source =" >> .coveragerc
echo " ." >> .coveragerc
echo " /ragflow" >> .coveragerc
coverage xml -o coverage-es-server.xml
rm .coveragerc
# Clean up for next run
sudo rm docker/ragflow-logs/.coverage
else
echo ".coverage file not found!"
fi
- name: Collect ragflow log Elasticsearch
if: ${{ !cancelled() }}
run: |
if [ -d docker/ragflow-logs ]; then
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-es
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
else
echo "No docker/ragflow-logs directory found; skipping log collection"
fi
sudo rm -rf docker/ragflow-logs || true
- name: Stop ragflow:nightly for Elasticsearch
if: always() # always run this step even if previous steps failed
run: |
# Sometimes `docker compose down` fail due to hang container, heavy load etc. Need to remove such containers to release resources(for example, listen ports).

38
.gitignore vendored
View File

@@ -7,7 +7,7 @@ hudet/
cv/
layout_app.py
api/flask_session
venv/
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
@@ -44,6 +44,7 @@ cl100k_base.tiktoken
chrome*
huggingface.co/
nltk_data/
uv-x86_64*.tar.gz
# Exclude hash-like temporary files like 9b5ad71b2ce5302211f9c61530b329a4922fc6a4
*[0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f][0-9a-f]*
@@ -51,6 +52,13 @@ nltk_data/
.venv
docker/data
# OceanBase data and conf
docker/oceanbase/conf
docker/oceanbase/data
# SeekDB data and conf
docker/seekdb
#--------------------------------------------------#
# The following was generated with gitignore.nvim: #
@@ -197,4 +205,30 @@ ragflow_cli.egg-info
backup
.hypothesis
*huqie.txt
.hypothesis
# Added by cargo
/target
# Do not include in PR (local dev / build artifacts)
ragflow.egg-info/
uv-aarch64*.tar.gz
uv-aarch64-unknown-linux-gnu.tar.gz
docker/launch_backend_service_windows.sh
# C++ build directories
internal/cpp/build/
internal/cpp/cmake-build-release/
internal/cpp/cmake-build-debug/
# Trae IDE config
.trae/
# Go server build output
bin/*
!bin/.gitkeep
.claude/settings.local.json

View File

@@ -35,7 +35,7 @@ The project uses **uv** for dependency management.
1. **Setup Environment**:
```bash
uv sync --python 3.12 --all-extras
uv run download_deps.py
uv run python3 download_deps.py
```
2. **Run Server**:

View File

@@ -5,14 +5,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It's a full-stack application with:
- Python backend (Flask-based API server)
- React/TypeScript frontend (built with UmiJS)
- React/TypeScript frontend (built with vitejs)
- Microservices architecture with Docker deployment
- Multiple data stores (MySQL, Elasticsearch/Infinity, Redis, MinIO)
## Architecture
### Backend (`/api/`)
- **Main Server**: `api/ragflow_server.py` - Flask application entry point
- **Apps**: Modular Flask blueprints in `api/apps/` for different functionalities:
- `kb_app.py` - Knowledge base management
@@ -24,29 +26,33 @@ RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on d
- **Models**: Database models in `api/db/db_models.py`
### Core Processing (`/rag/`)
- **Document Processing**: `deepdoc/` - PDF parsing, OCR, layout analysis
- **LLM Integration**: `rag/llm/` - Model abstractions for chat, embedding, reranking
- **RAG Pipeline**: `rag/flow/` - Chunking, parsing, tokenization
- **Graph RAG**: `graphrag/` - Knowledge graph construction and querying
- **Graph RAG**: `rag/graphrag/` - Knowledge graph construction and querying
### Agent System (`/agent/`)
- **Components**: Modular workflow components (LLM, retrieval, categorize, etc.)
- **Templates**: Pre-built agent workflows in `agent/templates/`
- **Tools**: External API integrations (Tavily, Wikipedia, SQL execution, etc.)
### Frontend (`/web/`)
- React/TypeScript with UmiJS framework
- Ant Design + shadcn/ui components
- React/TypeScript with vitejs framework
- shadcn/ui components
- State management with Zustand
- Tailwind CSS for styling
## Common Development Commands
### Backend Development
```bash
# Install Python dependencies
uv sync --python 3.12 --all-extras
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
# Start dependent services
@@ -66,6 +72,7 @@ ruff format
```
### Frontend Development
```bash
cd web
npm install
@@ -76,6 +83,7 @@ npm run test # Jest tests
```
### Docker Development
```bash
# Full stack with Docker
cd docker
@@ -104,6 +112,7 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
## Database Engines
RAGFlow supports switching between Elasticsearch (default) and Infinity:
- Set `DOC_ENGINE=infinity` in `docker/.env` to use Infinity
- Requires container restart: `docker compose down -v && docker compose up -d`
@@ -113,4 +122,13 @@ RAGFlow supports switching between Elasticsearch (default) and Infinity:
- Node.js >=18.20.4
- Docker & Docker Compose
- uv package manager
- 16GB+ RAM, 50GB+ disk space
- 16GB+ RAM, 50GB+ disk space
1. Think before acting. Read existing files before writing code.
2. Be concise in output but thorough in reasoning.
3. Prefer editing over rewriting whole files.
4. Do not re-read files you have already read.
5. Test your code before declaring done.
6. No sycophantic openers or closing fluff.
7. Keep solutions simple and direct.
8. User instructions always override this file.

View File

@@ -7,7 +7,7 @@ ARG NEED_MIRROR=0
WORKDIR /ragflow
# Copy models downloaded via download_deps.py
# copy models downloaded via download_deps.py
RUN mkdir -p /ragflow/rag/res/deepdoc /root/.ragflow
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/huggingface.co,target=/huggingface.co \
tar --exclude='.*' -cf - \
@@ -19,41 +19,50 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/huggingface.co
# This is the only way to run python-tika without internet access. Without this set, the default is to check the tika version and pull latest every time from Apache.
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps \
cp -r /deps/nltk_data /root/ && \
cp /deps/tika-server-standard-3.0.0.jar /deps/tika-server-standard-3.0.0.jar.md5 /ragflow/ && \
cp /deps/tika-server-standard-3.3.0.jar /deps/tika-server-standard-3.3.0.jar.md5 /ragflow/ && \
cp /deps/cl100k_base.tiktoken /ragflow/9b5ad71b2ce5302211f9c61530b329a4922fc6a4
ENV TIKA_SERVER_JAR="file:///ragflow/tika-server-standard-3.0.0.jar"
ENV TIKA_SERVER_JAR="file:///ragflow/tika-server-standard-3.3.0.jar"
ENV DEBIAN_FRONTEND=noninteractive
# Setup apt
# Python package and implicit dependencies:
# opencv-python: libglib2.0-0 libglx-mesa0 libgl1
# aspose-slides: pkg-config libicu-dev libgdiplus libssl1.1_1.1.1f-1ubuntu2_amd64.deb
# python-pptx: default-jdk tika-server-standard-3.0.0.jar
# python-pptx: default-jdk tika-server-standard-3.3.0.jar
# selenium: libatk-bridge2.0-0 chrome-linux64-121-0-6167-85
# Building C extensions: libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
apt update && \
apt --no-install-recommends install -y ca-certificates; \
if [ "$NEED_MIRROR" == "1" ]; then \
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.aliyun.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://security.ubuntu.com/ubuntu|https://mirrors.aliyun.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
fi; \
rm -f /etc/apt/apt.conf.d/docker-clean && \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache && \
chmod 1777 /tmp && \
apt update && \
apt install -y libglib2.0-0 libglx-mesa0 libgl1 && \
apt install -y pkg-config libicu-dev libgdiplus && \
apt install -y default-jdk && \
apt install -y libatk-bridge2.0-0 && \
apt install -y libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev && \
apt install -y libjemalloc-dev && \
apt install -y nginx unzip curl wget git vim less && \
apt install -y ghostscript && \
apt install -y pandoc && \
apt install -y texlive && \
apt install -y fonts-freefont-ttf fonts-noto-cjk
apt install -y \
build-essential libglib2.0-0 libglx-mesa0 libgl1 pkg-config libicu-dev libgdiplus default-jdk libatk-bridge2.0-0 libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev libjemalloc-dev gnupg unzip curl wget git vim less ghostscript pandoc texlive texlive-latex-extra texlive-xetex texlive-lang-chinese fonts-freefont-ttf fonts-noto-cjk postgresql-client
# Download resource from GitHub to /usr/share/infinity
RUN mkdir -p /usr/share/infinity/resource && \
if [ "$NEED_MIRROR" == "1" ]; then \
git clone --depth 1 --single-branch https://gitee.com/infiniflow/resource /tmp/resource; \
else \
git clone --depth 1 --single-branch https://github.com/infiniflow/resource.git /tmp/resource; \
fi && \
cp -r /tmp/resource/* /usr/share/infinity/resource && \
rm -rf /tmp/resource
ARG NGINX_VERSION=1.29.5-1~noble
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
mkdir -p /etc/apt/keyrings && \
curl --retry 5 --retry-delay 2 --retry-all-errors -fsSL https://nginx.org/keys/nginx_signing.key | gpg --dearmor -o /etc/apt/keyrings/nginx-archive-keyring.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nginx-archive-keyring.gpg] https://nginx.org/packages/mainline/ubuntu/ noble nginx" > /etc/apt/sources.list.d/nginx.list && \
apt -o Acquire::Retries=5 update && \
apt -o Acquire::Retries=5 install -y nginx=${NGINX_VERSION} && \
apt-mark hold nginx
# Install uv
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps \
@@ -61,41 +70,29 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps
mkdir -p /etc/uv && \
echo 'python-install-mirror = "https://registry.npmmirror.com/-/binary/python-build-standalone/"' > /etc/uv/uv.toml && \
echo '[[index]]' >> /etc/uv/uv.toml && \
echo 'url = "https://pypi.tuna.tsinghua.edu.cn/simple"' >> /etc/uv/uv.toml && \
echo 'url = "https://mirrors.aliyun.com/pypi/simple"' >> /etc/uv/uv.toml && \
echo 'default = true' >> /etc/uv/uv.toml; \
fi; \
tar xzf /deps/uv-x86_64-unknown-linux-gnu.tar.gz \
&& cp uv-x86_64-unknown-linux-gnu/* /usr/local/bin/ \
&& rm -rf uv-x86_64-unknown-linux-gnu \
&& uv python install 3.11
arch="$(uname -m)"; \
if [ "$arch" = "x86_64" ]; then uv_arch="x86_64"; else uv_arch="aarch64"; fi; \
tar xzf "/deps/uv-${uv_arch}-unknown-linux-gnu.tar.gz" \
&& cp "uv-${uv_arch}-unknown-linux-gnu/"* /usr/local/bin/ \
&& rm -rf "uv-${uv_arch}-unknown-linux-gnu" \
&& uv python install 3.12
ENV PYTHONDONTWRITEBYTECODE=1 DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
ENV PYTHONDONTWRITEBYTECODE=1 DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 \
UV_HTTP_TIMEOUT=200 \
UV_HTTP_RETRIES=3
ENV PATH=/root/.local/bin:$PATH
# nodejs 12.22 on Ubuntu 22.04 is too old
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \
apt purge -y nodejs npm cargo && \
apt purge -y nodejs npm && \
apt autoremove -y && \
apt update && \
apt install -y nodejs
# A modern version of cargo is needed for the latest version of the Rust compiler.
RUN apt update && apt install -y curl build-essential \
&& if [ "$NEED_MIRROR" == "1" ]; then \
# Use TUNA mirrors for rustup/rust dist files \
export RUSTUP_DIST_SERVER="https://mirrors.tuna.tsinghua.edu.cn/rustup"; \
export RUSTUP_UPDATE_ROOT="https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup"; \
echo "Using TUNA mirrors for Rustup."; \
fi; \
# Force curl to use HTTP/1.1 \
curl --proto '=https' --tlsv1.2 --http1.1 -sSf https://sh.rustup.rs | bash -s -- -y --profile minimal \
&& echo 'export PATH="/root/.cargo/bin:${PATH}"' >> /root/.bashrc
ENV PATH="/root/.cargo/bin:${PATH}"
RUN cargo --version && rustc --version
# Add msssql ODBC driver
# macOS ARM64 environment, install msodbcsql18.
# general x86_64 environment, install msodbcsql17.
@@ -125,8 +122,6 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/chromedriver-l
mv chromedriver /usr/local/bin/ && \
rm -f /usr/bin/google-chrome
# https://forum.aspose.com/t/aspose-slides-for-net-no-usable-version-of-libssl-found-with-linux-server/271344/13
# aspose-slides on linux/arm64 is unavailable
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps \
if [ "$(uname -m)" = "x86_64" ]; then \
dpkg -i /deps/libssl1.1_1.1.1f-1ubuntu2_amd64.deb; \
@@ -148,16 +143,19 @@ COPY pyproject.toml uv.lock ./
# uv records index url into uv.lock but doesn't failover among multiple indexes
RUN --mount=type=cache,id=ragflow_uv,target=/root/.cache/uv,sharing=locked \
if [ "$NEED_MIRROR" == "1" ]; then \
sed -i 's|pypi.org|pypi.tuna.tsinghua.edu.cn|g' uv.lock; \
sed -i 's|pypi.org|mirrors.aliyun.com/pypi|g' uv.lock; \
else \
sed -i 's|pypi.tuna.tsinghua.edu.cn|pypi.org|g' uv.lock; \
sed -i 's|mirrors.aliyun.com/pypi|pypi.org|g' uv.lock; \
fi; \
uv sync --python 3.12 --frozen
uv sync --python 3.12 --frozen && \
# Ensure pip is available in the venv for runtime package installation (fixes #12651)
.venv/bin/python3 -m ensurepip --upgrade
COPY web web
COPY docs docs
RUN --mount=type=cache,id=ragflow_npm,target=/root/.npm,sharing=locked \
cd web && npm install && npm run build
cd web && NODE_OPTIONS="--max-old-space-size=8192" npm install && \
NODE_OPTIONS="--max-old-space-size=8192" VITE_BUILD_SOURCEMAP=false VITE_MINIFY=esbuild npm run build
COPY .git /ragflow/.git
@@ -186,18 +184,23 @@ COPY conf conf
COPY deepdoc deepdoc
COPY rag rag
COPY agent agent
COPY graphrag graphrag
COPY agentic_reasoning agentic_reasoning
COPY pyproject.toml uv.lock ./
COPY mcp mcp
COPY plugin plugin
COPY common common
COPY memory memory
COPY bin bin
COPY docker/service_conf.yaml.template ./conf/service_conf.yaml.template
COPY docker/entrypoint.sh ./
RUN chmod +x ./entrypoint*.sh
# Copy nginx configuration for frontend serving
COPY docker/nginx/ragflow.conf.golang docker/nginx/ragflow.conf.python docker/nginx/ragflow.conf.hybrid docker/nginx/nginx.conf docker/nginx/proxy.conf /etc/nginx/
RUN mv /etc/nginx/ragflow.conf.golang /etc/nginx/conf.d/ragflow.conf.golang && \
mv /etc/nginx/ragflow.conf.python /etc/nginx/conf.d/ragflow.conf.python && \
mv /etc/nginx/ragflow.conf.hybrid /etc/nginx/conf.d/ragflow.conf.hybrid && \
rm -f /etc/nginx/sites-enabled/default
# Copy compiled web pages
COPY --from=builder /ragflow/web/dist /ragflow/web/dist

View File

@@ -3,7 +3,7 @@
FROM scratch
# Copy resources downloaded via download_deps.py
COPY chromedriver-linux64-121-0-6167-85 chrome-linux64-121-0-6167-85 cl100k_base.tiktoken libssl1.1_1.1.1f-1ubuntu2_amd64.deb libssl1.1_1.1.1f-1ubuntu2_arm64.deb tika-server-standard-3.0.0.jar tika-server-standard-3.0.0.jar.md5 libssl*.deb uv-x86_64-unknown-linux-gnu.tar.gz /
COPY chromedriver-linux64-121-0-6167-85 chrome-linux64-121-0-6167-85 cl100k_base.tiktoken libssl1.1_1.1.1f-1ubuntu2_amd64.deb libssl1.1_1.1.1f-1ubuntu2_arm64.deb tika-server-standard-3.3.0.jar tika-server-standard-3.3.0.jar.md5 libssl*.deb uv-x86_64-unknown-linux-gnu.tar.gz uv-aarch64-unknown-linux-gnu.tar.gz /
COPY nltk_data /nltk_data

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Table of Contents</b></summary>
- 💡 [What is RAGFlow?](#-what-is-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Get Started](#-get-started)
- 📌 [Latest Updates](#-latest-updates)
- 🌟 [Key Features](#-key-features)
- 🔎 [System Architecture](#-system-architecture)
- 🎬 [Get Started](#-get-started)
- 🎬 [Self-Hosting](#-self-hosting)
- 🔧 [Configurations](#-configurations)
- 🔧 [Build a Docker image](#-build-a-docker-image)
- 🔨 [Launch service from source for development](#-launch-service-from-source-for-development)
@@ -72,11 +74,11 @@
## 💡 What is RAGFlow?
[RAGFlow](https://ragflow.io/) is a leading open-source Retrieval-Augmented Generation (RAG) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs. It offers a streamlined RAG workflow adaptable to enterprises of any scale. Powered by a converged context engine and pre-built agent templates, RAGFlow enables developers to transform complex data into high-fidelity, production-ready AI systems with exceptional efficiency and precision.
[RAGFlow](https://ragflow.io/) is a leading open-source Retrieval-Augmented Generation ([RAG](https://ragflow.io/basics/what-is-rag)) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs. It offers a streamlined RAG workflow adaptable to enterprises of any scale. Powered by a converged [context engine](https://ragflow.io/basics/what-is-agent-context-engine) and pre-built agent templates, RAGFlow enables developers to transform complex data into high-fidelity, production-ready AI systems with exceptional efficiency and precision.
## 🎮 Demo
## 🎮 Get Started
Try our demo at [https://demo.ragflow.io](https://demo.ragflow.io).
Try our cloud service at [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@ Try our demo at [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Latest Updates
- 2026-04-24 Supports DeepSeek v4.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Provides an official skill for accessing RAGFlow datasets via OpenClaw.
- 2025-12-26 Supports 'Memory' for AI agent.
- 2025-11-19 Supports Gemini 3 Pro.
- 2025-11-12 Supports data synchronization from Confluence, S3, Notion, Discord, Google Drive.
@@ -140,7 +144,7 @@ releases! 🌟
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Get Started
## 🎬 Self-Hosting
### 📝 Prerequisites
@@ -188,15 +192,15 @@ releases! 🌟
> All Docker images are built for x86 platforms. We don't currently offer Docker images for ARM64.
> If you are on an ARM64 platform, follow [this guide](https://ragflow.io/docs/dev/build_docker_image) to build a Docker image compatible with your system.
> The command below downloads the `v0.23.1` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.23.1`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server.
> The command below downloads the `v0.25.3` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.25.3`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server.
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# This step ensures the **entrypoint.sh** file in the code matches the Docker image version.
# Use CPU for DeepDoc tasks:
$ docker compose -f docker-compose.yml up -d
@@ -325,7 +329,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
3. Launch the dependent services (MinIO, Elasticsearch, Redis, and MySQL) using Docker Compose:
@@ -389,8 +393,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -401,7 +405,7 @@ See the [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/1224
## 🏄 Community
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contributing

414
README_ar.md Normal file
View File

@@ -0,0 +1,414 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DBEDFA"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/License-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="license">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 جدول المحتويات</b></summary>
- 💡 [ما هو RAGFlow؟](#-what-is-ragflow)
- 🎮 [ابدأ](#-get-started)
- 📌 [آخر التحديثات](#-latest-updates)
- 🌟 [الميزات الرئيسية](#-key-features)
- 🔎 [بنية النظام](#-system-architecture)
- 🎬 [الاستضافة الذاتية](#-self-hosting)
- 🔧 [التكوينات](#-configurations)
- 🔧 [إنشاء صورة Docker](#-build-a-docker-image)
- 🔨 [إطلاق الخدمة من المصدر للتطوير](#-launch-service-from-source-for-development)
- 📚 [التوثيق](#-documentation)
- 📜 [Roadmap](#-roadmap)
- 🏄 [المجتمع](#-community)
- 🙌 [مساهمة](#-contributing)
</details>
## 💡 ما هو RAGFlow؟
يُعد مشروع [RAGFlow](https://ragflow.io/) محركًا رائدًا ومفتوح المصدر للاسترجاع المعزز بالتوليد (<bdi dir="ltr">RAG</bdi>)، ويجمع أحدث تقنيات <bdi dir="ltr">RAG</bdi> مع قدرات الوكلاء لبناء طبقة سياق متقدمة لنماذج <bdi dir="ltr">LLMs</bdi>. يوفّر سير عمل <bdi dir="ltr">RAG</bdi> مبسّطًا وقابلًا للتكيّف مع المؤسسات بمختلف أحجامها. وبالاعتماد على [محرك سياق موحّد](https://ragflow.io/basics/what-is-agent-context-engine) وقوالب وكلاء جاهزة، يتيح <bdi dir="ltr">RAGFlow</bdi> للمطورين تحويل البيانات المعقّدة إلى أنظمة <bdi dir="ltr">AI</bdi> عالية الدقة وجاهزة للإنتاج بكفاءة وموثوقية.
## 🎮 ابدأ
جرّب النسخة التجريبية على [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 آخر التحديثات
- 24-04-2026 يدعم DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — توفر مهارة رسمية للوصول إلى مجموعات بيانات RAGFlow عبر OpenClaw.
- 26-12-2025 يدعم ميزة "Memory" لوكلاء الذكاء الاصطناعي.
- 11-11-2025 يدعم Gemini 3 Pro.
- 12-11-2025 يدعم مزامنة البيانات من Confluence، S3، Notion، Discord، Google Drive.
- 23-10-2025 يدعم MinerU وDocling كطرق لتحليل المستندات.
- 15-10-2025 يدعم العرض الأوركسترالي pipeline.
- 08-08-2025 يدعم أحدث موديلات سلسلة OpenAI.
- 01-08-2025 يدعم سير العمل الوكيل وMCP.
- 23-05-2025 تمت إضافة مكون منفذ كود Python/JavaScript إلى Agent.
- 05-05-2025 يدعم الاستعلام بين اللغات.
- 19-03-2025 يدعم استخدام نموذج متعدد الوسائط لفهم الصور داخل ملفات PDF أو DOCX.
## 🎉 تابعونا
⭐️ قم بتمييز مستودعنا بنجمة لتبقى على اطلاع بالميزات والتحسينات الجديدة والمثيرة! احصل على إشعارات فورية بالجديد
الإصدارات! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 الميزات الرئيسية
### 🍭 **"الجودة في الداخل، الجودة في الخارج"**
- [الفهم العميق للمستندات](./deepdoc/README.md) لاستخراج المعرفة من البيانات غير المنظمة
ذات التنسيقات المعقدة.
- يجد "إبرة في كومة قش بيانات" من الرموز غير المحدودة حرفيًا.
### 🍱 **التقطيع القائم على القالب**
- ذكي وقابل للتفسير.
- الكثير من خيارات القالب للاختيار من بينها.
### 🌱 **استشهادات مؤرضة لتقليل الهلوسة**
- تصور تقطيع النص للسماح بالتدخل البشري.
- عرض سريع للمراجع الرئيسية والاستشهادات التي يمكن تتبعها لدعم الإجابات المبنية على أسس سليمة.
### 🍔 **التوافق مع مصادر البيانات غير المتجانسة**
- يدعم Word، والشرائح، وExcel، وtxt، والصور، والنسخ الممسوحة ضوئيًا، والبيانات المنظمة، وصفحات الويب، والمزيد.
### 🛀 **سير عمل RAG آلي وسهل**
- تنسيق RAG مبسط يلبي احتياجات الشركات الشخصية والكبيرة على حد سواء.
- نماذج LLMs قابلة للتكوين بالإضافة إلى نماذج embedding.
- الاستدعاء المتعدد المقترن بإعادة التصنيف المدمجة.
- APIs بديهي للتكامل السلس مع الأعمال.
## 🔎 هندسة النظام
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 الاستضافة الذاتية
### 📝 المتطلبات الأساسية
- CPU >= 4 مراكز
- الرام >= 16 جيجا
- القرص >= 50 جيجا بايت
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- [gVisor](https://gvisor.dev/docs/user_guide/install/): مطلوب فقط إذا كنت تنوي استخدام ميزة منفذ التعليمات البرمجية (وضع الحماية) لـ RAGFlow.
> [!TIP]
> إذا لم تقم بتثبيت Docker على جهازك المحلي (Windows أو Mac أو Linux)، راجع [تثبيت Docker Engine](https://docs.docker.com/engine/install/).
### 🚀 بدء تشغيل الخادم
1. تأكد من `vm.max_map_count` >= 262144:
> للتحقق من قيمة `vm.max_map_count`:
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> أعد تعيين `vm.max_map_count` إلى قيمة 262144 على الأقل إذا لم تكن كذلك.
>
> ```bash
> # In this case, we set it to 262144:
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> سيتم إعادة ضبط هذا التغيير بعد إعادة تشغيل النظام. لضمان بقاء التغيير دائمًا، قم بإضافة أو تحديث
> `vm.max_map_count` القيمة في **/etc/sysctl.conf** وفقًا لذلك:
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. استنساخ الريبو:
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. ابدأ تشغيل الخادم باستخدام صور Docker المعدة مسبقًا:
> [!CAUTION]
> جميع الصور Docker مصممة لمنصات x86. لا نعرض حاليًا صور Docker لـ ARM64.
> إذا كنت تستخدم نظامًا أساسيًا ARM64، فاتبع [هذا الدليل](https://ragflow.io/docs/dev/build_docker_image) لإنشاء صورة Docker متوافقة مع نظامك.
> يقوم الأمر أدناه بتنزيل إصدار `v0.25.3` من الصورة RAGFlow Docker. راجع الجدول التالي للحصول على أوصاف لإصدارات RAGFlow المختلفة. لتنزيل إصدار RAGFlow مختلف عن `v0.25.3`، قم بتحديث المتغير `RAGFLOW_IMAGE` وفقًا لذلك في **docker/.env** قبل استخدام `docker compose` لبدء تشغيل الخادم.
```bash
$ cd ragflow/docker
# git checkout v0.25.3
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# This step ensures the **entrypoint.sh** file in the code matches the Docker image version.
# Use CPU for DeepDoc tasks:
$ docker compose -f docker-compose.yml up -d
# To use GPU to accelerate DeepDoc tasks:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> ملاحظة: قبل `v0.22.0`، قدمنا ​​كلتا الصورتين بنماذج embedding وصورًا رفيعة بدون نماذج embedding. التفاصيل على النحو التالي:
| RAGFlow علامة الصورة | حجم الصورة (جيجابايت) | هل لديه نماذج embedding؟ | مستقر؟ |
|-------------------|-----------------|-----------------------|----------------|
| v0.21.1 | &approx;9 | ✔️ | إصدار مستقر |
| v0.21.1-slim | &approx;2 | ❌ | إصدار مستقر |
> بدءًا من `v0.22.0`، نقوم بشحن الإصدار النحيف فقط ولم نعد نلحق اللاحقة **-slim** بعلامة الصورة.
4. التحقق من حالة الخادم بعد تشغيل الخادم:
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_النتيجة التالية تؤكد الإطلاق الناجح للنظام:_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> إذا تخطيت خطوة التأكيد هذه وقمت بتسجيل الدخول مباشرة إلى RAGFlow، فقد يعرض متصفحك تنبيه `network abnormal`
> خطأ لأنه في تلك اللحظة، قد لا تتم تهيئة RAGFlow بشكل كامل.
>
5. في متصفح الويب الخاص بك، أدخل عنوان IP الخاص بالخادم الخاص بك وقم بتسجيل الدخول إلى RAGFlow.
> باستخدام الإعدادات الافتراضية، ما عليك سوى إدخال `http://IP_OF_YOUR_MACHINE` (**من دون** رقم المنفذ) كإعداد افتراضي
> HTTP يمكن حذف منفذ العرض `80` عند استخدام التكوينات الافتراضية.
>
6. في [service_conf.yaml.template](./docker/service_conf.yaml.template)، حدد المصنع LLM المطلوب في `user_default_llm` وقم بالتحديث
الحقل `API_KEY` مع مفتاح API المقابل.
> راجع [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) لمزيد من المعلومات.
>
_العرض بدأ!_
## 🔧 التكوينات
عندما يتعلق الأمر بتكوينات النظام، ستحتاج إلى إدارة الملفات التالية:
- [.env](./docker/.env): يحتفظ بالإعدادات الأساسية للنظام، مثل `SVR_HTTP_PORT`، `MYSQL_PASSWORD`، و
`MINIO_PASSWORD`.
- [service_conf.yaml.template](./docker/service_conf.yaml.template): تكوين الخدمات الخلفية. سيتم ملء متغيرات البيئة في هذا الملف تلقائيًا عند بدء تشغيل الحاوية Docker. ستكون أي متغيرات بيئة تم تعيينها داخل حاوية Docker متاحة للاستخدام، مما يسمح لك بتخصيص سلوك الخدمة استنادًا إلى بيئة النشر.
- [docker-compose.yml](./docker/docker-compose.yml): يعتمد النظام على [docker-compose.yml](./docker/docker-compose.yml) لبدء التشغيل.
> يوفر الملف [./docker/README](./docker/README.md) وصفًا تفصيليًا لإعدادات البيئة والخدمة
> التكوينات التي يمكن استخدامها كـ `${ENV_VARS}` في ملف [service_conf.yaml.template](./docker/service_conf.yaml.template).
لتحديث منفذ العرض الافتراضي HTTP (80)، انتقل إلى [docker-compose.yml](./docker/docker-compose.yml) وقم بتغيير `80:80`
إلى `<YOUR_SERVING_PORT>:80`.
تتطلب تحديثات التكوينات المذكورة أعلاه إعادة تشغيل جميع الحاويات لتصبح سارية المفعول:
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### تبديل محرك المستندات من Elasticsearch إلى Infinity
RAGFlow يستخدم Elasticsearch بشكل افتراضي لتخزين النص الكامل والمتجهات. للتبديل إلى [Infinity](https://github.com/infiniflow/infinity/)، اتبع الخطوات التالية:
1. إيقاف كافة الحاويات قيد التشغيل:
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` سوف يحذف docker وحدات تخزين الحاوية، وسيتم مسح البيانات الموجودة.
2. اضبط `DOC_ENGINE` في **docker/.env** على `infinity`.
3. ابدأ الحاويات:
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> التبديل إلى Infinity على جهاز Linux/arm64 غير مدعوم رسميًا بعد.
## 🔧 أنشئ صورة Docker
يبلغ حجم هذه الصورة حوالي 2 غيغابايت وتعتمد على خدمات LLM وembedding الخارجية.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
أو إذا كنت خلف وكيل، فيمكنك تمرير وسيطات الوكيل:
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://YOUR_PROXY:PORT \
--build-arg https_proxy=http://YOUR_PROXY:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 إطلاق الخدمة من المصدر للتطوير
1. قم بتثبيت `uv` و`pre-commit`، أو قم بتخطي هذه الخطوة إذا كانا مثبتين بالفعل:
```bash
pipx install uv pre-commit
```
2. استنساخ الكود المصدري وتثبيت تبعيات بايثون:
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. قم بتشغيل الخدمات التابعة (MinIO وElasticsearch وRedis وMySQL) باستخدام Docker Compose:
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
أضف السطر التالي إلى `/etc/hosts` لحل كافة المضيفين المحددين في **docker/.env** إلى `127.0.0.1`:
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. إذا لم تتمكن من الوصول إلى HuggingFace، فقم بتعيين متغير البيئة `HF_ENDPOINT` لاستخدام موقع مرآة:
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. إذا كان نظام التشغيل لديك لا يحتوي على jemalloc، فيرجى تثبيته على النحو التالي:
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. إطلاق الخدمة الخلفية:
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. تثبيت تبعيات الواجهة الأمامية:
```bash
cd web
npm install
```
8. إطلاق خدمة الواجهة الأمامية:
```bash
npm run dev
```
_النتيجة التالية تؤكد الإطلاق الناجح للنظام:_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. أوقف خدمة الواجهة الأمامية والخلفية RAGFlow بعد اكتمال التطوير:
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 التوثيق
- [البدء السريع](https://ragflow.io/docs/dev/)
- [التكوين](https://ragflow.io/docs/dev/configurations)
- [ملاحظات الإصدار](https://ragflow.io/docs/dev/release_notes)
- [أدلة المستخدم](https://ragflow.io/docs/category/user-guides)
- [أدلة المطورين](https://ragflow.io/docs/category/developer-guides)
- [المراجع](https://ragflow.io/docs/dev/category/references)
- [الأسئلة الشائعة](https://ragflow.io/docs/dev/faq)
## 📜 Roadmap
راجع [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 المجتمع
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [مناقشات جيثب](https://github.com/orgs/infiniflow/discussions)
## 🙌 المساهمة
RAGFlow يزدهر من خلال التعاون مفتوح المصدر. وبهذه الروح، فإننا نحتضن المساهمات المتنوعة من المجتمع.
إذا كنت ترغب في أن تكون جزءًا، فراجع [إرشادات المساهمة](https://ragflow.io/docs/dev/contributing) أولاً.

405
README_fr.md Normal file
View File

@@ -0,0 +1,405 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DBEDFA"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="suivre sur X(Twitter)">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Badge statique" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Dernière%20version" alt="Dernière version">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/License-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="licence">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Documentation</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 Table des matières</b></summary>
- 💡 [Qu'est-ce que RAGFlow?](#-quest-ce-que-ragflow)
- 🎮 [Démarrage](#-démarrage)
- 📌 [Dernières mises à jour](#-dernières-mises-à-jour)
- 🌟 [Fonctionnalités clés](#-fonctionnalités-clés)
- 🔎 [Architecture du système](#-architecture-du-système)
- 🎬 [Auto-hébergement](#-auto-hébergement)
- 🔧 [Configurations](#-configurations)
- 🔧 [Construire une image Docker](#-construire-une-image-docker)
- 🔨 [Lancer le service depuis les sources pour le développement](#-lancer-le-service-depuis-les-sources-pour-le-développement)
- 📚 [Documentation](#-documentation)
- 📜 [Roadmap](#-feuille-de-route)
- 🏄 [Communauté](#-communauté)
- 🙌 [Contribuer](#-contribuer)
</details>
## 💡 Qu'est-ce que RAGFlow?
[RAGFlow](https://ragflow.io/) est un moteur de [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source de premier plan qui fusionne les technologies RAG de pointe avec des capacités Agent pour créer une couche de contexte supérieure pour les LLM. Il offre un flux de travail RAG rationalisé, adaptable aux entreprises de toute taille. Alimenté par un [moteur de contexte](https://ragflow.io/basics/what-is-agent-context-engine) convergent et des modèles d'agents préconstruits, RAGFlow permet aux développeurs de transformer des données complexes en systèmes d'IA haute-fidélité, prêts pour la production, avec une efficacité et une précision exceptionnelles.
## 🎮 Démarrage
Essayez notre service cloud sur [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 Dernières mises à jour
- 24-04-2026 Prise en charge de DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Fournit un skill officiel pour accéder aux datasets RAGFlow via OpenClaw.
- 26-12-2025 Prise en charge de la « Mémoire » pour l'agent IA.
- 19-11-2025 Prise en charge de Gemini 3 Pro.
- 12-11-2025 Prise en charge de la synchronisation de données depuis Confluence, S3, Notion, Discord et Google Drive.
- 23-10-2025 Prise en charge de MinerU & Docling comme méthodes d'analyse de documents.
- 15-10-2025 Prise en charge du pipeline d'ingestion orchestrable.
- 08-08-2025 Prise en charge des derniers modèles de la série GPT-5 d'OpenAI.
- 01-08-2025 Prise en charge du flux de travail agentique et de MCP.
- 23-05-2025 Ajout d'un composant exécuteur de code Python/JavaScript à l'Agent.
- 05-05-2025 Prise en charge des requêtes inter-langues.
- 19-03-2025 Prise en charge de l'utilisation d'un modèle multi-modal pour analyser les images dans les fichiers PDF ou DOCX.
## 🎉 Restez informé
⭐️ Mettez une étoile à notre dépôt pour rester informé des nouvelles fonctionnalités et améliorations passionnantes ! Recevez des notifications instantanées pour les nouvelles versions ! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 Fonctionnalités clés
### 🍭 **"Quality in, quality out"**
- Extraction de connaissances basée sur la [compréhension approfondie des documents](./deepdoc/README.md) à partir de données non structurées aux formats complexes.
- Trouve "l'aiguille dans la meule de données" de tokens littéralement illimités.
### 🍱 **Découpage(Chunking) basé sur des templates**
- Intelligent et explicable.
- De nombreuses options de templates disponibles.
### 🌱 **Citations fondées avec réduction des hallucinations**
- Visualisation du découpage de texte pour permettre une intervention humaine.
- Aperçu rapide des références clés et citations traçables pour soutenir des réponses fondées.
### 🍔 **Compatibilité avec des sources de données hétérogènes**
- Prend en charge Word, présentations, Excel, txt, images, copies numérisées, données structurées, pages web, et plus encore.
### 🛀 **Flux de travail RAG automatisé et sans effort**
- Orchestration RAG rationalisée adaptée aux particuliers comme aux grandes entreprises.
- LLM et modèles d'embedding configurables.
- Rappel multiple associé à un ré-classement fusionné.
- APIs intuitives pour une intégration transparente avec les entreprises.
## 🔎 Architecture du système
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Auto-hébergement
### 📝 Prérequis
- CPU >= 4 cœurs
- RAM >= 16 Go
- Disque >= 50 Go
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- [gVisor](https://gvisor.dev/docs/user_guide/install/) : Requis uniquement si vous souhaitez utiliser la fonctionnalité d'exécuteur de code (sandbox) de RAGFlow.
> [!TIP]
> Si vous n'avez pas installé Docker sur votre machine locale (Windows, Mac ou Linux), consultez [Installer Docker Engine](https://docs.docker.com/engine/install/).
### 🚀 Démarrer le serveur
1. Assurez-vous que `vm.max_map_count` >= 262144 :
> Pour vérifier la valeur de `vm.max_map_count` :
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> Réinitialisez `vm.max_map_count` à une valeur d'au moins 262144 si ce n'est pas le cas.
>
> ```bash
> # Dans ce cas, nous le définissons à 262144 :
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> Ce changement sera réinitialisé après un redémarrage du système. Pour que votre modification reste permanente, ajoutez ou mettez à jour la valeur `vm.max_map_count` dans **/etc/sysctl.conf** :
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. Clonez le dépôt :
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. Démarrez le serveur en utilisant les images Docker préconstruites :
> [!CAUTION]
> Toutes les images Docker sont construites pour les plateformes x86. Nous ne proposons pas actuellement d'images Docker pour ARM64.
> Si vous êtes sur une plateforme ARM64, suivez [ce guide](https://ragflow.io/docs/dev/build_docker_image) pour construire une image Docker compatible avec votre système.
> La commande ci-dessous télécharge l'édition `v0.25.3` de l'image Docker RAGFlow. Consultez le tableau suivant pour les descriptions des différentes éditions de RAGFlow. Pour télécharger une édition de RAGFlow différente de `v0.25.3`, mettez à jour la variable `RAGFLOW_IMAGE` dans **docker/.env** avant d'utiliser `docker compose` pour démarrer le serveur.
```bash
$ cd ragflow/docker
# git checkout v0.25.3
# Optionnel : utiliser un tag stable (voir les versions : https://github.com/infiniflow/ragflow/releases)
# Cette étape garantit que le fichier **entrypoint.sh** dans le code correspond à la version de l'image Docker.
# Use CPU for DeepDoc tasks:
$ docker compose -f docker-compose.yml up -d
# To use GPU to accelerate DeepDoc tasks:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> Remarque : Avant `v0.22.0`, nous fournissions à la fois des images avec des modèles d'embedding et des images slim sans modèles d'embedding. Détails ci-dessous :
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
|-------------------|-----------------|-----------------------|----------------|
| v0.21.1 | &approx;9 | ✔️ | Stable release |
| v0.21.1-slim | &approx;2 | ❌ | Stable release |
> À partir de `v0.22.0`, nous ne distribuons que l'édition slim et ne rajoutons plus le suffixe **-slim** au tag d'image.
4. Vérifiez l'état du serveur après son démarrage :
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_La sortie suivante confirme un lancement réussi du système :_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> Si vous sautez cette étape de confirmation et vous connectez directement à RAGFlow, votre navigateur peut afficher une erreur `network abnormal`, car à ce moment-là, votre RAGFlow peut ne pas être entièrement initialisé.
>
5. Dans votre navigateur web, entrez l'adresse IP de votre serveur et connectez-vous à RAGFlow.
> Avec les paramètres par défaut, il vous suffit d'entrer `http://IP_OF_YOUR_MACHINE` (**sans** numéro de port), car le port HTTP par défaut `80` peut être omis lors de l'utilisation des configurations par défaut.
>
6. Dans [service_conf.yaml.template](./docker/service_conf.yaml.template), sélectionnez la fabrique LLM souhaitée dans `user_default_llm` et mettez à jour le champ `API_KEY` avec la clé API correspondante.
> Voir [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) pour plus d'informations.
>
_Le spectacle commence !_
## 🔧 Configurations
En ce qui concerne les configurations système, vous devrez gérer les fichiers suivants :
- [.env](./docker/.env) : Conserve les paramètres de base du système, tels que `SVR_HTTP_PORT`, `MYSQL_PASSWORD` et `MINIO_PASSWORD`.
- [service_conf.yaml.template](./docker/service_conf.yaml.template) : Configure les services back-end. Les variables d'environnement dans ce fichier seront automatiquement renseignées au démarrage du conteneur Docker. Toutes les variables d'environnement définies dans le conteneur Docker seront disponibles, vous permettant de personnaliser le comportement du service en fonction de l'environnement de déploiement.
- [docker-compose.yml](./docker/docker-compose.yml) : Le système s'appuie sur [docker-compose.yml](./docker/docker-compose.yml) pour démarrer.
> Le fichier [./docker/README](./docker/README.md) fournit une description détaillée des paramètres d'environnement et des configurations de services qui peuvent être utilisés comme `${ENV_VARS}` dans le fichier [service_conf.yaml.template](./docker/service_conf.yaml.template).
Pour mettre à jour le port HTTP de service par défaut (80), accédez à [docker-compose.yml](./docker/docker-compose.yml) et changez `80:80` en `<YOUR_SERVING_PORT>:80`.
Les mises à jour des configurations ci-dessus nécessitent un redémarrage de tous les conteneurs pour prendre effet :
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### Passer du moteur de documents Elasticsearch à Infinity
RAGFlow utilise Elasticsearch par défaut pour stocker le texte intégral et les vecteurs. Pour passer à [Infinity](https://github.com/infiniflow/infinity/), suivez ces étapes :
1. Arrêtez tous les conteneurs en cours d'exécution :
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` supprimera les volumes des conteneurs Docker, et les données existantes seront effacées.
2. Définissez `DOC_ENGINE` dans **docker/.env** sur `infinity`.
3. Démarrez les conteneurs :
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> Le passage à Infinity sur une machine Linux/arm64 n'est pas encore officiellement pris en charge.
## 🔧 Construire une image Docker
Cette image fait environ 2 Go et dépend de services LLM et d'embedding externes.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
Ou si vous êtes derrière un proxy, vous pouvez passer des arguments de proxy :
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://YOUR_PROXY:PORT \
--build-arg https_proxy=http://YOUR_PROXY:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 Lancer le service depuis les sources pour le développement
1. Installez `uv` et `pre-commit`, ou ignorez cette étape s'ils sont déjà installés :
```bash
pipx install uv pre-commit
```
2. Clonez le code source et installez les dépendances Python :
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. Lancez les services dépendants (MinIO, Elasticsearch, Redis et MySQL) avec Docker Compose :
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
Ajoutez la ligne suivante à `/etc/hosts` pour résoudre tous les hôtes spécifiés dans **docker/.env** vers `127.0.0.1` :
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. Si vous ne pouvez pas accéder à HuggingFace, définissez la variable d'environnement `HF_ENDPOINT` pour utiliser un site miroir :
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. Si votre système d'exploitation n'a pas jemalloc, installez-le comme suit :
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. Lancez le service back-end :
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. Installez les dépendances front-end :
```bash
cd web
npm install
```
8. Lancez le service front-end :
```bash
npm run dev
```
_La sortie suivante confirme un lancement réussi du système :_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. Arrêtez les services front-end et back-end de RAGFlow une fois le développement terminé :
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 Documentation
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
## 📜 Roadmap
Voir la [Feuille de route RAGFlow 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 Communauté
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contribuer
RAGFlow s'épanouit grâce à la collaboration open-source. Dans cet esprit, nous accueillons des contributions diverses de la communauté.
Si vous souhaitez en faire partie, consultez d'abord nos [Directives de contribution](https://ragflow.io/docs/dev/contributing).

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="Logo ragflow">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體中文版自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DBEDFA"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="Ikuti di X (Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Lencana Daring" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Lencana Daring" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Rilis%20Terbaru" alt="Rilis Terbaru">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Dokumentasi</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Peta Jalan</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Daftar Isi </b> </summary>
- 💡 [Apa Itu RAGFlow?](#-apa-itu-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Mulai](#-mulai)
- 📌 [Pembaruan Terbaru](#-pembaruan-terbaru)
- 🌟 [Fitur Utama](#-fitur-utama)
- 🔎 [Arsitektur Sistem](#-arsitektur-sistem)
- 🎬 [Mulai](#-mulai)
- 🎬 [Pengelolaan Mandiri](#-pengelolaan-mandiri)
- 🔧 [Konfigurasi](#-konfigurasi)
- 🔧 [Membangun Image Docker](#-membangun-docker-image)
- 🔨 [Meluncurkan aplikasi dari Sumber untuk Pengembangan](#-meluncurkan-aplikasi-dari-sumber-untuk-pengembangan)
@@ -72,11 +74,11 @@
## 💡 Apa Itu RAGFlow?
[RAGFlow](https://ragflow.io/) adalah mesin RAG (Retrieval-Augmented Generation) open-source terkemuka yang mengintegrasikan teknologi RAG mutakhir dengan kemampuan Agent untuk menciptakan lapisan kontekstual superior bagi LLM. Menyediakan alur kerja RAG yang efisien dan dapat diadaptasi untuk perusahaan segala skala. Didukung oleh mesin konteks terkonvergensi dan template Agent yang telah dipra-bangun, RAGFlow memungkinkan pengembang mengubah data kompleks menjadi sistem AI kesetiaan-tinggi dan siap-produksi dengan efisiensi dan presisi yang luar biasa.
[RAGFlow](https://ragflow.io/) adalah mesin [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source terkemuka yang mengintegrasikan teknologi RAG mutakhir dengan kemampuan Agent untuk menciptakan lapisan kontekstual superior bagi LLM. Menyediakan alur kerja RAG yang efisien dan dapat diadaptasi untuk perusahaan segala skala. Didukung oleh mesin konteks terkonvergensi dan template Agent yang telah dipra-bangun, RAGFlow memungkinkan pengembang mengubah data kompleks menjadi sistem AI kesetiaan-tinggi dan siap-produksi dengan efisiensi dan presisi yang luar biasa.
## 🎮 Demo
## 🎮 Mulai
Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
Coba layanan cloud kami di [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Pembaruan Terbaru
- 2026-04-24 Mendukung DeepSeek v4.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Menyediakan skill resmi untuk mengakses dataset RAGFlow melalui OpenClaw.
- 2025-12-26 Mendukung 'Memori' untuk agen AI.
- 2025-11-19 Mendukung Gemini 3 Pro.
- 2025-11-12 Mendukung sinkronisasi data dari Confluence, S3, Notion, Discord, Google Drive.
@@ -140,7 +144,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Mulai
## 🎬 Pengelolaan Mandiri
### 📝 Prasyarat
@@ -188,12 +192,12 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
> Semua gambar Docker dibangun untuk platform x86. Saat ini, kami tidak menawarkan gambar Docker untuk ARM64.
> Jika Anda menggunakan platform ARM64, [silakan gunakan panduan ini untuk membangun gambar Docker yang kompatibel dengan sistem Anda](https://ragflow.io/docs/dev/build_docker_image).
> Perintah di bawah ini mengunduh edisi v0.23.1 dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.23.1, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server.
> Perintah di bawah ini mengunduh edisi v0.25.3 dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.25.3, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server.
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# Opsional: gunakan tag stabil (lihat releases: https://github.com/infiniflow/ragflow/releases)
# This steps ensures the **entrypoint.sh** file in the code matches the Docker image version.
@@ -299,7 +303,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
3. Jalankan aplikasi yang diperlukan (MinIO, Elasticsearch, Redis, dan MySQL) menggunakan Docker Compose:
@@ -361,8 +365,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -373,7 +377,7 @@ Lihat [Roadmap RAGFlow 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 Komunitas
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Kontribusi

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體中文版自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DBEDFA"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -53,11 +55,11 @@
## 💡 RAGFlow とは?
[RAGFlow](https://ragflow.io/) は、先進的なRAGRetrieval-Augmented Generation技術と Agent 機能を融合し、大規模言語モデルLLMに優れたコンテキスト層を構築する最先端のオープンソース RAG エンジンです。あらゆる規模の企業に対応可能な合理化された RAG ワークフローを提供し、統合型コンテキストエンジンと事前構築されたAgentテンプレートにより、開発者が複雑なデータを驚異的な効率性と精度で高精細なプロダクションレディAIシステムへ変換することを可能にします。
[RAGFlow](https://ragflow.io/) は、先進的な[RAG](https://ragflow.io/basics/what-is-rag)Retrieval-Augmented Generation技術と Agent 機能を融合し、大規模言語モデルLLMに優れたコンテキスト層を構築する最先端のオープンソース RAG エンジンです。あらゆる規模の企業に対応可能な合理化された RAG ワークフローを提供し、統合型[コンテキストエンジン](https://ragflow.io/basics/what-is-agent-context-engine)と事前構築されたAgentテンプレートにより、開発者が複雑なデータを驚異的な効率性と精度で高精細なプロダクションレディAIシステムへ変換することを可能にします。
## 🎮 Demo
## 🎮 はじめに
デモをお試しください:[https://demo.ragflow.io](https://demo.ragflow.io)。
当社のクラウドサービスをぜひお試しください:[https://cloud.ragflow.io](https://cloud.ragflow.io)。
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -66,6 +68,8 @@
## 🔥 最新情報
- 2026-04-24 DeepSeek v4 をサポート。
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw経由でRAGFlowデータセットにアクセスする公式スキルを提供。
- 2025-12-26 AIエージェントの「メモリ」機能をサポート。
- 2025-11-19 Gemini 3 Proをサポートしています。
- 2025-11-12 Confluence、S3、Notion、Discord、Google Drive からのデータ同期をサポートします。
@@ -121,7 +125,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 初期設定
## 🎬 セルフホスティング
### 📝 必要条件
@@ -168,12 +172,12 @@
> 現在、公式に提供されているすべての Docker イメージは x86 アーキテクチャ向けにビルドされており、ARM64 用の Docker イメージは提供されていません。
> ARM64 アーキテクチャのオペレーティングシステムを使用している場合は、[このドキュメント](https://ragflow.io/docs/dev/build_docker_image)を参照して Docker イメージを自分でビルドしてください。
> 以下のコマンドは、RAGFlow Docker イメージの v0.23.1 エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.23.1 とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。
> 以下のコマンドは、RAGFlow Docker イメージの v0.25.3 エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.25.3 とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# 任意: 安定版タグを利用 (一覧: https://github.com/infiniflow/ragflow/releases)
# この手順は、コード内の entrypoint.sh ファイルが Docker イメージのバージョンと一致していることを確認します。
@@ -194,8 +198,8 @@
> `v0.22.0` 以降、当プロジェクトでは slim エディションのみを提供し、イメージタグに **-slim** サフィックスを付けなくなりました。
1. サーバーを立ち上げた後、サーバーの状態を確認する:
1. サーバーを立ち上げた後、サーバーの状態を確認する:
```bash
$ docker logs -f docker-ragflow-cpu-1
```
@@ -299,7 +303,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
3. Docker Compose を使用して依存サービスMinIO、Elasticsearch、Redis、MySQLを起動する:
@@ -361,8 +365,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -373,7 +377,7 @@ docker build --platform linux/amd64 \
## 🏄 コミュニティ
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 コントリビュート

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DBEDFA"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -54,11 +56,11 @@
## 💡 RAGFlow란?
[RAGFlow](https://ragflow.io/) 는 최첨단 RAG(Retrieval-Augmented Generation)와 Agent 기능을 융합하여 대규모 언어 모델(LLM)을 위한 우수한 컨텍스트 계층을 생성하는 선도적인 오픈소스 RAG 엔진입니다. 모든 규모의 기업에 적용 가능한 효율적인 RAG 워크플로를 제공하며, 통합 컨텍스트 엔진과 사전 구축된 Agent 템플릿을 통해 개발자들이 복잡한 데이터를 예외적인 효율성과 정밀도로 고급 구현도의 프로덕션 준비 완료 AI 시스템으로 변환할 수 있도록 지원합니다.
[RAGFlow](https://ragflow.io/) 는 최첨단 [RAG](https://ragflow.io/basics/what-is-rag)(Retrieval-Augmented Generation)와 Agent 기능을 융합하여 대규모 언어 모델(LLM)을 위한 우수한 컨텍스트 계층을 생성하는 선도적인 오픈소스 RAG 엔진입니다. 모든 규모의 기업에 적용 가능한 효율적인 RAG 워크플로를 제공하며, 통합 [컨텍스트 엔진](https://ragflow.io/basics/what-is-agent-context-engine)과 사전 구축된 Agent 템플릿을 통해 개발자들이 복잡한 데이터를 예외적인 효율성과 정밀도로 고급 구현도의 프로덕션 준비 완료 AI 시스템으로 변환할 수 있도록 지원합니다.
## 🎮 데모
## 🎮 시작하기
데모를 [https://demo.ragflow.io](https://demo.ragflow.io)에서 실행해 보세요.
[https://cloud.ragflow.io](https://cloud.ragflow.io)에서 저희 클라우드 서비스를 이용해 보세요.
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -67,6 +69,8 @@
## 🔥 업데이트
- 2026-04-24 DeepSeek v4를 지원합니다.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw를 통해 RAGFlow 데이터셋에 접근하는 공식 스킬 제공.
- 2025-12-26 AI 에이전트의 '메모리' 기능 지원.
- 2025-11-19 Gemini 3 Pro를 지원합니다.
- 2025-11-12 Confluence, S3, Notion, Discord, Google Drive에서 데이터 동기화를 지원합니다.
@@ -122,7 +126,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 시작하기
## 🎬 자체 호스팅
### 📝 사전 준비 사항
@@ -170,12 +174,12 @@
> 모든 Docker 이미지는 x86 플랫폼을 위해 빌드되었습니다. 우리는 현재 ARM64 플랫폼을 위한 Docker 이미지를 제공하지 않습니다.
> ARM64 플랫폼을 사용 중이라면, [시스템과 호환되는 Docker 이미지를 빌드하려면 이 가이드를 사용해 주세요](https://ragflow.io/docs/dev/build_docker_image).
> 아래 명령어는 RAGFlow Docker 이미지의 v0.23.1 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.23.1과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오.
> 아래 명령어는 RAGFlow Docker 이미지의 v0.25.3 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.25.3와 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오.
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# 이 단계는 코드의 entrypoint.sh 파일이 Docker 이미지 버전과 일치하도록 보장합니다.
@@ -294,7 +298,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
@@ -365,8 +369,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -377,7 +381,7 @@ docker build --platform linux/amd64 \
## 🏄 커뮤니티
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 컨트리뷰션

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DBEDFA"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="seguir no X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Badge Estático" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Badge Estático" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Última%20Relese" alt="Última Versão">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Documentação</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Índice</b></summary>
- 💡 [O que é o RAGFlow?](#-o-que-é-o-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Primeiros Passos](#-primeiros-passos)
- 📌 [Últimas Atualizações](#-últimas-atualizações)
- 🌟 [Principais Funcionalidades](#-principais-funcionalidades)
- 🔎 [Arquitetura do Sistema](#-arquitetura-do-sistema)
- 🎬 [Primeiros Passos](#-primeiros-passos)
- 🎬 [Auto-hospedagem](#-auto-hospedagem)
- 🔧 [Configurações](#-configurações)
- 🔧 [Construir uma imagem docker sem incorporar modelos](#-construir-uma-imagem-docker-sem-incorporar-modelos)
- 🔧 [Construir uma imagem docker incluindo modelos](#-construir-uma-imagem-docker-incluindo-modelos)
@@ -73,11 +75,11 @@
## 💡 O que é o RAGFlow?
[RAGFlow](https://ragflow.io/) é um mecanismo de RAG (Retrieval-Augmented Generation) open-source líder que fusiona tecnologias RAG de ponta com funcionalidades Agent para criar uma camada contextual superior para LLMs. Oferece um fluxo de trabalho RAG otimizado adaptável a empresas de qualquer escala. Alimentado por um motor de contexto convergente e modelos Agent pré-construídos, o RAGFlow permite que desenvolvedores transformem dados complexos em sistemas de IA de alta fidelidade e pronto para produção com excepcional eficiência e precisão.
[RAGFlow](https://ragflow.io/) é um mecanismo de [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source líder que fusiona tecnologias RAG de ponta com funcionalidades Agent para criar uma camada contextual superior para LLMs. Oferece um fluxo de trabalho RAG otimizado adaptável a empresas de qualquer escala. Alimentado por [um motor de contexto](https://ragflow.io/basics/what-is-agent-context-engine) convergente e modelos Agent pré-construídos, o RAGFlow permite que desenvolvedores transformem dados complexos em sistemas de IA de alta fidelidade e pronto para produção com excepcional eficiência e precisão.
## 🎮 Demo
## 🎮 Primeiros Passos
Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
Experimente o nosso serviço na nuvem em [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -86,6 +88,8 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Últimas Atualizações
- 24-04-2026 Suporta DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Fornece um skill oficial para acessar datasets do RAGFlow via OpenClaw.
- 26-12-2025 Suporte à função 'Memória' para agentes de IA.
- 19-11-2025 Suporta Gemini 3 Pro.
- 12-11-2025 Suporta a sincronização de dados do Confluence, S3, Notion, Discord e Google Drive.
@@ -141,7 +145,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Primeiros Passos
## 🎬 Auto-hospedagem
### 📝 Pré-requisitos
@@ -188,12 +192,12 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
> Todas as imagens Docker são construídas para plataformas x86. Atualmente, não oferecemos imagens Docker para ARM64.
> Se você estiver usando uma plataforma ARM64, por favor, utilize [este guia](https://ragflow.io/docs/dev/build_docker_image) para construir uma imagem Docker compatível com o seu sistema.
> O comando abaixo baixa a edição`v0.23.1` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.23.1`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor.
> O comando abaixo baixa a edição`v0.25.3` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.25.3`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor.
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# Opcional: use uma tag estável (veja releases: https://github.com/infiniflow/ragflow/releases)
# Esta etapa garante que o arquivo entrypoint.sh no código corresponda à versão da imagem do Docker.
@@ -316,7 +320,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # instala os módulos Python dependentes do RAGFlow
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
3. Inicie os serviços dependentes (MinIO, Elasticsearch, Redis e MySQL) usando Docker Compose:
@@ -378,8 +382,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -390,7 +394,7 @@ Veja o [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/12241
## 🏄 Comunidade
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contribuindo

409
README_tr.md Normal file
View File

@@ -0,0 +1,409 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DBEDFA"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="X(Twitter)'da takip et">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Çevrimiçi Demo" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Son%20Sürüm" alt="Son Sürüm">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/Lisans-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="lisans">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Dokümantasyon</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Yol Haritası</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 İçindekiler</b></summary>
- 💡 [RAGFlow Nedir?](#-ragflow-nedir)
- 🎮 [Başlarken](#-başlarken)
- 📌 [Son Güncellemeler](#-son-güncellemeler)
- 🌟 [Temel Özellikler](#-temel-özellikler)
- 🔎 [Sistem Mimarisi](#-sistem-mimarisi)
- 🎬 [Kendi Sunucusunda Barındırma](#-kendi-sunucusunda-barındırma)
- 🔧 [Yapılandırmalar](#-yapılandırmalar)
- 🔧 [Docker İmajı Oluşturma](#-docker-i̇majı-oluşturma)
- 🔨 [Geliştirme İçin Kaynaktan Hizmet Başlatma](#-geliştirme-i̇çin-kaynaktan-hizmet-başlatma)
- 📚 [Dokümantasyon](#-dokümantasyon)
- 📜 [Yol Haritası](#-yol-haritası)
- 🏄 [Topluluk](#-topluluk)
- 🙌 [Katkıda Bulunma](#-katkıda-bulunma)
</details>
## 💡 RAGFlow Nedir?
[RAGFlow](https://ragflow.io/), derin doküman anlayışına dayalı, açık kaynaklı ve öncü bir Artırılmış Üretim ile Bilgi Erişimi ([RAG](https://ragflow.io/basics/what-is-rag)) motorudur. En son RAG teknolojisini Ajan yetenekleriyle birleştirerek LLM'ler için üstün bir bağlam katmanı oluşturur. Her ölçekteki kuruluşa uyarlanabilir, kolaylaştırılmış bir RAG iş akışı sunar. Yakınsanmış bir [bağlam motoru](https://ragflow.io/basics/what-is-agent-context-engine) ve hazır ajan şablonlarıyla donatılmış RAGFlow, geliştiricilerin karmaşık verileri yüksek doğrulukta, üretime hazır yapay zeka sistemlerine olağanüstü verimlilik ve hassasiyetle dönüştürmesini sağlar.
## 🎮 Başlarken
Bulut hizmetimizi [https://cloud.ragflow.io](https://cloud.ragflow.io) adresinden deneyin.
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 Son Güncellemeler
- 2026-04-24 DeepSeek v4 desteği.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw üzerinden RAGFlow veri setlerine erişmek için resmi bir skill sağlar.
- 2025-12-26 Yapay zeka ajanı için 'Bellek' desteği eklendi.
- 2025-11-19 Gemini 3 Pro desteği eklendi.
- 2025-11-12 Confluence, S3, Notion, Discord, Google Drive'dan veri senkronizasyonu desteği eklendi.
- 2025-10-23 Doküman ayrıştırma yöntemi olarak MinerU ve Docling desteği eklendi.
- 2025-10-15 Düzenlenebilir veri alım hattı desteği eklendi.
- 2025-08-08 OpenAI'ın en yeni GPT-5 serisi modelleri için destek eklendi.
- 2025-08-01 Ajanlı iş akışı ve MCP desteği eklendi.
- 2025-05-23 Ajana Python/JavaScript kod çalıştırıcı bileşeni eklendi.
- 2025-05-05 Diller arası sorgu desteği eklendi.
- 2025-03-19 PDF veya DOCX dosyalarındaki görselleri yorumlamak için çok modlu model desteği eklendi.
## 🎉 Bizi Takip Edin
⭐️ Heyecan verici yeni özellikler ve iyileştirmelerden haberdar olmak için depomuzı yıldızlayın! Yeni sürümler için anında bildirim alın! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 Temel Özellikler
### 🍭 **"Kaliteli girdi, kaliteli çıktı"**
- Karmaşık formatlara sahip yapılandırılmamış verilerden [derin doküman anlayışı](./deepdoc/README.md) tabanlı bilgi çıkarımı.
- Kelimenin tam anlamıyla sınırsız token içinde "samanlıkta iğne bulma" yeteneği.
### 🍱 **Şablon tabanlı parçalama**
- Akıllı ve açıklanabilir.
- Aralarından seçim yapabileceğiniz çok sayıda şablon seçeneği.
### 🌱 **Azaltılmış halüsinasyonlarla temellendirilmiş alıntılar**
- İnsan müdahalesine olanak tanıyan metin parçalama görselleştirmesi.
- Temellendirilmiş yanıtları desteklemek için anahtar referansların hızlı görüntülenmesi ve izlenebilir alıntılar.
### 🍔 **Heterojen veri kaynaklarıyla uyumluluk**
- Word, slaytlar, Excel, txt, görseller, taranmış kopyalar, yapılandırılmış veriler, web sayfaları ve daha fazlasını destekler.
### 🛀 **Otomatik ve zahmetsiz RAG iş akışı**
- Hem bireysel hem de büyük işletmeler için özelleştirilmiş kolaylaştırılmış RAG düzenlemesi.
- Yapılandırılabilir LLM'ler ve gömme (embedding) modelleri.
- Birleştirilmiş yeniden sıralama ile çoklu geri çağırma.
- İş süreçlerine sorunsuz entegrasyon için sezgisel API'ler.
## 🔎 Sistem Mimarisi
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Kendi Sunucusunda Barındırma
### 📝 Ön Koşullar
- CPU >= 4 çekirdek
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- [gVisor](https://gvisor.dev/docs/user_guide/install/): Yalnızca RAGFlow'un kod çalıştırıcı (sandbox) özelliğini kullanmayı planlıyorsanız gereklidir.
> [!TIP]
> Yerel makinenize (Windows, Mac veya Linux) Docker yüklemediyseniz, [Docker Engine Kurulumu](https://docs.docker.com/engine/install/) sayfasına bakın.
### 🚀 Sunucuyu Başlatma
1. `vm.max_map_count` değerinin >= 262144 olduğundan emin olun:
> `vm.max_map_count` değerini kontrol etmek için:
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> Değer 262144'ten düşükse, en az 262144 olarak ayarlayın.
>
> ```bash
> # Bu örnekte 262144 olarak ayarlıyoruz:
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> Bu değişiklik sistem yeniden başlatıldığında sıfırlanacaktır. Değişikliğin kalıcı olmasını sağlamak için
> **/etc/sysctl.conf** dosyasındaki `vm.max_map_count` değerini buna göre ekleyin veya güncelleyin:
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. Depoyu klonlayın:
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. Önceden oluşturulmuş Docker imajlarını kullanarak sunucuyu başlatın:
> [!CAUTION]
> Tüm Docker imajları x86 platformları için oluşturulmuştur. Şu anda ARM64 için Docker imajı sunmuyoruz.
> ARM64 platformundaysanız, sisteminizle uyumlu bir Docker imajı oluşturmak için [bu kılavuzu](https://ragflow.io/docs/dev/build_docker_image) takip edin.
> Aşağıdaki komut RAGFlow Docker imajının `v0.25.3` sürümünü indirir. Farklı RAGFlow sürümleri için aşağıdaki tabloya bakın. `v0.25.3` dışında bir sürüm indirmek için, `docker compose` ile sunucuyu başlatmadan önce **docker/.env** dosyasındaki `RAGFLOW_IMAGE` değişkenini güncelleyin.
```bash
$ cd ragflow/docker
# git checkout v0.25.3
# İsteğe bağlı: Kararlı bir etiket kullanın (sürümler: https://github.com/infiniflow/ragflow/releases)
# Bu adım, koddaki **entrypoint.sh** dosyasının Docker imaj sürümüyle eşleşmesini sağlar.
# DeepDoc görevleri için CPU kullanımı:
$ docker compose -f docker-compose.yml up -d
# DeepDoc görevlerini hızlandırmak için GPU kullanımı:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> Not: `v0.22.0` öncesinde hem gömme modelleri içeren imajlar hem de gömme modelleri içermeyen ince (slim) imajlar sunuyorduk. Detaylar aşağıdadır:
| RAGFlow imaj etiketi | İmaj boyutu (GB) | Gömme modelleri var mı? | Kararlı mı? |
|-----------------------|-------------------|-------------------------|-----------------|
| v0.21.1 | &approx;9 | ✔️ | Kararlı sürüm |
| v0.21.1-slim | &approx;2 | ❌ | Kararlı sürüm |
> `v0.22.0`'dan itibaren yalnızca ince (slim) sürümü sunuyoruz ve imaj etiketine artık **-slim** son eki eklemiyoruz.
4. Sunucu çalışır duruma geldikten sonra sunucu durumunu kontrol edin:
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_Aşağıdaki çıktı, sistemin başarıyla başlatıldığını onaylar:_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> Bu onay adımını atlayıp doğrudan RAGFlow'a giriş yaparsanız, o anda RAGFlow tam olarak başlatılmamış olabileceğinden
> tarayıcınız `ağ hatası` uyarısı verebilir.
>
5. Web tarayıcınıza sunucunuzun IP adresini girin ve RAGFlow'a giriş yapın.
> Varsayılan ayarlarla, yalnızca `http://MAKİNENİZİN_IP_ADRESİ` girmeniz yeterlidir (port numarası **gerekmez**),
> çünkü varsayılan HTTP sunucu portu `80` varsayılan yapılandırmalar kullanıldığında ihmal edilebilir.
>
6. [service_conf.yaml.template](./docker/service_conf.yaml.template) dosyasında, `user_default_llm` içinde istediğiniz LLM sağlayıcısını seçin ve
`API_KEY` alanını ilgili API anahtarıyla güncelleyin.
> Daha fazla bilgi için [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) sayfasına bakın.
>
_Gösteri başlasın!_
## 🔧 Yapılandırmalar
Sistem yapılandırmaları söz konusu olduğunda, aşağıdaki dosyaları yönetmeniz gerekecektir:
- [.env](./docker/.env): `SVR_HTTP_PORT`, `MYSQL_PASSWORD` ve `MINIO_PASSWORD` gibi temel sistem ayarlarını içerir.
- [service_conf.yaml.template](./docker/service_conf.yaml.template): Arka uç hizmetlerini yapılandırır. Bu dosyadaki ortam değişkenleri, Docker konteyneri başladığında otomatik olarak doldurulacaktır. Docker konteyneri içinde ayarlanan tüm ortam değişkenleri kullanıma hazır olacak ve hizmet davranışını dağıtım ortamına göre özelleştirmenize olanak tanıyacaktır.
- [docker-compose.yml](./docker/docker-compose.yml): Sistem, başlatılmak için [docker-compose.yml](./docker/docker-compose.yml) dosyasına dayanır.
> [./docker/README](./docker/README.md) dosyası, [service_conf.yaml.template](./docker/service_conf.yaml.template) dosyasında `${ENV_VARS}` olarak kullanılabilen ortam ayarları ve hizmet yapılandırmalarının ayrıntılı bir açıklamasını sağlar.
Varsayılan HTTP sunucu portunu (80) değiştirmek için [docker-compose.yml](./docker/docker-compose.yml) dosyasında `80:80` ifadesini `<SUNUCU_PORTUNUZ>:80` olarak değiştirin.
Yukarıdaki yapılandırma değişikliklerinin etkili olması için tüm konteynerlerin yeniden başlatılması gerekir:
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### Doküman Motorunu Elasticsearch'ten Infinity'ye Geçirme
RAGFlow varsayılan olarak tam metin ve vektörlerin depolanması için Elasticsearch kullanır. [Infinity](https://github.com/infiniflow/infinity/)'ye geçmek için şu adımları izleyin:
1. Çalışan tüm konteynerleri durdurun:
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` seçeneği Docker konteyner birimlerini silecek ve mevcut veriler temizlenecektir.
2. **docker/.env** dosyasında `DOC_ENGINE` değerini `infinity` olarak ayarlayın.
3. Konteynerleri başlatın:
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> Linux/arm64 makinesinde Infinity'ye geçiş henüz resmi olarak desteklenmemektedir.
## 🔧 Docker İmajı Oluşturma
Bu imaj yaklaşık 2 GB boyutundadır ve harici LLM ile gömme hizmetlerine bağlıdır.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
Veya bir proxy arkasındaysanız, proxy parametrelerini iletebilirsiniz:
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://PROXY_ADRESINIZ:PORT \
--build-arg https_proxy=http://PROXY_ADRESINIZ:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 Geliştirme İçin Kaynaktan Hizmet Başlatma
1. `uv` ve `pre-commit` yükleyin veya zaten yüklüyse bu adımı atlayın:
```bash
pipx install uv pre-commit
```
2. Kaynak kodunu klonlayın ve Python bağımlılıklarını yükleyin:
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # RAGFlow'un bağımlı Python modüllerini yükler
uv run python3 download_deps.py
pre-commit install
```
3. Bağımlı hizmetleri (MinIO, Elasticsearch, Redis ve MySQL) Docker Compose kullanarak başlatın:
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
**docker/.env** dosyasında belirtilen tüm ana bilgisayar adlarını `127.0.0.1`'e çözümlemek için `/etc/hosts` dosyasına aşağıdaki satırı ekleyin:
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. HuggingFace'e erişemiyorsanız, bir ayna site kullanmak için `HF_ENDPOINT` ortam değişkenini ayarlayın:
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. İşletim sisteminizde jemalloc yoksa, aşağıdaki şekilde yükleyin:
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. Arka uç hizmetini başlatın:
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. Ön yüz bağımlılıklarını yükleyin:
```bash
cd web
npm install
```
8. Ön yüz hizmetini başlatın:
```bash
npm run dev
```
_Aşağıdaki çıktı, sistemin başarıyla başlatıldığını onaylar:_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. Geliştirme tamamlandıktan sonra RAGFlow ön yüz ve arka uç hizmetini durdurun:
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 Dokümantasyon
- [Hızlı Başlangıç](https://ragflow.io/docs/dev/)
- [Yapılandırma](https://ragflow.io/docs/dev/configurations)
- [Sürüm Notları](https://ragflow.io/docs/dev/release_notes)
- [Kullanıcı Kılavuzları](https://ragflow.io/docs/category/user-guides)
- [Geliştirici Kılavuzları](https://ragflow.io/docs/category/developer-guides)
- [Referanslar](https://ragflow.io/docs/dev/category/references)
- [SSS](https://ragflow.io/docs/dev/faq)
## 📜 Yol Haritası
[RAGFlow Yol Haritası 2026](https://github.com/infiniflow/ragflow/issues/12241) sayfasına bakın.
## 🏄 Topluluk
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Tartışmalar](https://github.com/orgs/infiniflow/discussions)
## 🙌 Katkıda Bulunma
RAGFlow, açık kaynak iş birliği sayesinde gelişmektedir. Bu anlayışla, topluluktan gelen çeşitli katkıları benimsiyoruz.
Bir parçası olmak istiyorsanız, önce [Katkıda Bulunma Kılavuzumuzu](https://ragflow.io/docs/dev/contributing) inceleyin.

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DBEDFA"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 目錄</b></summary>
- 💡 [RAGFlow 是什麼?](#-RAGFlow-是什麼)
- 🎮 [Demo-試用](#-demo-試用)
- 🎮 [快速開始](#-快速開始)
- 📌 [近期更新](#-近期更新)
- 🌟 [主要功能](#-主要功能)
- 🔎 [系統架構](#-系統架構)
- 🎬 [快速開始](#-快速開始)
- 🎬 [自行架設](#-自行架設)
- 🔧 [系統配置](#-系統配置)
- 🔨 [以原始碼啟動服務](#-以原始碼啟動服務)
- 📚 [技術文檔](#-技術文檔)
@@ -72,11 +74,11 @@
## 💡 RAGFlow 是什麼?
[RAGFlow](https://ragflow.io/) 是一款領先的開源 RAGRetrieval-Augmented Generation引擎通過融合前沿的 RAG 技術與 Agent 能力,為大型語言模型提供卓越的上下文層。它提供可適配任意規模企業的端到端 RAG 工作流,憑藉融合式上下文引擎與預置的 Agent 模板,助力開發者以極致效率與精度將複雜數據轉化為高可信、生產級的人工智能系統。
[RAGFlow](https://ragflow.io/) 是一款領先的開源 [RAG](https://ragflow.io/basics/what-is-rag)Retrieval-Augmented Generation引擎通過融合前沿的 RAG 技術與 Agent 能力,為大型語言模型提供卓越的上下文層。它提供可適配任意規模企業的端到端 RAG 工作流,憑藉融合式[上下文引擎](https://ragflow.io/basics/what-is-agent-context-engine)與預置的 Agent 模板,助力開發者以極致效率與精度將複雜數據轉化為高可信、生產級的人工智能系統。
## 🎮 Demo 試用
## 🎮 快速開始
請登入網址 [https://demo.ragflow.io](https://demo.ragflow.io) 試用 demo
請登入網址 [https://cloud.ragflow.io](https://cloud.ragflow.io) 試用雲服務
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@
## 🔥 近期更新
- 2026-04-24 支援 DeepSeek v4 版本。
- 2026-03-24 發布 [RAGFlow 官方 Skill](https://clawhub.ai/yingfeng/ragflow-skill) — 提供官方 Skill 以透過 OpenClaw 訪問 RAGFlow 數據集。
- 2025-12-26 支援AI代理的「記憶」功能。
- 2025-11-19 支援 Gemini 3 Pro。
- 2025-11-12 支援從 Confluence、S3、Notion、Discord、Google Drive 進行資料同步。
@@ -140,7 +144,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 快速開始
## 🎬 自行架設
### 📝 前提條件
@@ -187,12 +191,12 @@
> 所有 Docker 映像檔都是為 x86 平台建置的。目前,我們不提供 ARM64 平台的 Docker 映像檔。
> 如果您使用的是 ARM64 平台,請使用 [這份指南](https://ragflow.io/docs/dev/build_docker_image) 來建置適合您系統的 Docker 映像檔。
> 執行以下指令會自動下載 RAGFlow Docker 映像 `v0.23.1`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.23.1` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。
> 執行以下指令會自動下載 RAGFlow Docker 映像 `v0.25.3`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.25.3` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# 可選使用穩定版標籤查看發佈https://github.com/infiniflow/ragflow/releases
# 此步驟確保程式碼中的 entrypoint.sh 檔案與 Docker 映像版本一致。
@@ -326,7 +330,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
3. 透過 Docker Compose 啟動依賴的服務MinIO, Elasticsearch, Redis, and MySQL
@@ -392,8 +396,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -403,8 +407,8 @@ docker build --platform linux/amd64 \
## 🏄 開源社群
- [Discord](https://discord.gg/zd4qPW6t)
- [Twitter](https://twitter.com/infiniflowai)
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 貢獻指南

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.23.1">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.3">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 目录</b></summary>
- 💡 [RAGFlow 是什么?](#-RAGFlow-是什么)
- 🎮 [Demo](#-demo)
- 🎮 [快速开始](#-快速开始)
- 📌 [近期更新](#-近期更新)
- 🌟 [主要功能](#-主要功能)
- 🔎 [系统架构](#-系统架构)
- 🎬 [快速开始](#-快速开始)
- 🎬 [自主托管](#-自主托管)
- 🔧 [系统配置](#-系统配置)
- 🔨 [以源代码启动服务](#-以源代码启动服务)
- 📚 [技术文档](#-技术文档)
@@ -72,11 +74,11 @@
## 💡 RAGFlow 是什么?
[RAGFlow](https://ragflow.io/) 是一款领先的开源检索增强生成RAG引擎通过融合前沿的 RAG 技术与 Agent 能力,为大型语言模型提供卓越的上下文层。它提供可适配任意规模企业的端到端 RAG 工作流,凭借融合式上下文引擎与预置的 Agent 模板,助力开发者以极致效率与精度将复杂数据转化为高可信、生产级的人工智能系统。
[RAGFlow](https://ragflow.io/) 是一款领先的开源检索增强生成([RAG](https://ragflow.io/basics/what-is-rag))引擎,通过融合前沿的 RAG 技术与 Agent 能力,为大型语言模型提供卓越的上下文层。它提供可适配任意规模企业的端到端 RAG 工作流,凭借融合式[上下文引擎](https://ragflow.io/basics/what-is-agent-context-engine)与预置的 Agent 模板,助力开发者以极致效率与精度将复杂数据转化为高可信、生产级的人工智能系统。
## 🎮 Demo 试用
## 🎮 快速开始
请登录网址 [https://demo.ragflow.io](https://demo.ragflow.io) 试用 demo
请登录网址 [https://cloud.ragflow.io](https://cloud.ragflow.io) 体验云服务
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,7 +87,9 @@
## 🔥 近期更新
- 2025-12-26 支持AI代理的“记忆”功能。
- 2026-04-24 支持 DeepSeek v4.
- 2026-03-24 发布 [RAGFlow 官方 Skill](https://clawhub.ai/yingfeng/ragflow-skill) — 提供官方 Skill 以通过 OpenClaw 访问 RAGFlow 数据集。
- 2025-12-26 支持 AI 代理的"记忆"功能。
- 2025-11-19 支持 Gemini 3 Pro。
- 2025-11-12 支持从 Confluence、S3、Notion、Discord、Google Drive 进行数据同步。
- 2025-10-23 支持 MinerU 和 Docling 作为文档解析方法。
@@ -140,7 +144,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 快速开始
## 🎬 自主托管
### 📝 前提条件
@@ -188,12 +192,12 @@
> 请注意,目前官方提供的所有 Docker 镜像均基于 x86 架构构建,并不提供基于 ARM64 的 Docker 镜像。
> 如果你的操作系统是 ARM64 架构,请参考[这篇文档](https://ragflow.io/docs/dev/build_docker_image)自行构建 Docker 镜像。
> 运行以下命令会自动下载 RAGFlow Docker 镜像 `v0.23.1`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.23.1` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。
> 运行以下命令会自动下载 RAGFlow Docker 镜像 `v0.25.3`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.25.3` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。
```bash
$ cd ragflow/docker
# git checkout v0.23.1
# git checkout v0.25.3
# 可选使用稳定版本标签查看发布https://github.com/infiniflow/ragflow/releases
# 这一步确保代码中的 entrypoint.sh 文件与 Docker 镜像的版本保持一致。
@@ -204,7 +208,7 @@
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> 注意:在 `v0.22.0` 之前的版本,我们会同时提供包含 embedding 模型的镜像和不含 embedding 模型的 slim 镜像。具体如下:
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
@@ -326,7 +330,7 @@ docker build --platform linux/amd64 \
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv run python3 download_deps.py
pre-commit install
```
@@ -395,8 +399,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -406,8 +410,8 @@ docker build --platform linux/amd64 \
## 🏄 开源社区
- [Discord](https://discord.gg/zd4qPW6t)
- [Twitter](https://twitter.com/infiniflowai)
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 贡献指南

View File

@@ -21,7 +21,7 @@ cp pyproject.toml release/$PROJECT_NAME/pyproject.toml
cp README.md release/$PROJECT_NAME/README.md
mkdir release/$PROJECT_NAME/$SOURCE_DIR/$PACKAGE_DIR -p
cp admin_client.py release/$PROJECT_NAME/$SOURCE_DIR/$PACKAGE_DIR/admin_client.py
cp ragflow_cli.py release/$PROJECT_NAME/$SOURCE_DIR/$PACKAGE_DIR/ragflow_cli.py
if [ -d "release/$PROJECT_NAME/$SOURCE_DIR" ]; then
echo "✅ source dir: release/$PROJECT_NAME/$SOURCE_DIR"

779
admin/client/COMMAND.md Normal file
View File

@@ -0,0 +1,779 @@
# RAGFlow CLI User Command Reference
This document describes the user commands available in RAGFlow CLI. All commands must end with a semicolon (`;`).
## Command List
### ping_server
**Description**
Tests the connection status to the server.
**Usage**
```
PING;
```
**Parameters**
No parameters.
**Example**
```
ragflow> PING;
```
**Display Effect**
(Sample output will be provided by the user)
---
### show_current_user
**Description**
Displays information about the currently logged-in user.
**Usage**
```
SHOW CURRENT USER;
```
**Parameters**
No parameters.
**Example**
```
ragflow> SHOW CURRENT USER;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_model_provider
**Description**
Creates a new model provider.
**Usage**
```
CREATE MODEL PROVIDER <provider_name> <provider_key>;
```
**Parameters**
- `provider_name`: Provider name, quoted string.
- `provider_key`: Provider key, quoted string.
**Example**
```
ragflow> CREATE MODEL PROVIDER 'openai' 'sk-...';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_model_provider
**Description**
Deletes a model provider.
**Usage**
```
DROP MODEL PROVIDER <provider_name>;
```
**Parameters**
- `provider_name`: Name of the provider to delete, quoted string.
**Example**
```
ragflow> DROP MODEL PROVIDER 'openai';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_llm
**Description**
Sets the default LLM (Large Language Model).
**Usage**
```
SET DEFAULT LLM <llm_id>;
```
**Parameters**
- `llm_id`: LLM identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT LLM 'gpt-4';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_vlm
**Description**
Sets the default VLM (Vision Language Model).
**Usage**
```
SET DEFAULT VLM <vlm_id>;
```
**Parameters**
- `vlm_id`: VLM identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT VLM 'clip-vit-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_embedding
**Description**
Sets the default embedding model.
**Usage**
```
SET DEFAULT EMBEDDING <embedding_id>;
```
**Parameters**
- `embedding_id`: Embedding model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT EMBEDDING 'text-embedding-ada-002';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_reranker
**Description**
Sets the default reranker model.
**Usage**
```
SET DEFAULT RERANKER <reranker_id>;
```
**Parameters**
- `reranker_id`: Reranker model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT RERANKER 'bge-reranker-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_asr
**Description**
Sets the default ASR (Automatic Speech Recognition) model.
**Usage**
```
SET DEFAULT ASR <asr_id>;
```
**Parameters**
- `asr_id`: ASR model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT ASR 'whisper-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_tts
**Description**
Sets the default TTS (Text-to-Speech) model.
**Usage**
```
SET DEFAULT TTS <tts_id>;
```
**Parameters**
- `tts_id`: TTS model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT TTS 'tts-1';
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_llm
**Description**
Resets the default LLM to system default.
**Usage**
```
RESET DEFAULT LLM;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT LLM;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_vlm
**Description**
Resets the default VLM to system default.
**Usage**
```
RESET DEFAULT VLM;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT VLM;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_embedding
**Description**
Resets the default embedding model to system default.
**Usage**
```
RESET DEFAULT EMBEDDING;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT EMBEDDING;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_reranker
**Description**
Resets the default reranker model to system default.
**Usage**
```
RESET DEFAULT RERANKER;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT RERANKER;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_asr
**Description**
Resets the default ASR model to system default.
**Usage**
```
RESET DEFAULT ASR;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT ASR;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_tts
**Description**
Resets the default TTS model to system default.
**Usage**
```
RESET DEFAULT TTS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT TTS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_dataset_with_parser
**Description**
Creates a user dataset with the specified parser.
**Usage**
```
CREATE DATASET <dataset_name> WITH EMBEDDING <embedding> PARSER <parser_type>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
- `embedding`: Embedding model name, quoted string.
- `parser_type`: Parser type, quoted string.
**Example**
```
ragflow> CREATE DATASET 'my_dataset' WITH EMBEDDING 'text-embedding-ada-002' PARSER 'pdf';
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_dataset_with_pipeline
**Description**
Creates a user dataset with the specified pipeline.
**Usage**
```
CREATE DATASET <dataset_name> WITH EMBEDDING <embedding> PIPELINE <pipeline>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
- `embedding`: Embedding model name, quoted string.
- `pipeline`: Pipeline name, quoted string.
**Example**
```
ragflow> CREATE DATASET 'my_dataset' WITH EMBEDDING 'text-embedding-ada-002' PIPELINE 'standard';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_user_dataset
**Description**
Deletes a user dataset.
**Usage**
```
DROP DATASET <dataset_name>;
```
**Parameters**
- `dataset_name`: Name of the dataset to delete, quoted string.
**Example**
```
ragflow> DROP DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_datasets
**Description**
Lists all datasets for the current user.
**Usage**
```
LIST DATASETS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST DATASETS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_dataset_files
**Description**
Lists all files in the specified dataset.
**Usage**
```
LIST FILES OF DATASET <dataset_name>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> LIST FILES OF DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_agents
**Description**
Lists all agents for the current user.
**Usage**
```
LIST AGENTS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST AGENTS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_chats
**Description**
Lists all chat sessions for the current user.
**Usage**
```
LIST CHATS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST CHATS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_chat
**Description**
Creates a new chat session.
**Usage**
```
CREATE CHAT <chat_name>;
```
**Parameters**
- `chat_name`: Chat session name, quoted string.
**Example**
```
ragflow> CREATE CHAT 'my_chat';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_user_chat
**Description**
Deletes a chat session.
**Usage**
```
DROP CHAT <chat_name>;
```
**Parameters**
- `chat_name`: Name of the chat session to delete, quoted string.
**Example**
```
ragflow> DROP CHAT 'my_chat';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_model_providers
**Description**
Lists all model providers for the current user.
**Usage**
```
LIST MODEL PROVIDERS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST MODEL PROVIDERS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_default_models
**Description**
Lists all default model settings for the current user.
**Usage**
```
LIST DEFAULT MODELS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST DEFAULT MODELS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### import_docs_into_dataset
**Description**
Imports documents into the specified dataset.
**Usage**
```
IMPORT <document_list> INTO DATASET <dataset_name>;
```
**Parameters**
- `document_list`: List of document paths, multiple paths can be separated by commas, or as a space-separated quoted string.
- `dataset_name`: Target dataset name, quoted string.
**Example**
```
ragflow> IMPORT '/path/to/doc1.pdf,/path/to/doc2.pdf' INTO DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### search_on_datasets
**Description**
Searches in one or more specified datasets.
**Usage**
```
SEARCH <question> ON DATASETS <dataset_list>;
```
**Parameters**
- `question`: Search question, quoted string.
- `dataset_list`: List of dataset names, multiple names can be separated by commas, or as a space-separated quoted string.
**Example**
```
ragflow> SEARCH 'What is RAG?' ON DATASETS 'dataset1,dataset2';
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_docs
**Description**
Parses specified documents in a dataset.
**Usage**
```
PARSE <document_names> OF DATASET <dataset_name>;
```
**Parameters**
- `document_names`: List of document names, multiple names can be separated by commas, or as a space-separated quoted string.
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE 'doc1.pdf,doc2.pdf' OF DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_sync
**Description**
Synchronously parses the entire dataset.
**Usage**
```
PARSE DATASET <dataset_name> SYNC;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE DATASET 'my_dataset' SYNC;
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_async
**Description**
Asynchronously parses the entire dataset.
**Usage**
```
PARSE DATASET <dataset_name> ASYNC;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE DATASET 'my_dataset' ASYNC;
```
**Display Effect**
(Sample output will be provided by the user)
---
### benchmark
**Description**
Performs performance benchmark testing on the specified user command.
**Usage**
```
BENCHMARK <concurrency> <iterations> <user_command>;
```
**Parameters**
- `concurrency`: Concurrency number, positive integer.
- `iterations`: Number of iterations, positive integer.
- `user_command`: User command to test (must be a valid user command, such as `PING;`).
**Example**
```
ragflow> BENCHMARK 5 10 PING;
```
**Display Effect**
(Sample output will be provided by the user)
---
**Notes**
- All string parameters (such as names, IDs, paths) must be enclosed in single quotes (`'`) or double quotes (`"`).
- Commands must end with a semicolon (`;`).
- The prompt is `ragflow>`.

View File

@@ -48,7 +48,7 @@ It consists of a server-side Service and a command-line client (CLI), both imple
1. Ensure the Admin Service is running.
2. Install ragflow-cli.
```bash
pip install ragflow-cli==0.23.1
pip install ragflow-cli==0.25.3
```
3. Launch the CLI client:
```bash

View File

@@ -1,938 +0,0 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import argparse
import base64
import getpass
from cmd import Cmd
from typing import Any, Dict, List
import requests
from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5
from Cryptodome.PublicKey import RSA
from lark import Lark, Transformer, Tree
GRAMMAR = r"""
start: command
command: sql_command | meta_command
sql_command: list_services
| show_service
| startup_service
| shutdown_service
| restart_service
| list_users
| show_user
| drop_user
| alter_user
| create_user
| activate_user
| list_datasets
| list_agents
| create_role
| drop_role
| alter_role
| list_roles
| show_role
| grant_permission
| revoke_permission
| alter_user_role
| show_user_permission
| show_version
// meta command definition
meta_command: "\\" meta_command_name [meta_args]
meta_command_name: /[a-zA-Z?]+/
meta_args: (meta_arg)+
meta_arg: /[^\\s"']+/ | quoted_string
// command definition
LIST: "LIST"i
SERVICES: "SERVICES"i
SHOW: "SHOW"i
CREATE: "CREATE"i
SERVICE: "SERVICE"i
SHUTDOWN: "SHUTDOWN"i
STARTUP: "STARTUP"i
RESTART: "RESTART"i
USERS: "USERS"i
DROP: "DROP"i
USER: "USER"i
ALTER: "ALTER"i
ACTIVE: "ACTIVE"i
PASSWORD: "PASSWORD"i
DATASETS: "DATASETS"i
OF: "OF"i
AGENTS: "AGENTS"i
ROLE: "ROLE"i
ROLES: "ROLES"i
DESCRIPTION: "DESCRIPTION"i
GRANT: "GRANT"i
REVOKE: "REVOKE"i
ALL: "ALL"i
PERMISSION: "PERMISSION"i
TO: "TO"i
FROM: "FROM"i
FOR: "FOR"i
RESOURCES: "RESOURCES"i
ON: "ON"i
SET: "SET"i
VERSION: "VERSION"i
list_services: LIST SERVICES ";"
show_service: SHOW SERVICE NUMBER ";"
startup_service: STARTUP SERVICE NUMBER ";"
shutdown_service: SHUTDOWN SERVICE NUMBER ";"
restart_service: RESTART SERVICE NUMBER ";"
list_users: LIST USERS ";"
drop_user: DROP USER quoted_string ";"
alter_user: ALTER USER PASSWORD quoted_string quoted_string ";"
show_user: SHOW USER quoted_string ";"
create_user: CREATE USER quoted_string quoted_string ";"
activate_user: ALTER USER ACTIVE quoted_string status ";"
list_datasets: LIST DATASETS OF quoted_string ";"
list_agents: LIST AGENTS OF quoted_string ";"
create_role: CREATE ROLE identifier [DESCRIPTION quoted_string] ";"
drop_role: DROP ROLE identifier ";"
alter_role: ALTER ROLE identifier SET DESCRIPTION quoted_string ";"
list_roles: LIST ROLES ";"
show_role: SHOW ROLE identifier ";"
grant_permission: GRANT action_list ON identifier TO ROLE identifier ";"
revoke_permission: REVOKE action_list ON identifier FROM ROLE identifier ";"
alter_user_role: ALTER USER quoted_string SET ROLE identifier ";"
show_user_permission: SHOW USER PERMISSION quoted_string ";"
show_version: SHOW VERSION ";"
action_list: identifier ("," identifier)*
identifier: WORD
quoted_string: QUOTED_STRING
status: WORD
QUOTED_STRING: /'[^']+'/ | /"[^"]+"/
WORD: /[a-zA-Z0-9_\-\.]+/
NUMBER: /[0-9]+/
%import common.WS
%ignore WS
"""
class AdminTransformer(Transformer):
def start(self, items):
return items[0]
def command(self, items):
return items[0]
def list_services(self, items):
result = {"type": "list_services"}
return result
def show_service(self, items):
service_id = int(items[2])
return {"type": "show_service", "number": service_id}
def startup_service(self, items):
service_id = int(items[2])
return {"type": "startup_service", "number": service_id}
def shutdown_service(self, items):
service_id = int(items[2])
return {"type": "shutdown_service", "number": service_id}
def restart_service(self, items):
service_id = int(items[2])
return {"type": "restart_service", "number": service_id}
def list_users(self, items):
return {"type": "list_users"}
def show_user(self, items):
user_name = items[2]
return {"type": "show_user", "user_name": user_name}
def drop_user(self, items):
user_name = items[2]
return {"type": "drop_user", "user_name": user_name}
def alter_user(self, items):
user_name = items[3]
new_password = items[4]
return {"type": "alter_user", "user_name": user_name, "password": new_password}
def create_user(self, items):
user_name = items[2]
password = items[3]
return {"type": "create_user", "user_name": user_name, "password": password, "role": "user"}
def activate_user(self, items):
user_name = items[3]
activate_status = items[4]
return {"type": "activate_user", "activate_status": activate_status, "user_name": user_name}
def list_datasets(self, items):
user_name = items[3]
return {"type": "list_datasets", "user_name": user_name}
def list_agents(self, items):
user_name = items[3]
return {"type": "list_agents", "user_name": user_name}
def create_role(self, items):
role_name = items[2]
if len(items) > 4:
description = items[4]
return {"type": "create_role", "role_name": role_name, "description": description}
else:
return {"type": "create_role", "role_name": role_name}
def drop_role(self, items):
role_name = items[2]
return {"type": "drop_role", "role_name": role_name}
def alter_role(self, items):
role_name = items[2]
description = items[5]
return {"type": "alter_role", "role_name": role_name, "description": description}
def list_roles(self, items):
return {"type": "list_roles"}
def show_role(self, items):
role_name = items[2]
return {"type": "show_role", "role_name": role_name}
def grant_permission(self, items):
action_list = items[1]
resource = items[3]
role_name = items[6]
return {"type": "grant_permission", "role_name": role_name, "resource": resource, "actions": action_list}
def revoke_permission(self, items):
action_list = items[1]
resource = items[3]
role_name = items[6]
return {"type": "revoke_permission", "role_name": role_name, "resource": resource, "actions": action_list}
def alter_user_role(self, items):
user_name = items[2]
role_name = items[5]
return {"type": "alter_user_role", "user_name": user_name, "role_name": role_name}
def show_user_permission(self, items):
user_name = items[3]
return {"type": "show_user_permission", "user_name": user_name}
def show_version(self, items):
return {"type": "show_version"}
def action_list(self, items):
return items
def meta_command(self, items):
command_name = str(items[0]).lower()
args = items[1:] if len(items) > 1 else []
# handle quoted parameter
parsed_args = []
for arg in args:
if hasattr(arg, "value"):
parsed_args.append(arg.value)
else:
parsed_args.append(str(arg))
return {"type": "meta", "command": command_name, "args": parsed_args}
def meta_command_name(self, items):
return items[0]
def meta_args(self, items):
return items
def encrypt(input_string):
pub = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----"
pub_key = RSA.importKey(pub)
cipher = Cipher_pkcs1_v1_5.new(pub_key)
cipher_text = cipher.encrypt(base64.b64encode(input_string.encode("utf-8")))
return base64.b64encode(cipher_text).decode("utf-8")
def encode_to_base64(input_string):
base64_encoded = base64.b64encode(input_string.encode("utf-8"))
return base64_encoded.decode("utf-8")
class AdminCLI(Cmd):
def __init__(self):
super().__init__()
self.parser = Lark(GRAMMAR, start="start", parser="lalr", transformer=AdminTransformer())
self.command_history = []
self.is_interactive = False
self.admin_account = "admin@ragflow.io"
self.admin_password: str = "admin"
self.session = requests.Session()
self.access_token: str = ""
self.host: str = ""
self.port: int = 0
intro = r"""Type "\h" for help."""
prompt = "admin> "
def onecmd(self, command: str) -> bool:
try:
result = self.parse_command(command)
if isinstance(result, dict):
if "type" in result and result.get("type") == "empty":
return False
self.execute_command(result)
if isinstance(result, Tree):
return False
if result.get("type") == "meta" and result.get("command") in ["q", "quit", "exit"]:
return True
except KeyboardInterrupt:
print("\nUse '\\q' to quit")
except EOFError:
print("\nGoodbye!")
return True
return False
def emptyline(self) -> bool:
return False
def default(self, line: str) -> bool:
return self.onecmd(line)
def parse_command(self, command_str: str) -> dict[str, str]:
if not command_str.strip():
return {"type": "empty"}
self.command_history.append(command_str)
try:
result = self.parser.parse(command_str)
return result
except Exception as e:
return {"type": "error", "message": f"Parse error: {str(e)}"}
def verify_admin(self, arguments: dict, single_command: bool):
self.host = arguments["host"]
self.port = arguments["port"]
print("Attempt to access server for admin login")
url = f"http://{self.host}:{self.port}/api/v1/admin/login"
attempt_count = 3
if single_command:
attempt_count = 1
try_count = 0
while True:
try_count += 1
if try_count > attempt_count:
return False
if single_command:
admin_passwd = arguments["password"]
else:
admin_passwd = getpass.getpass(f"password for {self.admin_account}: ").strip()
try:
self.admin_password = encrypt(admin_passwd)
response = self.session.post(url, json={"email": self.admin_account, "password": self.admin_password})
if response.status_code == 200:
res_json = response.json()
error_code = res_json.get("code", -1)
if error_code == 0:
self.session.headers.update({"Content-Type": "application/json", "Authorization": response.headers["Authorization"], "User-Agent": "RAGFlow-CLI/0.23.1"})
print("Authentication successful.")
return True
else:
error_message = res_json.get("message", "Unknown error")
print(f"Authentication failed: {error_message}, try again")
continue
else:
print(f"Bad responsestatus: {response.status_code}, password is wrong")
except Exception as e:
print(str(e))
print("Can't access server for admin login (connection failed)")
def _format_service_detail_table(self, data):
if isinstance(data, list):
return data
if not all([isinstance(v, list) for v in data.values()]):
# normal table
return data
# handle task_executor heartbeats map, for example {'name': [{'done': 2, 'now': timestamp1}, {'done': 3, 'now': timestamp2}]
task_executor_list = []
for k, v in data.items():
# display latest status
heartbeats = sorted(v, key=lambda x: x["now"], reverse=True)
task_executor_list.append(
{
"task_executor_name": k,
**heartbeats[0],
}
if heartbeats
else {"task_executor_name": k}
)
return task_executor_list
def _print_table_simple(self, data):
if not data:
print("No data to print")
return
if isinstance(data, dict):
# handle single row data
data = [data]
columns = list(set().union(*(d.keys() for d in data)))
columns.sort()
col_widths = {}
def get_string_width(text):
half_width_chars = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\t\n\r"
width = 0
for char in text:
if char in half_width_chars:
width += 1
else:
width += 2
return width
for col in columns:
max_width = get_string_width(str(col))
for item in data:
value_len = get_string_width(str(item.get(col, "")))
if value_len > max_width:
max_width = value_len
col_widths[col] = max(2, max_width)
# Generate delimiter
separator = "+" + "+".join(["-" * (col_widths[col] + 2) for col in columns]) + "+"
# Print header
print(separator)
header = "|" + "|".join([f" {col:<{col_widths[col]}} " for col in columns]) + "|"
print(header)
print(separator)
# Print data
for item in data:
row = "|"
for col in columns:
value = str(item.get(col, ""))
if get_string_width(value) > col_widths[col]:
value = value[: col_widths[col] - 3] + "..."
row += f" {value:<{col_widths[col] - (get_string_width(value) - len(value))}} |"
print(row)
print(separator)
def run_interactive(self):
self.is_interactive = True
print("RAGFlow Admin command line interface - Type '\\?' for help, '\\q' to quit")
while True:
try:
command = input("admin> ").strip()
if not command:
continue
print(f"command: {command}")
result = self.parse_command(command)
self.execute_command(result)
if isinstance(result, Tree):
continue
if result.get("type") == "meta" and result.get("command") in ["q", "quit", "exit"]:
break
except KeyboardInterrupt:
print("\nUse '\\q' to quit")
except EOFError:
print("\nGoodbye!")
break
def run_single_command(self, command: str):
result = self.parse_command(command)
self.execute_command(result)
def parse_connection_args(self, args: List[str]) -> Dict[str, Any]:
parser = argparse.ArgumentParser(description="Admin CLI Client", add_help=False)
parser.add_argument("-h", "--host", default="localhost", help="Admin service host")
parser.add_argument("-p", "--port", type=int, default=9381, help="Admin service port")
parser.add_argument("-w", "--password", default="admin", type=str, help="Superuser password")
parser.add_argument("command", nargs="?", help="Single command")
try:
parsed_args, remaining_args = parser.parse_known_args(args)
if remaining_args:
command = remaining_args[0]
return {"host": parsed_args.host, "port": parsed_args.port, "password": parsed_args.password, "command": command}
else:
return {
"host": parsed_args.host,
"port": parsed_args.port,
}
except SystemExit:
return {"error": "Invalid connection arguments"}
def execute_command(self, parsed_command: Dict[str, Any]):
command_dict: dict
if isinstance(parsed_command, Tree):
command_dict = parsed_command.children[0]
else:
if parsed_command["type"] == "error":
print(f"Error: {parsed_command['message']}")
return
else:
command_dict = parsed_command
# print(f"Parsed command: {command_dict}")
command_type = command_dict["type"]
match command_type:
case "list_services":
self._handle_list_services(command_dict)
case "show_service":
self._handle_show_service(command_dict)
case "restart_service":
self._handle_restart_service(command_dict)
case "shutdown_service":
self._handle_shutdown_service(command_dict)
case "startup_service":
self._handle_startup_service(command_dict)
case "list_users":
self._handle_list_users(command_dict)
case "show_user":
self._handle_show_user(command_dict)
case "drop_user":
self._handle_drop_user(command_dict)
case "alter_user":
self._handle_alter_user(command_dict)
case "create_user":
self._handle_create_user(command_dict)
case "activate_user":
self._handle_activate_user(command_dict)
case "list_datasets":
self._handle_list_datasets(command_dict)
case "list_agents":
self._handle_list_agents(command_dict)
case "create_role":
self._create_role(command_dict)
case "drop_role":
self._drop_role(command_dict)
case "alter_role":
self._alter_role(command_dict)
case "list_roles":
self._list_roles(command_dict)
case "show_role":
self._show_role(command_dict)
case "grant_permission":
self._grant_permission(command_dict)
case "revoke_permission":
self._revoke_permission(command_dict)
case "alter_user_role":
self._alter_user_role(command_dict)
case "show_user_permission":
self._show_user_permission(command_dict)
case "show_version":
self._show_version(command_dict)
case "meta":
self._handle_meta_command(command_dict)
case _:
print(f"Command '{command_type}' would be executed with API")
def _handle_list_services(self, command):
print("Listing all services")
url = f"http://{self.host}:{self.port}/api/v1/admin/services"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to get all services, code: {res_json['code']}, message: {res_json['message']}")
def _handle_show_service(self, command):
service_id: int = command["number"]
print(f"Showing service: {service_id}")
url = f"http://{self.host}:{self.port}/api/v1/admin/services/{service_id}"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
res_data = res_json["data"]
if "status" in res_data and res_data["status"] == "alive":
print(f"Service {res_data['service_name']} is alive, ")
if isinstance(res_data["message"], str):
print(res_data["message"])
else:
data = self._format_service_detail_table(res_data["message"])
self._print_table_simple(data)
else:
print(f"Service {res_data['service_name']} is down, {res_data['message']}")
else:
print(f"Fail to show service, code: {res_json['code']}, message: {res_json['message']}")
def _handle_restart_service(self, command):
service_id: int = command["number"]
print(f"Restart service {service_id}")
def _handle_shutdown_service(self, command):
service_id: int = command["number"]
print(f"Shutdown service {service_id}")
def _handle_startup_service(self, command):
service_id: int = command["number"]
print(f"Startup service {service_id}")
def _handle_list_users(self, command):
print("Listing all users")
url = f"http://{self.host}:{self.port}/api/v1/admin/users"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to get all users, code: {res_json['code']}, message: {res_json['message']}")
def _handle_show_user(self, command):
username_tree: Tree = command["user_name"]
user_name: str = username_tree.children[0].strip("'\"")
print(f"Showing user: {user_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
table_data = res_json["data"]
table_data.pop("avatar")
self._print_table_simple(table_data)
else:
print(f"Fail to get user {user_name}, code: {res_json['code']}, message: {res_json['message']}")
def _handle_drop_user(self, command):
username_tree: Tree = command["user_name"]
user_name: str = username_tree.children[0].strip("'\"")
print(f"Drop user: {user_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}"
response = self.session.delete(url)
res_json = response.json()
if response.status_code == 200:
print(res_json["message"])
else:
print(f"Fail to drop user, code: {res_json['code']}, message: {res_json['message']}")
def _handle_alter_user(self, command):
user_name_tree: Tree = command["user_name"]
user_name: str = user_name_tree.children[0].strip("'\"")
password_tree: Tree = command["password"]
password: str = password_tree.children[0].strip("'\"")
print(f"Alter user: {user_name}, password: ******")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}/password"
response = self.session.put(url, json={"new_password": encrypt(password)})
res_json = response.json()
if response.status_code == 200:
print(res_json["message"])
else:
print(f"Fail to alter password, code: {res_json['code']}, message: {res_json['message']}")
def _handle_create_user(self, command):
user_name_tree: Tree = command["user_name"]
user_name: str = user_name_tree.children[0].strip("'\"")
password_tree: Tree = command["password"]
password: str = password_tree.children[0].strip("'\"")
role: str = command["role"]
print(f"Create user: {user_name}, password: ******, role: {role}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users"
response = self.session.post(url, json={"user_name": user_name, "password": encrypt(password), "role": role})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to create user {user_name}, code: {res_json['code']}, message: {res_json['message']}")
def _handle_activate_user(self, command):
user_name_tree: Tree = command["user_name"]
user_name: str = user_name_tree.children[0].strip("'\"")
activate_tree: Tree = command["activate_status"]
activate_status: str = activate_tree.children[0].strip("'\"")
if activate_status.lower() in ["on", "off"]:
print(f"Alter user {user_name} activate status, turn {activate_status.lower()}.")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}/activate"
response = self.session.put(url, json={"activate_status": activate_status})
res_json = response.json()
if response.status_code == 200:
print(res_json["message"])
else:
print(f"Fail to alter activate status, code: {res_json['code']}, message: {res_json['message']}")
else:
print(f"Unknown activate status: {activate_status}.")
def _handle_list_datasets(self, command):
username_tree: Tree = command["user_name"]
user_name: str = username_tree.children[0].strip("'\"")
print(f"Listing all datasets of user: {user_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}/datasets"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
table_data = res_json["data"]
for t in table_data:
t.pop("avatar")
self._print_table_simple(table_data)
else:
print(f"Fail to get all datasets of {user_name}, code: {res_json['code']}, message: {res_json['message']}")
def _handle_list_agents(self, command):
username_tree: Tree = command["user_name"]
user_name: str = username_tree.children[0].strip("'\"")
print(f"Listing all agents of user: {user_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name}/agents"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
table_data = res_json["data"]
for t in table_data:
t.pop("avatar")
self._print_table_simple(table_data)
else:
print(f"Fail to get all agents of {user_name}, code: {res_json['code']}, message: {res_json['message']}")
def _create_role(self, command):
role_name_tree: Tree = command["role_name"]
role_name: str = role_name_tree.children[0].strip("'\"")
desc_str: str = ""
if "description" in command:
desc_tree: Tree = command["description"]
desc_str = desc_tree.children[0].strip("'\"")
print(f"create role name: {role_name}, description: {desc_str}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles"
response = self.session.post(url, json={"role_name": role_name, "description": desc_str})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to create role {role_name}, code: {res_json['code']}, message: {res_json['message']}")
def _drop_role(self, command):
role_name_tree: Tree = command["role_name"]
role_name: str = role_name_tree.children[0].strip("'\"")
print(f"drop role name: {role_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles/{role_name}"
response = self.session.delete(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to drop role {role_name}, code: {res_json['code']}, message: {res_json['message']}")
def _alter_role(self, command):
role_name_tree: Tree = command["role_name"]
role_name: str = role_name_tree.children[0].strip("'\"")
desc_tree: Tree = command["description"]
desc_str: str = desc_tree.children[0].strip("'\"")
print(f"alter role name: {role_name}, description: {desc_str}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles/{role_name}"
response = self.session.put(url, json={"description": desc_str})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to update role {role_name} with description: {desc_str}, code: {res_json['code']}, message: {res_json['message']}")
def _list_roles(self, command):
print("Listing all roles")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to list roles, code: {res_json['code']}, message: {res_json['message']}")
def _show_role(self, command):
role_name_tree: Tree = command["role_name"]
role_name: str = role_name_tree.children[0].strip("'\"")
print(f"show role: {role_name}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles/{role_name}/permission"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to list roles, code: {res_json['code']}, message: {res_json['message']}")
def _grant_permission(self, command):
role_name_tree: Tree = command["role_name"]
role_name_str: str = role_name_tree.children[0].strip("'\"")
resource_tree: Tree = command["resource"]
resource_str: str = resource_tree.children[0].strip("'\"")
action_tree_list: list = command["actions"]
actions: list = []
for action_tree in action_tree_list:
action_str: str = action_tree.children[0].strip("'\"")
actions.append(action_str)
print(f"grant role_name: {role_name_str}, resource: {resource_str}, actions: {actions}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles/{role_name_str}/permission"
response = self.session.post(url, json={"actions": actions, "resource": resource_str})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to grant role {role_name_str} with {actions} on {resource_str}, code: {res_json['code']}, message: {res_json['message']}")
def _revoke_permission(self, command):
role_name_tree: Tree = command["role_name"]
role_name_str: str = role_name_tree.children[0].strip("'\"")
resource_tree: Tree = command["resource"]
resource_str: str = resource_tree.children[0].strip("'\"")
action_tree_list: list = command["actions"]
actions: list = []
for action_tree in action_tree_list:
action_str: str = action_tree.children[0].strip("'\"")
actions.append(action_str)
print(f"revoke role_name: {role_name_str}, resource: {resource_str}, actions: {actions}")
url = f"http://{self.host}:{self.port}/api/v1/admin/roles/{role_name_str}/permission"
response = self.session.delete(url, json={"actions": actions, "resource": resource_str})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to revoke role {role_name_str} with {actions} on {resource_str}, code: {res_json['code']}, message: {res_json['message']}")
def _alter_user_role(self, command):
role_name_tree: Tree = command["role_name"]
role_name_str: str = role_name_tree.children[0].strip("'\"")
user_name_tree: Tree = command["user_name"]
user_name_str: str = user_name_tree.children[0].strip("'\"")
print(f"alter_user_role user_name: {user_name_str}, role_name: {role_name_str}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name_str}/role"
response = self.session.put(url, json={"role_name": role_name_str})
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to alter user: {user_name_str} to role {role_name_str}, code: {res_json['code']}, message: {res_json['message']}")
def _show_user_permission(self, command):
user_name_tree: Tree = command["user_name"]
user_name_str: str = user_name_tree.children[0].strip("'\"")
print(f"show_user_permission user_name: {user_name_str}")
url = f"http://{self.host}:{self.port}/api/v1/admin/users/{user_name_str}/permission"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to show user: {user_name_str} permission, code: {res_json['code']}, message: {res_json['message']}")
def _show_version(self, command):
print("show_version")
url = f"http://{self.host}:{self.port}/api/v1/admin/version"
response = self.session.get(url)
res_json = response.json()
if response.status_code == 200:
self._print_table_simple(res_json["data"])
else:
print(f"Fail to show version, code: {res_json['code']}, message: {res_json['message']}")
def _handle_meta_command(self, command):
meta_command = command["command"]
args = command.get("args", [])
if meta_command in ["?", "h", "help"]:
self.show_help()
elif meta_command in ["q", "quit", "exit"]:
print("Goodbye!")
else:
print(f"Meta command '{meta_command}' with args {args}")
def show_help(self):
"""Help info"""
help_text = """
Commands:
LIST SERVICES
SHOW SERVICE <service>
STARTUP SERVICE <service>
SHUTDOWN SERVICE <service>
RESTART SERVICE <service>
LIST USERS
SHOW USER <user>
DROP USER <user>
CREATE USER <user> <password>
ALTER USER PASSWORD <user> <new_password>
ALTER USER ACTIVE <user> <on/off>
LIST DATASETS OF <user>
LIST AGENTS OF <user>
Meta Commands:
\\?, \\h, \\help Show this help
\\q, \\quit, \\exit Quit the CLI
"""
print(help_text)
def main():
import sys
cli = AdminCLI()
args = cli.parse_connection_args(sys.argv)
if "error" in args:
print("Error: Invalid connection arguments")
return
if "command" in args:
if "password" not in args:
print("Error: password is missing")
return
if cli.verify_admin(args, single_command=True):
command: str = args["command"]
# print(f"Run single command: {command}")
cli.run_single_command(command)
else:
if cli.verify_admin(args, single_command=False):
print(r"""
____ ___ ______________ ___ __ _
/ __ \/ | / ____/ ____/ /___ _ __ / | ____/ /___ ___ (_)___
/ /_/ / /| |/ / __/ /_ / / __ \ | /| / / / /| |/ __ / __ `__ \/ / __ \
/ _, _/ ___ / /_/ / __/ / / /_/ / |/ |/ / / ___ / /_/ / / / / / / / / / /
/_/ |_/_/ |_\____/_/ /_/\____/|__/|__/ /_/ |_\__,_/_/ /_/ /_/_/_/ /_/
""")
cli.cmdloop()
if __name__ == "__main__":
main()

182
admin/client/http_client.py Normal file
View File

@@ -0,0 +1,182 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import time
import json
import typing
from typing import Any, Dict, Optional
import requests
# from requests.sessions import HTTPAdapter
class HttpClient:
def __init__(
self,
host: str = "127.0.0.1",
port: int = 9381,
api_version: str = "v1",
api_key: Optional[str] = None,
connect_timeout: float = 5.0,
read_timeout: float = 60.0,
verify_ssl: bool = False,
) -> None:
self.host = host
self.port = port
self.api_version = api_version
self.api_key = api_key
self.login_token: str | None = None
self.connect_timeout = connect_timeout
self.read_timeout = read_timeout
self.verify_ssl = verify_ssl
def api_base(self) -> str:
return f"{self.host}:{self.port}/api/{self.api_version}"
def non_api_base(self) -> str:
return f"{self.host}:{self.port}/{self.api_version}"
def build_url(self, path: str, use_api_base: bool = True) -> str:
base = self.api_base() if use_api_base else self.non_api_base()
if self.verify_ssl:
return f"https://{base}/{path.lstrip('/')}"
else:
return f"http://{base}/{path.lstrip('/')}"
def _headers(self, auth_kind: Optional[str], extra: Optional[Dict[str, str]]) -> Dict[str, str]:
headers = {}
if auth_kind == "api" and self.api_key:
headers["Authorization"] = f"Bearer {self.api_key}"
elif auth_kind == "web" and self.login_token:
headers["Authorization"] = self.login_token
elif auth_kind == "admin" and self.login_token:
headers["Authorization"] = self.login_token
else:
pass
if extra:
headers.update(extra)
return headers
def request(
self,
method: str,
path: str,
*,
use_api_base: bool = True,
auth_kind: Optional[str] = "api",
headers: Optional[Dict[str, str]] = None,
json_body: Optional[Dict[str, Any]] = None,
data: Any = None,
files: Any = None,
params: Optional[Dict[str, Any]] = None,
stream: bool = False,
iterations: int = 1,
) -> requests.Response | dict:
url = self.build_url(path, use_api_base=use_api_base)
merged_headers = self._headers(auth_kind, headers)
# timeout: Tuple[float, float] = (self.connect_timeout, self.read_timeout)
session = requests.Session()
# adapter = HTTPAdapter(pool_connections=100, pool_maxsize=100)
# session.mount("http://", adapter)
http_function = typing.Any
match method:
case "GET":
http_function = session.get
case "POST":
http_function = session.post
case "PUT":
http_function = session.put
case "DELETE":
http_function = session.delete
case "PATCH":
http_function = session.patch
case _:
raise ValueError(f"Invalid HTTP method: {method}")
if iterations > 1:
response_list = []
total_duration = 0.0
for _ in range(iterations):
start_time = time.perf_counter()
response = http_function(url, headers=merged_headers, json=json_body, data=data, stream=stream)
# response = session.get(url, headers=merged_headers, json=json_body, data=data, stream=stream)
# response = requests.request(
# method=method,
# url=url,
# headers=merged_headers,
# json=json_body,
# data=data,
# files=files,
# params=params,
# stream=stream,
# verify=self.verify_ssl,
# )
end_time = time.perf_counter()
total_duration += end_time - start_time
response_list.append(response)
return {"duration": total_duration, "response_list": response_list}
else:
return http_function(url, headers=merged_headers, json=json_body, data=data, stream=stream)
# return session.get(url, headers=merged_headers, json=json_body, data=data, stream=stream)
# return requests.request(
# method=method,
# url=url,
# headers=merged_headers,
# json=json_body,
# data=data,
# files=files,
# params=params,
# stream=stream,
# verify=self.verify_ssl,
# )
def request_json(
self,
method: str,
path: str,
*,
use_api_base: bool = True,
auth_kind: Optional[str] = "api",
headers: Optional[Dict[str, str]] = None,
json_body: Optional[Dict[str, Any]] = None,
data: Any = None,
files: Any = None,
params: Optional[Dict[str, Any]] = None,
stream: bool = False,
) -> Dict[str, Any]:
response = self.request(
method,
path,
use_api_base=use_api_base,
auth_kind=auth_kind,
headers=headers,
json_body=json_body,
data=data,
files=files,
params=params,
stream=stream,
)
try:
return response.json()
except Exception as exc:
raise ValueError(f"Non-JSON response from {path}: {exc}") from exc
@staticmethod
def parse_json_bytes(raw: bytes) -> Dict[str, Any]:
try:
return json.loads(raw.decode("utf-8"))
except Exception as exc:
raise ValueError(f"Invalid JSON payload: {exc}") from exc

904
admin/client/parser.py Normal file
View File

@@ -0,0 +1,904 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from lark import Transformer
GRAMMAR = r"""
start: command
command: sql_command | meta_command
sql_command: login_user
| ping_server
| list_services
| show_service
| startup_service
| shutdown_service
| restart_service
| register_user
| list_users
| show_user
| drop_user
| alter_user
| create_user
| activate_user
| list_datasets
| list_agents
| create_role
| drop_role
| alter_role
| list_roles
| show_role
| grant_permission
| revoke_permission
| alter_user_role
| show_user_permission
| show_version
| grant_admin
| revoke_admin
| set_variable
| show_variable
| list_variables
| list_configs
| list_environments
| generate_key
| list_keys
| drop_key
| show_current_user
| set_default_llm
| set_default_vlm
| set_default_embedding
| set_default_reranker
| set_default_asr
| set_default_tts
| reset_default_llm
| reset_default_vlm
| reset_default_embedding
| reset_default_reranker
| reset_default_asr
| reset_default_tts
| create_model_provider
| drop_model_provider
| create_user_dataset_with_parser
| create_user_dataset_with_pipeline
| drop_user_dataset
| list_user_datasets
| list_user_dataset_files
| list_user_dataset_documents
| list_user_datasets_metadata
| list_user_documents_metadata_summary
| list_user_agents
| list_user_chats
| create_user_chat
| drop_user_chat
| create_dataset_table
| drop_dataset_table
| create_metadata_table
| drop_metadata_table
| list_user_model_providers
| list_user_default_models
| parse_dataset_docs
| parse_dataset_sync
| parse_dataset_async
| import_docs_into_dataset
| search_on_datasets
| get_chunk
| list_chunks
| insert_dataset_from_file
| insert_metadata_from_file
| update_chunk
| set_metadata
| remove_tags
| remove_chunks
| create_chat_session
| drop_chat_session
| list_chat_sessions
| chat_on_session
| list_server_configs
| show_fingerprint
| set_license
| set_license_config
| show_license
| check_license
| benchmark
// meta command definition
meta_command: "\\" meta_command_name [meta_args]
COMMA: ","
meta_command_name: /[a-zA-Z?]+/
meta_args: (meta_arg)+
meta_arg: /[^\s"',]+/ | quoted_string
// command definition
LOGIN: "LOGIN"i
REGISTER: "REGISTER"i
LIST: "LIST"i
SERVICES: "SERVICES"i
SHOW: "SHOW"i
CREATE: "CREATE"i
SERVICE: "SERVICE"i
SHUTDOWN: "SHUTDOWN"i
STARTUP: "STARTUP"i
RESTART: "RESTART"i
USERS: "USERS"i
DROP: "DROP"i
USER: "USER"i
ALTER: "ALTER"i
ACTIVE: "ACTIVE"i
ADMIN: "ADMIN"i
PASSWORD: "PASSWORD"i
DATASET_TABLE: "DATASET TABLE"i
DATASET: "DATASET"i
DATASETS: "DATASETS"i
OF: "OF"i
AGENTS: "AGENTS"i
ROLE: "ROLE"i
ROLES: "ROLES"i
DESCRIPTION: "DESCRIPTION"i
GRANT: "GRANT"i
REVOKE: "REVOKE"i
ALL: "ALL"i
PERMISSION: "PERMISSION"i
TO: "TO"i
FROM: "FROM"i
FOR: "FOR"i
RESOURCES: "RESOURCES"i
ON: "ON"i
SET: "SET"i
RESET: "RESET"i
VERSION: "VERSION"i
VAR: "VAR"i
VARS: "VARS"i
CONFIGS: "CONFIGS"i
ENVS: "ENVS"i
KEY: "KEY"i
KEYS: "KEYS"i
GENERATE: "GENERATE"i
MODEL: "MODEL"i
MODELS: "MODELS"i
PROVIDER: "PROVIDER"i
PROVIDERS: "PROVIDERS"i
DEFAULT: "DEFAULT"i
CHATS: "CHATS"i
CHAT: "CHAT"i
FILES: "FILES"i
DOCUMENT: "DOCUMENT"i
DOCUMENTS: "DOCUMENTS"i
METADATA: "METADATA"i
SUMMARY: "SUMMARY"i
AS: "AS"i
PARSE: "PARSE"i
IMPORT: "IMPORT"i
INTO: "INTO"i
IN: "IN"i
WITH: "WITH"i
VECTOR: "VECTOR"i
SIZE: "SIZE"i
PARSER: "PARSER"i
PIPELINE: "PIPELINE"i
SEARCH: "SEARCH"i
CURRENT: "CURRENT"i
LLM: "LLM"i
VLM: "VLM"i
EMBEDDING: "EMBEDDING"i
RERANKER: "RERANKER"i
ASR: "ASR"i
TTS: "TTS"i
ASYNC: "ASYNC"i
SYNC: "SYNC"i
BENCHMARK: "BENCHMARK"i
PING: "PING"i
SESSION: "SESSION"i
SESSIONS: "SESSIONS"i
SERVER: "SERVER"i
FINGERPRINT: "FINGERPRINT"i
LICENSE: "LICENSE"i
CHECK: "CHECK"i
CONFIG: "CONFIG"i
INDEX: "INDEX"i
TABLE: "TABLE"i
CHUNK: "CHUNK"i
CHUNKS: "CHUNKS"i
GET: "GET"i
INSERT: "INSERT"i
PAGE: "PAGE"i
KEYWORDS: "KEYWORDS"i
AVAILABLE: "AVAILABLE"i
FILE: "FILE"i
UPDATE: "UPDATE"i
REMOVE: "REMOVE"i
TAGS: "TAGS"i
login_user: LOGIN USER quoted_string (PASSWORD quoted_string)? ";"
list_services: LIST SERVICES ";"
show_service: SHOW SERVICE NUMBER ";"
startup_service: STARTUP SERVICE NUMBER ";"
shutdown_service: SHUTDOWN SERVICE NUMBER ";"
restart_service: RESTART SERVICE NUMBER ";"
register_user: REGISTER USER quoted_string AS quoted_string PASSWORD quoted_string ";"
list_users: LIST USERS ";"
drop_user: DROP USER quoted_string ";"
alter_user: ALTER USER PASSWORD quoted_string quoted_string ";"
show_user: SHOW USER quoted_string ";"
create_user: CREATE USER quoted_string quoted_string ";"
activate_user: ALTER USER ACTIVE quoted_string status ";"
list_datasets: LIST DATASETS OF quoted_string ";"
list_agents: LIST AGENTS OF quoted_string ";"
create_role: CREATE ROLE identifier [DESCRIPTION quoted_string] ";"
drop_role: DROP ROLE identifier ";"
alter_role: ALTER ROLE identifier SET DESCRIPTION quoted_string ";"
list_roles: LIST ROLES ";"
show_role: SHOW ROLE identifier ";"
grant_permission: GRANT identifier_list ON identifier TO ROLE identifier ";"
revoke_permission: REVOKE identifier_list ON identifier FROM ROLE identifier ";"
alter_user_role: ALTER USER quoted_string SET ROLE identifier ";"
show_user_permission: SHOW USER PERMISSION quoted_string ";"
show_version: SHOW VERSION ";"
grant_admin: GRANT ADMIN quoted_string ";"
revoke_admin: REVOKE ADMIN quoted_string ";"
generate_key: GENERATE KEY FOR USER quoted_string ";"
list_keys: LIST KEYS OF quoted_string ";"
drop_key: DROP KEY quoted_string OF quoted_string ";"
set_variable: SET VAR identifier identifier ";"
show_variable: SHOW VAR identifier ";"
list_variables: LIST VARS ";"
list_configs: LIST CONFIGS ";"
list_environments: LIST ENVS ";"
show_fingerprint: SHOW FINGERPRINT ";"
set_license: SET LICENSE quoted_string ";"
set_license_config: SET LICENSE CONFIG NUMBER NUMBER ";"
show_license: SHOW LICENSE ";"
check_license: CHECK LICENSE ";"
list_server_configs: LIST SERVER CONFIGS ";"
benchmark: BENCHMARK NUMBER NUMBER user_statement
user_statement: ping_server
| show_current_user
| create_model_provider
| drop_model_provider
| set_default_llm
| set_default_vlm
| set_default_embedding
| set_default_reranker
| set_default_asr
| set_default_tts
| reset_default_llm
| reset_default_vlm
| reset_default_embedding
| reset_default_reranker
| reset_default_asr
| reset_default_tts
| create_user_dataset_with_parser
| create_user_dataset_with_pipeline
| drop_user_dataset
| list_user_datasets
| list_user_dataset_files
| list_user_agents
| list_user_chats
| create_user_chat
| drop_user_chat
| list_user_model_providers
| list_user_default_models
| import_docs_into_dataset
| search_on_datasets
| update_chunk
| set_metadata
| remove_tags
| create_chat_session
| drop_chat_session
| list_chat_sessions
| chat_on_session
ping_server: PING ";"
show_current_user: SHOW CURRENT USER ";"
create_model_provider: CREATE MODEL PROVIDER quoted_string quoted_string ";"
drop_model_provider: DROP MODEL PROVIDER quoted_string ";"
set_default_llm: SET DEFAULT LLM quoted_string ";"
set_default_vlm: SET DEFAULT VLM quoted_string ";"
set_default_embedding: SET DEFAULT EMBEDDING quoted_string ";"
set_default_reranker: SET DEFAULT RERANKER quoted_string ";"
set_default_asr: SET DEFAULT ASR quoted_string ";"
set_default_tts: SET DEFAULT TTS quoted_string ";"
reset_default_llm: RESET DEFAULT LLM ";"
reset_default_vlm: RESET DEFAULT VLM ";"
reset_default_embedding: RESET DEFAULT EMBEDDING ";"
reset_default_reranker: RESET DEFAULT RERANKER ";"
reset_default_asr: RESET DEFAULT ASR ";"
reset_default_tts: RESET DEFAULT TTS ";"
list_user_datasets: LIST DATASETS ";"
create_user_dataset_with_parser: CREATE DATASET quoted_string WITH EMBEDDING quoted_string PARSER quoted_string ";"
create_user_dataset_with_pipeline: CREATE DATASET quoted_string WITH EMBEDDING quoted_string PIPELINE quoted_string ";"
drop_user_dataset: DROP DATASET quoted_string ";"
list_user_dataset_files: LIST FILES OF DATASET quoted_string ";"
list_user_dataset_documents: LIST DOCUMENTS OF DATASET quoted_string ";"
list_user_datasets_metadata: LIST METADATA OF DATASETS quoted_string (COMMA quoted_string)* ";"
list_user_documents_metadata_summary: LIST METADATA SUMMARY OF DATASET quoted_string (DOCUMENTS quoted_string (COMMA quoted_string)*)? ";"
list_user_agents: LIST AGENTS ";"
list_user_chats: LIST CHATS ";"
create_user_chat: CREATE CHAT quoted_string ";"
drop_user_chat: DROP CHAT quoted_string ";"
create_chat_session: CREATE CHAT quoted_string SESSION ";"
drop_chat_session: DROP CHAT quoted_string SESSION quoted_string ";"
list_chat_sessions: LIST CHAT quoted_string SESSIONS ";"
chat_on_session: CHAT quoted_string ON quoted_string SESSION quoted_string ";"
list_user_model_providers: LIST MODEL PROVIDERS ";"
list_user_default_models: LIST DEFAULT MODELS ";"
import_docs_into_dataset: IMPORT quoted_string INTO DATASET quoted_string ";"
search_on_datasets: SEARCH quoted_string ON DATASETS quoted_string ";"
get_chunk: GET CHUNK quoted_string ";"
list_chunks: LIST CHUNKS OF DOCUMENT quoted_string ("PAGE" NUMBER)? ("SIZE" NUMBER)? ("KEYWORDS" quoted_string)? ("AVAILABLE" NUMBER)? ";"
set_metadata: SET METADATA OF DOCUMENT quoted_string TO quoted_string ";"
remove_tags: REMOVE TAGS quoted_string (COMMA quoted_string)* FROM DATASET quoted_string ";"
remove_chunks: REMOVE CHUNKS quoted_string (COMMA quoted_string)* FROM DOCUMENT quoted_string ";"
| REMOVE ALL CHUNKS FROM DOCUMENT quoted_string ";"
parse_dataset_docs: PARSE quoted_string OF DATASET quoted_string ";"
parse_dataset_sync: PARSE DATASET quoted_string SYNC ";"
parse_dataset_async: PARSE DATASET quoted_string ASYNC ";"
// Internal CLI only for GO
create_dataset_table: CREATE DATASET TABLE quoted_string VECTOR SIZE NUMBER ";"
drop_dataset_table: DROP DATASET TABLE quoted_string ";"
create_metadata_table: CREATE METADATA TABLE ";"
drop_metadata_table: DROP METADATA TABLE ";"
insert_dataset_from_file: INSERT DATASET FROM FILE quoted_string ";"
insert_metadata_from_file: INSERT METADATA FROM FILE quoted_string ";"
update_chunk: UPDATE CHUNK quoted_string OF DATASET quoted_string SET quoted_string ";"
identifier_list: identifier (COMMA identifier)*
identifier: WORD
quoted_string: QUOTED_STRING
status: ON | WORD
QUOTED_STRING: /'[^']+'/ | /"[^"]+"/
WORD: /[a-zA-Z0-9_\-\.]+/
NUMBER: /[0-9]+/
%import common.WS
%ignore WS
"""
class RAGFlowCLITransformer(Transformer):
def start(self, items):
return items[0]
def command(self, items):
return items[0]
def login_user(self, items):
email = items[2].children[0].strip("'\"")
if len(items) == 5:
# With password: LOGIN USER email PASSWORD password
password = items[4].children[0].strip("'\"")
return {"type": "login_user", "email": email, "password": password}
else:
# Without password: LOGIN USER email
return {"type": "login_user", "email": email}
def ping_server(self, items):
return {"type": "ping_server"}
def list_services(self, items):
result = {"type": "list_services"}
return result
def show_service(self, items):
service_id = int(items[2])
return {"type": "show_service", "number": service_id}
def startup_service(self, items):
service_id = int(items[2])
return {"type": "startup_service", "number": service_id}
def shutdown_service(self, items):
service_id = int(items[2])
return {"type": "shutdown_service", "number": service_id}
def restart_service(self, items):
service_id = int(items[2])
return {"type": "restart_service", "number": service_id}
def register_user(self, items):
user_name: str = items[2].children[0].strip("'\"")
nickname: str = items[4].children[0].strip("'\"")
password: str = items[6].children[0].strip("'\"")
return {"type": "register_user", "user_name": user_name, "nickname": nickname, "password": password}
def list_users(self, items):
return {"type": "list_users"}
def show_user(self, items):
user_name = items[2]
return {"type": "show_user", "user_name": user_name}
def drop_user(self, items):
user_name = items[2]
return {"type": "drop_user", "user_name": user_name}
def alter_user(self, items):
user_name = items[3]
new_password = items[4]
return {"type": "alter_user", "user_name": user_name, "password": new_password}
def create_user(self, items):
user_name = items[2]
password = items[3]
return {"type": "create_user", "user_name": user_name, "password": password, "role": "user"}
def activate_user(self, items):
user_name = items[3]
activate_status = items[4]
return {"type": "activate_user", "activate_status": activate_status, "user_name": user_name}
def list_datasets(self, items):
user_name = items[3]
return {"type": "list_datasets", "user_name": user_name}
def list_agents(self, items):
user_name = items[3]
return {"type": "list_agents", "user_name": user_name}
def create_role(self, items):
role_name = items[2]
if len(items) > 4:
description = items[4]
return {"type": "create_role", "role_name": role_name, "description": description}
else:
return {"type": "create_role", "role_name": role_name}
def drop_role(self, items):
role_name = items[2]
return {"type": "drop_role", "role_name": role_name}
def alter_role(self, items):
role_name = items[2]
description = items[5]
return {"type": "alter_role", "role_name": role_name, "description": description}
def list_roles(self, items):
return {"type": "list_roles"}
def show_role(self, items):
role_name = items[2]
return {"type": "show_role", "role_name": role_name}
def grant_permission(self, items):
action_list = items[1]
resource = items[3]
role_name = items[6]
return {"type": "grant_permission", "role_name": role_name, "resource": resource, "actions": action_list}
def revoke_permission(self, items):
action_list = items[1]
resource = items[3]
role_name = items[6]
return {"type": "revoke_permission", "role_name": role_name, "resource": resource, "actions": action_list}
def alter_user_role(self, items):
user_name = items[2]
role_name = items[5]
return {"type": "alter_user_role", "user_name": user_name, "role_name": role_name}
def show_user_permission(self, items):
user_name = items[3]
return {"type": "show_user_permission", "user_name": user_name}
def show_version(self, items):
return {"type": "show_version"}
def grant_admin(self, items):
user_name = items[2]
return {"type": "grant_admin", "user_name": user_name}
def revoke_admin(self, items):
user_name = items[2]
return {"type": "revoke_admin", "user_name": user_name}
def generate_key(self, items):
user_name = items[4]
return {"type": "generate_key", "user_name": user_name}
def list_keys(self, items):
user_name = items[3]
return {"type": "list_keys", "user_name": user_name}
def drop_key(self, items):
key = items[2]
user_name = items[4]
return {"type": "drop_key", "key": key, "user_name": user_name}
def set_variable(self, items):
var_name = items[2]
var_value = items[3]
return {"type": "set_variable", "var_name": var_name, "var_value": var_value}
def show_variable(self, items):
var_name = items[2]
return {"type": "show_variable", "var_name": var_name}
def list_variables(self, items):
return {"type": "list_variables"}
def list_configs(self, items):
return {"type": "list_configs"}
def list_environments(self, items):
return {"type": "list_environments"}
def show_fingerprint(self, items):
return {"type": "show_fingerprint"}
def set_license(self, items):
license = items[2].children[0].strip("'\"")
return {"type": "set_license", "license": license}
def set_license_config(self, items):
value1: int = int(items[3])
value2: int = int(items[4])
return {"type": "set_license_config", "value1": value1, "value2": value2}
def show_license(self, items):
return {"type": "show_license"}
def check_license(self, items):
return {"type": "check_license"}
def list_server_configs(self, items):
return {"type": "list_server_configs"}
def create_model_provider(self, items):
provider_name = items[3].children[0].strip("'\"")
provider_key = items[4].children[0].strip("'\"")
return {"type": "create_model_provider", "provider_name": provider_name, "provider_key": provider_key}
def drop_model_provider(self, items):
provider_name = items[3].children[0].strip("'\"")
return {"type": "drop_model_provider", "provider_name": provider_name}
def show_current_user(self, items):
return {"type": "show_current_user"}
def set_default_llm(self, items):
llm_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "llm_id", "model_id": llm_id}
def set_default_vlm(self, items):
vlm_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "img2txt_id", "model_id": vlm_id}
def set_default_embedding(self, items):
embedding_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "embd_id", "model_id": embedding_id}
def set_default_reranker(self, items):
reranker_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "reranker_id", "model_id": reranker_id}
def set_default_asr(self, items):
asr_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "asr_id", "model_id": asr_id}
def set_default_tts(self, items):
tts_id = items[3].children[0].strip("'\"")
return {"type": "set_default_model", "model_type": "tts_id", "model_id": tts_id}
def reset_default_llm(self, items):
return {"type": "reset_default_model", "model_type": "llm_id"}
def reset_default_vlm(self, items):
return {"type": "reset_default_model", "model_type": "img2txt_id"}
def reset_default_embedding(self, items):
return {"type": "reset_default_model", "model_type": "embd_id"}
def reset_default_reranker(self, items):
return {"type": "reset_default_model", "model_type": "reranker_id"}
def reset_default_asr(self, items):
return {"type": "reset_default_model", "model_type": "asr_id"}
def reset_default_tts(self, items):
return {"type": "reset_default_model", "model_type": "tts_id"}
def list_user_datasets(self, items):
return {"type": "list_user_datasets"}
def create_user_dataset_with_parser(self, items):
dataset_name = items[2].children[0].strip("'\"")
embedding = items[5].children[0].strip("'\"")
parser_type = items[7].children[0].strip("'\"")
return {"type": "create_user_dataset", "dataset_name": dataset_name, "embedding": embedding,
"parser_type": parser_type}
def create_user_dataset_with_pipeline(self, items):
dataset_name = items[2].children[0].strip("'\"")
embedding = items[5].children[0].strip("'\"")
pipeline = items[7].children[0].strip("'\"")
return {"type": "create_user_dataset", "dataset_name": dataset_name, "embedding": embedding,
"pipeline": pipeline}
def drop_user_dataset(self, items):
dataset_name = items[2].children[0].strip("'\"")
return {"type": "drop_user_dataset", "dataset_name": dataset_name}
def list_user_dataset_files(self, items):
dataset_name = items[4].children[0].strip("'\"")
return {"type": "list_user_dataset_files", "dataset_name": dataset_name}
def list_user_dataset_documents(self, items):
dataset_name = items[4].children[0].strip("'\"")
return {"type": "list_user_dataset_documents", "dataset_name": dataset_name}
def list_user_datasets_metadata(self, items):
dataset_names = []
dataset_names.append(items[4].children[0].strip("'\""))
for i in range(5, len(items)):
if items[i] and hasattr(items[i], 'children') and items[i].children:
dataset_names.append(items[i].children[0].strip("'\""))
return {"type": "list_user_datasets_metadata", "dataset_names": dataset_names}
def list_user_documents_metadata_summary(self, items):
dataset_name = items[5].children[0].strip("'\"")
doc_ids = []
if len(items) > 6 and items[6] == "DOCUMENTS":
for i in range(7, len(items)):
if items[i] and hasattr(items[i], 'children') and items[i].children:
doc_id = items[i].children[0].strip("'\"")
doc_ids.append(doc_id)
return {"type": "list_user_documents_metadata_summary", "dataset_name": dataset_name, "document_ids": doc_ids}
def list_user_agents(self, items):
return {"type": "list_user_agents"}
def list_user_chats(self, items):
return {"type": "list_user_chats"}
def create_user_chat(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "create_user_chat", "chat_name": chat_name}
def drop_user_chat(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "drop_user_chat", "chat_name": chat_name}
def create_dataset_table(self, items):
dataset_name = None
vector_size = None
for i, item in enumerate(items):
if hasattr(item, 'data') and item.data == 'quoted_string':
dataset_name = item.children[0].strip("'\"")
if hasattr(item, 'type') and item.type == 'NUMBER':
if i > 0 and items[i-1].type == 'SIZE' and items[i-2].type == 'VECTOR':
vector_size = int(item)
return {"type": "create_dataset_table", "dataset_name": dataset_name, "vector_size": vector_size}
def drop_dataset_table(self, items):
dataset_name = None
for item in items:
if hasattr(item, 'data') and item.data == 'quoted_string':
dataset_name = item.children[0].strip("'\"")
return {"type": "drop_dataset_table", "dataset_name": dataset_name}
def create_metadata_table(self, items):
return {"type": "create_metadata_table"}
def drop_metadata_table(self, items):
return {"type": "drop_metadata_table"}
def list_user_model_providers(self, items):
return {"type": "list_user_model_providers"}
def list_user_default_models(self, items):
return {"type": "list_user_default_models"}
def parse_dataset_docs(self, items):
document_list_str = items[1].children[0].strip("'\"")
document_names = document_list_str.split(",")
if len(document_names) == 1:
document_names = document_names[0]
document_names = document_names.split(" ")
dataset_name = items[4].children[0].strip("'\"")
return {"type": "parse_dataset_docs", "dataset_name": dataset_name, "document_names": document_names}
def parse_dataset_sync(self, items):
dataset_name = items[2].children[0].strip("'\"")
return {"type": "parse_dataset", "dataset_name": dataset_name, "method": "sync"}
def parse_dataset_async(self, items):
dataset_name = items[2].children[0].strip("'\"")
return {"type": "parse_dataset", "dataset_name": dataset_name, "method": "async"}
def create_chat_session(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "create_chat_session", "chat_name": chat_name}
def drop_chat_session(self, items):
chat_name = items[2].children[0].strip("'\"")
session_id = items[4].children[0].strip("'\"")
return {"type": "drop_chat_session", "chat_name": chat_name, "session_id": session_id}
def list_chat_sessions(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "list_chat_sessions", "chat_name": chat_name}
def chat_on_session(self, items):
message = items[1].children[0].strip("'\"")
chat_name = items[3].children[0].strip("'\"")
session_id = items[5].children[0].strip("'\"")
return {"type": "chat_on_session", "message": message, "chat_name": chat_name, "session_id": session_id}
def import_docs_into_dataset(self, items):
document_list_str = items[1].children[0].strip("'\"")
document_paths = document_list_str.split(",")
if len(document_paths) == 1:
document_paths = document_paths[0]
document_paths = document_paths.split(" ")
dataset_name = items[4].children[0].strip("'\"")
return {"type": "import_docs_into_dataset", "dataset_name": dataset_name, "document_paths": document_paths}
def search_on_datasets(self, items):
question = items[1].children[0].strip("'\"")
datasets_str = items[4].children[0].strip("'\"")
datasets = datasets_str.split(",")
if len(datasets) == 1:
datasets = datasets[0]
datasets = datasets.split(" ")
return {"type": "search_on_datasets", "datasets": datasets, "question": question}
def get_chunk(self, items):
chunk_id = items[2].children[0].strip("'\"")
return {"type": "get_chunk", "chunk_id": chunk_id}
def insert_dataset_from_file(self, items):
file_path = items[4].children[0].strip("'\"")
return {"type": "insert_dataset_from_file", "file_path": file_path}
def insert_metadata_from_file(self, items):
file_path = items[4].children[0].strip("'\"")
return {"type": "insert_metadata_from_file", "file_path": file_path}
def update_chunk(self, items):
def get_quoted_value(item):
if hasattr(item, 'children') and item.children:
return item.children[0].strip("'\"")
return str(item).strip("'\"")
chunk_id = get_quoted_value(items[2])
dataset_name = get_quoted_value(items[5])
json_body = get_quoted_value(items[7])
return {"type": "update_chunk", "chunk_id": chunk_id, "dataset_name": dataset_name, "json_body": json_body}
def set_metadata(self, items):
doc_id = items[4].children[0].strip("'\"")
meta_json = items[6].children[0].strip("'\"")
return {"type": "set_metadata", "doc_id": doc_id, "meta": meta_json}
def remove_tags(self, items):
# items: REMOVE, TAGS, quoted_string(tag1), quoted_string(tag2), ..., FROM, DATASET, quoted_string(dataset_name), ";"
tags = []
# Start from index 2 (after TAGS keyword) and parse quoted strings until FROM
for i in range(2, len(items)):
item = items[i]
# Check for FROM token to stop
if hasattr(item, 'type') and item.type == 'FROM':
break
if hasattr(item, 'children') and item.children:
tag = item.children[0].strip("'\"")
tags.append(tag)
# Find dataset_name: quoted_string after DATASET
dataset_name = None
for i, item in enumerate(items):
# Check if item is a DATASET token
if hasattr(item, 'type') and item.type == 'DATASET':
# Next item should be quoted_string
dataset_name = items[i + 1].children[0].strip("'\"")
break
return {"type": "remove_tags", "dataset_name": dataset_name, "tags": tags}
def remove_chunks(self, items):
# Handle two cases:
# 1. REMOVE CHUNKS quoted_string (COMMA quoted_string)* FROM DOCUMENT quoted_string ";"
# 2. REMOVE ALL CHUNKS FROM DOCUMENT quoted_string ";"
# Check if it's "REMOVE ALL CHUNKS"
for item in items:
if hasattr(item, 'type') and item.type == 'ALL':
# Find doc_id
for j, inner_item in enumerate(items):
if hasattr(inner_item, 'type') and inner_item.type == 'DOCUMENT':
doc_id = items[j + 1].children[0].strip("'\"")
return {"type": "remove_chunks", "doc_id": doc_id, "delete_all": True}
# Otherwise, we have chunk_ids
chunk_ids = []
doc_id = None
for i, item in enumerate(items):
if hasattr(item, 'type') and item.type == 'DOCUMENT':
doc_id = items[i + 1].children[0].strip("'\"")
elif hasattr(item, 'children') and item.children:
val = item.children[0].strip("'\"")
# Skip if it's "FROM" or "DOCUMENT"
if val.upper() in ['FROM', 'DOCUMENT']:
continue
chunk_ids.append(val)
return {"type": "remove_chunks", "doc_id": doc_id, "chunk_ids": chunk_ids}
def list_chunks(self, items):
doc_id = items[4].children[0].strip("'\"")
result = {"type": "list_chunks", "doc_id": doc_id}
# Parse optional parameters: PAGE, SIZE, KEYWORDS, AVAILABLE
# items structure varies based on which params are present
for i, item in enumerate(items):
if str(item) == "PAGE":
result["page"] = int(items[i + 1])
elif str(item) == "SIZE":
result["size"] = int(items[i + 1])
elif str(item) == "KEYWORDS":
result["keywords"] = items[i + 1].children[0].strip("'\"")
elif str(item) == "AVAILABLE":
result["available_int"] = int(items[i + 1])
return result
def benchmark(self, items):
concurrency: int = int(items[1])
iterations: int = int(items[2])
command = items[3].children[0]
return {"type": "benchmark", "concurrency": concurrency, "iterations": iterations, "command": command}
def action_list(self, items):
return items
def meta_command(self, items):
command_name = str(items[0]).lower()
args = items[1:] if len(items) > 1 else []
# handle quoted parameter
parsed_args = []
for arg in args:
if hasattr(arg, "value"):
parsed_args.append(arg.value)
else:
parsed_args.append(str(arg))
return {"type": "meta", "command": command_name, "args": parsed_args}
def meta_command_name(self, items):
return items[0]
def meta_args(self, items):
return items

View File

@@ -1,6 +1,6 @@
[project]
name = "ragflow-cli"
version = "0.23.1"
version = "0.25.3"
description = "Admin Service's client of [RAGFlow](https://github.com/infiniflow/ragflow). The Admin Service provides user management and system monitoring. "
authors = [{ name = "Lynn", email = "lynn_inf@hotmail.com" }]
license = { text = "Apache License, Version 2.0" }
@@ -11,14 +11,17 @@ dependencies = [
"beartype>=0.20.0,<1.0.0",
"pycryptodomex>=3.10.0",
"lark>=1.1.0",
"requests-toolbelt>=1.0.0",
]
[dependency-groups]
test = [
"pytest>=8.3.5",
"requests>=2.32.3",
"requests-toolbelt>=1.0.0",
]
[tool.setuptools]
py-modules = ["ragflow_cli", "parser", "http_client", "ragflow_client", "user"]
[project.scripts]
ragflow-cli = "admin_client:main"
ragflow-cli = "ragflow_cli:main"

347
admin/client/ragflow_cli.py Normal file
View File

@@ -0,0 +1,347 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
import argparse
import base64
import getpass
import os
import atexit
import readline
from cmd import Cmd
from typing import Any, Dict, List
import requests
import warnings
from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5
from Cryptodome.PublicKey import RSA
from lark import Lark, Tree
from parser import GRAMMAR, RAGFlowCLITransformer
from http_client import HttpClient
from ragflow_client import RAGFlowClient, run_command
from user import login_user
warnings.filterwarnings("ignore", category=getpass.GetPassWarning)
def encrypt(input_string):
pub = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----"
pub_key = RSA.importKey(pub)
cipher = Cipher_pkcs1_v1_5.new(pub_key)
cipher_text = cipher.encrypt(base64.b64encode(input_string.encode("utf-8")))
return base64.b64encode(cipher_text).decode("utf-8")
def encode_to_base64(input_string):
base64_encoded = base64.b64encode(input_string.encode("utf-8"))
return base64_encoded.decode("utf-8")
class RAGFlowCLI(Cmd):
def __init__(self):
super().__init__()
self.parser = Lark(GRAMMAR, start="start", parser="lalr", transformer=RAGFlowCLITransformer())
self.command_history = []
self.account = "admin@ragflow.io"
self.account_password: str = "admin"
self.session = requests.Session()
self.host: str = ""
self.port: int = 0
self.mode: str = "admin"
self.ragflow_client = None
# History file for readline persistence
self.history_file = os.path.expanduser("~/.ragflow_cli_history")
# Load existing history
self._load_history()
# Register cleanup to save history on exit
atexit.register(self._save_history)
intro = r"""Type "\h" for help."""
prompt = "ragflow> "
def onecmd(self, command: str) -> bool:
try:
result = self.parse_command(command)
if isinstance(result, dict):
if "type" in result and result.get("type") == "empty":
return False
self.execute_command(result)
if isinstance(result, Tree):
return False
if result.get("type") == "meta" and result.get("command") in ["q", "quit", "exit"]:
return True
except KeyboardInterrupt:
print("\nUse '\\q' to quit")
except EOFError:
print("\nGoodbye!")
return True
return False
def emptyline(self) -> bool:
return False
def default(self, line: str) -> bool:
return self.onecmd(line)
def parse_command(self, command_str: str) -> dict[str, str]:
if not command_str.strip():
return {"type": "empty"}
self.command_history.append(command_str)
readline.add_history(command_str)
try:
result = self.parser.parse(command_str)
return result
except Exception as e:
return {"type": "error", "message": f"Parse error: {str(e)}"}
def verify_auth(self, arguments: dict, single_command: bool, auth: bool):
server_type = arguments.get("type", "admin")
http_client = HttpClient(arguments["host"], arguments["port"])
if not auth:
self.ragflow_client = RAGFlowClient(http_client, server_type)
return True
user_name = arguments["username"]
attempt_count = 3
if single_command:
attempt_count = 1
try_count = 0
while True:
try_count += 1
if try_count > attempt_count:
return False
if single_command:
user_password = arguments["password"]
else:
user_password = getpass.getpass(f"password for {user_name}: ").strip()
try:
token = login_user(http_client, server_type, user_name, user_password)
http_client.login_token = token
self.ragflow_client = RAGFlowClient(http_client, server_type)
return True
except Exception as e:
print(str(e))
print("Can't access server for login (connection failed)")
def _format_service_detail_table(self, data):
if isinstance(data, list):
return data
if not all([isinstance(v, list) for v in data.values()]):
# normal table
return data
# handle task_executor heartbeats map, for example {'name': [{'done': 2, 'now': timestamp1}, {'done': 3, 'now': timestamp2}]
task_executor_list = []
for k, v in data.items():
# display latest status
heartbeats = sorted(v, key=lambda x: x["now"], reverse=True)
task_executor_list.append(
{
"task_executor_name": k,
**heartbeats[0],
}
if heartbeats
else {"task_executor_name": k}
)
return task_executor_list
def _print_table_simple(self, data):
if not data:
print("No data to print")
return
if isinstance(data, dict):
# handle single row data
data = [data]
columns = list(set().union(*(d.keys() for d in data)))
columns.sort()
col_widths = {}
def get_string_width(text):
half_width_chars = " !\"#$%&'()*+,-./0123456789:;<=>?@ABCDEFGHIJKLMNOPQRSTUVWXYZ[\\]^_`abcdefghijklmnopqrstuvwxyz{|}~\t\n\r"
width = 0
for char in text:
if char in half_width_chars:
width += 1
else:
width += 2
return width
for col in columns:
max_width = get_string_width(str(col))
for item in data:
value_len = get_string_width(str(item.get(col, "")))
if value_len > max_width:
max_width = value_len
col_widths[col] = max(2, max_width)
# Generate delimiter
separator = "+" + "+".join(["-" * (col_widths[col] + 2) for col in columns]) + "+"
# Print header
print(separator)
header = "|" + "|".join([f" {col:<{col_widths[col]}} " for col in columns]) + "|"
print(header)
print(separator)
# Print data
for item in data:
row = "|"
for col in columns:
value = str(item.get(col, ""))
if get_string_width(value) > col_widths[col]:
value = value[: col_widths[col] - 3] + "..."
row += f" {value:<{col_widths[col] - (get_string_width(value) - len(value))}} |"
print(row)
print(separator)
def _load_history(self):
"""Load command history from file."""
try:
if os.path.exists(self.history_file):
readline.read_history_file(self.history_file)
except Exception:
pass # Ignore errors loading history
def _save_history(self):
"""Save command history to file."""
try:
readline.write_history_file(self.history_file)
except Exception:
pass # Ignore errors saving history
def run_interactive(self, args):
if self.verify_auth(args, single_command=False, auth=args["auth"]):
print(r"""
____ ___ ______________ ________ ____
/ __ \/ | / ____/ ____/ /___ _ __ / ____/ / / _/
/ /_/ / /| |/ / __/ /_ / / __ \ | /| / / / / / / / /
/ _, _/ ___ / /_/ / __/ / / /_/ / |/ |/ / / /___/ /____/ /
/_/ |_/_/ |_\____/_/ /_/\____/|__/|__/ \____/_____/___/
""")
self.cmdloop()
print("RAGFlow command line interface - Type '\\?' for help, '\\q' to quit")
def run_single_command(self, args):
if self.verify_auth(args, single_command=True, auth=args["auth"]):
command = args["command"]
result = self.parse_command(command)
self.execute_command(result)
def parse_connection_args(self, args: List[str]) -> Dict[str, Any]:
parser = argparse.ArgumentParser(description="RAGFlow CLI Client", add_help=False)
parser.add_argument("-h", "--host", default="127.0.0.1", help="Admin or RAGFlow service host")
parser.add_argument("-p", "--port", type=int, default=9381, help="Admin or RAGFlow service port")
parser.add_argument("-w", "--password", default="admin", type=str, help="Superuser password")
parser.add_argument("-t", "--type", default="admin", type=str, help="CLI mode, admin or user")
parser.add_argument("-u", "--username", default=None,
help="Username (email). In admin mode defaults to admin@ragflow.io, in user mode required.")
parser.add_argument("command", nargs="?", help="Single command")
try:
parsed_args, remaining_args = parser.parse_known_args(args)
# Determine username based on mode
username = parsed_args.username
if parsed_args.type == "admin":
if username is None:
username = "admin@ragflow.io"
if remaining_args:
if remaining_args[0] == "command":
command_str = ' '.join(remaining_args[1:]) + ';'
auth = True
if remaining_args[1] == "register":
auth = False
else:
if username is None:
print("Error: username (-u) is required in user mode")
return {"error": "Username required"}
return {
"host": parsed_args.host,
"port": parsed_args.port,
"password": parsed_args.password,
"type": parsed_args.type,
"username": username,
"command": command_str,
"auth": auth
}
else:
return {"error": "Invalid command"}
else:
auth = True
if username is None:
auth = False
return {
"host": parsed_args.host,
"port": parsed_args.port,
"type": parsed_args.type,
"username": username,
"auth": auth
}
except SystemExit:
return {"error": "Invalid connection arguments"}
def execute_command(self, parsed_command: Dict[str, Any]):
command_dict: dict
if isinstance(parsed_command, Tree):
command_dict = parsed_command.children[0]
else:
if parsed_command["type"] == "error":
print(f"Error: {parsed_command['message']}")
return
else:
command_dict = parsed_command
# print(f"Parsed command: {command_dict}")
run_command(self.ragflow_client, command_dict)
def main():
cli = RAGFlowCLI()
args = cli.parse_connection_args(sys.argv)
if "error" in args:
print("Error: Invalid connection arguments")
return
if "command" in args:
# single command mode
# for user mode, api key or password is ok
# for admin mode, only password
if "password" not in args:
print("Error: password is missing")
return
cli.run_single_command(args)
else:
cli.run_interactive(args)
if __name__ == "__main__":
main()

File diff suppressed because it is too large Load Diff

77
admin/client/user.py Normal file
View File

@@ -0,0 +1,77 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from http_client import HttpClient
class AuthException(Exception):
def __init__(self, message, code=401):
super().__init__(message)
self.code = code
self.message = message
def encrypt_password(password_plain: str) -> str:
try:
import base64
from Cryptodome.PublicKey import RSA
from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5
def crypt(line):
"""
decrypt(crypt(input_string)) == base64(input_string), which frontend and ragflow_cli use.
"""
pub = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----"
rsa_key = RSA.importKey(pub)
cipher = Cipher_pkcs1_v1_5.new(rsa_key)
password_base64 = base64.b64encode(line.encode('utf-8')).decode("utf-8")
encrypted_password = cipher.encrypt(password_base64.encode())
return base64.b64encode(encrypted_password).decode('utf-8')
except Exception as exc:
raise AuthException(
"Password encryption unavailable; install pycryptodomex (uv sync --python 3.12 --group test)."
) from exc
return crypt(password_plain)
def register_user(client: HttpClient, email: str, nickname: str, password: str) -> None:
password_enc = encrypt_password(password)
payload = {"email": email, "nickname": nickname, "password": password_enc}
res = client.request_json("POST", "/user/register", use_api_base=False, auth_kind=None, json_body=payload)
if res.get("code") == 0:
return
msg = res.get("message", "")
if "has already registered" in msg:
return
raise AuthException(f"Register failed: {msg}")
def login_user(client: HttpClient, server_type: str, email: str, password: str) -> str:
password_enc = encrypt_password(password)
payload = {"email": email, "password": password_enc}
if server_type == "admin":
response = client.request("POST", "/admin/login", use_api_base=True, auth_kind=None, json_body=payload)
else:
response = client.request("POST", "/user/login", use_api_base=False, auth_kind=None, json_body=payload)
try:
res = response.json()
except Exception as exc:
raise AuthException(f"Login failed: invalid JSON response ({exc})") from exc
if res.get("code") != 0:
raise AuthException(f"Login failed: {res.get('message')}")
token = response.headers.get("Authorization")
if not token:
raise AuthException("Login failed: missing Authorization header")
return token

138
admin/client/uv.lock generated
View File

@@ -1,6 +1,6 @@
version = 1
revision = 3
requires-python = ">=3.10, <3.13"
requires-python = ">=3.12, <3.15"
[[package]]
name = "beartype"
@@ -26,38 +26,6 @@ version = "3.4.4"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/13/69/33ddede1939fdd074bce5434295f38fae7136463422fe4fd3e0e89b98062/charset_normalizer-3.4.4.tar.gz", hash = "sha256:94537985111c35f28720e43603b8e7b43a6ecfb2ce1d3058bbe955b73404e21a", size = 129418, upload-time = "2025-10-14T04:42:32.879Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/b8/6d51fc1d52cbd52cd4ccedd5b5b2f0f6a11bbf6765c782298b0f3e808541/charset_normalizer-3.4.4-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:e824f1492727fa856dd6eda4f7cee25f8518a12f3c4a56a74e8095695089cf6d", size = 209709, upload-time = "2025-10-14T04:40:11.385Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/5c/af/1f9d7f7faafe2ddfb6f72a2e07a548a629c61ad510fe60f9630309908fef/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4bd5d4137d500351a30687c2d3971758aac9a19208fc110ccb9d7188fbe709e8", size = 148814, upload-time = "2025-10-14T04:40:13.135Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/79/3d/f2e3ac2bbc056ca0c204298ea4e3d9db9b4afe437812638759db2c976b5f/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:027f6de494925c0ab2a55eab46ae5129951638a49a34d87f4c3eda90f696b4ad", size = 144467, upload-time = "2025-10-14T04:40:14.728Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ec/85/1bf997003815e60d57de7bd972c57dc6950446a3e4ccac43bc3070721856/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f820802628d2694cb7e56db99213f930856014862f3fd943d290ea8438d07ca8", size = 162280, upload-time = "2025-10-14T04:40:16.14Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3e/8e/6aa1952f56b192f54921c436b87f2aaf7c7a7c3d0d1a765547d64fd83c13/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:798d75d81754988d2565bff1b97ba5a44411867c0cf32b77a7e8f8d84796b10d", size = 159454, upload-time = "2025-10-14T04:40:17.567Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/36/3b/60cbd1f8e93aa25d1c669c649b7a655b0b5fb4c571858910ea9332678558/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9d1bb833febdff5c8927f922386db610b49db6e0d4f4ee29601d71e7c2694313", size = 153609, upload-time = "2025-10-14T04:40:19.08Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/64/91/6a13396948b8fd3c4b4fd5bc74d045f5637d78c9675585e8e9fbe5636554/charset_normalizer-3.4.4-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9cd98cdc06614a2f768d2b7286d66805f94c48cde050acdbbb7db2600ab3197e", size = 151849, upload-time = "2025-10-14T04:40:20.607Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/7a/59482e28b9981d105691e968c544cc0df3b7d6133152fb3dcdc8f135da7a/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:077fbb858e903c73f6c9db43374fd213b0b6a778106bc7032446a8e8b5b38b93", size = 151586, upload-time = "2025-10-14T04:40:21.719Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/92/59/f64ef6a1c4bdd2baf892b04cd78792ed8684fbc48d4c2afe467d96b4df57/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:244bfb999c71b35de57821b8ea746b24e863398194a4014e4c76adc2bbdfeff0", size = 145290, upload-time = "2025-10-14T04:40:23.069Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6b/63/3bf9f279ddfa641ffa1962b0db6a57a9c294361cc2f5fcac997049a00e9c/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:64b55f9dce520635f018f907ff1b0df1fdc31f2795a922fb49dd14fbcdf48c84", size = 163663, upload-time = "2025-10-14T04:40:24.17Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/09/c9e38fc8fa9e0849b172b581fd9803bdf6e694041127933934184e19f8c3/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:faa3a41b2b66b6e50f84ae4a68c64fcd0c44355741c6374813a800cd6695db9e", size = 151964, upload-time = "2025-10-14T04:40:25.368Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d2/d1/d28b747e512d0da79d8b6a1ac18b7ab2ecfd81b2944c4c710e166d8dd09c/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6515f3182dbe4ea06ced2d9e8666d97b46ef4c75e326b79bb624110f122551db", size = 161064, upload-time = "2025-10-14T04:40:26.806Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/bb/9a/31d62b611d901c3b9e5500c36aab0ff5eb442043fb3a1c254200d3d397d9/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:cc00f04ed596e9dc0da42ed17ac5e596c6ccba999ba6bd92b0e0aef2f170f2d6", size = 155015, upload-time = "2025-10-14T04:40:28.284Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/f3/107e008fa2bff0c8b9319584174418e5e5285fef32f79d8ee6a430d0039c/charset_normalizer-3.4.4-cp310-cp310-win32.whl", hash = "sha256:f34be2938726fc13801220747472850852fe6b1ea75869a048d6f896838c896f", size = 99792, upload-time = "2025-10-14T04:40:29.613Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/eb/66/e396e8a408843337d7315bab30dbf106c38966f1819f123257f5520f8a96/charset_normalizer-3.4.4-cp310-cp310-win_amd64.whl", hash = "sha256:a61900df84c667873b292c3de315a786dd8dac506704dea57bc957bd31e22c7d", size = 107198, upload-time = "2025-10-14T04:40:30.644Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b5/58/01b4f815bf0312704c267f2ccb6e5d42bcc7752340cd487bc9f8c3710597/charset_normalizer-3.4.4-cp310-cp310-win_arm64.whl", hash = "sha256:cead0978fc57397645f12578bfd2d5ea9138ea0fac82b2f63f7f7c6877986a69", size = 100262, upload-time = "2025-10-14T04:40:32.108Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/27/c6491ff4954e58a10f69ad90aca8a1b6fe9c5d3c6f380907af3c37435b59/charset_normalizer-3.4.4-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6e1fcf0720908f200cd21aa4e6750a48ff6ce4afe7ff5a79a90d5ed8a08296f8", size = 206988, upload-time = "2025-10-14T04:40:33.79Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/94/59/2e87300fe67ab820b5428580a53cad894272dbb97f38a7a814a2a1ac1011/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f819d5fe9234f9f82d75bdfa9aef3a3d72c4d24a6e57aeaebba32a704553aa0", size = 147324, upload-time = "2025-10-14T04:40:34.961Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/07/fb/0cf61dc84b2b088391830f6274cb57c82e4da8bbc2efeac8c025edb88772/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a59cb51917aa591b1c4e6a43c132f0cdc3c76dbad6155df4e28ee626cc77a0a3", size = 142742, upload-time = "2025-10-14T04:40:36.105Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/62/8b/171935adf2312cd745d290ed93cf16cf0dfe320863ab7cbeeae1dcd6535f/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8ef3c867360f88ac904fd3f5e1f902f13307af9052646963ee08ff4f131adafc", size = 160863, upload-time = "2025-10-14T04:40:37.188Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/09/73/ad875b192bda14f2173bfc1bc9a55e009808484a4b256748d931b6948442/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d9e45d7faa48ee908174d8fe84854479ef838fc6a705c9315372eacbc2f02897", size = 157837, upload-time = "2025-10-14T04:40:38.435Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6d/fc/de9cce525b2c5b94b47c70a4b4fb19f871b24995c728e957ee68ab1671ea/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:840c25fb618a231545cbab0564a799f101b63b9901f2569faecd6b222ac72381", size = 151550, upload-time = "2025-10-14T04:40:40.053Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/55/c2/43edd615fdfba8c6f2dfbd459b25a6b3b551f24ea21981e23fb768503ce1/charset_normalizer-3.4.4-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:ca5862d5b3928c4940729dacc329aa9102900382fea192fc5e52eb69d6093815", size = 149162, upload-time = "2025-10-14T04:40:41.163Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/03/86/bde4ad8b4d0e9429a4e82c1e8f5c659993a9a863ad62c7df05cf7b678d75/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d9c7f57c3d666a53421049053eaacdd14bbd0a528e2186fcb2e672effd053bb0", size = 150019, upload-time = "2025-10-14T04:40:42.276Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/86/a151eb2af293a7e7bac3a739b81072585ce36ccfb4493039f49f1d3cae8c/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:277e970e750505ed74c832b4bf75dac7476262ee2a013f5574dd49075879e161", size = 143310, upload-time = "2025-10-14T04:40:43.439Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b5/fe/43dae6144a7e07b87478fdfc4dbe9efd5defb0e7ec29f5f58a55aeef7bf7/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:31fd66405eaf47bb62e8cd575dc621c56c668f27d46a61d975a249930dd5e2a4", size = 162022, upload-time = "2025-10-14T04:40:44.547Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/80/e6/7aab83774f5d2bca81f42ac58d04caf44f0cc2b65fc6db2b3b2e8a05f3b3/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:0d3d8f15c07f86e9ff82319b3d9ef6f4bf907608f53fe9d92b28ea9ae3d1fd89", size = 149383, upload-time = "2025-10-14T04:40:46.018Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4f/e8/b289173b4edae05c0dde07f69f8db476a0b511eac556dfe0d6bda3c43384/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:9f7fcd74d410a36883701fafa2482a6af2ff5ba96b9a620e9e0721e28ead5569", size = 159098, upload-time = "2025-10-14T04:40:47.081Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d8/df/fe699727754cae3f8478493c7f45f777b17c3ef0600e28abfec8619eb49c/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ebf3e58c7ec8a8bed6d66a75d7fb37b55e5015b03ceae72a8e7c74495551e224", size = 152991, upload-time = "2025-10-14T04:40:48.246Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1a/86/584869fe4ddb6ffa3bd9f491b87a01568797fb9bd8933f557dba9771beaf/charset_normalizer-3.4.4-cp311-cp311-win32.whl", hash = "sha256:eecbc200c7fd5ddb9a7f16c7decb07b566c29fa2161a16cf67b8d068bd21690a", size = 99456, upload-time = "2025-10-14T04:40:49.376Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/65/f6/62fdd5feb60530f50f7e38b4f6a1d5203f4d16ff4f9f0952962c044e919a/charset_normalizer-3.4.4-cp311-cp311-win_amd64.whl", hash = "sha256:5ae497466c7901d54b639cf42d5b8c1b6a4fead55215500d2f486d34db48d016", size = 106978, upload-time = "2025-10-14T04:40:50.844Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/9d/0710916e6c82948b3be62d9d398cb4fcf4e97b56d6a6aeccd66c4b2f2bd5/charset_normalizer-3.4.4-cp311-cp311-win_arm64.whl", hash = "sha256:65e2befcd84bc6f37095f5961e68a6f077bf44946771354a28ad434c2cce0ae1", size = 99969, upload-time = "2025-10-14T04:40:52.272Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f3/85/1637cd4af66fa687396e757dec650f28025f2a2f5a5531a3208dc0ec43f2/charset_normalizer-3.4.4-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:0a98e6759f854bd25a58a73fa88833fba3b7c491169f86ce1180c948ab3fd394", size = 208425, upload-time = "2025-10-14T04:40:53.353Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9d/6a/04130023fef2a0d9c62d0bae2649b69f7b7d8d24ea5536feef50551029df/charset_normalizer-3.4.4-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b5b290ccc2a263e8d185130284f8501e3e36c5e02750fc6b6bdeb2e9e96f1e25", size = 148162, upload-time = "2025-10-14T04:40:54.558Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/78/29/62328d79aa60da22c9e0b9a66539feae06ca0f5a4171ac4f7dc285b83688/charset_normalizer-3.4.4-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74bb723680f9f7a6234dcf67aea57e708ec1fbdf5699fb91dfd6f511b0a320ef", size = 144558, upload-time = "2025-10-14T04:40:55.677Z" },
@@ -74,6 +42,38 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a8/ef/89297262b8092b312d29cdb2517cb1237e51db8ecef2e9af5edbe7b683b1/charset_normalizer-3.4.4-cp312-cp312-win32.whl", hash = "sha256:5833d2c39d8896e4e19b689ffc198f08ea58116bee26dea51e362ecc7cd3ed26", size = 99694, upload-time = "2025-10-14T04:41:09.23Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3d/2d/1e5ed9dd3b3803994c155cd9aacb60c82c331bad84daf75bcb9c91b3295e/charset_normalizer-3.4.4-cp312-cp312-win_amd64.whl", hash = "sha256:a79cfe37875f822425b89a82333404539ae63dbdddf97f84dcbc3d339aae9525", size = 107131, upload-time = "2025-10-14T04:41:10.467Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d0/d9/0ed4c7098a861482a7b6a95603edce4c0d9db2311af23da1fb2b75ec26fc/charset_normalizer-3.4.4-cp312-cp312-win_arm64.whl", hash = "sha256:376bec83a63b8021bb5c8ea75e21c4ccb86e7e45ca4eb81146091b56599b80c3", size = 100390, upload-time = "2025-10-14T04:41:11.915Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/97/45/4b3a1239bbacd321068ea6e7ac28875b03ab8bc0aa0966452db17cd36714/charset_normalizer-3.4.4-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:e1f185f86a6f3403aa2420e815904c67b2f9ebc443f045edd0de921108345794", size = 208091, upload-time = "2025-10-14T04:41:13.346Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7d/62/73a6d7450829655a35bb88a88fca7d736f9882a27eacdca2c6d505b57e2e/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b39f987ae8ccdf0d2642338faf2abb1862340facc796048b604ef14919e55ed", size = 147936, upload-time = "2025-10-14T04:41:14.461Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/c5/adb8c8b3d6625bef6d88b251bbb0d95f8205831b987631ab0c8bb5d937c2/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3162d5d8ce1bb98dd51af660f2121c55d0fa541b46dff7bb9b9f86ea1d87de72", size = 144180, upload-time = "2025-10-14T04:41:15.588Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/91/ed/9706e4070682d1cc219050b6048bfd293ccf67b3d4f5a4f39207453d4b99/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:81d5eb2a312700f4ecaa977a8235b634ce853200e828fbadf3a9c50bab278328", size = 161346, upload-time = "2025-10-14T04:41:16.738Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d5/0d/031f0d95e4972901a2f6f09ef055751805ff541511dc1252ba3ca1f80cf5/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5bd2293095d766545ec1a8f612559f6b40abc0eb18bb2f5d1171872d34036ede", size = 158874, upload-time = "2025-10-14T04:41:17.923Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f5/83/6ab5883f57c9c801ce5e5677242328aa45592be8a00644310a008d04f922/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8a8b89589086a25749f471e6a900d3f662d1d3b6e2e59dcecf787b1cc3a1894", size = 153076, upload-time = "2025-10-14T04:41:19.106Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/75/1e/5ff781ddf5260e387d6419959ee89ef13878229732732ee73cdae01800f2/charset_normalizer-3.4.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc7637e2f80d8530ee4a78e878bce464f70087ce73cf7c1caf142416923b98f1", size = 150601, upload-time = "2025-10-14T04:41:20.245Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d7/57/71be810965493d3510a6ca79b90c19e48696fb1ff964da319334b12677f0/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f8bf04158c6b607d747e93949aa60618b61312fe647a6369f88ce2ff16043490", size = 150376, upload-time = "2025-10-14T04:41:21.398Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e5/d5/c3d057a78c181d007014feb7e9f2e65905a6c4ef182c0ddf0de2924edd65/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:554af85e960429cf30784dd47447d5125aaa3b99a6f0683589dbd27e2f45da44", size = 144825, upload-time = "2025-10-14T04:41:22.583Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e6/8c/d0406294828d4976f275ffbe66f00266c4b3136b7506941d87c00cab5272/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:74018750915ee7ad843a774364e13a3db91682f26142baddf775342c3f5b1133", size = 162583, upload-time = "2025-10-14T04:41:23.754Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d7/24/e2aa1f18c8f15c4c0e932d9287b8609dd30ad56dbe41d926bd846e22fb8d/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c0463276121fdee9c49b98908b3a89c39be45d86d1dbaa22957e38f6321d4ce3", size = 150366, upload-time = "2025-10-14T04:41:25.27Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e4/5b/1e6160c7739aad1e2df054300cc618b06bf784a7a164b0f238360721ab86/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:362d61fd13843997c1c446760ef36f240cf81d3ebf74ac62652aebaf7838561e", size = 160300, upload-time = "2025-10-14T04:41:26.725Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/10/f882167cd207fbdd743e55534d5d9620e095089d176d55cb22d5322f2afd/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a26f18905b8dd5d685d6d07b0cdf98a79f3c7a918906af7cc143ea2e164c8bc", size = 154465, upload-time = "2025-10-14T04:41:28.322Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/66/c7a9e1b7429be72123441bfdbaf2bc13faab3f90b933f664db506dea5915/charset_normalizer-3.4.4-cp313-cp313-win32.whl", hash = "sha256:9b35f4c90079ff2e2edc5b26c0c77925e5d2d255c42c74fdb70fb49b172726ac", size = 99404, upload-time = "2025-10-14T04:41:29.95Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c4/26/b9924fa27db384bdcd97ab83b4f0a8058d96ad9626ead570674d5e737d90/charset_normalizer-3.4.4-cp313-cp313-win_amd64.whl", hash = "sha256:b435cba5f4f750aa6c0a0d92c541fb79f69a387c91e61f1795227e4ed9cece14", size = 107092, upload-time = "2025-10-14T04:41:31.188Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/af/8f/3ed4bfa0c0c72a7ca17f0380cd9e4dd842b09f664e780c13cff1dcf2ef1b/charset_normalizer-3.4.4-cp313-cp313-win_arm64.whl", hash = "sha256:542d2cee80be6f80247095cc36c418f7bddd14f4a6de45af91dfad36d817bba2", size = 100408, upload-time = "2025-10-14T04:41:32.624Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2a/35/7051599bd493e62411d6ede36fd5af83a38f37c4767b92884df7301db25d/charset_normalizer-3.4.4-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:da3326d9e65ef63a817ecbcc0df6e94463713b754fe293eaa03da99befb9a5bd", size = 207746, upload-time = "2025-10-14T04:41:33.773Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/10/9a/97c8d48ef10d6cd4fcead2415523221624bf58bcf68a802721a6bc807c8f/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8af65f14dc14a79b924524b1e7fffe304517b2bff5a58bf64f30b98bbc5079eb", size = 147889, upload-time = "2025-10-14T04:41:34.897Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/10/bf/979224a919a1b606c82bd2c5fa49b5c6d5727aa47b4312bb27b1734f53cd/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74664978bb272435107de04e36db5a9735e78232b85b77d45cfb38f758efd33e", size = 143641, upload-time = "2025-10-14T04:41:36.116Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ba/33/0ad65587441fc730dc7bd90e9716b30b4702dc7b617e6ba4997dc8651495/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:752944c7ffbfdd10c074dc58ec2d5a8a4cd9493b314d367c14d24c17684ddd14", size = 160779, upload-time = "2025-10-14T04:41:37.229Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/67/ed/331d6b249259ee71ddea93f6f2f0a56cfebd46938bde6fcc6f7b9a3d0e09/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d1f13550535ad8cff21b8d757a3257963e951d96e20ec82ab44bc64aeb62a191", size = 159035, upload-time = "2025-10-14T04:41:38.368Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/67/ff/f6b948ca32e4f2a4576aa129d8bed61f2e0543bf9f5f2b7fc3758ed005c9/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ecaae4149d99b1c9e7b88bb03e3221956f68fd6d50be2ef061b2381b61d20838", size = 152542, upload-time = "2025-10-14T04:41:39.862Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/16/85/276033dcbcc369eb176594de22728541a925b2632f9716428c851b149e83/charset_normalizer-3.4.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb6254dc36b47a990e59e1068afacdcd02958bdcce30bb50cc1700a8b9d624a6", size = 149524, upload-time = "2025-10-14T04:41:41.319Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9e/f2/6a2a1f722b6aba37050e626530a46a68f74e63683947a8acff92569f979a/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c8ae8a0f02f57a6e61203a31428fa1d677cbe50c93622b4149d5c0f319c1d19e", size = 150395, upload-time = "2025-10-14T04:41:42.539Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/60/bb/2186cb2f2bbaea6338cad15ce23a67f9b0672929744381e28b0592676824/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:47cc91b2f4dd2833fddaedd2893006b0106129d4b94fdb6af1f4ce5a9965577c", size = 143680, upload-time = "2025-10-14T04:41:43.661Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7d/a5/bf6f13b772fbb2a90360eb620d52ed8f796f3c5caee8398c3b2eb7b1c60d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:82004af6c302b5d3ab2cfc4cc5f29db16123b1a8417f2e25f9066f91d4411090", size = 162045, upload-time = "2025-10-14T04:41:44.821Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/df/c5/d1be898bf0dc3ef9030c3825e5d3b83f2c528d207d246cbabe245966808d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7d8f6c26245217bd2ad053761201e9f9680f8ce52f0fcd8d0755aeae5b2152", size = 149687, upload-time = "2025-10-14T04:41:46.442Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a5/42/90c1f7b9341eef50c8a1cb3f098ac43b0508413f33affd762855f67a410e/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:799a7a5e4fb2d5898c60b640fd4981d6a25f1c11790935a44ce38c54e985f828", size = 160014, upload-time = "2025-10-14T04:41:47.631Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/76/be/4d3ee471e8145d12795ab655ece37baed0929462a86e72372fd25859047c/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:99ae2cffebb06e6c22bdc25801d7b30f503cc87dbd283479e7b606f70aff57ec", size = 154044, upload-time = "2025-10-14T04:41:48.81Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b0/6f/8f7af07237c34a1defe7defc565a9bc1807762f672c0fde711a4b22bf9c0/charset_normalizer-3.4.4-cp314-cp314-win32.whl", hash = "sha256:f9d332f8c2a2fcbffe1378594431458ddbef721c1769d78e2cbc06280d8155f9", size = 99940, upload-time = "2025-10-14T04:41:49.946Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4b/51/8ade005e5ca5b0d80fb4aff72a3775b325bdc3d27408c8113811a7cbe640/charset_normalizer-3.4.4-cp314-cp314-win_amd64.whl", hash = "sha256:8a6562c3700cce886c5be75ade4a5db4214fda19fede41d9792d100288d8f94c", size = 107104, upload-time = "2025-10-14T04:41:51.051Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/da/5f/6b8f83a55bb8278772c5ae54a577f3099025f9ade59d0136ac24a0df4bde/charset_normalizer-3.4.4-cp314-cp314-win_arm64.whl", hash = "sha256:de00632ca48df9daf77a2c65a484531649261ec9f25489917f09e455cb09ddb2", size = 100743, upload-time = "2025-10-14T04:41:52.122Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0a/4c/925909008ed5a988ccbb72dcc897407e5d6d3bd72410d69e051fc0c14647/charset_normalizer-3.4.4-py3-none-any.whl", hash = "sha256:7a32c560861a02ff789ad905a2fe94e3f840803362c84fecf1851cb4cf3dc37f", size = 53402, upload-time = "2025-10-14T04:42:31.76Z" },
]
@@ -86,18 +86,6 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "exceptiongroup"
version = "1.3.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
dependencies = [
{ name = "typing-extensions" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
]
[[package]]
name = "idna"
version = "3.11"
@@ -149,6 +137,17 @@ version = "3.23.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c9/85/e24bf90972a30b0fcd16c73009add1d7d7cd9140c2498a68252028899e41/pycryptodomex-3.23.0.tar.gz", hash = "sha256:71909758f010c82bc99b0abf4ea12012c98962fbf0583c2164f8b84533c2e4da", size = 4922157, upload-time = "2025-05-17T17:23:41.434Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2e/00/10edb04777069a42490a38c137099d4b17ba6e36a4e6e28bdc7470e9e853/pycryptodomex-3.23.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:7b37e08e3871efe2187bc1fd9320cc81d87caf19816c648f24443483005ff886", size = 2498764, upload-time = "2025-05-17T17:22:21.453Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6b/3f/2872a9c2d3a27eac094f9ceaa5a8a483b774ae69018040ea3240d5b11154/pycryptodomex-3.23.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:91979028227543010d7b2ba2471cf1d1e398b3f183cb105ac584df0c36dac28d", size = 1643012, upload-time = "2025-05-17T17:22:23.702Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/70/af/774c2e2b4f6570fbf6a4972161adbb183aeeaa1863bde31e8706f123bf92/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6b8962204c47464d5c1c4038abeadd4514a133b28748bcd9fa5b6d62e3cec6fa", size = 2187643, upload-time = "2025-05-17T17:22:26.37Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/de/a3/71065b24cb889d537954cedc3ae5466af00a2cabcff8e29b73be047e9a19/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a33986a0066860f7fcf7c7bd2bc804fa90e434183645595ae7b33d01f3c91ed8", size = 2273762, upload-time = "2025-05-17T17:22:28.313Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c9/0b/ff6f43b7fbef4d302c8b981fe58467b8871902cdc3eb28896b52421422cc/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7947ab8d589e3178da3d7cdeabe14f841b391e17046954f2fbcd941705762b5", size = 2313012, upload-time = "2025-05-17T17:22:30.57Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/02/de/9d4772c0506ab6da10b41159493657105d3f8bb5c53615d19452afc6b315/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c25e30a20e1b426e1f0fa00131c516f16e474204eee1139d1603e132acffc314", size = 2186856, upload-time = "2025-05-17T17:22:32.819Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/28/ad/8b30efcd6341707a234e5eba5493700a17852ca1ac7a75daa7945fcf6427/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:da4fa650cef02db88c2b98acc5434461e027dce0ae8c22dd5a69013eaf510006", size = 2347523, upload-time = "2025-05-17T17:22:35.386Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0f/02/16868e9f655b7670dbb0ac4f2844145cbc42251f916fc35c414ad2359849/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:58b851b9effd0d072d4ca2e4542bf2a4abcf13c82a29fd2c93ce27ee2a2e9462", size = 2272825, upload-time = "2025-05-17T17:22:37.632Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ca/18/4ca89ac737230b52ac8ffaca42f9c6f1fd07c81a6cd821e91af79db60632/pycryptodomex-3.23.0-cp313-cp313t-win32.whl", hash = "sha256:a9d446e844f08299236780f2efa9898c818fe7e02f17263866b8550c7d5fb328", size = 1772078, upload-time = "2025-05-17T17:22:40Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/73/34/13e01c322db027682e00986873eca803f11c56ade9ba5bbf3225841ea2d4/pycryptodomex-3.23.0-cp313-cp313t-win_amd64.whl", hash = "sha256:bc65bdd9fc8de7a35a74cab1c898cab391a4add33a8fe740bda00f5976ca4708", size = 1803656, upload-time = "2025-05-17T17:22:42.139Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/54/68/9504c8796b1805d58f4425002bcca20f12880e6fa4dc2fc9a668705c7a08/pycryptodomex-3.23.0-cp313-cp313t-win_arm64.whl", hash = "sha256:c885da45e70139464f082018ac527fdaad26f1657a99ee13eecdce0f0ca24ab4", size = 1707172, upload-time = "2025-05-17T17:22:44.704Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/dd/9c/1a8f35daa39784ed8adf93a694e7e5dc15c23c741bbda06e1d45f8979e9e/pycryptodomex-3.23.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:06698f957fe1ab229a99ba2defeeae1c09af185baa909a31a5d1f9d42b1aaed6", size = 2499240, upload-time = "2025-05-17T17:22:46.953Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/62/f5221a191a97157d240cf6643747558759126c76ee92f29a3f4aee3197a5/pycryptodomex-3.23.0-cp37-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b2c2537863eccef2d41061e82a881dcabb04944c5c06c5aa7110b577cc487545", size = 1644042, upload-time = "2025-05-17T17:22:49.098Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8c/fd/5a054543c8988d4ed7b612721d7e78a4b9bf36bc3c5ad45ef45c22d0060e/pycryptodomex-3.23.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:43c446e2ba8df8889e0e16f02211c25b4934898384c1ec1ec04d7889c0333587", size = 2186227, upload-time = "2025-05-17T17:22:51.139Z" },
@@ -160,11 +159,6 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8d/67/09ee8500dd22614af5fbaa51a4aee6e342b5fa8aecf0a6cb9cbf52fa6d45/pycryptodomex-3.23.0-cp37-abi3-win32.whl", hash = "sha256:189afbc87f0b9f158386bf051f720e20fa6145975f1e76369303d0f31d1a8d7c", size = 1771969, upload-time = "2025-05-17T17:23:07.115Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/69/96/11f36f71a865dd6df03716d33bd07a67e9d20f6b8d39820470b766af323c/pycryptodomex-3.23.0-cp37-abi3-win_amd64.whl", hash = "sha256:52e5ca58c3a0b0bd5e100a9fbc8015059b05cffc6c66ce9d98b4b45e023443b9", size = 1803124, upload-time = "2025-05-17T17:23:09.267Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f9/93/45c1cdcbeb182ccd2e144c693eaa097763b08b38cded279f0053ed53c553/pycryptodomex-3.23.0-cp37-abi3-win_arm64.whl", hash = "sha256:02d87b80778c171445d67e23d1caef279bf4b25c3597050ccd2e13970b57fd51", size = 1707161, upload-time = "2025-05-17T17:23:11.414Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f3/b8/3e76d948c3c4ac71335bbe75dac53e154b40b0f8f1f022dfa295257a0c96/pycryptodomex-3.23.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:ebfff755c360d674306e5891c564a274a47953562b42fb74a5c25b8fc1fb1cb5", size = 1627695, upload-time = "2025-05-17T17:23:17.38Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6a/cf/80f4297a4820dfdfd1c88cf6c4666a200f204b3488103d027b5edd9176ec/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:eca54f4bb349d45afc17e3011ed4264ef1cc9e266699874cdd1349c504e64798", size = 1675772, upload-time = "2025-05-17T17:23:19.202Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/42/1e969ee0ad19fe3134b0e1b856c39bd0b70d47a4d0e81c2a8b05727394c9/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4f2596e643d4365e14d0879dc5aafe6355616c61c2176009270f3048f6d9a61f", size = 1668083, upload-time = "2025-05-17T17:23:21.867Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6e/c3/1de4f7631fea8a992a44ba632aa40e0008764c0fb9bf2854b0acf78c2cf2/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fdfac7cda115bca3a5abb2f9e43bc2fb66c2b65ab074913643803ca7083a79ea", size = 1706056, upload-time = "2025-05-17T17:23:24.031Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f2/5f/af7da8e6f1e42b52f44a24d08b8e4c726207434e2593732d39e7af5e7256/pycryptodomex-3.23.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:14c37aaece158d0ace436f76a7bb19093db3b4deade9797abfc39ec6cd6cc2fe", size = 1806478, upload-time = "2025-05-17T17:23:26.066Z" },
]
[[package]]
@@ -182,12 +176,10 @@ version = "9.0.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "exceptiongroup", marker = "python_full_version < '3.11'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
{ name = "tomli", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/07/56/f013048ac4bc4c1d9be45afd4ab209ea62822fb1598f40687e6bf45dcea4/pytest-9.0.1.tar.gz", hash = "sha256:3e9c069ea73583e255c3b21cf46b8d3c56f6e3a1a8f6da94ccb0fcf57b9d73c8", size = 1564125, upload-time = "2025-11-12T13:05:09.333Z" }
wheels = [
@@ -196,7 +188,7 @@ wheels = [
[[package]]
name = "ragflow-cli"
version = "0.23.1"
version = "0.25.3"
source = { virtual = "." }
dependencies = [
{ name = "beartype" },
@@ -254,45 +246,11 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl", hash = "sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", size = 54481, upload-time = "2023-05-01T04:11:28.427Z" },
]
[[package]]
name = "tomli"
version = "2.3.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/52/ed/3f73f72945444548f33eba9a87fc7a6e969915e7b1acc8260b30e1f76a2f/tomli-2.3.0.tar.gz", hash = "sha256:64be704a875d2a59753d80ee8a533c3fe183e3f06807ff7dc2232938ccb01549", size = 17392, upload-time = "2025-10-08T22:01:47.119Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b3/2e/299f62b401438d5fe1624119c723f5d877acc86a4c2492da405626665f12/tomli-2.3.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45", size = 153236, upload-time = "2025-10-08T22:01:00.137Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/86/7f/d8fffe6a7aefdb61bced88fcb5e280cfd71e08939da5894161bd71bea022/tomli-2.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba", size = 148084, upload-time = "2025-10-08T22:01:01.63Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/47/5c/24935fb6a2ee63e86d80e4d3b58b222dafaf438c416752c8b58537c8b89a/tomli-2.3.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d1381caf13ab9f300e30dd8feadb3de072aeb86f1d34a8569453ff32a7dea4bf", size = 234832, upload-time = "2025-10-08T22:01:02.543Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/da/75dfd804fc11e6612846758a23f13271b76d577e299592b4371a4ca4cd09/tomli-2.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a0e285d2649b78c0d9027570d4da3425bdb49830a6156121360b3f8511ea3441", size = 242052, upload-time = "2025-10-08T22:01:03.836Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/70/8c/f48ac899f7b3ca7eb13af73bacbc93aec37f9c954df3c08ad96991c8c373/tomli-2.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0a154a9ae14bfcf5d8917a59b51ffd5a3ac1fd149b71b47a3a104ca4edcfa845", size = 239555, upload-time = "2025-10-08T22:01:04.834Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ba/28/72f8afd73f1d0e7829bfc093f4cb98ce0a40ffc0cc997009ee1ed94ba705/tomli-2.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:74bf8464ff93e413514fefd2be591c3b0b23231a77f901db1eb30d6f712fc42c", size = 245128, upload-time = "2025-10-08T22:01:05.84Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b6/eb/a7679c8ac85208706d27436e8d421dfa39d4c914dcf5fa8083a9305f58d9/tomli-2.3.0-cp311-cp311-win32.whl", hash = "sha256:00b5f5d95bbfc7d12f91ad8c593a1659b6387b43f054104cda404be6bda62456", size = 96445, upload-time = "2025-10-08T22:01:06.896Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0a/fe/3d3420c4cb1ad9cb462fb52967080575f15898da97e21cb6f1361d505383/tomli-2.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:4dc4ce8483a5d429ab602f111a93a6ab1ed425eae3122032db7e9acf449451be", size = 107165, upload-time = "2025-10-08T22:01:08.107Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ff/b7/40f36368fcabc518bb11c8f06379a0fd631985046c038aca08c6d6a43c6e/tomli-2.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d7d86942e56ded512a594786a5ba0a5e521d02529b3826e7761a05138341a2ac", size = 154891, upload-time = "2025-10-08T22:01:09.082Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f9/3f/d9dd692199e3b3aab2e4e4dd948abd0f790d9ded8cd10cbaae276a898434/tomli-2.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:73ee0b47d4dad1c5e996e3cd33b8a76a50167ae5f96a2607cbe8cc773506ab22", size = 148796, upload-time = "2025-10-08T22:01:10.266Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/60/83/59bff4996c2cf9f9387a0f5a3394629c7efa5ef16142076a23a90f1955fa/tomli-2.3.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:792262b94d5d0a466afb5bc63c7daa9d75520110971ee269152083270998316f", size = 242121, upload-time = "2025-10-08T22:01:11.332Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/45/e5/7c5119ff39de8693d6baab6c0b6dcb556d192c165596e9fc231ea1052041/tomli-2.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4f195fe57ecceac95a66a75ac24d9d5fbc98ef0962e09b2eddec5d39375aae52", size = 250070, upload-time = "2025-10-08T22:01:12.498Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/45/12/ad5126d3a278f27e6701abde51d342aa78d06e27ce2bb596a01f7709a5a2/tomli-2.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e31d432427dcbf4d86958c184b9bfd1e96b5b71f8eb17e6d02531f434fd335b8", size = 245859, upload-time = "2025-10-08T22:01:13.551Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/fb/a1/4d6865da6a71c603cfe6ad0e6556c73c76548557a8d658f9e3b142df245f/tomli-2.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7b0882799624980785240ab732537fcfc372601015c00f7fc367c55308c186f6", size = 250296, upload-time = "2025-10-08T22:01:14.614Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a0/b7/a7a7042715d55c9ba6e8b196d65d2cb662578b4d8cd17d882d45322b0d78/tomli-2.3.0-cp312-cp312-win32.whl", hash = "sha256:ff72b71b5d10d22ecb084d345fc26f42b5143c5533db5e2eaba7d2d335358876", size = 97124, upload-time = "2025-10-08T22:01:15.629Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/06/1e/f22f100db15a68b520664eb3328fb0ae4e90530887928558112c8d1f4515/tomli-2.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:1cb4ed918939151a03f33d4242ccd0aa5f11b3547d0cf30f7c74a408a5b99878", size = 107698, upload-time = "2025-10-08T22:01:16.51Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/77/b8/0135fadc89e73be292b473cb820b4f5a08197779206b33191e801feeae40/tomli-2.3.0-py3-none-any.whl", hash = "sha256:e95b1af3c5b07d9e643909b5abbec77cd9f1217e6d0bca72b0234736b9fb1f1b", size = 14408, upload-time = "2025-10-08T22:01:46.04Z" },
]
[[package]]
name = "typing-extensions"
version = "4.15.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
]
[[package]]
name = "urllib3"
version = "2.5.0"
version = "2.6.3"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/15/22/9ee70a2574a4f4599c47dd506532914ce044817c7752a79b6a51286319bc/urllib3-2.5.0.tar.gz", hash = "sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760", size = 393185, upload-time = "2025-06-18T14:07:41.644Z" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a7/c2/fe1e52489ae3122415c51f387e221dd0773709bad6c6cdaa599e8a2c5185/urllib3-2.5.0-py3-none-any.whl", hash = "sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc", size = 129795, upload-time = "2025-06-18T14:07:40.39Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" },
]

View File

@@ -14,12 +14,13 @@
# limitations under the License.
#
import time
start_ts = time.time()
import os
import signal
import logging
import time
import threading
import traceback
import faulthandler
from flask import Flask
@@ -56,7 +57,7 @@ if __name__ == '__main__':
os.environ.get("MAX_CONTENT_LENGTH", 1024 * 1024 * 1024)
)
Session(app)
logging.info(f'RAGFlow version: {get_ragflow_version()}')
logging.info(f'RAGFlow admin version: {get_ragflow_version()}')
show_configs()
login_manager = LoginManager()
login_manager.init_app(app)
@@ -66,17 +67,17 @@ if __name__ == '__main__':
SERVICE_CONFIGS.configs = load_configurations(SERVICE_CONF)
try:
logging.info("RAGFlow Admin service start...")
logging.info(f"RAGFlow admin is ready after {time.time() - start_ts}s initialization.")
run_simple(
hostname="0.0.0.0",
port=9381,
application=app,
threaded=True,
use_reloader=False,
use_debugger=True,
use_debugger=False,
)
except Exception:
traceback.print_exc()
except Exception as e:
logging.exception(f"Unhandled exception: {e}")
stop_event.set()
time.sleep(1)
os.kill(os.getpid(), signal.SIGKILL)

View File

@@ -22,11 +22,12 @@ from datetime import datetime
from flask import jsonify, request
from flask_login import current_user, login_user
from itsdangerous.url_safe import URLSafeTimedSerializer as Serializer
from api.common.exceptions import AdminException, UserNotFoundError
from api.common.base64 import encode_to_base64
from api.db.services import UserService
from api.db import UserTenantRole
from api.db.services.user_service import TenantService, UserTenantService
from common.constants import ActiveEnum, StatusEnum
from api.utils.crypt import decrypt
from common.misc_utils import get_uuid
@@ -38,18 +39,34 @@ from common import settings
def setup_auth(login_manager):
@login_manager.request_loader
def load_user(web_request):
jwt = Serializer(secret_key=settings.SECRET_KEY)
# Authorization header contains JWT-encoded access token
# First decode JWT to get the UUID, then query database
from itsdangerous.url_safe import URLSafeTimedSerializer as Serializer
from common import settings
authorization = web_request.headers.get("Authorization")
if authorization:
try:
access_token = str(jwt.loads(authorization))
# Strip "Bearer " prefix if present
jwt_token = authorization
if jwt_token.startswith("Bearer "):
jwt_token = jwt_token[7:]
if not access_token or not access_token.strip():
logging.warning("Authentication attempt with empty access token")
jwt_token = jwt_token.strip()
if not jwt_token:
logging.warning("Authentication attempt with empty JWT token")
return None
# Access tokens should be UUIDs (32 hex characters)
if len(access_token.strip()) < 32:
# Decode JWT to get the UUID access_token
jwt = Serializer(secret_key=settings.get_secret_key())
access_token = str(jwt.loads(jwt_token))
if not access_token or not access_token.strip():
logging.warning("Authentication attempt with empty access token after JWT decode")
return None
# Access tokens stored in database are UUIDs (32 hex characters)
if len(access_token) < 32:
logging.warning(f"Authentication attempt with invalid token format: {len(access_token)} chars")
return None
@@ -85,8 +102,45 @@ def init_default_admin():
}
if not UserService.save(**default_admin):
raise AdminException("Can't init admin.", 500)
add_tenant_for_admin(default_admin, UserTenantRole.OWNER)
elif not any([u.is_active == ActiveEnum.ACTIVE.value for u in users]):
raise AdminException("No active admin. Please update 'is_active' in db manually.", 500)
else:
default_admin_rows = [u for u in users if u.email == "admin@ragflow.io"]
if default_admin_rows:
default_admin = default_admin_rows[0].to_dict()
exist, default_admin_tenant = TenantService.get_by_id(default_admin["id"])
if not exist:
add_tenant_for_admin(default_admin, UserTenantRole.OWNER)
def add_tenant_for_admin(user_info: dict, role: str):
from api.db.services.tenant_llm_service import TenantLLMService
from api.db.services.llm_service import get_init_tenant_llm
tenant = {
"id": user_info["id"],
"name": user_info["nickname"] + "s Kingdom",
"llm_id": settings.CHAT_MDL,
"embd_id": settings.EMBEDDING_MDL,
"asr_id": settings.ASR_MDL,
"parser_ids": settings.PARSERS,
"img2txt_id": settings.IMAGE2TEXT_MDL,
"rerank_id": settings.RERANK_MDL,
}
usr_tenant = {
"tenant_id": user_info["id"],
"user_id": user_info["id"],
"invited_by": user_info["id"],
"role": role
}
tenant_llm = get_init_tenant_llm(user_info["id"])
TenantService.insert(**tenant)
UserTenantService.insert(**usr_tenant)
TenantLLMService.insert_many(tenant_llm)
logging.info(
f"Added tenant for email: {user_info['email']}, A default tenant has been set; changing the default models after login is strongly recommended.")
def check_admin_auth(func):

View File

@@ -264,6 +264,19 @@ def load_configurations(config_path: str) -> list[BaseConfig]:
db_name=database, detail_func_name="get_infinity_status")
configurations.append(config)
id_count += 1
case "minio_0":
name: str = 'minio_0'
url = v['host']
parts = url.split(':', 1)
host = parts[0]
port = int(parts[1])
user = v.get('user')
password = v.get('password')
config = MinioConfig(id=id_count, name=name, host=host, port=port, user=user, password=password,
service_type="file_store",
store_type="minio", detail_func_name="check_minio_alive")
configurations.append(config)
id_count += 1
case "minio":
name: str = 'minio'
url = v['host']
@@ -310,6 +323,14 @@ def load_configurations(config_path: str) -> list[BaseConfig]:
service_type="task_executor", detail_func_name="check_task_executor_alive")
configurations.append(config)
id_count += 1
case "rabbitmq":
name: str = 'rabbitmq'
host: str = v.get('host')
port: int = v.get('port')
config = RabbitMQConfig(id=id_count, name=name, host=host, port=port,
service_type="message_queue", mq_type="rabbitmq", detail_func_name="check_rabbitmq_alive")
configurations.append(config)
id_count += 1
case _:
logging.warning(f"Unknown configuration key: {k}")
continue

View File

@@ -15,29 +15,35 @@
#
import secrets
import logging
from typing import Any
from flask import Blueprint, request
from common.time_utils import current_timestamp, datetime_format
from datetime import datetime
from flask import Blueprint, Response, request
from flask_login import current_user, login_required, logout_user
from auth import login_verify, login_admin, check_admin_auth
from responses import success_response, error_response
from services import UserMgr, ServiceMgr, UserServiceMgr
from services import UserMgr, ServiceMgr, UserServiceMgr, SettingsMgr, ConfigMgr, EnvironmentsMgr, SandboxMgr
from roles import RoleMgr
from api.common.exceptions import AdminException
from common.versions import get_ragflow_version
from api.utils.api_utils import generate_confirmation_token
from common.log_utils import get_log_levels, set_log_level
admin_bp = Blueprint('admin', __name__, url_prefix='/api/v1/admin')
admin_bp = Blueprint("admin", __name__, url_prefix="/api/v1/admin")
@admin_bp.route('/ping', methods=['GET'])
@admin_bp.route("/ping", methods=["GET"])
def ping():
return success_response('PONG')
return success_response(message="pong")
@admin_bp.route('/login', methods=['POST'])
@admin_bp.route("/login", methods=["POST"])
def login():
if not request.json:
return error_response('Authorize admin failed.' ,400)
return error_response("Authorize admin failed.", 400)
try:
email = request.json.get("email", "")
password = request.json.get("password", "")
@@ -46,7 +52,7 @@ def login():
return error_response(str(e), 500)
@admin_bp.route('/logout', methods=['GET'])
@admin_bp.route("/logout", methods=["GET"])
@login_required
def logout():
try:
@@ -58,7 +64,7 @@ def logout():
return error_response(str(e), 500)
@admin_bp.route('/auth', methods=['GET'])
@admin_bp.route("/auth", methods=["GET"])
@login_verify
def auth_admin():
try:
@@ -67,7 +73,7 @@ def auth_admin():
return error_response(str(e), 500)
@admin_bp.route('/users', methods=['GET'])
@admin_bp.route("/users", methods=["GET"])
@login_required
@check_admin_auth
def list_users():
@@ -78,18 +84,18 @@ def list_users():
return error_response(str(e), 500)
@admin_bp.route('/users', methods=['POST'])
@admin_bp.route("/users", methods=["POST"])
@login_required
@check_admin_auth
def create_user():
try:
data = request.get_json()
if not data or 'username' not in data or 'password' not in data:
if not data or "username" not in data or "password" not in data:
return error_response("Username and password are required", 400)
username = data['username']
password = data['password']
role = data.get('role', 'user')
username = data["username"]
password = data["password"]
role = data.get("role", "user")
res = UserMgr.create_user(username, password, role)
if res["success"]:
@@ -105,7 +111,7 @@ def create_user():
return error_response(str(e))
@admin_bp.route('/users/<username>', methods=['DELETE'])
@admin_bp.route("/users/<username>", methods=["DELETE"])
@login_required
@check_admin_auth
def delete_user(username):
@@ -122,16 +128,16 @@ def delete_user(username):
return error_response(str(e), 500)
@admin_bp.route('/users/<username>/password', methods=['PUT'])
@admin_bp.route("/users/<username>/password", methods=["PUT"])
@login_required
@check_admin_auth
def change_password(username):
try:
data = request.get_json()
if not data or 'new_password' not in data:
if not data or "new_password" not in data:
return error_response("New password is required", 400)
new_password = data['new_password']
new_password = data["new_password"]
msg = UserMgr.update_user_password(username, new_password)
return success_response(None, msg)
@@ -141,15 +147,15 @@ def change_password(username):
return error_response(str(e), 500)
@admin_bp.route('/users/<username>/activate', methods=['PUT'])
@admin_bp.route("/users/<username>/activate", methods=["PUT"])
@login_required
@check_admin_auth
def alter_user_activate_status(username):
try:
data = request.get_json()
if not data or 'activate_status' not in data:
if not data or "activate_status" not in data:
return error_response("Activation status is required", 400)
activate_status = data['activate_status']
activate_status = data["activate_status"]
msg = UserMgr.update_user_activate_status(username, activate_status)
return success_response(None, msg)
except AdminException as e:
@@ -158,7 +164,39 @@ def alter_user_activate_status(username):
return error_response(str(e), 500)
@admin_bp.route('/users/<username>', methods=['GET'])
@admin_bp.route("/users/<username>/admin", methods=["PUT"])
@login_required
@check_admin_auth
def grant_admin(username):
try:
if current_user.email == username:
return error_response(f"can't grant current user: {username}", 409)
msg = UserMgr.grant_admin(username)
return success_response(None, msg)
except AdminException as e:
return error_response(e.message, e.code)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/users/<username>/admin", methods=["DELETE"])
@login_required
@check_admin_auth
def revoke_admin(username):
try:
if current_user.email == username:
return error_response(f"can't grant current user: {username}", 409)
msg = UserMgr.revoke_admin(username)
return success_response(None, msg)
except AdminException as e:
return error_response(e.message, e.code)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/users/<username>", methods=["GET"])
@login_required
@check_admin_auth
def get_user_details(username):
@@ -172,7 +210,7 @@ def get_user_details(username):
return error_response(str(e), 500)
@admin_bp.route('/users/<username>/datasets', methods=['GET'])
@admin_bp.route("/users/<username>/datasets", methods=["GET"])
@login_required
@check_admin_auth
def get_user_datasets(username):
@@ -186,7 +224,7 @@ def get_user_datasets(username):
return error_response(str(e), 500)
@admin_bp.route('/users/<username>/agents', methods=['GET'])
@admin_bp.route("/users/<username>/agents", methods=["GET"])
@login_required
@check_admin_auth
def get_user_agents(username):
@@ -200,7 +238,7 @@ def get_user_agents(username):
return error_response(str(e), 500)
@admin_bp.route('/services', methods=['GET'])
@admin_bp.route("/services", methods=["GET"])
@login_required
@check_admin_auth
def get_services():
@@ -211,7 +249,7 @@ def get_services():
return error_response(str(e), 500)
@admin_bp.route('/service_types/<service_type>', methods=['GET'])
@admin_bp.route("/service_types/<service_type>", methods=["GET"])
@login_required
@check_admin_auth
def get_services_by_type(service_type_str):
@@ -222,7 +260,7 @@ def get_services_by_type(service_type_str):
return error_response(str(e), 500)
@admin_bp.route('/services/<service_id>', methods=['GET'])
@admin_bp.route("/services/<service_id>", methods=["GET"])
@login_required
@check_admin_auth
def get_service(service_id):
@@ -233,7 +271,7 @@ def get_service(service_id):
return error_response(str(e), 500)
@admin_bp.route('/services/<service_id>', methods=['DELETE'])
@admin_bp.route("/services/<service_id>", methods=["DELETE"])
@login_required
@check_admin_auth
def shutdown_service(service_id):
@@ -244,7 +282,7 @@ def shutdown_service(service_id):
return error_response(str(e), 500)
@admin_bp.route('/services/<service_id>', methods=['PUT'])
@admin_bp.route("/services/<service_id>", methods=["PUT"])
@login_required
@check_admin_auth
def restart_service(service_id):
@@ -255,38 +293,38 @@ def restart_service(service_id):
return error_response(str(e), 500)
@admin_bp.route('/roles', methods=['POST'])
@admin_bp.route("/roles", methods=["POST"])
@login_required
@check_admin_auth
def create_role():
try:
data = request.get_json()
if not data or 'role_name' not in data:
if not data or "role_name" not in data:
return error_response("Role name is required", 400)
role_name: str = data['role_name']
description: str = data['description']
role_name: str = data["role_name"]
description: str = data["description"]
res = RoleMgr.create_role(role_name, description)
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/roles/<role_name>', methods=['PUT'])
@admin_bp.route("/roles/<role_name>", methods=["PUT"])
@login_required
@check_admin_auth
def update_role(role_name: str):
try:
data = request.get_json()
if not data or 'description' not in data:
if not data or "description" not in data:
return error_response("Role description is required", 400)
description: str = data['description']
description: str = data["description"]
res = RoleMgr.update_role_description(role_name, description)
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/roles/<role_name>', methods=['DELETE'])
@admin_bp.route("/roles/<role_name>", methods=["DELETE"])
@login_required
@check_admin_auth
def delete_role(role_name: str):
@@ -297,7 +335,7 @@ def delete_role(role_name: str):
return error_response(str(e), 500)
@admin_bp.route('/roles', methods=['GET'])
@admin_bp.route("/roles", methods=["GET"])
@login_required
@check_admin_auth
def list_roles():
@@ -308,7 +346,7 @@ def list_roles():
return error_response(str(e), 500)
@admin_bp.route('/roles/<role_name>/permission', methods=['GET'])
@admin_bp.route("/roles/<role_name>/permission", methods=["GET"])
@login_required
@check_admin_auth
def get_role_permission(role_name: str):
@@ -319,54 +357,54 @@ def get_role_permission(role_name: str):
return error_response(str(e), 500)
@admin_bp.route('/roles/<role_name>/permission', methods=['POST'])
@admin_bp.route("/roles/<role_name>/permission", methods=["POST"])
@login_required
@check_admin_auth
def grant_role_permission(role_name: str):
try:
data = request.get_json()
if not data or 'actions' not in data or 'resource' not in data:
if not data or "actions" not in data or "resource" not in data:
return error_response("Permission is required", 400)
actions: list = data['actions']
resource: str = data['resource']
actions: list = data["actions"]
resource: str = data["resource"]
res = RoleMgr.grant_role_permission(role_name, actions, resource)
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/roles/<role_name>/permission', methods=['DELETE'])
@admin_bp.route("/roles/<role_name>/permission", methods=["DELETE"])
@login_required
@check_admin_auth
def revoke_role_permission(role_name: str):
try:
data = request.get_json()
if not data or 'actions' not in data or 'resource' not in data:
if not data or "actions" not in data or "resource" not in data:
return error_response("Permission is required", 400)
actions: list = data['actions']
resource: str = data['resource']
actions: list = data["actions"]
resource: str = data["resource"]
res = RoleMgr.revoke_role_permission(role_name, actions, resource)
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/users/<user_name>/role', methods=['PUT'])
@admin_bp.route("/users/<user_name>/role", methods=["PUT"])
@login_required
@check_admin_auth
def update_user_role(user_name: str):
try:
data = request.get_json()
if not data or 'role_name' not in data:
if not data or "role_name" not in data:
return error_response("Role name is required", 400)
role_name: str = data['role_name']
role_name: str = data["role_name"]
res = RoleMgr.update_user_role(user_name, role_name)
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/users/<user_name>/permission', methods=['GET'])
@admin_bp.route("/users/<user_name>/permission", methods=["GET"])
@login_required
@check_admin_auth
def get_user_permission(user_name: str):
@@ -376,7 +414,140 @@ def get_user_permission(user_name: str):
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route('/version', methods=['GET'])
@admin_bp.route("/variables", methods=["PUT"])
@login_required
@check_admin_auth
def set_variable():
try:
data = request.get_json()
if not data and "var_name" not in data:
return error_response("Var name is required", 400)
if "var_value" not in data:
return error_response("Var value is required", 400)
var_name: str = data["var_name"]
var_value: str = data["var_value"]
SettingsMgr.update_by_name(var_name, var_value)
return success_response(None, "Set variable successfully")
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/variables", methods=["GET"])
@login_required
@check_admin_auth
def get_variable():
try:
if request.content_length is None or request.content_length == 0:
# list variables
res = list(SettingsMgr.get_all())
return success_response(res)
# get var
data = request.get_json()
if not data and "var_name" not in data:
return error_response("Var name is required", 400)
var_name: str = data["var_name"]
res = SettingsMgr.get_by_name(var_name)
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/configs", methods=["GET"])
@login_required
@check_admin_auth
def get_config():
try:
res = list(ConfigMgr.get_all())
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/environments", methods=["GET"])
@login_required
@check_admin_auth
def get_environments():
try:
res = list(EnvironmentsMgr.get_all())
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/users/<username>/keys", methods=["POST"])
@login_required
@check_admin_auth
def generate_user_api_key(username: str) -> tuple[Response, int]:
try:
user_details: list[dict[str, Any]] = UserMgr.get_user_details(username)
if not user_details:
return error_response("User not found!", 404)
tenants: list[dict[str, Any]] = UserServiceMgr.get_user_tenants(username)
if not tenants:
return error_response("Tenant not found!", 404)
tenant_id: str = tenants[0]["tenant_id"]
key: str = generate_confirmation_token()
obj: dict[str, Any] = {
"tenant_id": tenant_id,
"token": key,
"beta": generate_confirmation_token().replace("ragflow-", "")[:32],
"create_time": current_timestamp(),
"create_date": datetime_format(datetime.now()),
"update_time": None,
"update_date": None,
}
if not UserMgr.save_api_key(obj):
return error_response("Failed to generate API key!", 500)
return success_response(obj, "API key generated successfully")
except AdminException as e:
return error_response(e.message, e.code)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/users/<username>/keys", methods=["GET"])
@login_required
@check_admin_auth
def get_user_api_keys(username: str) -> tuple[Response, int]:
try:
api_keys: list[dict[str, Any]] = UserMgr.get_user_api_key(username)
return success_response(api_keys, "Get user API keys")
except AdminException as e:
return error_response(e.message, e.code)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/users/<username>/keys/<key>", methods=["DELETE"])
@login_required
@check_admin_auth
def delete_user_api_key(username: str, key: str) -> tuple[Response, int]:
try:
deleted = UserMgr.delete_api_key(username, key)
if deleted:
return success_response(None, "API key deleted successfully")
else:
return error_response("API key not found or could not be deleted", 404)
except AdminException as e:
return error_response(e.message, e.code)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/version", methods=["GET"])
@login_required
@check_admin_auth
def show_version():
@@ -385,3 +556,136 @@ def show_version():
return success_response(res)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/sandbox/providers", methods=["GET"])
@login_required
@check_admin_auth
def list_sandbox_providers():
"""List all available sandbox providers."""
try:
res = SandboxMgr.list_providers()
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/sandbox/providers/<provider_id>/schema", methods=["GET"])
@login_required
@check_admin_auth
def get_sandbox_provider_schema(provider_id: str):
"""Get configuration schema for a specific provider."""
try:
res = SandboxMgr.get_provider_config_schema(provider_id)
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/sandbox/config", methods=["GET"])
@login_required
@check_admin_auth
def get_sandbox_config():
"""Get current sandbox configuration."""
try:
res = SandboxMgr.get_config()
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/sandbox/config", methods=["POST"])
@login_required
@check_admin_auth
def set_sandbox_config():
"""Set sandbox provider configuration."""
try:
data = request.get_json()
if not data:
logging.error("set_sandbox_config: Request body is required")
return error_response("Request body is required", 400)
provider_type = data.get("provider_type")
if not provider_type:
logging.error("set_sandbox_config: provider_type is required")
return error_response("provider_type is required", 400)
config = data.get("config", {})
set_active = data.get("set_active", True) # Default to True for backward compatibility
logging.info(f"set_sandbox_config: provider_type={provider_type}, set_active={set_active}")
logging.info(f"set_sandbox_config: config keys={list(config.keys())}")
res = SandboxMgr.set_config(provider_type, config, set_active)
return success_response(res, "Sandbox configuration updated successfully")
except AdminException as e:
logging.exception("set_sandbox_config AdminException")
return error_response(str(e), 400)
except Exception as e:
logging.exception("set_sandbox_config unexpected error")
return error_response(str(e), 500)
@admin_bp.route("/sandbox/test", methods=["POST"])
@login_required
@check_admin_auth
def test_sandbox_connection():
"""Test connection to sandbox provider."""
try:
data = request.get_json()
if not data:
return error_response("Request body is required", 400)
provider_type = data.get("provider_type")
if not provider_type:
return error_response("provider_type is required", 400)
config = data.get("config", {})
res = SandboxMgr.test_connection(provider_type, config)
return success_response(res)
except AdminException as e:
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/log_levels", methods=["GET"])
@login_required
@check_admin_auth
def get_logger_levels():
"""Get current log levels for all packages."""
try:
res = get_log_levels()
return success_response(res, "Get log levels", 0)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/log_levels", methods=["PUT"])
@login_required
@check_admin_auth
def set_logger_level():
"""Set log level for a package."""
try:
data = request.get_json()
if not data or "pkg_name" not in data or "level" not in data:
return error_response("pkg_name and level are required", 400)
pkg_name = data["pkg_name"]
level = data["level"]
if not isinstance(pkg_name, str) or not isinstance(level, str):
return error_response("pkg_name and level must be strings", 400)
success = set_log_level(pkg_name, level)
if success:
return success_response({"pkg_name": pkg_name, "level": level}, "Log level updated successfully")
else:
return error_response(f"Invalid log level: {level}", 400)
except Exception as e:
return error_response(str(e), 500)

View File

@@ -14,16 +14,22 @@
# limitations under the License.
#
import json
import os
import logging
import re
from typing import Any
from werkzeug.security import check_password_hash
from common.constants import ActiveEnum
from api.db.services import UserService
from api.db.joint_services.user_account_service import create_new_user, delete_user_data
from api.db.services.canvas_service import UserCanvasService
from api.db.services.user_service import TenantService
from api.db.services.user_service import TenantService, UserTenantService
from api.db.services.knowledgebase_service import KnowledgebaseService
from api.db.services.system_settings_service import SystemSettingsService
from api.db.services.api_service import APITokenService
from api.db.db_models import APIToken
from api.utils.crypt import decrypt
from api.utils import health_utils
@@ -37,13 +43,15 @@ class UserMgr:
users = UserService.get_all_users()
result = []
for user in users:
result.append({
'email': user.email,
'nickname': user.nickname,
'create_date': user.create_date,
'is_active': user.is_active,
'is_superuser': user.is_superuser,
})
result.append(
{
"email": user.email,
"nickname": user.nickname,
"create_date": user.create_date,
"is_active": user.is_active,
"is_superuser": user.is_superuser,
}
)
return result
@staticmethod
@@ -52,19 +60,21 @@ class UserMgr:
users = UserService.query_user_by_email(username)
result = []
for user in users:
result.append({
'avatar': user.avatar,
'email': user.email,
'language': user.language,
'last_login_time': user.last_login_time,
'is_active': user.is_active,
'is_anonymous': user.is_anonymous,
'login_channel': user.login_channel,
'status': user.status,
'is_superuser': user.is_superuser,
'create_date': user.create_date,
'update_date': user.update_date
})
result.append(
{
"avatar": user.avatar,
"email": user.email,
"language": user.language,
"last_login_time": user.last_login_time,
"is_active": user.is_active,
"is_anonymous": user.is_anonymous,
"login_channel": user.login_channel,
"status": user.status,
"is_superuser": user.is_superuser,
"create_date": user.create_date,
"update_date": user.update_date,
}
)
return result
@staticmethod
@@ -126,8 +136,8 @@ class UserMgr:
# format activate_status before handle
_activate_status = activate_status.lower()
target_status = {
'on': ActiveEnum.ACTIVE.value,
'off': ActiveEnum.INACTIVE.value,
"on": ActiveEnum.ACTIVE.value,
"off": ActiveEnum.INACTIVE.value,
}.get(_activate_status)
if not target_status:
raise AdminException(f"Invalid activate_status: {activate_status}")
@@ -137,9 +147,84 @@ class UserMgr:
UserService.update_user(usr.id, {"is_active": target_status})
return f"Turn {_activate_status} user activate status successfully!"
@staticmethod
def get_user_api_key(username: str) -> list[dict[str, Any]]:
# use email to find user. check exist and unique.
user_list: list[Any] = UserService.query_user_by_email(username)
if not user_list:
raise UserNotFoundError(username)
elif len(user_list) > 1:
raise AdminException(f"More than one user with username '{username}' found!")
usr: Any = user_list[0]
# tenant_id is typically the same as user_id for the owner tenant
tenant_id: str = usr.id
# Query all API keys for this tenant
api_keys: Any = APITokenService.query(tenant_id=tenant_id)
result: list[dict[str, Any]] = []
for key in api_keys:
result.append(key.to_dict())
return result
@staticmethod
def save_api_key(api_key: dict[str, Any]) -> bool:
return APITokenService.save(**api_key)
@staticmethod
def delete_api_key(username: str, key: str) -> bool:
# use email to find user. check exist and unique.
user_list: list[Any] = UserService.query_user_by_email(username)
if not user_list:
raise UserNotFoundError(username)
elif len(user_list) > 1:
raise AdminException(f"Exist more than 1 user: {username}!")
usr: Any = user_list[0]
# tenant_id is typically the same as user_id for the owner tenant
tenant_id: str = usr.id
# Delete the API key
deleted_count: int = APITokenService.filter_delete([APIToken.tenant_id == tenant_id, APIToken.token == key])
return deleted_count > 0
@staticmethod
def grant_admin(username: str):
# use email to find user. check exist and unique.
user_list = UserService.query_user_by_email(username)
if not user_list:
raise UserNotFoundError(username)
elif len(user_list) > 1:
raise AdminException(f"Exist more than 1 user: {username}!")
# check activate status different from new
usr = user_list[0]
if usr.is_superuser:
return f"{usr} is already superuser!"
# update is_active
UserService.update_user(usr.id, {"is_superuser": True})
return "Grant successfully!"
@staticmethod
def revoke_admin(username: str):
# use email to find user. check exist and unique.
user_list = UserService.query_user_by_email(username)
if not user_list:
raise UserNotFoundError(username)
elif len(user_list) > 1:
raise AdminException(f"Exist more than 1 user: {username}!")
# check activate status different from new
usr = user_list[0]
if not usr.is_superuser:
return f"{usr} isn't superuser, yet!"
# update is_active
UserService.update_user(usr.id, {"is_superuser": False})
return "Revoke successfully!"
class UserServiceMgr:
@staticmethod
def get_user_datasets(username):
# use email to find user.
@@ -169,39 +254,43 @@ class UserServiceMgr:
tenant_ids = [m["tenant_id"] for m in tenants]
# filter permitted agents and owned agents
res = UserCanvasService.get_all_agents_by_tenant_ids(tenant_ids, usr.id)
return [{
'title': r['title'],
'permission': r['permission'],
'canvas_category': r['canvas_category'].split('_')[0],
'avatar': r['avatar']
} for r in res]
return [{"title": r["title"], "permission": r["permission"], "canvas_category": r["canvas_category"].split("_")[0], "avatar": r["avatar"]} for r in res]
@staticmethod
def get_user_tenants(email: str) -> list[dict[str, Any]]:
users: list[Any] = UserService.query_user_by_email(email)
if not users:
raise UserNotFoundError(email)
user: Any = users[0]
tenants: list[dict[str, Any]] = UserTenantService.get_tenants_by_user_id(user.id)
return tenants
class ServiceMgr:
@staticmethod
def get_all_services():
doc_engine = os.getenv('DOC_ENGINE', 'elasticsearch')
doc_engine = os.getenv("DOC_ENGINE", "elasticsearch")
result = []
configs = SERVICE_CONFIGS.configs
for service_id, config in enumerate(configs):
config_dict = config.to_dict()
if config_dict['service_type'] == 'retrieval':
if config_dict['extra']['retrieval_type'] != doc_engine:
if config_dict["service_type"] == "retrieval":
if config_dict["extra"]["retrieval_type"] != doc_engine:
continue
try:
service_detail = ServiceMgr.get_service_details(service_id)
if "status" in service_detail:
config_dict['status'] = service_detail['status']
config_dict["status"] = service_detail["status"]
else:
config_dict['status'] = 'timeout'
config_dict["status"] = "timeout"
except Exception as e:
logging.warning(f"Can't get service details, error: {e}")
config_dict['status'] = 'timeout'
if not config_dict['host']:
config_dict['host'] = '-'
if not config_dict['port']:
config_dict['port'] = '-'
config_dict["status"] = "timeout"
if not config_dict["host"]:
config_dict["host"] = "-"
if not config_dict["port"]:
config_dict["port"] = "-"
result.append(config_dict)
return result
@@ -217,11 +306,18 @@ class ServiceMgr:
raise AdminException(f"invalid service_index: {service_idx}")
service_config = configs[service_idx]
service_info = {'name': service_config.name, 'detail_func_name': service_config.detail_func_name}
detail_func = getattr(health_utils, service_info.get('detail_func_name'))
# exclude retrieval service if retrieval_type is not matched
doc_engine = os.getenv("DOC_ENGINE", "elasticsearch")
if service_config.service_type == "retrieval":
if service_config.retrieval_type != doc_engine:
raise AdminException(f"invalid service_index: {service_idx}")
service_info = {"name": service_config.name, "detail_func_name": service_config.detail_func_name}
detail_func = getattr(health_utils, service_info.get("detail_func_name"))
res = detail_func()
res.update({'service_name': service_info.get('name')})
res.update({"service_name": service_info.get("name")})
return res
@staticmethod
@@ -231,3 +327,397 @@ class ServiceMgr:
@staticmethod
def restart_service(service_id: int):
raise AdminException("restart_service: not implemented")
class SettingsMgr:
@staticmethod
def get_all():
settings = SystemSettingsService.get_all()
result = []
for setting in settings:
result.append(
{
"name": setting.name,
"source": setting.source,
"data_type": setting.data_type,
"value": setting.value,
}
)
return result
@staticmethod
def get_by_name(name: str):
settings = SystemSettingsService.get_by_name(name)
if len(settings) == 0:
raise AdminException(f"Can't get setting: {name}")
result = []
for setting in settings:
result.append(
{
"name": setting.name,
"source": setting.source,
"data_type": setting.data_type,
"value": setting.value,
}
)
return result
@staticmethod
def update_by_name(name: str, value: str):
settings = SystemSettingsService.get_by_name(name)
if len(settings) == 1:
setting = settings[0]
setting.value = value
setting_dict = setting.to_dict()
SystemSettingsService.update_by_name(name, setting_dict)
elif len(settings) > 1:
raise AdminException(f"Can't update more than 1 setting: {name}")
else:
# Create new setting if it doesn't exist
# Determine data_type based on name and value
if name.startswith("sandbox."):
data_type = "json"
elif name.endswith(".enabled"):
data_type = "boolean"
else:
data_type = "string"
new_setting = {
"name": name,
"value": str(value),
"source": "admin",
"data_type": data_type,
}
SystemSettingsService.save(**new_setting)
class ConfigMgr:
@staticmethod
def get_all():
result = []
configs = SERVICE_CONFIGS.configs
for config in configs:
config_dict = config.to_dict()
result.append(config_dict)
return result
class EnvironmentsMgr:
@staticmethod
def get_all():
result = []
env_kv = {"env": "DOC_ENGINE", "value": os.getenv("DOC_ENGINE")}
result.append(env_kv)
env_kv = {"env": "DEFAULT_SUPERUSER_EMAIL", "value": os.getenv("DEFAULT_SUPERUSER_EMAIL", "admin@ragflow.io")}
result.append(env_kv)
env_kv = {"env": "DB_TYPE", "value": os.getenv("DB_TYPE", "mysql")}
result.append(env_kv)
env_kv = {"env": "DEVICE", "value": os.getenv("DEVICE", "cpu")}
result.append(env_kv)
env_kv = {"env": "STORAGE_IMPL", "value": os.getenv("STORAGE_IMPL", "MINIO")}
result.append(env_kv)
return result
class SandboxMgr:
"""Manager for sandbox provider configuration and operations."""
# Provider registry with metadata
PROVIDER_REGISTRY = {
"self_managed": {
"name": "Self-Managed",
"description": "On-premise deployment using Daytona/Docker",
"tags": ["self-hosted", "low-latency", "secure"],
},
"aliyun_codeinterpreter": {
"name": "Aliyun Code Interpreter",
"description": "Aliyun Function Compute Code Interpreter - Code execution in serverless microVMs",
"tags": ["saas", "cloud", "scalable", "aliyun"],
},
"e2b": {
"name": "E2B",
"description": "E2B Cloud - Code Execution Sandboxes",
"tags": ["saas", "fast", "global"],
},
}
@staticmethod
def list_providers():
"""List all available sandbox providers."""
result = []
for provider_id, metadata in SandboxMgr.PROVIDER_REGISTRY.items():
result.append({
"id": provider_id,
**metadata
})
return result
@staticmethod
def get_provider_config_schema(provider_id: str):
"""Get configuration schema for a specific provider."""
from agent.sandbox.providers import (
SelfManagedProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
schemas = {
"self_managed": SelfManagedProvider.get_config_schema(),
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider.get_config_schema(),
"e2b": E2BProvider.get_config_schema(),
}
if provider_id not in schemas:
raise AdminException(f"Unknown provider: {provider_id}")
return schemas.get(provider_id, {})
@staticmethod
def get_config():
"""Get current sandbox configuration."""
try:
# Get active provider type
provider_type_settings = SystemSettingsService.get_by_name("sandbox.provider_type")
if not provider_type_settings:
# Return default config if not set
provider_type = "self_managed"
else:
provider_type = provider_type_settings[0].value
# Get provider-specific config
provider_config_settings = SystemSettingsService.get_by_name(f"sandbox.{provider_type}")
if not provider_config_settings:
provider_config = {}
else:
try:
provider_config = json.loads(provider_config_settings[0].value)
except json.JSONDecodeError:
provider_config = {}
return {
"provider_type": provider_type,
"config": provider_config,
}
except Exception as e:
raise AdminException(f"Failed to get sandbox config: {str(e)}")
@staticmethod
def set_config(provider_type: str, config: dict, set_active: bool = True):
"""
Set sandbox provider configuration.
Args:
provider_type: Provider identifier (e.g., "self_managed", "e2b")
config: Provider configuration dictionary
set_active: If True, also update the active provider. If False,
only update the configuration without switching providers.
Default: True
Returns:
Dictionary with updated provider_type and config
"""
from agent.sandbox.providers import (
SelfManagedProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
try:
# Validate provider type
if provider_type not in SandboxMgr.PROVIDER_REGISTRY:
raise AdminException(f"Unknown provider type: {provider_type}")
# Get provider schema for validation
schema = SandboxMgr.get_provider_config_schema(provider_type)
# Validate config against schema
for field_name, field_schema in schema.items():
if field_schema.get("required", False) and field_name not in config:
raise AdminException(f"Required field '{field_name}' is missing")
# Type validation
if field_name in config:
field_type = field_schema.get("type")
if field_type == "integer":
if not isinstance(config[field_name], int):
raise AdminException(f"Field '{field_name}' must be an integer")
elif field_type == "string":
if not isinstance(config[field_name], str):
raise AdminException(f"Field '{field_name}' must be a string")
elif field_type == "bool":
if not isinstance(config[field_name], bool):
raise AdminException(f"Field '{field_name}' must be a boolean")
# Range validation for integers
if field_type == "integer" and field_name in config:
min_val = field_schema.get("min")
max_val = field_schema.get("max")
if min_val is not None and config[field_name] < min_val:
raise AdminException(f"Field '{field_name}' must be >= {min_val}")
if max_val is not None and config[field_name] > max_val:
raise AdminException(f"Field '{field_name}' must be <= {max_val}")
# Provider-specific custom validation
provider_classes = {
"self_managed": SelfManagedProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
}
provider = provider_classes[provider_type]()
is_valid, error_msg = provider.validate_config(config)
if not is_valid:
raise AdminException(f"Provider validation failed: {error_msg}")
# Update provider_type only if set_active is True
if set_active:
SettingsMgr.update_by_name("sandbox.provider_type", provider_type)
# Always update the provider config
config_json = json.dumps(config)
SettingsMgr.update_by_name(f"sandbox.{provider_type}", config_json)
return {"provider_type": provider_type, "config": config}
except AdminException:
raise
except Exception as e:
raise AdminException(f"Failed to set sandbox config: {str(e)}")
@staticmethod
def test_connection(provider_type: str, config: dict):
"""
Test connection to sandbox provider by executing a simple Python script.
This creates a temporary sandbox instance and runs a test code to verify:
- Connection credentials are valid
- Sandbox can be created
- Code execution works correctly
Args:
provider_type: Provider identifier
config: Provider configuration dictionary
Returns:
dict with test results including stdout, stderr, exit_code, execution_time
"""
try:
from agent.sandbox.providers import (
SelfManagedProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
# Instantiate provider based on type
provider_classes = {
"self_managed": SelfManagedProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
}
if provider_type not in provider_classes:
raise AdminException(f"Unknown provider type: {provider_type}")
provider = provider_classes[provider_type]()
# Initialize with config
if not provider.initialize(config):
raise AdminException(f"Failed to initialize provider '{provider_type}'")
# Create a temporary sandbox instance for testing
instance = provider.create_instance(template="python")
if not instance or instance.status != "READY":
raise AdminException(f"Failed to create sandbox instance. Status: {instance.status if instance else 'None'}")
# Simple test code that exercises basic Python functionality
test_code = """
# Test basic Python functionality
import sys
import json
import math
print("Python version:", sys.version)
print("Platform:", sys.platform)
# Test basic calculations
result = 2 + 2
print(f"2 + 2 = {result}")
# Test JSON operations
data = {"test": "data", "value": 123}
print(f"JSON dump: {json.dumps(data)}")
# Test math operations
print(f"Math.sqrt(16) = {math.sqrt(16)}")
# Test error handling
try:
x = 1 / 1
print("Division test: OK")
except Exception as e:
print(f"Error: {e}")
# Return success indicator
print("TEST_PASSED")
"""
# Execute test code with timeout
execution_result = provider.execute_code(
instance_id=instance.instance_id,
code=test_code,
language="python",
timeout=10 # 10 seconds timeout
)
# Clean up the test instance (if provider supports it)
try:
if hasattr(provider, 'terminate_instance'):
provider.terminate_instance(instance.instance_id)
logging.info(f"Cleaned up test instance {instance.instance_id}")
else:
logging.warning(f"Provider {provider_type} does not support terminate_instance, test instance may leak")
except Exception as cleanup_error:
logging.warning(f"Failed to cleanup test instance {instance.instance_id}: {cleanup_error}")
# Build detailed result message
success = execution_result.exit_code == 0 and "TEST_PASSED" in execution_result.stdout
message_parts = [
f"Test {success and 'PASSED' or 'FAILED'}",
f"Exit code: {execution_result.exit_code}",
f"Execution time: {execution_result.execution_time:.2f}s"
]
if execution_result.stdout.strip():
stdout_preview = execution_result.stdout.strip()[:200]
message_parts.append(f"Output: {stdout_preview}...")
if execution_result.stderr.strip():
stderr_preview = execution_result.stderr.strip()[:200]
message_parts.append(f"Errors: {stderr_preview}...")
message = " | ".join(message_parts)
return {
"success": success,
"message": message,
"details": {
"exit_code": execution_result.exit_code,
"execution_time": execution_result.execution_time,
"stdout": execution_result.stdout,
"stderr": execution_result.stderr,
}
}
except AdminException:
raise
except Exception as e:
import traceback
error_details = traceback.format_exc()
raise AdminException(f"Connection test failed: {str(e)}\\n\\nStack trace:\\n{error_details}")

View File

@@ -15,6 +15,7 @@
#
import asyncio
import base64
import datetime
import inspect
import binascii
import json
@@ -28,9 +29,11 @@ from typing import Any, Union, Tuple
from agent.component import component_class
from agent.component.base import ComponentBase
from agent.dsl_migration import normalize_chunker_dsl
from api.db.services.file_service import FileService
from api.db.services.llm_service import LLMBundle
from api.db.services.task_service import has_canceled
from api.db.joint_services.tenant_model_service import get_tenant_default_model_by_type
from common.constants import LLMType
from common.misc_utils import get_uuid, hash_str2int
from common.exceptions import TaskCanceledException
@@ -78,13 +81,15 @@ class Graph:
}
"""
def __init__(self, dsl: str, tenant_id=None, task_id=None):
def __init__(self, dsl: str, tenant_id=None, task_id=None, custom_header=None):
self.path = []
self.components = {}
self.error = ""
self.dsl = json.loads(dsl)
# Accept legacy DSL on read, but keep the in-memory canvas in the latest schema.
self.dsl = normalize_chunker_dsl(json.loads(dsl))
self._tenant_id = tenant_id
self.task_id = task_id if task_id else get_uuid()
self.custom_header = custom_header
self._thread_pool = ThreadPoolExecutor(max_workers=5)
self.load()
@@ -94,6 +99,7 @@ class Graph:
for k, cpn in self.components.items():
cpn_nms.add(cpn["obj"]["component_name"])
param = component_class(cpn["obj"]["component_name"] + "Param")()
cpn["obj"]["params"]["custom_header"] = self.custom_header
param.update(cpn["obj"]["params"])
try:
param.check()
@@ -278,15 +284,17 @@ class Graph:
class Canvas(Graph):
def __init__(self, dsl: str, tenant_id=None, task_id=None, canvas_id=None):
def __init__(self, dsl: str, tenant_id=None, task_id=None, canvas_id=None, custom_header=None):
self.globals = {
"sys.query": "",
"sys.user_id": tenant_id,
"sys.conversation_turns": 0,
"sys.files": []
"sys.files": [],
"sys.history": [],
"sys.date": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
}
self.variables = {}
super().__init__(dsl, tenant_id, task_id)
super().__init__(dsl, tenant_id, task_id, custom_header=custom_header)
self._id = canvas_id
def load(self):
@@ -294,12 +302,18 @@ class Canvas(Graph):
self.history = self.dsl["history"]
if "globals" in self.dsl:
self.globals = self.dsl["globals"]
if "sys.history" not in self.globals:
self.globals["sys.history"] = []
if "sys.date" not in self.globals:
self.globals["sys.date"] = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
else:
self.globals = {
"sys.query": "",
"sys.user_id": "",
"sys.conversation_turns": 0,
"sys.files": []
"sys.files": [],
"sys.history": [],
"sys.date": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
}
if "variables" in self.dsl:
self.variables = self.dsl["variables"]
@@ -340,32 +354,35 @@ class Canvas(Graph):
key = k[4:]
if key in self.variables:
variable = self.variables[key]
if variable["value"]:
self.globals[k] = variable["value"]
value = variable.get("value")
if value is not None:
self.globals[k] = value
else:
if variable["type"] == "string":
self.globals[k] = ""
elif variable["type"] == "number":
var_type = variable.get("type", "")
if var_type == "number":
self.globals[k] = 0
elif variable["type"] == "boolean":
elif var_type == "boolean":
self.globals[k] = False
elif variable["type"] == "object":
elif var_type == "object":
self.globals[k] = {}
elif variable["type"].startswith("array"):
elif var_type.startswith("array"):
self.globals[k] = []
else:
else: # "string" or unknown
self.globals[k] = ""
else:
self.globals[k] = ""
async def run(self, **kwargs):
self.globals["sys.date"] = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
st = time.perf_counter()
self._loop = asyncio.get_running_loop()
self.message_id = get_uuid()
created_at = int(time.time())
self.add_user_input(kwargs.get("query"))
path_set = set(self.path)
for k, cpn in self.components.items():
self.components[k]["obj"].reset(True)
if k in path_set:
self.components[k]["obj"].reset(True)
if kwargs.get("webhook_payload"):
for k, cpn in self.components.items():
@@ -378,10 +395,16 @@ class Canvas(Graph):
continue
self.components[k]["obj"].set_output(kk, vv)
layout_recognize = None
for cpn in self.components.values():
if cpn["obj"].component_name.lower() == "begin":
layout_recognize = getattr(cpn["obj"]._param, "layout_recognize", None)
break
for k in kwargs.keys():
if k in ["query", "user_id", "files"] and kwargs[k]:
if k == "files":
self.globals[f"sys.{k}"] = await self.get_files_async(kwargs[k])
self.globals[f"sys.{k}"] = await self.get_files_async(kwargs[k], layout_recognize)
else:
self.globals[f"sys.{k}"] = kwargs[k]
if not self.globals["sys.conversation_turns"] :
@@ -419,9 +442,15 @@ class Canvas(Graph):
loop = asyncio.get_running_loop()
tasks = []
max_concurrency = getattr(self._thread_pool, "_max_workers", 5)
sem = asyncio.Semaphore(max_concurrency)
def _run_async_in_thread(coro_func, **call_kwargs):
return asyncio.run(coro_func(**call_kwargs))
async def _invoke_one(cpn_obj, sync_fn, call_kwargs, use_async: bool):
async with sem:
if use_async:
await cpn_obj.invoke_async(**(call_kwargs or {}))
return
await loop.run_in_executor(self._thread_pool, partial(sync_fn, **(call_kwargs or {})))
i = f
while i < t:
@@ -447,11 +476,9 @@ class Canvas(Graph):
if task_fn is None:
continue
invoke_async = getattr(cpn, "invoke_async", None)
if invoke_async and asyncio.iscoroutinefunction(invoke_async):
tasks.append(loop.run_in_executor(self._thread_pool, partial(_run_async_in_thread, invoke_async, **(call_kwargs or {}))))
else:
tasks.append(loop.run_in_executor(self._thread_pool, partial(task_fn, **(call_kwargs or {}))))
fn_invoke_async = getattr(cpn, "_invoke_async", None)
use_async = (fn_invoke_async and asyncio.iscoroutinefunction(fn_invoke_async)) or asyncio.iscoroutinefunction(getattr(cpn, "_invoke", None))
tasks.append(asyncio.create_task(_invoke_one(cpn, task_fn, call_kwargs, use_async)))
if tasks:
await asyncio.gather(*tasks)
@@ -490,7 +517,8 @@ class Canvas(Graph):
cpn_obj = self.get_component_obj(self.path[i])
if cpn_obj.component_name.lower() == "message":
if cpn_obj.get_param("auto_play"):
tts_mdl = LLMBundle(self._tenant_id, LLMType.TTS)
tts_model_config = get_tenant_default_model_by_type(self._tenant_id, LLMType.TTS)
tts_mdl = LLMBundle(self._tenant_id, tts_model_config)
if isinstance(cpn_obj.output("content"), partial):
_m = ""
buff_m = ""
@@ -535,18 +563,10 @@ class Canvas(Graph):
yield decorate("message", {"content": "", "audio_binary": self.tts(tts_mdl, buff_m)})
buff_m = ""
cpn_obj.set_output("content", _m)
cite = re.search(r"\[ID:[ 0-9]+\]", _m)
else:
yield decorate("message", {"content": cpn_obj.output("content")})
cite = re.search(r"\[ID:[ 0-9]+\]", cpn_obj.output("content"))
message_end = {}
if cpn_obj.get_param("status"):
message_end["status"] = cpn_obj.get_param("status")
if isinstance(cpn_obj.output("attachment"), dict):
message_end["attachment"] = cpn_obj.output("attachment")
if cite:
message_end["reference"] = self.get_reference()
message_end = self._build_message_end(cpn_obj)
yield decorate("message_end", message_end)
while partials:
@@ -638,6 +658,7 @@ class Canvas(Graph):
"created_at": st,
})
self.history.append(("assistant", self.get_component_obj(self.path[-1]).output()))
self.globals["sys.history"].append(f"{self.history[-1][0]}: {self.history[-1][1]}")
elif "Task has been canceled" in self.error:
yield decorate("workflow_finished",
{
@@ -715,6 +736,7 @@ class Canvas(Graph):
def add_user_input(self, question):
self.history.append(("user", question))
self.globals["sys.history"].append(f"{self.history[-1][0]}: {self.history[-1][1]}")
def get_prologue(self):
return self.components["begin"]["obj"]._param.prologue
@@ -734,30 +756,33 @@ class Canvas(Graph):
def get_component_input_elements(self, cpnnm):
return self.components[cpnnm]["obj"].get_input_elements()
async def get_files_async(self, files: Union[None, list[dict]]) -> list[str]:
async def get_files_async(self, files: Union[None, list[dict]], layout_recognize: str = None) -> list[str]:
if not files:
return []
def image_to_base64(file):
return "data:{};base64,{}".format(file["mime_type"],
base64.b64encode(FileService.get_blob(file["created_by"], file["id"])).decode("utf-8"))
def parse_file(file):
blob = FileService.get_blob(file["created_by"], file["id"])
return FileService.parse(file["name"], blob, True, file["created_by"], layout_recognize)
loop = asyncio.get_running_loop()
tasks = []
for file in files:
if file["mime_type"].find("image") >=0:
tasks.append(loop.run_in_executor(self._thread_pool, image_to_base64, file))
continue
tasks.append(loop.run_in_executor(self._thread_pool, FileService.parse, file["name"], FileService.get_blob(file["created_by"], file["id"]), True, file["created_by"]))
tasks.append(loop.run_in_executor(self._thread_pool, parse_file, file))
return await asyncio.gather(*tasks)
def get_files(self, files: Union[None, list[dict]]) -> list[str]:
def get_files(self, files: Union[None, list[dict]], layout_recognize: str = None) -> list[str]:
"""
Synchronous wrapper for get_files_async, used by sync component invoke paths.
"""
loop = getattr(self, "_loop", None)
if loop and loop.is_running():
return asyncio.run_coroutine_threadsafe(self.get_files_async(files), loop).result()
return asyncio.run_coroutine_threadsafe(self.get_files_async(files, layout_recognize), loop).result()
return asyncio.run(self.get_files_async(files))
return asyncio.run(self.get_files_async(files, layout_recognize))
def tool_use_callback(self, agent_id: str, func_name: str, params: dict, result: Any, elapsed_time=None):
agent_ids = agent_id.split("-->")
@@ -803,6 +828,22 @@ class Canvas(Graph):
return {"chunks": {}, "doc_aggs": {}}
return self.retrieval[-1]
def _has_reference(self) -> bool:
ref = self.get_reference()
if not isinstance(ref, dict):
return False
return bool(ref.get("chunks") or ref.get("doc_aggs"))
def _build_message_end(self, cpn_obj) -> dict:
message_end = {}
if cpn_obj.get_param("status"):
message_end["status"] = cpn_obj.get_param("status")
if isinstance(cpn_obj.output("attachment"), dict):
message_end["attachment"] = cpn_obj.output("attachment")
if self._has_reference():
message_end["reference"] = self.get_reference()
return message_end
def add_memory(self, user:str, assist:str, summ: str):
self.memory.append((user, assist, summ))

View File

@@ -20,19 +20,20 @@ import os
import re
from copy import deepcopy
from functools import partial
from timeit import default_timer as timer
from typing import Any
import json_repair
from timeit import default_timer as timer
from agent.tools.base import LLMToolPluginCallSession, ToolParamBase, ToolBase, ToolMeta
from agent.component.llm import LLM, LLMParam
from agent.tools.base import LLMToolPluginCallSession, ToolBase, ToolMeta, ToolParamBase
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from api.db.services.llm_service import LLMBundle
from api.db.services.tenant_llm_service import TenantLLMService
from api.db.services.mcp_server_service import MCPServerService
from api.db.services.tenant_llm_service import TenantLLMService
from common.connection_utils import timeout
from rag.prompts.generator import next_step_async, COMPLETE_TASK, \
citation_prompt, kb_prompt, citation_plus, full_question, message_fit_in, structured_output_prompt
from common.mcp_tool_call_conn import MCPToolCallSession, mcp_tool_metadata_to_openai_tool
from agent.component.llm import LLMParam, LLM
from rag.prompts.generator import citation_plus, citation_prompt, full_question, kb_prompt, message_fit_in, structured_output_prompt
class AgentParam(LLMParam, ToolParamBase):
@@ -41,41 +42,32 @@ class AgentParam(LLMParam, ToolParamBase):
"""
def __init__(self):
self.meta:ToolMeta = {
"name": "agent",
"description": "This is an agent for a specific task.",
"parameters": {
"user_prompt": {
"type": "string",
"description": "This is the order you need to send to the agent.",
"default": "",
"required": True
},
"reasoning": {
"type": "string",
"description": (
"Supervisor's reasoning for choosing the this agent. "
"Explain why this agent is being invoked and what is expected of it."
),
"required": True
},
"context": {
"type": "string",
"description": (
"All relevant background information, prior facts, decisions, "
"and state needed by the agent to solve the current query. "
"Should be as detailed and self-contained as possible."
),
"required": True
},
}
}
self.meta: ToolMeta = {
"name": "agent",
"description": "This is an agent for a specific task.",
"parameters": {
"user_prompt": {"type": "string", "description": "This is the order you need to send to the agent.", "default": "", "required": True},
"reasoning": {
"type": "string",
"description": ("Supervisor's reasoning for choosing the this agent. Explain why this agent is being invoked and what is expected of it."),
"required": True,
},
"context": {
"type": "string",
"description": (
"All relevant background information, prior facts, decisions, and state needed by the agent to solve the current query. Should be as detailed and self-contained as possible."
),
"required": True,
},
},
}
super().__init__()
self.function_name = "agent"
self.tools = []
self.mcp = []
self.max_rounds = 5
self.description = ""
self.custom_header = {}
class Agent(LLM, ToolBase):
@@ -89,13 +81,15 @@ class Agent(LLM, ToolBase):
original_name = cpn.get_meta()["function"]["name"]
indexed_name = f"{original_name}_{idx}"
self.tools[indexed_name] = cpn
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error,
max_rounds=self._param.max_rounds,
verbose_tool_use=True
)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id)
self.chat_mdl = LLMBundle(
self._canvas.get_tenant_id(),
chat_model_config,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error,
max_rounds=self._param.max_rounds,
verbose_tool_use=False,
)
self.tool_meta = []
for indexed_name, tool_obj in self.tools.items():
original_meta = tool_obj.get_meta()
@@ -105,16 +99,37 @@ class Agent(LLM, ToolBase):
for mcp in self._param.mcp:
_, mcp_server = MCPServerService.get_by_id(mcp["mcp_id"])
tool_call_session = MCPToolCallSession(mcp_server, mcp_server.variables)
custom_header = self._param.custom_header
tool_call_session = MCPToolCallSession(mcp_server, mcp_server.variables, custom_header)
for tnm, meta in mcp["tools"].items():
self.tool_meta.append(mcp_tool_metadata_to_openai_tool(meta))
self.tools[tnm] = tool_call_session
self.callback = partial(self._canvas.tool_use_callback, id)
self.toolcall_session = LLMToolPluginCallSession(self.tools, self.callback)
#self.chat_mdl.bind_tools(self.toolcall_session, self.tool_metas)
if self.tool_meta:
self.chat_mdl.bind_tools(self.toolcall_session, self.tool_meta)
def _fit_messages(self, prompt: str, msg: list[dict]) -> list[dict]:
_, fitted_messages = message_fit_in(
[{"role": "system", "content": prompt}, *msg],
int(self.chat_mdl.max_length * 0.97),
)
return fitted_messages
@staticmethod
def _append_system_prompt(msg: list[dict], extra_prompt: str) -> None:
if extra_prompt and msg and msg[0]["role"] == "system":
msg[0]["content"] += "\n" + extra_prompt
@staticmethod
def _clean_formatted_answer(ans: str) -> str:
ans = re.sub(r"^.*</think>", "", ans, flags=re.DOTALL)
ans = re.sub(r"^.*```json", "", ans, flags=re.DOTALL)
return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
def _load_tool_obj(self, cpn: dict) -> object:
from agent.component import component_class
tool_name = cpn["component_name"]
param = component_class(tool_name + "Param")()
param.update(cpn["params"])
@@ -127,19 +142,17 @@ class Agent(LLM, ToolBase):
return component_class(cpn["component_name"])(self._canvas, cpn_id, param)
def get_meta(self) -> dict[str, Any]:
self._param.function_name= self._id.split("-->")[-1]
self._param.function_name = self._id.split("-->")[-1]
m = super().get_meta()
if hasattr(self._param, "user_prompt") and self._param.user_prompt:
m["function"]["parameters"]["properties"]["user_prompt"] = self._param.user_prompt
# Keep the JSON schema valid; user_prompt is a string field, not a schema node.
m["function"]["parameters"]["properties"]["user_prompt"]["default"] = self._param.user_prompt
return m
def get_input_form(self) -> dict[str, dict]:
res = {}
for k, v in self.get_input_elements().items():
res[k] = {
"type": "line",
"name": v["name"]
}
res[k] = {"type": "line", "name": v["name"]}
for cpn in self._param.tools:
if not isinstance(cpn, LLM):
continue
@@ -172,7 +185,7 @@ class Agent(LLM, ToolBase):
def _invoke(self, **kwargs):
return asyncio.run(self._invoke_async(**kwargs))
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 20*60)))
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 20 * 60)))
async def _invoke_async(self, **kwargs):
if self.check_if_canceled("Agent processing"):
return
@@ -201,19 +214,17 @@ class Agent(LLM, ToolBase):
schema = json.dumps(output_schema, ensure_ascii=False, indent=2)
schema_prompt = structured_output_prompt(schema)
downstreams = self._canvas.get_component(self._id)["downstream"] if self._canvas.get_component(self._id) else []
component = self._canvas.get_component(self._id)
downstreams = component["downstream"] if component else []
ex = self.exception_handler()
if any([self._canvas.get_component_obj(cid).component_name.lower()=="message" for cid in downstreams]) and not (ex and ex["goto"]) and not output_schema:
has_message_downstream = any(self._canvas.get_component_obj(cid).component_name.lower() == "message" for cid in downstreams)
if has_message_downstream and not (ex and ex["goto"]) and not output_schema:
self.set_output("content", partial(self.stream_output_with_tools_async, prompt, deepcopy(msg), user_defined_prompt))
return
_, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
use_tools = []
ans = ""
async for delta_ans, _tk in self._react_with_tools_streamly_async_simple(prompt, msg, use_tools, user_defined_prompt,schema_prompt=schema_prompt):
if self.check_if_canceled("Agent processing"):
return
ans += delta_ans
msg = self._fit_messages(prompt, msg)
self._append_system_prompt(msg, schema_prompt)
ans = await self._generate_async(msg)
if ans.find("**ERROR**") >= 0:
logging.error(f"Agent._chat got error. response: {ans}")
@@ -227,14 +238,8 @@ class Agent(LLM, ToolBase):
error = ""
for _ in range(self._param.max_retries + 1):
try:
def clean_formated_answer(ans: str) -> str:
ans = re.sub(r"^.*</think>", "", ans, flags=re.DOTALL)
ans = re.sub(r"^.*```json", "", ans, flags=re.DOTALL)
return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
obj = json_repair.loads(clean_formated_answer(ans))
obj = json_repair.loads(self._clean_formatted_answer(ans))
self.set_output("structured", obj)
if use_tools:
self.set_output("use_tools", use_tools)
return obj
except Exception:
error = "The answer cannot be parsed as JSON"
@@ -245,330 +250,95 @@ class Agent(LLM, ToolBase):
self.set_output("_ERROR", error)
return
artifact_md = self._collect_tool_artifact_markdown(existing_text=ans)
if artifact_md:
ans += "\n\n" + artifact_md
self.set_output("content", ans)
if use_tools:
self.set_output("use_tools", use_tools)
return ans
async def stream_output_with_tools_async(self, prompt, msg, user_defined_prompt={}):
_, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
answer_without_toolcall = ""
use_tools = []
async for delta_ans, _ in self._react_with_tools_streamly_async_simple(prompt, msg, use_tools, user_defined_prompt):
if len(msg) > 3:
st = timer()
user_request = await full_question(messages=msg, chat_mdl=self.chat_mdl)
self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer() - st)
msg = [*msg[:-1], {"role": "user", "content": user_request}]
msg = self._fit_messages(prompt, msg)
need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
cited = False
if need2cite and len(msg) < 7:
self._append_system_prompt(msg, citation_prompt())
cited = True
answer = ""
async for delta in self._generate_streamly(msg):
if self.check_if_canceled("Agent streaming"):
return
if delta_ans.find("**ERROR**") >= 0:
if delta.find("**ERROR**") >= 0:
if self.get_exception_default_value():
self.set_output("content", self.get_exception_default_value())
yield self.get_exception_default_value()
fallback = self.get_exception_default_value()
self.set_output("content", fallback)
yield fallback
else:
self.set_output("_ERROR", delta_ans)
return
answer_without_toolcall += delta_ans
yield delta_ans
self.set_output("content", answer_without_toolcall)
if use_tools:
self.set_output("use_tools", use_tools)
async def _react_with_tools_streamly_async_simple(self, prompt, history: list[dict], use_tools, user_defined_prompt={}, schema_prompt: str = ""):
token_count = 0
tool_metas = self.tool_meta
hist = deepcopy(history)
last_calling = ""
if len(hist) > 3:
st = timer()
user_request = await full_question(messages=history, chat_mdl=self.chat_mdl)
self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer()-st)
else:
user_request = history[-1]["content"]
def build_task_desc(prompt: str, user_request: str, user_defined_prompt: dict | None = None) -> str:
"""Build a minimal task_desc by concatenating prompt, query, and tool schemas."""
user_defined_prompt = user_defined_prompt or {}
task_desc = (
"### Agent Prompt\n"
f"{prompt}\n\n"
"### User Request\n"
f"{user_request}\n\n"
)
if user_defined_prompt:
udp_json = json.dumps(user_defined_prompt, ensure_ascii=False, indent=2)
task_desc += "\n### User Defined Prompts\n" + udp_json + "\n"
return task_desc
async def use_tool_async(name, args):
nonlocal hist, use_tools, last_calling
logging.info(f"{last_calling=} == {name=}")
last_calling = name
tool_response = await self.toolcall_session.tool_call_async(name, args)
use_tools.append({
"name": name,
"arguments": args,
"results": tool_response
})
return name, tool_response
async def complete():
nonlocal hist
need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
if schema_prompt:
need2cite = False
cited = False
if hist and hist[0]["role"] == "system":
if schema_prompt:
hist[0]["content"] += "\n" + schema_prompt
if need2cite and len(hist) < 7:
hist[0]["content"] += citation_prompt()
cited = True
yield "", token_count
_hist = hist
if len(hist) > 12:
_hist = [hist[0], hist[1], *hist[-10:]]
entire_txt = ""
async for delta_ans in self._generate_streamly(_hist):
if not need2cite or cited:
yield delta_ans, 0
entire_txt += delta_ans
if not need2cite or cited:
self.set_output("_ERROR", delta)
self.set_output("content", delta)
yield delta
return
if not need2cite or cited:
yield delta
answer += delta
st = timer()
txt = ""
async for delta_ans in self._gen_citations_async(entire_txt):
if self.check_if_canceled("Agent streaming"):
return
yield delta_ans, 0
txt += delta_ans
self.callback("gen_citations", {}, txt, elapsed_time=timer()-st)
def build_observation(tool_call_res: list[tuple]) -> str:
"""
Build a Observation from tool call results.
No LLM involved.
"""
if not tool_call_res:
return ""
lines = ["Observation:"]
for name, result in tool_call_res:
lines.append(f"[{name} result]")
lines.append(str(result))
return "\n".join(lines)
def append_user_content(hist, content):
if hist[-1]["role"] == "user":
hist[-1]["content"] += content
else:
hist.append({"role": "user", "content": content})
if not need2cite or cited:
artifact_md = self._collect_tool_artifact_markdown(existing_text=answer)
if artifact_md:
yield "\n\n" + artifact_md
answer += "\n\n" + artifact_md
self.set_output("content", answer)
return
st = timer()
task_desc = build_task_desc(prompt, user_request, user_defined_prompt)
self.callback("analyze_task", {}, task_desc, elapsed_time=timer()-st)
for _ in range(self._param.max_rounds + 1):
cited_answer = ""
async for delta in self._gen_citations_async(answer):
if self.check_if_canceled("Agent streaming"):
return
response, tk = await next_step_async(self.chat_mdl, hist, tool_metas, task_desc, user_defined_prompt)
# self.callback("next_step", {}, str(response)[:256]+"...")
token_count += tk or 0
hist.append({"role": "assistant", "content": response})
try:
functions = json_repair.loads(re.sub(r"```.*", "", response))
if not isinstance(functions, list):
raise TypeError(f"List should be returned, but `{functions}`")
for f in functions:
if not isinstance(f, dict):
raise TypeError(f"An object type should be returned, but `{f}`")
tool_tasks = []
for func in functions:
name = func["name"]
args = func["arguments"]
if name == COMPLETE_TASK:
append_user_content(hist, f"Respond with a formal answer. FORGET(DO NOT mention) about `{COMPLETE_TASK}`. The language for the response MUST be as the same as the first user request.\n")
async for txt, tkcnt in complete():
yield txt, tkcnt
return
tool_tasks.append(asyncio.create_task(use_tool_async(name, args)))
results = await asyncio.gather(*tool_tasks) if tool_tasks else []
st = timer()
reflection = build_observation(results)
append_user_content(hist, reflection)
self.callback("reflection", {}, str(reflection), elapsed_time=timer()-st)
except Exception as e:
logging.exception(msg=f"Wrong JSON argument format in LLM ReAct response: {e}")
e = f"\nTool call error, please correct the input parameter of response format and call it again.\n *** Exception ***\n{e}"
append_user_content(hist, str(e))
logging.warning( f"Exceed max rounds: {self._param.max_rounds}")
final_instruction = f"""
{user_request}
IMPORTANT: You have reached the conversation limit. Based on ALL the information and research you have gathered so far, please provide a DIRECT and COMPREHENSIVE final answer to the original request.
Instructions:
1. SYNTHESIZE all information collected during this conversation
2. Provide a COMPLETE response using existing data - do not suggest additional research
3. Structure your response as a FINAL DELIVERABLE, not a plan
4. If information is incomplete, state what you found and provide the best analysis possible with available data
5. DO NOT mention conversation limits or suggest further steps
6. Focus on delivering VALUE with the information already gathered
Respond immediately with your final comprehensive answer.
"""
if self.check_if_canceled("Agent final instruction"):
return
append_user_content(hist, final_instruction)
async for txt, tkcnt in complete():
yield txt, tkcnt
# async def _react_with_tools_streamly_async(self, prompt, history: list[dict], use_tools, user_defined_prompt={}, schema_prompt: str = ""):
# token_count = 0
# tool_metas = self.tool_meta
# hist = deepcopy(history)
# last_calling = ""
# if len(hist) > 3:
# st = timer()
# user_request = await full_question(messages=history, chat_mdl=self.chat_mdl)
# self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer()-st)
# else:
# user_request = history[-1]["content"]
# async def use_tool_async(name, args):
# nonlocal hist, use_tools, last_calling
# logging.info(f"{last_calling=} == {name=}")
# last_calling = name
# tool_response = await self.toolcall_session.tool_call_async(name, args)
# use_tools.append({
# "name": name,
# "arguments": args,
# "results": tool_response
# })
# # self.callback("add_memory", {}, "...")
# #self.add_memory(hist[-2]["content"], hist[-1]["content"], name, args, str(tool_response), user_defined_prompt)
# return name, tool_response
# async def complete():
# nonlocal hist
# need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
# if schema_prompt:
# need2cite = False
# cited = False
# if hist and hist[0]["role"] == "system":
# if schema_prompt:
# hist[0]["content"] += "\n" + schema_prompt
# if need2cite and len(hist) < 7:
# hist[0]["content"] += citation_prompt()
# cited = True
# yield "", token_count
# _hist = hist
# if len(hist) > 12:
# _hist = [hist[0], hist[1], *hist[-10:]]
# entire_txt = ""
# async for delta_ans in self._generate_streamly(_hist):
# if not need2cite or cited:
# yield delta_ans, 0
# entire_txt += delta_ans
# if not need2cite or cited:
# return
# st = timer()
# txt = ""
# async for delta_ans in self._gen_citations_async(entire_txt):
# if self.check_if_canceled("Agent streaming"):
# return
# yield delta_ans, 0
# txt += delta_ans
# self.callback("gen_citations", {}, txt, elapsed_time=timer()-st)
# def append_user_content(hist, content):
# if hist[-1]["role"] == "user":
# hist[-1]["content"] += content
# else:
# hist.append({"role": "user", "content": content})
# st = timer()
# task_desc = await analyze_task_async(self.chat_mdl, prompt, user_request, tool_metas, user_defined_prompt)
# self.callback("analyze_task", {}, task_desc, elapsed_time=timer()-st)
# for _ in range(self._param.max_rounds + 1):
# if self.check_if_canceled("Agent streaming"):
# return
# response, tk = await next_step_async(self.chat_mdl, hist, tool_metas, task_desc, user_defined_prompt)
# # self.callback("next_step", {}, str(response)[:256]+"...")
# token_count += tk or 0
# hist.append({"role": "assistant", "content": response})
# try:
# functions = json_repair.loads(re.sub(r"```.*", "", response))
# if not isinstance(functions, list):
# raise TypeError(f"List should be returned, but `{functions}`")
# for f in functions:
# if not isinstance(f, dict):
# raise TypeError(f"An object type should be returned, but `{f}`")
# tool_tasks = []
# for func in functions:
# name = func["name"]
# args = func["arguments"]
# if name == COMPLETE_TASK:
# append_user_content(hist, f"Respond with a formal answer. FORGET(DO NOT mention) about `{COMPLETE_TASK}`. The language for the response MUST be as the same as the first user request.\n")
# async for txt, tkcnt in complete():
# yield txt, tkcnt
# return
# tool_tasks.append(asyncio.create_task(use_tool_async(name, args)))
# results = await asyncio.gather(*tool_tasks) if tool_tasks else []
# st = timer()
# reflection = await reflect_async(self.chat_mdl, hist, results, user_defined_prompt)
# append_user_content(hist, reflection)
# self.callback("reflection", {}, str(reflection), elapsed_time=timer()-st)
# except Exception as e:
# logging.exception(msg=f"Wrong JSON argument format in LLM ReAct response: {e}")
# e = f"\nTool call error, please correct the input parameter of response format and call it again.\n *** Exception ***\n{e}"
# append_user_content(hist, str(e))
# logging.warning( f"Exceed max rounds: {self._param.max_rounds}")
# final_instruction = f"""
# {user_request}
# IMPORTANT: You have reached the conversation limit. Based on ALL the information and research you have gathered so far, please provide a DIRECT and COMPREHENSIVE final answer to the original request.
# Instructions:
# 1. SYNTHESIZE all information collected during this conversation
# 2. Provide a COMPLETE response using existing data - do not suggest additional research
# 3. Structure your response as a FINAL DELIVERABLE, not a plan
# 4. If information is incomplete, state what you found and provide the best analysis possible with available data
# 5. DO NOT mention conversation limits or suggest further steps
# 6. Focus on delivering VALUE with the information already gathered
# Respond immediately with your final comprehensive answer.
# """
# if self.check_if_canceled("Agent final instruction"):
# return
# append_user_content(hist, final_instruction)
# async for txt, tkcnt in complete():
# yield txt, tkcnt
yield delta
cited_answer += delta
artifact_md = self._collect_tool_artifact_markdown(existing_text=cited_answer)
if artifact_md:
yield "\n\n" + artifact_md
cited_answer += "\n\n" + artifact_md
self.callback("gen_citations", {}, cited_answer, elapsed_time=timer() - st)
self.set_output("content", cited_answer)
async def _gen_citations_async(self, text):
retrievals = self._canvas.get_reference()
retrievals = {"chunks": list(retrievals["chunks"].values()), "doc_aggs": list(retrievals["doc_aggs"].values())}
formated_refer = kb_prompt(retrievals, self.chat_mdl.max_length, True)
async for delta_ans in self._generate_streamly([{"role": "system", "content": citation_plus("\n\n".join(formated_refer))},
{"role": "user", "content": text}
]):
async for delta_ans in self._generate_streamly([{"role": "system", "content": citation_plus("\n\n".join(formated_refer))}, {"role": "user", "content": text}]):
yield delta_ans
def _collect_tool_artifact_markdown(self, existing_text: str = "") -> str:
md_parts = []
for tool_obj in self.tools.values():
if not hasattr(tool_obj, "_param") or not hasattr(tool_obj._param, "outputs"):
continue
artifacts_meta = tool_obj._param.outputs.get("_ARTIFACTS", {})
artifacts = artifacts_meta.get("value") if isinstance(artifacts_meta, dict) else None
if not artifacts:
continue
for art in artifacts:
if not isinstance(art, dict):
continue
url = art.get("url", "")
if url and (f"![]({url})" in existing_text or f"![{art.get('name', '')}]({url})" in existing_text):
continue
if art.get("mime_type", "").startswith("image/"):
md_parts.append(f"![{art['name']}]({url})")
else:
md_parts.append(f"[Download {art['name']}]({url})")
return "\n\n".join(md_parts)
def reset(self, only_output=False):
"""
Reset all tools if they have a reset method. This avoids errors for tools like MCPToolCallSession.

View File

@@ -27,6 +27,10 @@ import pandas as pd
from agent import settings
from common.connection_utils import timeout
from common.misc_utils import thread_pool_exec
_FEEDED_DEPRECATED_PARAMS = "_feeded_deprecated_params"
_DEPRECATED_PARAMS = "_deprecated_params"
_USER_FEEDED_PARAMS = "_user_feeded_params"
@@ -362,6 +366,7 @@ class ComponentBase(ABC):
component_name: str
thread_limiter = asyncio.Semaphore(int(os.environ.get("MAX_CONCURRENT_CHATS", 10)))
variable_ref_patt = r"\{* *\{([a-zA-Z:0-9]+@[A-Za-z0-9_.-]+|sys\.[A-Za-z0-9_.]+|env\.[A-Za-z0-9_.]+)\} *\}*"
iteration_alias_patt = r"\{* *\{(item|index|result)\} *\}*"
def __str__(self):
"""
@@ -379,6 +384,7 @@ class ComponentBase(ABC):
def __init__(self, canvas, id, param: ComponentParamBase):
from agent.canvas import Graph # Local import to avoid cyclic dependency
assert isinstance(canvas, Graph), "canvas must be an instance of Canvas"
self._canvas = canvas
self._id = id
@@ -430,7 +436,7 @@ class ComponentBase(ABC):
elif asyncio.iscoroutinefunction(self._invoke):
await self._invoke(**kwargs)
else:
await asyncio.to_thread(self._invoke, **kwargs)
await thread_pool_exec(self._invoke, **kwargs)
except Exception as e:
if self.get_exception_default_value():
self.set_exception_default_value()
@@ -481,6 +487,10 @@ class ComponentBase(ABC):
continue
if isinstance(v, str) and self._canvas.is_reff(v):
self.set_input_value(var, self._canvas.get_variable_value(v))
elif isinstance(v, str) and re.search(self.variable_ref_patt, v):
elements = self.get_input_elements_from_text(v)
kv = {k: e.get('value', '') for k, e in elements.items()}
self.set_input_value(var, self.string_format(v, kv))
else:
self.set_input_value(var, v)
res[var] = self.get_input_value(var)
@@ -492,6 +502,23 @@ class ComponentBase(ABC):
return {var: self.get_input_value(var) for var, o in self.get_input_elements().items()}
def _resolve_iteration_alias_ref(self, exp: str) -> str | None:
if exp not in {"item", "index", "result"}:
return None
parent = self.get_parent()
if not parent or parent.component_name.lower() != "iteration":
return None
for cid, cpn in self._canvas.components.items():
if cpn.get("parent_id") != parent._id:
continue
if cpn["obj"].component_name.lower() != "iterationitem":
continue
return f"{cid}@{exp}"
return None
def get_input_elements_from_text(self, txt: str) -> dict[str, dict[str, str]]:
res = {}
for r in re.finditer(self.variable_ref_patt, txt, flags=re.IGNORECASE | re.DOTALL):
@@ -503,6 +530,20 @@ class ComponentBase(ABC):
"_retrieval": self._canvas.get_variable_value(f"{cpn_id}@_references") if cpn_id else None,
"_cpn_id": cpn_id
}
for r in re.finditer(self.iteration_alias_patt, txt, flags=re.IGNORECASE | re.DOTALL):
exp = r.group(1)
if exp in res:
continue
ref = self._resolve_iteration_alias_ref(exp)
if not ref:
continue
cpn_id, var_nm = ref.split("@", 1)
res[exp] = {
"name": (self._canvas.get_component_name(cpn_id) + f"@{var_nm}"),
"value": self._canvas.get_variable_value(ref),
"_retrieval": self._canvas.get_variable_value(f"{cpn_id}@_references"),
"_cpn_id": cpn_id
}
return res
def get_input_elements(self) -> dict[str, Any]:

View File

@@ -41,15 +41,19 @@ class Begin(UserFillUp):
if self.check_if_canceled("Begin processing"):
return
layout_recognize = self._param.layout_recognize or None
for k, v in kwargs.get("inputs", {}).items():
if self.check_if_canceled("Begin processing"):
return
if isinstance(v, dict) and v.get("type", "").lower().find("file") >=0:
if isinstance(v, dict) and v.get("type", "").lower().find("file") >= 0:
if v.get("optional") and v.get("value", None) is None:
v = None
else:
v = FileService.get_files([v["value"]])
file_value = v["value"]
# Support both single file (backward compatibility) and multiple files
files = file_value if isinstance(file_value, list) else [file_value]
v = FileService.get_files(files, layout_recognize=layout_recognize)
else:
v = v.get("value")
self.set_output(k, v)

View File

@@ -21,6 +21,7 @@ from abc import ABC
from common.constants import LLMType
from api.db.services.llm_service import LLMBundle
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from agent.component.llm import LLMParam, LLM
from common.connection_utils import timeout
from rag.llm.chat_model import ERROR_PREFIX
@@ -97,6 +98,13 @@ Here's description of each category:
class Categorize(LLM, ABC):
component_name = "Categorize"
def get_input_elements(self) -> dict[str, dict]:
query_key = self._param.query or "sys.query"
elements = self.get_input_elements_from_text(f"{{{query_key}}}")
if not elements:
logging.warning(f"[Categorize] input element not detected for query key: {query_key}")
return elements
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 10*60)))
async def _invoke_async(self, **kwargs):
if self.check_if_canceled("Categorize processing"):
@@ -105,14 +113,18 @@ class Categorize(LLM, ABC):
msg = self._canvas.get_history(self._param.message_history_window_size)
if not msg:
msg = [{"role": "user", "content": ""}]
if kwargs.get("sys.query"):
msg[-1]["content"] = kwargs["sys.query"]
self.set_input_value("sys.query", kwargs["sys.query"])
query_key = self._param.query or "sys.query"
if query_key in kwargs:
query_value = kwargs[query_key]
else:
msg[-1]["content"] = self._canvas.get_variable_value(self._param.query)
self.set_input_value(self._param.query, msg[-1]["content"])
query_value = self._canvas.get_variable_value(query_key)
if query_value is None:
query_value = ""
msg[-1]["content"] = query_value
self.set_input_value(query_key, msg[-1]["content"])
self._param.update_prompt()
chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.CHAT, self._param.llm_id)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), LLMType.CHAT, self._param.llm_id)
chat_mdl = LLMBundle(self._canvas.get_tenant_id(), chat_model_config)
user_prompt = """
---- Real Data ----
@@ -137,7 +149,7 @@ class Categorize(LLM, ABC):
category_counts[c] = count
cpn_ids = list(self._param.category_description.items())[-1][1]["to"]
max_category = list(self._param.category_description.keys())[0]
max_category = list(self._param.category_description.keys())[-1]
if any(category_counts.values()):
max_category = max(category_counts.items(), key=lambda x: x[1])[0]
cpn_ids = self._param.category_description[max_category]["to"]

View File

@@ -94,9 +94,9 @@ class DataOperations(ComponentBase,ABC):
def _recursive_eval(self, data):
if isinstance(data, dict):
return {k: self.recursive_eval(v) for k, v in data.items()}
return {k: self._recursive_eval(v) for k, v in data.items()}
if isinstance(data, list):
return [self.recursive_eval(item) for item in data]
return [self._recursive_eval(item) for item in data]
if isinstance(data, str):
try:
if (

File diff suppressed because it is too large Load Diff

View File

@@ -27,6 +27,7 @@ class UserFillUpParam(ComponentParamBase):
super().__init__()
self.enable_tips = True
self.tips = "Please fill up the form"
self.layout_recognize = ""
def check(self) -> bool:
return True
@@ -61,14 +62,18 @@ class UserFillUp(ComponentBase):
content = re.sub(r"\{%s\}"%k, ans, content)
self.set_output("tips", content)
layout_recognize = self._param.layout_recognize or None
for k, v in kwargs.get("inputs", {}).items():
if self.check_if_canceled("UserFillUp processing"):
return
if isinstance(v, dict) and v.get("type", "").lower().find("file") >=0:
if isinstance(v, dict) and v.get("type", "").lower().find("file") >= 0:
if v.get("optional") and v.get("value", None) is None:
v = None
else:
v = FileService.get_files([v["value"]])
file_value = v["value"]
# Support both single file (backward compatibility) and multiple files
files = file_value if isinstance(file_value, list) else [file_value]
v = FileService.get_files(files, layout_recognize=layout_recognize)
else:
v = v.get("value")
self.set_output(k, v)

View File

@@ -19,6 +19,7 @@ import os
import re
import time
from abc import ABC
from functools import partial
import requests
@@ -29,7 +30,7 @@ from deepdoc.parser import HtmlParser
class InvokeParam(ComponentParamBase):
"""
Define the Crawler component parameters.
Define the Invoke component parameters.
"""
def __init__(self):
@@ -41,7 +42,7 @@ class InvokeParam(ComponentParamBase):
self.url = ""
self.timeout = 60
self.clean_html = False
self.datatype = "json" # New parameter to determine data posting type
self.datatype = "json"
def check(self):
self.check_valid_value(self.method.lower(), "Type of content from the crawler", ["get", "post", "put"])
@@ -53,92 +54,196 @@ class InvokeParam(ComponentParamBase):
class Invoke(ComponentBase, ABC):
component_name = "Invoke"
header_variable_ref_patt = r"\{([a-zA-Z_][a-zA-Z0-9_.@-]*)\}"
@staticmethod
def _coerce_json_arg_if_possible(key, value):
raw_value = value
if isinstance(value, str):
try:
value = json.loads(value)
logging.debug(
"Invoke JSON arg coercion succeeded. key=%s parsed_type=%s",
key,
type(value).__name__,
)
except json.JSONDecodeError as exc:
logging.info(
"Invoke JSON arg coercion skipped; value is not valid JSON. key=%s raw=%r error=%s",
key,
raw_value,
exc,
)
return raw_value
try:
json.dumps(value, allow_nan=False)
except (TypeError, ValueError) as exc:
logging.warning(
"Invoke JSON arg is not JSON-serializable. key=%s value_type=%s value=%r error=%s",
key,
type(value).__name__,
value,
exc,
)
raise ValueError(f"Invoke JSON argument '{key}' is not JSON-serializable.") from exc
return value
def get_input_form(self) -> dict[str, dict]:
res = {}
for item in self._param.variables or []:
if not isinstance(item, dict):
continue
ref = (item.get("ref") or "").strip()
if not ref or ref in res:
continue
elements = self.get_input_elements_from_text("{" + ref + "}")
element = elements.get(ref, {})
res[ref] = {
"type": "line",
"name": element.get("name") or item.get("key") or ref,
}
return res
def _resolve_variable_value(self, variable_name: str, kwargs: dict | None = None):
kwargs = kwargs or {}
value = kwargs.get(variable_name, self._canvas.get_variable_value(variable_name))
if isinstance(value, partial):
value = "".join(value())
self.set_input_value(variable_name, value)
return "" if value is None else value
def _render_template(self, content: str, pattern: str, kwargs: dict | None = None, *, flags: int = 0) -> str:
content = content or ""
if not content:
return content
def replace_variable(match_obj):
return str(self._resolve_variable_value(match_obj.group(1), kwargs))
return re.sub(pattern, replace_variable, content, flags=flags)
def _resolve_template_text(self, content: str, kwargs: dict | None = None) -> str:
return self._render_template(content, self.variable_ref_patt, kwargs, flags=re.DOTALL)
def _resolve_header_text(self, content: str, kwargs: dict | None = None) -> str:
# Headers support plain {token} placeholders, so they cannot reuse the canvas variable regex.
return self._render_template(content, self.header_variable_ref_patt, kwargs)
def _resolve_arg_value(self, para: dict, kwargs: dict) -> object:
ref = (para.get("ref") or "").strip()
if ref and (ref in kwargs or self._canvas.get_variable_value(ref) is not None):
return self._resolve_variable_value(ref, kwargs)
if para.get("value") is not None:
value = para["value"]
if isinstance(value, str):
return self._resolve_template_text(value, kwargs)
return value
if ref:
return self._resolve_variable_value(ref, kwargs)
return ""
def _is_json_mode(self) -> bool:
return self._param.datatype.lower() == "json"
def _build_request_args(self, kwargs: dict) -> dict:
args = {}
for para in self._param.variables:
key = para["key"]
value = self._resolve_arg_value(para, kwargs)
if self._is_json_mode():
# JSON mode accepts stringified JSON so complex payloads can be passed through variables.
value = self._coerce_json_arg_if_possible(key, value)
args[key] = value
if para.get("ref"):
self.set_input_value(para["ref"], value)
return args
def _build_url(self, kwargs: dict) -> str:
url = self._resolve_template_text(self._param.url.strip(), kwargs)
if not url.startswith(("http://", "https://")):
url = "http://" + url
return url
def _build_headers(self, kwargs: dict) -> dict:
if not self._param.headers:
return {}
headers = json.loads(self._param.headers)
if not isinstance(headers, dict):
raise ValueError("Invoke headers must be a JSON object.")
return {key: self._resolve_header_text(value, kwargs) if isinstance(value, str) else value for key, value in headers.items()}
def _build_proxies(self) -> dict | None:
if not re.sub(r"https?:?/?/?", "", self._param.proxy):
return None
return {"http": self._param.proxy, "https": self._param.proxy}
def _send_request(self, url: str, args: dict, headers: dict, proxies: dict | None):
method = self._param.method.lower()
request = getattr(requests, method)
request_kwargs = {
"url": url,
"headers": headers,
"proxies": proxies,
"timeout": self._param.timeout,
}
# GET sends query params; POST/PUT send either JSON or form data based on datatype.
if method == "get":
request_kwargs["params"] = args
return request(**request_kwargs)
body_key = "json" if self._is_json_mode() else "data"
request_kwargs[body_key] = args
return request(**request_kwargs)
def _format_response(self, response) -> str:
if not self._param.clean_html:
return response.text
# HtmlParser keeps the Invoke output text-focused when the endpoint returns HTML.
sections = HtmlParser()(None, response.content)
return "\n".join(sections)
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 3)))
def _invoke(self, **kwargs):
if self.check_if_canceled("Invoke processing"):
return
args = {}
for para in self._param.variables:
if para.get("value"):
args[para["key"]] = para["value"]
else:
args[para["key"]] = self._canvas.get_variable_value(para["ref"])
args = self._build_request_args(kwargs)
url = self._build_url(kwargs)
headers = self._build_headers(kwargs)
proxies = self._build_proxies()
url = self._param.url.strip()
def replace_variable(match):
var_name = match.group(1)
try:
value = self._canvas.get_variable_value(var_name)
return str(value or "")
except Exception:
return ""
# {base_url} or {component_id@variable_name}
url = re.sub(r"\{([a-zA-Z_][a-zA-Z0-9_.@-]*)\}", replace_variable, url)
if url.find("http") != 0:
url = "http://" + url
method = self._param.method.lower()
headers = {}
if self._param.headers:
headers = json.loads(self._param.headers)
proxies = None
if re.sub(r"https?:?/?/?", "", self._param.proxy):
proxies = {"http": self._param.proxy, "https": self._param.proxy}
last_e = ""
last_error = None
for _ in range(self._param.max_retries + 1):
if self.check_if_canceled("Invoke processing"):
return
try:
if method == "get":
response = requests.get(url=url, params=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
sections = HtmlParser()(None, response.content)
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
if method == "put":
if self._param.datatype.lower() == "json":
response = requests.put(url=url, json=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
else:
response = requests.put(url=url, data=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
sections = HtmlParser()(None, response.content)
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
if method == "post":
if self._param.datatype.lower() == "json":
response = requests.post(url=url, json=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
else:
response = requests.post(url=url, data=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
return self.output("result")
response = self._send_request(url, args, headers, proxies)
result = self._format_response(response)
self.set_output("result", result)
return result
except Exception as e:
if self.check_if_canceled("Invoke processing"):
return
last_e = e
last_error = e
logging.exception(f"Http request error: {e}")
time.sleep(self._param.delay_after_error)
if last_e:
self.set_output("_ERROR", str(last_e))
return f"Http request error: {last_e}"
assert False, self.output()
if last_error:
self.set_output("_ERROR", str(last_error))
return f"Http request error: {last_error}"
def thoughts(self) -> str:
return "Waiting for the server respond..."

View File

@@ -54,7 +54,11 @@ class IterationItem(ComponentBase, ABC):
if self.check_if_canceled("IterationItem processing"):
return
self.set_output("item", arr[self._idx])
current_item = arr[self._idx]
self.set_output("item", current_item)
# Keep `result` as a compatibility alias because existing DSL examples
# and downstream references may still consume IterationItem via `@result`.
self.set_output("result", current_item)
self.set_output("index", self._idx)
self._idx += 1
@@ -69,7 +73,7 @@ class IterationItem(ComponentBase, ABC):
if p._id != pid:
continue
if p.component_name.lower() in ["categorize", "message", "switch", "userfillup", "interationitem"]:
if p.component_name.lower() in ["categorize", "message", "switch", "userfillup", "iterationitem"]:
continue
for k, o in p._param.outputs.items():

View File

@@ -10,8 +10,9 @@ class ListOperationsParam(ComponentParamBase):
def __init__(self):
super().__init__()
self.query = ""
self.operations = "topN"
self.n=0
self.operations = "nth"
self.n = 0
self.strict = False
self.sort_method = "asc"
self.filter = {
"operator": "=",
@@ -34,7 +35,11 @@ class ListOperationsParam(ComponentParamBase):
def check(self):
self.check_empty(self.query, "query")
self.check_valid_value(self.operations, "Support operations", ["topN","head","tail","filter","sort","drop_duplicates"])
self.check_valid_value(
self.operations,
"Support operations",
["nth", "head", "tail", "filter", "sort", "drop_duplicates"],
)
def get_input_form(self) -> dict[str, dict]:
return {}
@@ -51,8 +56,8 @@ class ListOperations(ComponentBase,ABC):
if not isinstance(self.inputs, list):
raise TypeError("The input of List Operations should be an array.")
self.set_input_value(inputs, self.inputs)
if self._param.operations == "topN":
self._topN()
if self._param.operations == "nth":
self._nth()
elif self._param.operations == "head":
self._head()
elif self._param.operations == "tail":
@@ -70,35 +75,74 @@ class ListOperations(ComponentBase,ABC):
return int(getattr(self._param, "n", 0))
except Exception:
return 0
def _is_strict(self):
strict = getattr(self._param, "strict", False)
if isinstance(strict, str):
return strict.strip().lower() in {"1", "true", "yes", "on"}
return bool(strict)
def _set_outputs(self, outputs):
self._param.outputs["result"]["value"] = outputs
self._param.outputs["first"]["value"] = outputs[0] if outputs else None
self._param.outputs["last"]["value"] = outputs[-1] if outputs else None
def _topN(self):
def _raise_strict_range_error(self, operation, n):
raise ValueError(
f"{operation} requires n to be within the valid range in strict mode, got {n}."
)
def _nth(self):
n = self._coerce_n()
if n < 1:
strict = self._is_strict()
if n == 0:
if strict:
self._raise_strict_range_error("nth", n)
outputs = []
elif n > 0:
if n <= len(self.inputs):
outputs = [self.inputs[n - 1]]
elif strict:
self._raise_strict_range_error("nth", n)
else:
outputs = []
else:
n = min(n, len(self.inputs))
outputs = self.inputs[:n]
if abs(n) <= len(self.inputs):
outputs = [self.inputs[n]]
elif strict:
self._raise_strict_range_error("nth", n)
else:
outputs = []
self._set_outputs(outputs)
def _head(self):
n = self._coerce_n()
if 1 <= n <= len(self.inputs):
outputs = [self.inputs[n - 1]]
strict = self._is_strict()
if strict:
if 1 <= n <= len(self.inputs):
outputs = self.inputs[:n]
else:
self._raise_strict_range_error("head", n)
else:
outputs = []
if n < 1:
outputs = []
else:
outputs = self.inputs[:n]
self._set_outputs(outputs)
def _tail(self):
n = self._coerce_n()
if 1 <= n <= len(self.inputs):
outputs = [self.inputs[-n]]
strict = self._is_strict()
if strict:
if 1 <= n <= len(self.inputs):
outputs = self.inputs[-n:]
else:
self._raise_strict_range_error("tail", n)
else:
outputs = []
if n < 1:
outputs = []
else:
outputs = self.inputs[-n:]
self._set_outputs(outputs)
def _filter(self):
@@ -107,7 +151,7 @@ class ListOperations(ComponentBase,ABC):
def _norm(self,v):
s = "" if v is None else str(v)
return s
def _eval(self, v, operator, value):
if operator == "=":
return v == value
@@ -163,6 +207,6 @@ class ListOperations(ComponentBase,ABC):
if isinstance(x, set):
return tuple(sorted(self._hashable(v) for v in x))
return x
def thoughts(self) -> str:
return "ListOperation in progress"

View File

@@ -25,6 +25,7 @@ from functools import partial
from common.constants import LLMType
from api.db.services.llm_service import LLMBundle
from api.db.services.tenant_llm_service import TenantLLMService
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from agent.component.base import ComponentBase, ComponentParamBase
from common.connection_utils import timeout
from rag.prompts.generator import tool_call_summary, message_fit_in, citation_prompt, structured_output_prompt
@@ -84,10 +85,10 @@ class LLM(ComponentBase):
def __init__(self, canvas, component_id, param: ComponentParamBase):
super().__init__(canvas, component_id, param)
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id),
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id)
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), chat_model_config,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error)
self.imgs = []
def get_input_form(self) -> dict[str, dict]:
@@ -125,23 +126,119 @@ class LLM(ComponentBase):
msg.append(p)
return msg, self.string_format(self._param.sys_prompt, args)
def _prepare_prompt_variables(self):
if self._param.visual_files_var:
self.imgs = self._canvas.get_variable_value(self._param.visual_files_var)
if not self.imgs:
self.imgs = []
self.imgs = [img for img in self.imgs if img[:len("data:image/")] == "data:image/"]
if self.imgs and TenantLLMService.llm_id2llm_type(self._param.llm_id) == LLMType.CHAT.value:
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.IMAGE2TEXT.value,
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
@staticmethod
def _extract_data_images(value) -> list[str]:
imgs = []
def walk(v):
if v is None:
return
if isinstance(v, str):
v = v.strip()
if v.startswith("data:image/"):
imgs.append(v)
return
if isinstance(v, (list, tuple, set)):
for item in v:
walk(item)
return
if isinstance(v, dict):
if "content" in v:
walk(v.get("content"))
else:
for item in v.values():
walk(item)
walk(value)
return imgs
@staticmethod
def _uniq_images(images: list[str]) -> list[str]:
seen = set()
uniq = []
for img in images:
if not isinstance(img, str):
continue
if not img.startswith("data:image/"):
continue
if img in seen:
continue
seen.add(img)
uniq.append(img)
return uniq
@classmethod
def _remove_data_images(cls, value):
if value is None:
return None
if isinstance(value, str):
return None if value.strip().startswith("data:image/") else value
if isinstance(value, list):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return cleaned
if isinstance(value, tuple):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return tuple(cleaned)
if isinstance(value, set):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return cleaned
if isinstance(value, dict):
if value.get("type") in {"image_url", "input_image", "image"} and cls._extract_data_images(value):
return None
cleaned = {}
for k, item in value.items():
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned[k] = v
return cleaned
return value
def _prepare_prompt_variables(self):
self.imgs = []
if self._param.visual_files_var:
visual_val = self._canvas.get_variable_value(self._param.visual_files_var)
self.imgs.extend(self._extract_data_images(visual_val))
args = {}
vars = self.get_input_elements() if not self._param.debug_inputs else self._param.debug_inputs
extracted_imgs = []
for k, o in vars.items():
args[k] = o["value"]
raw_value = o["value"]
extracted_imgs.extend(self._extract_data_images(raw_value))
args[k] = self._remove_data_images(raw_value)
if args[k] is None:
args[k] = ""
if not isinstance(args[k], str):
try:
args[k] = json.dumps(args[k], ensure_ascii=False)
@@ -149,6 +246,13 @@ class LLM(ComponentBase):
args[k] = str(args[k])
self.set_input_value(k, args[k])
self.imgs = self._uniq_images(self.imgs + extracted_imgs)
if self.imgs and TenantLLMService.llm_id2llm_type(self._param.llm_id) == LLMType.CHAT.value:
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.IMAGE2TEXT.value,
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
msg, sys_prompt = self._sys_prompt_and_msg(self._canvas.get_history(self._param.message_history_window_size)[:-1], args)
user_defined_prompt, sys_prompt = self._extract_prompts(sys_prompt)
if self._param.cite and self._canvas.get_reference()["chunks"]:

View File

@@ -14,8 +14,11 @@
# limitations under the License.
#
import asyncio
import nest_asyncio
nest_asyncio.apply()
try:
import nest_asyncio
nest_asyncio.apply()
except Exception:
pass
import inspect
import json
import os
@@ -27,7 +30,9 @@ from functools import partial
from typing import Any
from agent.component.base import ComponentBase, ComponentParamBase
from jinja2 import Template as Jinja2Template
from jinja2.sandbox import SandboxedEnvironment
_jinja2_sandbox = SandboxedEnvironment()
from common.connection_utils import timeout
from common.misc_utils import get_uuid
@@ -49,6 +54,9 @@ class MessageParam(ComponentParamBase):
self.outputs = {
"content": {
"type": "str"
},
"downloads": {
"type": "list"
}
}
@@ -61,15 +69,99 @@ class MessageParam(ComponentParamBase):
class Message(ComponentBase):
component_name = "Message"
@staticmethod
def _is_download_info(value: Any) -> bool:
return isinstance(value, dict) and all(
key in value for key in ("doc_id", "filename", "mime_type")
)
@staticmethod
def _download_info_includes_content(value: Any) -> bool:
return isinstance(value, dict) and bool(value.get("include_download_info_in_content"))
@staticmethod
def _normalize_download_info(value: Any) -> Any:
if isinstance(value, list):
return [Message._normalize_download_info(item) for item in value]
if not isinstance(value, dict):
return value
normalized = value.copy()
normalized.pop("include_download_info_in_content", None)
return normalized
def _extract_downloads(self, value: Any) -> list[dict[str, Any]]:
if isinstance(value, str):
try:
value = json.loads(value)
except Exception:
return []
if self._is_download_info(value):
return [value]
if isinstance(value, list) and all(self._is_download_info(item) for item in value):
return value
return []
def _stringify_message_value(
self,
value: Any,
delimiter: str = None,
downloads: list[dict[str, Any]] | None = None,
fallback_to_str: bool = False,
) -> str:
extracted_downloads = self._extract_downloads(value)
if extracted_downloads:
if downloads is not None:
downloads.extend(self._normalize_download_info(item) for item in extracted_downloads)
if any(self._download_info_includes_content(item) for item in extracted_downloads):
if isinstance(value, str):
try:
value = json.loads(value)
except Exception:
return value
try:
return json.dumps(self._normalize_download_info(value), ensure_ascii=False)
except Exception:
if fallback_to_str:
return str(value)
return ""
return ""
if value is None:
return ""
if isinstance(value, list) and delimiter:
return delimiter.join([str(vv) for vv in value])
if isinstance(value, str):
return value
try:
return json.dumps(value, ensure_ascii=False)
except Exception:
if fallback_to_str:
return str(value)
return ""
def get_input_elements(self) -> dict[str, Any]:
return self.get_input_elements_from_text("".join(self._param.content))
def get_kwargs(self, script:str, kwargs:dict = {}, delimiter:str=None) -> tuple[str, dict[str, str | list | Any]]:
def get_kwargs(
self,
script: str,
kwargs: dict = {},
delimiter: str = None,
downloads: list[dict[str, Any]] | None = None,
) -> tuple[str, dict[str, str | list | Any]]:
for k,v in self.get_input_elements_from_text(script).items():
if k in kwargs:
continue
v = v["value"]
if not v:
if v is None:
v = ""
ans = ""
if isinstance(v, partial):
@@ -79,15 +171,8 @@ class Message(ComponentBase):
else:
for t in iter_obj:
ans += t
elif isinstance(v, list) and delimiter:
ans = delimiter.join([str(vv) for vv in v])
elif not isinstance(v, str):
try:
ans = json.dumps(v, ensure_ascii=False)
except Exception:
pass
else:
ans = v
ans = self._stringify_message_value(v, delimiter, downloads)
if not ans:
ans = ""
kwargs[k] = ans
@@ -110,6 +195,7 @@ class Message(ComponentBase):
s = 0
all_content = ""
cache = {}
downloads = []
for r in re.finditer(self.variable_ref_patt, rand_cnt, flags=re.DOTALL):
if self.check_if_canceled("Message streaming"):
return
@@ -149,11 +235,9 @@ class Message(ComponentBase):
continue
elif inspect.isawaitable(v):
v = await v
elif not isinstance(v, str):
try:
v = json.dumps(v, ensure_ascii=False)
except Exception:
v = str(v)
v = self._stringify_message_value(
v, downloads=downloads, fallback_to_str=True
)
yield v
self.set_input_value(exp, v)
all_content += v
@@ -166,6 +250,7 @@ class Message(ComponentBase):
all_content += rand_cnt[s: ]
yield rand_cnt[s: ]
self.set_output("downloads", downloads)
self.set_output("content", all_content)
self._convert_content(all_content)
await self._save_to_memory(all_content)
@@ -186,12 +271,14 @@ class Message(ComponentBase):
self.set_output("content", partial(self._stream, rand_cnt))
return
rand_cnt, kwargs = self.get_kwargs(rand_cnt, kwargs)
template = Jinja2Template(rand_cnt)
downloads = []
rand_cnt, kwargs = self.get_kwargs(rand_cnt, kwargs, downloads=downloads)
template = _jinja2_sandbox.from_string(rand_cnt)
try:
content = template.render(kwargs)
except Exception:
pass
except Exception as e:
logging.warning(f"Jinja2 template rendering failed: {e}")
content = rand_cnt # fallback to unrendered content
if self.check_if_canceled("Message processing"):
return
@@ -199,6 +286,7 @@ class Message(ComponentBase):
for n, v in kwargs.items():
content = re.sub(n, v, content)
self.set_output("downloads", downloads)
self.set_output("content", content)
self._convert_content(content)
self._save_to_memory(content)
@@ -224,6 +312,38 @@ class Message(ComponentBase):
rows = []
headers = None
def _coerce_excel_cell_type(cell: str):
# Convert markdown cell text to native numeric types when safe,so Excel writes numeric cells instead of text.
if not isinstance(cell, str):
return cell
value = cell.strip()
if value == "":
return ""
# Keep values like "00123" as text to avoid losing leading zeros.
if re.match(r"^[+-]?0\d+$", value):
return cell
# Support thousand separators like 1,234 or 1,234.56
numeric_candidate = value
if re.match(r"^[+-]?\d{1,3}(,\d{3})+(\.\d+)?$", value):
numeric_candidate = value.replace(",", "")
if re.match(r"^[+-]?\d+$", numeric_candidate):
try:
return int(numeric_candidate)
except ValueError:
return cell
if re.match(r"^[+-]?(\d+\.\d+|\d+\.|\.\d+)([eE][+-]?\d+)?$", numeric_candidate) or re.match(r"^[+-]?\d+[eE][+-]?\d+$", numeric_candidate):
try:
return float(numeric_candidate)
except ValueError:
return cell
return cell
for line in table_lines:
# Split by | and clean up
@@ -234,6 +354,7 @@ class Message(ComponentBase):
if headers is None:
headers = cells
else:
cells = [_coerce_excel_cell_type(c) for c in cells]
rows.append(cells)
if headers and rows:
@@ -430,8 +551,15 @@ class Message(ComponentBase):
if not hasattr(self._param, "memory_ids") or not self._param.memory_ids:
return True, "No memory selected."
user_id = self._param.user_id if hasattr(self._param, "user_id") else ""
if user_id:
import re
# is variable
if re.match(r"^{.*}$", user_id):
user_id = self._canvas.get_variable_value(user_id)
message_dict = {
"user_id": self._canvas._tenant_id,
"user_id": user_id,
"agent_id": self._canvas._id,
"session_id": self._canvas.task_id,
"user_input": self._canvas.get_sys_query(),

View File

@@ -18,7 +18,9 @@ import re
from abc import ABC
from typing import Any
from jinja2 import Template as Jinja2Template
from jinja2.sandbox import SandboxedEnvironment
_jinja2_sandbox = SandboxedEnvironment()
from agent.component.base import ComponentParamBase
from common.connection_utils import timeout
from .message import Message
@@ -96,14 +98,14 @@ class StringTransform(Message, ABC):
script, kwargs = self.get_kwargs(script, kwargs, self._param.delimiters[0])
if self._is_jinjia2(script):
template = Jinja2Template(script)
template = _jinja2_sandbox.from_string(script)
try:
script = template.render(kwargs)
except Exception:
pass
for k,v in kwargs.items():
if not v:
if v is None:
v = ""
script = re.sub(k, lambda match: v, script)

View File

@@ -134,7 +134,7 @@ class Switch(ComponentBase, ABC):
except Exception:
return True if input <= value else False
raise ValueError('Not supported operator' + operator)
raise ValueError(f'Not supported operator: {operator}')
def thoughts(self) -> str:
return "Im weighing a few options and will pick the next step shortly."

View File

@@ -141,20 +141,18 @@ class VariableAssigner(ComponentBase,ABC):
return variable + parameter
def _remove_first(self,variable):
if len(variable)==0:
return variable
if not isinstance(variable,list):
return "ERROR:VARIABLE_NOT_LIST"
else:
return variable[1:]
if len(variable)==0:
return variable
return variable[1:]
def _remove_last(self,variable):
if len(variable)==0:
return variable
if not isinstance(variable,list):
return "ERROR:VARIABLE_NOT_LIST"
else:
return variable[:-1]
if len(variable)==0:
return variable
return variable[:-1]
def is_number(self, value):
if isinstance(value, bool):

178
agent/dsl_migration.py Normal file
View File

@@ -0,0 +1,178 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import copy
import re
# Keep all legacy chunker renames in one place so the migration rule stays readable.
COMPONENT_RENAMES = {
"Splitter": "TokenChunker",
"HierarchicalMerger": "TitleChunker",
"PDFGenerator": "DocGenerator",
}
NODE_TYPE_RENAMES = {
"splitterNode": "chunkerNode",
}
VARIABLE_REF_PATTERN = re.compile(r"(\{+\s*)([A-Za-z0-9:_-]+)(@[A-Za-z0-9_.-]+)(\s*\}+)")
def normalize_chunker_dsl(dsl: dict) -> dict:
"""
Rewrite legacy chunker component names and ids into the current DSL schema.
This is intentionally a pure migration step:
- it does not change business params
- it only rewrites structural identifiers used by the canvas/runtime
- custom human-authored names are preserved unless they are still the exact
built-in legacy operator name
"""
if not isinstance(dsl, dict):
return dsl
normalized = copy.deepcopy(dsl)
components = normalized.get("components")
if not isinstance(components, dict):
return normalized
component_id_map: dict[str, str] = {}
for component_id in components.keys():
new_component_id = component_id
for old_name, new_name in COMPONENT_RENAMES.items():
prefix = f"{old_name}:"
if component_id.startswith(prefix):
new_component_id = f"{new_name}:{component_id[len(prefix):]}"
break
component_id_map[component_id] = new_component_id
def rewrite_variable_refs(text: str) -> str:
if text in component_id_map:
return component_id_map[text]
def repl(match: re.Match[str]) -> str:
component_id = match.group(2)
return (
match.group(1)
+ component_id_map.get(component_id, component_id)
+ match.group(3)
+ match.group(4)
)
return VARIABLE_REF_PATTERN.sub(repl, text)
def rewrite_value(value):
if isinstance(value, str):
return rewrite_variable_refs(value)
if isinstance(value, list):
return [rewrite_value(item) for item in value]
if isinstance(value, dict):
return {key: rewrite_value(item) for key, item in value.items()}
return value
rewritten_components = {}
for old_component_id, component in components.items():
new_component_id = component_id_map[old_component_id]
new_component = rewrite_value(component)
if isinstance(new_component, dict):
obj = new_component.get("obj")
if isinstance(obj, dict):
component_name = obj.get("component_name")
obj["component_name"] = COMPONENT_RENAMES.get(component_name, component_name)
if isinstance(new_component.get("downstream"), list):
new_component["downstream"] = [
component_id_map.get(component_id, component_id)
for component_id in new_component["downstream"]
]
if isinstance(new_component.get("upstream"), list):
new_component["upstream"] = [
component_id_map.get(component_id, component_id)
for component_id in new_component["upstream"]
]
parent_id = new_component.get("parent_id")
if isinstance(parent_id, str):
new_component["parent_id"] = component_id_map.get(parent_id, parent_id)
rewritten_components[new_component_id] = new_component
normalized["components"] = rewritten_components
if isinstance(normalized.get("path"), list):
normalized["path"] = [
component_id_map.get(component_id, component_id)
for component_id in normalized["path"]
]
graph = normalized.get("graph")
if isinstance(graph, dict):
nodes = graph.get("nodes")
if isinstance(nodes, list):
for node in nodes:
if not isinstance(node, dict):
continue
node_id = node.get("id")
if isinstance(node_id, str):
node["id"] = component_id_map.get(node_id, node_id)
parent_id = node.get("parentId")
if isinstance(parent_id, str):
node["parentId"] = component_id_map.get(parent_id, parent_id)
node_type = node.get("type")
if isinstance(node_type, str):
node["type"] = NODE_TYPE_RENAMES.get(node_type, node_type)
data = node.get("data")
if not isinstance(data, dict):
continue
label = data.get("label")
if isinstance(label, str):
data["label"] = COMPONENT_RENAMES.get(label, label)
name = data.get("name")
if isinstance(name, str) and name in COMPONENT_RENAMES:
data["name"] = COMPONENT_RENAMES[name]
if "form" in data:
data["form"] = rewrite_value(data["form"])
edges = graph.get("edges")
if isinstance(edges, list):
replacements = sorted(component_id_map.items(), key=lambda item: len(item[0]), reverse=True)
for edge in edges:
if not isinstance(edge, dict):
continue
for key in ("source", "target"):
value = edge.get(key)
if isinstance(value, str):
edge[key] = component_id_map.get(value, value)
edge_id = edge.get("id")
if isinstance(edge_id, str):
for old_component_id, new_component_id in replacements:
edge_id = edge_id.replace(old_component_id, new_component_id)
edge["id"] = edge_id
for key in ("history", "messages", "reference"):
if key in normalized:
normalized[key] = rewrite_value(normalized[key])
return normalized

97
agent/plugin/README.md Normal file
View File

@@ -0,0 +1,97 @@
# Plugins
This directory contains the plugin mechanism for RAGFlow.
RAGFlow will load plugins from `embedded_plugins` subdirectory recursively.
## Supported plugin types
Currently, the only supported plugin type is `llm_tools`.
- `llm_tools`: A tool for LLM to call.
## How to add a plugin
Add a LLM tool plugin is simple: create a plugin file, put a class inherits the `LLMToolPlugin` class in it, then implement the `get_metadata` and the `invoke` methods.
- `get_metadata` method: This method returns a `LLMToolMetadata` object, which contains the description of this tool.
The description will be provided to LLM, and the RAGFlow web frontend for displaying.
- `invoke` method: This method accepts parameters generated by LLM, and return a `str` containing the tool execution result.
All the execution logic of this tool should go into this method.
When you start RAGFlow, you can see your plugin was loaded in the log:
```
2025-05-15 19:29:08,959 INFO 34670 Recursively importing plugins from path `/some-path/ragflow/agent/plugin/embedded_plugins`
2025-05-15 19:29:08,960 INFO 34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
```
Or it may contain some errors for you to fix your plugin.
### Demo
We will demonstrate how to add a plugin with a calculator tool which will give wrong answers.
First, create a plugin file `bad_calculator.py` under the `embedded_plugins/llm_tools` directory.
Then, we create a `BadCalculatorPlugin` class, extending the `LLMToolPlugin` base class:
```python
class BadCalculatorPlugin(LLMToolPlugin):
_version_ = "1.0.0"
```
The `_version_` field is required, which specifies the version of the plugin.
Our calculator has two numbers `a` and `b` as inputs, so we add a `invoke` method to our `BadCalculatorPlugin` class:
```python
def invoke(self, a: int, b: int) -> str:
return str(a + b + 100)
```
The `invoke` method will be called by LLM. It can have many parameters, but the return type must be a `str`.
Finally, we have to add a `get_metadata` method, to tell LLM how to use our `bad_calculator`:
```python
@classmethod
def get_metadata(cls) -> LLMToolMetadata:
return {
# Name of this tool, providing to LLM
"name": "bad_calculator",
# Display name of this tool, providing to RAGFlow frontend
"displayName": "$t:bad_calculator.name",
# Description of the usage of this tool, providing to LLM
"description": "A tool to calculate the sum of two numbers (will give wrong answer)",
# Description of this tool, providing to RAGFlow frontend
"displayDescription": "$t:bad_calculator.description",
# Parameters of this tool
"parameters": {
# The first parameter - a
"a": {
# Parameter type, options are: number, string, or whatever the LLM can recognise
"type": "number",
# Description of this parameter, providing to LLM
"description": "The first number",
# Description of this parameter, provding to RAGFlow frontend
"displayDescription": "$t:bad_calculator.params.a",
# Whether this parameter is required
"required": True
},
# The second parameter - b
"b": {
"type": "number",
"description": "The second number",
"displayDescription": "$t:bad_calculator.params.b",
"required": True
}
}
```
The `get_metadata` method is a `classmethod`. It will provide the description of this tool to LLM.
The fields start with `display` can use a special notation: `$t:xxx`, which will use the i18n mechanism in the RAGFlow frontend, getting text from the `llmTools` category. The frontend will display what you put here if you don't use this notation.
Now our tool is ready. You can select it in the `Generate` component and try it out.

99
agent/plugin/README_tr.md Normal file
View File

@@ -0,0 +1,99 @@
[English](./README.md) | [简体中文](./README_zh.md) | Türkçe
# Eklentiler
Bu klasör, RAGFlow'un eklenti mekanizmasını içerir.
RAGFlow, `embedded_plugins` alt klasöründen eklentileri özyinelemeli olarak yükleyecektir.
## Desteklenen eklenti türleri
Şu anda desteklenen tek eklenti türü `llm_tools`'dur.
- `llm_tools`: LLM'nin çağırması için bir araç.
## Eklenti nasıl eklenir
Bir LLM araç eklentisi eklemek basittir: bir eklenti dosyası oluşturun, içine `LLMToolPlugin` sınıfından türetilmiş bir sınıf koyun, ardından `get_metadata` ve `invoke` metodlarını uygulayın.
- `get_metadata` metodu: Bu metod, aracın açıklamasını içeren bir `LLMToolMetadata` nesnesi döndürür.
ıklama, LLM'ye çağrı için ve RAGFlow web ön yüzüne görüntüleme amacıyla sağlanacaktır.
- `invoke` metodu: Bu metod, LLM tarafından üretilen parametreleri kabul eder ve aracın yürütme sonucunu içeren bir `str` döndürür.
Bu aracın tüm yürütme mantığı bu metoda konulmalıdır.
RAGFlow'u başlattığınızda, günlükte eklentinizin yüklendiğini göreceksiniz:
```
2025-05-15 19:29:08,959 INFO 34670 Recursively importing plugins from path `/some-path/ragflow/agent/plugin/embedded_plugins`
2025-05-15 19:29:08,960 INFO 34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
```
Veya eklentinizi düzeltmeniz gereken hatalar da içerebilir.
### Örnek
Yanlış cevaplar veren bir hesap makinesi aracı ekleyerek eklenti ekleme sürecini göstereceğiz.
Önce, `embedded_plugins/llm_tools` klasörü altında `bad_calculator.py` adında bir eklenti dosyası oluşturun.
Ardından, `LLMToolPlugin` temel sınıfından türetilmiş bir `BadCalculatorPlugin` sınıfı oluşturuyoruz:
```python
class BadCalculatorPlugin(LLMToolPlugin):
_version_ = "1.0.0"
```
`_version_` alanı zorunludur ve eklentinin sürüm numarasını belirtir.
Hesap makinemizin girdileri olarak `a` ve `b` olmak üzere iki sayısı vardır, bu yüzden `BadCalculatorPlugin` sınıfımıza aşağıdaki `invoke` metodunu ekliyoruz:
```python
def invoke(self, a: int, b: int) -> str:
return str(a + b + 100)
```
`invoke` metodu LLM tarafından çağrılacaktır. Birçok parametreye sahip olabilir, ancak dönüş tipi `str` olmalıdır.
Son olarak, LLM'ye `bad_calculator` aracımızı nasıl kullanacağını anlatmak için bir `get_metadata` metodu eklememiz gerekiyor:
```python
@classmethod
def get_metadata(cls) -> LLMToolMetadata:
return {
# Bu aracın adı, LLM'ye sağlanır
"name": "bad_calculator",
# Bu aracın görüntüleme adı, RAGFlow ön yüzüne sağlanır
"displayName": "$t:bad_calculator.name",
# Bu aracın kullanım açıklaması, LLM'ye sağlanır
"description": "A tool to calculate the sum of two numbers (will give wrong answer)",
# Bu aracın açıklaması, RAGFlow ön yüzüne sağlanır
"displayDescription": "$t:bad_calculator.description",
# Bu aracın parametreleri
"parameters": {
# Birinci parametre - a
"a": {
# Parametre tipi, seçenekler: number, string veya LLM'nin tanıyabileceği herhangi bir tip
"type": "number",
# Bu parametrenin açıklaması, LLM'ye sağlanır
"description": "The first number",
# Bu parametrenin açıklaması, RAGFlow ön yüzüne sağlanır
"displayDescription": "$t:bad_calculator.params.a",
# Bu parametrenin zorunlu olup olmadığı
"required": True
},
# İkinci parametre - b
"b": {
"type": "number",
"description": "The second number",
"displayDescription": "$t:bad_calculator.params.b",
"required": True
}
}
```
`get_metadata` metodu bir `classmethod`'dur. Bu aracın açıklamasını LLM'ye sağlayacaktır.
`display` ile başlayan alanlar özel bir gösterim kullanabilir: `$t:xxx`, bu gösterim RAGFlow ön yüzündeki uluslararasılaştırma (i18n) mekanizmasını kullanarak `llmTools` kategorisinden metin alır. Bu gösterimi kullanmazsanız, ön yüz buraya yazdığınız metni doğrudan gösterecektir.
Artık aracımız hazırdır. `Yanıt Üret` bileşeninde seçip deneyebilirsiniz.

98
agent/plugin/README_zh.md Normal file
View File

@@ -0,0 +1,98 @@
# 插件
这个文件夹包含了RAGFlow的插件机制。
RAGFlow将会从`embedded_plugins`子文件夹中递归加载所有的插件。
## 支持的插件类型
目前,唯一支持的插件类型是`llm_tools`
- `llm_tools`用于供LLM进行调用的工具。
## 如何添加一个插件
添加一个LLM工具插件是很简单的创建一个插件文件向其中放一个继承自`LLMToolPlugin`的类,再实现它的`get_metadata``invoke`方法即可。
- `get_metadata`方法:这个方法返回一个`LLMToolMetadata`对象,其中包含了对这个工具的描述。
这些描述信息将被提供给LLM进行调用和RAGFlow的Web前端用作展示。
- `invoke`方法这个方法接受LLM生成的参数并且返回一个`str`对象,其中包含了这个工具的执行结果。
这个工具的所有执行逻辑都应当放到这个方法里。
当你启动RAGFlow时你会在日志中看见你的插件被加载了
```
2025-05-15 19:29:08,959 INFO 34670 Recursively importing plugins from path `/some-path/ragflow/agent/plugin/embedded_plugins`
2025-05-15 19:29:08,960 INFO 34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
```
也可能会报错,这时就需要根据报错对你的插件进行修复。
### 示例
我们将会添加一个会给出错误答案的计算器工具,来演示添加插件的过程。
首先,在`embedded_plugins/llm_tools`文件夹下创建一个插件文件`bad_calculator.py`
接下来,我们创建一个`BadCalculatorPlugin`类,继承基类`LLMToolPlugin`
```python
class BadCalculatorPlugin(LLMToolPlugin):
_version_ = "1.0.0"
```
`_version_`字段是必填的,用于指定这个插件的版本号。
我们的计算器拥有两个输入字段`a``b`,所以我们添加如下的`invoke`方法到`BadCalculatorPlugin`类中:
```python
def invoke(self, a: int, b: int) -> str:
return str(a + b + 100)
```
`invoke`方法将会被LLM所调用。这个方法可以有许多参数但它必须返回一个`str`
最后,我们需要添加一个`get_metadata`方法来告诉LLM怎样使用我们的`bad_calculator`工具:
```python
@classmethod
def get_metadata(cls) -> LLMToolMetadata:
return {
# 这个工具的名称会提供给LLM
"name": "bad_calculator",
# 这个工具的展示名称会提供给RAGFlow的Web前端
"displayName": "$t:bad_calculator.name",
# 这个工具的用法描述会提供给LLM
"description": "A tool to calculate the sum of two numbers (will give wrong answer)",
# 这个工具的描述会提供给RAGFlow的Web前端
"displayDescription": "$t:bad_calculator.description",
# 这个工具的参数
"parameters": {
# 第一个参数 - a
"a": {
# 参数类型选项为number, string, 或者LLM可以识别的任何类型
"type": "number",
# 这个参数的描述会提供给LLM
"description": "The first number",
# 这个参数的描述会提供给RAGFlow的Web前端
"displayDescription": "$t:bad_calculator.params.a",
# 这个参数是否是必填的
"required": True
},
# 第二个参数 - b
"b": {
"type": "number",
"description": "The second number",
"displayDescription": "$t:bad_calculator.params.b",
"required": True
}
}
```
`get_metadata`方法是一个`classmethod`。它会把这个工具的描述提供给LLM。
`display`开头的字段可以使用一种特殊写法`$t:xxx`这种写法将使用RAGFlow的国际化机制`llmTools`这个分类中获取文字。如果你不使用这种写法,那么前端将会显示此处的原始内容。
现在,我们的工具已经做好了,你可以在`生成回答`组件中选择这个工具来尝试一下。

View File

@@ -0,0 +1,37 @@
import logging
from agent.plugin.llm_tool_plugin import LLMToolMetadata, LLMToolPlugin
class BadCalculatorPlugin(LLMToolPlugin):
"""
A sample LLM tool plugin, will add two numbers with 100.
It only presents for demo purpose. Do not use it in production.
"""
_version_ = "1.0.0"
@classmethod
def get_metadata(cls) -> LLMToolMetadata:
return {
"name": "bad_calculator",
"displayName": "$t:bad_calculator.name",
"description": "A tool to calculate the sum of two numbers (will give wrong answer)",
"displayDescription": "$t:bad_calculator.description",
"parameters": {
"a": {
"type": "number",
"description": "The first number",
"displayDescription": "$t:bad_calculator.params.a",
"required": True
},
"b": {
"type": "number",
"description": "The second number",
"displayDescription": "$t:bad_calculator.params.b",
"required": True
}
}
}
def invoke(self, a: int, b: int) -> str:
logging.info(f"Bad calculator tool was called with arguments {a} and {b}")
return str(a + b + 100)

361
agent/sandbox/README.md Normal file
View File

@@ -0,0 +1,361 @@
# RAGFlow Sandbox
A secure, pluggable code execution backend for RAGFlow and beyond.
## 🔧 Features
-**Seamless RAGFlow Integration** — Out-of-the-box compatibility with the `code` component.
- 🔐 **High Security** — Leverages [gVisor](https://gvisor.dev/) for syscall-level sandboxing.
- 🔧 **Customizable Sandboxing** — Easily modify `seccomp` settings as needed.
- 🧩 **Pluggable Runtime Support** — Easily extend to support any programming language.
- ⚙️ **Developer Friendly** — Get started with a single command using `Makefile`.
## 🏗 Architecture
<p align="center">
<img src="asserts/code_executor_manager.svg" width="520" alt="Architecture Diagram">
</p>
## 🚀 Quick Start
### 📋 Prerequisites
#### Required
- Linux distro compatible with gVisor
- [gVisor](https://gvisor.dev/docs/user_guide/install/)
- Docker >= `25.0` (API 1.44+) — executor manager now bundles Docker CLI `29.1.0` to match newer daemons.
- Docker Compose >= `v2.26.1` like [RAGFlow](https://github.com/infiniflow/ragflow)
- [uv](https://docs.astral.sh/uv/) as package and project manager
#### Optional (Recommended)
- [GNU Make](https://www.gnu.org/software/make/) for simplified CLI management
---
> ⚠️ **New Docker CLI requirement**
>
> If you see `client version 1.43 is too old. Minimum supported API version is 1.44`, pull the latest `infiniflow/sandbox-executor-manager:latest` (rebuilt with Docker CLI `29.1.0`) or rebuild it in `./sandbox/executor_manager`. Older images shipped Docker 24.x, which cannot talk to newer Docker daemons.
### 🐳 Build Docker Base Images
We use isolated base images for secure containerized execution:
```bash
# Build base images manually
docker build -t sandbox-base-python:latest ./sandbox_base_image/python
docker build -t sandbox-base-nodejs:latest ./sandbox_base_image/nodejs
# OR use Makefile
make build
```
Then, build the executor manager image:
```bash
docker build -t sandbox-executor-manager:latest ./executor_manager
```
---
### 📦 Running with RAGFlow
1. Ensure gVisor is correctly installed.
2. Configure your `.env` in `docker/.env`:
- Uncomment sandbox-related variables.
- Enable sandbox profile at the bottom.
3. Add the following line to `/etc/hosts` as recommended:
```text
127.0.0.1 sandbox-executor-manager
```
4. Start RAGFlow service.
---
### 🧭 Running Standalone
#### Manual Setup
1. Initialize environment:
```bash
cp .env.example .env
```
2. Launch:
```bash
docker compose -f docker-compose.yml up
```
3. Test:
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
uv pip install -r executor_manager/requirements.txt
uv run tests/sandbox_security_tests_full.py
```
#### With Make
```bash
make # setup + build + launch + test
```
---
### 📈 Monitoring
```bash
docker logs -f sandbox-executor-manager # Manual
make logs # With Make
```
---
### 🧰 Makefile Toolbox
| Command | Description |
|-------------------|--------------------------------------------------|
| `make` | Setup, build, launch and test all at once |
| `make setup` | Initialize environment and install uv |
| `make ensure_env` | Auto-create `.env` if missing |
| `make ensure_uv` | Install `uv` package manager if missing |
| `make build` | Build all Docker base images |
| `make start` | Start services with safe env loading and testing |
| `make stop` | Gracefully stop all services |
| `make restart` | Shortcut for `stop` + `start` |
| `make test` | Run full test suite |
| `make logs` | Stream container logs |
| `make clean` | Stop and remove orphan containers and volumes |
---
## 🔐 Security
The RAGFlow sandbox is designed to balance security and usability, offering solid protection without compromising developer experience.
### ✅ gVisor Isolation
At its core, we use [gVisor](https://gvisor.dev/docs/architecture_guide/security/), a user-space kernel, to isolate code execution from the host system. gVisor intercepts and restricts syscalls, offering robust protection against container escapes and privilege escalations.
### 🔒 Optional seccomp Support (Advanced)
For users who need **zero-trust-level syscall control**, we support an additional `seccomp` profile. This feature restricts containers to only a predefined set of system calls, as specified in `executor_manager/seccomp-profile-default.json`.
> ⚠️ This feature is **disabled by default** to maintain compatibility and usability. Enabling it may cause compatibility issues with some dependencies.
#### To enable seccomp
1. Edit your `.env` file:
```dotenv
SANDBOX_ENABLE_SECCOMP=true
```
2. Customize allowed syscalls in:
```
executor_manager/seccomp-profile-default.json
```
This profile is passed to the container with:
```bash
--security-opt seccomp=/app/seccomp-profile-default.json
```
### 🧠 Python Code AST Inspection
In addition to sandboxing, Python code is **statically analyzed via AST (Abstract Syntax Tree)** before execution. Potentially malicious code (e.g. file operations, subprocess calls, etc.) is rejected early, providing an extra layer of protection.
---
This security model strikes a balance between **robust isolation** and **developer usability**. While `seccomp` can be highly restrictive, our default setup aims to keep things usable for most developers — no obscure crashes or cryptic setup required.
## 📦 Add Extra Dependencies for Supported Languages
Currently, the following languages are officially supported:
| Language | Priority |
|----------|----------|
| Python | High |
| Node.js | Medium |
### 🐍 Python
Pre-installed packages: `requests`, `numpy`, `pandas`, `matplotlib`.
> `matplotlib` uses the `Agg` (non-interactive) backend by default in the sandbox (`MPLBACKEND=Agg`). No display server is available, so always save figures to files (e.g. `fig.savefig("artifacts/chart.png")`) rather than calling `plt.show()`.
>
> Tip: if Chinese text renders as missing boxes/squares in `matplotlib`, install Debian package `fonts-noto-cjk` in your custom image. We do not preinstall it by default to keep the base image smaller. The sandbox base image ships a `matplotlibrc` that already lists common CJK fonts in the `font.sans-serif` fallback chain, so no code-level font configuration is needed — just install the font package and rebuild the image.
>
> Example:
>
> ```dockerfile
> RUN apt-get update && apt-get install -y --no-install-recommends fonts-noto-cjk && rm -rf /var/lib/apt/lists/*
> ```
To add more dependencies, edit:
```bash
sandbox_base_image/python/requirements.txt
```
Add any additional packages you need, one per line (just like a normal pip requirements file).
### 🟨 Node.js
Pre-installed packages: `axios`.
To add Node.js dependencies:
1. Navigate to the Node.js base image directory:
```bash
cd sandbox_base_image/nodejs
```
2. Use `npm` to install the desired packages. For example:
```bash
npm install lodash
```
3. The dependencies will be saved to `package.json` and `package-lock.json`, and included in the Docker image when rebuilt.
---
## Usage
### 🐍 A Python example
```python
def main(arg1: str, arg2: str) -> str:
return f"result: {arg1 + arg2}"
```
### 🟨 JavaScript examples
A simple sync function
```javascript
function main({arg1, arg2}) {
return arg1+arg2
}
```
Async funcion with aioxs
```javascript
const axios = require('axios');
async function main() {
try {
const response = await axios.get('https://github.com/infiniflow/ragflow');
return 'Body:' + response.data;
} catch (error) {
return 'Error:' + error.message;
}
}
```
---
## 📋 FAQ
### ❓Sandbox Not Working?
Follow this checklist to troubleshoot:
- [ ] **Is your machine compatible with gVisor?**
Ensure that your system supports gVisor. Refer to the [gVisor installation guide](https://gvisor.dev/docs/user_guide/install/).
- [ ] **Is gVisor properly installed?**
**Common error:**
`HTTPConnectionPool(host='sandbox-executor-manager', port=9385): Read timed out.`
Cause: `runsc` is an unknown or invalid Docker runtime.
**Fix:**
- Install gVisor
- Restart Docker
- Test with:
```bash
docker run --rm --runtime=runsc hello-world
```
- [ ] **Is `sandbox-executor-manager` mapped in `/etc/hosts`?**
**Common error:**
`HTTPConnectionPool(host='none', port=9385): Max retries exceeded.`
**Fix:**
Add the following entry to `/etc/hosts`:
```text
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
- [ ] **Are you running the latest executor manager image?**
**Common error:**
`docker: Error response from daemon: client version 1.43 is too old. Minimum supported API version is 1.44`
**Fix:**
Pull the refreshed image that bundles Docker CLI `29.1.0`, or rebuild it in `./sandbox/executor_manager`:
```bash
docker pull infiniflow/sandbox-executor-manager:latest
# or
docker build -t sandbox-executor-manager:latest ./sandbox/executor_manager
```
- [ ] **Have you enabled sandbox-related configurations in RAGFlow?**
Double-check that all sandbox settings are correctly enabled in your RAGFlow configuration.
- [ ] **Have you pulled the required base images for the runners?**
**Common error:**
`HTTPConnectionPool(host='sandbox-executor-manager', port=9385): Read timed out.`
Cause: no runner was started.
**Fix:**
Pull the necessary base images:
```bash
docker pull infiniflow/sandbox-base-nodejs:latest
docker pull infiniflow/sandbox-base-python:latest
```
- [ ] **Did you restart the service after making changes?**
Any changes to configuration or environment require a full service restart to take effect.
### ❓Container pool is busy?
All available runners are currently in use, executing tasks/running code. Please try again shortly, or consider increasing the pool size in the configuration to improve availability and reduce wait times.
## 🤝 Contribution
Contributions are welcome!

View File

Before

Width:  |  Height:  |  Size: 45 KiB

After

Width:  |  Height:  |  Size: 45 KiB

304
agent/sandbox/client.py Normal file
View File

@@ -0,0 +1,304 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Sandbox client for agent components.
This module provides a unified interface for agent components to interact
with the configured sandbox provider.
"""
import json
import logging
import os
from typing import Dict, Any, Optional
from api.db.services.system_settings_service import SystemSettingsService
from agent.sandbox.providers import ProviderManager
from agent.sandbox.providers.base import ExecutionResult, SandboxProviderConfigError
logger = logging.getLogger(__name__)
# Global provider manager instance
_provider_manager: Optional[ProviderManager] = None
def get_provider_manager() -> ProviderManager:
"""
Get the global provider manager instance.
Returns:
ProviderManager instance with active provider loaded
"""
global _provider_manager
if _provider_manager is not None:
return _provider_manager
# Initialize provider manager with system settings
_provider_manager = ProviderManager()
_load_provider_from_settings()
return _provider_manager
def _load_provider_from_settings() -> None:
"""
Load sandbox provider from system settings and configure the provider manager.
This function resolves the active provider type, then loads configuration
from system settings with environment overrides for that provider.
"""
global _provider_manager
if _provider_manager is None:
return
try:
provider_type, provider_type_from_env = _resolve_provider_type()
config = _load_provider_config(provider_type)
# Import and instantiate the provider
from agent.sandbox.providers import (
SelfManagedProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
LocalProvider,
)
provider_classes = {
"self_managed": SelfManagedProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
"local": LocalProvider,
}
if provider_type not in provider_classes:
if provider_type_from_env:
raise SandboxProviderConfigError(f"Unknown sandbox provider type: {provider_type}")
logger.error(f"Unknown provider type: {provider_type}")
return
provider_class = provider_classes[provider_type]
provider = provider_class()
# Initialize the provider
if not provider.initialize(config):
message = f"Failed to initialize sandbox provider: {provider_type}. Config keys: {list(config.keys())}"
if provider_type == "local" or provider_type_from_env:
raise SandboxProviderConfigError(message)
logger.error(message)
return
# Set the active provider
_provider_manager.set_provider(provider_type, provider)
logger.info(f"Sandbox provider '{provider_type}' initialized successfully")
except SandboxProviderConfigError:
raise
except Exception as e:
logger.error(f"Failed to load sandbox provider from settings: {e}")
import traceback
traceback.print_exc()
def _load_provider_config_from_settings(provider_type: str) -> Dict[str, Any]:
provider_config_settings = SystemSettingsService.get_by_name(f"sandbox.{provider_type}")
if not provider_config_settings:
logger.warning(f"No configuration found for provider: {provider_type}")
return {}
try:
return json.loads(provider_config_settings[0].value)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse sandbox config for {provider_type}: {e}")
return {}
def _resolve_provider_type() -> tuple[str, bool]:
provider_type = os.environ.get("SANDBOX_PROVIDER_TYPE", "").strip()
if provider_type:
return provider_type, True
provider_type_settings = SystemSettingsService.get_by_name("sandbox.provider_type")
if not provider_type_settings:
raise RuntimeError(
"Sandbox provider type not configured. Please set 'sandbox.provider_type' in system settings."
)
return provider_type_settings[0].value, False
def _load_provider_config(provider_type: str) -> Dict[str, Any]:
config = _load_provider_config_from_settings(provider_type)
env_config = _load_provider_config_from_env(provider_type)
if env_config:
config.update(env_config)
return config
def _load_provider_config_from_env(provider_type: str) -> Dict[str, Any]:
if provider_type == "local":
return _load_local_provider_config_from_env()
if provider_type == "self_managed":
return _load_self_managed_provider_config_from_env()
return {}
def _load_local_provider_config_from_env() -> Dict[str, Any]:
env_to_config = {
"SANDBOX_LOCAL_PYTHON_BIN": "python_bin",
"SANDBOX_LOCAL_NODE_BIN": "node_bin",
"SANDBOX_LOCAL_WORK_DIR": "work_dir",
"SANDBOX_LOCAL_TIMEOUT": "timeout",
"SANDBOX_LOCAL_MAX_MEMORY_MB": "max_memory_mb",
"SANDBOX_LOCAL_MAX_OUTPUT_BYTES": "max_output_bytes",
"SANDBOX_LOCAL_MAX_ARTIFACTS": "max_artifacts",
"SANDBOX_LOCAL_MAX_ARTIFACT_BYTES": "max_artifact_bytes",
}
config = {}
for env_name, config_name in env_to_config.items():
if env_name in os.environ:
config[config_name] = os.environ[env_name]
return config
def _load_self_managed_provider_config_from_env() -> Dict[str, Any]:
host = os.environ.get("SANDBOX_HOST", "").strip()
port = os.environ.get("SANDBOX_EXECUTOR_MANAGER_PORT", "").strip()
pool_size = os.environ.get("SANDBOX_EXECUTOR_MANAGER_POOL_SIZE", "").strip()
config = {}
if host:
config["endpoint"] = f"http://{host}:{port or '9385'}"
if pool_size:
config["pool_size"] = pool_size
return config
def reload_provider() -> None:
"""
Reload the sandbox provider from system settings.
Use this function when sandbox settings have been updated.
"""
global _provider_manager
_provider_manager = None
_load_provider_from_settings()
def execute_code(
code: str,
language: str = "python",
timeout: int = 30,
arguments: Optional[Dict[str, Any]] = None
) -> ExecutionResult:
"""
Execute code in the configured sandbox.
This is the main entry point for agent components to execute code.
Args:
code: Source code to execute
language: Programming language (python, nodejs, javascript)
timeout: Maximum execution time in seconds
arguments: Optional arguments dict to pass to main() function
Returns:
ExecutionResult containing stdout, stderr, exit_code, and metadata
Raises:
RuntimeError: If no provider is configured or execution fails
"""
provider_manager = get_provider_manager()
if not provider_manager.is_configured():
raise RuntimeError(
"No sandbox provider configured. Please configure sandbox settings in the admin panel."
)
provider = provider_manager.get_provider()
# Create a sandbox instance
instance = provider.create_instance(template=language)
try:
# Execute the code
result = provider.execute_code(
instance_id=instance.instance_id,
code=code,
language=language,
timeout=timeout,
arguments=arguments
)
return result
finally:
# Clean up the instance
try:
provider.destroy_instance(instance.instance_id)
except Exception as e:
logger.warning(f"Failed to destroy sandbox instance {instance.instance_id}: {e}")
def health_check() -> bool:
"""
Check if the sandbox provider is healthy.
Returns:
True if provider is configured and healthy, False otherwise
"""
try:
provider_manager = get_provider_manager()
if not provider_manager.is_configured():
return False
provider = provider_manager.get_provider()
return provider.health_check()
except Exception as e:
logger.error(f"Sandbox health check failed: {e}")
return False
def get_provider_info() -> Dict[str, Any]:
"""
Get information about the current sandbox provider.
Returns:
Dictionary with provider information:
- provider_type: Type of the active provider
- configured: Whether provider is configured
- healthy: Whether provider is healthy
"""
try:
provider_manager = get_provider_manager()
return {
"provider_type": provider_manager.get_provider_name(),
"configured": provider_manager.is_configured(),
"healthy": health_check(),
}
except Exception as e:
logger.error(f"Failed to get provider info: {e}")
return {
"provider_type": None,
"configured": False,
"healthy": False,
}

View File

@@ -0,0 +1,32 @@
services:
sandbox-executor-manager:
build:
context: ./executor_manager
dockerfile: Dockerfile
image: sandbox-executor-manager:latest
runtime: runc
privileged: true
ports:
- "${SANDBOX_EXECUTOR_MANAGER_PORT:-9385}:9385"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:
- sandbox-network
restart: always
security_opt:
- no-new-privileges:true
environment:
- SANDBOX_EXECUTOR_MANAGER_POOL_SIZE=${SANDBOX_EXECUTOR_MANAGER_POOL_SIZE:-5}
- SANDBOX_BASE_PYTHON_IMAGE=${SANDBOX_BASE_PYTHON_IMAGE-sandbox-base-python:latest}
- SANDBOX_BASE_NODEJS_IMAGE=${SANDBOX_BASE_NODEJS_IMAGE-sandbox-base-nodejs:latest}
- SANDBOX_ENABLE_SECCOMP=${SANDBOX_ENABLE_SECCOMP:-false}
- SANDBOX_MAX_MEMORY=${SANDBOX_MAX_MEMORY:-256m} # b, k, m, g
- SANDBOX_TIMEOUT=${SANDBOX_TIMEOUT:-10s} # s, m, 1m30s
healthcheck:
test: ["CMD-SHELL", "curl --fail http://localhost:9385/healthz || exit 1"]
interval: 10s
timeout: 5s
retries: 5
networks:
sandbox-network:
driver: bridge

View File

@@ -0,0 +1,41 @@
FROM python:3.11-slim-bookworm
ARG NEED_MIRROR=1
RUN if [ "$NEED_MIRROR" = 1 ]; then \
grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.tuna.tsinghua.edu.cn|g'; \
fi; \
apt-get update && \
apt-get install -y curl gcc && \
rm -rf /var/lib/apt/lists/*
ARG TARGETARCH
ARG TARGETVARIANT
RUN set -eux; \
case "${TARGETARCH}${TARGETVARIANT}" in \
amd64) DOCKER_ARCH=x86_64 ;; \
arm64) DOCKER_ARCH=aarch64 ;; \
armv7) DOCKER_ARCH=armhf ;; \
armv6) DOCKER_ARCH=armel ;; \
arm64v8) DOCKER_ARCH=aarch64 ;; \
arm64v7) DOCKER_ARCH=armhf ;; \
arm*) DOCKER_ARCH=armhf ;; \
ppc64le) DOCKER_ARCH=ppc64le ;; \
s390x) DOCKER_ARCH=s390x ;; \
*) echo "Unsupported architecture: ${TARGETARCH}${TARGETVARIANT}" && exit 1 ;; \
esac; \
echo "Downloading Docker for architecture: ${DOCKER_ARCH}"; \
curl -fsSL "https://download.docker.com/linux/static/stable/${DOCKER_ARCH}/docker-29.1.0.tgz" | \
tar xz -C /usr/local/bin --strip-components=1 docker/docker; \
ln -sf /usr/local/bin/docker /usr/bin/docker
COPY --from=ghcr.io/astral-sh/uv:0.7.5 /uv /uvx /bin/
WORKDIR /app
COPY . .
RUN if [ "$NEED_MIRROR" = 1 ]; then export UV_INDEX_URL="https://pypi.tuna.tsinghua.edu.cn/simple"; else export UV_INDEX_URL="https://pypi.org/simple"; fi && \
uv pip install --system -r requirements.txt
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9385"]

View File

@@ -0,0 +1,25 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from fastapi import APIRouter
from api.handlers import healthz_handler, run_code_handler
router = APIRouter()
router.get("/")(healthz_handler)
router.get("/healthz")(healthz_handler)
router.post("/run")(run_code_handler)

View File

@@ -0,0 +1,72 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import base64
from typing import Any, Optional
from pydantic import BaseModel, Field, field_validator
from models.enums import ResourceLimitType, ResultStatus, RuntimeErrorType, SupportLanguage, UnauthorizedAccessType
class ArtifactItem(BaseModel):
name: str
mime_type: str
size: int
content_b64: str
class ExecutionStructuredResult(BaseModel):
present: bool
value: Any = None
type: str = "json"
class CodeExecutionResult(BaseModel):
status: ResultStatus
stdout: str
stderr: str
exit_code: int
detail: Optional[str] = None
# Resource usage
time_used_ms: Optional[float] = None
memory_used_kb: Optional[float] = None
# Error details
resource_limit_type: Optional[ResourceLimitType] = None
unauthorized_access_type: Optional[UnauthorizedAccessType] = None
runtime_error_type: Optional[RuntimeErrorType] = None
# File artifacts produced by code execution (images, PDFs, CSVs, etc.)
artifacts: list[ArtifactItem] = []
# Structured return value produced by main()
result: Optional[ExecutionStructuredResult] = None
class CodeExecutionRequest(BaseModel):
code_b64: str = Field(..., description="Base64 encoded code string")
language: SupportLanguage = Field(default=SupportLanguage.PYTHON, description="Programming language")
arguments: Optional[dict] = Field(default={}, description="Arguments")
@field_validator("code_b64")
@classmethod
def validate_base64(cls, v: str) -> str:
try:
base64.b64decode(v, validate=True)
return v
except Exception as e:
raise ValueError(f"Invalid base64 encoding: {str(e)}")

View File

@@ -0,0 +1,429 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import asyncio
import base64
import json
import os
import time
import uuid
from core.config import TIMEOUT
from core.container import allocate_container_blocking, release_container
from core.logger import logger
from models.enums import ResourceLimitType, ResultStatus, RuntimeErrorType, SupportLanguage, UnauthorizedAccessType
from models.schemas import ArtifactItem, CodeExecutionRequest, CodeExecutionResult, ExecutionStructuredResult
from utils.common import async_run_command
RESULT_MARKER_PREFIX = "__RAGFLOW_RESULT__:"
def _extract_result_envelope(stdout: str) -> tuple[str, ExecutionStructuredResult | None]:
if not stdout:
return "", None
cleaned_lines: list[str] = []
envelope: ExecutionStructuredResult | None = None
for line in str(stdout).splitlines():
if line.startswith(RESULT_MARKER_PREFIX):
payload_b64 = line[len(RESULT_MARKER_PREFIX) :].strip()
if not payload_b64:
continue
try:
payload = base64.b64decode(payload_b64).decode("utf-8")
envelope = ExecutionStructuredResult.model_validate_json(payload)
except Exception as exc:
logger.warning(f"Failed to decode structured result marker: {exc}")
cleaned_lines.append(line)
continue
cleaned_lines.append(line)
cleaned_stdout = "\n".join(cleaned_lines)
if stdout.endswith("\n") and cleaned_stdout and not cleaned_stdout.endswith("\n"):
cleaned_stdout += "\n"
return cleaned_stdout, envelope
def _build_execution_bundle(req: CodeExecutionRequest, workdir: str) -> dict[str, str | bytes]:
arguments = req.arguments or {}
args_source = json.dumps(arguments, ensure_ascii=False)
args_name = "args.json"
code_bytes = base64.b64decode(req.code_b64)
if req.language == SupportLanguage.PYTHON:
code_name = "main.py"
runner_name = "runner.py"
runner_source = f"""import base64
import json
import os
import sys
os.makedirs(os.path.join(os.getcwd(), "artifacts"), exist_ok=True)
sys.path.insert(0, os.path.dirname(__file__))
from main import main
RESULT_MARKER_PREFIX = {RESULT_MARKER_PREFIX!r}
def emit_result(value):
payload = json.dumps(
{{
"present": True,
"value": value,
"type": "json",
}},
ensure_ascii=False,
separators=(",", ":"),
)
print(RESULT_MARKER_PREFIX + base64.b64encode(payload.encode("utf-8")).decode("ascii"))
if __name__ == "__main__":
with open(os.path.join(os.path.dirname(__file__), "args.json"), encoding="utf-8") as f:
args = json.load(f)
result = main(**args)
emit_result(result)
"""
elif req.language == SupportLanguage.NODEJS:
code_name = "main.js"
runner_name = "runner.js"
runner_source = """
const fs = require('fs');
const path = require('path');
const args = JSON.parse(fs.readFileSync(path.join(__dirname, 'args.json'), 'utf8'));
const mainPath = path.join(__dirname, 'main.js');
const RESULT_MARKER_PREFIX = '__RESULT_MARKER_PREFIX__';
function isPromise(value) {
return Boolean(value && typeof value.then === 'function');
}
function emitResult(value) {
if (typeof value === 'undefined') {
console.error('Error: main() must return a value. Use null for an empty result.');
process.exit(1);
}
const payload = JSON.stringify({ present: true, value, type: 'json' });
if (typeof payload === 'undefined') {
console.error('Error: main() returned a non-JSON-serializable value.');
process.exit(1);
}
console.log(RESULT_MARKER_PREFIX + Buffer.from(payload, 'utf8').toString('base64'));
}
if (fs.existsSync(mainPath)) {
const mod = require(mainPath);
const main = typeof mod === 'function' ? mod : mod.main;
if (typeof main !== 'function') {
console.error('Error: main is not a function');
process.exit(1);
}
if (typeof args === 'object' && args !== null) {
try {
const result = Promise.resolve(main(args));
if (isPromise(result)) {
result.then(output => {
emitResult(output);
}).catch(err => {
console.error('Error in async main function:', err);
process.exit(1);
});
} else {
emitResult(result);
}
} catch (err) {
console.error('Error when executing main:', err);
process.exit(1);
}
} else {
console.error('Error: args is not a valid object:', args);
process.exit(1);
}
} else {
console.error('main.js not found in the current directory');
process.exit(1);
}
"""
runner_source = runner_source.replace("__RESULT_MARKER_PREFIX__", RESULT_MARKER_PREFIX)
else:
assert False, "Will never reach here"
return {
"code_name": code_name,
"code_bytes": code_bytes,
"runner_name": runner_name,
"runner_source": runner_source,
"args_name": args_name,
"args_source": args_source,
}
def _build_container_run_args(language: SupportLanguage, task_id: str, container: str, runner_name: str) -> list[str]:
run_args = [
"docker",
"exec",
"--workdir",
f"/workspace/{task_id}",
container,
"timeout",
str(TIMEOUT),
language,
]
if language == SupportLanguage.PYTHON:
run_args.extend(["-I", "-B"])
run_args.append(runner_name)
return run_args
async def execute_code(req: CodeExecutionRequest):
language = req.language
container = await allocate_container_blocking(language)
if not container:
return CodeExecutionResult(
status=ResultStatus.PROGRAM_RUNNER_ERROR,
stdout="",
stderr="Container pool is busy",
exit_code=-10,
detail="no_available_container",
)
task_id = str(uuid.uuid4())
workdir = f"/tmp/sandbox_{task_id}"
os.makedirs(workdir, mode=0o700, exist_ok=True)
try:
bundle = _build_execution_bundle(req, workdir)
code_name = str(bundle["code_name"])
runner_name = str(bundle["runner_name"])
code_path = os.path.join(workdir, code_name)
with open(code_path, "wb") as f:
f.write(bundle["code_bytes"])
runner_path = os.path.join(workdir, runner_name)
with open(runner_path, "w", encoding="utf-8") as f:
f.write(str(bundle["runner_source"]))
args_path = os.path.join(workdir, str(bundle["args_name"]))
with open(args_path, "w", encoding="utf-8") as f:
f.write(str(bundle["args_source"]))
returncode, _, stderr = await async_run_command("docker", "exec", container, "mkdir", "-p", f"/workspace/{task_id}", timeout=5)
if returncode != 0:
raise RuntimeError(f"Directory creation failed: {stderr}")
tar_proc = await asyncio.create_subprocess_exec(
"tar", "czf", "-", "-C", workdir, code_name, runner_name, str(bundle["args_name"]), stdout=asyncio.subprocess.PIPE
)
tar_stdout, _ = await tar_proc.communicate()
docker_proc = await asyncio.create_subprocess_exec(
"docker", "exec", "-i", container, "tar", "xzf", "-", "-C", f"/workspace/{task_id}", stdin=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
)
stdout, stderr = await docker_proc.communicate(input=tar_stdout)
if docker_proc.returncode != 0:
raise RuntimeError(stderr.decode())
start_time = time.time()
try:
arguments = req.arguments or {}
logger.info("Passed in args keys=%s size_bytes=%s", list(arguments.keys()), len(json.dumps(arguments, ensure_ascii=False).encode("utf-8")))
run_args = _build_container_run_args(language=language, task_id=task_id, container=container, runner_name=runner_name)
returncode, stdout, stderr = await async_run_command(
*run_args,
timeout=TIMEOUT + 5,
)
time_used_ms = (time.time() - start_time) * 1000
logger.info("----------------------------------------------")
logger.info(f"Code: {str(base64.b64decode(req.code_b64))}")
logger.info(f"{returncode=}")
logger.info(f"{stdout=}")
logger.info(f"{stderr=}")
if returncode == 0:
clean_stdout, structured_result = _extract_result_envelope(stdout)
artifacts = await _collect_artifacts(container, task_id, workdir)
return CodeExecutionResult(
status=ResultStatus.SUCCESS,
stdout=clean_stdout,
stderr=stderr,
exit_code=0,
time_used_ms=time_used_ms,
artifacts=artifacts,
result=structured_result,
)
elif returncode == 124:
return CodeExecutionResult(
status=ResultStatus.RESOURCE_LIMIT_EXCEEDED,
stdout="",
stderr="Execution timeout",
exit_code=-124,
resource_limit_type=ResourceLimitType.TIME,
time_used_ms=time_used_ms,
)
elif returncode == 137:
return CodeExecutionResult(
status=ResultStatus.RESOURCE_LIMIT_EXCEEDED,
stdout="",
stderr="Memory limit exceeded (killed by OOM)",
exit_code=-137,
resource_limit_type=ResourceLimitType.MEMORY,
time_used_ms=time_used_ms,
)
return analyze_error_result(stderr, returncode)
except asyncio.TimeoutError:
await async_run_command("docker", "exec", container, "pkill", "-9", language)
return CodeExecutionResult(
status=ResultStatus.RESOURCE_LIMIT_EXCEEDED,
stdout="",
stderr="Execution timeout",
exit_code=-1,
resource_limit_type=ResourceLimitType.TIME,
time_used_ms=(time.time() - start_time) * 1000,
)
except Exception as e:
logger.error(f"Execution exception: {str(e)}")
return CodeExecutionResult(status=ResultStatus.PROGRAM_RUNNER_ERROR, stdout="", stderr=str(e), exit_code=-3, detail="internal_error")
finally:
cleanup_tasks = [async_run_command("docker", "exec", container, "rm", "-rf", f"/workspace/{task_id}"), async_run_command("rm", "-rf", workdir)]
await asyncio.gather(*cleanup_tasks, return_exceptions=True)
await release_container(container, language)
ALLOWED_ARTIFACT_EXTENSIONS = {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".svg": "image/svg+xml",
".pdf": "application/pdf",
".csv": "text/csv",
".json": "application/json",
".html": "text/html",
}
MAX_ARTIFACT_COUNT = 10
MAX_ARTIFACT_SIZE = 10 * 1024 * 1024 # 10MB per file
async def _collect_artifacts(container: str, task_id: str, host_workdir: str) -> list[ArtifactItem]:
artifacts_path = f"/workspace/{task_id}/artifacts"
# List files in the artifacts directory inside the container
returncode, stdout, _ = await async_run_command(
"docker", "exec", container, "find", artifacts_path,
"-maxdepth", "1", "-type", "f", timeout=5,
)
if returncode != 0 or not stdout.strip():
return []
raw_names = [line.split("/")[-1] for line in stdout.strip().splitlines() if line.strip()]
# Sanitize: reject names with path traversal or control characters
filenames = [n for n in raw_names if n and "/" not in n and "\\" not in n and ".." not in n and not n.startswith(".")]
if not filenames:
return []
items: list[ArtifactItem] = []
for fname in filenames[:MAX_ARTIFACT_COUNT]:
ext = os.path.splitext(fname)[1].lower()
mime_type = ALLOWED_ARTIFACT_EXTENSIONS.get(ext)
if not mime_type:
logger.warning(f"Skipping artifact with disallowed extension: {fname}")
continue
file_path = f"{artifacts_path}/{fname}"
# Check file size inside the container
returncode, size_str, _ = await async_run_command(
"docker", "exec", container, "stat", "-c", "%s", file_path, timeout=5,
)
if returncode != 0:
logger.warning(f"Failed to stat artifact {fname}")
continue
file_size = int(size_str.strip())
if file_size > MAX_ARTIFACT_SIZE:
logger.warning(f"Artifact {fname} too large ({file_size} bytes), skipping")
continue
if file_size == 0:
continue
# Read file content via docker exec (docker cp doesn't work with gVisor tmpfs)
returncode, content_b64, stderr = await async_run_command(
"docker", "exec", container, "base64", file_path, timeout=30,
)
if returncode != 0:
logger.warning(f"Failed to read artifact {fname}: {stderr}")
continue
content_b64 = content_b64.replace("\n", "").strip()
items.append(ArtifactItem(
name=fname,
mime_type=mime_type,
size=file_size,
content_b64=content_b64,
))
logger.info(f"Collected artifact: {fname} ({file_size} bytes, {mime_type})")
return items
def analyze_error_result(stderr: str, exit_code: int) -> CodeExecutionResult:
"""Analyze the error result and classify it"""
if "Permission denied" in stderr:
return CodeExecutionResult(
status=ResultStatus.UNAUTHORIZED_ACCESS,
stdout="",
stderr=stderr,
exit_code=exit_code,
unauthorized_access_type=UnauthorizedAccessType.FILE_ACCESS,
)
elif "Operation not permitted" in stderr:
return CodeExecutionResult(
status=ResultStatus.UNAUTHORIZED_ACCESS,
stdout="",
stderr=stderr,
exit_code=exit_code,
unauthorized_access_type=UnauthorizedAccessType.DISALLOWED_SYSCALL,
)
elif "MemoryError" in stderr:
return CodeExecutionResult(
status=ResultStatus.RESOURCE_LIMIT_EXCEEDED,
stdout="",
stderr=stderr,
exit_code=exit_code,
resource_limit_type=ResourceLimitType.MEMORY,
)
else:
return CodeExecutionResult(
status=ResultStatus.PROGRAM_ERROR,
stdout="",
stderr=stderr,
exit_code=exit_code,
runtime_error_type=RuntimeErrorType.NONZERO_EXIT,
)

View File

@@ -0,0 +1,207 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import ast
import re
from typing import List, Tuple
from core.logger import logger
from models.enums import SupportLanguage
class SecurePythonAnalyzer(ast.NodeVisitor):
"""
An AST-based analyzer for detecting unsafe Python code patterns.
"""
DANGEROUS_IMPORTS = {"os", "subprocess", "sys", "shutil", "socket", "ctypes", "pickle", "threading", "multiprocessing", "asyncio", "http.client", "ftplib", "telnetlib", "builtins"}
DANGEROUS_CALLS = {
"eval",
"exec",
"open",
"__import__",
"compile",
"input",
"system",
"popen",
"remove",
"rename",
"rmdir",
"chdir",
"chmod",
"chown",
"getattr",
"setattr",
"globals",
"locals",
"shutil.rmtree",
"subprocess.call",
"subprocess.Popen",
"ctypes",
"pickle.load",
"pickle.loads",
"pickle.dump",
"pickle.dumps",
}
def __init__(self):
self.unsafe_items: List[Tuple[str, int]] = []
def visit_Import(self, node: ast.Import):
"""Check for dangerous imports."""
for alias in node.names:
if alias.name.split(".")[0] in self.DANGEROUS_IMPORTS:
self.unsafe_items.append((f"Import: {alias.name}", node.lineno))
self.generic_visit(node)
def visit_ImportFrom(self, node: ast.ImportFrom):
"""Check for dangerous imports from specific modules."""
if node.module and node.module.split(".")[0] in self.DANGEROUS_IMPORTS:
self.unsafe_items.append((f"From Import: {node.module}", node.lineno))
self.generic_visit(node)
def visit_Call(self, node: ast.Call):
"""Check for dangerous function calls."""
if isinstance(node.func, ast.Name) and node.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append((f"Call: {node.func.id}", node.lineno))
elif isinstance(node.func, ast.Attribute) and node.func.attr in self.DANGEROUS_CALLS:
# Surface the attribute-style match in the analyzer log so that
# incident response can grep for it just like the other unsafe-item
# findings; the bare append is invisible to operators.
logger.warning(
"[SafeCheck] Attribute-style dangerous call detected: %s (line %s)",
node.func.attr,
node.lineno,
)
self.unsafe_items.append((f"Call: {node.func.attr}", node.lineno))
self.generic_visit(node)
def visit_Attribute(self, node: ast.Attribute):
"""Check for dangerous attribute access."""
if isinstance(node.value, ast.Name) and node.value.id in self.DANGEROUS_IMPORTS:
self.unsafe_items.append((f"Attribute Access: {node.value.id}.{node.attr}", node.lineno))
self.generic_visit(node)
def visit_BinOp(self, node: ast.BinOp):
"""Check for possible unsafe operations like concatenating strings with commands."""
# This could be useful to detect `eval("os." + "system")`
if isinstance(node.left, ast.Constant) and isinstance(node.right, ast.Constant):
self.unsafe_items.append(("Possible unsafe string concatenation", node.lineno))
self.generic_visit(node)
def visit_FunctionDef(self, node: ast.FunctionDef):
"""Check for dangerous function definitions (e.g., user-defined eval)."""
if node.name in self.DANGEROUS_CALLS:
self.unsafe_items.append((f"Function Definition: {node.name}", node.lineno))
self.generic_visit(node)
def visit_Assign(self, node: ast.Assign):
"""Check for assignments to variables that might lead to dangerous operations."""
for target in node.targets:
if isinstance(target, ast.Name) and target.id in self.DANGEROUS_CALLS:
self.unsafe_items.append((f"Assignment to dangerous variable: {target.id}", node.lineno))
self.generic_visit(node)
def visit_Lambda(self, node: ast.Lambda):
"""Check for lambda functions with dangerous operations."""
if isinstance(node.body, ast.Call) and isinstance(node.body.func, ast.Name) and node.body.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("Lambda with dangerous function call", node.lineno))
self.generic_visit(node)
def visit_ListComp(self, node: ast.ListComp):
"""Check for list comprehensions with dangerous operations."""
# First, visit the generators to check for any issues there
for elem in node.generators:
if isinstance(elem, ast.comprehension):
self.generic_visit(elem)
if isinstance(node.elt, ast.Call) and isinstance(node.elt.func, ast.Name) and node.elt.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("List comprehension with dangerous function call", node.lineno))
self.generic_visit(node)
def visit_DictComp(self, node: ast.DictComp):
"""Check for dictionary comprehensions with dangerous operations."""
# Check for dangerous calls in both the key and value expressions of the dictionary comprehension
if isinstance(node.key, ast.Call) and isinstance(node.key.func, ast.Name) and node.key.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("Dict comprehension with dangerous function call in key", node.lineno))
if isinstance(node.value, ast.Call) and isinstance(node.value.func, ast.Name) and node.value.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("Dict comprehension with dangerous function call in value", node.lineno))
# Visit other sub-nodes (e.g., the generators in the comprehension)
self.generic_visit(node)
def visit_SetComp(self, node: ast.SetComp):
"""Check for set comprehensions with dangerous operations."""
for elt in node.generators:
if isinstance(elt, ast.comprehension):
self.generic_visit(elt)
if isinstance(node.elt, ast.Call) and isinstance(node.elt.func, ast.Name) and node.elt.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("Set comprehension with dangerous function call", node.lineno))
self.generic_visit(node)
def visit_Yield(self, node: ast.Yield):
"""Check for yield statements that could be used to produce unsafe values."""
if isinstance(node.value, ast.Call) and isinstance(node.value.func, ast.Name) and node.value.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append(("Yield with dangerous function call", node.lineno))
self.generic_visit(node)
class SecureJavaScriptAnalyzer:
DANGEROUS_PATTERNS = [
(re.compile(r"""require\s*\(\s*['"`]child_process['"`]\s*\)"""), "Require: child_process"),
(re.compile(r"""require\s*\(\s*['"`]fs['"`]\s*\)"""), "Require: fs"),
(re.compile(r"""require\s*\(\s*['"`]worker_threads['"`]\s*\)"""), "Require: worker_threads"),
(re.compile(r"""\beval\s*\("""), "Call: eval"),
(re.compile(r"""\bFunction\s*\("""), "Call: Function"),
(re.compile(r"""\bprocess\s*\.\s*binding\s*\("""), "Call: process.binding"),
]
@classmethod
def analyze(cls, code: str) -> List[Tuple[str, int]]:
issues: List[Tuple[str, int]] = []
for pattern, description in cls.DANGEROUS_PATTERNS:
for match in pattern.finditer(code):
lineno = code.count("\n", 0, match.start()) + 1
issues.append((description, lineno))
return issues
def analyze_code_security(code: str, language: SupportLanguage) -> Tuple[bool, List[Tuple[str, int]]]:
"""
Analyze the provided code string and return whether it's safe and why.
:param code: The source code to analyze.
:param language: The programming language of the code.
:return: (is_safe: bool, issues: List of (description, line number))
"""
if language == SupportLanguage.PYTHON:
try:
tree = ast.parse(code)
analyzer = SecurePythonAnalyzer()
analyzer.visit(tree)
return len(analyzer.unsafe_items) == 0, analyzer.unsafe_items
except Exception as e:
logger.error(f"[SafeCheck] Python parsing failed: {str(e)}")
return False, [(f"Parsing Error: {str(e)}", -1)]
if language == SupportLanguage.NODEJS:
issues = SecureJavaScriptAnalyzer.analyze(code)
return len(issues) == 0, issues
logger.warning(f"[SafeCheck] Unsupported language for security analysis: {language}")
return False, [(f"Unsupported language for security analysis: {language}", -1)]

View File

@@ -0,0 +1,47 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Sandbox providers package.
This package contains:
- base.py: Base interface for all sandbox providers
- manager.py: Provider manager for managing active provider
- self_managed.py: Self-managed provider implementation (wraps existing executor_manager)
- aliyun_codeinterpreter.py: Aliyun Code Interpreter provider implementation
Official Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
- e2b.py: E2B provider implementation
- local.py: Local process provider implementation
"""
from .base import SandboxProvider, SandboxInstance, ExecutionResult, SandboxProviderConfigError
from .manager import ProviderManager
from .self_managed import SelfManagedProvider
from .aliyun_codeinterpreter import AliyunCodeInterpreterProvider
from .e2b import E2BProvider
from .local import LocalProvider
__all__ = [
"SandboxProvider",
"SandboxInstance",
"ExecutionResult",
"SandboxProviderConfigError",
"ProviderManager",
"SelfManagedProvider",
"AliyunCodeInterpreterProvider",
"E2BProvider",
"LocalProvider",
]

View File

@@ -0,0 +1,485 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Aliyun Code Interpreter provider implementation.
This provider integrates with Aliyun Function Compute Code Interpreter service
for secure code execution in serverless microVMs using the official agentrun-sdk.
Official Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
Official SDK: https://github.com/Serverless-Devs/agentrun-sdk-python
https://api.aliyun.com/api/AgentRun/2025-09-10/CreateTemplate?lang=PYTHON
https://api.aliyun.com/api/AgentRun/2025-09-10/CreateSandbox?lang=PYTHON
"""
import logging
import os
import time
import json
from typing import Dict, Any, List, Optional
from datetime import datetime, timezone
from agentrun.sandbox import TemplateType, CodeLanguage, Template, TemplateInput, Sandbox
from agentrun.utils.config import Config
from agentrun.utils.exception import ServerError
from agent.sandbox.result_protocol import build_javascript_wrapper, build_python_wrapper, extract_structured_result
from .base import SandboxProvider, SandboxInstance, ExecutionResult
logger = logging.getLogger(__name__)
class AliyunCodeInterpreterProvider(SandboxProvider):
"""
Aliyun Code Interpreter provider implementation.
This provider uses the official agentrun-sdk to interact with
Aliyun Function Compute's Code Interpreter service.
"""
def __init__(self):
self.access_key_id: Optional[str] = ""
self.access_key_secret: Optional[str] = ""
self.account_id: Optional[str] = ""
self.region: str = "cn-hangzhou"
self.template_name: str = ""
self.timeout: int = 30
self._initialized: bool = False
self._config: Optional[Config] = None
def initialize(self, config: Dict[str, Any]) -> bool:
"""
Initialize the provider with Aliyun credentials.
Args:
config: Configuration dictionary with keys:
- access_key_id: Aliyun AccessKey ID
- access_key_secret: Aliyun AccessKey Secret
- account_id: Aliyun primary account ID
- region: Region (default: "cn-hangzhou")
- template_name: Optional sandbox template name
- timeout: Request timeout in seconds (default: 30, max 30)
Returns:
True if initialization successful, False otherwise
"""
# Get values from config or environment variables
access_key_id = config.get("access_key_id") or os.getenv("AGENTRUN_ACCESS_KEY_ID")
access_key_secret = config.get("access_key_secret") or os.getenv("AGENTRUN_ACCESS_KEY_SECRET")
account_id = config.get("account_id") or os.getenv("AGENTRUN_ACCOUNT_ID")
region = config.get("region") or os.getenv("AGENTRUN_REGION", "cn-hangzhou")
self.access_key_id = access_key_id
self.access_key_secret = access_key_secret
self.account_id = account_id
self.region = region
self.template_name = config.get("template_name", "")
self.timeout = min(config.get("timeout", 30), 30) # Max 30 seconds
logger.info(f"Aliyun Code Interpreter: Initializing with account_id={self.account_id}, region={self.region}")
# Validate required fields
if not self.access_key_id or not self.access_key_secret:
logger.error("Aliyun Code Interpreter: Missing access_key_id or access_key_secret")
return False
if not self.account_id:
logger.error("Aliyun Code Interpreter: Missing account_id (primary account ID)")
return False
# Create SDK configuration
try:
logger.info(f"Aliyun Code Interpreter: Creating Config object with account_id={self.account_id}")
self._config = Config(
access_key_id=self.access_key_id,
access_key_secret=self.access_key_secret,
account_id=self.account_id,
region_id=self.region,
timeout=self.timeout,
)
logger.info("Aliyun Code Interpreter: Config object created successfully")
# Verify connection with health check
if not self.health_check():
logger.error(f"Aliyun Code Interpreter: Health check failed for region {self.region}")
return False
self._initialized = True
logger.info(f"Aliyun Code Interpreter: Initialized successfully for region {self.region}")
return True
except Exception as e:
logger.error(f"Aliyun Code Interpreter: Initialization failed - {str(e)}")
return False
def create_instance(self, template: str = "python") -> SandboxInstance:
"""
Create a new sandbox instance in Aliyun Code Interpreter.
Args:
template: Programming language (python, javascript)
Returns:
SandboxInstance object
Raises:
RuntimeError: If instance creation fails
"""
if not self._initialized or not self._config:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# Normalize language
language = self._normalize_language(template)
try:
# Get or create template
if self.template_name:
# Use existing template
template_name = self.template_name
else:
# Try to get default template, or create one if it doesn't exist
default_template_name = f"ragflow-{language}-default"
try:
# Check if template exists
Template.get_by_name(default_template_name, config=self._config)
template_name = default_template_name
except Exception:
# Create default template if it doesn't exist
template_input = TemplateInput(
template_name=default_template_name,
template_type=TemplateType.CODE_INTERPRETER,
)
Template.create(template_input, config=self._config)
template_name = default_template_name
# Create sandbox directly
sandbox = Sandbox.create(
template_type=TemplateType.CODE_INTERPRETER,
template_name=template_name,
sandbox_idle_timeout_seconds=self.timeout,
config=self._config,
)
instance_id = sandbox.sandbox_id
return SandboxInstance(
instance_id=instance_id,
provider="aliyun_codeinterpreter",
status="READY",
metadata={
"language": language,
"region": self.region,
"account_id": self.account_id,
"template_name": template_name,
"created_at": datetime.now(timezone.utc).isoformat(),
},
)
except ServerError as e:
raise RuntimeError(f"Failed to create sandbox instance: {str(e)}")
except Exception as e:
raise RuntimeError(f"Unexpected error creating instance: {str(e)}")
def execute_code(self, instance_id: str, code: str, language: str, timeout: int = 10, arguments: Optional[Dict[str, Any]] = None) -> ExecutionResult:
"""
Execute code in the Aliyun Code Interpreter instance.
Args:
instance_id: ID of the sandbox instance
code: Source code to execute
language: Programming language (python, javascript)
timeout: Maximum execution time in seconds (max 30)
arguments: Optional arguments dict to pass to main() function
Returns:
ExecutionResult containing stdout, stderr, exit_code, and metadata
Raises:
RuntimeError: If execution fails
TimeoutError: If execution exceeds timeout
"""
if not self._initialized or not self._config:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# Normalize language
normalized_lang = self._normalize_language(language)
# Enforce 30-second hard limit
timeout = min(timeout or self.timeout, 30)
try:
# Connect to existing sandbox instance
sandbox = Sandbox.connect(sandbox_id=instance_id, config=self._config)
# agentrun-sdk 0.0.26 only exposes CodeLanguage.PYTHON; keep JS as string fallback.
code_language = CodeLanguage.PYTHON if normalized_lang == "python" else "javascript"
# Wrap code to call main() function
# Matches self_managed provider behavior: call main(**arguments)
args_json = json.dumps(arguments or {})
wrapped_code = (
build_python_wrapper(code, args_json)
if normalized_lang == "python"
else build_javascript_wrapper(code, args_json)
)
logger.debug(f"Aliyun Code Interpreter: Wrapped code (first 200 chars): {wrapped_code[:200]}")
start_time = time.time()
# Execute code using SDK's simplified execute endpoint
logger.info(f"Aliyun Code Interpreter: Executing code (language={normalized_lang}, timeout={timeout})")
logger.debug(f"Aliyun Code Interpreter: Original code (first 200 chars): {code[:200]}")
result = sandbox.context.execute(
code=wrapped_code,
language=code_language,
timeout=timeout,
)
execution_time = time.time() - start_time
logger.info(f"Aliyun Code Interpreter: Execution completed in {execution_time:.2f}s")
logger.debug(f"Aliyun Code Interpreter: Raw SDK result: {result}")
# Parse execution result
results = result.get("results", []) if isinstance(result, dict) else []
logger.info(f"Aliyun Code Interpreter: Parsed {len(results)} result items")
# Extract stdout and stderr from results
stdout_parts = []
stderr_parts = []
exit_code = 0
execution_status = "ok"
for item in results:
result_type = item.get("type", "")
text = item.get("text", "")
if result_type == "stdout":
stdout_parts.append(text)
elif result_type == "stderr":
stderr_parts.append(text)
exit_code = 1 # Error occurred
elif result_type == "endOfExecution":
execution_status = item.get("status", "ok")
if execution_status != "ok":
exit_code = 1
elif result_type == "error":
stderr_parts.append(text)
exit_code = 1
stdout = "\n".join(stdout_parts)
stderr = "\n".join(stderr_parts)
stdout, structured_result = extract_structured_result(stdout)
logger.info(f"Aliyun Code Interpreter: stdout length={len(stdout)}, stderr length={len(stderr)}, exit_code={exit_code}")
if stdout:
logger.debug(f"Aliyun Code Interpreter: stdout (first 200 chars): {stdout[:200]}")
if stderr:
logger.debug(f"Aliyun Code Interpreter: stderr (first 200 chars): {stderr[:200]}")
return ExecutionResult(
stdout=stdout,
stderr=stderr,
exit_code=exit_code,
execution_time=execution_time,
metadata={
"instance_id": instance_id,
"language": normalized_lang,
"context_id": result.get("contextId") if isinstance(result, dict) else None,
"timeout": timeout,
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
},
)
except ServerError as e:
if "timeout" in str(e).lower():
raise TimeoutError(f"Execution timed out after {timeout} seconds")
raise RuntimeError(f"Failed to execute code: {str(e)}")
except Exception as e:
raise RuntimeError(f"Unexpected error during execution: {str(e)}")
def destroy_instance(self, instance_id: str) -> bool:
"""
Destroy an Aliyun Code Interpreter instance.
Args:
instance_id: ID of the instance to destroy
Returns:
True if destruction successful, False otherwise
"""
if not self._initialized or not self._config:
raise RuntimeError("Provider not initialized. Call initialize() first.")
try:
# Delete sandbox by ID directly
Sandbox.delete_by_id(sandbox_id=instance_id)
logger.info(f"Successfully destroyed sandbox instance {instance_id}")
return True
except ServerError as e:
logger.error(f"Failed to destroy instance {instance_id}: {str(e)}")
return False
except Exception as e:
logger.error(f"Unexpected error destroying instance {instance_id}: {str(e)}")
return False
def health_check(self) -> bool:
"""
Check if the Aliyun Code Interpreter service is accessible.
Returns:
True if provider is healthy, False otherwise
"""
if not self._initialized and not (self.access_key_id and self.account_id):
return False
try:
# Try to list templates to verify connection
from agentrun.sandbox import Template
templates = Template.list(config=self._config)
return templates is not None
except Exception as e:
logger.warning(f"Aliyun Code Interpreter health check failed: {str(e)}")
# If we get any response (even an error), the service is reachable
return "connection" not in str(e).lower()
def get_supported_languages(self) -> List[str]:
"""
Get list of supported programming languages.
Returns:
List of language identifiers
"""
return ["python", "javascript"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
"""
Return configuration schema for Aliyun Code Interpreter provider.
Returns:
Dictionary mapping field names to their schema definitions
"""
return {
"access_key_id": {
"type": "string",
"required": True,
"label": "Access Key ID",
"placeholder": "LTAI5t...",
"description": "Aliyun AccessKey ID for authentication",
"secret": False,
},
"access_key_secret": {
"type": "string",
"required": True,
"label": "Access Key Secret",
"placeholder": "••••••••••••••••",
"description": "Aliyun AccessKey Secret for authentication",
"secret": True,
},
"account_id": {
"type": "string",
"required": True,
"label": "Account ID",
"placeholder": "1234567890...",
"description": "Aliyun primary account ID, required for API calls",
},
"region": {
"type": "string",
"required": False,
"label": "Region",
"default": "cn-hangzhou",
"description": "Aliyun region for Code Interpreter service",
"options": ["cn-hangzhou", "cn-beijing", "cn-shanghai", "cn-shenzhen", "cn-guangzhou"],
},
"template_name": {
"type": "string",
"required": False,
"label": "Template Name",
"placeholder": "my-interpreter",
"description": "Optional sandbox template name for pre-configured environments",
},
"timeout": {
"type": "integer",
"required": False,
"label": "Execution Timeout (seconds)",
"default": 30,
"min": 1,
"max": 30,
"description": "Code execution timeout (max 30 seconds - hard limit)",
},
}
def validate_config(self, config: Dict[str, Any]) -> tuple[bool, Optional[str]]:
"""
Validate Aliyun-specific configuration.
Args:
config: Configuration dictionary to validate
Returns:
Tuple of (is_valid, error_message)
"""
# Validate access key format
access_key_id = config.get("access_key_id", "")
if access_key_id and not access_key_id.startswith("LTAI"):
return False, "Invalid AccessKey ID format (should start with 'LTAI')"
# Validate account ID
account_id = config.get("account_id", "")
if not account_id:
return False, "Account ID is required"
# Validate region
valid_regions = ["cn-hangzhou", "cn-beijing", "cn-shanghai", "cn-shenzhen", "cn-guangzhou"]
region = config.get("region", "cn-hangzhou")
if region and region not in valid_regions:
return False, f"Invalid region. Must be one of: {', '.join(valid_regions)}"
# Validate timeout range (max 30 seconds)
timeout = config.get("timeout", 30)
if isinstance(timeout, int) and (timeout < 1 or timeout > 30):
return False, "Timeout must be between 1 and 30 seconds"
return True, None
def _normalize_language(self, language: str) -> str:
"""
Normalize language identifier to Aliyun format.
Args:
language: Language identifier (python, python3, javascript, nodejs)
Returns:
Normalized language identifier
"""
if not language:
return "python"
lang_lower = language.lower()
if lang_lower in ("python", "python3"):
return "python"
elif lang_lower in ("javascript", "nodejs"):
return "javascript"
else:
return language

View File

@@ -0,0 +1,216 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Base interface for sandbox providers.
Each sandbox provider (self-managed, SaaS) implements this interface
to provide code execution capabilities.
"""
from abc import ABC, abstractmethod
from dataclasses import dataclass
from typing import Dict, Any, Optional, List
class SandboxProviderConfigError(Exception):
"""Raised when the selected provider is explicitly configured but unusable."""
@dataclass
class SandboxInstance:
"""Represents a sandbox execution instance"""
instance_id: str
provider: str
status: str # running, stopped, error
metadata: Dict[str, Any]
def __post_init__(self):
if self.metadata is None:
self.metadata = {}
@dataclass
class ExecutionResult:
"""Result of code execution in a sandbox"""
stdout: str
stderr: str
exit_code: int
execution_time: float # in seconds
metadata: Dict[str, Any]
def __post_init__(self):
if self.metadata is None:
self.metadata = {}
class SandboxProvider(ABC):
"""
Base interface for all sandbox providers.
Each provider implementation (self-managed, Aliyun OpenSandbox, E2B, etc.)
must implement these methods to provide code execution capabilities.
"""
@abstractmethod
def initialize(self, config: Dict[str, Any]) -> bool:
"""
Initialize the provider with configuration.
Args:
config: Provider-specific configuration dictionary
Returns:
True if initialization successful, False otherwise
"""
pass
@abstractmethod
def create_instance(self, template: str = "python") -> SandboxInstance:
"""
Create a new sandbox instance.
Args:
template: Programming language/template for the instance
(e.g., "python", "nodejs", "bash")
Returns:
SandboxInstance object representing the created instance
Raises:
RuntimeError: If instance creation fails
"""
pass
@abstractmethod
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10,
arguments: Optional[Dict[str, Any]] = None
) -> ExecutionResult:
"""
Execute code in a sandbox instance.
Args:
instance_id: ID of the sandbox instance
code: Source code to execute
language: Programming language (python, javascript, etc.)
timeout: Maximum execution time in seconds
arguments: Optional arguments dict to pass to main() function
Returns:
ExecutionResult containing stdout, stderr, exit_code, and metadata
Raises:
RuntimeError: If execution fails
TimeoutError: If execution exceeds timeout
"""
pass
@abstractmethod
def destroy_instance(self, instance_id: str) -> bool:
"""
Destroy a sandbox instance.
Args:
instance_id: ID of the instance to destroy
Returns:
True if destruction successful, False otherwise
Raises:
RuntimeError: If destruction fails
"""
pass
@abstractmethod
def health_check(self) -> bool:
"""
Check if the provider is healthy and accessible.
Returns:
True if provider is healthy, False otherwise
"""
pass
@abstractmethod
def get_supported_languages(self) -> List[str]:
"""
Get list of supported programming languages.
Returns:
List of language identifiers (e.g., ["python", "javascript", "go"])
"""
pass
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
"""
Return configuration schema for this provider.
The schema defines what configuration fields are required/optional,
their types, validation rules, and UI labels.
Returns:
Dictionary mapping field names to their schema definitions.
Example:
{
"endpoint": {
"type": "string",
"required": True,
"label": "API Endpoint",
"placeholder": "http://localhost:9385"
},
"timeout": {
"type": "integer",
"default": 30,
"label": "Timeout (seconds)",
"min": 5,
"max": 300
}
}
"""
return {}
def validate_config(self, config: Dict[str, Any]) -> tuple[bool, Optional[str]]:
"""
Validate provider-specific configuration.
This method allows providers to implement custom validation logic beyond
the basic schema validation. Override this method to add provider-specific
checks like URL format validation, API key format validation, etc.
Args:
config: Configuration dictionary to validate
Returns:
Tuple of (is_valid, error_message):
- is_valid: True if configuration is valid, False otherwise
- error_message: Error message if invalid, None if valid
Example:
>>> def validate_config(self, config):
>>> endpoint = config.get("endpoint", "")
>>> if not endpoint.startswith(("http://", "https://")):
>>> return False, "Endpoint must start with http:// or https://"
>>> return True, None
"""
# Default implementation: no custom validation
return True, None

View File

@@ -0,0 +1,233 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
E2B provider implementation.
This provider integrates with E2B Cloud for cloud-based code execution
using Firecracker microVMs.
"""
import uuid
from typing import Dict, Any, List
from .base import SandboxProvider, SandboxInstance, ExecutionResult
class E2BProvider(SandboxProvider):
"""
E2B provider implementation.
This provider uses E2B Cloud service for secure code execution
in Firecracker microVMs.
"""
def __init__(self):
self.api_key: str = ""
self.region: str = "us"
self.timeout: int = 30
self._initialized: bool = False
def initialize(self, config: Dict[str, Any]) -> bool:
"""
Initialize the provider with E2B credentials.
Args:
config: Configuration dictionary with keys:
- api_key: E2B API key
- region: Region (us, eu) (default: "us")
- timeout: Request timeout in seconds (default: 30)
Returns:
True if initialization successful, False otherwise
"""
self.api_key = config.get("api_key", "")
self.region = config.get("region", "us")
self.timeout = config.get("timeout", 30)
# Validate required fields
if not self.api_key:
return False
# TODO: Implement actual E2B API client initialization
# For now, we'll mark as initialized but actual API calls will fail
self._initialized = True
return True
def create_instance(self, template: str = "python") -> SandboxInstance:
"""
Create a new sandbox instance in E2B.
Args:
template: Programming language template (python, nodejs, go, bash)
Returns:
SandboxInstance object
Raises:
RuntimeError: If instance creation fails
"""
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# Normalize language
language = self._normalize_language(template)
# TODO: Implement actual E2B API call
# POST /sandbox with template
instance_id = str(uuid.uuid4())
return SandboxInstance(
instance_id=instance_id,
provider="e2b",
status="running",
metadata={
"language": language,
"region": self.region,
}
)
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10
) -> ExecutionResult:
"""
Execute code in the E2B instance.
Args:
instance_id: ID of the sandbox instance
code: Source code to execute
language: Programming language (python, nodejs, go, bash)
timeout: Maximum execution time in seconds
Returns:
ExecutionResult containing stdout, stderr, exit_code, and metadata
Raises:
RuntimeError: If execution fails
TimeoutError: If execution exceeds timeout
"""
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# TODO: Implement actual E2B API call
# POST /sandbox/{sandboxID}/execute
raise RuntimeError(
"E2B provider is not yet fully implemented. "
"Please use the self-managed provider or implement the E2B API integration. "
"See https://github.com/e2b-dev/e2b for API documentation."
)
def destroy_instance(self, instance_id: str) -> bool:
"""
Destroy an E2B instance.
Args:
instance_id: ID of the instance to destroy
Returns:
True if destruction successful, False otherwise
"""
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# TODO: Implement actual E2B API call
# DELETE /sandbox/{sandboxID}
return True
def health_check(self) -> bool:
"""
Check if the E2B service is accessible.
Returns:
True if provider is healthy, False otherwise
"""
if not self._initialized:
return False
# TODO: Implement actual E2B health check API call
# GET /healthz or similar
# For now, return True if initialized with API key
return bool(self.api_key)
def get_supported_languages(self) -> List[str]:
"""
Get list of supported programming languages.
Returns:
List of language identifiers
"""
return ["python", "nodejs", "javascript", "go", "bash"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
"""
Return configuration schema for E2B provider.
Returns:
Dictionary mapping field names to their schema definitions
"""
return {
"api_key": {
"type": "string",
"required": True,
"label": "API Key",
"placeholder": "e2b_sk_...",
"description": "E2B API key for authentication",
"secret": True,
},
"region": {
"type": "string",
"required": False,
"label": "Region",
"default": "us",
"description": "E2B service region (us or eu)",
},
"timeout": {
"type": "integer",
"required": False,
"label": "Request Timeout (seconds)",
"default": 30,
"min": 5,
"max": 300,
"description": "API request timeout for code execution",
}
}
def _normalize_language(self, language: str) -> str:
"""
Normalize language identifier to E2B template format.
Args:
language: Language identifier
Returns:
Normalized language identifier
"""
if not language:
return "python"
lang_lower = language.lower()
if lang_lower in ("python", "python3"):
return "python"
elif lang_lower in ("javascript", "nodejs"):
return "nodejs"
else:
return language

View File

@@ -0,0 +1,310 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import base64
import json
import mimetypes
import os
import shutil
import signal
import subprocess
import time
import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional
from agent.sandbox.result_protocol import build_javascript_wrapper, build_python_wrapper, extract_structured_result
from .base import ExecutionResult, SandboxInstance, SandboxProvider, SandboxProviderConfigError
ALLOWED_ARTIFACT_EXTENSIONS = {
".csv",
".html",
".jpeg",
".jpg",
".json",
".pdf",
".png",
".svg",
}
LOCAL_PYTHON_THREAD_ENV_VARS = (
"OPENBLAS_NUM_THREADS",
"OMP_NUM_THREADS",
"MKL_NUM_THREADS",
"NUMEXPR_NUM_THREADS",
"BLIS_NUM_THREADS",
"VECLIB_MAXIMUM_THREADS",
)
def _env_enabled(name: str) -> bool:
return os.environ.get(name, "").strip().lower() in {"1", "true", "yes", "on"}
class LocalProvider(SandboxProvider):
"""
Execute code as a local child process.
This provider is intentionally gated by SANDBOX_LOCAL_ENABLED because it is
not a sandbox boundary. Use a low-privilege runtime account.
"""
def __init__(self):
self.python_bin = "python3"
self.node_bin = "node"
self.work_dir = Path("/tmp/ragflow-codeexec")
self.timeout = 30
self.max_memory_mb = 512
self.max_output_bytes = 1024 * 1024
self.max_artifacts = 20
self.max_artifact_bytes = 10 * 1024 * 1024
self._initialized = False
self._instances: dict[str, Path] = {}
def initialize(self, config: Dict[str, Any]) -> bool:
if not _env_enabled("SANDBOX_LOCAL_ENABLED"):
raise SandboxProviderConfigError("Local code execution is disabled. Set SANDBOX_LOCAL_ENABLED=true to enable it.")
self.python_bin = str(self._resolve_config_value(config, "python_bin", "SANDBOX_LOCAL_PYTHON_BIN", "python3"))
self.node_bin = str(self._resolve_config_value(config, "node_bin", "SANDBOX_LOCAL_NODE_BIN", "node"))
self.work_dir = Path(self._resolve_config_value(config, "work_dir", "SANDBOX_LOCAL_WORK_DIR", "/tmp/ragflow-codeexec")).resolve()
self.timeout = int(self._resolve_config_value(config, "timeout", "SANDBOX_LOCAL_TIMEOUT", 30))
self.max_memory_mb = int(self._resolve_config_value(config, "max_memory_mb", "SANDBOX_LOCAL_MAX_MEMORY_MB", 512))
self.max_output_bytes = int(self._resolve_config_value(config, "max_output_bytes", "SANDBOX_LOCAL_MAX_OUTPUT_BYTES", 1024 * 1024))
self.max_artifacts = int(self._resolve_config_value(config, "max_artifacts", "SANDBOX_LOCAL_MAX_ARTIFACTS", 20))
self.max_artifact_bytes = int(self._resolve_config_value(config, "max_artifact_bytes", "SANDBOX_LOCAL_MAX_ARTIFACT_BYTES", 10 * 1024 * 1024))
self._validate_limits()
self.work_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
self._initialized = True
return True
def create_instance(self, template: str = "python") -> SandboxInstance:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
language = self._normalize_language(template)
instance_id = str(uuid.uuid4())
instance_dir = self.work_dir / instance_id
instance_dir.mkdir(mode=0o700)
(instance_dir / "artifacts").mkdir(mode=0o700)
self._instances[instance_id] = instance_dir
return SandboxInstance(
instance_id=instance_id,
provider="local",
status="running",
metadata={"language": language, "work_dir": str(instance_dir)},
)
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10,
arguments: Optional[Dict[str, Any]] = None,
) -> ExecutionResult:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
normalized_lang = self._normalize_language(language)
instance_dir = self._instances[instance_id]
args_json = json.dumps(arguments or {}, ensure_ascii=False)
command, script_path = self._prepare_script(instance_dir, normalized_lang, code, args_json)
requested_timeout = self.timeout if timeout is None else int(timeout)
if requested_timeout <= 0:
raise RuntimeError(f"Execution timeout must be greater than 0 seconds, got {requested_timeout}.")
exec_timeout = min(requested_timeout, self.timeout)
start_time = time.time()
process = subprocess.Popen(
command,
cwd=instance_dir,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
errors="replace",
env=self._build_child_env(instance_dir),
preexec_fn=self._limit_child_process if os.name == "posix" else None,
start_new_session=os.name == "posix",
)
try:
stdout, stderr = process.communicate(timeout=exec_timeout)
except subprocess.TimeoutExpired:
if os.name == "posix":
os.killpg(process.pid, signal.SIGKILL)
else:
process.kill()
process.communicate()
raise TimeoutError(f"Execution timed out after {exec_timeout} seconds")
execution_time = time.time() - start_time
self._validate_output_size(stdout, stderr)
stdout, structured_result = extract_structured_result(stdout)
return ExecutionResult(
stdout=stdout,
stderr=stderr,
exit_code=process.returncode,
execution_time=execution_time,
metadata={
"instance_id": instance_id,
"language": normalized_lang,
"script_path": str(script_path),
"status": "ok" if process.returncode == 0 else "error",
"timeout": exec_timeout,
"artifacts": self._collect_artifacts(instance_dir / "artifacts"),
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
},
)
def destroy_instance(self, instance_id: str) -> bool:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
instance_dir = self._instances.pop(instance_id)
shutil.rmtree(instance_dir)
return True
def health_check(self) -> bool:
return self._initialized and self.work_dir.exists() and os.access(self.work_dir, os.W_OK)
def get_supported_languages(self) -> List[str]:
return ["python", "javascript", "nodejs"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
return {
"python_bin": {"type": "string", "required": False, "default": "python3"},
"node_bin": {"type": "string", "required": False, "default": "node"},
"work_dir": {"type": "string", "required": False, "default": "/tmp/ragflow-codeexec"},
"timeout": {"type": "integer", "required": False, "default": 30},
"max_memory_mb": {"type": "integer", "required": False, "default": 512},
"max_output_bytes": {"type": "integer", "required": False, "default": 1048576},
"max_artifacts": {"type": "integer", "required": False, "default": 20},
"max_artifact_bytes": {"type": "integer", "required": False, "default": 10485760},
}
def _validate_limits(self) -> None:
if self.timeout <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_TIMEOUT must be greater than 0.")
if self.max_memory_mb <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_MEMORY_MB must be greater than 0.")
if self.max_output_bytes <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_OUTPUT_BYTES must be greater than 0.")
if self.max_artifacts < 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_ARTIFACTS must be greater than or equal to 0.")
if self.max_artifact_bytes <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_ARTIFACT_BYTES must be greater than 0.")
def _prepare_script(self, instance_dir: Path, language: str, code: str, args_json: str) -> tuple[list[str], Path]:
if language == "python":
script_path = instance_dir / "main.py"
script_path.write_text(build_python_wrapper(code, args_json), encoding="utf-8")
return [self.python_bin, str(script_path)], script_path
if language in {"javascript", "nodejs"}:
script_path = instance_dir / "main.js"
script_path.write_text(build_javascript_wrapper(code, args_json), encoding="utf-8")
return [self.node_bin, str(script_path)], script_path
raise RuntimeError(f"Unsupported language for local provider: {language}")
@staticmethod
def _resolve_config_value(config: Dict[str, Any], key: str, env_name: str, default: Any) -> Any:
value = config.get(key)
if value is not None:
return value
return os.environ.get(env_name, default)
def _build_child_env(self, instance_dir: Path) -> dict[str, str]:
env = {
"HOME": str(instance_dir),
"MPLBACKEND": "Agg",
"PATH": os.environ.get("PATH", ""),
"PYTHONUNBUFFERED": "1",
"TMPDIR": str(instance_dir),
}
for name in LOCAL_PYTHON_THREAD_ENV_VARS:
value = os.environ.get(name)
if value is not None:
env[name] = value
return env
def _limit_child_process(self) -> None:
import resource
self._set_resource_limit(resource.RLIMIT_CPU, self.timeout + 1)
self._set_resource_limit(resource.RLIMIT_AS, self.max_memory_mb * 1024 * 1024)
self._set_resource_limit(resource.RLIMIT_FSIZE, self.max_artifact_bytes)
self._set_resource_limit(resource.RLIMIT_NOFILE, 64)
@staticmethod
def _set_resource_limit(kind: int, value: int) -> None:
import resource
_, hard = resource.getrlimit(kind)
limit = value if hard == resource.RLIM_INFINITY else min(value, hard)
resource.setrlimit(kind, (limit, limit))
def _validate_output_size(self, stdout: str, stderr: str) -> None:
output_size = len((stdout or "").encode("utf-8")) + len((stderr or "").encode("utf-8"))
if output_size > self.max_output_bytes:
raise RuntimeError(f"Local execution output exceeded {self.max_output_bytes} bytes.")
def _collect_artifacts(self, artifacts_dir: Path) -> list[dict[str, Any]]:
artifacts: list[dict[str, Any]] = []
for path in sorted(artifacts_dir.rglob("*")):
if path.is_symlink():
raise RuntimeError(f"Artifact symlinks are not allowed: {path.name}")
if path.is_dir():
continue
if not path.is_file():
raise RuntimeError(f"Unsupported artifact entry: {path.name}")
if len(artifacts) >= self.max_artifacts:
raise RuntimeError(f"Local execution produced more than {self.max_artifacts} artifacts.")
size = path.stat().st_size
if size > self.max_artifact_bytes:
raise RuntimeError(f"Artifact exceeds {self.max_artifact_bytes} bytes: {path.name}")
ext = path.suffix.lower()
if ext not in ALLOWED_ARTIFACT_EXTENSIONS:
raise RuntimeError(f"Unsupported artifact type: {path.name}")
artifacts.append(
{
"name": path.relative_to(artifacts_dir).as_posix(),
"content_b64": base64.b64encode(path.read_bytes()).decode("ascii"),
"mime_type": mimetypes.guess_type(path.name)[0] or "application/octet-stream",
"size": size,
}
)
return artifacts
@staticmethod
def _normalize_language(language: str) -> str:
lang_lower = (language or "python").lower()
if lang_lower in {"python", "python3"}:
return "python"
if lang_lower in {"javascript", "nodejs"}:
return "nodejs"
return lang_lower

View File

@@ -0,0 +1,78 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Provider manager for sandbox providers.
Since sandbox configuration is global (system-level), we only use one
active provider at a time. This manager is a thin wrapper that holds a reference
to the currently active provider.
"""
from typing import Optional
from .base import SandboxProvider
class ProviderManager:
"""
Manages the currently active sandbox provider.
With global configuration, there's only one active provider at a time.
This manager simply holds a reference to that provider.
"""
def __init__(self):
"""Initialize an empty provider manager."""
self.current_provider: Optional[SandboxProvider] = None
self.current_provider_name: Optional[str] = None
def set_provider(self, name: str, provider: SandboxProvider):
"""
Set the active provider.
Args:
name: Provider identifier (e.g., "self_managed", "e2b")
provider: Provider instance
"""
self.current_provider = provider
self.current_provider_name = name
def get_provider(self) -> Optional[SandboxProvider]:
"""
Get the active provider.
Returns:
Currently active SandboxProvider instance, or None if not set
"""
return self.current_provider
def get_provider_name(self) -> Optional[str]:
"""
Get the active provider name.
Returns:
Provider name (e.g., "self_managed"), or None if not set
"""
return self.current_provider_name
def is_configured(self) -> bool:
"""
Check if a provider is configured.
Returns:
True if a provider is set, False otherwise
"""
return self.current_provider is not None

View File

@@ -0,0 +1,364 @@
#
# Copyright 2025 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
"""
Self-managed sandbox provider implementation.
This provider wraps the existing executor_manager HTTP API which manages
a pool of Docker containers with gVisor for secure code execution.
"""
import base64
import time
import uuid
from typing import Dict, Any, List, Optional
import requests
from .base import SandboxProvider, SandboxInstance, ExecutionResult
class SelfManagedProvider(SandboxProvider):
"""
Self-managed sandbox provider using Daytona/Docker.
This provider communicates with the executor_manager HTTP API
which manages a pool of containers for code execution.
"""
def __init__(self):
self.endpoint: str = "http://localhost:9385"
self.timeout: int = 30
self.max_retries: int = 3
self.pool_size: int = 10
self._initialized: bool = False
def initialize(self, config: Dict[str, Any]) -> bool:
"""
Initialize the provider with configuration.
Args:
config: Configuration dictionary with keys:
- endpoint: HTTP endpoint (default: "http://localhost:9385")
- timeout: Request timeout in seconds (default: 30)
- max_retries: Maximum retry attempts (default: 3)
- pool_size: Container pool size for info (default: 10)
Returns:
True if initialization successful, False otherwise
"""
self.endpoint = config.get("endpoint", "http://localhost:9385")
self.timeout = config.get("timeout", 30)
self.max_retries = config.get("max_retries", 3)
self.pool_size = config.get("pool_size", 10)
# Validate endpoint is accessible
if not self.health_check():
# Try to fall back to SANDBOX_HOST from settings if we are using localhost
if "localhost" in self.endpoint or "127.0.0.1" in self.endpoint:
try:
from common import settings
if settings.SANDBOX_HOST and settings.SANDBOX_HOST not in self.endpoint:
original_endpoint = self.endpoint
self.endpoint = f"http://{settings.SANDBOX_HOST}:9385"
if self.health_check():
import logging
logging.warning(f"Sandbox self_managed: Connected using settings.SANDBOX_HOST fallback: {self.endpoint} (original: {original_endpoint})")
self._initialized = True
return True
else:
self.endpoint = original_endpoint # Restore if fallback also fails
except ImportError:
pass
return False
self._initialized = True
return True
def create_instance(self, template: str = "python") -> SandboxInstance:
"""
Create a new sandbox instance.
Note: For self-managed provider, instances are managed internally
by the executor_manager's container pool. This method returns
a logical instance handle.
Args:
template: Programming language (python, nodejs)
Returns:
SandboxInstance object
Raises:
RuntimeError: If instance creation fails
"""
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# Normalize language
language = self._normalize_language(template)
# The executor_manager manages instances internally via container pool
# We create a logical instance ID for tracking
instance_id = str(uuid.uuid4())
return SandboxInstance(
instance_id=instance_id,
provider="self_managed",
status="running",
metadata={
"language": language,
"endpoint": self.endpoint,
"pool_size": self.pool_size,
}
)
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10,
arguments: Optional[Dict[str, Any]] = None
) -> ExecutionResult:
"""
Execute code in the sandbox.
Args:
instance_id: ID of the sandbox instance (not used for self-managed)
code: Source code to execute
language: Programming language (python, nodejs, javascript)
timeout: Maximum execution time in seconds
arguments: Optional arguments dict to pass to main() function
Returns:
ExecutionResult containing stdout, stderr, exit_code, and metadata
Raises:
RuntimeError: If execution fails
TimeoutError: If execution exceeds timeout
"""
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
# Normalize language
normalized_lang = self._normalize_language(language)
# Prepare request
code_b64 = base64.b64encode(code.encode("utf-8")).decode("utf-8")
payload = {
"code_b64": code_b64,
"language": normalized_lang,
"arguments": arguments or {}
}
url = f"{self.endpoint}/run"
exec_timeout = timeout or self.timeout
start_time = time.time()
try:
response = requests.post(
url,
json=payload,
timeout=exec_timeout,
headers={"Content-Type": "application/json"}
)
execution_time = time.time() - start_time
if response.status_code != 200:
raise RuntimeError(
f"HTTP {response.status_code}: {response.text}"
)
result = response.json()
structured_result = result.get("result") or {}
return ExecutionResult(
stdout=result.get("stdout", ""),
stderr=result.get("stderr", ""),
exit_code=result.get("exit_code", 0),
execution_time=execution_time,
metadata={
"status": result.get("status"),
"time_used_ms": result.get("time_used_ms"),
"memory_used_kb": result.get("memory_used_kb"),
"detail": result.get("detail"),
"instance_id": instance_id,
"artifacts": result.get("artifacts", []),
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
}
)
except requests.Timeout:
execution_time = time.time() - start_time
raise TimeoutError(
f"Execution timed out after {exec_timeout} seconds"
)
except requests.RequestException as e:
raise RuntimeError(f"HTTP request failed: {str(e)}")
def destroy_instance(self, instance_id: str) -> bool:
"""
Destroy a sandbox instance.
Note: For self-managed provider, instances are returned to the
internal pool automatically by executor_manager after execution.
This is a no-op for tracking purposes.
Args:
instance_id: ID of the instance to destroy
Returns:
True (always succeeds for self-managed)
"""
# The executor_manager manages container lifecycle internally
# Container is returned to pool after execution
return True
def health_check(self) -> bool:
"""
Check if the provider is healthy and accessible.
Returns:
True if provider is healthy, False otherwise
"""
try:
url = f"{self.endpoint}/healthz"
response = requests.get(url, timeout=5)
return response.status_code == 200
except Exception:
return False
def get_supported_languages(self) -> List[str]:
"""
Get list of supported programming languages.
Returns:
List of language identifiers
"""
return ["python", "nodejs", "javascript"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
"""
Return configuration schema for self-managed provider.
Returns:
Dictionary mapping field names to their schema definitions
"""
return {
"endpoint": {
"type": "string",
"required": True,
"label": "Executor Manager Endpoint",
"placeholder": "http://localhost:9385",
"default": "http://localhost:9385",
"description": "HTTP endpoint of the executor_manager service"
},
"timeout": {
"type": "integer",
"required": False,
"label": "Request Timeout (seconds)",
"default": 30,
"min": 5,
"max": 300,
"description": "HTTP request timeout for code execution"
},
"max_retries": {
"type": "integer",
"required": False,
"label": "Max Retries",
"default": 3,
"min": 0,
"max": 10,
"description": "Maximum number of retry attempts for failed requests"
},
"pool_size": {
"type": "integer",
"required": False,
"label": "Container Pool Size",
"default": 10,
"min": 1,
"max": 100,
"description": "Size of the container pool (configured in executor_manager)"
}
}
def _normalize_language(self, language: str) -> str:
"""
Normalize language identifier to executor_manager format.
Args:
language: Language identifier (python, python3, nodejs, javascript)
Returns:
Normalized language identifier
"""
if not language:
return "python"
lang_lower = language.lower()
if lang_lower in ("python", "python3"):
return "python"
elif lang_lower in ("javascript", "nodejs"):
return "nodejs"
else:
return language
def validate_config(self, config: dict) -> tuple[bool, Optional[str]]:
"""
Validate self-managed provider configuration.
Performs custom validation beyond the basic schema validation,
such as checking URL format.
Args:
config: Configuration dictionary to validate
Returns:
Tuple of (is_valid, error_message)
"""
# Validate endpoint URL format
endpoint = config.get("endpoint", "")
if endpoint:
# Check if it's a valid HTTP/HTTPS URL or localhost
import re
url_pattern = r'^(https?://|http://localhost|http://[\d\.]+:[a-z]+:[/]|http://[\w\.]+:)'
if not re.match(url_pattern, endpoint):
return False, f"Invalid endpoint format: {endpoint}. Must start with http:// or https://"
# Validate pool_size is positive
pool_size = config.get("pool_size", 10)
if isinstance(pool_size, int) and pool_size <= 0:
return False, "Pool size must be greater than 0"
# Validate timeout is reasonable
timeout = config.get("timeout", 30)
if isinstance(timeout, int) and (timeout < 1 or timeout > 600):
return False, "Timeout must be between 1 and 600 seconds"
# Validate max_retries
max_retries = config.get("max_retries", 3)
if isinstance(max_retries, int) and (max_retries < 0 or max_retries > 10):
return False, "Max retries must be between 0 and 10"
return True, None

View File

@@ -0,0 +1,28 @@
[project]
name = "gvisor-sandbox"
version = "0.1.0"
description = "Add your description here"
readme = "README.md"
requires-python = ">=3.12,<3.15"
dependencies = [
"fastapi>=0.115.12",
"httpx>=0.28.1",
"pydantic>=2.11.4",
"requests>=2.32.4",
"slowapi>=0.1.9",
"uvicorn>=0.34.2",
]
[[tool.uv.index]]
url = "https://pypi.tuna.tsinghua.edu.cn/simple"
[dependency-groups]
dev = [
"basedpyright>=1.29.1",
]
[tool.ruff]
line-length = 200
[tool.ruff.lint]
extend-select = ["C4", "SIM", "TCH"]

View File

@@ -0,0 +1,85 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import base64
import json
from typing import Any
RESULT_MARKER_PREFIX = "__RAGFLOW_RESULT__:"
def build_python_wrapper(code: str, args_json: str) -> str:
return f'''{code}
if __name__ == "__main__":
import base64
import json
result = main(**{args_json})
payload = json.dumps({{"present": True, "value": result, "type": "json"}}, ensure_ascii=False, separators=(",", ":"))
print("{RESULT_MARKER_PREFIX}" + base64.b64encode(payload.encode("utf-8")).decode("ascii"))
'''
def build_javascript_wrapper(code: str, args_json: str) -> str:
return f'''{code}
const __ragflowArgs = {args_json};
(async () => {{
const __ragflowMain = typeof main !== 'undefined' ? main : module.exports && module.exports.main;
if (typeof __ragflowMain !== 'function') {{
throw new Error('main() must be defined or exported.');
}}
const output = await Promise.resolve(__ragflowMain(__ragflowArgs));
if (typeof output === 'undefined') {{
throw new Error('main() must return a value. Use null for an empty result.');
}}
const payload = JSON.stringify({{ present: true, value: output, type: 'json' }});
if (typeof payload === 'undefined') {{
throw new Error('main() returned a non-JSON-serializable value.');
}}
console.log('{RESULT_MARKER_PREFIX}' + Buffer.from(payload, 'utf8').toString('base64'));
}})();
'''
def extract_structured_result(stdout: str) -> tuple[str, dict[str, Any]]:
if not stdout:
return "", {}
cleaned_lines: list[str] = []
structured_result: dict[str, Any] = {}
for line in str(stdout).splitlines():
if line.startswith(RESULT_MARKER_PREFIX):
payload_b64 = line[len(RESULT_MARKER_PREFIX) :].strip()
if not payload_b64:
cleaned_lines.append(line)
continue
try:
payload = base64.b64decode(payload_b64, validate=True).decode("utf-8")
structured_result = json.loads(payload)
except Exception:
cleaned_lines.append(line)
continue
cleaned_lines.append(line)
cleaned_stdout = "\n".join(cleaned_lines)
if stdout.endswith("\n") and cleaned_stdout and not cleaned_stdout.endswith("\n"):
cleaned_stdout += "\n"
return cleaned_stdout, structured_result

View File

@@ -0,0 +1,23 @@
FROM node:24.13-bookworm-slim
ARG NEED_MIRROR=1
RUN if [ "$NEED_MIRROR" = 1 ]; then \
npm config set registry https://registry.npmmirror.com; \
else \
npm config set registry https://registry.npmjs.org; \
fi
# RUN grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.ustc.edu.cn|g' && \
# apt-get update && \
# apt-get install -y curl gcc make
WORKDIR /app
COPY package.json package-lock.json .
RUN npm install
CMD ["sleep", "infinity"]

View File

@@ -0,0 +1,295 @@
{
"name": "nodejs",
"version": "1.0.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "nodejs",
"version": "1.0.0",
"license": "ISC",
"dependencies": {
"axios": "^1.9.0"
}
},
"node_modules/asynckit": {
"version": "0.4.0",
"resolved": "https://registry.npmjs.org/asynckit/-/asynckit-0.4.0.tgz",
"integrity": "sha512-Oei9OH4tRh0YqU3GxhX79dM/mwVgvbZJaSNaRk+bshkj0S5cfHcgYakreBjrHwatXKbz+IoIdYLxrKim2MjW0Q==",
"license": "MIT"
},
"node_modules/axios": {
"version": "1.13.6",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.13.6.tgz",
"integrity": "sha512-ChTCHMouEe2kn713WHbQGcuYrr6fXTBiu460OTwWrWob16g1bXn4vtz07Ope7ewMozJAnEquLk5lWQWtBig9DQ==",
"license": "MIT",
"dependencies": {
"follow-redirects": "^1.15.11",
"form-data": "^4.0.5",
"proxy-from-env": "^1.1.0"
}
},
"node_modules/call-bind-apply-helpers": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/call-bind-apply-helpers/-/call-bind-apply-helpers-1.0.2.tgz",
"integrity": "sha512-Sp1ablJ0ivDkSzjcaJdxEunN5/XvksFJ2sMBFfq6x0ryhQV/2b/KwFe21cMpmHtPOSij8K99/wSfoEuTObmuMQ==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/combined-stream": {
"version": "1.0.8",
"resolved": "https://registry.npmjs.org/combined-stream/-/combined-stream-1.0.8.tgz",
"integrity": "sha512-FQN4MRfuJeHf7cBbBMJFXhKSDq+2kAArBlmRBvcvFE5BB1HZKXtSFASDhdlz9zOYwxh8lDdnvmMOe/+5cdoEdg==",
"license": "MIT",
"dependencies": {
"delayed-stream": "~1.0.0"
},
"engines": {
"node": ">= 0.8"
}
},
"node_modules/delayed-stream": {
"version": "1.0.0",
"resolved": "https://registry.npmjs.org/delayed-stream/-/delayed-stream-1.0.0.tgz",
"integrity": "sha512-ZySD7Nf91aLB0RxL4KGrKHBXl7Eds1DAmEdcoVawXnLD7SDhpNgtuII2aAkg7a7QS41jxPSZ17p4VdGnMHk3MQ==",
"license": "MIT",
"engines": {
"node": ">=0.4.0"
}
},
"node_modules/dunder-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/dunder-proto/-/dunder-proto-1.0.1.tgz",
"integrity": "sha512-KIN/nDJBQRcXw0MLVhZE9iQHmG68qAVIBg9CqmUYjmQIhgij9U5MFvrqkUL5FbtyyzZuOeOt0zdeRe4UY7ct+A==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.1",
"es-errors": "^1.3.0",
"gopd": "^1.2.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-define-property": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/es-define-property/-/es-define-property-1.0.1.tgz",
"integrity": "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-errors": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/es-errors/-/es-errors-1.3.0.tgz",
"integrity": "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-object-atoms": {
"version": "1.1.1",
"resolved": "https://registry.npmjs.org/es-object-atoms/-/es-object-atoms-1.1.1.tgz",
"integrity": "sha512-FGgH2h8zKNim9ljj7dankFPcICIK9Cp5bm+c2gQSYePhpaG5+esrLODihIorn+Pe6FGJzWhXQotPv73jTaldXA==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/es-set-tostringtag": {
"version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-set-tostringtag/-/es-set-tostringtag-2.1.0.tgz",
"integrity": "sha512-j6vWzfrGVfyXxge+O0x5sh6cvxAog0a/4Rdd2K36zCMV5eJ+/+tOAngRO8cODMNWbVRdVlmGZQL2YS3yR8bIUA==",
"license": "MIT",
"dependencies": {
"es-errors": "^1.3.0",
"get-intrinsic": "^1.2.6",
"has-tostringtag": "^1.0.2",
"hasown": "^2.0.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/follow-redirects": {
"version": "1.15.11",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz",
"integrity": "sha512-deG2P0JfjrTxl50XGCDyfI97ZGVCxIpfKYmfyrQ54n5FO/0gfIES8C/Psl6kWVDolizcaaxZJnTS0QSMxvnsBQ==",
"funding": [
{
"type": "individual",
"url": "https://github.com/sponsors/RubenVerborgh"
}
],
"license": "MIT",
"engines": {
"node": ">=4.0"
},
"peerDependenciesMeta": {
"debug": {
"optional": true
}
}
},
"node_modules/form-data": {
"version": "4.0.5",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz",
"integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==",
"license": "MIT",
"dependencies": {
"asynckit": "^0.4.0",
"combined-stream": "^1.0.8",
"es-set-tostringtag": "^2.1.0",
"hasown": "^2.0.2",
"mime-types": "^2.1.12"
},
"engines": {
"node": ">= 6"
}
},
"node_modules/function-bind": {
"version": "1.1.2",
"resolved": "https://registry.npmjs.org/function-bind/-/function-bind-1.1.2.tgz",
"integrity": "sha512-7XHNxH7qX9xG5mIwxkhumTox/MIRNcOgDrxWsMt2pAr23WHp6MrRlN7FBSFpCpr+oVO0F744iUgR82nJMfG2SA==",
"license": "MIT",
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-intrinsic": {
"version": "1.3.0",
"resolved": "https://registry.npmjs.org/get-intrinsic/-/get-intrinsic-1.3.0.tgz",
"integrity": "sha512-9fSjSaos/fRIVIp+xSJlE6lfwhES7LNtKaCBIamHsjr2na1BiABJPo0mOjjz8GJDURarmCPGqaiVg5mfjb98CQ==",
"license": "MIT",
"dependencies": {
"call-bind-apply-helpers": "^1.0.2",
"es-define-property": "^1.0.1",
"es-errors": "^1.3.0",
"es-object-atoms": "^1.1.1",
"function-bind": "^1.1.2",
"get-proto": "^1.0.1",
"gopd": "^1.2.0",
"has-symbols": "^1.1.0",
"hasown": "^2.0.2",
"math-intrinsics": "^1.1.0"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/get-proto": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/get-proto/-/get-proto-1.0.1.tgz",
"integrity": "sha512-sTSfBjoXBp89JvIKIefqw7U2CCebsc74kiY6awiGogKtoSGbgjYE/G/+l9sF3MWFPNc9IcoOC4ODfKHfxFmp0g==",
"license": "MIT",
"dependencies": {
"dunder-proto": "^1.0.1",
"es-object-atoms": "^1.0.0"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/gopd": {
"version": "1.2.0",
"resolved": "https://registry.npmjs.org/gopd/-/gopd-1.2.0.tgz",
"integrity": "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-symbols": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/has-symbols/-/has-symbols-1.1.0.tgz",
"integrity": "sha512-1cDNdwJ2Jaohmb3sg4OmKaMBwuC48sYni5HUw2DvsC8LjGTLK9h+eb1X6RyuOHe4hT0ULCW68iomhjUoKUqlPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/has-tostringtag": {
"version": "1.0.2",
"resolved": "https://registry.npmjs.org/has-tostringtag/-/has-tostringtag-1.0.2.tgz",
"integrity": "sha512-NqADB8VjPFLM2V0VvHUewwwsw0ZWBaIdgo+ieHtK3hasLz4qeCRjYcqfB6AQrBggRKppKF8L52/VqdVsO47Dlw==",
"license": "MIT",
"dependencies": {
"has-symbols": "^1.0.3"
},
"engines": {
"node": ">= 0.4"
},
"funding": {
"url": "https://github.com/sponsors/ljharb"
}
},
"node_modules/hasown": {
"version": "2.0.2",
"resolved": "https://registry.npmjs.org/hasown/-/hasown-2.0.2.tgz",
"integrity": "sha512-0hJU9SCPvmMzIBdZFqNPXWa6dqh7WdH0cII9y+CyS8rG3nL48Bclra9HmKhVVUHyPWNH5Y7xDwAB7bfgSjkUMQ==",
"license": "MIT",
"dependencies": {
"function-bind": "^1.1.2"
},
"engines": {
"node": ">= 0.4"
}
},
"node_modules/math-intrinsics": {
"version": "1.1.0",
"resolved": "https://registry.npmjs.org/math-intrinsics/-/math-intrinsics-1.1.0.tgz",
"integrity": "sha512-/IXtbwEk5HTPyEwyKX6hGkYXxM9nbj64B+ilVJnC/R6B0pH5G4V3b0pVbL7DBj4tkhBAppbQUlf6F6Xl9LHu1g==",
"license": "MIT",
"engines": {
"node": ">= 0.4"
}
},
"node_modules/mime-db": {
"version": "1.52.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.52.0.tgz",
"integrity": "sha512-sPU4uV7dYlvtWJxwwxHD0PuihVNiE7TyAbQ5SWxDCB9mUYvOgroQOwYQQOKPJ8CIbE+1ETVlOoK1UC2nU3gYvg==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
"version": "2.1.35",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-2.1.35.tgz",
"integrity": "sha512-ZDY+bPm5zTTF+YpCrAU9nK0UgICYPT0QtT1NZWFv4s++TNkcgVaT0g6+4R2uI4MjQjzysHB1zxuWL50hzaeXiw==",
"license": "MIT",
"dependencies": {
"mime-db": "1.52.0"
},
"engines": {
"node": ">= 0.6"
}
},
"node_modules/proxy-from-env": {
"version": "1.1.0",
"resolved": "https://registry.npmmirror.com/proxy-from-env/-/proxy-from-env-1.1.0.tgz",
"integrity": "sha512-D+zkORCbA9f1tdWRK0RaCR3GPv50cMxcrz4X8k5LTSUD1Dkw47mKJEZQNunItRTkWwgtaUSo1RVFRIG9ZXiFYg==",
"license": "MIT"
}
}
}

Some files were not shown because too many files have changed in this diff Show More