Compare commits

...

1133 Commits

Author SHA1 Message Date
writinwaters
8f0632c8d9 Docs: v0.25.6 release notes draft (#15255)
### What problem does this PR solve?

v0.25.6 release notes draft updated.

### Type of change

- [x] Documentation Update
2026-05-26 20:56:36 +08:00
oktofeesh
5ae41dc1eb fix(go-models): route hosted OCR providers through drivers (#15233)
## Summary
- route hosted MinerU.Net and PaddleOCR.Net provider names to their
existing Go drivers
- add regression coverage for loading the hosted OCR provider configs
through ProviderManager

## What changed
- Added canonical provider-name aliases for the hosted OCR provider
display names.
- Covered both bundled configs with a focused provider-manager test.

## Why
The hosted provider configs use display names with `.Net`, while model
factory dispatch lowercases the provider name. Without aliases, those
configs fall through to `DummyModel` instead of using the existing
MinerU and PaddleOCR drivers.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 20:40:40 +08:00
Wang Qi
303221c1f4 Fix: show tag list for chunk (#15251) 2026-05-26 20:24:22 +08:00
oktofeesh
22a3b8cdf9 feat(go-models): list LongCat models (#15241)
## Summary
- Add LongCat model-list support through the documented
OpenAI-compatible models endpoint.

## What changed
- Add the LongCat `models` URL suffix for `/openai/v1/models`.
- Implement `ListModels` for the LongCat Go driver.
- Delegate `CheckConnection` to the lightweight model-list request.
- Add focused regression coverage for successful, malformed, oversized,
and missing-key responses.

## Why
LongCat documents a models endpoint under the OpenAI-compatible API
surface, but the Go driver still returned `no such method` for model
listing and connection checks.

## Validation
- `go test ./internal/entity/models -run TestLongCat -count=1`
- `go test -race ./internal/entity/models -run TestLongCat -count=1`
- `go test ./internal/entity -count=1`
- `git diff --check`

## Notes
- Related to the broader Go model provider tracking in #14736, but this
PR only handles LongCat model listing.
- `go test ./internal/entity/models -count=1` is currently blocked by an
unrelated Astraflow test panic outside this LongCat change.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 19:58:53 +08:00
oktofeesh
557024e7d4 fix(go-models): add xAI model listing suffix (#15236)
## Summary
- add the xAI `models` URL suffix used by the existing Go `ListModels`
implementation
- return a clear error when the xAI models suffix is missing
- add focused xAI model-listing and connection-check regression tests

## What changed
- Added `url_suffix.models` to `conf/models/xai.json`.
- Normalized the configured models suffix before building the request
URL.
- Covered config loading, successful model listing, upstream errors,
API-key validation, missing suffix handling, and `CheckConnection`
delegation.

## Why
`XAIModel.ListModels` already builds requests from `URLSuffix.Models`,
and `CheckConnection` delegates to that method. The bundled xAI config
did not define that suffix, which left the model-listing path unable to
call the provider `/models` endpoint from the existing provider config.

## Validation
- `go test ./internal/entity/models -run TestXAI -count=1`
- `go test ./internal/entity -count=1`
- `git diff HEAD~1..HEAD --check`

## Notes
- `go test ./internal/entity/models -count=1` currently fails in
unchanged Astraflow coverage: `TestAstraflowEmbedReturnsNoSuchMethod`
panics before reaching any xAI assertions.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 19:58:20 +08:00
writinwaters
af48a22ff4 Docs: Initial draft for v0.25.6 release notes. (#15250)
### What problem does this PR solve?

Initial draft: v0.25.6 release notes.

### Type of change

- [x] Documentation Update
2026-05-26 19:46:40 +08:00
Liu An
0639dba89a Docs: Update version references to v0.25.6 in READMEs and docs (#15248)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.5 to v0.25.6
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-26 19:45:43 +08:00
Haruko386
3619ceca01 Go: implement provider: OrcaRouter (#15235)
### What problem does this PR solve?

implement provider `OrcaRouter`
**The following functionalities are now supported:**

**Cohere:**
- [x] Chat / Think Chat / Stream Chat / Stream Think Chat
- [x] Model listing
- [x] TTS
- [ ] Balance


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 18:20:33 +08:00
dripsmvcp
a48bcf814d Go: implement provider: ModelScope (#15041)
Closes #15040.

ModelScope was listed unchecked in the Go-rewrite tracker #14736 and
already had an llm_factories.json entry (tags: LLM) but no Go driver, so
the new Go API server could not route ModelScope instances. The Python
side has supported it through the OpenAI-compatible base at
rag/llm/chat_model.py:618 (ModelScopeChat), which requires a
user-supplied base URL and appends /v1.

This adds:
- internal/entity/models/modelscope.go: self-hosted OpenAI-compatible
driver with chat (sync + SSE stream with idle-timeout cancellation),
list_models, and check_connection. Auth header is optional, matching the
xinference pattern, so deployments without auth and auth-enabled
deployments both work. Base URL is normalized so users can configure
either the root endpoint or the /v1 endpoint.
- internal/entity/models/modelscope_test.go: 12 tests covering name, URL
normalization, factory routing, chat happy path / auth header /
reasoning_content extraction, stream happy path / stream=false rejection
/ idle cancellation, list_models + check_connection, missing-base-URL
clear error, and the no-such-method sentinels.
- conf/models/modelscope.json: shipped config (class: "local",
url_suffix v1/chat/completions and v1/models).
- internal/entity/models/factory.go: case "modelscope" →
ModelScopeModel.
- internal/service/llm.go: ModelScope added to the selfDeployed map
alongside Ollama, Xinference, LocalAI, LM-Studio, GPUStack — the Python
side requires user-supplied URL with no default, so the Go side
classifies it the same way.

Follow-on issues will add Embed and Rerank, in line with how Novita,
NVIDIA, TogetherAI, and other providers landed method-by-method.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 18:18:46 +08:00
Hz_
84add43208 Add HuaweiCloud model provider (#15237)
### What problem does this PR solve?

  This PR adds HuaweiCloud provider integration in RAGFlow.

  Supported capabilities:

  - [x] Chat / Think Chat / Stream Chat / Stream Think Chat
  - [x] Embedding
  - [x] Rerank
  - [x] Model listing
  - [x] Provider connection checking

  Verified examples from the CLI:

  ```
  check instance 'test' from 'HuaweiCloud';

  chat with 'deepseek-v4-flash@test@HuaweiCloud' message 'hello';

  think chat with 'deepseek-v4-flash@test@HuaweiCloud' message 'hello';

  stream chat with 'deepseek-v4-flash@test@HuaweiCloud' message 'hello';

stream think chat with 'deepseek-v4-flash@test@HuaweiCloud' message
'hello';

embed text 'what is rag' 'who are you' with 'bge-m3@test@HuaweiCloud'
dimension 1024;

rerank query 'what is rag' document 'rag is retrieval augmented
generation' 'rag need llm' 'famous rag
project includes ragflow' with 'bge-reranker-v2-m3@test@HuaweiCloud' top
3;

  list supported models from 'HuaweiCloud' 'test';

  LIST MODELS FROM 'HuaweiCloud' 'test';
```
  ### Type of change

  - [x] New Feature
  - [x] Provider integration
2026-05-26 17:13:15 +08:00
ghost
a7d25391dc fix(tokenhub): wire Go driver and harden requests (#15224)
## Summary
- Wire the Go TokenHub provider through the model factory.
- Harden TokenHub request handling for chat, streaming, embeddings, and
model listing.
- Add focused TokenHub unit coverage for factory wiring and provider
behavior.

## Notes
- Refs #14736.
- Follows up #15159.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 17:12:37 +08:00
Jake Armstrong
0fb85a66bc feat(go-models): add AWS Bedrock provider driver (#15166)
## Summary

Closes #15165.

Implements the AWS Bedrock model provider for the Go API server, tracked
under #14736. Adds Converse + Converse-Stream chat and foundation-model
listing, with SigV4 signing over a hand-rolled `net/http` path that
matches the established pattern in `internal/entity/models/` (no new
direct `go.mod` deps).

## Linked tracker

Tracked under #14736 (Implement model providers of RAGFlow API server in
Go). Closes #15165.
2026-05-26 17:10:06 +08:00
Idriss Sbaaoui
036ed5b236 Feat: add new tests and tescases for restful api suite (#15230)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-26 13:24:22 +08:00
chanx
bce11527c3 Fix: Fixed metadata issue (#15226)
### What problem does this PR solve?

Fix: Fixed metadata issue

- The dataset's built-in metadata is now active, but it appears to be
disabled in the individual file configuration.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-26 13:16:15 +08:00
Wang Qi
619b971785 Fix: empty file with better message (#15232)
Fix: empty file with better message
2026-05-26 12:28:53 +08:00
天海蒼灆
0d2a17254c fix(api): allow canvas_type in agent create and update APIs (#15201)
### What problem does this PR solve?

Creating or updating an agent via `POST /api/v1/agents` and `PUT
/api/v1/agents/{agent_id}` did not persist `canvas_type` because the
handler `req` dict never assigned the field before
`UserCanvasService.save` / `update_by_id`.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 11:31:46 +08:00
glorydavid03023
3dbd874a79 Go: implement Rerank in DeepInfra driver (#15185)
### What problem does this PR solve?

The Go DeepInfra driver returned a stub error for `Rerank()` even though
DeepInfra serves reranker models at `POST /v1/inference/{model}` with
`query`, `documents`, and a `scores[]` response.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-26 10:52:09 +08:00
sxxtony
67f7d87dff Go: implement provider: FuturMix (#15013)
### What problem does this PR solve?

Add a Go driver for **FuturMix** (https://futurmix.ai/docs), one of the
unchecked providers on the umbrella tracking issue #14736. FuturMix is
documented as an "OpenAI-compatible API" aggregator over Claude / GPT /
Gemini / DeepSeek (~22 models per their `/models` page).

Until this PR, a tenant who configured `futurmix` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

---------

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 10:51:29 +08:00
Renzo
806414df43 Go: validate Baidu OCR inputs (#15168)
### What problem does this PR solve?

Closes #15167.

The Baidu Go provider advertises OCR support through
`paddleocr-vl-0.9b`, but `BaiduModel.OCRFile` dereferenced required
inputs before validating them. Calling OCR with a missing API config,
API key, or model name could panic instead of returning a normal error.

This PR adds explicit input validation for those required values.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-26 10:51:05 +08:00
Jake Armstrong
b961810e79 Go: implement OCR in ZhipuAI driver (#15143)
### What problem does this PR solve?

Closes #15142.

ZhipuAI lists `glm-ocr` as an OCR model, but the Go driver still
returned `no such method` from `OCRFile`. This wires the advertised
model to Z.AI's documented `layout_parsing` endpoint and returns the
`md_results` Markdown output through the existing `OCRFileResponse.Text`
field.

This PR also adds focused tests for URL input, raw file-content base64
input, and validation errors.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Test

- [x] `go test -vet=off ./internal/entity/models -run
'TestZhipuAIOCRFile'`
2026-05-26 10:50:06 +08:00
Idriss Sbaaoui
c3b38d397f Feat: add new tests and tescases for restful api suite (#15223)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-26 10:08:45 +08:00
Jay Xu
54c3d23513 Fix [Bug]: Save parser configs in dataset configuration page is not working #15175 (#15177)
### What problem does this PR solve?

Fix [Bug]: Save parser configs in dataset configuration page is not
working #15175

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-26 10:04:43 +08:00
wdeveloper16
4b36801b53 fix: resolve asyncio correctness issues (fire-and-forget tasks, event loop nesting) (#14761)
## Summary

Fixes the confirmed asyncio anti-patterns from #14755. Only the three
verified bugs are addressed; patterns already correctly using
`asyncio.new_event_loop()` in a fresh thread are left untouched.

### Changes

**`api/apps/restful_apis/tenant_api.py` — fire-and-forget
`send_invite_email`**

`asyncio.create_task()` was called without storing the `Task` reference.
CPython's GC can collect an unfinished task, silently cancelling it and
swallowing exceptions. Fixed by storing the task in a module-level
`_background_tasks: set[Task]` with a `done_callback` to discard it on
completion — the standard Python idiom for safe background tasks.

**`api/apps/restful_apis/agent_api.py` — fire-and-forget
`background_run`**

Same root cause in the webhook "Immediately" execution path. Same fix
applied.

**`rag/llm/chat_model.py` (`LocalLLM._stream_response`) —
`asyncio.get_event_loop()` on running loop**

`asyncio.get_event_loop()` returns Quart's running event loop when
called from an async context.
Calling `loop.run_until_complete()` on it raises `RuntimeError`.
Replaced with `asyncio.new_event_loop()` so the generator
uses a dedicated fresh loop, closed in a `finally` block.

## What was NOT changed

- `llm_service._sync_from_async_stream` and
`evaluation_service._sync_from_async_gen`: both already correctly use
`asyncio.new_event_loop()` inside a fresh thread.
- `llm_service._run_coroutine_sync`: only caller is `rag/app/resume.py`
(sync context), so `thread.join()` is correct there.
- `requests` in agent tools: sync methods dispatched through thread
pools; httpx migration is a separate, larger refactor.

## Test plan

- [ ] Invite a team member and confirm the email is sent with no task
warnings in logs.
- [ ] Trigger a webhook agent in "Immediately" mode; confirm canvas
state is persisted after background run.
- [ ] Verify `LocalLLM` (Jina backend) chat and streaming work
end-to-end.

Closes #14755

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
2026-05-25 22:45:40 +08:00
balibabu
ed179ce684 Fix: The prompt variable for the agent operator disappears after input. (#15218)
### What problem does this PR solve?

Fix: The prompt variable for the agent operator disappears after input.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 20:36:51 +08:00
writinwaters
67e43e7df7 Docs: Minimum required Python version increased to 3.13. (#15219)
### What problem does this PR solve?

Minimum Python version increased to 3.13.

### Type of change


- [x] Documentation Update
2026-05-25 20:23:30 +08:00
qinling0210
af85aa9c7b Implement Elasticsearch functions in GO (#15160)
### What problem does this PR solve?

Implement Elasticsearch functions in GO (except for Search)

### Type of change

- [x] Refactoring
2026-05-25 19:15:07 +08:00
Idriss Sbaaoui
7d200d5bd7 Feat: add new tests and tescases for restful api suite (#15208)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-25 19:03:56 +08:00
balibabu
c7c75c0a87 Feat: Enable agent messages to display base64 images (#15212)
### What problem does this PR solve?

Feat: Enable agent messages to display base64 images

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-05-25 19:02:03 +08:00
Wang Qi
f4d36f7082 Fix #15170 cannot filter document status (#15216)
Fix #15170 cannot filter document status

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 18:58:37 +08:00
Haruko386
4783ce9951 fix(Go): rewrite chat, listmodels, embed for Ollama (#15213)
### What problem does this PR solve?

IDK how to implement **`Ollama`** on #14580 but it's totally wrong.
This is the rewrite version for **`Ollama`**

**Verified from CLI**
```
# Embed
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'nomic-embed-text:latest@test12@ollama' dimension 1024;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 768       | 0     |
| 768       | 1     |
+-----------+-------+

# Chat
RAGFlow(user)> think chat with 'qwen3:0.6b@test12@ollama' message 'who r u'
Thinking: Okay, the user asked, "Who r u?" I need to respond appropriately. First, I should acknowledge their question. Since I'm an AI, I don't have a physical form, but I can confirm that I'm a large language model. I should keep the response friendly and offer help. Let me make sure I'm not making up any information and that the response is natural. Also, I should check for any typos and ensure clarity. Alright, that should cover it.

Answer: I'm an AI language model, and I don't have a physical form. However, I can tell you that I'm designed to assist with questions and tasks. How can I help you today?
Time: 2.914285


RAGFlow(user)> stream think chat with 'qwen3:0.6b@test12@ollama' message 'who r u'
Thinking: , the user asked, "Who are you?" I need to respond appropriately. Since I'm an AI assistant, I should mention that I don't have a physical form or a mind. I should also clarify that I can help with various tasks like answering questions or providing information. It's important to keep the response friendly and informative while maintaining the correct tone.
Answer:  don't have a physical form or a mind, but I'm here to help with your questions or tasks! What can I do for you today?
Time: 1.740047

# LisyModels
RAGFlow(user)> list supported models from 'ollama' 'test12'
+-------------------------+
| model_name              |
+-------------------------+
| nomic-embed-text:latest |
| qwen3:0.6b              |
+-------------------------+
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-25 18:55:03 +08:00
balibabu
0f92353bd9 Fix: Replace the red highlight at the top of the PDF document with yellow. (#15203)
### What problem does this PR solve?

Fix: Replace the red highlight at the top of the PDF document with
yellow.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 17:21:36 +08:00
Wang Qi
4776bfa8a2 Fix: Correct the API path (#15204)
Follow on PR #15146 to reslove the backwad compatability issue.

1. /agents/<attachment_id>/download ->
/agents/attachments/<attachment_id>/download

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 17:11:24 +08:00
Jonathan Chang
9d1006e4ec fix: The output of the parser in the ingestion pipeline contains HTML tags (#14920)
## Summary
This change fixes ingestion quality issues where MinerU parser output
may contain HTML fragments (for example, table-related tags like `<tr>`,
`<td>`, `<br>`), which were previously passed directly into
chunking/tokenization and degraded chunk quality.

The fix adds a sanitization step in the MinerU parser path so parsed
sections are normalized to clean text before chunking.

## Change Type (select all)
- [x] Bug fix
- [x] Ingestion pipeline improvement
- [x] Parser/chunking quality fix

## Related Issue
- https://github.com/infiniflow/ragflow/issues/14831
2026-05-25 16:06:36 +08:00
Ahmad Intisar
e6068a7f7e Fix: table parser metadata (#15127)
### What problem does this PR solve?

This PR improves the table upload flow for CSV/Excel files by allowing
table column role configuration at upload time.

Previously, users had to:
1. Upload and parse a table file.
2. Open parser settings and manually set table column roles.
3. Re-parse the file for the roles to take effect.

This was inefficient and required an unnecessary second parse.

With this change:
1. When the knowledge base uses table parsing, the upload dialog
extracts CSV/Excel headers client-side.
2. Users can choose Auto mode or Manual mode.
3. In Manual mode, users can assign per-column roles before upload.
4. The selected parser config is sent with the upload request and
applied server-side during document creation.

Result: configured table column roles are applied from the first parse.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-05-25 16:05:38 +08:00
nickmopen
e7d45dd645 Feat: Expose Doc Generator file metadata as discrete outputs (#15080)
Declare doc_id, filename, mime_type, and size as separate outputs on the
Document Generation component so downstream nodes (e.g., the Code
component) can consume them via the variable picker. The existing
download JSON blob is preserved unchanged for the Message component's
download-chip rendering.

### What problem does this PR solve?

The Document Generation component previously exposed only a single
`download` output —
a JSON-encoded blob containing the file's `doc_id`, `filename`,
`mime_type`, `size`,
and base64 payload. On top of that, the variable picker actively hides
this `download`
entry from every consumer except the Message component (because the
embedded base64 is
  too heavy to splat into arbitrary downstream nodes).

The combined effect: users wiring the Doc Generator's output into a Code
component had
no way to retrieve basic file info such as `file_name` or `doc_id` from
the picker,
blocking workflows that need to post-process the generated file (e.g.,
registering it
  elsewhere, custom delivery, follow-up API calls).

This PR declares `doc_id`, `filename`, `mime_type`, and `size` as
**discrete outputs**
on the Document Generation component, alongside the existing `download`
blob. The new
  fields:

- Appear in the variable picker for **all** downstream nodes, including
the Code
  component, so users can bind them directly to script arguments.
- Are cheap scalars only — no base64 payload leaks into other
components.
- Leave the existing `download` JSON blob completely untouched, so the
Message
component's download-chip rendering (which parses that blob via
`_is_download_info`)
  keeps working with no behavior change.

  Changes:
- `agent/component/docs_generator.py` — declare the four new outputs in
  `DocGeneratorParam` and emit them via `set_output(...)` in `_invoke`.
- `web/src/pages/agent/constant/index.tsx` — extend
`initialDocGeneratorValues.outputs`
   with the new keys.
- `web/src/pages/agent/form/doc-generator-form/index.tsx` — mirror the
new outputs in
  the zod schema so the form is valid.

No changes needed to the picker's existing `download`-hiding filter — it
matches only
on the literal output name `download`, so the new metadata entries fall
through
  naturally.

  Reported in: https://github.com/infiniflow/ragflow/issues/14461.
  ### Type of change

  - [x] New Feature (non-breaking change which adds functionality)
2026-05-25 16:05:00 +08:00
Haruko386
69f301b84a Go: implement embed for Tencent Hunyuan (#15207)
### What problem does this PR solve?

Implement embed for Tencent Hunyuan

**Verified from CLI**
```
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'hunyuan-embedding@test1@hunyuan' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 1024      | 0     |
| 1024      | 1     |
+-----------+-------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-25 16:04:17 +08:00
ちー
bb6cfc14e6 feat[go]: implement provider: TokenHub (#15159)
### What problem does this PR solve?

implement provider TokenHub

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-25 16:02:50 +08:00
Wang Qi
5069561abc Fix /chat/completions to allow send only the latest message (#15197)
### What problem does this PR solve?

1. Fix /chat/completions to send only the latest message
2. Allo chat stream=False

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 14:23:33 +08:00
Wang Qi
bb148edf4c Revert "Fix: /openai/<chat_id>/chat/completions not aware of session_id" (#15205)
Reverts infiniflow/ragflow#15155 because this is never supported, keep
it as it is.
2026-05-25 14:23:10 +08:00
Jin Hai
f8c626bbc8 Go: add ingestion server (#15094)
### What problem does this PR solve?

1. Go ingestion server will connected with admin server with gRPC stream
2. Go ingestion server will be responsible for ingestion tasks
```

RAGFlow(admin)> list ingestors;
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+
| address         | cpu_usage | id                               | last_heartbeat            | name     | process_id | rss_usage    | status | task_count | vms_usage     |
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+
| 127.0.0.1:58564 | 0         | bdd1870eea2646e0aacb8a2cd3307aa2 | 2026-05-24T18:16:17+08:00 | ingestor | 680152     | 212.72265625 | active | 0          | 2589.12109375 |
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+

RAGFlow(admin)> start ingestion 'abc';
+----------------------------------+
| task_id                          |
+----------------------------------+
| e714777639ca4760ab427b5f211e81ad |
+----------------------------------+

RAGFlow(admin)> stop ingestion 'f7bd39d0a724457eb5fdce6d81699776';
+----------------------------------+
| task_id                          |
+----------------------------------+
| f7bd39d0a724457eb5fdce6d81699776 |
+----------------------------------+

RAGFlow(admin)> list tasks;
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+
| ETA | assign_to                        | error | from | id                               | last_update               | start_time | status     |
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | eae6431da72a40e796cff3a03008091b | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 6cccdd174bd049ecb05a774bbb47593f | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | ef360d777e57485799adb96b30f2b4b8 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | bcc5c5448cb64de48b6b6171c36fb790 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | bfc25384c43a443294fe2da979a38ac2 | 2026-05-24T19:46:03+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 84960537b85d413b8990a9efd5952d67 | 2026-05-24T19:46:04+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 3d223c1b51e24b36861a3bfb2f1d58d4 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | e433b0e356b846c89c301621a3c54494 | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 7c93a3880f074ebd8eca14e6b51bb7ef | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | df2e4ef51aaf4390bff9a23f2692486e | 2026-05-24T19:46:04+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 7377c53010194ef7a83aa206698d66ff | 2026-05-24T19:46:05+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | df64d1a1f9d348e3a2f174c4d7d69e73 | 2026-05-24T19:46:05+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | b59834512e2847e1bdf13ace04b8a456 | 2026-05-24T19:46:06+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 0064bb0ab69344028d1ecfda053826f4 | 2026-05-24T19:46:03+08:00 |            | QUEUED     |
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+


```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-25 14:00:08 +08:00
Haruko386
5d022d83e8 Go: implement provider: PaddleOCR_Local (#15158)
### What problem does this PR solve?

Go: implement provider: PaddleOCR_Local

**Verified from CLI**

```
RAGFlow(user)> ocr with 'PaddleOCR-VL@test@paddleocr_local' file './internal/test1.jpg'
+----------------------+
| text                 |
+----------------------+
| ## Parallel to these |
+----------------------+
```

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
- [X] New Feature (non-breaking change which adds functionality)
- [X] Refactoring
2026-05-25 12:12:57 +08:00
dripsmvcp
8d8ea71877 Go: implement provider: Tencent Hunyuan (#15092)
## Summary
- Adds a `Hunyuan` Go driver so the new API server can route Tencent
Hunyuan chat instances (registered in `conf/llm_factories.json:3830` as
`Tencent Hunyuan`). Follows the same SaaS-driver shape used for
Astraflow, Avian, Novita, TogetherAI, Replicate, DeepInfra, Upstage, and
LongCat.

Closes #15087

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-25 11:04:39 +08:00
Wang Qi
0ce6655789 Fix: /chat/completions not aware of conversation_id (#15162)
### What problem does this PR solve?

Fix /chat/completions not aware of conversation_id

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 10:47:08 +08:00
VincentLambert
50424df48e feat(i18n): complete French translation — add ~1400 missing keys (#15192)
## Summary

- Brings the French locale (`web/src/locales/fr.ts`) to full parity with
the English reference
- Adds ~1400 missing translation keys across **all sections**: `common`,
`chat`, `header`, `login`, `admin`, `setting`, `flow`,
`knowledgeDetails`, `knowledgeConfiguration`, `memory`, `skills`,
`skillSearch`, `chunk`, `mcp`, `fileManager`, `search`,
`dataflowParser`, `datasetOverview`, `deleteModal`, `empty`, `explore`,
`memories`, `pagination`, `language`, `knowledgeList`
- All strings containing French apostrophes use double-quote delimiters
(prevents JS syntax errors)

## Test plan

- [ ] `npx esbuild src/locales/fr.ts --bundle=false` — no errors
- [ ] `npx eslint src/locales/fr.ts` — no errors
- [ ] Switch UI language to French and verify key sections render
correctly (chat, knowledge base, admin panel, agent flow)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-25 10:32:45 +08:00
bitloi
432e966414 fix(go): support OpenAI audio endpoints (#15104)
### What problem does this PR solve?

Closes #15102.

OpenAI's Go provider config advertises `whisper-1` as ASR and `tts-1` as
TTS, but the Go driver returned `openai, no such method` for both audio
paths and did not define `url_suffix.asr` / `url_suffix.tts`.

This PR:

- adds OpenAI audio URL suffixes for `audio/transcriptions` and
`audio/speech`
- implements non-streaming `TranscribeAudio` using multipart form
uploads
- implements non-streaming `AudioSpeech` using the OpenAI speech JSON
request shape
- keeps streaming TTS explicitly unsupported instead of sending binary
audio through the text SSE sender
- adds focused tests for config coverage, ASR/TTS request shape,
required TTS voice validation, and unsupported streaming TTS


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-25 10:25:53 +08:00
Wang Qi
e6dd397531 Fix: /openai/<chat_id>/chat/completions not aware of session_id (#15155)
### What problem does this PR solve?

Fix: /openai/<chat_id>/chat/completions not aware of session_id

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-22 20:38:56 +08:00
Tohka
302f97de50 Go: implement reasoning_chat, TTS, ASR for Groq (#15153)
### What problem does this PR solve?

Go: implement reasoning_chat, TTS, ASR for Groq

**Verify from CLI**
```
RAGFlow(user)> think chat with 'qwen/qwen3-32b@test@groq' message 'who r u'
Thinking: Okay, the user asked, who r u. I need to determine what the user is asking. They may be asking about my identity. I should introduce my name and basic functions. The user might want to know what I can do, so I should list some common use cases, such as answering questions, creating writing, coding, and expressing opinions. The user may be curious about how they can interact with me, so they can be advised to ask any questions or provide instructions. Keep your answers conversational, avoid overly technical terms, keep answers concise, and encourage further interaction. Check if there's any ambiguity in the answer and make sure it's accurate and meets the user's needs. Also consider if there are other aspects the user may be interested in, such as my training data or performance. But since the question is basic, I'll focus on the essentials first and invite the user to ask more. In summary, respond to the user's questions by introducing yourself, your functions, and encouraging further interaction.

Answer: Hello! I'm Qwen. I am a large-scale language model developed by Tongyi Lab, designed to assist you in various ways, such as answering questions, creating text, logical reasoning, programming, and more. I aim to provide clear, accurate, and helpful information and support. How can I assist you today? Feel free to ask any questions or give me tasks! 😊
Time: 2.199908


RAGFlow(user)> stream think chat with 'openai/gpt-oss-20b@test@groq' message 'who r u'
Thinking:  to respond politely.
Answer: ’m ChatGPT—an AI language model created by OpenAI. I’m here to answer questions, offer explanations, and help with a wide range of topics. How can I assist you today?


RAGFlow(user)> tts with 'canopylabs/orpheus-arabic-saudi@test@groq' text 'hello? show yourself' play format 'wav' param '{"voice": "fahad"}'
SUCCESS


RAGFlow(user)> asr with 'whisper-large-v3-turbo@test@groq' audio './internal/test.wav' param '{"language": "en"}'
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
|  The examination and testimony of the experts enabled the Commission to conclude that five shots may have been fired |
+----------------------------------------------------------------------------------------------------------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-22 18:02:30 +08:00
Haruko386
3f02ca7ba1 Go: implement embed, rerank, tts for AstraFlow (#15135)
### What problem does this PR solve?

implement embed, rerank, tts for AstraFlow

**Verify from CLI**

```
# Astraflow
RAGFlow(user)> tts with 'IndexTeam/IndexTTS-2@test3@astraflow' text 'hello? show yourself' play format 'wav' param '{"voice": "jack_cheng"}'
SUCCESS

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'bge-reranker-v2-m3@test3@astraflow' top 3;
+-------+---------------------+
| index | relevance_score     |
+-------+---------------------+
| 0     | 0.9837390184402466  |
| 2     | 0.06322699040174484 |
| 1     | 0.04663187265396118 |
+-------+---------------------+

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'text-embedding-3-large@test3@astraflow' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 3072      | 0     |
| 3072      | 1     |
+-----------+-------+

# Xinference


```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-22 18:02:01 +08:00
writinwaters
bf9297a343 Docs: Added a guide on integrating Discord. (#15156)
### What problem does this PR solve?

How to ingest messages from your Discord server.

### Type of change

- [x] Documentation Update
2026-05-22 17:49:18 +08:00
Wang Qi
87918650ff Refactor: Move API files (#15151)
Refactor: Move API files
2026-05-22 17:44:05 +08:00
Wang Qi
7e6844118b Fix search vector_similarity_weight (#15108)
### What problem does this PR solve?

Fix search vector_similarity_weight

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-22 16:05:13 +08:00
ghost
f9ce07ced1 feat(go-models): add Groq provider driver (#15097)
### What problem does this PR solve?

Closes #15088.

Adds Groq support to the Go model-provider layer so Groq instances can
be routed through the Go API server with the same OpenAI-compatible
chat, streaming, model listing, and connection-check flow used by other
SaaS providers.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Summary

- Added a Groq Go model driver.
- Added the Groq provider catalog and default OpenAI-compatible API URL.
- Registered Groq in the model factory.
- Added focused provider tests.

## What changed

- Implemented chat completions, SSE streaming, ListModels, and
CheckConnection for Groq.
- Covered request shape, stream termination, reasoning fallback, model
listing, custom base URLs, safe transport setup, and unsupported
methods.
- Kept the provider catalog scoped to current Groq chat-capable model
IDs.
- Cleaned up pre-existing Go model package validation blockers so the
package can be tested normally with vet enabled.

## Why

The existing Python/provider catalog path includes Groq, but the Go
model-provider layer did not have a Groq driver, so the Go API server
could not instantiate or use Groq as requested in #15088.

## Notes

The model package now validates without disabling vet.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-22 15:24:52 +08:00
Lynn
893980ed8f Fix: add model_type into llm_setting (#15141)
### What problem does this PR solve?

Add model_type into llm_setting

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-22 15:23:07 +08:00
buua436
71a52d579c fix: move agent attachment download api (#15146)
### What problem does this PR solve?

move agent attachment download api to the correct route and update
frontend callers

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Notes

- Move the attachment download endpoint from document routes to agent
routes.
- Update frontend download callers to use the agent attachment endpoint.
- Reuse the shared file response header helper instead of duplicating it
in `agent_api.py`.
2026-05-22 15:22:05 +08:00
dripsmvcp
ed04893415 Go: implement provider: TokenPony (#15091)
## Summary
- Adds a `TokenPony` Go driver so the new API server can route TokenPony
chat instances, matching the existing Python `TokenPonyChat`
(`rag/llm/chat_model.py:1210`). Follows the same SaaS-driver shape used
for Astraflow, Avian, Novita, TogetherAI, Replicate, DeepInfra, Upstage,
and LongCat.

Closes #15086

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-22 15:21:45 +08:00
kpdev
faf77a5a8a feat(evaluation): track token usage in evaluation results (#13487)
## Summary

Implements the TODO in `evaluation_service.py`: **Track token usage** in
evaluation results.

## Changes

- **Import** `num_tokens_from_string` from `common.token_utils`
- **Prompt tokens**: Use the full prompt returned by `async_chat` when
available (includes system prompt + knowledge base + query), otherwise
fall back to the question token count
- **Completion tokens**: Count tokens in the generated answer
- **Storage**: Store `token_usage` as `{prompt_tokens,
completion_tokens, total_tokens}` in each `EvaluationResult` instead of
`None`

## Why

The evaluation pipeline previously saved `token_usage: None` for every
result. This change allows downstream consumers (e.g. evaluation
dashboards, cost tracking) to see approximate token usage per test case
using the same tokenizer (tiktoken cl100k_base) used elsewhere in
RAGFlow.

## Testing

- No new tests added; existing evaluation flow unchanged
- Token counting uses existing `num_tokens_from_string` utility

---------

Co-authored-by: kiannidev <kiannidev@users.noreply.github.com>
2026-05-22 15:19:53 +08:00
Jake Armstrong
b1ef5d365f Go: implement ASR in OpenRouter driver (#15067)
### What problem does this PR solve?

Fixes #15066

OpenRouter now exposes an official speech-to-text endpoint at `POST
/api/v1/audio/transcriptions`, but the Go model driver still returned
`openrouter, no such method` from `TranscribeAudio`. This left
OpenRouter ASR models unavailable through the Go API server even though
the provider already has OpenRouter audio support for TTS.

Related provider-tracking context: #14736

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-22 15:19:38 +08:00
Full Stack Developer
8f90740d2e feat: pass chat_template_kwargs through agent chat completion (#14542)
### What problem does this PR solve?

The agent API currently does not pass chat_template_kwargs to the
underlying LLM call path, so clients cannot control template-level model
behavior (such as thinking-mode toggles) when invoking
/agents/chat/completion. This PR adds passthrough support for
chat_template_kwargs across agent execution flows (session and
non-session, streaming and non-streaming) by propagating it through
canvas runtime state and into LLM invocation kwargs. This addresses the
feature gap raised in [Issue
#14182](https://github.com/infiniflow/ragflow/issues/14182).

Closes #14182 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-22 15:15:49 +08:00
dale053
c33d0b8081 fix: prevent sensitive fields from leaking in user API responses (#14792)
Closes #14789

### What problem does this PR solve?

User API endpoints (`login`, `user_profile`, `user_add`,
`forget_reset_password`) were returning full user objects via
`to_json()` / `to_dict()`, which included sensitive fields like
`password` and `access_token` in the response body. This leaks
credentials to the client.

This PR adds a `to_safe_dict()` method on the `User` model that strips
sensitive fields (`password`, `access_token`) and replaces all affected
call sites to use it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-22 15:14:26 +08:00
Wang Qi
f4e63ef33f Refactor: enahnce CI (#15147)
### What problem does this PR solve?

Refactor: enahnce CI

### Type of change

- [x] Refactoring
2026-05-22 14:45:09 +08:00
Wang Qi
a9ec78cb9c Refactor: enahnce retry and timeout (#14983)
### What problem does this PR solve?

1. Enhance retry and timeout, and adjust the default timeout
2. NER: spacy do not batch chunks
3. extract _has_cancel_and_exit
4. enhance log messages

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-22 13:16:39 +08:00
Calixto Ong
11ff848b04 feat: Add SDK and cURL examples for chunk management, chat assistant, and retrieval (#4310) (#14208)
Closes #4310

### What problem does this PR solve?

Issue #4310 requests practical examples for the RAGFlow SDK and HTTP API
to help developers get started faster. The existing `example/sdk/`
folder only contains `dataset_example.py`. This PR fills the remaining
gaps by adding examples for three key API areas not yet covered in
`main` or by other open PRs (#13904, #13284):

- **Chunk management** — add, list, update, delete, and retrieve chunks
within a dataset
- **Chat assistant** — create a chat assistant, open a session, send
messages (streaming and non-streaming), and clean up
- **Retrieval** — perform semantic retrieval across one or multiple
datasets

### Type of change

- [x] Documentation Update
- [x] New Feature (non-breaking change which adds functionality)
2026-05-22 12:13:00 +08:00
dale053
6ab25bf715 fix: block SSRF in misc_utils.download_img for OAuth avatars (#14868)
### What problem does this PR solve?

Closes #14865

`download_img` in `common/misc_utils.py` is used for OAuth avatar URLs.
The previous implementation called `async_request` from
`common.http_client`, which followed redirects without re-validating
each hop and did not apply the same SSRF protections as this path needs.
That made it possible to reach non-public or disallowed targets (for
example via redirects or unsafe URLs) when fetching avatars.

This change replaces that flow with an explicit, bounded fetch: each URL
(including every redirect target) is checked with
`common.ssrf_guard.assert_url_is_safe`, DNS is pinned with
`pin_dns_global`, `httpx` streams the body with `follow_redirects=False`
and a manual redirect loop (capped by
`RAGFLOW_OAUTH_AVATAR_MAX_REDIRECTS`), and total response size is capped
(`RAGFLOW_OAUTH_AVATAR_MAX_BYTES`). Timeouts, proxy, and user agent
align with `HTTP_CLIENT_*` env vars without importing `http_client`, so
lightweight tests stay simple.

Unit tests cover empty/None URLs, loopback, cloud metadata-style
addresses, and disallowed schemes so SSRF regressions are caught early.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-22 12:12:04 +08:00
Jake Armstrong
b2bf9155ed Go: implement ASR in ZhipuAI driver (#15134)
### What problem does this PR solve?

This PR implements ASR and TTS support for the ZhipuAI Go driver.

The ZhipuAI model config already advertises `glm-asr-2512` as an ASR
model, but the Go driver returned `zhipu, no such method` from
`TranscribeAudio`. This adds the documented audio transcription endpoint
suffix and sends multipart transcription requests with `model`,
`stream=false`, and `file` fields.

Per maintainer review, this also adds the ZhipuAI TTS endpoint suffix
and implements `AudioSpeech` / `AudioSpeechWithSender` for `glm-tts`.

Closes #15133

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-22 11:53:18 +08:00
ghost
b2053cc3c7 feat(go-models): add PPIO provider driver (#15099)
### What problem does this PR solve?

Closes #15089.

Adds PPIO support to the Go model-provider layer so PPIO instances can
be routed through the Go API server with the same OpenAI-compatible
chat, streaming, model listing, and connection-check flow used by other
SaaS providers.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Summary

- Added a PPIO Go model driver.
- Added the PPIO provider catalog and default OpenAI-compatible API URL.
- Registered PPIO in the model factory.
- Added focused provider and provider-manager tests.

## What changed

- Implemented chat completions, SSE streaming, ListModels, and
CheckConnection for PPIO.
- Covered request shape, stream termination, reasoning fallback, model
listing, custom base URLs, safe transport setup, unsupported methods,
and provider config loading.
- Kept the provider catalog aligned with the existing RAGFlow PPIO
factory model set.
- Cleaned up pre-existing Go model package validation blockers so the
scoped provider tests can run normally with vet enabled.

## Why

The existing Python/provider catalog path includes PPIO, but the Go
model-provider layer did not have a PPIO driver, so the Go API server
could not instantiate or use PPIO as requested in #15089.
2026-05-22 11:52:18 +08:00
buua436
04bdb41909 Fix: guard missing task language (#15136)
### What problem does this PR solve?

guard missing task language

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-22 11:46:38 +08:00
buua436
ea1764a7dc Revert "fix(api): infer /documents/{id}/download Content-Type from filename when ext is omitted (#15052)" (#15138)
Reverts infiniflow/ragflow#15053
2026-05-22 11:46:01 +08:00
writinwaters
57ddd79183 Docs: Fixed a deployment issue (#15114)
### What problem does this PR solve?

Fixed a docusaurus deployment issue.

### Type of change

- [x] Documentation Update
2026-05-21 22:43:49 +08:00
writinwaters
8995662ee6 Docs: Updated v0.25.5 release notes (#15109)
### What problem does this PR solve?

Updated v0.25.5 release notes.

### Type of change


- [x] Documentation Update
2026-05-21 22:04:44 +08:00
Haruko386
1ece1c81da Go: implement rerank, asr, tts for TogetherAI (#15107)
### What problem does this PR solve?

implement rerank, asr, tts for TogetherAI

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-21 20:57:04 +08:00
Wang Qi
c5a46fda44 Fix: <asyncio.locks.Semaphore object at 0xabcd [locked]> is bound to a different event loop (#15100)
Fix: <asyncio.locks.Semaphore object at 0xabcd [locked]> is bound to a
different event loop
2026-05-21 19:23:41 +08:00
Jin Hai
775ea55679 Docs: update python version to 3.13 (#15103)
### What problem does this PR solve?

1. update python version to 3.13
2. upgrade ormsgpack to 1.6.0

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 19:09:19 +08:00
Haruko386
a725e114f9 Go: implement ASR and TTS for Xinference (#15096)
### What problem does this PR solve?

implement ASR and TTS for Xinference

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-21 18:28:06 +08:00
Jonathan Hill
111cdc77b5 fix: guard LLM response against empty choices (fixes #14711) (#14988)
## Summary

Fixes 10 unguarded `response.choices[0]` accesses that cause
`IndexError` or `AttributeError` when the LLM returns an empty `choices`
list — the scenario described in #14711.

- `rag/llm/cv_model.py`
- `rag/llm/chat_model.py`

Each access site is now guarded with:
```python
if not response.choices:
    raise ValueError("LLM returned empty response")
```

## Verification

Detected and verified by [pact](https://github.com/qizwiz/pact) — a
sheaf-cohomological LLM contract checker using Z3 as a local theory
solver.

**pact sheaf-cohomological proof status after fix:**

| File | Ȟ¹ (after) | Z3 |
|------|-----------|-----|
| `rag/llm/cv_model.py` | 0 | UNSAT ✓ |
| `rag/llm/chat_model.py` | 0 | UNSAT ✓ |

All access sites proven safe (Z3 UNSAT certificate).

The checker was also used to verify the autogen streaming-None fix in
[microsoft/autogen#7711](https://github.com/microsoft/autogen/pull/7711).

## Test plan
- [ ] Existing test suite passes
- [ ] Manually test with a provider that returns empty `choices` under
load (e.g. Vertex AI)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Signed-off-by: Jonathan Hill <jonathan.f.hill@gmail.com>
2026-05-21 15:37:19 +08:00
dripsmvcp
12a148d541 fix(api): guard against missing session in get_agent_session (#15011)
`GET /agents/<agent_id>/sessions/<session_id>` crashed with
`AttributeError: 'NoneType' object has no attribute 'to_dict'` when the
session lookup failed: `_, conv =
API4ConversationService.get_by_id(...)` returned `(False, None)`, then
`conv.to_dict()` was called unconditionally.

This is reachable in multi-instance deployments: the session row may not
yet be visible on the node servicing the immediate follow-up GET after a
session is created on a different node.

Add the same `if not exists` guard already used by every other call site
of `API4ConversationService.get_by_id` (see agent_api.py:1147,
sdk/session.py:179, conversation_service.py:248, canvas_service.py:323).

Closes #14989

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-21 15:37:10 +08:00
dripsmvcp
ce9a4425d2 fix(imap): handle multi-address headers in _parse_singular_addr (#15006)
Replace the RuntimeError with a warning + first-address fallback so a
single email whose From header contains multiple addresses no longer
crashes the entire IMAP sync task. Also add regression tests covering:

- #14963: RFC 5322 quoted display names with commas (e.g. "Schlüter,
Sabine" <s@x>) parsed as one address, not two.
- #14964: multi-address headers warn instead of raising.

Closes #14964
Refs #14963
2026-05-21 15:37:02 +08:00
dripsmvcp
85caad5558 fix(docker): bump nginx to 1.31.0 (CVE-2026-42945) (#15007)
## Summary
- Bump pinned nginx in `Dockerfile` from `1.29.5-1~noble` (vulnerable)
to `1.31.0-1~noble` to remediate **CVE-2026-42945**.

## Root Cause
`Dockerfile:58` pinned `ARG NGINX_VERSION=1.29.5-1~noble`. Per the
official nginx security advisory, **CVE-2026-42945** is a buffer
overflow in `ngx_http_rewrite_module` triggered via the `rewrite` and
`set` directives, affecting nginx **0.6.27 through 1.30.0**. `1.29.5`
falls inside that range, so the shipped image is vulnerable.

References:
- nginx security advisories:
https://nginx.org/en/security_advisories.html
- Vendor advisory: https://my.f5.com/manage/s/article/K000161019
- Fixed versions: `1.31.0` (mainline) and `1.30.1` (stable)

## Fix
Single-line change in `Dockerfile:58`:

```diff
-ARG NGINX_VERSION=1.29.5-1~noble
+ARG NGINX_VERSION=1.31.0-1~noble
2026-05-21 15:36:51 +08:00
Prateek Jain
bf4864e614 fix(infinity): declare extra field + serialize dict on write to unbreak RAPTOR (#14998)
### What problem does this PR solve?

Fixes #14997.

RAPTOR builds on the Infinity backend have been broken since v0.25.2
introduced the `extra` field in code (`rag/svr/task_executor.py:1011`)
without declaring it in `conf/infinity_mapping.json`. Every RAPTOR job
fails with:

```
infinity.common.InfinityException: (3013, 'Fail to bind the expression: extra@src/planner/expression_binder_impl.cpp:99')
```

The auto-migration in
`common/doc_store/infinity_conn_base.py:_migrate_db()` adds any columns
it finds in the mapping JSON to existing tables — so the only thing
standing between users and a working RAPTOR build is that one missing
declaration. OceanBase, ES, and OpenSearch were unaffected because they
store `extra` as a native JSON type; only Infinity (which has a strict
`varchar`/`integer`/`float` schema) needed the addition.

### The fix

Two-part change:

1. **`conf/infinity_mapping.json`**: declare `"extra": {"type":
"varchar", "default": ""}`. On next startup, `_migrate_db()` adds the
column to all existing chunk tables — no manual DDL needed for upgrading
installations.
2. **`rag/utils/infinity_conn.py` `insert()`**: serialize the `extra`
dict to a JSON string at write time, since Infinity's `varchar` can't
store a Python dict directly. Modelled on the existing `chunk_data`
handling a few lines above.

The read path (`rag/utils/raptor_utils.py:_as_extra_dict`) already
normalises both dict and JSON-string inputs, so no read-side change is
needed. Other backends are untouched — `task_executor.py` still writes
the dict, and the OceanBase/ES/OpenSearch insert paths handle dicts
natively.

### Verification

Tested on a v0.25.4 deployment with the Infinity backend by applying the
same two changes via mounted-volume override:

- Confirmed `_migrate_db()` adds the `extra` column to all pre-existing
chunk tables on startup (column visible via Infinity's
`show_columns()`).
- Triggered RAPTOR builds on four datasets (~21k chunks total) via `POST
/api/v1/datasets/<id>/index?type=raptor`.
- All four progressed past the previously-failing
`get_raptor_chunk_methods()` call into actual entity-extraction and
clustering work without the (3013) error.
- GraphRAG builds (which can trigger the same path indirectly via
`task_executor.py:857`) also progressed cleanly.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-05-21 15:36:15 +08:00
tmimmanuel
38a8bc3dab fix(upstage): extract reasoning delta from streaming responses (#14817)
### What problem does this PR solve?

`UpstageModel.ChatStreamlyWithSender` (in the driver merged via #14819)
only extracted `delta.content` from each SSE event. For the `solar-pro3`
reasoning family (and any future Upstage model that follows the same
wire shape), the chain-of-thought is streamed in a **separate
`delta.reasoning` field**, and the driver was silently dropping all of
it.

The non-streaming path already extracts `message.reasoning` into
`ChatResponse.ReasonContent` (added earlier in this PR's history), so
the same model produced **inconsistent behavior** between streaming and
non-streaming: a tenant calling `solar-pro3` with `reasoning_effort:
high` would see the reasoning trace if they used `ChatWithMessages` but
not if they used `ChatStreamlyWithSender`.

### Live evidence

Probed against `api.upstage.ai/v1/chat/completions` with `solar-pro3` +
`reasoning_effort: high` + `stream: true` (8000-token budget so the
reasoning has room to finish):

```
$ curl -sN -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
       -X POST https://api.upstage.ai/v1/chat/completions \
       -d '{"model":"solar-pro3","messages":[{"role":"user","content":"Compute 15% of 80."}],
            "max_tokens":8000,"stream":true,"reasoning_effort":"high"}'

# across 168 SSE events:
#   delta keys seen: [content reasoning role]
#   delta.content total len:   121 chars   (the visible answer)
#   delta.reasoning total len: 159 chars   (the chain-of-thought) <- driver dropped this
```

A representative event showing both fields side by side:

```json
data: {"choices":[{"index":0,"delta":{"reasoning":"15% = 0.15."}}]}
data: {"choices":[{"index":0,"delta":{"content":"15% of 80 is "}}]}
```

The 159 chars of reasoning were arriving on the wire and being thrown
away. `solar-pro2` was also probed (625 events); it does **not** emit
`delta.reasoning` — its reasoning is inlined into `delta.content` — so
this change is a no-op for it and for `solar-mini`.

### What this PR includes

- `internal/entity/models/upstage.go`: in the SSE scanner loop, extract
`delta.reasoning` before `delta.content` and forward each non-empty
chunk via the sender's second arg (the existing `reasonContent` channel
the non-stream path already populates).

The ordering contract is documented inline: reasoning chunks within a
single SSE event are emitted before content chunks, so a UI that pipes
both sees the chain-of-thought start before the answer for that token,
matching the wire order Upstage emits.

- `internal/entity/models/upstage_test.go`: three new tests pinning the
new behavior:
- `TestUpstageStreamExtractsReasoningDelta` — reasoning + content
forwarded to the right sender args; one-of invariant per call
- `TestUpstageStreamReasoningChunksArriveBeforeContent` — ordering
pinned within a single SSE event that carries both fields
- `TestUpstageStreamWithoutReasoningStillWorks` — regression net:
non-reasoning models (`solar-mini`, `solar-pro2`) continue to work; the
reason callback never fires

No interface change. No factory change. No config change.

### How was this tested?

```
$ go test -vet=off -run TestUpstage -count=1 -v ./internal/entity/models/...
... (existing tests 1..9 still pass) ...
=== RUN   TestUpstageStreamExtractsReasoningDelta
--- PASS: TestUpstageStreamExtractsReasoningDelta (0.01s)
=== RUN   TestUpstageStreamReasoningChunksArriveBeforeContent
--- PASS: TestUpstageStreamReasoningChunksArriveBeforeContent (0.01s)
=== RUN   TestUpstageStreamWithoutReasoningStillWorks
--- PASS: TestUpstageStreamWithoutReasoningStillWorks (0.00s)
PASS
ok      ragflow/internal/entity/models  0.034s
```

12/12 Upstage tests pass on go 1.25. `go build
./internal/entity/models/...` exits 0.

**Live integration test** (smoke test not committed) — the patched
driver was run directly against `api.upstage.ai/v1` with the same prompt
that produced the curl evidence above:

```
=== RUN   TestUpstageStreamReasoningLiveSmoke
    [OK] visible content: 50 chunks, 84 chars
    [OK] reasoning:       39 chunks, 90 chars
    content head 200:   "\\(15\\% = \\frac{15}{100}=0.15\\).\n\n\\[\n0.15 \\times 80 = 12.\n\\]\n\n**15 % of 80 is 12.**"
    reasoning head 200: "We need to compute 15% of 80. That's 0.15 * 80 = 12. So answer is 12. Provide explanation."

UPSTAGE STREAM REASONING SMOKE PASSED
--- PASS: TestUpstageStreamReasoningLiveSmoke (1.97s)
```

Before this fix, the same call would have produced **0 reasoning
chunks**. The 90 chars of reasoning that the patched driver now surfaces
are the chain-of-thought solar-pro3 emits when reasoning_effort is high.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-21 15:33:21 +08:00
tmimmanuel
85d0b46d8e fix(mistral): handle structured content from magistral reasoning models (#14805)
### What problem does this PR solve?

`MistralModel.ChatWithMessages` (in the driver merged via #14807)
assumes that `choices[0].message.content` from `/v1/chat/completions` is
always a string and falls through to `return nil, fmt.Errorf("invalid
content format")` on anything else.

That assumption breaks for the **magistral reasoning family**
(`magistral-small-*`, `magistral-medium-*`). When the model needs a
chain-of-thought to answer, Mistral returns `content` as a **structured
array of typed parts**:

```json
"content": [
  {"type": "thinking",
   "thinking": [{"type": "text", "text": "Combined speed is 150 mph. 300 / 150 = 2 hours."}],
   "closed": true},
  {"type": "text", "text": "They will meet after **2 hours**."}
]
```

Concretely, this is what the live API returns today (probed against
`api.mistral.ai/v1`):

```
$ curl -H "Authorization: Bearer <key>" -H "Content-Type: application/json" \
       -X POST https://api.mistral.ai/v1/chat/completions \
       -d '{"model":"magistral-medium-latest",
            "messages":[{"role":"user","content":"two trains 60mph and 90mph, 300mi apart, when do they meet? step by step."}],
            "max_tokens":1024}'
HTTP 200
{ "choices":[{"message":{
    "role":"assistant",
    "content":[
      {"type":"thinking","thinking":[{"type":"text","text":"Okay, let's see..."}],"closed":true},
      {"type":"text","text":"To determine when the two trains meet..."}
    ]}}] }
```

With the current driver, every call like that returns the generic
`"invalid content format"` error. Trivial prompts that happen to fit in
a string answer still succeed, so the breakage is **non-deterministic
from the tenant's POV**: same model, same provider, sometimes works,
sometimes 500s with no useful error.

A secondary issue: `conf/models/mistral.json` does not include any
magistral model. The picker hid the broken path, which is why this
wasn't caught during #14807's review.

### What this PR includes

- New helper `extractMistralContent(raw interface{}) (answer,
reasonContent string, err error)` in
`internal/entity/models/mistral.go`, which normalizes both shapes
Mistral can return:
- `string` → historical path. `Answer = content`, `ReasonContent = ""`.
Preserves behavior for every non-reasoning model (`mistral-large-*`,
`mistral-small-*`, `ministral-*`, `codestral-*`, `pixtral-*`,
`open-mistral-nemo`).
- `[]interface{}` → walk the parts. Concatenate every `{"type":"text",
"text":...}` part into `Answer`; concatenate the inner text inside every
`{"type":"thinking", "thinking":[...]}` part into `ReasonContent`.
- `ChatWithMessages` now calls the helper instead of doing the raw
`.(string)` cast.
- Unknown part types are **skipped, not failed**. Mistral has been
adding new content variants quickly (audio chunks, citations, etc.);
this driver should not 500 every call when a new part type appears.
- `conf/models/mistral.json`: add `magistral-medium-latest` and
`magistral-small-latest`. Both are visible in `/v1/models` today.

No interface change. No factory change. No new dependencies.

### How was this tested?

**Unit tests** — 5 new tests in `internal/entity/models/mistral_test.go`
on top of the 27 already shipped via #14807:

- `TestMistralChatHandlesStringContent` — regression net for the
historical path
- `TestMistralChatExtractsReasoningFromStructuredContent` — the fixture
body is a trimmed copy of the actual `magistral-medium-latest` response
captured above; asserts both `Answer` and `ReasonContent` are populated
correctly
- `TestMistralChatHandlesStructuredContentWithoutThinking` —
`magistral-*` with a trivial answer returns a structured shape that has
only a `text` part; `ReasonContent` must stay empty
- `TestMistralChatIgnoresUnknownContentPartTypes` — `audio_url` and
`future_part_type` parts are skipped, `text` parts still flow through
- `TestExtractMistralContent` — table-driven unit coverage of the helper
for string, empty string, nil, empty array, text-only, thinking+text,
unsupported root type

```
$ go test -vet=off -run "TestMistral|TestExtractMistralContent" -count=1 -v ./internal/entity/models/...
=== RUN   TestMistralChatHandlesStringContent
--- PASS: TestMistralChatHandlesStringContent (0.00s)
=== RUN   TestMistralChatExtractsReasoningFromStructuredContent
--- PASS: TestMistralChatExtractsReasoningFromStructuredContent (0.00s)
=== RUN   TestMistralChatHandlesStructuredContentWithoutThinking
--- PASS: TestMistralChatHandlesStructuredContentWithoutThinking (0.00s)
=== RUN   TestMistralChatIgnoresUnknownContentPartTypes
--- PASS: TestMistralChatIgnoresUnknownContentPartTypes (0.00s)
=== RUN   TestExtractMistralContent
=== RUN   TestExtractMistralContent/plain_string
=== RUN   TestExtractMistralContent/empty_string
=== RUN   TestExtractMistralContent/nil
=== RUN   TestExtractMistralContent/empty_array
=== RUN   TestExtractMistralContent/text_only
=== RUN   TestExtractMistralContent/thinking_then_text
=== RUN   TestExtractMistralContent/unknown_root_type
--- PASS: TestExtractMistralContent (0.00s)
PASS
ok      ragflow/internal/entity/models  0.046s
```

All 32 Mistral tests pass on go 1.25. `go build
./internal/entity/models/...` exits 0.

**Live integration test** — driver exercised against `api.mistral.ai/v1`
with the patched code:

```
=== RUN   TestMistralMagistralSmoke
    [OK] "magistral-small-latest" present upstream
    [OK] "magistral-medium-latest" present upstream
    [OK trivial]    Answer="7"  ReasonContent=""
    [OK reasoning]  Answer len=797   head="To determine when the two trains meet, we can follow these steps:\n\n1. **Identify..."
                    ReasonContent len=1069 head="Okay, let's see. There are two trains, one going 60 mph and the other going 90 mph. They're moving towards each other, s..."

MAGISTRAL SMOKE PASSED
--- PASS: TestMistralMagistralSmoke (18.09s)
PASS
ok      ragflow/internal/entity/models  18.112s
```

What the live run proves on the wire:

- `magistral-small-latest` with a trivial prompt still uses the
string-content shape; the regression-net path is exercised against the
real server, not just the mock.
- `magistral-medium-latest` with a reasoning prompt uses the
structured-array shape; the new code path extracts a 1069-character
reasoning trace into `ChatResponse.ReasonContent` and a 797-character
visible answer into `ChatResponse.Answer`. Before this fix, the same
call returned `"invalid content format"` and the caller saw nothing.

The smoke-test file itself is not committed (live tests live outside the
PR diff, same convention used for prior provider PRs).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-21 15:33:14 +08:00
sapienza yoan
9d37234953 build(go): make bash build.sh work on macOS arm64 (Homebrew) (#15009)
## Problem

The Go server build pipeline (`build.sh` + CMake + CGO bindings) was
tested on Ubuntu only. On macOS arm64 with Homebrew it fails in five
orthogonal places. None of these require platform-specific code paths —
the same source builds on both Linux and Darwin after these fixes.

## Reproduction (before)

```
$ uname -a
Darwin … 25.4.0 arm64
$ brew install cmake pcre2 simde
$ bash build.sh
…
error: 'simde/x86/sse4.1.h' file not found
error: implicit instantiation of undefined template 'std::basic_istringstream<char>'
error: no matching function for call to 'Join'
…
clang: error: no such file or directory: '/usr/local/lib/libpcre2-8.a'
```

## Fix (5 small, orthogonal changes)

### 1. `internal/cpp/CMakeLists.txt` — find Homebrew + libpcre2-8
portably

- Detect Apple platforms via `if(APPLE)`, call `brew --prefix` once, add
`${HOMEBREW_PREFIX}/include` and `${HOMEBREW_PREFIX}/lib`. No effect on
Linux.
- Replace the literal `libpcre2-8.a` link token (which only the Linux
linker finds in `/usr/local/lib` by default) with
`find_library(PCRE2_LIB NAMES pcre2-8 REQUIRED)`. Works on
`/usr/lib/x86_64-linux-gnu` (Debian/Ubuntu), `/usr/local/lib` (Intel Mac
& legacy Linux), `/opt/homebrew/lib` (Apple Silicon).

### 2. `internal/cpp/wordnet_lemmatizer.cpp` +
`internal/cpp/rag_analyzer.cpp` — explicit `#include <sstream>`

libstdc++ (Linux) pulls `<sstream>` in transitively via `<fstream>`;
libc++ (Apple Clang) doesn't, so the existing `std::istringstream` /
`std::ostringstream` uses fail to compile on macOS. One-line include in
each file.

### 3. `internal/cpp/rag_analyzer.cpp` — `Join` template overload fix

`Join(tokens, start, tokens.size(), delim)` at line 146 passes `size_t`
to an `int` parameter. C++23 strict mode in Apple Clang refuses the
implicit narrowing and reports the 4-arg overload as a substitution
failure, leaving the call ambiguous between the 3-arg and 4-arg
templates. Fix: explicit `static_cast<int>(tokens.size())`. Behaviour
identical on libstdc++ — the narrowing was always intentional.

### 4. `internal/binding/rag_analyzer.go` — split darwin CGO LDFLAGS

The existing `#cgo darwin LDFLAGS: ... /usr/local/lib/libpcre2-8.a` only
matches Intel Macs. Apple Silicon Homebrew installs to `/opt/homebrew`.
Split into `darwin,arm64` and `darwin,amd64` build constraints with the
right absolute path on each.

### 5. `build.sh` — accept Homebrew path in the pcre2 sanity check

The sanity check looked at two Linux paths only and then fell through to
`sudo apt -y install libpcre2-dev` on failure. Added
`/opt/homebrew/lib/libpcre2-8.a`, and on Darwin failure now exits
cleanly with the right `brew install pcre2` hint instead of trying
`apt`.

## Verified

- `bash build.sh` now completes on macOS arm64 (Apple Silicon, brew 4.x,
cmake 4.x, Apple Clang 17, Go 1.25, pcre2 10.x, simde 0.8.x).
- Produced binaries: `bin/server_main`, `bin/admin_server`,
`bin/ragflow_cli`.
- `bin/server_main` boots, connects MySQL, runs migrations, loads the 64
model provider configs cleanly.
- Still builds on Linux — the CMake additions are inside an `if(APPLE)`
guard, the `find_library` call matches Linux paths too, the build.sh
check still tries `apt` when not on Darwin.

## Out of scope

The Go server itself currently fails at runtime when not pointing at
Elasticsearch (`Failed to initialize doc engine: failed to ping
Elasticsearch`), but that's the placeholder Infinity engine documented
in `internal/engine/README.md` — unrelated to this build patchset.

---

Happy to split this into smaller PRs if you'd prefer (one per file). The
five changes are independent.
2026-05-21 15:33:09 +08:00
BitToby
bd4ce39038 Go: implement provider: Perplexity (#15008)
## What
- Add Perplexity as a chat and embedding provider backed by its
OpenAI-compatible `/chat/completions` and `/v1/embeddings` APIs
- Register Perplexity in the Go model factory and provider config
- Support non-streaming chat, SSE streaming chat, embeddings, model
listing, and connection checks

Refs #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 15:33:02 +08:00
dripsmvcp
d5ba14a128 feat(go): implement provider Astraflow (#15062) (#15064)
- Adds an `Astraflow` Go driver so the new API server can route
Astraflow (UCloud ModelVerse) chat instances, matching the existing
Python `AstraflowChat` (`rag/llm/chat_model.py:1237`). Follows the same
SaaS-driver shape used for Avian, Novita, TogetherAI, Replicate,
DeepInfra, Upstage, and LongCat.

Closes #15062

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 15:32:56 +08:00
dripsmvcp
5a18df0fd0 Go: implement provider: Avian (#15045)
Closes #15044.

Avian was listed unchecked in the Go-rewrite tracker #14736 and already
had an llm_factories.json entry with 4 preconfigured chat models
(deepseek-v3.2, kimi-k2.5, glm-5, minimax-m2.5), but the Go API server
had no driver to route them. The Python side has supported Avian at
rag/llm/chat_model.py:1220 (AvianChat) via the LiteLLM openai/ provider
with default base https://api.avian.io/v1.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 15:32:49 +08:00
sxxtony
7740ec6c95 Go: implement Embed (embeddings) in Replicate driver (#15073)
### What problem does this PR solve?

`ReplicateModel.Embed` in `internal/entity/models/replicate.go` was a
`"replicate, no such method"` stub. Tracking issue #14736 lists
Replicate's embedding surface as not implemented. This PR wires it up
against Replicate's documented embedding schema.

Until this PR, a tenant who selected a Replicate embedding model got the
sentinel error on every embed call.

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 15:32:41 +08:00
天海蒼灆
3e5b11a523 Feat(browser control):Add new agent component 'browser' to control browser by AI (#14888)
### What problem does this PR solve?
This PR adds a new `Browser` operator to Agent workflows, enabling
prompt-driven browser automation in RAGFlow.Technically based
‘Browser-Use’

It includes:
- Backend browser component execution with tenant LLM integration
- Upload source support (file IDs, URLs, variables, CSV/JSON array)
- Downloaded file persistence to RAGFlow storage
- Frontend node/operator integration, form config, icon, and i18n
updates
- Unit tests for upload/download and ID parsing logic
- Dependency and Docker updates for browser-use runtime support

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-21 15:32:32 +08:00
Stephen Hu
da112e3db0 Refactor:improve dify retrieval logic (#15036)
### What problem does this PR solve?

improve dify retrieval logic for o(n) io to o(1)

### Type of change
- [x] Refactoring
2026-05-21 15:32:24 +08:00
Kevin Hu
e7544562cc Feat: @tool decorator for chat-model tool registration (#15047)
## Summary

- Adds a lightweight `@tool` decorator and `FunctionToolSession` adapter
in `rag/llm/tool_decorator.py` that let callers register plain Python
functions as LLM tools without hand-writing OpenAI function schemas or
building an MCP-style session.
- Refactors `Base.bind_tools` and `LiteLLMBase.bind_tools` in
`rag/llm/chat_model.py` to accept either the new decorator form
`bind_tools(tools=[fn1, fn2])` or the existing `(toolcall_session,
tools_schemas)` form, so existing agent/dialog call-sites in
`agent/component/agent_with_tools.py`, `api/db/services/llm_service.py`,
and `api/db/services/dialog_service.py` are unaffected.
- Adds 8 unit tests in `test/unit_test/rag/llm/test_tool_decorator.py`
covering schema shape, required/optional inference, sync + async
dispatch, and bad-input rejection.

## Usage

```python
from rag.llm.tool_decorator import tool

@tool
def get_weather(city: str) -> str:
    """Get current weather for a city.

    :param city: City name to look up.
    """
    return f"{city}: 21 C, partly cloudy"

chat_mdl.bind_tools(tools=[get_weather])
ans, tk = await chat_mdl.async_chat_with_tools(system, history)
```

The decorator introspects `inspect.signature` + type hints + the
docstring (`:param name:` style) and attaches an OpenAI-format
`openai_schema` to the callable. `FunctionToolSession` duck-types the
existing `ToolCallSession` protocol, dispatching async callables
directly and sync ones through `thread_pool_exec` so the event loop is
never blocked.

## Design notes

- `tool_decorator.py` deliberately does **not** live inside
`rag/llm/__init__.py` to avoid forcing every consumer through the heavy
provider auto-discovery loop and to sidestep a circular import
(`__init__.py` imports `chat_model`, which would otherwise need symbols
from `__init__.py`).
- `FunctionToolSession` is duck-typed against
`common.mcp_tool_call_conn.ToolCallSession` rather than explicitly
inheriting from it, so importing the decorator doesn't pull the MCP
client SDK into the import graph.
- Docstring parsing is intentionally minimal (`:param name:` only) to
keep this dependency-free; Google/NumPy styles can be added later via
`docstring_parser` if needed.

## Test plan

- [x] `python -m pytest test/unit_test/rag/llm/test_tool_decorator.py
-v` — 8 passed
- [x] `python -m pytest test/unit_test/rag/llm/
--ignore=test/unit_test/rag/llm/test_perplexity_embed.py` — 11 passed
(the ignored test has a pre-existing `numpy` import that's unrelated)
- [ ] Reviewer: smoke-test the new path end-to-end with a live model via
`chat_mdl.bind_tools(tools=[my_fn])` to confirm the OpenAI-format
schemas pass through unchanged

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-21 15:32:17 +08:00
bitloi
a6186244ee fix: handle missing SDK authorization headers (#15050)
### What problem does this PR solve?

Closes #15048.

Several SDK session routes in `api/apps/sdk/session.py` called
`.split()` directly on `request.headers.get("Authorization")`. When
clients omitted the header, the handlers raised `AttributeError` before
returning the existing `Authorization is not valid!` response.

This PR centralizes SDK Authorization parsing in a small helper and
keeps the existing error response for missing, empty, or malformed
headers.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Tests

- `ZHIPU_AI_API_KEY=dummy uv run --python 3.13 --group test pytest
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py::test_sdk_session_routes_missing_authorization_unit
-q`
- `uv run --python 3.13 --group test ruff check api/apps/sdk/session.py
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`
- `python3 -m py_compile api/apps/sdk/session.py
test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`
- `git diff --check`
2026-05-21 15:32:00 +08:00
kingloon
da4eaf9fb0 Fix: remove duplicate function definitions (#15063)
### What problem does this PR solve?

Remove duplicate function definitions in
`api/db/services/dialog_service.py`.

**Problem:** Two helper functions were defined twice in the same file,
but with different parameter orders:

- First definition (line 57):
`_resolve_reference_metadata(request_payload=None, config=None)`
- Second definition (line 136): `_resolve_reference_metadata(config,
request_payload=None)`

**Solution:** Keep the second definition (which is actually used by
other modules) and remove the first one to avoid confusion.

Additionally, remove duplicate `_enrich_chunks_with_document_metadata`
definition (keep line 140 version).
<img width="1584" height="313" alt="image"
src="https://github.com/user-attachments/assets/7daee832-244f-4bb2-8488-e3b65012a3f9"
/>
<img width="1672" height="359" alt="image"
src="https://github.com/user-attachments/assets/4fd2f523-273c-4b20-a7c9-ab35740b7834"
/>


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-21 15:31:51 +08:00
kpdev
6932615852 fix(api): infer /documents/{id}/download Content-Type from filename when ext is omitted (#15052) (#15053)
## Summary

- Align **GET `/api/v1/documents/<doc_id>/download`** with
**`/preview`**: resolve extension and MIME type from the stored document
name when the **`ext` query parameter is omitted**, instead of
defaulting to `markdown`.
- When **`?ext=`** is present, behavior stays the same as before
(explicit extension / `Content-Type` mapping).
- Enforce the same access + document lookup pattern as preview
(**`accessible`** + **`get_by_id`**).
- Extend unit tests for the no-`ext` PDF filename case.

## Test plan

- [x] `uv run pytest
test/testcases/test_web_api/test_document_app/test_document_metadata.py::TestDocumentMetadataUnit::test_download_attachment_success_and_exception_unit`
- [x] Optional: `curl -sSI` against
`/api/v1/documents/<pdf_doc_id>/download` without `ext` and confirm
`Content-Type: application/pdf`

Fixes #15052.
2026-05-21 15:31:36 +08:00
dripsmvcp
440153c378 fix(api): check kb ownership in /dify/retrieval (#15028)
POST /api/v1/dify/retrieval resolved the caller via @apikey_required
(injecting tenant_id) but then fetched the requested knowledge_id with
no tenant filter and ran the full retrieval pipeline against
kb.tenant_id (the owner). Any valid Dify-compatible API key could
retrieve chunks from any tenant whose KB UUID was known. Adds the
missing ownership check.

## Root Cause
api/apps/sdk/dify_retrieval.py line 253:
KnowledgebaseService.get_by_id(kb_id) fetched the KB by id alone, then
the handler used kb.tenant_id (the OWNER) to build the embedding model
and call the retriever. The caller tenant_id was only used downstream at
line 278 for retrieval_by_children, well after cross-tenant data was
already retrieved.

grep confirmed there was no KnowledgebaseService.accessible call
anywhere in the handler.

## Fix
Two-line guard immediately after the existing get_by_id lookup,
mirroring the pattern PR #14749 lands for the sibling sdk/doc.py routes
(download, parse, stop_parsing, retrieval_test):

    e, kb = KnowledgebaseService.get_by_id(kb_id)
    if not e:
return build_error_result(message="Knowledgebase not found!",
code=RetCode.NOT_FOUND)
+   if not KnowledgebaseService.accessible(kb_id, tenant_id):
+ return build_error_result(message="No authorization.",
code=RetCode.AUTHENTICATION_ERROR)
    if kb.tenant_embd_id:
        ...

KnowledgebaseService.accessible already handles solo-tenant ownership,
team membership via TenantService.get_joined_tenants_by_user_id, and the
permission=ME distinction. No behavior change for legitimate callers;
cross-tenant callers now receive RetCode.AUTHENTICATION_ERROR (109).

## Test Plan
- [x] Regression test added:
test/unit_test/api/apps/sdk/test_dify_retrieval.py
- test_cross_tenant_request_is_rejected -- attacker tenant calling owner
tenant KB gets 109; retriever is not invoked
- test_same_tenant_request_succeeds -- owner tenant gets the records
back
- test_missing_knowledge_base_returns_not_found -- missing KB returns
404 BEFORE the access check fires (legit callers see the clearer
message)
- [x] All 3 tests pass after the fix
- [x] Cross-tenant test FAILS on pre-fix main (KeyError on result[code]
because handler leaks records dict instead of returning auth error)
- [x] ruff check clean on both changed files
- [x] No drive-by reformatting in dify_retrieval.py -- only the 2 added
lines

### Post-fix output

    test_cross_tenant_request_is_rejected           PASSED [ 33%]
    test_same_tenant_request_succeeds               PASSED [ 66%]
    test_missing_knowledge_base_returns_not_found   PASSED [100%]

============================== 3 passed in 0.04s
===============================

Closes #15027
2026-05-21 13:29:00 +08:00
Chan
0c93161a14 fix: prevent session user_id spoofing via request body (#15077)
### What problem does this PR solve?

Closes #15076 

Two endpoints in `api/apps/restful_apis/chat_api.py` accepted a
`user_id` field from the request body and used it directly when creating
a session:

```python
# before (vulnerable)
"user_id": req.get("user_id", current_user.id)          # create_session
conv = await _create_session_for_completion(chat_id, dia, req.get("user_id", current_user.id))  # session_completion
```

Any authenticated caller could supply an arbitrary `user_id` and have
the new session attributed to a different user — effectively spoofing
session ownership. Both call sites are now fixed to always use
`current_user.id`, which is set by the authentication middleware and
cannot be tampered with via the request payload.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

| File | Change |
|------|--------|
| `api/apps/restful_apis/chat_api.py` | Remove `req.get("user_id", ...)`
fallback in `create_session` and `session_completion`; always use
`current_user.id` |
|
`test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`
| Add `test_create_session_user_id_not_spoofable` and
`test_session_completion_user_id_not_spoofable` (both `@pytest.mark.p2`)
|

### Testing

Two new unit tests assert that a `user_id` value supplied in the request
body is silently ignored and the session is always owned by the
authenticated user:

```
test_create_session_user_id_not_spoofable
test_session_completion_user_id_not_spoofable
```

Run with:

```bash
uv run pytest test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py -k "spoofable" -v
```
2026-05-21 13:28:14 +08:00
web-dev0521
2d3a1a4483 feat(go-models): add Azure OpenAI model driver (#15022)
## What problem does this PR solve?

Closes #15021.

The Go model-provider layer had no support for **Azure OpenAI**. Azure
OpenAI is *not* a drop-in base-URL swap of the OpenAI driver — it
differs in authentication, endpoint structure, and how models are listed
— so it needs its own `ModelDriver` implementation.

## Type of change

- [x] New feature (non-breaking change which adds functionality)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 11:52:56 +08:00
Renzo
c7ac9b7171 Go: implement provider: GPUStack (chat) (#15024)
### What problem does this PR solve?

Fixes #15023

GPUStack is listed as unchecked in the Go-rewrite tracker #14736, and
`internal/service/llm.go:171` already classifies it as a self-deployed
provider alongside Ollama, Xinference, LocalAI, and LM Studio — but
`internal/entity/models/` had no `gpustack.go` driver, so the new Go API
server could not route GPUStack instances. This PR adds the chat surface
for GPUStack so it lines up with the existing self-hosted Go drivers.

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 11:49:18 +08:00
Renzo
394cd5d116 Go: implement Embed in Xinference driver (#14932)
## Summary

- Replaces the `"no such method"` stub on `XinferenceModel.Embed`
(`internal/entity/models/xinference.go`) with a real implementation
against Xinference's OpenAI-compatible `/v1/embeddings` endpoint.
- Adds the `"embedding": "v1/embeddings"` URL suffix to
`conf/models/xinference.json`.
- Mirrors the Python `XinferenceEmbed` class in
`rag/llm/embedding_model.py:407` for payload shape (OpenAI-compatible
`model + input` → `data[*].index + data[*].embedding`) and tolerates the
same no-auth default Xinference deployments use. Authorization is only
sent when a non-empty API key is configured, via the existing
`setXinferenceAuth` helper.
- Reuses the existing `normalizeXinferenceBaseURL` + `baseURLForRegion`
helpers so both `http://127.0.0.1:9997` and `http://127.0.0.1:9997/v1`
resolve to the same `/v1/embeddings` target without doubled `/v1`.
- Validates response indices — duplicate, missing, or out-of-range
`data[*].index` values fail with a clear error rather than silently
producing misaligned vectors.
- Returns `[]EmbeddingData` in original input order (placed by `Index`)
so downstream callers can index positionally without re-sorting.
- Forwards `EmbeddingConfig.Dimension` as `dimensions` when `> 0`,
matching the OpenAI cluster pattern.

Closes #14810

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-21 11:47:30 +08:00
Renzo
fec0b968e7 Go: implement Rerank in Novita driver (#15014)
### What problem does this PR solve?

Fixes #15012

The Novita Go driver landed in #14850 and shipped a stub `Rerank` method
that returned `"novita, no such method"`, so Novita could not be used as
a rerank provider in RAGFlow. This PR fills that gap, in the same way
#14895 filled the Embed gap on the same driver.

Novita exposes a public rerank endpoint at `POST
https://api.novita.ai/openai/v1/rerank` that accepts the
Cohere-compatible request shape (`{model, query, documents, top_n}`)
with `Authorization: Bearer <api_key>`. `baai/bge-reranker-v2-m3` is
documented in Novita's model library with a 1024-token limit.
2026-05-21 10:19:17 +08:00
Renzo
536ed07d27 Go: implement Rerank in Xinference driver (#15032)
### What problem does this PR solve?

Fixes #14816

The Xinference Go driver landed chat in #14938 and Embed is in review in
#14932, but `Rerank` shipped as a stub that returns `"xinference, no
such method"`. Tenants who launch a rerank model with `--model-type
rerank` on their Xinference instance cannot route it through the Go API
server. This PR fills the gap.

Xinference exposes an OpenAI-compatible REST API. The rerank endpoint is
at `POST <base>/v1/rerank` and accepts the Cohere-shaped body `{model,
query, documents, top_n}`, returning `{results: [{index,
relevance_score}]}` — the same wire shape used by the merged NVIDIA
(#14778), Aliyun (#14676), Gitee (#14656), ZhipuAI (#14608), Novita
(#15014), and LocalAI (#14813) Rerank implementations. Documented in
[Xinference rerank
docs](https://inference.readthedocs.io/en/v1.6.1/models/model_abilities/rerank.html);
the [builtin rerank model
catalog](https://inference.readthedocs.io/en/stable/models/builtin/rerank/)
lists `bge-reranker-base`, `bge-reranker-large`, `bge-reranker-v2-m3`,
and others.
2026-05-21 10:14:30 +08:00
sxxtony
63db30f0d9 Go: implement provider: n1n.ai (#15010)
### What problem does this PR solve?

Add a Go driver for **n1n.ai** (https://docs.n1n.ai), one of the
unchecked providers on the umbrella tracking issue #14736. n1n.ai is an
OpenAI-compatible aggregator hosting a 450+ model catalog (GPT, Claude,
Gemini, DeepSeek, Kimi, Qwen, embedding + reranker families) under
`https://api.n1n.ai/v1`.

Until this PR, a tenant who configured `n1n` as a model provider in the
Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

---------

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
2026-05-21 10:13:15 +08:00
Jack Storment
dc01e0e51c Go: implement Embed (embeddings) in TogetherAI driver (#15017)
### What problem does this PR solve?

Fixes #15015

The TogetherAI Go driver in `internal/entity/models/togetherai.go`
shipped a stub `Embed` method that returned `"TogetherAI, no such
method"`, so TogetherAI could not be used as an embedding provider in
RAGFlow. This PR fills that gap.

TogetherAI exposes a public OpenAI-compatible embeddings endpoint at
`POST https://api.together.ai/v1/embeddings` that accepts the standard
`{model, input}` shape with `Authorization: Bearer <api_key>` (confirmed
in TogetherAI's official docs:
https://docs.together.ai/docs/embeddings-overview). Documented embedding
models include `intfloat/multilingual-e5-large-instruct`,
`BAAI/bge-large-en-v1.5`, and `BAAI/bge-base-en-v1.5`.

### Changes

- `internal/entity/models/togetherai.go`: implement
`TogetherAIModel.Embed`.
- Validate inputs (api key, model name) and short-circuit on empty
texts.
  - Resolve region with the existing `baseURLForRegion` helper.
  - Build URL from `URLSuffix.Embedding`.
- Send `{model, input}` POST body, add `dimensions` when
`embeddingConfig.Dimension > 0` (matches the pattern in #14735).
  - Bearer auth + JSON content type, mirroring the chat path.
- Parse `{data: [{embedding, index}]}` and reorder by `index`, rejecting
out-of-range indices, duplicates, and missing entries so the output
always lines up with the input. Same shape as the merged Mistral,
Upstage, and Novita Embed implementations.
- `conf/models/togetherai.json`:
  - Add `"embedding": "embeddings"` to `url_suffix`.
- Add default embedding model entries for
`intfloat/multilingual-e5-large-instruct`, `BAAI/bge-large-en-v1.5`, and
`BAAI/bge-base-en-v1.5`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-20 20:48:44 +08:00
chanx
aa0a3d6988 Fix: The logs on the data source details page are not fully displayed. (#15056)
### What problem does this PR solve?

Fix: The logs on the data source details page are not fully displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-20 20:32:31 +08:00
qinling0210
dbef3e361f Update chunk/metadata cli (#15055)
### What problem does this PR solve?

Update chunk/metadata cli

### Type of change

- [ ] Refactoring
2026-05-20 20:32:06 +08:00
Jin Hai
90c76e73d0 Docs: Update version references to v0.25.5 in READMEs and docs (#15059)
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-20 20:05:45 +08:00
writinwaters
31bf29bc38 Docs: Initial draft of v0.25.5 release notes. (#15058)
### What problem does this PR solve?

Intial draft of v0.25.5 release notes.

### Type of change

- [x] Documentation Update
2026-05-20 19:44:40 +08:00
Haruko386
4a91ca5349 Go: implement provider: MinerU_Local (#15051)
### What problem does this PR solve?
1. Add model types when add model
---
```
RAGFlow(user)> add model 'pipeline' to provider 'mineru_local' instance 'test' with tokens 131072 doc_parse;
SUCCESS
```

2. implement provider: MinerU_Local
---
**Verified from CLI**
```
RAGFlow(user)> parse with 'pipeline@test@mineru_local' file './internal/test.pdf'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| c7260e31-b6e2-4b36-955d-e9c60510c669 |
+--------------------------------------+
RAGFlow(user)> show 'test@mineru_local' task 'c7260e31-b6e2-4b36-955d-e9c60510c669'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| content                                                                                                                                                                                                                                                          | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke Anton Obukhov Shengyu Huang Nando Metzger Rodrigo Caye Daudt Konrad Schindler Photogrammetry and Remote Sensing, ETH Zurich ¨

![](images/ae256101419715b544d13722...  | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-20 19:21:57 +08:00
Wang Qi
6ce76e6799 Fix discord async issue (#15054)
### What problem does this PR solve?

RuntimeError: Cannot run the event loop while another loop is running

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-20 19:21:19 +08:00
Magicbook1108
b28e134944 Feat: add local & ssh provider in admin panel (#15039)
### What problem does this PR solve?

Feat: add local & ssh provider in admin panel

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-20 16:56:20 +08:00
bitloi
6499bce2a6 fix: Langfuse chat observation (#15026)
### What problem does this PR solve?

Closes #15025

Langfuse-enabled `dialog_service.async_chat()` regressed to
`langfuse_tracer.start_generation(...)` after the earlier Langfuse v4
migration. Langfuse v4 uses `start_observation(as_type="generation")`,
so the remaining `start_generation` call can fail when chat tracing is
enabled.

This restores the migrated `start_observation(as_type="generation")`
call for chat observations while preserving the existing trace context,
model, input payload, and update/end flow. It also adds a regression
test with a fake Langfuse v4-style client that exposes
`start_observation()` but not `start_generation()`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Tests

- `.venv/bin/pytest
test/unit_test/api/db/services/test_dialog_service_final_answer.py -q`
- `.venv/bin/ruff check api/db/services/dialog_service.py
test/unit_test/api/db/services/test_dialog_service_final_answer.py`
2026-05-20 15:01:19 +08:00
balibabu
1ed8a118cf Fix: The folder tree menu for moving folders cannot be scrolled. (#15037)
### What problem does this PR solve?

Fix: The folder tree menu for moving folders cannot be scrolled.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-20 14:59:36 +08:00
bitloi
d69518ea42 fix(go): guard custom base URL driver creation (#15030)
### What problem does this PR solve?

Closes #15029.

Some custom `base_url` paths in `ModelProviderService` call
`NewInstance(newURL)` and then immediately invoke methods on the
returned driver. Several real Go model drivers still return `nil` from
`NewInstance`, so those paths can panic instead of returning a normal
error.

This PR:

- centralizes custom base URL driver creation in `model_service.go`
- skips request-local driver creation when `base_url` is blank or
whitespace
- preserves the existing region key behavior when building the
request-local base URL map
- returns a clear error when the provider driver is missing or
`NewInstance` returns `nil`
- routes list/check/task and active model paths through the guarded
helper
- adds focused unit coverage for empty-region preservation, regional
base URLs, blank base URLs, nil drivers, and nil `NewInstance` results

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Test plan

- [x] `git diff --check upstream/main...HEAD`
- [x] `/root/go/bin/gofmt -w internal/service/model_service.go
internal/service/model_service_test.go`
- [x] `GOPATH=/root/gopath GOTOOLCHAIN=local /root/go/bin/go test
./internal/service -run TestNewModelDriverForBaseURL -count=1 -vet=off`
- [x] `GOPATH=/root/gopath GOTOOLCHAIN=local /root/go/bin/go build
./internal/service/... ./internal/entity/models/...`

Note: the same targeted `go test` command without `-vet=off` is
currently blocked by an existing unrelated vet finding in
`internal/service/llm.go:355` (`non-constant format string in call to
fmt.Errorf`).
2026-05-20 14:58:20 +08:00
Idriss Sbaaoui
aea90f4e39 Feat: add new tests and tescases for restful api suite (#15038)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-20 14:56:55 +08:00
Haruko386
2836a934b5 Go: implement provider: 302.AI and JieKou-AI (#15034)
### What problem does this PR solve?

This PR implement implement provider 302.AI and JieKouAI

**The following functionalities are now supported:**

**302.ai**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] ASR
- [x] ListModels
- [x] Provider connection checking
- [x] Balance
- [x] Rerank
- [x] OCR
- [x] Doc Parse
- [x] Show task 
- [ ]  ~~List Tasks!~~
- [ ] TTS

**JieKouAI**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] Rerank
- [x] ListModels

**Verified examples from the CLI:**
```palintext
# jiekouAI

RAGFlow(user)> stream think chat with 'zai-org/glm-4.5@test@jiekouai' message 'Hi'
Thinking: Let me think about how to respond to this simple greeting. The user just said "Hi", which is a basic and friendly way to start a conversation. I should respond in a similarly warm and welcoming manner.First, I need to acknowledge their greeting and reciprocate with enthusiasm. Something like "Hello!" or "Hi there!" would work well to create a positive atmosphere right from the start.Next, I should make it clear that I'm ready to help. Since they haven't asked anything specific yet, I'll keep it open-ended and inviting. Perhaps offering assistance with a question or task would encourage them to engage further.I should also maintain a professional yet approachable tone. Being an AI assistant, I want to convey that I'm knowledgeable and capable, but also friendly and easy to talk to.Let me put this all together into a concise response. I'll start with a cheerful greeting, express my readiness to help, and finish with an open invitation for them to share what's on their mind. This should create a welcoming environment for whatever they want to discuss next.
Answer: ! I'm Claude, an AI assistant created by Anthropic. I'm here to help you with information, answer questions, or assist you with tasks. What can I help you with today?

RAGFlow(user)> think chat with 'zai-org/glm-4.5@test@jiekouai' message 'Hi'
Thinking: Let me consider how to respond to this greeting. The user initiated with a simple "Hi," so a friendly and open response would be most appropriate to encourage further conversation. I should maintain a welcoming tone while offering assistance.

The response should accomplish a few key things: return the greeting warmly, show openness to conversation, and offer specific ways I can help. This approach demonstrates both approachability and usefulness.

I'll start with a greeting in return, then express my availability to help, and finish by suggesting some areas where I can provide assistance. This creates a natural flow from acknowledgment to support.

It's important to keep the response concise but inviting. Since the user hasn't specified their needs yet, I'll present a few broad categories of assistance to spark their thinking about what they might want to discuss or ask about.

The response should end with an encouraging note that prompts them to share what's on their mind, keeping the conversational ball in their court while making it clear I'm ready to engage with whatever they need.
Answer: Hello! How can I help you today? Whether you have questions, need information, or just want to chat, I'm here to assist.

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'text-embedding-3-large@test@jiekouai' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 3072      | 0     |
| 3072      | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'baai/bge-reranker-v2-m3@test@jiekouai' top 3
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.9830034       |
| 2     | 0.06399203      |
| 1     | 0.04665664      |
+-------+-----------------+


# 302.ai

RAGFlow(user)> think chat with 'kimi-k2.6@test@302.ai' message 'who r u'
Thinking: The user is asking "who r u" which is a casual way of asking "who are you." I need to identify myself as an AI assistant created by Moonshot AI. I should be friendly, concise, and helpful.

Key points to include:
- I am Kimi, an AI assistant made by Moonshot AI
- I can help with various tasks like answering questions, writing, analysis, coding, etc.
- Keep it casual but informative since the user used "r u" (text speak)

I should not:
- Pretend to be human
- Claim to have personal experiences or emotions
- Be overly formal or robotic

Simple, friendly response is best.
Answer: I'm Kimi, an AI assistant made by Moonshot AI. I can help you with answering questions, writing, coding, analysis, or just chatting. What can I do for you?
Time: 17.687750

RAGFlow(user)> stream think chat with 'kimi-k2.6@test@302.ai' message 'who r u'
Thinking:  user asked "who r u" which is a casual way of asking "who are you." I should introduce myself as Kimi, an AI assistant developed by Moonshot AI. I need to be friendly, concise, and accurate. I should mention my capabilities briefly and keep the tone helpful. Since the user used casual text speak ("r u"), I can match that energy with a friendly but still informative tone.Key points:- I'm Kimi, an AI assistant made by Moonshot AI- I can help with various tasks like answering questions, writing, coding, analysis, etc.- Keep it brief but warm- Don't claim to be human- Don't over-explainDraft:"I'm Kimi, an AI assistant created by Moonshot AI. I can help with answering questions, writing, coding, analysis, brainstorming, and lots of other tasks. What can I do for you?"This is good - direct, accurate, and inviting.
Answer:  Kimi, an AI assistant made by Moonshot AI. I can help with answering questions, writing, coding, analysis, brainstorming, and lots of other stuff. What can I do for you?
Time: 14.912576

RAGFlow(user)> asr with 'whisper-v3-turbo@test@302.ai' audio './internal/test.wav' param ''
+---------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                |
+---------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the Commission to conclude that five shots may have been fired |
+---------------------------------------------------------------------------------------------------------------------+

RAGFlow(user)> ocr with 'mistral-ocr-latest@test@302.ai' file './internal/test.pdf'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke

Nando Metzger

Anton Obukhov

Rodrigo Caye Daudt

Shengyu Huang

Konrad Schindler

Photogrammetry and Remote Sensing, ETH Zürich

![img-0.jpeg](img-0.jpeg)
Figur...  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


RAGFlow(user)> parse with 'vlm@test@302.ai' file 'https://arxiv.org/pdf/2505.09358'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| 6de6eae6-c122-4b67-91e8-b061a0b8c087 |
+--------------------------------------+
RAGFlow(user)> show 'test@302.ai' task '6de6eae6-c122-4b67-91e8-b061a0b8c087'
+----------------------------------------------------------------------------+-------+
| content                                                                    | index |
+----------------------------------------------------------------------------+-------+
| https://file.302.ai/gpt/imgs/20260519/b340fdff4774699c287fe4ee4658b317.zip | 0     |
+----------------------------------------------------------------------------+-------+

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'jina-embeddings-v3@test@302.ai' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 1024      | 0     |
| 1024      | 1     |
+-----------+-------+
RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'jina-reranker-v2-base-multilingual@test@302.ai' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.74167407      |
| 2     | 0.18832397      |
| 1     | 0.15713684      |
+-------+-----------------+
```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-20 14:10:15 +08:00
qinling0210
77834870fc Refact functions in engine in GO (#14981)
### What problem does this PR solve?

Refact functions in engine in GO
### Type of change

- [x] Refactoring
2026-05-19 17:34:59 +08:00
Idriss Sbaaoui
6b2fcb4116 Feat: add new tests and tescases for restful api suite (#14996)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-19 17:17:31 +08:00
plind
6796a47b8d feat(sdk): make Begin inputs discoverable on Session.ask (#14842)
### What problem does this PR solve?

Closes #14751.

The user reported that after adding a variable (e.g. `key1`) to an
agent's **Begin** component, the Python SDK gave them no way to pass it:
their call `session.ask(question=user_question, stream=False)` had no
parameter for `key1`, and the `ask()` signature was just `(question,
stream, **kwargs)` with a docstring that only described streaming
behavior.

The functionality already works — `_ask_agent` does
`json_data.update(kwargs)` and the server reads `inputs` from the
request body at `agent_api.py:902`. The canonical shape is also in the
public API docs (`docs/references/python_api_reference.md:1817-1840`):

```python
session.ask(
    "",
    stream=False,
    inputs={"line_var": {"type": "line", "value": "I am line_var"}},
    return_trace=True,
)
```

But because `inputs`, `release`, and `return_trace` were hidden behind
`**kwargs`, they did not appear in IDE signature help, and the docstring
did not mention them. Users had no path from "I added a key in the UI"
to "I need to pass `inputs=...` with this exact shape."

This PR promotes the three most relevant Begin-related arguments to
named parameters and rewrites the docstring with a worked example.

### What this PR changes

- `sdk/python/ragflow_sdk/modules/session.py`:
- `Session.ask()` signature becomes `ask(question="", stream=False,
inputs=None, release=None, return_trace=None, **kwargs)`.
- These three new named params are forwarded into the existing `kwargs`
dict before dispatch, so the wire format and downstream behavior are
unchanged.
- Docstring rewritten in numpy style, including the structured `{"type":
..., "value": ...}` shape that the Begin component requires (see
`agent/component/begin.py:45-60`).

No backend changes. `**kwargs` is preserved for forward compatibility
with other body fields (`session_id`, `files`, `user_id`,
`custom_header`, …).

### Test plan

- [ ] `session.ask(question="hi", stream=False)` — existing call still
works
- [ ] `session.ask("", stream=False, inputs={"key1": {"type": "line",
"value": "v"}})` — Begin component receives `key1 = "v"`
- [ ] `session.ask("", stream=True, return_trace=True)` — streaming
response includes trace events
- [ ] IDE / `help(Session.ask)` now shows `inputs`, `release`,
`return_trace` with descriptions

### Type of change

- [x] Refactoring
- [x] Documentation Update
2026-05-19 16:14:57 +08:00
Rene Arredondo
f58e0b3eca Feat: VLM image descriptions in MinerU parser (#14869) (#14946)
## Summary

Closes #14869.

Adds VLM-based semantic descriptions to **image chunks produced by the
MinerU parser**, closing a long-standing parity gap with the deepdoc
parser's `VisionFigureParser`. A maintainer flagged this in #13342
("We may add the VLM enhancement to MinerU parser as well") and an
earlier proposal exists in #13824; this PR lands the change end-to-end
inside the existing parser plumbing.

## Why

Today the MinerU parser returns image chunks containing only the
native `image_caption` and `image_footnote` strings from MinerU's
JSON. When neither is present (or when both are sparse), the chunk
carries effectively no searchable content for the figure and
retrieval misses it entirely. Users who configured a local VLM
(reporter's case: Gemma-4-31B) had to post-process MinerU's
`tmp/*.json` themselves.

The deepdoc parser already solves this via
[`VisionFigureParser`](deepdoc/parser/figure_parser.py): when the
tenant has an `IMAGE2TEXT` model configured, each figure gets a
semantic description merged into its chunk. This PR brings the same
behavior to MinerU.

## What changed

### `deepdoc/parser/mineru_parser.py`

- **New method `_enhance_images_with_vlm(outputs, vision_model,
callback=None)`** —
  collects every `IMAGE` block with a readable `img_path`, runs
  `rag.app.picture.vision_llm_chunk` in a 10-worker
  `ThreadPoolExecutor` using the existing
  `vision_llm_figure_describe_prompt`, and writes the result back as
  `vlm_description`. Per-image failures are logged and skipped — they
  never abort the run.
- **`_transfer_to_sections` (IMAGE branch)** — folds
  `vlm_description` into the section text alongside caption +
  footnote, so the description becomes part of the chunk and is
  searchable / retrievable.
- **`parse_pdf`** — after `_read_output`, calls
  `_enhance_images_with_vlm(outputs, vision_model, callback=callback)`
  when a `vision_model` kwarg is supplied. Wrapped in `try / except`
  so a VLM outage cannot break parsing.

### `rag/app/naive.py` (`by_mineru`)

After successfully resolving the MinerU OCR parser, also resolves the
tenant's default `LLMType.IMAGE2TEXT` model via
`get_tenant_default_model_by_type`, wraps it in an `LLMBundle`, and
injects it as `kwargs["vision_model"]` before delegating to
`parse_pdf`.

## Behavior

| Tenant config | Behavior |
|---|---|
| `IMAGE2TEXT` model configured | MinerU image chunks contain `caption +
footnote + VLM description`. Retrieval against figures now actually
works. |
| No `IMAGE2TEXT` model configured | Exact same output as today (caption
+ footnote only). Lookup fails silently with an info log; no error, no
regression. |
| VLM call fails for a single image | That image silently falls back to
caption + footnote; other images proceed. |
| Caller already passes `vision_model` in kwargs | We don't override it
— `if "vision_model" not in kwargs` guards the lookup. |

## Files

- `deepdoc/parser/mineru_parser.py` (+56)
- `rag/app/naive.py` (+13)
2026-05-19 16:08:10 +08:00
Idriss Sbaaoui
95b56e73f2 Feat: add new tests and tescases for restful api suite (#14993)
### What problem does this PR solve?

extend restful api suite

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-19 15:43:15 +08:00
Rene Arredondo
ce3402cbb9 Fix: restore saved api_key fallback in add_llm (#14921) (#14941)
## Summary

Closes #14921.

Reconfiguring an existing LLM provider to enable **tool call** or
**vision** fails with `Your API key is invalid. Fail to access model.`
even when the saved API key is correct. The most visible report is
VLLM ("Cannot add vllm model" once `--enable-auto-tool-choice` /
vision is toggled on), but the bug applies to every provider whose
api_key field stays blank in edit mode.

## Root cause

PR #14885 ("Fix: llm add api key overridden") removed the existing-key
lookup in `api/apps/llm_app.py::add_llm`. The intent was correct —
stop the saved key from clobbering a user-provided new one — but the
removal was unconditional, so the edit path now has no fallback at all:

1. `web/src/pages/user-setting/setting-model/hooks.tsx:230` sets the
   initial `api_key` form value to `''` in edit mode (the real key is
   never returned to the browser).
2. The user toggles `is_tools` / `vision` without retyping the key.
3. `hooks.tsx:183-185` strips the empty `api_key` from the payload.
4. `add_llm` defaults to the placeholder `"x"`
   (`api/apps/llm_app.py:182`).
5. The upstream provider rejects `"x"` with `Your API key is invalid`.

## Fix

Restore the fallback **narrowly**, before any factory-specific handler
runs:

- If `req.get("api_key") is None`, look up the tenant's existing record
  (using the correctly suffixed `llm_name` for VLLM /
  OpenAI-API-Compatible / LocalAI / HuggingFace).
- Decode the saved blob with `_decode_api_key_config` and write **only
  the decoded `api_key` string** back into `req["api_key"]`. Never use
  the raw JSON payload — that was the exact thing PR #14885 was trying
  to avoid.
- When the user **does** type a new key, `req.get("api_key")` is not
  `None` and the fallback is skipped, so PR #14885's fix is preserved.

| Scenario | Before this PR | After this PR |
|---|---|---|
| Plain factory (VLLM, Ollama, …), retype key | OK | OK |
| Plain factory, blank key in edit (the bug) | Fails with "API key is
invalid" | Recovers saved key, validates against the real one |
| OpenRouter / Bedrock, change `provider_order` only | Fails |
`apikey_json([...])` rebuilds the JSON with saved `api_key` + new field
|
| User clears the form and types a brand-new key | OK (key replaced) |
OK (key replaced — fallback skipped) |

## Files changed

- `api/apps/llm_app.py` — restored fallback in `add_llm` (no other call
sites touched).

## Test plan

- [ ] Add a VLLM chat model with a valid api_key, no toggles → save
succeeds.
- [ ] Edit the same model, toggle **tool call** on, leave api_key blank
      → save succeeds, validation runs against the saved key.
- [ ] Edit again, toggle **vision** on (model_type → `image2text`),
      leave api_key blank → save succeeds.
- [ ] Edit again and **type a new api_key** → the new key replaces the
      saved one (`is None` check skips the fallback). Verify via the DB
      row or by deliberately typing a wrong key and observing the
      validation failure.
- [ ] Repeat the blank-key edit with **OpenRouter**, changing only
      `provider_order` → resulting api_key JSON contains the saved
      `api_key` and the new `provider_order`.
- [ ] First-time add of a new model name → no existing record, fallback
      no-ops, behaves as before.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-19 15:32:09 +08:00
tmimmanuel
243d9ed281 Add TogetherAI chat provider (#14957)
## What
- Add TogetherAI as a chat provider backed by its OpenAI-compatible
`/v1/chat/completions` API
- Register TogetherAI in the Go model factory and provider config
- Support non-streaming chat, SSE streaming chat, model listing, and
connection checks

## Notes
- Uses the current TogetherAI OpenAI-compatible base URL
`https://api.together.ai/v1`
- Forwards documented chat parameters from `ChatConfig`: `max_tokens`,
`temperature`, `top_p`, `stop`, and GPT-OSS `reasoning_effort`
- Routes Together reasoning traces from `reasoning` /
`reasoning_content` into `ReasonContent`

## Tests
- `go test -vet=off -run TestTogetherAI -count=1
./internal/entity/models`
- `go test -vet=off -count=1 ./internal/entity/models`

Refs #14736
2026-05-19 15:10:42 +08:00
tmimmanuel
09a06f1b00 Go: implement provider: Xinference (#14938)
### What problem does this PR solve?

Closes #14808.

Adds a Go model driver for Xinference so self-hosted Xinference chat
models can be used through the Go provider layer instead of falling
through to the dummy driver. Xinference exposes an OpenAI-compatible API
under `/v1`; the driver accepts either a root endpoint such as
`http://127.0.0.1:9997` or an OpenAI-compatible endpoint such as
`http://127.0.0.1:9997/v1` and normalizes it before calling chat or
model-listing routes.

### What is changed?

- Add `internal/entity/models/xinference.go` implementing `ModelDriver`
for Xinference chat.
- Route provider name `xinference` in
`internal/entity/models/factory.go`.
- Add `conf/models/xinference.json` as a local provider config.
- Add focused unit tests in `internal/entity/models/xinference_test.go`.

Initial method coverage:

- `ChatWithMessages`: POST `/v1/chat/completions`.
- `ChatStreamlyWithSender`: SSE streaming from `/v1/chat/completions`.
- `ListModels`: GET `/v1/models`.
- `CheckConnection`: lightweight `ListModels` probe.
- Optional auth: send `Authorization: Bearer <api_key>` only when a
non-empty key is configured, matching Xinference no-auth and
auth-enabled deployments.
- `Balance`, `Embed`, `Rerank`, ASR, TTS, and OCR return `no such
method` for this initial chat-provider PR.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Bug Fix (non-breaking change which fixes an issue)

### Tests

- `go test -vet=off -run TestXinference -count=1
./internal/entity/models/...`
- `go test -vet=off -count=1 ./internal/entity/models/...`

### References

- Xinference docs:
https://inference.readthedocs.io/zh-cn/latest/index.html
- OpenAI-compatible chat usage:
https://inference.readthedocs.io/zh-cn/latest/getting_started/using_xinference.html
- API key auth:
https://inference.readthedocs.io/zh-cn/latest/user_guide/auth_system.html

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-19 15:10:13 +08:00
plind
7edabdf7c3 fix(retrieval): keep manual metadata filter reusable inside Iteration (#14849)
## What problem does this PR solve?

Closes #12582.

When a Retrieval component sits inside an Iteration with a **manual**
metadata filter that references the iteration variable (e.g.
`{IterationItem:abc@item}`), every iteration reuses the value resolved
on the **first** pass.

Root cause: [`_resolve_manual_filter` in
`agent/tools/retrieval.py`](https://github.com/infiniflow/ragflow/blob/main/agent/tools/retrieval.py#L144-L171)
mutated `flt["value"]` in place. The `filters` list passed in is the
live `self._param.meta_data_filter["manual"]` (see
[`apply_meta_data_filter` in
`common/metadata_utils.py:257-261`](https://github.com/infiniflow/ragflow/blob/main/common/metadata_utils.py#L257-L261)),
so after the first iteration the param dict permanently held the
resolved string instead of the original variable reference.

```text
iter #1: flt["value"] = "{IterationItem:abc@item}"  →  resolved to "AI"
         after mutation: flt["value"] = "AI"        ← written back into _param

iter #2: flt["value"] = "AI"                         ← no {…} matches
         retrieval keeps filtering by "AI" forever
```

This PR returns a shallow copy with the resolved value instead, leaving
the original filter (and its variable reference) intact for the next
iteration.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## Test plan

- [ ] Build an agent: `Agent (structured output → list of areas) →
Iteration → Retrieval (manual filter: Area = {IterationItem/Item}) →
Message`. Run with a multi-area query and confirm each iteration's
Retrieval result matches its own item, not the first item.
- [ ] Regression: Retrieval with a manual metadata filter outside an
Iteration still resolves the variable correctly on each request.
- [ ] Regression: Retrieval with no metadata filter and with `auto` /
`semi_auto` filters behave unchanged.
2026-05-19 15:08:31 +08:00
plind
f169ab4b39 feat(tts): cache synthesized speech in Redis to avoid redundant calls (#14851)
## What problem does this PR solve?

Closes #12017.

TTS output is deterministic for a given `(model, text)` pair, so
re-running the same text through the same TTS model produces the same
bytes — yet `Canvas.tts` and `dialog_service.tts` re-synthesized on
every request. That's slow and wastes provider quota whenever the same
assistant response is replayed, shared across users, or repeated within
a session.

### Change

New helper `rag/utils/tts_cache.py` with `synthesize_with_cache(tts_mdl,
cleaned_text)`:

- **Key:** `tts:cache:{model_id}:{sha256(text)}` — separate namespace
per model, identical cleaned text reuses a single entry across both call
sites.
- **Value:** the hex-encoded audio blob both call sites already
returned. No format change for downstream consumers.
- **TTL:** 7 days by default, configurable via
`RAGFLOW_TTS_CACHE_TTL_SECONDS`.
- **Failure modes:** a Redis hiccup falls back to direct synthesis; a
failed synthesis still returns `None` (existing contract preserved).


[`Canvas.tts`](https://github.com/infiniflow/ragflow/blob/main/agent/canvas.py#L683-L724)
and
[`dialog_service.tts`](https://github.com/infiniflow/ragflow/blob/main/api/db/services/dialog_service.py#L1367-L1380)
now route through the helper; the per-file bytes-accumulation/hex-encode
loop has been removed in favor of one shared implementation.

## Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Test plan

- [ ] **Cache hit, chat path:** Configure a dialog with TTS enabled, ask
the same question twice with `stream=false`. Verify the second response
returns the same `audio_binary` and that the second invocation doesn't
hit the TTS provider (e.g., observe provider-side logs / usage counters;
check no `LLMBundle.tts can't update token usage` log line on the second
run).
- [ ] **Cache hit, agent path:** Same exercise via a Conversational
Agent that includes a Message component playing back the answer.
- [ ] **Cache isolation per model:** Switch tenant's `tts_id` between
two models, run the same text against each — confirm the second model's
first synthesis still happens (no cross-model hits).
- [ ] **TTL override:** Set `RAGFLOW_TTS_CACHE_TTL_SECONDS=120`, confirm
the entry expires after 2 minutes.
- [ ] **Redis unavailable:** Stop Redis (or break the connection).
Verify the TTS endpoint still works — synthesis falls back to direct
calls, with a `TTS cache lookup failed` / `TTS cache store failed`
warning logged.
- [ ] **Failure path:** Configure a TTS model with an invalid API key,
ensure the response still returns successfully with `audio_binary=None`
(no regression vs. current behavior).
2026-05-19 14:20:40 +08:00
OrbisAI Security
f17a66d4f0 fix: the opencc c library uses fgets() to read dicti... in text.c (#13970)
## Summary
Fix critical severity security issue in
`internal/cpp/opencc/dictionary/text.c`.

## Vulnerability
| Field | Value |
|-------|-------|
| **ID** | V-001 |
| **Severity** | CRITICAL |
| **Scanner** | multi_agent_ai |
| **Rule** | `V-001` |
| **File** | `internal/cpp/opencc/dictionary/text.c:107` |

**Description**: The OpenCC C library uses fgets() to read dictionary
and configuration files without proper bounds validation on subsequent
buffer operations. While fgets() itself is bounds-checked, the sprintf()
call at config_reader.c:174 constructs file paths by concatenating
home_path and filename without verifying the result fits in pkg_filename
buffer. An attacker providing malformed OpenCC configuration files with
excessively long path components can overflow the fixed-size buffer,
overwriting adjacent memory including return addresses and function
pointers.

## Changes
- `internal/cpp/opencc/config_reader.c`
- `internal/cpp/opencc/dictionary/text.c`
- `internal/cpp/opencc/utils.c`

## Verification
- [x] Build passes
- [x] Scanner re-scan confirms fix
- [x] LLM code review passed

---
*Automated security fix by [OrbisAI Security](https://orbisappsec.com)*


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved error detection and handling for malformed configuration and
dictionary entries during file parsing.
* Enhanced memory cleanup in error recovery paths to prevent potential
issues.
* Strengthened robustness of string operations and buffer handling
throughout the library.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Ubuntu <ubuntu@ip-172-31-32-15.us-west-2.compute.internal>
2026-05-19 13:55:33 +08:00
刘康伟
c6e3a2e713 Fix: MinerU vlm-http-client backend output file detection (#14240)
## Problem
When using MinerU with `vlm-http-client` backend, the parser fails to
find the output files because they are located in a `vlm/` subdirectory,
but the `_read_output`
  method doesn't check this location.

  ## Error Message
  [ERROR]MinerU not found.
  [MinerU] Missing output file, tried: ...

  ## Root Cause
The MinerU API with `vlm-http-client` backend returns output files in
the following structure:
  output_dir/
    vlm/
      filename_content_list.json
      filename.md
      images/

  However, the `_read_output` method in `mineru_parser.py` only checks:
  1. `output_dir/filename_content_list.json`
  2. `output_dir/sanitized_filename_content_list.json`
3. `output_dir/sanitized_filename/sanitized_filename_content_list.json`

  It doesn't check the `vlm/` subdirectory.

  ## Solution
  Added two additional fallback paths to check the `vlm/` subdirectory:
  - `output_dir/vlm/filename_content_list.json`
  - `output_dir/vlm/sanitized_filename_content_list.json`

  ## Testing
Tested with MinerU API using `vlm-http-client` backend. The parser now
successfully finds and processes the output files.

  ## Related
  This issue occurs specifically when using:
  - MinerU backend: `vlm-http-client`
  - MinerU server URL configured for remote vLLM inference
2026-05-19 12:28:31 +08:00
buua436
87d22a4415 Fix: agent session log message (#14991)
### What problem does this PR solve?

agent session log message
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-19 12:00:02 +08:00
tmimmanuel
4c9529ef36 Add Replicate chat provider (#14958)
## What
- Add Replicate as a chat provider backed by the documented predictions
API
- Register Replicate in the Go model factory and provider config
- Support non-streaming chat through sync predictions, polling fallback,
streaming through `urls.stream`, model listing, and connection checks

## Notes
- Uses `POST /v1/predictions` with Replicate model identifiers in
`version`, which supports official and community model identifiers
- Maps RAGFlow messages into Replicate prompt-shaped inputs (`prompt`,
optional `system_prompt`) and forwards common documented LLM inputs:
`max_new_tokens`, `temperature`, `top_p`
- Preserves whitespace in SSE output chunks and emits RAGFlow `[DONE]`
at stream completion

## Tests
- `go test -vet=off -run TestReplicate -count=1
./internal/entity/models`
- `go test -vet=off -count=1 ./internal/entity/models`

Refs #14736
2026-05-19 11:10:36 +08:00
Haruko386
db9e782747 Go: implement provider: MinerU (#14990)
### What problem does this PR solve?

Implement MinerU Provider

**The following functionalities are now supported:**

**MinerU**
----
- [x] Parse file
- [x] Show task
- [ ] ~~List tasks~~

**Verified examples from the CLI:**
```plaintext
RAGFlow(user)> parse with 'vlm@test@mineru' file 'https://arxiv.org/pdf/2505.09358'
+--------------------------------------+
| task_id                              |
+--------------------------------------+
| 142ac8ea-d9d0-4a68-a2d1-d3af67635dc9 |
+--------------------------------------+

RAGFlow(user)> show 'test@mineru' task '142ac8ea-d9d0-4a68-a2d1-d3af67635dc9'
+--------------------------------------------+-------+
| content                                    | index |
+--------------------------------------------+-------+
| Task is running... Progress: 17 / 18 pages | 0     |
+--------------------------------------------+-------+

RAGFlow(user)> show 'test@mineru' task '142ac8ea-d9d0-4a68-a2d1-d3af67635dc9'
+--------------------------------------------------------------------------------------------+-------+
| content                                                                                    | index |
+--------------------------------------------------------------------------------------------+-------+
| https://cdn-mineru.openxlab.org.cn/pdf/2026-05-18/142ac8ea-d9d0-4a68-a2d1-d3af67635dc9.zip | 0     |
+--------------------------------------------------------------------------------------------+-------+

```


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-19 10:49:33 +08:00
kingloon
525a87be0f Misc: fix some typos (#14987)
### What problem does this PR solve?

Fix minor code quality issues:

1. Fix typo in assertion error message: "Can't fine" → "Can't find"
2. Remove duplicate line in common/connection_utils.py

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-19 10:47:06 +08:00
jony376
198f3c4b9a Fix: validate memory tenant model IDs on update and enforce tenant scope in memory pipeline (#14923)
### Related issues

Closes #14922

### What problem does this PR solve?

`POST /memories` already resolves `tenant_llm_id` and `tenant_embd_id`
through `ensure_tenant_model_id_for_params`, but `PUT
/memories/<memory_id>` accepted client-supplied `tenant_llm_id` /
`tenant_embd_id` without checking that those `tenant_llm` rows belong to
the memory owner’s tenant. A caller could persist another tenant’s row
IDs and later trigger extraction or embedding that loaded foreign model
credentials via `get_model_config_by_id(tenant_model_id)` with no tenant
allow-list.

This change aligns the update path with create: updates that change
models must go through `llm_id` / `embd_id` and
`ensure_tenant_model_id_for_params` scoped to the **memory’s**
`tenant_id` (not only the current user, so team-access cases stay
correct). Direct `tenant_*` fields in the body without `llm_id` /
`embd_id` are rejected. As defense in depth, `memory_message_service`
passes `allowed_tenant_ids` / `requester_tenant_id` into
`get_model_config_by_id` for LLM and embedding resolution so mismatched
IDs cannot be used even if bad data existed. A regression test rejects
payloads that set only `tenant_llm_id` / `tenant_embd_id`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-19 10:11:46 +08:00
Magicbook1108
b69a6a5d80 Feat: full optimization on connector dashboard (#14979)
### What problem does this PR solve?

This PR improves the connector dashboard task management experience and
adds better visibility into connector execution logs.

### Overview:

#### Before
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/e4a8ed6f-2e18-4f0f-8528-41a514550052"
/>

#### Now:
<img width="700" alt="Screenshot from 2026-05-18 16-31-30"
src="https://github.com/user-attachments/assets/d4ca193b-847a-49ae-9e4f-5fbca60ea627"
/>

### 1. Add a new logging page to the connector dashboard

A new logging page has been added so users can view connector task
execution logs directly from the connector dashboard.

### 2. Merge the Resume button into Confirm

The separate **Resume** button has been removed. The **Confirm** button
now represents different actions depending on the current task state:

- **Save**: Save form changes and reschedule tasks.
- **Stop**: Cancel currently scheduled or running tasks.
- **Resume**: Create new scheduled tasks after the previous tasks have
been stopped.
- **Start**: Start tasks when no task has been started yet.

### 3. Separate syncing and pruning tasks

Connector tasks are now separated into **syncing** and **pruning**.

Pruning is controlled by the **Sync deleted files** option:

- When **Sync deleted files** is disabled, only syncing tasks are shown.
- When **Sync deleted files** is enabled, both syncing and pruning tasks
are shown.

**Now: Sync deleted files disabled**

<img width="700" alt="Sync deleted files disabled"
src="https://github.com/user-attachments/assets/dbd9232e-614a-407f-a0b1-c109e5fa567d"
/>

**Now: Sync deleted files enabled**

<img width="700" alt="Sync deleted files enabled"
src="https://github.com/user-attachments/assets/1f527f48-ccb3-4ee8-97ca-086891489296"
/>

### 4. Update logs in backend

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/10a95a3f-98c1-4e67-8afa-ddf6cda5b0b2"
/>

### 5. Remove connector resume API

- Removed: `POST /v1/connectors/<connector_id>/resume`
- Replaced by: `PATCH /v1/connectors/<connector_id>`


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-19 10:07:11 +08:00
buua436
41a9fc0030 Go: add dataset graph api (#14984)
### What problem does this PR solve?

add dataset graph api

### Type of change

- [x] Refactoring
2026-05-18 20:02:53 +08:00
buua436
d7fb4bdb4e Go: align document list response (#14982)
### What problem does this PR solve?

align document list response

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 20:00:11 +08:00
buua436
3290257014 Go: fix forgetting policy validation and fix memory update diff checks (#14976)
### What problem does this PR solve?

 fix forgetting policy validation and fix memory update diff checks

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 19:21:47 +08:00
Jake Armstrong
93d3deb5e4 Fix admin CLI system variable commands (#14956)
## What

Fixes #12409.

Implements admin CLI support for:

- `list vars;`
- `show var <name-or-prefix>;`
- `set var <name> <value>;`

## Changes

- Wire Go CLI variable commands to the admin API.
- Support integer and quoted string values in `SET VAR`.
- Return variable rows as `data_type`, `name`, `setting_type`, and
`value`.
- Add exact-name lookup with prefix fallback for `SHOW VAR`.
- Validate values by stored data type: `string`, `integer`, `bool`, and
`json`.
- Keep the legacy Python admin CLI/server behavior aligned.
- Update admin CLI docs and add focused tests.

## Verification

- `go test -count=1 ./internal/cli`
- `python3.12 -m py_compile admin/server/services.py
admin/server/routes.py api/db/services/system_settings_service.py
admin/client/parser.py admin/client/ragflow_client.py`
- Python admin CLI parser smoke test for `SET VAR`, quoted values, `SHOW
VAR`, and `LIST VARS`.
- Attempted `./run_go_tests.sh`; local environment is missing native
tokenizer/linker artifacts:
  - `internal/cpp/cmake-build-release/librag_tokenizer_c_api.a`
  - `-lstdc++`

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-18 19:08:45 +08:00
Wang Qi
732e4741c4 Bugfix: fix tag show (#14980)
### What problem does this PR solve?

Bugfix: fix tag show

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 18:55:01 +08:00
Hamza Amin Khokhar
2dbe3b8a62 fix: metadata_condition returning all docs when filter matches nothing (#14967)
### What problem does this PR solve?

When _parse_doc_id_filter_with_metadata returns [], the empty list is
falsy so the WHERE id IN (...) clause was silently skipped, causing the
full dataset to be returned instead of an empty result.

Change `if doc_ids:` to `if doc_ids is not None:` in both get_list() and
get_by_kb_id() to distinguish between no filter (None) and a filter that
matched zero documents ([]).

Fixes #14962

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 18:54:30 +08:00
Haruko386
92145dc764 Go: implement provider: DeepInfra, XunFei (#14978)
### What problem does this PR solve?

This PR implement implement provider  and Mistral, DeepInfra, XunFei

**The following functionalities are now supported:**

**DeepInfra**

- [x] chat / think chat / stream chat / stream think chat
- [x] Embedding
- [x] ASR
- [x] TTS
- [x] ListModels
- [x] Provider connection checking
- [x] Balance
- [ ] ~~Rerank~~

**XunFei**

- [x] chat / think chat / stream chat / stream think chat

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-18 16:57:42 +08:00
buua436
b8ac997606 Go: add restful api route aliases (#14977)
### What problem does this PR solve?

add restful api route aliases

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-18 16:57:14 +08:00
Wang Qi
13b422037f Refactor: enhance graphrag - part 2 (#14972)
### What problem does this PR solve?
1. expose batch_chunk_token_size for configuration
2. retrieve chunks when build subgraph for the doc, not retreive all
docs chunks at the begining
3. get all chunks for a document, used to be hard coded 10000
4. delete not used method run_graphrag

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

Follow on: #14617
2026-05-18 16:10:21 +08:00
dev
b12eaee38b fix(api): enforce tenant access for connector routes (#14747)
### What problem does this PR solve?

Fixes #14746.

Adds tenant access checks for connector-by-id REST routes before reading
connector details, mutating connector config/status, deleting
connectors, rebuilding, or listing sync logs. Unauthorized callers now
receive `RetCode.AUTHENTICATION_ERROR` with `No authorization.` without
reaching the connector/log mutation paths.

Validation:
- `python3 -m pytest
--confcutdir=test/testcases/test_web_api/test_connector_app
test/testcases/test_web_api/test_connector_app/test_connector_routes_unit.py`
- `uvx ruff check api/apps/restful_apis/connector_api.py
api/db/services/connector_service.py
test/testcases/test_web_api/test_connector_app/test_connector_routes_unit.py`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: dev111-actor <dev111-actor@users.noreply.github.com>
2026-05-18 16:09:26 +08:00
Wang Qi
56d73d0c2c Refactor: speed up ragflow server, save startup memory (#14973)
### What problem does this PR solve?

Refactor: speed up ragflow server, save startup memory, saved 200MiB,
and 5-9 seconds start time.

##### Before
1241292  |   |           \_ python3 api/ragflow_server.py
RAGFlow server is ready after 25.61845850944519s initialization.

##### After
1019968  |   |           \_ python3 api/ragflow_server.py
RAGFlow server is ready after 16.205134391784668s initialization.

### Type of change

- [x] Refactoring
2026-05-18 15:55:59 +08:00
buua436
b40b0bf996 Go: fix siliconflow embedding response (#14975)
### What problem does this PR solve?

fix siliconflow embedding response

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 15:07:07 +08:00
dale053
fe82a96193 Fix: add SSRF guard for agent test_db_connection endpoint (#14860)
### What problem does this PR solve?

Closes #14858

The `test_db_connection` endpoint in the agent API accepts a
user-supplied `host` and connects to it directly via database drivers
(MySQL/PostgreSQL) without any validation. This allows an attacker to
probe internal network addresses (e.g. `127.0.0.1`, `10.x.x.x`,
link-local, etc.) through the server — a classic Server-Side Request
Forgery (SSRF) vulnerability.

This PR adds an SSRF guard that resolves the host and rejects any
address that is not globally routable before the database connection is
attempted.

**Changes:**
- **`common/ssrf_guard.py`** — Added `assert_host_is_safe()`, a
host-level counterpart of the existing `assert_url_is_safe()`, designed
for non-HTTP protocols (database drivers) where there is no URL to
parse.
- **`api/apps/restful_apis/agent_api.py`** — Call
`assert_host_is_safe(req["host"])` at the top of `test_db_connection` so
that non-public hosts are rejected early with a clear error message.

Fixes #14858

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-18 14:32:44 +08:00
tmimmanuel
b09da6e347 Go: implement provider: CometAPI (#14930)
### What problem does this PR solve?

Adds the Go model provider driver for CometAPI, which is listed as
unchecked in the Go provider tracking issue #14736 and requested in
#14804. Without this, the Go layer falls back to the dummy driver for
the `cometapi` provider.

Fixes #14804

### What this PR includes

- New `internal/entity/models/cometapi.go` implementing `ModelDriver`
for CometAPI.
- New `conf/models/cometapi.json` with CometAPI base URLs and
representative chat / embedding models from the public catalog.
- `factory.go`: route `"cometapi"` to `NewCometAPIModel`.
- Unit tests in `internal/entity/models/cometapi_test.go`.

### Method coverage

- `ChatWithMessages`: `POST /v1/chat/completions`.
- `ChatStreamlyWithSender`: SSE streaming on the same endpoint.
- `Embed`: `POST /v1/embeddings`, including optional `dimensions`.
- `ListModels`: `GET /api/models` public catalog.
- `Balance`: `GET https://query.cometapi.com/user/quota?key=...`.
- `CheckConnection`: delegates to the quota query to verify the key.
- `Rerank`, ASR, TTS, OCR: return `no such method` for now.

No ModelDriver interface change. No new dependencies.

### How was this tested?

```bash
go test -vet=off -run TestCometAPI -count=1 ./internal/entity/models/...
go test -vet=off -count=1 ./internal/entity/models/...
```

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Signed-off-by: majiayu000 <1835304752@qq.com>
Co-authored-by: 加帆 <Jiafan@users.noreply.github.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: bulexu <baiheng527@gmail.com>
Co-authored-by: xubh <xubh@wikiflyer.cn>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Carve_ <75568342+Rynzie02@users.noreply.github.com>
Co-authored-by: Paul Y Hui <paulhui@seismic.com>
Co-authored-by: LIRUI YU <128563231+LiruiYu33@users.noreply.github.com>
Co-authored-by: yun.kou <koopking@gmail.com>
Co-authored-by: Yun.kou <yunkou@deepglint.com>
Co-authored-by: Ahmad Intisar <168020872+ahmadintisar@users.noreply.github.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: Syed Shahmeer Ali <syedshahmeerali196@gmail.com>
Co-authored-by: Octopus <liyuan851277048@icloud.com>
Co-authored-by: lif <1835304752@qq.com>
2026-05-18 14:31:16 +08:00
qinling0210
f1d2383572 Push metadata filters down to Infinity (#14974)
### What problem does this PR solve?

Push metadata filters down to Infinity

### Type of change

- [x] Refactoring
2026-05-18 14:22:04 +08:00
Kevin Hu
7cdc74bbe5 Refactor: Drop the vector fetch for ES (#14970)
## Summary
- Stop pulling chunk vectors (`q_*_vec`) back from Elasticsearch in the
main retrieval path. ES already knows them; shipping them was pure
bandwidth/memory overhead.
- Recover the per-chunk cosine similarity via a second KNN-only ES call
filtered by the candidate chunk ids. The new `_score` is merged with
locally computed term similarity using the user-configured
`vector_similarity_weight`.
- Lazily fetch the chunk embedding only for the chunks
`insert_citations` actually needs.

## Details
**`rag/nlp/search.py`**
- `Dealer.search`: no longer appends `q_*_vec` to the ES select list.
OceanBase still gets it (its rerank path is unchanged).
- New `Dealer._knn_scores(sres, idx_names, kb_ids)`: a `MatchDenseExpr`
over the cached query vector filtered by `id IN sres.ids`, returning
`{chunk_id: cosine_score}` via ES `_score`.
- New `Dealer.rerank_with_knn(...)`: term similarity from
`qryr.token_similarity` plus the ES-supplied KNN score, combined with
`tkweight`/`vtweight` and the existing rank-feature bonus.
- New `Dealer.fetch_chunk_vectors(chunk_ids, tenant_ids, kb_ids, dim)`:
on-demand vector fetch for citation use.
- `Dealer.retrieval` routes Infinity → unchanged, OceanBase → existing
local `rerank`, ES → new KNN-score path.

**`common/doc_store/es_conn_base.py`**
- New `get_scores(res)` helper returning `{_id: _score}` directly from
hit headers (ES doesn't surface `_score` through `get_fields`).

**`api/db/services/dialog_service.py`**
- New top-level `_hydrate_chunk_vectors(...)` helper. On ES it
back-fills `ck["vector"]` from `fetch_chunk_vectors` right before
`insert_citations`. No-op on Infinity / OB (their chunks already carry
vectors).
- Both `decorate_answer` closures became `async` and are `await`-ed at
all call sites in `async_chat` and `async_ask`.

## Backend behavior
| Backend | Returns chunk vec in main search | Sim source | Vectors for
citations |
|---|---|---|---|
| ES | No | second KNN call (`_score`) merged with term sim | fetched on
demand |
| Infinity | No (unchanged) | normalized `_score` | already on chunks |
| OceanBase | Yes (kept) | local hybrid rerank | already on chunks |

## Test plan
2026-05-18 14:21:56 +08:00
Rene Arredondo
9f2fb4611f Fix: guard empty/whitespace embedding inputs in LLMBundle (#14428) (#14924)
Closes #14428 


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 14:11:54 +08:00
carlos4s
2eba2c4d75 Add Anthropic Go model provider (#14940)
### What problem does this PR solve?

Adds the missing Anthropic provider implementation for the Go model
provider layer.

Closes #14939

### What changed

- Add `conf/models/anthropic.json` with Anthropic Claude chat/vision
models and API endpoints.
- Add `internal/entity/models/anthropic.go` implementing non-streaming
Messages API chat, model listing, and connection checking.
- Register `anthropic` in the Go model factory.
- Add httptest coverage for headers, payload mapping, response parsing,
validation errors, provider errors, model listing, connection checking,
factory registration, and unsupported methods.

### Notes

Streaming chat is left as an explicit `no such method` follow-up because
this initial provider focuses on non-streaming chat and connection
checking.

### Tests

- `docker run --rm -v
/home/ubuntu/Documents/gitTensor_repos/carlos/ragflow:/work -v
/tmp/ragflow-go-cache:/go/pkg/mod -v
/tmp/ragflow-go-build:/root/.cache/go-build -w /work golang:1.25 go test
-vet=off ./internal/entity/models -run Anthropic -count=1 -v`
- `docker run --rm -v
/home/ubuntu/Documents/gitTensor_repos/carlos/ragflow:/work -v
/tmp/ragflow-go-cache:/go/pkg/mod -v
/tmp/ragflow-go-build:/root/.cache/go-build -w /work golang:1.25 go test
-vet=off ./internal/entity -count=1`
- `git diff --check`
- `jq . conf/models/anthropic.json >/dev/null`

Plain `go test ./internal/entity/models` currently hits pre-existing
unrelated vet findings in other provider files (`baidu.go`, `cohere.go`,
`fishaudio.go`, `openrouter.go`).

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-18 12:03:33 +08:00
Jake Armstrong
fe1433d1ff Go: add Jina chat completions support (#14935)
### What problem does this PR solve?

This PR adds non-streaming chat support for the Jina Go model provider.

The Jina provider was added with embedding, rerank, model listing, and
connection checking, but `ChatWithMessages` still returned a
not-implemented error even though Jina exposes an OpenAI-compatible
`/v1/chat/completions` endpoint.

Closes #14933

**The following functionalities are now supported:**

### **Jina:**
- [x] Chat
- [ ] Stream Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

### **Implementation details:**
- Implements `JinaModel.ChatWithMessages`
- Sends `Authorization: Bearer <api-key>` and JSON chat completion
requests
- Validates API key, model name, messages, and configured region before
making requests
- Forwards supported chat config fields: `max_tokens`, `temperature`,
`top_p`, and `stop`
- Parses the first chat completion choice into `ChatResponse.Answer`
- Adds `jina-ai/jina-vlm` as a chat-capable model in
`conf/models/jina.json`
- Adds focused unit tests for request construction, auth, response
parsing, validation errors, provider errors, and region handling

**Verification:**

```plaintext
docker run --rm -v $PWD:/repo -w /repo golang:1.25 sh -c '/usr/local/go/bin/gofmt -w internal/entity/models/jina.go internal/entity/models/jina_test.go && /usr/local/go/bin/go test -vet=off ./internal/entity/models -run TestJina -count=1'
ok  	ragflow/internal/entity/models	0.037s
```

Note: `go test ./internal/entity/models -run TestJina -count=1`
currently hits unrelated existing vet findings in other provider files,
so the focused Jina tests were run with `-vet=off`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-18 12:03:12 +08:00
Panda Dev
6794ad2f70 Go: implement Embed (embeddings) in Novita driver (#14895)
### What problem does this PR solve?

Fixes #14893

The Novita Go driver landed in #14850 and shipped a stub `Embed` method
that returned `"novita, no such method"`, so Novita could not be used as
an embedding provider in RAGFlow. This PR fills that gap.

Novita exposes a public embeddings endpoint at `POST
https://api.novita.ai/v3/embeddings` that accepts the standard
OpenAI-compatible request shape (`{model, input}`) with `Authorization:
Bearer <api_key>`. Two embedding models are documented in Novita's model
library: `baai/bge-m3` (multilingual, 8192 tokens) and
`baai/bge-large-en-v1.5`.

### Changes

- `internal/entity/models/novita.go`: implement `NovitaModel.Embed`.
- Validate inputs (api key, model name) and short-circuit on empty
texts.
  - Resolve region with the existing `baseURLForRegion` helper.
- Build URL from `URLSuffix.Embedding` (the embeddings path lives under
`/v3/`, separate from the chat path under `/openai/v1/`).
- Send `{model, input}` POST body, add `dimensions` when
`embeddingConfig.Dimension > 0` (matches the pattern in #14735).
  - Bearer auth + JSON content type, mirroring the chat path.
- Parse `{data: [{embedding, index}]}` and reorder by `index`, rejecting
out-of-range indices, duplicates, and missing entries so the output
always lines up with the input. Same shape as the merged Mistral and
Upstage Embed implementations.
- `conf/models/novita.json`:
  - Add `"embedding": "v3/embeddings"` to `url_suffix`.
- Add default embedding model entries for `baai/bge-m3` and
`baai/bge-large-en-v1.5` so they appear in the model picker.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-18 12:02:28 +08:00
Idriss Sbaaoui
e98f3e5c0d Fix session deletion leaking chat-upload blobs (#14969)
### What problem does this PR solve?

This fixes a bug where files uploaded in chat were left in storage after
the session was deleted. It now removes those chat-uploaded blobs during
session deletion. fixes #14965

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-18 11:14:27 +08:00
qinling0210
9d94527b1d Bump to infinity v0.7.0 (#14968)
### What problem does this PR solve?

Upgrade infinity

### Type of change

- [x] Refactoring
2026-05-18 10:25:59 +08:00
07heco
e194027b01 refactor: optimize BaseTitleChunker to improve RAG document chunk quality (#14247)
## RAG Optimization Description
Optimize the core `BaseTitleChunker` in
`rag/flow/chunker/title_chunker/common.py` to improve RAG document
chunking quality and retrieval accuracy.

## Key Changes
1. **Format-branched text processing**: Preserve original whitespace &
indentation for Markdown/HTML payloads to maintain document semantics
and chunk fidelity; only perform full whitespace cleaning on plain text
content.
2. **Empty chunk filtering**: Thoroughly filter invalid pure-blank lines
to reduce noisy data in vector database.
3. **Code deduplication**: Unified markdown/text/html payload extraction
logic, removed redundant repeated code blocks.
4. **None serialization fix**: Avoid converting `None` value into
literal `"None"` string in chunk text fields.
5. **Production logging**: Added input/output line count logging for
filter logic, observable in online environment.
6. **100% backward compatible**: No changes to chunking hierarchy rules,
output format and all existing workflows.

## RAG Business Value
- Preserves document format fidelity for structured Markdown/HTML files
- Reduces invalid noisy chunks → improves RAG retrieval precision
- Cleans plain text data → optimizes vector embedding quality
- Improves code maintainability with no breaking changes
- Provides observable logging for chunk filtering behavior

## Compatibility
-  No API changes
-  No chunk logic modifications
-  All document parsing/chunking workflows unaffected
-  All pre-checks passed, no code conflicts

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-05-18 10:00:18 +08:00
Ricardo-M-L
ff318aba7a fix: correct literal_eval dispatch and bool isinstance ordering in agent components (#13988)
## Summary

This PR fixes 3 bugs in agent components:

### Bug 1: `DataOperations._invoke()` dispatches `"literal_eval"` to
wrong handler

**File:** `agent/component/data_operations.py`, line 76

The `_invoke()` method compares `self._param.operations` against
`"recursive_eval"` (line 76), but the valid value defined in
`DataOperationsParam.__init__()` (line 29) and validated in `check()`
(line 43) is `"literal_eval"`. This means selecting the `literal_eval`
operation from the frontend would never match, and the method
`_literal_eval()` would never be called.

**Fix:** Change `"recursive_eval"` to `"literal_eval"` in the dispatch
condition.

### Bug 2: `VariableAssigner._clear()` — `bool` branch unreachable

**File:** `agent/component/variable_assigner.py`, lines 95–100

In Python, `bool` is a subclass of `int` (`True` is `isinstance(True,
int) == True`). The `isinstance(variable, int)` check on line 95 catches
boolean values before the `isinstance(variable, bool)` check on line 99,
making the bool branch unreachable. A boolean variable would be cleared
to `0` instead of `False`.

**Fix:** Move the `isinstance(variable, bool)` check before
`isinstance(variable, int)`.

### Bug 3: `LoopItem.evaluate_condition()` — `bool` branch unreachable

**File:** `agent/component/loopitem.py`, lines 67–93

Same issue as Bug 2: `isinstance(var, (int, float))` on line 67 catches
boolean values before `isinstance(var, bool)` on line 85. Boolean
variables would be evaluated with numeric operators (`=`, `≠`, `>`,
etc.) instead of boolean operators (`is`, `is not`).

**Fix:** Move the `isinstance(var, bool)` check before `isinstance(var,
(int, float))`.

## Test plan

- [ ] Verify `DataOperations` with `literal_eval` operation correctly
invokes `_literal_eval()`
- [ ] Verify `VariableAssigner._clear()` returns `False` for boolean
variables (not `0`)
- [ ] Verify `LoopItem.evaluate_condition()` uses boolean operators for
`True`/`False` values


🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed operation routing logic to correctly dispatch the "literal_eval"
operation to its handler.

* **Refactor**
* Reorganized conditional branch ordering in agent components to improve
code structure and maintainability without affecting functional
behavior.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-18 09:58:45 +08:00
小熊
09d45046e5 Feat/web markdown UI updates (#14214)
### What problem does this PR solve?

LLM/chat and search UIs render Markdown in several places (document
preview, floating chat widget, next-search, etc.). Plugin lists and
behavior were duplicated or inconsistent, and single newlines in model
output were not always rendered as visible line breaks, which hurts
readability for chat-style content.

This PR centralizes shared **remark/rehype** configuration (including
**`remark-breaks`** for newline handling) and wires the main Markdown
surfaces to use it, so behavior is consistent and easier to maintain.


### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-05-15 22:29:44 +08:00
Haruko386
bf41d35729 Go: implement PaddleOCR provider and implement ASR for CoHere (#14954)
### What problem does this PR solve?

This PR implement implement OCR for Baidu and Mistral, implement
PaddleOCR provider and implement ASR for CoHere

**Verified examples from the CLI:**

```
RAGFlow(user)> ocr with 'mistral-ocr-2512@test@mistral' file './internal/text.jpg'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parallel to these organizational innovations there were significant complementary technical innovations (e.g., improved methods of manufacturing cast-iron pipe and of coating interiors for pressure maintenance, and newer paving and construction material... |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+


RAGFlow(user)> ocr with 'paddleocr-vl-0.9b@test@baidu' file './internal/text.jpg'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Parallel to these organizational innovations there were significant complementary technical innovations (e.g., improved methods of manufacturing cast-iron pipe and of coating interiors for pressure maintenance, and newer paving and construction material... |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

# PaddleOCR
RAGFlow(user)> ocr with 'PaddleOCR-VL-1.5@test@paddleocr' file './internal/test.pdf'
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                                                                                                                                                             |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| # Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

Bingxin Ke

Nando Metzger

Photogra

Anton Obukhov

Rodrigo Caye Daudt

netry and Remote Sensing,

Shengyu Huang

Konrad Schindler

ETH Zürich





<div style="text-align: c...  |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+

# Cohere

RAGFlow(user)> asr with 'cohere-transcribe-03-2026@test@cohere' audio './internal/test.wav' param '{"language": "en"}'
+-----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                  |
+-----------------------------------------------------------------------------------------------------------------------+
|  The examination and testimony of the experts enabled the Commission to conclude that five shots may have been fired. |
+-----------------------------------------------------------------------------------------------------------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-15 18:41:43 +08:00
wdeveloper16
14c0985182 feat: bump Python minimum from 3.12 to 3.13, drop strenum backport (#14767)
Closes #14753

## What changed

| File | Change |
|---|---|
| `pyproject.toml` | `requires-python` → `>=3.13,<3.15`; remove
`strenum==0.4.15` |
| `Dockerfile` | `uv python install 3.13`, `uv sync --python 3.13` |
| `.github/workflows/tests.yml` | `uv sync --python 3.13` on both matrix
legs |
| `CLAUDE.md` | dev setup command + requirements note updated |
| `deepdoc/parser/mineru_parser.py` | `from strenum import StrEnum` →
`from enum import StrEnum` |
| `agent/tools/code_exec.py` | same |

`StrEnum` has been in the stdlib since Python 3.11 — the `strenum`
backport package is no longer needed once the floor is 3.13.

## Why uv.lock is not regenerated

`uv lock --python 3.13` fails because:

1. The infiniflow/graspologic fork pins `numpy>=1.26.4,<2.0.0`
2. `tensorflow-cpu>=2.20.0` (the first release with cp313 wheels)
depends on `ml-dtypes>=0.5.1`, which requires `numpy>=2.1.0`
3. These two constraints are irreconcilable on Python 3.13

The lockfile regeneration requires loosening the `numpy` upper bound in
the `infiniflow/graspologic` fork. Once that fork commit is updated and
the SHA in `pyproject.toml:49` is bumped, `uv lock --python 3.13` will
succeed.

## RFC corrections

Two claims in the original RFC (#14753) did not hold up under code
review:

- **"graspologic hard-blocks 3.13"** — the infiniflow fork at the pinned
commit has no `<3.13` Python constraint. The blocker is the transitive
`numpy<2.0.0` conflict with tensorflow-cpu's test dependency, not a
direct Python version cap.
- **"free-threading throughput gains for I/O-bound workload"** — Python
3.13 free-threading requires a special `--disable-gil` build and
provides no benefit for async I/O code (the GIL is already released
during I/O). The real motivation is forward compatibility and improved
error messages.
2026-05-15 14:40:53 +08:00
Ricardo-M-L
cb606e1c38 fix: correct attribute name typo model_speciess to model_species (#13929)
## Summary
- Rename misspelled attribute `model_speciess` to `model_species` across
4 files
- The extra `s` is a typo — `species` is already plural

## Test plan
- [ ] Verify PDF parsing with laws/manual/paper parser types still works
correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-15 14:19:41 +08:00
Haruko386
c2863173b0 Go: implement TTS, ASR for Siliconflow and TTs for StepFun (#14944)
### What problem does this PR solve?

This PRimplement TTS, ASR for Siliconflow and TTs for StepFun

**The following functionalities are now supported:**

**SiliConFlow:**
- [x] Text To Speech
- [x] Audio To Text
- [x] Stream Audio To Text

**StrepFun:**

- [x] Audio To Text
- [x] Stream Audio To Text

**Verified examples from the CLI:**
```plaintext
# SiliconFlow

RAGFlow(user)> tts with 'FunAudioLLM/CosyVoice2-0.5B@test@Siliconflow' text 'hello? show yourself' play format 'wav' param '{"voice": "fnlp/MOSS-TTSD-v0.5:alex"}'
SUCCESS

RAGFlow(user)> asr with 'FunAudioLLM/SenseVoiceSmall@test@siliconflow' audio './internal/test.wav' param ''
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+

RAGFlow(user)> stream asr with 'FunAudioLLM/SenseVoiceSmall@test@siliconflow' audio './internal/test.wav' param ''
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-15 14:03:33 +08:00
Jin Hai
335dd5a263 Go: add cli command, list dataset documents (#14948)
### What problem does this PR solve?
```
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
| created_at          | id                               | meta_fields | name            | size    | status | type |
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
| 2026-05-08 19:35:08 | f6aa38bb4ad111f1ba6338a74640adcc | map[]       |  abc.pdf        | 3387987 | 1      | pdf  |
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-15 14:00:45 +08:00
SnakeEye-sudo (Er. Sangam Krishna)
1a25191b13 docs: add FAQ entry for using Ollama with RAGFlow (#14557)
### What problem does this PR solve?

Users frequently ask how to use Ollama for local LLM inference with
RAGFlow. This FAQ entry provides step-by-step instructions for setting
up Ollama as a local model provider.

### Type of change

- [x] Documentation update

### Description

Adds a new FAQ entry: "How do I use Ollama with RAGFlow for local LLM
inference?"

Covers:
1. Starting Ollama and pulling a model
2. Configuring Ollama as a model provider in RAGFlow Settings
3. Using the Ollama model in an assistant
2026-05-15 13:54:09 +08:00
Hunnyboy1217
86bcf9767d Go: implement Rerank in vLLM driver (#14878) (#14880)
### What problem does this PR solve?

Closes #14878.

`VllmModel.Rerank()` in
[internal/entity/models/vllm.go:551](internal/entity/models/vllm.go#L551)
is currently a stub returning `nil, fmt.Errorf("%s, Rerank not
implemented", z.Name())`, and
[conf/models/vllm.json](conf/models/vllm.json) is missing a `rerank`
entry in `url_suffix`. Chat (long-standing) and embeddings (#14688)
already work, so rerank is the last missing leg of the retrieval
pipeline for operators running everything on a single self-hosted vLLM
server — today they have to point rerank at a different provider, which
defeats the point of a fully local deployment.

Upstream vLLM has supported a Jina/Cohere-compatible `POST /v1/rerank`
endpoint since v0.7
([vllm-project/vllm#12376](https://github.com/vllm-project/vllm/pull/12376)).
The request/response shape is essentially identical to the NVIDIA driver
landed in #14778, so this PR mirrors that structure with two
vLLM-specific adjustments.

This PR replaces the stub with a real implementation against vLLM's
`/v1/rerank`:

- `POST {baseURL}/rerank`
- Request body: `{"model": "<modelName>", "query": "<query>",
"documents": [...], "top_n": <int>}` — documents are a flat `[]string`,
**not** wrapped as `{text: "..."}` like NVIDIA's `/ranking`.
- Response body: `{"results": [{"index": int, "relevance_score": float},
...]}` (Jina-compatible; the optional `document` field is ignored since
callers reconstruct text via `Index`).
- `Authorization: Bearer <ApiKey>` is set **only when `APIConfig.ApiKey`
is non-empty**, matching the existing `Embed`/`ListModels` behaviour in
this file. vLLM is a local driver and can be deployed without an API
key.

The return shape matches the existing `*RerankResponse` contract used by
the NVIDIA ([nvidia.go:461](internal/entity/models/nvidia.go#L461)),
Aliyun ([aliyun.go:507](internal/entity/models/aliyun.go#L507)), and
ZhipuAI ([zhipu-ai.go:554](internal/entity/models/zhipu-ai.go#L554))
drivers, i.e. `Data []RerankResult` carrying `{Index, RelevanceScore}`
in the API's ranking order. Callers that need original-input order sort
by `Index`.

Behaviour requirements from the issue, all covered:

1. Empty `documents` → returns `&RerankResponse{}` without an HTTP call.
2. Missing `modelName` → `"model name is required"` validation error.
3. `rerankConfig.TopN` honored when `0 < TopN < len(documents)`;
otherwise `top_n` defaults to `len(documents)` so callers get a score
per input.
4. Non-200 responses return an error including upstream status and body
(`"vLLM rerank API error: <status>, body: <body>"`).
5. Response `index` values are bounds-checked against `len(documents)`.

**Scope:**

- [internal/entity/models/vllm.go](internal/entity/models/vllm.go) —
replaces the `Rerank` stub at line 551 with a real implementation; adds
`vllmRerankRequest`/`vllmRerankResponse` types for the slim subset of
the payload we need. Region/baseURL resolution, 30s context timeout,
conditional bearer header, and error wrapping all follow the existing
patterns in this file.
- [conf/models/vllm.json](conf/models/vllm.json) — adds `"rerank":
"rerank"` to `url_suffix`, joined to the operator-configured vLLM base
URL the same way the NVIDIA driver joins at
[nvidia.go:485](internal/entity/models/nvidia.go#L485).
-
[internal/entity/models/vllm_rerank_test.go](internal/entity/models/vllm_rerank_test.go)
— adds 7 `httptest`-backed tests mirroring `nvidia_rerank_test.go`:
happy path (out-of-order ranking → Index preservation), `top_n` clamp to
`RerankConfig.TopN`, empty-documents short-circuit, missing-model-name
validation, HTTP error propagation, out-of-range index rejection, and a
vLLM-specific `TestVllmRerankWithoutAPIKey` locking in the optional-auth
behaviour that distinguishes this driver from NVIDIA.

**Out of scope:** no interface change, no DDL, no frontend change. Chat,
embeddings, and balance paths are untouched. No new user-facing docs
required beyond the existing rerank model setup page — vLLM joins the
list of providers whose rerank model can be selected once `/v1/rerank`
is exposed by the server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-15 13:27:22 +08:00
Octopus
eaa5d9921b fix: enable GitHub connector to sync PRs and issues by default (#14062)
Fixes #13975

## Problem
The GitHub data source connector had both `include_pull_requests` and
`include_issues` defaulting to `false` in both the frontend form and the
backend sync code. This meant that with the default configuration, **no
content was synced at all** from a GitHub repository — silently
producing zero results.

Additionally, the form field labels contained a typo: "Inlcude" instead
of "Include".

## Solution
- Changed `include_pull_requests` default from `false` to `true` in the
frontend form fields and default values
- Changed `include_issues` default from `false` to `true` in the
frontend form fields and default values
- Changed both backend defaults in `sync_data_source.py` from `False` to
`True`
- Fixed label typos: "Inlcude Pull Requests" → "Include Pull Requests"
and "Inlcude Issues" → "Include Issues"

This makes the GitHub connector consistent with the GitLab connector,
which already defaults `include_mrs`, `include_issues`, and
`include_code_files` all to `true`.

## Testing
- The connector now syncs both pull requests and issues by default when
a new GitHub data source is created
- Users who want to exclude PRs or issues can uncheck the corresponding
checkboxes in the form

Co-authored-by: octo-patch <octo-patch@github.com>
2026-05-15 13:26:31 +08:00
plind
c9622d0924 fix(agentbot): aggregate structured output in non-streaming completions (#14848)
## What problem does this PR solve?

Closes #13384.

The `/api/v1/agentbots/<agent_id>/completions` non-streaming path
returned the first yielded SSE chunk and exited:

```python
async for answer in agent_completion(objs[0].tenant_id, agent_id, **req):
    return get_result(data=answer)
```

That meant structured output, the full assistant message, and reference
data were all dropped when an agent was called with `stream=false`.
Streaming worked because each event was forwarded individually;
non-streaming was returning a raw SSE-formatted string from a single
early event.

The v1 endpoint at
[`agent_api.py:1006-1050`](https://github.com/infiniflow/ragflow/blob/main/api/apps/restful_apis/agent_api.py#L1006-L1050)
already handles this correctly. This PR mirrors that aggregation in the
SDK beta endpoint: parse each SSE line, accumulate `content` from
`message` events, merge `reference`, collect `outputs.structured` from
each `node_finished` event keyed by `component_id`, and attach all of
them to the final response.

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)

## Test plan

- [ ] Build an agent with a node that emits structured output, call
`POST /api/v1/agentbots/<agent_id>/completions` with `stream=false` and
a beta API token, verify `data.structured.<component_id>` is present in
the response.
- [ ] Same agent with `stream=true` — verify behavior is unchanged.
- [ ] Agent without structured output — verify `data.structured` is
omitted, `content` and `reference` still aggregated correctly.
2026-05-15 12:42:33 +08:00
Jin Hai
3a5df08c76 Go: add file parse command (#14892)
### What problem does this PR solve?

```
RAGFlow(user)> ocr with 'hunyuanocr@test@gitee' file './picture.png'
+----------------------------------------------------------+
| text                                                     |
+----------------------------------------------------------+
| 生活不是等待风暴过去,而是学会在雨中翩翩起舞。
——佚名                                                       |
+----------------------------------------------------------+

RAGFlow(user)> list 'test@gitee' tasks;
+---------+----------------------------------+
| status  | task_id                          |
+---------+----------------------------------+
| success | C3FX4MQNKY5MGC6ZFMIXIAMJKHCEBQB5 |
+---------+----------------------------------+
RAGFlow(user)> show 'test@gitee' task 'C3FX4MQNKY5MGC6ZFMIXIAMJKHCEBQB5';
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| content                                                                                                                                                                                                                                                          | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| # PDF 1: Purpose of RAGFlow  

RAGFlow is an open source Retrieval-Augmented Generation (RAG) engine designed to turn raw documents into reliable context for large language models.Its purpose is to make it practical to build an Al assistant that can ans... | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-15 12:29:52 +08:00
Sebastion
547b8cf9d8 security: always use RestrictedUnpickler in deserialize_b64 (CWE-502) (#14803)
## Summary

Harden `api/utils/configs.deserialize_b64` so that it always routes
pickle data through the existing `RestrictedUnpickler`
(`restricted_loads`) rather than falling back to bare `pickle.loads()`.

- **CWE-502** — Deserialization of Untrusted Data
- **File / function**: `api/utils/configs.py` → `deserialize_b64`
- **Caller**: `SerializedField.python_value` in `api/db/db_models.py`
(invoked by Peewee whenever a pickled DB column is read)

## The issue

Before this change, `deserialize_b64` consulted a
`use_deserialize_safe_module` config flag that **defaults to `False`**
and is not set anywhere in the repository:

```python
use_deserialize_safe_module = get_base_config('use_deserialize_safe_module', False)
if use_deserialize_safe_module:
    return restricted_loads(src)
return pickle.loads(src)   # <-- default path
```

So the default code path was unrestricted `pickle.loads()` on bytes read
from a MySQL `SerializedField(serialized_type=PICKLE)` column. Any
attacker who can influence those bytes (SQL injection elsewhere,
compromised DB credentials, a backup restored from an untrusted source,
or a compromised replication peer) can craft a pickle payload that
achieves arbitrary code execution on the ragflow application server when
the field is next read.

Today no model in-tree instantiates a `SerializedField` with the default
PICKLE type — only `JsonSerializedField` is used in practice — so the
attack surface is currently **latent** rather than actively reachable
through an HTTP endpoint. But the insecure-by-default behaviour is a
sharp edge: any future field that uses the default PICKLE serialization
would silently inherit RCE-on-read semantics.

## The fix

```diff
-    use_deserialize_safe_module = get_base_config(
-        'use_deserialize_safe_module', False)
-    if use_deserialize_safe_module:
-        return restricted_loads(src)
-    return pickle.loads(src)
+    return restricted_loads(src)
```

`restricted_loads` is the existing `RestrictedUnpickler` already defined
in the same file, which limits permitted modules to `numpy` and
`rag_flow`. The config flag (and the now-dead `get_base_config` import)
are removed.

Diff is 1 insertion / 6 deletions, scoped to a single function.

## Testing

- Built a malicious pickle whose `__reduce__` resolves to
`posix.system('id')`. Pre-fix: executes. Post-fix: `restricted_loads`
raises `UnpicklingError: global 'posix.system' is forbidden`.
- Round-tripped a benign `numpy.ndarray` through `serialize_b64` →
`deserialize_b64`. Values preserved bit-for-bit.
- Confirmed `use_deserialize_safe_module` is not set in any config file
in the tree, so removing the flag does not change any operator-facing
knob that was actually in use.

## A note on `restricted_loads` itself

The existing `SECURITY.md` notes that `restricted_loads`'s `numpy`
allow-list can still be reached via `numpy.f2py.diagnose.run_command`.
This PR does **not** attempt to fix that — it is a separate hardening
question about tightening the allow-list to specific symbols rather than
whole modules. The change here strictly improves on the status quo (bare
`pickle.loads`) and brings the default path in line with what the
`restricted_loads` helper was clearly designed for. Happy to follow up
with a separate PR narrowing the allow-list if that direction is
welcome.

## Adversarial review

Before submitting, we tried to argue this finding away. The two
strongest objections are (1) "no field uses PICKLE today, so this is
unreachable" — true, but the default behaviour of a security-sensitive
helper still matters because new fields silently inherit it; and (2)
"the attacker already needs DB write access, which is game over" —
partially true, but pickle-RCE meaningfully escalates *data tampering*
into *code execution on the application host* (filesystem, internal
network, in-process secrets), which is not equivalent. The fix is one
line of real code, has no behavioural cost for legitimate callers, and
removes an insecure default. We decided it was worth filing.

---

<sub>_Submitted by Sebastion — autonomous open-source security research
from [Foundation Machines](https://foundationmachines.ai). Free for
public repos via the [Sebastion AI GitHub
App](https://github.com/marketplace/sebastion-ai)._</sub>
2026-05-15 10:58:27 +08:00
yingjianzh
4c68a6b86c fix(agent): pass top_k and fix similarity weight slider behavior (#14760)
### What problem does this PR solve?

This PR fixes two issues in Agent Retrieval behavior and configuration
UX:

1. `top_k` configured in Agent Retrieval was not passed down to the
backend retriever call, so retrieval could ignore the configured vector
recall limit.
2. Similarity weight slider semantics were confusing in Agent forms
because the Agent field stores `keywords_similarity_weight` while UI
interactions were interpreted as vector weight. This could cause
displayed values and actual behavior to diverge.

This PR ensures Agent retrieval uses configured `top_k`, and makes the
slider behavior consistent and explicit for both vector and keyword
weight modes.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-15 10:49:14 +08:00
sham-sr
ef2969a462 fix(llm): Tongyi-Qianwen embeddings use correct DashScope native API for intl URLs (#14784)
## Summary
- Fixes **Tongyi-Qianwen** (`QWenEmbed`) text embeddings when the
configured `base_url` points at DashScope **international**
(`dashscope-intl.aliyuncs.com`) or **China** (`dashscope.aliyuncs.com`)
hosts, including values copied from Model Studio that use the
**OpenAI-compatible** path (`.../compatible-mode/v1`).
- The `dashscope` Python SDK (`TextEmbedding.call`) expects the
**native** HTTP root (`https://<host>/api/v1`), not the
OpenAI-compatible base URL. Without mapping, international accounts
could hit the wrong host or path.

## Implementation
- Added `_dashscope_native_http_api_url()` to normalize known DashScope
hosts to `.../api/v1`, and wired `QWenEmbed` to set
`dashscope.base_http_api_url` before each embedding call (document and
query).

## Notes
- In-code comments document the Tongyi-Qianwen / DashScope intl vs CN
behavior for future maintainers.

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-15 10:07:48 +08:00
Octopus
d887b578c5 fix: preserve uploaded file attachments after subsequent assistant messages (#13993)
## Problem

When a user uploads a file attachment in their first message (Q1) and
then sends a follow-up message (Q2) that triggers a backend response,
the uploaded file attachment disappears from Q1 in the chat UI.

Fixes #13959

## Root Cause

In `single-chat-box.tsx`, a `useEffect` hook syncs `derivedMessages`
from `conversation?.messages` whenever the conversation data changes
(e.g., after a new assistant reply arrives):

```typescript
useEffect(() => {
  const messages = conversation?.messages;
  if (Array.isArray(messages)) {
    setDerivedMessages(messages); // ← overwrites local state
  }
}, [conversation?.messages, setDerivedMessages]);
```

The problem is that `conversation.messages` comes from the server, which
stores messages as plain JSON. Browser `File` objects (uploaded by the
user) cannot be serialized to JSON, so they are never stored on the
server. Each time the server data is applied to local state, the `files`
array on the user's first message is lost.

## Fix

Instead of replacing the local messages wholesale, preserve any `files`
entries from the previous local state by ID before applying the server
data:

```typescript
useEffect(() => {
  const messages = conversation?.messages;
  if (Array.isArray(messages)) {
    setDerivedMessages((prevMessages) => {
      const filesMap = new Map(
        prevMessages
          .filter((m) => m.files?.length)
          .map((m) => [m.id, m.files]),
      );
      if (filesMap.size === 0) {
        return messages;
      }
      return messages.map((m) => ({
        ...m,
        files: filesMap.get(m.id) ?? m.files,
      }));
    });
  }
}, [conversation?.messages, setDerivedMessages]);
```

This is a minimal, targeted fix: when there are no local files to
preserve the behavior is identical to before (early return with plain
assignment). When local file objects exist they are re-attached to the
corresponding server messages by ID.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved search query processing to properly handle special characters
and apostrophes in search terms and synonyms.
* Fixed chat message file attachments to persist when syncing with
server.

* **Refactor**
  * Simplified OCR detection return values by removing timing metadata.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ximi <octo-patch@github.com>
2026-05-15 09:53:35 +08:00
buua436
58819f5d3e fix: add document download endpoint and refactor existing download function (#14927)
### What problem does this PR solve?

add document download endpoint and refactor existing download function

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-15 09:36:58 +08:00
writinwaters
5a5bbee948 Doc: Finalized v0.25.4 release notes (#14929)
### What problem does this PR solve?

v0.25.4 release notes. Final.

### Type of change


- [x] Documentation Update
2026-05-14 21:08:39 +08:00
balibabu
41072ed44d Feat: This enables SelectWithSearch to search by label. (#14925)
### What problem does this PR solve?

Feat: This enables SelectWithSearch to search by label.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: balibabu <assassin_cike@163.com>
2026-05-14 20:33:11 +08:00
Haruko386
106f4b777e Go: implement TTS for fishaudio, openrouter and asr for fishaudio (#14926)
### What problem does this PR solve?

This PR implement TTS for FishAudio and MiniMax provider and ASR for
FishAudio

**The following functionalities are now supported:**

**FishAudio:**
- [x] Text To Speech
- [x] Stream Text To Speech
- [x] Audio To Text

**OpenRouter:**

- [x] Text To Speech

**Verified examples from the CLI:**
```plaintext

**FishAudio**

RAGFlow(user)> tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> stream tts with 's1@test@fishaudio' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"reference_id": "90e65eaaf50e4470b8e6d43ee6afd7d5", "temperature": 0.7, "top_p": 0.7, "prosody": {"speed": 1, "volume": 0, "normalize_loudness": true}, "chunk_length": 300, "normalize": true, "sample_rate": 44100, "mp3_bitrate": 128, "latency": "normal", "max_new_tokens": 1024, "repetition_penalty": 1.2, "min_chunk_length": 50, "condition_on_previous_chunks": true, "early_stop_threshold": 1}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/s1_output.wav
SUCCESS

RAGFlow(user)> asr with 'transcribe-1@test@fishaudio' audio './internal/test.wav' param '{"language": "en", "ignore_timestamps": true}'
+----------------------------------------------------------------------------------------------------------------------+
| text                                                                                                                 |
+----------------------------------------------------------------------------------------------------------------------+
| The examination and testimony of the experts enabled the commission to conclude that five shots may have been fired. |
+----------------------------------------------------------------------------------------------------------------------+

```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-14 18:58:00 +08:00
wdeveloper16
a98994ff91 fix: close db connections reliably in test_db_connection (#14777)
## Summary

- Fixes resource-management bugs in the `POST
/agents/test_db_connection` endpoint where database connections could be
left open on error (part of #14750)

## Changes

- `api/apps/restful_apis/agent_api.py` — `test_db_connection`:
- mysql / mariadb / oceanbase / postgres: replaced bare `db.connect()` /
`db.close()` fallthrough with `with db.connection_context()` and a probe
`SELECT 1` — guaranteed close on both success and exception
- mssql: nested `try/finally` blocks so `cursor.close()` and
`db.close()` are always called even when `cursor.execute()` raises
  - trino: wrapped cursor ops in `try/finally` for the same reason
- Removed the `if req["db_type"] != "mssql": db.connect(); db.close()`
shared fallthrough block — each branch now owns its teardown
- Consolidated to a single `return get_json_result(...)` after the
if/elif chain
2026-05-14 16:45:44 +08:00
eviaaaaa
63df01fe3f fix(agent): handle duplicate MCP tool names (#14217)
### What problem does this PR solve?

When multiple MCP servers expose tools with the same name, the agent
currently registers those tools using their original MCP names. This can
lead to two issues:

- later MCP tools may overwrite earlier ones in the agent tool map
- duplicate function names may be exposed to the LLM

This PR fixes duplicate MCP tool-name handling by applying the same
indexed naming strategy already used for native agent tools. Native
tools are exposed with generated names such as `<tool_name>_<index>` to
avoid collisions, and MCP tools now follow the same convention for
consistency.

Specifically, this PR:

- assigns unique indexed function names to MCP tools exposed to the LLM
- preserves each MCP tool's original server-side name in an
`MCPToolBinding`
- dispatches MCP calls using the original MCP tool name while keeping
the indexed name in the agent tool map
- allows MCP metadata conversion to override only the OpenAI function
name without modifying the original MCP tool metadata

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


### Validation

The validation was performed using two MCP servers. Both servers exposed
a tool with the same name: `mcp0`. Both tools take no input parameters.

**MCP Server One:**
<img width="1780" height="625" alt="ONE"
src="https://github.com/user-attachments/assets/801a2654-fc10-4b71-b31c-81841fd40c55"
/>

**MCP Server Two:**
<img width="1777" height="624" alt="Second"
src="https://github.com/user-attachments/assets/c095151d-7bdf-47c8-9bfe-6aaf4a01b944"
/>

**Before the fix:**
When invoking `mcp0`, only the `mcp0` tool from the MCP server injected
later could be called successfully. As shown below, both `mcp0` tools
were present, but only the later-registered one was actually invokable.

<img width="694" height="935" alt="Three"
src="https://github.com/user-attachments/assets/3b9d7ab2-1765-492c-b8e0-bf05a69933ca"
/>

**After the fix:**
Both `mcp0` tools can now be invoked correctly.

<img width="737" height="1095" alt="F"
src="https://github.com/user-attachments/assets/6e896627-2b7f-41bb-becc-daa0c73ff58f"
/>

<img width="730" height="1090" alt="six"
src="https://github.com/user-attachments/assets/aba75593-26ae-4e3b-951d-b45ff177fd32"
/>
2026-05-14 15:28:39 +08:00
dale053
bd99a22661 fix: atomic chunk/token counter updates for documents and knowledge b… (#14867)
### What problem does this PR solve?

Fixes #14866.

Previously, `DocumentService.increment_chunk_num` and
`decrement_chunk_num` updated the `Document` row and its parent
`Knowledgebase` row in two separate, non-transactional statements. If
the second update failed (DB error, connection drop, etc.) after the
first one succeeded, the document and knowledge base chunk/token
counters would drift apart and stay inconsistent.

There was also a behavioral asymmetry between the two methods:

- `increment_chunk_num` only logged a warning when the document row was
missing and returned a value that callers usually treated as success.
- `decrement_chunk_num` raised `LookupError` in the same situation.

This PR makes the counter updates atomic and aligns the missing-document
behavior between the two methods:

- Wrap the `Document` and `Knowledgebase` updates in
`increment_chunk_num` / `decrement_chunk_num` inside a `DB.atomic()`
block so both succeed or both roll back together.
- Raise `LookupError` from `increment_chunk_num` when the target
document no longer exists, matching `decrement_chunk_num`.
- Update `reset_document_for_reparse` in `document_api_service.py` to
catch the new `LookupError` and return a proper "Document not found!"
API error instead of propagating the exception.

No schema changes, no API contract changes for the success path; only
the failure mode for a missing document during reparse is now a clean
error response instead of an uncaught exception.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 14:48:52 +08:00
buua436
3c68ad03be Go: update user settings fields (#14918)
### What problem does this PR solve?

update user settings fields

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 14:47:15 +08:00
Ethan T.
ba8cb9dd4a fix: replace mutable default arguments with None in LLM chat models (#13513)
## Summary
- Replace `gen_conf={}` with `gen_conf=None` + guard in
`rag/llm/chat_model.py` (12 instances across Base, BaiChuanChat,
LocalLLM, MistralChat, ReplicateChat, BaiduYiyanChat, GoogleChat
classes)
- Replace `doc_ids=[]` with `doc_ids=None` + guard in
`api/db/services/document_service.py` (1 instance)
- Mutable default arguments are shared across all calls, causing
potential cross-request state contamination
- See Python docs:
https://docs.python.org/3/faq/programming.html#why-are-default-values-shared-between-objects

## Test plan
- [x] Verify LLM calls work with and without explicit gen_conf
- [x] No behavior change for existing callers — `None` is replaced with
`{}` at function entry

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-14 14:46:47 +08:00
buua436
0450400efd Go: fix LastLoginTime update (#14917)
### What problem does this PR solve?

fix LastLoginTime update

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 14:46:39 +08:00
dale053
714f777fa0 Fix: missing authentication on agent file upload and download endpoints (#14854)
### What problem does this PR solve?

Closes #14853

The `/agents/download` and `/agents/<agent_id>/upload` endpoints in the
agent API are missing `@login_required` and `@add_tenant_id_to_kwargs`
decorators, allowing unauthenticated access. This is a security issue —
any user can upload files to or download files from an agent without
being logged in. Additionally, the upload endpoint bypasses canvas
access control (`@_require_canvas_access_async`).
This PR adds the missing authentication and authorization decorators to
both endpoints and replaces the manual `user_id` / `created_by` lookups
with the `tenant_id` provided by the auth middleware, making these
endpoints consistent with the rest of the agent API.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 13:48:41 +08:00
buua436
f0122179dd GO: align time units with Python and centralize timestamp injection in BaseModel (#14875)
### What problem does this PR solve?

align time units with Python and centralize timestamp injection in
BaseModel

### Type of change

- [x] Refactoring
2026-05-14 13:46:46 +08:00
buua436
82e06db8c3 Doc: code component output section (#14915)
### What problem does this PR solve?

code component output section

### Type of change

- [x] Documentation Update
2026-05-14 13:42:40 +08:00
Ricardo-M-L
48b4aa3e93 Fix WebDriver resource leak in HTML-to-PDF conversion (#14310)
### What problem does this PR solve?

In `api/utils/web_utils.py`, `__get_pdf_from_html()` creates a Chrome
WebDriver but only calls `driver.quit()` inside the `TimeoutException`
handler. If the page element becomes stale before the timeout (no
exception raised), the WebDriver is never quit, leaking the Chrome
browser process and returning `None`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- Move the PDF printing logic and `driver.quit()` outside the `except`
block so they execute on all code paths
- Use `try/finally` to ensure `driver.quit()` is always called, even if
the `Page.printToPDF` DevTools call fails

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-14 13:28:58 +08:00
Ricardo-M-L
4bfdb1e123 fix: correct nested path traversal in set_variable_param_value (#13986)
## Summary

`Graph.set_variable_param_value()` in `agent/canvas.py` has a bug in its
nested path traversal logic. The `for` loop iterates through **all**
keys in the path (including the last one), descending into every level.
After the loop, it then tries to set `cur[keys[-1]] = value`, but `cur`
has already descended one level too deep.

**Example:** For `path = "a.b"`, `value = "hello"`:
- **Before (bug):** `obj["a"]["b"]` becomes `{"b": "hello"}` instead of
`"hello"`
- **After (fix):** `obj["a"]["b"]` becomes `"hello"` as expected

The fix changes `for key in keys:` to `for key in keys[:-1]:`, so the
loop only navigates to the parent dict, and the final key is set
directly. This is consistent with how the read-side counterpart
`get_variable_param_value()` works.

This method is called by `set_variable_value()` when assigning to nested
variable paths (e.g., `component@root.nested.key`), which is used by the
`VariableAssigner` component.

## Test plan
- [ ] Create a canvas with a VariableAssigner that writes to a nested
path (e.g., `component@obj.nested.key`)
- [ ] Verify the value is set correctly at the expected path, not
wrapped in an extra dict layer
- [ ] Verify single-key paths (e.g., `component@key`) still work
correctly

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Fixed a bug in variable parameter assignment where nested structures
were being incorrectly modified, ensuring values are now properly set at
their intended locations without unintended overwrites.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-05-14 13:27:04 +08:00
Haruko386
ef46005ef1 Go: implement TTS for MiniMax provider and CLI testing for TTS (#14911)
### What problem does this PR solve?

This PR implement TTS for MiniMax provider and CLI testing for TTS

**The following functionalities are now supported:**

**MiniMax:**
- [x] Chat / Stream Chat 
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [x] Text To Speech
- [ ] OCRFile
- [ ] ~~Audio To Text~~
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> tts with 'speech-2.8-hd@test@minimax' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"voice_setting": {"voice_id": "English_radiant_girl", "speed": 1, "vol": 1, "pitch": 0}, "audio_setting": {"sample_rate": 32000, "bitrate": 128000, "format": "wav", "channel": 1}, "output_format": "hex"}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/speech-2.8-hd_output.wav
SUCCESS

RAGFlow(user)> stream tts with 'speech-2.8-hd@test@minimax' text 'He who desires but acts not, breeds pestilence.' play format 'wav' save './internal' param '{"voice_setting": {"voice_id": "English_radiant_girl", "speed": 1, "vol": 1, "pitch": 0}, "audio_setting": {"sample_rate": 32000, "bitrate": 128000, "format": "wav", "channel": 1}, "output_format": "hex"}'
Saved to directory: /home/infiniflow/Documents/development/ragflow/internal/speech-2.8-hd_output.wav
SUCCESS
```
Set `Play` to play audio in CLI
Set `Save` `PATH_TO_SAVE` to save file
Set `format` to save file in wav or mp3
Set `Param` align with official request body

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-14 13:19:31 +08:00
Br1an
d46bbd30f7 Fix: send input and output token usage to Langfuse (#13294)
### What problem does this PR solve?

Closes #9837

The Langfuse integration currently only sends the output text to
`langfuse_generation.update()` without including token usage
information. This means Langfuse cannot track input/output token
consumption for cost analysis and monitoring.

### Solution

Add the `usage` parameter to `langfuse_generation.update()` with:
- `input`: approximate input token count from `message_fit_in()`
- `output`: approximate output token count from
`num_tokens_from_string(answer)`
- `total`: sum of input and output

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-14 13:11:37 +08:00
Ricardo-M-L
cc21dc7f00 fix: replace broken assert with raise ValueError in variable_assigner and loop (#13906)
\`assert \"string\"\` always passes in Python because non-empty strings
are truthy. This silently skips input validation:

- **variable_assigner.py line 51**: \`assert \"Variable is not
complete.\"\` → \`raise ValueError(\"Variable is not complete.\")\`
- **loop.py line 59**: \`assert \"Loop Variable is not complete.\"\` →
\`raise ValueError(\"Loop Variable is not complete.\")\`

Without this fix, incomplete variables pass validation silently and
cause a confusing KeyError on the next line.
2026-05-14 12:33:17 +08:00
07heco
8dc5b1b42d fix: optimize reranking module robustness and bug fixes (#14264)
## Description
This PR fixes critical bugs and improves the robustness of the RAG
reranking module while maintaining **100% backward compatibility** with
all existing functionality and providers.

## Key Changes
1. **Network Stability**: Added 30s timeout to all API requests to
prevent service blocking
2. **Boundary Protection**: Added empty query/text validation for all
rerank models
3. **Response Fault Tolerance**: Replaced hardcoded key access with
`.get()` to avoid KeyError crashes
4. **Bug Fixes**:
   - Fixed `Ai302Rerank` (completely non-functional before)
   - Fixed `GPUStackRerank` incorrect exception catching
   - Fixed `_normalize_rank` empty array crash
5. **Code Specification**: Added type annotations, standardized
unimplemented class prompts

## Compatibility
-  No changes to any class/method names
-  All rerank providers (Jina/Cohere/NVIDIA/HuggingFace etc.) work as
before
-  No breaking changes, zero impact on existing workflows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-14 11:56:09 +08:00
writinwaters
851b16b913 Docs: Added v0.25.4 release notes draft. (#14914)
### What problem does this PR solve?

v0.25.4 release notes draft.

### Type of change


- [x] Documentation Update
2026-05-14 11:24:20 +08:00
Liu An
f038a34154 Docs: Update version references to v0.25.4 in READMEs and docs (#14912)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.3 to v0.25.4
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-14 11:07:08 +08:00
buua436
b89878c593 Fix: dataset document download route (#14910)
### What problem does this PR solve?

dataset document download route
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 10:59:06 +08:00
writinwaters
1c0eaa504b Docs: Finalized v0.25.3 release notes (#14913)
### What problem does this PR solve?

0.25.3 release notes, final.

### Type of change

- [x] Documentation Update
2026-05-14 10:57:43 +08:00
sirj0k3r
b2b63600f1 Adds gpt-5.4-mini and gpt-5.4-nano (#14908)
### What problem does this PR solve?
Includes gpt-5.4-mini and gpt-5.4-nano to the OpenAI model list

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-14 10:16:24 +08:00
Yingfeng
e577901388 Fix doc format (#14909)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-14 09:49:45 +08:00
tmimmanuel
cb01529d8b Go: implement provider: Voyage AI (#14811)
### What problem does this PR solve?

Add a Go driver for Voyage AI (https://voyageai.com), one of the
unchecked providers on the umbrella tracking issue #14736. Voyage AI is
**embed + rerank only** — no chat, no streaming, no `/v1/models`
endpoint. It's the first provider in the Go layer of this shape.

Until this PR, a tenant who configured `voyage` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/voyage.go` with a `VoyageModel`
implementing the `ModelDriver` interface.
- New `conf/models/voyage.json` with 6 embedding models (`voyage-3.5`,
`voyage-3.5-lite`, `voyage-3-large`, `voyage-code-3`, `voyage-law-2`,
`voyage-finance-2`) and 2 rerank models (`rerank-2`, `rerank-2-lite`).
- `factory.go`: route `"voyage"` to `NewVoyageModel`.
- `internal/entity/models/voyage_test.go`: 19 unit tests.

### How the driver works

- **Embed**: `POST /v1/embeddings`. Response is OpenAI-shaped (`{data:
[{embedding, index, object, text}], model, usage}`). Driver reorders by
`index`, rejects duplicate / out-of-range / missing slots, and
short-circuits empty input without an HTTP call.
- **Rerank**: `POST /v1/rerank`. Voyage uses **`top_k`** as the request
param name (not `top_n` like Aliyun/SiliconFlow); the driver translates
`RerankConfig.TopN` → `top_k`. Response is Cohere-shaped (`{data:
[{relevance_score, index}], model}`), so the existing
`RerankResponse{Data: []RerankResult{Index, RelevanceScore}}` shape fits
cleanly.
- **`ListModels`**: returns a hardcoded list of `voyageKnownModels`.
Voyage does **not** expose `/v1/models` (probed live, returns 404), so
the driver synthesizes the list from the same set the config ships. New
upstream models are added by extending one slice.
- **`CheckConnection`**: pings a 1-input embed call against
`voyage-3.5`. Without `/v1/models`, this is the cheapest way to verify
the API key + network path before a tenant tries a real workload.
- **`ChatWithMessages` / `ChatStreamlyWithSender` / `Balance` /
`TranscribeAudio` / `AudioSpeech` / `OCRFile`**: all return `"no such
method"`. Voyage does not host any of these surfaces.

No interface change. No new dependencies.

### How was this tested?

**19 unit tests** in `internal/entity/models/voyage_test.go` — all pass
on go 1.25:

```
$ go test -vet=off -run TestVoyage -count=1 ./internal/entity/models/...
ok      ragflow/internal/entity/models  0.036s
```

Coverage: Name; Embed (happy path, reorder, empty-input, missing
key/model, duplicate index, out-of-range index, missing slot); Rerank
(happy path with `top_k` assertion, default-to-len-documents, empty
documents, out-of-range index); ListModels (static list, missing key);
CheckConnection (happy, 401); chat methods sentinels; Balance sentinel;
audio/OCR sentinels.

`go build ./internal/entity/models/...` exits 0.

**Live integration test** against `api.voyageai.com`:

```
=== RUN   TestVoyageLiveSmoke
    [OK] Name() = "voyage"
    [OK] ListModels (static): 8 models -> [voyage-3.5 voyage-3.5-lite voyage-3-large voyage-code-3 voyage-law-2 voyage-finance-2 rerank-2 rerank-2-lite]
    [OK] CheckConnection
    [OK] Embed vectors=3 dim=1024 indices=[0 1 2]
    [OK] Embed(empty) -> 0 vectors
    [OK] Rerank results=3 scores=[0.8125 0.59765625 0.39453125]
    [OK] ChatWithMessages returns voyage, no such method
    [OK] Balance returns voyage, no such method

VOYAGE LIVE SMOKE PASSED
--- PASS: TestVoyageLiveSmoke (0.81s)
```

What the live run proves on the wire:

- Auth (`Bearer <key>`) accepted by `api.voyageai.com`.
- Embed `voyage-3.5` on 3 inputs returns 3 vectors at dim 1024 with
`index` field preserved as `[0, 1, 2]` — the reorder-by-index code is
exercised on real data.
- Empty input short-circuits without an HTTP call (mock server would
have been hit if it did).
- Rerank `rerank-2` on 3 docs returns 3 real `relevance_score` floats
`[0.8125, 0.598, 0.395]`. The `top_k` translation works on the live
wire.
- All sentinel methods return the documented `"no such method"` strings.

### Note on PR history

This branch was previously named for LocalAI Embed work which is now
consolidated into PR #14813. The branch was reset to `upstream/main` and
rebuilt for Voyage. Diff against `main` is a clean +838 lines across 4
files.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-14 09:46:54 +08:00
plind
dd76653dc1 feat: add tag management for Agents with filtering and sorting (#14774) (#14799)
## Summary
Closes #14774.

Adds free-form tags on agents (UserCanvas) with full UI + API:
- Stored as comma-separated `tags` column on `UserCanvas` with online
migration.
- New endpoints: `GET /v1/agents/tags` (aggregate counts) and `PUT
/v1/agent/<id>/tags` (write). `GET /v1/agents` accepts a `tags=` query.
- "Edit tags" item in agent dropdown opens a chip-style editor dialog;
tags render as badges on each agent card.
- New "Tags" facet in the agents filter bar, with counts.

## Implementation notes
- **Tag matching is exact-token**: the SQL filter wraps stored tags as
`,…,` and matches `,ml,` so `ml` doesn't match `ml-ops`.
- **Server-side normalization** in `UserCanvasService.update_tags`:
dedup (case-insensitive), per-tag cap of 64 chars, total length capped
at 512 chars to fit the column, commas inside tag values are replaced
with spaces.
- **Tenant authorization**: `PUT /v1/agent/<id>/tags` gates on
`UserCanvasService.accessible(canvas_id, tenant_id)`.
- **Tag listing scope**: `UserCanvasService.list_tags` follows the same
own + team-shared rule as `get_by_tenant_ids`.
- **i18n**: keys added to `en.ts` and `zh.ts` only (per project
convention; other locales fall back).
- **`HomeCard`** gets a non-breaking `extra?: ReactNode` slot for the
chip row; no `src/components/ui/` files modified.

## Test plan
- [ ] Backend boot runs `migrate_db` → confirm `user_canvas.tags` column
exists (`DESCRIBE user_canvas`).
- [ ] Agents page renders cards normally (no console error from missing
field).
- [ ] `⋯ → Edit tags` opens a dialog that stays open (regression: dialog
was unmounting with the dropdown).
- [ ] Typing a tag without pressing Enter and clicking Save persists it
(regression: last typed tag was being dropped).
- [ ] Chip input supports Enter/comma to commit, Backspace on empty to
remove, `×` to remove individual chip.
- [ ] Tag containing a comma sent via API is stored with the comma
replaced by a space.
- [ ] 20 long tags sent via API does not error (length cap silently
truncates).
- [ ] "Tags" filter in the filter bar shows counts and narrows the list.
- [ ] Filtering by `ml` does **not** return agents tagged `ml-ops`.
- [ ] UI in Chinese shows 编辑标签 / 添加标签以整理和筛选你的智能体 etc.
- [ ] `PUT /v1/agent/<other-tenant-id>/tags` returns `Agent not found or
no permission.`
2026-05-13 21:41:32 +08:00
writinwaters
cb49f47c38 Docs: Editorial updates to the v0.25.3 release notes draft. (#14903)
### What problem does this PR solve?


v0.25.3 release notes. To be continued.

### Type of change


- [x] Documentation Update
2026-05-13 21:36:34 +08:00
47NoahThompson
9e0f976729 Add widget customization and persistence (#14603)
Introduce comprehensive floating widget customization: add new widget
settings (title, subtitle, footer, colors, mute, streaming) with types
and defaults, and expose them via EmbedDialog UI (split into Embed Setup
and Widget Customization tabs). Persist and load settings through Agent
page by reading/writing globals and wiring an onSaveWidgetSettings
handler to setAgent; show a loading ButtonLoading for saving. Update
embed iframe query params and FloatingChatWidget to honor URL params
(colors, text, mute/streaming) with validation/normalization, color
darkening for gradients, footer link normalization, and improved
styling. Also add copy-to-clipboard in message toolbar, adjust syntax
highlighter layout and Copy button, and add i18n key for muteWidget.

### What problem does this PR solve?

Adds a few fields to the embed widget modal to customize the appearance
of the floating widget when embedded into a page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Noah <Noah.Thompson@ecn.forces.gc.ca>
2026-05-13 21:13:11 +08:00
Ethan T.
8c5845f6ca fix: use context manager for pdfplumber to prevent resource leak (#13512)
## Summary
- Convert `pdfplumber.open()` to use `with` context manager in
`api/utils/file_utils.py` (`thumbnail_img` function)
- If any exception occurs between `open()` and `close()`, the PDF file
handle leaks
- The rest of the codebase (e.g. `read_potential_broken_pdf` in the same
file) already uses `with pdfplumber.open(...)` correctly

## Test plan
- [x] PDF thumbnail generation works correctly with context manager
- [x] Resources properly cleaned up on exceptions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-13 21:09:51 +08:00
Ahmad Intisar
e994051eb9 Feature/generic api connector (#13545)
# feat: Add Generic REST API Connector

## What problem does this PR solve?

RAGFlow supports many specific data source connectors (MySQL, Slack,
Google Drive, etc.), but there was no way to connect an arbitrary REST
API as a data source. Users with custom or third-party APIs had to write
a new connector class for each one.

This PR adds a **generic, configuration-driven REST API connector** that
lets users connect any REST API as a data source entirely through the UI
— no code changes needed per API.

---

## Features

### Core Connector (`common/data_source/rest_api_connector.py`)

- Implements `LoadConnector` and `PollConnector` interfaces for full and
incremental sync
- **Configurable authentication:** None, API Key (custom header), Bearer
Token, Basic Auth
- **Pluggable pagination:** Page-based, Offset-based, Cursor-based, or
None
- Smart page-size inference from user's query parameters to avoid
duplicate/conflicting params
- Configurable request delay between pages to prevent API rate limiting
- Auto-detection of the items array in JSON responses (`items`,
`results`, `data`, `records`, or first list found)
- **Advanced field mapping** with dot-notation (`country.name`), array
wildcards (`newsType[*].name`), type hints, and default values
- Optional content template rendering (`"Title: {title}\nBody: {body}"`)
- HTML stripping for content fields
- Stable document IDs via `hash128` from a configurable ID field or
auto-generated from item content
- Pydantic configuration schema with automatic coercion of UI string
inputs to dicts/lists

### Backend Registration (`rag/svr/sync_data_source.py`,
`common/constants.py`, `common/data_source/config.py`)

- `REST_API` sync class wired into RAGFlow's `func_factory`
- Full sync (`load_from_state`) and incremental polling (`poll_source`)
support
- Credentials and config passed from task to connector following
existing patterns (MySQL, SeaFile, etc.)

### Test Connection Endpoint (`api/apps/connector_app.py`)

- `POST /v1/connector/<id>/test` validates config schema,
authentication, and API connectivity without triggering a sync
- Clear error messages for auth failures vs. config issues

### Frontend UI (`web/src/pages/user-setting/data-source/constant/`)

- **Postman-style configuration:** Base URL, Query Parameters (key=value
per line), Auth, Content Fields, Metadata Fields, Pagination Type
- Auth-type-aware form: fields for API key header/value, Bearer token,
or Basic username/password appear only when relevant
- **Advanced Settings** toggle for: Custom Headers, Max Pages, Request
Delay, Poll Timestamp Field, Request Body (POST)
- Connector icon (SVG) and i18n strings (English)
- **"Test Connection"** button to validate before syncing

---

## Controls & Safety

- Configurable max pages safety cap (default: 1000, adjustable in UI)
- Configurable request delay between pages (default: 0.5s, adjustable in
UI)
- Auth errors (401/403) fail immediately without retries; transient
errors retry with exponential backoff
- Diagnostic logging: auth setup confirmation, request details on
failure, content field extraction status

---

## Type of change

- [x] New Feature (non-breaking change which adds functionality)


##Visual Screenshots of Features
<img width="482" height="510" alt="Screenshot 2026-03-11 at 5 19 52 PM"
src="https://github.com/user-attachments/assets/dcb7ab4a-1622-44f3-bb02-d6f0527314c4"
/>
(Connector can be configured within the external data sources tab)

Configuration Parameters:
<img width="661" height="682" alt="Screenshot 2026-03-11 at 5 20 46 PM"
src="https://github.com/user-attachments/assets/5e154e71-4ab5-4872-bfb2-04f02b73c18a"
/>
<img width="661" height="682" alt="Screenshot 2026-03-11 at 5 20 54 PM"
src="https://github.com/user-attachments/assets/00cb14b7-0bcf-4b94-9d71-34e93369ecb2"
/>

Connection can be tested before attaching to dataset:
<img width="981" height="681" alt="Screenshot 2026-03-11 at 5 21 40 PM"
src="https://github.com/user-attachments/assets/aaa6eeeb-89a7-4349-bc34-2423bf8be9ee"
/>

Ingestion tested with API connector (works perfectly fine):
<img width="1062" height="705" alt="Screenshot 2026-03-11 at 5 22 30 PM"
src="https://github.com/user-attachments/assets/afcd0d58-cadd-4152-badc-d2f14d96fbec"
/>

Search & Retrieval works as well with metadata flow:
<img width="1062" height="705" alt="Screenshot 2026-03-11 at 5 23 05 PM"
src="https://github.com/user-attachments/assets/d41ee935-dcf7-4456-b317-22a76ca032c0"
/>

---------

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-05-13 20:35:01 +08:00
Jin Hai
30d1c1dc28 Fix go compilation (#14900)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 20:05:56 +08:00
jony376
7f699d1202 Fix: enforce tenant authorization for tenant_rerank_id in retrieval flows (#14782)
### Related issues

Closes #14781 

### What problem does this PR solve?

Some retrieval endpoints accepted caller-supplied `tenant_rerank_id` and
resolved it through `get_model_config_by_id(...)`. That helper loaded
`TenantLLM` rows by global database id and returned decoded model
configuration without checking whether the model belonged to the
authenticated tenant or the dataset owner tenant.

This meant dataset access was validated, but rerank-model selection was
not. A caller who knew or could guess another tenant's
`tenant_rerank_id` could attempt retrieval with a foreign rerank model
config, creating a cross-tenant authorization gap for model usage.

This PR closes that gap by making `tenant_rerank_id` resolution
tenant-aware across the retrieval paths that accept it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Solution

- Extend `get_model_config_by_id(...)` to accept an optional
`allowed_tenant_ids` set and reject `TenantLLM` rows whose `tenant_id`
is outside that set.
- Pass the allowed tenant scope from retrieval endpoints that accept
`tenant_rerank_id`:
  - `api/apps/sdk/doc.py`
  - `api/apps/sdk/session.py`
  - `api/apps/services/dataset_api_service.py`
- Use the authenticated tenant plus dataset-owner tenant ids already
derived by each retrieval flow as the authorization boundary for rerank
model selection.
- Add focused unit coverage to assert unauthorized `tenant_rerank_id`
values are rejected and that the allowed tenant set is propagated
correctly.

### Testing

- `python -m py_compile` on:
  - `api/db/joint_services/tenant_model_service.py`
  - `api/apps/services/dataset_api_service.py`
  - `api/apps/sdk/doc.py`
  - `api/apps/sdk/session.py`
- Added unit tests in:
-
`test/testcases/test_http_api/test_file_management_within_dataset/test_doc_sdk_routes_unit.py`
-
`test/testcases/test_http_api/test_session_management/test_session_sdk_routes_unit.py`

### Notes for reviewers

- This change is intentionally narrow: it affects only the
`tenant_rerank_id` path, not the normal `rerank_id` name-based
resolution path.
- Local lint/syntax checks passed.
- Full pytest execution could not be completed in this environment
because the local test runtime is missing `strenum`, so the route-test
files fail during collection before exercising the updated cases.

---------

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-13 19:53:08 +08:00
writinwaters
bb1a6259d3 Docs: Updated v0.25.3 release notes draft (#14899)
### What problem does this PR solve?

Draft v0.25.3 release notes

### Type of change


- [x] Documentation Update
2026-05-13 19:49:07 +08:00
Jin Hai
87516edadf Bump to infinity v0.7.0-dev7 (#14897)
### What problem does this PR solve?

Upgrade infinity

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:42:50 +08:00
writinwaters
9ed4da74b8 Docs: Draft 0.25.3 release notes (#14898)
### What problem does this PR solve?

A draft 0.25.3 release note.

### Type of change


- [x] Documentation Update

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:35:55 +08:00
tmimmanuel
0a4b733b2a Go: implement Rerank in LocalAI driver (#14813)
### What problem does this PR solve?

The LocalAI Go driver landed in #14809 and Embed landed in #14811.
`Rerank` was left as a stub that returns `"not implemented"`. This PR
fills the gap.

LocalAI exposes a public rerank endpoint at `<tenant-url>/v1/rerank`
with a Cohere-shaped request and response (`{model, query, documents,
top_n}` → `{results: [{index, relevance_score}]}`). The Python side has
had `LocalAIRerank` in `rag/llm/rerank_model.py` for a long time. Until
this PR, a tenant who wanted to use LocalAI for reranking in the Go
layer got `"not implemented"`.

### What this PR includes

- `conf/models/localai.json`: add `"rerank": "rerank"` under
`url_suffix` so the driver can build the URL from config. This matches
the `URLSuffix.Rerank` field already used by aliyun and siliconflow.
- `internal/entity/models/localai.go`: replace the `Rerank` stub with a
real implementation that POSTs to `/v1/rerank`. Adds local
request/response types `localAIRerankRequest` and
`localAIRerankResponse`.

No factory change. No interface change.

### How the implementation works

- Validate the model name and resolve the tenant-supplied base URL with
the existing `resolveBaseURL` helper.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern `ChatWithMessages`,
`ListModels`, and `Embed` already use in this file.
- Only set the `Authorization` header when a non-empty API key was
supplied. LocalAI accepts an empty key by default, so this preserves the
optional-auth contract.
- Default `top_n` to `len(documents)` when `rerankConfig.TopN == 0`,
matching the existing Aliyun and SiliconFlow rerank implementations.
- Validate every `results[].index` against `len(documents)`. If the
upstream returns an out-of-range index, fail clearly instead of silently
writing past the slice.
- An empty `documents` slice returns `&RerankResponse{}` with no HTTP
call.
- Non-200 responses propagate the upstream status line and body.

### Note on stacking

This PR builds on #14809 (LocalAI driver) and #14811 (LocalAI Embed).
Until both merge, this PR's diff on GitHub will include all three
commits. After #14809 and #14811 land on `main`, GitHub will auto-reduce
this PR to only the `Rerank` changes (one commit, ~99 line diff in
`localai.go` plus 1 line in `localai.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `LocalAIModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Aliyun Rerank
(`internal/entity/models/aliyun.go`) and SiliconFlow Rerank
(`internal/entity/models/siliconflow.go`) implementations.

Closes #14812
Depends on #14809, #14811
Tracking: #14736

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 19:35:19 +08:00
Liu An
3182fd0789 Docs: Update version references to v0.25.3 in READMEs and docs (#14896)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.2 to v0.25.3
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-13 18:42:42 +08:00
Wang Qi
f3b3596c29 Speed up ragflow server (#14894)
### What problem does this PR solve?

Speed up ragflow server

### Type of change

- [ ] Refactoring
2026-05-13 18:01:33 +08:00
Jin Hai
b18640d228 Go: fix OCR command (#14891)
### What problem does this PR solve?

RAGFlow(user)> ocr with 'hunyuanocr@test@gitee' file './picture.png'
+----------------------------------------------------------+
| text                                                     |
+----------------------------------------------------------+
| 生活不是等待风暴过去,而是学会在雨中翩翩起舞。

——佚名                                                       |
+----------------------------------------------------------+

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 17:29:53 +08:00
buua436
8cb2bf04fb Fix: llm add api key overridden (#14885)
### What problem does this PR solve?

llm add api key overridden

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 17:15:32 +08:00
Panda Dev
bf90c8948a Go: implement ListModels in ZhipuAI driver (#14886)
### What problem does this PR solve?

Fixes #14884

The ZhipuAI Go driver in `internal/entity/models/zhipu-ai.go` had a stub
`ListModels` method that always returned `"zhipu-ai, no such method"`.
The DeepSeek, Gitee, NVIDIA, OpenAI, SiliconFlow, and OpenRouter drivers
in the same package already implement `ListModels` against the
OpenAI-compatible `/models` endpoint, and the model picker UI relies on
it. This PR brings ZhipuAI in line with that pattern.

### Changes

- `internal/entity/models/zhipu-ai.go`: implement
`ZhipuAIModel.ListModels`.
  - Resolve region with default fallback.
- GET `${BaseURL[region]}/${URLSuffix.Models}` (resolves to
`https://open.bigmodel.cn/api/paas/v4/models` with the default region).
- Send `Authorization: Bearer <api_key>` when an API key is configured.
Omit the header when the key is empty, so an unauthenticated caller gets
a clear `401` from upstream.
- Surface non-200 responses with the upstream status line and body,
matching the other Go drivers.
- Parse the response via the package-level `DSModelList` / `DSModel`
types already used by DeepSeek, Gitee, and SiliconFlow.
- When the response includes `owned_by`, render the entry as
`id@owned_by`, matching the convention of Gitee and SiliconFlow.
- `conf/models/zhipu-ai.json`: add `"models": "models"` to `url_suffix`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-13 16:39:14 +08:00
chanx
bbb0798c41 Fix: Set embedded models during form initialization. (#14889)
### What problem does this PR solve?

Fix: Set embedded models during form initialization.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 16:34:25 +08:00
tmimmanuel
8b53960819 Go: implement provider: LongCat (#14809)
### What problem does this PR solve?

Add a Go driver for LongCat (Meituan, https://longcat.chat), one of the
unchecked providers on the umbrella tracking issue #14736. LongCat
exposes an OpenAI-compatible REST API at
`https://api.longcat.chat/openai/v1` with three public chat models
including `LongCat-Flash-Thinking`, a reasoning model that returns
chain-of-thought in `reasoning_content` (OpenAI o-series shape).

Until this PR, a tenant who configured `longcat` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/longcat.go` with a `LongCatModel`
implementing the `ModelDriver` interface.
- New `conf/models/longcat.json` with the 3 public chat models
(Flash-Chat, Flash-Lite, Flash-Thinking) and `url_suffix` for `chat` and
`models`.
- `factory.go`: route `"longcat"` to `NewLongCatModel`.

Method coverage:

- `ChatWithMessages`: `POST /openai/v1/chat/completions`, non-streaming
- `ChatStreamlyWithSender`: SSE stream against the same endpoint
- `ListModels` / `CheckConnection`: `GET /openai/v1/models`
- **Reasoning extraction**: `message.reasoning_content` (non-stream) and
`delta.reasoning_content` (stream) flow into
`ChatResponse.ReasonContent` / the sender's second arg. Matches the
OpenAI o-series convention also used by kimi-k2.6 and DeepSeek-R1.
- **`reasoning_effort` propagation**: `ChatConfig.Effort` → request body
`reasoning_effort` (LongCat-Flash-Thinking honors it; non-reasoning
models ignore it).
- `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` return `"no such method"` (LongCat does not expose any of
these surfaces).

No interface change. No new dependencies.

### How was this tested?

**21 unit tests** in `internal/entity/models/longcat_test.go` — all
pass:

```
$ go test -vet=off -run TestLongCat -count=1 -v ./internal/entity/models/...
=== RUN   TestLongCatName
--- PASS: TestLongCatName (0.00s)
=== RUN   TestLongCatChatHappyPath
--- PASS: TestLongCatChatHappyPath (0.00s)
=== RUN   TestLongCatChatExtractsReasoningContent
--- PASS: TestLongCatChatExtractsReasoningContent (0.00s)
=== RUN   TestLongCatChatPropagatesReasoningEffort
--- PASS: TestLongCatChatPropagatesReasoningEffort (0.00s)
=== RUN   TestLongCatChatOmitsReasoningEffortWhenUnset
--- PASS: TestLongCatChatOmitsReasoningEffortWhenUnset (0.00s)
=== RUN   TestLongCatChatRequiresAPIKey
--- PASS: TestLongCatChatRequiresAPIKey (0.00s)
=== RUN   TestLongCatChatRequiresMessages
--- PASS: TestLongCatChatRequiresMessages (0.00s)
=== RUN   TestLongCatChatRejectsHTTPError
--- PASS: TestLongCatChatRejectsHTTPError (0.00s)
=== RUN   TestLongCatStreamHappyPath
--- PASS: TestLongCatStreamHappyPath (0.00s)
=== RUN   TestLongCatStreamExtractsReasoningContent
--- PASS: TestLongCatStreamExtractsReasoningContent (0.00s)
=== RUN   TestLongCatStreamRejectsExplicitFalse
--- PASS: TestLongCatStreamRejectsExplicitFalse (0.00s)
=== RUN   TestLongCatStreamRequiresSender
--- PASS: TestLongCatStreamRequiresSender (0.00s)
=== RUN   TestLongCatStreamFailsWithoutTerminal
--- PASS: TestLongCatStreamFailsWithoutTerminal (0.00s)
=== RUN   TestLongCatListModelsHappyPath
--- PASS: TestLongCatListModelsHappyPath (0.00s)
=== RUN   TestLongCatListModelsRequiresAPIKey
--- PASS: TestLongCatListModelsRequiresAPIKey (0.00s)
=== RUN   TestLongCatCheckConnectionDelegatesToListModels
--- PASS: TestLongCatCheckConnectionDelegatesToListModels (0.00s)
=== RUN   TestLongCatEmbedReturnsNoSuchMethod
--- PASS: TestLongCatEmbedReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatRerankReturnsNoSuchMethod
--- PASS: TestLongCatRerankReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatBalanceReturnsNoSuchMethod
--- PASS: TestLongCatBalanceReturnsNoSuchMethod (0.00s)
=== RUN   TestLongCatAudioOCRReturnNoSuchMethod
--- PASS: TestLongCatAudioOCRReturnNoSuchMethod (0.00s)
PASS
ok      ragflow/internal/entity/models  0.020s
```

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.longcat.chat`:

```
=== RUN   TestLongCatLiveSmoke
    [OK] Name() = "longcat"
    [OK] CheckConnection
    [OK] ListModels: 5 models -> [LongCat-Flash-Lite LongCat-Flash-Chat LongCat-Flash-Thinking-2601 LongCat-Flash-Omni-2603 LongCat-2.0-Preview]
    [OK] Chat (Flash-Chat) answer="Got it! Let me know if you" reason=""
    [OK] Chat (Flash-Thinking) answer len=443 head="To find 15 % of 80, follow these steps:\n\n1. **Convert the percentage to a frac..."
                  ReasonContent len=557 head="The user asks: \"15% of 80?\" They want step by step reasoning and final answer in \\boxed{}. So we need to compute 15% of ..."
    [OK] Stream content: 78 chunks, 351 chars
    [OK] Stream reasoning: 107 chunks, 537 chars
    [OK] Balance returns longcat, no such method
    [OK] Embed returns longcat, no such method
    [OK] Rerank returns longcat, no such method

LONGCAT LIVE SMOKE PASSED
--- PASS: TestLongCatLiveSmoke (31.01s)
```

What the live run proves on the wire:

- Auth header (`Bearer <key>`) is accepted by `api.longcat.chat`.
- `/openai/v1/models` parser handles the real 5-model response (note:
live API returns versioned aliases `LongCat-Flash-Thinking-2601`,
`LongCat-Flash-Omni-2603`, `LongCat-2.0-Preview` plus the un-versioned
`LongCat-Flash-Chat` and `LongCat-Flash-Lite`).
- Non-stream chat against `LongCat-Flash-Chat`: visible answer parses
correctly, `ReasonContent` correctly empty.
- Non-stream chat against `LongCat-Flash-Thinking`: 443-char answer
flows into `Answer`, 557-char chain-of-thought flows into
`ReasonContent` via the new `message.reasoning_content` extraction.
- Streaming chat against `LongCat-Flash-Thinking`: 107 reasoning chunks
(537 chars) reach the sender's second arg via `delta.reasoning_content`;
78 content chunks (351 chars) reach the first arg. Before this code, the
reasoning chunks would have been silently dropped.
- All sentinel methods (Balance, Embed, Rerank, audio/OCR) return the
documented `"no such method"` strings.

### Note on PR history

This branch was previously named for LocalAI work which is now
consolidated into PR #14813. The branch was reset to `upstream/main` and
rebuilt for LongCat. The diff against `main` is a clean +969 lines
across 4 files.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-13 16:27:56 +08:00
Wang Qi
ff685d3131 Delete duplicate route (#14883)
### What problem does this PR solve?

The delete /graph is duplicated of
`/datasets/<dataset_id>/<index_type>`, delete it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 15:57:44 +08:00
Idriss Sbaaoui
09e1fd290a Chore: migrate tests to restful api (#14871)
### What problem does this PR solve?

add new testing suite for the new restful api endpoints meant to replace
http and web api tests

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test
2026-05-13 15:07:23 +08:00
tmimmanuel
d63d3bb7d2 Go: implement provider: Novita.ai (#14850)
### What problem does this PR solve?

Add a Go driver for Novita.ai (https://novita.ai), one of the unchecked
providers on the umbrella tracking issue #14736. Novita exposes an
OpenAI-compatible REST API at `https://api.novita.ai/v3/openai` and
proxies a large catalog of third-party models (DeepSeek, Llama, Qwen3,
Kimi, Gemma, Mistral, MiniMax, GLM, etc.) behind a single OpenAI-shaped
surface — 102 models live at the time of writing.

Until this PR, a tenant who configured `novita` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver.

### What this PR includes

- New `internal/entity/models/novita.go` with a `NovitaModel`
implementing the `ModelDriver` interface (~520 lines).
- New `conf/models/novita.json` with 7 representative chat models
(DeepSeek-V4, Llama-3.3-70B, Qwen3-30B/235B reasoning, Kimi-K2,
Gemma-3-27B, Mistral-Nemo).
- `factory.go`: route `"novita"` to `NewNovitaModel`.
- `internal/entity/models/novita_test.go`: 23 unit tests.

### Notable design point: `<think>...</think>` reasoning extraction

Novita-routed reasoning models like `qwen3-*` and `deepseek-r1-*` embed
their chain-of-thought **inline inside content as `<think>...</think>`
tags**, rather than in a separate `reasoning_content` field. Verified
live by probing `api.novita.ai`:

```
content head 200: <think>
Okay, let's see. I need to find 15% of 80. Hmm, percentages can sometimes be tricky, but I think
content tail 100: h, that works.
Alternatively, 0.15 × 80. If I move the decimal two places to the left for </think>
```

Without handling, a tenant picking qwen3 via Novita would see raw
`<think>` tags in their UI answer — different from every other reasoning
provider in the Go layer.

The driver detects those tags and routes the inner text to
`ChatResponse.ReasonContent` (non-stream) or the sender's second arg
(stream), keeping the visible answer clean of tag clutter:

- **`splitNovitaThink`** — scans a complete content string. Used by the
non-streaming path. Handles multiple `<think>` blocks, unclosed tags
(the model got cut off mid-reasoning), pure-text content with no tags.
- **`novitaThinkExtractor`** — stateful streaming version. Buffers
trailing bytes that might be the start of a tag (e.g. `<thi` held back
when the next chunk completes `nk>`), then emits segments in routing
order so callers can pipe them to a UI. Tested with byte-level chunk
boundaries and tag-spanning scenarios.

### Method coverage

| Method | Behavior |
|---|---|
| `ChatWithMessages` | `POST /v3/openai/chat/completions`, `<think>`
extraction on response |
| `ChatStreamlyWithSender` | SSE stream, stateful `<think>` extraction
across deltas |
| `ListModels` / `CheckConnection` | `GET /v3/openai/models` (102 live)
|
| `Embed` / `Rerank` / `Balance` / `TranscribeAudio` / `AudioSpeech` /
`OCRFile` | `"no such method"` — Novita's OpenAI-compatible surface does
not expose any |

No interface change. No new dependencies.

### How was this tested?

**23 unit tests** in `internal/entity/models/novita_test.go` — all pass:

```
$ go test -vet=off -run "TestNovita|TestSplitNovita" -count=1 ./internal/entity/models/...
ok      ragflow/internal/entity/models  0.020s
```

Coverage:

- `splitNovitaThink` (5 cases: pure text, single block, leading text,
multiple blocks, unclosed tag)
- `novitaThinkExtractor` (6 cases: single-chunk, opening tag span,
closing tag span, byte-level chunking, no tags, lone `<` not as tag
start)
- `ChatWithMessages`: pure text, with `<think>` tags, missing API key,
empty messages, HTTP error
- `ChatStreamlyWithSender`: tag-stripping with spanning deltas, pure
content, sender-required, stream-true-required
- `ListModels` / `CheckConnection` (happy paths)
- All sentinel methods

`go build ./internal/entity/models/...` exits 0 on go 1.25.

**Live integration test** against `api.novita.ai/v3/openai`:

```
=== RUN   TestNovitaLiveSmoke
    [OK] Name() = "novita"
    [OK] CheckConnection
    [OK] ListModels: 102 models (showing first 6) [deepseek/deepseek-v4-pro deepseek/deepseek-v4-flash deepseek/deepseek-v3.2 xiaomimimo/mimo-v2.5-pro moonshotai/kimi-k2.6 zai-org/glm-5.1]
    [OK] Chat (llama-3.3) answer="ok" reason=""
    [OK] Chat (qwen3) answer len=0 head=""
                  ReasonContent len=1657 head="Okay, so I need to figure out what 15% of 80 is. Hmm, percentages can sometimes trip me up, but let ..."
    [OK] Stream content: 0 chunks, 0 chars; reasoning: 600 chunks, 1667 chars
    [OK] Embed/Rerank/Balance/TranscribeAudio/AudioSpeech/OCRFile all return "novita, no such method"

NOVITA LIVE SMOKE PASSED
--- PASS: TestNovitaLiveSmoke (26.18s)
```

What the live run proves on the wire:

- Auth (`Bearer <key>`) accepted by `api.novita.ai`.
- `/v3/openai/models` parser handles the real 102-model response.
- Non-stream chat against `meta-llama/llama-3.3-70b-instruct`: clean
string answer, empty ReasonContent (non-reasoning model, pure-text
path).
- Non-stream chat against `qwen/qwen3-30b-a3b-fp8`: 1657-char reasoning
extracted from `<think>...</think>` and routed to
`ChatResponse.ReasonContent`. Visible answer is 0 chars in this run
because qwen3 spent its 600-token budget entirely on reasoning before
reaching the answer phase — that's the model's behavior, not a driver
bug. The important thing: **no `<think>` tags leaked into the visible
Answer field**.
- Streaming against qwen3: 600 reasoning chunks (1667 chars) emitted via
the sender's 2nd arg across SSE deltas; **no `<think>` tag fragments
leaked into the content channel** despite tag boundaries crossing chunk
boundaries on the wire.
- All 6 sentinel methods return the documented `"no such method"`
strings.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Tracking: #14736
2026-05-13 14:10:50 +08:00
Jackie
71d327b11c Fix: The text field resizing function in the knowledge block creation… (#14212)
… modal

- Add vertical resizing functionality for the text field

### What problem does this PR solve?

_Fix the issue where the text content of the knowledge base editing
parsing block is too long to scroll._

<img width="701" height="775" alt="image"
src="https://github.com/user-attachments/assets/b258422e-fbc1-466d-abab-062e642c21d5"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: chenyun <chenyun@chenyundemacbook-pro.local>
2026-05-13 13:57:05 +08:00
Wang Qi
45d676bc05 Fix delete graphrag not take effect in UI (#14879)
### What problem does this PR solve?

Fix delete graphrag not take effect in UI

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 13:49:16 +08:00
Joseff
733d75d6a7 Fix(Go): make Baidu Encode fail loudly on malformed responses (#14721)
### What problem does this PR solve?

The Baidu (Qianfan) `Encode` method silently swallowed malformed
responses. If a `data[]` item from the API was missing a field (`index`,
`embedding`, or unexpected shape), the loop did `continue` instead of
returning an error, leaving `nil` entries in the result slice. Callers
got back partial results with no indication anything went wrong, which
then crashes downstream consumers when they try to use a `nil` vector.

Concrete gaps fixed:

- No count-mismatch check between `data` length and input texts (only
checked for empty)
- No duplicate-index detection (a duplicate would silently overwrite)
- No missing-index final scan
- No empty-embedding rejection
- No per-call context timeout
- `EmbeddingConfig.Dimension` (added in #14735) was not propagated

This PR replaces `map[string]interface{}` parsing with a typed
`baiduEmbeddingResponse` struct, applies the standard four-layer
validation (count → out-of-range → duplicate → empty → final
missing-index scan), adds `context.WithTimeout(nonStreamCallTimeout)`,
and forwards `embeddingConfig.Dimension` as the `dimensions` parameter
(Baidu Qianfan v2 uses an OpenAI-compatible API).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 12:54:00 +08:00
shawnxiao105-afk
8b6dd6a5c2 fix: guard whitespace-only chunks before embedding (#13938)
## Problem

When parsing DOCX files with many tables, DeepDOC generates chunks
containing only empty HTML table tags, such as:

```html
<table><tr><td></td></tr><tr><td></td></tr><tr><td></td></tr><tr><td></td></tr></table>
```

After the regex cleanup at `task_executor.py:584`, this becomes `" "`
(whitespace only).

The guard at line 585 (`if not c`) only catches empty strings `""`, but
whitespace strings are truthy in Python and pass through. When sent to
Zhipu `embedding-3` API, it rejects them with error 1213:
`未正常接收到prompt参数`.

## Root Cause

```python
c = re.sub(r"</?(table|td|caption|tr|th)( [^<>]{0,12})?>", " ", c)
if not c:       # ← only catches "", not "   " / "\n" / "\t"
    c = "None"
```

Verified with Zhipu `embedding-3`:
| Input | Result |
|---|---|
| `""` | error 1213 |
| `" "` | error 1213 |
| `"\n"` | error 1213 |
| `"None"` | OK |

## Fix

```diff
- if not c:
+ if not c.strip():
      c = "None"
```

## Testing

Reproduced with a 678KB DOCX file (166 tables, 270 chunks). Chunk #89 is
the empty table above. After fix, `"None"` is sent instead and embedding
succeeds.

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-13 11:47:50 +08:00
Wang Qi
64bd0130d3 Add REST API backward compatibility (#14872)
### What problem does this PR solve?

Add REST API backward compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 11:44:40 +08:00
dale053
5a5e766386 fix(api): authorize owner_ids for list chats and search apps (#14775)
Closes #14768
### What problem does this PR solve?
The `list_chats` and `list_searches` REST API endpoints did not enforce
authorization on the `owner_ids` query parameter. Any authenticated user
could pass arbitrary tenant IDs to `owner_ids` and retrieve chats or
search apps belonging to other tenants they are not a member of.

This PR resolves the issue by:
1. Looking up the current user's authorized tenants via
`TenantService.get_joined_tenants_by_user_id` and rejecting any
`owner_ids` that fall outside that set.
2. When no `owner_ids` are provided, scoping the query to only the
user's authorized tenants instead of returning an unfiltered result.
3. Adding unit tests that verify unauthorized `owner_ids` are rejected
with `OPERATING_ERROR`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-13 09:43:44 +08:00
Paul Yao
c34c81e8e6 fix: remove duplicate .wav and .aac in audio supported extensions list (#14791)
What problem does this PR solve?

In rag/app/audio.py, the supported audio extensions list contains
duplicate entries: .wav appears twice (positions 3 and 5) and .aac
appears twice (positions 6 and 14). While this does not affect runtime
behavior, it is redundant and makes the code harder to maintain.

This PR removes the duplicate entries to keep the list clean and
consistent.

Type of change

 - [X]  Bug Fix (non-breaking change which fixes an issue)
2026-05-13 09:42:31 +08:00
writinwaters
5e46457c28 Docs: How to add Bitbucket as data source. (#14846)
### What problem does this PR solve?

Added a guide on integrating Bitbucket as an external data source.

### Type of change

- [x] Documentation Update
2026-05-12 20:48:30 +08:00
Jin Hai
ad4717f40a Go: fix model type check when use the model (#14843)
### What problem does this PR solve?
```
RAGFlow(user)> chat with 'glm-ocr@test@zhipu-ai' message 'what is this'
CLI error: expect  model glm-ocr@zhipu-ai is a chat or multimodal model
```
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 19:44:01 +08:00
Wang Qi
76d5240fb5 Fix #14801 to allow search dataset list when add (#14841)
### What problem does this PR solve?

Fix #14801 to allow search dataset list when add, following on #14825

<img width="2172" height="857" alt="image"
src="https://github.com/user-attachments/assets/65ea7647-56f4-4c16-8437-121b834811f0"
/>


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 19:36:23 +08:00
balibabu
3f41f8cfae Feat: When a Wait Node precedes a Message Node within a Loop Node, the outgoing message is split into two separate messages. (#14839)
### What problem does this PR solve?

Feat: When a Wait Node precedes a Message Node within a Loop Node, the
outgoing message is split into two separate messages.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-05-12 18:48:44 +08:00
0xτensor
127aeac4aa fix: expose gpt-5.5 and gpt-5.4 in OpenAI model list (#14828)
### What problem does this PR solve?

OpenAI model catalogs used in provider selection flows were missing the
latest GPT models (`gpt-5.5` and `gpt-5.4`).
Because model availability is driven by seeded catalog data
(`conf/llm_factories.json` → DB seed → API response), these models were
not selectable in the UI or `/llm/list` responses.

This PR updates and synchronizes the OpenAI catalog definitions across
configuration sources and ensures the new models are correctly exposed
through the API layer and validated in tests.

---

### Type of change

* [x] New Feature (non-breaking change which adds functionality)

---

### Changes Made

* Added `gpt-5.5` and `gpt-5.4` to OpenAI catalog definitions in:

  * `conf/llm_factories.json`
  * `conf/models/openai.json` (chat + vision support)
* Ensured consistency between DB-seeded factory config and provider
model configuration
* Updated test coverage in:

  * `test_llm_list_unit.py`

    * seeded OpenAI catalog entries
* added response-level assertion validating `/llm/list` includes both
new model IDs under OpenAI grouping

---

### Root Cause

OpenAI model listings in selection flows are generated from catalog data
seeded via `conf/llm_factories.json`.
The catalog had not been updated to include the latest GPT models,
resulting in missing availability in UI and API responses.

---

### Testing

* Created isolated test environment:

  * `python -m venv .venv-review`
  * installed `pytest`
* Ran targeted and full test suite:

  * `test_list_app_grouping_availability_and_merge`:  passed
  * Full `test_llm_list_unit.py`:  10 passed

---

### Risks / Limitations

* Adding models to the catalog does not guarantee upstream provider
availability or account entitlement.
* Environments with pre-seeded DB catalogs may require reseed or refresh
to reflect updated configuration.

---

### Notes

* Changes are minimal and scoped strictly to catalog configuration and
related test coverage.
* Ensures `/llm/list` API remains aligned with expected latest OpenAI
model availability.
* Closes #14827
2026-05-12 18:03:47 +08:00
Haruko386
45ee5ca9cd Go: implement provider: Jina (#14838)
### What problem does this PR solve?

This PR completes the Jina provider

**The following functionalities are now supported:**

**Jina:**
- [ ] Chat / Stream Chat (Not available for now: [(Jina chat API
docs)](https://api.jina.ai/docs#/Search%20Foundation%20Models/chat_completions_v1_chat_completions_post))
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'jina-embeddings-v2-base-en@test@jina' dimension 16
+-----------+-------+
| dimension | index |
+-----------+-------+
| 768       | 0     |
| 768       | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'jina-reranker-v2-base-multilingual@test@jina' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.74316794      |
| 2     | 0.18713269      |
| 1     | 0.15817434      |
+-------+-----------------+

RAGFlow(user)> list supported models from 'jina' 'test'
+---------------------------------------------+
| model_name                                  |
+---------------------------------------------+
| Jina AI: Jina VLM                           |
| Jina AI: Jina Reranker v3                   |
| Jina AI: Jina Code Embeddings 0.5b          |
| Jina AI: Jina Code Embeddings 1.5b          |
| Jina AI: Jina Embeddings v4                 |
| Jina AI: Jina Reranker M0                   |
| Jina AI: ReaderLM v2                        |
| Jina AI: Jina Clip v2                       |
| Jina AI: Jina Embeddings v3                 |
| Jina AI: Jina Colbert v2                    |
| Jina AI: Reader LM 0.5b                     |
| Jina AI: Reader LM 1.5b                     |
| Jina AI: Jina Reranker v2 Base Multilingual |
| Jina AI: Jina Clip v1                       |
| Jina AI: Jina Reranker v1 Tiny EN           |
| Jina AI: Jina Reranker v1 Turbo EN          |
| Jina AI: Jina Reranker v1 Base EN           |
| Jina AI: Jina Colbert v1 EN                 |
| Jina AI: Jina Embeddings v2 Base ES         |
| Jina AI: Jina Embeddings v2 Base Code       |
| Jina AI: Jina Embeddings v2 Base DE         |
| Jina AI: Jina Embeddings v2 Base ZH         |
| Jina AI: Jina Embeddings v2 Base EN         |
| Jina AI: Jina Embedding B EN v1             |
| Jina AI: Jina Embeddings v5 Text Small      |
| Jina AI: Jina Embeddings v5 Omni Small      |
| Jina AI: Jina Embeddings v5 Omni Nano       |
| Jina AI: Jina Embeddings v5 Text Nano       |
+---------------------------------------------+

RAGFlow(user)> check instance 'test' from 'jina'
SUCCESS
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-12 18:03:05 +08:00
tmimmanuel
7d3836907a Go: implement Embed (embeddings) in Mistral driver (#14807)
### What problem does this PR solve?

The Mistral Go driver landed in #14805 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

`conf/models/mistral.json` did not list any embedding model out of the
box, so a tenant who wanted to use Mistral end to end (chat +
embeddings) could not run an embedding call. This PR adds
`mistral-embed` to the config and a real `/v1/embeddings`
implementation.

### What this PR includes

- `conf/models/mistral.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, siliconflow,
zhipu-ai), and add a `mistral-embed` entry under `models`
(1024-dimensional vectors, 8192 max input tokens).
- `internal/entity/models/mistral.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/embeddings`. Adds local response
types `mistralEmbeddingData` and `mistralEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Mistral API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14805 (the Mistral driver). Until #14805 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14805 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~111 line diff in `mistral.go` plus 8
lines in `mistral.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `MistralModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing OpenAI Embed implementation
(`internal/entity/models/openai.go`).

Closes #14806
Depends on #14805
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 17:45:48 +08:00
buua436
14332dd75c Go: fix dataset time unit (#14837)
### What problem does this PR solve?

fix dataset time unit

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 17:22:16 +08:00
Jin Hai
d08bf02d9b Go: add ASR, TTS, OCR command (#14836)
### What problem does this PR solve?

```
RAGFlow(user)> asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method
RAGFlow(user)> stream asr with 'glm-asr-2512@test@zhipu-ai' audio './speech.wav';
CLI error: zhipu, no such method

RAGFlow(user)> tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> stream tts with 'glm-tts@test@zhipu-ai' text 'how are you';
CLI error: zhipu, no such method

RAGFlow(user)> ocr with 'glm-ocr@test@zhipu-ai' file './test.log';
CLI error: zhipu, no such method
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 17:17:44 +08:00
buua436
9ee481807f GO: implement GET /api/v1/datasets/:dataset_id (#14834)
### What problem does this PR solve?

implement GET /api/v1/datasets/:dataset_id

### Type of change

- [x] Refactoring
2026-05-12 17:16:48 +08:00
Wang Qi
4374e07a29 Speed up start time (#14833)
### What problem does this PR solve?

Speed up start time

### Type of change
- [x] Refactoring
2026-05-12 17:00:45 +08:00
tmimmanuel
eaa2e46b1e Go: implement Embed (embeddings) in Upstage driver (#14819)
### What problem does this PR solve?

The Upstage Go driver landed in #14817 with chat, list models, and check
connection. `Embed` was left as a stub that returns `"not implemented"`.
This PR fills the gap.

Upstage exposes an OpenAI-compatible embeddings endpoint at
`https://api.upstage.ai/v1/solar/embeddings` via the
`solar-embedding-1-large` family (`solar-embedding-1-large-query` for
queries, `solar-embedding-1-large-passage` for passages), and the Python
side has had `UpstageEmbed(OpenAIEmbed)` in `rag/llm/embedding_model.py`
for a long time targeting this same path. The existing
`conf/models/upstage.json` did not list any embedding model out of the
box, so a tenant who wanted to use Upstage end to end could not run an
embedding call. This PR fills the gap.

### What this PR includes

- `conf/models/upstage.json`: add `"embedding": "embeddings"` under
`url_suffix` so the driver can build the URL from config (matches the
`URLSuffix.Embedding` field already used by openai, mistral,
siliconflow, zhipu-ai), and add `solar-embedding-1-large-query` and
`solar-embedding-1-large-passage` entries under `models`.
- `internal/entity/models/upstage.go`: replace the `Embed` stub with a
real implementation that POSTs to `/v1/solar/embeddings`. Adds local
response types `upstageEmbeddingData` and `upstageEmbeddingResponse`.

No factory change. No interface change.

### How the implementation works

- Validate `apiConfig`, the API key, and the model name. Use the
existing `baseURLForRegion` helper so an unknown region fails fast with
a clear error.
- Wrap the request with `context.WithTimeout(nonStreamCallTimeout)` so
the call has a clear deadline. Same pattern as `ChatWithMessages` and
`ListModels` already use in this file.
- Send all input texts in one request. The Upstage API accepts the
`input` field as an array.
- Parse `data[*].embedding` and copy each slice into a `[]EmbeddingData`
indexed by `data[*].index` so the output order matches the input order
even if the API returns items in a different order.
- An empty input slice returns `[]EmbeddingData{}` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still empty, return a clear error so the caller does not silently use a
zero vector.

### Note on stacking

This PR builds on #14817 (the Upstage driver). Until #14817 merges, this
PR's diff on GitHub will include both that PR's commits and this one.
After #14817 lands on `main`, GitHub will auto-reduce this PR to only
the `Embed` changes (one commit, ~119 line diff in `upstage.go` plus ~15
lines in `upstage.json`).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 on go 1.25 (the
`go.mod` minimum).
- The full method set on `UpstageModel` still matches the `ModelDriver`
interface.
- Pattern parity with the existing Mistral Embed
(`internal/entity/models/mistral.go`) and OpenAI Embed
(`internal/entity/models/openai.go`) implementations.

Closes #14818
Depends on #14817
Tracking: #14736

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-12 16:11:06 +08:00
Haruko386
ebab3513c4 Go: implement provider: Baichuan (#14832)
### What problem does this PR solve?

This PR completes the Baichuan provider

**The following functionalities are now supported:**

**Baichuan:**
- [x] Chat / Stream Chat
- [x] Embedding
- [ ] ~~Rerank~~
- [ ] ~~Model listing~~
- [ ] ~~Provider connection checking~~
- [ ] ~~Balance~~

**Verified examples from the CLI:**

```plaintext
# Baichuan

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'Baichuan-Text-Embedding@test@baichuan' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 1024      | 0     |
| 1024      | 1     |
+-----------+-------+

AGFlow(user)> chat with 'Baichuan-M2@test@baichuan' message 'who r u'
Answer: I'm BaiChuan, a helpful AI assistant created by Baichuan-AI. I'm designed to be a knowledgeable, friendly, and reliable assistant for various tasks like answering questions, explaining concepts, writing content, and more. Feel free to ask me anything! 😊
Time: 1.637975

RAGFlow(user)> stream chat with 'Baichuan-M2@test@baichuan' message 'who r u'
Answer: I'm BaiChuan-m2, an AI assistant developed by Baichuan-AI. My purpose is to help you with a wide range of tasks by providing information, answering questions, solving problems, and assisting with creative projects. Think of me as a helpful digital companion! If you have any questions or need assistance, just let me know.😊
Time: 1.692321
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-12 16:10:32 +08:00
Achieve3318
2cc206ee85 Test : aggregation edge cases for list and scalar values (#14170)
This PR adds focused unit tests for aggregate_by_field in OceanBase
memory utilities to improve behavior coverage for real-world input
shapes.

- Adds test coverage for list-valued aggregation fields, including
whitespace trimming and skipping invalid list entries.
- Adds test coverage for scalar field values to ensure blank/non-string
values are ignored.
- Confirms aggregation output remains correct and stable for
mixed-quality message payloads.

### Why this helps
It strengthens regression protection for aggregation logic used by
memory retrieval flows, with no production code changes and minimal
review risk.
2026-05-12 15:53:35 +08:00
Magicbook1108
f85e18afbc Refact: sandbox quickstart.md & add tutorial for code exec component (#14786)
### What problem does this PR solve?

Refact: sandbox quickstart.md && add tutorial for code exec component

### Type of change

- [x] Refactoring


<img width="700" alt="img_v3_0211j_dcff835b-e3bb-4c77-9bc5-3b31a983229g"
src="https://github.com/user-attachments/assets/7842fc0f-639a-458f-b164-bc81a99ce4a5"
/>

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2026-05-12 14:42:20 +08:00
buua436
e8adc977bd Fix: some agent bug (#14829)
### What problem does this PR solve?

fix: 
update null checks to use 'is None' for better clarity
replace RAGFlowSelect with SelectWithSearch in DebugContent
add max height and overflow to DialogContent in ParameterDialog
 remove unused types from DataOperationsForm

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-12 14:41:49 +08:00
lif
a02b456720 fix(docs): correct broken knowledge graph construction link (#13838)
Fixes #13817

### What problem does this PR solve?

The "knowledge graph construction" link on line 21 of
`docs/guides/dataset/run_retrieval_test.md` points to
`./construct_knowledge_graph.md`, which doesn't exist. The actual file
is at `./advanced/construct_knowledge_graph.md`.

### Type of change

- [x] Documentation Update

Signed-off-by: majiayu000 <1835304752@qq.com>
2026-05-12 14:27:56 +08:00
tmimmanuel
558ea51a0f Go: implement provider: StepFun (#14815)
### What problem does this PR solve?

Add a Go driver for StepFun (阶跃星辰), one of the unchecked providers on
the umbrella tracking issue #14736.

Until this PR, a tenant who configured `stepfun` as a model provider in
the Go layer fell through to the default branch of
`internal/entity/models/factory.go` and got the dummy driver. Chat, list
models, and check connection all returned `"not implemented"` instead of
reaching the StepFun API.

The Python side has had StepFun registered in `rag/llm/__init__.py` as a
`SupportedLiteLLMProvider` with base URL `https://api.stepfun.com/v1`,
plus `StepFunCV` for vision and `StepFunSeq2txt` for ASR, but no Go
path. StepFun's chat API is OpenAI-compatible, so the implementation
pattern is the same as the merged Moonshot driver (#14433) and OpenAI
driver (#14605).

### What this PR includes

- New file `internal/entity/models/stepfun.go` with a `StepFunModel`
that implements the `ModelDriver` interface.
- `factory.go`: route the `"stepfun"` provider name to
`NewStepFunModel`.
- New `conf/models/stepfun.json` with the public StepFun chat models
(step-2-16k, step-1 family in 8k/32k/128k/256k context lengths,
step-1-flash, and the step-1v / step-1o vision models) and `url_suffix`
entries for `chat` and `models`.

### How the driver works

- StepFun exposes the OpenAI-compatible API at
`https://api.stepfun.com/v1`.
- `ChatWithMessages` and `ChatStreamlyWithSender` post to
`/chat/completions` in the same shape as the merged moonshot,
openrouter, and openai drivers.
- `ListModels` and `CheckConnection` call `/models` to list available
ids and confirm the API key works.
- `Embed` is left as `"not implemented"`. StepFun has not advertised a
public embeddings endpoint in the API reference linked from the umbrella
issue
(`https://platform.stepfun.com/docs/en/api-reference/chat/chat-completion-create`
is the chat endpoint), so any real implementation belongs in a separate
follow-up only after the endpoint is verified.
- `Rerank` and `Balance` return `"no such method"` because StepFun does
not expose either.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` returns exit 0 with no errors
on go 1.25 (the `go.mod` minimum).
- Method set of `StepFunModel` matches the `ModelDriver` interface:
`NewInstance`, `Name`, `ChatWithMessages`, `ChatStreamlyWithSender`,
`Embed`, `Rerank`, `ListModels`, `Balance`, `CheckConnection`.
- Pattern parity with the merged moonshot (#14433), openai (#14605),
openrouter (#14652), and xai (#14550) drivers.

Closes #14814
Tracking: #14736
2026-05-12 13:49:35 +08:00
hyl64
02c2587ca4 fix(agent): support iteration item aliases in child nodes (#14146)
## Summary
This PR fixes the iteration variable mismatch reported in #14142.

Changes:
- restore compatibility for `IterationItem@result` by exposing `result`
alongside `item`
- support bare iteration aliases like `{item}`, `{index}`, and
`{result}` inside iteration child-node inputs
- add focused unit/runtime tests covering both alias styles and
multi-item iteration execution

## Validation
```bash
pytest -q --noconftest \
  test/testcases/test_web_api/test_canvas_app/test_iterationitem_unit.py \
  test/testcases/test_web_api/test_canvas_app/test_iteration_runtime_unit.py \
  test/testcases/test_web_api/test_canvas_app/test_invoke_component_unit.py
```

Result: `12 passed`

Closes #14142
2026-05-12 13:05:21 +08:00
Haruko386
128a64eae5 Refactor(Go): remove hardcode in huggingface provider (#14822)
### What problem does this PR solve?

remove hardcode in `huggingface` provider

### Type of change

- [x] Refactoring
2026-05-12 11:35:26 +08:00
dependabot[bot]
139b76d2b1 Chore(deps): Bump urllib3 from 2.6.3 to 2.7.0 in /agent/sandbox (#14824)
Bumps [urllib3](https://github.com/urllib3/urllib3) from 2.6.3 to 2.7.0.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/releases">urllib3's
releases</a>.</em></p>
<blockquote>
<h2>2.7.0</h2>
<h2>🚀 urllib3 is fundraising for HTTP/2 support</h2>
<p><a
href="https://sethmlarson.dev/urllib3-is-fundraising-for-http2-support">urllib3
is raising ~$40,000 USD</a> to release HTTP/2 support and ensure
long-term sustainable maintenance of the project after a sharp decline
in financial support. If your company or organization uses Python and
would benefit from HTTP/2 support in Requests, pip, cloud SDKs, and
thousands of other projects <a
href="https://opencollective.com/urllib3">please consider contributing
financially</a> to ensure HTTP/2 support is developed sustainably and
maintained for the long-haul.</p>
<p>Thank you for your support.</p>
<h2>Security</h2>
<p>Addressed high-severity security issues. Impact was limited to
specific use cases detailed in the accompanying advisories; overall user
exposure was estimated to be marginal.</p>
<ul>
<li>
<p>Decompression-bomb safeguards of the streaming API were bypassed:</p>
<ol>
<li>When <code>HTTPResponse.drain_conn()</code> was called after the
response had been read and decompressed partially. (Reported by <a
href="https://github.com/Cycloctane"><code>@​Cycloctane</code></a>)</li>
<li>During the second <code>HTTPResponse.read(amt=N)</code> or
<code>HTTPResponse.stream(amt=N)</code> call when the response was
decompressed using the official <a
href="https://pypi.org/project/brotli/">Brotli</a> library. (Reported by
<a
href="https://github.com/kimkou2024"><code>@​kimkou2024</code></a>)</li>
</ol>
<p>See GHSA-mf9v-mfxr-j63j for details.</p>
</li>
<li>
<p>HTTP pools created using
<code>ProxyManager.connection_from_url</code> did not strip sensitive
headers specified in <code>Retry.remove_headers_on_redirect</code> when
redirecting to a different host. (GHSA-qccp-gfcp-xxvc reported by <a
href="https://github.com/christos-spearbit"><code>@​christos-spearbit</code></a>)</p>
</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Used <code>FutureWarning</code> instead of
<code>DeprecationWarning</code> for better visibility of existing
deprecation notices. Rescheduled the removal of deprecated features to
version 3.0. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3763">urllib3/urllib3#3763</a>)</li>
<li>Removed support for end-of-life Python 3.9. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3720">urllib3/urllib3#3720</a>)</li>
<li>Removed support for end-of-life PyPy3.10. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4979">urllib3/urllib3#4979</a>)</li>
<li>Bumped the minimum supported pyOpenSSL version to 19.0.0. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3777">urllib3/urllib3#3777</a>)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Fixed a bug where <code>HTTPResponse.read(amt=None)</code> was
ignoring decompressed data buffered from previous partial reads. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3636">urllib3/urllib3#3636</a>)</li>
<li>Fixed a bug where <code>HTTPResponse.read()</code> could cache only
part of the response after a partial read when
<code>cache_content=True</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4967">urllib3/urllib3#4967</a>)</li>
<li>Fixed <code>HTTPResponse.stream()</code> and
<code>HTTPResponse.read_chunked()</code> to handle <code>amt=0</code>.
(<a
href="https://redirect.github.com/urllib3/urllib3/issues/3793">urllib3/urllib3#3793</a>)</li>
<li>Updated <code>_TYPE_BODY</code> type alias to include missing
<code>Iterable[str]</code>, matching the documented and runtime behavior
of chunked request bodies. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3798">urllib3/urllib3#3798</a>)</li>
<li>Fixed <code>LocationParseError</code> when paths resembling
schemeless URIs were passed to
<code>HTTPConnectionPool.urlopen()</code>. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3352">urllib3/urllib3#3352</a>)</li>
<li>Fixed <code>BaseHTTPResponse.readinto()</code> type annotation to
accept <code>memoryview</code> in addition to <code>bytearray</code>,
matching the <code>io.RawIOBase.readinto</code> contract and enabling
use with <code>io.BufferedReader</code> without type errors. (<a
href="https://redirect.github.com/urllib3/urllib3/issues/3764">urllib3/urllib3#3764</a>)</li>
</ul>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/urllib3/urllib3/blob/main/CHANGES.rst">urllib3's
changelog</a>.</em></p>
<blockquote>
<h1>2.7.0 (2026-05-07)</h1>
<h2>Security</h2>
<p>Addressed high-severity security issues.
Impact was limited to specific use cases detailed in the accompanying
advisories; overall user exposure was estimated to be marginal.</p>
<ul>
<li>
<p>Decompression-bomb safeguards of the streaming API were bypassed:</p>
<ol>
<li>When <code>HTTPResponse.drain_conn()</code> was called after the
response had been
read and decompressed partially.</li>
<li>During the second <code>HTTPResponse.read(amt=N)</code> or
<code>HTTPResponse.stream(amt=N)</code> call when the response was
decompressed
using the official <code>Brotli
&lt;https://pypi.org/project/brotli/&gt;</code>__ library.</li>
</ol>
<p>See <code>GHSA-mf9v-mfxr-j63j
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-mf9v-mfxr-j63j&gt;</code>__
for details.</p>
</li>
<li>
<p>HTTP pools created using
<code>ProxyManager.connection_from_url</code> did not strip
sensitive headers specified in
<code>Retry.remove_headers_on_redirect</code> when
redirecting to a different host.
(<code>GHSA-qccp-gfcp-xxvc
&lt;https://github.com/urllib3/urllib3/security/advisories/GHSA-qccp-gfcp-xxvc&gt;</code>__)</p>
</li>
</ul>
<h2>Deprecations and Removals</h2>
<ul>
<li>Used <code>FutureWarning</code> instead of
<code>DeprecationWarning</code> for better
visibility of existing deprecation notices. Rescheduled the removal of
deprecated features to version 3.0.
(<code>[#3763](https://github.com/urllib3/urllib3/issues/3763)
&lt;https://github.com/urllib3/urllib3/issues/3763&gt;</code>__)</li>
<li>Removed support for end-of-life Python 3.9.
(<code>[#3720](https://github.com/urllib3/urllib3/issues/3720)
&lt;https://github.com/urllib3/urllib3/issues/3720&gt;</code>__)</li>
<li>Removed support for end-of-life PyPy3.10.
(<code>[#4979](https://github.com/urllib3/urllib3/issues/4979)
&lt;https://github.com/urllib3/urllib3/issues/4979&gt;</code>__)</li>
<li>Bumped the minimum supported pyOpenSSL version to 19.0.0.
(<code>[#3777](https://github.com/urllib3/urllib3/issues/3777)
&lt;https://github.com/urllib3/urllib3/issues/3777&gt;</code>__)</li>
</ul>
<h2>Bugfixes</h2>
<ul>
<li>Fixed a bug where <code>HTTPResponse.read(amt=None)</code> was
ignoring decompressed
data buffered from previous partial reads.
(<code>[#3636](https://github.com/urllib3/urllib3/issues/3636)
&lt;https://github.com/urllib3/urllib3/issues/3636&gt;</code>__)</li>
<li>Fixed a bug where <code>HTTPResponse.read()</code> could cache only
part of the
response after a partial read when <code>cache_content=True</code>.</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="9a950b92d9"><code>9a950b9</code></a>
Release 2.7.0</li>
<li><a
href="5ec0de499b"><code>5ec0de4</code></a>
Merge commit from fork</li>
<li><a
href="2bdcc44d1e"><code>2bdcc44</code></a>
Merge commit from fork</li>
<li><a
href="f45b0df09d"><code>f45b0df</code></a>
Fix a misleading example for <code>ProxyManager</code> (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4970">#4970</a>)</li>
<li><a
href="577193ca02"><code>577193c</code></a>
Switch to nightly PyPy3.11 in CI for now (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4984">#4984</a>)</li>
<li><a
href="e90af45bb0"><code>e90af45</code></a>
Avoid infinite loop in <code>HTTPResponse.read_chunked</code> when
<code>amt=0</code> (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4974">#4974</a>)</li>
<li><a
href="67ed74fdae"><code>67ed74f</code></a>
Bump dev dependencies (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4972">#4972</a>)</li>
<li><a
href="3abd481097"><code>3abd481</code></a>
Upgrade mypy to version 1.20.2 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4978">#4978</a>)</li>
<li><a
href="2b8725dfca"><code>2b8725d</code></a>
Drop support for EOL PyPy3.10 (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4979">#4979</a>)</li>
<li><a
href="2944b2a0a6"><code>2944b2a</code></a>
Upgrade <code>setup-chrome</code> and <code>setup-firefox</code> to fix
warnings (<a
href="https://redirect.github.com/urllib3/urllib3/issues/4973">#4973</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/urllib3/urllib3/compare/2.6.3...2.7.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=urllib3&package-manager=uv&previous-version=2.6.3&new-version=2.7.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-05-12 11:10:15 +08:00
CaptainTimon
2717ee283f feat(raptor): add Psi tree builder with original-space ranking and safe migration (#14679)
### What problem does this PR solve?

Closes #14674.

This PR improves RAPTOR configuration and tree construction while
preserving the existing RAPTOR behavior as the default.

RAPTOR currently builds summary layers with the original UMAP + GMM
clustering path. This PR keeps that default path, and adds:

- A hidden backend tree-builder option:
  - `tree_builder="raptor"`: default, existing RAPTOR behavior.
- `tree_builder="psi"`: rank-aware Psi-style tree builder using original
embedding-space cosine ranking.
- A user-facing clustering method option for the default RAPTOR builder:
  - `clustering_method="gmm"`: existing default.
- `clustering_method="ahc"`: agglomerative hierarchical clustering path.
- A RAPTOR UI setting for `Clustering method` and `Max cluster`.

### What changed

#### Backend

- Added `tree_builder` support for RAPTOR/Psi.
- Added `clustering_method` support for GMM/AHC.
- Kept existing RAPTOR + GMM as the default.
- Added Psi tree building from original-space cosine similarity.
- Added bucketed Psi building controls for large inputs:
  - `raptor.ext.psi_exact_max_leaves`
  - `raptor.ext.psi_bucket_size`
- Added method-aware RAPTOR summary metadata using existing
`extra.raptor_method`.
- Avoided adding a dedicated DB schema field for experimental method
tracking.
- Added cleanup/migration logic to avoid mixing stale RAPTOR summary
trees.
- Added defensive checks for Psi tree construction and summary failures.

#### Frontend/UI

- Added `Clustering method` in RAPTOR settings with `GMM` and `AHC`.
- Added/kept `Max cluster` in RAPTOR settings.
- Enlarged max cluster UI limit to `1024`, matching backend validation.
- Kept AHC editable even when a RAPTOR task has already finished.
- Fixed the UI save payload so `clustering_method` and `tree_builder`
are serialized through `parser_config.raptor.ext`, avoiding backend
validation errors for extra top-level RAPTOR fields.

Example saved RAPTOR config:

```json
{
  "raptor": {
    "max_cluster": 317,
    "ext": {
      "clustering_method": "ahc",
      "tree_builder": "raptor"
    }
  }
}

Co-authored-by: CaptainTimon <CaptainTimon@users.noreply.github.com>
2026-05-12 09:42:31 +08:00
黄圣祺
415169d497 fix(dify): add GET method support to /dify/retrieval for health check (#13837)
## Summary
- Add GET method handler to `/api/v1/dify/retrieval` endpoint for Dify
external knowledge base connectivity verification
- GET requests return a simple success response; POST requests retain
existing retrieval logic unchanged

## Problem
When Dify integrates with RAGFlow as an external knowledge base, it
sends periodic GET requests to the retrieval endpoint for
health/connectivity checks. The endpoint only accepted POST, causing
werkzeug to return `405 Method Not Allowed`. After several successful
POST retrievals, the failing GET health checks trigger Dify's circuit
breaker, causing all subsequent requests to fail.

Traceback from the issue:
```
werkzeug.exceptions.MethodNotAllowed: 405 Method Not Allowed: The method is not allowed for the requested URL.
```

## Changes
- `api/apps/sdk/dify_retrieval.py`: Added a separate GET route handler
(`retrieval_health_check`) that returns `get_json_result(data=True)`

## Test plan
- [ ] Verify `GET /api/v1/dify/retrieval` returns `{"code": 0,
"message": "success", "data": true}`
- [ ] Verify `POST /api/v1/dify/retrieval` with valid API key and body
still works as before
- [ ] Verify Dify external knowledge base integration no longer returns
405 errors

Closes #13788

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-12 09:37:07 +08:00
Ramin M.
765cdc2ec2 [Bug]: REDIS error #12870 (#13875)
Fix for: [Bug]: REDIS error #12870
2026-05-12 09:31:47 +08:00
Jin Hai
2f2d1569e6 Go: fix retrieval test error (#14794)
### What problem does this PR solve?

1. Add region check in zhipu-ai embed method
2. Fix retrieval test

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 20:19:08 +08:00
Haruko386
3e90d303e0 Go: implement provider: CoHere and FishAudio (#14790)
### What problem does this PR solve?

This PR completes the Cohere provider integration (upgrading to the new
Cohere V2 API) and enhances the Fish Audio provider in RAGFlow.

**The following functionalities are now supported:**

**Cohere:**
- [x] Chat / Think Chat / Stream Chat / Stream Think Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

**Fish Audio:**
- [x] Model listing (`ListModels`)
- [x] Balance (`Balance`)

-----

**Verified examples from the CLI:**

```plaintext

# Cohere

RAGFlow(user)> think chat with 'command-a-reasoning-08-2025@test3@cohere' message 'jumperwho'
Thinking: Okay, the user wrote "jumperwho". Let me try to figure out what they might be asking. First, I'll check if it's a misspelling. "Jumper" ...... Hmm. Since the query is unclear, the best approach is to ask the user to provide more context or correct any possible typos.
Answer: It seems there might be a typo or missing context in your query "jumperwho." Could you clarify what you're referring to? For example:
- Are you asking about a **jumper** (a type of sweater, a person who jumps, or a component in electronics)?
- Is this related to a specific context, like a movie (e.g., the 2008 film *Jumper*) or a game?
- Did you mean to ask about a person ("who") associated with jumping (e.g., a parachutist)?

Let me know so I can provide a helpful response! 😊
Time: 6.710331

RAGFlow(user)> stream think chat with 'command-a-reasoning-08-2025@test3@cohere' message 'jumperwho'
Thinking: , the user mentioned "jumperwho". Let me try to figure out what they're referring to. First, I'll check if it's a misspelling. "Jumper" could be a typo for "jumper" or maybe a username. Alternatively, it might be a combination of words like "jumper who",....... the best approach is to inform the user that I don't recognize the term and ask if they can provide more context or clarify what they mean by "jumperwho". That way, I can assist them better once I have more information.
Answer:  seems "jumperwho" isn't a widely recognized term, proper noun, or acronym in common usage. Could you provide more context or clarify what you mean by "jumperwho"? This will help me understand your question or request better!
Time: 4.513596

RAGFlow(user)> embed text 'walkerwhat' 'jumperwho' with 'embed-v4.0@test3@cohere' dimension 16;
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| embedding                                                                                                                                                                                                                                                        | index |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+
| [-0.016643638 -0.001957038 0.0055713872 0.009027058 0.05275187 -0.024542313 -0.044006906 0.024119169 0.0014192933 0.006558722 0.0019129605 -0.021016119 -0.026516981 -0.017489925 0.021298215 0.017772019 0.04569948 0.008886009 0.012059584 -0.0014721862 0.... | 0     |
| [0.018778935 -0.0063459855 -0.0006839742 0.0046623563 0.0067668925 -0.018001877 -0.03963003 0.035744734 -0.014246088 -0.0020721585 -0.006313608 0.025124922 -0.010749322 0.01217393 -0.010231283 -0.025254432 0.021498645 -0.028880708 0.019167464 -0.0058279... | 1     |
+------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'rerank-v4.0-pro@test@cohere' top 3;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 0.91744334      |
| 1     | 0.7458429       |
| 2     | 0.68729424      |
+-------+-----------------+

RAGFlow(user)> list supported models from 'cohere' 'test'
+-------------------------------------+
| model_name                          |
+-------------------------------------+
| c4ai-aya-expanse-32b                |
| c4ai-aya-vision-32b                 |
| cohere-transcribe-03-2026           |
| command-a-03-2025                   |
| command-a-reasoning-08-2025         |
| command-a-translate-08-2025         |
| command-a-vision-07-2025            |
| command-r-08-2024                   |
| command-r-plus-08-2024              |
| command-r7b-12-2024                 |
| command-r7b-arabic-02-2025          |
| embed-english-light-v3.0            |
| embed-english-light-v3.0-image      |
| embed-english-v3.0                  |
| embed-english-v3.0-image            |
| embed-multilingual-light-v3.0       |
| embed-multilingual-light-v3.0-image |
| embed-multilingual-v3.0             |
| embed-multilingual-v3.0-image       |
| embed-v4.0                          |
+-------------------------------------+

RAGFlow(user)> check instance 'test' from 'cohere'
SUCCESS


# FishAudio

RAGFlow(user)> list supported models from 'fishaudio' 'test'
+----------------------------------------+
| model_name                             |
+----------------------------------------+
| Valentino Narración Biblica Fer        |
| Super Smash Bros. 4/Ultimate Announcer |
| Farid Dieck                            |
| عصام الشوالي                           |
| ALEX_CHIKNA                            |
| Energetic Male                         |
| voz de locutor k                       |
| يي                                     |
| ELITE                                  |
| Mortal Kombat                          |
+----------------------------------------+

RAGFlow(user)> show balance from 'fishaudio' 'test'
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+
| _id                              | created_at                  | credit | has_free_credit | has_phone_sha256 | updated_at                  | user_id                          |
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+
| 82ffec12cf984d88a30ec504d7909812 | 2026-05-09T07:52:16.119000Z | 0      |                 | false            | 2026-05-09T07:52:16.119000Z | 2578ab1126804d6eaa630552400d7ff3 |
+----------------------------------+-----------------------------+--------+-----------------+------------------+-----------------------------+----------------------------------+

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-11 20:18:38 +08:00
buua436
daf8a58c4b Fix: add codeexec attachments output (#14787)
### What problem does this PR solve?

add codeexec attachments output

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 19:16:33 +08:00
Renzo
39ee2fb120 Go: implement Rerank in NVIDIA driver (#14778)
## Summary

- Replaces the `"no such method"` stub on `NvidiaModel.Rerank`
(`internal/entity/models/nvidia.go`) with a real implementation against
NVIDIA NIM's `/ranking` endpoint.
- Mirrors the existing Python `NvidiaRerank` class at
`rag/llm/rerank_model.py:149-190` for behavior parity: same
`passages`/`query.text`/`logit` payload shape; `top_n` set to
`len(documents)` so every input gets a score returned in original order
(the issue body's spec omitted `top_n`, which would cause silent data
loss).
- Adds the `"rerank": "ranking"` URL suffix and two NIM rerank model
entries (`nvidia/nv-rerankqa-mistral-4b-v3`,
`nvidia/llama-3.2-nv-rerankqa-1b-v2`) to `conf/models/nvidia.json` so
the picker exposes them.
- Follows the same shape as the recently merged Aliyun (#14676), Gitee
(#14656), and ZhipuAI (#14608) Rerank implementations: lowercase
per-driver request/response types, conversion to the project-wide
`RerankResponse{Data: []RerankResult}`, per-call `context.WithTimeout`
of 30s.

Closes #14720

## Test plan

- [x] `gofmt -l internal/entity/models/nvidia.go` — clean
- [x] `go vet ./internal/entity/models/...` — no new errors introduced
(the two pre-existing vet errors in `baidu.go:642` and
`openrouter.go:566` are unrelated to this PR)
- [x] `go build ./internal/entity/models/...` — succeeds
- [x] `python3 -c "import json;
json.load(open('conf/models/nvidia.json'))"` — JSON valid
- [ ] Live smoke test against NVIDIA NIM with a real API key (requires
reviewer with NIM credentials)

## Notes for reviewers

- The issue body suggested omitting `top_n`. The Python reference
includes it (`top_n: len(texts)`), and without it NVIDIA returns only
the default top-K rankings rather than scores for every input. This PR
follows the Python.
- The URL host is `integrate.api.nvidia.com` (kept consistent with the
existing chat/embeddings BaseURL in `nvidia.go`), not the legacy
`ai.api.nvidia.com` host the Python uses. NIM's unified endpoint accepts
the model names as-is, so no per-model URL transform is needed.
2026-05-11 17:21:16 +08:00
Jin Hai
9b3850339b Go: add development guide document (#14785)
### What problem does this PR solve?

As the title suggests.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 17:20:41 +08:00
tmimmanuel
663fc1d42c fix(opensearch): implement doc-meta dispatch surface on OSConnection (#14577)
### What problem does this PR solve?

Fixes #14570. On OpenSearch backends (`DOC_ENGINE=opensearch`) every
document-metadata write failed with `'OSConnection' object has no
attribute 'create_doc_meta_idx'`, so both `PATCH
/api/v1/datasets/{ds}/documents/{doc}` with `meta_fields` and `POST
/api/v1/datasets/{ds}/metadata/update` were unusable while every other
document operation (retrieval, parsing, name update, chunk management)
worked correctly on the same OpenSearch cluster.

The bug runs deeper than the missing method name in the error message
suggests. `DocMetadataService` also reached into
`settings.docStoreConn.es.*` directly for the index refresh, the
scripted partial update, and the count call, which means that even after
adding `create_doc_meta_idx` to `OSConnection` the very next call in the
same metadata flow would still raise `AttributeError` because
`OSConnection` exposes `self.os` rather than `self.es`. Fixing only the
reported symptom would have moved the failure one line down without
restoring the feature.

This PR adds a uniform document-metadata dispatch surface to both
connection classes so they present the same abstract API, and routes the
service layer through that surface via `getattr` guards instead of
poking at backend-specific attributes. The four new methods on
`OSConnection` and `ESConnectionBase` are `create_doc_meta_idx`,
`refresh_idx`, `count_idx`, and `replace_meta_fields`.
`OSConnection.create_doc_meta_idx` reuses the existing
`conf/doc_meta_es_mapping.json` schema in the OpenSearch `body=` form
because OpenSearch and Elasticsearch share the same index-creation
payload, and `replace_meta_fields` emits a full scripted assignment
(`ctx._source.meta_fields = params.meta_fields`) on both backends so
removed keys actually disappear instead of being preserved by deep-merge
semantics.

The `getattr`-guarded dispatch in `DocMetadataService` keeps the
existing fall-through paths intact for Infinity and OceanBase, which
continue to rely on their search-based count fallback and on the
delete-then-insert metadata replacement they used before, so this change
is strictly additive for those two backends.

Verification: `pytest
test/unit_test/rag/utils/test_opensearch_doc_meta.py` runs 16 new unit
tests that pass locally and pin the `OSConnection` dispatch surface, the
`create_doc_meta_idx` short-circuit when the index already exists, the
mapping-file payload routing, the `IndicesClient.create` failure path,
the `refresh_idx` and `count_idx` success and error sentinels, and the
full-assignment script emitted by `replace_meta_fields`. The test module
stubs `common.settings` and `rag.nlp` at import time so the suite runs
without the heavy backend SDKs that the rest of the repository pulls in
transitively.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: tmimmanuel <tmimmanuel@users.noreply.github.com>
2026-05-11 17:04:28 +08:00
box4wangjing
292b0b8bce chore: fix some comments to improve readability (#14756)
### What problem does this PR solve?

fix some comments to improve readability

### Type of change

- [x] Documentation Update

---------

Signed-off-by: box4wangjing <box4wangjing@outlook.com>
2026-05-11 16:48:48 +08:00
Octopus
c58906b69e fix: OCR.detect() returns truthy None-tuple causing NoneType subscript crash (#13951)
Fixes #13851

## Problem

`OCR.detect()` in `deepdoc/vision/ocr.py` returns `None, None,
time_dict` (a truthy 3-tuple) when the text detector fails or receives a
`None` image. However, the caller in `pdf_parser.py:__ocr()` checks:

```python
bxs = self.ocr.detect(np.array(img), device_id)
if not bxs:  # False! (None, None, time_dict) is a non-empty tuple → truthy
    self.boxes.append([])
    return
bxs = [(line[0], line[1][0]) for line in bxs]  # iterates (None, None, time_dict)
# line = None → None[0] → TypeError: 'NoneType' object is not subscriptable
```

This causes the `NoneType object is not subscriptable` error that
appears after "OCR started" in the chunking pipeline when using PDF +
General parser.

## Solution

Simplified `OCR.detect()` to return `None` (falsy) instead of `None,
None, time_dict` on failure. The `time_dict` was unused by the only
caller of this method. The early-return guard `if not bxs:` in
`pdf_parser.py` then correctly catches it.

## Testing

- The method's only caller (`pdf_parser.py:__ocr`) already has a `if not
bxs:` guard that handles the `None` return correctly.
- No other callers of `OCR.detect()` exist in the codebase.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Modified OCR detection function return behavior to streamline output.
The function now returns detection results only, without timing
metadata. Error cases now return `None` instead of empty tuple values.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-05-11 16:19:28 +08:00
Nie WeiYang
1e80be77a2 fix(web): fix incomplete Docx preview in citation reference (#14122)
This PR fixes a UI issue where the .docx document preview was displayed
incompletely when clicking on a citation/reference link during a
knowledge base conversation.

### What problem does this PR solve?

The Issue:
In the chat interface, when a user clicks the source citation at the end
of an answer, the DocPreviewer opens. However, for .docx files, if the
content exceeded the window height, it was truncated and unscrollable,
preventing users from reading the full referenced text.

Changes:
web/src/components/document-preview/doc-preview.tsx: Added the
overflow-auto Tailwind class to the DocPreviewer root container to
ensure scrollbars appear automatically when content overflows.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: nie.weiyang <nie.weiyang@embedway.com>
2026-05-11 16:17:48 +08:00
as-ondewo
6fb8c31c22 Fix: Document parse status set to DONE before chunks are retrievable (#13352)
### What problem does this PR solve?

The document parse status was set to DONE before the document chunks
were actually retrievable from Elasticsearch/Opensearch because it did
not wait for the index refresh. This meant that it was possible that the
document parse status returned by the API was DONE but when trying to
retrieve chunks there were none. Since the index refreshes every 1
second this was quite likely to happen when wait for document parsing by
polling with a short interval and then immediately trying to retrieve
chunks once the status was DONE.

I fixed this bug and added a test case that would have caught it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 16:04:08 +08:00
Sank
592dba1489 Refact: Added a private helper _visibility_and_status_filter (#13627)
### What problem does this PR solve?

Added a private helper _visibility_and_status_filter(joined_tenant_ids,
user_id) that returns the Peewee condition: visible to user (team or
own) and status is VALID.

### Type of change
- [x] Refactoring

---------

Co-authored-by: Serobabov Aleksandr <40SerobabovAS@region.cbr.ru>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-05-11 15:21:41 +08:00
tmimmanuel
6ce014c23b fix: offload blocking DB/Redis calls to thread pool for high-concurrency support (#13825) (#13941)
### What problem does this PR solve?

Addresses event-loop blocking under high concurrency reported in #13825.
When multiple requests hit the API simultaneously, synchronous DB/Redis
calls block the async event loop, preventing Quart from handling other
requests and causing cascading 502/504 timeouts.

This PR wraps all remaining blocking DB/Redis calls in `canvas_app.py`,
`chat_api.py`, `session.py`, and `canvas_service.py` with `await
thread_pool_exec()`
- Offload all synchronous `Service.*`, `REDIS_CONN.*`, and
`APIToken.query` calls to the thread pool
- Convert sync endpoint handlers (`list_chats`, `get_chat`, `templates`,
`sessions`, etc.) to `async def`
- Convert sync helper functions (`_ensure_owned_chat`,
`_validate_llm_id`, `_validate_dataset_ids`, etc.) to async - no
duplicate sync/async pairs
- Wrap `CanvasReplicaService` Redis IO calls (`bootstrap`,
`replace_for_set`, `commit_after_run`)
- Use `asyncio.gather()` for concurrent file uploads and chat response
building

**Note:** This fixes the code-level event-loop blocking, which is a
prerequisite for handling concurrent requests. For the full "30
concurrent requests without 502/504" goal described in the issue, users
should also tune deployment config:
- `WS=4` or higher (HTTP worker processes, default 1)
- `MAX_CONCURRENT_CHATS=50` (default 10)
- `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` for workflow-heavy workloads

### Performance verification

Reviewer asked for a before-vs-after comparison
([comment](https://github.com/infiniflow/ragflow/pull/13941#issuecomment-4393667231)).
I built a self-contained microbenchmark that reproduces the exact
failure mode this PR targets: an async handler that performs blocking
DB/Redis-style calls (50 ms each, 3 per request, 30 concurrent requests)
is run twice — once with the pre-PR pattern (sync call directly inside
the async handler) and once with the post-PR pattern (`await
thread_pool_exec(...)`). The benchmark imports nothing from RAGFlow
except `thread_pool_exec` itself, so it is hermetic and reproducible
(`THREAD_POOL_MAX_WORKERS=128`, Python 3.13.12).

**Throughput — wall-clock for 30 concurrent requests (lower is better)**

| flavour | wall(s) | p50(s) | p95(s) | max(s) |
|---|---:|---:|---:|---:|
| before | 4.986 | 0.158 | 0.207 | 0.269 |
| after  | 0.248 | 0.181 | 0.230 | 0.231 |

The pre-PR handler serializes the entire load on the event-loop thread,
so 30 × 3 × 50 ms ≈ 4.5 s shows up as the wall time. The post-PR handler
parallelizes the blocking work across the thread pool and finishes the
same load in 248 ms — a **~20× speedup** on this workload.

**Event-loop responsiveness — latency of an unrelated probe coroutine
while the 30 slow requests are running (lower is better)**

| flavour | samples | probe p50 (ms) | probe p95 (ms) | probe max (ms) |
|---|---:|---:|---:|---:|
| before | 1 | 5442.26 | 5442.26 | 5442.26 |
| after  | 28 | 0.88 | 11.53 | 98.02 |

This is the metric that maps directly to "the API still answers other
requests while one is busy". A 5 ms-interval probe was scheduled while
the 30 slow handlers ran. With the pre-PR code the event loop was frozen
for the entire duration of the blocking work, so only one probe sample
was ever picked up and it waited **5,442 ms**. After the PR, 28 probe
samples landed with **p50 0.88 ms / p95 11.53 ms**, meaning unrelated
requests are no longer starved by the slow ones. That is the regression
mode behind the cascading 502/504s reported in #13825.

<details>
<summary>Raw benchmark output</summary>

```
config: 30 concurrent requests, 3 blocking calls of 50ms each per request, THREAD_POOL_MAX_WORKERS=128

=== Throughput (lower wall is better) ===
flavour       wall(s)     p50(s)     p95(s)     max(s)
before          4.986      0.158      0.207      0.269
after           0.248      0.181      0.230      0.231

=== Event-loop responsiveness (lower probe latency is better) ===
flavour       samples    probe p50(ms)    probe p95(ms)    probe max(ms)
before              1          5442.26          5442.26          5442.26
after              28             0.88            11.53            98.02
```

</details>

The benchmark script is included as a comment on the PR for
reproducibility.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Performance Improvement

Closes [#13825](https://github.com/infiniflow/ragflow/issues/13825)

---------

Co-authored-by: tmimmanuel <tmimmanuel@users.noreply.github.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 15:08:55 +08:00
Paul Y Hui
a0efc453f3 Fix: safe argument guard and remove redundant redis call (#14060)
### What problem does this PR solve?

- Moved if not all([email, new_pwd, new_pwd2]) guard to the top, before
any decryption that could crash on None value
- Removed the redundant REDIS_CONN.get() call — one call is sufficient

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-11 15:02:24 +08:00
Jin Hai
c55e23e7e2 Go: refactor embedding interface (#14757)
### What problem does this PR solve?

Provide embedding index according to the input text

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 14:45:30 +08:00
Ricardo-M-L
5ef7f50eef fix: use context manager for ThreadPoolExecutor in file_service.py (#14144)
## Summary
- Wrap 2 `ThreadPoolExecutor` instances in `file_service.py` with `with`
statement
- Ensures threads are properly shut down after all futures complete

## Problem

`parse_docs()` (line 532) and the file processing method (line 694)
create `ThreadPoolExecutor` instances that are never shut down. In a
long-running server process, this leaks thread resources on every
invocation — threads remain alive consuming memory even after all
submitted work is complete.

## Fix

Replace bare `ThreadPoolExecutor()` with `with ThreadPoolExecutor() as
exe:` context manager, which calls `executor.shutdown(wait=True)` on
exit.

## Test plan
- [x] Verified both call sites use `with` statement after fix
- [x] No remaining bare `ThreadPoolExecutor` in `file_service.py`
- [x] `document_service.py:1066` is a module-level executor (different
pattern, not changed in this PR)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 14:02:45 +08:00
buua436
a03b95f8c4 Fix: shared dataset chunk index lookup (#14764)
### What problem does this PR solve?

shared dataset chunk index lookup

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:50:08 +08:00
buua436
024c8cb0b5 Fix: dataset search rerank id type (#14759)
### What problem does this PR solve?
issue: https://github.com/infiniflow/ragflow/issues/14748

change: dataset search rerank id type
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:48:05 +08:00
jony376
46897d6fa4 Fix: bind memory message user_id to authenticated user for JWT auth (#14745)
### Related issues

Closes #14744

### What problem does this PR solve?

The Memory REST endpoint `POST /api/v1/messages` previously persisted
whatever `user_id` the client sent in the JSON body. Memory rows were
therefore attributed to an arbitrary string, even when the caller
authenticated as a normal workspace user via JWT (browser/session-style
bearer token decoded into an access token). That broke attribution and
audit semantics for shared memories (team visibility): any authorized
writer could spoof another subject id.

The Python SDK already sends an optional `user_id` for integrations
using **API keys** (`APIToken`) to tag an external subject distinct from
the tenant owner user.

### Solution

- Record **`g.auth_via_api_token`** in `_load_user`
(`api/apps/__init__.py`): set `True` only when authentication resolves
via `APIToken`, otherwise `False` after JWT-based login succeeds.
- In **`POST /messages`** (`memory_api.add_message`): if the request was
authenticated with an API key, keep accepting optional `user_id` from
the body (default empty string). For JWT-authenticated users, **always**
set stored `user_id` to **`current_user.id`** and ignore the client
field.
- Guard reads of `g` with **`RuntimeError`** handling so isolated
imports or tests without a Quart application context do not fail when
resolving `user_id`.
- Document on **`RAGFlow.add_message`** that `user_id` is only
meaningful for API-key authentication.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- `python -m py_compile` on modified modules (`api/apps/__init__.py`,
`api/apps/restful_apis/memory_api.py`).
- Recommended: run web/SDK memory message tests (`test_add_message`,
`test_message_routes_unit`) against a full environment with `quart` and
configured services.

### Notes for reviewers

- Behavior change **only** for callers using JWT-style authorization on
`POST /messages`; API-key callers keep prior optional `user_id`
semantics.

Co-authored-by: jony376 <jony376@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-11 13:26:05 +08:00
Achieve3318
16354f4e14 fix(dify): guard retrieval argument error behavior (#14169)
## What problem does this PR solve?

The Dify-compatible `/dify/retrieval` endpoint recently gained stricter
parsing and validation for its request payload, including:
- Normalized `retrieval_setting.top_k` and
`retrieval_setting.score_threshold` types.
- Clear separation between malformed arguments vs missing required
fields.
Previously, there was no unit test explicitly guarding the exact error
code and message contract for these cases.

## What does this PR change?

- **Add guard-style unit test** in `test_dify_retrieval_routes_unit.py`:
  - `test_retrieval_argument_error_messages`:
    - Sends a request with malformed numeric options:
- `retrieval_setting = {"top_k": "not-int", "score_threshold":
"not-float"}`
      - Asserts `code == RetCode.ARGUMENT_ERROR` and message contains  
        `"invalid or malformed arguments:"`.
    - Sends a request with required fields missing:
      - Empty payload (`{}`)
      - Asserts `code == RetCode.ARGUMENT_ERROR` and message contains  
        `"required arguments are missing:"`.

This test encodes the intended behavior of the Dify retrieval API so
future refactors cannot silently regress error handling.

## Type of change

- [x] Tests (add coverage and guardrails for existing behavior)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 13:17:42 +08:00
FPlust
0734fd793a fix: scope pending_cell_images by sheet in excel parser (#14120)
pending_cell_images should be scoped by sheet

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 13:17:14 +08:00
Wang Qi
3838770e7a GraphRAG feature - Part 1 - add spacy to extract entity and relation (#14670)
### What problem does this PR solve?

GraphRAG feature - Part 1 - add spacy to extract entity and relation

<img width="1621" height="1288" alt="image"
src="https://github.com/user-attachments/assets/aadeddad-94da-46c6-adad-9c3784181f61"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 12:59:59 +08:00
web-dev0521
cc207b5b05 Refactor: tidy up ThreadPoolExecutor lifecycle in file_service and task executor (#14668)
## Summary
- Wrap the `ThreadPoolExecutor` instances in `FileService.parse_docs`
and `FileService.get_files` with `with ... as exe:` blocks for
deterministic cleanup
- Replace the `concurrent.futures.ThreadPoolExecutor` in
`do_handle_task` with `asyncio.create_task(asyncio.to_thread(build_TOC,
...))`, preserving the existing parallelism with chunk insertion while
leveraging the surrounding async context
- Drop the now-unused `import concurrent` and the
`executor.shutdown(wait=False)` call in the `finally` block

Closes #14622.

No behavioral change, no public API change. Net diff: ~19 insertions /
25 deletions across two files.

## Test plan
- [ ] `uv run ruff check api/db/services/file_service.py
rag/svr/task_executor.py` passes
- [ ] Upload a multi-file batch through the chat/file endpoint and
confirm `FileService.parse_docs` still returns combined parsed text
- [ ] Trigger `FileService.get_files` via the chat reference flow with a
mix of image and non-image files; verify both `raw=True` and `raw=False`
paths return correctly
- [ ] Run a `naive`-parser document task with `toc_extraction: true` and
confirm the TOC chunk is generated and inserted exactly as before
- [ ] Run a `naive`-parser document task with `toc_extraction: false`
and confirm the path with `toc_thread = None` is unaffected
- [ ] Cancel a running task to exercise the `finally` block and confirm
cleanup still works without the executor shutdown call

---------

Co-authored-by: web-dev0521 <jasonpette1783@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-05-11 12:59:00 +08:00
Joseff
13e6554901 Fix(Go): make OpenRouter Encode fail loudly on malformed responses (#14717)
### What problem does this PR solve?

The OpenRouter `Encode` method silently swallowed malformed responses.
If a `data[]` item from the API was missing a field (`index`,
`embedding`, or unexpected shape), the loop did `continue` instead of
returning an error — leaving `nil` entries in the result slice. Callers
got back partial results with no indication anything went wrong, which
then crashes downstream consumers when they try to use a `nil` vector.
There were three concrete gaps:

- No count-mismatch check between `data` length and input texts (only
checked for empty)
- No duplicate-index detection (a duplicate would silently overwrite)
- Parse failures on individual items returned partial slices instead of
erroring

This PR replaces `map[string]interface{}` parsing with a typed
`openrouterEmbeddingResponse` struct and applies the same 3-layer
validation used in the other drivers (count mismatch → out-of-range
index → duplicate index), so any malformed response produces a clear
error instead of corrupted data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 12:57:11 +08:00
Panda Dev
530edbac99 Go: implement Encode (embeddings) in LM Studio driver (#14694)
### What problem does this PR solve?

The LM Studio Go driver shipped with a stub \`Encode\` method that
returned \`no such method\`, even though LM Studio is one of the most
common local LLM runners on macOS and Windows and exposes an
OpenAI-compatible embeddings endpoint at \`/v1/embeddings\`.

LM Studio users routinely load local embedding models such as
\`nomic-ai/nomic-embed-text-v1.5\`,
\`mixedbread-ai/mxbai-embed-large-v1\`, or \`BAAI/bge-m3\`. They run on
the same \`/v1\` namespace as chat. The existing \`ListModels\` already
discovers them, but because \`Encode\` was a stub, a tenant who picked
one of these models in the Go layer could not actually run an embedding
call.

This finishes the local-LLM trio: Ollama Encode (#14664) and vLLM Encode
(#14688) are already in flight, both using the
same OpenAI-compatible \`/embeddings\` shape.

### What this PR includes

- \`conf/models/lmstudio.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the driver can build the URL from config.
- \`internal/entity/models/lmstudio.go\`: replace the \`Encode\` stub
with a real implementation. Adds a small local response type that
matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for local LM Studio,
so the Authorization header is only set when both \`apiConfig\` and
\`ApiKey\` are non-nil and non-empty, the same pattern the recently
merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
Aliyun Encode (#14647) and the in-flight Ollama Encode (#14664) and vLLM
Encode (#14688) use.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`LmStudioModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the merged Aliyun Encode (#14647), the in-flight
Ollama Encode (#14664) and vLLM Encode (#14688), and the existing
SiliconFlow Encode.

Closes #14693
2026-05-11 12:55:57 +08:00
Joseff
0580c137fa Perf(Go): batch SiliconFlow Encode requests with 32-item chunking (#14719)
### What problem does this PR solve?

The SiliconFlow `Encode` method sent one HTTP request per text, which is
wasteful and slow when indexing many documents (e.g., 100 docs = 100
round-trips).

SiliconFlow's `/v1/embeddings` is OpenAI-compatible and accepts an array
of strings in `input` (officially documented at
https://docs.siliconflow.cn/en/api-reference/embeddings/create-embeddings,
with a documented max array size of 32). This PR batches the requests up
to that limit, reducing 100 docs to ~4 round-trips, and replaces
`map[string]interface{}` parsing with a typed struct using the same
3-layer validation (count mismatch, out-of-range index, duplicate index)
used in the other drivers.

### Type of change

- [x] Performance Improvement
2026-05-11 12:55:27 +08:00
BitToby
4b96362092 Go: implement Encode (embeddings) in NVIDIA driver (#14700)
### What problem does this PR solve?

The NVIDIA Go driver in `internal/entity/models/nvidia.go` shipped with
a stub `Encode`
method that returned `no such method`. `conf/models/nvidia.json` already
lists
`nvidia/llama-3.2-nemoretriever-1b-vlm-embed-v1` as an embedding model,
but the conf had
no `embedding` URL suffix, so the picker had nothing wired even if
`Encode` worked.

A tenant who wanted to use NVIDIA NIM for chat (already working) and
embeddings from a
single provider could not, even though the upstream endpoint is public
at
`https://integrate.api.nvidia.com/v1/embeddings` and uses an
OpenAI-compatible request
body extended with the NVIDIA-specific `input_type` and `truncate`
fields. Several other
Go drivers already implement `Encode` (siliconflow, zhipu-ai, aliyun),
so the interface
and the pattern are well-established.

This PR fills the gap.

### What this PR includes

* `conf/models/nvidia.json`: declare the `embedding` URL suffix
alongside the existing
`chat` and `models` entries. The embedding model entry was already
present, so no
  model addition is needed.
* `internal/entity/models/nvidia.go`: replace the `Encode` stub with a
real
implementation. Adds a small local response type that matches the
OpenAI-compatible
  shape NVIDIA NIM returns.

No factory change. No interface change.

### How the driver works

* Validates `apiConfig` and the API key, validates the model name,
resolves the region
with a default fallback (matching the pattern the merged `ListModels`
and
`CheckConnection` paths in this driver already use), and builds the URL
from
  `BaseURL[region] + URLSuffix.Embedding`.
* Sends all input texts in one request as the `input` array, with the
NVIDIA-specific `input_type: "query"`, `encoding_format: "float"`, and
`truncate: "END"`
  fields, mirroring the Python `NvidiaEmbed` reference.
* Parses `data[*].embedding` and copies each slice into `[][]float64`
indexed by
`data[*].index` so the output order matches the input order even if the
API returns
  items in a different order.
* Handles both `float64` and `float32` element types.
* Empty input returns `[][]float64{}` with no HTTP call.
* Non-200 responses propagate the upstream status line and body.
* A final pass checks every input slot got a vector and returns a clear
error if any
  slot is still nil.
* Per-call 30s context deadline so a slow call cannot block forever.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

* `go build ./internal/entity/models/...` returns exit 0.
* `go vet ./internal/entity/models/...` is clean.
* `gofmt -l internal/entity/models/nvidia.go` is clean.
* The full method set on `NvidiaModel` still matches the `ModelDriver`
interface.
* Pattern parity with the just-merged Aliyun `Encode` (#14647).

Closes #14699
2026-05-11 12:50:50 +08:00
Jack Storment
8ff623fbc4 Go: implement Encode (embeddings) in Ollama driver (#14664)
### What problem does this PR solve?

The Ollama Go driver shipped with a stub \`Encode\` method that returned
\`no such method\`, even though Ollama is one of the most common local
LLM runners and exposes an OpenAI-compatible embeddings endpoint at
\`/v1/embeddings\`.

Ollama users routinely run local embedding models such as
\`nomic-embed-text\`, \`mxbai-embed-large\`, or \`bge-m3\`.
Pulled with \`ollama pull <model>\` and served on the same \`/v1\`
namespace as chat. The existing \`ListModels\` already
discovers them, but because \`Encode\` was a stub, a tenant who picked
one of these models in the Go layer could not
actually run an embedding call.

### What this PR includes

- \`conf/models/ollama.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the
  driver can build the URL from config.
- \`internal/entity/models/ollama.go\`: replace the \`Encode\` stub with
a real implementation. Adds a small local response
  type that matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for local Ollama, so
the Authorization header is only set when both
\`apiConfig\` and \`ApiKey\` are non-nil and non-empty, the same pattern
the recently merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
  Aliyun Encode (#14647) uses.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`OllamaModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the merged Aliyun Encode (#14647) and the existing
SiliconFlow Encode.

Closes #14662
2026-05-11 12:50:15 +08:00
hyl64
77ce88dfcc fix(prompt): reserve system budget in message_fit_in (#14164)
## Summary
This PR fixes the `message_fit_in()` truncation bug reported in #13607.

Changes:
- fix the user-message truncation branch to reserve room for the system
prompt token budget
- guard the zero-token edge case to avoid dividing by zero in the
truncation ratio check
- add focused regression tests covering both the user-dominant
truncation path and the zero-token boundary case

## Validation
```bash
pytest -q --noconftest test/unit_test/rag/prompts/test_generator_message_fit_in.py
```

Result: `2 passed`

Closes #13607
2026-05-11 12:44:27 +08:00
07heco
e46989832e fix: complete robustness fixes for rerank module addressing all review comments (#14265)
## Summary
This PR fully addresses all CodeRabbit review feedback and enhances the
robustness of the reranking module with 100% backward compatibility.

## Key Fixes
1. Fixed JinaRerank hardcoded base_url to support subclass endpoint
overrides
2. Corrected GPUStackRerank exception handling to use proper requests
exceptions and preserve stack traces
3. Added 30s timeout to all API calls to prevent service hanging
4. Added empty input validation for all rerank providers
5. Replaced direct dict key access with .get() to eliminate KeyError
crashes
6. Fixed _normalize_rank edge case for empty arrays
7. Implemented missing functionality for Ai302Rerank
8. Standardized type hints and fixed typo issues

## Compatibility
- No breaking changes to any existing functionality
- All rerank providers work as originally intended
- Fully compatible with existing configurations and workflows

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 12:40:41 +08:00
Panda Dev
fa53b93dd5 Go: implement Encode (embeddings) in vLLM driver (#14688)
### What problem does this PR solve?

The vLLM Go driver shipped with a stub \`Encode\` method that returned
\`not implemented\`, even though vLLM is one of the most common
production-grade self-hosted inference servers and exposes an
OpenAI-compatible embeddings endpoint at \`/v1/embeddings\`.

Users who self-host \`BAAI/bge-m3\`, \`Qwen3-Embedding-*\`,
\`NV-Embed-v2\`, or similar models on vLLM could not run an embedding
call through the Go layer. The existing \`ListModels\` already discovers
the loaded models, but the embedding path failed because \`Encode\` was
a stub.

### What this PR includes

- \`conf/models/vllm.json\`: add \`\"embedding\": \"embeddings\"\` under
\`url_suffix\` so the driver can build the URL from config.
- \`internal/entity/models/vllm.go\`: replace the \`Encode\` stub with a
real implementation. Adds a small local response
  type that matches the OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate the model name. The API key is optional for self-hosted vLLM,
so the Authorization header is only set when both \`apiConfig\` and
\`ApiKey\` are non-nil and non-empty, the same pattern the recently
merged CheckConnection PR (#14614) uses.
- Resolve the region with a default fallback. Return a clear "missing
base URL" error when the user has not configured
  the local access address yet.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, the same pattern the merged
  Aliyun Encode (#14647) and in-flight Ollama Encode (#14664) use.
- Send \`{model, input: [texts]}\` in one request.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\`, so the output
  order matches the input order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Length mismatch between input and result, out-of-range index, and any
missing slot all return clear errors instead
  of silent zero vectors.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`VllmModel\` still matches the \`ModelDriver\`
interface.
- Pattern parity with the merged Aliyun Encode (#14647), the in-flight
Ollama Encode (#14664), and the existing
  SiliconFlow Encode.

Closes #14687
2026-05-11 12:09:17 +08:00
Qinsanz
d6660cf156 fix(keyword_extraction): accept Chinese commas/semicolons/newlines as keyword delimiters (#14540)
## What
Widen the keyword delimiter in `rag/svr/task_executor.py`:
both `build_chunks` (LLM `keyword_extraction` cache parsing) and
`run_dataflow` (chunk-level `keywords` ingestion) now split on
`, , ; ; 、 \r \n` instead of only ASCII comma.

## Why
`rag/prompts/keyword_prompt.md` instructs the LLM:

> The keywords are delimited by ENGLISH COMMA.

In practice, Chinese-leaning models (Qwen / Tongyi-Qianwen, GLM,
etc.) frequently ignore this instruction when the source content is
Chinese and emit Chinese commas (`,`) instead. Result:
`cached.split(",")` sees the full LLM output as a *single* keyword.

Repro: `auto_keywords>=4` + Chinese docs + `qwen-plus@Tongyi-Qianwen`.
We observed entries in `important_kwd` like
`"功能介绍,配置说明,参数详解,问题排查"` — one bucket instead of four.

## Impact
- Silent data-quality bug; no exception thrown.
- BM25 `important_kwd^30` boost effectively stops firing — the
  indexed term is the whole list, never matches user query tokens.
- Any downstream aggregating `important_kwd` (tagging, analytics,
  candidate-keyword review UIs) sees garbage.

## Compatibility
- Pure widening of the splitter; ASCII-comma-only outputs continue
  to work identically.
- No schema / API change.

## Test plan
Manually verified against `qwen-plus@Tongyi-Qianwen` with
`auto_keywords=10` on Chinese .txt files:

- Before: `important_kwd` contains one element per chunk that is the
  full LLM string with `,`-separated phrases inside.
- After: `important_kwd` contains N elements, one per phrase, as the
  LLM intended.
2026-05-11 12:05:24 +08:00
BitToby
bfb4a0eea2 Go: implement Encode (embeddings) in Gitee AI driver (#14698)
### What problem does this PR solve?

The Gitee AI Go driver in `internal/entity/models/gitee.go` shipped with
a stub `Encode` method that returned `gitee, no such method`, even
though `conf/models/gitee.json` already wires the `embedding` URL
suffix. The conf also listed no embedding models, so the picker had
nothing to select.

This blocked any tenant who wanted to use Gitee AI for chat, rerank
(already working, see #14656), and embeddings from a single provider.

This PR fills the gap, mirroring the just-merged Aliyun `Encode`
(#14647):

- `internal/entity/models/gitee.go`: replace the `Encode` stub with a
real implementation.
Validates inputs, resolves the region with a default fallback, POSTs the
standard OpenAI-compatible `{"model", "input": [...]}` body to
`BaseURL[region] + URLSuffix.Embedding`, parses `data[*].embedding`
indexed by `data[*].index` so output order matches input order, handles
both `float64` and `float32` element types, and uses a 30s per-call
context deadline matching the merged `Rerank`.
- `conf/models/gitee.json`: add `BAAI/bge-m3` so the embedding picker
has something to select.

No factory change. No interface change. No URL suffix change.

Verified with `go build`, `go vet`, and `gofmt -l` : all clean.

Closes #14697

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 11:56:46 +08:00
VincentLambert
b83e2ae5a2 fix: handle missing parent chunk in retrieval_by_children (#14556)
### What problem does this PR solve?

`retrieval_by_children()` in `rag/nlp/search.py` crashes with a
`TypeError: 'NoneType' object is not subscriptable` when a parent
("mom") chunk referenced by child chunks is missing from the index.

This happens when the index is in an inconsistent state — for example
after a partial re-index, a document deletion that didn't clean up all
children, or a race condition during ingestion. `dataStore.get()`
returns `None` for the missing parent, and the subsequent access to
`chunk["content_with_weight"]` raises a `TypeError`.

**Stack trace:**
```
TypeError: 'NoneType' object is not subscriptable
  File "rag/nlp/search.py", line 792, in retrieval_by_children
    "content_with_weight": chunk["content_with_weight"],
```

### Type of change

- [x] Bug Fix

### Fix

When `dataStore.get()` returns `None` for a parent chunk, fall back to
using the child chunks directly and continue processing the remaining
parents. This preserves retrieval results for all other chunks rather
than aborting the entire query with an exception.

```python
chunk = self.dataStore.get(id, idx_nms[0], [ck["kb_id"] for ck in cks])
if chunk is None:
    chunks.extend(cks)
    continue
```

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 11:55:44 +08:00
Sp1kyss
e6cb9faace fix: close two security analyzer bypass paths in sandbox executor (#14690)
## Summary

Two bypass vectors in the sandbox code security analyzer allowed
malicious code to pass the safety check undetected and reach the Docker
executor.

### 1. JavaScript: template-literal bypass of `require()` block

The `SecureJavaScriptAnalyzer` regex patterns used `['"]` to match
module names, covering only single and double quotes. An attacker could
use ES6 template literals to bypass all three `require` checks:

`javascript
const cp = require(`child_process`);
async function main() {
  return cp.execSync('cat /etc/passwd').toString();
}
`

The same bypass applied to `fs` and `worker_threads`.

**Fix:** Updated all three `require` patterns from `['"]` to `['"\]` to
also match backtick template literals.

### 2. Python: `builtins` not blocked + attribute-call blind spot in
`visit_Call`

`visit_Call` only checked `ast.Name` nodes, so attribute-style calls
like `module.func()` were invisible to the analyzer. Additionally,
`builtins` was absent from `DANGEROUS_IMPORTS`. Combined, this allowed:

`python
import builtins
def main():
    builtins.exec('import os; os.system("id")')
`

Neither the import nor the exec call triggered any flag.

**Fix:** Added `builtins` to `DANGEROUS_IMPORTS` and added an
`ast.Attribute` branch to `visit_Call` so that `module.dangerous_func()`
style calls are caught alongside bare `dangerous_func()` calls.

## Tests

Added four regression tests covering each new bypass vector:
- `test_javascript_child_process_template_literal_is_rejected`
- `test_javascript_fs_template_literal_is_rejected`
- `test_python_builtins_import_is_rejected`
- `test_python_attribute_eval_call_is_rejected`

---------

Co-authored-by: bounty-hunter <bounty@hunter.local>
2026-05-11 11:46:27 +08:00
Joseff
827cceccba Fix(Go): correct Name() and region URL fallback in Aliyun driver (#14673)
### What problem does this PR solve?

Two bugs in the Aliyun Go driver:

1. **`Name()` returns `"siliconflow"`** — a copy-paste bug from when the
driver was created. `Name()` is used in error messages and log output,
so every Aliyun error incorrectly attributed itself to SiliconFlow.

2. **Silent empty URL for unknown regions in `ChatWithMessages`,
`ChatStreamlyWithSender`, and `ListModels`** — all three methods
construct the request URL as `z.BaseURL[region]` without checking
whether the key exists. For an unrecognised region this returns `""`,
producing a malformed URL like `"/chat/completions"` that the HTTP
transport rejects with a confusing error. `Encode` and `Rerank` (already
merged) correctly fall back to `"default"` and return a clear error.
This PR applies the same pattern to the remaining three methods.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 11:26:24 +08:00
Carmen Fernández Ruiz
f852a7524e fix(go): wire Google CheckConnection to ListModels (#14660)
### What problem does this PR solve?

Closes #14703

`GoogleModel.CheckConnection` currently returns a hardcoded `no such
method` error even though the Google Go driver already supports
`ListModels`. This makes provider connection checks fail regardless of
whether the configured API key can list Google models.

This PR makes `CheckConnection` call `ListModels`, adds a small API-key
guard for nil, empty, and whitespace-only keys, and keeps `ListModels`
useful by following paginated Google model responses.

### What stays unchanged

* Google model listing still uses the Google GenAI SDK with
`genai.BackendGeminiAPI`.
* Model names still come from `models.Items[*].Name`.
* `Balance`, `Encode`, chat, streaming, provider config, and factory
wiring are unchanged.

### Tests and validation

Added focused unit coverage for:

* `CheckConnection` delegating to `ListModels` and returning its error
* nil, missing, empty, and whitespace-only API key validation
* model-name passthrough from the list-models adapter
* paginated model listing, empty-result preservation, and next-page
error propagation

Validated current PR head `17ceef43515ba8c46c254dd349b9085bf26dcbea`
locally with Go 1.25.0:

* `go test ./internal/entity/models -run
'TestGoogleModel|TestCollectGoogleModelNames' -count=1 -v` - PASS
* `go test ./internal/entity/models -count=1` - PASS
* `go test -race ./internal/entity/models -count=1` - PASS
* `gofmt -w internal/entity/models/google.go
internal/entity/models/google_test.go` - PASS, no diff
* `git diff --check` - PASS

### Type of change

* [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 11:25:17 +08:00
Joseff
f4f8bed9f7 Go: implement Encode (embeddings) in Google Gemini driver (#14682)
### What problem does this PR solve?

- Implements the `Encode` method in the Google Gemini driver, which was
previously a stub returning `not implemented`
- Uses the `google.golang.org/genai` SDK's `EmbedContent` API, which
routes to the `batchEmbedContents` endpoint internally — all texts are
sent in a single request
- Adds `text-embedding-004` (max 2048 tokens) to
`conf/models/google.json`
- Response values are `[]float32` from the SDK and are cast to
`[]float64` to satisfy the `ModelDriver` interface

## Files changed

- `internal/entity/models/google.go` — full `Encode` implementation
- `conf/models/google.json` — adds `text-embedding-004` embedding model

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 11:24:21 +08:00
Ricardo-M-L
13922209e6 fix(llm): add timeout to HTTP requests in LLM integration layer (#14313)
### What problem does this PR solve?

Multiple `requests.post()` calls across the LLM integration layer lack a
`timeout` parameter. Without a timeout, a single unresponsive upstream
service can block the calling thread **indefinitely**, eventually
exhausting the thread pool and degrading the entire system.

This is a well-known issue — Python's `requests` library defaults to
`timeout=None` (infinite wait), and [the library docs explicitly
recommend](https://requests.readthedocs.io/en/latest/user/advanced/#timeouts)
always setting a timeout.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Change

Added `timeout` to all `requests.post()` calls missing it:

| File | Calls fixed | Timeout |
|------|-------------|---------|
| `rag/llm/rerank_model.py` | 9 | 30s |
| `rag/llm/embedding_model.py` | 8 | 30s |
| `rag/llm/cv_model.py` | 3 | 60s |
| `rag/llm/tts_model.py` | 2 | 60s |
| `rag/llm/sequence2txt_model.py` | 2 | 60s |

Embedding/rerank calls use 30s (lightweight API calls). Vision, TTS, and
audio transcription use 60s (heavier workloads with file uploads).

Note: other files in the codebase (e.g. `check_minio_alive`,
`check_ragflow_server_alive`) already use `timeout=10`, so this PR
brings the LLM layer in line with existing practice.

Signed-off-by: Ricardo-M-L <Sibyl_Hartmanbnb@webname.com>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 11:19:07 +08:00
Paras Sondhi
51b73850e1 feat: make sandbox Dockerfile mirrors optional with ARG (#14553)
### What problem does this PR solve?

Resolves #14447. *(Note: This supersedes stalled PR #14448 and
implements the requested CodeRabbitAI fixes).*

Currently, the Dockerfiles inside `agent/sandbox/sandbox_base_image`
(both Python and Node.js) have hardcoded Chinese package mirrors. This
forces the mirrors on all users globally, which causes build network
timeouts for contributors outside of China.

This PR introduces an enhancement to fix the issue by:
1. Implementing the `NEED_MIRROR` build argument in the sandbox
Dockerfiles.
2. Replacing static `ENV` instructions with conditional shell logic
inside `RUN` blocks to dynamically set the package registries.
3. Allowing the build to cleanly fall back to default global registries
(`pypi.org` and `npmjs.org`) when `--build-arg NEED_MIRROR=0` is passed.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 11:01:43 +08:00
BitToby
39a1773f7f Go: implement ListModels in Volcengine driver (#14702)
### What problem does this PR solve?

The VolcEngine Go driver in `internal/entity/models/volcengine.go`
shipped with a
`ListModels` stub that returned `volcengine, no such method`.
`conf/models/volcengine.json`
also did not declare a `models` URL suffix, so the model picker had
nothing to call even
if the method body were filled in.

A tenant who configured Volcengine (Doubao / Ark) as a provider could
not see the list of
available endpoints from the RAGFlow UI. Several other Go drivers
already implement
`ListModels` against the OpenAI-compatible `/models` endpoint (deepseek,
gitee, nvidia,
openai, siliconflow), so the interface and pattern are well-established.

This PR fills the gap.

### What this PR includes

* `conf/models/volcengine.json`: declare the `models` URL suffix
alongside the existing
  `chat`, `files`, and `embedding` entries. The Ark v3 API exposes
`https://ark.cn-beijing.volces.com/api/v3/models`, so the suffix is just
`models`.
* `internal/entity/models/volcengine.go`: replace the `ListModels` stub
with a real
implementation. Reuses the package-level `DSModelList` / `DSModel` types
that
DeepSeek, Gitee, and SiliconFlow already use to parse the
OpenAI-compatible models
  response shape.

No factory change. No interface change.

### How the driver works

* Resolves the region with a default fallback, the same way the other
VolcEngine methods
  in this driver already do.
* Builds the URL from `BaseURL[region] + URLSuffix.Models`, with
`strings.TrimSuffix` on
  the base to keep the join robust.
* Issues a `GET` with optional `Authorization: Bearer <api_key>` (the
header is omitted
when no key is configured, mirroring the existing NVIDIA `ListModels`).
* Reads the response body once, surfaces a non-200 with the upstream
status line plus
  body, and parses the JSON via the shared `DSModelList` type.
* Returns the model id list in input order. When the response includes
an `owned_by`
field, the entry is rendered as `id@owned_by`, matching the convention
used by the
  other Go drivers.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


### How was this tested?

* `go build ./internal/entity/models/...` returns exit 0.
* `go vet ./internal/entity/models/...` is clean.
* `gofmt -l internal/entity/models/volcengine.go` is clean.
* The full method set on `VolcEngine` still matches the `ModelDriver`
interface.
* Endpoint reachability check: `GET
https://ark.cn-beijing.volces.com/api/v3/models`
returns `401 Unauthorized` without an API key, confirming the path
exists and accepts
  Bearer authentication.
* Pattern parity with DeepSeek, Gitee, NVIDIA, and SiliconFlow
`ListModels`.

Fixes #14701

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-11 10:59:18 +08:00
VincentLambert
08bb53bbb1 Feat: add BedrockCV for vision/image2text inference via LiteLLM (#14705)
## Summary

- `CvModel["Bedrock"]` was absent from `rag/llm/cv_model.py`, causing
`model_instance()` to return `None` when a Bedrock model was used as a
PDF parser — even after correct model resolution.
- This PR adds `BedrockCV`, enabling Bedrock vision models (e.g.
`amazon.nova-pro-v1:0`, `anthropic.claude-3-5-sonnet`) to be used as PDF
parsers.

## What problem does this PR solve?

When a Bedrock model is selected as the PDF parser in a knowledge base,
ingestion failed with:

```
'LiteLLMBase' object has no attribute 'describe_with_prompt'
```

The root cause: `LiteLLMBase` (the Bedrock chat implementation) was the
only registered handler for the Bedrock factory. It does not implement
`describe_with_prompt`. `CvModel` had no Bedrock entry, so
`model_instance()` returned `None` for `image2text` requests.

## Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Changes

**`rag/llm/cv_model.py`**

Adds `BedrockCV(Base)` with `_FACTORY_NAME = "Bedrock"`:

- Uses `litellm.completion` with the `bedrock/` prefix (consistent with
`LiteLLMBase`)
- Parses AWS credentials from the JSON key assembled by `add_llm`
(`auth_mode`, `bedrock_ak`, `bedrock_sk`, `bedrock_region`,
`aws_role_arn`)
- Supports three auth modes: `access_key_secret`, `iam_role` (via STS
`assume_role`), and default credential chain (IRSA, instance profile)
- Implements `describe_with_prompt` and `describe`

## Test plan

- [ ] Configure a Bedrock vision model (e.g. `amazon.nova-pro-v1:0`)
with valid AWS credentials
- [ ] Select it as PDF parser in a knowledge base
- [ ] Verify ingestion of a PDF document completes without errors
- [ ] Verify `CvModel["Bedrock"]` resolves to `BedrockCV`

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-11 10:29:58 +08:00
Ahmad Intisar
3c4d1da98f Feature/table parser column roles (#13710)
### What problem does this PR solve?

The table file parser (CSV/Excel) currently treats all columns
identically — every column is both vectorized (embedded in chunk text)
and stored as filterable metadata. There's no way for users to control
which columns should be searchable by semantic meaning versus which
should only be filterable attributes.

For example, when ingesting a news articles CSV with columns like title,
content, country, category, source, etc., the embedding includes
metadata fields like country: Brazil and source: Reuters in the chunk
text, which dilutes the semantic quality of the embedding without adding
retrieval value.

The RDBMS connector (MySQL/PostgreSQL) already supports content_columns
/ metadata_columns, but this capability was missing for file-based table
ingestion.

This PR adds column-level control (vectorize / metadata / both) for the
table file parser, following RAGFlow's existing patterns.

Backward compatible: Datasets without table_column_roles or with
table_column_mode: auto behave exactly as before (all columns = both).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-11 10:06:04 +08:00
Igor Ilinskii
889aba6a32 fix base_url handling in HuggingfaceRerank (#14555)
### What problem does this PR solve?

HuggingfaceRerank.post() unconditionally prepends `http://` to base_url,
which already contains a protocol. This creates invalid URLs like
http://http://127.0.0.1:8080/rerank, breaking all requests. The fix
normalizes URL handling to match the rest of the codebase, removing
redunant `http://`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Related Issues
- #7318 
- #7796

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-11 10:04:40 +08:00
Tim Wang
ed01ac9994 Fix: resolve template strings in tool component parameters (#14601)
## Summary

- Tool-type components (Email, Invoke, etc.) fail to resolve template
strings that mix variable references with literal text in their
parameters.
- This adds template string resolution to `get_input()` in
`ComponentBase`, reusing existing `get_input_elements_from_text()` and
`string_format()` methods.

## Problem

`get_input()` in `ComponentBase` handles two cases:
1. **Pure reference** (`{Component:ID@field}`) — resolved via
`is_reff()` + `get_variable_value()`
2. **Literal value** — passed through as-is

But template strings like `{UserFillUp:X@name}@duke.edu` or `Question
from {Agent:Y@topic}` fall through to the literal branch because
`is_reff()` returns `False` (it expects the entire string to be a single
reference). The unresolved template is passed directly to the tool.

This affects **all** tool components (Email, Invoke, etc.) that need
mixed reference + text parameters — for example, constructing email
addresses or subjects dynamically.

## Fix

```python
# In get_input(), between is_reff check and literal fallback:
elif isinstance(v, str) and re.search(self.variable_ref_patt, v):
    elements = self.get_input_elements_from_text(v)
    kv = {k: e.get('value', '') for k, e in elements.items()}
    self.set_input_value(var, self.string_format(v, kv))
```

This reuses `get_input_elements_from_text()` and `string_format()` which
are already used by `Message` components for the same purpose. The fix
only activates when the string contains at least one variable reference
pattern but is not a pure reference.

## Test plan

- [x] Pure references (`{Component:ID@field}`) still resolve correctly
via `is_reff()` path
- [x] Literal values without references pass through unchanged
- [x] Template strings like `{ref}@duke.edu` resolve the reference and
keep the literal suffix
- [x] Template strings like `Question from {ref}` resolve correctly
- [x] Multiple references in one string (`{ref1} and {ref2}`) both
resolve
- [x] Message components unaffected (they use their own template
resolution in `_run`)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-11 10:01:41 +08:00
Mehmet Karakose
7ec87f7cb7 fix(auth): fall back to session-based auth in _load_user (#14569)
## Summary

Closes #13663.

OAuth / OIDC callbacks call `login_user(user)` which writes `_user_id`
into the session cookie, but `_load_user()` in `api/apps/__init__.py`
only ever looked at the `Authorization` header. The SPA's response
interceptor wipes the Authorization value from `localStorage` on the
first 401 it sees — meaning that during the post-redirect window after
an OAuth login, a single transient 401 sends every subsequent request
back to the login page even though `login_user()` had already
established a perfectly good server-side session.

The reporter's analysis traces this all the way through the redirect →
`navigate('/')` → first request → empty header → 401 → `removeAll()` →
infinite-redirect-to-login chain.

## What changed

- New `_load_user_from_session()` helper that reads
`session["_user_id"]`, looks up the user in `UserService` (with the same
`StatusEnum.VALID` and `access_token` checks already used elsewhere),
and assigns `g.user`.
- Every `return None` path in `_load_user()` now routes through that
helper before giving up:
  - missing `Authorization` header
  - malformed `bearer ` prefix
  - empty / too-short JWT payload
  - JWT signature failure
  - JWT-resolved user not found / has no `access_token`
  - `APIToken.query()` fallback exhausted

The JWT and API-token paths still take precedence — the session is only
consulted when those can't authenticate the request. So existing
local-login and SDK callers see no behaviour change; only OAuth / OIDC
users that hit the original race now stay logged in.

The Bearer-prefix issue called out in #13663 (lines 103-110) is already
handled in the current code, so this PR only addresses the second half
of the report.

## Test plan

- [ ] Configure OIDC under `oauth` in `service_conf.yaml`
- [ ] Click the OIDC login button, complete auth at the IdP
- [ ] Confirm that navigating between pages no longer bounces back to
`/login`
- [ ] Confirm local email/password login still issues + accepts JWTs
- [ ] Confirm SDK/API key callers still authenticate via `Authorization:
Bearer <api-token>`

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-05-11 09:59:52 +08:00
很拉风的James
6cb4bc2947 Fix: Radio.Group cloneElement crashes on non-element children (#14407)
### What problem does this PR solve?

`Radio.Group` in `web/src/components/ui/radio.tsx` injects the parent's
`disabled` prop into each child via `React.cloneElement` with
`as React.ReactElement` and no validation.

This throws at runtime when a consumer passes strings, numbers, `null`,
`false`, or other non-element nodes, while the cast hides the unsafe
access from TypeScript.

Use `React.isValidElement<RadioProps>(child)` as a type guard before
calling `cloneElement`. Non-element children pass through unchanged,
and `child.props` access becomes type-checked without an `as` cast.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-11 09:54:42 +08:00
Panda Dev
6bfe0f9a10 Go: implement Encode (embeddings) in OpenAI driver (#14630)
### What problem does this PR solve?

The OpenAI Go driver landed in #14605 with chat, list models, and check
connection. Encode was left as a stub that returns \`not implemented\`.

\`conf/models/openai.json\` already lists three embedding models out of
the box:

- text-embedding-ada-002
- text-embedding-3-small
- text-embedding-3-large

So a tenant who picked one of these in the Go layer could not actually
run an embedding call. This PR fills the gap.

### What this PR includes

- \`conf/models/openai.json\`: add \`\"embedding\": \"embeddings\"\`
under \`url_suffix\` so the driver can build the URL from config. This
matches the \`URLSuffix.Embedding\` field used by other drivers
(siliconflow, zhipu-ai).
- \`internal/entity/models/openai.go\`: replace the Encode stub with a
real implementation that POSTs to \`/v1/embeddings\`. Adds a small local
response type \`openaiEmbeddingResponse\`.

No factory change. No interface change.

### How the implementation works

- Validate \`apiConfig\` and the API key, validate the model name. Use
the existing \`baseURLForRegion\` helper so an unknown region fails fast
with a clear error.
- Wrap the request with \`context.WithTimeout(nonStreamCallTimeout)\` so
the call has a clear deadline. Same pattern as \`ChatWithMessages\` and
\`ListModels\` already use in this file.
- Send all input texts in one request. The OpenAI API accepts the
\`input\` field as an array.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\` so the output order matches the input order
even if the API returns items in a different order.
- Handle both \`float64\` and \`float32\` element types, the way the
SiliconFlow driver does.
- An empty input slice returns \`[][]float64{}\` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks that every input slot got a vector. If any slot is
still nil, return a clear error so the caller does not silently use a
zero vector.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`OpenAIModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Encode implementation
(\`internal/entity/models/siliconflow.go\`).

Closes #14629

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-10 10:31:37 +08:00
Jin Hai
048ec2fc5c Go: fix siliconflow rerank issue (#14743)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 20:45:53 +08:00
Jin Hai
779cd83862 Go: fix Baidu rerank issue (#14742)
### What problem does this PR solve?

top_n is missing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 20:05:57 +08:00
Hunnyboy1217
782084780e feat(connectors): ETag-based bypass for incremental S3 ingestion (#14628) (#14677)
### What problem does this PR solve?

S3-family connector syncs currently re-download every in-window object
just so we can compute `xxhash128(blob)` and compare against
`Document.content_hash`. Anything that bumps `LastModified` without
changing bytes (`aws s3 cp` touches, bucket re-encryption, etc.) pays
full bandwidth and re-parses files that didn't actually change. #14628
covers the broader incremental-ingestion redesign; this PR is the first
slice.

The fix is a pre-listing short-circuit. `BlobStorageConnector` (S3 / R2
/ GCS / OCI / S3-compat) now implements a new `FingerprintConnector`
interface: `list_keys()` paginates `list_objects_v2` and yields
`KeyRecord(key, fingerprint)` where `fingerprint = xxhash128(ETag)`. The
orchestrator joins those against the connector's existing `{doc_id:
content_hash}` map and only calls `get_value(key)` when the fingerprint
differs. Unchanged keys are skipped entirely — no `GetObject`, no
re-parse.

No DDL. xxhash128(ETag) is 32 hex chars and reuses the existing
`Document.content_hash` column per @yingfeng's suggestion; the connector
decides at listing time whether to populate it. Local uploads and
connectors that don't opt in fall through to the existing post-download
`xxhash128(blob)` path with no behavior change.

This is PR-1 of a 4-PR series — full design lives on #14628. Subsequent
PRs extend tier 1 to local FS / WebDAV / Dropbox / Seafile / RDBMS
(PR-2), wire up tier 2 cursor connectors with `SyncLogs.next_checkpoint`
(PR-3), and unify deletion via `KeyRecord(deleted=True)` reconciliation
(PR-4). Holding those back keeps this PR additive and reviewable on its
own.

#### Files touched

- `common/data_source/models.py` — new `KeyRecord`; optional
`fingerprint` on `Document`
- `common/data_source/interfaces.py` — `IncrementalCapability` enum,
`FingerprintConnector` ABC
- `common/data_source/blob_connector.py` — `BlobStorageConnector`
implements `FingerprintConnector`; per-object download factored into
`_build_document_from_obj()` so `_yield_blob_objects`, `list_keys`,
`get_value` all share it
- `rag/svr/sync_data_source.py` —
`_BlobLikeBase._fingerprint_filtered_generator` does the bypass loop;
`_run_task_logic` plumbs `doc.fingerprint` into the upload dict
- `api/db/services/document_service.py` —
`list_id_content_hash_map_by_kb_and_source_type()` helper
- `api/db/services/connector_service.py` + `file_service.py` —
fingerprint flows through `duplicate_and_parse → upload_document` and
lands in `content_hash`
- `test/unit_test/common/test_blob_connector_fingerprint.py` — 14 tests
covering ETag normalization (single-part, multipart, quoted, empty),
`list_keys()` not calling `GetObject`, `get_value()` materializing with
fingerprint, deterministic/stable fingerprints, and the bypass loop
asserting `GetObject` is *not* called on a match

#### Worth flagging for review

Old `_BlobLikeBase._generate` called `poll_source(start, now)` with a
`LastModified` window when `poll_range_start` was set. New code uses
`_fingerprint_filtered_generator` (full bucket listing + fingerprint
compare) outside of explicit `reindex=1`. Strictly better for
unchanged-bucket cases since it skips `GetObject`, but it does mean
every sync now does a full `list_objects_v2` paginate. Should still be
cheap for most buckets — flagging in case anyone has a very large bucket
where the time-window filter was meaningful.

On migration: existing rows have `content_hash = xxhash128(blob)` from
the old code. The first sync after this lands sees ETag-derived
fingerprints that don't match, re-fetches every object once, and writes
the new fingerprint. From the second sync onward the bypass works as
expected. "Slow day one, fast every day after." A `fingerprint_backfill:
trust` opt-out is sketched in the design doc but not in this PR.

#### Test plan

- [x] `uv run ruff check` — clean on all 8 touched files
- [x] `uv run pytest
test/unit_test/common/test_blob_connector_fingerprint.py -v` — 14 passed
- [x] Broader unit-test suite — no regressions in anything I touched
- [ ] Manual smoke against a real S3 bucket — configure a connector, run
sync twice, expect the second sync to log `bypassed=N, fetched=0` and no
`GetObject` calls in CloudTrail / bucket access logs
- [ ] Manual smoke with `reindex=1` — confirm the full re-download path
still works

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-05-09 20:03:56 +08:00
Haruko386
7931b693dc Go: implement provider: Baidu (#14741)
### What problem does this PR solve?

This PR completes the Baidu Qianfan provider integration in RAGFlow.

**The following functionalities are now supported:**

- [x] Chat / Think Chat / Stream Chat / Stream Think Chat
- [x] Embedding
- [x] Rerank
- [x] Model listing
- [x] Provider connection checking
- [ ] Balance

-----

**Verified examples from the CLI:**

```plaintext
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'embedding-3@test@zhipu-ai' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 16        | 0     |
| 16        | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'qwen3-reranker-4b@test@baidu' top 2;
+-------+---------------------+
| index | relevance_score     |
+-------+---------------------+
| 0     | 0.974821150302887   |
| 1     | 0.14223189651966095 |
| 2     | 0.08632347732782364 |
+-------+---------------------+

RAGFlow(user)> think chat with 'deepseek-v3.2@test@baidu' message 'who r u'
Thinking: Hmm, the user is asking for a simple introduction. This is straightforward – no need for overcomplication. 

I should give a clear, friendly response that covers my basic identity as an AI assistant, my purpose, and my capabilities. Keeping it concise but informative is key here. 

Mentioning my creator Anthropic adds credibility, and ending with an offer to help invites further interaction. No need for technical details unless the user asks later.
Answer: Hello! I'm an AI assistant created by Anthropic, designed to help with a wide variety of tasks. You can think of me as a helpful digital companion—I can answer questions, assist with writing, help solve problems, provide explanations, and engage in conversation on many topics. I'm here to help with whatever you need! How can I assist you today?
Time: 8.103902

RAGFlow(user)> stream think chat with 'deepseek-v3.2@test@baidu' message 'who r u'
Thinking: mm, the user is asking "who r u" with casual spelling. This is a straightforward identity question. should give a clear, friendly introduction without overcomplicating it. Can start with my core function as an AI assistant, mention my creator, and briefly state my key capabilities. response should be welcoming and invite further interaction since this seems like an introductory question. Keeping it concise but covering the essentials: who I am, what I do, and how I can help.
Answer: ! I am DeepSeek, an AI assistant created by DeepSeek Company. I'm designed to help answer questions, provide information, assist with various tasks, and engage in conversations on a wide range of topics. I'm here to assist you with whatever you need - whether it's answering questions, helping with analysis, writing, coding, or just having a friendly chat!Is there anything specific I can help you with today? 😊
Time: 7.219703

RAGFlow(user)> list supported models from 'baidu' 'test'
+--------------------------------------+
| model_name                           |
+--------------------------------------+
| ernie-3.5-8k-preview                 |
| ernie-4.0-8k                         |
| ernie-4.0-turbo-8k-latest            |
| ernie-4.0-turbo-8k-preview           |
| ernie-4.0-8k-preview                 |
| ernie-speed-pro-128k                 |
| ernie-char-fiction-8k                |
| ernie-3.5-8k                         |
| ernie-3.5-128k                       |
| ernie-lite-pro-128k                  |
| ernie-novel-8k                       |
| ernie-4.0-turbo-8k                   |
| ernie-4.0-turbo-128k                 |
| ernie-4.0-8k-latest                  |
| irag-1.0                             |
| ...........                          |
| glm-5.1                              |
| ernie-image-turbo                    |
| deepseek-v4-pro                      |
| deepseek-v4-flash                    |
| ernie-5.1                            |
+--------------------------------------+

RAGFlow(user)> check instance 'test' from 'baidu'
SUCCESS
```

Additionally, this PR fixes an incorrect error message typo:

Before:

```go
fmt.Errorf("API requestssss failed with status %d: %s : %s", ...)
```

After:

```go
fmt.Errorf("API request failed with status %d: %s", ...)
```

This PR mainly improves provider compatibility, API completeness, and
runtime stability.

### Type of change

* [x] Bug Fix (non-breaking change which fixes an issue)
* [x] New Feature (non-breaking change which adds functionality)
* [x] Refactoring
2026-05-09 19:21:13 +08:00
Liu An
57b24be6d6 Docs: Update version references to v0.25.2 in READMEs and docs (#14731)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.1 to v0.25.2
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-05-09 19:06:05 +08:00
writinwaters
a3de873617 Docs: Updated release date (#14740)
### What problem does this PR solve?

Updated v0.25.2 release date.

### Type of change


- [x] Documentation Update
2026-05-09 18:49:33 +08:00
euvre
f4b8f53b6d Fix: restore embedding model switching for datasets with existing chunks (#14732)
### What problem does this PR solve?

## Problem

During the REST API refactoring (#13690), the
`/api/v2/kb/check_embedding` endpoint was removed and never migrated to
the new RESTful structure. The frontend was pointed to the
`/api/v1/datasets/{id}/embedding` endpoint (which is `run_embedding` — a
completely different function). Additionally, a hard guard was
introduced that rejects any `embd_id` change when `chunk_num > 0`,
making it impossible to switch embedding models on datasets with
existing chunks.

## Root Cause

1. **Missing endpoint**: The old `check_embedding` logic (sample random
chunks, re-embed with the new model, compare cosine similarity) was not
carried over to the new REST API service layer.
2. **Wrong frontend URL**: `checkEmbedding` in `api.ts` pointed to
`/datasets/{id}/embedding` (`run_embedding`) instead of a dedicated
check endpoint.
3. **Overly restrictive guard**: `dataset_api_service.py` line 310
blocked all `embd_id` updates when `chunk_num > 0`. This check did not
exist in the pre-refactor code — it was incorrectly introduced during
the refactor.

## Changes

### Backend

- **`api/apps/services/dataset_api_service.py`**
  - Remove the `chunk_num > 0` hard guard on `embd_id` updates
- Add `check_embedding()` service function: samples random chunks,
re-embeds them with the candidate model, computes cosine similarity,
returns compatibility result (avg ≥ 0.9 = compatible)
  - Add `import re` for the `_clean()` helper

- **`api/apps/restful_apis/dataset_api.py`**
- Add `POST /datasets/<dataset_id>/embedding/check` endpoint following
the new REST API conventions
  - Clean up unused top-level imports (`random`, `re`, `numpy`)

### Frontend

- **`web/src/utils/api.ts`**
- Fix `checkEmbedding` URL from `/datasets/${datasetId}/embedding` →
`/datasets/${datasetId}/embedding/check`

### Tests

-
**`test/testcases/test_http_api/test_dataset_management/test_update_dataset.py`**
- Update `test_embedding_model_with_existing_chunks` to assert success
(`code == 0`) instead of expecting the old `102` error

-
**`test/testcases/test_web_api/test_dataset_management/test_dataset_sdk_routes_unit.py`**
- Update `test_update_route_branch_matrix_unit` to assert
`RetCode.SUCCESS` when updating `embd_id` on a chunked dataset,
replacing the old `chunk_num` error assertion

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-05-09 18:48:57 +08:00
buua436
330257b611 Fix: Add legacy system healthz route (#14738)
### What problem does this PR solve?

Add legacy system healthz route

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:49:26 +08:00
Jin Hai
17d71e5d79 Go CLI: embed and rerank (#14735)
### What problem does this PR solve?

```
RAGFlow(user)> embed text 'what is rag' 'who are you' with 'embedding-3@test@zhipu-ai' dimension 16;
+-----------+-------+
| dimension | index |
+-----------+-------+
| 16        | 0     |
| 16        | 1     |
+-----------+-------+

RAGFlow(user)> rerank query 'what is rag' document 'rag is retrieval augment generation' 'rag need llm' 'famous rag project includes ragflow' with 'rerank@test@zhipu-ai' top 2;
+-------+-----------------+
| index | relevance_score |
+-------+-----------------+
| 0     | 1               |
| 2     | 0.99999976      |
+-------+-----------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 17:41:54 +08:00
Lynn
efe6d23d61 Fix: handle id as keyword (#14729)
### What problem does this PR solve?

Update mapping.json to treat id as a keyword.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:41:08 +08:00
chanx
8ac14b597f Fix: Some bugs (#14734)
### What problem does this PR solve?

Fix: Some bugs
- Error during batch modification of metadata in the Knowledge Base
- Manually configured metadata is not displayed in search settings

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 17:40:22 +08:00
akie
c11650bb4c Fix IDOR: Add permission checks to file ancestry endpoints (#14725)
Close #14292

## Issue

File ancestry endpoints return folder metadata without validating tenant
permissions, allowing any authenticated user to query arbitrary
`file_id` values across tenant boundaries.

## Affected Endpoints
- `GET /v1/file/parent_folder?file_id={file_id}`
- `GET /v1/file/all_parent_folder?file_id={file_id}`  
- `GET /api/v1/files/{id}/ancestors`

## Root Cause

These endpoints **skip the permission check** that other file operations
(Delete, Download, Move) perform.

## Expected Permission Check

All file operations should follow this 3-step validation:

- Check file.tenant_id
- Check if user_id belongs to this tenant (via user_tenant join table)
- Check KB permission type (team permission)


**Code reference:** This is implemented in `checkFileTeamPermission()`
and used by Delete/Download/Move, but **missing** from
GetParentFolder/GetAllParentFolders.

## Reproduction

```bash
# User B (tenant: BBB) accessing User A's file (tenant: AAA)
curl -H "Authorization: Bearer USER_B_TOKEN" \
  "http://localhost:9384/v1/file/parent_folder?file_id=AAA_FILE_123"

# Result: Returns User A's folder metadata 
# Expected: "No authorization." 
Fix
Pass userID from handler to service and call checkFileTeamPermission() — same as Download/Delete/Move handlers.

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-09 16:03:23 +08:00
writinwaters
6465753968 Docs: Added v0.25.2 release notes (#14727)
### What problem does this PR solve?

Added v0.25.2 release notes.

### Type of change

- [x] Documentation Update
2026-05-09 15:13:01 +08:00
Magicbook1108
f7e8c39dcc Fix: filter api in dataset document (#14728)
### What problem does this PR solve?

Fix: filter api in dataset document

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 14:45:40 +08:00
buua436
de2abe9ed8 Fix: tag parser id (#14724)
### What problem does this PR solve?
tag parser id
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 14:29:09 +08:00
Haruko386
ee0de58204 Go: implement provider: HuggingFace (#14722)
### What problem does this PR solve?

Implement `HuggingFace` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-09 13:36:03 +08:00
jony376
3b6eeabb09 Fix: private dataset authorization bypass in shared dataset access checks (#14645)
### Related issues
Closes #14644

### What problem does this PR solve?

This PR fixes an authorization bug where datasets marked with
`permission = me` could still be accessed by other members of the same
tenant through APIs that relied on `KnowledgebaseService.accessible()`
or `DocumentService.accessible()`.

Before this change, those shared access helpers only checked tenant
membership and did not enforce the dataset's permission mode. As a
result, a non-owner who knew a private `dataset_id` could still reach
downstream document and chunk operations even though the dataset was
intended to be owner-only.

This change updates the central access checks so that:

- dataset owners always retain access
- joined tenant members only get access when the dataset permission is
`TEAM`
- private datasets with `permission = me` remain inaccessible to
non-owners
- document-level access follows the same dataset permission rules

The PR also adds regression coverage for private-vs-team dataset access
behavior.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- Added
`test/unit_test/api/db/services/test_dataset_access_permissions.py`
- Attempted to run: `python -m pytest
test\\unit_test\\api\\db\\services\\test_dataset_access_permissions.py
-q`
- Local execution in this workspace is currently blocked during test
collection because the environment is missing the `strenum` dependency

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: jony376 <jony376@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
Co-authored-by: d 🔹 <liusway405@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Magicbook1108 <newyorkupperbay@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: sxxtony <166789813+sxxtony@users.noreply.github.com>
Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
Co-authored-by: Baki Burak Öğün <63836730+bakiburakogun@users.noreply.github.com>
Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
Co-authored-by: Panda Dev <56657208+pandadev66@users.noreply.github.com>
Co-authored-by: Haruko386 <tryeverypossible@163.com>
Co-authored-by: D2758695161 <13510221939@163.com>
Co-authored-by: Hunter <hunter@yitong.ai>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: buua436 <sz_buua@foxmail.com>
Co-authored-by: web-dev0521 <jasonpette1783@gmail.com>
Co-authored-by: Tim Wang <38489718+wanghualoong@users.noreply.github.com>
Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: qinling0210 <88864212+qinling0210@users.noreply.github.com>
Co-authored-by: dale053 <star05223@outlook.com>
2026-05-09 13:30:14 +08:00
Ricardo-M-L
1046042e01 fix(llm): replace mutable default gen_conf={} with None + defensive copy (#14566)
### What

19 methods across `rag/llm/chat_model.py` and `rag/llm/cv_model.py`
declare `gen_conf={}` (or `gen_conf: dict = {}`) as a parameter default
and then mutate `gen_conf` in place — typically `del
gen_conf["max_tokens"]`, `gen_conf["penalty_score"] = ...`, or
`gen_conf.pop(...)` as part of provider-specific normalization.

### The two bugs in this pattern

**1. Mutable default argument (Python footgun).** Python evaluates
default values **once** at function-definition time, so the single `{}`
dict is *shared* across every caller that doesn't pass `gen_conf`. The
first such call's mutations leak into the default seen by every
subsequent call.

```python
# Before
def chat_streamly(self, system, history, gen_conf={}, **kwargs):
    if "max_tokens" in gen_conf:
        del gen_conf["max_tokens"]   # mutates the SHARED default dict
    ...
```

After call N with `max_tokens` set, call N+1 that omits `gen_conf` no
longer sees `max_tokens` — even though the caller never touched it.

**2. Caller-dict pollution.** When the caller *does* pass a `gen_conf`
dict, the same in-place mutations modify the caller's dict. A reused
`gen_conf` (very common for chat-loop callers that build the config once
and pass it on every turn) silently loses `max_tokens`,
`presence_penalty`, etc. after the first round.

### The fix

In every affected method:

- Change `gen_conf={}` (or `gen_conf: dict = {}`) → `gen_conf=None`.
- Add `gen_conf = dict(gen_conf or {})` as the first statement of the
body so all subsequent mutations operate on a fresh local copy.

```python
# After
def chat_streamly(self, system, history, gen_conf=None, **kwargs):
    gen_conf = dict(gen_conf or {})
    if "max_tokens" in gen_conf:
        del gen_conf["max_tokens"]   # local copy — safe
    ...
```

This is byte-for-byte identical provider-side behavior for callers that
already pass a fresh `gen_conf` per call. The new `dict(...)` copy is
O(small constant) per call.

### Files changed

- `rag/llm/chat_model.py` — 17 methods
- `rag/llm/cv_model.py` — 2 methods

### Tests

Adds `test/unit_test/rag/llm/test_gen_conf_no_mutable_default.py` — an
`ast`-based regression guard that walks both modules and asserts no
parameter named `gen_conf` ever has a mutable literal (`{}` or `[]`) as
its default. The test caught **five additional `gen_conf: dict = {}`
sites** that an initial `gen_conf={}` text grep had missed (annotated
parameters with whitespace), and would fail again if the pattern is ever
reintroduced.

```
$ pytest test/unit_test/rag/llm/test_gen_conf_no_mutable_default.py -v
============================== 3 passed in 0.04s ===============================
```

`ruff check` passes on all touched files.

### Notes

- This PR is intentionally focused on **just** the `gen_conf` default +
copy fix. There's a related (but separate) `history.insert(0, ...)`
pattern in the same files that mutates the caller's history list in 12
places — left for a follow-up so this PR stays mechanical and easy to
review.

### Latest revision (`700bb54a7`) — addresses CodeRabbit review

- Type annotation: `gen_conf: dict = None` → `gen_conf: dict | None =
None` (5 occurrences in `chat_model.py`). The old annotation was a
static-checker mismatch since `None` isn't a `dict`.
- Regression test: the AST check accessed `default.keys` directly.
`ast.List` has no `.keys` attribute — a future `gen_conf=[]` would crash
with `AttributeError` instead of being caught. Use `getattr` for both
`.keys` (Dict) and `.elts` (List). Manually verified the updated check
correctly catches both `gen_conf={}` and `gen_conf=[]` while ignoring
`gen_conf=None` and non-empty literals.

---------

Co-authored-by: Ricardo <ricardo@example.com>
2026-05-09 13:11:44 +08:00
Wang Qi
42504fa18c Bugfix: keep document api backward compatible (#14726)
### What problem does this PR solve?

Bugfix: keep document api backward compatible

Fix 1: https://github.com/infiniflow/ragflow/issues/14634 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-09 13:03:09 +08:00
Yingfeng
3234a0ef35 Update README (#14723)
### Type of change

- [x] Documentation Update
2026-05-09 11:28:44 +08:00
VincentLambert
4f3711d37f fix: handle missing 'total' key causing KeyError in deep research retrieval (#13942)
## Summary

- When KB retrieval fails (e.g. ES `AssertionError` on empty
`index_names`), `kbinfos` falls back to a dict without a `total` key
- `_async_update_chunk_info` then iterates over `chunk_info.keys()`
(which includes `total`) and tries `kbinfos['total']`, raising a
`KeyError`
- This error surfaces when using Tavily web retrieval in a chat with no
knowledge base attached

## Changes

- Add `'total': 0` to all default `kbinfos` dicts in
`_retrieve_information`
- Add `setdefault('total', 0)` guard after successful KB retrieval to
handle cases where the retrieval result omits the key
- Accumulate `total` correctly in the merge branch of
`_async_update_chunk_info`

## Test plan

- [ ] Start a chat with Tavily configured and no knowledge base
- [ ] Verify no `KeyError: 'total'` is raised
- [ ] Verify Tavily results are returned correctly

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:57:51 +08:00
VincentLambert
870bc59365 Fix: Bedrock api_key overridden by existing-key fallback in add_llm (#14707)
## Summary

- Adding a Bedrock model from the frontend fails with `Fail to access
model(Bedrock/<model>).Expecting value: line 1 column 1 (char 0)`.
- The assembled Bedrock JSON credentials are silently replaced by `"x"`
before the connection test, causing `json.loads("x")` to raise a
`JSONDecodeError`.

## What problem does this PR solve?

Commit `050113482` introduced a fallback in `add_llm()` that reuses the
existing DB key when `req.get("api_key") is None`:

```python
if req.get("api_key") is None:
    api_key = existing_api_key if existing_api_key is not None else "x"
```

For Bedrock, credentials are sent as separate fields (`auth_mode`,
`bedrock_ak`, `bedrock_sk`, `bedrock_region`, `aws_role_arn`) — the
frontend does not send an `api_key` field. The function correctly
assembles the JSON key:

```python
api_key = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])
```

But since `req.get("api_key")` is `None`, the override immediately
replaces `api_key` with `"x"` (or a stale DB value). `LiteLLMBase` then
calls `json.loads("x")` for Bedrock auth → `JSONDecodeError`.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

## Changes

**`api/apps/llm_app.py`**

Write the assembled key into `req["api_key"]` so the `None` check
evaluates to `False` and the override is skipped — consistent with how
`Tencent Cloud` is already handled.

```python
# Before
api_key = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])

# After
req["api_key"] = apikey_json(["auth_mode", "bedrock_ak", "bedrock_sk", "bedrock_region", "aws_role_arn"])
api_key = req["api_key"]
```

## Test plan

- [ ] Configure a Bedrock provider in Model Providers with valid AWS
credentials
- [ ] Add a Bedrock chat model — verify no `Expecting value` error
- [ ] Update the same model — verify the existing key is reused
correctly when credentials fields are left empty

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:54:58 +08:00
Xing Hong
c428187350 Fix: validate kb_ids as UUIDs before SQL interpolation in use_sql (#14087)
### What problem does this PR solve?

The use_sql() function in dialog_service.py constructed SQL WHERE
clauses and Infinity table names by directly interpolating kb_id values
using Python f-strings, with no validation of the input values. A
malformed or maliciously crafted kb_id (introduced via a compromised
admin account or a separate injection vector) could alter the structure
of the generated SQL query, potentially leading to unauthorized data
access or data manipulation.

This PR adds strict UUID format validation for all kb_id values before
they are interpolated into any SQL string, causing requests with invalid
IDs to fail fast with a ValueError rather than executing a tampered
query.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-05-09 10:52:06 +08:00
VincentLambert
c44dc85143 Fix: IMAGE2TEXT→CHAT fallback with model_type normalization in tenant_model_service (#14704)
## Summary

- When a model is registered as `chat` in `tenant_llm` but has the
`IMAGE2TEXT` tag in `llm_factories.json`, requesting it as `image2text`
(e.g. PDF parser) fails with `Tenant Model with name <model> and type
image2text not found`.
- After resolution via the new fallback, the returned
`config_dict["model_type"]` was still `"chat"`, causing
`tenant_llm_service.model_instance()` to instantiate `ChatModel` instead
of `CvModel` — breaking `describe_with_prompt` at ingestion time.

## What problem does this PR solve?

RAGFlow already has a `CHAT→IMAGE2TEXT` fallback: when a chat model is
not found, it retries with `image2text`. The symmetric fallback
(`IMAGE2TEXT→CHAT`) was missing.

This matters for multimodal models declared as `model_type: "chat"` with
an `IMAGE2TEXT` tag in `llm_factories.json` (e.g. models added after
tenant creation, or providers where a single model serves both
purposes). The frontend PDF parser selector correctly surfaces these
models via the `IMAGE2TEXT` tag, but the backend fails to resolve them
at runtime.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

## Changes

**`api/db/joint_services/tenant_model_service.py`**

1. Add `IMAGE2TEXT→CHAT` fallback in
`get_model_config_by_type_and_name`: when an `image2text` model is not
found in `tenant_llm`, retry with `chat` — but only if the `llm` table
confirms `IMAGE2TEXT` capability via the `tags` field. This mirrors the
philosophy of the existing `CHAT→IMAGE2TEXT` fallback: substitution is
only allowed when the model has declared the required capability.

2. Normalize `config_dict["model_type"]` to `image2text` after the
fallback, so the caller (`model_instance`) correctly routes to `CvModel`
instead of `ChatModel`.

3. Extend the type validation guard to allow `(requested=image2text,
found=chat)` alongside the existing `(requested=chat, found=image2text)`
exception.

## Test plan

- [ ] Add a model with `model_type=chat` and `tags` containing
`IMAGE2TEXT` to a tenant
- [ ] Select it as PDF parser in a knowledge base
- [ ] Verify ingestion succeeds without `image2text not found` or
`describe_with_prompt` errors
- [ ] Verify the same model still works correctly in chat context

🤖 Generated with [Claude Code](https://claude.ai/claude-code)

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-09 10:40:58 +08:00
Octopus
653b00b94c fix(sync): scope document IDs per connector to prevent cross-KB collisions (#14378)
Fixes #14360

## Problem

When the same blob storage bucket is connected to multiple knowledge
bases (each through a different data source connector), the sync
pipeline hashes only the blob path
(`bucket_type:bucket_name:object_key`) to derive the document ID. Every
connector pointing at the same bucket therefore produces **identical
IDs** for the same object. The collision guard in
`FileService.upload_document` then fires for the second knowledge base:

```
Existing document id collision with another knowledge base; skipping update.
```

This makes it impossible to index the same bucket into more than one KB
simultaneously.

## Solution

Include `connector_id` in the hash input so that each connector produces
a distinct document ID even when the underlying blob path is identical:

```python
# Before
"id": hash128(doc.id),

# After
"id": hash128(f"{task['connector_id']}:{doc.id}"),
```

Because each KB connection uses its own connector (with a unique
`connector_id`), documents are now namespaced per connector and no
collision occurs.

**Note:** This is a breaking change for existing synced data sources.
After upgrading, a re-sync will create new documents with the updated ID
format. Old documents (indexed under the previous format) will remain in
the database but can be manually deleted or cleaned up via a re-sync
with reindex enabled.

## Testing

- Verified that the one-line change produces unique IDs for two
connectors pointing at the same S3 path.
- Existing unit test
`test_upload_document_skips_cross_kb_document_id_collision` continues to
pass — the collision guard in `FileService` is still valid for genuinely
colliding IDs from other sources.

---------

Co-authored-by: octo-patch <octo-patch@github.com>
2026-05-09 10:33:54 +08:00
writinwaters
d487a7f190 Docs: Added a guide on configuring SSL certificates (#14696)
### What problem does this PR solve?

### Type of change

- [x] Documentation Update
2026-05-09 10:08:14 +08:00
Jin Hai
b6abce50b1 Go: Admin list ingestion tasks (#14695)
### What problem does this PR solve?

```
RAGFlow(admin)> list tasks;
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
| chunk_count | digest           | document_id                      | duration    | from_page | id                               | priority | progress             | retry_count | task_type | to_page |
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
| 16          | 8a0016a0dc3cbdbb | f6aa38bb4ad111f1ba6338a74640adcc | 1511.156966 | 0         | f91e4f104ad111f1aaaf38a74640adcc | 0        | 1                    | 1           |           | 12      |
+-------------+------------------+----------------------------------+-------------+-----------+----------------------------------+----------+----------------------+-------------+-----------+---------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-09 10:03:23 +08:00
Jin Hai
5e96c5cae6 Fix go cli: search on datasets (#14692)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 20:25:14 +08:00
Joseff
2ad854c586 Go: implement Rerank in Aliyun driver (#14676)
### What problem does this PR solve?

The Aliyun Go driver has a stub `Rerank` method that always returns
`"Aliyun, Rerank not implemented"`. DashScope exposes an
OpenAI-compatible rerank endpoint (`compatible-mode/v1/rerank`) and
hosts dedicated bilingual rerankers (`gte-rerank-v2`, `gte-rerank`) that
are a natural pairing with the embedding models already in
`aliyun.json`. Without this, Aliyun users cannot use reranking within
RAGFlow.

Closes #14675

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 20:21:04 +08:00
Wang Qi
0552b1695a Fix UI search multiple datasets (#14689)
### What problem does this PR solve?

Fix UI search multiple datasets

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 20:20:09 +08:00
chanx
cacb7f2c18 Fix: Route error in dataset files page (#14691)
### What problem does this PR solve?

Fix: Route error in dataset  files page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 20:19:26 +08:00
Wang Qi
7d35e40c7b Refactor : Allow search multiple datasets (#14685)
### What problem does this PR solve?

Refactor : Allow search multiple datasets
1. support /datasets/search
2. get rid of /graph/search, use /graph

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-05-08 19:01:35 +08:00
dale053
26d70189b6 fix: enforce tenant-scoped authorization for chatbot SDK endpoints (#14592)
Closes #14590

## Self Checks

- [x] I have searched for existing issues [search for existing
issues](https://github.com/infiniflow/ragflow/issues), including closed
ones.
- [x] I confirm that I am using English to submit this report ([Language
Policy](https://github.com/infiniflow/ragflow/issues/5910)).
- [x] Non-english title submitions will be closed directly (
非英文标题的提交将会被直接关闭 ) ([Language
Policy](https://github.com/infiniflow/ragflow/issues/5910)).
- [x] Please do not modify this template :) and fill in all the required
fields.

## RAGFlow workspace code commit ID

`a1b2c3d4e5f67890123456789abcdef12345678`

## RAGFlow image version

`0.13.1`

## Other environment information

- Hardware parameters: N/A
- OS type: Linux 6.17.0-22-generic
- Others: API key authentication via `Authorization: Bearer <token>`

## Actual behavior

The chatbot API endpoints:

- `POST /chatbots/<dialog_id>/completions`
- `GET /chatbots/<dialog_id>/info`

validate only that the bearer token exists in `APIToken`, but do not
verify that `dialog_id` belongs to the same tenant as that token.

Current flow (simplified):

1. Route extracts bearer token and checks `APIToken.query(beta=token)`.
2. If token exists, request is accepted.
3. Downstream service resolves dialog globally by ID
(`DialogService.get_by_id(dialog_id)` in `conversation_service.py`).
4. No tenant ownership check is enforced for `dialog_id`.

Impact: Any user with a valid API key can attempt arbitrary `dialog_id`
values and access/invoke chatbots outside their own tenant boundary if
IDs are known/guessed/leaked.

Security classification:

- Vulnerability class: Broken Access Control (IDOR, OWASP Top 10 A01)
- Severity recommendation: Critical
- Exploit prerequisite: any valid API key + discoverable target
`dialog_id`

## Expected behavior

Requests to `/chatbots/<dialog_id>/completions` and
`/chatbots/<dialog_id>/info` must be authorized only when:

1. bearer token is valid, and
2. `dialog_id` belongs to the same `tenant_id` as the token.

Otherwise, reject with authorization failure (e.g., 403 or
404-equivalent policy).

## Steps to reproduce

1. Prepare two tenants:
   - Tenant A with API key `TOKEN_A`
   - Tenant B with chatbot `dialog_id = DIALOG_B`
2. Send request from Tenant A to Tenant B chatbot completion endpoint:

```bash
curl -X POST "https://<host>/chatbots/DIALOG_B/completions" \
  -H "Authorization: Bearer TOKEN_A" \
  -H "Content-Type: application/json" \
  -d '{"question":"hello","stream":false}'
```

3. Observe request is processed (or reaches dialog resolution) without
tenant ownership rejection.
4. Repeat against info endpoint:

```bash
curl -X GET "https://<host>/chatbots/DIALOG_B/info" \
  -H "Authorization: Bearer TOKEN_A"
```

5. Observe the same missing ownership enforcement.

## Additional information

Affected code paths:

- `api/apps/sdk/session.py`
  - `chatbot_completions(dialog_id)`
  - `chatbots_inputs(dialog_id)`
- `api/db/services/conversation_service.py`
  - `async_iframe_completion(...)` uses global dialog lookup

Suggested fix:

1. In both chatbot endpoints:
   - Resolve `tenant_id = objs[0].tenant_id` from validated token.
- Fetch dialog with tenant-scoped query
(`DialogService.query(id=dialog_id, tenant_id=tenant_id)`).
   - Reject if dialog is not found/owned by tenant.
2. Defense in depth:
- Require and enforce `tenant_id` in service-layer dialog resolution for
external flows.
- Avoid global `get_by_id(dialog_id)` where user-controlled dialog IDs
are reachable.
3. Add regression tests:
   - Positive: same-tenant token + dialog succeeds.
   - Negative: cross-tenant token + dialog fails for both endpoints.
2026-05-08 18:00:18 +08:00
Lynn
ada6d47880 Fix: move file check (#14681)
### What problem does this PR solve?

Restrict file move operations: prevent moving a folder to itself or to
one of its own subfolders.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 17:58:37 +08:00
qinling0210
4d6e8dffac Do not bypass threshold for rerank when metadata filter is enabled (#14684)
### What problem does this PR solve?

Do not bypass threshold for rerank when metadata filter is enabled

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 17:48:30 +08:00
web-dev0521
a32ebf32bd Fix: handle null document_metadata in kb_prompt to prevent citation crash (#14651) (#14666)
### What problem does this PR solve?

Fixes #14651.

`kb_prompt()` in `rag/prompts/generator.py` crashes with
`AttributeError: 'NoneType' object has no attribute 'items'` during
agent citation generation when a retrieved chunk carries
`document_metadata: null`.

**Root cause.** The crash happens at `rag/prompts/generator.py:132-133`:

```python
meta = ck.get("document_metadata", {})
for k, v in meta.items():
```

`dict.get(key, default)` only returns the default when the key is
*missing*. When the key is present with an explicit `None` value,
`.get()` returns `None`, and `.items()` crashes.

**How the chunk gets `None`.** It's a round-trip inside RAGFlow itself,
not bad input from retrieval:

1. The agent stores retrieved chunks via `agent/canvas.py:814`, which
routes them through `chunks_format()`.
2. `rag/prompts/generator.py:61` canonicalizes the field with
`chunk.get("document_metadata")` (no default), so chunks without
metadata become `{"document_metadata": None, ...}`.
3. `agent/component/agent_with_tools.py:314` feeds those canonicalized
chunks back into `kb_prompt()` for citation generation, and
`.get("document_metadata", {})` no longer protects us.

**Fix.** One-line change at `rag/prompts/generator.py:132`: use
`ck.get("document_metadata") or {}` so an explicit `None` is also
coerced to `{}`.

The line-61 `None` is intentionally part of the API/UI contract — the
frontend handles it via optional chaining
(`web/src/components/markdown-content/index.tsx:184`,
`web/src/pages/next-search/search-view.tsx:217`) — so the fix belongs at
the consumer, not the producer.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-08 16:54:33 +08:00
Jin Hai
ce2ec86b5e Go: fix CLI logout command (#14672)
### What problem does this PR solve?

```
RAGFlow(user)> logout;
SUCCESS
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 16:47:25 +08:00
Haruko386
94f82acd03 Fix(Go): prevent global state pollution in local model connection check (#14669)
### What problem does this PR solve?

1. **Fix Global State Pollution in Local Providers (Critical Bug):** -
Resolved a severe concurrency and architecture issue in
`model_service.go`. Previously, `ListSupportedModels` would permanently
overwrite the global provider singleton with a localized URL instance
(`driver.NewInstance`). This caused cross-request contamination in
multi-tenant environments.
- Fixed `CheckProviderConnection` for local models (LM Studio, vLLM,
Ollama). It now properly creates a localized driver copy and injects the
`base_url` before testing the connection, entirely eliminating the
false-positive `missing base URL` error without polluting the global
state.
2. **Implement `VolcEngine` Embeddings:** - Fully implemented the
`Encode` method for the `volcengine` provider, enabling text embedding
capabilities for VolcEngine models.
3. **Enhance Region Validation in `SiliconFlow`:** - Added a strict
empty string check (`*apiConfig.Region != ""`) alongside the existing
`nil` check when parsing regions. This ensures that if an empty string
is passed, the system safely falls back to the `"default"` region,
preventing malformed URL requests and `unsupported protocol scheme`
errors.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 15:54:27 +08:00
Jin Hai
ee5ae6f1a4 Go CLI: fix register user (#14665)
### What problem does this PR solve?

1. Update API URL
2. Add password encryption

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 15:53:06 +08:00
Lynn
69197d4a8f Fix: type of tenant_rerank_id (#14667)
### What problem does this PR solve?

Update the type of tenant_rerank_id in validation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 15:32:34 +08:00
Tim Wang
decb5dcb6f Fix: path-aware reset in canvas.run() to preserve cross-run outputs (#14600)
## Summary

- When an agent workflow has multiple `UserFillUp` pause points,
`canvas.run()` calls `reset(True)` on **all** components at the start of
each run. This clears outputs from components that completed in prior
runs, so downstream references like `{Agent:XXX@content}` resolve to
`None`.
- This fix only resets components on the **current execution path**
(`self.path`), preserving outputs from previously completed components.

## Problem

In a multi-step agent (e.g. draft email → user confirms → send email):

1. First `run()`: Agent drafts content, UserFillUp pauses for user input
→ Agent output is saved
2. Second `run()`: User submits input, but `reset(True)` clears **all**
components including the Agent that already completed
3. Email component references `{Agent:XXX@content}` → gets `None`
instead of the draft

This affects **all** agents that reference upstream component outputs
after a UserFillUp pause point.

## Fix

```python
# Before: reset ALL components
for k, cpn in self.components.items():
    self.components[k]["obj"].reset(True)

# After: only reset components on current execution path
path_set = set(self.path)
for k, cpn in self.components.items():
    if k in path_set:
        self.components[k]["obj"].reset(True)
```

`self.path` already tracks the current execution path. For agents
without UserFillUp (single run), `path` contains all components, so
behavior is unchanged.

## Test plan

- [x] Agent with single UserFillUp: outputs from prior components are
preserved after resume
- [x] Agent with multiple UserFillUp: each resume preserves all
previously completed outputs
- [x] Agent without UserFillUp: behavior unchanged (all components in
path, all reset)
- [x] Webhook-triggered agents: unaffected (path includes all components
on first run)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-08 15:10:15 +08:00
buua436
d843035c8b Fix: add compatibility route for document download under /v1 (#14663)
### What problem does this PR solve?

add compatibility route for document download under /v1

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 14:44:02 +08:00
Tim Wang
1bcb6deb6f Fix: collapsible thinking display and separate deep research retrieval tag (#14613)
## Summary

- **Collapsible thinking**: Replace `<section>` with `<details>` for
`<think>` content, so model thinking output is collapsed by default
(click to expand). Works for all models that output `<think>` tags
(Qwen3, DeepSeek, Gemini, Claude, etc.).
- **Fix double thinking tags**: When reasoning/deep research mode is
enabled in knowledge base chat, both the retrieval progress and model
thinking were wrapped in `<think>` tags, producing two "Thinking..."
blocks. Now retrieval progress uses a dedicated `<retrieving>` tag
rendered as a separate "Retrieving..." collapsible with a distinct green
accent.

### Before
- Thinking content displayed as flat gray-bordered `<section>`,
occupying significant screen space
- Deep research + model thinking both use `<think>` → two identical
"Thinking..." blocks

### After
- Thinking content collapsed by default in a `<details>` element, click
"Thinking..." to expand
- Deep research shows "Retrieving..." (green border), model thinking
shows "Thinking..." (gray border)

## Changes

**Backend (`api/db/services/dialog_service.py`)**
- Deep research callback: replace `start_to_think`/`end_to_think` marker
flags with direct `<retrieving>`/`</retrieving>` answer text

**Frontend**
- `web/src/utils/chat.ts`: `replaceThinkToSection()` now uses
`<details>` instead of `<section>`; add new
`replaceRetrievingToSection()`
- 4 tsx files: import and pipe `replaceRetrievingToSection`, whitelist
`details`, `summary`, `retrieving` in DOMPurify `ADD_TAGS`
- 4 less files: `section.think` → `details.think` with `<summary>`
styles; add `details.retrieving` with green accent; dark mode and RTL
variants

## Test plan
- [ ] Open a chat WITHOUT knowledge base, ask a question to a model with
thinking (e.g. Qwen3) → thinking content should be collapsed by default,
click "Thinking..." to expand
- [ ] Open a chat WITH knowledge base and reasoning enabled, ask a
question → "Retrieving..." (green) shows retrieval progress,
"Thinking..." (gray) shows model thinking, each independently
collapsible
- [ ] Verify dark mode renders correctly for both collapsible blocks
- [ ] Verify RTL layout renders correctly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: wanghualoong <wanghualoong@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-05-08 14:40:00 +08:00
web-dev0521
d51fb88573 Fix: enforce tenant authorization on document download endpoint (#14618) (#14625)
### What problem does this PR solve?

Closes #14618.

The `GET /v1/document/get/<doc_id>` endpoint in
`api/apps/document_app.py` was protected only by `@login_required` and
called `DocumentService.get_by_id(doc_id)` without verifying that the
document's knowledge base belonged to the requesting user's tenant. Any
authenticated user who knew (or guessed) a document ID could download
files belonging to any other tenant — a cross-tenant IDOR.

This PR adds a `DocumentService.accessible(doc_id, current_user.id)`
check before serving the file. The helper already exists and joins
`Document` → `Knowledgebase` → `UserTenant` to verify the requesting
user belongs to the tenant that owns the document's KB. The same pattern
is already used by `api/apps/restful_apis/document_api.py` and mirrors
the tenant scoping in the SDK route at `api/apps/sdk/doc.py`.

The check returns the existing `"Document not found!"` error for both
non-existent and inaccessible documents, so attackers cannot use the
response to enumerate valid doc IDs across tenants.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Other (please describe): Security fix (cross-tenant IDOR /
authorization bypass)
2026-05-08 14:24:03 +08:00
Panda Dev
a82ae4a991 Go: implement Encode (embeddings) in Aliyun driver (#14647)
### What problem does this PR solve?

The Aliyun Go driver shipped with a stub \`Encode\` method that returned
\`no such method\`, even though \`conf/models/aliyun.json\` already
wires the OpenAI-compatible embeddings URL suffix at
\`compatible-mode/v1/embeddings\`. The same config also did not list any
embedding models, so the picker had nothing to select.

So an Aliyun tenant who wanted to use Tongyi text-embedding-v3 or v4 in
the Go layer could not, even though the upstream endpoint is public and
uses the standard \`POST /v1/embeddings\` shape that the SiliconFlow and
ZhipuAI
drivers already support.

This PR fills the gap.

### What this PR includes

- \`conf/models/aliyun.json\`: add \`text-embedding-v4\` and
\`text-embedding-v3\` to the \`models\` array.
- \`internal/entity/models/aliyun.go\`: replace the \`Encode\` stub with
a real implementation. Adds a small local response type that matches the
OpenAI-compatible shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, validate the model name,
resolve the region with a default fallback, build the
  URL from \`BaseURL[region] + URLSuffix.Embedding\`.
- Send all input texts in one request as the \`input\` array, the same
OpenAI-compatible shape the SiliconFlow \`Encode\`
  uses.
- Parse \`data[*].embedding\` and copy each slice into a \`[][]float64\`
indexed by \`data[*].index\` so the output order matches the input order
even if the API returns items in a different order.
- Handle both \`float64\` and \`float32\` element types.
- Empty input returns \`[][]float64{}\` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.
- A final pass checks every input slot got a vector and returns a clear
error if any slot is still nil.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`AliyunModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Encode implementation.

Closes #14646

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 13:58:25 +08:00
Haruko386
d13a240dc0 Go: implement remaining interface for OpenRouter (#14657)
### What problem does this PR solve?

1. implement `rerank`, `embedding`, `balance`, `checkConnet` method for
`OpenRouter`
2. delete `chat` method in `internal/entity/models/volcengine.go`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-08 13:56:45 +08:00
Jin Hai
731c887ba0 Fix cli login (#14658)
### What problem does this PR solve?

Since API is updated, CLI login failed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 13:56:19 +08:00
jony376
6547751936 Fix: missing authorization checks in /files/link-to-datasets (#14649)
### Related issues
Closes #14648

### What problem does this PR solve?

This PR fixes an authorization flaw in `POST /files/link-to-datasets`.

Before this change, the endpoint only checked whether the supplied
`file_ids` and `kb_ids` existed. It did not verify whether the
authenticated user was actually allowed to access those files or target
datasets. As a result, an authenticated user who knew valid IDs could
relink another user's files to arbitrary datasets.

This was especially risky because the relinking flow is state-changing:
the background worker removes existing file-document mappings and then
recreates documents under the attacker-supplied dataset IDs.

This change makes the route enforce the same permission model already
used by nearby file and document operations:

- each resolved file must pass `check_file_team_permission(...)`
- each target dataset must pass `check_kb_team_permission(...)`
- authorization is enforced before scheduling background relinking work

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Testing

- Added regression coverage in
`test/testcases/test_web_api/test_file_app/test_file2document_routes_unit.py`
- Covered:
  - unauthorized file access is rejected
  - unauthorized dataset access is rejected
- existing success path still returns immediately after scheduling
background work
- Attempted to run:
- `python -m pytest
test\\testcases\\test_web_api\\test_file_app\\test_file2document_routes_unit.py
-q`
- Local execution in this workspace is currently blocked by missing test
dependencies during bootstrap, including `ragflow_sdk`

---------

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-08 13:49:23 +08:00
buua436
f703169117 Refa: migrate document preview/download to RESTful API (#14633)
### What problem does this PR solve?

migrate document preview/download to RESTful API

### Type of change
- [x] Refactoring
2026-05-08 13:26:13 +08:00
Lynn
412fae7ac2 Fix: display error (#14654)
### What problem does this PR solve?

Use right key in error text.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-08 13:11:59 +08:00
Panda Dev
d8d49df35e Go: implement Rerank in Gitee AI driver (#14656)
### What problem does this PR solve?

The Gitee AI Go driver shipped with a stub \`Rerank\` method that
returned \`Rerank not implemented\`, even though
\`conf/models/gitee.json\` already wires the rerank URL suffix at
\`\"rerank\": \"rerank\"\`. The same config did not list any
rerank model, so the picker had nothing to select.

So a Gitee tenant could not use BAAI/bge-reranker-v2-m3 as a reranker
through the Go layer today, even though the
infrastructure was one config entry and one method body away.

### What this PR includes

- \`conf/models/gitee.json\`: add \`BAAI/bge-reranker-v2-m3\` to the
\`models\` array.
- \`internal/entity/models/gitee.go\`: replace the \`Rerank\` stub with
a real implementation. Adds two small local types
that match the OpenAI-compatible \`/rerank\` shape already used by the
SiliconFlow and ZhipuAI drivers.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, validate the model name,
resolve the region with a default fallback, build the
  URL from \`BaseURL[region] + URLSuffix.Rerank\`.
- Use a per-call \`context.WithTimeout(30s)\` and
\`http.NewRequestWithContext\`, matching the pattern the
  recently merged Aliyun Encode and the OpenAI driver already use.
- Send \`{model, query, documents, top_n, return_documents:false}\` in
the body.
- Parse \`results[*].relevance_score\` and copy each score into the
output slice indexed by \`results[*].index\`, so the
output order matches the input order even if the API returns items in a
different order.
- Empty input returns \`[]float64{}\` with no HTTP call.
- An out-of-range result index returns a clear error rather than
silently skipping the entry.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`GiteeModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing SiliconFlow Rerank and the recently
merged ZhipuAI Rerank (#14608).

Closes #14655
2026-05-08 13:08:22 +08:00
D2758695161
f063e03a24 fix: add bucket prefix to Azure Blob SPN and SAS storage operations (#14347)
## Summary

Fixes file collision between different datasets when using Azure Blob
storage (SPN or SAS authentication).

## Bug

azure_spn_conn.py and zure_sas_conn.py ignored the ucket parameter
entirely, storing all files flat with just the filename. This caused
files with the same name from different datasets (knowledge bases) to
overwrite each other.

## Fix

Prepend bucket/ as a path prefix in all methods (put, 
m, get, obj_exist, get_presigned_url, health) to match the behavior of
MinIO and S3 implementations.

## Changes

- **rag/utils/azure_spn_conn.py**: Added {bucket}/ prefix to file paths
in all operations
- **rag/utils/azure_sas_conn.py**: Same fix applied for consistency
(also noted in the original issue)

## Testing

Manual verification: files from different datasets now stored under
distinct bucket/ prefixes, preventing collisions.

Fixes #14159

Co-authored-by: Hunter <hunter@yitong.ai>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-08 12:06:28 +08:00
Panda Dev
c7ddc8c039 fix(go): implement ListModels and CheckConnection in NVIDIA driver (#14636)
### What problem does this PR solve?

The NVIDIA Go driver added in #14623 has a real chat path, but
\`ListModels\` and \`CheckConnection\` are stubs that always return \`no
such method\`. So:

- The model picker cannot auto-populate available NVIDIA NIM model ids.
Users have to type the full id by hand (e.g.
  \`abacusai/dracarys-llama-3.1-70b-instruct\`).
- The "Check connection" button always fails for NVIDIA, even when the
base URL is reachable and the API key is accepted.

NVIDIA NIM is OpenAI-compatible. \`/v1/models\` works with the same
Bearer token used for chat. The
\`conf/models/nvidia.json\` file already wires the \`models\`
url_suffix, so no config change is needed.

### What this PR includes

- \`internal/entity/models/nvidia.go\`:
  - \`ListModels\` now calls
    \`GET ${BaseURL}/${URLSuffix.Models}\`, parses
    \`response.data[*].id\`, and returns the list. Same shape
    as the moonshot, xai, and openai drivers.
  - \`CheckConnection\` now calls \`ListModels\` and returns its
    error. Same pattern xai, moonshot, deepseek, aliyun, and
    gitee already use.

\`Balance\`, \`Encode\`, and \`Rerank\` are still stubs in this PR and
can be added in follow-ups.

No JSON change. No factory change. No interface change.

### How the implementation works

- Region resolution falls back to \`default\` when the supplied region
is unknown, so a stray region value does not break a valid request.
- The Authorization header is only set when \`apiConfig\` and \`ApiKey\`
are non-nil and non-empty. This avoids a nil-pointer dereference and
lets self-hosted NIM deployments without a key still work.
- Non-200 responses propagate the upstream status line and body so the
user sees a real error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`NvidiaModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing xai, moonshot, deepseek, aliyun,
gitee, and openai drivers.

Closes #14635
2026-05-08 12:04:28 +08:00
Panda Dev
e729eced45 Go: implement Balance in DeepSeek driver (#14632)
Closes #14631 

### What problem does this PR solve?

The DeepSeek Go driver shipped with a stub \`Balance\` method that
returned \`no such method\`, even though DeepSeek exposes a public \`GET
/user/balance\` endpoint that works with the same Bearer token used for
chat.

So the "Balance" panel in the model provider UI always shows an error
for DeepSeek tenants, while it already works for Moonshot and Gitee.
This PR fills the gap.

### What this PR includes

- \`conf/models/deepseek.json\`: add \`\"balance\": \"user/balance\"\`
under \`url_suffix\` so the driver can build the URL from config the
same way the other endpoints do.
- \`internal/entity/models/deepseek.go\`: replace the \`Balance\` stub
with a real implementation. Adds a small local response type
\`deepseekBalanceResponse\` that matches the upstream shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, resolve the region (with a
\`default\` fallback), and build the URL from \`BaseURL[region] +
URLSuffix.Balance\`.
- GET the URL with \`Authorization: Bearer <api_key>\`.
- Parse the upstream response:

  \`\`\`json
  {
    \"is_available\": true,
    \"balance_infos\": [
      {\"currency\": \"USD\", \"total_balance\": \"10.00\", ...},
      {\"currency\": \"CNY\", \"total_balance\": \"70.00\", ...}
    ]
  }
  \`\`\`

\`total_balance\` is a string in the upstream API, so the driver parses
it with \`strconv.ParseFloat\`.
- Return the first balance entry as \`{\"balance\": <float>,
\"currency\": <string>}\`, the same shape the Moonshot driver returns.
The UI can render it with no provider-specific code.

### Edge cases

- Missing or empty API key returns a clear local error before any HTTP
call.
- Empty \`balance_infos\` returns a clear \"no balance info in
response\" error rather than a zero-value silent success.
- Non-numeric \`total_balance\` returns a clear parse error.
- Non-200 responses propagate the upstream status line and body so the
user can see why the call failed.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
(the go.mod minimum) returns exit 0.
- The full method set on \`DeepSeekModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing Moonshot and Gitee Balance
implementations.
2026-05-08 12:03:39 +08:00
Haruko386
a377512110 Go: implement provider: OpenRouter (#14652)
### What problem does this PR solve?

1. **Implement `OpenRouter` Provider:** Fully support OpenRouter AI
models (e.g., `gemma`, `minimax`). Includes robust handling of
Server-Sent Events (SSE) streams, error event interception, and proper
parsing of both `reasoning_content` and standard `content`.
2. **Fix BaseURL Resolution Bug:** Fixed a critical edge case in region
configuration parsing. Added a strict empty string check
(`*apiConfig.Region != ""`) alongside the `nil` check. This ensures that
if the UI passes an empty string, the system correctly falls back to the
`"default"` region, preventing `unsupported protocol scheme ""` errors
during HTTP requests.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-08 12:02:37 +08:00
Panda Dev
a86e0ca0ca Go: implement Balance in SiliconFlow driver (#14643)
### What problem does this PR solve?

The SiliconFlow Go driver shipped with a stub \`Balance\` method that
returned \`no such method\`, even though SiliconFlow exposes a public
\`GET /v1/user/info\` endpoint that returns the account balance per
currency.

So the "Balance" panel in the model provider UI always shows an error
for SiliconFlow tenants, while it already works for
Moonshot and Gitee. This PR fills the gap.

### What this PR includes

- \`conf/models/siliconflow.json\`: add \`\"balance\": \"user/info\"\`
under \`url_suffix\` so the driver builds the URL from config.
- \`internal/entity/models/siliconflow.go\`: replace the \`Balance\`
stub with a real implementation. Adds a small local response type that
matches the upstream shape.

No factory change. No interface change.

### How the driver works

- Validate \`apiConfig\` and the API key, resolve the region with a
default fallback, and build the URL from \`BaseURL[region] +
URLSuffix.Balance\`.
- GET the URL with \`Authorization: Bearer <api_key>\`.
- Parse the upstream response. SiliconFlow returns balance fields as
strings, so the driver parses them with \`strconv.ParseFloat\`. It
prefers \`totalBalance\` over \`balance\` when both are present.
- Return \`{\"balance\": <float>, \"currency\": \"CNY\"}\`, the same
shape the Moonshot driver returns. The UI can render it
  with no provider-specific code.

### Edge cases

- Missing or empty API key returns a clear local error before any HTTP
call.
- An unknown region falls back to the default base URL.
- Empty \`balance\` and \`totalBalance\` returns a clear "no balance
info in response" error rather than a zero-value silent success.
- Non-numeric balance string returns a clear parse error.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- \`go build ./internal/entity/models/...\` in a clean go 1.25 image
returns exit 0.
- The full method set on \`SiliconflowModel\` still matches the
\`ModelDriver\` interface.
- Pattern parity with the existing Moonshot and Gitee Balance
implementations.

Closes #14642
2026-05-08 12:01:10 +08:00
Panda Dev
2fd8cdc3cc fix(go): wire CheckConnection to ListModels in ollama, lm-studio, and vllm (#14614)
### What problem does this PR solve?

Three Go drivers had `CheckConnection` returning a hardcoded `no such
method` error, even though each one already has a working `ListModels`
that hits the configured base URL with the configured API key. So the
"Check connection" button in the model provider UI always failed for
these three providers, even when the underlying setup was fine.

Affected drivers:

- `internal/entity/models/ollama.go`
- `internal/entity/models/lmstudio.go`
- `internal/entity/models/vllm.go`

This is a real user-facing gap because Ollama and LM Studio are two of
the most popular local LLM runners, and vLLM is widely used for
self-hosted deployments.

### What this PR includes

For each of the three drivers, replace the stub with a small
implementation that calls `ListModels` and returns its error:

```go
func (o *OllamaModel) CheckConnection(apiConfig *APIConfig) error {
    _, err := o.ListModels(apiConfig)
    return err
}
```

This is the exact pattern that xai, moonshot, deepseek, aliyun, and
gitee already use for the same method.

No JSON change. No factory change. No interface change.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- `go build ./internal/entity/models/...` in a clean go 1.25 image (the
go.mod minimum) returns exit 0.
- The full ModelDriver interface still resolves on each driver
(NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender, Encode,
Rerank, ListModels, Balance, CheckConnection).
- Pattern parity with the existing xai, moonshot, deepseek, aliyun, and
gitee CheckConnection methods.

Closes #14609
2026-05-08 12:00:10 +08:00
Baki Burak Öğün
5d28bb0701 feat: update Turkish localization strings (#14650)
## Summary
Update the Turkish locale file to match the latest English locale keys.

## Changes
- Add missing Turkish translations for the new Skills and Skill Search
sections
- Add newly introduced common, header, dataset, settings, and agent
workflow strings
- Align renamed flow keys such as file format options and list
operations with the English source
- Add empty-state strings for skill spaces

## Validation
- Compared web/src/locales/en.ts and web/src/locales/tr.ts: 0 missing
keys, 0 extra keys
- Checked jsonjoy-builder locale: Turkish is already complete
- Checked translated README variants: no new Turkish-specific
documentation gap found
- VS Code diagnostics: no errors in web/src/locales/tr.ts

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
2026-05-08 09:56:04 +08:00
sxxtony
59c35100c5 Perf: push metadata filters down to Elasticsearch (#14576)
### What problem does this PR solve?

Fixes #14412.

`common.metadata_utils.meta_filter` evaluates user-defined metadata
conditions in Python after `DocMetadataService.get_flatted_meta_by_kbs`
loads the entire `meta_fields` table into memory. Past a few thousand
documents per knowledge base this becomes a memory bottleneck and a
wasted ES round-trip — every filter request currently fetches up to
10000 metadata rows even when the resulting `doc_ids` list is tiny.

This PR adds an ES push-down path that translates the same filter
language into a `bool` query and returns just the matching document IDs.

**Changes**

- `common/metadata_es_filter.py` *(new)*: pure-Python translator from
the RAGflow filter list to ES DSL. Covers every operator the in-memory
path supports (`=`, `≠`, `>`, `<`, `≥`, `≤`, `in`, `not in`, `contains`,
`not contains`, `start with`, `end with`, `empty`, `not empty`) with
`case_insensitive: true` on `prefix` and `wildcard` for parity with the
existing lower-cased Python comparisons. User wildcard metacharacters
are escaped before being injected into `wildcard` patterns. Negative
operators (`≠`, `not in`, `not contains`, ranges) are wrapped with an
`exists` guard so they do not accidentally match documents missing the
key, matching the legacy `if k not in metas` behaviour.
- `api/db/services/doc_metadata_service.py`: new
`DocMetadataService.filter_doc_ids_by_meta_pushdown(kb_ids, filters,
logic)` that returns the doc IDs ES matched, or `None` to signal the
caller should fall back to the in-memory path. Returns `None` when the
active doc store is Infinity (`meta_fields` is a JSON column, not a
dotted-object mapping), when any filter cannot be expressed in DSL
(`UnsupportedMetaFilter`), or when the ES request or metadata index
lookup errors.
- `common/metadata_utils.py`: `apply_meta_data_filter` accepts an
optional `kb_ids` argument. When supplied, conditions go through
push-down first via a new `_try_meta_pushdown` helper; on `None` the
function falls back to the original `meta_filter` call. Default
behaviour is unchanged for callers that don't pass `kb_ids`.
- Updated all four callers (`agent/tools/retrieval.py`,
`api/db/services/dialog_service.py` ×2,
`api/apps/services/dataset_api_service.py`, `api/apps/sdk/session.py`)
to forward `kb_ids` so the push-down path is exercised in production.
- `test/unit_test/common/test_metadata_es_filter.py` *(new)*: 35 unit
tests covering every operator's DSL shape, value coercion
(`ast.literal_eval`, lowercasing, ISO-date pass-through), wildcard
escaping, OR-logic wrapping that protects negative clauses, and the
doc-ID extractor.

**Behaviour preserved**

- The in-memory `meta_filter` is untouched and still services every
fallback case (Infinity backend, unknown operators, ES outages).
- The eligibility / credibility / issue-multiplier semantics described
in the LLM-driven `auto` and `semi_auto` modes still hand the LLM the
full in-memory `metas` dict to choose conditions from. Only the
*evaluation* of those generated conditions is pushed down.
- Existing tests in
`test/unit_test/common/test_metadata_filter_operators.py` continue to
pass (14/14).

**Test plan**

- `pytest test/unit_test/common/test_metadata_es_filter.py` — 35 passed.
- `pytest test/unit_test/common/test_metadata_filter_operators.py` — 14
passed.
- `ruff check` clean on every modified file.
- Reviewer please validate the ES query shapes against a live cluster —
particularly `case_insensitive` on `wildcard` and `prefix` (requires ES
7.10+) and the `exists` + `must_not` pairing for `≠`.

**Notes**

- The first cut caps each push-down request at 10000 results, matching
the existing `get_flatted_meta_by_kbs` limit, and logs a warning when
the cap is hit. A `search_after` follow-up would let us drop the cap
entirely once the push-down path is validated.
- Operator parity with the in-memory path is exact for the canonical
unicode operators (`≥`, `≤`, `≠`) used internally; the ASCII aliases
(`>=`, `<=`, `!=`) are normalised by `convert_conditions` before they
reach the translator.

### Type of change

- [x] Performance Improvement

---------

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
2026-05-07 21:23:43 +08:00
chanx
805a2daac2 Fix: Change route name (#14639)
### What problem does this PR solve?

Fix: Change route name

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 21:23:29 +08:00
Magicbook1108
c29335cbff Feat: support local provider for code exec component & remove some outdated models (#14637)
### What problem does this PR solve?

Feat: support local provider for code exec component & remove some
outdated models

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 21:23:13 +08:00
d 🔹
057806d7f1 fix: prepend bucket prefix to Azure SPN and SAS storage paths (#14185)
## Summary

Fixes #14159 — files from different datasets can overwrite each other in
Azure Blob storage.

## Problem

Both `azure_spn_conn.py` and `azure_sas_conn.py` ignore the `bucket`
parameter in all storage operations (`put`, `get`, `rm`, `obj_exist`,
`get_presigned_url`). Files are stored flat using only the filename, so
two datasets containing a file with the same name will overwrite each
other.

The MinIO and S3 implementations correctly use the bucket (typically the
knowledge base ID) as a path prefix to create logical folder isolation:
- MinIO: uses `use_prefix_path` decorator → `{orig_bucket}/{fnm}`
- S3: uses `use_prefix_path` decorator → `{prefix_path}/{bucket}/{fnm}`

## Fix

Prepend `{bucket}/` to the file path in all 5 operations across both
Azure connector files:

| File | Methods fixed |
|------|---------------|
| `azure_spn_conn.py` | `put`, `get`, `rm`, `obj_exist`,
`get_presigned_url` |
| `azure_sas_conn.py` | `put`, `get`, `rm`, `obj_exist`,
`get_presigned_url` |

This matches the existing convention where `bucket` is the knowledge
base ID used as a directory prefix.

## ⚠️ Migration Note

Existing Azure SPN/SAS deployments have files stored without the bucket
prefix. After this fix, new files will be stored under
`{bucket}/{filename}` while existing files remain at `{filename}`. A
one-time migration script or manual file move may be needed for existing
deployments. New deployments are unaffected.

## Testing

- Verified the fix is consistent across all 5 methods in both files
- The `health()` method is intentionally left unchanged as it uses a
hardcoded test filename without bucket semantics

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 20:48:32 +08:00
Panda Dev
bb10b83e61 Go: implement Rerank in ZhipuAI driver (#14608)
### What problem does this PR solve?

The ZhipuAI Go driver had a stub Rerank method that returned "not
implemented", even though conf/models/zhipu-ai.json already ships
glm-rerank as a rerank model and the rerank URL suffix is already wired
in url_suffix:

```json
"url_suffix": {
  ...
  "rerank": "rerank"
},
"models": [
  {"name": "glm-rerank", "model_types": ["rerank"]},
  ...
]
```

So the config was ready but the driver was not. A tenant who picked
glm-rerank in the Go layer could not actually run a rerank call. This PR
fills the gap so the listed model works end to end.

### What this PR includes

- `internal/entity/models/zhipu-ai.go`: real implementation of
`ZhipuAIModel.Rerank`, plus two small local types (`zhipuRerankRequest`,
`zhipuRerankResponse`) that mirror the standard OpenAI-compatible rerank
shape used by SiliconFlow.

No factory change. No JSON change. No interface change.

### How the driver works

- POST to `${BaseURL}/${URLSuffix.Rerank}` (resolves to
`https://open.bigmodel.cn/api/paas/v4/rerank` with the default config),
reusing the existing httpClient on the driver.
- Validate apiConfig and the API key, validate the model name, and
resolve the region. Return a clear local error before any HTTP call when
something is missing.
- Send `{model, query, documents, top_n, return_documents: false}` in
the body, the same shape the SiliconFlow driver already uses.
- Walk `results[*].relevance_score` and copy each score into the output
slice indexed by `results[*].index`, so the output order matches the
input order even if the API returns results in a different order.
- Empty `texts` input returns an empty `[]float64` with no HTTP call.
- Non-200 responses propagate the upstream status line and body.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- `go build ./internal/entity/models/...` in a clean go 1.25 image (the
go.mod minimum) returns exit 0.
- The full method set on `ZhipuAIModel` still matches the `ModelDriver`
interface (NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender,
Encode, ListModels, Balance, CheckConnection, Rerank).
- Pattern parity with the existing SiliconFlow Rerank implementation
(`internal/entity/models/siliconflow.go`).

Closes #14607
2026-05-07 17:56:30 +08:00
Jack Storment
59bb184e63 feat(moodle): support deleted-file sync (#14548)
Fixes #14551 

### What problem does this PR solve?

The Moodle connector did not let the sync runner clean up indexed
documents that were deleted from the source. Other connectors such as
dropbox, seafile, webdav, and rss already do this through a slim
snapshot pass. This PR adds the same support for Moodle.

When `sync_deleted_files` is on, the runner now asks the Moodle
connector for a lightweight list of every module id that could be
indexed. The runner then compares this list with the index and removes
any indexed document whose id is not in the list.

The slim pass does not download files. It only goes through courses and
modules and yields ids. The id format matches the ids that the loader
produces, so the match is exact.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Notes

- `MoodleConnector` now also implements `SlimConnectorWithPermSync`.
- New `retrieve_all_slim_docs_perm_sync` yields slim docs with the same
ids the loader uses (`moodle_resource_<id>`, `moodle_forum_<id>`,
`moodle_page_<id>`, `moodle_book_<id>`, `moodle_assign_<id>`,
`moodle_quiz_<id>`).
- The `Moodle` sync class now returns `(document_generator, file_list)`
so the runner can do the cleanup. If the slim snapshot fails,
`file_list` is set back to `None` and the run continues without cleanup.
- The web data source map exposes `syncDeletedFiles` for Moodle so the
option shows up in the UI.

### How was this tested?

- `ruff check` passes on the changed Python files.
- Manual review of the produced slim ids against the ids the loader
builds in `_process_resource`, `_process_forum`, `_process_page`,
`_process_book`, and `_process_activity`.
- Behavior parity with the merged dropbox (#14476), seafile (#14499),
webdav (#14491), and rss (#14493) PRs.
2026-05-07 17:44:46 +08:00
Jin Hai
94324afee9 Go: fix auth issue in hybrid mode (#14611)
### What problem does this PR solve?

Since secret key get and set logic is updated, the go server also need
to update.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 17:14:22 +08:00
Octopus
5c9124c3ef fix: prepend bucket prefix in Azure Blob (SAS/SPN) to prevent cross-dataset file overwrites (#14174)
Fixes #14159

## Problem

The `put()`, `get()`, `rm()`, and `obj_exist()` methods in both
`azure_spn_conn.py` and `azure_sas_conn.py` ignore the `bucket`
parameter entirely, storing all files flat using only the filename. This
causes files from different datasets to overwrite each other when they
share the same filename.

By contrast, the MinIO and S3 implementations correctly use the bucket
(typically the knowledge base ID) as a path prefix, creating logical
folder isolation like `{kb_id}/{filename}`.

## Solution

Prepend the `bucket` parameter as a path prefix to all file operations
in both Azure storage implementations:

- `azure_spn_conn.py`: `create_file`, `delete_file`, `get_file_client`
now use `f"{bucket}/{fnm}"`
- `azure_sas_conn.py`: `upload_blob`, `delete_blob`, `download_blob`,
`get_blob_client` now use `f"{bucket}/{fnm}"`

This matches the behavior of all other storage backends (MinIO, S3) and
prevents filename collisions across knowledge bases.

## Testing

- Verified the fix aligns with how MinIO/S3 connectors handle the bucket
parameter
- The `health()` method is left unchanged as it uses a fixed test path
for connectivity checks only

Co-authored-by: octo-patch <octo-patch@github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 17:13:43 +08:00
buua436
0501134820 Fix: support tool call config (#14616)
### What problem does this PR solve?
support tool call config

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:54:57 +08:00
buua436
5b162a0c46 Fix: preserve doc generator download metadata in message (#14626)
### What problem does this PR solve?

preserve doc generator download metadata

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:48:36 +08:00
Wang Qi
c50028b1f3 Fix team member cannot edit agent (#14612)
### What problem does this PR solve?

Follow on PR: https://github.com/infiniflow/ragflow/pull/14602
to fix: team member cannot edit agent.
new behavior: beside delete, everything is allowed for team member.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-07 15:09:13 +08:00
Wang Qi
1d114f034b Allow more task logs for #14617 (#14624)
### What problem does this PR solve?

Allow more task logs for #14617

### Type of change

- [x] Refactoring
2026-05-07 15:03:08 +08:00
Haruko386
078ea3bf4a Go: implement provider: Nvidia (#14623)
### What problem does this PR solve?

1. **Implement `Nvidia` Provider:** Fully support NVIDIA NIM APIs with
robust parameter handling (including the `thinking` parameter) and safe
URL merging in `NewInstance`.
2. **Fix Misleading CLI Errors:** Corrected a bug in `common_command.go`
where failed chat requests inaccurately reported `failed to list
instance models`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 14:17:57 +08:00
Magicbook1108
911671cef0 Feat: enable sync deleted files for RDBMS & fix remove last file issue (#14615)
### What problem does this PR solve?

Feat: enable sync deleted files for RDBMS & fix remove last file issue

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 13:31:05 +08:00
Panda Dev
b8b741555f Go: implement provider: OpenAI (#14605)
### What problem does this PR solve?

Add a Go driver for OpenAI (GPT models).

The config file conf/models/openai.json has been in the repo for a while
with the full GPT-5 model list, but
internal/entity/models/factory.go had no case for "openai". So any
tenant that configured OpenAI as a model provider in the Go layer fell
through to the default branch and got the dummy driver. Chat, list
models, and check connection all returned dummy responses instead of
reaching the API.

OpenAI is the most commonly requested provider and the JSON config
already ships with the repo, so this gap is high impact even though the
JSON has been there for some time.

### What this PR includes

- New file internal/entity/models/openai.go with an OpenAIModel that
implements the ModelDriver interface.
- factory.go: route the "openai" provider name to NewOpenAIModel.
- conf/models/openai.json: add "models": "models" under url_suffix so
ListModels can hit /v1/models with no hardcoded fallback.

### How the driver works

- OpenAI exposes the canonical OpenAI-compatible API at
https://api.openai.com/v1.
- ChatWithMessages and ChatStreamlyWithSender post to /chat/completions
in the same shape the moonshot, vllm, and xai drivers use.
- ListModels and CheckConnection call /models to list available ids and
confirm the API key works.
- reasoning_content is passed through for the o-series and other
reasoning models, in both the non-stream and stream paths.
- Encode (embeddings) is left as "not implemented" for now, the same way
the other recent provider drivers do it. Rerank and Balance are not part
of OpenAI's public API surface in this layer and return a clear "not
implemented" or "no such method" error.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- go build ./internal/entity/models/... in a clean go 1.25 image (the
go.mod minimum) returns exit 0 with no errors.
- Method set of OpenAIModel matches the ModelDriver interface:
NewInstance, Name, ChatWithMessages, ChatStreamlyWithSender, Encode,
Rerank, ListModels, Balance, CheckConnection.
- Pattern parity with the merged moonshot (#14433), volcengine (#14460),
minimax (#14478), vllm (#14532), xai (#14550), and lm-studio (#14586)
PRs.

Closes #14604
2026-05-07 13:09:51 +08:00
Zhichang Yu
86fe78c73f feat(llm): add MiniMax GroupId header support (#14610)
## Summary
- Add MiniMax provider GroupId query parameter support in `LiteLLMBase`
- Extract `group_id` from key configuration in `__init__`
- Append `GroupId` as query parameter to `api_base` in
`_construct_complete_args`

## Why this change is needed

MiniMax provides an OpenAI-compatible API endpoint
(`/v1/chat/completions`), but `GroupId` is a MiniMax-specific account
identifier required for billing and rate limiting - it is not part of
the OpenAI standard.

Looking at LiteLLM's `MinimaxChatConfig`:
- `get_complete_url()` only constructs the base URL (e.g.,
`https://api.minimaxi.com/v1/chat/completions`)
- LiteLLM does **not** automatically inject `GroupId` into requests
- This must be handled by the caller (ragflow's chat_model.py)

The implementation appends `GroupId` as a query parameter to `api_base`:
```python
api_base = completion_args.get("api_base", self.base_url)
separator = "&" if "?" in api_base else "?"
completion_args["api_base"] = f"{api_base}{separator}GroupId={self.group_id}"
```

This matches MiniMax's official API format (as documented by
LlamaFactory):
```bash
curl --location 'https://api.minimaxi.chat/v1/text/chatcompletion?GroupId=你的GroupId' \
  --header 'Authorization: Bearer 你的API_Key'
```

## Test plan
- [ ] Verify MiniMax API calls work with GroupId query parameter
- [ ] Verify backward compatibility for other providers

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-07 11:54:49 +08:00
qinling0210
12f80f170c Bump to infinity v0.7.0-dev6 (#14606)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev6

(uv lock --upgrade-package infinity-sdk)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-07 10:51:17 +08:00
Stephen Hu
53a4edfded refactor: use warp to improve canvas access check logic (#14587)
### What problem does this PR solve?

use warp to improve canvas access check logic

### Type of change

- [x] Refactoring
2026-05-07 10:46:43 +08:00
Jin Hai
1d0519d025 Fix secret key inconsistency cross the RAGFlow servers (#14591)
### What problem does this PR solve?

A and B, two API servers and a REDIS server.
If A and REDIS restart, B will hold the obsolete secret key and will
lead to error.

TODO:
app.config['SECRET_KEY'] and app.secret_key still hold obsolete secret
key.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-07 10:10:02 +08:00
Wang Qi
15dcdd7b5b Revert "Fix agent permission issue" (#14602)
Reverts infiniflow/ragflow#14597
2026-05-06 20:52:54 +08:00
buua436
3e396c0a72 Fix: add base64 to doc generator output (#14599)
### What problem does this PR solve?
add base64 to doc generator output
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:33:08 +08:00
buua436
faae91d34f Fix: support non-stream runtime agent completion (#14596)
### What problem does this PR solve?

support non-stream runtime agent completion

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:29:15 +08:00
Wang Qi
67e1de50ab Fix agent permission issue (#14597)
### What problem does this PR solve?

Fix agent permission issue.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 20:17:36 +08:00
Wang Qi
04c5f1b3b6 Bug fix: Support question and custom_header (#14594)
### What problem does this PR solve?

Support question and custom_header

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 19:26:29 +08:00
Haruko386
dd7a0ce1d3 Go: implement provider: lm-studio (#14586)
### What problem does this PR solve?

implement `lm-studio` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-06 19:23:11 +08:00
Vivek Dubey
33d8320ce8 fix: normalize double-escaped LaTeX backslashes and HTML entities (#14564)
Fixes #14562

## Problem
LLMs like DeepSeek V4 Flash and Qwen3-MAX return \\( and \\[ 
(double backslash) in LaTeX output. The preprocessLaTeX() function 
only handled single backslash delimiters, so equations showed as raw
text.
HTML entities like &lt; and &gt; were also not decoded.

## Solution
Added normalization step before existing delimiter conversion:
- \\( → \(  and  \\[ → \[
- &lt; → <  and  &gt; → >  and  &amp; → &

---------

Co-authored-by: Vivek <viveksantoshkumardubey@email.com>
2026-05-06 19:14:34 +08:00
buua436
c9513e5ecb Fix: bootstrap agent replica on demand (#14588)
### What problem does this PR solve?

bootstrap agent replica on demand

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 19:07:50 +08:00
Wang Qi
f32034e83e Refactor: completion -> completions (#14584)
### What problem does this PR solve?

Keep only /completions, deprecated /completion

### Type of change

- [x] Refactoring
2026-05-06 17:19:22 +08:00
buua436
a190a6d67f Fix: add file convert backward compatibility (#14583)
### What problem does this PR solve?

add file convert backward compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 15:19:38 +08:00
Preston Percival
e8f19aa338 feat(graphrag): fix merge concurrency and add resume-from-checkpoint (#14238)
This PR addresses three related GraphRAG reliability issues that
together allow long-running GraphRAG tasks (10+ hours of LLM extraction)
to be resumed after a crash or pause without re-doing completed work. It
builds on #14096 (per-doc subgraph cache) and extends the same idea to
the resolution and community-detection phases.

Fixes #14236.

## 1. Fix concurrent merge crash

Long GraphRAG runs would crash near the end of entity resolution with:
```
RuntimeError: dictionary keys changed during iteration
```
in `Extractor._merge_graph_nodes`. Two changes:

- `rag/graphrag/general/extractor.py`: snapshot `graph.neighbors(node1)`
via `list(...)` before iterating, so concurrent `add_edge` /
`remove_node` mutations on the shared `nx.Graph` cannot invalidate the
iterator. Also tracks each redirected neighbour in `node0_neighbors` so
a later merged node sharing the same external neighbour takes the
edge-merge branch instead of overwriting via `add_edge`.
- `rag/graphrag/entity_resolution.py`: serialize the merge step with a
dedicated `asyncio.Semaphore(1)`. `nx.Graph` is not thread-safe and
concurrent merges on overlapping neighbourhoods can produce incorrect
results even with the snapshot fix.

## 2. Don't wipe partial graph on pause

Previously the pause / cancel UI path called
`settings.docStoreConn.delete({"knowledge_graph_kwd": [...]}, ...)`,
destroying every subgraph, entity, relation, and graph row.
Re-triggering then started GraphRAG from scratch even though #14096 had
already added `load_subgraph_from_store`.

After main was merged in (which deleted `api/apps/kb_app.py` per
#14394), the pause path now lives on the new REST surface `DELETE
/v1/datasets/<id>/<index_type>`:

- `api/apps/services/dataset_api_service.py`: `delete_index` accepts a
`wipe: bool = True` parameter. When `False` the doc-store rows and
GraphRAG phase markers are left intact and only the running task is
cancelled. Default preserves historical behaviour.
- `api/apps/restful_apis/dataset_api.py`: parses `?wipe=false|0|no|off`
from the query string and forwards it.
- `web/src/utils/api.ts` + `web/src/services/knowledge-service.ts`:
`unbindPipelineTask` appends `?wipe=false` when explicitly false.
- The GraphRAG pause action in
`web/src/pages/dataset/dataset/generate-button/hook.ts` passes `wipe:
false` for `KnowledgeGraph`; raptor is unchanged.

**UX impact:** the pause icon next to a running GraphRAG task no longer
wipes graph data. The only path that still wipes is the explicit Delete
action in `GenerateLogButton` (trash icon behind a confirmation modal).

## 3. Phase-completion markers (`rag/graphrag/phase_markers.py`)

A small Redis-backed marker layer at
`graphrag:phase:{kb_id}:{resolution_done|community_done}` (7-day TTL).
`run_graphrag_for_kb` consults the markers on entry and skips phases
that already completed in a prior run. Markers are cleared automatically
when:
- new docs are merged into the graph (which invalidates prior resolution
and community results),
- `delete_index` wipes the graph, or
- `delete_knowledge_graph` is called.

Redis failures never block a run -- markers are an optimization, not a
gate.

## 4. Idempotent community detection

`extract_community` previously did `delete-then-insert` on
`community_report` rows; a crash mid-insert left the dataset with no
reports. Now report IDs are derived deterministically from `(kb_id,
community.title)`, the existing report IDs are snapshotted before
insert, new rows are written, then only stale rows are pruned. A failure
at any step leaves either the prior or the new report set intact --
never a partial mix.

## 5. Tunable doc-store insert pipeline

The GraphRAG insert loop in `rag/graphrag/utils.py` and the
`community_report` insert in `rag/graphrag/general/index.py` were both
hardcoded to `es_bulk_size = 4` and ran strictly sequentially. On a real
KB this meant 1077 chunks took ~21 minutes for a 100-chunk slice -- pure
round-trip overhead.

- New `insert_chunks_bounded()` helper in `rag/graphrag/utils.py`
batches inserts via a bounded `asyncio.Semaphore`. Same retry / timeout
semantics as the prior loop.
- Defaults: 64 docs per batch, 4 batches in flight (matches the regular
ingest pipeline in `document_service.py`). Tunable per-deployment via
`GRAPHRAG_INSERT_BULK_SIZE` and `GRAPHRAG_INSERT_CONCURRENCY`.
- Both `set_graph` and `extract_community` now use the helper.

This dropped the same 1077-chunk insert from minutes to seconds in local
testing without measurable extra pressure on Infinity (total in-flight
docs ≤ `BULK_SIZE × CONCURRENCY` = 256 by default).

## Tests

- `test/unit_test/rag/graphrag/test_merge_graph_nodes.py` (3 tests):
dense neighbourhood merge, neighbour-snapshot regression, concurrent
serialized merges.
- `test/unit_test/rag/graphrag/test_phase_markers.py` (4 tests): set/has
round-trip, kb-scoped clear, no-op on empty input, graceful Redis
failure.
-
`test/testcases/test_web_api/test_dataset_management/test_dataset_sdk_routes_unit.py`:
new `test_delete_index_wipe_flag_unit` covers `wipe=false` for both
GraphRAG and raptor on the new REST route, and confirms the default
still wipes and clears phase markers.

## Compatibility

- Backward compatible: tasks queued before this change behave
identically (default `wipe=true`, no markers expected).
- No schema/migration changes; all new state lives in Redis.
- New optional REST query param `wipe` on `DELETE
/v1/datasets/<id>/<index_type>`.
- New optional env vars `GRAPHRAG_INSERT_BULK_SIZE` and
`GRAPHRAG_INSERT_CONCURRENCY`; defaults preserve safe behaviour.

## Example of resume

Screenshot below shows a test resuming knowledge graph generation after
applying the concurrency fix and re-deploying.

<img width="521" height="677" alt="image"
src="https://github.com/user-attachments/assets/9ef0d405-cbb3-420d-a1a1-e51f3e7e9b7a"
/>

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-05-06 15:01:01 +08:00
Idriss Sbaaoui
38f6484e98 Fix OpenDataLoader naive parsing by normalizing @OpenDataLoader and filtering unsupported parser kwargs (#14581)
### What problem does this PR solve?
This PR fixes a bug where `layout_recognize="<name>@OpenDataLoader"` was
misrouted and then failed during parsing in the naive parser path. It
now routes correctly to OpenDataLoader and avoids passing unsupported
arguments that caused runtime errors. fixes #14572

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 15:00:55 +08:00
Sebastion
7e83c5f421 fix: authorize beta document downloads by tenant (#14496)
## Summary

This fixes a missing authorization check in the beta API document
download endpoint:

- **CWE:** CWE-862 (Missing Authorization)
- **Severity:** Medium
- **Affected route/file:** `GET /api/v1/documents/<document_id>` in
`api/apps/sdk/doc.py`
- **Data flow:** the route reads a bearer beta API token, resolves the
token with `APIToken.query(beta=token)`, accepts `document_id` directly
from the URL, loads the document with
`DocumentService.query(id=document_id)`, and then fetches the backing
object through `File2DocumentService.get_storage_address()` /
`settings.STORAGE_IMPL.get()`.

Before this change, that flow verified that the API token was valid, but
it did not verify that the token's tenant owned the document's knowledge
base. A caller with any valid beta API token and a known document ID
could therefore reach storage for a document belonging to another
tenant.

## Fix

The endpoint now takes the tenant ID from the resolved API token and
checks the document's knowledge base with:

```python
KnowledgebaseService.query(id=doc[0].kb_id, tenant_id=tenant_id)
```

If the knowledge base is not owned by the token tenant, the request
returns an access error before any storage lookup occurs. This mirrors
the tenant-scoped ownership checks used by the dataset-scoped document
download path and keeps the patch small.

## Tests

Added unit coverage for `download_doc()` to assert that:

- the beta token tenant ID is used in the knowledge-base ownership
lookup;
- cross-tenant access returns `You do not have access to this
document.`;
- storage resolution is not called before tenant authorization succeeds;
- the existing same-tenant empty-file and successful-download paths
still run after the authorization gate passes.

I also verified the final patch is limited to `api/apps/sdk/doc.py` and
the related document SDK route unit test. A local `pytest` invocation
could not complete in this checkout because the shared test fixture
attempts to log in to a RAGFlow server at `127.0.0.1:9380`, which was
not running in the local environment.

## Security analysis

This is exploitable when an attacker has a valid beta API token for
their own tenant and obtains or guesses a document ID from another
tenant. The token alone should not grant access to other tenants' files,
but the direct document route previously authorized only the token
itself and not the requested resource. The new tenant-scoped
knowledge-base check binds the requested document back to the token
tenant before storage is accessed, preventing cross-tenant document
downloads through this endpoint.

Before submitting, we attempted to disprove this by checking whether
existing dataset-scoped routes, token validation, or framework
protections already enforced ownership. They do not apply to this direct
document-ID route: it bypassed the dataset path parameter and used only
`DocumentService.query(id=document_id)` before reading storage.

cc @lewiswigmore
2026-05-06 14:55:41 +08:00
alfaadriel
5e01feb755 fix(connector_service): add TIMEZONE setting and correct interval log… (#14446)
### What problem does this PR solve?



### Type of change

- [v] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: wiratama <dafa.wiratama@bankraya.co.id>
2026-05-06 14:40:35 +08:00
euvre
8269fa01b4 Fix AttributeError when appending non-streaming tool calls to chat history in Agentic Agent (#14456)
### What problem does this PR solve?

Fix #14340 

## Problem Description

When using an **Agentic Agent** (not Workflow) with one or more
Retrieval tools (e.g., Dataset Retrieval + Memory Retrieval), the agent
silently returns an empty response (`agent_response: ""`) after hanging
for several minutes. The server logs show:

```
AttributeError: 'ChatCompletionMessageToolCall' object has no attribute 'index'
```

This error propagates as a `GENERIC_ERROR`, causing the canvas to return
an empty response. The subsequent Memory save task then receives the
empty `agent_response` and logs:

```
Document for referred_document_id XXXX not found
```

## Reproduction Steps

1. Set `DOC_ENGINE=infinity` (or `elasticsearch` — the engine itself is
not the root cause).
2. Create a blank **Agentic Agent** (not a Workflow).
3. Add **two Retrieval tools** to the Agent node:
   - `Retrieval_DS` → Dataset (Knowledge Base)
   - `Retrieval_Mem` → Memory component
4. Add a **Message** node with **Save to Memory** enabled.
5. Launch the agent and send any message (e.g., "hola").
6. The agent hangs and returns an empty response.

## Root Cause Analysis

The crash occurs in `_append_history` and `_append_history_batch` inside
`rag/llm/chat_model.py`. These methods directly access `.index` on tool
call objects:

```python
# _append_history_batch
{
    "index": tc.index,   # <-- crashes here
    ...
}
```

However, **non-streaming** LLM responses (`stream=False`) return
`ChatCompletionMessageToolCall` objects, which **do not have an `index`
field** according to the OpenAI API specification. The `index` field
only exists on `ChoiceDeltaToolCall` objects returned in **streaming**
responses (`stream=True`).

When the agentic agent triggers an internal `full_question` call (used
to compress multi-turn conversation history), the request is incorrectly
routed through `async_chat_with_tools` because `is_tools=True` is set at
the `LLMBundle` level. If the LLM decides to emit `tool_calls` during
this auxiliary request, the code enters the non-streaming tool loop and
crashes when trying to append history.

## Fix

Replaced all direct `.index` accesses with `getattr(..., "index", None)`
for safe, backward-compatible access:

| Method | File | Line | Change |
|--------|------|------|--------|
| `_append_history` | `rag/llm/chat_model.py` | ~L304 |
`tool_call.index` → `getattr(tool_call, "index", None)` |
| `_append_history_batch` | `rag/llm/chat_model.py` | ~L332 | `tc.index`
→ `getattr(tc, "index", None)` |
| `_append_history` | `rag/llm/chat_model.py` | ~L1467 |
`tool_call.index` → `getattr(tool_call, "index", None)` |
| `_append_history_batch` | `rag/llm/chat_model.py` | ~L1496 |
`tc.index` → `getattr(tc, "index", None)` |

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-05-06 14:39:40 +08:00
Shiyao Huang
406b36a452 fix(#14389): normalize list metadata values for in filters (#14410)
## Summary
- normalize string items for list-valued metadata filters in
`meta_filter`
- fix `in` / `not in` case asymmetry when document metadata is
lowercased but filter list values are not
- add regression tests that cover the original issue scenario using
uppercase list values

## Validation
- `PYTHONPATH=external/ragflow pytest
external/ragflow/test/unit_test/common/test_metadata_filter_operators.py
-q`

## Notes
- I commented on #14389 before opening this PR to claim the issue.
- The new tests use `value=["F2", "F11"]` so they fail on the old
implementation and pass with this fix.
- This also benefits other non-comparison operators that flow through
the same normalization path.

Co-authored-by: copizza <copizza@users.noreply.github.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-05-06 14:28:25 +08:00
buua436
e4aee25b4b Fix: add legacy agent completion API compatibility (#14582)
### What problem does this PR solve?

add legacy agent completion API compatibility

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 14:22:48 +08:00
jony376
94f8779a00 Memory API: enforce tenant permissions on memory and message endpoints (#14535)
### What problem does this PR solve?

This PR fixes missing authorization checks in the Memory API.
Previously, several authenticated endpoints accepted caller-supplied
`tenant_id`, `owner_ids`, or `memory_id` values and used them directly
to list, read, update, delete, or search Memory data.

That could allow an authenticated user to access or mutate another
tenant's Memory records if they knew a tenant ID or memory ID. The fix
centralizes Memory access checks and applies them consistently across
Memory and Memory-message operations.

The change:

- Adds helper logic to parse list filters and compute tenant IDs
accessible to `current_user`.
- Requires direct `memory_id` operations to pass Memory access checks
before reading, updating, deleting, or changing message state.
- Filters list/search/recent-message requests to accessible memories
only.
- Applies Memory visibility filtering before count and pagination in
`MemoryService.get_by_filter`.
- Accepts `owner_ids` in the Memory list route, matching the frontend
owner filter while still intersecting values with the caller's
accessible tenants.
- 

### Related issues
Closes #14534 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: jony376 <jony376@gmail.com>
2026-05-06 14:10:47 +08:00
buua436
5672be0652 Feat: add IMAP deleted document sync (#14539)
### What problem does this PR solve?

add IMAP deleted document sync

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-06 14:06:46 +08:00
NeedmeFordev
89961962c0 feat(dingtalk-ai-table): support deleted-file sync via slim snapshot (#14525)
### What problem does this PR solve?

Incremental DingTalk AI Table (Notable) sync did not reconcile rows
removed on the remote side with documents already in the knowledge base.
This follows the coordinated datasource work in #14362 (“sync deleted
files”).

This PR adds a **full slim snapshot**
(`retrieve_all_slim_docs_perm_sync`) that lists **current record IDs for
all sheets** without building document blobs, using the same logical
document IDs as full ingest
(`dingtalk_ai_table:{table_id}:{sheet_id}:{record_id}`). When
**`sync_deleted_files`** is enabled on incremental runs,
`DingTalkAITable._generate` returns **`(document_generator,
file_list)`** so **`SyncBase`** can run
**`cleanup_stale_documents_for_task`** and remove KB rows that no longer
exist remotely.

Design notes:

- **`_document_id`** centralizes the ID string so slim snapshots and
**`_convert_record_to_document`** stay aligned with
**`hash128(doc.id)`** semantics used during ingestion/cleanup.
- **`end_ts`** is captured before building **`file_list`**, then
**`poll_source`** uses the same upper bound (consistent with other
Dropbox-style connectors).
- **`batch_size`** from connector config is coerced to a positive
**`int`** before constructing the connector.
- Slim snapshot failures are caught in **`_generate`**; **`file_list`**
is set to **`None`** so cleanup is skipped rather than running on
partial/error state.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Files changed (summary)

| Area | Change |
|------|--------|
| `common/data_source/dingtalk_ai_table_connector.py` |
`SlimConnectorWithPermSync`, `retrieve_all_slim_docs_perm_sync`,
`_document_id` shared with document conversion |
| `rag/svr/sync_data_source.py` | `DingTalkAITable._generate`: slim
snapshot + tuple return; `batch_size` validation; shared `end_ts` with
`poll_source` |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`syncDeletedFiles` for DingTalk AI Table in
`DataSourceFeatureVisibilityMap` |

Closes / relates to: #14362
2026-05-06 14:06:23 +08:00
Idriss Sbaaoui
c502001d9e Fix MinerU output fallback and NameError regression (#14538)
### What problem does this PR solve?

This fixes a MinerU parsing failure where output JSON was not found in
nested v0.24.0 layouts, and also fixes a `content_names` NameError in
`_read_output()`. As a result, successful MinerU API runs no longer end
with false “MinerU not found” parsing failures.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-06 14:03:57 +08:00
sapienza yoan
c0fc8b32f2 Fix: retry RocksDB metadata contention on concurrent CREATE/DROP (#14563)
Concurrent CREATE TABLE / CREATE INDEX / DROP TABLE on the same Infinity
instance can race on the catalog counter (e.g. db|1|next_table_id) and
fail with error 9003 "Resource busy" instead of waiting on a lock. Two
users creating a knowledge base at the same instant, or any deployment
with multiple backend workers behind one Infinity, can hit it.

Wrap the metadata paths in create_idx, create_doc_meta_idx, and
delete_idx with exponential backoff + jitter (5 attempts, 50ms base).
The wrapped operations already use ConflictType.Ignore, so retrying is
idempotent — worst case the second attempt is a no-op against an
already-created table. Tunable via INFINITY_META_RETRY_MAX /
INFINITY_META_RETRY_BASE_DELAY_MS.

Repro: stress 30 concurrent POST /api/v1/datasets against a 4-worker
backend → ~50% of requests fail without the patch (Resource busy from
the second worker that hits the counter), 100% succeed with it. At 100
concurrent requests, all 100 succeed in ~1.2s; the retry budget never
exhausted in our tests.

Scope is limited to metadata paths only — data-path operations (INSERT
chunks, SELECT for retrieval) go through per-table code paths and don't
share the contended counter.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: yoan sapienza <Yoan Sapienza yoan.sapienza@orange.fr Yoan Sapienza zappy@macbookpro.home>
2026-05-06 13:32:20 +08:00
Jack Storment
c2ad672c09 Go: implement provider: xAI (#14550)
Closes #14552 

### What problem does this PR solve?

Add a Go driver for xAI (Grok models).

The config file conf/models/xai.json has been in the repo since the
early Go provider work, but internal/entity/models/factory.go had no
case for "xai". So any xAI request fell through to the dummy driver
and never reached the API. This PR adds the missing driver and wires it
up.

### What this PR includes

- New file internal/entity/models/xai.go with an XAIModel that
implements the ModelDriver interface.
- factory.go: route the "xai" provider name to NewXAIModel.

### How the driver works

- xAI exposes an OpenAI-compatible API at https://api.x.ai/v1.
- ChatWithMessages and ChatStreamlyWithSender post to /chat/completions
in the same shape the moonshot and deepseek drivers use.
- ListModels and CheckConnection call /models to confirm the API key
works and to list available model ids.
- reasoning_content is passed through for grok-3-mini and other xAI
reasoning models, both in the non-stream and stream paths.
- Encode, Rerank, and Balance are not part of the public xAI API at the
moment, so they return a clear "not implemented" or "no such method"
error.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### How was this tested?

- go build ./internal/entity/models/... in a clean go 1.25 image (the
  go.mod minimum) returns exit 0 with no errors.
- Method set of XAIModel matches the ModelDriver interface: NewInstance,
  Name, ChatWithMessages, ChatStreamlyWithSender, Encode, Rerank,
  ListModels, Balance, CheckConnection.
- Pattern parity with the merged moonshot (#14433), volcengine (#14460),
  minimax (#14478), and vllm (#14532) PRs.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 12:16:37 +08:00
Haruko386
cd54c08e84 Go: implement provider: Ollama (#14580)
### What problem does this PR solve?

implement `Ollama` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 12:03:58 +08:00
Yingfeng
28388993a4 Update README (#14547)
### Type of change

- [x] Documentation Update
2026-05-06 11:57:29 +08:00
qinling0210
7335916868 Use GetChatModel, remove duplicate functions in model_service.go (#14546)
### What problem does this PR solve?

Use GetChatModel, remove duplicate functions in model_service.go

### Type of change

- [x] Refactoring

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 11:33:32 +08:00
dependabot[bot]
9e4f3614de Chore(deps-dev): Bump pillow from 12.1.1 to 12.2.0 (#14578)
As title
2026-05-06 11:08:38 +08:00
Jin Hai
aa57b5bd8b Go: move logger to common module (#14545)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 10:41:58 +08:00
Jin Hai
3a51c27a75 Go: CLI chat with text, image, video (#14573)
### What problem does this PR solve?

```
RAGFlow(user)> chat with 'glm-4.6v-flash@test@zhipu-ai' message 'What are the pics talk about?' image 'https://cdn.bigmodel.cn/static/logo/register.png' 'https://cdn.bigmodel.cn/static/logo/api-key.png'
Answer: The first picture shows a login/register modal with options for phone number login, account login, and WeChat QR code login, along with a prompt for new users to get a 20 million tokens experience package. The second picture displays the API keys management page of a platform, including a warning about API key security and a table listing existing API keys with details like creation time and usage history.
Time: 31.600545
RAGFlow(user)> chat with 'glm-4.6v-flash@test@zhipu-ai' message 'What are the video talk about?' video 'https://cdn.bigmodel.cn/agent-demos/lark/113123.mov'
Answer: Based on the sequence of frames provided, the video is a demonstration of a web search and navigation process.

1.  The video starts with a blank Google search page.
2.  The user types "智谱" (which is the Chinese name for the company Zhipu AI) into the search box.
3.  The search is initiated and the page shows "About 0 results".
4.  The search results load, showing information about Zhipu AI, including its website.
5.  The user clicks on the main website link (www.zhipuai.cn).
6.  The video ends by showing the homepage of Zhipu AI's website, titled "Z.ai GLM Large Model Open Platform".

In summary, the video is about searching for the company "智谱" (Zhipu AI) on Google and then navigating to its official website.
Time: 76.582520
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-05 18:14:39 +08:00
Attili-sys
24af0875e5 Feat/configurable metadata display (#13464)
### What problem does this PR solve?

Currently, RAGFlow's Search and Chat interfaces display only raw
vectorized text chunks during retrieval, without contextual information
about their source documents. Users cannot see document titles, page
numbers, upload dates, or custom metadata fields that would help them
understand and trust the retrieved results.

This PR introduces an **optional metadata display feature** that
enriches retrieved chunks with document-level metadata in both the
Search tab and Chatbot interface.

**Key improvements:**
- **Search results**: Display document metadata as styled badges beneath
chunk snippets
- **Chat citations**: Show metadata in citation popovers and reference
lists for better source context
- **LLM context**: Metadata is injected into the LLM prompt to enable
more accurate, citation-aware responses
- **External API support**: Applications using RAGFlow's SDK retrieval
endpoints (`/v1/retrieval`, `/v1/searchbots/retrieval_test`) can opt-in
via request parameters
- **User control**: Multi-select dropdown UI allows users to choose
which metadata fields to display

**Implementation approach:**
-  Reuses existing `DocMetadataService` infrastructure (no new database
tables or indices)
-  Settings stored in existing JSON configuration fields
(`search_config.reference_metadata`, `prompt_config.reference_metadata`)
-  No database migrations required
-  Disabled by default (fully opt-in and backward-compatible)
-  Dynamic metadata field selection populated from actual document
metadata keys
-  Fixed critical bug where Python's builtin `set()` was shadowed by a
route handler function

**Modified endpoints (all backward-compatible):**
- `POST /v1/retrieval` (Public SDK)
- `POST /v1/searchbots/retrieval_test` (Searchbots)
- `POST /v1/chunk/retrieval_test` (UI/Internal)
- Chat completions endpoints (via `extra_body.reference_metadata` or
`prompt_config`)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


###Images
-
<img width="879" height="1275" alt="image"
src="https://github.com/user-attachments/assets/95b2d731-31ae-45a1-b081-bf5893f52aeb"
/>
<br><br>
<br><br>

<img width="1532" height="362" alt="image"
src="https://github.com/user-attachments/assets/9cebc65b-b7a7-459f-b25e-3b13fa9b638e"
/>
<br><br>
<br><br>

<img width="2586" height="1320" alt="image"
src="https://github.com/user-attachments/assets/2153d493-d899-461f-a7a9-041391e07776"
/>

---------

Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Attili-sys <Attili-sys@users.noreply.github.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-04-30 23:13:27 +08:00
writinwaters
d38d6e7931 Doc: RAGFlow now supports DeepSeek v4 (#14544)
### What problem does this PR solve?

RAGFlow now supports DeepSeek v4.

### Type of change

- [x] Documentation Update
2026-04-30 20:12:29 +08:00
writinwaters
f14abf858e Doc: Minor editorial updates (#14543)
### What problem does this PR solve?

Minor editorial updates.

### Type of change


- [x] Documentation Update
2026-04-30 20:06:28 +08:00
qinling0210
12af73f2ca Support stream for multimodal chat (#14537)
### What problem does this PR solve?

Support stream for multimodal chat

### Type of change

- [x] Refactoring
2026-04-30 19:33:57 +08:00
Magicbook1108
5fd4579a2f Fix: sync data source empty list (#14530)
### What problem does this PR solve?

Fix: sync data source empty list

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 18:56:43 +08:00
buua436
05ee7f8bb6 Fix: remove delete_documents uuid validation (#14533)
### What problem does this PR solve?

remove delete_documents uuid validation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 18:56:33 +08:00
bitloi
a69e0c73c7 feat(rss): support deleted-file sync (#14493)
### What problem does this PR solve?

Partially addresses #14362.

This PR enables syncing deleted files for RSS data sources.

Previously, RSS incremental sync only returned feed entries whose
timestamps were inside the poll window. If an entry was removed from the
RSS feed, RAGFlow had no full current RSS snapshot to pass into the
shared stale-document cleanup path, so the deleted remote entry could
remain in the knowledge base.

This PR:
- adds `retrieve_all_slim_docs_perm_sync()` to `RSSConnector`
- reuses the same `rss:<md5(stable_key)>` document ID derivation used by
normal RSS ingest
- returns `(document_generator, file_list)` for incremental RSS sync
when `sync_deleted_files` is enabled
- captures the poll end timestamp before snapshot/poll so cleanup does
not race against the same sync window
- adds start/end logs around RSS slim snapshot collection
- exposes the deleted-file sync toggle for RSS in the data source UI

Per maintainer request on related datasource PRs, this PR contains no
test-case changes. Local verification was run with an external script.

Validation:
- `uv run ruff check common/data_source/rss_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`
- `git diff --check`
- `uv run python /tmp/verify_rss_deleted_sync.py --repo
/root/74/ragflow`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 18:56:13 +08:00
NeedmeFordev
bedf9592ef feat(webdav): support deleted-file sync via slim snapshot (#14491)
## What problem does this PR solve?

Incremental WebDAV sync only ingested files whose modification time fell
inside the poll window; documents removed on the WebDAV server were
never removed from the knowledge base. This aligns with
[#14362](https://github.com/infiniflow/ragflow/issues/14362)
(coordinated datasource “sync deleted files” work).

This PR adds a **full-tree slim snapshot**
(`retrieve_all_slim_docs_perm_sync`) that enumerates current remote
paths **without downloading file contents**, using the same logical
document IDs as full ingest (`webdav:{base_url}:{file_path}`). When
**`sync_deleted_files`** is enabled on incremental runs, sync returns
**`(document_generator, file_list)`** so **`SyncBase`** runs
**`cleanup_stale_documents_for_task`** and removes KB rows no longer
present remotely.

Design notes:

- **`_list_files_recursive`** gains **`filter_by_mtime`**: snapshot
passes **`filter_by_mtime=False`** (full tree under **`remote_path`**);
**`poll_source`** keeps mtime-window filtering as before.
- Slim snapshot applies the same **extension** and **`size_threshold`**
rules as **`_yield_webdav_documents`** so retain IDs match what would be
indexed.
- **`end_ts`** is captured before building **`file_list`**, then
**`poll_source`** uses the same upper bound (consistent with
Dropbox-style connectors).

## Type of change

- [x] New Feature (non-breaking change which adds functionality)

## Files changed

| Area | Change |
|------|--------|
| `common/data_source/webdav_connector.py` |
`SlimConnectorWithPermSync`, `retrieve_all_slim_docs_perm_sync`,
`filter_by_mtime` on `_list_files_recursive` |
| `rag/svr/sync_data_source.py` | WebDAV `_generate`: `file_list` +
tuple return; pass **`batch_size`** from connector config |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`syncDeletedFiles` for WebDAV in `DataSourceFeatureVisibilityMap` |
2026-04-30 17:26:27 +08:00
Wang Qi
c363e43664 Fix #14443 (#14536)
### What problem does this PR solve?

Use variable from os.env

### Type of change

- [ ] Documentation Update
2026-04-30 17:21:28 +08:00
Haruko386
93f3b90121 Go: implement provider: Vllm (#14532)
### What problem does this PR solve?

Implement the vLLM model provider for RAGFlow to fully support local and
self-hosted open-source models (e.g., Qwen, GLM, Llama) via the vLLM
framework, and fix several critical bugs related to model instance
management and API requests.

**Key changes and fixes:**
1. **Added Standard vLLM Provider (`vllm.go`, `vllm.json`):**
- Implemented `VllmModel` driver strictly adhering to the OpenAI API
specification.
- Removed hardcoded and dangerous routing logic (e.g., forcing
`AsyncChat` for Qwen/GLM prefixes), ensuring standard
`/v1/chat/completions` compatibility.
- Refactored `ListModels` to use safe JSON parsing (resolving nil
pointer panics) and standard `GET` requests without bodies.
- Added `APIConfig.Region` fallback logic to prevent empty `base_url`
fetching when checking models.

2. **Fixed `ChatToModelStreamWithSender` Bug (`model_service.go`):**
- Resolved the `model is disabled` error when streaming chat with local
database-saved models.
- Added the missing `if modelInfo.Status == "active"` block to correctly
invoke `NewInstance` and inject the dynamic `base_url` into the provider
driver before starting the SSE stream.

3. **Fixed `ListSupportedModels` Bug (`model_service.go`):**
- Added dynamic `NewInstance` injection for `base_url`. Previously, the
list models function used the static JSON config without injecting
user-configured dynamic URLs from the database, resulting in an
`unsupported protocol scheme ""` error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 16:30:14 +08:00
balibabu
00e03a1945 Fix: LaTeX formulas cannot be displayed on the chat page. (#14531)
### What problem does this PR solve?

Fix: LaTeX formulas cannot be displayed on the chat page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 16:12:13 +08:00
qinling0210
265f92c83e Simplify chat and support multimodal chat (#14523)
### What problem does this PR solve?

Simplify chat and support multimodal chat

### Type of change

- [x] Refactoring
2026-04-30 15:25:01 +08:00
Wang Qi
f45ce00347 Not allow to sort by id (#14526)
### What problem does this PR solve?

id as "text", not a "keyword", order by it will cause error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 14:52:43 +08:00
Jin Hai
45d77dc778 Fix version info (#14529)
### What problem does this PR solve?

Fix docker image version info in comment

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-30 14:47:28 +08:00
bitloi
17eda04b8d feat(zendesk): support deleted-file sync (#14487)
### What problem does this PR solve?

Refs #14362.

This PR enables syncing deleted files for Zendesk data sources.

Previously, Zendesk incremental sync never returned a slim remote
snapshot to the shared stale-document cleanup path, so deleted remote
Zendesk records could remain in RAGFlow. The existing Zendesk slim
snapshot also included records that ingestion intentionally skips, such
as draft articles, articles without bodies, skipped-label articles,
empty-body articles, and tickets with `status == "deleted"`.

This PR:
- exposes the deleted-file sync option for Zendesk in the data source UI
- returns Zendesk slim snapshots during incremental sync when
`sync_deleted_files` is enabled
- reuses Zendesk indexability rules so cleanup compares against the same
records ingestion can materialize
- adds start/end logs around Zendesk slim snapshot collection for
operational visibility

Per maintainer request, this PR contains no test-case changes. Manual
verification recording will be provided separately.

Validation:
- `uv run ruff check common/data_source/zendesk_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-30 14:44:05 +08:00
bitloi
8f75e52bbf feat(asana): support deleted-file sync (#14468)
### What problem does this PR solve?

Partially addresses #14362.

Adds deleted-file sync support for the Asana data source. Asana already
indexes task attachments as documents, but it did not provide the slim
document snapshot required by stale-document reconciliation, and the
sync wrapper never returned a `file_list` for cleanup.

This PR:
- adds `retrieve_all_slim_docs_perm_sync()` to `AsanaConnector`
- builds slim IDs with the same `asana:{task_id}:{attachment_gid}`
format used by indexed documents
- avoids downloading attachment blobs during the snapshot
- aborts the snapshot if Asana API errors occur, preventing partial
snapshots from deleting valid local docs
- captures the incremental poll end time before snapshotting and makes
`poll_source()` respect that boundary
- exposes the deleted-file sync toggle for Asana in the data source UI

Per maintainer request, this PR contains no test-case changes. Manual
verification recording will be provided separately.

Validation:
- `uv run ruff check common/data_source/asana_connector.py
rag/svr/sync_data_source.py`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`
- `git diff --check`

### Type of change

- [x] New Feature
2026-04-30 14:41:36 +08:00
orbisai0security
e992fe39b2 fix: the oceanbase database connector constructs sql... in ob_conn.py (#14470)
## Summary
Fix critical severity security issue in `rag/utils/ob_conn.py`.

## Vulnerability
| Field | Value |
|-------|-------|
| **ID** | V-003 |
| **Severity** | CRITICAL |
| **Scanner** | multi_agent_ai |
| **Rule** | `V-003` |
| **File** | `rag/utils/ob_conn.py:691` |

**Description**: The OceanBase database connector constructs SQL WHERE
clauses by directly embedding user-controlled filter expressions using
Python f-strings at lines 726, 777, 781, 787, 793, 821, and 827. No
parameterization or allowlist validation is applied before the
expressions are incorporated into live SQL queries. This is the most
critical vulnerability in the codebase because it directly exposes the
RAG knowledge base — the platform's core business asset — to complete
compromise.

## Changes
- `rag/utils/ob_conn.py`

## Verification
- [x] Build passes
- [x] Scanner re-scan confirms fix
- [x] LLM code review passed

---
*Automated security fix by [OrbisAI Security](https://orbisappsec.com)*
2026-04-30 14:25:17 +08:00
Yingfeng
4ee0702aed Feat: add skills space to context engine (#13908)
### What problem does this PR solve?

issue #13714

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:36:03 +08:00
Magicbook1108
bb3b99f0a5 Feat: add button for remove header & footer in pipeline (#14486)
### What problem does this PR solve?

Feat: add button for remove header & footer in pipeline

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:30:41 +08:00
NeedmeFordev
2932b65da6 feat(seafile): support deleted-file sync via slim snapshot (#14499)
### What problem does this PR solve?

Incremental Seafile sync only ingests files whose modification time
falls in the poll window; documents removed in Seafile were never
removed from the knowledge base. This contributes to
[#14362](https://github.com/infiniflow/ragflow/issues/14362) (datasource
“sync deleted files” coordination).

This PR adds a **slim snapshot** (`retrieve_all_slim_docs_perm_sync`)
that enumerates current remote file IDs **without downloading content**,
using the same logical IDs as full ingest
(`seafile:{repo_id}:{file_id}`). When **`sync_deleted_files`** is
enabled on incremental runs, **`SeaFile._generate`** returns
**`(document_generator, file_list)`** so **`SyncBase`** can run
**`cleanup_stale_documents_for_task`** and remove stale KB documents.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
### What changed

- **`common/data_source/seafile_connector.py`**: `SeaFileConnector`
implements **`SlimConnectorWithPermSync`**;
**`_list_files_recursive(..., filter_by_mtime=...)`** supports full-tree
listing for snapshots; **`retrieve_all_slim_docs_perm_sync()`** reuses
the same library/root scan as ingest and applies the same **size**
ceiling; logging for snapshot start/end and counts.
- **`rag/svr/sync_data_source.py`**: **`SeaFile._generate`** validates
**`batch_size`**, captures **`end_ts`** before snapshot +
**`poll_source`**, wraps slim retrieval in **`try`/`except`** (
**`file_list = None`** on failure so ingest continues), returns
**`(generator, file_list)`**.
- **`web/src/pages/user-setting/data-source/constant/index.tsx`**:
**`syncDeletedFiles`** for Seafile in
**`DataSourceFeatureVisibilityMap`**.
2026-04-30 12:05:12 +08:00
writinwaters
8aaf0942b1 Doc: Updated v0.25.1 release notes (#14519)
### What problem does this PR solve?

Updated v0.25.1 release notes.

### Type of change


- [x] Documentation Update
2026-04-30 11:59:53 +08:00
Idriss Sbaaoui
9075872435 Fix: Manual/Naive outline tuple unpack crash (#14518)
### What problem does this PR solve?

This fixes a crash in Manual and Naive parsing when PDF outlines include
page numbers as a third tuple value. It makes outline unpacking accept
extra values so parsing no longer fails. fixes #14411

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:55:02 +08:00
dependabot[bot]
71952b6b58 Chore(deps): Bump go.opentelemetry.io/otel from 1.39.0 to 1.41.0 (#14516)
Bumps
[go.opentelemetry.io/otel](https://github.com/open-telemetry/opentelemetry-go)
from 1.39.0 to 1.41.0.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 11:43:24 +08:00
buua436
06c6da5d94 Fix: add document delete permission check (#14472)
### What problem does this PR solve?

add document delete permission check

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:01:09 +08:00
buua436
47129fdd08 Fix: optimize file batch delete (#14473)
### What problem does this PR solve?

optimize file batch delete

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 11:00:39 +08:00
sapienza yoan
811e9826d0 perf: avoid O(n²) array growth in embedding accumulation (#14369)
### What problem does this PR solve?

Both tokenizer (`rag/flow/tokenizer/tokenizer.py`) and
`BuiltinEmbed.encode`
(`rag/llm/embedding_model.py`) currently accumulate embedding batches
via
`np.concatenate` inside the per-batch loop. `np.concatenate` allocates a
new
array and copies all existing data on every call, so accumulating N
batches
is O(N²) in both time and peak memory.

Replacing the incremental concatenate with a list-of-batches + a single
`np.vstack` at the end gives O(N) total work.

For tokenizer the title-vector broadcast `np.concatenate([vts[0]] * N)`
is
also replaced by `np.tile`, which does the same job with a single
contiguous
allocation instead of building a Python list of references.

This is purely a CPU/memory optimisation — output shape and dtype are
unchanged. Measured impact grows with document size:
  -   1k chunks (batch 512, 2 iters):    ~negligible
  -  10k chunks (20 iters):              ~10× speedup on this stage
  - 100k chunks (195 iters):             ~100× speedup, and peak RAM
drops from O(N) extra to near-zero

### Type of change

- [x] Performance Improvement

Co-authored-by: yoan sapienza <Yoan Sapienza yoan.sapienza@orange.fr Yoan Sapienza zappy@macbookpro.home>
2026-04-30 11:00:10 +08:00
FuturMix
2548c28d65 feat: add FuturMix as model provider (#14419)
## Summary

Add [FuturMix](https://futurmix.ai) as a new model provider. FuturMix is
an OpenAI-compatible unified AI gateway that provides access to 22+
models (GPT, Claude, Gemini, DeepSeek, and more) through a single API
endpoint and key.

- **API Base**: `https://futurmix.ai/v1` (OpenAI-compatible)
- **Supported capabilities**: Chat, Embedding, Image2Text, TTS,
Speech2Text, Rerank

### Changes

| File | Change |
|------|--------|
| `rag/llm/__init__.py` | Add `FuturMix` to `SupportedLiteLLMProvider`
enum, `FACTORY_DEFAULT_BASE_URL`, and `LITELLM_PROVIDER_PREFIX` |
| `rag/llm/chat_model.py` | Add `FuturMixChat(Base)` — follows
Astraflow/Avian pattern |
| `rag/llm/embedding_model.py` | Add `FuturMixEmbed(OpenAIEmbed)` —
follows Astraflow pattern |
| `rag/llm/cv_model.py` | Add `FuturMixCV(GptV4)` — follows
SILICONFLOW/OpenRouter pattern |
| `rag/llm/tts_model.py` | Add `FuturMixTTS(OpenAITTS)` — follows
CometAPI/DeerAPI pattern |
| `rag/llm/sequence2txt_model.py` | Add `FuturMixSeq2txt(GPTSeq2txt)` —
follows StepFun pattern |
| `rag/llm/rerank_model.py` | Add `FuturMixRerank(OpenAI_APIRerank)` |
| `conf/llm_factories.json` | Add factory config with 8 chat, 2
embedding, 1 image2text, 2 TTS, 1 speech2text models |
| `docs/guides/models/supported_models.mdx` | Add FuturMix to supported
models table |

### Models included

- **Chat**: claude-sonnet-4-20250514, claude-3.5-haiku, gpt-4o,
gpt-4o-mini, gemini-2.5-flash, gemini-2.0-flash, deepseek-chat,
deepseek-reasoner
- **Embedding**: text-embedding-3-small, text-embedding-3-large
- **Image2Text**: gpt-4o
- **TTS**: tts-1, tts-1-hd
- **Speech2Text**: whisper-1

## Test plan

- [ ] Verify FuturMix appears in the model provider list in RAGFlow UI
- [ ] Configure FuturMix with API key and test chat completion
- [ ] Test embedding model with document indexing
- [ ] Test image2text with a sample image

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-30 10:59:37 +08:00
Liu An
ce4c782fd7 Docs: Update version references to v0.25.1 in READMEs and docs (#14488)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.25.0 to v0.25.1
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-04-30 10:49:26 +08:00
balibabu
7c0584a2b7 Fix: The GraphRAG icon is not displaying. (#14514)
### What problem does this PR solve?

Fix: The GraphRAG icon is not displaying.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 10:44:05 +08:00
dependabot[bot]
0fa2bd539d Chore(deps): Bump google.golang.org/grpc from 1.66.2 to 1.79.3 (#14513)
Bumps [google.golang.org/grpc](https://github.com/grpc/grpc-go) from
1.66.2 to 1.79.3.

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-30 10:35:03 +08:00
euvre
6dd38eca6a fix: file logs not displayed in dataset ingestion page (#14479)
### What problem does this PR solve?

## Summary

Fixed a bug where the **File Logs** tab in the dataset ingestion page
always showed "No logs" even after files were parsed successfully.

## Root Cause

Both the **File Logs** and **Dataset Logs** tabs on the frontend called
the same backend endpoint `/datasets/{dataset_id}/ingestions`. However,
the backend only queried `get_dataset_logs_by_kb_id`, which
hard-filtered records by `document_id == GRAPH_RAPTOR_FAKE_DOC_ID`
(dataset-level logs). As a result, real file-level logs were never
returned, causing the table to appear empty.

## Changes

### Backend

- **`api/apps/restful_apis/dataset_api.py`**
  - Added two new query parameters to `list_ingestion_logs`:
    - `log_type` — `"file"` or `"dataset"` (default: `"dataset"`)
    - `keywords` — search keyword for filtering by document / task name

- **`api/apps/services/dataset_api_service.py`**
- Updated `list_ingestion_logs` signature to accept `log_type` and
`keywords`.
  - Added conditional routing:
- When `log_type == "file"`, call
`PipelineOperationLogService.get_file_logs_by_kb_id`
- Otherwise, call
`PipelineOperationLogService.get_dataset_logs_by_kb_id`

- **`api/db/services/pipeline_operation_log_service.py`**
- Extended `get_dataset_logs_by_kb_id` with an optional `keywords`
parameter so dataset logs can also be searched.

### Frontend

- **`web/src/pages/dataset/dataset-overview/hook.ts`**
- Removed the separate API function switching (`listPipelineDatasetLogs`
vs `listDataPipelineLogDocument`).
- Unified both tabs to call `listDataPipelineLogDocument` with the new
`log_type` query parameter (`"file"` or `"dataset"`).
  - Ensured `keywords` and filter values are passed through correctly.

## Behavior After Fix

| Tab | `log_type` | Returned Records | Searchable Field |
|---|---|---|---|
| File Logs | `file` | Real document-level logs | `document_name` (file
name) |
| Dataset Logs | `dataset` | GraphRAG / RAPTOR / MindMap logs |
`document_name` (task type) |
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-04-29 22:10:24 +08:00
Wang Qi
5018459112 Fix metadata config (#14480)
### What problem does this PR solve?

Fix metadata config

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 21:09:54 +08:00
writinwaters
d4147efc66 Docs: (#14492)
### What problem does this PR solve?

Added v0.25.1 release notes

### Type of change


- [x] Documentation Update
2026-04-29 20:29:58 +08:00
Wang Qi
c4d0b0ebcf Fix visit dataset error (#14490)
### What problem does this PR solve?

Fix visit dataset error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 20:17:00 +08:00
balibabu
1692f0928f Fix: The pipeline column header in the FileLogsTable is displaying incorrectly. (#14489)
### What problem does this PR solve?
Fix: The pipeline column header in the FileLogsTable is displaying
incorrectly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 19:52:28 +08:00
writinwaters
9280c64518 Docs: Updated Title chunker references (#14483)
### What problem does this PR solve?

Updated Title chunker references

### Type of change

- [x] Documentation Update
2026-04-29 19:37:24 +08:00
Jin Hai
261be81127 Go: add drop instance models (#14485)
### What problem does this PR solve?

1. drop instance model
2. Fix issue of drop instance but not drop models.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 19:18:49 +08:00
Haruko386
0e1477eb23 Go: implement provider: MiniMax (#14478)
### What problem does this PR solve?

implement MiniMax provider

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:06:40 +08:00
Magicbook1108
de8c6ad0f3 Feat: enable sync deleted file for Discord (#14451)
### What problem does this PR solve?

Feat: enable sync deleted file for Discord

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:05:40 +08:00
bitloi
2bc8c6d35e feat(dropbox): support deleted-file sync (#14476)
### What problem does this PR solve?

Partially addresses #14362 by adding deleted-file sync support for the
Dropbox data source.

Dropbox previously did not provide the slim current-file snapshot
required by stale document reconciliation, and its sync runner returned
only document batches. As a result, enabling deleted-file sync could not
remove local documents that had been deleted from Dropbox.

This PR:
- Adds `retrieve_all_slim_docs_perm_sync()` to `DropboxConnector`.
- Reuses Dropbox metadata traversal to collect current remote file IDs
without downloading file contents.
- Wires incremental Dropbox sync to return `(document_generator,
file_list)` when `sync_deleted_files` is enabled.
- Enables the deleted-file sync toggle for Dropbox in the data source
settings UI.
- Adds regression coverage for slim snapshots, nested folders, paginated
listings, duplicate filenames, and full reindex behavior.

Tests:
- `uv run pytest test/unit_test/common/test_dropbox_connector.py -q`
- `uv run pytest test/unit_test/rag/test_sync_data_source.py -q`
- `uv run pytest test/unit_test/common/test_dropbox_connector.py
test/unit_test/rag/test_sync_data_source.py -q`
- `uv run ruff check common/data_source/dropbox_connector.py
rag/svr/sync_data_source.py
test/unit_test/common/test_dropbox_connector.py
test/unit_test/rag/test_sync_data_source.py`
- `./node_modules/.bin/eslint
src/pages/user-setting/data-source/constant/index.tsx`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:05:11 +08:00
Magicbook1108
db1a73b255 Feat: enable sync deleted files in gitlab (#14481)
### What problem does this PR solve?

Feat: enable sync deleted files in gitlab

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 19:04:10 +08:00
euvre
a0f9ae16d2 Fix: RAPTOR "Generation scope" reset to "Single file" when selecting "Dataset" (#14477)
## Problem
In the Dataset Configuration page, changing the RAPTOR **Generation
scope** from "Single file" to "Dataset" and clicking **Save** did not
persist the change. After refreshing or re-entering the page, the scope
always reverted to "Single file".

## Root Cause
1. **Backend**: The `RaptorConfig` Pydantic model in
`api/utils/validation_utils.py` was configured with `extra="forbid"` but
did not declare a `scope` field. When the frontend sent `"scope":
"dataset"`, Pydantic rejected the request.
2. **Frontend**: The `extractRaptorConfigExt` utility in
`web/src/hooks/parser-config-utils.ts` treated `scope` as an unknown
field and moved it into the nested `ext` object. Consequently, the
backend could not read `raptor_config.get("scope", "file")` correctly,
so the default `"file"` was always used.

## Changes
- Added `scope: Literal["file", "dataset"]` to the backend
`RaptorConfig` model with a default of `"file"`.
- Added `scope` to the known-field whitelist in the frontend
`extractRaptorConfigExt` helper so it is transmitted as a top-level
raptor field instead of being buried in `ext`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-29 18:46:28 +08:00
Wang Qi
1b84892e3a Fix delete graph (#14484)
### What problem does this PR solve?

Fix delete graph

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 18:09:10 +08:00
Wang Qi
3991bdfaf5 Fix graph task type (#14475)
### What problem does this PR solve?
Fix graph task type

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:05:56 +08:00
Jin Hai
bb05a8bd7e Update create model instance command (#14441)
### What problem does this PR solve?

1. support command:

```
RAGFlow(user)> create provider 'vllm' instance 'test' key 'test-key' url 'base-url' region 'abc';
SUCCESS
RAGFlow(user)> list instances from 'vllm';
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| apiKey   | extra                                  | id                               | instanceName | providerID                       | status |
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| test-key | {"base_url":"base-url","region":"abc"} | 40213c89430311f1a7cf38a74640adcc | test         | b4d40e6142d311f1a4f938a74640adcc | enable |
+----------+----------------------------------------+----------------------------------+--------------+----------------------------------+--------+
```
2. support add vllm model
```
RAGFlow(user)> add model 'Qwen/Qwen2-0.5B' to provider 'vllm' instance 'test' with tokens 131072 chat;
SUCCESS
```
3. add vllm chat

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 17:05:08 +08:00
qinling0210
486ca463aa Port PR14454 to GO (PruneDeletedChunks) (#14463)
### What problem does this PR solve?

Port PR14454 to GO (PruneDeletedChunks)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:04:22 +08:00
Magicbook1108
e0b3070012 Feat: enable sync deleted files for Gmail && fix google drive issues (#14462)
### What problem does this PR solve?

Feat: enable sync deleted files for Gmail && fix google drive issues

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: bill <yibie_jingnian@163.com>
Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-29 17:03:56 +08:00
balibabu
a736948493 Fix: Clicking the button in the bottom-right corner of the /chats/widget page fails to display the dialog box. (#14465)
### What problem does this PR solve?

Fix: Clicking the button in the bottom-right corner of the
`/chats/widget` page fails to display the dialog box.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 17:03:33 +08:00
Wang Qi
6afb1957d8 Fix query param type (#14471)
### What problem does this PR solve?

Fix query param type

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 16:53:28 +08:00
Wang Qi
9690923516 Fix delete graphrag raptor (#14469)
### What problem does this PR solve?

Fix delete graphrag raptor

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 16:47:42 +08:00
Haruko386
decf673049 Go: implement provider: volcengine (#14460)
### What problem does this PR solve?

implement `volcengine` provider 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 15:45:08 +08:00
Wang Qi
b684c89950 Add backward compat APIs (#14427)
### What problem does this PR solve?

Add backward compat APIs:

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 15:15:49 +08:00
buua436
c08ced09a7 Fix: add retrieval fallback comments (#14457)
### What problem does this PR solve?

add retrieval fallback comments

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 14:44:31 +08:00
qinling0210
f3c232cf47 Remove model_bundle.go, modify chat_session.go (#14458)
### What problem does this PR solve?

Remove model_bundle.go, modify chat_session.go

### Type of change

- [x] Refactoring
2026-04-29 14:44:12 +08:00
balibabu
ce933357c6 Fix: Dataset: When configuring the "general chunk method," options such as chunk size and parent-child slicing are unavailable. (#14459)
### What problem does this PR solve?

Fix: Dataset: When configuring the "general chunk method," options such
as chunk size and parent-child slicing are unavailable.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-29 14:37:48 +08:00
buua436
a7ce1b1677 Fix: prune deleted doc chunks from retrieval (#14454)
### What problem does this PR solve?

prune deleted doc chunks from retrieval

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-29 13:03:09 +08:00
Jin Hai
b493a33316 Go: update chat URL (#14453)
### What problem does this PR solve?

Update the URL to: /api/v1/chat/completions

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-29 11:45:06 +08:00
Magicbook1108
3b7a6eaa6c Feat: sync deleted files in Bitbucket (#14450)
### What problem does this PR solve?

Feat: sync deleted files in Bitbucket

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-29 11:29:17 +08:00
Paras Sondhi
74fa54f122 feat(google-drive): optimize memory payload and enable sync deletion (#14372)
**Addresses the Google Drive integration for #14362**

This PR completely overhauls the Google Drive sync logic to accurately
detect remote deletions, while drastically reducing the memory footprint
during the snapshot phase.

### What changed under the hood:

* **Killed the memory bloat:** Swapped out the massive document
dictionary objects for a lightweight `collections.namedtuple` (`SlimDoc
= namedtuple('SlimDoc', ['id'])`). This prevents RAM spikes during
`retrieve_all_slim_docs_perm_sync` on massive enterprise drives.
* **Flawless downstream integration:** The `SlimDoc` object relies on
simple duck typing. It perfectly delivers the `.id` attribute required
by `ConnectorService.cleanup_stale_documents_for_task`, meaning your
core `hash128` vector cleanup logic runs natively without modification.
* **Fixed the Shared Drive blindspot:** The standard API query was
missing team folders. Injected the `corpora="allDrives"` and
`includeItemsFromAllDrives=True` override flags so the connector now
accurately maps state across both personal workspaces and organizational
Shared Drives.

### Testing:
Isolated the Google API retrieval logic locally to prove the `SlimDoc`
mapping works and correctly registers state drops when a file is trashed
remotely.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Performance Improvement
2026-04-29 10:04:36 +08:00
Stephen Hu
345bec812d refactor: improve QwenRerank logic (#14388)
### What problem does this PR solve?

improve QwenRerank logic

### Type of change

- [x] Refactoring
2026-04-28 20:17:34 +08:00
Magicbook1108
0d18b293f5 Fix: enable sync deleted file in airtable (#14438)
### What problem does this PR solve?

Fix: enable sync deleted file in airtable

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 20:09:08 +08:00
Magicbook1108
926efbd29b Fix: update based on #14436 (#14440)
### What problem does this PR solve?

Fix: update based on #14436

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 20:08:42 +08:00
euvre
35f6d81b73 Refactor: migrate chunk retrieval_test and knowledge_graph to REST API endpoints (#14402)
### What problem does this PR solve?

## Summary

Migrate two web API endpoints to REST-style HTTP API endpoints,
following the pattern established in #14222:

| Old Endpoint | New Endpoint |
|---|---|
| `POST /v1/chunk/retrieval_test` | `POST
/api/v1/datasets/<dataset_id>/search` |
| `GET /v1/chunk/knowledge_graph` | `GET
/api/v1/datasets/<dataset_id>/graph` |
2026-04-28 20:00:26 +08:00
Magicbook1108
85575259ac Fix: google authentication - gmail && google-drive (#14422)
### What problem does this PR solve?

Fix: google authentication - gmail && google-drive

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 18:09:02 +08:00
qinling0210
dcce864d4c Simplify Encode (#14437)
### What problem does this PR solve?

Simplify Encode

### Type of change

- [x] Refactoring
2026-04-28 18:07:42 +08:00
Magicbook1108
d532151be0 Feat: more model for paddle (#14436)
### What problem does this PR solve?

Feat: more model for paddle
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 18:07:00 +08:00
Haruko386
4e5a093ac5 Go: implement provider: Moonshot (#14433)
### What problem does this PR solve?

implement `Moonshot` provider

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 18:06:25 +08:00
Jack
c330005659 Fix: document level auto metadata config missing after save (#14421)
### What problem does this PR solve?

Steps to re-produce (existing bug before API migration):

create a new dataset
upload a file 
click on "General" in "Parse" column and then click on "switch or
configure ingestion pipeline"
click on "Settings" (at right of "Auto metadata")
click "Add" to add new metadata
click on "Save"
re-open "Settings" and the newly added metadata is not there

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 17:09:23 +08:00
buua436
e6e80041f5 Fix: agent toolcall null response & schema validation & DeepSeek think history (#14425)
### What problem does this PR solve?
agent toolcall null response & schema validation & DeepSeek think
history

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 17:09:08 +08:00
Jin Hai
f670913bb4 Refactor model type to model class (#14426)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 16:05:15 +08:00
Jin Hai
7c25870923 Go: update db model (#14423)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 16:04:55 +08:00
Magicbook1108
18fbfafca6 Feat: enable sync deleted files for more connectors (#14353)
### What problem does this PR solve?

Feat: enable sync delted files for connectors

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-28 15:07:14 +08:00
NeedmeFordev
0df65d358a Fix case-insensitive matching for manual meta_data_filter in / not in list values (#14397)
## Summary

Fixes case-asymmetric matching for manual `meta_data_filter` when using
**`in`** / **`not in`** with a **list** `value`. Document metadata
strings were lowercased, but list elements were not, so values like
`"F2"` failed to match `["F2", "F11"]` even though **`=`** behaved
correctly.

Closes #14389

## Changes

- **`common/metadata_utils.py`**: For **`in`** / **`not in`**, normalize
string elements when `value` and/or `input` is a list, consistent with
scalar string lowercasing.
- **`test/unit_test/common/test_metadata_filter_operators.py`**:
Regression tests for list `value` case-insensitivity and **`not in`**.

## Type of change

- [x] Bug fix (non-breaking)
2026-04-28 14:51:48 +08:00
Idriss Sbaaoui
2a37562791 Fix manual naive parser position extraction fallback (#14420)
### What problem does this PR solve?
This PR fixes a regression where Manual pipeline + Naive (Plain Text)
PDF parsing crashed with `AttributeError: 'PlainParser' object has no
attribute 'extract_positions'` in `rag/app/manual.py`.
fixes #14411 
### Type of change:
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 14:21:30 +08:00
Jin Hai
ae420f6358 Go: fix compilation (#14418)
### What problem does this PR solve?

Add methods to volcengine

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 13:21:05 +08:00
qinling0210
effc84a042 Refactor model in GO (#14398)
### What problem does this PR solve?

Refactor model in GO

### Type of change

- [x] Refactoring
2026-04-28 12:59:01 +08:00
Wang Qi
5885691c68 Always return success if no such task id (#14417)
### What problem does this PR solve?

Always return success if no such task id to follow existing code logic.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:55:24 +08:00
buua436
444e564329 Fix: align chat recommendation and thumbup APIs (#14413)
### What problem does this PR solve?
align chat recommendation and thumbup APIs
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:55:16 +08:00
buua436
7a70a0fd85 Fix: preserve infinity available_int zero filter (#14416)
### What problem does this PR solve?

preserve infinity available_int zero filter

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-28 12:54:32 +08:00
Jin Hai
819257f257 Go: add volcengine (#14409)
### What problem does this PR solve?

1. Refactor server_main
2. Add volcengine

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 12:12:58 +08:00
Jack
2d522ccb36 Fix: thumbnails issue in chat (#14415)
[Uploading part_4-13.pdf…]()
### What problem does this PR solve?

In chat, the thumbnails didn't display correctly

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)

Steps to reproduce:
1. create dataset and upload a file (see attached)
2. parse the document
3. once parsing completed, create a chat and associate it with the
dataset
4. ask a question (DAP VS DAPE comparison)
5. check result
2026-04-28 11:39:29 +08:00
writinwaters
0cf105da8d Doc: Added a database schema and migration guide. (#14404)
### What problem does this PR solve?

Added a database schema and migration guide.

### Type of change


- [x] Documentation Update
2026-04-28 09:54:33 +08:00
Jack
c81081f8ef Refactor: Doc change parser (#14327)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/change_parser
HTTP API: PATCH /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API
PATCH /api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-27 23:42:57 +08:00
Jack
872ff08304 Fix: add executor.shutdown (#14403)
### What problem does this PR solve?

Add executor shutdown in finally clause to free resources.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 22:38:43 +08:00
Jack
c5116b90e5 Refactor: migrate document thumbnails API (#14344)
### What problem does this PR solve?

Before migration: GET /v1/document/thumbnails
After migration:  GET /api/v1/thumbnails

### Type of change

- [x] Refactoring
2026-04-27 21:29:09 +08:00
Jack
49912a156e Refactor: migrate document run api (#14351)
### What problem does this PR solve?

Before migration: POST /v1/document/run
After migration: POST /api/v1/documents/ingest/

### Type of change

- [x] Refactoring
2026-04-27 21:25:58 +08:00
Jin Hai
965717c4fb Go: add new provider: google (#14395)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-27 20:35:47 +08:00
Jack
343bda1119 Refactor: deco document upload_and_parse API (#14366)
### What problem does this PR solve?

remove unused "POST /v1/document/upload_and_parse"

### Type of change

- [x] Refactoring
2026-04-27 20:35:00 +08:00
euvre
d78013964a tests: add missing HTTP API tests for dataset management endpoints removed in #14222 (#14390)
### What problem does this PR solve?

### Summary

PR #14222 consolidated KB (web) API endpoints into RESTful Dataset
(HTTP) API endpoints and deleted the web API test suite under
`test_web_api/test_kb_app/` and `test_web_api/test_document_app/`. While
most test coverage was migrated to the HTTP API test suite, some tests
were not ported over. This PR adds back the missing coverage.

### Route migration reference

| Old Web API | New HTTP API | Missing tests |
|---|---|---|
| `POST /v1/kb/update_metadata_setting` | `PUT
/api/v1/datasets/<id>/metadata/config` | auth & error paths |
| `GET /api/v1/datasets/<id>/auto_metadata` | `GET
/api/v1/datasets/<id>/metadata/config` | auth & CRUD |
| `PUT /api/v1/datasets/<id>/auto_metadata` | `PUT
/api/v1/datasets/<id>/metadata/config` | auth & CRUD |
| `GET /v1/kb/<kb_id>/basic_info` | `GET
/api/v1/datasets/<id>/ingestions/summary` | covered |
| `POST /v1/kb/list_pipeline_logs` | `GET
/api/v1/datasets/<id>/ingestions` | edge cases missing |

### Changes

#### `test_file_management_within_dataset/test_metadata_config.py` (new,
10 tests)

Covers `GET/PUT /datasets/<id>/metadata/config` (migrated from
`test_kb_tags_meta.py`'s `test_update_metadata_setting` and
`test_document_metadata.py`'s negative tests):
- Authorization for dataset metadata config GET/PUT
- Authorization for document metadata config PUT
- Success, invalid dataset, missing payload, not found scenarios

#### `test_dataset_management/test_ingestion_logs.py` (extended, +2
tests)

Covers `GET /datasets/<id>/ingestions` edge cases (migrated from
`test_kb_pipeline_tasks.py`):
- Missing dataset ID
- Abnormal date filter

### Type of change

- [x] Other: Test coverage improvement

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 20:01:28 +08:00
Jack
a536980e22 Refactor: Doc batch change status (#14337)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/change_status

After consolidation, Restful API
POST /api/v1/datasets/<dataset_id>/documents/batch-update-status 

### Type of change

- [x] Refactoring
2026-04-27 20:00:23 +08:00
buua436
c949096db0 Refactor: optimize agent reset conversation variable defaults (#14401)
### What problem does this PR solve?
optimize agent reset conversation variable defaults
### Type of change
- [x] Refactoring
2026-04-27 19:57:56 +08:00
Wang Qi
488c3ef6a3 Add task API (#14393)
### What problem does this PR solve?

Add task API

### Type of change

- [x] Refactor
2026-04-27 19:16:37 +08:00
buua436
82313020c7 Refa: align list operations and strict mode (#14387)
### What problem does this PR solve?

align list operations and strict mode

### Type of change
- [x] Refactoring
2026-04-27 19:13:00 +08:00
Jack
c1941fd503 Refactor: deco doc-parse API that is not used any more (#14367)
### What problem does this PR solve?

Delete un-used API "POST /v1/document/parse"

### Type of change

- [x] Refactoring
2026-04-27 18:54:49 +08:00
buua436
4f6651968a Fix: prioritize explore session ID and reset default conversation variables (#14399)
### What problem does this PR solve?

 prioritize explore session ID and reset default conversation variables

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 18:52:40 +08:00
mginfn
10e28e5c5f Helm template ragflow.yaml: fix nginx-config-volume mountPath according to Dockerfile v0.25.0 (#14361)
### What problem does this PR solve?

Dockerfile v0.25.0 expects nginx conf at path
/etc/nginx/ragflow.conf.python, see
[Dockerfile#L200](ca01c7a745/Dockerfile (L200))
However current helm template mount the conf at path
/etc/nginx/ragflow.conf causing runtime error at startup time.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Mauro Gattari <mauro.gattari@infn.it>
2026-04-27 18:51:55 +08:00
buua436
0f2778efe7 Fix: support release in agent update api (#14396)
### What problem does this PR solve?

support release in agent update api

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 17:35:35 +08:00
Jack
61a24a2c14 Refactor: migrate doc upload info used in chat (#14359)
### What problem does this PR solve?

Before migration: POST /v1/document/upload_info/
After migration: POST /api/v1/documentss/upload/

### Type of change

- [x] Refactoring
2026-04-27 16:58:42 +08:00
Zhichang Yu
c446c403de perf: lazy img_np loading and chunked parse_into_bboxes for large PDFs (#14385)
## Summary

- **Lazy img_np loading**: `np.array(img)` is now deferred until the
first OCR text extraction is actually needed, avoiding unnecessary
memory allocation for pages that already have text.
- **Chunked parse_into_bboxes**: Large PDFs (>50 pages, configurable via
`PDF_PARSER_PAGE_BATCH_SIZE`) are processed in batches. Each chunk's
boxes are normalized with `_to_global_boxes` to produce globally
consistent page numbers and position tags.
- **DLA early init**: Move remote-client initialization before model
loading in `LayoutRecognizer.__init__` so `DEEPDOC_URL` (or legacy
`TENSORRT_DLA_SVR`) short-circuits unnecessary model download for parser
containers relying on remote inference.
- **Fix outline regression**: Restore `self.outlines =
extract_pdf_outlines(fnm)` in `parse_into_bboxes`; this was dropped
during refactoring and is required by downstream `remove_toc` and
metadata handling in `rag/flow/parser/parser.py`.

## Test plan

- [ ] Small PDF (<=50 pages): verify parse succeeds and `self.outlines`
is populated
- [ ] Large PDF (>50 pages): verify chunked processing produces globally
consistent page numbers
- [ ] With `DEEPDOC_URL` set: verify remote DLA client is used and local
model is not downloaded
- [ ] With legacy `TENSORRT_DLA_SVR` set: verify backward compatibility

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-27 16:52:43 +08:00
Idriss Sbaaoui
4303be223f Fix metadata parsing regression for upgraded v0.24 datasets (#14383)
### What problem does this PR solve?

This PR fixes issue #14371 where file parsing failed after upgrading
from v0.24.0 to v0.25.0, because metadata config could be a JSON Schema
object but was handled like a list and later caused `KeyError:
'properties'`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-27 16:18:06 +08:00
Wang Qi
d88f7ac8d2 Remove evaluation_app.py and kb_app.py (#14394)
### What problem does this PR solve?

Delete not used APIs

### Type of change

- [x] Refactoring
2026-04-27 16:08:54 +08:00
Jack
290f0294d6 Refactor: migrate artifact API (#14348)
### What problem does this PR solve?

Before migration: GET /v1/document/artifact/<filename>
After migration:  GET /api/v1/documents/artifact/<filename>

### Type of change

- [x] Refactoring
2026-04-27 15:19:41 +08:00
euvre
2846a93998 Fix: Remove hardcoded page limits causing parsing failures on large PDFs (>300 pages) (#14382)
### What problem does this PR solve?

Fixes #14196

## Problem

When using DeepDOC to parse large PDFs (over 1000 pages), the parser
silently truncated processing at 300 pages due to a hardcoded default
`page_to=299` in `RAGFlowPdfParser.__images__()`. This caused:

- **Errors** on pages beyond the limit
- **Poor image quality** as the parser attempted to compensate with
missing page data
- **Inconsistent chunk splitting** between full PDF imports and partial
imports

Additionally, the codebase scattered magic numbers (`299`, `600`,
`10000`, `100000`, `100000000`, `10000000000`, `10**9`) across 22 files
as sentinel values for "parse all pages", making future maintenance
error-prone.

## Root Cause

```python
# deepdoc/parser/pdf_parser.py (before)
def __images__(self, fnm, zoomin=3, page_from=0, page_to=299, callback=None):
    # Only the first 300 pages were rendered; everything beyond was silently dropped
```

While most callers in `rag/app/*.py` correctly passed `to_page=100000`,
the base class `RAGFlowPdfParser.__call__()` and `parse_into_bboxes()`
invoked `__images__` **without** forwarding `page_from`/`page_to`,
falling back to the restrictive default of 299.

## Solution

### 1. Define constants in `common/constants.py`

```python
MAXIMUM_PAGE_NUMBER = 100000                        # Used by the parsing layer
MAXIMUM_TASK_PAGE_NUMBER = MAXIMUM_PAGE_NUMBER * 1000  # Used by the task/DB layer
```

### 2. Replace all hardcoded sentinel values

| Layer | Files Changed | Old Values | New Value |
|---|---|---|---|
| **Deepdoc parsers** | `pdf_parser.py`, `mineru_parser.py`,
`docling_parser.py`, `opendataloader_parser.py`, `paddleocr_parser.py`,
`docx_parser.py` | `299`, `600`, `10**9`, `100000000` |
`MAXIMUM_PAGE_NUMBER` |
| **Chunk parsers** | `naive.py`, `book.py`, `qa.py`, `one.py`,
`manual.py`, `paper.py`, `presentation.py`, `laws.py`, `resume.py`,
`email.py`, `table.py` | `100000`, `10000`, `10000000000` |
`MAXIMUM_PAGE_NUMBER` |
| **Task/DB layer** | `db_models.py`, `task_service.py`,
`document_service.py`, `file_service.py` | `100000000` |
`MAXIMUM_TASK_PAGE_NUMBER` |

### 3. Fix `parse_into_bboxes()` missing parameters

Added `from_page`/`to_page` parameters to `parse_into_bboxes()` so that
the `rag/flow/parser/parser.py` DeepDOC path no longer falls back to the
restrictive default.

## Files Changed (22)

- `common/constants.py`
- `deepdoc/parser/pdf_parser.py`
- `deepdoc/parser/mineru_parser.py`
- `deepdoc/parser/docling_parser.py`
- `deepdoc/parser/opendataloader_parser.py`
- `deepdoc/parser/paddleocr_parser.py`
- `deepdoc/parser/docx_parser.py`
- `rag/app/naive.py`
- `rag/app/book.py`
- `rag/app/qa.py`
- `rag/app/one.py`
- `rag/app/manual.py`
- `rag/app/paper.py`
- `rag/app/presentation.py`
- `rag/app/laws.py`
- `rag/app/resume.py`
- `rag/app/email.py`
- `rag/app/table.py`
- `api/db/db_models.py`
- `api/db/services/task_service.py`
- `api/db/services/document_service.py`
- `api/db/services/file_service.py`

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 14:57:20 +08:00
Jin Hai
c3eac4103a Go: aliyun model provider (#14379)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-27 14:53:33 +08:00
buua436
0b46ab07c5 Refa: restore openai-compatible chat completions api (#14380)
### What problem does this PR solve?
restore openai-compatible chat completions api
### Type of change

- [x] Refactoring
2026-04-27 14:02:19 +08:00
LeonTung
6a23dfeec1 chore(CLAUDE.md): add shared UI component lock convention to CLAUDE.md (#14381)
### What problem does this PR solve?

AI coding agents (Claude, Copilot, etc.) tend to directly edit files in
`src/components/ui/` when asked to tweak styles or add props, treating
them like ordinary feature code. This silently breaks the shared
component library that both shadcn primitives and project-authored
common components live in.

This PR adds a `Shared UI Component Lock` convention to `web/CLAUDE.md`
to instruct AI agents to treat the entire `src/components/ui/` directory
as read-only. Any customization must be done via wrappers or composition
outside the directory; exceptions require explicit user approval.

### Type of change
- [x] Other (please describe): Update `CLAUDE.md`
2026-04-27 12:03:32 +08:00
yuch85
0d87cecae2 feat: persist PDF bookmark outline as document metadata (#13287)
## Summary

PDF files often contain a bookmark/outline tree (table of contents built
into the file by the authoring tool). RAGFlow's `pdf_parser.outlines`
already extracts these `(title, depth)` tuples via pypdf, but they are
used ephemerally during chunking (`manual` parser uses them for
hierarchy detection) and then discarded.

This PR persists the outline as `doc.meta_fields["outline"]` — a JSON
array of `{"title": str, "depth": int}` objects — so downstream features
can use the structural information.

### Why this matters

- **Complementary to `toc_extraction`** — the existing `toc_extraction`
feature uses LLM calls to generate a TOC and only works for the `naive`
parser. The raw PDF outline is free (already extracted by pypdf), works
for all parsers, and captures the author's original document structure.
- **Document navigation** — frontends can render a clickable TOC from
the outline
- **Entity extraction** — the outline provides a structural map for
identifying document sections and key topics
- **Search result context** — knowing which section a chunk belongs to
helps users evaluate relevance

### Changes

| File | Change | LOC |
|------|--------|-----|
| `rag/app/naive.py` | Attach `pdf_parser.outlines` as `__outline__` on
first chunk dict | ~7 |
| `rag/app/manual.py` | Same for the manual parser | ~5 |
| `rag/svr/task_executor.py` | Extract `__outline__`, persist via
`DocMetadataService.update_document_metadata()` | ~12 |

### Design decisions

- **Transient key pattern**: The outline is passed from parser →
task_executor via `__outline__` on the first chunk dict, then removed
before indexing. This follows the same pattern as `metadata_obj` for
LLM-generated metadata.
- **No schema changes**: Uses the existing `meta_fields` JSON column on
the document table.
- **Graceful degradation**: If a PDF has no outline (common for scanned
docs), nothing is stored. If persistence fails, it logs a warning and
continues — parsing is not interrupted.

### Backward compatibility

- **Fully backward compatible** — no existing fields, behavior, or
schemas changed
- PDFs without outlines are unaffected
- Existing `meta_fields` data is preserved (merged, not overwritten)

## Test plan

- [ ] Parse a PDF with bookmarks (e.g. any multi-chapter document),
verify `meta_fields["outline"]` is populated
- [ ] Parse a PDF without bookmarks, verify no errors and no outline key
in meta_fields
- [ ] Verify existing `meta_fields` data is preserved (not overwritten)
when outline is added
- [ ] Verify `manual` parser also persists outlines
- [ ] Verify outline JSON structure: `[{"title": "Chapter 1", "depth":
0}, ...]`

Related: #9921 (Deterministic Document Access Layer)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuch85 <yuch85.1@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-04-27 11:57:06 +08:00
euvre
f3b7d55a1e fix: handle Infinity table-not-exist error (3022) in update() methods (#14153)
### What problem does this PR solve?

## Summary

Closes #6102

When using Infinity as the document store engine (GPU version), calling
`update()` on a non-existent table throws an unhandled
`InfinityException` with error code 3022 (`TABLE_NOT_EXIST`). This
causes users to see a raw "3022" error when clicking on a parsed
document.

## Root Cause

The `update()` methods in both `rag/utils/infinity_conn.py` and
`memory/utils/infinity_conn.py` call `db_instance.get_table(table_name)`
without catching `InfinityException`. In contrast, other CRUD methods
(`insert`, `delete`, `search`) all handle this exception gracefully:

| Method   | Handles table-not-exist? | Behavior |
|----------|--------------------------|----------|
| `insert`  |  Yes | Auto-creates the table |
| `search`  |  Yes | Skips the table |
| `delete`  |  Yes | Returns 0 |
| `update`  |  **No** | Crashes with 3022 |

Additionally, `api/apps/document_app.py` worked around this with a
fragile string match (`"3022" in msg`) to detect the error.

## Changes

- **`rag/utils/infinity_conn.py`**: Catch `InfinityException` in
`update()`. When `TABLE_NOT_EXIST` is detected, log a warning and return
`False` — consistent with `delete()`.
- **`memory/utils/infinity_conn.py`**: Apply the same fix to its
`update()` method.
- **`api/apps/document_app.py`**: Remove the fragile `"3022"`
string-matching workaround. Table-not-exist is now handled by the `if
not ok` path with an improved error message.

### Type of change

- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 11:52:22 +08:00
euvre
33bb464ce3 fix: skip canvas SSE fetch in chat shared page to eliminate spurious 103 error (#14190)
## What does this PR do?

Fixes the `hint : 103 Only owner of canvas authorized for this
operation` error that appears when opening a **Chat** shared link
(`/chats/share?shared_id=...&from=chat`).

## Root Cause

The Chat shared page (`web/src/pages/next-chats/share/index.tsx`)
unconditionally calls `useFetchFlowSSE()`, which requests
`/api/canvas/getsse/{sharedId}`. This is an Agent Canvas endpoint that
validates canvas ownership. When sharing a **Chat** dialog (not an
Agent):

1. `sharedId` is a `dialog_id`, not a `canvas_id`
2. The API token's `tenant_id` doesn't match any canvas owner
3. The backend returns `code: 103, message: "Only owner of canvas
authorized for this operation."`
4. The global error interceptor in `request.ts` displays it as a
notification: `hint : 103 Only owner of canvas authorized for this
operation.`

## Changes

- **`web/src/hooks/use-agent-request.ts`**: Added an `enabled` parameter
to `useFetchFlowSSE` so callers can conditionally skip the query.
- **`web/src/pages/next-chats/share/index.tsx`**: Only enable
`useFetchFlowSSE` when `from === SharedFrom.Agent`. For Chat shares, the
hook is disabled, avoiding the unnecessary canvas API call entirely.

## Related Issue

Closes #14115

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 11:27:39 +08:00
yuch85
3ad3241ae0 feat: persist RAPTOR layer metadata on summary chunks (#13286)
## Summary

RAPTOR's recursive clustering builds a `layers` list tracking
`(start_idx, end_idx)` boundaries per level, but currently discards this
information — only the flat `chunks` list is returned. This makes it
impossible to distinguish leaf-level summaries from top-level ones.

This PR:
- Returns `(chunks, layers)` tuple from `raptor.py`'s `__call__`
- Annotates each RAPTOR summary chunk with `raptor_layer_int` (1 = first
summary level, 2 = summary-of-summaries, etc.)
- Adds `raptor_layer_int` to `infinity_mapping.json` (Elasticsearch
handles it via existing `*_int` dynamic template)

### Why this matters

Downstream features need to know which RAPTOR layer a summary belongs
to:
- **Retrieving the top-level document summary** for entity extraction,
search snippets, or document comparison
- **Filtering by abstraction level** — users may want only high-level
summaries or only leaf-level cluster summaries
- **RAPTOR recall quality** — #10951 reports summaries not being
recalled for definition queries; layer metadata enables targeted
retrieval

### Changes

| File | Change | LOC |
|------|--------|-----|
| `rag/raptor.py` | Return `(chunks, layers)` tuple | ~3 |
| `rag/svr/task_executor.py` | Build `chunk_layer` mapping, set
`raptor_layer_int` | ~12 |
| `conf/infinity_mapping.json` | Add `raptor_layer_int` integer field |
~1 |

### Backward compatibility

- **Additive only** — no existing fields or behavior changed
- Existing RAPTOR chunks continue to work (they'll have
`raptor_layer_int = 0` by default)
- New RAPTOR chunks get layer metadata automatically

## Test plan

- [ ] Parse a document with RAPTOR enabled, verify `raptor_layer_int` is
set on indexed chunks
- [ ] Verify `raptor_layer_int` values increase with abstraction level
(layer 1 < layer 2 < ...)
- [ ] Verify existing RAPTOR deletion (`delete by raptor_kwd`) still
works
- [ ] Verify Infinity backend accepts the new field

Fixes #7488
Related: #4104, #11191, #10951

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuch85 <yuch85.1@gmail.com>
Co-authored-by: Wang Qi <wangq8@outlook.com>
2026-04-27 10:20:46 +08:00
buua436
a9e5724b46 Refa: unify document create flows under REST documents API (#14345)
### What problem does this PR solve?

unify document create flows under REST documents API

### Type of change

- [x] Refactoring
2026-04-27 10:18:16 +08:00
euvre
4dcc42e0e1 feat(api): add unified index API and dataset management endpoints (#14222)
### What problem does this PR solve?

## Summary

Refactor the dataset API layer into a clean service/REST separation
pattern, add a unified `/index` API for graph/raptor/mindmap operations,
and introduce several new dataset management endpoints with full test
coverage.

## Changes

### Service Layer (`dataset_api_service.py`)

- Added `trace_index(dataset_id, tenant_id, index_type)` — unified trace
function for all index types
- Added `run_index`, `delete_index` service functions
- Added `get_dataset`, `get_ingestion_summary`, `list_ingestion_logs`,
`get_ingestion_log`
- Added `run_embedding`, `list_tags`, `aggregate_tags`, `delete_tags`,
`rename_tag`
- Added `get_flattened_metadata`, `get_auto_metadata`,
`update_auto_metadata`

### REST API Layer (`dataset_api.py`)

**New unified routes:**

| Method | Route | Description |
|--------|-------|-------------|
| POST | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Run index
task |
| GET | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Trace index
task |
| DELETE | `/datasets/<id>/<index_type>` | Delete index |
| GET | `/datasets/<id>` | Get dataset details |
| GET | `/datasets/<id>/ingestions/summary` | Ingestion summary |
| GET | `/datasets/<id>/ingestions` | List ingestion logs |
| GET | `/datasets/<id>/ingestions/<log_id>` | Get single ingestion log
|
| POST | `/datasets/<id>/embedding` | Run embedding |
| GET | `/datasets/<id>/tags` | List tags |
| GET | `/datasets/tags/aggregation` | Aggregate tags across datasets |
| DELETE | `/datasets/<id>/tags` | Delete tags |
| PUT | `/datasets/<id>/tags` | Rename tag |
| GET | `/datasets/metadata/flattened` | Get flattened metadata |
| GET/PUT | `/datasets/<id>/metadata/config` | New metadata config path
|

**Removed routes (replaced by unified `/index`):**

- `POST /datasets/<id>/mindmap`
- `GET /datasets/<id>/mindmap`

**Preserved legacy routes (backward compatibility):**

- `/run_graphrag`, `/trace_graphrag`, `/run_raptor`, `/trace_raptor`
- `/auto_metadata` GET/PUT

### Test Suite

- Updated `common.py` helpers: added `trace_index`, removed
`run_mindmap`/`trace_mindmap`
- Added 7 new test files with 39 test cases total:

| Test File | Cases |
|-----------|-------|
| `test_get_dataset.py` | 4 |
| `test_ingestion_summary.py` | 2 |
| `test_ingestion_logs.py` | 5 |
| `test_index_api.py` | 14 |
| `test_embedding.py` | 2 |
| `test_tags.py` | 8 |
| `test_flattened_metadata.py` | 4 |

- Deleted `test_mindmap_tasks.py` (covered by unified index tests)

## Design Decisions

1. **Unified `/index?type=...`** — single endpoint replaces 3 separate
route pairs for graph/raptor/mindmap
2. **Backward compatibility** — old routes (`/run_graphrag`,
`/run_raptor`, `/auto_metadata`) preserved alongside new paths
3. **`_VALID_INDEX_TYPES = {"graph", "raptor", "mindmap"}`** — input
validation via constant set
4. **`_INDEX_TYPE_TO_TASK_ID_FIELD`** — maps index type to KB model task
ID field for clean dispatch

## Files Changed

- `api/apps/restful_apis/dataset_api.py`
- `api/apps/services/dataset_api_service.py`
- `sdk/python/ragflow_sdk/modules/dataset.py`
- `test/testcases/test_http_api/common.py`
- `test/testcases/test_http_api/test_dataset_management/` (7 new files)
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 09:38:01 +08:00
Xing Hong
fb95136f39 Fix: validate URL scheme and resolved IP before crawling to prevent SSRF (#14090)
### What problem does this PR solve?

The POST /upload_info?url=<url> endpoint accepted a user-supplied URL
and passed it directly to AsyncWebCrawler without any validation. There
were no restrictions on URL scheme, destination hostname, or resolved IP
address. This allowed any authenticated user to instruct the server to
make outbound HTTP requests to internal infrastructure — including RFC
1918 private networks, loopback addresses, and cloud metadata services
such as http://169.254.169.254 — effectively using the server as a proxy
for internal network reconnaissance or credential theft.

This PR adds an SSRF guard (_validate_url_for_crawl) that runs before
any crawl is initiated. It enforces an allowlist of safe schemes
(http/https), resolves the hostname at validation time, and rejects any
URL whose resolved IP falls within a private or reserved network range.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-25 14:30:15 +08:00
wdeveloper16
78188ce9e9 Feat: add OpenDataLoader PDF parser backend (#14058) (#14097)
### What problem does this PR solve?

Closes #14058.

RAGFlow supports multiple PDF parsing backends (DeepDOC, MinerU,
Docling, TCADP, PaddleOCR). This PR adds **OpenDataLoader**
([opendataloader-project/opendataloader-pdf](https://github.com/opendataloader-project/opendataloader-pdf))
as a new optional backend, giving users a deterministic, local-first
alternative with competitive table extraction accuracy.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---

### Changes

#### Backend
- `deepdoc/parser/opendataloader_parser.py` — new `OpenDataLoaderParser`
class inheriting `RAGFlowPdfParser`. Implements `check_installation()`
(guards Python package + Java 11+ runtime), `parse_pdf()` with
JSON-first extraction (heading/paragraph/table/list/image/formula) and
Markdown fallback, position-tag generation compatible with the shared
`@@page\tx0\tx1\ty0\ty1##` format, and temp-dir lifecycle with cleanup.
- `rag/app/naive.py` — new `by_opendataloader()` wrapper, registered in
`PARSERS` dict, added to `chunk_token_num=0` override list.
- `rag/flow/parser/parser.py` — `"opendataloader"` branch in the
pipeline PDF handler + check validation list.

#### Infrastructure
- `docker/entrypoint.sh` — `ensure_opendataloader()` function: opt-in
via `USE_OPENDATALOADER=true`, skips gracefully if Java is not on PATH.

#### Frontend
- `web/src/components/layout-recognize-form-field.tsx` —
`OpenDataLoader` added to `ParseDocumentType` enum and parser dropdown.
Cascades automatically to the pipeline editor's Parser component.

#### Docs
- `docs/guides/dataset/select_pdf_parser.md` — added OpenDataLoader
entry and full env-var reference.

---

### Environment variables

| Variable | Default | Description |
|---|---|---|
| `USE_OPENDATALOADER` | `false` | Set `true` to install
`opendataloader-pdf` on container startup |
| `OPENDATALOADER_VERSION` | latest | Pin the PyPI release (e.g.
`==2.2.1`) |
| `OPENDATALOADER_HYBRID` | _(unset)_ | Enable hybrid AI mode (e.g.
`docling-fast`) |
| `OPENDATALOADER_IMAGE_OUTPUT` | _(unset)_ | `off` / `embedded` /
`external` |
| `OPENDATALOADER_OUTPUT_DIR` | _(tmp)_ | Persistent output dir; temp
dir used + cleaned if unset |
| `OPENDATALOADER_DELETE_OUTPUT` | `1` | `0` to retain intermediate
files for debugging |
| `OPENDATALOADER_SANITIZE` | _(unset)_ | `1` to filter prompt-injection
patterns from output |

---

### Dependencies

- **Runtime**: `opendataloader-pdf` (PyPI, Apache 2.0) — opt-in, not
added to `pyproject.toml` core deps. Installed by
`ensure_opendataloader()` at container startup when
`USE_OPENDATALOADER=true`.
- **System**: Java 11+ on PATH (JVM is the underlying engine). The
installer skips with a warning if `java` is not found.

---

### How to test

**Standalone parser:**
```bash
source .venv/bin/activate
uv pip install opendataloader-pdf
python3 -c "
import sys; sys.path.insert(0, '.')
from deepdoc.parser.opendataloader_parser import OpenDataLoaderParser
p = OpenDataLoaderParser()
print('available:', p.check_installation())
s, t = p.parse_pdf('path/to/test.pdf', parse_method='pipeline')
print(f'sections={len(s)} tables={len(t)}')
"

```
### Benchmark vs Docling
```
file                      parser            secs  sections  tables
----------------------------------------------------------------------
text-heavy.pdf            docling           45.29       148      10
text-heavy.pdf            opendataloader     3.14       559       0
table-heavy.pdf           docling           7.05        76       3
table-heavy.pdf           opendataloader     3.71        90       0
complex.pdf               docling            42.67       114       8
complex.pdf               opendataloader     3.51       180       0
```
2026-04-25 00:33:02 +08:00
Lynn
e22cf333ed Fix: allow search id or _id (#14356)
### What problem does this PR solve?

Allow search id or _id when using es as doc_engine.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 21:38:19 +08:00
Magicbook1108
25089600d0 Feat: introduce minimum type check for pipeline (#14354)
### What problem does this PR solve?

Feat: introduce minimum type check for pipeline

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 21:12:50 +08:00
Jin Hai
1c244df90d Go: add gitee and siliconflow as model provider (#14336)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-24 20:59:30 +08:00
writinwaters
e5cfe7fb8f Doc: Updated a 0.25-specific faq (#14365)
### What problem does this PR solve?

Updated a 0.25 faq.

### Type of change


- [x] Documentation Update
2026-04-24 20:57:32 +08:00
Wang Qi
7fb6a12067 Update API document (#14364)
### What problem does this PR solve?

Update API document

### Type of change

- [ ] Documentation Update
2026-04-24 20:36:47 +08:00
balibabu
3ccd58f28c Fix: The button styles in the PaddleOCR dialog are not applying correctly. (#14350)
### What problem does this PR solve?

Fix: The button styles in the PaddleOCR dialog are not applying
correctly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Copilot <copilot@github.com>
2026-04-24 20:17:01 +08:00
writinwaters
1870c934c6 Refact: Updated rootAsHeadingTip (#14363)
### What problem does this PR solve?

Updated rootASHeadingTip.

### Type of change

- [x] Documentation Update
2026-04-24 20:08:44 +08:00
Idriss Sbaaoui
ca01c7a745 Fix blob sync: skip unsupported files before download (#14357)
### What problem does this PR solve?

Blob storage sync was downloading unsupported files first and rejecting
them later, which wasted bandwidth and made sync slower. This PR skips
unsupported extensions before download and applies `allow_images` in
blob sync. fixes #14338

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 19:22:32 +08:00
Cocoon-Break
620088be2f fix: check isinstance before len in VariableAssigner _remove_first/_remove_last (#14281)
fix: check isinstance before len in VariableAssigner _remove_first/_remove_last
2026-04-24 19:09:44 +08:00
Paras Sondhi
eeb89d604e feat: route docling parsing through native chunking endpoints (#14218)
Resolves #14211

**Background:** Currently, RAGFlow routes all Docling parsing through
the standard `/convert/source` endpoint. For large documents, this
returns massive, unchunked text that exceeds RAGFlow's internal
embedding model context limits, causing pipeline failures.

**Solution:**
This PR updates the `_parse_pdf_remote` ingestion logic in
`docling_parser.py` to prioritize `docling-serve`'s native chunking
endpoints (`/v1/chunk/source` and `/v1alpha/chunk/source`).
- By receiving pre-sliced chunk objects directly from Docling, RAGFlow
natively bypasses token limit overflows.
- Included a graceful fallback mechanism to the standard
`/convert/source` endpoints to maintain backwards compatibility for
users running older versions of the Docling server that return 404s on
the new routes.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 19:03:19 +08:00
Lynn
beb2406b86 Fix: allow use image2text as chat model (#14331)
### What problem does this PR solve?

Allow image2text models (multimodal) to be used as chat models.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 17:58:25 +08:00
buua436
9ad752f497 Refa:migrate agent webhook routes to REST APIs (#14330)
### What problem does this PR solve?

migrate agent webhook routes to REST APIs

### Type of change
- [x] Refactoring
2026-04-24 17:55:53 +08:00
Wang Qi
b8d831c1c3 Fix api user patch verb does not work (#14358)
### What problem does this PR solve?

Fix api user patch verb does not work

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 17:27:41 +08:00
Mukunda Rao Katta
8a2f63e77d docs: fix API key guide typo (#14352)
Fixes a small typo in the RAGFlow API key guide: `This documents
provides` -> `This document provides`.
2026-04-24 16:59:25 +08:00
qinling0210
1473000135 Implement retrieval_test in GO (#14231)
### What problem does this PR solve?

Implement retrieval_test in GO

### Type of change

- [x] Refactoring
2026-04-24 15:30:14 +08:00
Magicbook1108
aadd9a333f Feat: deepseek v4 (#14346)
### What problem does this PR solve?

Feat: deepseek v4
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-24 13:07:59 +08:00
Wang Qi
199fbceb72 Refactor user REST API (#14334)
### What problem does this PR solve?
Refactor user REST API

### Type of change
- [x] Refactoring
2026-04-24 10:25:15 +08:00
RazmikGevorgyan
c41b5e8a5d fix: migrate Langfuse integration from start_generation to start_obse… (#14205)
The Langfuse Python SDK v3+ removed `start_generation()` method.
RagFlow's code called this non-existent method, causing AttributeError
when Langfuse tracing is enabled.

Replace all `start_generation()` calls with
`start_observation(as_type="generation")` which is the correct v4 SDK
API.

Affected files:
- api/db/services/llm_service.py (12 occurrences)
- api/db/services/dialog_service.py (1 occurrence)

Fixes #14204
Related to #9243

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-24 10:03:57 +08:00
Magicbook1108
c74aece63c Feat: Agent api (#14157)
### What problem does this PR solve?

1. **List agents**  
   **Prev API**:  
   - `/v1/canvas/list GET`  
   - `/api/v1/agents GET`  
   **Current API**: `/api/v2/agents GET`

2. **Get canvas template**  
   **Prev API**: `/v1/canvas/templates GET`  
   **Current API**: `/api/v2/agents/templates GET`

3. **Delete an agent**  
   **Prev API**: 
    - `/v1/canvas/rm POST`  
    - `/api/v1/agents/<agent_id> DELETE`
   **Current API**: `/api/v2/agents/<agent_id> DELETE`

4. **Update an agent**  
   **Prev API**: 
    - `/api/v1/agents/<agent_id> PUT`   
    - `/v1/canvas/setting POST `
   **Current API**: `/api/v2/agents/<agent_id> PATCH`


5. **Create an agent**  
   **Prev API**: 
    - `/v1/canvas/set POST`  
    - `/api/v1/agents POST`
   **Current API**: `/api/v2/agents POST`


6. **Get an agent**  
   **Prev API**: 
    - `/v1/canvas/get/<canvas_id> GET `  
   **Current API**: `/api/v2/agents/<agent_id> GET`


7. **Reset an agent**  
   **Prev API**: 
    - `/v1/canvas/reset POST`  
   **Current API**: `/api/v2/agents/<agent_id>/reset POST`


8. **Upload a file to an agent**  
   **Prev API**: 
    - `/v1/canvas/upload/<canvas_id> POST`  
   **Current API**: `/api/v2/agents/<agent_id>/upload POST`


9. **Input form**  
   **Prev API**: 
    - `/v1/canvas/input_form GET`  
**Current API**:
`/api/v2/agents/<agent_id>/components/<component_id>/input-form GET`


10. **Debug an agent**  
   **Prev API**: 
    - `/v1/canvas/debug POST`  
**Current API**:
`/api/v2/agents/<agent_id>/components/<component_id>/debug POST`


11. **Trace an agent**  
   **Prev API**: 
    - `/v1/canvas/trace GET`  
   **Current API**: `/api/v2/agents/<agent_id>/logs/<message_id> GET`


12. **Get an agent version list**  
   **Prev API**: 
    - `/v1/canvas/getlistversion/<canvas_id>`  
   **Current API**: `/api/v2/agents/<agent_id>/versions GET`


13. **Get a version of agent**  
   **Prev API**: 
    - `/v1/canvas/getversion/<version_id>`  
**Current API**: `/api/v2/agents/<agent_id>/versions/<version_id> GET`


14. **Test db connection**  
   **Prev API**: 
    - `/v1/canvas/test_db_connect POST`  
   **Current API**: `/api/v2/agents/test_db_connection`


15. **Rerun the agent**  
   **Prev API**: 
    - `/v1/canvas/rerun POST`  
   **Current API**: `/api/v2/agents/rerun POST`


16. **Get prompts**  
   **Prev API**: 
    - `/v1/canvas/prompts GET`  
   **Current API**: `/api/v2/agents/prompts GET`

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chanx <1243304602@qq.com>
2026-04-24 10:02:22 +08:00
newyangyang
d84438fd53 fix azure blob put method param (#14329)
### What problem does this PR solve?

when use azure blob as the file container, when click parse file, it
calls:

```python
partial(settings.STORAGE_IMPL.put, tenant_id=task["tenant_id"])
```
So any storage backend used there must accept tenant_id as a kwarg. 
RAGFlowAzureSasBlob.put() did not, causing:
```
TypeError: ... got an unexpected keyword argument 'tenant_id'
```
Now it does, so parsing should proceed past this point.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-23 20:40:54 +08:00
buua436
d4fa57311c Refa: remove legacy MCP server web API (#14322)
### What problem does this PR solve?

remove legacy MCP server web API

### Type of change

- [x] Refactoring
2026-04-23 19:01:22 +08:00
Magicbook1108
75a5548b85 Feat: optimize title chunk (#14325)
### What problem does this PR solve?

Feat: optimize title chunk
1. Add a new button to enable "Use root chunk as H0 heading", so that
the first chunk is carried on to all remaining chunks.
2. Update resume agent template

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<img width="700" alt="img_v3_02111_63b04951-b3d7-4001-a08b-539db6d5298g"
src="https://github.com/user-attachments/assets/4179ac4d-90e7-4353-9b93-d649a455e634"
/>

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/c0ba0f3c-05aa-4f2c-b418-e808ca1a2641"
/>
2026-04-23 18:55:55 +08:00
Wang Qi
ba47c13eb5 Fix commit override from #14298 of api-key to api_key (#14328)
### What problem does this PR solve?

Fix commit override from
https://github.com/infiniflow/ragflow/pull/14298/ of `api-key` to `api_key`

### Type of change

- [x] Refactoring
2026-04-23 17:16:32 +08:00
Wang Qi
4458763a93 API refactor: stats_api and plugin_api (#14324)
### What problem does this PR solve?

API refactor: stats_api and plugin_api

### Type of change

- [x] Refactoring
2026-04-23 17:16:04 +08:00
buua436
7817b0d779 Refa: migrate chunk APIs to RESTful routes (#14291)
### What problem does this PR solve?

migrate chunk APIs to RESTful routes

### Type of change
- [x] Refactoring
2026-04-23 14:17:23 +08:00
Magicbook1108
76b017ca32 Refact: system apis (#14298)
### What problem does this PR solve?
Refact: system apis

### Type of change

- [x] Refactoring
2026-04-23 14:09:42 +08:00
Ricardo-M-L
57f527eb02 Add missing timeout to ragflow server health check (#14311)
### What problem does this PR solve?

`check_ragflow_server_alive()` in `api/utils/health_utils.py` calls
`requests.get(url)` without a `timeout` parameter. Unlike
`check_minio_alive()` which correctly specifies `timeout=10`, this
health check can hang indefinitely if the server is unresponsive.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

Added `timeout=10` to the `requests.get()` call, consistent with
`check_minio_alive()`.

Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-23 14:08:52 +08:00
dependabot[bot]
8901c18cb8 Build(deps): Bump lxml from 6.0.2 to 6.1.0 in /sdk/python (#14318)
Bumps [lxml](https://github.com/lxml/lxml) from 6.0.2 to 6.1.0.
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/lxml/lxml/blob/master/CHANGES.txt">lxml's
changelog</a>.</em></p>
<blockquote>
<h1>6.1.0 (2026-04-17)</h1>
<p>This release fixes a possible external entity injection (XXE)
vulnerability in
<code>iterparse()</code> and the <code>ETCompatXMLParser</code>.</p>
<h2>Features added</h2>
<ul>
<li>
<p>GH#486: The HTML ARIA accessibility attributes were added to the set
of safe attributes
in <code>lxml.html.defs</code>. This allows <code>lxml_html_clean</code>
to pass them through.
Patch by oomsveta.</p>
</li>
<li>
<p>The default chunk size for reading from file-likes in
<code>iterparse()</code> is now configurable
with a new <code>chunk_size</code> argument.</p>
</li>
</ul>
<h2>Bugs fixed</h2>
<ul>
<li>LP#2146291: The <code>resolve_entities</code> option was still set
to <code>True</code> for
<code>iterparse</code> and <code>ETCompatXMLParser</code>, allowing for
external entity injection (XXE)
when using these parsers without setting this option explicitly.
The default was now changed to <code>'internal'</code> only (as for the
normal XML and HTML parsers
since lxml 5.0).
Issue found by Sihao Qiu as CVE-2026-41066.</li>
</ul>
<h1>6.0.4 (2026-04-12)</h1>
<h2>Bugs fixed</h2>
<ul>
<li>LP#2148019: Spurious MemoryError during namespace cleanup.</li>
</ul>
<h1>6.0.3 (2026-04-09)</h1>
<h2>Bugs fixed</h2>
<ul>
<li>
<p>Several out of memory error cases now raise <code>MemoryError</code>
that were not handled before.</p>
</li>
<li>
<p>Slicing with large step values (outside of <code>+/-
sys.maxsize</code>) could trigger undefined C behaviour.</p>
</li>
<li>
<p>LP#2125399: Some failing tests were fixed or disabled in PyPy.</p>
</li>
<li>
<p>LP#2138421: Memory leak in error cases when setting the
<code>public_id</code> or <code>system_url</code> of a document.</p>
</li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="43722f4402"><code>43722f4</code></a>
Update changelog.</li>
<li><a
href="87470409b1"><code>8747040</code></a>
Name version of option change in docstring.</li>
<li><a
href="6c36e6cef7"><code>6c36e6c</code></a>
Fix pypistats URL in download statistics script.</li>
<li><a
href="c7d76d6cb8"><code>c7d76d6</code></a>
Change security policy to point to Github security advisories.</li>
<li><a
href="378ccf82db"><code>378ccf8</code></a>
Update project income report.</li>
<li><a
href="315270b810"><code>315270b</code></a>
Docs: Reduce TOC depth of package pages and move module contents
first.</li>
<li><a
href="6dbba7f3c7"><code>6dbba7f</code></a>
Docs: Show current year in copyright line.</li>
<li><a
href="e4385bfa5d"><code>e4385bf</code></a>
Update project income report.</li>
<li><a
href="5bed1e1a22"><code>5bed1e1</code></a>
Validate file hashes in release download script.</li>
<li><a
href="c13ee10a42"><code>c13ee10</code></a>
Prepare release of 6.1.0.</li>
<li>Additional commits viewable in <a
href="https://github.com/lxml/lxml/compare/lxml-6.0.2...lxml-6.1.0">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=lxml&package-manager=uv&previous-version=6.0.2&new-version=6.1.0)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-23 13:23:12 +08:00
Wang Qi
224574831c Add REDIS zcard (#14316)
### What problem does this PR solve?

As description.

### Type of change

- [x] Refactoring
2026-04-23 12:51:55 +08:00
buua436
aa4526266f Refa: migrate MCP APIs to RESTful api (#14317)
### What problem does this PR solve?

migrate MCP APIs to RESTful api

### Type of change

- [x] Refactoring
2026-04-23 12:51:27 +08:00
Jack
dbf8c6ed90 Refactor: Doc metadata update (#14289)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/metadata/update

After migration, Restful API
PATCH /api/v2/datasets/<dataset_id>/documents/metadatas 

### Type of change

- [x] Refactoring
2026-04-23 12:04:34 +08:00
Wang Qi
aae45b959b Refactor: API file2document (#14306)
Refactor: API file2document
2026-04-23 11:40:45 +08:00
Wang Qi
e79b896637 Refactor: REST API langfuse api-key (#14315)
REST API langfuse api-key
2026-04-23 11:36:16 +08:00
balibabu
f98597a19e Fix: Recall Test Page Metadata Not Displaying. (#14297)
### What problem does this PR solve?

Fix: Recall Test Page Metadata Not Displaying.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-23 10:57:05 +08:00
Jin Hai
2b029882d7 Go: add new provider minimax (#14296)
### What problem does this PR solve?

1. Add new provider minimax
2. Add new command: CHECK INSTANCE 'instance_name' FROM 'provider_name';
```
RAGFlow(user)> check instance 'test' from 'minimax';
SUCCESS
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-23 10:16:20 +08:00
chanx
387e2903d3 Fix: Some bugs (#14287)
### What problem does this PR solve?

Fix: Some bugs

- Pipeline runtime log files could not be viewed
- Corrected TOC terminology errors in the English translation

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-23 10:15:26 +08:00
balibabu
ffa8738a78 Fix: Remove duplicate text output from the thought model on the chat page. (#14301)
### What problem does this PR solve?

Fix: Remove duplicate text output from the thought model on the chat
page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 23:22:51 +08:00
Wang Qi
01753b8f31 Refactor: API connectors (#14228)
### What problem does this PR solve?

Refactor /api/v1/connectors to be more RESTful.

### Type of change
- [x] Refactoring
2026-04-22 20:42:41 +08:00
Jack
c08cd8e090 Refactor: Migrate document metadata config update API (#14286)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/update_metadata_setting

After consolidation, Restful API
PUT
/api/v1/datasets/<dataset_id>/documents/<document_id>/metadata/config

### Type of change

- [x] Refactoring
2026-04-22 20:01:31 +08:00
Magicbook1108
d1c62fc19d Refact: Tenant api (#14288)
### What problem does this PR solve?

Refact: Tenant api

### Type of change

- [x] Refactoring
2026-04-22 20:00:32 +08:00
writinwaters
1434f8ade8 Doc: two PDF parser optimizers are supported as of v0.25.0. (#14261)
### What problem does this PR solve?

Multi-column layout detection is supported in v0.25.0

### Type of change


- [x] Documentation Update
2026-04-22 20:00:06 +08:00
Wang Qi
b52c518ec9 Set image tag v0.25.0 (#14299)
### What problem does this PR solve?

AD

### Type of change
2026-04-22 19:12:21 +08:00
NeedmeFordev
38e45a1117 Fix: serialize GraphRAG entity resolution merges to avoid graph mutation races (#14237)
### What problem does this PR solve?

This PR fixes the merge-phase crash reported in #14236 during GraphRAG
entity resolution.

The issue happens after candidate pair resolution completes, when
multiple merge coroutines mutate the same shared `networkx` graph
concurrently. In `_merge_graph_nodes`, the code iterates over
`graph.neighbors(node1)` and also awaits during edge/description
merging. That allows another coroutine to modify the graph adjacency
structure in between, which can trigger `RuntimeError: dictionary keys
changed during iteration` and can also lead to unsafe shared-graph
mutation.

This change keeps the PR scoped to that single issue by:
- serializing merge-time graph mutations with a dedicated merge lock
- snapshotting `graph.neighbors(node1)` with `list(...)` before
iteration

Together, these changes prevent concurrent mutation of the shared graph
during the merge phase and make the merge loop safe against live-view
invalidation.

Fixes #14236

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 16:42:53 +08:00
bohdansolovie
e0f0eb277d Fix upload stream handling to prevent truncated files (#14267)
## Summary
- Replace single `Read()` call in Go upload service with `io.ReadAll()`.
- Prevent potential truncated/corrupted file content during multipart
upload.
- Keep existing API behavior unchanged while fixing data integrity risk.

## Root Cause
`io.Reader.Read()` may return fewer bytes than requested without an
error. The previous implementation read once into a full buffer and
assumed all bytes were populated.

## Test plan
- Upload files of multiple sizes and verify uploaded content integrity.
- Confirm upload endpoint still returns successful responses.
- Verify downstream document parsing works on uploaded files.

## Issues
Closes #14266
2026-04-22 16:32:38 +08:00
Jin Hai
b8660b9919 Add deepseek and moonshot model json (#14290)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 15:59:41 +08:00
ucloudnb666
f853a39b40 feat: Add Astraflow provider support (global + China endpoints) (#14270)
## Add Astraflow Provider Support

This PR integrates [Astraflow](https://astraflow.ucloud.cn/) (by UCloud
/ 优刻得) as a new AI model provider in RAGFlow, with support for both
global and China endpoints.

### About Astraflow
Astraflow is an OpenAI-compatible AI model aggregation platform
supporting 200+ models from major providers including DeepSeek, Qwen,
GPT, Claude, Gemini, Llama, Mistral, and more.

| Variant | Factory Name | Endpoint | Env Var |
|---------|-------------|----------|---------|
| Global | `Astraflow` | `https://api-us-ca.umodelverse.ai/v1` |
`ASTRAFLOW_API_KEY` |
| China | `Astraflow-CN` | `https://api.modelverse.cn/v1` |
`ASTRAFLOW_CN_API_KEY` |

- **API key signup**: https://astraflow.ucloud.cn/

---

### Files Changed

| File | Change |
|------|--------|
| `rag/llm/__init__.py` | Register `Astraflow` and `Astraflow-CN` in
`SupportedLiteLLMProvider` enum, `FACTORY_DEFAULT_BASE_URL`, and
`LITELLM_PROVIDER_PREFIX` |
| `rag/llm/chat_model.py` | Add `AstraflowChat` and `AstraflowCNChat`
(OpenAI-compatible `Base` subclass) |
| `rag/llm/embedding_model.py` | Add `AstraflowEmbed` and
`AstraflowCNEmbed` (subclasses of `OpenAIEmbed`) |
| `rag/llm/rerank_model.py` | Add `AstraflowRerank` and
`AstraflowCNRerank` (subclasses of `OpenAI_APIRerank`) |
| `rag/llm/cv_model.py` | Add `AstraflowCV` and `AstraflowCNCV`
(subclasses of `GptV4`) |
| `rag/llm/tts_model.py` | Add `AstraflowTTS` and `AstraflowCNTTS`
(subclasses of `OpenAITTS`) |
| `rag/llm/sequence2txt_model.py` | Add `AstraflowSeq2txt` and
`AstraflowCNSeq2txt` (subclasses of `GPTSeq2txt`) |
| `conf/llm_factories.json` | Register `Astraflow` and `Astraflow-CN`
factories with a curated list of popular models |

---

### Supported Model Types
-  **Chat / LLM** — DeepSeek-V3/R1, Qwen3, GPT-4o/4.1, Claude 3.5/3.7,
Gemini 2.0/2.5 Flash, Llama 3.3/4, Mistral, and 200+ more
-  **Text Embedding** — text-embedding-3-small/large
-  **Image / Vision (IMAGE2TEXT)** — GPT-4o, GPT-4.1, Claude, Gemini,
Llama-4, etc.
-  **Text Re-Rank**
-  **TTS** — tts-1
-  **Speech-to-Text (SPEECH2TEXT)** — whisper-1

### Implementation Notes
- Uses the `openai/` LiteLLM prefix — consistent with other
OpenAI-compatible aggregation platforms (SILICONFLOW, DeerAPI, CometAPI,
OpenRouter, n1n, Avian, etc.)
- `Astraflow` (global, rank 250) and `Astraflow-CN` (China, rank 249)
are separate factory entries, allowing users to choose the optimal
endpoint based on their region.
- All model classes cleanly subclass existing base classes (`Base`,
`OpenAIEmbed`, `OpenAI_APIRerank`, `GptV4`, `OpenAITTS`, `GPTSeq2txt`)
with no custom logic needed — the provider is fully OpenAI-compatible.

---------

Co-authored-by: user <user@xzaaaMacBook-Air.local>
2026-04-22 15:38:34 +08:00
Idriss Sbaaoui
d5d162b374 Fix: MinerU 3.x output discovery and API contract (#14282)
### What problem does this PR solve?

update MinerU parser to most recent minerU v3 logic

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 14:44:41 +08:00
Lynn
3ce1e44b2d Fix: document and sdk support of searching message with user_id (#14283)
### What problem does this PR solve?

Add document of search message with user_id, add sdk support.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-04-22 14:43:38 +08:00
Jin Hai
01c5437fdf Fix uv.lock (#14285)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 13:09:21 +08:00
Wang Qi
61d756e1b5 Fix #14213 create folder does not accept FOLDER (#14276)
### What problem does this PR solve?

As description.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:55:10 +08:00
writinwaters
69d8aed792 Doc: v0.25.0 release notes. (#14284)
### What problem does this PR solve?

Added v0.25.0 release notes

### Type of change


- [x] Documentation Update
2026-04-22 11:48:28 +08:00
Idriss Sbaaoui
77a843503d Fix: switch MinerU API endpoint to /pdf_parse (#14272)
### What problem does this PR solve?

update MinerU endpoint to /pdf_parse which has been exposed since v3.x.
fixes #14263

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:15:46 +08:00
buua436
ff29484d42 fix: normalize think tags in final chat answer (#14271)
### What problem does this PR solve?

normalize think tags in final chat answer

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-22 11:15:08 +08:00
Jack
3d8a82c0aa Refactor: Consolidation WEB API & HTTP API for document delete api (#14254)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/rm
Http API - DELETE /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- DELETE
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-22 10:49:52 +08:00
buua436
6baf74afc1 Refa: align chat and search restful APIs (#14229)
### What problem does this PR solve?

Refactor /api/v1/chats to be more RESTful.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 10:49:11 +08:00
Jin Hai
bfac0195df Update release note (#14275)
### What problem does this PR solve?

As title.

### Type of change

- [x] Documentation Update

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-22 10:47:43 +08:00
Jin Hai
74b44e1aa3 Go: add balance command (#14262)
### What problem does this PR solve?

```
RAGFlow(user)> list supported models from 'moonshot' 'test';
+---------------------------------+
| model_name                      |
+---------------------------------+
| moonshot-v1-32k-vision-preview  |
| kimi-k2.6                       |
| moonshot-v1-8k                  |
| moonshot-v1-auto                |
| moonshot-v1-128k                |
| moonshot-v1-32k                 |
| kimi-k2.5                       |
| moonshot-v1-8k-vision-preview   |
| moonshot-v1-128k-vision-preview |
+---------------------------------+
RAGFlow(user)> show balance from 'moonshot' 'test';
+---------+----------+
| balance | currency |
+---------+----------+
| 0       | CNY      |
+---------+----------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-21 21:31:50 +08:00
Jack
2d05475693 Refactor: Consolidation WEB API & HTTP API for document infos (#14239)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/infos
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents?ids=id1&ids=id2

### Type of change

- [ ] Refactoring
2026-04-21 19:35:11 +08:00
writinwaters
779deadf76 Docs: User-level memory is supported in v0.25.0 (#14259)
### What problem does this PR solve?

v0.25.0 supports linking User ID with conversations.

### Type of change


- [x] Documentation Update
2026-04-21 18:59:00 +08:00
hyl64
b439f8a74d docs: add DeepWiki developer guide page (#14244)
Closes #14165

Add a short documentation page under Developer Guides introducing
DeepWiki as a resource for developers doing secondary development or
exploring RAGFlow's codebase internals.

---------

Co-authored-by: hyl64 <hyl64@users.noreply.github.com>
2026-04-21 18:57:20 +08:00
Jack
009e538a4e Refactor: Consolidation WEB API & HTTP API for document get_filter (#14248)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/filter
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents?type=filter
### Type of change

- [x] Refactoring
2026-04-21 18:55:30 +08:00
Liu An
a33d0737cd Docs: Update version references to v0.25.0 in READMEs and docs (#14257)
### What problem does this PR solve?

- Update version tags in README files (including translations) from
v0.24.0 to v0.25.0
- Modify Docker image references and documentation to reflect new
version
- Update version badges and image descriptions
- Maintain consistency across all language variants of README files

### Type of change

- [x] Documentation Update
2026-04-21 17:26:50 +08:00
Lynn
afdf0814d7 Fix: get metadata conf (#14250)
### What problem does this PR solve?

Get metadata configuration from union of custom metadata and
built_in_metadata.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 17:22:42 +08:00
writinwaters
0db2d544a9 Docs: 0.25.0 agent apps can be published. (#14252)
### What problem does this PR solve?

Agent apps can be published.

### Type of change

- [x] Documentation Update
2026-04-21 16:56:11 +08:00
balibabu
4841ce4239 Fix: Component definition is missing display name. (#14255)
### What problem does this PR solve?

Fix: Component definition is missing display name.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 16:53:08 +08:00
Jin Hai
e48d75987c Go: add stream / think chat (#14242)
### What problem does this PR solve?

1. Supports stream and non-stream chat
2. Supports think and non-think chat
3. List supported models from DeepSeek service. (This command can be
used to verify the API validity)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-21 16:52:32 +08:00
balibabu
a2bea30749 Fix: Editing an empty response in the retrieval operator will cause the focus to shift to the metadata input box. (#14253)
### What problem does this PR solve?

Fix: Editing an empty response in the retrieval operator will cause the
focus to shift to the metadata input box.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 16:19:55 +08:00
chanx
05c39b90a8 Fix: pipeline parser log not display (#14251)
### What problem does this PR solve?

Fix: pipeline parser log not display

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 15:24:13 +08:00
Liu An
6e33d8722f Revert "Fix: forwarding highlight param" (#14249)
Reverts infiniflow/ragflow#14112
2026-04-21 15:23:18 +08:00
balibabu
78b800e685 Fix: Fix: The minimum value for the "Suggested text block size" input box is set to 1. (#14246)
### What problem does this PR solve?

Fix: The minimum value for the "Suggested text block size" input box is
set to 1.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 14:06:36 +08:00
Magicbook1108
b3891ba6a4 Fix audio/video in pipeline (#14241)
### What problem does this PR solve?

Fix audio/video in pipeline

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 12:17:57 +08:00
Wang Qi
8aab158942 OpenSource Resume is supported only with Elasticsearch. (#14233)
### What problem does this PR solve?

OpenSource Resume is supported only with Elasticsearch.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-21 10:05:47 +08:00
Jin Hai
f269ee9739 Go: add thinking features to zhipu-ai (#14234)
### What problem does this PR solve?

```
RAGFlow(user)> list models from 'zhipu-ai';
+------------+------------+---------------+----------------+
| features   | max_tokens | model_types   | name           |
+------------+------------+---------------+----------------+
| [thinking] | 128000     | [chat]        | glm-4.7        |
| [thinking] | 128000     | [chat]        | glm-4.5        |
| [thinking] | 128000     | [chat vision] | glm-4.6v-Flash |
| [thinking] | 128000     | [chat]        | glm-4.5-x      |
| [thinking] | 128000     | [chat]        | glm-4.5-air    |
| [thinking] | 128000     | [chat]        | glm-4.5-airx   |
| [thinking] | 128000     | [chat]        | glm-4.5-flash  |
| [thinking] | 64000      | [vision]      | glm-4.5v       |
|            | 128000     | [chat]        | glm-4-plus     |
|            | 128000     | [chat]        | glm-4-0520     |
|            | 128000     | [chat]        | glm-4          |
|            | 8000       | [chat]        | glm-4-airx     |
|            | 128000     | [chat]        | glm-4-air      |
|            | 128000     | [chat]        | glm-4-flash    |
|            | 128000     | [chat]        | glm-4-flashx   |
|            | 1000000    | [chat]        | glm-4-long     |
|            | 128000     | [chat]        | glm-3-turbo    |
|            | 2000       | [vision]      | glm-4v         |
|            | 8192       | [chat]        | glm-4-9b       |
|            | 512        | [embedding]   | embedding-2    |
|            | 512        | [embedding]   | embedding-3    |
|            | 4096       | [asr]         | glm-asr        |
|            | 0          | [tts]         | glm-tts        |
|            | 0          | [ocr]         | glm-ocr        |
|            | 0          | [rerank]      | glm-rerank     |
+------------+------------+---------------+----------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-20 21:53:27 +08:00
balibabu
c43367eca3 Fix: The number of chunks in the file list is not displayed. (#14232)
### What problem does this PR solve?

Fix: The number of chunks in the file list is not displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 19:24:20 +08:00
balibabu
5265def967 Fix: The mind map on the search page does not display completely upon initial loading. (#14226)
### What problem does this PR solve?

Fix: The mind map on the search page does not display completely upon
initial loading.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 19:24:13 +08:00
Julian
ba7d3f6c31 Add debugpy dependency to pyproject.toml (#14225)
In order to attach the debugger to a running docker container it has to
be inside the docker image

### What problem does this PR solve?

[#14224](https://github.com/infiniflow/ragflow/issues/14224)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 18:05:17 +08:00
NeedmeFordev
78c3583964 Fix memory resolution regression for multimodal Gemini models (#14209)
### What problem does this PR solve?

Fixes #14206.

This issue is a regression. PR #9520 previously changed Gemini models
from `image2text` to `chat` to fix chat-side resolution, but PR #13073
later restored those Gemini entries to `image2text` during model-list
updates, which reintroduced the bug.

The underlying problem is that Gemini models are multimodal and
advertise both `CHAT` and `IMAGE2TEXT`, while tenant model resolution
still depends on a single stored `model_type`. That makes chat-only
flows such as memory extraction fragile when a compatible model is
stored as `image2text`.

This PR fixes the issue at the model resolution layer instead of
changing `llm_factories.json` again:
- keep the stored tenant model type unchanged
- try exact `model_type` lookup first
- if no exact match is found, fall back only when the model metadata
shows the requested capability is supported
- coerce the runtime config to the requested type for chat callers
- fail fast in memory creation instead of silently persisting
`tenant_llm_id=0`

This preserves existing multimodal and `image2text` behavior while
restoring chat compatibility for memory-related flows.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Testing

- Re-checked the current memory creation and memory message extraction
paths against the updated resolution logic
- Verified locally that a Gemini-style tenant model stored as
`image2text` but tagged with `CHAT` can still be resolved for `chat`
- Verified `get_model_config_by_type_and_name(..., CHAT, ...)` returns a
chat-compatible runtime config
- Verified `get_model_config_by_id(..., CHAT)` also returns a
chat-compatible runtime config
- Verified strict resolution still fails when the model metadata does
not advertise chat capability
2026-04-20 16:37:36 +08:00
Magicbook1108
9c7c105007 Fix: Doc generator (#14223)
### What problem does this PR solve?

Doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 16:37:33 +08:00
Jin Hai
af2ed416a7 Add extra field to model instance (#14203)
### What problem does this PR solve?

Now each model support region with different URL

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-20 15:31:12 +08:00
Jack
939933649a Refactor: Consolidation WEB API & HTTP API for document list_docs (#14176)
### What problem does this PR solve?

Before consolidation
Web API: POST /v1/document/list
Http API - GET /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- GET
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-20 14:54:40 +08:00
Magicbook1108
d053317c4d Fix: variable in doc generator (#14180)
### What problem does this PR solve?

Fix: variable in doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 14:19:42 +08:00
Magicbook1108
19eedeec61 Fix: accept empty value as 0 chunk (#14220)
### What problem does this PR solve?

Fix: accept empty value as 0 chunk
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-20 12:53:47 +08:00
LeonTung
f554f6ae85 chore(docs): tips for installing CN fonts (#14189)
### What problem does this PR solve?
Add tips for installing Chinse fonts under code sandbox. Otherwise,
`matplotlib `won't render Chinese correctly.

<img width="2082" height="1186" alt="sales_analysis"
src="https://github.com/user-attachments/assets/57e675ab-1e92-4662-9aeb-ad72a6121eb5"
/>



### Type of change

- [x] Documentation Update
2026-04-20 12:11:23 +08:00
Lynn
0f806dc3ca Feat: mysql sync (#14200)
### What problem does this PR solve?

Add a script to sync db schema with peewee_migrate.

### Type of change

- [x] Other (please describe): tool script
2026-04-20 11:40:01 +08:00
rhinoceros.xn
4e992de91f Add tongyi gte-rerank-v2 (#14215)
https://bailian.console.aliyun.com/cn-beijing?tab=api#/api/?type=model&url=2780056

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change
- [x] Other (please describe): add gte-rerank-v2、qwen3-rerank
2026-04-20 11:39:17 +08:00
Liu An
d5c306de30 Fix: remove unit test checkpoint resume (#14216)
### What problem does this PR solve?

remove unit test checkpoint resume

### Type of change

- [x] Performance Improvement
2026-04-20 11:27:40 +08:00
euvre
84b6069ec7 fix: escape single quotes in Infinity SQL filter conditions (#14186)
### What problem does this PR solve?

## Summary

Fixes #5939

Entity names containing single quotes (e.g., `投影直线L'`) caused SQL syntax
errors when building filter conditions for Infinity queries, due to
unescaped string interpolation in `equivalent_condition_to_str`.

## Changes

In `common/doc_store/infinity_conn_base.py`, added `.replace("'", "''")`
escaping for string values in two branches of
`equivalent_condition_to_str` where it was missing:

1. **`field_keyword` branch with non-list value** (line 190): The list
branch already escaped single quotes on line 183, but the single-string
branch did not.
2. **Plain string value branch** (line 209): Direct f-string
interpolation `{k}='{v}'` was vulnerable to unescaped quotes.

Both fixes use the same SQL-standard escape pattern (`'` → `''`) already
applied elsewhere in this method.

## How to Test

1. Upload a document containing entity names with single quotes.
2. Enable Knowledge Graph (GraphRAG) in the parsing configuration.
3. Initiate document parsing — it should complete without SQL syntax
errors.

## Note

The original issue also reported a typo (`dge_graph_kwd` instead of
`knowledge_graph_kwd`), which has already been fixed in the current
codebase.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-20 10:04:07 +08:00
balibabu
6712b504e6 Fix: Clicking on the empty dialog box on the agent exploration page will result in an error. (#14198)
### What problem does this PR solve?

Fix: Clicking on the empty dialog box on the agent exploration page will
result in an error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 23:52:13 +08:00
Lynn
c3387cd5b8 Fix: parent child config (#14199)
### What problem does this PR solve?

Correctly set and display parent-child config in parser_config, and
allow to pass `tenant_id` in PATCH `/api/v1/chats`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 23:02:42 +08:00
balibabu
09622c6353 Fix: Spaces cannot be entered in the code editor of the code operator. (#14183)
### What problem does this PR solve?

Fix: Spaces cannot be entered in the code editor of the code operator.

[Monaco Editor with XYFlow fails to accept most space bar keypresses,
who is at fault?
#5204](https://github.com/microsoft/monaco-editor/discussions/5204)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:41:39 +08:00
balibabu
fa644c5a15 Fix: The embedded page for search is inaccessible. (#14194)
### What problem does this PR solve?

Fix: The embedded page for search is inaccessible.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:37:34 +08:00
chanx
60506ef7a5 fix: Add internationalization configurations related to text segmentation identifiers. (#14201)
### What problem does this PR solve?

fix: Add internationalization configurations related to text
segmentation identifiers.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:37:14 +08:00
balibabu
3a4d17cb0d Fix: The placeholder in PromptEditor is obscured. (#14179)
### What problem does this PR solve?

Fix: The placeholder in PromptEditor is obscured.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 21:02:41 +08:00
Daniil Sivak
22c6648348 Fix: forwarding highlight param (#14112)
Closes #9078

### What problem does this PR solve?

The `retrieval_test` endpoint in `chunk_app.py` never forwarded the
`highlight` request parameter to `retriever.retrieval()`, so the search
engine never produced highlight snippets. Additionally, the frontend
always rendered `content_with_weight` instead of preferring the
`highlight` field, and the CSS rule color `var(--accent-primary)` didn't
work because the variable stores an RGB triplet `(45,212,191)` requiring
the `rgb()` wrapper.

### Before

- Search page: displayed raw content_with_weight as a wall of plain
white text with no term highlighting, including markdown headings
rendered as literal text
- Retrieval testing page: showed `content_with_weight` in a plain `<p>`
tag, no `<em>` tags rendered, no highlight coloring
- Children chunks: when child chunks were consolidated into a parent via
`retrieval_by_children`, any highlight data from children was discarded
- TOC chunks: chunks fetched via `retrieval_by_toc` had no `highlight`
field, appearing as plain text while other chunks had highlights

**Retrieval testing**:
<img width="1449" height="1178"
alt="before-retrieval-no-highlight-cropped"
src="https://github.com/user-attachments/assets/5c6f5a5e-6c11-461a-bdb4-049d7dfb7a33"
/>

**Search**:
<img width="1378" height="711" alt="before-search-no-highlight-cropped"
src="https://github.com/user-attachments/assets/be7b5152-72ef-40da-a8fd-921e997ae7d3"
/>

### After

- Search page: displays the highlight field with search terms rendered
in teal/cyan color (`rgb(var(--accent-primary))`)
- Retrieval testing page: sends highlight: true in the request, uses
`HighLightMarkdown` component to render `<em>` tags with proper coloring
- Children chunks: highlights from child chunks are joined and preserved
on the parent
- TOC chunks: when other chunks have highlights, TOC-fetched chunks use
`content_with_weight` as a highlight fallback

**Retrieval testing**:
<img width="1410" height="1015" alt="05-retrieval-testing-results"
src="https://github.com/user-attachments/assets/f0cff8cf-0962-4320-b559-cd5037f622d2"
/>

**Search**:
<img width="1294" height="455" alt="03-search-highlight-results"
src="https://github.com/user-attachments/assets/a90e0e3e-3837-46be-8ddd-2412ff7cbc19"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 20:59:20 +08:00
Yongteng Lei
fac46ef67f Refa: change Minimax base url to mainland by default to align with UI (#14195)
### What problem does this PR solve?

Change Minimax base url to mainland by default to align with UI.

### Type of change

- [x] Refactoring
2026-04-17 19:08:57 +08:00
dependabot[bot]
b34a726acd Build(deps): Bump pypdf from 6.9.2 to 6.10.2 (#14184)
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.9.2 to 6.10.2.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/releases">pypdf's
releases</a>.</em></p>
<blockquote>
<h2>Version 6.10.2, 2026-04-15</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.1...6.10.2">Full
Changelog</a></p>
<h2>Version 6.10.1, 2026-04-14</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use new parameter names for compress_identical_objects by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.0...6.10.1">Full
Changelog</a></p>
<h2>Version 6.10.0, 2026-04-10</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Disallow custom XML entity declarations for XMP metadata (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3724">#3724</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>New Features (ENH)</h3>
<ul>
<li>Skip MD5 key derivation for AES-256 encrypted PDFs (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3694">#3694</a>)
by <a href="https://github.com/Ygnas"><code>@​Ygnas</code></a></li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use remove_orphans in compress_identical_objects (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3310">#3310</a>)
by <a href="https://github.com/j-t-1"><code>@​j-t-1</code></a></li>
<li>Fix PdfReadError when xref table contains comments before trailer
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3710">#3710</a>)
by <a href="https://github.com/rassie"><code>@​rassie</code></a></li>
<li>Correctly verify AES padding during decryption (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3699">#3699</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
<li>Fix stale object cache from non-authoritative object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3698">#3698</a>)
by <a
href="https://github.com/astahlman"><code>@​astahlman</code></a></li>
<li>Fix extract_links pairing when annotations include non-links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3687">#3687</a>)
by <a
href="https://github.com/ReinerBRO"><code>@​ReinerBRO</code></a></li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Add AI policy (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3717">#3717</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.0">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md">pypdf's
changelog</a>.</em></p>
<blockquote>
<h2>Version 6.10.2, 2026-04-15</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)</li>
<li>Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.1...6.10.2">Full
Changelog</a></p>
<h2>Version 6.10.1, 2026-04-14</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)</li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)</li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Use new parameter names for compress_identical_objects</li>
</ul>
<p><a
href="https://github.com/py-pdf/pypdf/compare/6.10.0...6.10.1">Full
Changelog</a></p>
<h2>Version 6.10.0, 2026-04-10</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Disallow custom XML entity declarations for XMP metadata (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3724">#3724</a>)</li>
</ul>
<h3>New Features (ENH)</h3>
<ul>
<li>Skip MD5 key derivation for AES-256 encrypted PDFs (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3694">#3694</a>)</li>
</ul>
<h3>Bug Fixes (BUG)</h3>
<ul>
<li>Use remove_orphans in compress_identical_objects (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3310">#3310</a>)</li>
<li>Fix PdfReadError when xref table contains comments before trailer
(<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3710">#3710</a>)</li>
<li>Correctly verify AES padding during decryption (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3699">#3699</a>)</li>
<li>Fix stale object cache from non-authoritative object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3698">#3698</a>)</li>
<li>Fix extract_links pairing when annotations include non-links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3687">#3687</a>)</li>
</ul>
<h3>Documentation (DOC)</h3>
<ul>
<li>Add AI policy (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3717">#3717</a>)</li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.0">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="c476b4f293"><code>c476b4f</code></a>
REL: 6.10.2</li>
<li><a
href="c50a0104cf"><code>c50a010</code></a>
SEC: Do not rely on possibly invalid /Size for incremental cloning (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3735">#3735</a>)</li>
<li><a
href="ac734dab4e"><code>ac734da</code></a>
SEC: Introduce limits for FlateDecode parameters and image decoding (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3734">#3734</a>)</li>
<li><a
href="b49e7eb454"><code>b49e7eb</code></a>
REL: 6.10.1</li>
<li><a
href="62338e9d36"><code>62338e9</code></a>
SEC: Limit the allowed size of xref and object streams (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3733">#3733</a>)</li>
<li><a
href="5dcc0aebaa"><code>5dcc0ae</code></a>
DEV: Update pytest-benchmark to 5.2.3</li>
<li><a
href="b42e4aa98a"><code>b42e4aa</code></a>
DEV: Update pinned pillow and pytest where possible (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3732">#3732</a>)</li>
<li><a
href="717446b121"><code>717446b</code></a>
ROB: Consider strict mode setting for decryption errors (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3731">#3731</a>)</li>
<li><a
href="9e461d361b"><code>9e461d3</code></a>
DEV: Bump softprops/action-gh-release from 2 to 3 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3730">#3730</a>)</li>
<li><a
href="500d09d92f"><code>500d09d</code></a>
TST: Update <code>test_embedded_file__basic</code> to use
<code>tmp_path</code> fixture (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3726">#3726</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/py-pdf/pypdf/compare/6.9.2...6.10.2">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypdf&package-manager=uv&previous-version=6.9.2&new-version=6.10.2)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-04-17 18:43:19 +08:00
Jin Hai
94106646e7 Go: set and list default models (#14191)
### What problem does this PR solve?

```
RAGFlow(user)> set default vlm "zhipu-ai" "ccc" "glm-4.6v-flash";
SUCCESS
RAGFlow(user)> list default models;
+--------+----------------+----------------+----------------+------------+
| enable | model_instance | model_name     | model_provider | model_type |
+--------+----------------+----------------+----------------+------------+
| true   | ccc            | glm-4.6v-flash | zhipu-ai       | llm        |
| true   | ccc            | glm-4.6v-flash | zhipu-ai       | image2text |
+--------+----------------+----------------+----------------+------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-17 18:05:33 +08:00
Wang Qi
28d8b1c883 [Fix] trivial fix log creation (#14181)
### What problem does this PR solve?

Trivial fix log creation, follow on PR:
https://github.com/infiniflow/ragflow/pull/14136

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 13:13:41 +08:00
Magicbook1108
797aa6076a Fix: keyword extraction (#14177)
### What problem does this PR solve?

Fix: keyword extraction

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 11:32:48 +08:00
LeonTung
c3bf8d9d60 feat(templates): add a data analysis agent template (#14130)
### What problem does this PR solve?

Add a new agent template that demonstrates how to leverage the
`CodeExec` component to do the data analysis.

### Type of change

- [x] Other (please describe): Agent template
2026-04-17 11:32:04 +08:00
writinwaters
0df5d830d4 Refact: Updated agent template descriptions. (#14175)
### What problem does this PR solve?

Updated ingestion pipeline template descriptions for better technical
accuracy and readability.

### Type of change

- [x] Refactoring
2026-04-17 10:46:06 +08:00
Lynn
f194a09cd6 Fix: dataset update parent child (#14167)
### What problem does this PR solve?

Correctly set parent child config in parser_config.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 10:41:50 +08:00
Jin Hai
e03212fd7a Fix go cli models command and api (#14166)
### What problem does this PR solve?

```
RAGFlow(user)> list providers;
+--------------------------------------+----------+-------------------------------------------+--------------+
| base_url                             | name     | tags                                      | total_models |
+--------------------------------------+----------+-------------------------------------------+--------------+
| https://open.bigmodel.cn/api/paas/v4 | ZHIPU-AI | LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION | 21           |
| https://api.x.ai/v1                  | xAI      | LLM                                       | 6            |
+--------------------------------------+----------+-------------------------------------------+--------------+
RAGFlow(user)> show provider 'zhipu-ai';
+--------------------------------------+----------+-------------------------------------------+--------------+
| base_url                             | name     | tags                                      | total_models |
+--------------------------------------+----------+-------------------------------------------+--------------+
| https://open.bigmodel.cn/api/paas/v4 | ZHIPU-AI | LLM,TEXT EMBEDDING,SPEECH2TEXT,MODERATION | 21           |
+--------------------------------------+----------+-------------------------------------------+--------------+
RAGFlow(user)> delete provider 'zhipu-ai';
SUCCESS
RAGFlow(user)> add provider 'zhipu-ai';
SUCCESS
RAGFlow(user)> create provider 'zhipu-ai' instance 'ccc' 'ccxxccxx';
SUCCESS
RAGFlow(user)> list instances from 'zhipu-ai';
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| apiKey                                            | id                               | instanceName | providerID                       | status |
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
| ccxxccxx | 640dd7ee398711f1bdd838a74640adcc | ccc          | d1d59de5398411f1bdd838a74640adcc | active |
+---------------------------------------------------+----------------------------------+--------------+----------------------------------+--------+
RAGFlow(user)> list models from 'zhipu-ai';
+----------+------------+---------------+---------------+
| features | max_tokens | model_types   | name          |
+----------+------------+---------------+---------------+
| map[]    | 128000     | [chat]        | glm-4.7       |
| map[]    | 128000     | [chat]        | glm-4.5       |
| map[]    | 128000     | [chat]        | glm-4.5-x     |
| map[]    | 128000     | [chat]        | glm-4.5-air   |
| map[]    | 128000     | [chat]        | glm-4.5-airx  |
| map[]    | 128000     | [chat]        | glm-4.5-flash |
| map[]    | 64000      | [image2text]  | glm-4.5v      |
| map[]    | 128000     | [chat]        | glm-4-plus    |
| map[]    | 128000     | [chat]        | glm-4-0520    |
| map[]    | 128000     | [chat]        | glm-4         |
| map[]    | 8000       | [chat]        | glm-4-airx    |
| map[]    | 128000     | [chat]        | glm-4-air     |
| map[]    | 128000     | [chat]        | glm-4-flash   |
| map[]    | 128000     | [chat]        | glm-4-flashx  |
| map[]    | 1000000    | [chat]        | glm-4-long    |
| map[]    | 128000     | [chat]        | glm-3-turbo   |
| map[]    | 2000       | [image2text]  | glm-4v        |
| map[]    | 8192       | [chat]        | glm-4-9b      |
| map[]    | 512        | [embedding]   | embedding-2   |
| map[]    | 512        | [embedding]   | embedding-3   |
| map[]    | 4096       | [speech2text] | glm-asr       |
+----------+------------+---------------+---------------+
RAGFlow(user)> disable model 'glm-4.5-flash' from 'zhipu-ai' 'ccc';
SUCCESS
RAGFlow(user)> drop instance 'ccc' from 'zhipu-ai';
SUCCESS
RAGFlow(user)> list instances from 'zhipu-ai';
No data to print
```

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-17 09:55:25 +08:00
Wang Qi
96a23d2fd0 [Bug fix] fix bug found in regression when view chunks for document that not parsed in infinity, it would fail in UI (#14168)
### What problem does this PR solve?
See title, the fail image:
<img width="2667" height="915" alt="20260416-205718"
src="https://github.com/user-attachments/assets/0c564237-5ed0-49af-bf4c-d3b5519abc6e"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 09:51:23 +08:00
Magicbook1108
f906a203bb Fix doc generator (#14160)
### What problem does this PR solve?

Fix doc generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:38 +08:00
balibabu
4a9bfd18bc Fix: The PromptEditor's placeholder is only half displayed. (#14161)
### What problem does this PR solve?

Fix: The PromptEditor's placeholder is only half displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:16 +08:00
Magicbook1108
ea8de1bb47 Fix: different llm in chat (#14162)
### What problem does this PR solve?

Fix: different llm in chat

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 20:37:01 +08:00
writinwaters
8a874c7a09 Doc: Added Ingetrating Notion connector (#14163)
### What problem does this PR solve?

Added How to integrate Notion to RAGFlow.

### Type of change

- [x] Documentation Update
2026-04-16 20:06:02 +08:00
Lynn
655dd2f8c6 Fix: simplify _load_user (#14154)
### What problem does this PR solve?

Simplify _load_user, remove unused fallback.

### Type of change

- [x] Refactoring
2026-04-16 18:47:43 +08:00
balibabu
4cf4d444d2 Fix: Login page type error. (#14156)
### What problem does this PR solve?

Fix: Login page type error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 18:46:52 +08:00
Magicbook1108
901023a80a Fix: literal eval http request input (#14145)
### What problem does this PR solve?

Fix: literal eval http request input

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

<img width="700" alt="img_v3_0210q_f4b49ff7-e670-4054-ab0e-9443a09215fg"
src="https://github.com/user-attachments/assets/089300be-06f9-4bb6-97af-61bf5f4a5e8c"
/>


<img width="700" alt="img_v3_0210q_398cd52a-2ad9-42be-8d5b-4e6e68a7d22g"
src="https://github.com/user-attachments/assets/239b43cd-a2a5-49d8-9200-991bb26336c8"
/>
2026-04-16 16:52:34 +08:00
euvre
9a785b26bd fix: change file size column from IntegerField to BigIntegerField to support files > 2GB (#14148)
### What problem does this PR solve?

Fixes #6034

Changes the `size` field in both `Document` and `File` models from
`IntegerField` (32-bit, max ~2GB) to `BigIntegerField` (64-bit, max
~9.2EB), and adds corresponding database migrations.

## Problem

When uploading a file larger than 2GB, the `size` value overflows a
32-bit signed integer (max 2,147,483,647). This causes:

- The stored `size` wraps around to an incorrect value (e.g., a 3GB file
shows as 2,097,152 KB in File Management).
- Subsequent file operations (e.g., download) fail because the corrupted
size leads to invalid storage lookups.

## Changes

- `Document.size`: `IntegerField` → `BigIntegerField`
- `File.size`: `IntegerField` → `BigIntegerField`
- Added `alter_db_column_type` migrations in `migrate_db()` for both
`document.size` and `file.size` columns to ensure existing deployments
are upgraded automatically.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-16 15:43:29 +08:00
euvre
0cd49e14dd fix: make Infinity connection pool size configurable and add retry logic for GraphRAG write bursts (#14143)
### What problem does this PR solve?

Resolve #14137 .

### Problem

Graph resolution succeeds (nodes/edges merged, pagerank updated), but
the subsequent burst of Infinity write operations in `set_graph`
exhausts the connection pool with `TOO_MANY_CONNECTIONS` errors. Root
causes:

1. **Hardcoded pool size** — `infinity_conn_pool.py` hardcoded
`ConnectionPool(max_size=4)` on initial creation and `max_size=32` on
refresh. Operators cannot tune this without patching code.
2. **No retry on transient failures** — a single `TOO_MANY_CONNECTIONS`
on edge deletes or chunk inserts kills the entire resolution+community
pipeline with no retry.

### Changes

#### `common/doc_store/infinity_conn_pool.py`

- Read `ConnectionPool` `max_size` from the `INFINITY_POOL_MAX_SIZE`
environment variable (default: `4`), applied consistently to both
initial creation and refresh paths.
- Log the actual pool size on startup for easier debugging.

#### `rag/graphrag/utils.py` — `set_graph()`

- **Edge deletes**: add exponential-backoff retry (3 attempts, 1s/2s/4s
delays) so transient `TOO_MANY_CONNECTIONS` errors are retried instead
of failing the entire job. Concurrency continues to be gated by the
existing `chat_limiter`.
- **Batch inserts**: add exponential-backoff retry (3 attempts, 1s/2s/4s
delays) for the same reason.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-16 15:40:54 +08:00
Qi Wang
969ce3a79f [Bug fix #14133] fix graph rag, raptor, mindmap log cannot show correctly in UI (#14136)
### What problem does this PR solve?
Fix #14133, knowledge graph, raptor, mindmap log cannot show correctly
in UI
<img width="1930" height="982" alt="Image"
src="https://github.com/user-attachments/assets/d2f8e6c1-d82d-4b00-a377-949aada545ca"
/>
After Fix:
<img width="2108" height="805" alt="image"
src="https://github.com/user-attachments/assets/b37426c1-83d3-4a32-a83c-9d340d69e0e6"
/>
<img width="2173" height="1067" alt="image"
src="https://github.com/user-attachments/assets/30105222-3310-43a0-9f83-1e320d05e413"
/>

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 13:08:36 +08:00
Yongteng Lei
356ba5650a Fix: sandbox don't attach attachment metadata (#14135)
### What problem does this PR solve?

Sandbox don't attach attachment metadata

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-16 12:08:54 +08:00
balibabu
53154b2cc3 Feat: Add a title prefix to the testid on the login page. (#14129)
### What problem does this PR solve?

Feat: Add a title prefix to the testid on the login page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-16 12:08:44 +08:00
Magicbook1108
944a90d645 Feat: add button to turn off vlm parsing (#14125)
### What problem does this PR solve?

Feat: add button to turn off vlm parsing

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: chanx <1243304602@qq.com>
2026-04-15 19:06:00 +08:00
chanx
dce0b1c030 Fix: Pipeline page style optimizations (#14128)
### What problem does this PR solve?

Fix: Pipeline page style optimizations

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-15 19:05:54 +08:00
Daniil Sivak
c93ec0a1f3 Fix: reject empty/space-only content in update_chunk API (#14082)
Closes #6541

### What problem does this PR solve?

Add content validation to `update_chunk` (SDK and non-SDK) to reject
empty or whitespace-only content before it reaches the embedding model.

**Before:** Calling `update_chunk` with space-only content (like `" "`,
`""`, `"\n"`) bypassed validation and was sent directly to the embedding
model, which returned an error. This was the same bug previously fixed
for `add_chunk` in #6390, but `update_chunk` was missed.

**After:** Empty/whitespace-only content is caught by validation and
returns an error: `` `content` is required ``

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-15 18:43:53 +08:00
Magicbook1108
d51789e2be Feat: update templates && add resume template (#14124)
### What problem does this PR solve?

Feat: update templates  && add resume template

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-15 18:42:29 +08:00
balibabu
c56a7f99d1 Fix: The pop-up menu of the PromptEditor will be blocked. #14126 (#14127)
### What problem does this PR solve?

Fix: The pop-up menu of the PromptEditor will be blocked.  #14126

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-15 18:42:02 +08:00
writinwaters
2520065c5a Doc: Added Integrate Confluence (#14131)
### What problem does this PR solve?

Added a guide on integrating Confluence as connector.

### Type of change

- [x] Documentation Update
2026-04-15 18:38:36 +08:00
Minal Mahala
f930389311 Refact: improve task resume mechanism for graphrag (#14096)
### What problem does this PR solve?

Addresses review feedback on #14074 (Checkpoint mechanism for
long-running workflow jobs, issue #12494).

**Changes based on @yuzhichang's review:**

1. **Renamed `checkpoint_service.py` → `task_checkpoint.py`** as
suggested.
2. **Replaced Redis with direct docEngine queries** as suggested — the
subgraph already gets persisted to the doc store by
`generate_subgraph()`, so we just query for it instead of maintaining a
separate checkpoint in Redis. This is simpler, has no extra dependency,
and uses a single source of truth.

**Changes based on CodeRabbit review:**

3. **Fixed `source_id` query format mismatch** — subgraphs are stored
with `source_id: [doc_id]` (list), but the original query used
`source_id: doc_id` (string). Now follows the same pattern as
`does_graph_contains()` in `rag/graphrag/utils.py`: filter by
`knowledge_graph_kwd` only, then match `source_id` in Python. This
avoids ambiguity across Elasticsearch / Infinity / OceanBase backends.

### Changes

| File | Change |
|---|---|
| `api/db/services/task_checkpoint.py` (new) |
`load_subgraph_from_store()` and `has_raptor_chunks()` — docEngine-based
checkpoint queries |
| `rag/graphrag/general/index.py` | `build_one()` calls
`load_subgraph_from_store()` before running LLM extraction |
| `rag/svr/task_executor.py` | RAPTOR per-doc loop calls
`has_raptor_chunks()` before processing |
| `test/unit_test/rag/graphrag/test_checkpoint_resume.py` (new) | 10
unit tests covering subgraph loading, source_id filtering, edge cases |

### How it works

- **GraphRAG:** Before running expensive LLM entity/relation extraction
for a doc, checks the doc store for an existing subgraph (saved by a
previous interrupted run). If found, loads it directly and skips LLM
calls.
- **RAPTOR:** Before processing a doc, checks if RAPTOR chunks
(`raptor_kwd="raptor"`) already exist for it. If yes, skips.

### Testing

- 10 new unit tests — all passing
- Full existing suite: 617 passed

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-04-15 17:37:28 +08:00
euvre
3364d86e6b Auto-inject knowledge parameter in async_chat when prompt_config is missing it (#14121)
### What problem does this PR solve?

Resolve #14115 .

## Problem

On the shared chat link page (`/chats/share?shared_id=...`), querying
the knowledge base returns "no relevant information was found", while
the same query works correctly on the editor chat page.

## Root Cause

Knowledge base retrieval in `async_chat()` is gated by the check `if
"knowledge" in param_keys` (line 598), where `param_keys` is derived
from `prompt_config["parameters"]`. If `parameters` is empty or missing
the `{"key": "knowledge", "optional": false}` entry, retrieval is
entirely skipped.

This can happen because `_apply_prompt_defaults()` — which ensures
`parameters` contains the `knowledge` entry — is only called in the
`create` (POST) and `update_chat` (PUT) handlers, but **not** in
`patch_chat` (PATCH). If a chat's `prompt_config` was updated via PATCH
without including `parameters`, the `knowledge` entry would be absent.
Additionally, `prompt_config["parameters"]` would raise a `KeyError` if
the key was missing entirely.

## Fix

Added a defensive safety net in `async_chat()`
(`api/db/services/dialog_service.py`) that auto-injects the `knowledge`
parameter when:
- `dialog.kb_ids` is set (knowledge bases are configured)
- `"knowledge"` is not already in `param_keys`
- `{knowledge}` placeholder exists in the system prompt

Also changed `prompt_config["parameters"]` to
`prompt_config.get("parameters", [])` to prevent `KeyError` when the key
is absent.

## Files Changed

- `api/db/services/dialog_service.py` — added auto-injection of
`knowledge` parameter and safe `.get()` access for `parameters`


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-15 17:31:31 +08:00
Ea001
38cefd88e2 Fix tag_feas code injection in retrieval ranking (#13923)
## Summary
- remove eval-based parsing from retrieval rank feature scoring
- validate `tag_feas` at write time in chunk APIs and SDK routes
- add regression tests for safe parsing and malicious payload rejection

## Details
`tag_feas` is intended to be structured rank-feature data, but the
retrieval ranking path was evaluating stored values as Python
expressions. This change treats `tag_feas` strictly as data.

### What changed
- replace `eval()` in `rag/nlp/search.py` with safe parsing via
`json.loads()` and optional `ast.literal_eval()` compatibility for
legacy Python-dict strings
- strictly filter parsed values down to `dict[str, finite number]`
- reject invalid `tag_feas` payloads at write time in web chunk routes
and SDK document chunk routes
- add focused regression tests to prove executable strings are ignored
and invalid payloads are rejected

## Validation
- `python -m pytest test/unit_test/common/test_tag_feature_utils.py
test/unit_test/rag/test_rank_feature_scores.py -q`

---------

Co-authored-by: unknown <zhenglinkai@CCN.Local>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-04-15 16:31:11 +08:00
Eden
1f33ca1099 fix(dialog): restore decorated answer in async_ask final SSE event (#13917)
## What's the problem

Both `async_chat()` and `async_ask()` call `decorate_answer()` to build
the final SSE payload — it inserts citation markers (`##N$$`) into the
answer text and prunes `doc_aggs` to only the cited documents.
Immediately after, both functions overwrite `final["answer"]` with `""`:

```python
# async_chat(), line ~774  (issue #13828)
final = decorate_answer(thought + full_answer)
final["final"] = True
final["audio_binary"] = None
final["answer"] = ""   # discards decorated text
yield final

# async_ask(), line ~1444  (same bug, different path)
final = decorate_answer(full_answer)
final["final"] = True
final["answer"] = ""   # discards decorated text
yield final
```

The client receives filtered references (built for a citation-decorated
answer it never sees) while displaying the raw, undecorated streaming
text. Citations can never match.

## Root cause

`final["answer"] = ""` was left over from an earlier design where
clients were meant to reconstruct the full answer purely from delta
events. Once `decorate_answer()` started placing citation markers, this
blank-out broke the contract: the final event is where the decorated
answer should land.

## Fix

Remove the two blank-override lines — one in `async_chat()`, one in
`async_ask()`:

```diff
-    final["answer"] = ""
```

`decorate_answer()` already sets `final["answer"]` to the correct
decorated string; there is nothing to override.

## Relation to #13828

Issue #13828 and PR #13835 identify the bug in `async_chat()`. This PR
absorbs that fix and also corrects the identical pattern in
`async_ask()` (used by the `/retrieval` route in `chat_api.py`), which
PR #13835 does not touch.

## Regression test

Added
`test/unit_test/api/db/services/test_dialog_service_final_answer.py`
with three tests:

| Test | Purpose |
|------|---------|
| `test_buggy_pattern_drops_answer` | Documents the old behaviour:
blank-override empties the final answer |
| `test_fixed_pattern_preserves_decorated_answer` | Core invariant:
final event carries the decorated text from `decorate_answer()` |
| `test_final_event_reference_matches_decorated_result` | Citation
markers in the answer must match the pruned `doc_aggs` in the same event
|

Local run result:

```
test_dialog_service_final_answer.py::test_buggy_pattern_drops_answer         PASSED
test_dialog_service_final_answer.py::test_fixed_pattern_preserves_decorated_answer PASSED
test_dialog_service_final_answer.py::test_final_event_reference_matches_decorated_result PASSED

3 passed in 0.04s
```

`ruff check` passes with no issues on all changed files.

---------

Co-authored-by: edenfunf <edenfunf@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-15 14:10:36 +08:00
balibabu
f08d13287a Feat: Edit the code of the code operator from a broad perspective. (#14116)
### What problem does this PR solve?

Feat: Edit the code of the code operator from a broad perspective.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-15 11:51:17 +08:00
chanx
2d291cd841 fix(flow): Fix text descriptions for multi-column layout options. (#14107)
### What problem does this PR solve?

fix(flow): Fix text descriptions for multi-column layout options.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-15 11:50:58 +08:00
Jin Hai
a0a4029f01 Fix document (#14118)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-15 11:35:16 +08:00
Jack
bc5f78996b Consolidateion of document upload API (#14106)
### What problem does this PR solve?

Consolidation WEB API & HTTP API for document upload

Before consolidation
Web API: POST /v1/document/upload
Http API - POST /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API -- POST
/api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-15 11:27:43 +08:00
xinmotlanthua
e1dede1366 fix(web): replace hardcoded English strings with i18n in floating chat widget (#14095)
## Summary
- Replace 3 hardcoded English strings in `floating-chat-widget.tsx` with
`react-i18next` `t()` calls so the widget respects the `locale` query
parameter
- Add `useTranslation` hook to the component
- Add translation keys (`chat.chatSupport`, `chat.replyInstantly`,
`chat.typeYourMessage`) to all 14 locale files

## Strings replaced
| Original | i18n key |
|---|---|
| `'Chat Support'` | `t('chat.chatSupport')` |
| `'We typically reply instantly'` | `t('chat.replyInstantly')` |
| `'Type your message...'` | `t('chat.typeYourMessage')` |

Closes #14072

Co-authored-by: khanhkhanhlele <namkhanh2172@gmail.com>
2026-04-14 20:12:56 +08:00
akie
a98b64326c Add warning log when metadata query hits 10000 result limit (#14109)
## What problem does this PR solve?

Add a warning log when `get_flatted_meta_by_kbs` returns 10,000 results,
which indicates the query limit has been reached and metadata may be
silently truncated.


## Type of change
- [x] Improvement (non-breaking change which improves observability)
2026-04-14 20:04:32 +08:00
NeedmeFordev
1a1b5aa53e Fix: respect the internet toggle before running Tavily web search (#14051) (#14052)
### What problem does this PR solve?

Fixes #14051.

The chat UI already sends an `internet` flag with each request, but the
backend previously triggered Tavily web retrieval whenever
`prompt_config.tavily_api_key` was configured. As a result, web search
could still run even when the internet toggle was off.

This PR makes web search an explicit opt-in at request time:
- `tavily_api_key` only indicates that web search is available
- Tavily retrieval runs only when `internet` is explicitly enabled
- the same behavior now applies to both the normal retrieval path and
the deep-research / reasoning path

This also fixes the no-KB fallback case so chats without KBs fall back
to normal solo chat when `internet` is off.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 19:55:20 +08:00
Jin Hai
8e9cef3687 Remove unused API (#14046)
### What problem does this PR solve?

1. Remove unused token related API
2. Fix typo

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-14 19:32:16 +08:00
chanx
912fedc9b9 Fix: metadata bug (#14105)
### What problem does this PR solve?

Fix: metadata bug

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 18:45:09 +08:00
writinwaters
1c0c1f27ef Doc: Updated FAQ (#14108)
### What problem does this PR solve?

Updated frequently asked questions.

### Type of change


- [x] Documentation Update
2026-04-14 18:42:16 +08:00
balibabu
1bc4868abe Fix: The file count in the file header did not change after uploading or deleting files. (#14034)
### What problem does this PR solve?
Fix: The file count in the file header did not change after uploading or
deleting files.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-14 18:07:32 +08:00
Jack
576431de99 Refactor: Change update doc from PUT to patch (#14067)
### What problem does this PR solve?

Before change, update_document in api/apps/restful_apis/document_api.py
is using "PUT".
After change, it will use "PATCH" which is more suitable.

### Type of change

- [x] Refactoring
2026-04-14 17:12:23 +08:00
Qi Wang
57aec2e65d Fix bug: run Knowledge graph or RAPTOR, it will update an existing task (#14102)
### What problem does this PR solve?

It fixed the bug: https://github.com/infiniflow/ragflow/issues/14101
When run Knowledge graph or RAPTOR, the last document running status
will be wrongly set, see below:
It should never touch existing document result.

![Image](https://github.com/user-attachments/assets/14fe1f9e-0541-4093-8111-ed0bd25b87ba)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 16:37:41 +08:00
balibabu
27ebc64ec0 Feat: Adapted for the upgraded knowledge graph of @antv/g6. (#14103)
### What problem does this PR solve?

Feat: Adapted for the upgraded knowledge graph of @antv/g6.

### Type of change

- [x] Refactoring
2026-04-14 16:33:52 +08:00
Magicbook1108
1376c004a9 Fix: update docs generator (#14070)
### What problem does this PR solve?

Refactor: update docs generator

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

1. Support multiple document generator components and correctly display
messages in the message component. The document generator will not
overwrite other messages.

<img width="700" alt="Screenshot from 2026-04-13 13-56-17"
src="https://github.com/user-attachments/assets/3f3e06e8-33ce-4df1-8b05-510c86af70a4"
/>

2. Support Chinese content and ensure correct Markdown rendering in PDF
and DOCX
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/69bf1f7b-261d-48e5-a9f3-8e94462b90ed"
/>

3. Simplify configuration page and support more output format
 
<img height="700" alt="image"
src="https://github.com/user-attachments/assets/8647374c-c055-4daa-ad71-cd9052eb138e"
/>

4. Hide download from other components except for message 
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/a723dfcb-b60d-4eb5-b2f6-d41ca5955eb4"
/>

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/a8762ac4-807b-4f0b-9287-65f82f7c9c98"
/>

5. Sanitize filename
 
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/df49509f-37c0-40f9-b03d-bd6ce7fdefa8"
/>


6. And more changes on usability
2026-04-14 15:24:43 +08:00
chanx
1031aebc8f feat(file): Add file ancestor directory lookup feature by go (#14037)
### What problem does this PR solve?

feat(file): Add file ancestor directory lookup feature by go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-14 15:22:03 +08:00
chanx
6aec8058bb refactor: Remove knowledge base-related API handlers that are already included in the dataset. (#14094)
### What problem does this PR solve?

refactor: Remove knowledge base-related API handlers that are already
included in the dataset.

### Type of change

- [x] Refactoring
2026-04-14 15:19:31 +08:00
Jin Hai
2b6c50734f Sync code from EE (#14080)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-14 15:03:46 +08:00
Ricardo-M-L
c22811f096 fix: close file handles in json.load() calls in resume parser (#14061)
## Summary
- Replace `json.load(open(...))` with `with open(...) as f:
json.load(f)` in 2 resume parser files
- Fixes 4 leaked file descriptors in `corporations.py` (3) and
`schools.py` (1)

## Why
In a long-running server process like RAGFlow, leaked file handles can
accumulate and hit the OS file descriptor limit (`OSError: [Errno 24]
Too many open files`). The other instances mentioned in the issue
(`infinity_conn_base.py` and `init_data.py`) have already been fixed.

## Test plan
- [x] Verified affected files use `with` statement after fix
- [x] Grep confirms no remaining `json.load(open(` patterns in codebase

Fixes #13996

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-14 11:43:58 +08:00
Idriss Sbaaoui
de6a8e789a Fix: rerank overflow by enforcing top_k and 64 cap (#14084)
### What problem does this PR solve?

This fixes rerank overflow where retrieval could send more documents
than allowed (for example 66 when `page_size=6`), causing provider 400
errors and bypassing the user’s `top_k` intent in rerank-enabled paths.
this pr fixes #14081

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 10:47:25 +08:00
Idriss Sbaaoui
d6987b4d8f Fix p3 ci fails (#14069)
### What problem does this PR solve?

fix issue with stale tests on p3 level

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-14 10:47:07 +08:00
balibabu
d2b744facd Fix: The indented tree text generated on the search page overlaps. #14077 (#14078)
### What problem does this PR solve?

Fix: The indented tree text generated on the search page overlaps.
#14077

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-14 10:02:00 +08:00
Magicbook1108
8723c3aa86 Feat: more templates (#14075)
### What problem does this PR solve?

Feat: more templates
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/533e88f1-fc56-4337-a026-6623fc978893"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
2026-04-14 10:00:55 +08:00
chanx
6ffa566ec3 Refactor: Standardize naming convention to camelCase (#14079)
### What problem does this PR solve?

Refactor: Standardize naming convention to camelCase

### Type of change

- [x] Refactoring
2026-04-13 21:07:07 +08:00
balibabu
9a38af7cbf Feat: Hide the download button embedded in the agent page. (#14083)
### What problem does this PR solve?

Feat: Hide the download button embedded in the agent page.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-13 21:06:41 +08:00
Syed Shahmeer Ali
c7ce062ea8 Fix: model_type not passed in ensure_tenant_model_id_for_params causing wrong tenant model lookup (#13782)
Summary

When setting a default model for an OpenAI-API-Compatible provider,
ensure_tenant_model_id_for_params called get_api_key
without a model_type filter. If the same model name was registered under
multiple types (e.g., both chat and embedding),
it could return the wrong tenant_llm_id, leading to Model(@None) not
authorized errors during chat.

This applies the same type-scoped fix that PR #13569 introduced in
get_model_config_by_type_and_name — now consistently
  in tenant_utils.py as well.

  Changes

  - Added _KEY_TO_MODEL_TYPE mapping in tenant_utils.py
- Each model key (llm_id, embd_id, etc.) now passes its correct LLMType
to get_api_key

  Fixes #13775
2026-04-13 20:57:28 +08:00
天海蒼灆
356d45fda1 Feat: add cell type coercion for Excel export (#13808)
### What problem does this PR solve?

- Implemented a helper function to convert markdown cell text to native
numeric types for Excel output.
- Ensured that leading zeros are preserved and handled various numeric
formats, including those with thousand separators and scientific
notation.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-13 20:54:57 +08:00
Lynn
47d3741dcc Feat: migrate script (#14076)
### What problem does this PR solve?

Add command line arguments for mysql config.

### Type of change

- [x] Other (please describe): tool scripts.
2026-04-13 20:45:11 +08:00
bitloi
853021ff2a feat: support multiple canvas_types for agent templates and remove duplicate files (#14030)
### What problem does this PR solve?

Closes #13907

The template catalog had duplicate files (e.g. `*_r.json`) only to place
the same template into multiple sidebar groups.
This increases maintenance cost and makes template updates error-prone.

This PR adds first-class support for multiple template categories in a
single file via `canvas_types`, then removes duplicate template files.

What changed:
- Added `canvas_types` to `CanvasTemplate` model and DB migration.
- Added normalization logic when loading templates:
  - accepts legacy `canvas_type`
  - accepts new `canvas_types`
  - merges/deduplicates values
- preserves backward compatibility by keeping `canvas_type` as first
normalized value.
- Updated template import flow to load only `.json` files and in stable
sorted order.
- Updated frontend template filtering to match on `canvas_types` first,
with fallback to legacy `canvas_type`.
- Consolidated duplicated template pairs into single files and removed:
  - `deep_search_r.json`
  - `reflective_academic_paper_generator_r.json`
  - `seo_article_writer_r.json`
- Added regression/edge-case tests for category normalization and route
serialization expectations.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-13 20:26:30 +08:00
writinwaters
ef07faea80 Doc: Updated frequently asked questions and answers. (#14085)
### What problem does this PR solve?

Updated frequently asked questions. 

### Type of change

- [x] Documentation Update
2026-04-13 20:26:16 +08:00
Tong Liu
6fdca2d212 [Security] Fix jinja2 SSTI vulnerability using SandboxedEnvironment (#14068) 2026-04-13 19:24:13 +08:00
balibabu
a023305b96 Fix: The chat page is not displaying the meta tags. (#14071)
### What problem does this PR solve?

Fix: The chat page is not displaying the meta tags.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-13 16:18:25 +08:00
Krishna Chaitanya
5ece2d8aa8 Fix: upgrade Apache Tika from 3.2.3 to 3.3.0 to address GHSA-72hv-8253-57qq (#13769)
### What problem does this PR solve?

Upgrades Apache Tika from 3.2.3 to 3.3.0 to address the security
vulnerability GHSA-72hv-8253-57qq (TIKA-4687).

Closes #13601

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- `Dockerfile`: Updated tika JAR filename and `TIKA_SERVER_JAR` env var
from 3.2.3 to 3.3.0
- `Dockerfile.deps`: Updated tika JAR filename in COPY instruction from
3.2.3 to 3.3.0
- `download_deps.py`: Updated both Maven Central and Huawei Cloud mirror
download URLs from 3.2.3 to 3.3.0

### References

- Apache Tika 3.3.0 release:
https://www.apache.org/dyn/closer.lua/tika/3.3.0/tika-app-3.3.0.jar
- TIKA-4687: https://issues.apache.org/jira/browse/TIKA-4687
- GHSA-72hv-8253-57qq
2026-04-13 16:01:08 +08:00
Jin Hai
3e787b3b09 Go: update search (#14023)
### What problem does this PR solve?

Update search

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-13 15:07:04 +08:00
Yongteng Lei
1638083e18 Fix: sandbox cannot accept large args list (#14063)
### What problem does this PR solve?

Sandbox cannot accept large args list.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-13 14:14:08 +08:00
Jack
51ce6aab01 Consolidate set_meta into update_document (#14045)
### What problem does this PR solve?

Consolidate "set_meta" API into "update_document" .

Before consolidation
Web API: POST /api/v1/document/set_meta
Http API - PUT /v1/datasets/<dataset_id>/document/<document_id>

After consolidation, Restful API -- PUT
/v1/datasets/<dataset_id>/document/<document_id>

### Type of change

- [x] Refactoring
2026-04-13 12:47:17 +08:00
akie
3911d90993 Fix: agent application can not show Cite (#14047)
Close #14018

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Problem
In Agent applications, even with the cite option enabled, only inline
[ID: x] citation markers are visible (showing chunk content on hover).
The Agent does not display the referenced file cards below the response,
unlike Chat applications.

### Root Cause
The Agent's Retrieval tool (agent/tools/retrieval.py) calls
retriever.retrieval() with aggs=False, which means the retrieval results
do not include doc_aggs (document aggregation) data. Without doc_aggs,
the frontend ReferenceDocumentList component has no data to render the
file cards.

In contrast, the Chat application (api/db/services/dialog_service.py)
calls the same retriever.retrieval() method with aggs=True.

### Fix
Changed aggs=False to aggs=True in agent/tools/retrieval.py so that
document aggregation data is returned along with the retrieved chunks.
2026-04-13 11:06:14 +08:00
writinwaters
52442c8eb5 Docs: Added a guide on adding Github repo as data source (#14048)
### What problem does this PR solve?

Added a guide on adding Github repo as data source

### Type of change


- [x] Documentation Update
2026-04-10 21:32:26 +08:00
balibabu
462be53b76 Fix: When creating a dataset, if no chunk_method is selected, there is no indication that this is a required field. (#14039)
### What problem does this PR solve?
Fix: When creating a dataset, if no `chunk_method` is selected, there is
no indication that this is a required field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 19:05:14 +08:00
Magicbook1108
82d74fd276 Refact: update pipeline template (#14036)
### What problem does this PR solve?

Refact: update pipeline template

### Type of change

- [x] Refactoring
2026-04-10 19:04:52 +08:00
Jack
4046a4cfb6 Consolidateion metadata summary API (#14031)
### What problem does this PR solve?

Consolidation WEB API & HTTP API for document metadata summary

Before consolidation
Web API: POST /api/v1/document/metadata/summary
Http API - GET /v1/datasets/<dataset_id>/metadata/summary

After consolidation, Restful API -- GET
/v1/datasets/<dataset_id>/metadata/summary

### Type of change

- [x] Refactoring
2026-04-10 18:41:30 +08:00
balibabu
11c89d87da Fix: The dataset on the search page is not displaying the required field error message. (#14041)
### What problem does this PR solve?

Fix: The dataset on the search page is not displaying the required field
error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 18:20:50 +08:00
Zhichang Yu
a9ca4ea1a1 Disable flask and quart debug (#14042)
### What problem does this PR solve?

Visit
`http://127.0.0.1:9381/?__debugger__=yes&cmd=resource&f=debugger.js`
will expose the flask code:
```
docReady(() => {
  if (!EVALEX_TRUSTED) {
    initPinBox();
  }
  // if we are in console mode, show the console.
  if (CONSOLE_MODE && EVALEX) {
    createInteractiveConsole();
  }

  const frames = document.querySelectorAll("div.traceback div.frame");
  if (EVALEX) {
    addConsoleIconToFrames(frames);
  }
  addEventListenersToElements(document.querySelectorAll("div.detail"), "click", () =>
    document.querySelector("div.traceback").scrollIntoView(false)
  );
  addToggleFrameTraceback(frames);
  addToggleTraceTypesOnClick(document.querySelectorAll("h2.traceback"));
  addInfoPrompt(document.querySelectorAll("span.nojavascript"));
  wrapPlainTraceback();
});

function addToggleFrameTraceback(frames) {
  frames.forEach((frame) => {
    frame.addEventListener("click", () => {
      frame.getElementsByTagName("pre")[0].parentElement.classList.toggle("expanded");
    });
  })
}

```

### Type of change

- [x] Other (please describe): Fix security risk
2026-04-10 18:01:49 +08:00
Jin Hai
cfc2928de2 Go: remove unused API route (#14028)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 18:00:41 +08:00
Jin Hai
3d59448b0d Go: add parameter parsing of list chats (#14026)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 14:33:32 +08:00
Magicbook1108
18cafff790 Fix: markdown parser in pipeline (#14032)
### What problem does this PR solve?

Fix: markdown parser in pipeline

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 14:11:14 +08:00
Magicbook1108
9ce293a736 Refact: update exesql notification (#14027)
### What problem does this PR solve?

Refact: update exesql notification

### Type of change


- [x] Refactoring
2026-04-10 13:42:57 +08:00
Magicbook1108
87a87a7122 Feat: pipeline support ONE chunking method (#14024)
### What problem does this PR solve?

Feat: pipeline support ONE chunking method

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-10 13:11:22 +08:00
Jin Hai
a37605cbd2 Go: add get chat (#14025)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 13:06:51 +08:00
eason
aa92abe73c fix: close file handles properly in json.load() calls (#13997)
## Summary

Fixes #13996

Replace `json.load(open(...))` with `with open(...) as f: json.load(f)`
in two files to ensure file descriptors are properly closed.

**Affected files:**
- `common/doc_store/infinity_conn_base.py` — schema loading for Infinity
doc store
- `api/db/init_data.py` — agent template loading at startup

## Why this matters

In a long-running server process like RAGFlow, leaked file descriptors
from `json.load(open(...))` can accumulate over time. While CPython's
refcounting usually cleans these up, it's not guaranteed (especially
under memory pressure or with alternative Python runtimes), and can lead
to `OSError: [Errno 24] Too many open files`.

## Test plan

- [ ] Verify Infinity doc store schema loading still works correctly
- [ ] Verify agent templates load correctly on startup

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Improved file handling in internal data processing to ensure proper
resource cleanup.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: easonysliu <easonysliu@tencent.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-04-10 12:16:49 +08:00
chanx
4538910b52 feat: Implement file-related functionality (#14011)
### What problem does this PR solve?

feat: Implement file-related functionality

- Implement file deletion API and business logic
- Add context support for file deletion operations and prevent root
folder deletion
-  Implement file move functionality
-  Add File Download API Endpoints and Utility Functions

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-10 12:15:27 +08:00
corevibe555
e7d044413f Fix: Google Drive connector missing new files after initial sync (#13943)
Closes https://github.com/infiniflow/ragflow/issues/13939

## What problem does this PR solve?

The Google Drive connector fails to detect new files after the initial
sync (#13939). The root cause is that `generate_time_range_filter()`
applies a strict `modifiedTime > poll_range_start` cutoff when querying
the Google Drive API. Files uploaded to Google Drive that retain their
original `modifiedTime` (common behavior) get silently excluded if their
timestamp predates the last sync's cutoff.

Unlike the Confluence and Jira connectors which use a configurable time
buffer (`CONFLUENCE_SYNC_TIME_BUFFER_SECONDS`) to offset
`poll_range_start` backward, the Google Drive connector had no such
mechanism — resulting in a razor-sharp timestamp boundary with zero
tolerance for overlap.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


## Summary

* **New Features**
* Added a configurable time buffer for Google Drive synchronization to
address timing delays and improve sync reliability.
* Improved file detection logic to include recently created files
alongside modified ones, reducing missed synchronizations.
2026-04-10 11:39:19 +08:00
NeedmeFordev
7315d25cbc Fix retrieval API handling for omitted dataset IDs (#13990)
### What problem does this PR solve?

This PR fixes a mismatch between the MCP retrieval contract and the
backend retrieval API.

`ragflow_retrieval` already describes `dataset_ids` as optional, but
`/api/v1/retrieval` still rejected omitted or empty `dataset_ids` with
`` `dataset_ids` is required. ``. That made MCP retrieval fail even
though the tool schema promised that the request could search across all
available datasets.

This change updates `/api/v1/retrieval` to accept missing or empty
`dataset_ids`, resolve all accessible datasets for the authenticated
user, and keep the route schema aligned with the new runtime behavior.
It also adds focused unit coverage for the fallback resolution path and
the no-accessible-datasets case.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Fixes: #13981

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved dataset resolution to reliably discover all accessible
datasets through proper pagination, replacing the previous parsing
method.
* Enhanced error handling with clearer messaging when no datasets are
available for retrieval.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-10 11:34:15 +08:00
Magicbook1108
27329b40ed Refact: refact on parser structure (#14012)
### What problem does this PR solve?

Refact: refact on parser structure

### Type of change

- [x] Refactoring
2026-04-10 10:03:44 +08:00
Jin Hai
cd04467b9b Go: add delete search (#14014)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-10 09:42:37 +08:00
balibabu
56810ec5a3 Fix: The knowledge base selected by the retrieval node is not displayed. (#14013)
### What problem does this PR solve?

Fix: The knowledge base selected by the retrieval node is not displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 09:40:35 +08:00
Magicbook1108
52f5880d21 Fix: support vlm fall back in pipeline (#14007)
### What problem does this PR solve?

Fix: support vlm fall back in pipeline for img/table parsing

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-09 20:20:11 +08:00
Jin Hai
5951e2b564 Go: Add create search (#13998)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 20:04:06 +08:00
Yongteng Lei
b33d2fdea5 Refa: GraphRAG to use async chat methods instead of thread pool execution (#14002)
### What problem does this PR solve?

GraphRAG _async_chat.

### Type of change

- [x] Refactoring
- [x] Performance Improvement


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* Unified chat calls to an async invocation across extractors, improving
timeout handling and ensuring task IDs propagate reliably.
* **Tests**
* Added and expanded unit tests and mocks to cover extractor behavior,
timeout scenarios, and safe test-package imports, reducing regression
risk.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-09 19:57:35 +08:00
Octopus
c2ce49e037 fix: strip single quotes from synonym terms to prevent Infinity TokenError (#13969)
Fixes #13823

## Problem

When querying with words like `cat`, RAGFlow's query expansion system
looks up synonyms via WordNet, which can return terms containing single
quotes (e.g., `cat-o'-nine-tails`). When using Infinity as the document
store, these unescaped single quotes in the query string cause a
`TokenError` because Infinity's lexer treats `'` as a string delimiter.

```
TokenError: Error tokenizing ' OR "big cat" OR "computerized tomography")^0.7)': Missing ' from 1:531
```

## Solution

Strip single quotes from synonym terms before they are inserted into
query expressions, consistent with how single quotes are already
stripped from the input query text (line 51 of `query.py`):

- **`common/query_base.py`**: In `sub_special_char()`, strip `'` before
escaping other special characters. This fixes the Chinese text
processing path and the `paragraph()` method.
- **`rag/nlp/query.py`**: In the English text path, strip `'` from
tokenized synonym terms.
- **`memory/services/query.py`**: Same fix for the memory query English
text path.

## Testing

The fix can be verified by:
1. Using Infinity as the document store (`DOC_ENGINE=infinity`)
2. Creating a dataset and running a retrieval test with the keyword
`cat`
3. Confirming no `TokenError` is raised and results are returned
normally

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced special character handling in query processing and synonym
expansion by properly sanitizing single quotes before text processing.
* Simplified OCR detection output by removing timing metadata while
preserving core detection accuracy.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: ximi <octo-patch@github.com>
2026-04-09 19:10:34 +08:00
Jin Hai
e2b879b258 Fix tiny issues (#14006)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Improved authentication error logging to better distinguish between
JWT and API token failures.
* Enhanced code documentation with clarifying comments for better
maintainability.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 19:01:36 +08:00
balibabu
3c5a3e5fb4 Feat: Integrate the name, avatar, and description of chat and search into a single component. (#14008)
### What problem does this PR solve?

Feat: Integrate the name, avatar, and description of chat and search
into a single component.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
  * Inline-editable avatar, name, and description fields
  * Expandable content blocks in search results
  * New RAGFlow heading/logo component

* **Refactor**
* Replaced scattered form fields with a composed Avatar/Name/Description
component
  * Mindmap drawer converted to a sheet-based drawer and layout cleanup
* Simplified search page controls and layout; improved scroll viewport
handling

* **Chores**
* Added/updated English and Chinese localization keys (placeholders,
view more/less)
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-09 18:51:45 +08:00
eviaaaaa
1e83c8c051 Fix: align MCP tool call timeout and handle empty content (#13899)
### What problem does this PR solve?
Resolves #12105
This PR fixes two MCP tool call issues in
`common/mcp_tool_call_conn.py`.
First, the timeout passed to `tool_call(..., timeout=...)` was only
applied to the outer `future.result(...)` wait, but was not forwarded to
the internal MCP request. As a result, callers could pass a longer
timeout while the actual MCP request still failed after the default
internal timeout.
Second, the MCP tool call result handling assumed `result.content[0]`
always existed. If an MCP server returned an empty content list, this
could raise an exception unexpectedly.
This PR fixes both issues by:
- forwarding the external `timeout` value to the internal MCP request
timeout
- returning a clear message when the MCP server returns empty content
instead of indexing into an empty list

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update                                             
- [ ] Refactoring                               
- [ ] Performance Improvement
- [ ] Other (please describe)
2026-04-09 18:44:04 +08:00
Zhichang Yu
b7744e053e fix: support dense_vector from ES fields response (ES 9.x compatibility) (#13972)
fix: support dense_vector from ES fields response (ES 9.x compatibility)

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Configuration Chore (non-breaking change which updates
configuration)


## Summary by CodeRabbit

* **Bug Fixes**
* More accurate handling and unwrapping of dense-vector fields so
returned values have correct shapes.
* Field selection reliably limits returned data and falls back to
alternate result locations when needed.
* Use of consistent result IDs and tolerant handling when score values
are missing.

* **Chores / Configuration**
* Increased build memory and adjusted build-time flags for the frontend
build.
* Simplified runtime model/GPU checks and removed an automated runtime
GPU-install attempt.

* **Build Fixes**
* `web/vite.config.ts`: make `build.minify` and `build.sourcemap`
respect `VITE_MINIFY` and `VITE_BUILD_SOURCEMAP` env vars from
Dockerfile instead of hardcoding `terser` and `true`.

* **Environment**
* Allow stack version override and default the runtime image tag to
"latest".

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Correct unwrapping of dense-vector fields and reliable field selection
with fallback locations.
* Consistent use of hit-level IDs and tolerant handling when score
values are missing.

* **Chores / Configuration**
* Increased frontend build memory and added build-time minify/sourcemap
flags; build minification and sourcemap now configurable.
* Removed runtime GPU detection for model initialization; force CPU
initialization.

* **Environment**
* Allow stack version override and default runtime image tag to
"latest".

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 17:44:13 +08:00
Magicbook1108
107fe6cf90 Feat: support doc for pipeline parser in word (#14005)
### What problem does this PR solve?

Feat: support doc for pipeline parser in word

### Type of change

- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added support for processing legacy Word `.doc` file formats,
extending document compatibility.

* **Bug Fixes**
* Enhanced error handling during document parsing to improve reliability
and prevent processing failures.
2026-04-09 16:40:42 +08:00
Magicbook1108
8d52ef2893 Feat: enable sync deleted files for connector (#14000)
### What problem does this PR solve?

Feat: enable sync deleted files for connector
1. first comes with github

### Type of change

- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added "sync deleted files" feature for data sources, enabling
automatic removal of files deleted from the source system.
* Added multilingual support for the new sync deleted files setting
across multiple languages.

* **UI Improvements**
  * Improved checkbox form field rendering and layout.
  * Enhanced full-width display for authentication token input fields.
2026-04-09 16:40:14 +08:00
Jack
577c96bf2a Refactor: Merge document update API (#13962)
### What problem does this PR solve?

Refactor: merge document.rename into document.update_document

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a unified document update API (PUT) supporting name, metadata,
parser/chunk settings, and status changes.

* **Breaking Changes**
* Legacy single-parameter rename endpoint removed; renames now require
dataset + document identifiers.
  * `/list` now reads dataset id from a different query parameter.

* **Validation / Bug Fixes**
* Stricter meta_fields and parser-config validation; unauthenticated
requests return 401.

* **Frontend**
  * UI now sends dataset id when saving document names.

* **Tests**
* Numerous unit and HTTP tests adjusted or removed to match new API and
validations.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: MkDev11 <94194147+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
Co-authored-by: Qi Wang <wangq8@outlook.com>
Co-authored-by: dataCenter430 <161712630+dataCenter430@users.noreply.github.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
2026-04-09 11:17:38 +08:00
Ricardo-M-L
c13f8856a1 fix: correct typos in agent component filename and templates (#13930)
## Summary
- Rename misspelled file `varaiable_aggregator.py` →
`variable_aggregator.py`
- Fix `unkown` → `unknown` in template and frontend constant (3
instances)
- Fix `Finale` → `Final` in customer feedback template (2 instances)

## Test plan
- [ ] Verify variable aggregator component loads correctly
- [ ] Verify agent templates render properly

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-09 11:06:01 +08:00
Lynn
dbfb439239 Feat: migrate script (#13976)
### What problem does this PR solve?

Add stage for migrate tenant_llm data into table tenant_model_instance
and tenant_model.

### Type of change

- [x] Other (please describe): tool script


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Chores**
* Added two new migration stages to move tenant model and instance
records into new target tables, with dry-run, full-execute, and "create
table only" modes; migration skips already-migrated rows to avoid
duplicates.
* **Bug Fixes**
  * Cleaned up migration header logging for clearer output.
* **Documentation**
* Added usage guide describing stages, options, modes, config format,
examples, and expected logs.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-09 11:03:39 +08:00
Magicbook1108
c5871c1078 Fix: dsl import/export (#13992)
### What problem does this PR solve?

Fix: dsl import/export
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Enhanced JSON import functionality for agents to automatically
populate components from imported graph structures.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 10:55:22 +08:00
qinling0210
82fa85c837 Implement Delete in GO and refactor functions (#13974)
### What problem does this PR solve?

Implement Delete in GO and refactor functions

### Type of change

- [x] Refactoring

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Added a remove_chunks command to delete specific or all chunks from a
document.
  * Added new endpoints for chunk removal and chunk update.

* **Refactor**
* Renamed index commands to dataset/metadata table terminology and
updated REST routes accordingly.
* Updated chunk update flow to a JSON POST style and improved metadata
error messages.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
2026-04-09 09:52:31 +08:00
Jack
3b7723855c Fix: revert xgboost version to 1.6.0 (#13984)
### What problem does this PR solve?

Revert xgboost version to 1.6.0

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Updated xgboost dependency from version 3.2.0 to 1.6.0

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 19:53:47 +08:00
Jin Hai
5fe6f7c9ac Go CLI: Add list configs and set log level command (#13983)
### What problem does this PR solve?

1. list configs
2. set log level debug/info/warn/error/fatal/panic

```

RAGFlow(user)> list configs;
+--------------------+-----------------------+
| key                | value                 |
+--------------------+-----------------------+
| redis_host         | localhost:6379        |
| doc_engine         | elasticsearch         |
| elasticsearch_host | http://localhost:1200 |
| log_level          | info                  |
| database           | mysql                 |
| database_host      | localhost:3306        |
| admin              | 0.0.0.0:9383          |
| storage_engine     | minio                 |
| minio_host         | localhost:9000        |
+--------------------+-----------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added `LIST CONFIGS` command to view system configuration details
(Redis, database, log level, storage engine, and host settings).
* Added `SET LOG LEVEL` command to adjust logging verbosity at runtime.

* **Improvements**
* Enhanced log level configuration defaults and runtime state
management.
* Reorganized token management and system endpoints under `/system/`
routes for better API organization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 19:32:53 +08:00
balibabu
86900dca99 Refactor: Remove unused API code (#13978)
### What problem does this PR solve?

Refactor: Remove unused API code

### Type of change


- [x] New Feature (non-breaking change which adds functionality)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Style**
* Updated table header styling in dataset settings by removing a
hard-coded background color class, allowing the header to use default or
inherited component styling instead.

* **Refactor**
* Removed token management endpoints from the API service. Token
creation, listing, and removal functions are no longer available.
  * Removed the statistics data endpoint from available API routes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 18:46:08 +08:00
balibabu
c0c3287af4 Fix: Error message: Use 'const' instead. (#13982)
### What problem does this PR solve?

Fix: Linter error message: Use 'const' instead.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Updated variable declarations across form components, agent utilities,
memory management hooks, and data handling functions to enhance code
consistency and maintainability throughout the application codebase.

* **Style**
* Added ESLint suppressions to document intentional constant-condition
patterns in asynchronous event streaming operations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 18:13:14 +08:00
Yongteng Lei
3064895bbb Fix: import error in sandbox provider (#13971)
### What problem does this PR solve?

Fix import error in sandbox provider.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
* Updated internal configuration import mechanism for sandbox provider
initialization. No end-user impact.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 15:35:30 +08:00
Jin Hai
fa75aee3b9 Refactor system API (#13958)
### What problem does this PR solve?

- ping
- token
- log level

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Refactor**
* System endpoints consolidated under /api/v1/system: ping, health
check, and token management moved to the centralized API surface.
* Token management unified at /api/v1/system/tokens with
list/create/delete behavior.

* **Documentation**
  * API reference updated to reflect the new /api/v1/system paths.

* **Tests**
* Client fixtures and test utilities updated to use
/api/v1/system/tokens; one unit test for health/oceanbase status
removed.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 15:26:18 +08:00
Jin Hai
ad789f5c43 Fix list files (#13960)
### What problem does this PR solve?

As title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Standardized the query parameter used when listing documents so
listings behave consistently across the web and client interfaces.
* Clarified the error message shown when a required dataset ID is
missing to give clearer guidance to users.

* **Tests**
* Updated test coverage to reflect the standardized dataset identifier
usage.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 13:38:30 +08:00
balibabu
b8764cfa11 Fix: The document management table cannot be displayed. (#13967)
### What problem does this PR solve?

Fix: The document management table cannot be displayed.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved table layout and overflow behavior in the files view to
ensure proper scrolling and display.

* **Chores**
* Removed unused system status functionality and cleaned up service
methods.
  * Updated TypeScript configuration for compatibility.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 11:37:27 +08:00
dataCenter430
62a1333cf2 Feat: expose parent-child chunking configuration via HTTP API and Python SDK (#13940)
…
### What problem does this PR solve?

Closes #13857

Parent-child chunking was introduced in v0.23.0 but is only configurable
through the web UI. Users managing datasets programmatically cannot
enable it via the HTTP API or Python SDK because `ParserConfig` uses
`extra="forbid"`, rejecting the `children_delimiter` field at
validation.

### What does this PR change?

Adds a `parent_child` nested config to `ParserConfig`, following the
same pattern as `raptor` and `graphrag`:

```json
"parser_config": {
  "parent_child": {
    "use_parent_child": true,
    "children_delimiter": "\n"
  }
}
```

- api/utils/validation_utils.py — new ParentChildConfig model, added to
ParserConfig
- api/utils/api_utils.py — naive defaults + flatten to
children_delimiter for the execution layer
- api/apps/services/dataset_api_service.py — flatten on the update path
- test/testcases/configs.py — updated DEFAULT_PARSER_CONFIG
-
test/testcases/test_http_api/test_dataset_management/test_create_dataset.py
— 4 valid + 2 invalid test cases

No changes to the execution layer (rag/app/naive.py, rag/nlp/search.py).
Existing UI flow via ext is unaffected.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
* Added parent-child chunking configuration for dataset creation and
updates with new `use_parent_child` toggle and customizable
`children_delimiter` setting to specify how parent chunks are split into
child chunks.

* **Documentation**
* Updated HTTP and Python API references with parent-child chunking
configuration details and examples.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 11:36:57 +08:00
Qi Wang
0ced071a0b Use uv run python3 x.py instead of uv run x.py (#13966)
### Use uv run python3 x.py instead of uv run x.py

When directly call `uv run x.py` it will use the python in shebang, it
does not work if the default python lack of some packages, so change it
to best practices `uv run python3 x.py`

### Type of change

- [x] Documentation Update

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **Documentation**
* Updated development setup instructions across all README files
(English and multiple language translations) to use explicit Python
interpreter invocation for the dependency download command.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-04-08 10:33:46 +08:00
MkDev11
cfee2bc9db feat: Auto-adjust chunk recall weights based on user feedback (#12689)
### What problem does this PR solve?

Implements automatic adjustment of knowledge base chunk recall weights
based on user feedback (upvotes/downvotes). When users upvote or
downvote a response, the system locates the corresponding knowledge
snippets and adjusts their recall weight to improve future retrieval
quality.

**Closes #12670**

**How it works:**
1. User upvotes/downvotes a response via `POST /thumbup`
2. System extracts chunk IDs from the conversation reference
3. For each referenced chunk:
   - Reads current `pagerank_fea` value from document store
   - Increments (+1) for upvote or decrements (-1) for downvote
   - Clamps weight to [0, 100] range
   - Updates chunk in ES/Infinity/OceanBase
4. Future retrievals score these chunks higher/lower based on
accumulated feedback

**Files changed:**
- `api/db/services/chunk_feedback_service.py` - New service for updating
chunk pagerank weights
- `api/apps/conversation_app.py` - Integrated feedback service into
thumbup endpoint
- `test/testcases/test_web_api/test_chunk_feedback/` - Unit tests

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Chat message feedback now updates per-chunk relevance weights
(feature-flag gated), with configurable weighting and atomic updates
across storage backends.

* **Bug Fixes**
* Stricter validation for message feedback inputs and more robust
handling of feedback transitions.

* **Tests**
* Expanded test coverage for chunk-feedback behavior, weighting
strategies, storage backends, and thumb-flip scenarios.

* **Chores**
  * CI workflow extended to run the new chunk-feedback web API tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
2026-04-08 09:52:18 +08:00
Jin Hai
4a2a17c27a Fix typos (#13961)
### What problem does this PR solve?

as title.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Chores**
  * Internal code quality improvements with no user-facing changes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 23:16:52 +08:00
Jin Hai
931021875a Refactor system/version API to RESTful style (#13956)
### What problem does this PR solve?

Refactor version API to RESTful style. Python and go server API also
updated.
### Type of change

- [x] Refactoring



<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

## Release Notes

* **Refactor**
* Migrated core API endpoints to the `/api/v1/` namespace for improved
consistency and organization.
* Standardized system version, search, and chat list endpoints under the
new API versioning structure.

* **New Features**
* Added MinIO region configuration support, allowing specification of
storage engine regional settings via environment variables or
configuration files.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 19:07:47 +08:00
Yang_Ming
bc8d67ce78 feat: add region parameter support to MinIO connection (#13954)
## Summary
- Add optional `region` parameter to `Minio()` client constructor in
`rag/utils/minio_conn.py`
- Reads from `MINIO.region` in settings, defaults to `None` when not
configured
- Required by some S3-compatible storage services (e.g., AWS S3, Tencent
COS) for proper bucket access

## Motivation
When using RAGFlow with S3-compatible storage that requires a region
(such as AWS S3 or Tencent Cloud COS), the MinIO client fails to access
buckets because the `region` parameter is not passed through.

The `Minio()` Python client already supports the `region` parameter
natively — this PR simply wires it up from the RAGFlow configuration.

## Changes
- `rag/utils/minio_conn.py`: Pass `region=settings.MINIO.get("region",
None) or None` to `Minio()` constructor

## Backward Compatibility
- No breaking changes. When `region` is not configured, it defaults to
`None`, preserving the existing behavior exactly.

## Test Plan
- [ ] Verified with MinIO (no region set) — works as before
- [x] Verified with S3-compatible storage requiring region — bucket
access succeeds

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Enhanced MinIO client initialization with regional configuration
support for improved compatibility with region-specific deployments.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: Jarry Wang <code-better-life@users.noreply.github.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 16:38:23 +08:00
Jin Hai
68f665be7a CLI: Add float parsing (#13955)
### What problem does this PR solve?

Add float parsing

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 15:09:45 +08:00
Jin Hai
393efa9b7c Refactor variable of front end (#13953)
### What problem does this PR solve?

api_host -> webAPI
ExternalApi -> restAPIv1

### Type of change

- [x] Refactoring


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Refactor**
* Updated internal API endpoint configuration to use consolidated base
URL constants for improved maintainability and consistency across the
application.

* **Chores**
* Updated server-side protocol validation for admin connectivity checks.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 15:08:11 +08:00
balibabu
38acf34724 Fix: The agent selected a knowledge base, but the API returned the error: "No dataset is selected". (#13950)
### What problem does this PR solve?

Fix: The agent selected a knowledge base, but the API returned the
error: "No dataset is selected".

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-04-07 14:16:37 +08:00
auyua9
fa08fa2a17 docs: fix broken internal links in guides (#13935)
### What problem does this PR solve?

This fixes two broken internal documentation links in the guides:

- `docs/develop/mcp/launch_mcp_server.md` linked
`./acquire_ragflow_api_key.md`, but the target page lives one level up
as `../acquire_ragflow_api_key.md`.
- `docs/guides/dataset/run_retrieval_test.md` linked
`./construct_knowledge_graph.md`, but the actual page lives under
`./advanced/construct_knowledge_graph.md`.

These broken links make it harder to follow the MCP and retrieval-test
docs from the local docs tree.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-04-07 14:01:12 +08:00
Jin Hai
9ac5d28f06 Refactor context command (#13952)
### What problem does this PR solve?

Refactor context search command

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 13:59:27 +08:00
Ricardo-M-L
424aee5bec fix: correct typos in code comments, docstrings and docs (#13931)
## Summary
- Fix `a image` → `an image` in README and log message
- Fix `colomn` → `column` in table structure recognizer comment
- Fix `formated` → `formatted` in confluence connector docstring
- Fix `tabel of content` → `table of contents` in TOC prompt

## Test plan
- [ ] Documentation and comment changes, no functional impact

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 13:05:39 +08:00
Ricardo-M-L
29cf8aba48 fix: correct typos in locale files and search hooks (#13932)
## Summary
- Fix `Refrence` → `Reference` in zh, id, zh-traditional locale files
(en.ts already correct)
- Fix `from from` → `from` and `this files` → `this file` in en.ts
- Fix variable name `reponse` → `response` in search hooks

## Test plan
- [ ] Verify UI strings display correctly
- [ ] Verify search functionality works with renamed variable

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: yuj <yuj@ztjzsoft.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 12:26:25 +08:00
Yongteng Lei
112007243d Refa: refine code_exec component (#13925)
### What problem does this PR solve?

Refine code_exec component.

### Type of change

- [x] Refactoring
2026-04-07 11:48:29 +08:00
Jack
c4b0aaa874 Fix: #6098 - Add validation logic for parser_config when update document (#13911)
### What problem does this PR solve?

Add validation logic for parser_config.
Refactor the processing flow. Before change, validation logics and
update logics are mixed up - some validation logis executes followed by
some update logic executes and then another such
"validation-and-then-update" which is not good. After change, all
validation logic executes firstly. Update logic will be executed after
ALL validation logic executed.
Validation logic for parameters (that come from front end) will be
checked using Pydantic. For validation logic that depends on data from
DB, they will be in separate methods.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-04-07 11:33:05 +08:00
Jin Hai
5673245134 Refactor context command (#13948)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 11:30:09 +08:00
Idriss Sbaaoui
ff27ce86d6 fix: gpt-5 name-based config clearing from base chat path (#13949)
### What problem does this PR solve?

fix #13944 where OpenAI-compatible custom endpoints failed verification
when model names contained `gpt-5` becauser of incorrect name-based
handling in the Base/backend=`base` path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-07 11:24:47 +08:00
buildearth
a0be7c7ca7 Fix(connector): expose id_column, timestamp_column, metadata_columns for MySQL/PostgreSQL incremental sync (#13849)
### What problem does this PR solve?
The MySQL and PostgreSQL sync classes in `sync_data_source.py` were not
passing `id_column`, `timestamp_column`, and `metadata_columns` to
`RDBMSConnector`,
making incremental sync and document update impossible even when
configured.
   
- Without `id_column`: updated records generate new documents instead of
overwriting existing ones (doc ID is derived from content hash, so any
change produces a new ID).
- Without `timestamp_column`: `poll_source` always falls back to full
sync,
ignoring the configured time range.
- The three fields existed in the frontend default values but had no
form
inputs, so users had no way to fill them in.
### Type of change
  - [x] Bug Fix (non-breaking change which fixes an issue)        
  - [x] New Feature (non-breaking change which adds functionality)

### Changes
   
- **Backend** (`rag/svr/sync_data_source.py`): pass `id_column`,
    `timestamp_column`, and `metadata_columns` from `self.conf` to
`RDBMSConnector` for both `MySQL` and `PostgreSQL` sync classes.
- **Frontend**
(`web/src/pages/user-setting/data-source/constant/index.tsx`):
add `ID Column`, `Timestamp Column`, and `Metadata Columns` form fields
    to MySQL and PostgreSQL data source configuration UI with tooltips.

Signed-off-by: lixintao <lixintao@uniontech.com>
Co-authored-by: lixintao <lixintao@uniontech.com>
2026-04-07 10:24:30 +08:00
qinling0210
49386bc1b5 Implement UpdateDataset and UpdateMetadata in GO (#13928)
### What problem does this PR solve?

Implement UpdateDataset and UpdateMetadata in GO

Add cli:
UPDATE CHUNK <chunk_id> OF DATASET <dataset_name> SET <update_fields>
REMOVE TAGS 'tag1', 'tag2' from DATASET 'dataset_name';
SET METADATA OF DOCUMENT <doc_id> TO <meta>


### Type of change

- [ ] Refactoring
2026-04-07 09:44:51 +08:00
Lynn
60ec5880e5 Feat: mysql data migrate script (#13927)
### What problem does this PR solve?

Add a script to migrate data in tenant_llm into tenant_model_provider.

### Type of change

- [x] Other (please describe): tool script.
2026-04-03 20:01:37 +08:00
Magicbook1108
69264b3a70 Feat: Refact pipeline (#13826)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 19:26:45 +08:00
Jin Hai
6d9430a125 Add think chat to CLI (#13922)
### What problem does this PR solve?

Now user can use 'think mode' to chat with LLM

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-03 18:11:23 +08:00
Yingfeng
e518c20736 Update README (#13924)
### Type of change

- [x] Documentation Update
2026-04-03 17:29:48 +08:00
akie
35b2a714f9 Fix: tag datasets not visible in tag sets dropdown (#13921)
## Problem Description

When a user creates Dataset A using the **Tag parser** (for CSV/Excel
files with tag definitions), and then creates Dataset B, the Tag Sets
dropdown in Dataset B's Configuration page cannot display Dataset A.

### Steps to Reproduce
1. Create Dataset A with **Tag** as the chunking method
2. Upload a CSV file to Dataset A to generate tags
3. Create Dataset B
4. Navigate to Dataset B → Configuration → Tag Sets
5. **Expected**: Dataset A should appear in the dropdown
6. **Actual**: The dropdown is empty, Dataset A is not visible

---

## Root Cause Analysis

After thorough code review, **the original code logic is correct**. The
`chunk_method` field flows properly through the system:

### Data Flow

```mermaid
sequenceDiagram
    participant Frontend
    participant Pydantic
    participant API
    participant Database

    Note over Frontend,Database: Creating a Tag Dataset
    Frontend->>Pydantic: POST {chunk_method: "tag"}
    Pydantic->>API: serialization_alias converts<br/>chunk_method → parser_id
    API->>Database: INSERT {parser_id: "tag"}

    Note over Frontend,Database: Querying Datasets
    Frontend->>API: GET /api/v1/datasets
    API->>Database: SELECT parser_id, ...
    Database-->>API: Returns {parser_id: "tag"}
    API->>API: remap_dictionary_keys()<br/>parser_id → chunk_method
    API-->>Frontend: {chunk_method: "tag"}

    Note over Frontend: Filter: x.chunk_method === 'tag'
    Note over Frontend:  Match found!
```

### Field Mapping

**Location**: `api/utils/api_utils.py:657-662`
```python
DEFAULT_KEY_MAP = {
    "chunk_num": "chunk_count",
    "doc_num": "document_count",
    "parser_id": "chunk_method",  # Maps DB field to API response
    "embd_id": "embedding_model",
}
```

### Frontend Filtering (Already Correct)

**Location**:
`web/src/pages/dataset/dataset-setting/components/tag-item.tsx:24`
```typescript
const knowledgeOptions = knowledgeList
  .filter((x) => x.chunk_method === 'tag')  //  Correct field
  .map((x) => ({...}));
```

---

## Actual Issue

The most likely causes for the "bug" are:

1. **Browser Cache**: Old data cached before proper deployment
2. **Stale Data**: Datasets created before the code was fully deployed
3. **Container Not Restarted**: Changes not applied to running container

---

## Resolution

**No code changes are needed.** The existing code correctly:

1. Accepts `chunk_method` from frontend
2. Converts to `parser_id` via Pydantic serialization_alias
3. Stores in database as `parser_id`
4. Maps back to `chunk_method` in API response
5. Frontend filters by `chunk_method === 'tag'`
2026-04-03 17:29:10 +08:00
LeonTung
0b724be521 chore(templates): Update the customer feedback dispatcher template (#13919)
### What problem does this PR solve?
Update the customer feedback dispatcher template and introduce a new
operator `Variable Aggregator`.

### Type of change

- [x] Other (please describe): Template change

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-04-03 16:51:39 +08:00
balibabu
5b43c7cf16 Feat: Place the language configuration in web/.env for easy user configuration. (#13920)
### What problem does this PR solve?

Feat: Place the language configuration in web/.env for easy user
configuration.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-04-03 16:50:18 +08:00
Ricardo-M-L
354108922b fix: use f-string with separator in switch operator error message (#13915)
\`switch.py\` line 137 concatenates the operator directly after the text
without separator:
\`'Not supported operator' + operator\` → produces \`"Not supported
operatorXXX"\`

Changed to: \`f'Not supported operator: {operator}'\`
2026-04-03 16:49:28 +08:00
chanx
21af67f6f9 feat(File Management): Refactor File List API and Add Knowledge Base Document Initialization (#13914)
### What problem does this PR solve?

feat(File Management): Refactor File List API and Add Knowledge Base
Document Initialization

- Migrate the file list API endpoint from `/v1/file/list` to
`/api/v1/files` to align with the Python implementation.
- Add logic for initializing knowledge base documents; automatically
create the `.knowledgebase` folder and associated documents when
retrieving the root directory.
- Enhance parameter validation and error handling, including the
introduction of a new `CodeParamError` error code.
- Optimize the file list response structure to match the implementation
on the Python side.
- Update the Vite configuration to support proxying the new
`/api/v1/files` endpoint.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-03 15:08:43 +08:00
writinwaters
6263857c1e Agent templates regrouped and renamed (#13873)
### What problem does this PR solve?

Regrouped and renamed agent templates to increase user engagement.

### Type of change


- [x] Refactoring
2026-04-03 13:43:25 +08:00
Zhichang Yu
ab358fe949 feat: make Azure cloud authority configurable for SPN auth (#13898)
## Summary
- The Azure SPN storage handler hardcoded
`AzureAuthorityHosts.AZURE_CHINA`, preventing users in Azure Public
Cloud regions (UK-South, EU, US, etc.) from authenticating
- Add a `cloud` config option (env: `AZURE_CLOUD`) supporting all four
Azure sovereignties: `public`, `china`, `government`, `germany`
- Defaults to `public` (global Azure) — the most common international
use case

Closes #13259

## Test plan
- [ ] Verify default (`cloud: public`) connects to Azure Public Cloud
endpoints
- [ ] Verify `cloud: china` retains existing behavior for Azure China
users
- [ ] Verify `AZURE_CLOUD` env var overrides the config file value

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-03 12:51:26 +08:00
Zhichang Yu
384fa6fc6e Replace MinIO official image with pgsty/minio fork (#13896)
## Summary

- Replace `quay.io/minio/minio` with `pgsty/minio` community fork in
`docker/docker-compose-base.yml`

MinIO stopped distributing pre-built Docker images and changed its
license. The pgsty/minio fork provides drop-in compatible images under
AGPLv3.

Closes #13840

## Test plan

- [x] Verify `docker compose -f docker/docker-compose-base.yml up -d`
pulls the pgsty/minio image successfully
- [ ] Verify MinIO console accessible on port 9001
- [ ] Verify RAGFlow backend can connect to MinIO and perform file
operations normally

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-02 22:03:02 +08:00
Yongteng Lei
b7daf6285b Refa: Chat conversations /convsersation API to RESTFul (#13893)
### What problem does this PR solve?

Chat conversations /convsersation API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-02 20:49:23 +08:00
chanx
bbb9b1df85 feat: Implement file upload and folder creation features by GO (#13903)
### What problem does this PR solve?

feat: Implement file upload and folder creation features

- Add file upload route in router.go
- Add file operation methods in dao/file.go
- Add util/file.go for file type detection and filename handling
- Implement file upload and folder creation endpoints in handler/file.go
- Implement file upload and folder creation logic in service/file.go
- Modify response message format in memory.go
- Add document count method in dao/document.go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-02 20:21:04 +08:00
Jin Hai
6c29128de1 Refactor model provider and command (#13887)
### What problem does this PR solve?

Introduce 5 new tables, including model groups and provider instance.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-02 20:20:35 +08:00
qinling0210
f02f5fa435 Get ROW_ID from search() in Infinity (#13901)
### What problem does this PR solve?

1. Search() in Infinity can return row_id now

2.  To Get ROW_ID from search(), refer to handling of retrieval_test.

example
```
$ curl -s -X POST "http://localhost:$PORT/v1/chunk/retrieval_test" -H "Authorization: $TOKEN" -H "Content-Type: application/json" -d '{"kb_id": "4fcd01582ca911f1954184ba59049aa3", "question": "曹操"}'
```


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 18:56:43 +08:00
Idriss Sbaaoui
ee1bb8a8b5 Fix: overlapping document parse race that can clear chunks (#13900)
### What problem does this PR solve?

This PR fixes a race in batch document parsing where overlapping parse
requests for the same document could clear/rewrite chunk state and make
previously parsed content appear lost. It adds an atomic per-document
parse guard so only one parse can run at a time for that document (Fixes
#13864 ).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 18:50:56 +08:00
writinwaters
3b96cedece Docs: Updated chat-specific APIs (#13888)
### What problem does this PR solve?

Chat-specific API descriptions updated.

### Type of change

- [x] Documentation Update
2026-04-02 14:15:09 +08:00
NeedmeFordev
6b7989b4b4 Add file type validation (#13802)
### What problem does this PR solve?

This PR fixes WebDAV sync behavior for unsupported file types
([#13795](https://github.com/infiniflow/ragflow/issues/13795)).

Previously, the WebDAV connector selected files primarily by modified
time (and size threshold) and could still pass unsupported extensions
into the download/document-generation path. This caused unnecessary
processing and inconsistent behavior compared with connectors that
validate file type earlier.

This change adds extension validation in two places:

1. **Early filter during recursive listing** to skip unsupported files
before they enter the download flow.
2. **Defensive filter before download/document creation** to prevent
unsupported files from being processed if any listing edge case slips
through.

It also wires `allow_images` into the WebDAV sync path so image
extension handling follows connector policy.

Scope is intentionally limited to WebDAV for a focused bug-fix PR.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### How was this tested?

- Manual verification with mixed file types under the configured WebDAV
path:
  - supported: `.pdf`, `.txt`, `.md`
  - unsupported: `.exe`, `.bin`, `.dat`
- Triggered full sync and polling sync.
- Confirmed unsupported files are skipped before download.
- Confirmed supported files are still indexed normally.
- Confirmed image handling follows `allow_images` setting.

Fixes: #13795
2026-04-02 14:12:27 +08:00
Idriss Sbaaoui
dd529137eb Fix: markdown table double extraction in parser (#13892)
### What problem does this PR solve?

Fixes markdown tables being parsed twice (once as markdown and again as
generated HTML), which caused duplicate table chunks in the chunk list
UI.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-02 13:31:56 +08:00
Sank
1c2c4b337e [RU] Add schema synchronization and translate (#13891)
### What problem does this PR solve?

Add schema synchronization and translate

### Type of change

- [x] Translation into Russian
2026-04-02 11:18:27 +08:00
Ricardo-M-L
09a09a5b20 fix: correct typo in IterationItem name check and incomplete error message (#13890)
Two small fixes:

1. **iterationitem.py line 72**: Typo "interationitem" → "iterationitem"
(missing 't'). The component name check never matched IterationItem
components.

2. **raptor.py line 94**: Error message "Embedding error: " had a
trailing colon with no details. Changed to "Embedding error: empty
embeddings returned".
2026-04-02 10:35:28 +08:00
balibabu
af40be68c3 Fix: The dataset on the list page cannot be renamed. (#13886)
### What problem does this PR solve?

Fix: The dataset on the list page cannot be renamed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-01 20:53:05 +08:00
Yongteng Lei
b622c47ed6 Refa: Chats /chat API to RESTFul (#13881)
### What problem does this PR solve?

 Refactor Chats /chat API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-01 20:10:37 +08:00
qinling0210
bb4a06f759 Implement InsertDataset and InsertMetadata in GO (#13883)
### What problem does this PR solve?

Implement InsertDataset and InsertMetadata in GO

new internal cli for go:

INSERT DATASET FROM FILE "file_name"
INSERT METADATA FROM FILE "file_name"

### Type of change

- [x] Refactoring
2026-04-01 16:16:25 +08:00
Liu An
b1d28b5898 Revert "Refa: Chats /chat API to RESTFul (#13871)" (#13877)
### What problem does this PR solve?

This reverts commit 1a608ac411.

### Type of change

- [x] Other (please describe):
2026-04-01 11:05:29 +08:00
Yongteng Lei
1a608ac411 Refa: Chats /chat API to RESTFul (#13871)
### What problem does this PR solve?

Chats /chat API to RESTFul.

### Type of change

- [x] Refactoring
2026-04-01 10:50:22 +08:00
balibabu
00b62dd587 Feat: If a model configured in the agent is deleted from the user center, a notification will be displayed on the canvas with a red border. (#13872)
### What problem does this PR solve?

Feat: If a model configured in the agent is deleted from the user
center, a notification will be displayed on the canvas with a red
border.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-31 18:43:24 +08:00
Jin Hai
efd6ecc3e5 New provider and models API and CLI (#13865)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-31 18:42:12 +08:00
Sank
68b4287892 Add translate [RU] for MinerU (#13832)
add translate for MinerU in knowledgeConfiguration

### Type of change

- [X] Other (please describe):
2026-03-31 17:03:31 +08:00
balibabu
36513313f8 Fix: The agent form sheet will be obscured by the message log sheet. (#13870)
### What problem does this PR solve?

Fix: The agent form sheet will be obscured by the message log sheet.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-31 16:18:43 +08:00
Paul Y Hui
3e702c6265 fix: guard against missing/malformed Authorization header in apikey_required (#13860)
### What problem does this PR solve?

Previously, `apikey_required` called
`request.headers.get('Authorization').split()[1]` without checking for
None or insufficient parts, causing an unhandled AttributeError or
IndexError (500) instead of a proper 403 JSON response.

This applies the same guarding pattern already used by `token_required`
in the same file.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-03-31 15:25:00 +08:00
balibabu
4f27090289 Fix: Unable to reconnect after deleting the connection between begin and parser #13868 (#13869)
### What problem does this PR solve?

Fix: Unable to reconnect after deleting the connection between begin and
parser #13868
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-31 14:44:47 +08:00
writinwaters
db5ab7bbe8 Docs: Image2text is supported by GPUStack. (#13856)
### What problem does this PR solve?

 Image2text is supported by GPUStack. #9515 

### Type of change

- [x] Documentation Update
2026-03-30 20:39:02 +08:00
balibabu
3a4f0d1a83 Fix: The chat settings are not displayed correctly on the first page load. (#13855)
### What problem does this PR solve?
Fix: The chat settings are not displayed correctly on the first page
load.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-30 20:16:52 +08:00
qinling0210
620fe215a4 Fix python metadata search (#13727)
### What problem does this PR solve?

Fix python metadata search

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-30 19:37:19 +08:00
qinling0210
0462c20113 Fix special characters in matching text of search() (#13852)
### What problem does this PR solve?

Fix special characters in matching text of search(). We should escape
some special characters(such as ?, *,:) before passing to matching_text
of search()

Fix https://github.com/infiniflow/ragflow/issues/13729

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-30 18:47:10 +08:00
Zhichang Yu
0d85a8e7aa feat: add dynamic log level adjustment APIs (#13850)
Add REST APIs to dynamically query and modify log levels at runtime for
both Python (Flask) and Go servers.

Changes:
- common/log_utils.py: add set_log_level() and get_log_levels()
functions
- admin/server/routes.py: add GET/PUT /api/v1/admin/log_levels endpoints
- api/apps/system_app.py: add GET/PUT /api/{version}/system/log_levels
endpoints
- internal/logger/logger.go: add GetLevel() and SetLevel() with atomic
level support
- internal/handler/system.go: add GetLogLevel, SetLogLevel, Health
handlers
- internal/router/router.go: route /health to systemHandler
- internal/admin/handler.go: add GetLogLevel, SetLogLevel handlers
- internal/admin/router.go: add /api/v1/admin/log_level routes

### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 18:40:58 +08:00
黄圣祺
534729546e fix(html-parser): correct h4 heading mapping from ##### to #### (#13833)
## Summary

- Fix incorrect Markdown heading mapping for `h4` in `TITLE_TAGS`
dictionary
- `h4` was mapped to `"#####"` (h5 level) instead of `"####"` (correct
h4 level)

Closes #13819

## Details

In `deepdoc/parser/html_parser.py`, the `TITLE_TAGS` dictionary had a
typo where `h4` was assigned 5 `#` characters instead of 4, causing h4
headings to be converted to h5-level Markdown headings during HTML
parsing.

## Test plan

- [ ] Parse an HTML document containing `<h4>` tags and verify the
output uses `####` (4 hashes)
- [ ] Verify other heading levels remain correct

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-30 13:17:32 +08:00
Jin Hai
2faaa9f9ce Update docker container start printout (#13847)
### What problem does this PR solve?

Printout RAGFlow version

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-30 12:59:42 +08:00
Jin Hai
e20cf39735 Refactor Go server model provider reading and access (#13831)
### What problem does this PR solve?

1. Refactor model provider json file format
2. Use memory data structure to replace database
3. Add CLI command to access

```
RAGFlow(user)> list pool models from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| features                                                                            | max_tokens | model_types | name                  |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| map[]                                                                               | 256000     | [llm]       | grok-4                |
| map[]                                                                               | 131072     | [llm]       | grok-3                |
| map[]                                                                               | 131072     | [llm]       | grok-3-fast           |
| map[]                                                                               | 131072     | [llm]       | grok-3-mini           |
| map[]                                                                               | 131072     | [llm]       | grok-3-mini-mini-fast |
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768      | [vlm]       | grok-2-vision         |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
RAGFlow(user)> show pool model 'grok-2-vision' from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| features                                                                            | max_tokens | model_types | name          |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768      | [vlm]       | grok-2-vision |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
RAGFlow(user)> list pool providers;
+--------+------------------------------------------------------------+---------------------------+
| name   | tags                                                       | url                       |
+--------+------------------------------------------------------------+---------------------------+
| OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | https://api.openai.com/v1 |
| xAI    | LLM                                                        | https://api.x.ai/v1       |
+--------+------------------------------------------------------------+---------------------------+
RAGFlow(user)> show pool provider 'openai';
+---------------------------+--------+------------------------------------------------------------+--------------+
| base_url                  | name   | tags                                                       | total_models |
+---------------------------+--------+------------------------------------------------------------+--------------+
| https://api.openai.com/v1 | OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | 27           |
+---------------------------+--------+------------------------------------------------------------+--------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-30 12:00:49 +08:00
qinling0210
a8bbe167a9 Bump to infinity v0.7.0-dev5 (#13846)
### What problem does this PR solve?

Bump to infinity v0.7.0-dev5

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-30 10:19:06 +08:00
Heyang Wang
641b319647 feat: support reading tags via API (#12891) (#13732)
### What problem does this PR solve?

Enable reading Tag Set tags via API (expose tag_kwd field). The result
of the queried list chunks is as shown below:

<img width="1422" height="818" alt="image"
src="https://github.com/user-attachments/assets/abd1960a-fe34-489e-9d72-525f8e574938"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: heyang.why <heyang.why@alibaba-inc.com>
2026-03-29 20:17:01 +08:00
KeJun
cb78ce0a7b feat: support rss datasource (#13721)
### What problem does this PR solve?

Supporting public RSS/Atom feed URLs as data sources for RagFlow.

link https://github.com/infiniflow/ragflow/issues/12313

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-27 22:58:44 +08:00
Jin Hai
f32a832f92 Add rename model directory to entity to avoid name misunderstanding (#13829)
### What problem does this PR solve?

Model-> entity

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-27 19:25:18 +08:00
balibabu
308b3a1299 Feat: Remove antd-related code and upgrade lucide-react to the latest version. (#13830)
### What problem does this PR solve?

Feat: Remove antd-related code and upgrade lucide-react to the latest
version.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-27 19:24:52 +08:00
Jin Hai
1fff48b656 Add minio go test (#13800)
### What problem does this PR solve?

1. Add go test
2. Update CI process

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-27 18:12:56 +08:00
Liu An
2240fc778c Fix: add missing "mom" field to infinity_mapping.json for parent-child chunker (#13821)
### What problem does this PR solve?

When using Infinity as DOC_ENGINE with parent-child chunker enabled,
vector insertion fails because the "mom" field is missing from the index
mapping. This fix adds the required field to resolve the issue.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-27 13:06:18 +08:00
Krishna Chaitanya
cdbbd2620c Fix: upgrade pyasn1 from 0.6.2 to 0.6.3 to address CVE-2026-30922 (#13773)
## Summary

- Adds `pyasn1>=0.6.3` as a `[tool.uv.constraint-dependencies]` entry to
mitigate **CVE-2026-30922** (CVSS 7.5 HIGH)
- Regenerates `uv.lock` so the resolved pyasn1 version moves from
**0.6.2 to 0.6.3**

## Details

**CVE-2026-30922** is a Denial of Service vulnerability in pyasn1 caused
by unbounded recursion when decoding ASN.1 data with deeply nested
structures. An attacker can send crafted payloads with thousands of
nested SEQUENCE or SET tags to trigger a `RecursionError` crash or
memory exhaustion.

- **Severity:** HIGH (CVSS 7.5)
- **Affected versions:** pyasn1 < 0.6.3
- **Fixed in:** pyasn1 >= 0.6.3
- **NVD:** https://nvd.nist.gov/vuln/detail/CVE-2026-25769

`pyasn1` is not a direct dependency of RAGFlow but is pulled in
transitively via `google-auth` -> `rsa` -> `pyasn1-modules` -> `pyasn1`.
The `constraint-dependencies` mechanism in uv is the correct way to
enforce a minimum version for transitive dependencies without polluting
the direct dependency list.

## Test plan

- [x] `pyproject.toml` passes TOML validation
- [x] `uv lock` resolves successfully with the new constraint
- [x] pyasn1 version in `uv.lock` is now 0.6.3
- [ ] Existing CI/CD tests continue to pass

Closes #13686
2026-03-27 10:37:34 +08:00
chanx
8a9bbf3d6d Feat: add memory function by go (#13754)
### What problem does this PR solve?

Feat: Add Memory function by go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-27 09:49:50 +08:00
黄圣祺
406339af1f Fix(paddleocr): load all PDF pages for image cropping instead of first 100 (#13811)
## Summary

Closes #13803

The `__images__` method in `paddleocr_parser.py` defaulted to
`page_to=100`, only loading the first 100 pages for image cropping.
However, the PaddleOCR API processes **all** pages of the PDF. For PDFs
with more than 100 pages, page indices beyond 99 were rejected as out of
range during crop validation, causing content loss.

## Root Cause

```
__images__(page_to=100) → loads pages 0-99 → page_images has 100 entries
PaddleOCR API → processes all 226 pages → tags reference pages 1-226
extract_positions() → converts tag "101" to index 100
crop() validation → 0 <= 100 < 100 → False → "All page indices [100] out of range"
```

## Fix

Changed `page_to` default from `100` to `10**9`, so all PDF pages are
loaded for cropping. Python's list slicing safely handles oversized
indices.

## Test plan

- [ ] Parse a PDF with >100 pages using PaddleOCR — no more "out of
range" warnings
- [ ] Parse a PDF with <100 pages — behavior unchanged
- [ ] Verify cropped images are generated correctly for all pages

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Asksksn <Asksksn@noreply.gitcode.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-27 09:33:11 +08:00
Sank
992a15146d Add translate autoMetadata (#13807)
### What problem does this PR solve?

add translate autoMetadata in Russia

### Type of change

- [x] Other
2026-03-27 09:31:20 +08:00
Renzo
f3aa3381a2 Fix username line break in SharedBadge component (#13794)
## Summary
- Added Tailwind truncation classes (`inline-block max-w-[120px]
truncate align-middle`) to the username `<span>` in `SharedBadge` to
prevent long usernames from wrapping onto multiple lines
- Added `title` attribute to show the full username on hover when
truncated


![ragflow](https://github.com/user-attachments/assets/8b3d8c03-d605-4957-bcf0-8b4d81fc4e70)


## Test plan
- [x] Verify long usernames display truncated with ellipsis (`...`)
- [x] Verify hovering over a truncated username shows the full name as a
tooltip
- [x] Verify short usernames display normally without truncation

Closes #13748
2026-03-27 09:31:08 +08:00
Yingfeng
6e309f9d0a Feat: Initialize context engine CLI (#13776)
### What problem does this PR solve?

- Add multiple output format to ragflow_cli
- Initialize contextengine to Go module
  - ls datasets/ls files
  - cat file
  - search -d dir -q query

issue: #13714

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-26 21:07:06 +08:00
Idriss Sbaaoui
3b1e77a6d4 Fix: shared KB embedding authorization for team members (#13809)
### What problem does this PR solve?

fixes issue #13799 where team members get model not authorized when
running RAG on an admin-shared knowledge base after the admin changes
the KB embedding model (for example to bge-m3).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 21:01:07 +08:00
Lynn
8d4a3d0dfe Fix: create dataset with chunk_method or pipeline (#13814)
### What problem does this PR solve?

Allow create datasets with parse_type == 1/None and chunk_method, or
parse_type == 2 and pipeline_id.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 20:43:53 +08:00
Lynn
6a4a9debd2 Fix: allow create dataset with resume chunk_method (#13798)
### What problem does this PR solve?

Allow create dataset with resume chunk_method.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 19:06:51 +08:00
balibabu
8402fcac6b Fix: The chunk method of the knowledge base cannot be saved. (#13813)
### What problem does this PR solve?

Fix: The chunk method of the knowledge base cannot be saved.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-26 19:05:49 +08:00
Syed Shahmeer Ali
ff92b5575b Fix: /file2document/convert blocks event loop on large folders causing 504 timeout (#13784)
Problem

The /file2document/convert endpoint ran all file lookups, document
deletions, and insertions synchronously inside the
request cycle. Linking a large folder (~1.7GB with many files) caused
504 Gateway Timeout because the blocking DB loop
  held the HTTP connection open for too long.

  Fix

- Extracted the heavy DB work into a plain sync function _convert_files
- Inputs are validated and folder file IDs expanded upfront (fast path)
- The blocking work is dispatched to a thread pool via
get_running_loop().run_in_executor() and the endpoint returns 200
  immediately
- Frontend only checks data.code === 0 so the response change
(file2documents list → True) has no impact

  Fixes #13781

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-26 16:45:10 +08:00
Jin Hai
e705ac6643 Add logout (#13796)
### What problem does this PR solve?

Add command: logout

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-26 11:54:23 +08:00
qinling0210
ebf36950e4 Implement Create/Drop Index/Metadata index in GO (#13791)
### What problem does this PR solve?

Implement Create/Drop Index/Metadata index in GO

New API handling in GO:
POST/kb/index 
DELETE /kb/index
POST /tenant/doc_meta_index
DELETE /tenant/doc_meta_index

CREATE INDEX FOR DATASET 'dataset_name' VECTOR_SIZE 1024;
DROP INDEX FOR DATASET 'dataset_name';
CREATE INDEX DOC_META;
DROP INDEX DOC_META;

### Type of change

- [x] Refactoring
2026-03-26 11:54:10 +08:00
Yongteng Lei
d19ca71b43 Refa: Searches /search API to RESTFul (#13770)
### What problem does this PR solve?

Searches /search API to RESTFul

### Type of change

- [x] Documentation Update
- [x] Refactoring

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-26 01:07:41 +08:00
Yongteng Lei
ea1430bec5 Security: do not use litellm 1.82.7 and 1.82.8 (#13768)
### What problem does this PR solve?

See [issue](https://github.com/BerriAI/litellm/issues/24518) from
Litellm.

Upgraded from `1.81.15` to `1.82.6`, so RAGFlow is safe as always. 

### Type of change

- [x] Security

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-25 22:39:33 +08:00
Jin Hai
61cc5ffef2 Add api tokens commands of go admin cli (#13765)
### What problem does this PR solve?

- GENERATE TOKENS OF USER 'xxx@xxx.com'
- DROP KEY 'ragflow-yyyyy' OF 'xxx@xxx.com'
- LIST KEYS OF 'xxx@xxx.com'

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 21:39:14 +08:00
balibabu
33948b9dd8 Fix: Fix the issue of errors when creating datasets. (#13787)
### What problem does this PR solve?

Fix: Fix the issue of errors when creating datasets.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-25 21:37:58 +08:00
balibabu
8f45398422 Fix: Using AvatarUpload in a dialog and pressing Enter will cause a file selection pop-up to appear. #13779 (#13780)
### What problem does this PR solve?
Fix: Using AvatarUpload in a dialog and pressing Enter will cause a file
selection pop-up to appear. #13779

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 19:02:51 +08:00
Jin Hai
24fcd6bbc7 Update CI (#13774)
### What problem does this PR solve?

CI isn't stable, try to fix it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-25 18:17:52 +08:00
Idriss Sbaaoui
f3b4d6ab0e Fix: ci fails (#13778)
### What problem does this PR solve?

fix tests failing at p2 and p3

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 17:56:13 +08:00
Liu An
543d164e9b Fix: add build-essential for Python C extension packages (#13772)
### What problem does this PR solve?

The removal of cargo in commit f59d96f87 also removed build-essential
which was needed to compile C extension packages like datrie.

Use aliyun mirror for coverage pip install

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 17:14:48 +08:00
chanx
a2e6daa8d6 Fix: Metadata,chunk,dataset Related bugs (#13760)
### What problem does this PR solve?

Fix: Metadata,chunk,dataset Related bugs
- metadata not show add button #13731
- chunk edit question style
- dataset modified chunk method bug
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-25 10:47:34 +08:00
Yongteng Lei
1b29522279 Fix: migrate_add_unique_email silently skips unique constraint (#13744)
### What problem does this PR solve?

Fix
migrate_add_unique_email-silently-skips-unique-constraint-when-non-unique-user_email-index-exists.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 20:24:24 +08:00
NeedmeFordev
840cc8fbe9 fix(asana): use project memberships endpoint for project IDs in connector (#13746)
### What problem does this PR solve?

Fixes a bug in the Asana connector where providing `Project IDs` caused
sync to fail with:

`project_membership: Not a recognized ID: <PROJECT_GID>`

Root cause: the connector called `get_project_membership(project_gid)`,
but that API expects a **project membership gid**, not a **project
gid**.
This PR switches to the correct project-scoped API and adds regression
tests.

Fixes: [#13669](https://github.com/infiniflow/ragflow/issues/13669)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes made

- Updated `common/data_source/asana_connector.py`:
- Replaced `get_project_membership(pid, ...)` with
`get_project_memberships_for_project(pid, ...)`
- Trimmed and filtered `asana_project_ids` parsing to avoid
empty/whitespace IDs
  - Normalized `asana_team_id` by trimming whitespace
  - Used safer access for membership email extraction (`m.get("user")`)
- Added `test/unit_test/common/test_asana_connector.py`:
  - Verifies the correct project-membership API method is called
  - Verifies empty `project_ids` path returns workspace emails
  - Verifies project/team input normalization behavior

### Compatibility / risk

- Non-breaking bug fix
- No API contract changes
- Existing behavior for empty `Project IDs` remains unchanged
2026-03-24 20:21:31 +08:00
qinling0210
7c8927c4fb Implement GetChunk() in Infinity in GO (#13758)
### What problem does this PR solve?

Implement GetChunk() in Infinity in GO

Add cli:
GET CHUNK 'XXX';
LIST CHUNKS OF DOCUMENT 'XXX';

### Type of change

- [x] Refactoring
2026-03-24 20:10:21 +08:00
Jin Hai
b308cd3a02 Update go cli (#13717)
### What problem does this PR solve?

Go cli

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 20:08:36 +08:00
balibabu
d84b688b91 Fix: This resolves the issue where selecting a knowledge base in chat could not differentiate between different users. (#13764)
### What problem does this PR solve?

Fix: This resolves the issue where selecting a knowledge base in chat
could not differentiate between different users.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 20:07:06 +08:00
Yongteng Lei
3d10e2075c Refa: files /file API to RESTFul style (#13741)
### What problem does this PR solve?

Files /file API to RESTFul style.

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 19:24:41 +08:00
Idriss Sbaaoui
10a36d6443 Tests : add tests for dataset settings (#13747)
### What problem does this PR solve?

add tests

### Type of change

- [x] Other (please describe): test

Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 19:04:04 +08:00
Yongteng Lei
1b1f1bc69f Fix: minor fix of refacotr excel parser use lazy image loader (#13752)
### What problem does this PR solve?

Minor fix.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Hu Di <812791840@qq.com>
2026-03-24 19:03:54 +08:00
Baki Burak Öğün
8a4da41406 docs: add Turkish README translation (README_tr.md) (#13750)
## Summary
Add a complete Turkish translation of the README and include a Turkish
language badge across all existing README files.

## Changes
- **New file**: `README_tr.md` - Full Turkish translation of README.md,
covering all sections (What is RAGFlow, Demo, Latest Updates, Key
Features, System Architecture, Get Started, Configurations, Docker
Image, Development from Source, Documentation, Roadmap, Community,
Contributing)
- **Updated 9 existing README files** (README.md, README_zh.md,
README_tzh.md, README_ja.md, README_ko.md, README_id.md,
README_pt_br.md, README_fr.md, README_ar.md) to include the Turkish
language badge in the language selector

## Impact
- 10 files changed, 417 insertions
- Follows the same structure and conventions as other language-specific
README files (README_ja.md, README_ko.md, etc.)
- Turkish badge uses the same styling pattern (highlighted with DBEDFA
in README_tr.md, standard DFE0E5 in others)

---------

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
2026-03-24 19:00:48 +08:00
Baki Burak Öğün
1319a25416 feat: complete Turkish localization (#13749)
## Summary
Complete and improve the existing Turkish (tr.ts) localization to fully
match the English (en.ts) reference file.

## Changes
- **Translate 6 English model tips** in the setting section
(chatModelTip, embeddingModelTip, img2txtModelTip, sequence2txtModelTip,
rerankModelTip, ttsModelTip) to Turkish
- **Expand all 13 truncated parser HTML descriptions** (book, laws,
manual, naive, paper, presentation, qa, resume, table, picture, one,
knowledgeGraph, tag) to match the full en.ts structure
- **Expand shortened tooltips** across knowledgeDetails,
knowledgeConfiguration, chat, and setting sections (~40+ tooltips
expanded)
- **Add missing translation details** for data source connectors
(SeaFile, Jira, Gmail, Moodle, Dropbox, Google Drive, etc.)

## Impact
- 182 insertions, 71 deletions in web/src/locales/tr.ts
- No structural changes, only translation content improvements
- All application terminology maintained consistently

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 18:58:58 +08:00
Jin Hai
f59d96f879 Remove rust/cargo install in docker (#13739)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 17:04:57 +08:00
balibabu
48c60b8ce5 Fix: Fixed the issue where agent log time could not be selected. (#13756)
### What problem does this PR solve?
Fix: Fixed the issue where agent log time could not be selected.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-24 16:02:26 +08:00
Jin Hai
9eb11bf65d Fix ping response (#13757)
### What problem does this PR solve?

As title to be compatible with go server

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 15:15:21 +08:00
Stephen Hu
d32967eda8 refactor: let excel use lazy image loader (#13558)
### What problem does this PR solve?

let excel use lazy image loader

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-23 21:24:40 +08:00
Magicbook1108
f991cd362e Fix: type check in resume parsing method (#13740)
### What problem does this PR solve?

Fix: type check in resume parsing method
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 21:19:09 +08:00
Idriss Sbaaoui
df2cc32f51 Fix: dataset settings save (#13745)
### What problem does this PR solve?

Saving dataset settings failed with validation error 101 (Extra inputs
are not permitted)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 17:46:41 +08:00
qinling0210
ac542da505 Fix tokenizer in cpp (#13735)
### What problem does this PR solve?

Tokenzier in Infinity is modified in
https://github.com/infiniflow/infinity/pull/3330, sync the code change
to cpp files in ragflow

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-23 15:40:35 +08:00
qinling0210
7b86f577be Implement metadata search in Infinity in GO (#13706)
### What problem does this PR solve?

Add cli

LIST DOCUMENTS OF DATASET quoted_string ";"
LIST METADATA OF DATASETS quoted_string ("," quoted_string)* ";"
LIST METADATA SUMMARY OF DATASET quoted_string (DOCUMENTS quoted_string
("," quoted_string)*)? ";"

### Type of change

- [x] Refactoring
2026-03-21 18:10:00 +08:00
Lynn
db57155b30 Fix: get user_id from variables (#13716)
### What problem does this PR solve?

Get user_id from canvas variable when input a {} pattern value.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-20 23:39:34 +08:00
Yongteng Lei
dd839f30e8 Fix: code supports matplotlib (#13724)
### What problem does this PR solve?

Code as "final" node: 

![img_v3_02vs_aece4caf-8403-4939-9e68-9845a22c2cfg](https://github.com/user-attachments/assets/9d87b8df-da6b-401c-bf6d-8b807fe92c22)

Code as "mid" node:

![img_v3_02vv_f74f331f-d755-44ab-a18c-96fff8cbd34g](https://github.com/user-attachments/assets/c94ef3f9-2a6c-47cb-9d2b-19703d2752e4)


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-20 20:32:00 +08:00
balibabu
0507463f4e Fix: The retrieval_test interface is continuously requested when the user enters a question. #13719 (#13720)
### What problem does this PR solve?

Fix: The retrieval_test interface is continuously requested when the
user enters a question. #13719

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-20 15:46:41 +08:00
Jin Hai
9ce766192f Init storage engine (#13707)
### What problem does this PR solve?

1. Init Minio / S3 / OSS
2. Fix minio / s3 / oss config

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-20 13:15:41 +08:00
Jin Hai
04a60a41e0 Allow default admin user login ragflow user of go server (#13715)
### What problem does this PR solve?

1. Allow admin@ragflow.io login go ragflow server
2. Fix go server start error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-20 12:02:44 +08:00
tmimmanuel
13d0df1562 feat: add Perplexity contextualized embeddings API as a new model provider (#13709)
### What problem does this PR solve?

Adds Perplexity contextualized embeddings API as a new model provider,
as requested in #13610.

- `PerplexityEmbed` provider in `rag/llm/embedding_model.py` supporting
both standard (`/v1/embeddings`) and contextualized
(`/v1/contextualizedembeddings`) endpoints
- All 4 Perplexity embedding models registered in
`conf/llm_factories.json`: `pplx-embed-v1-0.6b`, `pplx-embed-v1-4b`,
`pplx-embed-context-v1-0.6b`, `pplx-embed-context-v1-4b`
- Frontend entries (enum, icon mapping, API key URL) in
`web/src/constants/llm.ts`
- Updated `docs/guides/models/supported_models.mdx`
- 22 unit tests in `test/unit_test/rag/llm/test_perplexity_embed.py`

Perplexity's API returns `base64_int8` encoded embeddings (not
OpenAI-compatible), so this uses a custom `requests`-based
implementation. Contextualized vs standard model is auto-detected from
the model name.

Closes #13610

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-03-20 10:47:48 +08:00
Zhicheng Wu
456b1bbf66 fix: row selection leaks across pages in dataset and file list tables (#13668)
### What problem does this PR solve?

When using pagination in the Dataset file list or File Manager,
selecting row N on page 1 would incorrectly cause row N on page 2 (and
subsequent pages) to also appear selected. This is a state pollution
bug.

### Root Cause

TanStack React Table defaults to using array indices (0, 1, 2...) as
`rowSelection` keys. With server-side (manual) pagination, each page's
rows start from index 0, so a selection like `{2: true}` on page 1 also
matches index 2 on every other page.

### Fix

- Added `getRowId: (row) => row.id` to `useReactTable` in both
`DatasetTable` and `FilesTable`, so selection state is keyed by unique
document/file IDs instead of positional indices.
- Updated the `useSelectedIds` helper to support ID-based selection keys
while maintaining backward compatibility with index-based keys.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Files Changed

| File | Change |
|------|--------|
| `web/src/pages/dataset/dataset/dataset-table.tsx` | Added `getRowId`
to table config |
| `web/src/pages/files/files-table.tsx` | Added `getRowId` to table
config |
| `web/src/hooks/logic-hooks/use-row-selection.ts` | Updated
`useSelectedIds` to handle ID-based selection |
2026-03-19 21:08:09 +08:00
chanx
e1dbfb8a9c fix(dao): Remove unnecessary status filter conditions in user queries (#13698)
### What problem does this PR solve?

Fix: Enhanced the user deletion function to return detailed deletion
information.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 21:05:15 +08:00
Magicbook1108
cfe6ea6f56 Feat: CREATE / DELETE / LIST dataset api in Go (#13695)
### What problem does this PR solve?

Feat: CREATE / DELETE / LIST dataset api in Go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 20:48:32 +08:00
Lynn
f06e332c44 Fix: allow on (#13704)
### What problem does this PR solve?

Allow input on/ON as status.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 20:41:02 +08:00
writinwaters
b5e0b37d69 Refact: Renamed 'Agent flow' to 'Workflow' (#13705)
### What problem does this PR solve?

'Agent flow' rebranded.

### Type of change

- [x] Refactoring
2026-03-19 20:17:25 +08:00
Jin Hai
8d50ee632d Add environments reading (#13701)
### What problem does this PR solve?

environment variable > config file

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-19 18:50:28 +08:00
yH
757d8d42dd Fix: use configured OrderByExpr in _community_retrieval_ (#13683)
The `odr` variable was configured with `desc("weight_flt")` but a new
empty `OrderByExpr()` was passed to `dataStore.search()` instead,
causing the descending sort to have no effect.

### What problem does this PR solve?

In `_community_retrieval_`, the configured `OrderByExpr` with
`desc("weight_flt")` was discarded — a new empty `OrderByExpr()` was
passed to `dataStore.search()` instead, so community reports were never
sorted by weight.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 17:55:40 +08:00
Lynn
e12147f5b9 Fix: admin client (#13699)
### What problem does this PR solve?

Define a crypt function in admin directory, remove import from
api.utils. And move requests-toolbelt to dependency.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 17:06:54 +08:00
Lynn
4bb1acaa5b Refactor: dataset / kb API to RESTFul style (#13690)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 14:41:36 +08:00
Idriss Sbaaoui
7827f0fce5 fix : empty mind map (#13693)
### What problem does this PR solve?

Fix graphrag extractor chat response parsing and skip truncated cache
values

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-19 13:53:06 +08:00
Jin Hai
7ebe1d2722 Fix docker building (#13681)
### What problem does this PR solve?

1. Refactor go server log
2. Update docker building, since nginx config should be set according to
the deployment.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-19 10:25:35 +08:00
NeedmeFordev
c3f79dbcb0 fix(jira): prevent missed incremental updates after issue edits (#13674)
### What problem does this PR solve?

Fixes [#13505](https://github.com/infiniflow/ragflow/issues/13505): Jira
incremental sync could miss updated issues after initial sync,
especially near time boundaries.

Root cause:
- Jira JQL uses minute-level precision for `updated` filters.
- Incremental windows had no overlap buffer, so boundary updates could
be skipped.
- Sync log cursor tracking used a backward-facing update for
`poll_range_start`.
- Existing-doc updates in `upload_document` lacked a KB ownership guard
for doc-id collisions.

What changed:
- Added Jira incremental overlap buffer (`time_buffer_seconds`,
defaulting to `JIRA_SYNC_TIME_BUFFER_SECONDS`) when building JQL
lower-bound time.
- Preserved second-level post-filtering to avoid duplicate reprocessing
while still catching boundary updates.
- Improved Jira sync logging to include start/end window and overlap
configuration.
- Updated sync cursor tracking in `increase_docs` to keep
`poll_range_start` moving forward with max update time.
- Added KB ID safety check before updating existing document records in
`upload_document`.

Verification performed:
- Python syntax compile checks passed for modified files.
- Manual verification flow:
  1. Run full Jira sync.
  2. Edit an already-indexed Jira issue.
  3. Run next incremental sync.
  4. Confirm updated content is re-ingested into KB.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 23:31:05 +08:00
Daniil Sivak
dee68c571b Feat: support variable interpolation in headers (#13680)
Closes #13277

### What problem does this PR solve?

Adds `{variable_name}` (and `{component@variable}`) interpolation
support to HTTP header values in the `Invoke` component, matching the
existing URL interpolation behavior.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

<img width="1280" height="867" alt="image"
src="https://github.com/user-attachments/assets/8ab7b4e9-7cc0-4a7f-8a5f-f838a15a5fda"
/>

---------

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
2026-03-18 22:38:20 +08:00
Mustafa YILDIZ
e4d8cdaff3 feat: add Turkish language support (#13670)
### What problem does this PR solve?
RAGFlow had no Turkish language support. This PR adds Turkish (tr)
locale translations to the UI.

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?

Co-authored-by: Mustafa YILDIZ <mustafa.yildiz@cilek.com>
2026-03-18 21:09:32 +08:00
writinwaters
bbd0cd80e4 Docs: Updated Add Google Drive as data source (#13684)
### What problem does this PR solve?

Gave an editorial pass to the Add Google Drive document.

### Type of change

- [x] Documentation Update
2026-03-18 21:05:25 +08:00
Octopus
f171554c0a feat: upgrade MiniMax default model to M2.7 (#13676)
## Summary
Upgrade MiniMax model configuration to include the latest M2.7 model.

## Changes
- Add `MiniMax-M2.7` and `MiniMax-M2.7-highspeed` to the model selection
list in `conf/llm_factories.json`
- Place M2.7 models at the top of the list as the recommended default
- Retain all previous models (M2.5, M2.5-highspeed, M2.1, M2) as
available alternatives

## Why
MiniMax-M2.7 is the latest flagship model with enhanced reasoning and
coding capabilities. This update ensures RAGFlow users can access the
newest model while maintaining backward compatibility with existing
configurations.

## Testing
- JSON config validated (well-formed)
- No existing MiniMax-specific unit tests affected
- Model entries follow the same structure as existing entries

Co-authored-by: PR Bot <pr-bot@minimaxi.com>
2026-03-18 19:20:10 +08:00
Idriss Sbaaoui
9070408b04 Fix : model-specific handling (#13675)
### What problem does this PR solve?

add a handler for gpt 5 models that do not accept parameters by dropping
them, and centralize all models with specific paramter handling function
into a single helper.
solves issue #13639 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-03-18 17:28:20 +08:00
Yongteng Lei
53e395ca2e Fix: cannot debug invoke component (#13649)
### What problem does this PR solve?

Cannot debug invoke component.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-18 14:22:13 +08:00
Jin Hai
74866371ef Fix compatiblity issue (#13667)
### What problem does this PR solve?

1. Change go admin server port from 9385 to 9383 to avoid conflicts
2. Start go server after python servers are started completely, in
entrypoint.sh
3. Fix some database migration issue
4. Add more API routes in web to compliant with EE.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-18 11:51:03 +08:00
Daniil Sivak
60ad32a0c2 Feat: support epub parsing (#13650)
Closes #1398

### What problem does this PR solve?

Adds native support for EPUB files. EPUB content is extracted in spine
(reading) order and parsed using the existing HTML parser. No new
dependencies required.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

To check this parser manually:

```python
uv run --python 3.12 python -c "
from deepdoc.parser import EpubParser

with open('$HOME/some_epub_book.epub', 'rb') as f:
  data = f.read()

sections = EpubParser()(None, binary=data, chunk_token_num=512)
print(f'Got {len(sections)} sections')
for i, s in enumerate(sections[:5]):
  print(f'\n--- Section {i} ---')
  print(s[:200])
"
```
2026-03-17 20:14:06 +08:00
Idriss Sbaaoui
1399c60164 fix builtin model fail when parsing (#13657)
### What problem does this PR solve?

using builtin model when parsing gave an error because it expects
fid==builtin. split_model_name_and_factory returns id=None. pr allows
the model to be accepted wheter with or without @Builtin

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 19:38:54 +08:00
balibabu
6cae364ac2 Feat: Export Agent Logs. (#13658)
### What problem does this PR solve?
Feat: Export Agent Logs.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-17 18:51:26 +08:00
balibabu
fc4f1e2488 Fix: The dataset description should not be a required field. (#13655)
### What problem does this PR solve?

Fix: The dataset description should not be a required field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 18:51:18 +08:00
Idriss Sbaaoui
ad6bdb5bfe Fix: left preview containment regression for file previews (#13652)
### What problem does this PR solve?

Fix left preview containment regression for file previews

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 17:21:13 +08:00
Yongteng Lei
ca6c3218c3 Refa: follow-up expose agent structured outputs in non-stream completions (#13524)
### What problem does this PR solve?

Follow-up expose agent structured outputs in non-stream completions
#13389.

### Type of change

- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-17 17:11:27 +08:00
qinling0210
ca182dc188 Implement Search() in Infinity in GO (#13645)
### What problem does this PR solve?

Implement Search() in Infinity in GO.

The function can handle the following request. 
"search '曹操' on datasets 'infinity'" 
"search '常胜将军' on datasets 'infinity'"
"search '卓越儒雅' on datasets 'infinity'"
"search '辅佐刘禅北伐中原' on datasets 'infinity'"

The output is exactly the same as  request to python Search()

### Type of change

- [ ] New Feature (non-breaking change which adds functionality)
2026-03-17 16:45:45 +08:00
balibabu
549833b8a4 Fix: Fixed an issue where agent template titles were not displayed in Chinese mode. (#13647)
### What problem does this PR solve?

Fix: Fixed an issue where agent template titles were not displayed in
Chinese mode.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 15:56:57 +08:00
Stephen Hu
77483b1e58 refactor: remove useless variable in raptor (#13648)
### What problem does this PR solve?

remove useless variable in raptor

### Type of change


- [x] Refactoring
2026-03-17 15:56:51 +08:00
Jin Hai
986dcf1cc8 Revert "Refactor: dataset / kb API to RESTFul style" (#13646)
Reverts infiniflow/ragflow#13619
2026-03-17 12:09:48 +08:00
balibabu
fdf2d84ffc Fix: Fixed an issue where the agent could not publish. (#13644)
### What problem does this PR solve?

Fix: Fixed an issue where the agent could not publish.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-17 11:44:01 +08:00
Lynn
1db5409d82 Refactor: dataset / kb API to RESTFul style (#13619)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-16 22:51:34 +08:00
Yingfeng
73bc9b91de Limit max recursion depth for rag analyzer#3318 (#13637)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 22:49:56 +08:00
chanx
5403f142ae Feat: Add chunk also supports uploading image. (#13628)
### What problem does this PR solve?

Feat: Add chunk also supports uploading image.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-16 20:15:49 +08:00
Yongteng Lei
af7e24ba8c Feat: add_chunk supports add image (#13629)
### What problem does this PR solve?

Add_chunk supports add image.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 20:15:36 +08:00
Magicbook1108
09ff1bc2b0 Fix: paddle ocr coordinate lower > upper (#13630)
### What problem does this PR solve?

Fix: paddle ocr coordinate lower > upper #13618 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 20:15:26 +08:00
Jin Hai
0545801251 Update CI process (#13632)
### What problem does this PR solve?

This pull request updates the GitHub Actions workflow for testing,
primarily to simplify Docker Compose usage and environment file
management. The main changes focus on removing unnecessary subdirectory
references, updating environment file handling, and streamlining the
workflow steps.


### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-16 19:00:28 +08:00
Idriss Sbaaoui
d5ed179d15 Playwright : add test ids and chat test (#13432)
### What problem does this PR solve?


### Type of change

- [x] Other
2026-03-16 16:39:05 +08:00
balibabu
f4d126acb0 Fix: Shared chat link triggers infinite POST loop with empty question, input disabled #13606 (#13625)
### What problem does this PR solve?

Fix: Shared chat link triggers infinite POST loop with empty question,
input disabled #13606

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 15:43:18 +08:00
balibabu
fa48ffe5de Feat: Translate embedded dialog text. (#13623)
### What problem does this PR solve?

Feat: Translate embedded dialog text.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 15:43:13 +08:00
Idriss Sbaaoui
c98ad2f0d8 fix multimodel input bars slowly being pushed offscreen (#13620)
### What problem does this PR solve?

when the conversation starts to get long on multimodel chat, the
conversation pushes the input bar offscreem

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 15:03:50 +08:00
Idriss Sbaaoui
bb962e67b0 fix : build error (#13622)
### What problem does this PR solve?

add timeout to fix fail at build during uvsync step

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 15:03:25 +08:00
Yingfeng
b686a60713 Switch from demo.ragflow.io to cloud.ragflow.io (#13624)
### What problem does this PR solve?

Switch from demo.ragflow.io to cloud.ragflow.io

### Type of change

- [x] Documentation Update
2026-03-16 14:44:39 +08:00
Liu An
5b3bb25010 Fix: switch Python package mirror from Tsinghua to Aliyun (#13617)
### What problem does this PR solve?

Replace pypi.tuna.tsinghua.edu.cn with mirrors.aliyun.com to resolve
issues with missing packages on the Tsinghua mirror.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-16 12:12:25 +08:00
Jin Hai
a2d72202cf Revert "Refactor dataset / kb API to RESTFul style" (#13614)
Reverts infiniflow/ragflow#13263
2026-03-16 10:44:38 +08:00
Ram Mourya
ae9b1c7f6a Docs : Fixed the links for user and developer guide in readme files (#13609)
Fixed the links for user and developer guide in readme files.
2026-03-16 10:23:52 +08:00
Yongteng Lei
287637162c Revert "fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416" (#13613)
Reverts infiniflow/ragflow#13583 which cause uv sync fails.
2026-03-16 10:19:29 +08:00
Lynn
7c32e206be Refactor dataset / kb API to RESTFul style (#13263)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-13 20:02:35 +08:00
apps-lycusinc
8b984c9d5f Fixing WordNetCorpusReader object has no attribute _LazyCorpusLoader_… (#13600)
### What problem does this PR solve?

Forces NLTK to load the corpus synchronously once, preventing concurrent
tasks from triggering the lazy-loading race condition that cause Fixing
WordNetCorpusReader object has no attribute _LazyCorpusLoader_… #13590


### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: shakeel <shakeel@lollylaw.com>
2026-03-13 19:55:01 +08:00
Magicbook1108
161659becc Fix: model selecton rule in get_model_config_by_type_and_name (#13569)
### What problem does this PR solve?

Fix: model selecton rule in get_model_config_by_type_and_name

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 19:46:13 +08:00
balibabu
cb49cd30c4 Feat: Add the user_id field to the agent log table and the embedded page. (#13596)
### What problem does this PR solve?

Feat: Add the `user_id` field to the agent log table and the embedded
page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-13 19:06:18 +08:00
Jin Hai
cc7e94ffb6 Use different API route according the ENV (#13597)
### What problem does this PR solve?

1. Fix go server date precision
2. Use API_SCHEME_PROXY to control the web API route

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-13 19:05:30 +08:00
balibabu
1569ed82f8 Refactor: Delete flow.ts (#13498)
### What problem does this PR solve?

Feat: Delete flow.ts

### Type of change


- [x] Refactoring
2026-03-13 18:04:54 +08:00
chanx
a3e6c2e84a Fix: Enhanced user management functionality and cascading data deletion. (#13594)
### What problem does this PR solve?
Fix: Enhanced user management functionality and cascading data deletion.

Added tenant and related data initialization functionality during user
creation, including tenants, user-tenant relationships, LLM
configuration, and root folder.
Added cascading deletion logic for user deletion, ensuring that all
associated data is cleaned up simultaneously when a user is deleted.
Implemented a Werkzeug-compatible password hash algorithm (scrypt) and
verification functionality.
Added multiple DAO methods to support batch data operations and
cascading deletion.
Improved user login processing and added token signing functionality.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 16:53:54 +08:00
Sank
a67fa03584 fix CVE-2026-28804 CVE-2026-31826 (#13592)
What problem does this PR solve?

fix CVE-2026-28804 CVE-2026-31826

 Bug Fix (non-breaking change which fixes an issue)

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 16:34:28 +08:00
balibabu
717f1f1362 Feat: Modify the style of the release confirmation box. (#13542)
### What problem does this PR solve?

Feat: Modify the style of the release confirmation box.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: balibabu <assassin_cike@163.com>
Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
2026-03-13 16:31:17 +08:00
Lynn
02070bab2a Feat: record user_id in memory (#13585)
### What problem does this PR solve?

Get user_id from canvas and record it.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-13 15:38:35 +08:00
Jin Hai
5c955a31cc Update go server (#13589)
### What problem does this PR solve?

1. Add more CLI command
2. Add some license hooks

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-13 14:41:02 +08:00
Idriss Sbaaoui
ef94a9c291 Fix : remove min value for description field (#13587)
### What problem does this PR solve?

min value and message force users to input a descript in datasets. Also
had a wrong error message.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 13:11:54 +08:00
Ethan T.
71804bf5bc fix(db_models): guard MySQL-specific SQL in migration with DB_TYPE check (fixes #13544) (#13582)
## Summary

Fixes #13544: PostgreSQL startup crash because
`update_tenant_llm_to_id_primary_key()` unconditionally uses
MySQL-specific SQL.

- Split `update_tenant_llm_to_id_primary_key()` into
`_update_tenant_llm_to_id_primary_key_mysql()` and
`_update_tenant_llm_to_id_primary_key_postgres()`, dispatching on
`settings.DATABASE_TYPE`
- MySQL path: unchanged (existing `DATABASE()`, `SET @row = 0`,
`AUTO_INCREMENT`, `DROP PRIMARY KEY` logic)
- PostgreSQL path: uses `current_database()`, `ROW_NUMBER() OVER (ORDER
BY ...)` for sequential IDs, `CREATE SEQUENCE` + `nextval()` for
auto-increment, and `information_schema.table_constraints` to find the
PK constraint name
- Also fix `migrate_add_unique_email()`: MySQL-only
`information_schema.statistics` is replaced with `pg_indexes` on
PostgreSQL

## Test plan

- [ ] Start RAGFlow with `DB_TYPE=postgres` — startup should complete
without `function database() does not exist` error
- [ ] Start RAGFlow with `DB_TYPE=mysql` (default) — existing behaviour
unchanged, migration runs as before
- [ ] Fresh PostgreSQL install: verify `tenant_llm.id` column is created
as a serial primary key after migration
- [ ] Idempotency: running migration twice on PostgreSQL should be a
no-op (column already exists check passes)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: gambletan <gambletan@github>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-13 11:53:01 +08:00
Sank
e90f0e8910 fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416 (#13583)
### What problem does this PR solve?

fix CVE-2026-26216. CVE-2026-26217 CVE 2025-66416

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 11:17:39 +08:00
Liu An
667f9c1c3a fix: remove duplicate "arabic" key in French translations (#13529)
### What problem does this PR solve?

Removed duplicate key that caused build warning during Vite build.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 10:57:19 +08:00
Idriss Sbaaoui
810692dfa3 fix: restore cross_languages default chat-model fallback for retrieval (#13471)
### What problem does this PR solve?

issue #13465 
POST /api/v1/retrieval failed with
{"code":100,...,"message":"Exception('Model Name is required')"} when
cross_languages was provided and no explicit llm_id was passed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 10:52:37 +08:00
Jimmy Ben Klieve
1a4dee4313 refactor(ui): unify top level pages structure, use standard language codes and time zones (#13573)
### What problem does this PR solve?

- Unify top level pages structure
- Standardize locale language codes (BCP 47) and time zones (IANA tz)


> **Note:** 
> Newly created user info brings non-standard default values `timezone:
"UTC+8\tAsia/Shanghai"` and `language: "English"`.


### Type of change

- [x] Refactoring
2026-03-12 21:01:09 +08:00
Ethan Clarke
35cd56f990 feat: add MiniMax-M2.5 and M2.5-highspeed models (#13557)
## Summary

Add MiniMax's latest M2.5 model family to the model registry and update
the default API base URL to the international endpoint for broader
accessibility.

## Changes

- **Add MiniMax-M2.5 models** to `conf/llm_factories.json`:
- `MiniMax-M2.5` — Peak Performance. Ultimate Value. Master the Complex.
  - `MiniMax-M2.5-highspeed` — Same performance, faster and more agile.
- Both support 204,800 token context window and tool calling (`is_tools:
true`).
- **Update default MiniMax API base URL** in `rag/llm/__init__.py`:
- From `https://api.minimaxi.com/v1` (domestic) to
`https://api.minimax.io/v1` (international).
- Chinese users can still override via the Base URL field in the UI
settings (as documented in existing i18n strings).

## Supported Models

| Model | Context Window | Tool Calling | Description |
|-------|---------------|-------------|-------------|
| `MiniMax-M2.5` | 204,800 tokens | Yes | Peak Performance. Ultimate
Value. |
| `MiniMax-M2.5-highspeed` | 204,800 tokens | Yes | Same performance,
faster and more agile. |

## API Documentation

- OpenAI Compatible API:
https://platform.minimax.io/docs/api-reference/text-openai-api

## Testing

- [x] JSON validation passes
- [x] Python syntax validation passes
- [x] Ruff lint passes
- [x] MiniMax-M2.5 API call verified (returns valid response)
- [x] MiniMax-M2.5-highspeed API call verified (returns valid response)

Co-authored-by: PR Bot <pr-bot@minimaxi.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 20:41:46 +08:00
Jin Hai
3fbf8bc3d4 Expose go version server and admin server port out of docker in CI (#13572)
### What problem does this PR solve?

- Print Go version log when start server
- Expose the server port in CI docker container

### Type of change

- [x] Other (please describe): For CI

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:39:57 +08:00
Jin Hai
d688b72dff Go: Add admin server status checking (#13571)
### What problem does this PR solve?

RAGFlow server isn't available when admin server isn't connected.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:02:50 +08:00
chanx
1df804a14a Feature (System Settings): Implemented system settings management functionality (#13556)
### What problem does this PR solve?

Feature (System Settings): Implemented system settings management
functionality

- Added a new SystemSettings model, including creation and update time
fields.

- Implemented SystemSettingsDAO, providing CRUD operations and
transaction support.

- Implemented management interfaces for variables, configurations, and
environment variables in the admin service.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 19:06:20 +08:00
guptas6est
7c79602c77 fix(web): upgrade lodash to 4.17.23 and dompurify to 3.3.2 to fix CVE-2026-0540 and CVE-2025-13465 (#13488)
### What problem does this PR solve?

This PR fixes two security vulnerabilities in web dependencies
identified by Trivy:

1. CVE-2025-13465 (lodash): Prototype pollution vulnerability in _.unset
and _.omit functions
2. CVE-2026-0540 (dompurify): Cross-site scripting (XSS) vulnerability

**Changes:**
- Upgraded lodash from 4.17.21 to 4.17.23
- Upgraded dompurify from 3.3.1 to 3.3.2
- Added npm override to force monaco-editor's transitive dependency on
dompurify to use 3.3.2 (monaco-editor still depends on vulnerable 3.2.7)

Both upgrades are backward-compatible patch versions. Build verified
successfully with no breaking changes.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 19:04:26 +08:00
Ray Zhang
375f62a6c3 docs(migration): add project name (-p) usage to backup & migration guide (#13565)
## Summary

- Add documentation for the `-p project_name` flag in the migration
script, covering all steps (stop, backup, restore, start)
- Add a note explaining how Docker volume name prefixes relate to the
Compose project name
- Update `docker-compose` to `docker compose` (Compose V2 syntax) for
consistency
- Fix `sh` to `bash` to match the script's shebang line

This is the documentation follow-up to #12187 which added `-p` project
name support to `docker/migration.sh`.

## Test plan

- [ ] Verify the documentation renders correctly on the docs site
- [ ] Confirm all example commands are accurate against the current
`migration.sh`
2026-03-12 19:01:25 +08:00
qinling0210
1be07a0a34 Fix "Result window is too large" during meta data search (#13521)
### What problem does this PR solve?

Fix
https://github.com/infiniflow/ragflow/issues/13210#issuecomment-3982878498

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 18:59:56 +08:00
Jin Hai
cebf5892ec Create go version storage component, but not used (#13561)
### What problem does this PR solve?

Implement: minio, s3, oss, azure_sas, azure_spn, gcs, opendal

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 18:58:25 +08:00
Jinghan Xu
f6b06fab72 Fix: allow document parsing status recovery after transient errors (#13341)
### What problem does this PR solve?

Fixes #13285

When an LLM returns a transient error (e.g. overloaded) during parsing,
the task progress is set to -1. Previously, the progress could never be
updated again, leaving the document permanently stuck in FAIL status
even after the task successfully recovered and completed.

Three coordinated changes address this:

1. task_service.update_progress: relax the progress update guard to
accept prog >= 1 even when current progress is -1, so a task that
recovers from a transient failure can report completion.

2. document_service.get_unfinished_docs: include documents that are
marked FAIL (progress == -1) but still have at least one non-failed task
(task.progress >= 0) in the polling set, so their status can be
re-synced once a task recovers. Documents where all tasks have
permanently failed are excluded to avoid unnecessary polling.

3. document_service.update_progress: explicitly set document status to
RUNNING when not all tasks have finished, instead of preserving whatever
stale status (potentially FAIL) the document previously had.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 18:02:12 +08:00
Yongteng Lei
13a34d7689 Feat: inject sys.date into canvas (#13567)
### What problem does this PR solve?

Inject sys.date into canvas.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-12 17:49:13 +08:00
Magicbook1108
eda7835d47 Fix: image pdf in ingestion pipeline (#13563)
### What problem does this PR solve?

Fix: image pdf in ingestion pipeline #13550


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 17:49:02 +08:00
NeedmeFordev
387b0b27c4 feat(parser): support external Docling server via DOCLING_SERVER_URL (#13527)
### What problem does this PR solve?

This PR adds support for parsing PDFs through an external Docling
server, so RAGFlow can connect to remote `docling serve` deployments
instead of relying only on local in-process Docling.

It addresses the feature request in
[#13426](https://github.com/infiniflow/ragflow/issues/13426) and aligns
with the external-server usage pattern already used by MinerU.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### What is changed?

- Add external Docling server support in `DoclingParser`:
  - Use `DOCLING_SERVER_URL` to enable remote parsing mode.
- Try `POST /v1/convert/source` first, and fallback to
`/v1alpha/convert/source`.
- Keep existing local Docling behavior when `DOCLING_SERVER_URL` is not
set.
- Wire Docling env settings into parser invocation paths:
  - `rag/app/naive.py`
  - `rag/flow/parser/parser.py`
- Add Docling env hints in constants and update docs:
  - `docs/guides/dataset/select_pdf_parser.md`
  - `docs/guides/agent/agent_component_reference/parser.md`
  - `docs/faq.mdx`

### Why this approach?

This keeps the change focused on one issue and one capability (external
Docling connectivity), without introducing unrelated provider-model
plumbing.

### Validation

- Static checks:
  - `python -m py_compile` on changed Python files
  - `python -m ruff check` on changed Python files
- Functional checks:
  - Remote v1 endpoint path works
  - v1alpha fallback works
  - Local Docling path remains available when server URL is unset

### Related links

- Feature request: [Support external Docling server (issue
#13426)](https://github.com/infiniflow/ragflow/issues/13426)
- Compare view for this branch:
[main...feat/docling-server](https://github.com/infiniflow/ragflow/compare/main...spider-yamet:ragflow:feat/docling-server?expand=1)

##### Fixes [#13426](https://github.com/infiniflow/ragflow/issues/13426)
2026-03-12 17:09:03 +08:00
Josh
a353c7bdd7 Fix: avoid empty doc filter in knowledge retrieval (#13484)
## Summary
Fix knowledge-base chat retrieval when no individual document IDs are
selected.

## Root Cause
`async_chat()` initialized `doc_ids` as an empty list when the request
did not explicitly select documents. That empty list was then forwarded
into retrieval as an active `doc_id` filter, effectively becoming
`doc_id IN []` and suppressing all chunk matches.

## Changes
- treat missing selected document IDs as `None` instead of `[]`
- keep explicit document filtering when IDs are actually provided
- add regression coverage for the shared chat retrieval path

## Validation
- `python3 -m py_compile api/db/services/dialog_service.py
test/unit_test/api/db/services/test_dialog_service_use_sql_source_columns.py`
- `.venv/bin/python -m pytest
test/unit_test/api/db/services/test_dialog_service_use_sql_source_columns.py`
- manually verified that chat completions again inject retrieved
knowledge into the prompt

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-12 16:03:30 +08:00
cambrianlee
227c852e67 Fix typo: documnet_keyword -> document_keyword in Chunk class (#13531)
### What problem does this PR solve?
The Chunk class had a typo in the attribute name 'documnet_keyword',
which caused the document_name field to remain empty when retrieving
chunks via the SDK. This fix corrects the spelling to
'document_keyword'.

Changes:
- Line 36: Changed self.documnet_keyword to self.document_keyword
- Line 52: Updated backward compatibility code to use
self.document_keyword


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 15:23:55 +08:00
Jin Hai
e78938c72c Update go admin server default port to 9383 (#13559)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 13:41:08 +08:00
Jimmy Ben Klieve
31a8184f63 refactor(ui): update ui for user settings, etc. (#13532)
### What problem does this PR solve?

Update UI styles:
- **User settings**
- Component styles: 
   - `ui/button.tsx`
   - `ui/checkbox.tsx`
   - `avatar-upload.tsx`
   - `file-uploader.tsx`
   - `icon-font.tsx`

### Type of change

- [x] Refactoring
2026-03-12 13:33:36 +08:00
chanx
0da9c4618d feat(cli): Enhance CLI functionality and add administrator mode support (#13539)
### What problem does this PR solve?

feat(cli): Enhance CLI functionality and add administrator mode support

- Modify `parseActivateUser` in `parser.go` to support 'on'/'off' states
- Add administrator mode switching and host port settings functionality
to `cli.go`
- Implement user management API calls in `client.go`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-12 13:33:13 +08:00
chanx
4bd5bb141d Fix: data-source-detail page style (#13507)
### What problem does this PR solve?

Fix: data-source-detail page style

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 13:32:39 +08:00
Jin Hai
5cbdfc5f17 Fix Gitee embedding model URL error (#13553)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 13:13:06 +08:00
Yongteng Lei
375a910bcf Fix: add deadlock retry (#13552)
### What problem does this PR solve?

 Add deadlock retry.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 12:39:01 +08:00
Jin Hai
90afce192c Add license and fingerprint API hook (#13548)
### What problem does this PR solve?

For EE

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 11:52:39 +08:00
Jin Hai
2fb1360d9d Add command line parameter and fix error message (#13526)
### What problem does this PR solve?

`./server_main -p 9380`

`./server_main -h`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 09:50:57 +08:00
Yongteng Lei
e1b632a7bb Feat: add delete all support for delete operations (#13530)
### What problem does this PR solve?

Add delete all support for delete operations.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-12 09:47:42 +08:00
qinling0210
d201a81db7 Add command history in ragflow cli (#13538)
### What problem does this PR solve?

In ragflow cli,  use Up/Down arrows to navigate command history,

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-11 19:14:18 +08:00
Liu An
852393c114 Test: Lower priority of chat assistant and chunk list API tests (#13540)
### What problem does this PR solve?

Mark test cases as lower priority (p3) for:
- Creating chat assistants
- Deleting chat assistants
- Listing chat assistants
- Listing chunks within datasets

### Type of change

- [x] Update testcases
2026-03-11 19:00:18 +08:00
foyou
f75dc6a452 Docs: Fix normalization of case and some code blocks (#13520)
### What problem does this PR solve?

Standardize term capitalization in `deploy_local_llm.mdx` and improve
code block formatting.

### Type of change

- [x] Documentation Update
2026-03-11 17:51:13 +08:00
Ethan T.
1cee8b1a7b fix: use context managers for file handles to prevent resource leaks (#13514)
## Summary
- Convert bare `open()` calls to `with` context managers or
`Path.read_text()`
- File handles leak if not properly closed, especially on exceptions
- Fixes in crypt.py, sequence2txt_model.py, term_weight.py,
deepdoc/vision/__init__.py

## Test plan
- [x] File operations work correctly with context managers
- [x] Resources properly cleaned up on exceptions

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-11 16:47:06 +08:00
Attili-sys
6afd13ff29 Feat/arabic language support (#13516)
### What problem does this PR solve?

This PR implements comprehensive Arabic language support for the RAGFlow
application. The changes include:
- Complete Arabic translation of all UI text elements in the web
interface
- RTL (right-to-left) layout support for Arabic content
- Localization updates for all supported languages (ar, bg, de, en, es,
fr, id, it, ja, pt-br, ru, vi, zh-traditional, zh)
- UI component adjustments to properly display Arabic text and support
RTL layout

The implementation ensures that Arabic-speaking users can fully interact
with the application in their native language with proper text rendering
and layout direction.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

<img width="2866" height="1617" alt="image"
src="https://github.com/user-attachments/assets/f2751b34-1b65-4867-b81d-a1068c17b9b7"
/>

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-11 15:06:07 +08:00
chanx
9ca2bac984 Feat: Implement user creation, deletion, and permission management functionality. (#13519)
### What problem does this PR solve?

Feat: Implement user creation, deletion, and permission management
functionality.

- Added the `ListByEmail` method to `user.go` to query users by email
address.

- Updated the user activation status handling logic in `handler.go`,
adding input validation.

- Added RSA password decryption functionality to `password.go`.

- Implemented complete user management functionality in `service.go`,
including user creation, deletion, password modification, activation
status, and permission management.

- Added input validation and error handling logic.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-11 14:04:00 +08:00
Jin Hai
2028e895fd Add license and time record DAO (#13522)
### What problem does this PR solve?

1. Change go server default port to 9382
2. Compatible with EE data model.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-11 14:02:24 +08:00
qinling0210
1815f5950b Call get_flatted_meta_by_kbs in dify retrieval (#13509)
### What problem does this PR solve?

Fix https://github.com/infiniflow/ragflow/issues/13388

Call get_flatted_meta_by_kbs in dify retrieval. Remove get_meta_by_kbs.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-11 13:42:24 +08:00
Josh
2d2d3cdbcf Fix document metadata loading for paged listings (#13515)
## Summary
- scope normal document-list metadata lookups to the current page's
document IDs
- keep the `return_empty_metadata=True` path dataset-wide because it
needs full knowledge of docs that already have metadata
- add unit tests for both paged listing paths and the unchanged
empty-metadata behavior

## Why
`DocumentService.get_list()` and the normal `get_by_kb_id()` path were
calling `DocMetadataService.get_metadata_for_documents(None, kb_id)`,
which loads metadata for the entire dataset on every page request.

That becomes especially problematic on large datasets. The metadata scan
path paginates through the full metadata index without an explicit sort,
while the ES helper only switches to `search_after` beyond `10000`
results when a sort is present. In practice this can lead to unnecessary
full-dataset metadata work, slower document-list loading, and unreliable
`meta_fields` in list responses for large KBs.

This change keeps the existing empty-metadata filter behavior intact,
but scopes normal list responses to metadata for the current page only.
2026-03-11 13:42:16 +08:00
Jimmy Ben Klieve
507ba4ea20 refactor(ui): update knowledge graph, chunk, metadata, agent log styles (#13518)
### What problem does this PR solve?

Update UI styles:
- **Dataset** > **Knowledge graph** tooltip
- **Dataset** > **Files** > **Manage metadata** modal
- **Dataset** > **Files** > **Modify Chunking Method** > **Auto
metadata** > **Manage generation settings** modal
- **Agent** > **Canvas (Ingestion pipeline)** > **Dataflow result**

### Type of change

- [x] Refactoring
2026-03-11 11:27:20 +08:00
Jin Hai
2133fd76a8 Add auth middleware (#13506)
### What problem does this PR solve?

Use auth middle-ware to check authorization.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-11 11:23:13 +08:00
eviaaaaa
d0ca388bec Refa: implement unified lazy image loading for Docx parsers (qa/manual) (#13329)
## Summary
This PR is the direct successor to the previous `docx` lazy-loading
implementation. It addresses the technical debt intentionally left out
in the last PR by fully migrating the `qa` and `manual` parsing
strategies to the new lazy-loading model.

Additionally, this PR comprehensively refactors the underlying `docx`
parsing pipeline to eliminate significant code redundancy and introduces
robust fallback mechanisms to handle completely corrupted image streams
safely.


## What's Changed

* **Centralized Abstraction (`docx_parser.py`)**: Moved the
`get_picture` extraction logic up to the `RAGFlowDocxParser` base class.
Previously, `naive`, `qa`, and `manual` parsers maintained separate,
redundant copies of this method. All downstream strategies now natively
gather raw blobs and return `LazyDocxImage` objects automatically.
* **Robust Corrupted Image Fallback (`docx_parser.py`)**: Handled edge
cases where `python-docx` encounters critically malformed magic headers.
Implemented an explicit `try-except` structure that safely intercepts
`UnrecognizedImageError` (and similar exceptions) and seamlessly falls
back to retrieving the raw binary via `getattr(related_part, "blob",
None)`, preventing parser crashes on damaged documents.

* **Legacy Code & Redundancy Purge**:
* Removed the duplicate `get_picture` methods from `naive.py`, `qa.py`,
and `manual.py`.
* Removed the standalone, immediate-decoding `concat_img` method in
`manual.py`. It has been completely replaced by the globally unified,
lazy-loading-compatible `rag.nlp.concat_img`.
* Cleaned up unused legacy imports (e.g., `PIL.Image`, docx exception
packages) across all updated strategy files.

## Scope
To keep this PR focused, I have restricted these changes strictly to the
unification of `docx` extraction logic and the lazy-load migration of
`qa` and `manual`.

## Validation & Testing
I've tested this to ensure no regressions and validated the fallback
logic:

* **Output Consistency**: Compared identical `.docx` inputs using `qa`
and `manual` strategies before and after this branch: chunk counts,
extracted text, table HTML, and attached images match perfectly.
* **Memory Footprint Drop**: Confirmed a noticeable drop in peak memory
usage when processing image-dense documents through the `qa` and
`manual` pipelines, bringing them up to parity with the `naive`
strategy's performance gains.

## Breaking Changes
* None.
2026-03-11 10:00:07 +08:00
balibabu
d36e3c97d1 Feat: Add a user_id field to the message and retrieval operators. (#13508)
### What problem does this PR solve?

Feat: Add a user_id field to the message and retrieval operators.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-10 22:18:27 +08:00
Yongteng Lei
3c80a0ae09 Fix: support vLLM's new reasoning field (#13493)
### What problem does this PR solve?

Support vLLM's new reasoning field

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 21:13:14 +08:00
yzy
07c9cf6cbe Fix: return structured JSON output for non-streaming agent API (#13389)
### What problem does this PR solve?

Previously, when an Agent component was configured with structured
output, the non-streaming /agents/{agent_id}/completions API never
returned the structured field in its response.

The root cause: the non-streaming code path only collected message
events to build full_content, then returned the workflow_finished
payload — which only contains the output of the last component in the
execution path (typically a Message component).
Any structured output set by upstream components (e.g., Agent or LLM)
was silently discarded.

This PR fixes the non-streaming handler to iterate node_finished events
and collect structured output from intermediate components.
If any component produced a non-empty structured value, it is included
in the final response under data.structured. The streaming path is
unaffected, as it already exposes node_finished events to the caller.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 19:22:04 +08:00
Heyang Wang
08f83ff331 Feat: Support get aggregated parsing status to dataset via the API (#13481)
### What problem does this PR solve?

Support getting aggregated parsing status to dataset via the API

Issue: #12810

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: heyang.why <heyang.why@alibaba-inc.com>
2026-03-10 18:05:45 +08:00
Liu An
68a623154a Fix: bin directory cannot be copied to docker image introduced by #13444 (#13502)
### What problem does this PR solve?

bin directory cannot be copied to docker image introduced by

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 17:31:20 +08:00
chanx
f14b53c764 feat(admin): Implemented default administrator initialization and login functionality. (#13504)
### What problem does this PR solve?

feat(admin): Implemented default administrator initialization and login
functionality.

Added support for default administrator configuration, including super
user nickname, email, and password.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 17:30:21 +08:00
balibabu
81461b4505 Fix: The number of deleted session prompts is displayed incorrectly. #13499 (#13500)
### What problem does this PR solve?

Fix: The number of deleted session prompts is displayed incorrectly.
#13499
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 16:01:31 +08:00
Magicbook1108
675810e0cf Refact: optimize confluence performance (#13497)
### What problem does this PR solve?

Refact: optimize confluence performance #13494

### Type of change

- [x] Refactoring
2026-03-10 15:02:24 +08:00
Alexander Vostres
9ba43ae4ee Fix "Coordinate lower is less than upper" error with MinerU (#13483)
### What problem does this PR solve?

Fixes #6004 #7142 #11959

Unlike #9207 we actually normalize the coordinates here

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 15:02:01 +08:00
balibabu
aaf900cf16 Feat: Display release status in agent version history. (#13479)
### What problem does this PR solve?
Feat: Display release status in agent version history.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-10 14:25:27 +08:00
Idriss Sbaaoui
249b78561b Fix missmatch docnm_kwd in raptor chunks (#13451)
### What problem does this PR solve?

issue #13393 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 14:24:33 +08:00
qinling0210
185ab0d4ef Fix delete_document_metadata (#13496)
### What problem does this PR solve?

Avoid getting doc in function delete_document_metadata as the doc might
have been removed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 13:44:24 +08:00
Magicbook1108
7143954b48 Fix: chats_openai in none stream condition (#13495)
### What problem does this PR solve?

Fix: chats_openai in none stream condition #13453

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 13:44:17 +08:00
qinling0210
7c92f51133 Fix retrieval function when metadata_condtion is specified in retrieval API (#13473)
### What problem does this PR solve?

Fix https://github.com/infiniflow/ragflow/issues/13388

The following command returns empty when there is doc with the meta data
```
curl --request POST \
     --url http://localhost:9222/api/v1/retrieval \
     --header 'Content-Type: application/json' \
     --header 'Authorization: Bearer ragflow-fO3mPFePfLgUYg8-9gjBVVXbvHqrvMPLGaW0P86PvAk' \
     --data '{
          "question": "any question",
          "dataset_ids": ["9bb4f0591b8811f18a4a84ba59049aa3"],
           "metadata_condition": {
            "logic": "and",
            "conditions": [
              {
                "name": "character",
                "comparison_operator": "is",
                "value": "刘备"
              }
            ]
          }
     }'
```

When metadata_condtion is specified in the retrieval API, it is
converted to doc_ids and doc_ids is passed to retrieval function.
In retrieval funciton, when doc_ids is explicitly provided , we should
bypass threshold.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 11:57:32 +08:00
tunsuy
292a1a8566 fix: detect and fallback garbled PDF text to OCR (#13366) (#13404)
## Problem

When PDF fonts lack ToUnicode/CMap mappings, pdfplumber (pdfminer)
cannot map CIDs to correct Unicode characters, outputting PUA characters
(U+E000~U+F8FF) or `(cid:xxx)` placeholders. The original code fully
trusted pdfplumber text without any garbled detection, causing garbled
output in the final parsed result.

Relates to #13366

## Solution

### 1. Garbled text detection functions
- `_is_garbled_char(ch)`: Detects PUA characters (BMP/Plane 15/16),
replacement character U+FFFD, control characters, and
unassigned/surrogate codepoints
- `_is_garbled_text(text, threshold)`: Calculates garbled ratio and
detects `(cid:xxx)` patterns

### 2. Box-level fallback (in `__ocr()`)
When a text box has ≥50% garbled characters, discard pdfplumber text and
fallback to OCR recognition.

### 3. Page-level detection (in `__images__()`)
Sample characters from each page; if garbled rate ≥30%, clear all
pdfplumber characters for that page, forcing full OCR.

### 4. Layout recognizer CID filtering
Filter out `(cid:xxx)` patterns in `layout_recognizer.py` text
processing to prevent them from polluting layout analysis.

## Testing
- 29 unit tests covering: normal CJK/English text, PUA characters, CID
patterns, mixed text, boundary thresholds, edge cases
- All 85 existing project unit tests pass without regression
2026-03-10 11:20:31 +08:00
Jin Hai
7f6a9e8ee9 Update ext field type of heartbeat message (#13490)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 10:49:39 +08:00
chanx
02108772d8 refactor: Moves the LLM factory initialization logic to the dao package. (#13476)
### What problem does this PR solve?

refactor: Moves the LLM factory initialization logic to the `dao`
package.

Removes the `init_data` package and integrates the LLM factory
initialization functionality into the `dao` package.
Adds a `utility` package to provide general utility functions.
Updates `server_main.go` to use the new initialization path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 10:35:55 +08:00
atian8179
88a40b95a2 fix: include missing modules in ragflow-cli PyPI package (#13457)
## Problem

The `ragflow-cli` PyPI package (v0.24.0) is missing `http_client.py`,
`ragflow_client.py`, and `user.py`, causing import errors when installed
from PyPI.

## Root Cause

`pyproject.toml` only lists `ragflow_cli` and `parser` in
`[tool.setuptools] py-modules`.

## Fix

Add the three missing modules to `py-modules`.

Fixes #13456

Co-authored-by: atian8179 <atian8179@users.noreply.github.com>
2026-03-10 10:02:21 +08:00
Jin Hai
4fe706876c Service list and minio status (#13480)
### What problem does this PR solve?

1. Resolve standard user can access admin service
2. Get RAGFlow service status
3. Fix minio status fetching

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 09:56:43 +08:00
writinwaters
4f507c0058 Docs: Updated Switch chunk availability (#13482)
### What problem does this PR solve?

A quick editorial pass.

### Type of change

- [x] Documentation Update
2026-03-09 21:14:45 +08:00
Yongteng Lei
7484298c82 Refa: convert download_img to async (#13477)
### What problem does this PR solve?

Convert download_img to async.

### Type of change

- [x] Refactoring
- [x] Performance Improvement
2026-03-09 19:00:17 +08:00
Jin Hai
52bcd98d29 Add scheduled tasks (#13470)
### What problem does this PR solve?

1. RAGFlow server will send heartbeat periodically.
2. This PR will including:
- Scheduled task
- API server message sending
- Admin server API to receive the message.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 17:48:29 +08:00
Jin Hai
c732a1c8e0 Refactor the go_binding to binding (#13469)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 15:52:31 +08:00
chanx
25ace613b0 feat: Added LLM factory initialization functionality and knowledge base related API interfaces (#13472)
### What problem does this PR solve?

feat: Added LLM factory initialization functionality and knowledge base
related API interfaces

refactor(dao): Refactored the TenantLLMDAO query method
feat(handler): Implemented knowledge base related API endpoints
feat(service): Added LLM API key setting functionality
feat(model): Extended the knowledge base model definition
feat(config): Added default user LLM configuration

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-09 15:52:14 +08:00
Stephen Hu
d0465ba909 refactor: improve paddle ocr logic (#13467)
### What problem does this PR solve?

improve paddle ocr logic

### Type of change
- [x] Refactoring
2026-03-09 14:16:57 +08:00
天海蒼灆
3ce236c4e3 Feat: add switch_chunks endpoint to manage chunk availability (#13435)
### What problem does this commit solve?

This commit introduces a new API endpoint
`/datasets/<dataset_id>/documents/<document_id>/chunks/switch` that
allows users to switch the availability status of specified chunks in a
document as same as chunk_app.py

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-09 12:36:45 +08:00
guptas6est
32d31284cc Fix: upgrade pypdf to 6.7.5 and migrate from deprecated pypdf2 to fix CVE-2026-28804 and CVE-2023-36464 (#13454)
### What problem does this PR solve?

This PR addresses security vulnerabilities in PDF processing
dependencies identified by Trivy security scan:

1. CVE-2026-28804 (MEDIUM): pypdf 6.7.4 vulnerable to inefficient
decoding of ASCIIHexDecode streams
2. CVE-2023-36464 (MEDIUM): pypdf2 3.0.1 susceptible to infinite loop
when parsing malformed comments

Since pypdf2 is deprecated with no available fixes, this PR migrates all
pypdf2 usage to the actively maintained pypdf library (version 6.7.5),
which resolves
both vulnerabilities.


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-09 12:06:00 +08:00
JiangNan
2634cfc06f Fix: undefined variable and wrong method name in agent components (#13462)
## Summary

This PR fixes two runtime bugs in agent components:

**Bug 1: `agent/component/invoke.py` — `NameError` in POST +
`clean_html` path**

The POST method's `clean_html` branch uses the variable `sections`
without ever defining it. Both the GET and PUT branches correctly call
`sections = HtmlParser()(None, response.content)` before referencing
`sections`, but this line was missing from the POST branch (copy-paste
omission). This causes a `NameError` whenever a user configures an
Invoke component with `method="post"` and `clean_html=True`.

**Bug 2: `agent/component/data_operations.py` — `AttributeError` in
`_recursive_eval`**

The `_recursive_eval` method recursively calls `self.recursive_eval()`
(without the leading underscore) instead of `self._recursive_eval()`.
Since the method is defined as `_recursive_eval`, this causes an
`AttributeError` at runtime when the `literal_eval` operation processes
nested dicts or lists.

## Test plan

- [ ] Configure an Invoke node with `method=post` and `clean_html=True`,
verify HTML is parsed correctly without `NameError`
- [ ] Configure a DataOperations node with `operations=literal_eval` on
nested data, verify no `AttributeError`

---------

Signed-off-by: JiangNan <1394485448@qq.com>
2026-03-09 11:09:47 +08:00
Jin Hai
610c1b507d Add more API of admin server of go (#13403)
### What problem does this PR solve?

Add APIs to admin server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 10:44:53 +08:00
Eden
ab6ca75245 fix(agent): ensure database connections are properly closed in ExeSQL tool (#13427)
## Summary

Fix a database connection and cursor resource leak in the ExeSQL agent
tool.

When SQL execution raises an exception (for example syntax error or
missing table),
the existing code path skips `cursor.close()` and `db.close()`, causing
database
connections to accumulate over time.

This can eventually lead to connection exhaustion in long-running agent
workflows.

## Root Cause

The cleanup logic for database cursors and connections is placed after
the SQL
execution loop without `try/finally` protection. If an exception occurs
during
`cursor.execute()`, `fetchmany()`, or result processing, the cleanup
code is not
reached and the connection remains open.

The same issue also exists in the IBM DB2 execution path where
`ibm_db.close(conn)`
may be skipped when exceptions occur.

## Fix

- Wrap SQL execution logic in `try/finally` blocks to guarantee resource
cleanup.
- Ensure `cursor.close()` and `db.close()` are always executed.
- Add explicit `db.close()` when `db.cursor()` creation fails.
- Remove redundant close calls in early-return branches since `finally`
now handles cleanup.

## Impact

- No change to normal execution behavior.
- Ensures database resources are always released when errors occur.
- Prevents connection leaks in long-running workflows.
- Only affects `agent/tools/exesql.py`.

## Testing

Manual test scenarios:

1. Valid SQL execution
2. SQL syntax error
3. Query against a non-existing table
4. Execution cancellation during query

In all scenarios the database cursor and connection are properly closed.

Code quality checks:

- `ruff check` passed
- No new warnings introduced
2026-03-09 10:36:02 +08:00
Liu An
89e495e1bc Chore: update release workflow configuration (#13466)
### What problem does this PR solve?

update release workflow configuration

### Type of change

- [x] Update CI
2026-03-09 10:32:51 +08:00
Heyang Wang
c217b8f3d8 Feat: add DingTalk AI Table connector and integration for data synch… (#13413)
### What problem does this PR solve?

Add DingTalk AI Table connector and integration for data synchronization

Issue #13400

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: wangheyang <wangheyang@corp.netease.com>
2026-03-06 21:13:23 +08:00
Jimmy Ben Klieve
094eae3cf5 refactor(ui): adjust dataset page styles (#13452)
### What problem does this PR solve?

- Adjust UI styles in **Dataset** pages.
- Adjust several shared components styles
- Modify files and directory structure in `src/layouts`

### Type of change

- [x] Refactoring
2026-03-06 21:13:14 +08:00
Liu An
7166a7e50e Test: adjust test priority markers for API tests (#13450)
### What problem does this PR solve?

Changed test priority markers from p1/p2 to p3 in three test files:
- test_table_parser_dataset_chat.py: Adjusted priority for table parser
dataset chat test
- test_delete_chunks.py: Updated priority for chunk deletion test with
invalid IDs
- test_retrieval_chunks.py: Modified priority for chunks retrieval
pagination test

These changes demote the priority of specific test cases to p3,
indicating they are lower priority tests that can run later in the test
suite execution.

### Type of change

- [x] Test update
2026-03-06 20:17:39 +08:00
chanx
ae4645e01b Fix: Add folder upload #9743 (#13448)
### What problem does this PR solve?

Fix: Add folder upload  #9743

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 20:17:29 +08:00
balibabu
82a616589b Feat: Add PublishConfirmDialog (#13447)
### What problem does this PR solve?

Feat: Add PublishConfirmDialog

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 20:17:21 +08:00
Achieve3318
45cf24cd2f feat(memory): implement get_highlight for OceanBase memory (#13449)
### What problem does this PR solve?

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 20:17:11 +08:00
Jin Hai
01a100bb29 Fix data models (#13444)
### What problem does this PR solve?

Since database model is updated in python version, go server also need
to update

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-06 20:05:10 +08:00
OliverW
3ed91345aa fix(auth): return HTTP 401 for token-auth failures (#13420)
Follow-up to #12488 #13386

### What problem does this PR solve?

Previously, token authentication failures returned HTTP 200 with an
error code in the response body.

This PR updates `token_required` to raise `Unauthorized` and relies on
the global error handler to return a structured JSON response with HTTP
401 status.

The response body structure (`code`, `message`, `data`) remains
unchanged to preserve compatibility with the official SDK.

Frontend logic has been updated to handle HTTP 401 responses in addition
to checking `data.code`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 18:18:14 +08:00
Yongteng Lei
51be1f1442 Refa: empty ids means no-op operation (#13439)
### What problem does this PR solve?

Empty ids means no-op operation.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Documentation Update
- [x] Refactoring

---------

Co-authored-by: writinwaters <cai.keith@gmail.com>
2026-03-06 18:16:42 +08:00
Zhichang Yu
7781c51a21 Revert aliyun registry to registry.cn-hangzhou.aliyuncs.com (#13445)
## Summary
- Revert aliyun registry from
`infiniflow-registry.cn-shanghai.cr.aliyuncs.com` back to
`registry.cn-hangzhou.aliyuncs.com`

## Test plan
- [ ] Verify the docker/.env file contains the correct registry URL

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-06 18:03:35 +08:00
Magicbook1108
826af383b4 Fix: paddle ocr missing outlines (#13441)
### What problem does this PR solve?

Fix: paddle ocr missing outlines #13422

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 17:19:51 +08:00
Jin Hai
2504c3adde Fix docker file (#13438)
### What problem does this PR solve?

To copy infinity/resource into docker images

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-06 16:56:12 +08:00
chanx
81fd1811b8 Feat:Using Go to implement user registration logic (#13431)
### What problem does this PR solve?

Feat:Using Go to implement user registration logic

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 16:42:49 +08:00
Achieve3318
37eb533fea Feat(memory): implement get_aggregation for OceanBase memory (#13428)
### What problem does this PR solve?

- Add aggregation_utils.aggregate_by_field for pure aggregation logic
- Wire OBConnection.get_aggregation to use it (unwrap tuple, pass
messages)
- Add unit tests for aggregate_by_field (no DB/heavy deps)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 12:51:22 +08:00
BitToby
383986dc5f fix: re-chunk documents when data source content is updated (#12918)
Closes: #12889 

### What problem does this PR solve?

When syncing external data sources (e.g., Jira, Confluence, Google
Drive), updated documents were not being re-chunked. The raw content was
correctly updated in blob storage, but the vector database retained
stale chunks, causing search results to return outdated information.

**Root cause:** The task digest used for chunk reuse optimization was
calculated only from parser configuration fields (`parser_id`,
`parser_config`, `kb_id`, etc.), without any content-dependent fields.
When a document's content changed but the parser configuration remained
the same, the system incorrectly reused old chunks instead of
regenerating new ones.

**Example scenario:**
1. User syncs a Jira issue: "Meeting scheduled for Monday"
2. User updates the Jira issue to: "Meeting rescheduled to Friday"
3. User triggers sync again
4. Raw content panel shows updated text ✓
5. Chunk panel still shows old text "Monday" ✗

**Solution:**
1. Include `update_time` and `size` in the chunking config, so the task
digest changes when document content is updated
2. Track updated documents separately in `upload_document()` and return
them for processing
3. Process updated documents through the re-parsing pipeline to
regenerate chunks


[1.webm](https://github.com/user-attachments/assets/d21d4dcd-e189-4d39-8700-053bae0ca5a0)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 12:48:47 +08:00
Lynn
0214257886 Fix: init func (#13430)
### What problem does this PR solve?

Fix update_cnt add error in init_data.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 11:42:31 +08:00
balibabu
6849d35bf5 Feat: Optimize the style of the chat page. (#13429)
### What problem does this PR solve?

Feat: Optimize the style of the chat page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 11:42:25 +08:00
Jonah Hartmann
6023eb27ac feat: add Ragcon provider (#13425)
### What problem does this PR solve?

This PR aims to extend the list of possible providers. Adds new Provider
"RAGcon" within the Ollama Modal. It provides all model types except OCR
via Openai-compatible endpoints.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Jakob <16180662+hauberj@users.noreply.github.com>
2026-03-06 09:37:27 +08:00
guptas6est
c35b210c3a fix(security): upgrade requests to 2.32.5 in agent/sandbox to fix CVE-2024-47081 (#13424)
### What problem does this PR solve?

This PR remediates CVE-2024-47081 (MEDIUM severity) in the agent/sandbox
component by upgrading the requests library from version 2.32.3 to
2.32.5. The vulnerability allows .netrc credentials to leak via
malicious URLs.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 09:29:18 +08:00
guptas6est
aa57bcf92a fix: upgrade urllib3 to 2.6.3 to resolve CVE-2025-66418, CVE-2025-66471, CVE-2026-21441 (#13423)
### What problem does this PR solve?

This PR remediates three HIGH severity vulnerabilities in urllib3
affecting the admin client and Python SDK:
- **CVE-2025-66418**: Unbounded decompression chain leads to resource
exhaustion
- **CVE-2025-66471**: Streaming API improperly handles highly compressed
data
- **CVE-2026-21441**: Decompression-bomb safeguard bypass when following
HTTP redirects
Trivy security scan identified urllib3 v2.5.0 as vulnerable in both
`admin/client/uv.lock` and `sdk/python/uv.lock`. This PR updates urllib3
to v2.6.3 to eliminate these security risks.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 09:29:10 +08:00
Jimmy Ben Klieve
ef4cbe72a3 refactor(ui): adjust global navigation bar style (#13419)
### What problem does this PR solve?

Renovate global navigation bar, align styles to the design.
(May causes minor layout issues in sub-pages, will check and fix soon)

### Type of change

- [x] Refactoring
2026-03-05 20:47:29 +08:00
leonardlin
9e0e128ce5 Add checksum/values annotation to ragflow.yaml (#13409)
Add checksum annotation for values in ragflow.yaml

### What problem does this PR solve?

This PR is about this ticket: #13408

Ragflow helm charts do not include the Values.yaml in the list of
watched changes.
If you update the Values.yaml for an existing deployment, helm will not
detect it and not update the deployment.

This PR fixes that.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 20:27:38 +08:00
writinwaters
963e31e9b5 Refact: Updated the doc structure. (#13414)
### What problem does this PR solve?

Updated the doc structure.

### Type of change


- [x] Documentation Update
2026-03-05 19:04:56 +08:00
Idriss Sbaaoui
d90d6026af Playwright : new chat multi model test (#13402)
### What problem does this PR solve?

new test for chat multiple model and other chat parameters under
playwright

### Type of change

- [x] Other (please describe): new test/ data-testid
2026-03-05 18:51:57 +08:00
Yongteng Lei
d9785ea2ce Fix: Alibaba cloud OSS config issue (#13406)
### What problem does this PR solve?

 Alibaba Could OSS config issue #13390.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 18:13:45 +08:00
chanx
8b534c895e Fix: UI Placeholder and Hint Optimization (#13416)
### What problem does this PR solve?

Fix: UI Placeholder and Hint Optimization

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 18:13:19 +08:00
chanx
35fc5edc93 feat: Adds the tenant model ID field to the interface definition. (#13274)
### What problem does this PR solve?

feat: Adds the tenant model ID field to the interface definition

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 17:27:34 +08:00
Lynn
62cb292635 Feat/tenant model (#13072)
### What problem does this PR solve?

Add id for table tenant_llm and apply in LLMBundle.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-05 17:27:17 +08:00
Magicbook1108
47540a4147 Feat: published agent version control (#13410)
### What problem does this PR solve?

Feat: published agent version control

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 17:26:39 +08:00
guptas6est
8c9b080499 fix: update axios to 1.13.5+ to remediate CVE-2026-25639 DoS vulnerability (#13380)
### What problem does this PR solve?

This PR remediates CVE-2026-25639, a HIGH severity Denial of Service
vulnerability in axios caused by __proto__ pollution in the mergeConfig
function. The vulnerability affects both the web frontend and the
sandbox nodejs environment.

Trivy security scan identified axios versions below 1.13.5 as
vulnerable. This PR updates axios to secure versions (1.13.6 in web,
1.13.5 in sandbox) to eliminate the security risk.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 17:26:04 +08:00
Yongteng Lei
f13a1fb007 Refa: improve model verification ux (#13392)
### What problem does this PR solve?

Improve model verification UX. #13395 

### Type of change

- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-05 17:23:47 +08:00
Liu An
3124fa955e chore: add bin and internal dirs to .gitignore for Go server build output (#13407)
### What problem does this PR solve?

add bin and internal dirs to .gitignore for Go server build output
2026-03-05 15:52:01 +08:00
tunsuy
e1f1184b01 test: add unit tests for graphrag/utils.py (87 test cases) (#13328)
Add comprehensive unit tests for `graphrag/utils.py`, covering 15
functions/classes with 87 test cases.

Tested functions:
- clean_str, dict_has_keys_with_types, perform_variable_replacements
- get_from_to, compute_args_hash, is_float_regex
- GraphChange dataclass
- handle_single_entity_extraction, handle_single_relationship_extraction
- graph_merge, tidy_graph
- split_string_by_multi_markers, pack_user_ass_to_openai_messages
- is_continuous_subsequence, merge_tuples, flat_uniq_list

All 327 existing + new tests pass with no regressions.
2026-03-05 15:30:43 +08:00
Jin Hai
3e3b665b89 RAGFlow admin server go version (#13394)
### What problem does this PR solve?

1. init go admin server
2. refactor api server router
3. add benchmark CI to 450s time limit
4. remove docker builder container after building

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-05 15:18:40 +08:00
Liu An
6f5bd4d2e9 feat: add bin and internal dirs to .gitignore for Go server build output (#13391)
### What problem does this PR solve?

add bin and internal dirs to .gitignore for Go server build output
2026-03-05 14:26:40 +08:00
天海蒼灆
118f737b3a Feat:Enhance chunk management by adding support for 'available', 'tag_kwd' and 'tag_feas' (#13383)
### What problem does this PR solve?

Enhance chunk management by adding support for 'available', 'tag_kwd'
and 'tag_feas' fields in list, add, and update chunk functions just like
chunk_app.py.This improves data handling and flexibility in chunk
processing.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-05 13:45:39 +08:00
orbcom-pedroferreira
61209ff3bf Feat: File uploads for future conversations on SDK API (#13378)
### What problem does this PR solve?

This PR aims to:

1. Enable file uploads for the public API, similarly to what
/document/upload_info accomplishes for the frontend;
2. Enable files sent to the /chat/:chat_id/completions endpoint to be
used within the conversation.
We classify the first item as a new future, while classifying the second
one as a bug fix.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

*The work related to this PR was co-authored by*

[Bruno Ferreira](https://github.com/brunopferreira): Custom Solutions
Manager @ [Orbcom](https://orbcom.pt/)
[Pedro Ferreira](https://github.com/sirj0k3r): Lead Software Developer @
[Orbcom](https://orbcom.pt/)
[Pedro Cardoso](https://github.com/pedromiguel4560): Associate Software
Developer @ [Orbcom](https://orbcom.pt/)

*This PR replaces #13248*

---------

Co-authored-by: Pedro Cardoso <pedrocardoso@orbcom.pt>
Co-authored-by: Pedro Ferreira <pedroferreira@orbcom.pt>
2026-03-04 22:26:58 +08:00
tunsuy
020068dd16 Fix: preserve field boundaries in chunked documents from MySQL… (#13369)
### What problem does this PR solve?

When multiple columns are used as content columns in RDBMS connector,
the generated document text gets chunked by TxtParser which strips
newline delimiters during merge. This causes field names and values from
different columns to be concatenated without any separator, making the
content unreadable.

Changes:
- txt_parser.py: restore newline separator when merging adjacent text
segments within a chunk, so that split sections are not directly
concatenated
- rdbms_connector.py: use double newline between fields and place field
value on a new line after the field name bracket, giving TxtParser
clearer boundaries to work with

Closes #13001

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: tunsuytang <tunsuytang@tencent.com>
2026-03-04 21:42:02 +08:00
writinwaters
9deb3a6249 Refact: Fine tweaks to the doc structure. (#13379)
### What problem does this PR solve?

Fine tweaks to the doc structure.

### Type of change


- [x] Documentation Update
2026-03-04 21:30:28 +08:00
balibabu
be231faec0 Feat: Write the row and column numbers into the element's data attribute for easy code location. (#13368)
### What problem does this PR solve?

Feat: Write the row and column numbers into the element's data attribute
for easy code location.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 20:50:58 +08:00
Idriss Sbaaoui
b3a7332c08 playwright : add data-testids for new test (#13364)
### What problem does this PR solve?

add data-testids for new test

### Type of change

- [x] Other (please describe): add data-testids for new test
2026-03-04 19:28:36 +08:00
Yao Wei
c99b53064d fix: remove company info from resume_summary to prevent over-retrieval (#13358)
### What problem does this PR solve?

Problem: When searching for a specific company name like(Daofeng
Technology), the search would incorrectly return unrelated resumes
containing generic terms like (Technology) in their company names

Root Cause: The `corporation_name_tks` field was included in the
identity fields that are redundantly written to every chunk. This caused
common words like "科技" to match across all chunks, leading to
over-retrieval of irrelevant resumes.

Solution: Remove `corporation_name_tks` from the `_IDENTITY_FIELDS`
list. Company information is still preserved in the "Work Overview"
chunk where it belongs, allowing proper company-based searches while
preventing false positives from generic terms.

---------

Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 19:24:49 +08:00
Jin Hai
70e9743ef1 RAGFlow go API server (#13240)
# RAGFlow Go Implementation Plan 🚀

This repository tracks the progress of porting RAGFlow to Go. We'll
implement core features and provide performance comparisons between
Python and Go versions.

## Implementation Checklist

- [x] User Management APIs
- [x] Dataset Management Operations
- [x] Retrieval Test
- [x] Chat Management Operations
- [x] Infinity Go SDK

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-03-04 19:17:16 +08:00
Idriss Sbaaoui
2508c46c8f Playwright : add new test for configuration tab in datasets (#13365)
### What problem does this PR solve?

this pr adds new tests, for the full configuration tab in datasests

### Type of change

- [x] Other (please describe): new tests
2026-03-04 19:10:06 +08:00
Idriss Sbaaoui
88e8509159 benchmark fail in ci (#13377)
### What problem does this PR solve?
ci fails in elastic search because of benchmark

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 19:01:41 +08:00
Stephen Hu
c7d17c84b2 Refa:improve excel parser logic (#13372)
### What problem does this PR solve?

improve excel parser logic

### Type of change
- [x] Refactoring
2026-03-04 18:00:17 +08:00
Jin Hai
6bb00e2762 Update graspologic to gitee (#13362)
### What problem does this PR solve?

Accelerate python module downloading

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 17:48:47 +08:00
Good0987
8a7272f423 Test: add scenario for embedding_model update when chunk_count > 0 (#13351)
### What problem does this PR solve?

Guard embedding_model change when dataset has existing chunks. API must
return code 102 with message 'When chunk_num (N) > 0, embedding_model
must remain <current_model>' to prevent silent embedding drift.

### Type of change

- [x] Add Testcases

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 17:41:35 +08:00
Jin Hai
f47c47df99 Disable benchmark (#13370)
### What problem does this PR solve?

benchmark always failed in new CI machine. please enable it after the
issue is fixed.

### Type of change

- [x] Other (please describe): disable benchmark

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 16:36:42 +08:00
yiminghub2024
5eb602166c Enhance local model deployment documentation support gpustack guide (#13339)
### Type of change

- [X] Documentation Update:Enhance local model deployment documentation
support gpustack guide
2026-03-04 13:54:20 +08:00
少卿
54ae5b4a27 Fix Dify external retrieval by providing metadata.document_id (#13337)
### What problem does this PR solve?

## Summary                                                           
  Dify’s external retrieval expects `records[].metadata.document_id` to
  be a non-empty string.                                               
  RAGFlow currently only sets `metadata.doc_id`, which causes Dify     
  validation to fail.                                                  
                                                                       
  This PR adds `metadata.document_id` (mapped from `doc_id`) in the    
  Dify-compatible retrieval response.                                  
                                                                       
  ## Changes                                                           
- Add `meta["document_id"] = c["doc_id"]` in
`api/apps/sdk/dify_retrieval.py`
                                                                       
  ## Testing                                                           
  - Not run (logic-only change).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 13:23:37 +08:00
Jin Hai
b9ad014f63 Supports login cross multiple RAGFlow servers (#13322)
### What problem does this PR solve?

1. Use redis to store the secret key.
2. During startup API server will read the secret from redis. If no such
secret key, generate one and store it into redis, atomically.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-04 13:07:45 +08:00
balibabu
5f8966608d Fix: The dropdown menu for large models does not automatically focus on the search box. #13313 (#13360)
### What problem does this PR solve?

Fix: The dropdown menu for large models does not automatically focus on
the search box. #13313

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 12:48:35 +08:00
Magicbook1108
93d621a666 Fix: Correct PDF chunking parameter name in naive (#13357)
### What problem does this PR solve?

Fix: Correct PDF chunking parameter name in naive #13325

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 11:51:10 +08:00
balibabu
733a64f0d6 Fix: Change the background color of the message notification button. (#13344)
### What problem does this PR solve?

Fix: Change the background color of the message notification button.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-04 11:10:05 +08:00
statxc
839b603768 feat: Add PDF parser selection to Agent Begin and Await Response comp… (#13325)
### Issue: #12756

### What problem does this PR solve?

When users upload files through Agent's Begin or Await Response
components, the parsing is hardcoded to "Plain Text", ignoring all other
available parsers (DeepDOC, TCADP, Docling, MinerU, PaddleOCR). This PR
adds a PDF parser dropdown to these components so users can select the
appropriate parser for their file inputs.


### Changes

**Backend**
- `agent/component/fillup.py` - Added `layout_recognize` param to
`UserFillUpParam`, forwarded to `FileService.get_files()`
- `agent/component/begin.py` - Same forwarding in `Begin._invoke()`
- `agent/canvas.py` - Extract Begin's `layout_recognize` for `sys.files`
parsing, added param to `get_files_async()` / `get_files()`
- `api/db/services/file_service.py` - Added `layout_recognize` param to
`parse()` and `get_files()`, replacing hardcoded `"Plain Text"`
- `rag/app/naive.py` - Added `"plain text"` and `"tcadp parser"` aliases
to PARSERS dict to match dropdown values after `.lower()`

**Frontend**
- `web/src/pages/agent/form/begin-form/index.tsx` - Show
`LayoutRecognizeFormField` dropdown when file inputs exist
- `web/src/pages/agent/form/begin-form/schema.ts` - Added
`layout_recognize` to Zod schema
- `web/src/pages/agent/form/user-fill-up-form/index.tsx` - Same dropdown
for Await Response component


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-04 11:09:33 +08:00
Liu An
7715bad04e refactor: reorganize unit test files into appropriate directories (#13343)
### What problem does this PR solve?

Move test files from utils/ to their corresponding functional
directories:
- api/db/ for database related tests
- api/utils/ for API utility tests
- rag/utils/ for RAG utility tests

### Type of change

- [x] Refactoring
2026-03-04 11:02:56 +08:00
Copilot
33ba955b02 Translate Chinese text to English in agent/sandbox (#13356)
Chinese text remained in generated code comments, log messages, field
descriptions, and documentation files under `agent/sandbox/`.

### Changes

- **`tests/MIGRATION_GUIDE.md`** — Full EN translation (migration guide
from OpenSandbox → Code Interpreter)
- **`tests/QUICKSTART.md`** — Full EN translation (quick test guide for
Aliyun sandbox provider)
- **`providers/aliyun_codeinterpreter.py`** — Removed `(主账号ID)` from
docstring, error log, and config field description
- **`sandbox_spec.md`** — Removed `(主账号ID)` from `account_id` field
description
- **`tests/test_aliyun_codeinterpreter_integration.py`** — Removed
`(主账号ID)` from inline comment

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

<!-- START COPILOT CODING AGENT TIPS -->
---

💡 You can make Copilot smarter by setting up custom instructions,
customizing its development environment and configuring Model Context
Protocol (MCP) servers. Learn more [Copilot coding agent
tips](https://gh.io/copilot-coding-agent-tips) in the docs.

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: yuzhichang <153784+yuzhichang@users.noreply.github.com>
2026-03-04 10:49:38 +08:00
wyou
0a4c0c38c7 Feat: expose admin service in helm configuration (#13345)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
For helm deployment, there is also requirement to enable the Admin
Service for administrative operations.
So expose the ability of enable/disable this function by helm
configuration.
        When it's enabled (by default),
<img width="486" height="190" alt="image"
src="https://github.com/user-attachments/assets/4db0dc3d-bd94-4ad9-bb5d-a240aac5e1c5"
/>
        Admin access and operations would be feasible like below,
<img width="2530" height="876" alt="image"
src="https://github.com/user-attachments/assets/3e948e1b-7522-4f8d-8dc0-c80a22242022"
/>
Something like 'user management' is very much important for Ragflow
User/Owner to control their clients.
2026-03-04 10:26:10 +08:00
Idriss Sbaaoui
2f4ca38adf Fix : make playwright tests idempotent (#13332)
### What problem does this PR solve?

Playwright tests previously depended on cross-file execution order
(`auth -> provider -> dataset -> chat`).
This change makes setup explicit and idempotent via fixtures so tests
can run independently.

- Added/standardized prerequisite fixtures in
`test/playwright/conftest.py`:
- `ensure_auth_context`, `ensure_model_provider_configured`,
`ensure_dataset_ready`, `ensure_chat_ready`
- Made provisioning reusable/idempotent with `RUN_ID`-based resource
naming.
- Synced auth envs (`E2E_ADMIN_EMAIL`, `E2E_ADMIN_PASSWORD`) into seeded
creds.
- Fixed provider cache freshness (`auth_header`/`page` refresh on cache
hit).

Also included minimal stability fixes:
- dataset create stale-element click handling,
- search wait logic for results/empty-state,
- agent create-menu handling,
- agent run-step retry when run UI doesn’t open first click.

### Type of change

- [x] Test fix
- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-04 10:07:14 +08:00
writinwaters
1c87f97dde Docs: Minor document structure tweak. (#13346)
### What problem does this PR solve?

Refactored the document architecture.

### Type of change

- [x] Documentation Update
2026-03-03 20:09:34 +08:00
writinwaters
f7c808383f Docs: Refactored documentation (#13340)
### What problem does this PR solve?

Refactored documentation. 

### Type of change

- [x] Documentation Update
2026-03-03 17:48:48 +08:00
Yao Wei
48755a3352 Fix: (resume) Cross-verify project experience and work experience, and remove duplicate text (#13323)
Cross-verify project experience and work experience, and remove
duplicate text

---------

Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
2026-03-03 14:53:46 +08:00
balibabu
eca60208e3 Fix: The document generation node cannot generate the output content of a large model to a file. #13321 (#13326)
### What problem does this PR solve?

Fix: The document generation node cannot generate the output content of
a large model to a file. #13321
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-03 11:05:24 +08:00
Magicbook1108
4f09b3e2a4 Fix: pipeline canvas category (#13319)
### What problem does this PR solve?

Fix: pipeline canvas category

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 20:27:36 +08:00
Yongteng Lei
707de2461a Fix: use async_chat with sync wrapper in resume parser (#13320)
### What problem does this PR solve?

Fix AttributeError when calling llm.chat() in resume parser. LLMBundle
only has async_chat method, not chat method. Use `_run_coroutine_sync`
wrapper to call async_chat synchronously.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 19:51:06 +08:00
chanx
ef264b52c7 Fix: Fixed some errors in the console (#13317)
### What problem does this PR solve?

Fix: Fixed some errors in the console
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 19:19:15 +08:00
Yingfeng
a806f7b707 Potential fix for code scanning alert no. 71: Incomplete URL substring sanitization (#13318)
Potential fix for
[https://github.com/infiniflow/ragflow/security/code-scanning/71](https://github.com/infiniflow/ragflow/security/code-scanning/71)

In general, instead of using `String.prototype.includes` on the entire
URL string, parse the URL and make decisions based on its `host` (or
`hostname`) field. This avoids cases where the trusted domain appears in
the path, query, or as part of a different hostname.

Here, `payload.source_fid` is set to `'siliconflow_intl'` if
`postBody.base_url` “contains” `api.siliconflow.com`. To keep behavior
for correct inputs but close the hole, we should:

1. Safely parse `postBody.base_url` using the standard `URL` class.
2. Extract the hostname (`url.hostname`).
3. Compare it appropriately:
- If we only want the exact host `api.siliconflow.com`, use strict
equality.
- If international endpoints may include subdomains like
`foo.api.siliconflow.com`, allow those via suffix check on the hostname.
4. Fall back to `LLMFactory.SILICONFLOW` if parsing fails or the host
does not match.

Concretely, in `web/src/pages/user-setting/setting-model/hooks.tsx`, in
the `onApiKeySavingOk` callback where `payload.source_fid` is set,
replace the `toLowerCase().includes('api.siliconflow.com')` logic with a
small block that:

- Initializes a local `let sourceFid = LLMFactory.SILICONFLOW;`
- If `postBody.base_url` is present, attempts `new
URL(postBody.base_url)` inside a `try/catch`, lowercases `url.hostname`,
and checks whether it equals `api.siliconflow.com` or ends with
`.api.siliconflow.com`.
- Assigns `payload.source_fid = sourceFid`.

No new external dependencies are required; `URL` is available in modern
browsers and Node, and TypeScript understands it.


_Suggested fixes powered by Copilot Autofix. Review carefully before
merging._

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
2026-03-02 19:11:52 +08:00
Idriss Sbaaoui
b0ace2c5d0 feat: enable Arabic in production UI and add complete Arabic documentation (#13315)
### What problem does this PR solve?

This PR adds end-to-end Arabic support in production. It also adds a
full Arabic README

### Type of change

 - [x] New Feature (non-breaking change which adds functionality)
 - [x] Documentation Update
2026-03-02 19:10:11 +08:00
Yao Wei
f8c91e8854 Refa: Resume parsing module (architectural optimizations based on SmartResume Pipeline) (#13255)
Core optimizations (refer to arXiv:2510.09722):

1. PDF text fusion: Metadata + OCR dual-path extraction and fusion

2. Page-aware reconstruction: YOLOv10 page segmentation + hierarchical
sorting + line number indexing

3. Parallel task decomposition: Basic information/work
experience/educational background three-way parallel LLM extraction

4. Index pointer mechanism: LLM returns a range of line numbers instead
of generating the full text, reducing the illusion of full text.

---------

Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
Co-authored-by: Aron.Yao <yaowei@192.168.1.68>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 19:05:50 +08:00
balibabu
7d6f20585f Feat: Modify the style of the classification operator and fix some console errors. (#13314)
### What problem does this PR solve?

Feat: Modify the style of the classification operator and fix some
console errors.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 16:53:24 +08:00
Magicbook1108
5fc3bd38b0 Feat: Support siliconflow.com (#13308)
### What problem does this PR solve?

Feat: Support siliconflow.com

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 15:37:42 +08:00
Magicbook1108
1db221f19e Feat: add more models for siliconflow and tongyi-qwen (#13311)
### What problem does this PR solve?

Feat: add more models for siliconflow and tongyi-qwen

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 15:37:08 +08:00
liuxiaoyusky
8ba66dd62a Fix: respect user-configured chunk_token_num for MinerU/docling/paddleocr parsers (#13234)
## Summary

When using MinerU, docling, TCADP, or paddleocr as the PDF parser with
the General (naive) chunk method, the user-configured `chunk_token_num`
is **unconditionally overwritten to 0** at
[rag/app/naive.py#L858-L859](https://github.com/infiniflow/ragflow/blob/main/rag/app/naive.py#L858-L859),
effectively disabling chunk merging regardless of what the user sets in
the UI.

### Problem

A user sets `chunk_token_num = 2048` in the dataset configuration UI,
expecting small parser blocks to be merged into larger chunks. However,
this line:

```python
if name in ["tcadp", "docling", "mineru", "paddleocr"]:
    parser_config["chunk_token_num"] = 0
```

silently overrides the user's setting. As a result, every MinerU output
block becomes its own chunk. For short documents (e.g. a 3-page PDF fund
factsheet parsed by MinerU), this produces **47 tiny chunks** — some as
small as 11 characters (`"July 2025"`) or 15 characters (`"CIES
Eligible"`).

This severely degrades retrieval quality: vector embeddings of such
short fragments have minimal semantic value, and keyword search produces
excessive noise.

### Fix

Only apply the `chunk_token_num = 0` override when the user has **not**
explicitly configured a positive value:

```python
if name in ["tcadp", "docling", "mineru", "paddleocr"]:
    if int(parser_config.get("chunk_token_num", 0)) <= 0:
        parser_config["chunk_token_num"] = 0
```

This preserves the original default behavior (no merging) while
respecting the user's explicit configuration.

### Before / After (MinerU, 3-page PDF, chunk_token_num=2048)

| | Before | After |
|---|---|---|
| Chunks produced | 47 | ~8 (merged by token limit) |
| Smallest chunk | 11 chars | ~500 chars |
| User setting respected | No | Yes |

## Test plan

- [ ] Parse a PDF with MinerU and `chunk_token_num = 2048` → verify
chunks are merged up to token limit
- [ ] Parse a PDF with MinerU and `chunk_token_num = 0` (or default) →
verify original behavior (no merging)
- [ ] Parse a PDF with DeepDOC parser → verify no change in behavior
(not affected by this code path)
- [ ] Repeat with docling/paddleocr if available
2026-03-02 15:31:40 +08:00
少卿
d430446e69 fix:absolute page index mix-up in DeepDoc PDF parser (#12848)
### What problem does this PR solve?

Summary:
This PR addresses critical indexing issues in
deepdoc/parser/pdf_parser.py that occur when parsing long PDFs with
chunk-based pagination:

Normalize rotated table page numbering: Rotated-table re-OCR now writes
page_number in chunk-local 1-based form, eliminating double-addition of
page_from offset that caused misalignment between table positions and
document boxes.
Convert absolute positions to chunk-local coordinates: When inserting
tables/figures extracted via _extract_table_figure, positions are now
converted from absolute (0-based) to chunk-local indices before distance
matching and box insertion. This prevents IndexError and out-of-range
accesses during paged parsing of long documents.

Root Cause:
The parser mixed absolute (0-based, document-global) and relative
(1-based, chunk-local) page numbering systems. Table/figure positions
from layout extraction carried absolute page numbers, but insertion
logic expected chunk-local coordinates aligned with self.boxes and
page_cum_height.


Testing(I do):

Manual verification: Parse a 200+ page PDF with from_page > 0 and table
rotation enabled. Confirm that:

Tables and figures appear on correct pages
No IndexError or position mismatches occur
Page numbers in output match expected chunk-local offsets


Automated testing: 我没做


## Separate Discussion: Memory Optimization Strategy(from codex-5.2-max
and claude 4.5 opus and me)

### Context

The current implementation loads entire page ranges into memory
(`__images__`, `page_chars`, intermediates), which can cause RAM
exhaustion on large documents. While the page numbering fix resolves
correctness issues, scalability remains a concern.

### Proposed Architecture

**Pipeline-Driven Chunking with Explicit Resource Management:**

1. **Authoritative chunk planning**: Accept page-range specifications
from upstream pipeline as the single source of truth. The parser should
be a stateless worker that processes assigned chunks without making
independent pagination decisions.

2. **Granular memory lifecycle**:
   ```python
   for chunk_spec in chunk_plan:
       # Load only chunk_spec.pages into __images__
       page_images = load_page_range(chunk_spec.start, chunk_spec.end)
       
       # Process with offset tracking
       results = process_chunk(page_images, offset=chunk_spec.start)
       
       # Explicit cleanup before next iteration
       del page_images, page_chars, layout_intermediates
       gc.collect()  # Force collection of large objects
   ```

3. **Persistent lightweight state**: Keep model instances (layout
detector, OCR engine), document metadata (outlines, PDF structure), and
configuration across chunks to avoid reinitialization overhead (~2-5s
per chunk for model loading).

4. **Adaptive fallback**: Provide `max_pages_per_chunk` (default: 50)
only when pipeline doesn't supply a plan. Never exceed
pipeline-specified ranges to maintain predictable memory bounds.

5. **Optional: Dynamic budgeting**: Expose a memory budget parameter
that adjusts chunk size based on observed image dimensions and format
(e.g., reduce chunk size for high-DPI scanned documents).

### Benefits

- **Predictable memory footprint**: RAM usage bounded by `chunk_size ×
avg_page_size` rather than total document size
- **Horizontal scalability**: Enables parallel chunk processing across
workers
- **Failure isolation**: Page extraction errors affect only current
chunk, not entire document
- **Cloud-friendly**: Works within container memory limits (e.g., 2-4GB
per worker)

### Trade-offs

- **Increased I/O**: Re-opening PDF for each chunk vs. keeping file
handle (mitigated by page-range seeks)
- **Complexity**: Requires careful offset tracking and stateful
coordination between pipeline and parser
- **Warmup cost**: Model initialization overhead amortized across chunks
(acceptable for documents >100 pages)

### Implementation Priority

This optimization should be **deferred to a separate PR** after the
current correctness fix is merged, as:
1. It requires broader architectural changes across the pipeline
2. Current fix is critical for correctness and can be backported
3. Memory optimization needs comprehensive benchmarking on
representative document corpus


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 14:58:37 +08:00
Ahmad Intisar
184388879d feat: Add disable_password_login configuration to support SSO-only authentication (#13151)
### What problem does this PR solve?

Enterprise deployments that use an external Identity Provider (e.g.,
Microsoft Entra ID, Okta, Keycloak) need the ability to enforce SSO-only
authentication by hiding the email/password login form. Currently, the
login page always shows the password form alongside OAuth buttons, with
no way to disable it.

This PR adds a `disable_password_login` configuration option under the
existing `authentication` section in `service_conf.yaml`. When set to
`true`, the login page only displays configured OAuth/SSO buttons and
hides the email/password form, "Remember me" checkbox, and "Sign up"
link.

The flag can be set via:
- `service_conf.yaml` (`authentication.disable_password_login: true`)
- Environment variable (`DISABLE_PASSWORD_LOGIN=true`)

Default behavior is unchanged (`false`).

### Behavior

| `disable_password_login` | OAuth configured | Result |
|---|---|---|
| `false` (default) | No | Standard email/password form |
| `false` | Yes | Email/password form + SSO buttons below |
| `true` | Yes | **SSO buttons only** (no form, no sign up link) |
| `true` | No | Empty card (admin should configure OAuth first) |

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Files changed (5)

1. `docker/service_conf.yaml.template` — added `disable_password_login:
false` under authentication
2. `common/settings.py` — added `DISABLE_PASSWORD_LOGIN` global variable
and loader in `init_settings()`
3. `common/config_utils.py` — fixed `TypeError` in `show_configs()` when
authentication section contains non-dict values (e.g., booleans)
4. `api/apps/system_app.py` — exposed `disablePasswordLogin` flag in
`/config` endpoint
5. `web/src/pages/login/index.tsx` — conditionally render password form
based on config flag; OAuth buttons always render when channels exist

---------

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-03-02 14:06:03 +08:00
Magicbook1108
daec36e935 Fix: add soft limit for graph rag size (#13252)
### What problem does this PR solve?

Fix: add soft limit for graph rag size #13258 Q2

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 14:02:36 +08:00
huber
8a6b5ced6b fix: add missing chunk_data column to OceanBase schema migration (#13306)
### What problem does this PR solve?

When using OceanBase as the document storage engine, parsing and
inserting chunks with chunk_data (e.g., table parser row data) fails
with the following error:
```
[ERROR][Exception]: Insert chunk error: ['Unconsumed column names: chunk_data']
This happens because the chunk_data column was recently introduced but was omitted from the EXTRA_COLUMNS list in 
rag/utils/ob_conn.py
```
As a result, the automatic schema migration for existing OceanBase
tables does not append the missing chunk_data column, causing the
underlying pyobvector or SQLAlchemy to raise an unconsumed column names
error during data insertion.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### What is the solution?
Added column_chunk_data to the EXTRA_COLUMNS list in 
```
rag/utils/ob_conn.py
```
This ensures that the OceanBase connection wrapper can correctly detect
the missing column and automatically alter existing chunk tables to
include the chunk_data field during initialization.
2026-03-02 13:25:11 +08:00
Magicbook1108
f0dd12289c Feat: add preprocess parameters for ingestion pipeline (#13300)
### What problem does this PR solve?
Feat: add preprocess parameters for ingestion pipeline

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 13:18:57 +08:00
Yihang Wang
7fc97da610 security: Adopt Jinja2 SandboxedEnvironment for template rendering. (#13305) 2026-03-02 13:17:29 +08:00
Idriss Sbaaoui
860c4bd0bb Feat: UI testing automation with playwright (#12749)
### What problem does this PR solve?

This PR helps automate the testing of the ui interface using pytest
Playwright

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Other (please describe): test automation infrastructure

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-03-02 13:04:08 +08:00
Attili-sys
21bc1ab7ec Feature rtl support (#13118)
### What problem does this PR solve?

This PR adds comprehensive **Right-to-Left (RTL) language support**,
primarily targeting Arabic and other RTL scripts (Hebrew, Persian, Urdu,
etc.).

Previously, RTL content had multiple rendering issues:

- Incorrect sentence splitting for Arabic punctuation in citation logic
- Misaligned text in chat messages and markdown components  
- Improper positioning of blockquotes and “think” sections  
- Incorrect table alignment  
- Citation placement ambiguity in RTL prompts  
- UI layout inconsistencies when mixing LTR and RTL text  

This PR introduces backend and frontend improvements to properly detect,
render, and style RTL content while preserving existing LTR behavior.

#### Backend
- Updated sentence boundary regex in `rag/nlp/search.py` to include
Arabic punctuation:
  - `،` (comma)
  - `؛` (semicolon)
  - `؟` (question mark)
  - `۔` (Arabic full stop)
- Ensures citation insertion works correctly in RTL sentences.
- Updated citation prompt instructions to clarify citation placement
rules for RTL languages.

#### Frontend
- Introduced a new utility: `text-direction.ts`
  - Detects text direction based on Unicode ranges.
  - Supports Arabic, Hebrew, Syriac, Thaana, and related scripts.
  - Provides `getDirAttribute()` for automatic `dir` assignment.

- Applied dynamic `dir` attributes across:
  - Markdown rendering
  - Chat messages
  - Search results
  - Tables
  - Hover cards and reference popovers

- Added proper RTL styling in LESS:
  - Text alignment adjustments
  - Blockquote border flipping
  - Section indentation correction
  - Table direction switching
  - Use of `<bdi>` for figure labels to prevent bidirectional conflicts

#### DevOps / Environment
- Added Windows backend launch script with retry handling.
- Updated dependency metadata.
- Adjusted development-only React debugging behavior.

---

### Type of change

- [x] Bug Fix (non-breaking change which fixes RTL rendering and
citation issues)
- [x] New Feature (non-breaking change which adds RTL detection and
dynamic direction handling)

---------

Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Ahmad Intisar <168020872+ahmadintisar@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-02 13:03:44 +08:00
balibabu
a897aedea9 Feat: Modify the form styles for retrieval and conditional operators. (#13299)
### What problem does this PR solve?

Feat: Modify the form styles for retrieval and conditional operators.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 12:05:27 +08:00
chanx
0cdddea59a feat: pipeline add preprocess (#13302)
### What problem does this PR solve?

feat: pipeline add preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:50:48 +08:00
balibabu
cf3d3c7c89 Feat: When exporting the agent DSL, the tailkey, password, and history fields need to be cleared. #13281 (#13282)
### What problem does this PR solve?
Feat: When exporting the agent DSL, the tailkey, password, and history
fields need to be cleared. #13281

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:41:38 +08:00
dependabot[bot]
b956ad180c Build(deps): Bump pypdf from 6.7.3 to 6.7.4 (#13298)
Bumps [pypdf](https://github.com/py-pdf/pypdf) from 6.7.3 to 6.7.4.
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/releases">pypdf's
releases</a>.</em></p>
<blockquote>
<h2>Version 6.7.4, 2026-02-27</h2>
<h2>What's new</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)
by <a
href="https://github.com/stefan6419846"><code>@​stefan6419846</code></a></li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/py-pdf/pypdf/blob/main/CHANGELOG.md">pypdf's
changelog</a>.</em></p>
<blockquote>
<h2>Version 6.7.4, 2026-02-27</h2>
<h3>Security (SEC)</h3>
<ul>
<li>Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)</li>
</ul>
<h3>Robustness (ROB)</h3>
<ul>
<li>Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)</li>
</ul>
<p><a href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">Full
Changelog</a></p>
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="1650bc31e8"><code>1650bc3</code></a>
REL: 6.7.4</li>
<li><a
href="f309c60037"><code>f309c60</code></a>
SEC: Allow limiting output length for RunLengthDecode filter (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3664">#3664</a>)</li>
<li><a
href="993f052748"><code>993f052</code></a>
DEV: Bump actions/upload-artifact from 6 to 7 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3662">#3662</a>)</li>
<li><a
href="a3c996bffc"><code>a3c996b</code></a>
DEV: Bump actions/download-artifact from 7 to 8 (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3663">#3663</a>)</li>
<li><a
href="37de32022e"><code>37de320</code></a>
ROB: Deal with invalid annotations in extract_links (<a
href="https://redirect.github.com/py-pdf/pypdf/issues/3659">#3659</a>)</li>
<li>See full diff in <a
href="https://github.com/py-pdf/pypdf/compare/6.7.3...6.7.4">compare
view</a></li>
</ul>
</details>
<br />


[![Dependabot compatibility
score](https://dependabot-badges.githubapp.com/badges/compatibility_score?dependency-name=pypdf&package-manager=uv&previous-version=6.7.3&new-version=6.7.4)](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores)

Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore this major version` will close this PR and stop
Dependabot creating any more for this major version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop
Dependabot creating any more for this minor version (unless you reopen
the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop
Dependabot creating any more for this dependency (unless you reopen the
PR or upgrade to it yourself)
You can disable automated security fix PRs for this repo from the
[Security Alerts
page](https://github.com/infiniflow/ragflow/network/alerts).

</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:32:12 +08:00
Idriss Sbaaoui
9d78d3ddb1 Tests: fix failling http in CI (#13301)
### What problem does this PR solve?
test_doc_sdk_routes_unit had two flaky/incorrect branch assumptions:

1. parse/stop_parsing production logic gates on doc.run, but tests used
progress, causing branch mismatch and unintended fallthrough into
mutation/DB paths.
2. stop_parsing invalid-state test asserted an outdated message
fragment, making the contract brittle.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-02 10:44:33 +08:00
Jimmy Ben Klieve
7e0dd906f2 refactor: update admin ui (#13280)
### What problem does this PR solve?

Update for Admin UI:
- Update file picker input in **Registration whitelist** > **Import from
Excel** modal
- Modify DOM structure of **Sandbox Settings** and move several
hardcoded texts into translation files

### Type of change

- [x] Refactoring
2026-02-28 19:21:51 +08:00
Idriss Sbaaoui
e62552d482 Added some React IDs for playwright e2e tests (#13265)
### What problem does this PR solve?

Necessary ids for implementing the new testing suite with playwright for
UI

### Type of change

- [x] Other (please describe): Testing IDs

Co-authored-by: Liu An <asiro@qq.com>
2026-02-28 15:13:47 +08:00
Magicbook1108
1027916bfe Fix: inconsistent state handling for multi-user single-canvas access (#13267)
### What problem does this PR solve?

<img width="700" alt="image"
src="https://github.com/user-attachments/assets/1db7412e-4554-44bc-84ba-16421949aacc"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-02-28 15:09:21 +08:00
Yongteng Lei
c91e803a38 Fix: close detached PIL image on JPEG save failure in encode_image (#13278)
### What problem does this PR solve?

Properly close detached PIL image on JPEG save failure in encode_image.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 14:43:35 +08:00
天海蒼灆
983150b936 Fix (api): fix the document parsing status check logic (#12504)
### What problem does this PR solve?
When the original code terminates the parsing task halfway, the progress
may not be 0 or 1, which will result in the inability to call the
interface to parse again

-Change the document parsing progress check to task status check, and
use TaskStatus.RUNNING.value to judge
-Update the condition judgment for stopping parsing documents, and check
whether the task is running instead


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 14:38:55 +08:00
Jin Hai
32ec950ca8 Fix create / drop chat session syntax (#13279)
### What problem does this PR solve?

This pull request refactors the chat session creation and deletion logic
in both the parser and client code to use unique session IDs instead of
session names. It also updates the corresponding command syntax and
payloads, ensuring more robust and unambiguous session management.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 14:18:21 +08:00
Jin Hai
d9d4825079 Add chat sessions related command (#13268)
### What problem does this PR solve?

1. Create / Drop / List chat sessions
2. Chat with LLM and datasets

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 12:52:45 +08:00
Jin Hai
54094771a3 Fix streaming chat on web API (#13275)
### What problem does this PR solve?

This pull request makes a small but important fix to how streaming
requests are handled in the `completion` endpoint of
`conversation_app.py`. The main change ensures that the `stream`
argument is not passed twice, which could cause errors.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-28 12:16:38 +08:00
Yongteng Lei
0110151e12 Fix: document remove race condition (#13242)
### What problem does this PR solve?

Fix document remove race condition.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-28 11:23:24 +08:00
eviaaaaa
fa71f8d0c7 refactor(word): lazy-load DOCX images to reduce peak memory without changing output (#13233)
**Summary**
This PR tackles a significant memory bottleneck when processing
image-heavy Word documents. Previously, our pipeline eagerly decoded
DOCX images into `PIL.Image` objects, which caused high peak memory
usage. To solve this, I've introduced a **lazy-loading approach**:
images are now stored as raw blobs and only decoded exactly when and
where they are consumed.

This successfully reduces the memory footprint while keeping the parsing
output completely identical to before.

**What's Changed**
Instead of a dry file-by-file list, here is the logical breakdown of the
updates:

* **The Core Abstraction (`lazy_image.py`)**: Introduced `LazyDocxImage`
along with helper APIs to handle lazy decoding, image-type checks, and
NumPy compatibility. It also supports `.close()` and detached PIL access
to ensure safe lifecycle management and prevent memory leaks.
* **Pipeline Integration (`naive.py`, `figure_parser.py`, etc.)**:
Updated the general DOCX picture extraction to return these new lazy
images. Downstream consumers (like the figure/VLM flow and base64
encoding paths) now decode images right at the use site using detached
PIL instances, avoiding shared-instance side effects.
* **Compatibility Hooks (`operators.py`, `book.py`, etc.)**: Added
necessary compatibility conversions so these lazy images flow smoothly
through existing merging, filtering, and presentation steps without
breaking.

**Scope & What is Intentionally Left Out**
To keep this PR focused, I have restricted these changes strictly to the
**general Word pipeline** and its downstream consumers.
The `QA` and `manual` Word parsing pipelines are explicitly **not
modified** in this PR. They can be safely migrated to this new lazy-load
model in a subsequent, standalone PR.

**Design Considerations**
I briefly considered adding image compression during processing, but
decided against it to avoid any potential quality degradation in the
derived outputs. I also held off on a massive pipeline re-architecture
to avoid overly invasive changes right now.

**Validation & Testing**
I've tested this to ensure no regressions:

* Compared identical DOCX inputs before and after this branch: chunk
counts, extracted text, table HTML, and image descriptions match
perfectly.
* **Confirmed a noticeable drop in peak memory usage when processing
image-dense documents.** For a 30MB Word document containing 243 1080p
screenshots, memory consumption is reduced by approximately 1.5GB.

**Breaking Changes**
None.
2026-02-28 11:22:31 +08:00
SFL79
4f0c892b32 feat(ui): add individual model delete buttons across all providers (#13271)
### What problem does this PR solve?

Added the option to delete models individually from providers.
For additional context, see
[issue-13184](https://github.com/infiniflow/ragflow/issues/13184)

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Note: when deleting a selected model, it leaves the full model name as
text as seen here:
<img width="676" height="90" alt="image"
src="https://github.com/user-attachments/assets/c11c7c1b-3f2a-4119-b20c-bb8148a8ad16"
/>

If attempting to use ragflow with that deleted model, ragflow will throw
an unauthorized model error as expected.
I left it like that on purpose, so it's easier for the user to
understand what he deleted and that he needs to replace it with another
model.

Co-authored-by: Shahar Flumin <shahar@Shahars-MacBook-Air.local>
2026-02-28 10:51:39 +08:00
Yesid Cano Castro
d1afcc9e71 feat(seafile): add library and directory sync scope support (#13153)
### What problem does this PR solve?

The SeaFile connector currently synchronises the entire account — every
library
visible to the authenticated user. This is impractical for users who
only need
a subset of their data indexed, especially on large SeaFile instances
with many
shared libraries.

This PR introduces granular sync scope support, allowing users to choose
between
syncing their entire account, a single library, or a specific directory
within a
library. It also adds support for SeaFile library-scoped API tokens
(`/api/v2.1/via-repo-token/` endpoints), enabling tighter access control
without
exposing account-level credentials.


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Test

```
from seafile_connector import SeaFileConnector
import logging
import os

logging.basicConfig(level=logging.DEBUG)

URL = os.environ.get("SEAFILE_URL", "https://seafile.example.com")
TOKEN = os.environ.get("SEAFILE_TOKEN", "")
REPO_ID = os.environ.get("SEAFILE_REPO_ID", "")
SYNC_PATH = os.environ.get("SEAFILE_SYNC_PATH", "/Documents")
REPO_TOKEN = os.environ.get("SEAFILE_REPO_TOKEN", "")

def _test_scope(scope, repo_id=None, sync_path=None):
    print(f"\n{'='*50}")
    print(f"Testing scope: {scope}")
    print(f"{'='*50}")

    creds = {"seafile_token": TOKEN} if TOKEN else {}
    if REPO_TOKEN and scope in ("library", "directory"):
        creds["repo_token"] = REPO_TOKEN

    connector = SeaFileConnector(
        seafile_url=URL,
        batch_size=5,
        sync_scope=scope,
        include_shared = False,
        repo_id=repo_id,
        sync_path=sync_path,
    )
    connector.load_credentials(creds)
    connector.validate_connector_settings()

    count = 0
    for batch in connector.load_from_state():
        for doc in batch:
            count += 1
            print(f"  [{count}] {doc.semantic_identifier} "
                  f"({doc.size_bytes} bytes, {doc.extension})")

    print(f"\n-> {scope} scope: {count} document(s) found.\n")

# 1. Account scope
if TOKEN:
    _test_scope("account")
else:
    print("\nSkipping account scope (set SEAFILE_TOKEN)")

# 2. Library scope
if REPO_ID and (TOKEN or REPO_TOKEN):
    _test_scope("library", repo_id=REPO_ID)
else:
    print("\nSkipping library scope (set SEAFILE_REPO_ID + token)")

# 3. Directory scope
if REPO_ID and SYNC_PATH and (TOKEN or REPO_TOKEN):
    _test_scope("directory", repo_id=REPO_ID, sync_path=SYNC_PATH)
else:
    print("\nSkipping directory scope (set SEAFILE_REPO_ID + SEAFILE_SYNC_PATH + token)")
```
2026-02-28 10:24:28 +08:00
Stephen Hu
aec2ef4232 refactor:improve tts model's codes (#13137)
### What problem does this PR solve?

improve tts model's codes

### Type of change

- [x] Refactoring
2026-02-28 10:18:00 +08:00
Stephen Hu
9577753c10 Refactor: improve the logic about docling parser extract box (#13215)
### What problem does this PR solve?
 improve the logic about docling parser extract box

### Type of change
- [x] Refactoring
2026-02-28 10:05:24 +08:00
chanx
510ff89661 Fix: remove unused files (#13232)
### What problem does this PR solve?

Fix: remove unused files

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 23:05:40 +08:00
Jimmy Ben Klieve
c0823e8d6d refactor: update chat ui (#13269)
### What problem does this PR solve?

Update **Chat** UI:
- Align to the design.
- Update `<AudioButton>` visualizer logic.
- Fix keyboard navigation issue.

### Type of change

- [x] Refactoring
2026-02-27 22:26:19 +08:00
Enes Delibalta
4e48aba5c4 fix: update DoclingParser return type hint (#13243)
### What problem does this PR solve?

The _transfer_to_sections method was throwing a type hint violation
because it occasionally returns 3-item tuples instead of 2. Adjusted to
list[tuple[str, ...]] to prevent runtime crashes.

Error: 

20:53:21 Page(1~10): [ERROR]Internal server error while chunking:
Method
deepdoc.parser.docling_parser.DoclingParser._transfer_to_sections()
return [(1. JIRA Nasıl Kullanılır?, text,
@@1\t70.8\t194.9\t70.9\t85.5##), (1.1. Proje O...##)] violates type
hint list[tuple[str, str]], as list index
15 item tuple tuple (Gelen ekran
üzerinden alanları isterlerine göre doldurduğunuz taktirde Create
düğmesi i...##) length 3 != 2.
20:53:21 [ERROR][Exception]: Method
deepdoc.parser.docling_parser.DoclingParser._transfer_to_sections()
return [('1. JIRA Nasıl Kullanılır?', 'text',
'@@1\t70.8\t194.9\t70.9\t85.5##'), ('1.1. Proje O...##')] violates
type hint list[tuple[str, str]], as list index
15 item tuple tuple ('Gelen ekran
üzerinden alanları isterlerine göre doldurduğunuz taktirde Create
düğmesi i...##') length 3 != 2.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Enes Delibalta <enes.delibalta@pentanom.com>
2026-02-27 20:13:50 +08:00
Yuxing Deng
51b180d991 fix: adding GPUStack chat model requires v1 suffix (#13237)
### What problem does this PR solve?

Refer to issue: #13236
The base url for GPUStack chat model requires `/v1` suffix. For the
other model type like `Embedding` or `Rerank`, the `/v1` suffix is not
required and will be appended in code.
So keep the same logic for chat model as other model type.

### Type of change

- [X] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 20:13:07 +08:00
as-ondewo
194e076e26 Fix: init superuser can create duplicate users (#13221)
### What problem does this PR solve?

This PR fixes 2 bugs related to RAGFlow's init superuser functionality.

#### Bug 1

When the RAGFlow server was started with the `--init-superuser` option
it would always create a new admin user even if it already exists
resulting in duplicate users.

To fix this, I added an additional check before create the superuser and
added the *unique* constraint to the email column of the database, to
mitigate potential TOCTOU race conditions. Since existing databases
could contain duplicate emails I added email de-duplication to the
database migration.

#### Bug 2

When the RAGFlow server was started with the `--init-superuser` option
but without configured default LLM and embedding models it would fail to
start because the `init_superuser` function would always make test
request to the models even if they were not set.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 19:55:51 +08:00
balibabu
6d0100ca67 Fix: The output content of the multi-model comparison will disappear. #13227 (#13241)
### What problem does this PR solve?

Fix: The output content of the multi-model comparison will disappear.
#13227
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 19:18:40 +08:00
balibabu
861ebfc6e1 Feat: Make the embedded page of chat compatible with mobile devices. (#13262)
### What problem does this PR solve?
Feat: Make the embedded page of chat compatible with mobile devices.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-27 19:17:41 +08:00
avianion
5f53fbe0f1 feat: Add Avian as an LLM provider (#13256)
### What problem does this PR solve?

This PR adds [Avian](https://avian.io) as a new LLM provider to RAGFlow.
Avian provides an OpenAI-compatible API with competitive pricing,
offering access to models like DeepSeek V3.2, Kimi K2.5, GLM-5, and
MiniMax M2.5.

**Provider details:**
- API Base URL: `https://api.avian.io/v1`
- Auth: Bearer token via API key
- OpenAI-compatible (chat completions, streaming, function calling)
- Models:
  - `deepseek/deepseek-v3.2` — 164K context, $0.26/$0.38 per 1M tokens
  - `moonshotai/kimi-k2.5` — 131K context, $0.45/$2.20 per 1M tokens
  - `z-ai/glm-5` — 131K context, $0.30/$2.55 per 1M tokens
  - `minimax/minimax-m2.5` — 1M context, $0.30/$1.10 per 1M tokens

**Changes:**
- `rag/llm/chat_model.py` — Add `AvianChat` class extending `Base`
- `rag/llm/__init__.py` — Register in `SupportedLiteLLMProvider`,
`FACTORY_DEFAULT_BASE_URL`, `LITELLM_PROVIDER_PREFIX`
- `conf/llm_factories.json` — Add Avian factory with model definitions
- `web/src/constants/llm.ts` — Add to `LLMFactory` enum, `IconMap`,
`APIMapUrl`
- `web/src/components/svg-icon.tsx` — Register SVG icon
- `web/src/assets/svg/llm/avian.svg` — Provider icon
- `docs/references/supported_models.mdx` — Add to supported models table

This follows the same pattern as other OpenAI-compatible providers
(e.g., n1n #12680, TokenPony).

cc @KevinHuSh @JinHai-CN

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-02-27 17:36:55 +08:00
6ba3i
bb59a27e55 Doc : Add french Readme (#13254)
### What problem does this PR solve?

Add fench Readme

### Type of change

- [x] Documentation Update
2026-02-27 11:34:13 +08:00
qinling0210
8b6d363a98 Use pagination in _search_metadata (#13238)
### What problem does this PR solve?

Fix [#13210](https://github.com/infiniflow/ragflow/issues/13210)

Remove limit in _search_metadata, use pagination in _search_metadata.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-27 11:24:49 +08:00
Jin Hai
a1549c0fdc Fix UI (#13239)
### What problem does this PR solve?

This pull request makes a minor update to the English locale strings for
the Table of Contents toggle buttons, changing the labels from "Show
TOC"/"Hide TOC" to "Show content"/"Hide content" for improved clarity.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-26 19:21:08 +08:00
Magicbook1108
c03c537bf8 Feat: optimize gmail/google-drive (#13230)
### What problem does this PR solve?

Feat: optimize gmail/google-drive

Now:
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/0c4b6044-7209-4c4f-ac0c-32070b79daf7"
/>
<img width="700" alt="image"
src="https://github.com/user-attachments/assets/406f93d8-9b0f-4f5a-b8bb-3936990f558c"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 19:19:40 +08:00
6ba3i
22c4d72891 tests: improve RAGFlow coverage based on Codecov report (#13219)
### What problem does this PR solve?

Codecov’s coverage report shows that several RAGFlow code paths are
currently untested or under-tested. This makes it easier for regressions
to slip in during refactors and feature work.
This PR adds targeted automated tests to cover the files and branches
highlighted by Codecov, improving confidence in core behavior while
keeping runtime functionality unchanged.

### Type of change

- [x] Other (please describe): Test coverage improvement (adds/extends
unit and integration tests to address Codecov-reported gaps)
2026-02-26 19:03:26 +08:00
Magicbook1108
1aa49a11f0 Feat: support AWS SES smtp (#13195)
### What problem does this PR solve?

Support AWS SES smtp #13179

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 13:49:53 +08:00
writinwaters
74dc43406f Docs: After careful consideration, the RAGFlow team decided to hold o… (#13226)
…ff publishing this guide.

### What problem does this PR solve?

Removed failsure mode checklist per your request. @JinHai-CN 

### Type of change


- [x] Documentation Update
2026-02-26 12:39:58 +08:00
balibabu
d2dd0b7e50 Fix: The agent is embedded in the webpage; interrupting its operation will redirect to the login page. #12697 (#13224)
### What problem does this PR solve?

Fix: The agent is embedded in the webpage; interrupting its operation
will redirect to the login page. #12697

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 12:39:28 +08:00
chanx
8bce212284 Fix: error in retrieval testing page (#13225)
### What problem does this PR solve?

Fix: error in retrieval testing page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 12:39:09 +08:00
Angel98518
024edba1b8 fix(web): prevent duplicate i18n languageChanged listeners (#13218)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring
2026-02-26 10:45:50 +08:00
PandaMan
d43aebe701 Fix/13142 auto metadata (#13217)
### What problem does this PR solve?

Close #13142

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 10:25:48 +08:00
Angel98518
b54260bcd7 fix(web): correct initial chat variable enabled state (#13214)
## Summary

Fixes the initial enabled/disabled state of chat variable checkboxes by
correcting a helper function that previously always returned .

## Problem

 in  had two  statements:



Because of the early , the function always returned , so all chat
variable checkboxes were initially disabled regardless of the field.
This also made the helper inconsistent with , which enables all fields
by default except .

## Fix

Update  to use the same condition as :



This ensures:
- All chat variable checkboxes are enabled by default
-  remains the only field disabled by default
- Behavior is consistent between the helper and the checkbox map
initialization in .

No API or backend changes are involved; this is a small, isolated
frontend bugfix.
2026-02-26 10:25:14 +08:00
Magicbook1108
158503a1aa Feat: optimize ingestion pipeline with preprocess (#13211)
### What problem does this PR solve?

Feat: optimize ingestion pipeline with preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 10:24:13 +08:00
PSBigBig × MiniPS
b7eca981d4 docs: add RAG failure modes checklist guide (refs #13138) (#13204)
### What problem does this PR solve?

This PR adds a new guide: **"RAG failure modes checklist"**.

RAG systems often fail in ways that are not immediately visible from a
single metric like accuracy or latency. In practice, debugging
production RAG applications requires identifying recurring failure
patterns across retrieval, routing, evaluation, and deployment stages.

This guide introduces a structured, pattern-based checklist (P01–P12) to
help users interpret traces, evaluation results, and dataset behavior
within RAGFlow. The goal is to provide a practical way to classify
incidents (e.g., retrieval hallucination, chunking issues, index
staleness, routing misalignment) and reason about minimal structural
fixes rather than ad-hoc prompt changes.

The change is documentation-only and does not modify any code or
configuration.

Refs #13138


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-02-25 19:35:15 +08:00
writinwaters
f9e0eb38ec Refact: Updated ingestion pipeline UI. (#13216)
### What problem does this PR solve?

Updated ingestion pipeline-specific UI tips.

### Type of change

- [x] Refactoring
2026-02-25 19:29:04 +08:00
6ba3i
38011f2c16 tests: improve RAGFlow coverage based on Codecov report (#13200)
### What problem does this PR solve?

Codecov’s coverage report shows that several RAGFlow code paths are
currently untested or under-tested. This makes it easier for regressions
to slip in during refactors and feature work.
This PR adds targeted automated tests to cover the files and branches
highlighted by Codecov, improving confidence in core behavior while
keeping runtime functionality unchanged.

### Type of change

- [x] Other (please describe): Test coverage improvement (adds/extends
unit and integration tests to address Codecov-reported gaps)
2026-02-25 19:12:11 +08:00
balibabu
2a5ddf064d Fix: Note component text area does not resize with component #13065 (#13212)
### What problem does this PR solve?

Fix: Note component text area does not resize with component #13065

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 17:24:07 +08:00
Jimmy Ben Klieve
220e611e33 refactor: ux improvements for variable picker in prompt editor (#13213)
### What problem does this PR solve?

User experience enhancement for variable picker in prompt editor:

- Add case-insensitive string search for variables.
- Add basic keyboard navigation in variable picker:
   - Hit <kbd>UpArrow</kbd> and <kbd>DownArrow</kbd> for navigating.
- Hit <kbd>Tab</kbd> or <kbd>Enter</kbd> for selecting focused item into
editor.
- Fix unexpectedly inserting invalid variable into editor by hitting
<kbd>Tab</kbd>.

_Note: you still need to pick variables inside secondary menu (agent
structured output, etc.) by using your pointing device. May finish these
later._

### Type of change

- [x] Refactoring
2026-02-25 17:22:48 +08:00
He Wang
394ff16b66 fix: OceanBase metadata not returned in document list API (#13209)
### What problem does this PR solve?

Fix #13144.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 15:29:17 +08:00
Phives
4ceb668d40 feat(api/utils): Harden file_utils for robustness and edge cases (#12915)
## Summary
Improves robustness and edge-case handling in `api.utils.file_utils` to
avoid crashes, DoS/OOM risks, and timeouts when processing user-provided
filenames, paths, and file blobs.

## Changes

### Resource limits & timeouts
- **`MAX_BLOB_SIZE_THUMBNAIL`** (50 MiB) and **`MAX_BLOB_SIZE_PDF`**
(100 MiB) to reject oversized inputs before thumbnail/PDF processing.
- **`GHOSTSCRIPT_TIMEOUT_SEC`** (120 s) for
`repair_pdf_with_ghostscript` subprocess to avoid hangs on malicious or
broken PDFs.

### `filename_type`
- Handles `None`, empty string, non-string (e.g. int/list), and
path-only input via new **`_normalize_filename_for_type()`**.
- Uses basename for type detection (e.g. `a/b/c.pdf` → PDF).
- Enforces **`FILE_NAME_LEN_LIMIT`**; invalid input returns
`FileType.OTHER`.

### `thumbnail_img`
- Rejects `None`/empty/oversized blob and invalid filename; returns
`None` instead of raising.
- Wraps PDF, image, and PPT handling in try/except so corrupt or
malformed files return `None`.
- Ensures PDF has pages and PPT has slides before use.
- Normalizes PIL image mode (RGBA/P/LA → RGB) for safe PNG export.

### `repair_pdf_with_ghostscript`
- Handles `None`/empty input; skips repair when input size exceeds
limit.
- Uses `subprocess.run(..., timeout=GHOSTSCRIPT_TIMEOUT_SEC)` and
catches `TimeoutExpired`.
- Returns original bytes when Ghostscript output is empty.

### `read_potential_broken_pdf`
- `None` → `b""`; non–sequence-like (no `len`) → `b""`; empty → return
as-is.
- Oversized blob returned as-is (no repair) to avoid DoS.

### `sanitize_path`
- Explicit `None` and non-string check; strips whitespace before
normalizing.

## Testing
- **`test/unit_test/utils/test_api_file_utils.py`** added with 36 unit
tests covering the above behavior (filename_type, sanitize_path,
read_potential_broken_pdf, thumbnail_img, thumbnail,
repair_pdf_with_ghostscript, constants).
- All tests pass.

---------

Co-authored-by: Gittensor Miner <miner@gittensor.io>
2026-02-25 14:34:47 +08:00
PentaFDevs
8ad47bf242 feat: add 'Open in new tab' button for agents (#13044)
- Add new button in agent management dropdown to open agent in new tab
- Implement token-based authentication for shared agent access
- Add translations for 9 languages (en, zh, zh-tw, de, fr, it, ru,
pt-br, vi)
- Keep existing 'Embed into webpage' functionality intact

### What problem does this PR solve?

This allows users to open agents in a separate tab to work in background
while continuing to use other parts of the application.

<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/ca1719c8-2f00-4570-a730-1321fa0bfd57"
/>
<img width="254" height="222" alt="image"
src="https://github.com/user-attachments/assets/b3dd6d9f-b7e7-46b0-83e7-f0ea86e7b156"
/>
<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/e94e99f9-9039-43f7-b2d9-862b9448630c"
/>

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-25 13:39:02 +08:00
Yao Wei
cf6fd6f115 fix: When using OceanBase as storage, the list_chunk sorting is abnormal. #13198 (#13208)
Actual behavior
When using OceanBase as storage, the list_chunk sorting is abnormal. The
following is the SQL statement.

SELECT id, content_with_weight, important_kwd, question_kwd, img_id,
available_int, position_int, doc_type_kwd, create_timestamp_flt,
create_time, array_to_string(page_num_int, ',') AS page_num_int_sort,
array_to_string(top_int, ',') AS top_int_sort FROM
rag_store_284250730805059584 WHERE doc_id = '' AND kb_id IN ('') ORDER
BY page_num_int_sort ASC, top_int_sort ASC, create_timestamp_flt DESC
LIMIT 0, 20

<img width="1610" height="740" alt="image"
src="https://github.com/user-attachments/assets/84e14c30-a97f-4e8f-8c8c-6ccac915d97d"
/>

Co-authored-by: Aron.Yao <yaowei@yaoweideMacBook-Pro.local>
2026-02-25 13:36:18 +08:00
Ray Zhang
cbe64402db feat(migration): support docker compose -p project name for backup/restore (#13191)
### What problem does this PR solve?

When users start RAGFlow with `docker compose -p <alias>`, Docker
creates volumes prefixed with the alias (e.g., `myproject_mysql_data`).
The migration script (`docker/migration.sh`) previously hardcoded the
`docker_` prefix in volume names, causing backup/restore to silently
skip all volumes for any non-default project name.

This PR adds a `-p <project_name>` option so the script correctly
targets volumes regardless of the Docker Compose project name used.

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Changes

- Add `-p <project_name>` flag (default: `docker`) for specifying Docker
Compose project name
- Build volume names dynamically: `${project_name}_${base_name}`
- Update help text with new option documentation and examples
- Show project-aware `docker compose` commands in error messages
- Fix deprecated `docker-compose` to `docker compose` in hints
- Use dynamic step count instead of hardcoded `4`
- Fully backward compatible — existing usage without `-p` works
unchanged

### Usage

```bash
# Existing usage (unchanged)
./migration.sh backup
./migration.sh restore my_backup

# New: custom project name
./migration.sh -p myproject backup
./migration.sh -p myproject restore my_backup
```
2026-02-25 13:18:47 +08:00
Yongteng Lei
2bf2abfdbc Fix: authorization bypass (IDOR) in /v1/document/web_crawl (#13203)
### What problem does this PR solve?

Fix authorization bypass (IDOR) in `/v1/document/web_crawl` allows
Cross-Tenant Dataset Modification.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 12:59:41 +08:00
Ahmad Intisar
99d1c9725c Bug mysql connector empty content resolved: Semantic ID Issue (#13206)
The RDBMS (MySQL/PostgreSQL) connector generates document filenames
using the first 100 characters of the content column
(semantic_identifier). When the content contains newline characters
(\n), the resulting filename includes those newlines — for example:
Category: غير صحيح كليًا\nTitle: تفنيد حقائق....txt
RAGFlow's filename_type() function uses re.match(r".*\.txt$", filename)
to detect file types, but .* does not match newline characters by
default in Python regex. This causes the regex to fail, returning
FileType.OTHER, which triggers:
pythonraise RuntimeError("This type of file has not been supported
yet!")
As a result, all documents synced via the MySQL/PostgreSQL connector are
silently discarded. The sync logs report success (e.g., "399 docs
synchronized"), but zero documents actually appear in the dataset. This
is the root cause of issue #13001.
Root cause trace:

rdbms_connector.py → _row_to_document() sets semantic_identifier from
raw content (may contain \n)
connector_service.py → duplicate_and_parse() uses semantic_identifier as
the filename
file_service.py → upload_document() calls filename_type(filename)
file_utils.py → filename_type() regex .*\.txt$ fails on newlines →
returns FileType.OTHER
upload_document() raises "This type of file has not been supported yet!"

Fix: Sanitize the semantic_identifier in _row_to_document() by replacing
newlines and carriage returns with spaces before truncating to 100
characters.
Relates to: #13001, #12817
Type of change

 Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-02-25 12:55:04 +08:00
Yongteng Lei
72b89304c1 Fix: LFI vulnerability in document parsing API (#13196)
### What problem does this PR solve?

Fix LFI vulnerability in document parsing API.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 09:47:39 +08:00
PandaMan
f4cbdc3a3b fix(api): MinIO health check use dynamic scheme and verify (Closes #13159 and #13158) (#13197)
## Summary

Fixes MinIO SSL/TLS support in two places: the MinIO **client**
connection and the **health check** used by the Admin/Service Health
dashboard. Both now respect the `secure` and `verify` settings from the
MinIO configuration.

Closes #13158
Closes #13159

---

## Problem

**#13158 – MinIO client:** The client in `rag/utils/minio_conn.py` was
hardcoded with `secure=False`, so RAGFlow could not connect to MinIO
over HTTPS even when `secure: true` was set in config. There was also no
way to disable certificate verification for self-signed certs.

**#13159 – MinIO health check:** In `api/utils/health_utils.py`, the
MinIO liveness check always used `http://` for the health URL. When
MinIO was configured with SSL, the health check failed and the dashboard
showed "timeout" even though MinIO was reachable over HTTPS.

---

## Solution

### MinIO client (`rag/utils/minio_conn.py`)

- Read `MINIO.secure` (default `false`) and pass it into the `Minio()`
constructor so HTTPS is used when configured.
- Add `_build_minio_http_client()` that reads `MINIO.verify` (default
`true`). When `verify` is false, return an `urllib3.PoolManager` with
`cert_reqs=ssl.CERT_NONE` and pass it as `http_client` to `Minio()` so
self-signed certificates are accepted.
- Support string values for `secure` and `verify` (e.g. `"true"`,
`"false"`).

### MinIO health check (`api/utils/health_utils.py`)

- Add `_minio_scheme_and_verify()` to derive URL scheme (http/https) and
the `verify` flag from `MINIO.secure` and `MINIO.verify`.
- Update `check_minio_alive()` to use the correct scheme, pass `verify`
into `requests.get(..., verify=verify)`, and use `timeout=10`.

### Config template (`docker/service_conf.yaml.template`)

- Add commented optional MinIO keys `secure` and `verify` (and env vars
`MINIO_SECURE`, `MINIO_VERIFY`) so deployers know they can enable HTTPS
and optional cert verification.

### Tests

- **`test/unit_test/utils/test_health_utils_minio.py`** – Tests for
`_minio_scheme_and_verify()` and `check_minio_alive()` (scheme, verify,
status codes, timeout, errors).
- **`test/unit_test/utils/test_minio_conn_ssl.py`** – Tests for
`_build_minio_http_client()` (verify true/false/missing, string values,
`CERT_NONE` when verify is false).

---

## Testing

- Unit tests added/updated as above; run with the project's test runner.
- Manually: configure MinIO with HTTPS and `secure: true` (and
optionally `verify: false` for self-signed); confirm client operations
work and the Service Health dashboard shows MinIO as alive instead of
timeout.
2026-02-25 09:47:12 +08:00
Yongteng Lei
c292d617ca Fix: stored XSS via HTML File upload and inline Rendering in file get (#13202)
### What problem does this PR solve?

Fix stored XSS via HTML file upload and inline rendering in
/v1/file/get/<id>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-25 09:46:48 +08:00
ksufer
5a8fa7cf31 Fix #13119: Use email.utils to fix IMAP parsing for names with commas (#13120)
## Type of Change
- [x] Bug fix

## Description
Closes #13119

The current IMAP connector uses `split(',')` to parse email headers,
which crashes when a sender's display name contains a comma inside
quotes (e.g., `"Doe, John" <john@example.com>`).

This PR replaces the manual string splitting with Python's standard
`email.utils.getaddresses`. This correctly handles RFC 5322 quoted
strings and prevents the `RuntimeError: Expected a singular address`.

## Checklist
- [x] I have checked the code and it works as expected.

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-24 19:18:55 +08:00
as-ondewo
0a7c520579 Fix: empty response from OpenAI chat completion endpoint (#13166)
### What problem does this PR solve?

When using a chat assistant that has a hardcoded `empty_response`, that
response was not returned correctly in streaming mode when no
information is found in the knowledge base. In this case only one
response with `"content": null` was yielded. If `"references": true`,
then the `empty_response` is still put into the `final_content` so there
is technically some content returned, but when `"references": false` no
content at all is returned.

I update the OpenAI chat completion endpoint to yield an additional
response with the `empty_response` in the content.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:18:12 +08:00
Magicbook1108
5de92e57d3 Fix: 'None None' in log (#13192)
### What problem does this PR solve?

Fix: 'None None' in log

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:15:20 +08:00
Magicbook1108
46dec98f52 Fix: Chat/Agent embedded page (#13199)
### What problem does this PR solve?

Fix: Chat/Agent embedded page #13190

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 19:14:24 +08:00
tuandang-diag
d89ad8b79d fix: handle null response in LLM and improve JSON parsing in agent (#13187)
Fixes AttributeError in _remove_reasoning_content() when LLM returns
None, and improves JSON parsing regex for markdown code fences in
agent_with_tools.py
2026-02-24 13:15:09 +08:00
Lynn
67befc9119 Fix: add back MCP tool custom header (#13188)
### What problem does this PR solve?

Add back custom header when use MCP.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 13:14:21 +08:00
chanx
7db2fb200c Fix: Metadata mult-selected display error (#13189)
### What problem does this PR solve?

Fix: Metadata mult-selected display error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 13:10:32 +08:00
PandaMan
f462a9d85a fix(web): prevent LaTeX from being cut at \right] or \big) in agent o… (#13155)
### What problem does this PR solve?

- Use negative lookbehind (?<![a-zA-Z]) so \] and \) inside commands
(e.g. \right], \big)) are not treated as block/inline delimiters
- Use greedy matching to capture up to the last valid delimiter, fixing
truncated formulas (e.g. C_{seq}(y|x) = \frac{1}{|y|} ...)
- Add unit tests for preprocessLaTeX

Closes #13134


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 12:30:06 +08:00
Bradley Boveinis
3280772934 fix(helm): exclude password keys from env range loop to prevent duplicate YAML keys (#13136)
## Summary

- Fix duplicate YAML mapping keys in `helm/templates/env.yaml` that
cause deployment failures with strict YAML parsers

## Problem

The `range` loop in `env.yaml` iterates over all `.Values.env` keys and
emits them into a Secret. The exclusion filter skips host/port/user
keys, but does **not** skip password keys (`MYSQL_PASSWORD`,
`REDIS_PASSWORD`, `MINIO_PASSWORD`, `ELASTIC_PASSWORD`,
`OPENSEARCH_PASSWORD`). These same keys are then explicitly defined
again later in the template, producing duplicate YAML mapping keys.

Go's `yaml.v3` (used by Flux's helm-controller for post-rendering)
rejects duplicate keys per the YAML spec:

```
Helm install failed: yaml: unmarshal errors:
  mapping key "MINIO_PASSWORD" already defined
  mapping key "MYSQL_PASSWORD" already defined
  mapping key "REDIS_PASSWORD" already defined
```

Plain `helm install` does not surface this because Helm's internal
parser (`yaml.v2`) silently accepts duplicate keys (last value wins).

## Fix

Add password keys to the exclusion filter on line 12 so they are only
emitted by their explicit definitions later in the template.

Note: `MINIO_ROOT_USER` is intentionally **not** excluded — it is only
emitted by the range loop and has no explicit definition elsewhere.
Excluding it causes MinIO to crash with `Missing credential environment
variable, "MINIO_ROOT_USER"`.

## Test plan

- [ ] Deploy with Flux helm-controller (uses yaml.v3) — no duplicate key
errors
- [ ] Verify all passwords are present in the rendered Secret
- [ ] Verify `MINIO_ROOT_USER` is present in the rendered Secret
- [ ] Test with `DOC_ENGINE=elasticsearch` (ELASTIC_PASSWORD)
- [ ] Test with `DOC_ENGINE=opensearch` (OPENSEARCH_PASSWORD)

Fixes #13135
2026-02-24 11:09:31 +08:00
as-ondewo
91d1a81937 fix: error during admin tenant creation when using Postgres (#13164)
### What problem does this PR solve?

This fixes the bug described in #13130. When starting RAGFlow with
Postgres the admin tenant create failed because the rerank model was not
set.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 10:57:31 +08:00
chanx
59e9e77061 fix: Add admin proxy (#13186)
### What problem does this PR solve?

fix: Add admin proxy

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 10:29:58 +08:00
Magicbook1108
98e1d5aa5c Refact: switch from google-generativeai to google-genai (#13140)
### What problem does this PR solve?

Refact: switch from oogle-generativeai to google-genai  #13132
Refact: commnet out unused pywencai.

### Type of change

- [x] Refactoring
2026-02-24 10:28:33 +08:00
zagnaan
45aa3a0e89 fix(compose): use official opensearch image instead of hub.icert.top mirror (#13131)
### What problem does this PR solve?

The Docker Compose configuration was using hub.icert.top as the registry
for the OpenSearch image. That registry is not reachable in our
environment, which causes podman pull and docker compose pull to fail
with a connection refused error. As a result, the application cannot
start because the OpenSearch image cannot be downloaded.

This PR updates the image reference to use the official Docker Hub image
(opensearchproject/opensearch:2.19.1) instead of the hub.icert.top
mirror. After this change, the image pulls successfully and the services
start as expected.


![photo_2026-02-12_15-11-56](https://github.com/user-attachments/assets/6db736a5-b701-450f-96c1-9c23f092c3ab)


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Co-authored-by: Shynggys Samarkhanov <shynggys.samarkhanov@nixs.com>
2026-02-24 09:50:02 +08:00
Trifon
ce71d87867 Add Bulgarian language support (#13147)
### What problem does this PR solve?

RAGFlow supports 12 UI languages but does not include Bulgarian. This PR
adds Bulgarian (`bg` / `Български`) as the 13th supported language,
covering the full UI translation (2001 keys across all 26 sections) and
OCR/PDF parser language mapping.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Changes

- **`web/src/constants/common.ts`** — Registered Bulgarian in all 5
language data structures (`LanguageList`, `LanguageMap`,
`LanguageAbbreviation` enum, `LanguageAbbreviationMap`,
`LanguageTranslationMap`)
- **`web/src/locales/config.ts`** — Added lazy-loading dynamic import
for the `bg` locale
- **`web/src/locales/bg.ts`** *(new)* — Full Bulgarian translation file
with all 26 sections, matching the English source (`en.ts`). All
interpolation placeholders, HTML tags, and technical terms are preserved
as-is
- **`deepdoc/parser/mineru_parser.py`** — Mapped `'Bulgarian'` to
`'cyrillic'` in `LANGUAGE_TO_MINERU_MAP` for OCR/PDF parser support

### How it works

The language selector automatically picks up the new entry. When a user
selects "Български", the translation bundle is lazy-loaded on demand.
The preference is persisted to the database and localStorage across
sessions.
2026-02-14 16:51:29 +08:00
chanx
f612d2254d Refactor: i18n language pack for on-demand import (#13139)
### What problem does this PR solve?

Refactor:   i18n language pack for on-demand import

### Type of change

- [x] Refactoring

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-13 18:42:48 +08:00
chanx
f2a1d59c36 Refactor: Remove ant design component (#13143)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring
2026-02-13 18:40:41 +08:00
writinwaters
bc9ed24a85 Docs: Updated v0.24.0 release notes. (#13129)
### What problem does this PR solve?

Added more details to v0.24.0 release notes.

### Type of change

- [x] Documentation Update
2026-02-12 20:14:05 +08:00
Levi
6d6c54db19 fix(metadata): handle unhashable list values in metadata split (#13116)
### What problem does this PR solve?

This PR fixes missing metadata on documents synced from the Moodle
connector, especially for **Book** modules.

Background:
- Moodle Book metadata includes fields like `chapters`, which is a
`list[dict]`.
- During metadata normalization in
`DocMetadataService._split_combined_values`, list deduplication used
`dict.fromkeys(...)`.
- `dict.fromkeys(...)` fails for unhashable values (like `dict`),
causing metadata update to fail.
- Result: documents were imported, but metadata was not saved for
affected module types (notably Books).

What this PR changes:
- Replaces hash-based list deduplication with `dedupe_list(...)`, which
safely handles unhashable list items while preserving order.
- This allows Book metadata (and other complex list metadata) to be
persisted correctly.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

Contribution during my time at RAGcon GmbH.
2026-02-12 19:48:51 +08:00
chanx
b922a5cbdf Fix: replace session page icons and fix nested list search functionality in filters (#13127)
### What problem does this PR solve?

Fix: replace session page icons and fix nested list search functionality
in filters

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 19:48:35 +08:00
Ahmad Intisar
5885f150ab fix: register WebDAVConnector in data_source __init__.py (#13121)
What problem does this PR solve?
The sync_data_source.py module imports WebDAVConnector from
common.data_source, but WebDAVConnector was never registered in the
package's __init__.py. This causes an ImportError at startup, crashing
the data sync service:
ImportError: cannot import name 'WebDAVConnector' from
'common.data_source'
The webdav_connector.py file already exists in the common/data_source/
directory — it just wasn't exported. This PR adds the import and
registers it in __all__.
Type of change

 Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
2026-02-12 16:05:58 +08:00
Magicbook1108
e89fd686e2 Improve: optimize file name (with path) in box container. (#13124)
### What problem does this PR solve?

Refact: optimize file name (with path) in box container. 

### Type of change

- [x] Performance Improvement

<img width="2357" height="1258" alt="image"
src="https://github.com/user-attachments/assets/f4c5c90b-d885-4514-b7bc-f17ab62b045f"
/>
2026-02-12 15:40:55 +08:00
疯癫
e72291bc9a Fix the bug where the mcp service tools/list does not return knowledge base IDs information. (#13123)
Fix the issue where the server-side parameter validation fails when the
id parameter is None in the asynchronous list_datasets method.

### What problem does this PR solve?

Fix the issue where the server-side parameter validation fails when the
id parameter is None in the asynchronous list_datasets method.

### Type of change

- [√ ] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 15:40:15 +08:00
Lynn
6e7bcf58bc Refactor: split message apis to gateway and service (#13126)
### What problem does this PR solve?

Split message apis to gateway and service

### Type of change

- [x] Refactoring
2026-02-12 14:43:52 +08:00
chanx
7210178620 Fix: Bugs fixed (#13109) (#13122)
### What problem does this PR solve?

Fix: Bugs fixed (#13109)
- chat pdf preview error
- data source add box error
- change route next-chat -> chat , next-search->search ...

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 13:42:12 +08:00
Liu An
65ebc06956 Refa: test file location for better organization (#13107)
### What problem does this PR solve?

Renamed test/unit/test_delete_query_construction.py to
test/unit_test/common/test_delete_query_construction.py to align with
the project's directory structure and improve test categorization.

### Type of change

- [x] Refactoring
2026-02-12 10:15:09 +08:00
Lynn
30d5fc1a07 Refactor: split memory API into gateway and service layers (#13111)
### What problem does this PR solve?

Decouple the memory API into a gateway layer (for routing/param parse)
and a service layer (for business logic).

### Type of change

- [x] Refactoring
2026-02-12 10:11:50 +08:00
Levi
4b50b8c579 Fix: persist SSO auth token on root route loader (#12784)
### What problem does this PR solve?

This PR fixes SSO/OIDC login persistence after the Vite migration
#12568. Because wrappers are ignored by React Router, the OAuth callback
never stored the auth token in localStorage, causing auth to only work
while ?auth= stayed in the URL. We move that logic into a route loader
and remove the Bearer prefix for the signed token so the backend accepts
it.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Contribution during my time at RAGcon GmbH.

Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com>
2026-02-12 10:09:35 +08:00
TheoG
67937a668e Fix graphrag extraction (#13113)
### What problem does this PR solve?

Fix error when extracting the graph.
A string is expected, but a tuple was provided.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-11 20:11:56 +08:00
writinwaters
57dc25f150 Docs: Updated v0.24.0 release notes (#13115)
### What problem does this PR solve?

Updated v0.24.0 release notes.

### Type of change

- [x] Documentation Update
2026-02-11 18:08:56 +08:00
writinwaters
a9272c26eb Docs: Updated sandbox reference (#13114)
### What problem does this PR solve?

Updated sandbox reference.

### Type of change

- [x] Documentation Update
2026-02-11 17:58:08 +08:00
Kevin Hu
88920d23e6 Refa: Change aliyun repo. (#13103)
### Type of change

- [x] Refactoring
2026-02-11 11:21:03 +08:00
Jim Smith
7029b8ca81 Fix: Make time_utils tests timezone-independent (#13100)
## Summary
- Replace hardcoded CST (UTC+8) expected values in `test_time_utils.py`
with dynamically computed local-time expectations using
`time.localtime()` and `time.mktime()`
- Tests previously failed in any timezone other than UTC+8; they now
pass regardless of the system's local timezone

## Test plan
- [x] `uv run pytest test/unit_test/ -v` — 317 passed, 25 skipped

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Jim Smith <jhsmith0@me.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:51:53 +08:00
Ahmad Intisar
99ed8e759d Fix: Correct Gemini embedding model name in llm_factories.json (#13051)
## Problem
   RAGFlow was using incorrect model names for Google Gemini embeddings:
   - `embedding-001` (missing `gemini-` prefix)
   - `text-embedding-004` (OpenAI model name, not Gemini)
   
   This caused API errors when users tried to use Gemini embeddings.
   
   ## Solution
- Updated `conf/llm_factories.json` to use the correct model name:
`gemini-embedding-001`
   - Removed the incorrect `text-embedding-004` entry
- Added volume mount in `docker-compose.yml` to ensure config changes
persist
   
   ## Testing
Tested with a valid Gemini API key and confirmed embeddings now work
correctly.

## Changes
- Modified `conf/llm_factories.json`
- Modified `docker/docker-compose.yml`

---------

Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-11 09:49:48 +08:00
Magicbook1108
109441628b Fix: upload image files (#13071)
### What problem does this PR solve?

Fix: upload image files

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-11 09:47:33 +08:00
writinwaters
630f05b8a1 Docs: Added v0.24.0 release notes (#13096)
### What problem does this PR solve?

Added v0.24.0 release notes.

### Type of change


- [x] Documentation Update
2026-02-10 17:38:27 +08:00
1968 changed files with 368796 additions and 58248 deletions

192
.agents/rules/named.md Normal file
View File

@@ -0,0 +1,192 @@
# Go Naming Best Practices
## 1. Package Naming
- **All lowercase, no underscores**: `package user`, not `package userService` or `package user_service`
- **Short and meaningful**: `package http`, `package json`, `package dao`
- **Avoid plurals**: `package user` not `package users`
- **Avoid generic names**: Avoid `package util`, `package common`, `package base`
```go
// Recommended
package user
package handler
package service
// Not recommended
package UserService
package user_service
package utils
```
## 2. File Naming
- **All lowercase, underscore separated**: `user_handler.go`, `user_service.go`
- **Test files**: `user_handler_test.go`
- **Platform-specific**: `user_linux.go`, `user_windows.go`
```
user/
├── user_handler.go
├── user_service.go
├── user_dao.go
└── user_test.go
```
## 3. Directory Naming
- **All lowercase, no underscores or hyphens**: `internal/`, `pkg/`, `cmd/`
- **Short and descriptive**: `handler/`, `service/`, `dao/`
```
project/
├── cmd/ # Main entry point
│ └── server_main.go
├── internal/ # Private code
│ ├── handler/
│ ├── service/
│ ├── dao/
│ ├── model/
│ └── middleware/
├── pkg/ # Public code
└── api/ # API definitions
```
## 4. Interface Naming
- **Single-method interfaces end with "-er"**: `Reader`, `Writer`, `Handler`
- **Verb form**: `Reader`, `Executor`, `Validator`
```go
// Recommended
type Reader interface {
Read(p []byte) (n int, err error)
}
type UserService interface {
Register(req *RegisterRequest) (*User, error)
Login(req *LoginRequest) (*User, error)
}
// Not recommended
type UserInterface interface {}
type IUserService interface {}
```
## 5. Struct Naming
- **CamelCase**: `UserService`, `UserHandler`
- **Avoid redundant prefixes**: `User` not `UserModel`
```go
// Recommended
type UserService struct {}
type UserHandler struct {}
type RegisterRequest struct {}
// Not recommended
type user_service struct {}
type SUserService struct {}
type UserModel struct {}
```
## 6. Method/Function Naming
- **CamelCase**
- **Start with verb**: `GetUser`, `CreateUser`, `DeleteUser`
- **Boolean returns use Is/Has/Can prefix**: `IsValid`, `HasPermission`
```go
// Recommended
func (s *UserService) Register(req *RegisterRequest) (*User, error)
func (s *UserService) GetUserByID(id uint) (*User, error)
func (s *UserService) IsEmailExists(email string) bool
// Not recommended
func (s *UserService) register_user()
func (s *UserService) get_user_by_id()
func (s *UserService) CheckEmailExists() // Should use Is/Has
```
## 7. Constant Naming
- **CamelCase**: `const MaxRetryCount = 3`
- **Enum constants**: `const StatusActive = "active"`
```go
// Recommended
const (
StatusActive = "1"
StatusInactive = "0"
MaxRetryCount = 3
)
// Not recommended
const (
STATUS_ACTIVE = "1" // Not all uppercase
status_active = "1" // Not all lowercase
)
```
## 8. Error Variable Naming
- **Start with "Err"**: `ErrNotFound`, `ErrInvalidInput`
```go
// Recommended
var (
ErrNotFound = errors.New("not found")
ErrInvalidInput = errors.New("invalid input")
ErrUnauthorized = errors.New("unauthorized")
)
```
## 9. Acronyms Keep Consistent Case
```go
// Recommended
type HTTPHandler struct {}
var URL string
func GetHTTPClient() {}
func ParseJSON() {}
// Not recommended
type HttpHandler struct {}
var Url string
func GetHttpClient() {}
```
## 10. Project Structure Naming
```
project-name/
├── cmd/ # Main programs
│ └── app_name/
│ └── main.go
├── internal/ # Private code
│ ├── handler/ # HTTP handlers
│ ├── service/ # Business logic
│ ├── repository/ # Data access
│ ├── model/ # Data models
│ └── config/ # Configuration
├── pkg/ # Public code
├── api/ # API definitions
├── configs/ # Config files
├── scripts/ # Scripts
├── docs/ # Documentation
├── go.mod
└── go.sum
```
## Summary Table
| Type | Rule | Example |
| -------------- | ----------------------------------- | ------------------- |
| Package | All lowercase, no underscores | `package user` |
| File | All lowercase, underscore separated | `user_service.go` |
| Directory | All lowercase, no separators | `internal/handler/` |
| Struct | CamelCase, capitalized first letter | `UserService` |
| Interface | CamelCase, -er suffix | `Reader`, `Writer` |
| Method | CamelCase, verb prefix | `GetUserByID` |
| Constant | CamelCase | `MaxRetryCount` |
| Error Variable | Err prefix | `ErrNotFound` |

View File

@@ -0,0 +1,6 @@
---
name: go-naming
description: Go naming conventions and best practices. Use this skill when working with Go code and need to name packages, files, directories, structs, interfaces, functions, variables, or constants. Provides comprehensive naming guidelines following Go community standards.
---
Strictly follow the naming conventions in [rules/named.md](rules/named.md)

View File

@@ -23,7 +23,7 @@ concurrency:
jobs:
release:
runs-on: [ "self-hosted", "ragflow-test" ]
runs-on: [ "self-hosted", "ragflow-release" ]
steps:
- name: Ensure workspace ownership
run: echo "chown -R ${USER} ${GITHUB_WORKSPACE}" && sudo chown -R ${USER} ${GITHUB_WORKSPACE}

View File

@@ -129,20 +129,24 @@ jobs:
fi
fi
- name: Run unit test
- name: Build ragflow go server
run: |
uv sync --python 3.12 --group test --frozen
source .venv/bin/activate
which pytest || echo "pytest not in PATH"
echo "Start to run unit test"
python3 run_tests.py
BUILDER_CONTAINER=ragflow_build_$(od -An -N4 -tx4 /dev/urandom | tr -d ' ')
echo "BUILDER_CONTAINER=${BUILDER_CONTAINER}" >> ${GITHUB_ENV}
TZ=${TZ:-$(readlink -f /etc/localtime | awk -F '/zoneinfo/' '{print $2}')}
sudo docker run --privileged -d --name ${BUILDER_CONTAINER} -e TZ=${TZ} -e UV_INDEX=https://mirrors.aliyun.com/pypi/simple -v ${PWD}:/ragflow -v ${PWD}/internal/cpp/resource:/usr/share/infinity/resource infiniflow/infinity_builder:ubuntu22_clang20
sudo docker exec ${BUILDER_CONTAINER} bash -c "git config --global safe.directory \"*\" && cd /ragflow && ./build.sh --cpp"
./build.sh --go
if [[ -n "${BUILDER_CONTAINER}" ]]; then
sudo docker rm -f -v "${BUILDER_CONTAINER}"
fi
- name: Build ragflow:nightly
run: |
RUNNER_WORKSPACE_PREFIX=${RUNNER_WORKSPACE_PREFIX:-${HOME}}
RAGFLOW_IMAGE=infiniflow/ragflow:${GITHUB_RUN_ID}
echo "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> ${GITHUB_ENV}
sudo docker pull ubuntu:22.04
sudo docker pull ubuntu:24.04
sudo DOCKER_BUILDKIT=1 docker build --build-arg NEED_MIRROR=1 --build-arg HTTPS_PROXY=${HTTPS_PROXY} --build-arg HTTP_PROXY=${HTTP_PROXY} -f Dockerfile -t ${RAGFLOW_IMAGE} .
if [[ ${GITHUB_EVENT_NAME} == "schedule" ]]; then
export HTTP_API_TEST_LEVEL=p3
@@ -152,88 +156,338 @@ jobs:
echo "HTTP_API_TEST_LEVEL=${HTTP_API_TEST_LEVEL}" >> ${GITHUB_ENV}
echo "RAGFLOW_CONTAINER=${GITHUB_RUN_ID}-ragflow-cpu-1" >> ${GITHUB_ENV}
- name: Start ragflow:nightly
- name: Run unit test
run: |
uv sync --python 3.13 --group test --frozen
source .venv/bin/activate
which pytest || echo "pytest not in PATH"
echo "Start to run unit test"
python3 run_tests.py -i
- name: Prepare function test environment
working-directory: docker
run: |
# Determine runner number (default to 1 if not found)
RUNNER_NUM=$(sudo docker inspect $(hostname) --format '{{index .Config.Labels "com.docker.compose.container-number"}}' 2>/dev/null || true)
RUNNER_NUM=${RUNNER_NUM:-1}
RUNNER_NUM=$(sudo docker inspect $(hostname) --format '{{index .Config.Labels "com.docker.compose.container-number"}}' 2>/dev/null || true)
RUNNER_NUM=${RUNNER_NUM:-1}
# Compute port numbers using bash arithmetic
ES_PORT=$((1200 + RUNNER_NUM * 10))
OS_PORT=$((1201 + RUNNER_NUM * 10))
INFINITY_THRIFT_PORT=$((23817 + RUNNER_NUM * 10))
INFINITY_HTTP_PORT=$((23820 + RUNNER_NUM * 10))
INFINITY_PSQL_PORT=$((5432 + RUNNER_NUM * 10))
EXPOSE_MYSQL_PORT=$((5455 + RUNNER_NUM * 10))
MINIO_PORT=$((9000 + RUNNER_NUM * 10))
MINIO_CONSOLE_PORT=$((9001 + RUNNER_NUM * 10))
REDIS_PORT=$((6379 + RUNNER_NUM * 10))
TEI_PORT=$((6380 + RUNNER_NUM * 10))
KIBANA_PORT=$((6601 + RUNNER_NUM * 10))
SVR_HTTP_PORT=$((9380 + RUNNER_NUM * 10))
ADMIN_SVR_HTTP_PORT=$((9381 + RUNNER_NUM * 10))
SVR_MCP_PORT=$((9382 + RUNNER_NUM * 10))
SANDBOX_EXECUTOR_MANAGER_PORT=$((9385 + RUNNER_NUM * 10))
SVR_WEB_HTTP_PORT=$((80 + RUNNER_NUM * 10))
SVR_WEB_HTTPS_PORT=$((443 + RUNNER_NUM * 10))
# Per-runner base plus per-workflow-run offset avoids port clashes when
# multiple CI jobs share the same self-hosted runner concurrently.
PORT_OFFSET=$(( (GITHUB_RUN_ID % 400) + RUNNER_NUM * 10 ))
ES_PORT=$((1200 + PORT_OFFSET))
OS_PORT=$((1201 + PORT_OFFSET))
INFINITY_THRIFT_PORT=$((23817 + PORT_OFFSET))
INFINITY_HTTP_PORT=$((23820 + PORT_OFFSET))
INFINITY_PSQL_PORT=$((5432 + PORT_OFFSET))
EXPOSE_MYSQL_PORT=$((5455 + PORT_OFFSET))
MINIO_PORT=$((9000 + PORT_OFFSET))
MINIO_CONSOLE_PORT=$((9001 + PORT_OFFSET))
REDIS_PORT=$((6379 + PORT_OFFSET))
TEI_PORT=$((6380 + PORT_OFFSET))
KIBANA_PORT=$((6601 + PORT_OFFSET))
SVR_HTTP_PORT=$((9380 + PORT_OFFSET))
ADMIN_SVR_HTTP_PORT=$((9381 + PORT_OFFSET))
SVR_MCP_PORT=$((9382 + PORT_OFFSET))
GO_HTTP_PORT=$((9384 + PORT_OFFSET))
GO_ADMIN_PORT=$((9383 + PORT_OFFSET))
SANDBOX_EXECUTOR_MANAGER_PORT=$((9385 + PORT_OFFSET))
SVR_WEB_HTTP_PORT=$((80 + PORT_OFFSET))
SVR_WEB_HTTPS_PORT=$((443 + PORT_OFFSET))
# Persist computed ports into docker/.env so docker-compose uses the correct host bindings
echo "" >> docker/.env
echo -e "ES_PORT=${ES_PORT}" >> docker/.env
echo -e "OS_PORT=${OS_PORT}" >> docker/.env
echo -e "INFINITY_THRIFT_PORT=${INFINITY_THRIFT_PORT}" >> docker/.env
echo -e "INFINITY_HTTP_PORT=${INFINITY_HTTP_PORT}" >> docker/.env
echo -e "INFINITY_PSQL_PORT=${INFINITY_PSQL_PORT}" >> docker/.env
echo -e "EXPOSE_MYSQL_PORT=${EXPOSE_MYSQL_PORT}" >> docker/.env
echo -e "MINIO_PORT=${MINIO_PORT}" >> docker/.env
echo -e "MINIO_CONSOLE_PORT=${MINIO_CONSOLE_PORT}" >> docker/.env
echo -e "REDIS_PORT=${REDIS_PORT}" >> docker/.env
echo -e "TEI_PORT=${TEI_PORT}" >> docker/.env
echo -e "KIBANA_PORT=${KIBANA_PORT}" >> docker/.env
echo -e "SVR_HTTP_PORT=${SVR_HTTP_PORT}" >> docker/.env
echo -e "ADMIN_SVR_HTTP_PORT=${ADMIN_SVR_HTTP_PORT}" >> docker/.env
echo -e "SVR_MCP_PORT=${SVR_MCP_PORT}" >> docker/.env
echo -e "SANDBOX_EXECUTOR_MANAGER_PORT=${SANDBOX_EXECUTOR_MANAGER_PORT}" >> docker/.env
echo -e "SVR_WEB_HTTP_PORT=${SVR_WEB_HTTP_PORT}" >> docker/.env
echo -e "SVR_WEB_HTTPS_PORT=${SVR_WEB_HTTPS_PORT}" >> docker/.env
# Persist computed ports into .env so docker-compose uses the correct host bindings
echo "" >> .env
echo -e "ES_PORT=${ES_PORT}" >> .env
echo -e "OS_PORT=${OS_PORT}" >> .env
echo -e "INFINITY_THRIFT_PORT=${INFINITY_THRIFT_PORT}" >> .env
echo -e "INFINITY_HTTP_PORT=${INFINITY_HTTP_PORT}" >> .env
echo -e "INFINITY_PSQL_PORT=${INFINITY_PSQL_PORT}" >> .env
echo -e "EXPOSE_MYSQL_PORT=${EXPOSE_MYSQL_PORT}" >> .env
echo -e "MINIO_PORT=${MINIO_PORT}" >> .env
echo -e "MINIO_CONSOLE_PORT=${MINIO_CONSOLE_PORT}" >> .env
echo -e "REDIS_PORT=${REDIS_PORT}" >> .env
echo -e "TEI_PORT=${TEI_PORT}" >> .env
echo -e "KIBANA_PORT=${KIBANA_PORT}" >> .env
echo -e "SVR_HTTP_PORT=${SVR_HTTP_PORT}" >> .env
echo -e "ADMIN_SVR_HTTP_PORT=${ADMIN_SVR_HTTP_PORT}" >> .env
echo -e "SVR_MCP_PORT=${SVR_MCP_PORT}" >> .env
echo -e "GO_HTTP_PORT=${GO_HTTP_PORT}" >> .env
echo -e "GO_ADMIN_PORT=${GO_ADMIN_PORT}" >> .env
echo -e "SANDBOX_EXECUTOR_MANAGER_PORT=${SANDBOX_EXECUTOR_MANAGER_PORT}" >> .env
echo -e "SVR_WEB_HTTP_PORT=${SVR_WEB_HTTP_PORT}" >> .env
echo -e "SVR_WEB_HTTPS_PORT=${SVR_WEB_HTTPS_PORT}" >> .env
echo -e "COMPOSE_PROFILES=\${COMPOSE_PROFILES},tei-cpu" >> .env
echo -e "TEI_MODEL=BAAI/bge-small-en-v1.5" >> .env
echo -e "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> .env
echo "HOST_ADDRESS=http://host.docker.internal:${SVR_HTTP_PORT}" >> ${GITHUB_ENV}
echo -e "COMPOSE_PROFILES=\${COMPOSE_PROFILES},tei-cpu" >> docker/.env
echo -e "TEI_MODEL=BAAI/bge-small-en-v1.5" >> docker/.env
echo -e "RAGFLOW_IMAGE=${RAGFLOW_IMAGE}" >> docker/.env
echo "HOST_ADDRESS=http://host.docker.internal:${SVR_HTTP_PORT}" >> ${GITHUB_ENV}
# Patch entrypoint.sh for coverage
sed -i '/"\$PY" api\/ragflow_server.py \${INIT_SUPERUSER_ARGS} &/c\ echo "Ensuring coverage is installed..."\n "$PY" -m pip install coverage -i https://mirrors.aliyun.com/pypi/simple\n export COVERAGE_FILE=/ragflow/logs/.coverage\n echo "Starting ragflow_server with coverage..."\n "$PY" -m coverage run --source=./api/apps --omit="*/tests/*,*/migrations/*" -a api/ragflow_server.py ${INIT_SUPERUSER_ARGS} &' ./entrypoint.sh
cd ..
uv sync --python 3.13 --group test --frozen && uv pip install -e sdk/python
# Patch entrypoint.sh for coverage
sed -i '/"\$PY" api\/ragflow_server.py \${INIT_SUPERUSER_ARGS} &/c\ echo "Ensuring coverage is installed..."\n "$PY" -m pip install coverage\n export COVERAGE_FILE=/ragflow/logs/.coverage\n echo "Starting ragflow_server with coverage..."\n "$PY" -m coverage run --source=./api/apps --omit="*/tests/*,*/migrations/*" -a api/ragflow_server.py ${INIT_SUPERUSER_ARGS} &' docker/entrypoint.sh
- name: Start ragflow:nightly for Infinity
run: |
sed -i 's/^DOC_ENGINE=.*$/DOC_ENGINE=infinity/' docker/.env
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
- name: Run sdk tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
echo "Start to run test sdk on Infinity"
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} --junitxml=pytest-infinity-sdk.xml --cov=sdk/python/ragflow_sdk --cov-branch --cov-report=xml:coverage-infinity-sdk.xml test/testcases/test_sdk_api 2>&1 | tee infinity_sdk_test.log
- name: Run New RESTFUL api tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/restful_api 2>&1 | tee infinity_restful_api_test.log
- name: RAGFlow CLI retrieval test Infinity
env:
PYTHONPATH: ${{ github.workspace }}
run: |
set -euo pipefail
source .venv/bin/activate
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
EMAIL="ci-${GITHUB_RUN_ID}@example.com"
PASS="ci-pass-${GITHUB_RUN_ID}"
DATASET="ci_dataset_${GITHUB_RUN_ID}"
CLI="python admin/client/ragflow_cli.py"
LOG_FILE="infinity_cli_test.log"
: > "${LOG_FILE}"
ERROR_RE='Traceback|ModuleNotFoundError|ImportError|Parse error|Bad response|Fail to|code:\\s*[1-9]'
run_cli() {
local logfile="$1"
shift
local allow_re=""
if [[ "${1:-}" == "--allow" ]]; then
allow_re="$2"
shift 2
fi
local cmd_display="$*"
echo "===== $(date -u +\"%Y-%m-%dT%H:%M:%SZ\") CMD: ${cmd_display} =====" | tee -a "${logfile}"
local tmp_log
tmp_log="$(mktemp)"
set +e
timeout 500s "$@" 2>&1 | tee "${tmp_log}"
local status=${PIPESTATUS[0]}
set -e
cat "${tmp_log}" >> "${logfile}"
if grep -qiE "${ERROR_RE}" "${tmp_log}"; then
if [[ -n "${allow_re}" ]] && grep -qiE "${allow_re}" "${tmp_log}"; then
echo "Allowed CLI error markers in ${logfile}"
rm -f "${tmp_log}"
return 0
fi
echo "Detected CLI error markers in ${logfile}"
rm -f "${tmp_log}"
exit 1
fi
rm -f "${tmp_log}"
return ${status}
}
set -a
source docker/.env
set +a
HOST_ADDRESS="http://host.docker.internal:${SVR_HTTP_PORT}"
USER_HOST="$(echo "${HOST_ADDRESS}" | sed -E 's#^https?://([^:/]+).*#\1#')"
USER_PORT="${SVR_HTTP_PORT}"
ADMIN_HOST="${USER_HOST}"
ADMIN_PORT="${ADMIN_SVR_HTTP_PORT}"
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
admin_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "ping"; then
admin_ready=1
break
fi
sleep 1
done
if [[ "${admin_ready}" -ne 1 ]]; then
echo "Admin service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "show version"
ALLOW_USER_EXISTS_RE='already exists|already exist|duplicate|already.*registered|exist(s)?'
run_cli "${LOG_FILE}" --allow "${ALLOW_USER_EXISTS_RE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "create user '$EMAIL' '$PASS'"
user_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "ping"; then
user_ready=1
break
fi
sleep 1
done
if [[ "${user_ready}" -ne 1 ]]; then
echo "User service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "show version"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "create dataset '$DATASET' with embedding 'BAAI/bge-small-en-v1.5@Builtin' parser 'auto'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "import 'test/benchmark/test_docs/Doc1.pdf,test/benchmark/test_docs/Doc2.pdf' into dataset '$DATASET'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "parse dataset '$DATASET' sync"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "Benchmark 16 100 search 'what are these documents about' on datasets '$DATASET'"
- name: Stop ragflow to save coverage Infinity
if: ${{ !cancelled() }}
run: |
# Send SIGINT to ragflow_server.py to trigger coverage save
PID=$(sudo docker exec ${RAGFLOW_CONTAINER} ps aux | grep "ragflow_server.py" | grep -v grep | awk '{print $2}' | head -n 1)
if [ -n "$PID" ]; then
echo "Sending SIGINT to ragflow_server.py (PID: $PID)..."
sudo docker exec ${RAGFLOW_CONTAINER} kill -INT $PID
# Wait for process to exit and coverage file to be written
sleep 10
else
echo "ragflow_server.py not found!"
fi
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} stop
- name: Generate server coverage report Infinity
if: ${{ !cancelled() }}
run: |
# .coverage file should be in docker/ragflow-logs/.coverage
if [ -f docker/ragflow-logs/.coverage ]; then
echo "Found .coverage file"
cp docker/ragflow-logs/.coverage .coverage
source .venv/bin/activate
# Create .coveragerc to map container paths to host paths
echo "[paths]" > .coveragerc
echo "source =" >> .coveragerc
echo " ." >> .coveragerc
echo " /ragflow" >> .coveragerc
coverage xml -o coverage-infinity-server.xml
rm .coveragerc
else
echo ".coverage file not found!"
fi
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
if: ${{ !cancelled() }}
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
- name: Collect ragflow log Infinity
if: ${{ !cancelled() }}
run: |
if [ -d docker/ragflow-logs ]; then
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-infinity
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
else
echo "No docker/ragflow-logs directory found; skipping log collection"
fi
sudo rm -rf docker/ragflow-logs || true
- name: Stop ragflow:nightly for Infinity
if: always() # always run this step even if previous steps failed
run: |
# Sometimes `docker compose down` fail due to hang container, heavy load etc. Need to remove such containers to release resources(for example, listen ports).
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
- name: Start ragflow:nightly for Elasticsearch
run: |
sed -i 's/^DOC_ENGINE=.*$/DOC_ENGINE=elasticsearch/' docker/.env
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
uv sync --python 3.12 --group test --frozen && uv pip install -e sdk/python
- name: Run sdk tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
echo "Start to run test sdk on Elasticsearch"
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} --junitxml=pytest-infinity-sdk.xml --cov=sdk/python/ragflow_sdk --cov-branch --cov-report=xml:coverage-es-sdk.xml test/testcases/test_sdk_api 2>&1 | tee es_sdk_test.log
- name: Run web api tests against Elasticsearch
- name: Run New RESTFUL api tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_web_api 2>&1 | tee es_web_api_test.log
- name: Run http api tests against Elasticsearch
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api 2>&1 | tee es_http_api_test.log
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
source .venv/bin/activate && set -o pipefail; pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/restful_api 2>&1 | tee es_restful_api_test.log
- name: RAGFlow CLI retrieval test Elasticsearch
env:
@@ -267,7 +521,7 @@ jobs:
local tmp_log
tmp_log="$(mktemp)"
set +e
timeout 180s "$@" 2>&1 | tee "${tmp_log}"
timeout 500s "$@" 2>&1 | tee "${tmp_log}"
local status=${PIPESTATUS[0]}
set -e
cat "${tmp_log}" >> "${logfile}"
@@ -295,10 +549,20 @@ jobs:
ADMIN_HOST="${USER_HOST}"
ADMIN_PORT="${ADMIN_SVR_HTTP_PORT}"
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
svc_ready=0
for i in $(seq 1 60); do
if sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/api/v1/system/ping > /dev/null 2>&1; then
svc_ready=1
break
fi
echo "Waiting for service to be available... ($i/120)"
sleep 5
done
if [ "$svc_ready" -ne 1 ]; then
echo "Service did not become ready after 5 minutes. Docker logs:"
sudo docker logs ${RAGFLOW_CONTAINER}
exit 1
fi
admin_ready=0
for i in $(seq 1 30); do
@@ -383,198 +647,7 @@ jobs:
fi
sudo rm -rf docker/ragflow-logs || true
- name: Stop ragflow:nightly
if: always() # always run this step even if previous steps failed
run: |
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} down -v || true
sudo docker ps -a --filter "label=com.docker.compose.project=${GITHUB_RUN_ID}" -q | xargs -r sudo docker rm -f
- name: Start ragflow:nightly
run: |
sed -i '1i DOC_ENGINE=infinity' docker/.env
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} up -d
- name: Run sdk tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} --junitxml=pytest-infinity-sdk.xml --cov=sdk/python/ragflow_sdk --cov-branch --cov-report=xml:coverage-infinity-sdk.xml test/testcases/test_sdk_api 2>&1 | tee infinity_sdk_test.log
- name: Run web api tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_web_api/test_api_app 2>&1 | tee infinity_web_api_test.log
- name: Run http api tests against Infinity
run: |
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
source .venv/bin/activate && set -o pipefail; DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api 2>&1 | tee infinity_http_api_test.log
- name: RAGFlow CLI retrieval test Infinity
env:
PYTHONPATH: ${{ github.workspace }}
run: |
set -euo pipefail
source .venv/bin/activate
export http_proxy=""; export https_proxy=""; export no_proxy=""; export HTTP_PROXY=""; export HTTPS_PROXY=""; export NO_PROXY=""
EMAIL="ci-${GITHUB_RUN_ID}@example.com"
PASS="ci-pass-${GITHUB_RUN_ID}"
DATASET="ci_dataset_${GITHUB_RUN_ID}"
CLI="python admin/client/ragflow_cli.py"
LOG_FILE="infinity_cli_test.log"
: > "${LOG_FILE}"
ERROR_RE='Traceback|ModuleNotFoundError|ImportError|Parse error|Bad response|Fail to|code:\\s*[1-9]'
run_cli() {
local logfile="$1"
shift
local allow_re=""
if [[ "${1:-}" == "--allow" ]]; then
allow_re="$2"
shift 2
fi
local cmd_display="$*"
echo "===== $(date -u +\"%Y-%m-%dT%H:%M:%SZ\") CMD: ${cmd_display} =====" | tee -a "${logfile}"
local tmp_log
tmp_log="$(mktemp)"
set +e
timeout 180s "$@" 2>&1 | tee "${tmp_log}"
local status=${PIPESTATUS[0]}
set -e
cat "${tmp_log}" >> "${logfile}"
if grep -qiE "${ERROR_RE}" "${tmp_log}"; then
if [[ -n "${allow_re}" ]] && grep -qiE "${allow_re}" "${tmp_log}"; then
echo "Allowed CLI error markers in ${logfile}"
rm -f "${tmp_log}"
return 0
fi
echo "Detected CLI error markers in ${logfile}"
rm -f "${tmp_log}"
exit 1
fi
rm -f "${tmp_log}"
return ${status}
}
set -a
source docker/.env
set +a
HOST_ADDRESS="http://host.docker.internal:${SVR_HTTP_PORT}"
USER_HOST="$(echo "${HOST_ADDRESS}" | sed -E 's#^https?://([^:/]+).*#\1#')"
USER_PORT="${SVR_HTTP_PORT}"
ADMIN_HOST="${USER_HOST}"
ADMIN_PORT="${ADMIN_SVR_HTTP_PORT}"
until sudo docker exec ${RAGFLOW_CONTAINER} curl -s --connect-timeout 5 ${HOST_ADDRESS}/v1/system/ping > /dev/null; do
echo "Waiting for service to be available..."
sleep 5
done
admin_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "ping"; then
admin_ready=1
break
fi
sleep 1
done
if [[ "${admin_ready}" -ne 1 ]]; then
echo "Admin service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "show version"
ALLOW_USER_EXISTS_RE='already exists|already exist|duplicate|already.*registered|exist(s)?'
run_cli "${LOG_FILE}" --allow "${ALLOW_USER_EXISTS_RE}" $CLI --type admin --host "$ADMIN_HOST" --port "$ADMIN_PORT" --username "admin@ragflow.io" --password "admin" command "create user '$EMAIL' '$PASS'"
user_ready=0
for i in $(seq 1 30); do
if run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "ping"; then
user_ready=1
break
fi
sleep 1
done
if [[ "${user_ready}" -ne 1 ]]; then
echo "User service did not become ready"
exit 1
fi
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "show version"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "create dataset '$DATASET' with embedding 'BAAI/bge-small-en-v1.5@Builtin' parser 'auto'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "import 'test/benchmark/test_docs/Doc1.pdf,test/benchmark/test_docs/Doc2.pdf' into dataset '$DATASET'"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "parse dataset '$DATASET' sync"
run_cli "${LOG_FILE}" $CLI --type user --host "$USER_HOST" --port "$USER_PORT" --username "$EMAIL" --password "$PASS" command "Benchmark 16 100 search 'what are these documents about' on datasets '$DATASET'"
- name: Stop ragflow to save coverage Infinity
if: ${{ !cancelled() }}
run: |
# Send SIGINT to ragflow_server.py to trigger coverage save
PID=$(sudo docker exec ${RAGFLOW_CONTAINER} ps aux | grep "ragflow_server.py" | grep -v grep | awk '{print $2}' | head -n 1)
if [ -n "$PID" ]; then
echo "Sending SIGINT to ragflow_server.py (PID: $PID)..."
sudo docker exec ${RAGFLOW_CONTAINER} kill -INT $PID
# Wait for process to exit and coverage file to be written
sleep 10
else
echo "ragflow_server.py not found!"
fi
sudo docker compose -f docker/docker-compose.yml -p ${GITHUB_RUN_ID} stop
- name: Generate server coverage report Infinity
if: ${{ !cancelled() }}
run: |
# .coverage file should be in docker/ragflow-logs/.coverage
if [ -f docker/ragflow-logs/.coverage ]; then
echo "Found .coverage file"
cp docker/ragflow-logs/.coverage .coverage
source .venv/bin/activate
# Create .coveragerc to map container paths to host paths
echo "[paths]" > .coveragerc
echo "source =" >> .coveragerc
echo " ." >> .coveragerc
echo " /ragflow" >> .coveragerc
coverage xml -o coverage-infinity-server.xml
rm .coveragerc
else
echo ".coverage file not found!"
fi
- name: Upload coverage reports to Codecov
uses: codecov/codecov-action@v5
if: ${{ !cancelled() }}
with:
token: ${{ secrets.CODECOV_TOKEN }}
fail_ci_if_error: false
- name: Collect ragflow log
if: ${{ !cancelled() }}
run: |
if [ -d docker/ragflow-logs ]; then
cp -r docker/ragflow-logs ${ARTIFACTS_DIR}/ragflow-logs-infinity
echo "ragflow log" && tail -n 200 docker/ragflow-logs/ragflow_server.log || true
else
echo "No docker/ragflow-logs directory found; skipping log collection"
fi
sudo rm -rf docker/ragflow-logs || true
- name: Stop ragflow:nightly
- name: Stop ragflow:nightly for Elasticsearch
if: always() # always run this step even if previous steps failed
run: |
# Sometimes `docker compose down` fail due to hang container, heavy load etc. Need to remove such containers to release resources(for example, listen ports).

24
.gitignore vendored
View File

@@ -7,7 +7,7 @@ hudet/
cv/
layout_app.py
api/flask_session
venv/
# Remove Cargo.lock from gitignore if creating an executable, leave it for libraries
# More information here https://doc.rust-lang.org/cargo/guide/cargo-toml-vs-cargo-lock.html
Cargo.lock
@@ -21,6 +21,7 @@ Cargo.lock
.idea/
.vscode/
.cursor/settings.json
# Exclude Mac generated files
.DS_Store
@@ -205,9 +206,30 @@ ragflow_cli.egg-info
backup
*huqie.txt
.hypothesis
# Added by cargo
/target
# Do not include in PR (local dev / build artifacts)
ragflow.egg-info/
uv-aarch64*.tar.gz
uv-aarch64-unknown-linux-gnu.tar.gz
docker/launch_backend_service_windows.sh
# C++ build directories
internal/cpp/build/
internal/cpp/cmake-build-release/
internal/cpp/cmake-build-debug/
# Trae IDE config
.trae/
# Go server build output
bin/*
!bin/.gitkeep
.claude/settings.local.json

View File

@@ -34,8 +34,8 @@ The project uses **uv** for dependency management.
1. **Setup Environment**:
```bash
uv sync --python 3.12 --all-extras
uv run download_deps.py
uv sync --python 3.13 --all-extras
uv run python3 download_deps.py
```
2. **Run Server**:

View File

@@ -5,14 +5,16 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
## Project Overview
RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on deep document understanding. It's a full-stack application with:
- Python backend (Flask-based API server)
- React/TypeScript frontend (built with UmiJS)
- React/TypeScript frontend (built with vitejs)
- Microservices architecture with Docker deployment
- Multiple data stores (MySQL, Elasticsearch/Infinity, Redis, MinIO)
## Architecture
### Backend (`/api/`)
- **Main Server**: `api/ragflow_server.py` - Flask application entry point
- **Apps**: Modular Flask blueprints in `api/apps/` for different functionalities:
- `kb_app.py` - Knowledge base management
@@ -24,29 +26,33 @@ RAGFlow is an open-source RAG (Retrieval-Augmented Generation) engine based on d
- **Models**: Database models in `api/db/db_models.py`
### Core Processing (`/rag/`)
- **Document Processing**: `deepdoc/` - PDF parsing, OCR, layout analysis
- **LLM Integration**: `rag/llm/` - Model abstractions for chat, embedding, reranking
- **RAG Pipeline**: `rag/flow/` - Chunking, parsing, tokenization
- **Graph RAG**: `rag/graphrag/` - Knowledge graph construction and querying
### Agent System (`/agent/`)
- **Components**: Modular workflow components (LLM, retrieval, categorize, etc.)
- **Templates**: Pre-built agent workflows in `agent/templates/`
- **Tools**: External API integrations (Tavily, Wikipedia, SQL execution, etc.)
### Frontend (`/web/`)
- React/TypeScript with UmiJS framework
- Ant Design + shadcn/ui components
- React/TypeScript with vitejs framework
- shadcn/ui components
- State management with Zustand
- Tailwind CSS for styling
## Common Development Commands
### Backend Development
```bash
# Install Python dependencies
uv sync --python 3.12 --all-extras
uv run download_deps.py
uv sync --python 3.13 --all-extras
uv run python3 download_deps.py
pre-commit install
# Start dependent services
@@ -66,6 +72,7 @@ ruff format
```
### Frontend Development
```bash
cd web
npm install
@@ -76,6 +83,7 @@ npm run test # Jest tests
```
### Docker Development
```bash
# Full stack with Docker
cd docker
@@ -104,13 +112,23 @@ docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly
## Database Engines
RAGFlow supports switching between Elasticsearch (default) and Infinity:
- Set `DOC_ENGINE=infinity` in `docker/.env` to use Infinity
- Requires container restart: `docker compose down -v && docker compose up -d`
## Development Environment Requirements
- Python 3.10-3.12
- Python 3.10-3.13
- Node.js >=18.20.4
- Docker & Docker Compose
- uv package manager
- 16GB+ RAM, 50GB+ disk space
1. Think before acting. Read existing files before writing code.
2. Be concise in output but thorough in reasoning.
3. Prefer editing over rewriting whole files.
4. Do not re-read files you have already read.
5. Test your code before declaring done.
6. No sycophantic openers or closing fluff.
7. Keep solutions simple and direct.
8. User instructions always override this file.

View File

@@ -7,7 +7,7 @@ ARG NEED_MIRROR=0
WORKDIR /ragflow
# Copy models downloaded via download_deps.py
# copy models downloaded via download_deps.py
RUN mkdir -p /ragflow/rag/res/deepdoc /root/.ragflow
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/huggingface.co,target=/huggingface.co \
tar --exclude='.*' -cf - \
@@ -19,49 +19,49 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/huggingface.co
# This is the only way to run python-tika without internet access. Without this set, the default is to check the tika version and pull latest every time from Apache.
RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps \
cp -r /deps/nltk_data /root/ && \
cp /deps/tika-server-standard-3.2.3.jar /deps/tika-server-standard-3.2.3.jar.md5 /ragflow/ && \
cp /deps/tika-server-standard-3.3.0.jar /deps/tika-server-standard-3.3.0.jar.md5 /ragflow/ && \
cp /deps/cl100k_base.tiktoken /ragflow/9b5ad71b2ce5302211f9c61530b329a4922fc6a4
ENV TIKA_SERVER_JAR="file:///ragflow/tika-server-standard-3.2.3.jar"
ENV TIKA_SERVER_JAR="file:///ragflow/tika-server-standard-3.3.0.jar"
ENV DEBIAN_FRONTEND=noninteractive
# Setup apt
# Python package and implicit dependencies:
# opencv-python: libglib2.0-0 libglx-mesa0 libgl1
# python-pptx: default-jdk tika-server-standard-3.2.3.jar
# python-pptx: default-jdk tika-server-standard-3.3.0.jar
# selenium: libatk-bridge2.0-0 chrome-linux64-121-0-6167-85
# Building C extensions: libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
apt update && \
apt --no-install-recommends install -y ca-certificates; \
if [ "$NEED_MIRROR" == "1" ]; then \
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://security.ubuntu.com/ubuntu|https://mirrors.tuna.tsinghua.edu.cn/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://archive.ubuntu.com/ubuntu|https://mirrors.aliyun.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
sed -i 's|http://security.ubuntu.com/ubuntu|https://mirrors.aliyun.com/ubuntu|g' /etc/apt/sources.list.d/ubuntu.sources; \
fi; \
rm -f /etc/apt/apt.conf.d/docker-clean && \
echo 'Binary::apt::APT::Keep-Downloaded-Packages "true";' > /etc/apt/apt.conf.d/keep-cache && \
chmod 1777 /tmp && \
apt update && \
apt install -y libglib2.0-0 libglx-mesa0 libgl1 && \
apt install -y pkg-config libicu-dev libgdiplus && \
apt install -y default-jdk && \
apt install -y libatk-bridge2.0-0 && \
apt install -y libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev && \
apt install -y libjemalloc-dev && \
apt install -y gnupg unzip curl wget git vim less && \
apt install -y ghostscript && \
apt install -y pandoc && \
apt install -y texlive && \
apt install -y fonts-freefont-ttf fonts-noto-cjk && \
apt install -y postgresql-client
apt install -y \
build-essential libglib2.0-0 libglx-mesa0 libgl1 pkg-config libicu-dev libgdiplus default-jdk libatk-bridge2.0-0 libpython3-dev libgtk-4-1 libnss3 xdg-utils libgbm-dev libjemalloc-dev gnupg unzip curl wget git vim less ghostscript pandoc texlive texlive-latex-extra texlive-xetex texlive-lang-chinese fonts-freefont-ttf fonts-noto-cjk postgresql-client
ARG NGINX_VERSION=1.29.5-1~noble
# Download resource from GitHub to /usr/share/infinity
RUN mkdir -p /usr/share/infinity/resource && \
if [ "$NEED_MIRROR" == "1" ]; then \
git clone --depth 1 --single-branch https://gitee.com/infiniflow/resource /tmp/resource; \
else \
git clone --depth 1 --single-branch https://github.com/infiniflow/resource.git /tmp/resource; \
fi && \
cp -r /tmp/resource/* /usr/share/infinity/resource && \
rm -rf /tmp/resource
ARG NGINX_VERSION=1.31.0-1~noble
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
mkdir -p /etc/apt/keyrings && \
curl -fsSL https://nginx.org/keys/nginx_signing.key | gpg --dearmor -o /etc/apt/keyrings/nginx-archive-keyring.gpg && \
curl --retry 5 --retry-delay 2 --retry-all-errors -fsSL https://nginx.org/keys/nginx_signing.key | gpg --dearmor -o /etc/apt/keyrings/nginx-archive-keyring.gpg && \
echo "deb [signed-by=/etc/apt/keyrings/nginx-archive-keyring.gpg] https://nginx.org/packages/mainline/ubuntu/ noble nginx" > /etc/apt/sources.list.d/nginx.list && \
apt update && \
apt install -y nginx=${NGINX_VERSION} && \
apt -o Acquire::Retries=5 update && \
apt -o Acquire::Retries=5 install -y nginx=${NGINX_VERSION} && \
apt-mark hold nginx
# Install uv
@@ -70,7 +70,7 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps
mkdir -p /etc/uv && \
echo 'python-install-mirror = "https://registry.npmmirror.com/-/binary/python-build-standalone/"' > /etc/uv/uv.toml && \
echo '[[index]]' >> /etc/uv/uv.toml && \
echo 'url = "https://pypi.tuna.tsinghua.edu.cn/simple"' >> /etc/uv/uv.toml && \
echo 'url = "https://mirrors.aliyun.com/pypi/simple"' >> /etc/uv/uv.toml && \
echo 'default = true' >> /etc/uv/uv.toml; \
fi; \
arch="$(uname -m)"; \
@@ -78,35 +78,21 @@ RUN --mount=type=bind,from=infiniflow/ragflow_deps:latest,source=/,target=/deps
tar xzf "/deps/uv-${uv_arch}-unknown-linux-gnu.tar.gz" \
&& cp "uv-${uv_arch}-unknown-linux-gnu/"* /usr/local/bin/ \
&& rm -rf "uv-${uv_arch}-unknown-linux-gnu" \
&& uv python install 3.12
&& uv python install 3.13
ENV PYTHONDONTWRITEBYTECODE=1 DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1
ENV PYTHONDONTWRITEBYTECODE=1 DOTNET_SYSTEM_GLOBALIZATION_INVARIANT=1 \
UV_HTTP_TIMEOUT=200 \
UV_HTTP_RETRIES=3
ENV PATH=/root/.local/bin:$PATH
# nodejs 12.22 on Ubuntu 22.04 is too old
RUN --mount=type=cache,id=ragflow_apt,target=/var/cache/apt,sharing=locked \
curl -fsSL https://deb.nodesource.com/setup_20.x | bash - && \
apt purge -y nodejs npm cargo && \
apt purge -y nodejs npm && \
apt autoremove -y && \
apt update && \
apt install -y nodejs
# A modern version of cargo is needed for the latest version of the Rust compiler.
RUN apt update && apt install -y curl build-essential \
&& if [ "$NEED_MIRROR" == "1" ]; then \
# Use TUNA mirrors for rustup/rust dist files \
export RUSTUP_DIST_SERVER="https://mirrors.tuna.tsinghua.edu.cn/rustup"; \
export RUSTUP_UPDATE_ROOT="https://mirrors.tuna.tsinghua.edu.cn/rustup/rustup"; \
echo "Using TUNA mirrors for Rustup."; \
fi; \
# Force curl to use HTTP/1.1 \
curl --proto '=https' --tlsv1.2 --http1.1 -sSf https://sh.rustup.rs | bash -s -- -y --profile minimal \
&& echo 'export PATH="/root/.cargo/bin:${PATH}"' >> /root/.bashrc
ENV PATH="/root/.cargo/bin:${PATH}"
RUN cargo --version && rustc --version
# Add msssql ODBC driver
# macOS ARM64 environment, install msodbcsql18.
# general x86_64 environment, install msodbcsql17.
@@ -157,19 +143,19 @@ COPY pyproject.toml uv.lock ./
# uv records index url into uv.lock but doesn't failover among multiple indexes
RUN --mount=type=cache,id=ragflow_uv,target=/root/.cache/uv,sharing=locked \
if [ "$NEED_MIRROR" == "1" ]; then \
sed -i 's|pypi.org|pypi.tuna.tsinghua.edu.cn|g' uv.lock; \
sed -i 's|pypi.org|mirrors.aliyun.com/pypi|g' uv.lock; \
else \
sed -i 's|pypi.tuna.tsinghua.edu.cn|pypi.org|g' uv.lock; \
sed -i 's|mirrors.aliyun.com/pypi|pypi.org|g' uv.lock; \
fi; \
uv sync --python 3.12 --frozen && \
uv sync --python 3.13 --frozen && \
# Ensure pip is available in the venv for runtime package installation (fixes #12651)
.venv/bin/python3 -m ensurepip --upgrade
COPY web web
COPY docs docs
RUN --mount=type=cache,id=ragflow_npm,target=/root/.npm,sharing=locked \
export NODE_OPTIONS="--max-old-space-size=4096" && \
cd web && npm install && npm run build
cd web && NODE_OPTIONS="--max-old-space-size=8192" npm install && \
NODE_OPTIONS="--max-old-space-size=8192" VITE_BUILD_SOURCEMAP=false VITE_MINIFY=esbuild npm run build
COPY .git /ragflow/.git
@@ -202,11 +188,19 @@ COPY pyproject.toml uv.lock ./
COPY mcp mcp
COPY common common
COPY memory memory
COPY bin bin
COPY docker/service_conf.yaml.template ./conf/service_conf.yaml.template
COPY docker/entrypoint.sh ./
RUN chmod +x ./entrypoint*.sh
# Copy nginx configuration for frontend serving
COPY docker/nginx/ragflow.conf.golang docker/nginx/ragflow.conf.python docker/nginx/ragflow.conf.hybrid docker/nginx/nginx.conf docker/nginx/proxy.conf /etc/nginx/
RUN mv /etc/nginx/ragflow.conf.golang /etc/nginx/conf.d/ragflow.conf.golang && \
mv /etc/nginx/ragflow.conf.python /etc/nginx/conf.d/ragflow.conf.python && \
mv /etc/nginx/ragflow.conf.hybrid /etc/nginx/conf.d/ragflow.conf.hybrid && \
rm -f /etc/nginx/sites-enabled/default
# Copy compiled web pages
COPY --from=builder /ragflow/web/dist /ragflow/web/dist

View File

@@ -3,7 +3,7 @@
FROM scratch
# Copy resources downloaded via download_deps.py
COPY chromedriver-linux64-121-0-6167-85 chrome-linux64-121-0-6167-85 cl100k_base.tiktoken libssl1.1_1.1.1f-1ubuntu2_amd64.deb libssl1.1_1.1.1f-1ubuntu2_arm64.deb tika-server-standard-3.2.3.jar tika-server-standard-3.2.3.jar.md5 libssl*.deb uv-x86_64-unknown-linux-gnu.tar.gz uv-aarch64-unknown-linux-gnu.tar.gz /
COPY chromedriver-linux64-121-0-6167-85 chrome-linux64-121-0-6167-85 cl100k_base.tiktoken libssl1.1_1.1.1f-1ubuntu2_amd64.deb libssl1.1_1.1.1f-1ubuntu2_arm64.deb tika-server-standard-3.3.0.jar tika-server-standard-3.3.0.jar.md5 libssl*.deb uv-x86_64-unknown-linux-gnu.tar.gz uv-aarch64-unknown-linux-gnu.tar.gz /
COPY nltk_data /nltk_data

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Table of Contents</b></summary>
- 💡 [What is RAGFlow?](#-what-is-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Get Started](#-get-started)
- 📌 [Latest Updates](#-latest-updates)
- 🌟 [Key Features](#-key-features)
- 🔎 [System Architecture](#-system-architecture)
- 🎬 [Get Started](#-get-started)
- 🎬 [Self-Hosting](#-self-hosting)
- 🔧 [Configurations](#-configurations)
- 🔧 [Build a Docker image](#-build-a-docker-image)
- 🔨 [Launch service from source for development](#-launch-service-from-source-for-development)
@@ -74,9 +76,9 @@
[RAGFlow](https://ragflow.io/) is a leading open-source Retrieval-Augmented Generation ([RAG](https://ragflow.io/basics/what-is-rag)) engine that fuses cutting-edge RAG with Agent capabilities to create a superior context layer for LLMs. It offers a streamlined RAG workflow adaptable to enterprises of any scale. Powered by a converged [context engine](https://ragflow.io/basics/what-is-agent-context-engine) and pre-built agent templates, RAGFlow enables developers to transform complex data into high-fidelity, production-ready AI systems with exceptional efficiency and precision.
## 🎮 Demo
## 🎮 Get Started
Try our demo at [https://demo.ragflow.io](https://demo.ragflow.io).
Try our cloud service at [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@ Try our demo at [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Latest Updates
- 2026-04-24 Supports DeepSeek v4.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Provides an official skill for accessing RAGFlow datasets via OpenClaw.
- 2025-12-26 Supports 'Memory' for AI agent.
- 2025-11-19 Supports Gemini 3 Pro.
- 2025-11-12 Supports data synchronization from Confluence, S3, Notion, Discord, Google Drive.
@@ -140,7 +144,7 @@ releases! 🌟
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Get Started
## 🎬 Self-Hosting
### 📝 Prerequisites
@@ -148,6 +152,7 @@ releases! 🌟
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): Required only if you intend to use the code executor (sandbox) feature of RAGFlow.
> [!TIP]
@@ -188,12 +193,12 @@ releases! 🌟
> All Docker images are built for x86 platforms. We don't currently offer Docker images for ARM64.
> If you are on an ARM64 platform, follow [this guide](https://ragflow.io/docs/dev/build_docker_image) to build a Docker image compatible with your system.
> The command below downloads the `v0.24.0` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.24.0`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server.
> The command below downloads the `v0.25.6` edition of the RAGFlow Docker image. See the following table for descriptions of different RAGFlow editions. To download a RAGFlow edition different from `v0.25.6`, update the `RAGFLOW_IMAGE` variable accordingly in **docker/.env** before using `docker compose` to start the server.
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# This step ensures the **entrypoint.sh** file in the code matches the Docker image version.
@@ -324,8 +329,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. Launch the dependent services (MinIO, Elasticsearch, Redis, and MySQL) using Docker Compose:
@@ -389,8 +394,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -401,7 +406,7 @@ See the [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/1224
## 🏄 Community
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contributing

415
README_ar.md Normal file
View File

@@ -0,0 +1,415 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DBEDFA"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/License-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="license">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 جدول المحتويات</b></summary>
- 💡 [ما هو RAGFlow؟](#-what-is-ragflow)
- 🎮 [ابدأ](#-get-started)
- 📌 [آخر التحديثات](#-latest-updates)
- 🌟 [الميزات الرئيسية](#-key-features)
- 🔎 [بنية النظام](#-system-architecture)
- 🎬 [الاستضافة الذاتية](#-self-hosting)
- 🔧 [التكوينات](#-configurations)
- 🔧 [إنشاء صورة Docker](#-build-a-docker-image)
- 🔨 [إطلاق الخدمة من المصدر للتطوير](#-launch-service-from-source-for-development)
- 📚 [التوثيق](#-documentation)
- 📜 [Roadmap](#-roadmap)
- 🏄 [المجتمع](#-community)
- 🙌 [مساهمة](#-contributing)
</details>
## 💡 ما هو RAGFlow؟
يُعد مشروع [RAGFlow](https://ragflow.io/) محركًا رائدًا ومفتوح المصدر للاسترجاع المعزز بالتوليد (<bdi dir="ltr">RAG</bdi>)، ويجمع أحدث تقنيات <bdi dir="ltr">RAG</bdi> مع قدرات الوكلاء لبناء طبقة سياق متقدمة لنماذج <bdi dir="ltr">LLMs</bdi>. يوفّر سير عمل <bdi dir="ltr">RAG</bdi> مبسّطًا وقابلًا للتكيّف مع المؤسسات بمختلف أحجامها. وبالاعتماد على [محرك سياق موحّد](https://ragflow.io/basics/what-is-agent-context-engine) وقوالب وكلاء جاهزة، يتيح <bdi dir="ltr">RAGFlow</bdi> للمطورين تحويل البيانات المعقّدة إلى أنظمة <bdi dir="ltr">AI</bdi> عالية الدقة وجاهزة للإنتاج بكفاءة وموثوقية.
## 🎮 ابدأ
جرّب النسخة التجريبية على [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 آخر التحديثات
- 24-04-2026 يدعم DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — توفر مهارة رسمية للوصول إلى مجموعات بيانات RAGFlow عبر OpenClaw.
- 26-12-2025 يدعم ميزة "Memory" لوكلاء الذكاء الاصطناعي.
- 11-11-2025 يدعم Gemini 3 Pro.
- 12-11-2025 يدعم مزامنة البيانات من Confluence، S3، Notion، Discord، Google Drive.
- 23-10-2025 يدعم MinerU وDocling كطرق لتحليل المستندات.
- 15-10-2025 يدعم العرض الأوركسترالي pipeline.
- 08-08-2025 يدعم أحدث موديلات سلسلة OpenAI.
- 01-08-2025 يدعم سير العمل الوكيل وMCP.
- 23-05-2025 تمت إضافة مكون منفذ كود Python/JavaScript إلى Agent.
- 05-05-2025 يدعم الاستعلام بين اللغات.
- 19-03-2025 يدعم استخدام نموذج متعدد الوسائط لفهم الصور داخل ملفات PDF أو DOCX.
## 🎉 تابعونا
⭐️ قم بتمييز مستودعنا بنجمة لتبقى على اطلاع بالميزات والتحسينات الجديدة والمثيرة! احصل على إشعارات فورية بالجديد
الإصدارات! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 الميزات الرئيسية
### 🍭 **"الجودة في الداخل، الجودة في الخارج"**
- [الفهم العميق للمستندات](./deepdoc/README.md) لاستخراج المعرفة من البيانات غير المنظمة
ذات التنسيقات المعقدة.
- يجد "إبرة في كومة قش بيانات" من الرموز غير المحدودة حرفيًا.
### 🍱 **التقطيع القائم على القالب**
- ذكي وقابل للتفسير.
- الكثير من خيارات القالب للاختيار من بينها.
### 🌱 **استشهادات مؤرضة لتقليل الهلوسة**
- تصور تقطيع النص للسماح بالتدخل البشري.
- عرض سريع للمراجع الرئيسية والاستشهادات التي يمكن تتبعها لدعم الإجابات المبنية على أسس سليمة.
### 🍔 **التوافق مع مصادر البيانات غير المتجانسة**
- يدعم Word، والشرائح، وExcel، وtxt، والصور، والنسخ الممسوحة ضوئيًا، والبيانات المنظمة، وصفحات الويب، والمزيد.
### 🛀 **سير عمل RAG آلي وسهل**
- تنسيق RAG مبسط يلبي احتياجات الشركات الشخصية والكبيرة على حد سواء.
- نماذج LLMs قابلة للتكوين بالإضافة إلى نماذج embedding.
- الاستدعاء المتعدد المقترن بإعادة التصنيف المدمجة.
- APIs بديهي للتكامل السلس مع الأعمال.
## 🔎 هندسة النظام
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 الاستضافة الذاتية
### 📝 المتطلبات الأساسية
- CPU >= 4 مراكز
- الرام >= 16 جيجا
- القرص >= 50 جيجا بايت
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- بايثون >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): مطلوب فقط إذا كنت تنوي استخدام ميزة منفذ التعليمات البرمجية (وضع الحماية) لـ RAGFlow.
> [!TIP]
> إذا لم تقم بتثبيت Docker على جهازك المحلي (Windows أو Mac أو Linux)، راجع [تثبيت Docker Engine](https://docs.docker.com/engine/install/).
### 🚀 بدء تشغيل الخادم
1. تأكد من `vm.max_map_count` >= 262144:
> للتحقق من قيمة `vm.max_map_count`:
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> أعد تعيين `vm.max_map_count` إلى قيمة 262144 على الأقل إذا لم تكن كذلك.
>
> ```bash
> # In this case, we set it to 262144:
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> سيتم إعادة ضبط هذا التغيير بعد إعادة تشغيل النظام. لضمان بقاء التغيير دائمًا، قم بإضافة أو تحديث
> `vm.max_map_count` القيمة في **/etc/sysctl.conf** وفقًا لذلك:
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. استنساخ الريبو:
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. ابدأ تشغيل الخادم باستخدام صور Docker المعدة مسبقًا:
> [!CAUTION]
> جميع الصور Docker مصممة لمنصات x86. لا نعرض حاليًا صور Docker لـ ARM64.
> إذا كنت تستخدم نظامًا أساسيًا ARM64، فاتبع [هذا الدليل](https://ragflow.io/docs/dev/build_docker_image) لإنشاء صورة Docker متوافقة مع نظامك.
> يقوم الأمر أدناه بتنزيل إصدار `v0.25.6` من الصورة RAGFlow Docker. راجع الجدول التالي للحصول على أوصاف لإصدارات RAGFlow المختلفة. لتنزيل إصدار RAGFlow مختلف عن `v0.25.6`، قم بتحديث المتغير `RAGFLOW_IMAGE` وفقًا لذلك في **docker/.env** قبل استخدام `docker compose` لبدء تشغيل الخادم.
```bash
$ cd ragflow/docker
# git checkout v0.25.6
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# This step ensures the **entrypoint.sh** file in the code matches the Docker image version.
# Use CPU for DeepDoc tasks:
$ docker compose -f docker-compose.yml up -d
# To use GPU to accelerate DeepDoc tasks:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> ملاحظة: قبل `v0.22.0`، قدمنا ​​كلتا الصورتين بنماذج embedding وصورًا رفيعة بدون نماذج embedding. التفاصيل على النحو التالي:
| RAGFlow علامة الصورة | حجم الصورة (جيجابايت) | هل لديه نماذج embedding؟ | مستقر؟ |
|-------------------|-----------------|-----------------------|----------------|
| v0.21.1 | &approx;9 | ✔️ | إصدار مستقر |
| v0.21.1-slim | &approx;2 | ❌ | إصدار مستقر |
> بدءًا من `v0.22.0`، نقوم بشحن الإصدار النحيف فقط ولم نعد نلحق اللاحقة **-slim** بعلامة الصورة.
4. التحقق من حالة الخادم بعد تشغيل الخادم:
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_النتيجة التالية تؤكد الإطلاق الناجح للنظام:_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> إذا تخطيت خطوة التأكيد هذه وقمت بتسجيل الدخول مباشرة إلى RAGFlow، فقد يعرض متصفحك تنبيه `network abnormal`
> خطأ لأنه في تلك اللحظة، قد لا تتم تهيئة RAGFlow بشكل كامل.
>
5. في متصفح الويب الخاص بك، أدخل عنوان IP الخاص بالخادم الخاص بك وقم بتسجيل الدخول إلى RAGFlow.
> باستخدام الإعدادات الافتراضية، ما عليك سوى إدخال `http://IP_OF_YOUR_MACHINE` (**من دون** رقم المنفذ) كإعداد افتراضي
> HTTP يمكن حذف منفذ العرض `80` عند استخدام التكوينات الافتراضية.
>
6. في [service_conf.yaml.template](./docker/service_conf.yaml.template)، حدد المصنع LLM المطلوب في `user_default_llm` وقم بالتحديث
الحقل `API_KEY` مع مفتاح API المقابل.
> راجع [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) لمزيد من المعلومات.
>
_العرض بدأ!_
## 🔧 التكوينات
عندما يتعلق الأمر بتكوينات النظام، ستحتاج إلى إدارة الملفات التالية:
- [.env](./docker/.env): يحتفظ بالإعدادات الأساسية للنظام، مثل `SVR_HTTP_PORT`، `MYSQL_PASSWORD`، و
`MINIO_PASSWORD`.
- [service_conf.yaml.template](./docker/service_conf.yaml.template): تكوين الخدمات الخلفية. سيتم ملء متغيرات البيئة في هذا الملف تلقائيًا عند بدء تشغيل الحاوية Docker. ستكون أي متغيرات بيئة تم تعيينها داخل حاوية Docker متاحة للاستخدام، مما يسمح لك بتخصيص سلوك الخدمة استنادًا إلى بيئة النشر.
- [docker-compose.yml](./docker/docker-compose.yml): يعتمد النظام على [docker-compose.yml](./docker/docker-compose.yml) لبدء التشغيل.
> يوفر الملف [./docker/README](./docker/README.md) وصفًا تفصيليًا لإعدادات البيئة والخدمة
> التكوينات التي يمكن استخدامها كـ `${ENV_VARS}` في ملف [service_conf.yaml.template](./docker/service_conf.yaml.template).
لتحديث منفذ العرض الافتراضي HTTP (80)، انتقل إلى [docker-compose.yml](./docker/docker-compose.yml) وقم بتغيير `80:80`
إلى `<YOUR_SERVING_PORT>:80`.
تتطلب تحديثات التكوينات المذكورة أعلاه إعادة تشغيل جميع الحاويات لتصبح سارية المفعول:
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### تبديل محرك المستندات من Elasticsearch إلى Infinity
RAGFlow يستخدم Elasticsearch بشكل افتراضي لتخزين النص الكامل والمتجهات. للتبديل إلى [Infinity](https://github.com/infiniflow/infinity/)، اتبع الخطوات التالية:
1. إيقاف كافة الحاويات قيد التشغيل:
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` سوف يحذف docker وحدات تخزين الحاوية، وسيتم مسح البيانات الموجودة.
2. اضبط `DOC_ENGINE` في **docker/.env** على `infinity`.
3. ابدأ الحاويات:
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> التبديل إلى Infinity على جهاز Linux/arm64 غير مدعوم رسميًا بعد.
## 🔧 أنشئ صورة Docker
يبلغ حجم هذه الصورة حوالي 2 غيغابايت وتعتمد على خدمات LLM وembedding الخارجية.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
أو إذا كنت خلف وكيل، فيمكنك تمرير وسيطات الوكيل:
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://YOUR_PROXY:PORT \
--build-arg https_proxy=http://YOUR_PROXY:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 إطلاق الخدمة من المصدر للتطوير
1. قم بتثبيت `uv` و`pre-commit`، أو قم بتخطي هذه الخطوة إذا كانا مثبتين بالفعل:
```bash
pipx install uv pre-commit
```
2. استنساخ الكود المصدري وتثبيت تبعيات بايثون:
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. قم بتشغيل الخدمات التابعة (MinIO وElasticsearch وRedis وMySQL) باستخدام Docker Compose:
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
أضف السطر التالي إلى `/etc/hosts` لحل كافة المضيفين المحددين في **docker/.env** إلى `127.0.0.1`:
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. إذا لم تتمكن من الوصول إلى HuggingFace، فقم بتعيين متغير البيئة `HF_ENDPOINT` لاستخدام موقع مرآة:
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. إذا كان نظام التشغيل لديك لا يحتوي على jemalloc، فيرجى تثبيته على النحو التالي:
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. إطلاق الخدمة الخلفية:
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. تثبيت تبعيات الواجهة الأمامية:
```bash
cd web
npm install
```
8. إطلاق خدمة الواجهة الأمامية:
```bash
npm run dev
```
_النتيجة التالية تؤكد الإطلاق الناجح للنظام:_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. أوقف خدمة الواجهة الأمامية والخلفية RAGFlow بعد اكتمال التطوير:
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 التوثيق
- [البدء السريع](https://ragflow.io/docs/dev/)
- [التكوين](https://ragflow.io/docs/dev/configurations)
- [ملاحظات الإصدار](https://ragflow.io/docs/dev/release_notes)
- [أدلة المستخدم](https://ragflow.io/docs/category/user-guides)
- [أدلة المطورين](https://ragflow.io/docs/category/developer-guides)
- [المراجع](https://ragflow.io/docs/dev/category/references)
- [الأسئلة الشائعة](https://ragflow.io/docs/dev/faq)
## 📜 Roadmap
راجع [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 المجتمع
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [مناقشات جيثب](https://github.com/orgs/infiniflow/discussions)
## 🙌 المساهمة
RAGFlow يزدهر من خلال التعاون مفتوح المصدر. وبهذه الروح، فإننا نحتضن المساهمات المتنوعة من المجتمع.
إذا كنت ترغب في أن تكون جزءًا، فراجع [إرشادات المساهمة](https://ragflow.io/docs/dev/contributing) أولاً.

406
README_fr.md Normal file
View File

@@ -0,0 +1,406 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DBEDFA"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="suivre sur X(Twitter)">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Badge statique" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Dernière%20version" alt="Dernière version">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/License-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="licence">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Documentation</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 Table des matières</b></summary>
- 💡 [Qu'est-ce que RAGFlow?](#-quest-ce-que-ragflow)
- 🎮 [Démarrage](#-démarrage)
- 📌 [Dernières mises à jour](#-dernières-mises-à-jour)
- 🌟 [Fonctionnalités clés](#-fonctionnalités-clés)
- 🔎 [Architecture du système](#-architecture-du-système)
- 🎬 [Auto-hébergement](#-auto-hébergement)
- 🔧 [Configurations](#-configurations)
- 🔧 [Construire une image Docker](#-construire-une-image-docker)
- 🔨 [Lancer le service depuis les sources pour le développement](#-lancer-le-service-depuis-les-sources-pour-le-développement)
- 📚 [Documentation](#-documentation)
- 📜 [Roadmap](#-feuille-de-route)
- 🏄 [Communauté](#-communauté)
- 🙌 [Contribuer](#-contribuer)
</details>
## 💡 Qu'est-ce que RAGFlow?
[RAGFlow](https://ragflow.io/) est un moteur de [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source de premier plan qui fusionne les technologies RAG de pointe avec des capacités Agent pour créer une couche de contexte supérieure pour les LLM. Il offre un flux de travail RAG rationalisé, adaptable aux entreprises de toute taille. Alimenté par un [moteur de contexte](https://ragflow.io/basics/what-is-agent-context-engine) convergent et des modèles d'agents préconstruits, RAGFlow permet aux développeurs de transformer des données complexes en systèmes d'IA haute-fidélité, prêts pour la production, avec une efficacité et une précision exceptionnelles.
## 🎮 Démarrage
Essayez notre service cloud sur [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 Dernières mises à jour
- 24-04-2026 Prise en charge de DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Fournit un skill officiel pour accéder aux datasets RAGFlow via OpenClaw.
- 26-12-2025 Prise en charge de la « Mémoire » pour l'agent IA.
- 19-11-2025 Prise en charge de Gemini 3 Pro.
- 12-11-2025 Prise en charge de la synchronisation de données depuis Confluence, S3, Notion, Discord et Google Drive.
- 23-10-2025 Prise en charge de MinerU & Docling comme méthodes d'analyse de documents.
- 15-10-2025 Prise en charge du pipeline d'ingestion orchestrable.
- 08-08-2025 Prise en charge des derniers modèles de la série GPT-5 d'OpenAI.
- 01-08-2025 Prise en charge du flux de travail agentique et de MCP.
- 23-05-2025 Ajout d'un composant exécuteur de code Python/JavaScript à l'Agent.
- 05-05-2025 Prise en charge des requêtes inter-langues.
- 19-03-2025 Prise en charge de l'utilisation d'un modèle multi-modal pour analyser les images dans les fichiers PDF ou DOCX.
## 🎉 Restez informé
⭐️ Mettez une étoile à notre dépôt pour rester informé des nouvelles fonctionnalités et améliorations passionnantes ! Recevez des notifications instantanées pour les nouvelles versions ! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 Fonctionnalités clés
### 🍭 **"Quality in, quality out"**
- Extraction de connaissances basée sur la [compréhension approfondie des documents](./deepdoc/README.md) à partir de données non structurées aux formats complexes.
- Trouve "l'aiguille dans la meule de données" de tokens littéralement illimités.
### 🍱 **Découpage(Chunking) basé sur des templates**
- Intelligent et explicable.
- De nombreuses options de templates disponibles.
### 🌱 **Citations fondées avec réduction des hallucinations**
- Visualisation du découpage de texte pour permettre une intervention humaine.
- Aperçu rapide des références clés et citations traçables pour soutenir des réponses fondées.
### 🍔 **Compatibilité avec des sources de données hétérogènes**
- Prend en charge Word, présentations, Excel, txt, images, copies numérisées, données structurées, pages web, et plus encore.
### 🛀 **Flux de travail RAG automatisé et sans effort**
- Orchestration RAG rationalisée adaptée aux particuliers comme aux grandes entreprises.
- LLM et modèles d'embedding configurables.
- Rappel multiple associé à un ré-classement fusionné.
- APIs intuitives pour une intégration transparente avec les entreprises.
## 🔎 Architecture du système
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Auto-hébergement
### 📝 Prérequis
- CPU >= 4 cœurs
- RAM >= 16 Go
- Disque >= 50 Go
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/) : Requis uniquement si vous souhaitez utiliser la fonctionnalité d'exécuteur de code (sandbox) de RAGFlow.
> [!TIP]
> Si vous n'avez pas installé Docker sur votre machine locale (Windows, Mac ou Linux), consultez [Installer Docker Engine](https://docs.docker.com/engine/install/).
### 🚀 Démarrer le serveur
1. Assurez-vous que `vm.max_map_count` >= 262144 :
> Pour vérifier la valeur de `vm.max_map_count` :
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> Réinitialisez `vm.max_map_count` à une valeur d'au moins 262144 si ce n'est pas le cas.
>
> ```bash
> # Dans ce cas, nous le définissons à 262144 :
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> Ce changement sera réinitialisé après un redémarrage du système. Pour que votre modification reste permanente, ajoutez ou mettez à jour la valeur `vm.max_map_count` dans **/etc/sysctl.conf** :
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. Clonez le dépôt :
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. Démarrez le serveur en utilisant les images Docker préconstruites :
> [!CAUTION]
> Toutes les images Docker sont construites pour les plateformes x86. Nous ne proposons pas actuellement d'images Docker pour ARM64.
> Si vous êtes sur une plateforme ARM64, suivez [ce guide](https://ragflow.io/docs/dev/build_docker_image) pour construire une image Docker compatible avec votre système.
> La commande ci-dessous télécharge l'édition `v0.25.6` de l'image Docker RAGFlow. Consultez le tableau suivant pour les descriptions des différentes éditions de RAGFlow. Pour télécharger une édition de RAGFlow différente de `v0.25.6`, mettez à jour la variable `RAGFLOW_IMAGE` dans **docker/.env** avant d'utiliser `docker compose` pour démarrer le serveur.
```bash
$ cd ragflow/docker
# git checkout v0.25.6
# Optionnel : utiliser un tag stable (voir les versions : https://github.com/infiniflow/ragflow/releases)
# Cette étape garantit que le fichier **entrypoint.sh** dans le code correspond à la version de l'image Docker.
# Use CPU for DeepDoc tasks:
$ docker compose -f docker-compose.yml up -d
# To use GPU to accelerate DeepDoc tasks:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> Remarque : Avant `v0.22.0`, nous fournissions à la fois des images avec des modèles d'embedding et des images slim sans modèles d'embedding. Détails ci-dessous :
| RAGFlow image tag | Image size (GB) | Has embedding models? | Stable? |
|-------------------|-----------------|-----------------------|----------------|
| v0.21.1 | &approx;9 | ✔️ | Stable release |
| v0.21.1-slim | &approx;2 | ❌ | Stable release |
> À partir de `v0.22.0`, nous ne distribuons que l'édition slim et ne rajoutons plus le suffixe **-slim** au tag d'image.
4. Vérifiez l'état du serveur après son démarrage :
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_La sortie suivante confirme un lancement réussi du système :_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> Si vous sautez cette étape de confirmation et vous connectez directement à RAGFlow, votre navigateur peut afficher une erreur `network abnormal`, car à ce moment-là, votre RAGFlow peut ne pas être entièrement initialisé.
>
5. Dans votre navigateur web, entrez l'adresse IP de votre serveur et connectez-vous à RAGFlow.
> Avec les paramètres par défaut, il vous suffit d'entrer `http://IP_OF_YOUR_MACHINE` (**sans** numéro de port), car le port HTTP par défaut `80` peut être omis lors de l'utilisation des configurations par défaut.
>
6. Dans [service_conf.yaml.template](./docker/service_conf.yaml.template), sélectionnez la fabrique LLM souhaitée dans `user_default_llm` et mettez à jour le champ `API_KEY` avec la clé API correspondante.
> Voir [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) pour plus d'informations.
>
_Le spectacle commence !_
## 🔧 Configurations
En ce qui concerne les configurations système, vous devrez gérer les fichiers suivants :
- [.env](./docker/.env) : Conserve les paramètres de base du système, tels que `SVR_HTTP_PORT`, `MYSQL_PASSWORD` et `MINIO_PASSWORD`.
- [service_conf.yaml.template](./docker/service_conf.yaml.template) : Configure les services back-end. Les variables d'environnement dans ce fichier seront automatiquement renseignées au démarrage du conteneur Docker. Toutes les variables d'environnement définies dans le conteneur Docker seront disponibles, vous permettant de personnaliser le comportement du service en fonction de l'environnement de déploiement.
- [docker-compose.yml](./docker/docker-compose.yml) : Le système s'appuie sur [docker-compose.yml](./docker/docker-compose.yml) pour démarrer.
> Le fichier [./docker/README](./docker/README.md) fournit une description détaillée des paramètres d'environnement et des configurations de services qui peuvent être utilisés comme `${ENV_VARS}` dans le fichier [service_conf.yaml.template](./docker/service_conf.yaml.template).
Pour mettre à jour le port HTTP de service par défaut (80), accédez à [docker-compose.yml](./docker/docker-compose.yml) et changez `80:80` en `<YOUR_SERVING_PORT>:80`.
Les mises à jour des configurations ci-dessus nécessitent un redémarrage de tous les conteneurs pour prendre effet :
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### Passer du moteur de documents Elasticsearch à Infinity
RAGFlow utilise Elasticsearch par défaut pour stocker le texte intégral et les vecteurs. Pour passer à [Infinity](https://github.com/infiniflow/infinity/), suivez ces étapes :
1. Arrêtez tous les conteneurs en cours d'exécution :
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` supprimera les volumes des conteneurs Docker, et les données existantes seront effacées.
2. Définissez `DOC_ENGINE` dans **docker/.env** sur `infinity`.
3. Démarrez les conteneurs :
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> Le passage à Infinity sur une machine Linux/arm64 n'est pas encore officiellement pris en charge.
## 🔧 Construire une image Docker
Cette image fait environ 2 Go et dépend de services LLM et d'embedding externes.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
Ou si vous êtes derrière un proxy, vous pouvez passer des arguments de proxy :
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://YOUR_PROXY:PORT \
--build-arg https_proxy=http://YOUR_PROXY:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 Lancer le service depuis les sources pour le développement
1. Installez `uv` et `pre-commit`, ou ignorez cette étape s'ils sont déjà installés :
```bash
pipx install uv pre-commit
```
2. Clonez le code source et installez les dépendances Python :
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. Lancez les services dépendants (MinIO, Elasticsearch, Redis et MySQL) avec Docker Compose :
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
Ajoutez la ligne suivante à `/etc/hosts` pour résoudre tous les hôtes spécifiés dans **docker/.env** vers `127.0.0.1` :
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. Si vous ne pouvez pas accéder à HuggingFace, définissez la variable d'environnement `HF_ENDPOINT` pour utiliser un site miroir :
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. Si votre système d'exploitation n'a pas jemalloc, installez-le comme suit :
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. Lancez le service back-end :
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. Installez les dépendances front-end :
```bash
cd web
npm install
```
8. Lancez le service front-end :
```bash
npm run dev
```
_La sortie suivante confirme un lancement réussi du système :_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. Arrêtez les services front-end et back-end de RAGFlow une fois le développement terminé :
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 Documentation
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
## 📜 Roadmap
Voir la [Feuille de route RAGFlow 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 Communauté
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contribuer
RAGFlow s'épanouit grâce à la collaboration open-source. Dans cet esprit, nous accueillons des contributions diverses de la communauté.
Si vous souhaitez en faire partie, consultez d'abord nos [Directives de contribution](https://ragflow.io/docs/dev/contributing).

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="Logo ragflow">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體中文版自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DBEDFA"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="Ikuti di X (Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Lencana Daring" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Lencana Daring" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Rilis%20Terbaru" alt="Rilis Terbaru">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Dokumentasi</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Peta Jalan</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Daftar Isi </b> </summary>
- 💡 [Apa Itu RAGFlow?](#-apa-itu-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Mulai](#-mulai)
- 📌 [Pembaruan Terbaru](#-pembaruan-terbaru)
- 🌟 [Fitur Utama](#-fitur-utama)
- 🔎 [Arsitektur Sistem](#-arsitektur-sistem)
- 🎬 [Mulai](#-mulai)
- 🎬 [Pengelolaan Mandiri](#-pengelolaan-mandiri)
- 🔧 [Konfigurasi](#-konfigurasi)
- 🔧 [Membangun Image Docker](#-membangun-docker-image)
- 🔨 [Meluncurkan aplikasi dari Sumber untuk Pengembangan](#-meluncurkan-aplikasi-dari-sumber-untuk-pengembangan)
@@ -74,9 +76,9 @@
[RAGFlow](https://ragflow.io/) adalah mesin [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source terkemuka yang mengintegrasikan teknologi RAG mutakhir dengan kemampuan Agent untuk menciptakan lapisan kontekstual superior bagi LLM. Menyediakan alur kerja RAG yang efisien dan dapat diadaptasi untuk perusahaan segala skala. Didukung oleh mesin konteks terkonvergensi dan template Agent yang telah dipra-bangun, RAGFlow memungkinkan pengembang mengubah data kompleks menjadi sistem AI kesetiaan-tinggi dan siap-produksi dengan efisiensi dan presisi yang luar biasa.
## 🎮 Demo
## 🎮 Mulai
Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
Coba layanan cloud kami di [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Pembaruan Terbaru
- 2026-04-24 Mendukung DeepSeek v4.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Menyediakan skill resmi untuk mengakses dataset RAGFlow melalui OpenClaw.
- 2025-12-26 Mendukung 'Memori' untuk agen AI.
- 2025-11-19 Mendukung Gemini 3 Pro.
- 2025-11-12 Mendukung sinkronisasi data dari Confluence, S3, Notion, Discord, Google Drive.
@@ -140,7 +144,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Mulai
## 🎬 Pengelolaan Mandiri
### 📝 Prasyarat
@@ -148,6 +152,7 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): Hanya diperlukan jika Anda ingin menggunakan fitur eksekutor kode (sandbox) dari RAGFlow.
> [!TIP]
@@ -188,12 +193,12 @@ Coba demo kami di [https://demo.ragflow.io](https://demo.ragflow.io).
> Semua gambar Docker dibangun untuk platform x86. Saat ini, kami tidak menawarkan gambar Docker untuk ARM64.
> Jika Anda menggunakan platform ARM64, [silakan gunakan panduan ini untuk membangun gambar Docker yang kompatibel dengan sistem Anda](https://ragflow.io/docs/dev/build_docker_image).
> Perintah di bawah ini mengunduh edisi v0.24.0 dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.24.0, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server.
> Perintah di bawah ini mengunduh edisi v0.25.6 dari gambar Docker RAGFlow. Silakan merujuk ke tabel berikut untuk deskripsi berbagai edisi RAGFlow. Untuk mengunduh edisi RAGFlow yang berbeda dari v0.25.6, perbarui variabel RAGFLOW_IMAGE di docker/.env sebelum menggunakan docker compose untuk memulai server.
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# Opsional: gunakan tag stabil (lihat releases: https://github.com/infiniflow/ragflow/releases)
# This steps ensures the **entrypoint.sh** file in the code matches the Docker image version.
@@ -298,8 +303,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. Jalankan aplikasi yang diperlukan (MinIO, Elasticsearch, Redis, dan MySQL) menggunakan Docker Compose:
@@ -361,8 +366,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -373,7 +378,7 @@ Lihat [Roadmap RAGFlow 2026](https://github.com/infiniflow/ragflow/issues/12241)
## 🏄 Komunitas
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Kontribusi

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體中文版自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DBEDFA"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,9 +57,9 @@
[RAGFlow](https://ragflow.io/) は、先進的な[RAG](https://ragflow.io/basics/what-is-rag)Retrieval-Augmented Generation技術と Agent 機能を融合し、大規模言語モデルLLMに優れたコンテキスト層を構築する最先端のオープンソース RAG エンジンです。あらゆる規模の企業に対応可能な合理化された RAG ワークフローを提供し、統合型[コンテキストエンジン](https://ragflow.io/basics/what-is-agent-context-engine)と事前構築されたAgentテンプレートにより、開発者が複雑なデータを驚異的な効率性と精度で高精細なプロダクションレディAIシステムへ変換することを可能にします。
## 🎮 Demo
## 🎮 はじめに
デモをお試しください:[https://demo.ragflow.io](https://demo.ragflow.io)。
当社のクラウドサービスをぜひお試しください:[https://cloud.ragflow.io](https://cloud.ragflow.io)。
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -66,6 +68,8 @@
## 🔥 最新情報
- 2026-04-24 DeepSeek v4 をサポート。
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw経由でRAGFlowデータセットにアクセスする公式スキルを提供。
- 2025-12-26 AIエージェントの「メモリ」機能をサポート。
- 2025-11-19 Gemini 3 Proをサポートしています。
- 2025-11-12 Confluence、S3、Notion、Discord、Google Drive からのデータ同期をサポートします。
@@ -121,7 +125,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 初期設定
## 🎬 セルフホスティング
### 📝 必要条件
@@ -129,6 +133,7 @@
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): RAGFlowのコード実行サンドボックス機能を利用する場合のみ必要です。
> [!TIP]
@@ -168,12 +173,12 @@
> 現在、公式に提供されているすべての Docker イメージは x86 アーキテクチャ向けにビルドされており、ARM64 用の Docker イメージは提供されていません。
> ARM64 アーキテクチャのオペレーティングシステムを使用している場合は、[このドキュメント](https://ragflow.io/docs/dev/build_docker_image)を参照して Docker イメージを自分でビルドしてください。
> 以下のコマンドは、RAGFlow Docker イメージの v0.24.0 エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.24.0 とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。
> 以下のコマンドは、RAGFlow Docker イメージの v0.25.6 エディションをダウンロードします。異なる RAGFlow エディションの説明については、以下の表を参照してください。v0.25.6 とは異なるエディションをダウンロードするには、docker/.env ファイルの RAGFLOW_IMAGE 変数を適宜更新し、docker compose を使用してサーバーを起動してください。
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# 任意: 安定版タグを利用 (一覧: https://github.com/infiniflow/ragflow/releases)
# この手順は、コード内の entrypoint.sh ファイルが Docker イメージのバージョンと一致していることを確認します。
@@ -298,8 +303,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. Docker Compose を使用して依存サービスMinIO、Elasticsearch、Redis、MySQLを起動する:
@@ -361,8 +366,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -373,7 +378,7 @@ docker build --platform linux/amd64 \
## 🏄 コミュニティ
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 コントリビュート

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DBEDFA"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -56,9 +58,9 @@
[RAGFlow](https://ragflow.io/) 는 최첨단 [RAG](https://ragflow.io/basics/what-is-rag)(Retrieval-Augmented Generation)와 Agent 기능을 융합하여 대규모 언어 모델(LLM)을 위한 우수한 컨텍스트 계층을 생성하는 선도적인 오픈소스 RAG 엔진입니다. 모든 규모의 기업에 적용 가능한 효율적인 RAG 워크플로를 제공하며, 통합 [컨텍스트 엔진](https://ragflow.io/basics/what-is-agent-context-engine)과 사전 구축된 Agent 템플릿을 통해 개발자들이 복잡한 데이터를 예외적인 효율성과 정밀도로 고급 구현도의 프로덕션 준비 완료 AI 시스템으로 변환할 수 있도록 지원합니다.
## 🎮 데모
## 🎮 시작하기
데모를 [https://demo.ragflow.io](https://demo.ragflow.io)에서 실행해 보세요.
[https://cloud.ragflow.io](https://cloud.ragflow.io)에서 저희 클라우드 서비스를 이용해 보세요.
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -67,6 +69,8 @@
## 🔥 업데이트
- 2026-04-24 DeepSeek v4를 지원합니다.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw를 통해 RAGFlow 데이터셋에 접근하는 공식 스킬 제공.
- 2025-12-26 AI 에이전트의 '메모리' 기능 지원.
- 2025-11-19 Gemini 3 Pro를 지원합니다.
- 2025-11-12 Confluence, S3, Notion, Discord, Google Drive에서 데이터 동기화를 지원합니다.
@@ -122,7 +126,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 시작하기
## 🎬 자체 호스팅
### 📝 사전 준비 사항
@@ -130,6 +134,7 @@
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): RAGFlow의 코드 실행기(샌드박스) 기능을 사용하려는 경우에만 필요합니다.
> [!TIP]
@@ -170,12 +175,12 @@
> 모든 Docker 이미지는 x86 플랫폼을 위해 빌드되었습니다. 우리는 현재 ARM64 플랫폼을 위한 Docker 이미지를 제공하지 않습니다.
> ARM64 플랫폼을 사용 중이라면, [시스템과 호환되는 Docker 이미지를 빌드하려면 이 가이드를 사용해 주세요](https://ragflow.io/docs/dev/build_docker_image).
> 아래 명령어는 RAGFlow Docker 이미지의 v0.24.0 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.24.0과 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오.
> 아래 명령어는 RAGFlow Docker 이미지의 v0.25.6 버전을 다운로드합니다. 다양한 RAGFlow 버전에 대한 설명은 다음 표를 참조하십시오. v0.25.6와 다른 RAGFlow 버전을 다운로드하려면, docker/.env 파일에서 RAGFLOW_IMAGE 변수를 적절히 업데이트한 후 docker compose를 사용하여 서버를 시작하십시오.
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# Optional: use a stable tag (see releases: https://github.com/infiniflow/ragflow/releases)
# 이 단계는 코드의 entrypoint.sh 파일이 Docker 이미지 버전과 일치하도록 보장합니다.
@@ -293,8 +298,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
@@ -365,8 +370,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -377,7 +382,7 @@ docker build --platform linux/amd64 \
## 🏄 커뮤니티
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 컨트리뷰션

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DBEDFA"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="seguir no X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Badge Estático" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Badge Estático" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Última%20Relese" alt="Última Versão">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Documentação</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 Índice</b></summary>
- 💡 [O que é o RAGFlow?](#-o-que-é-o-ragflow)
- 🎮 [Demo](#-demo)
- 🎮 [Primeiros Passos](#-primeiros-passos)
- 📌 [Últimas Atualizações](#-últimas-atualizações)
- 🌟 [Principais Funcionalidades](#-principais-funcionalidades)
- 🔎 [Arquitetura do Sistema](#-arquitetura-do-sistema)
- 🎬 [Primeiros Passos](#-primeiros-passos)
- 🎬 [Auto-hospedagem](#-auto-hospedagem)
- 🔧 [Configurações](#-configurações)
- 🔧 [Construir uma imagem docker sem incorporar modelos](#-construir-uma-imagem-docker-sem-incorporar-modelos)
- 🔧 [Construir uma imagem docker incluindo modelos](#-construir-uma-imagem-docker-incluindo-modelos)
@@ -75,9 +77,9 @@
[RAGFlow](https://ragflow.io/) é um mecanismo de [RAG](https://ragflow.io/basics/what-is-rag) (Retrieval-Augmented Generation) open-source líder que fusiona tecnologias RAG de ponta com funcionalidades Agent para criar uma camada contextual superior para LLMs. Oferece um fluxo de trabalho RAG otimizado adaptável a empresas de qualquer escala. Alimentado por [um motor de contexto](https://ragflow.io/basics/what-is-agent-context-engine) convergente e modelos Agent pré-construídos, o RAGFlow permite que desenvolvedores transformem dados complexos em sistemas de IA de alta fidelidade e pronto para produção com excepcional eficiência e precisão.
## 🎮 Demo
## 🎮 Primeiros Passos
Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
Experimente o nosso serviço na nuvem em [https://cloud.ragflow.io](https://cloud.ragflow.io).
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -86,6 +88,8 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
## 🔥 Últimas Atualizações
- 24-04-2026 Suporta DeepSeek v4.
- 24-03-2026 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — Fornece um skill oficial para acessar datasets do RAGFlow via OpenClaw.
- 26-12-2025 Suporte à função 'Memória' para agentes de IA.
- 19-11-2025 Suporta Gemini 3 Pro.
- 12-11-2025 Suporta a sincronização de dados do Confluence, S3, Notion, Discord e Google Drive.
@@ -141,7 +145,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Primeiros Passos
## 🎬 Auto-hospedagem
### 📝 Pré-requisitos
@@ -149,6 +153,7 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
- RAM >= 16 GB
- Disco >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): Necessário apenas se você pretende usar o recurso de executor de código (sandbox) do RAGFlow.
> [!TIP]
@@ -188,12 +193,12 @@ Experimente nossa demo em [https://demo.ragflow.io](https://demo.ragflow.io).
> Todas as imagens Docker são construídas para plataformas x86. Atualmente, não oferecemos imagens Docker para ARM64.
> Se você estiver usando uma plataforma ARM64, por favor, utilize [este guia](https://ragflow.io/docs/dev/build_docker_image) para construir uma imagem Docker compatível com o seu sistema.
> O comando abaixo baixa a edição`v0.24.0` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.24.0`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor.
> O comando abaixo baixa a edição`v0.25.6` da imagem Docker do RAGFlow. Consulte a tabela a seguir para descrições de diferentes edições do RAGFlow. Para baixar uma edição do RAGFlow diferente da `v0.25.6`, atualize a variável `RAGFLOW_IMAGE` conforme necessário no **docker/.env** antes de usar `docker compose` para iniciar o servidor.
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# Opcional: use uma tag estável (veja releases: https://github.com/infiniflow/ragflow/releases)
# Esta etapa garante que o arquivo entrypoint.sh no código corresponda à versão da imagem do Docker.
@@ -315,8 +320,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # instala os módulos Python dependentes do RAGFlow
uv run download_deps.py
uv sync --python 3.13 # instala os módulos Python dependentes do RAGFlow
uv run python3 download_deps.py
pre-commit install
```
3. Inicie os serviços dependentes (MinIO, Elasticsearch, Redis e MySQL) usando Docker Compose:
@@ -378,8 +383,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -390,7 +395,7 @@ Veja o [RAGFlow Roadmap 2026](https://github.com/infiniflow/ragflow/issues/12241
## 🏄 Comunidade
- [Discord](https://discord.gg/NjYzJD3GM3)
- [Twitter](https://twitter.com/infiniflowai)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 Contribuindo

410
README_tr.md Normal file
View File

@@ -0,0 +1,410 @@
<div align="center">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="520" alt="ragflow logo">
</a>
</div>
<p align="center">
<a href="./README.md"><img alt="README in English" src="https://img.shields.io/badge/English-DFE0E5"></a>
<a href="./README_zh.md"><img alt="简体中文版自述文件" src="https://img.shields.io/badge/简体中文-DFE0E5"></a>
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DBEDFA"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="X(Twitter)'da takip et">
</a>
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Çevrimiçi Demo" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Son%20Sürüm" alt="Son Sürüm">
</a>
<a href="https://github.com/infiniflow/ragflow/blob/main/LICENSE">
<img height="21" src="https://img.shields.io/badge/Lisans-Apache--2.0-ffffff?labelColor=d4eaf7&color=2e6cc4" alt="lisans">
</a>
<a href="https://deepwiki.com/infiniflow/ragflow">
<img alt="Ask DeepWiki" src="https://deepwiki.com/badge.svg">
</a>
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Dokümantasyon</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Yol Haritası</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/ragflow-octoverse.png" width="1200"/>
</div>
<div align="center">
<a href="https://trendshift.io/repositories/9064" target="_blank"><img src="https://trendshift.io/api/badge/repositories/9064" alt="infiniflow%2Fragflow | Trendshift" style="width: 250px; height: 55px;" width="250" height="55"/></a>
</div>
<details open>
<summary><b>📕 İçindekiler</b></summary>
- 💡 [RAGFlow Nedir?](#-ragflow-nedir)
- 🎮 [Başlarken](#-başlarken)
- 📌 [Son Güncellemeler](#-son-güncellemeler)
- 🌟 [Temel Özellikler](#-temel-özellikler)
- 🔎 [Sistem Mimarisi](#-sistem-mimarisi)
- 🎬 [Kendi Sunucusunda Barındırma](#-kendi-sunucusunda-barındırma)
- 🔧 [Yapılandırmalar](#-yapılandırmalar)
- 🔧 [Docker İmajı Oluşturma](#-docker-i̇majı-oluşturma)
- 🔨 [Geliştirme İçin Kaynaktan Hizmet Başlatma](#-geliştirme-i̇çin-kaynaktan-hizmet-başlatma)
- 📚 [Dokümantasyon](#-dokümantasyon)
- 📜 [Yol Haritası](#-yol-haritası)
- 🏄 [Topluluk](#-topluluk)
- 🙌 [Katkıda Bulunma](#-katkıda-bulunma)
</details>
## 💡 RAGFlow Nedir?
[RAGFlow](https://ragflow.io/), derin doküman anlayışına dayalı, açık kaynaklı ve öncü bir Artırılmış Üretim ile Bilgi Erişimi ([RAG](https://ragflow.io/basics/what-is-rag)) motorudur. En son RAG teknolojisini Ajan yetenekleriyle birleştirerek LLM'ler için üstün bir bağlam katmanı oluşturur. Her ölçekteki kuruluşa uyarlanabilir, kolaylaştırılmış bir RAG iş akışı sunar. Yakınsanmış bir [bağlam motoru](https://ragflow.io/basics/what-is-agent-context-engine) ve hazır ajan şablonlarıyla donatılmış RAGFlow, geliştiricilerin karmaşık verileri yüksek doğrulukta, üretime hazır yapay zeka sistemlerine olağanüstü verimlilik ve hassasiyetle dönüştürmesini sağlar.
## 🎮 Başlarken
Bulut hizmetimizi [https://cloud.ragflow.io](https://cloud.ragflow.io) adresinden deneyin.
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/agentic-dark.gif" width="1200"/>
</div>
## 🔥 Son Güncellemeler
- 2026-04-24 DeepSeek v4 desteği.
- 2026-03-24 [RAGFlow Skill on OpenClaw](https://clawhub.ai/yingfeng/ragflow-skill) — OpenClaw üzerinden RAGFlow veri setlerine erişmek için resmi bir skill sağlar.
- 2025-12-26 Yapay zeka ajanı için 'Bellek' desteği eklendi.
- 2025-11-19 Gemini 3 Pro desteği eklendi.
- 2025-11-12 Confluence, S3, Notion, Discord, Google Drive'dan veri senkronizasyonu desteği eklendi.
- 2025-10-23 Doküman ayrıştırma yöntemi olarak MinerU ve Docling desteği eklendi.
- 2025-10-15 Düzenlenebilir veri alım hattı desteği eklendi.
- 2025-08-08 OpenAI'ın en yeni GPT-5 serisi modelleri için destek eklendi.
- 2025-08-01 Ajanlı iş akışı ve MCP desteği eklendi.
- 2025-05-23 Ajana Python/JavaScript kod çalıştırıcı bileşeni eklendi.
- 2025-05-05 Diller arası sorgu desteği eklendi.
- 2025-03-19 PDF veya DOCX dosyalarındaki görselleri yorumlamak için çok modlu model desteği eklendi.
## 🎉 Bizi Takip Edin
⭐️ Heyecan verici yeni özellikler ve iyileştirmelerden haberdar olmak için depomuzı yıldızlayın! Yeni sürümler için anında bildirim alın! 🌟
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/18c9707e-b8aa-4caf-a154-037089c105ba" width="1200"/>
</div>
## 🌟 Temel Özellikler
### 🍭 **"Kaliteli girdi, kaliteli çıktı"**
- Karmaşık formatlara sahip yapılandırılmamış verilerden [derin doküman anlayışı](./deepdoc/README.md) tabanlı bilgi çıkarımı.
- Kelimenin tam anlamıyla sınırsız token içinde "samanlıkta iğne bulma" yeteneği.
### 🍱 **Şablon tabanlı parçalama**
- Akıllı ve açıklanabilir.
- Aralarından seçim yapabileceğiniz çok sayıda şablon seçeneği.
### 🌱 **Azaltılmış halüsinasyonlarla temellendirilmiş alıntılar**
- İnsan müdahalesine olanak tanıyan metin parçalama görselleştirmesi.
- Temellendirilmiş yanıtları desteklemek için anahtar referansların hızlı görüntülenmesi ve izlenebilir alıntılar.
### 🍔 **Heterojen veri kaynaklarıyla uyumluluk**
- Word, slaytlar, Excel, txt, görseller, taranmış kopyalar, yapılandırılmış veriler, web sayfaları ve daha fazlasını destekler.
### 🛀 **Otomatik ve zahmetsiz RAG iş akışı**
- Hem bireysel hem de büyük işletmeler için özelleştirilmiş kolaylaştırılmış RAG düzenlemesi.
- Yapılandırılabilir LLM'ler ve gömme (embedding) modelleri.
- Birleştirilmiş yeniden sıralama ile çoklu geri çağırma.
- İş süreçlerine sorunsuz entegrasyon için sezgisel API'ler.
## 🔎 Sistem Mimarisi
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 Kendi Sunucusunda Barındırma
### 📝 Ön Koşullar
- CPU >= 4 çekirdek
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): Yalnızca RAGFlow'un kod çalıştırıcı (sandbox) özelliğini kullanmayı planlıyorsanız gereklidir.
> [!TIP]
> Yerel makinenize (Windows, Mac veya Linux) Docker yüklemediyseniz, [Docker Engine Kurulumu](https://docs.docker.com/engine/install/) sayfasına bakın.
### 🚀 Sunucuyu Başlatma
1. `vm.max_map_count` değerinin >= 262144 olduğundan emin olun:
> `vm.max_map_count` değerini kontrol etmek için:
>
> ```bash
> $ sysctl vm.max_map_count
> ```
>
> Değer 262144'ten düşükse, en az 262144 olarak ayarlayın.
>
> ```bash
> # Bu örnekte 262144 olarak ayarlıyoruz:
> $ sudo sysctl -w vm.max_map_count=262144
> ```
>
> Bu değişiklik sistem yeniden başlatıldığında sıfırlanacaktır. Değişikliğin kalıcı olmasını sağlamak için
> **/etc/sysctl.conf** dosyasındaki `vm.max_map_count` değerini buna göre ekleyin veya güncelleyin:
>
> ```bash
> vm.max_map_count=262144
> ```
>
2. Depoyu klonlayın:
```bash
$ git clone https://github.com/infiniflow/ragflow.git
```
3. Önceden oluşturulmuş Docker imajlarını kullanarak sunucuyu başlatın:
> [!CAUTION]
> Tüm Docker imajları x86 platformları için oluşturulmuştur. Şu anda ARM64 için Docker imajı sunmuyoruz.
> ARM64 platformundaysanız, sisteminizle uyumlu bir Docker imajı oluşturmak için [bu kılavuzu](https://ragflow.io/docs/dev/build_docker_image) takip edin.
> Aşağıdaki komut RAGFlow Docker imajının `v0.25.6` sürümünü indirir. Farklı RAGFlow sürümleri için aşağıdaki tabloya bakın. `v0.25.6` dışında bir sürüm indirmek için, `docker compose` ile sunucuyu başlatmadan önce **docker/.env** dosyasındaki `RAGFLOW_IMAGE` değişkenini güncelleyin.
```bash
$ cd ragflow/docker
# git checkout v0.25.6
# İsteğe bağlı: Kararlı bir etiket kullanın (sürümler: https://github.com/infiniflow/ragflow/releases)
# Bu adım, koddaki **entrypoint.sh** dosyasının Docker imaj sürümüyle eşleşmesini sağlar.
# DeepDoc görevleri için CPU kullanımı:
$ docker compose -f docker-compose.yml up -d
# DeepDoc görevlerini hızlandırmak için GPU kullanımı:
# sed -i '1i DEVICE=gpu' .env
# docker compose -f docker-compose.yml up -d
```
> Not: `v0.22.0` öncesinde hem gömme modelleri içeren imajlar hem de gömme modelleri içermeyen ince (slim) imajlar sunuyorduk. Detaylar aşağıdadır:
| RAGFlow imaj etiketi | İmaj boyutu (GB) | Gömme modelleri var mı? | Kararlı mı? |
|-----------------------|-------------------|-------------------------|-----------------|
| v0.21.1 | &approx;9 | ✔️ | Kararlı sürüm |
| v0.21.1-slim | &approx;2 | ❌ | Kararlı sürüm |
> `v0.22.0`'dan itibaren yalnızca ince (slim) sürümü sunuyoruz ve imaj etiketine artık **-slim** son eki eklemiyoruz.
4. Sunucu çalışır duruma geldikten sonra sunucu durumunu kontrol edin:
```bash
$ docker logs -f docker-ragflow-cpu-1
```
_Aşağıdaki çıktı, sistemin başarıyla başlatıldığını onaylar:_
```bash
____ ___ ______ ______ __
/ __ \ / | / ____// ____// /____ _ __
/ /_/ // /| | / / __ / /_ / // __ \| | /| / /
/ _, _// ___ |/ /_/ // __/ / // /_/ /| |/ |/ /
/_/ |_|/_/ |_|\____//_/ /_/ \____/ |__/|__/
* Running on all addresses (0.0.0.0)
```
> Bu onay adımını atlayıp doğrudan RAGFlow'a giriş yaparsanız, o anda RAGFlow tam olarak başlatılmamış olabileceğinden
> tarayıcınız `ağ hatası` uyarısı verebilir.
>
5. Web tarayıcınıza sunucunuzun IP adresini girin ve RAGFlow'a giriş yapın.
> Varsayılan ayarlarla, yalnızca `http://MAKİNENİZİN_IP_ADRESİ` girmeniz yeterlidir (port numarası **gerekmez**),
> çünkü varsayılan HTTP sunucu portu `80` varsayılan yapılandırmalar kullanıldığında ihmal edilebilir.
>
6. [service_conf.yaml.template](./docker/service_conf.yaml.template) dosyasında, `user_default_llm` içinde istediğiniz LLM sağlayıcısını seçin ve
`API_KEY` alanını ilgili API anahtarıyla güncelleyin.
> Daha fazla bilgi için [llm_api_key_setup](https://ragflow.io/docs/dev/llm_api_key_setup) sayfasına bakın.
>
_Gösteri başlasın!_
## 🔧 Yapılandırmalar
Sistem yapılandırmaları söz konusu olduğunda, aşağıdaki dosyaları yönetmeniz gerekecektir:
- [.env](./docker/.env): `SVR_HTTP_PORT`, `MYSQL_PASSWORD` ve `MINIO_PASSWORD` gibi temel sistem ayarlarını içerir.
- [service_conf.yaml.template](./docker/service_conf.yaml.template): Arka uç hizmetlerini yapılandırır. Bu dosyadaki ortam değişkenleri, Docker konteyneri başladığında otomatik olarak doldurulacaktır. Docker konteyneri içinde ayarlanan tüm ortam değişkenleri kullanıma hazır olacak ve hizmet davranışını dağıtım ortamına göre özelleştirmenize olanak tanıyacaktır.
- [docker-compose.yml](./docker/docker-compose.yml): Sistem, başlatılmak için [docker-compose.yml](./docker/docker-compose.yml) dosyasına dayanır.
> [./docker/README](./docker/README.md) dosyası, [service_conf.yaml.template](./docker/service_conf.yaml.template) dosyasında `${ENV_VARS}` olarak kullanılabilen ortam ayarları ve hizmet yapılandırmalarının ayrıntılı bir açıklamasını sağlar.
Varsayılan HTTP sunucu portunu (80) değiştirmek için [docker-compose.yml](./docker/docker-compose.yml) dosyasında `80:80` ifadesini `<SUNUCU_PORTUNUZ>:80` olarak değiştirin.
Yukarıdaki yapılandırma değişikliklerinin etkili olması için tüm konteynerlerin yeniden başlatılması gerekir:
> ```bash
> $ docker compose -f docker-compose.yml up -d
> ```
### Doküman Motorunu Elasticsearch'ten Infinity'ye Geçirme
RAGFlow varsayılan olarak tam metin ve vektörlerin depolanması için Elasticsearch kullanır. [Infinity](https://github.com/infiniflow/infinity/)'ye geçmek için şu adımları izleyin:
1. Çalışan tüm konteynerleri durdurun:
```bash
$ docker compose -f docker/docker-compose.yml down -v
```
> [!WARNING]
> `-v` seçeneği Docker konteyner birimlerini silecek ve mevcut veriler temizlenecektir.
2. **docker/.env** dosyasında `DOC_ENGINE` değerini `infinity` olarak ayarlayın.
3. Konteynerleri başlatın:
```bash
$ docker compose -f docker-compose.yml up -d
```
> [!WARNING]
> Linux/arm64 makinesinde Infinity'ye geçiş henüz resmi olarak desteklenmemektedir.
## 🔧 Docker İmajı Oluşturma
Bu imaj yaklaşık 2 GB boyutundadır ve harici LLM ile gömme hizmetlerine bağlıdır.
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
docker build --platform linux/amd64 -f Dockerfile -t infiniflow/ragflow:nightly .
```
Veya bir proxy arkasındaysanız, proxy parametrelerini iletebilirsiniz:
```bash
docker build --platform linux/amd64 \
--build-arg http_proxy=http://PROXY_ADRESINIZ:PORT \
--build-arg https_proxy=http://PROXY_ADRESINIZ:PORT \
-f Dockerfile -t infiniflow/ragflow:nightly .
```
## 🔨 Geliştirme İçin Kaynaktan Hizmet Başlatma
1. `uv` ve `pre-commit` yükleyin veya zaten yüklüyse bu adımı atlayın:
```bash
pipx install uv pre-commit
```
2. Kaynak kodunu klonlayın ve Python bağımlılıklarını yükleyin:
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.13 # RAGFlow'un bağımlı Python modüllerini yükler
uv run python3 download_deps.py
pre-commit install
```
3. Bağımlı hizmetleri (MinIO, Elasticsearch, Redis ve MySQL) Docker Compose kullanarak başlatın:
```bash
docker compose -f docker/docker-compose-base.yml up -d
```
**docker/.env** dosyasında belirtilen tüm ana bilgisayar adlarını `127.0.0.1`'e çözümlemek için `/etc/hosts` dosyasına aşağıdaki satırı ekleyin:
```
127.0.0.1 es01 infinity mysql minio redis sandbox-executor-manager
```
4. HuggingFace'e erişemiyorsanız, bir ayna site kullanmak için `HF_ENDPOINT` ortam değişkenini ayarlayın:
```bash
export HF_ENDPOINT=https://hf-mirror.com
```
5. İşletim sisteminizde jemalloc yoksa, aşağıdaki şekilde yükleyin:
```bash
# Ubuntu
sudo apt-get install libjemalloc-dev
# CentOS
sudo yum install jemalloc
# OpenSUSE
sudo zypper install jemalloc
# macOS
sudo brew install jemalloc
```
6. Arka uç hizmetini başlatın:
```bash
source .venv/bin/activate
export PYTHONPATH=$(pwd)
bash docker/launch_backend_service.sh
```
7. Ön yüz bağımlılıklarını yükleyin:
```bash
cd web
npm install
```
8. Ön yüz hizmetini başlatın:
```bash
npm run dev
```
_Aşağıdaki çıktı, sistemin başarıyla başlatıldığını onaylar:_
![](https://github.com/user-attachments/assets/0daf462c-a24d-4496-a66f-92533534e187)
9. Geliştirme tamamlandıktan sonra RAGFlow ön yüz ve arka uç hizmetini durdurun:
```bash
pkill -f "ragflow_server.py|task_executor.py"
```
## 📚 Dokümantasyon
- [Hızlı Başlangıç](https://ragflow.io/docs/dev/)
- [Yapılandırma](https://ragflow.io/docs/dev/configurations)
- [Sürüm Notları](https://ragflow.io/docs/dev/release_notes)
- [Kullanıcı Kılavuzları](https://ragflow.io/docs/category/user-guides)
- [Geliştirici Kılavuzları](https://ragflow.io/docs/category/developer-guides)
- [Referanslar](https://ragflow.io/docs/dev/category/references)
- [SSS](https://ragflow.io/docs/dev/faq)
## 📜 Yol Haritası
[RAGFlow Yol Haritası 2026](https://github.com/infiniflow/ragflow/issues/12241) sayfasına bakın.
## 🏄 Topluluk
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Tartışmalar](https://github.com/orgs/infiniflow/discussions)
## 🙌 Katkıda Bulunma
RAGFlow, açık kaynak iş birliği sayesinde gelişmektedir. Bu anlayışla, topluluktan gelen çeşitli katkıları benimsiyoruz.
Bir parçası olmak istiyorsanız, önce [Katkıda Bulunma Kılavuzumuzu](https://ragflow.io/docs/dev/contributing) inceleyin.

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DBEDFA"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 目錄</b></summary>
- 💡 [RAGFlow 是什麼?](#-RAGFlow-是什麼)
- 🎮 [Demo-試用](#-demo-試用)
- 🎮 [快速開始](#-快速開始)
- 📌 [近期更新](#-近期更新)
- 🌟 [主要功能](#-主要功能)
- 🔎 [系統架構](#-系統架構)
- 🎬 [快速開始](#-快速開始)
- 🎬 [自行架設](#-自行架設)
- 🔧 [系統配置](#-系統配置)
- 🔨 [以原始碼啟動服務](#-以原始碼啟動服務)
- 📚 [技術文檔](#-技術文檔)
@@ -74,9 +76,9 @@
[RAGFlow](https://ragflow.io/) 是一款領先的開源 [RAG](https://ragflow.io/basics/what-is-rag)Retrieval-Augmented Generation引擎通過融合前沿的 RAG 技術與 Agent 能力,為大型語言模型提供卓越的上下文層。它提供可適配任意規模企業的端到端 RAG 工作流,憑藉融合式[上下文引擎](https://ragflow.io/basics/what-is-agent-context-engine)與預置的 Agent 模板,助力開發者以極致效率與精度將複雜數據轉化為高可信、生產級的人工智能系統。
## 🎮 Demo 試用
## 🎮 快速開始
請登入網址 [https://demo.ragflow.io](https://demo.ragflow.io) 試用 demo
請登入網址 [https://cloud.ragflow.io](https://cloud.ragflow.io) 試用雲服務
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,6 +87,8 @@
## 🔥 近期更新
- 2026-04-24 支援 DeepSeek v4 版本。
- 2026-03-24 發布 [RAGFlow 官方 Skill](https://clawhub.ai/yingfeng/ragflow-skill) — 提供官方 Skill 以透過 OpenClaw 訪問 RAGFlow 數據集。
- 2025-12-26 支援AI代理的「記憶」功能。
- 2025-11-19 支援 Gemini 3 Pro。
- 2025-11-12 支援從 Confluence、S3、Notion、Discord、Google Drive 進行資料同步。
@@ -140,7 +144,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 快速開始
## 🎬 自行架設
### 📝 前提條件
@@ -148,6 +152,7 @@
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): 僅在您打算使用 RAGFlow 的代碼執行器(沙箱)功能時才需要安裝。
> [!TIP]
@@ -187,12 +192,12 @@
> 所有 Docker 映像檔都是為 x86 平台建置的。目前,我們不提供 ARM64 平台的 Docker 映像檔。
> 如果您使用的是 ARM64 平台,請使用 [這份指南](https://ragflow.io/docs/dev/build_docker_image) 來建置適合您系統的 Docker 映像檔。
> 執行以下指令會自動下載 RAGFlow Docker 映像 `v0.24.0`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.24.0` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。
> 執行以下指令會自動下載 RAGFlow Docker 映像 `v0.25.6`。請參考下表查看不同 Docker 發行版的說明。如需下載不同於 `v0.25.6` 的 Docker 映像,請在執行 `docker compose` 啟動服務之前先更新 **docker/.env** 檔案內的 `RAGFLOW_IMAGE` 變數。
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# 可選使用穩定版標籤查看發佈https://github.com/infiniflow/ragflow/releases
# 此步驟確保程式碼中的 entrypoint.sh 檔案與 Docker 映像版本一致。
@@ -325,8 +330,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
3. 透過 Docker Compose 啟動依賴的服務MinIO, Elasticsearch, Redis, and MySQL
@@ -392,8 +397,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -403,8 +408,8 @@ docker build --platform linux/amd64 \
## 🏄 開源社群
- [Discord](https://discord.gg/zd4qPW6t)
- [Twitter](https://twitter.com/infiniflowai)
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 貢獻指南

View File

@@ -1,5 +1,5 @@
<div align="center">
<a href="https://demo.ragflow.io/">
<a href="https://cloud.ragflow.io/">
<img src="web/src/assets/logo-with-text.svg" width="350" alt="ragflow logo">
</a>
</div>
@@ -10,19 +10,22 @@
<a href="./README_tzh.md"><img alt="繁體版中文自述文件" src="https://img.shields.io/badge/繁體中文-DFE0E5"></a>
<a href="./README_ja.md"><img alt="日本語のREADME" src="https://img.shields.io/badge/日本語-DFE0E5"></a>
<a href="./README_ko.md"><img alt="한국어" src="https://img.shields.io/badge/한국어-DFE0E5"></a>
<a href="./README_fr.md"><img alt="README en Français" src="https://img.shields.io/badge/Français-DFE0E5"></a>
<a href="./README_id.md"><img alt="Bahasa Indonesia" src="https://img.shields.io/badge/Bahasa Indonesia-DFE0E5"></a>
<a href="./README_pt_br.md"><img alt="Português(Brasil)" src="https://img.shields.io/badge/Português(Brasil)-DFE0E5"></a>
<a href="./README_ar.md"><img alt="README in Arabic" src="https://img.shields.io/badge/Arabic-DFE0E5"></a>
<a href="./README_tr.md"><img alt="Türkçe README" src="https://img.shields.io/badge/Türkçe-DFE0E5"></a>
</p>
<p align="center">
<a href="https://x.com/intent/follow?screen_name=infiniflowai" target="_blank">
<img src="https://img.shields.io/twitter/follow/infiniflow?logo=X&color=%20%23f5f5f5" alt="follow on X(Twitter)">
</a>
<a href="https://demo.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Online-Demo-4e6b99">
<a href="https://cloud.ragflow.io" target="_blank">
<img alt="Static Badge" src="https://img.shields.io/badge/Get-Started-4e6b99">
</a>
<a href="https://hub.docker.com/r/infiniflow/ragflow" target="_blank">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.24.0">
<img src="https://img.shields.io/docker/pulls/infiniflow/ragflow?label=Docker%20Pulls&color=0db7ed&logo=docker&logoColor=white&style=flat-square" alt="docker pull infiniflow/ragflow:v0.25.6">
</a>
<a href="https://github.com/infiniflow/ragflow/releases/latest">
<img src="https://img.shields.io/github/v/release/infiniflow/ragflow?color=blue&label=Latest%20Release" alt="Latest Release">
@@ -36,11 +39,10 @@
</p>
<h4 align="center">
<a href="https://cloud.ragflow.io">Cloud</a> |
<a href="https://ragflow.io/docs/dev/">Document</a> |
<a href="https://github.com/infiniflow/ragflow/issues/12241">Roadmap</a> |
<a href="https://twitter.com/infiniflowai">Twitter</a> |
<a href="https://discord.gg/NjYzJD3GM3">Discord</a> |
<a href="https://demo.ragflow.io">Demo</a>
<a href="https://discord.gg/NjYzJD3GM3">Discord</a>
</h4>
<div align="center" style="margin-top:20px;margin-bottom:20px;">
@@ -55,11 +57,11 @@
<summary><b>📕 目录</b></summary>
- 💡 [RAGFlow 是什么?](#-RAGFlow-是什么)
- 🎮 [Demo](#-demo)
- 🎮 [快速开始](#-快速开始)
- 📌 [近期更新](#-近期更新)
- 🌟 [主要功能](#-主要功能)
- 🔎 [系统架构](#-系统架构)
- 🎬 [快速开始](#-快速开始)
- 🎬 [自主托管](#-自主托管)
- 🔧 [系统配置](#-系统配置)
- 🔨 [以源代码启动服务](#-以源代码启动服务)
- 📚 [技术文档](#-技术文档)
@@ -74,9 +76,9 @@
[RAGFlow](https://ragflow.io/) 是一款领先的开源检索增强生成([RAG](https://ragflow.io/basics/what-is-rag))引擎,通过融合前沿的 RAG 技术与 Agent 能力,为大型语言模型提供卓越的上下文层。它提供可适配任意规模企业的端到端 RAG 工作流,凭借融合式[上下文引擎](https://ragflow.io/basics/what-is-agent-context-engine)与预置的 Agent 模板,助力开发者以极致效率与精度将复杂数据转化为高可信、生产级的人工智能系统。
## 🎮 Demo 试用
## 🎮 快速开始
请登录网址 [https://demo.ragflow.io](https://demo.ragflow.io) 试用 demo
请登录网址 [https://cloud.ragflow.io](https://cloud.ragflow.io) 体验云服务
<div align="center" style="margin-top:20px;margin-bottom:20px;">
<img src="https://raw.githubusercontent.com/infiniflow/ragflow-docs/refs/heads/image/image/chunking.gif" width="1200"/>
@@ -85,7 +87,9 @@
## 🔥 近期更新
- 2025-12-26 支持AI代理的“记忆”功能。
- 2026-04-24 支持 DeepSeek v4.
- 2026-03-24 发布 [RAGFlow 官方 Skill](https://clawhub.ai/yingfeng/ragflow-skill) — 提供官方 Skill 以通过 OpenClaw 访问 RAGFlow 数据集。
- 2025-12-26 支持 AI 代理的"记忆"功能。
- 2025-11-19 支持 Gemini 3 Pro。
- 2025-11-12 支持从 Confluence、S3、Notion、Discord、Google Drive 进行数据同步。
- 2025-10-23 支持 MinerU 和 Docling 作为文档解析方法。
@@ -140,7 +144,7 @@
<img src="https://github.com/user-attachments/assets/31b0dd6f-ca4f-445a-9457-70cb44a381b2" width="1000"/>
</div>
## 🎬 快速开始
## 🎬 自主托管
### 📝 前提条件
@@ -148,6 +152,7 @@
- RAM >= 16 GB
- Disk >= 50 GB
- Docker >= 24.0.0 & Docker Compose >= v2.26.1
- Python >= 3.13
- [gVisor](https://gvisor.dev/docs/user_guide/install/): 仅在你打算使用 RAGFlow 的代码执行器(沙箱)功能时才需要安装。
> [!TIP]
@@ -188,12 +193,12 @@
> 请注意,目前官方提供的所有 Docker 镜像均基于 x86 架构构建,并不提供基于 ARM64 的 Docker 镜像。
> 如果你的操作系统是 ARM64 架构,请参考[这篇文档](https://ragflow.io/docs/dev/build_docker_image)自行构建 Docker 镜像。
> 运行以下命令会自动下载 RAGFlow Docker 镜像 `v0.24.0`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.24.0` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。
> 运行以下命令会自动下载 RAGFlow Docker 镜像 `v0.25.6`。请参考下表查看不同 Docker 发行版的描述。如需下载不同于 `v0.25.6` 的 Docker 镜像,请在运行 `docker compose` 启动服务之前先更新 **docker/.env** 文件内的 `RAGFLOW_IMAGE` 变量。
```bash
$ cd ragflow/docker
# git checkout v0.24.0
# git checkout v0.25.6
# 可选使用稳定版本标签查看发布https://github.com/infiniflow/ragflow/releases
# 这一步确保代码中的 entrypoint.sh 文件与 Docker 镜像的版本保持一致。
@@ -325,8 +330,8 @@ docker build --platform linux/amd64 \
```bash
git clone https://github.com/infiniflow/ragflow.git
cd ragflow/
uv sync --python 3.12 # install RAGFlow dependent python modules
uv run download_deps.py
uv sync --python 3.13 # install RAGFlow dependent python modules
uv run python3 download_deps.py
pre-commit install
```
@@ -395,8 +400,8 @@ docker build --platform linux/amd64 \
- [Quickstart](https://ragflow.io/docs/dev/)
- [Configuration](https://ragflow.io/docs/dev/configurations)
- [Release notes](https://ragflow.io/docs/dev/release_notes)
- [User guides](https://ragflow.io/docs/dev/category/guides)
- [Developer guides](https://ragflow.io/docs/dev/category/developers)
- [User guides](https://ragflow.io/docs/category/user-guides)
- [Developer guides](https://ragflow.io/docs/category/developer-guides)
- [References](https://ragflow.io/docs/dev/category/references)
- [FAQs](https://ragflow.io/docs/dev/faq)
@@ -406,8 +411,8 @@ docker build --platform linux/amd64 \
## 🏄 开源社区
- [Discord](https://discord.gg/zd4qPW6t)
- [Twitter](https://twitter.com/infiniflowai)
- [Discord](https://discord.gg/NjYzJD3GM3)
- [X](https://x.com/infiniflowai)
- [GitHub Discussions](https://github.com/orgs/infiniflow/discussions)
## 🙌 贡献指南

779
admin/client/COMMAND.md Normal file
View File

@@ -0,0 +1,779 @@
# RAGFlow CLI User Command Reference
This document describes the user commands available in RAGFlow CLI. All commands must end with a semicolon (`;`).
## Command List
### ping_server
**Description**
Tests the connection status to the server.
**Usage**
```
PING;
```
**Parameters**
No parameters.
**Example**
```
ragflow> PING;
```
**Display Effect**
(Sample output will be provided by the user)
---
### show_current_user
**Description**
Displays information about the currently logged-in user.
**Usage**
```
SHOW CURRENT USER;
```
**Parameters**
No parameters.
**Example**
```
ragflow> SHOW CURRENT USER;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_model_provider
**Description**
Creates a new model provider.
**Usage**
```
CREATE MODEL PROVIDER <provider_name> <provider_key>;
```
**Parameters**
- `provider_name`: Provider name, quoted string.
- `provider_key`: Provider key, quoted string.
**Example**
```
ragflow> CREATE MODEL PROVIDER 'openai' 'sk-...';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_model_provider
**Description**
Deletes a model provider.
**Usage**
```
DROP MODEL PROVIDER <provider_name>;
```
**Parameters**
- `provider_name`: Name of the provider to delete, quoted string.
**Example**
```
ragflow> DROP MODEL PROVIDER 'openai';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_llm
**Description**
Sets the default LLM (Large Language Model).
**Usage**
```
SET DEFAULT LLM <llm_id>;
```
**Parameters**
- `llm_id`: LLM identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT LLM 'gpt-4';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_vlm
**Description**
Sets the default VLM (Vision Language Model).
**Usage**
```
SET DEFAULT VLM <vlm_id>;
```
**Parameters**
- `vlm_id`: VLM identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT VLM 'clip-vit-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_embedding
**Description**
Sets the default embedding model.
**Usage**
```
SET DEFAULT EMBEDDING <embedding_id>;
```
**Parameters**
- `embedding_id`: Embedding model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT EMBEDDING 'text-embedding-ada-002';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_reranker
**Description**
Sets the default reranker model.
**Usage**
```
SET DEFAULT RERANKER <reranker_id>;
```
**Parameters**
- `reranker_id`: Reranker model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT RERANKER 'bge-reranker-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_asr
**Description**
Sets the default ASR (Automatic Speech Recognition) model.
**Usage**
```
SET DEFAULT ASR <asr_id>;
```
**Parameters**
- `asr_id`: ASR model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT ASR 'whisper-large';
```
**Display Effect**
(Sample output will be provided by the user)
---
### set_default_tts
**Description**
Sets the default TTS (Text-to-Speech) model.
**Usage**
```
SET DEFAULT TTS <tts_id>;
```
**Parameters**
- `tts_id`: TTS model identifier, quoted string.
**Example**
```
ragflow> SET DEFAULT TTS 'tts-1';
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_llm
**Description**
Resets the default LLM to system default.
**Usage**
```
RESET DEFAULT LLM;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT LLM;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_vlm
**Description**
Resets the default VLM to system default.
**Usage**
```
RESET DEFAULT VLM;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT VLM;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_embedding
**Description**
Resets the default embedding model to system default.
**Usage**
```
RESET DEFAULT EMBEDDING;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT EMBEDDING;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_reranker
**Description**
Resets the default reranker model to system default.
**Usage**
```
RESET DEFAULT RERANKER;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT RERANKER;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_asr
**Description**
Resets the default ASR model to system default.
**Usage**
```
RESET DEFAULT ASR;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT ASR;
```
**Display Effect**
(Sample output will be provided by the user)
---
### reset_default_tts
**Description**
Resets the default TTS model to system default.
**Usage**
```
RESET DEFAULT TTS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> RESET DEFAULT TTS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_dataset_with_parser
**Description**
Creates a user dataset with the specified parser.
**Usage**
```
CREATE DATASET <dataset_name> WITH EMBEDDING <embedding> PARSER <parser_type>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
- `embedding`: Embedding model name, quoted string.
- `parser_type`: Parser type, quoted string.
**Example**
```
ragflow> CREATE DATASET 'my_dataset' WITH EMBEDDING 'text-embedding-ada-002' PARSER 'pdf';
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_dataset_with_pipeline
**Description**
Creates a user dataset with the specified pipeline.
**Usage**
```
CREATE DATASET <dataset_name> WITH EMBEDDING <embedding> PIPELINE <pipeline>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
- `embedding`: Embedding model name, quoted string.
- `pipeline`: Pipeline name, quoted string.
**Example**
```
ragflow> CREATE DATASET 'my_dataset' WITH EMBEDDING 'text-embedding-ada-002' PIPELINE 'standard';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_user_dataset
**Description**
Deletes a user dataset.
**Usage**
```
DROP DATASET <dataset_name>;
```
**Parameters**
- `dataset_name`: Name of the dataset to delete, quoted string.
**Example**
```
ragflow> DROP DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_datasets
**Description**
Lists all datasets for the current user.
**Usage**
```
LIST DATASETS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST DATASETS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_dataset_files
**Description**
Lists all files in the specified dataset.
**Usage**
```
LIST FILES OF DATASET <dataset_name>;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> LIST FILES OF DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_agents
**Description**
Lists all agents for the current user.
**Usage**
```
LIST AGENTS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST AGENTS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_chats
**Description**
Lists all chat sessions for the current user.
**Usage**
```
LIST CHATS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST CHATS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### create_user_chat
**Description**
Creates a new chat session.
**Usage**
```
CREATE CHAT <chat_name>;
```
**Parameters**
- `chat_name`: Chat session name, quoted string.
**Example**
```
ragflow> CREATE CHAT 'my_chat';
```
**Display Effect**
(Sample output will be provided by the user)
---
### drop_user_chat
**Description**
Deletes a chat session.
**Usage**
```
DROP CHAT <chat_name>;
```
**Parameters**
- `chat_name`: Name of the chat session to delete, quoted string.
**Example**
```
ragflow> DROP CHAT 'my_chat';
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_model_providers
**Description**
Lists all model providers for the current user.
**Usage**
```
LIST MODEL PROVIDERS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST MODEL PROVIDERS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### list_user_default_models
**Description**
Lists all default model settings for the current user.
**Usage**
```
LIST DEFAULT MODELS;
```
**Parameters**
No parameters.
**Example**
```
ragflow> LIST DEFAULT MODELS;
```
**Display Effect**
(Sample output will be provided by the user)
---
### import_docs_into_dataset
**Description**
Imports documents into the specified dataset.
**Usage**
```
IMPORT <document_list> INTO DATASET <dataset_name>;
```
**Parameters**
- `document_list`: List of document paths, multiple paths can be separated by commas, or as a space-separated quoted string.
- `dataset_name`: Target dataset name, quoted string.
**Example**
```
ragflow> IMPORT '/path/to/doc1.pdf,/path/to/doc2.pdf' INTO DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### search_on_datasets
**Description**
Searches in one or more specified datasets.
**Usage**
```
SEARCH <question> ON DATASETS <dataset_list>;
```
**Parameters**
- `question`: Search question, quoted string.
- `dataset_list`: List of dataset names, multiple names can be separated by commas, or as a space-separated quoted string.
**Example**
```
ragflow> SEARCH 'What is RAG?' ON DATASETS 'dataset1,dataset2';
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_docs
**Description**
Parses specified documents in a dataset.
**Usage**
```
PARSE <document_names> OF DATASET <dataset_name>;
```
**Parameters**
- `document_names`: List of document names, multiple names can be separated by commas, or as a space-separated quoted string.
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE 'doc1.pdf,doc2.pdf' OF DATASET 'my_dataset';
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_sync
**Description**
Synchronously parses the entire dataset.
**Usage**
```
PARSE DATASET <dataset_name> SYNC;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE DATASET 'my_dataset' SYNC;
```
**Display Effect**
(Sample output will be provided by the user)
---
### parse_dataset_async
**Description**
Asynchronously parses the entire dataset.
**Usage**
```
PARSE DATASET <dataset_name> ASYNC;
```
**Parameters**
- `dataset_name`: Dataset name, quoted string.
**Example**
```
ragflow> PARSE DATASET 'my_dataset' ASYNC;
```
**Display Effect**
(Sample output will be provided by the user)
---
### benchmark
**Description**
Performs performance benchmark testing on the specified user command.
**Usage**
```
BENCHMARK <concurrency> <iterations> <user_command>;
```
**Parameters**
- `concurrency`: Concurrency number, positive integer.
- `iterations`: Number of iterations, positive integer.
- `user_command`: User command to test (must be a valid user command, such as `PING;`).
**Example**
```
ragflow> BENCHMARK 5 10 PING;
```
**Display Effect**
(Sample output will be provided by the user)
---
**Notes**
- All string parameters (such as names, IDs, paths) must be enclosed in single quotes (`'`) or double quotes (`"`).
- Commands must end with a semicolon (`;`).
- The prompt is `ragflow>`.

View File

@@ -48,7 +48,7 @@ It consists of a server-side Service and a command-line client (CLI), both imple
1. Ensure the Admin Service is running.
2. Install ragflow-cli.
```bash
pip install ragflow-cli==0.24.0
pip install ragflow-cli==0.25.6
```
3. Launch the CLI client:
```bash

View File

@@ -77,10 +77,17 @@ sql_command: login_user
| drop_user_dataset
| list_user_datasets
| list_user_dataset_files
| list_user_dataset_documents
| list_user_datasets_metadata
| list_user_documents_metadata_summary
| list_user_agents
| list_user_chats
| create_user_chat
| drop_user_chat
| create_dataset_table
| drop_dataset_table
| create_metadata_table
| drop_metadata_table
| list_user_model_providers
| list_user_default_models
| parse_dataset_docs
@@ -88,15 +95,35 @@ sql_command: login_user
| parse_dataset_async
| import_docs_into_dataset
| search_on_datasets
| get_chunk
| list_chunks
| insert_dataset_from_file
| insert_metadata_from_file
| update_chunk
| set_metadata
| remove_tags
| remove_chunks
| create_chat_session
| drop_chat_session
| list_chat_sessions
| chat_on_session
| list_server_configs
| show_fingerprint
| set_license
| set_license_config
| show_license
| check_license
| benchmark
// meta command definition
meta_command: "\\" meta_command_name [meta_args]
COMMA: ","
meta_command_name: /[a-zA-Z?]+/
meta_args: (meta_arg)+
meta_arg: /[^\\s"']+/ | quoted_string
meta_arg: /[^\s"',]+/ | quoted_string
// command definition
@@ -117,6 +144,7 @@ ALTER: "ALTER"i
ACTIVE: "ACTIVE"i
ADMIN: "ADMIN"i
PASSWORD: "PASSWORD"i
DATASET_TABLE: "DATASET TABLE"i
DATASET: "DATASET"i
DATASETS: "DATASETS"i
OF: "OF"i
@@ -151,11 +179,18 @@ DEFAULT: "DEFAULT"i
CHATS: "CHATS"i
CHAT: "CHAT"i
FILES: "FILES"i
DOCUMENT: "DOCUMENT"i
DOCUMENTS: "DOCUMENTS"i
METADATA: "METADATA"i
SUMMARY: "SUMMARY"i
AS: "AS"i
PARSE: "PARSE"i
IMPORT: "IMPORT"i
INTO: "INTO"i
IN: "IN"i
WITH: "WITH"i
VECTOR: "VECTOR"i
SIZE: "SIZE"i
PARSER: "PARSER"i
PIPELINE: "PIPELINE"i
SEARCH: "SEARCH"i
@@ -170,8 +205,28 @@ ASYNC: "ASYNC"i
SYNC: "SYNC"i
BENCHMARK: "BENCHMARK"i
PING: "PING"i
SESSION: "SESSION"i
SESSIONS: "SESSIONS"i
SERVER: "SERVER"i
FINGERPRINT: "FINGERPRINT"i
LICENSE: "LICENSE"i
CHECK: "CHECK"i
CONFIG: "CONFIG"i
INDEX: "INDEX"i
TABLE: "TABLE"i
CHUNK: "CHUNK"i
CHUNKS: "CHUNKS"i
GET: "GET"i
INSERT: "INSERT"i
PAGE: "PAGE"i
KEYWORDS: "KEYWORDS"i
AVAILABLE: "AVAILABLE"i
FILE: "FILE"i
UPDATE: "UPDATE"i
REMOVE: "REMOVE"i
TAGS: "TAGS"i
login_user: LOGIN USER quoted_string ";"
login_user: LOGIN USER quoted_string (PASSWORD quoted_string)? ";"
list_services: LIST SERVICES ";"
show_service: SHOW SERVICE NUMBER ";"
startup_service: STARTUP SERVICE NUMBER ";"
@@ -209,12 +264,20 @@ generate_key: GENERATE KEY FOR USER quoted_string ";"
list_keys: LIST KEYS OF quoted_string ";"
drop_key: DROP KEY quoted_string OF quoted_string ";"
set_variable: SET VAR identifier identifier ";"
set_variable: SET VAR identifier variable_value ";"
show_variable: SHOW VAR identifier ";"
list_variables: LIST VARS ";"
list_configs: LIST CONFIGS ";"
list_environments: LIST ENVS ";"
show_fingerprint: SHOW FINGERPRINT ";"
set_license: SET LICENSE quoted_string ";"
set_license_config: SET LICENSE CONFIG NUMBER NUMBER ";"
show_license: SHOW LICENSE ";"
check_license: CHECK LICENSE ";"
list_server_configs: LIST SERVER CONFIGS ";"
benchmark: BENCHMARK NUMBER NUMBER user_statement
user_statement: ping_server
@@ -246,6 +309,13 @@ user_statement: ping_server
| list_user_default_models
| import_docs_into_dataset
| search_on_datasets
| update_chunk
| set_metadata
| remove_tags
| create_chat_session
| drop_chat_session
| list_chat_sessions
| chat_on_session
ping_server: PING ";"
show_current_user: SHOW CURRENT USER ";"
@@ -270,24 +340,47 @@ create_user_dataset_with_parser: CREATE DATASET quoted_string WITH EMBEDDING quo
create_user_dataset_with_pipeline: CREATE DATASET quoted_string WITH EMBEDDING quoted_string PIPELINE quoted_string ";"
drop_user_dataset: DROP DATASET quoted_string ";"
list_user_dataset_files: LIST FILES OF DATASET quoted_string ";"
list_user_dataset_documents: LIST DOCUMENTS OF DATASET quoted_string ";"
list_user_datasets_metadata: LIST METADATA OF DATASETS quoted_string (COMMA quoted_string)* ";"
list_user_documents_metadata_summary: LIST METADATA SUMMARY OF DATASET quoted_string (DOCUMENTS quoted_string (COMMA quoted_string)*)? ";"
list_user_agents: LIST AGENTS ";"
list_user_chats: LIST CHATS ";"
create_user_chat: CREATE CHAT quoted_string ";"
drop_user_chat: DROP CHAT quoted_string ";"
create_chat_session: CREATE CHAT quoted_string SESSION ";"
drop_chat_session: DROP CHAT quoted_string SESSION quoted_string ";"
list_chat_sessions: LIST CHAT quoted_string SESSIONS ";"
chat_on_session: CHAT quoted_string ON quoted_string SESSION quoted_string ";"
list_user_model_providers: LIST MODEL PROVIDERS ";"
list_user_default_models: LIST DEFAULT MODELS ";"
import_docs_into_dataset: IMPORT quoted_string INTO DATASET quoted_string ";"
search_on_datasets: SEARCH quoted_string ON DATASETS quoted_string ";"
get_chunk: GET CHUNK quoted_string ";"
list_chunks: LIST CHUNKS OF DOCUMENT quoted_string ("PAGE" NUMBER)? ("SIZE" NUMBER)? ("KEYWORDS" quoted_string)? ("AVAILABLE" NUMBER)? ";"
set_metadata: SET METADATA OF DOCUMENT quoted_string TO quoted_string ";"
remove_tags: REMOVE TAGS quoted_string (COMMA quoted_string)* FROM DATASET quoted_string ";"
remove_chunks: REMOVE CHUNKS quoted_string (COMMA quoted_string)* FROM DOCUMENT quoted_string ";"
| REMOVE ALL CHUNKS FROM DOCUMENT quoted_string ";"
parse_dataset_docs: PARSE quoted_string OF DATASET quoted_string ";"
parse_dataset_sync: PARSE DATASET quoted_string SYNC ";"
parse_dataset_async: PARSE DATASET quoted_string ASYNC ";"
identifier_list: identifier ("," identifier)*
// Internal CLI only for GO
create_dataset_table: CREATE DATASET TABLE quoted_string VECTOR SIZE NUMBER ";"
drop_dataset_table: DROP DATASET TABLE quoted_string ";"
create_metadata_table: CREATE METADATA TABLE ";"
drop_metadata_table: DROP METADATA TABLE ";"
insert_dataset_from_file: INSERT DATASET FROM FILE quoted_string ";"
insert_metadata_from_file: INSERT METADATA FROM FILE quoted_string ";"
update_chunk: UPDATE CHUNK quoted_string OF DATASET quoted_string SET quoted_string ";"
identifier_list: identifier (COMMA identifier)*
identifier: WORD
variable_value: WORD | NUMBER | QUOTED_STRING
quoted_string: QUOTED_STRING
status: WORD
status: ON | WORD
QUOTED_STRING: /'[^']+'/ | /"[^"]+"/
WORD: /[a-zA-Z0-9_\-\.]+/
@@ -307,7 +400,13 @@ class RAGFlowCLITransformer(Transformer):
def login_user(self, items):
email = items[2].children[0].strip("'\"")
return {"type": "login_user", "email": email}
if len(items) == 5:
# With password: LOGIN USER email PASSWORD password
password = items[4].children[0].strip("'\"")
return {"type": "login_user", "email": email, "password": password}
else:
# Without password: LOGIN USER email
return {"type": "login_user", "email": email}
def ping_server(self, items):
return {"type": "ping_server"}
@@ -459,6 +558,27 @@ class RAGFlowCLITransformer(Transformer):
def list_environments(self, items):
return {"type": "list_environments"}
def show_fingerprint(self, items):
return {"type": "show_fingerprint"}
def set_license(self, items):
license = items[2].children[0].strip("'\"")
return {"type": "set_license", "license": license}
def set_license_config(self, items):
value1: int = int(items[3])
value2: int = int(items[4])
return {"type": "set_license_config", "value1": value1, "value2": value2}
def show_license(self, items):
return {"type": "show_license"}
def check_license(self, items):
return {"type": "check_license"}
def list_server_configs(self, items):
return {"type": "list_server_configs"}
def create_model_provider(self, items):
provider_name = items[3].children[0].strip("'\"")
provider_key = items[4].children[0].strip("'\"")
@@ -538,6 +658,28 @@ class RAGFlowCLITransformer(Transformer):
dataset_name = items[4].children[0].strip("'\"")
return {"type": "list_user_dataset_files", "dataset_name": dataset_name}
def list_user_dataset_documents(self, items):
dataset_name = items[4].children[0].strip("'\"")
return {"type": "list_user_dataset_documents", "dataset_name": dataset_name}
def list_user_datasets_metadata(self, items):
dataset_names = []
dataset_names.append(items[4].children[0].strip("'\""))
for i in range(5, len(items)):
if items[i] and hasattr(items[i], 'children') and items[i].children:
dataset_names.append(items[i].children[0].strip("'\""))
return {"type": "list_user_datasets_metadata", "dataset_names": dataset_names}
def list_user_documents_metadata_summary(self, items):
dataset_name = items[5].children[0].strip("'\"")
doc_ids = []
if len(items) > 6 and items[6] == "DOCUMENTS":
for i in range(7, len(items)):
if items[i] and hasattr(items[i], 'children') and items[i].children:
doc_id = items[i].children[0].strip("'\"")
doc_ids.append(doc_id)
return {"type": "list_user_documents_metadata_summary", "dataset_name": dataset_name, "document_ids": doc_ids}
def list_user_agents(self, items):
return {"type": "list_user_agents"}
@@ -552,6 +694,30 @@ class RAGFlowCLITransformer(Transformer):
chat_name = items[2].children[0].strip("'\"")
return {"type": "drop_user_chat", "chat_name": chat_name}
def create_dataset_table(self, items):
dataset_name = None
vector_size = None
for i, item in enumerate(items):
if hasattr(item, 'data') and item.data == 'quoted_string':
dataset_name = item.children[0].strip("'\"")
if hasattr(item, 'type') and item.type == 'NUMBER':
if i > 0 and items[i-1].type == 'SIZE' and items[i-2].type == 'VECTOR':
vector_size = int(item)
return {"type": "create_dataset_table", "dataset_name": dataset_name, "vector_size": vector_size}
def drop_dataset_table(self, items):
dataset_name = None
for item in items:
if hasattr(item, 'data') and item.data == 'quoted_string':
dataset_name = item.children[0].strip("'\"")
return {"type": "drop_dataset_table", "dataset_name": dataset_name}
def create_metadata_table(self, items):
return {"type": "create_metadata_table"}
def drop_metadata_table(self, items):
return {"type": "drop_metadata_table"}
def list_user_model_providers(self, items):
return {"type": "list_user_model_providers"}
@@ -575,6 +741,25 @@ class RAGFlowCLITransformer(Transformer):
dataset_name = items[2].children[0].strip("'\"")
return {"type": "parse_dataset", "dataset_name": dataset_name, "method": "async"}
def create_chat_session(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "create_chat_session", "chat_name": chat_name}
def drop_chat_session(self, items):
chat_name = items[2].children[0].strip("'\"")
session_id = items[4].children[0].strip("'\"")
return {"type": "drop_chat_session", "chat_name": chat_name, "session_id": session_id}
def list_chat_sessions(self, items):
chat_name = items[2].children[0].strip("'\"")
return {"type": "list_chat_sessions", "chat_name": chat_name}
def chat_on_session(self, items):
message = items[1].children[0].strip("'\"")
chat_name = items[3].children[0].strip("'\"")
session_id = items[5].children[0].strip("'\"")
return {"type": "chat_on_session", "message": message, "chat_name": chat_name, "session_id": session_id}
def import_docs_into_dataset(self, items):
document_list_str = items[1].children[0].strip("'\"")
document_paths = document_list_str.split(",")
@@ -593,6 +778,103 @@ class RAGFlowCLITransformer(Transformer):
datasets = datasets.split(" ")
return {"type": "search_on_datasets", "datasets": datasets, "question": question}
def get_chunk(self, items):
chunk_id = items[2].children[0].strip("'\"")
return {"type": "get_chunk", "chunk_id": chunk_id}
def insert_dataset_from_file(self, items):
file_path = items[4].children[0].strip("'\"")
return {"type": "insert_dataset_from_file", "file_path": file_path}
def insert_metadata_from_file(self, items):
file_path = items[4].children[0].strip("'\"")
return {"type": "insert_metadata_from_file", "file_path": file_path}
def update_chunk(self, items):
def get_quoted_value(item):
if hasattr(item, 'children') and item.children:
return item.children[0].strip("'\"")
return str(item).strip("'\"")
chunk_id = get_quoted_value(items[2])
dataset_name = get_quoted_value(items[5])
json_body = get_quoted_value(items[7])
return {"type": "update_chunk", "chunk_id": chunk_id, "dataset_name": dataset_name, "json_body": json_body}
def set_metadata(self, items):
doc_id = items[4].children[0].strip("'\"")
meta_json = items[6].children[0].strip("'\"")
return {"type": "set_metadata", "doc_id": doc_id, "meta": meta_json}
def remove_tags(self, items):
# items: REMOVE, TAGS, quoted_string(tag1), quoted_string(tag2), ..., FROM, DATASET, quoted_string(dataset_name), ";"
tags = []
# Start from index 2 (after TAGS keyword) and parse quoted strings until FROM
for i in range(2, len(items)):
item = items[i]
# Check for FROM token to stop
if hasattr(item, 'type') and item.type == 'FROM':
break
if hasattr(item, 'children') and item.children:
tag = item.children[0].strip("'\"")
tags.append(tag)
# Find dataset_name: quoted_string after DATASET
dataset_name = None
for i, item in enumerate(items):
# Check if item is a DATASET token
if hasattr(item, 'type') and item.type == 'DATASET':
# Next item should be quoted_string
dataset_name = items[i + 1].children[0].strip("'\"")
break
return {"type": "remove_tags", "dataset_name": dataset_name, "tags": tags}
def remove_chunks(self, items):
# Handle two cases:
# 1. REMOVE CHUNKS quoted_string (COMMA quoted_string)* FROM DOCUMENT quoted_string ";"
# 2. REMOVE ALL CHUNKS FROM DOCUMENT quoted_string ";"
# Check if it's "REMOVE ALL CHUNKS"
for item in items:
if hasattr(item, 'type') and item.type == 'ALL':
# Find doc_id
for j, inner_item in enumerate(items):
if hasattr(inner_item, 'type') and inner_item.type == 'DOCUMENT':
doc_id = items[j + 1].children[0].strip("'\"")
return {"type": "remove_chunks", "doc_id": doc_id, "delete_all": True}
# Otherwise, we have chunk_ids
chunk_ids = []
doc_id = None
for i, item in enumerate(items):
if hasattr(item, 'type') and item.type == 'DOCUMENT':
doc_id = items[i + 1].children[0].strip("'\"")
elif hasattr(item, 'children') and item.children:
val = item.children[0].strip("'\"")
# Skip if it's "FROM" or "DOCUMENT"
if val.upper() in ['FROM', 'DOCUMENT']:
continue
chunk_ids.append(val)
return {"type": "remove_chunks", "doc_id": doc_id, "chunk_ids": chunk_ids}
def list_chunks(self, items):
doc_id = items[4].children[0].strip("'\"")
result = {"type": "list_chunks", "doc_id": doc_id}
# Parse optional parameters: PAGE, SIZE, KEYWORDS, AVAILABLE
# items structure varies based on which params are present
for i, item in enumerate(items):
if str(item) == "PAGE":
result["page"] = int(items[i + 1])
elif str(item) == "SIZE":
result["size"] = int(items[i + 1])
elif str(item) == "KEYWORDS":
result["keywords"] = items[i + 1].children[0].strip("'\"")
elif str(item) == "AVAILABLE":
result["available_int"] = int(items[i + 1])
return result
def benchmark(self, items):
concurrency: int = int(items[1])
iterations: int = int(items[2])

View File

@@ -1,6 +1,6 @@
[project]
name = "ragflow-cli"
version = "0.24.0"
version = "0.25.6"
description = "Admin Service's client of [RAGFlow](https://github.com/infiniflow/ragflow). The Admin Service provides user management and system monitoring. "
authors = [{ name = "Lynn", email = "lynn_inf@hotmail.com" }]
license = { text = "Apache License, Version 2.0" }
@@ -11,17 +11,17 @@ dependencies = [
"beartype>=0.20.0,<1.0.0",
"pycryptodomex>=3.10.0",
"lark>=1.1.0",
"requests-toolbelt>=1.0.0",
]
[dependency-groups]
test = [
"pytest>=8.3.5",
"requests>=2.32.3",
"requests-toolbelt>=1.0.0",
]
[tool.setuptools]
py-modules = ["ragflow_cli", "parser"]
py-modules = ["ragflow_cli", "parser", "http_client", "ragflow_client", "user"]
[project.scripts]
ragflow-cli = "ragflow_cli:main"

View File

@@ -18,6 +18,9 @@ import sys
import argparse
import base64
import getpass
import os
import atexit
import readline
from cmd import Cmd
from typing import Any, Dict, List
@@ -61,6 +64,12 @@ class RAGFlowCLI(Cmd):
self.port: int = 0
self.mode: str = "admin"
self.ragflow_client = None
# History file for readline persistence
self.history_file = os.path.expanduser("~/.ragflow_cli_history")
# Load existing history
self._load_history()
# Register cleanup to save history on exit
atexit.register(self._save_history)
intro = r"""Type "\h" for help."""
prompt = "ragflow> "
@@ -99,6 +108,7 @@ class RAGFlowCLI(Cmd):
return {"type": "empty"}
self.command_history.append(command_str)
readline.add_history(command_str)
try:
result = self.parser.parse(command_str)
@@ -210,6 +220,21 @@ class RAGFlowCLI(Cmd):
print(separator)
def _load_history(self):
"""Load command history from file."""
try:
if os.path.exists(self.history_file):
readline.read_history_file(self.history_file)
except Exception:
pass # Ignore errors loading history
def _save_history(self):
"""Save command history to file."""
try:
readline.write_history_file(self.history_file)
except Exception:
pass # Ignore errors saving history
def run_interactive(self, args):
if self.verify_auth(args, single_command=False, auth=args["auth"]):
print(r"""

File diff suppressed because it is too large Load Diff

View File

@@ -26,10 +26,22 @@ class AuthException(Exception):
def encrypt_password(password_plain: str) -> str:
try:
from api.utils.crypt import crypt
import base64
from Cryptodome.PublicKey import RSA
from Cryptodome.Cipher import PKCS1_v1_5 as Cipher_pkcs1_v1_5
def crypt(line):
"""
decrypt(crypt(input_string)) == base64(input_string), which frontend and ragflow_cli use.
"""
pub = "-----BEGIN PUBLIC KEY-----\nMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArq9XTUSeYr2+N1h3Afl/z8Dse/2yD0ZGrKwx+EEEcdsBLca9Ynmx3nIB5obmLlSfmskLpBo0UACBmB5rEjBp2Q2f3AG3Hjd4B+gNCG6BDaawuDlgANIhGnaTLrIqWrrcm4EMzJOnAOI1fgzJRsOOUEfaS318Eq9OVO3apEyCCt0lOQK6PuksduOjVxtltDav+guVAA068NrPYmRNabVKRNLJpL8w4D44sfth5RvZ3q9t+6RTArpEtc5sh5ChzvqPOzKGMXW83C95TxmXqpbK6olN4RevSfVjEAgCydH6HN6OhtOQEcnrU97r9H0iZOWwbw3pVrZiUkuRD1R56Wzs2wIDAQAB\n-----END PUBLIC KEY-----"
rsa_key = RSA.importKey(pub)
cipher = Cipher_pkcs1_v1_5.new(rsa_key)
password_base64 = base64.b64encode(line.encode('utf-8')).decode("utf-8")
encrypted_password = cipher.encrypt(password_base64.encode())
return base64.b64encode(encrypted_password).decode('utf-8')
except Exception as exc:
raise AuthException(
"Password encryption unavailable; install pycryptodomex (uv sync --python 3.12 --group test)."
"Password encryption unavailable; install pycryptodomex (uv sync --python 3.13 --group test)."
) from exc
return crypt(password_plain)

138
admin/client/uv.lock generated
View File

@@ -1,6 +1,6 @@
version = 1
revision = 3
requires-python = ">=3.10, <3.13"
requires-python = ">=3.12, <3.15"
[[package]]
name = "beartype"
@@ -26,38 +26,6 @@ version = "3.4.4"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/13/69/33ddede1939fdd074bce5434295f38fae7136463422fe4fd3e0e89b98062/charset_normalizer-3.4.4.tar.gz", hash = "sha256:94537985111c35f28720e43603b8e7b43a6ecfb2ce1d3058bbe955b73404e21a", size = 129418, upload-time = "2025-10-14T04:42:32.879Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/b8/6d51fc1d52cbd52cd4ccedd5b5b2f0f6a11bbf6765c782298b0f3e808541/charset_normalizer-3.4.4-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:e824f1492727fa856dd6eda4f7cee25f8518a12f3c4a56a74e8095695089cf6d", size = 209709, upload-time = "2025-10-14T04:40:11.385Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/5c/af/1f9d7f7faafe2ddfb6f72a2e07a548a629c61ad510fe60f9630309908fef/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:4bd5d4137d500351a30687c2d3971758aac9a19208fc110ccb9d7188fbe709e8", size = 148814, upload-time = "2025-10-14T04:40:13.135Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/79/3d/f2e3ac2bbc056ca0c204298ea4e3d9db9b4afe437812638759db2c976b5f/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:027f6de494925c0ab2a55eab46ae5129951638a49a34d87f4c3eda90f696b4ad", size = 144467, upload-time = "2025-10-14T04:40:14.728Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ec/85/1bf997003815e60d57de7bd972c57dc6950446a3e4ccac43bc3070721856/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:f820802628d2694cb7e56db99213f930856014862f3fd943d290ea8438d07ca8", size = 162280, upload-time = "2025-10-14T04:40:16.14Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3e/8e/6aa1952f56b192f54921c436b87f2aaf7c7a7c3d0d1a765547d64fd83c13/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:798d75d81754988d2565bff1b97ba5a44411867c0cf32b77a7e8f8d84796b10d", size = 159454, upload-time = "2025-10-14T04:40:17.567Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/36/3b/60cbd1f8e93aa25d1c669c649b7a655b0b5fb4c571858910ea9332678558/charset_normalizer-3.4.4-cp310-cp310-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:9d1bb833febdff5c8927f922386db610b49db6e0d4f4ee29601d71e7c2694313", size = 153609, upload-time = "2025-10-14T04:40:19.08Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/64/91/6a13396948b8fd3c4b4fd5bc74d045f5637d78c9675585e8e9fbe5636554/charset_normalizer-3.4.4-cp310-cp310-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:9cd98cdc06614a2f768d2b7286d66805f94c48cde050acdbbb7db2600ab3197e", size = 151849, upload-time = "2025-10-14T04:40:20.607Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b7/7a/59482e28b9981d105691e968c544cc0df3b7d6133152fb3dcdc8f135da7a/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_aarch64.whl", hash = "sha256:077fbb858e903c73f6c9db43374fd213b0b6a778106bc7032446a8e8b5b38b93", size = 151586, upload-time = "2025-10-14T04:40:21.719Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/92/59/f64ef6a1c4bdd2baf892b04cd78792ed8684fbc48d4c2afe467d96b4df57/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_armv7l.whl", hash = "sha256:244bfb999c71b35de57821b8ea746b24e863398194a4014e4c76adc2bbdfeff0", size = 145290, upload-time = "2025-10-14T04:40:23.069Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6b/63/3bf9f279ddfa641ffa1962b0db6a57a9c294361cc2f5fcac997049a00e9c/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_ppc64le.whl", hash = "sha256:64b55f9dce520635f018f907ff1b0df1fdc31f2795a922fb49dd14fbcdf48c84", size = 163663, upload-time = "2025-10-14T04:40:24.17Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/09/c9e38fc8fa9e0849b172b581fd9803bdf6e694041127933934184e19f8c3/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_riscv64.whl", hash = "sha256:faa3a41b2b66b6e50f84ae4a68c64fcd0c44355741c6374813a800cd6695db9e", size = 151964, upload-time = "2025-10-14T04:40:25.368Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d2/d1/d28b747e512d0da79d8b6a1ac18b7ab2ecfd81b2944c4c710e166d8dd09c/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_s390x.whl", hash = "sha256:6515f3182dbe4ea06ced2d9e8666d97b46ef4c75e326b79bb624110f122551db", size = 161064, upload-time = "2025-10-14T04:40:26.806Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/bb/9a/31d62b611d901c3b9e5500c36aab0ff5eb442043fb3a1c254200d3d397d9/charset_normalizer-3.4.4-cp310-cp310-musllinux_1_2_x86_64.whl", hash = "sha256:cc00f04ed596e9dc0da42ed17ac5e596c6ccba999ba6bd92b0e0aef2f170f2d6", size = 155015, upload-time = "2025-10-14T04:40:28.284Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/f3/107e008fa2bff0c8b9319584174418e5e5285fef32f79d8ee6a430d0039c/charset_normalizer-3.4.4-cp310-cp310-win32.whl", hash = "sha256:f34be2938726fc13801220747472850852fe6b1ea75869a048d6f896838c896f", size = 99792, upload-time = "2025-10-14T04:40:29.613Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/eb/66/e396e8a408843337d7315bab30dbf106c38966f1819f123257f5520f8a96/charset_normalizer-3.4.4-cp310-cp310-win_amd64.whl", hash = "sha256:a61900df84c667873b292c3de315a786dd8dac506704dea57bc957bd31e22c7d", size = 107198, upload-time = "2025-10-14T04:40:30.644Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b5/58/01b4f815bf0312704c267f2ccb6e5d42bcc7752340cd487bc9f8c3710597/charset_normalizer-3.4.4-cp310-cp310-win_arm64.whl", hash = "sha256:cead0978fc57397645f12578bfd2d5ea9138ea0fac82b2f63f7f7c6877986a69", size = 100262, upload-time = "2025-10-14T04:40:32.108Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ed/27/c6491ff4954e58a10f69ad90aca8a1b6fe9c5d3c6f380907af3c37435b59/charset_normalizer-3.4.4-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:6e1fcf0720908f200cd21aa4e6750a48ff6ce4afe7ff5a79a90d5ed8a08296f8", size = 206988, upload-time = "2025-10-14T04:40:33.79Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/94/59/2e87300fe67ab820b5428580a53cad894272dbb97f38a7a814a2a1ac1011/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:5f819d5fe9234f9f82d75bdfa9aef3a3d72c4d24a6e57aeaebba32a704553aa0", size = 147324, upload-time = "2025-10-14T04:40:34.961Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/07/fb/0cf61dc84b2b088391830f6274cb57c82e4da8bbc2efeac8c025edb88772/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:a59cb51917aa591b1c4e6a43c132f0cdc3c76dbad6155df4e28ee626cc77a0a3", size = 142742, upload-time = "2025-10-14T04:40:36.105Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/62/8b/171935adf2312cd745d290ed93cf16cf0dfe320863ab7cbeeae1dcd6535f/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:8ef3c867360f88ac904fd3f5e1f902f13307af9052646963ee08ff4f131adafc", size = 160863, upload-time = "2025-10-14T04:40:37.188Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/09/73/ad875b192bda14f2173bfc1bc9a55e009808484a4b256748d931b6948442/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d9e45d7faa48ee908174d8fe84854479ef838fc6a705c9315372eacbc2f02897", size = 157837, upload-time = "2025-10-14T04:40:38.435Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6d/fc/de9cce525b2c5b94b47c70a4b4fb19f871b24995c728e957ee68ab1671ea/charset_normalizer-3.4.4-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:840c25fb618a231545cbab0564a799f101b63b9901f2569faecd6b222ac72381", size = 151550, upload-time = "2025-10-14T04:40:40.053Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/55/c2/43edd615fdfba8c6f2dfbd459b25a6b3b551f24ea21981e23fb768503ce1/charset_normalizer-3.4.4-cp311-cp311-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:ca5862d5b3928c4940729dacc329aa9102900382fea192fc5e52eb69d6093815", size = 149162, upload-time = "2025-10-14T04:40:41.163Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/03/86/bde4ad8b4d0e9429a4e82c1e8f5c659993a9a863ad62c7df05cf7b678d75/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:d9c7f57c3d666a53421049053eaacdd14bbd0a528e2186fcb2e672effd053bb0", size = 150019, upload-time = "2025-10-14T04:40:42.276Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1f/86/a151eb2af293a7e7bac3a739b81072585ce36ccfb4493039f49f1d3cae8c/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_armv7l.whl", hash = "sha256:277e970e750505ed74c832b4bf75dac7476262ee2a013f5574dd49075879e161", size = 143310, upload-time = "2025-10-14T04:40:43.439Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b5/fe/43dae6144a7e07b87478fdfc4dbe9efd5defb0e7ec29f5f58a55aeef7bf7/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_ppc64le.whl", hash = "sha256:31fd66405eaf47bb62e8cd575dc621c56c668f27d46a61d975a249930dd5e2a4", size = 162022, upload-time = "2025-10-14T04:40:44.547Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/80/e6/7aab83774f5d2bca81f42ac58d04caf44f0cc2b65fc6db2b3b2e8a05f3b3/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_riscv64.whl", hash = "sha256:0d3d8f15c07f86e9ff82319b3d9ef6f4bf907608f53fe9d92b28ea9ae3d1fd89", size = 149383, upload-time = "2025-10-14T04:40:46.018Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4f/e8/b289173b4edae05c0dde07f69f8db476a0b511eac556dfe0d6bda3c43384/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_s390x.whl", hash = "sha256:9f7fcd74d410a36883701fafa2482a6af2ff5ba96b9a620e9e0721e28ead5569", size = 159098, upload-time = "2025-10-14T04:40:47.081Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d8/df/fe699727754cae3f8478493c7f45f777b17c3ef0600e28abfec8619eb49c/charset_normalizer-3.4.4-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:ebf3e58c7ec8a8bed6d66a75d7fb37b55e5015b03ceae72a8e7c74495551e224", size = 152991, upload-time = "2025-10-14T04:40:48.246Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1a/86/584869fe4ddb6ffa3bd9f491b87a01568797fb9bd8933f557dba9771beaf/charset_normalizer-3.4.4-cp311-cp311-win32.whl", hash = "sha256:eecbc200c7fd5ddb9a7f16c7decb07b566c29fa2161a16cf67b8d068bd21690a", size = 99456, upload-time = "2025-10-14T04:40:49.376Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/65/f6/62fdd5feb60530f50f7e38b4f6a1d5203f4d16ff4f9f0952962c044e919a/charset_normalizer-3.4.4-cp311-cp311-win_amd64.whl", hash = "sha256:5ae497466c7901d54b639cf42d5b8c1b6a4fead55215500d2f486d34db48d016", size = 106978, upload-time = "2025-10-14T04:40:50.844Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/9d/0710916e6c82948b3be62d9d398cb4fcf4e97b56d6a6aeccd66c4b2f2bd5/charset_normalizer-3.4.4-cp311-cp311-win_arm64.whl", hash = "sha256:65e2befcd84bc6f37095f5961e68a6f077bf44946771354a28ad434c2cce0ae1", size = 99969, upload-time = "2025-10-14T04:40:52.272Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f3/85/1637cd4af66fa687396e757dec650f28025f2a2f5a5531a3208dc0ec43f2/charset_normalizer-3.4.4-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:0a98e6759f854bd25a58a73fa88833fba3b7c491169f86ce1180c948ab3fd394", size = 208425, upload-time = "2025-10-14T04:40:53.353Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9d/6a/04130023fef2a0d9c62d0bae2649b69f7b7d8d24ea5536feef50551029df/charset_normalizer-3.4.4-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:b5b290ccc2a263e8d185130284f8501e3e36c5e02750fc6b6bdeb2e9e96f1e25", size = 148162, upload-time = "2025-10-14T04:40:54.558Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/78/29/62328d79aa60da22c9e0b9a66539feae06ca0f5a4171ac4f7dc285b83688/charset_normalizer-3.4.4-cp312-cp312-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74bb723680f9f7a6234dcf67aea57e708ec1fbdf5699fb91dfd6f511b0a320ef", size = 144558, upload-time = "2025-10-14T04:40:55.677Z" },
@@ -74,6 +42,38 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a8/ef/89297262b8092b312d29cdb2517cb1237e51db8ecef2e9af5edbe7b683b1/charset_normalizer-3.4.4-cp312-cp312-win32.whl", hash = "sha256:5833d2c39d8896e4e19b689ffc198f08ea58116bee26dea51e362ecc7cd3ed26", size = 99694, upload-time = "2025-10-14T04:41:09.23Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3d/2d/1e5ed9dd3b3803994c155cd9aacb60c82c331bad84daf75bcb9c91b3295e/charset_normalizer-3.4.4-cp312-cp312-win_amd64.whl", hash = "sha256:a79cfe37875f822425b89a82333404539ae63dbdddf97f84dcbc3d339aae9525", size = 107131, upload-time = "2025-10-14T04:41:10.467Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d0/d9/0ed4c7098a861482a7b6a95603edce4c0d9db2311af23da1fb2b75ec26fc/charset_normalizer-3.4.4-cp312-cp312-win_arm64.whl", hash = "sha256:376bec83a63b8021bb5c8ea75e21c4ccb86e7e45ca4eb81146091b56599b80c3", size = 100390, upload-time = "2025-10-14T04:41:11.915Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/97/45/4b3a1239bbacd321068ea6e7ac28875b03ab8bc0aa0966452db17cd36714/charset_normalizer-3.4.4-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:e1f185f86a6f3403aa2420e815904c67b2f9ebc443f045edd0de921108345794", size = 208091, upload-time = "2025-10-14T04:41:13.346Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7d/62/73a6d7450829655a35bb88a88fca7d736f9882a27eacdca2c6d505b57e2e/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:6b39f987ae8ccdf0d2642338faf2abb1862340facc796048b604ef14919e55ed", size = 147936, upload-time = "2025-10-14T04:41:14.461Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/c5/adb8c8b3d6625bef6d88b251bbb0d95f8205831b987631ab0c8bb5d937c2/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:3162d5d8ce1bb98dd51af660f2121c55d0fa541b46dff7bb9b9f86ea1d87de72", size = 144180, upload-time = "2025-10-14T04:41:15.588Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/91/ed/9706e4070682d1cc219050b6048bfd293ccf67b3d4f5a4f39207453d4b99/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:81d5eb2a312700f4ecaa977a8235b634ce853200e828fbadf3a9c50bab278328", size = 161346, upload-time = "2025-10-14T04:41:16.738Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d5/0d/031f0d95e4972901a2f6f09ef055751805ff541511dc1252ba3ca1f80cf5/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:5bd2293095d766545ec1a8f612559f6b40abc0eb18bb2f5d1171872d34036ede", size = 158874, upload-time = "2025-10-14T04:41:17.923Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f5/83/6ab5883f57c9c801ce5e5677242328aa45592be8a00644310a008d04f922/charset_normalizer-3.4.4-cp313-cp313-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a8a8b89589086a25749f471e6a900d3f662d1d3b6e2e59dcecf787b1cc3a1894", size = 153076, upload-time = "2025-10-14T04:41:19.106Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/75/1e/5ff781ddf5260e387d6419959ee89ef13878229732732ee73cdae01800f2/charset_normalizer-3.4.4-cp313-cp313-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:bc7637e2f80d8530ee4a78e878bce464f70087ce73cf7c1caf142416923b98f1", size = 150601, upload-time = "2025-10-14T04:41:20.245Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d7/57/71be810965493d3510a6ca79b90c19e48696fb1ff964da319334b12677f0/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_aarch64.whl", hash = "sha256:f8bf04158c6b607d747e93949aa60618b61312fe647a6369f88ce2ff16043490", size = 150376, upload-time = "2025-10-14T04:41:21.398Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e5/d5/c3d057a78c181d007014feb7e9f2e65905a6c4ef182c0ddf0de2924edd65/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_armv7l.whl", hash = "sha256:554af85e960429cf30784dd47447d5125aaa3b99a6f0683589dbd27e2f45da44", size = 144825, upload-time = "2025-10-14T04:41:22.583Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e6/8c/d0406294828d4976f275ffbe66f00266c4b3136b7506941d87c00cab5272/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_ppc64le.whl", hash = "sha256:74018750915ee7ad843a774364e13a3db91682f26142baddf775342c3f5b1133", size = 162583, upload-time = "2025-10-14T04:41:23.754Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d7/24/e2aa1f18c8f15c4c0e932d9287b8609dd30ad56dbe41d926bd846e22fb8d/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_riscv64.whl", hash = "sha256:c0463276121fdee9c49b98908b3a89c39be45d86d1dbaa22957e38f6321d4ce3", size = 150366, upload-time = "2025-10-14T04:41:25.27Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/e4/5b/1e6160c7739aad1e2df054300cc618b06bf784a7a164b0f238360721ab86/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_s390x.whl", hash = "sha256:362d61fd13843997c1c446760ef36f240cf81d3ebf74ac62652aebaf7838561e", size = 160300, upload-time = "2025-10-14T04:41:26.725Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/10/f882167cd207fbdd743e55534d5d9620e095089d176d55cb22d5322f2afd/charset_normalizer-3.4.4-cp313-cp313-musllinux_1_2_x86_64.whl", hash = "sha256:9a26f18905b8dd5d685d6d07b0cdf98a79f3c7a918906af7cc143ea2e164c8bc", size = 154465, upload-time = "2025-10-14T04:41:28.322Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/66/c7a9e1b7429be72123441bfdbaf2bc13faab3f90b933f664db506dea5915/charset_normalizer-3.4.4-cp313-cp313-win32.whl", hash = "sha256:9b35f4c90079ff2e2edc5b26c0c77925e5d2d255c42c74fdb70fb49b172726ac", size = 99404, upload-time = "2025-10-14T04:41:29.95Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c4/26/b9924fa27db384bdcd97ab83b4f0a8058d96ad9626ead570674d5e737d90/charset_normalizer-3.4.4-cp313-cp313-win_amd64.whl", hash = "sha256:b435cba5f4f750aa6c0a0d92c541fb79f69a387c91e61f1795227e4ed9cece14", size = 107092, upload-time = "2025-10-14T04:41:31.188Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/af/8f/3ed4bfa0c0c72a7ca17f0380cd9e4dd842b09f664e780c13cff1dcf2ef1b/charset_normalizer-3.4.4-cp313-cp313-win_arm64.whl", hash = "sha256:542d2cee80be6f80247095cc36c418f7bddd14f4a6de45af91dfad36d817bba2", size = 100408, upload-time = "2025-10-14T04:41:32.624Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2a/35/7051599bd493e62411d6ede36fd5af83a38f37c4767b92884df7301db25d/charset_normalizer-3.4.4-cp314-cp314-macosx_10_13_universal2.whl", hash = "sha256:da3326d9e65ef63a817ecbcc0df6e94463713b754fe293eaa03da99befb9a5bd", size = 207746, upload-time = "2025-10-14T04:41:33.773Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/10/9a/97c8d48ef10d6cd4fcead2415523221624bf58bcf68a802721a6bc807c8f/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:8af65f14dc14a79b924524b1e7fffe304517b2bff5a58bf64f30b98bbc5079eb", size = 147889, upload-time = "2025-10-14T04:41:34.897Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/10/bf/979224a919a1b606c82bd2c5fa49b5c6d5727aa47b4312bb27b1734f53cd/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_armv7l.manylinux_2_17_armv7l.manylinux_2_31_armv7l.whl", hash = "sha256:74664978bb272435107de04e36db5a9735e78232b85b77d45cfb38f758efd33e", size = 143641, upload-time = "2025-10-14T04:41:36.116Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ba/33/0ad65587441fc730dc7bd90e9716b30b4702dc7b617e6ba4997dc8651495/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_ppc64le.manylinux_2_17_ppc64le.manylinux_2_28_ppc64le.whl", hash = "sha256:752944c7ffbfdd10c074dc58ec2d5a8a4cd9493b314d367c14d24c17684ddd14", size = 160779, upload-time = "2025-10-14T04:41:37.229Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/67/ed/331d6b249259ee71ddea93f6f2f0a56cfebd46938bde6fcc6f7b9a3d0e09/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_s390x.manylinux_2_17_s390x.manylinux_2_28_s390x.whl", hash = "sha256:d1f13550535ad8cff21b8d757a3257963e951d96e20ec82ab44bc64aeb62a191", size = 159035, upload-time = "2025-10-14T04:41:38.368Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/67/ff/f6b948ca32e4f2a4576aa129d8bed61f2e0543bf9f5f2b7fc3758ed005c9/charset_normalizer-3.4.4-cp314-cp314-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:ecaae4149d99b1c9e7b88bb03e3221956f68fd6d50be2ef061b2381b61d20838", size = 152542, upload-time = "2025-10-14T04:41:39.862Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/16/85/276033dcbcc369eb176594de22728541a925b2632f9716428c851b149e83/charset_normalizer-3.4.4-cp314-cp314-manylinux_2_31_riscv64.manylinux_2_39_riscv64.whl", hash = "sha256:cb6254dc36b47a990e59e1068afacdcd02958bdcce30bb50cc1700a8b9d624a6", size = 149524, upload-time = "2025-10-14T04:41:41.319Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/9e/f2/6a2a1f722b6aba37050e626530a46a68f74e63683947a8acff92569f979a/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_aarch64.whl", hash = "sha256:c8ae8a0f02f57a6e61203a31428fa1d677cbe50c93622b4149d5c0f319c1d19e", size = 150395, upload-time = "2025-10-14T04:41:42.539Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/60/bb/2186cb2f2bbaea6338cad15ce23a67f9b0672929744381e28b0592676824/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_armv7l.whl", hash = "sha256:47cc91b2f4dd2833fddaedd2893006b0106129d4b94fdb6af1f4ce5a9965577c", size = 143680, upload-time = "2025-10-14T04:41:43.661Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7d/a5/bf6f13b772fbb2a90360eb620d52ed8f796f3c5caee8398c3b2eb7b1c60d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_ppc64le.whl", hash = "sha256:82004af6c302b5d3ab2cfc4cc5f29db16123b1a8417f2e25f9066f91d4411090", size = 162045, upload-time = "2025-10-14T04:41:44.821Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/df/c5/d1be898bf0dc3ef9030c3825e5d3b83f2c528d207d246cbabe245966808d/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_riscv64.whl", hash = "sha256:2b7d8f6c26245217bd2ad053761201e9f9680f8ce52f0fcd8d0755aeae5b2152", size = 149687, upload-time = "2025-10-14T04:41:46.442Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a5/42/90c1f7b9341eef50c8a1cb3f098ac43b0508413f33affd762855f67a410e/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_s390x.whl", hash = "sha256:799a7a5e4fb2d5898c60b640fd4981d6a25f1c11790935a44ce38c54e985f828", size = 160014, upload-time = "2025-10-14T04:41:47.631Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/76/be/4d3ee471e8145d12795ab655ece37baed0929462a86e72372fd25859047c/charset_normalizer-3.4.4-cp314-cp314-musllinux_1_2_x86_64.whl", hash = "sha256:99ae2cffebb06e6c22bdc25801d7b30f503cc87dbd283479e7b606f70aff57ec", size = 154044, upload-time = "2025-10-14T04:41:48.81Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b0/6f/8f7af07237c34a1defe7defc565a9bc1807762f672c0fde711a4b22bf9c0/charset_normalizer-3.4.4-cp314-cp314-win32.whl", hash = "sha256:f9d332f8c2a2fcbffe1378594431458ddbef721c1769d78e2cbc06280d8155f9", size = 99940, upload-time = "2025-10-14T04:41:49.946Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/4b/51/8ade005e5ca5b0d80fb4aff72a3775b325bdc3d27408c8113811a7cbe640/charset_normalizer-3.4.4-cp314-cp314-win_amd64.whl", hash = "sha256:8a6562c3700cce886c5be75ade4a5db4214fda19fede41d9792d100288d8f94c", size = 107104, upload-time = "2025-10-14T04:41:51.051Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/da/5f/6b8f83a55bb8278772c5ae54a577f3099025f9ade59d0136ac24a0df4bde/charset_normalizer-3.4.4-cp314-cp314-win_arm64.whl", hash = "sha256:de00632ca48df9daf77a2c65a484531649261ec9f25489917f09e455cb09ddb2", size = 100743, upload-time = "2025-10-14T04:41:52.122Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0a/4c/925909008ed5a988ccbb72dcc897407e5d6d3bd72410d69e051fc0c14647/charset_normalizer-3.4.4-py3-none-any.whl", hash = "sha256:7a32c560861a02ff789ad905a2fe94e3f840803362c84fecf1851cb4cf3dc37f", size = 53402, upload-time = "2025-10-14T04:42:31.76Z" },
]
@@ -86,18 +86,6 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/d6/3965ed04c63042e047cb6a3e6ed1a63a35087b6a609aa3a15ed8ac56c221/colorama-0.4.6-py2.py3-none-any.whl", hash = "sha256:4f1d9991f5acc0ca119f9d443620b77f9d6b33703e51011c16baf57afb285fc6", size = 25335, upload-time = "2022-10-25T02:36:20.889Z" },
]
[[package]]
name = "exceptiongroup"
version = "1.3.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
dependencies = [
{ name = "typing-extensions" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/50/79/66800aadf48771f6b62f7eb014e352e5d06856655206165d775e675a02c9/exceptiongroup-1.3.1.tar.gz", hash = "sha256:8b412432c6055b0b7d14c310000ae93352ed6754f70fa8f7c34141f91c4e3219", size = 30371, upload-time = "2025-11-21T23:01:54.787Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8a/0e/97c33bf5009bdbac74fd2beace167cab3f978feb69cc36f1ef79360d6c4e/exceptiongroup-1.3.1-py3-none-any.whl", hash = "sha256:a7a39a3bd276781e98394987d3a5701d0c4edffb633bb7a5144577f82c773598", size = 16740, upload-time = "2025-11-21T23:01:53.443Z" },
]
[[package]]
name = "idna"
version = "3.11"
@@ -149,6 +137,17 @@ version = "3.23.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c9/85/e24bf90972a30b0fcd16c73009add1d7d7cd9140c2498a68252028899e41/pycryptodomex-3.23.0.tar.gz", hash = "sha256:71909758f010c82bc99b0abf4ea12012c98962fbf0583c2164f8b84533c2e4da", size = 4922157, upload-time = "2025-05-17T17:23:41.434Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/2e/00/10edb04777069a42490a38c137099d4b17ba6e36a4e6e28bdc7470e9e853/pycryptodomex-3.23.0-cp313-cp313t-macosx_10_13_universal2.whl", hash = "sha256:7b37e08e3871efe2187bc1fd9320cc81d87caf19816c648f24443483005ff886", size = 2498764, upload-time = "2025-05-17T17:22:21.453Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6b/3f/2872a9c2d3a27eac094f9ceaa5a8a483b774ae69018040ea3240d5b11154/pycryptodomex-3.23.0-cp313-cp313t-macosx_10_13_x86_64.whl", hash = "sha256:91979028227543010d7b2ba2471cf1d1e398b3f183cb105ac584df0c36dac28d", size = 1643012, upload-time = "2025-05-17T17:22:23.702Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/70/af/774c2e2b4f6570fbf6a4972161adbb183aeeaa1863bde31e8706f123bf92/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:6b8962204c47464d5c1c4038abeadd4514a133b28748bcd9fa5b6d62e3cec6fa", size = 2187643, upload-time = "2025-05-17T17:22:26.37Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/de/a3/71065b24cb889d537954cedc3ae5466af00a2cabcff8e29b73be047e9a19/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:a33986a0066860f7fcf7c7bd2bc804fa90e434183645595ae7b33d01f3c91ed8", size = 2273762, upload-time = "2025-05-17T17:22:28.313Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/c9/0b/ff6f43b7fbef4d302c8b981fe58467b8871902cdc3eb28896b52421422cc/pycryptodomex-3.23.0-cp313-cp313t-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c7947ab8d589e3178da3d7cdeabe14f841b391e17046954f2fbcd941705762b5", size = 2313012, upload-time = "2025-05-17T17:22:30.57Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/02/de/9d4772c0506ab6da10b41159493657105d3f8bb5c53615d19452afc6b315/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_aarch64.whl", hash = "sha256:c25e30a20e1b426e1f0fa00131c516f16e474204eee1139d1603e132acffc314", size = 2186856, upload-time = "2025-05-17T17:22:32.819Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/28/ad/8b30efcd6341707a234e5eba5493700a17852ca1ac7a75daa7945fcf6427/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_i686.whl", hash = "sha256:da4fa650cef02db88c2b98acc5434461e027dce0ae8c22dd5a69013eaf510006", size = 2347523, upload-time = "2025-05-17T17:22:35.386Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0f/02/16868e9f655b7670dbb0ac4f2844145cbc42251f916fc35c414ad2359849/pycryptodomex-3.23.0-cp313-cp313t-musllinux_1_2_x86_64.whl", hash = "sha256:58b851b9effd0d072d4ca2e4542bf2a4abcf13c82a29fd2c93ce27ee2a2e9462", size = 2272825, upload-time = "2025-05-17T17:22:37.632Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ca/18/4ca89ac737230b52ac8ffaca42f9c6f1fd07c81a6cd821e91af79db60632/pycryptodomex-3.23.0-cp313-cp313t-win32.whl", hash = "sha256:a9d446e844f08299236780f2efa9898c818fe7e02f17263866b8550c7d5fb328", size = 1772078, upload-time = "2025-05-17T17:22:40Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/73/34/13e01c322db027682e00986873eca803f11c56ade9ba5bbf3225841ea2d4/pycryptodomex-3.23.0-cp313-cp313t-win_amd64.whl", hash = "sha256:bc65bdd9fc8de7a35a74cab1c898cab391a4add33a8fe740bda00f5976ca4708", size = 1803656, upload-time = "2025-05-17T17:22:42.139Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/54/68/9504c8796b1805d58f4425002bcca20f12880e6fa4dc2fc9a668705c7a08/pycryptodomex-3.23.0-cp313-cp313t-win_arm64.whl", hash = "sha256:c885da45e70139464f082018ac527fdaad26f1657a99ee13eecdce0f0ca24ab4", size = 1707172, upload-time = "2025-05-17T17:22:44.704Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/dd/9c/1a8f35daa39784ed8adf93a694e7e5dc15c23c741bbda06e1d45f8979e9e/pycryptodomex-3.23.0-cp37-abi3-macosx_10_9_universal2.whl", hash = "sha256:06698f957fe1ab229a99ba2defeeae1c09af185baa909a31a5d1f9d42b1aaed6", size = 2499240, upload-time = "2025-05-17T17:22:46.953Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7a/62/f5221a191a97157d240cf6643747558759126c76ee92f29a3f4aee3197a5/pycryptodomex-3.23.0-cp37-abi3-macosx_10_9_x86_64.whl", hash = "sha256:b2c2537863eccef2d41061e82a881dcabb04944c5c06c5aa7110b577cc487545", size = 1644042, upload-time = "2025-05-17T17:22:49.098Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8c/fd/5a054543c8988d4ed7b612721d7e78a4b9bf36bc3c5ad45ef45c22d0060e/pycryptodomex-3.23.0-cp37-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:43c446e2ba8df8889e0e16f02211c25b4934898384c1ec1ec04d7889c0333587", size = 2186227, upload-time = "2025-05-17T17:22:51.139Z" },
@@ -160,11 +159,6 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/8d/67/09ee8500dd22614af5fbaa51a4aee6e342b5fa8aecf0a6cb9cbf52fa6d45/pycryptodomex-3.23.0-cp37-abi3-win32.whl", hash = "sha256:189afbc87f0b9f158386bf051f720e20fa6145975f1e76369303d0f31d1a8d7c", size = 1771969, upload-time = "2025-05-17T17:23:07.115Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/69/96/11f36f71a865dd6df03716d33bd07a67e9d20f6b8d39820470b766af323c/pycryptodomex-3.23.0-cp37-abi3-win_amd64.whl", hash = "sha256:52e5ca58c3a0b0bd5e100a9fbc8015059b05cffc6c66ce9d98b4b45e023443b9", size = 1803124, upload-time = "2025-05-17T17:23:09.267Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f9/93/45c1cdcbeb182ccd2e144c693eaa097763b08b38cded279f0053ed53c553/pycryptodomex-3.23.0-cp37-abi3-win_arm64.whl", hash = "sha256:02d87b80778c171445d67e23d1caef279bf4b25c3597050ccd2e13970b57fd51", size = 1707161, upload-time = "2025-05-17T17:23:11.414Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f3/b8/3e76d948c3c4ac71335bbe75dac53e154b40b0f8f1f022dfa295257a0c96/pycryptodomex-3.23.0-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:ebfff755c360d674306e5891c564a274a47953562b42fb74a5c25b8fc1fb1cb5", size = 1627695, upload-time = "2025-05-17T17:23:17.38Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6a/cf/80f4297a4820dfdfd1c88cf6c4666a200f204b3488103d027b5edd9176ec/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:eca54f4bb349d45afc17e3011ed4264ef1cc9e266699874cdd1349c504e64798", size = 1675772, upload-time = "2025-05-17T17:23:19.202Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/d1/42/1e969ee0ad19fe3134b0e1b856c39bd0b70d47a4d0e81c2a8b05727394c9/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:4f2596e643d4365e14d0879dc5aafe6355616c61c2176009270f3048f6d9a61f", size = 1668083, upload-time = "2025-05-17T17:23:21.867Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/6e/c3/1de4f7631fea8a992a44ba632aa40e0008764c0fb9bf2854b0acf78c2cf2/pycryptodomex-3.23.0-pp310-pypy310_pp73-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:fdfac7cda115bca3a5abb2f9e43bc2fb66c2b65ab074913643803ca7083a79ea", size = 1706056, upload-time = "2025-05-17T17:23:24.031Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f2/5f/af7da8e6f1e42b52f44a24d08b8e4c726207434e2593732d39e7af5e7256/pycryptodomex-3.23.0-pp310-pypy310_pp73-win_amd64.whl", hash = "sha256:14c37aaece158d0ace436f76a7bb19093db3b4deade9797abfc39ec6cd6cc2fe", size = 1806478, upload-time = "2025-05-17T17:23:26.066Z" },
]
[[package]]
@@ -182,12 +176,10 @@ version = "9.0.1"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
dependencies = [
{ name = "colorama", marker = "sys_platform == 'win32'" },
{ name = "exceptiongroup", marker = "python_full_version < '3.11'" },
{ name = "iniconfig" },
{ name = "packaging" },
{ name = "pluggy" },
{ name = "pygments" },
{ name = "tomli", marker = "python_full_version < '3.11'" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/07/56/f013048ac4bc4c1d9be45afd4ab209ea62822fb1598f40687e6bf45dcea4/pytest-9.0.1.tar.gz", hash = "sha256:3e9c069ea73583e255c3b21cf46b8d3c56f6e3a1a8f6da94ccb0fcf57b9d73c8", size = 1564125, upload-time = "2025-11-12T13:05:09.333Z" }
wheels = [
@@ -196,7 +188,7 @@ wheels = [
[[package]]
name = "ragflow-cli"
version = "0.24.0"
version = "0.25.6"
source = { virtual = "." }
dependencies = [
{ name = "beartype" },
@@ -254,45 +246,11 @@ wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/3f/51/d4db610ef29373b879047326cbf6fa98b6c1969d6f6dc423279de2b1be2c/requests_toolbelt-1.0.0-py2.py3-none-any.whl", hash = "sha256:cccfdd665f0a24fcf4726e690f65639d272bb0637b9b92dfd91a5568ccf6bd06", size = 54481, upload-time = "2023-05-01T04:11:28.427Z" },
]
[[package]]
name = "tomli"
version = "2.3.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/52/ed/3f73f72945444548f33eba9a87fc7a6e969915e7b1acc8260b30e1f76a2f/tomli-2.3.0.tar.gz", hash = "sha256:64be704a875d2a59753d80ee8a533c3fe183e3f06807ff7dc2232938ccb01549", size = 17392, upload-time = "2025-10-08T22:01:47.119Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b3/2e/299f62b401438d5fe1624119c723f5d877acc86a4c2492da405626665f12/tomli-2.3.0-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:88bd15eb972f3664f5ed4b57c1634a97153b4bac4479dcb6a495f41921eb7f45", size = 153236, upload-time = "2025-10-08T22:01:00.137Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/86/7f/d8fffe6a7aefdb61bced88fcb5e280cfd71e08939da5894161bd71bea022/tomli-2.3.0-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:883b1c0d6398a6a9d29b508c331fa56adbcdff647f6ace4dfca0f50e90dfd0ba", size = 148084, upload-time = "2025-10-08T22:01:01.63Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/47/5c/24935fb6a2ee63e86d80e4d3b58b222dafaf438c416752c8b58537c8b89a/tomli-2.3.0-cp311-cp311-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:d1381caf13ab9f300e30dd8feadb3de072aeb86f1d34a8569453ff32a7dea4bf", size = 234832, upload-time = "2025-10-08T22:01:02.543Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/89/da/75dfd804fc11e6612846758a23f13271b76d577e299592b4371a4ca4cd09/tomli-2.3.0-cp311-cp311-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:a0e285d2649b78c0d9027570d4da3425bdb49830a6156121360b3f8511ea3441", size = 242052, upload-time = "2025-10-08T22:01:03.836Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/70/8c/f48ac899f7b3ca7eb13af73bacbc93aec37f9c954df3c08ad96991c8c373/tomli-2.3.0-cp311-cp311-musllinux_1_2_aarch64.whl", hash = "sha256:0a154a9ae14bfcf5d8917a59b51ffd5a3ac1fd149b71b47a3a104ca4edcfa845", size = 239555, upload-time = "2025-10-08T22:01:04.834Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ba/28/72f8afd73f1d0e7829bfc093f4cb98ce0a40ffc0cc997009ee1ed94ba705/tomli-2.3.0-cp311-cp311-musllinux_1_2_x86_64.whl", hash = "sha256:74bf8464ff93e413514fefd2be591c3b0b23231a77f901db1eb30d6f712fc42c", size = 245128, upload-time = "2025-10-08T22:01:05.84Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/b6/eb/a7679c8ac85208706d27436e8d421dfa39d4c914dcf5fa8083a9305f58d9/tomli-2.3.0-cp311-cp311-win32.whl", hash = "sha256:00b5f5d95bbfc7d12f91ad8c593a1659b6387b43f054104cda404be6bda62456", size = 96445, upload-time = "2025-10-08T22:01:06.896Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/0a/fe/3d3420c4cb1ad9cb462fb52967080575f15898da97e21cb6f1361d505383/tomli-2.3.0-cp311-cp311-win_amd64.whl", hash = "sha256:4dc4ce8483a5d429ab602f111a93a6ab1ed425eae3122032db7e9acf449451be", size = 107165, upload-time = "2025-10-08T22:01:08.107Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/ff/b7/40f36368fcabc518bb11c8f06379a0fd631985046c038aca08c6d6a43c6e/tomli-2.3.0-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:d7d86942e56ded512a594786a5ba0a5e521d02529b3826e7761a05138341a2ac", size = 154891, upload-time = "2025-10-08T22:01:09.082Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f9/3f/d9dd692199e3b3aab2e4e4dd948abd0f790d9ded8cd10cbaae276a898434/tomli-2.3.0-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:73ee0b47d4dad1c5e996e3cd33b8a76a50167ae5f96a2607cbe8cc773506ab22", size = 148796, upload-time = "2025-10-08T22:01:10.266Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/60/83/59bff4996c2cf9f9387a0f5a3394629c7efa5ef16142076a23a90f1955fa/tomli-2.3.0-cp312-cp312-manylinux2014_aarch64.manylinux_2_17_aarch64.manylinux_2_28_aarch64.whl", hash = "sha256:792262b94d5d0a466afb5bc63c7daa9d75520110971ee269152083270998316f", size = 242121, upload-time = "2025-10-08T22:01:11.332Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/45/e5/7c5119ff39de8693d6baab6c0b6dcb556d192c165596e9fc231ea1052041/tomli-2.3.0-cp312-cp312-manylinux2014_x86_64.manylinux_2_17_x86_64.manylinux_2_28_x86_64.whl", hash = "sha256:4f195fe57ecceac95a66a75ac24d9d5fbc98ef0962e09b2eddec5d39375aae52", size = 250070, upload-time = "2025-10-08T22:01:12.498Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/45/12/ad5126d3a278f27e6701abde51d342aa78d06e27ce2bb596a01f7709a5a2/tomli-2.3.0-cp312-cp312-musllinux_1_2_aarch64.whl", hash = "sha256:e31d432427dcbf4d86958c184b9bfd1e96b5b71f8eb17e6d02531f434fd335b8", size = 245859, upload-time = "2025-10-08T22:01:13.551Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/fb/a1/4d6865da6a71c603cfe6ad0e6556c73c76548557a8d658f9e3b142df245f/tomli-2.3.0-cp312-cp312-musllinux_1_2_x86_64.whl", hash = "sha256:7b0882799624980785240ab732537fcfc372601015c00f7fc367c55308c186f6", size = 250296, upload-time = "2025-10-08T22:01:14.614Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a0/b7/a7a7042715d55c9ba6e8b196d65d2cb662578b4d8cd17d882d45322b0d78/tomli-2.3.0-cp312-cp312-win32.whl", hash = "sha256:ff72b71b5d10d22ecb084d345fc26f42b5143c5533db5e2eaba7d2d335358876", size = 97124, upload-time = "2025-10-08T22:01:15.629Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/06/1e/f22f100db15a68b520664eb3328fb0ae4e90530887928558112c8d1f4515/tomli-2.3.0-cp312-cp312-win_amd64.whl", hash = "sha256:1cb4ed918939151a03f33d4242ccd0aa5f11b3547d0cf30f7c74a408a5b99878", size = 107698, upload-time = "2025-10-08T22:01:16.51Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/77/b8/0135fadc89e73be292b473cb820b4f5a08197779206b33191e801feeae40/tomli-2.3.0-py3-none-any.whl", hash = "sha256:e95b1af3c5b07d9e643909b5abbec77cd9f1217e6d0bca72b0234736b9fb1f1b", size = 14408, upload-time = "2025-10-08T22:01:46.04Z" },
]
[[package]]
name = "typing-extensions"
version = "4.15.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/72/94/1a15dd82efb362ac84269196e94cf00f187f7ed21c242792a923cdb1c61f/typing_extensions-4.15.0.tar.gz", hash = "sha256:0cea48d173cc12fa28ecabc3b837ea3cf6f38c6d1136f85cbaaf598984861466", size = 109391, upload-time = "2025-08-25T13:49:26.313Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/18/67/36e9267722cc04a6b9f15c7f3441c2363321a3ea07da7ae0c0707beb2a9c/typing_extensions-4.15.0-py3-none-any.whl", hash = "sha256:f0fa19c6845758ab08074a0cfa8b7aecb71c999ca73d62883bc25cc018c4e548", size = 44614, upload-time = "2025-08-25T13:49:24.86Z" },
]
[[package]]
name = "urllib3"
version = "2.5.0"
version = "2.6.3"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/15/22/9ee70a2574a4f4599c47dd506532914ce044817c7752a79b6a51286319bc/urllib3-2.5.0.tar.gz", hash = "sha256:3fc47733c7e419d4bc3f6b3dc2b4f890bb743906a30d56ba4a5bfa4bbff92760", size = 393185, upload-time = "2025-06-18T14:07:41.644Z" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/a7/c2/fe1e52489ae3122415c51f387e221dd0773709bad6c6cdaa599e8a2c5185/urllib3-2.5.0-py3-none-any.whl", hash = "sha256:e6b01673c0fa6a13e374b50871808eb3bf7046c4b125b216f6bf1cc604cff0dc", size = 129795, upload-time = "2025-06-18T14:07:40.39Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" },
]

View File

@@ -21,7 +21,6 @@ import os
import signal
import logging
import threading
import traceback
import faulthandler
from flask import Flask
@@ -58,7 +57,7 @@ if __name__ == '__main__':
os.environ.get("MAX_CONTENT_LENGTH", 1024 * 1024 * 1024)
)
Session(app)
logging.info(f'RAGFlow version: {get_ragflow_version()}')
logging.info(f'RAGFlow admin version: {get_ragflow_version()}')
show_configs()
login_manager = LoginManager()
login_manager.init_app(app)
@@ -75,10 +74,10 @@ if __name__ == '__main__':
application=app,
threaded=True,
use_reloader=False,
use_debugger=True,
use_debugger=False,
)
except Exception:
traceback.print_exc()
except Exception as e:
logging.exception(f"Unhandled exception: {e}")
stop_event.set()
time.sleep(1)
os.kill(os.getpid(), signal.SIGKILL)

View File

@@ -22,7 +22,6 @@ from datetime import datetime
from flask import jsonify, request
from flask_login import current_user, login_user
from itsdangerous.url_safe import URLSafeTimedSerializer as Serializer
from api.common.exceptions import AdminException, UserNotFoundError
from api.common.base64 import encode_to_base64
@@ -40,18 +39,34 @@ from common import settings
def setup_auth(login_manager):
@login_manager.request_loader
def load_user(web_request):
jwt = Serializer(secret_key=settings.SECRET_KEY)
# Authorization header contains JWT-encoded access token
# First decode JWT to get the UUID, then query database
from itsdangerous.url_safe import URLSafeTimedSerializer as Serializer
from common import settings
authorization = web_request.headers.get("Authorization")
if authorization:
try:
access_token = str(jwt.loads(authorization))
# Strip "Bearer " prefix if present
jwt_token = authorization
if jwt_token.startswith("Bearer "):
jwt_token = jwt_token[7:]
if not access_token or not access_token.strip():
logging.warning("Authentication attempt with empty access token")
jwt_token = jwt_token.strip()
if not jwt_token:
logging.warning("Authentication attempt with empty JWT token")
return None
# Access tokens should be UUIDs (32 hex characters)
if len(access_token.strip()) < 32:
# Decode JWT to get the UUID access_token
jwt = Serializer(secret_key=settings.get_secret_key())
access_token = str(jwt.loads(jwt_token))
if not access_token or not access_token.strip():
logging.warning("Authentication attempt with empty access token after JWT decode")
return None
# Access tokens stored in database are UUIDs (32 hex characters)
if len(access_token) < 32:
logging.warning(f"Authentication attempt with invalid token format: {len(access_token)} chars")
return None
@@ -110,7 +125,8 @@ def add_tenant_for_admin(user_info: dict, role: str):
"embd_id": settings.EMBEDDING_MDL,
"asr_id": settings.ASR_MDL,
"parser_ids": settings.PARSERS,
"img2txt_id": settings.IMAGE2TEXT_MDL
"img2txt_id": settings.IMAGE2TEXT_MDL,
"rerank_id": settings.RERANK_MDL,
}
usr_tenant = {
"tenant_id": user_info["id"],

View File

@@ -264,6 +264,19 @@ def load_configurations(config_path: str) -> list[BaseConfig]:
db_name=database, detail_func_name="get_infinity_status")
configurations.append(config)
id_count += 1
case "minio_0":
name: str = 'minio_0'
url = v['host']
parts = url.split(':', 1)
host = parts[0]
port = int(parts[1])
user = v.get('user')
password = v.get('password')
config = MinioConfig(id=id_count, name=name, host=host, port=port, user=user, password=password,
service_type="file_store",
store_type="minio", detail_func_name="check_minio_alive")
configurations.append(config)
id_count += 1
case "minio":
name: str = 'minio'
url = v['host']
@@ -310,6 +323,14 @@ def load_configurations(config_path: str) -> list[BaseConfig]:
service_type="task_executor", detail_func_name="check_task_executor_alive")
configurations.append(config)
id_count += 1
case "rabbitmq":
name: str = 'rabbitmq'
host: str = v.get('host')
port: int = v.get('port')
config = RabbitMQConfig(id=id_count, name=name, host=host, port=port,
service_type="message_queue", mq_type="rabbitmq", detail_func_name="check_rabbitmq_alive")
configurations.append(config)
id_count += 1
case _:
logging.warning(f"Unknown configuration key: {k}")
continue

View File

@@ -30,13 +30,14 @@ from roles import RoleMgr
from api.common.exceptions import AdminException
from common.versions import get_ragflow_version
from api.utils.api_utils import generate_confirmation_token
from common.log_utils import get_log_levels, set_log_level
admin_bp = Blueprint("admin", __name__, url_prefix="/api/v1/admin")
@admin_bp.route("/ping", methods=["GET"])
def ping():
return success_response("PONG")
return success_response(message="pong")
@admin_bp.route("/login", methods=["POST"])
@@ -420,7 +421,7 @@ def get_user_permission(user_name: str):
def set_variable():
try:
data = request.get_json()
if not data and "var_name" not in data:
if not data or "var_name" not in data:
return error_response("Var name is required", 400)
if "var_value" not in data:
@@ -448,7 +449,7 @@ def get_variable():
# get var
data = request.get_json()
if not data and "var_name" not in data:
if not data or "var_name" not in data:
return error_response("Var name is required", 400)
var_name: str = data["var_name"]
res = SettingsMgr.get_by_name(var_name)
@@ -652,3 +653,39 @@ def test_sandbox_connection():
return error_response(str(e), 400)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/log_levels", methods=["GET"])
@login_required
@check_admin_auth
def get_logger_levels():
"""Get current log levels for all packages."""
try:
res = get_log_levels()
return success_response(res, "Get log levels", 0)
except Exception as e:
return error_response(str(e), 500)
@admin_bp.route("/log_levels", methods=["PUT"])
@login_required
@check_admin_auth
def set_logger_level():
"""Set log level for a package."""
try:
data = request.get_json()
if not data or "pkg_name" not in data or "level" not in data:
return error_response("pkg_name and level are required", 400)
pkg_name = data["pkg_name"]
level = data["level"]
if not isinstance(pkg_name, str) or not isinstance(level, str):
return error_response("pkg_name and level must be strings", 400)
success = set_log_level(pkg_name, level)
if success:
return success_response({"pkg_name": pkg_name, "level": level}, "Log level updated successfully")
else:
return error_response(f"Invalid log level: {level}", 400)
except Exception as e:
return error_response(str(e), 500)

View File

@@ -330,36 +330,65 @@ class ServiceMgr:
class SettingsMgr:
@staticmethod
def _format_setting(setting):
return {
"data_type": setting.data_type,
"name": setting.name,
"setting_type": "config",
"value": setting.value,
}
@staticmethod
def _validate_value(name: str, data_type: str, value: str):
data_type = data_type.lower()
value = str(value)
if data_type == "string":
return
if data_type == "integer":
try:
int(value)
except ValueError:
raise AdminException(f"Invalid integer value for {name}: {value}")
return
if data_type in {"bool", "boolean"}:
if value not in {"true", "false"}:
raise AdminException(f"Invalid bool value for {name}: expected true or false")
return
if data_type == "json":
try:
json.loads(value)
except json.JSONDecodeError:
raise AdminException(f"Invalid JSON value for {name}")
return
raise AdminException(f"Unsupported data type for {name}: {data_type}")
@staticmethod
def _infer_data_type(name: str):
if name.startswith("sandbox."):
return "json"
if name.endswith(".enabled"):
return "bool"
return "string"
@staticmethod
def get_all():
settings = SystemSettingsService.get_all()
settings = SystemSettingsService.get_all(reverse=False, order_by="name")
result = []
for setting in settings:
result.append(
{
"name": setting.name,
"source": setting.source,
"data_type": setting.data_type,
"value": setting.value,
}
)
result.append(SettingsMgr._format_setting(setting))
return result
@staticmethod
def get_by_name(name: str):
settings = SystemSettingsService.get_by_name(name)
if len(settings) == 0:
raise AdminException(f"Can't get setting: {name}")
settings = SystemSettingsService.get_by_name_prefix(name)
if len(settings) == 0:
raise AdminException(f"Can't get setting: {name}")
result = []
for setting in settings:
result.append(
{
"name": setting.name,
"source": setting.source,
"data_type": setting.data_type,
"value": setting.value,
}
)
result.append(SettingsMgr._format_setting(setting))
return result
@staticmethod
@@ -367,6 +396,7 @@ class SettingsMgr:
settings = SystemSettingsService.get_by_name(name)
if len(settings) == 1:
setting = settings[0]
SettingsMgr._validate_value(name, setting.data_type, value)
setting.value = value
setting_dict = setting.to_dict()
SystemSettingsService.update_by_name(name, setting_dict)
@@ -376,12 +406,8 @@ class SettingsMgr:
# Create new setting if it doesn't exist
# Determine data_type based on name and value
if name.startswith("sandbox."):
data_type = "json"
elif name.endswith(".enabled"):
data_type = "boolean"
else:
data_type = "string"
data_type = SettingsMgr._infer_data_type(name)
SettingsMgr._validate_value(name, data_type, value)
new_setting = {
"name": name,
@@ -431,11 +457,21 @@ class SandboxMgr:
# Provider registry with metadata
PROVIDER_REGISTRY = {
"local": {
"name": "Local",
"description": "Execute code directly on the current host process.",
"tags": ["local", "host", "minimal"],
},
"self_managed": {
"name": "Self-Managed",
"description": "On-premise deployment using Daytona/Docker",
"tags": ["self-hosted", "low-latency", "secure"],
},
"ssh": {
"name": "SSH",
"description": "Execute code on a remote machine over SSH.",
"tags": ["remote", "ssh", "custom-runtime"],
},
"aliyun_codeinterpreter": {
"name": "Aliyun Code Interpreter",
"description": "Aliyun Function Compute Code Interpreter - Code execution in serverless microVMs",
@@ -463,13 +499,17 @@ class SandboxMgr:
def get_provider_config_schema(provider_id: str):
"""Get configuration schema for a specific provider."""
from agent.sandbox.providers import (
LocalProvider,
SelfManagedProvider,
SSHProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
schemas = {
"local": LocalProvider.get_config_schema(),
"self_managed": SelfManagedProvider.get_config_schema(),
"ssh": SSHProvider.get_config_schema(),
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider.get_config_schema(),
"e2b": E2BProvider.get_config_schema(),
}
@@ -486,7 +526,6 @@ class SandboxMgr:
# Get active provider type
provider_type_settings = SystemSettingsService.get_by_name("sandbox.provider_type")
if not provider_type_settings:
# Return default config if not set
provider_type = "self_managed"
else:
provider_type = provider_type_settings[0].value
@@ -501,6 +540,15 @@ class SandboxMgr:
except json.JSONDecodeError:
provider_config = {}
if not provider_config:
schema = SandboxMgr.get_provider_config_schema(provider_type)
provider_config = {}
for field_name, field_schema in schema.items():
if field_schema.get("readonly"):
continue
if field_schema.get("default") is not None:
provider_config[field_name] = field_schema["default"]
return {
"provider_type": provider_type,
"config": provider_config,
@@ -524,7 +572,9 @@ class SandboxMgr:
Dictionary with updated provider_type and config
"""
from agent.sandbox.providers import (
LocalProvider,
SelfManagedProvider,
SSHProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
@@ -551,7 +601,7 @@ class SandboxMgr:
elif field_type == "string":
if not isinstance(config[field_name], str):
raise AdminException(f"Field '{field_name}' must be a string")
elif field_type == "bool":
elif field_type == "boolean":
if not isinstance(config[field_name], bool):
raise AdminException(f"Field '{field_name}' must be a boolean")
@@ -566,7 +616,9 @@ class SandboxMgr:
# Provider-specific custom validation
provider_classes = {
"local": LocalProvider,
"self_managed": SelfManagedProvider,
"ssh": SSHProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
}
@@ -582,6 +634,8 @@ class SandboxMgr:
# Always update the provider config
config_json = json.dumps(config)
SettingsMgr.update_by_name(f"sandbox.{provider_type}", config_json)
from agent.sandbox.client import reload_provider
reload_provider()
return {"provider_type": provider_type, "config": config}
except AdminException:
@@ -608,14 +662,18 @@ class SandboxMgr:
"""
try:
from agent.sandbox.providers import (
LocalProvider,
SelfManagedProvider,
SSHProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
)
# Instantiate provider based on type
provider_classes = {
"local": LocalProvider,
"self_managed": SelfManagedProvider,
"ssh": SSHProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
}
@@ -631,59 +689,40 @@ class SandboxMgr:
# Create a temporary sandbox instance for testing
instance = provider.create_instance(template="python")
if not instance:
raise AdminException("Failed to create sandbox instance.")
if not instance or instance.status != "READY":
raise AdminException(f"Failed to create sandbox instance. Status: {instance.status if instance else 'None'}")
# Simple test code that exercises basic Python functionality
test_code = """
# Test basic Python functionality
import sys
try:
# Simple test code that exercises provider wrapping via main().
test_code = """
import json
import math
import sys
print("Python version:", sys.version)
print("Platform:", sys.platform)
# Test basic calculations
result = 2 + 2
print(f"2 + 2 = {result}")
# Test JSON operations
data = {"test": "data", "value": 123}
print(f"JSON dump: {json.dumps(data)}")
# Test math operations
print(f"Math.sqrt(16) = {math.sqrt(16)}")
# Test error handling
try:
x = 1 / 1
print("Division test: OK")
except Exception as e:
print(f"Error: {e}")
# Return success indicator
print("TEST_PASSED")
def main() -> dict:
print("Python version:", sys.version)
print("Platform:", sys.platform)
print(f"2 + 2 = {2 + 2}")
print(f"JSON dump: {json.dumps({'test': 'data', 'value': 123})}")
print(f"Math.sqrt(16) = {math.sqrt(16)}")
print("TEST_PASSED")
return {"ok": True, "provider_test": "TEST_PASSED"}
"""
# Execute test code with timeout
execution_result = provider.execute_code(
instance_id=instance.instance_id,
code=test_code,
language="python",
timeout=10 # 10 seconds timeout
)
# Clean up the test instance (if provider supports it)
try:
if hasattr(provider, 'terminate_instance'):
provider.terminate_instance(instance.instance_id)
# Execute test code with timeout
execution_result = provider.execute_code(
instance_id=instance.instance_id,
code=test_code,
language="python",
timeout=10,
)
finally:
try:
provider.destroy_instance(instance.instance_id)
logging.info(f"Cleaned up test instance {instance.instance_id}")
else:
logging.warning(f"Provider {provider_type} does not support terminate_instance, test instance may leak")
except Exception as cleanup_error:
logging.warning(f"Failed to cleanup test instance {instance.instance_id}: {cleanup_error}")
except Exception as cleanup_error:
logging.warning(f"Failed to cleanup test instance {instance.instance_id}: {cleanup_error}")
# Build detailed result message
success = execution_result.exit_code == 0 and "TEST_PASSED" in execution_result.stdout

View File

@@ -15,8 +15,8 @@
#
import asyncio
import base64
import datetime
import inspect
import binascii
import json
import logging
import re
@@ -28,14 +28,17 @@ from typing import Any, Union, Tuple
from agent.component import component_class
from agent.component.base import ComponentBase
from agent.dsl_migration import normalize_chunker_dsl
from api.db.services.file_service import FileService
from api.db.services.llm_service import LLMBundle
from api.db.services.task_service import has_canceled
from api.db.joint_services.tenant_model_service import get_tenant_default_model_by_type
from common.constants import LLMType
from common.misc_utils import get_uuid, hash_str2int
from common.exceptions import TaskCanceledException
from rag.prompts.generator import chunks_format
from rag.utils.redis_conn import REDIS_CONN
from rag.utils.tts_cache import synthesize_with_cache
class Graph:
"""
@@ -82,7 +85,8 @@ class Graph:
self.path = []
self.components = {}
self.error = ""
self.dsl = json.loads(dsl)
# Accept legacy DSL on read, but keep the in-memory canvas in the latest schema.
self.dsl = normalize_chunker_dsl(json.loads(dsl))
self._tenant_id = tenant_id
self.task_id = task_id if task_id else get_uuid()
self.custom_header = custom_header
@@ -259,7 +263,7 @@ class Graph:
keys = path.split('.')
if not path:
return value
for key in keys:
for key in keys[:-1]:
if key not in cur or not isinstance(cur[key], dict):
cur[key] = {}
cur = cur[key]
@@ -286,7 +290,8 @@ class Canvas(Graph):
"sys.user_id": tenant_id,
"sys.conversation_turns": 0,
"sys.files": [],
"sys.history": []
"sys.history": [],
"sys.date": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
}
self.variables = {}
super().__init__(dsl, tenant_id, task_id, custom_header=custom_header)
@@ -299,13 +304,16 @@ class Canvas(Graph):
self.globals = self.dsl["globals"]
if "sys.history" not in self.globals:
self.globals["sys.history"] = []
if "sys.date" not in self.globals:
self.globals["sys.date"] = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
else:
self.globals = {
"sys.query": "",
"sys.user_id": "",
"sys.conversation_turns": 0,
"sys.files": [],
"sys.history": []
"sys.history": [],
"sys.date": datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
}
if "variables" in self.dsl:
self.variables = self.dsl["variables"]
@@ -346,34 +354,35 @@ class Canvas(Graph):
key = k[4:]
if key in self.variables:
variable = self.variables[key]
if variable["type"] == "string":
self.globals[k] = ""
variable["value"] = ""
elif variable["type"] == "number":
self.globals[k] = 0
variable["value"] = 0
elif variable["type"] == "boolean":
self.globals[k] = False
variable["value"] = False
elif variable["type"] == "object":
self.globals[k] = {}
variable["value"] = {}
elif variable["type"].startswith("array"):
self.globals[k] = []
variable["value"] = []
value = variable.get("value")
if value is not None:
self.globals[k] = value
else:
self.globals[k] = ""
var_type = variable.get("type", "")
if var_type == "number":
self.globals[k] = 0
elif var_type == "boolean":
self.globals[k] = False
elif var_type == "object":
self.globals[k] = {}
elif var_type.startswith("array"):
self.globals[k] = []
else: # "string" or unknown
self.globals[k] = ""
else:
self.globals[k] = ""
async def run(self, **kwargs):
self.globals["sys.date"] = datetime.datetime.now(datetime.timezone.utc).strftime("%Y-%m-%d %H:%M:%S")
st = time.perf_counter()
self._loop = asyncio.get_running_loop()
self.message_id = get_uuid()
created_at = int(time.time())
self.add_user_input(kwargs.get("query"))
path_set = set(self.path)
for k, cpn in self.components.items():
self.components[k]["obj"].reset(True)
if k in path_set:
self.components[k]["obj"].reset(True)
if kwargs.get("webhook_payload"):
for k, cpn in self.components.items():
@@ -386,10 +395,16 @@ class Canvas(Graph):
continue
self.components[k]["obj"].set_output(kk, vv)
layout_recognize = None
for cpn in self.components.values():
if cpn["obj"].component_name.lower() == "begin":
layout_recognize = getattr(cpn["obj"]._param, "layout_recognize", None)
break
for k in kwargs.keys():
if k in ["query", "user_id", "files"] and kwargs[k]:
if k in ["query", "user_id", "files", "chat_template_kwargs"] and kwargs[k]:
if k == "files":
self.globals[f"sys.{k}"] = await self.get_files_async(kwargs[k])
self.globals[f"sys.{k}"] = await self.get_files_async(kwargs[k], layout_recognize)
else:
self.globals[f"sys.{k}"] = kwargs[k]
if not self.globals["sys.conversation_turns"] :
@@ -502,7 +517,8 @@ class Canvas(Graph):
cpn_obj = self.get_component_obj(self.path[i])
if cpn_obj.component_name.lower() == "message":
if cpn_obj.get_param("auto_play"):
tts_mdl = LLMBundle(self._tenant_id, LLMType.TTS)
tts_model_config = get_tenant_default_model_by_type(self._tenant_id, LLMType.TTS)
tts_mdl = LLMBundle(self._tenant_id, tts_model_config)
if isinstance(cpn_obj.output("content"), partial):
_m = ""
buff_m = ""
@@ -547,18 +563,10 @@ class Canvas(Graph):
yield decorate("message", {"content": "", "audio_binary": self.tts(tts_mdl, buff_m)})
buff_m = ""
cpn_obj.set_output("content", _m)
cite = re.search(r"\[ID:[ 0-9]+\]", _m)
else:
yield decorate("message", {"content": cpn_obj.output("content")})
cite = re.search(r"\[ID:[ 0-9]+\]", cpn_obj.output("content"))
message_end = {}
if cpn_obj.get_param("status"):
message_end["status"] = cpn_obj.get_param("status")
if isinstance(cpn_obj.output("attachment"), dict):
message_end["attachment"] = cpn_obj.output("attachment")
if cite:
message_end["reference"] = self.get_reference()
message_end = self._build_message_end(cpn_obj)
yield decorate("message_end", message_end)
while partials:
@@ -706,14 +714,7 @@ class Canvas(Graph):
text = clean_tts_text(text)
if not text:
return None
bin = b""
try:
for chunk in tts_mdl.tts(text):
bin += chunk
except Exception as e:
logging.error(f"TTS failed: {e}, text={text!r}")
return None
return binascii.hexlify(bin).decode("utf-8")
return synthesize_with_cache(tts_mdl, text)
def get_history(self, window_size):
convs = []
@@ -748,7 +749,7 @@ class Canvas(Graph):
def get_component_input_elements(self, cpnnm):
return self.components[cpnnm]["obj"].get_input_elements()
async def get_files_async(self, files: Union[None, list[dict]]) -> list[str]:
async def get_files_async(self, files: Union[None, list[dict]], layout_recognize: str = None) -> list[str]:
if not files:
return []
def image_to_base64(file):
@@ -756,7 +757,7 @@ class Canvas(Graph):
base64.b64encode(FileService.get_blob(file["created_by"], file["id"])).decode("utf-8"))
def parse_file(file):
blob = FileService.get_blob(file["created_by"], file["id"])
return FileService.parse(file["name"], blob, True, file["created_by"])
return FileService.parse(file["name"], blob, True, file["created_by"], layout_recognize)
loop = asyncio.get_running_loop()
tasks = []
for file in files:
@@ -766,15 +767,15 @@ class Canvas(Graph):
tasks.append(loop.run_in_executor(self._thread_pool, parse_file, file))
return await asyncio.gather(*tasks)
def get_files(self, files: Union[None, list[dict]]) -> list[str]:
def get_files(self, files: Union[None, list[dict]], layout_recognize: str = None) -> list[str]:
"""
Synchronous wrapper for get_files_async, used by sync component invoke paths.
"""
loop = getattr(self, "_loop", None)
if loop and loop.is_running():
return asyncio.run_coroutine_threadsafe(self.get_files_async(files), loop).result()
return asyncio.run_coroutine_threadsafe(self.get_files_async(files, layout_recognize), loop).result()
return asyncio.run(self.get_files_async(files))
return asyncio.run(self.get_files_async(files, layout_recognize))
def tool_use_callback(self, agent_id: str, func_name: str, params: dict, result: Any, elapsed_time=None):
agent_ids = agent_id.split("-->")
@@ -820,6 +821,22 @@ class Canvas(Graph):
return {"chunks": {}, "doc_aggs": {}}
return self.retrieval[-1]
def _has_reference(self) -> bool:
ref = self.get_reference()
if not isinstance(ref, dict):
return False
return bool(ref.get("chunks") or ref.get("doc_aggs"))
def _build_message_end(self, cpn_obj) -> dict:
message_end = {}
if cpn_obj.get_param("status"):
message_end["status"] = cpn_obj.get_param("status")
if isinstance(cpn_obj.output("attachment"), dict):
message_end["attachment"] = cpn_obj.output("attachment")
if self._has_reference():
message_end["reference"] = self.get_reference()
return message_end
def add_memory(self, user:str, assist:str, summ: str):
self.memory.append((user, assist, summ))

View File

@@ -20,19 +20,20 @@ import os
import re
from copy import deepcopy
from functools import partial
from timeit import default_timer as timer
from typing import Any
import json_repair
from timeit import default_timer as timer
from agent.tools.base import LLMToolPluginCallSession, ToolParamBase, ToolBase, ToolMeta
from agent.component.llm import LLM, LLMParam
from agent.tools.base import LLMToolPluginCallSession, ToolBase, ToolMeta, ToolParamBase
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from api.db.services.llm_service import LLMBundle
from api.db.services.tenant_llm_service import TenantLLMService
from api.db.services.mcp_server_service import MCPServerService
from api.db.services.tenant_llm_service import TenantLLMService
from common.connection_utils import timeout
from rag.prompts.generator import next_step_async, COMPLETE_TASK, \
citation_prompt, kb_prompt, citation_plus, full_question, message_fit_in, structured_output_prompt
from common.mcp_tool_call_conn import MCPToolCallSession, mcp_tool_metadata_to_openai_tool
from agent.component.llm import LLMParam, LLM
from common.mcp_tool_call_conn import MCPToolBinding, MCPToolCallSession, mcp_tool_metadata_to_openai_tool
from rag.prompts.generator import citation_plus, citation_prompt, full_question, kb_prompt, message_fit_in, structured_output_prompt
class AgentParam(LLMParam, ToolParamBase):
@@ -41,35 +42,25 @@ class AgentParam(LLMParam, ToolParamBase):
"""
def __init__(self):
self.meta:ToolMeta = {
"name": "agent",
"description": "This is an agent for a specific task.",
"parameters": {
"user_prompt": {
"type": "string",
"description": "This is the order you need to send to the agent.",
"default": "",
"required": True
},
"reasoning": {
"type": "string",
"description": (
"Supervisor's reasoning for choosing the this agent. "
"Explain why this agent is being invoked and what is expected of it."
),
"required": True
},
"context": {
"type": "string",
"description": (
"All relevant background information, prior facts, decisions, "
"and state needed by the agent to solve the current query. "
"Should be as detailed and self-contained as possible."
),
"required": True
},
}
}
self.meta: ToolMeta = {
"name": "agent",
"description": "This is an agent for a specific task.",
"parameters": {
"user_prompt": {"type": "string", "description": "This is the order you need to send to the agent.", "default": "", "required": True},
"reasoning": {
"type": "string",
"description": ("Supervisor's reasoning for choosing the this agent. Explain why this agent is being invoked and what is expected of it."),
"required": True,
},
"context": {
"type": "string",
"description": (
"All relevant background information, prior facts, decisions, and state needed by the agent to solve the current query. Should be as detailed and self-contained as possible."
),
"required": True,
},
},
}
super().__init__()
self.function_name = "agent"
self.tools = []
@@ -79,7 +70,6 @@ class AgentParam(LLMParam, ToolParamBase):
self.custom_header = {}
class Agent(LLM, ToolBase):
component_name = "Agent"
@@ -91,13 +81,15 @@ class Agent(LLM, ToolBase):
original_name = cpn.get_meta()["function"]["name"]
indexed_name = f"{original_name}_{idx}"
self.tools[indexed_name] = cpn
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error,
max_rounds=self._param.max_rounds,
verbose_tool_use=True
)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id)
self.chat_mdl = LLMBundle(
self._canvas.get_tenant_id(),
chat_model_config,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error,
max_rounds=self._param.max_rounds,
verbose_tool_use=False,
)
self.tool_meta = []
for indexed_name, tool_obj in self.tools.items():
original_meta = tool_obj.get_meta()
@@ -105,19 +97,42 @@ class Agent(LLM, ToolBase):
indexed_meta["function"]["name"] = indexed_name
self.tool_meta.append(indexed_meta)
tool_idx = len(self.tools)
for mcp in self._param.mcp:
_, mcp_server = MCPServerService.get_by_id(mcp["mcp_id"])
custom_header = self._param.custom_header
tool_call_session = MCPToolCallSession(mcp_server, mcp_server.variables, custom_header)
for tnm, meta in mcp["tools"].items():
self.tool_meta.append(mcp_tool_metadata_to_openai_tool(meta))
self.tools[tnm] = tool_call_session
indexed_name = f"{tnm}_{tool_idx}"
tool_idx += 1
self.tool_meta.append(mcp_tool_metadata_to_openai_tool(meta, function_name=indexed_name))
self.tools[indexed_name] = MCPToolBinding(tool_call_session, tnm)
self.callback = partial(self._canvas.tool_use_callback, id)
self.toolcall_session = LLMToolPluginCallSession(self.tools, self.callback)
#self.chat_mdl.bind_tools(self.toolcall_session, self.tool_metas)
if self.tool_meta:
self.chat_mdl.bind_tools(self.toolcall_session, self.tool_meta)
def _fit_messages(self, prompt: str, msg: list[dict]) -> list[dict]:
_, fitted_messages = message_fit_in(
[{"role": "system", "content": prompt}, *msg],
int(self.chat_mdl.max_length * 0.97),
)
return fitted_messages
@staticmethod
def _append_system_prompt(msg: list[dict], extra_prompt: str) -> None:
if extra_prompt and msg and msg[0]["role"] == "system":
msg[0]["content"] += "\n" + extra_prompt
@staticmethod
def _clean_formatted_answer(ans: str) -> str:
ans = re.sub(r"^.*</think>", "", ans, flags=re.DOTALL)
ans = re.sub(r"^.*```json", "", ans, flags=re.DOTALL)
return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
def _load_tool_obj(self, cpn: dict) -> object:
from agent.component import component_class
tool_name = cpn["component_name"]
param = component_class(tool_name + "Param")()
param.update(cpn["params"])
@@ -130,19 +145,17 @@ class Agent(LLM, ToolBase):
return component_class(cpn["component_name"])(self._canvas, cpn_id, param)
def get_meta(self) -> dict[str, Any]:
self._param.function_name= self._id.split("-->")[-1]
self._param.function_name = self._id.split("-->")[-1]
m = super().get_meta()
if hasattr(self._param, "user_prompt") and self._param.user_prompt:
m["function"]["parameters"]["properties"]["user_prompt"] = self._param.user_prompt
# Keep the JSON schema valid; user_prompt is a string field, not a schema node.
m["function"]["parameters"]["properties"]["user_prompt"]["default"] = self._param.user_prompt
return m
def get_input_form(self) -> dict[str, dict]:
res = {}
for k, v in self.get_input_elements().items():
res[k] = {
"type": "line",
"name": v["name"]
}
res[k] = {"type": "line", "name": v["name"]}
for cpn in self._param.tools:
if not isinstance(cpn, LLM):
continue
@@ -175,7 +188,7 @@ class Agent(LLM, ToolBase):
def _invoke(self, **kwargs):
return asyncio.run(self._invoke_async(**kwargs))
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 20*60)))
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 20 * 60)))
async def _invoke_async(self, **kwargs):
if self.check_if_canceled("Agent processing"):
return
@@ -204,19 +217,17 @@ class Agent(LLM, ToolBase):
schema = json.dumps(output_schema, ensure_ascii=False, indent=2)
schema_prompt = structured_output_prompt(schema)
downstreams = self._canvas.get_component(self._id)["downstream"] if self._canvas.get_component(self._id) else []
component = self._canvas.get_component(self._id)
downstreams = component["downstream"] if component else []
ex = self.exception_handler()
if any([self._canvas.get_component_obj(cid).component_name.lower()=="message" for cid in downstreams]) and not (ex and ex["goto"]) and not output_schema:
has_message_downstream = any(self._canvas.get_component_obj(cid).component_name.lower() == "message" for cid in downstreams)
if has_message_downstream and not (ex and ex["goto"]) and not output_schema:
self.set_output("content", partial(self.stream_output_with_tools_async, prompt, deepcopy(msg), user_defined_prompt))
return
_, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
use_tools = []
ans = ""
async for delta_ans, _tk in self._react_with_tools_streamly_async_simple(prompt, msg, use_tools, user_defined_prompt,schema_prompt=schema_prompt):
if self.check_if_canceled("Agent processing"):
return
ans += delta_ans
msg = self._fit_messages(prompt, msg)
self._append_system_prompt(msg, schema_prompt)
ans = await self._generate_async(msg)
if ans.find("**ERROR**") >= 0:
logging.error(f"Agent._chat got error. response: {ans}")
@@ -230,14 +241,8 @@ class Agent(LLM, ToolBase):
error = ""
for _ in range(self._param.max_retries + 1):
try:
def clean_formated_answer(ans: str) -> str:
ans = re.sub(r"^.*</think>", "", ans, flags=re.DOTALL)
ans = re.sub(r"^.*```json", "", ans, flags=re.DOTALL)
return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
obj = json_repair.loads(clean_formated_answer(ans))
obj = json_repair.loads(self._clean_formatted_answer(ans))
self.set_output("structured", obj)
if use_tools:
self.set_output("use_tools", use_tools)
return obj
except Exception:
error = "The answer cannot be parsed as JSON"
@@ -248,330 +253,95 @@ class Agent(LLM, ToolBase):
self.set_output("_ERROR", error)
return
artifact_md = self._collect_tool_artifact_markdown(existing_text=ans)
if artifact_md:
ans += "\n\n" + artifact_md
self.set_output("content", ans)
if use_tools:
self.set_output("use_tools", use_tools)
return ans
async def stream_output_with_tools_async(self, prompt, msg, user_defined_prompt={}):
_, msg = message_fit_in([{"role": "system", "content": prompt}, *msg], int(self.chat_mdl.max_length * 0.97))
answer_without_toolcall = ""
use_tools = []
async for delta_ans, _ in self._react_with_tools_streamly_async_simple(prompt, msg, use_tools, user_defined_prompt):
if len(msg) > 3:
st = timer()
user_request = await full_question(messages=msg, chat_mdl=self.chat_mdl)
self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer() - st)
msg = [*msg[:-1], {"role": "user", "content": user_request}]
msg = self._fit_messages(prompt, msg)
need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
cited = False
if need2cite and len(msg) < 7:
self._append_system_prompt(msg, citation_prompt())
cited = True
answer = ""
async for delta in self._generate_streamly(msg):
if self.check_if_canceled("Agent streaming"):
return
if delta_ans.find("**ERROR**") >= 0:
if delta.find("**ERROR**") >= 0:
if self.get_exception_default_value():
self.set_output("content", self.get_exception_default_value())
yield self.get_exception_default_value()
fallback = self.get_exception_default_value()
self.set_output("content", fallback)
yield fallback
else:
self.set_output("_ERROR", delta_ans)
return
answer_without_toolcall += delta_ans
yield delta_ans
self.set_output("content", answer_without_toolcall)
if use_tools:
self.set_output("use_tools", use_tools)
async def _react_with_tools_streamly_async_simple(self, prompt, history: list[dict], use_tools, user_defined_prompt={}, schema_prompt: str = ""):
token_count = 0
tool_metas = self.tool_meta
hist = deepcopy(history)
last_calling = ""
if len(hist) > 3:
st = timer()
user_request = await full_question(messages=history, chat_mdl=self.chat_mdl)
self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer()-st)
else:
user_request = history[-1]["content"]
def build_task_desc(prompt: str, user_request: str, user_defined_prompt: dict | None = None) -> str:
"""Build a minimal task_desc by concatenating prompt, query, and tool schemas."""
user_defined_prompt = user_defined_prompt or {}
task_desc = (
"### Agent Prompt\n"
f"{prompt}\n\n"
"### User Request\n"
f"{user_request}\n\n"
)
if user_defined_prompt:
udp_json = json.dumps(user_defined_prompt, ensure_ascii=False, indent=2)
task_desc += "\n### User Defined Prompts\n" + udp_json + "\n"
return task_desc
async def use_tool_async(name, args):
nonlocal hist, use_tools, last_calling
logging.info(f"{last_calling=} == {name=}")
last_calling = name
tool_response = await self.toolcall_session.tool_call_async(name, args)
use_tools.append({
"name": name,
"arguments": args,
"results": tool_response
})
return name, tool_response
async def complete():
nonlocal hist
need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
if schema_prompt:
need2cite = False
cited = False
if hist and hist[0]["role"] == "system":
if schema_prompt:
hist[0]["content"] += "\n" + schema_prompt
if need2cite and len(hist) < 7:
hist[0]["content"] += citation_prompt()
cited = True
yield "", token_count
_hist = hist
if len(hist) > 12:
_hist = [hist[0], hist[1], *hist[-10:]]
entire_txt = ""
async for delta_ans in self._generate_streamly(_hist):
if not need2cite or cited:
yield delta_ans, 0
entire_txt += delta_ans
if not need2cite or cited:
self.set_output("_ERROR", delta)
self.set_output("content", delta)
yield delta
return
if not need2cite or cited:
yield delta
answer += delta
st = timer()
txt = ""
async for delta_ans in self._gen_citations_async(entire_txt):
if self.check_if_canceled("Agent streaming"):
return
yield delta_ans, 0
txt += delta_ans
self.callback("gen_citations", {}, txt, elapsed_time=timer()-st)
def build_observation(tool_call_res: list[tuple]) -> str:
"""
Build a Observation from tool call results.
No LLM involved.
"""
if not tool_call_res:
return ""
lines = ["Observation:"]
for name, result in tool_call_res:
lines.append(f"[{name} result]")
lines.append(str(result))
return "\n".join(lines)
def append_user_content(hist, content):
if hist[-1]["role"] == "user":
hist[-1]["content"] += content
else:
hist.append({"role": "user", "content": content})
if not need2cite or cited:
artifact_md = self._collect_tool_artifact_markdown(existing_text=answer)
if artifact_md:
yield "\n\n" + artifact_md
answer += "\n\n" + artifact_md
self.set_output("content", answer)
return
st = timer()
task_desc = build_task_desc(prompt, user_request, user_defined_prompt)
self.callback("analyze_task", {}, task_desc, elapsed_time=timer()-st)
for _ in range(self._param.max_rounds + 1):
cited_answer = ""
async for delta in self._gen_citations_async(answer):
if self.check_if_canceled("Agent streaming"):
return
response, tk = await next_step_async(self.chat_mdl, hist, tool_metas, task_desc, user_defined_prompt)
# self.callback("next_step", {}, str(response)[:256]+"...")
token_count += tk or 0
hist.append({"role": "assistant", "content": response})
try:
functions = json_repair.loads(re.sub(r"```.*", "", response))
if not isinstance(functions, list):
raise TypeError(f"List should be returned, but `{functions}`")
for f in functions:
if not isinstance(f, dict):
raise TypeError(f"An object type should be returned, but `{f}`")
tool_tasks = []
for func in functions:
name = func["name"]
args = func["arguments"]
if name == COMPLETE_TASK:
append_user_content(hist, f"Respond with a formal answer. FORGET(DO NOT mention) about `{COMPLETE_TASK}`. The language for the response MUST be as the same as the first user request.\n")
async for txt, tkcnt in complete():
yield txt, tkcnt
return
tool_tasks.append(asyncio.create_task(use_tool_async(name, args)))
results = await asyncio.gather(*tool_tasks) if tool_tasks else []
st = timer()
reflection = build_observation(results)
append_user_content(hist, reflection)
self.callback("reflection", {}, str(reflection), elapsed_time=timer()-st)
except Exception as e:
logging.exception(msg=f"Wrong JSON argument format in LLM ReAct response: {e}")
e = f"\nTool call error, please correct the input parameter of response format and call it again.\n *** Exception ***\n{e}"
append_user_content(hist, str(e))
logging.warning( f"Exceed max rounds: {self._param.max_rounds}")
final_instruction = f"""
{user_request}
IMPORTANT: You have reached the conversation limit. Based on ALL the information and research you have gathered so far, please provide a DIRECT and COMPREHENSIVE final answer to the original request.
Instructions:
1. SYNTHESIZE all information collected during this conversation
2. Provide a COMPLETE response using existing data - do not suggest additional research
3. Structure your response as a FINAL DELIVERABLE, not a plan
4. If information is incomplete, state what you found and provide the best analysis possible with available data
5. DO NOT mention conversation limits or suggest further steps
6. Focus on delivering VALUE with the information already gathered
Respond immediately with your final comprehensive answer.
"""
if self.check_if_canceled("Agent final instruction"):
return
append_user_content(hist, final_instruction)
async for txt, tkcnt in complete():
yield txt, tkcnt
# async def _react_with_tools_streamly_async(self, prompt, history: list[dict], use_tools, user_defined_prompt={}, schema_prompt: str = ""):
# token_count = 0
# tool_metas = self.tool_meta
# hist = deepcopy(history)
# last_calling = ""
# if len(hist) > 3:
# st = timer()
# user_request = await full_question(messages=history, chat_mdl=self.chat_mdl)
# self.callback("Multi-turn conversation optimization", {}, user_request, elapsed_time=timer()-st)
# else:
# user_request = history[-1]["content"]
# async def use_tool_async(name, args):
# nonlocal hist, use_tools, last_calling
# logging.info(f"{last_calling=} == {name=}")
# last_calling = name
# tool_response = await self.toolcall_session.tool_call_async(name, args)
# use_tools.append({
# "name": name,
# "arguments": args,
# "results": tool_response
# })
# # self.callback("add_memory", {}, "...")
# #self.add_memory(hist[-2]["content"], hist[-1]["content"], name, args, str(tool_response), user_defined_prompt)
# return name, tool_response
# async def complete():
# nonlocal hist
# need2cite = self._param.cite and self._canvas.get_reference()["chunks"] and self._id.find("-->") < 0
# if schema_prompt:
# need2cite = False
# cited = False
# if hist and hist[0]["role"] == "system":
# if schema_prompt:
# hist[0]["content"] += "\n" + schema_prompt
# if need2cite and len(hist) < 7:
# hist[0]["content"] += citation_prompt()
# cited = True
# yield "", token_count
# _hist = hist
# if len(hist) > 12:
# _hist = [hist[0], hist[1], *hist[-10:]]
# entire_txt = ""
# async for delta_ans in self._generate_streamly(_hist):
# if not need2cite or cited:
# yield delta_ans, 0
# entire_txt += delta_ans
# if not need2cite or cited:
# return
# st = timer()
# txt = ""
# async for delta_ans in self._gen_citations_async(entire_txt):
# if self.check_if_canceled("Agent streaming"):
# return
# yield delta_ans, 0
# txt += delta_ans
# self.callback("gen_citations", {}, txt, elapsed_time=timer()-st)
# def append_user_content(hist, content):
# if hist[-1]["role"] == "user":
# hist[-1]["content"] += content
# else:
# hist.append({"role": "user", "content": content})
# st = timer()
# task_desc = await analyze_task_async(self.chat_mdl, prompt, user_request, tool_metas, user_defined_prompt)
# self.callback("analyze_task", {}, task_desc, elapsed_time=timer()-st)
# for _ in range(self._param.max_rounds + 1):
# if self.check_if_canceled("Agent streaming"):
# return
# response, tk = await next_step_async(self.chat_mdl, hist, tool_metas, task_desc, user_defined_prompt)
# # self.callback("next_step", {}, str(response)[:256]+"...")
# token_count += tk or 0
# hist.append({"role": "assistant", "content": response})
# try:
# functions = json_repair.loads(re.sub(r"```.*", "", response))
# if not isinstance(functions, list):
# raise TypeError(f"List should be returned, but `{functions}`")
# for f in functions:
# if not isinstance(f, dict):
# raise TypeError(f"An object type should be returned, but `{f}`")
# tool_tasks = []
# for func in functions:
# name = func["name"]
# args = func["arguments"]
# if name == COMPLETE_TASK:
# append_user_content(hist, f"Respond with a formal answer. FORGET(DO NOT mention) about `{COMPLETE_TASK}`. The language for the response MUST be as the same as the first user request.\n")
# async for txt, tkcnt in complete():
# yield txt, tkcnt
# return
# tool_tasks.append(asyncio.create_task(use_tool_async(name, args)))
# results = await asyncio.gather(*tool_tasks) if tool_tasks else []
# st = timer()
# reflection = await reflect_async(self.chat_mdl, hist, results, user_defined_prompt)
# append_user_content(hist, reflection)
# self.callback("reflection", {}, str(reflection), elapsed_time=timer()-st)
# except Exception as e:
# logging.exception(msg=f"Wrong JSON argument format in LLM ReAct response: {e}")
# e = f"\nTool call error, please correct the input parameter of response format and call it again.\n *** Exception ***\n{e}"
# append_user_content(hist, str(e))
# logging.warning( f"Exceed max rounds: {self._param.max_rounds}")
# final_instruction = f"""
# {user_request}
# IMPORTANT: You have reached the conversation limit. Based on ALL the information and research you have gathered so far, please provide a DIRECT and COMPREHENSIVE final answer to the original request.
# Instructions:
# 1. SYNTHESIZE all information collected during this conversation
# 2. Provide a COMPLETE response using existing data - do not suggest additional research
# 3. Structure your response as a FINAL DELIVERABLE, not a plan
# 4. If information is incomplete, state what you found and provide the best analysis possible with available data
# 5. DO NOT mention conversation limits or suggest further steps
# 6. Focus on delivering VALUE with the information already gathered
# Respond immediately with your final comprehensive answer.
# """
# if self.check_if_canceled("Agent final instruction"):
# return
# append_user_content(hist, final_instruction)
# async for txt, tkcnt in complete():
# yield txt, tkcnt
yield delta
cited_answer += delta
artifact_md = self._collect_tool_artifact_markdown(existing_text=cited_answer)
if artifact_md:
yield "\n\n" + artifact_md
cited_answer += "\n\n" + artifact_md
self.callback("gen_citations", {}, cited_answer, elapsed_time=timer() - st)
self.set_output("content", cited_answer)
async def _gen_citations_async(self, text):
retrievals = self._canvas.get_reference()
retrievals = {"chunks": list(retrievals["chunks"].values()), "doc_aggs": list(retrievals["doc_aggs"].values())}
formated_refer = kb_prompt(retrievals, self.chat_mdl.max_length, True)
async for delta_ans in self._generate_streamly([{"role": "system", "content": citation_plus("\n\n".join(formated_refer))},
{"role": "user", "content": text}
]):
async for delta_ans in self._generate_streamly([{"role": "system", "content": citation_plus("\n\n".join(formated_refer))}, {"role": "user", "content": text}]):
yield delta_ans
def _collect_tool_artifact_markdown(self, existing_text: str = "") -> str:
md_parts = []
for tool_obj in self.tools.values():
if not hasattr(tool_obj, "_param") or not hasattr(tool_obj._param, "outputs"):
continue
artifacts_meta = tool_obj._param.outputs.get("_ARTIFACTS", {})
artifacts = artifacts_meta.get("value") if isinstance(artifacts_meta, dict) else None
if not artifacts:
continue
for art in artifacts:
if not isinstance(art, dict):
continue
url = art.get("url", "")
if url and (f"![]({url})" in existing_text or f"![{art.get('name', '')}]({url})" in existing_text):
continue
if art.get("mime_type", "").startswith("image/"):
md_parts.append(f"![{art['name']}]({url})")
else:
md_parts.append(f"[Download {art['name']}]({url})")
return "\n\n".join(md_parts)
def reset(self, only_output=False):
"""
Reset all tools if they have a reset method. This avoids errors for tools like MCPToolCallSession.

View File

@@ -366,6 +366,7 @@ class ComponentBase(ABC):
component_name: str
thread_limiter = asyncio.Semaphore(int(os.environ.get("MAX_CONCURRENT_CHATS", 10)))
variable_ref_patt = r"\{* *\{([a-zA-Z:0-9]+@[A-Za-z0-9_.-]+|sys\.[A-Za-z0-9_.]+|env\.[A-Za-z0-9_.]+)\} *\}*"
iteration_alias_patt = r"\{* *\{(item|index|result)\} *\}*"
def __str__(self):
"""
@@ -486,6 +487,10 @@ class ComponentBase(ABC):
continue
if isinstance(v, str) and self._canvas.is_reff(v):
self.set_input_value(var, self._canvas.get_variable_value(v))
elif isinstance(v, str) and re.search(self.variable_ref_patt, v):
elements = self.get_input_elements_from_text(v)
kv = {k: e.get('value', '') for k, e in elements.items()}
self.set_input_value(var, self.string_format(v, kv))
else:
self.set_input_value(var, v)
res[var] = self.get_input_value(var)
@@ -497,6 +502,23 @@ class ComponentBase(ABC):
return {var: self.get_input_value(var) for var, o in self.get_input_elements().items()}
def _resolve_iteration_alias_ref(self, exp: str) -> str | None:
if exp not in {"item", "index", "result"}:
return None
parent = self.get_parent()
if not parent or parent.component_name.lower() != "iteration":
return None
for cid, cpn in self._canvas.components.items():
if cpn.get("parent_id") != parent._id:
continue
if cpn["obj"].component_name.lower() != "iterationitem":
continue
return f"{cid}@{exp}"
return None
def get_input_elements_from_text(self, txt: str) -> dict[str, dict[str, str]]:
res = {}
for r in re.finditer(self.variable_ref_patt, txt, flags=re.IGNORECASE | re.DOTALL):
@@ -508,6 +530,20 @@ class ComponentBase(ABC):
"_retrieval": self._canvas.get_variable_value(f"{cpn_id}@_references") if cpn_id else None,
"_cpn_id": cpn_id
}
for r in re.finditer(self.iteration_alias_patt, txt, flags=re.IGNORECASE | re.DOTALL):
exp = r.group(1)
if exp in res:
continue
ref = self._resolve_iteration_alias_ref(exp)
if not ref:
continue
cpn_id, var_nm = ref.split("@", 1)
res[exp] = {
"name": (self._canvas.get_component_name(cpn_id) + f"@{var_nm}"),
"value": self._canvas.get_variable_value(ref),
"_retrieval": self._canvas.get_variable_value(f"{cpn_id}@_references"),
"_cpn_id": cpn_id
}
return res
def get_input_elements(self) -> dict[str, Any]:

View File

@@ -41,6 +41,7 @@ class Begin(UserFillUp):
if self.check_if_canceled("Begin processing"):
return
layout_recognize = self._param.layout_recognize or None
for k, v in kwargs.get("inputs", {}).items():
if self.check_if_canceled("Begin processing"):
return
@@ -52,7 +53,7 @@ class Begin(UserFillUp):
file_value = v["value"]
# Support both single file (backward compatibility) and multiple files
files = file_value if isinstance(file_value, list) else [file_value]
v = FileService.get_files(files)
v = FileService.get_files(files, layout_recognize=layout_recognize)
else:
v = v.get("value")
self.set_output(k, v)

716
agent/component/browser.py Normal file
View File

@@ -0,0 +1,716 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import asyncio
import hashlib
import inspect
import json
import logging
import os
import re
import shutil
import tempfile
from abc import ABC
from pathlib import Path
from typing import Any
from urllib.error import HTTPError, URLError
from urllib.parse import unquote, urlparse
from urllib.request import Request, urlopen
from agent.component.base import ComponentBase
from agent.component.llm import LLMParam
from api.db import FileType
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from api.db.services import duplicate_name
from api.db.services.file_service import FileService
from api.db.services.tenant_llm_service import TenantLLMService
from api.utils.file_utils import filename_type
from common import settings
from common.connection_utils import timeout
from common.misc_utils import get_uuid
from rag.llm import FACTORY_DEFAULT_BASE_URL
class BrowserParam(LLMParam):
"""
Parameters for Browser node.
"""
def __init__(self):
super().__init__()
self.prompts = "{sys.query}"
self.max_steps = 30
self.headless = True
self.enable_default_extensions = False
self.chromium_sandbox = False
# Reuse browser profile across runs of the same agent node by default.
self.persist_session = True
self.upload_sources = []
self.outputs = {
"content": {"type": "string", "value": ""},
"downloaded_files": {"type": "Array<Object>", "value": []},
}
def check(self):
self.check_empty(self.llm_id, "[Browser] LLM")
self.check_positive_integer(self.max_steps, "[Browser] Max steps")
self.check_boolean(self.headless, "[Browser] Headless")
self.check_boolean(self.enable_default_extensions, "[Browser] Enable default extensions")
self.check_boolean(self.chromium_sandbox, "[Browser] Chromium sandbox")
self.check_boolean(self.persist_session, "[Browser] Persist session")
self.check_empty(self.prompts, "[Browser] Prompts")
return True
def get_input_form(self) -> dict[str, dict]:
return {
"prompts": {"type": "text", "name": "Prompts"},
"upload_sources": {"type": "line", "name": "Upload sources"},
}
class Browser(ComponentBase, ABC):
component_name = "Browser"
def get_input_elements(self) -> dict[str, dict]:
text_parts = [
str(self._param.prompts or ""),
json.dumps(self._param.upload_sources, ensure_ascii=False),
]
return self.get_input_elements_from_text("\n".join(text_parts))
def _resolve_param_value(self, value: Any) -> Any:
if isinstance(value, str):
direct_ref = value.strip()
if direct_ref.startswith("{") and direct_ref.endswith("}") and self._canvas.is_reff(direct_ref):
return self._canvas.get_variable_value(direct_ref)
return value
return value
def _extract_ids(self, value: Any) -> list[str]:
ids: list[str] = []
value = self._resolve_param_value(value)
def collect(item: Any):
if item is None:
return
if isinstance(item, str):
token = item.strip()
if not token:
return
if token.startswith("{") and token.endswith("}") and self._canvas.is_reff(token):
collect(self._canvas.get_variable_value(token))
return
if token.startswith("[") and token.endswith("]"):
try:
parsed = json.loads(token)
collect(parsed)
return
except Exception:
pass
if self._is_http_url(token):
ids.append(token)
return
if "," in token:
for part in token.split(","):
collect(part)
return
ids.append(token)
return
if isinstance(item, dict):
for k in ("file_id", "id", "url", "value"):
if k in item:
collect(item[k])
return
for v in item.values():
collect(v)
return
if isinstance(item, (list, tuple, set)):
for v in item:
collect(v)
return
token = str(item).strip()
if token:
ids.append(token)
collect(value)
deduped: list[str] = []
visited = set()
for item in ids:
if item in visited:
continue
visited.add(item)
deduped.append(item)
return deduped
@staticmethod
def _is_http_url(value: str) -> bool:
token = str(value or "").strip()
if not token:
return False
parsed = urlparse(token)
return parsed.scheme in {"http", "https"} and bool(parsed.netloc)
@staticmethod
def _extract_url_filename(url: str, headers: Any) -> str:
content_disposition = str(getattr(headers, "get", lambda *_args, **_kwargs: "")("Content-Disposition", "") or "")
if content_disposition:
# Prefer RFC 5987 encoded filename*=UTF-8''... when present.
m = re.search(r"filename\*\s*=\s*(?:UTF-8''|utf-8'')?([^;]+)", content_disposition)
if m:
name = unquote(m.group(1).strip().strip('"'))
if name:
return os.path.basename(name)
m = re.search(r'filename\s*=\s*"([^"]+)"', content_disposition)
if m:
name = m.group(1).strip()
if name:
return os.path.basename(name)
m = re.search(r"filename\s*=\s*([^;]+)", content_disposition)
if m:
name = m.group(1).strip().strip('"')
if name:
return os.path.basename(name)
parsed = urlparse(url)
raw_name = os.path.basename(parsed.path or "")
name = unquote(raw_name).strip()
if name:
return name
return f"url_file_{get_uuid()[:8]}.bin"
@staticmethod
def _resolve_upload_url_max_bytes() -> int:
raw = str(os.getenv("RAGFLOW_BROWSER_UPLOAD_URL_MAX_BYTES", "") or "").strip()
default_max_bytes = 100 * 1024 * 1024
if not raw:
return default_max_bytes
try:
parsed = int(raw)
return parsed if parsed > 0 else default_max_bytes
except (TypeError, ValueError):
return default_max_bytes
@staticmethod
def _restore_env_var(key: str, value: str | None):
if value is None:
os.environ.pop(key, None)
return
os.environ[key] = value
def _prepare_upload_url_file(self, url: str, upload_dir: str) -> dict[str, Any] | None:
max_bytes = self._resolve_upload_url_max_bytes()
local_path = ""
local_name = ""
total_size = 0
try:
req = Request(url, headers={"User-Agent": "RAGFlow-Browser-Node/1.0"})
with urlopen(req, timeout=30) as response:
local_name = self._extract_url_filename(url, response.headers)
local_path = os.path.join(upload_dir, local_name)
index = 1
while os.path.exists(local_path):
stem, ext = os.path.splitext(local_name)
local_path = os.path.join(upload_dir, f"{stem}_{index}{ext}")
index += 1
with open(local_path, "wb") as f:
while True:
chunk = response.read(1024 * 1024)
if not chunk:
break
total_size += len(chunk)
if total_size > max_bytes:
raise ValueError(f"upload url file exceeds max size limit: {max_bytes}")
f.write(chunk)
except (HTTPError, URLError, OSError, TimeoutError, ValueError) as e:
if local_path and os.path.exists(local_path):
try:
os.remove(local_path)
except OSError:
pass
logging.warning("Browser failed to fetch upload url. url=%s, error=%s", url, e)
return None
if total_size <= 0:
if local_path and os.path.exists(local_path):
try:
os.remove(local_path)
except OSError:
pass
logging.warning("Browser upload url returned empty content: %s", url)
return None
return {
"file_id": "",
"name": local_name,
"size": total_size,
"local_path": local_path,
"source_url": url,
}
def _resolve_text(self, raw_text: Any) -> str:
text = str(self._resolve_param_value(raw_text) or "")
vars_map = self.get_input_elements_from_text(text)
kv = {}
for key, meta in vars_map.items():
val = meta.get("value", "")
if isinstance(val, str):
kv[key] = val
else:
kv[key] = json.dumps(val, ensure_ascii=False)
return self.string_format(text, kv)
@staticmethod
def _as_model_config_dict(cfg_obj: Any) -> dict[str, Any]:
if cfg_obj is None:
return {}
if isinstance(cfg_obj, dict):
return cfg_obj
if hasattr(cfg_obj, "to_dict") and callable(cfg_obj.to_dict):
try:
result = cfg_obj.to_dict()
return result if isinstance(result, dict) else {}
except (AttributeError, TypeError, ValueError):
return {}
result = {}
for key in ("model", "model_name", "llm_name", "llm_factory", "api_key", "base_url", "api_base", "temperature"):
val = getattr(cfg_obj, key, None)
if val not in (None, ""):
result[key] = val
return result
@staticmethod
def _error_chain(exc: Exception) -> str:
parts = []
cur = exc
depth = 0
while cur is not None and depth < 6:
parts.append(f"{type(cur).__name__}: {cur}")
cur = cur.__cause__ or cur.__context__
depth += 1
return " <- ".join(parts)
@staticmethod
def _resolve_browser_executable() -> str:
explicit_candidates = [
os.getenv("BROWSER_USE_EXECUTABLE_PATH", "").strip(),
os.getenv("BROWSER_USE_BROWSER_BINARY_PATH", "").strip(),
os.getenv("BROWSER_USE_CHROME_BINARY_PATH", "").strip(),
]
for explicit in explicit_candidates:
if explicit and os.path.isfile(explicit) and os.access(explicit, os.X_OK):
return explicit
candidates = [
"/opt/chrome/chrome",
"/usr/local/bin/chrome",
"/usr/local/bin/google-chrome",
"/usr/bin/google-chrome",
"/usr/bin/google-chrome-stable",
"/usr/bin/chromium",
"/usr/bin/chromium-browser",
]
for path in candidates:
if os.path.isfile(path) and os.access(path, os.X_OK):
return path
for cmd in ("chrome", "google-chrome", "google-chrome-stable", "chromium", "chromium-browser"):
path = shutil.which(cmd)
if path and os.path.isfile(path) and os.access(path, os.X_OK):
return path
return ""
@staticmethod
def _normalize_model_name(model: Any) -> str:
name = str(model or "").strip()
if not name:
return ""
if name.startswith("bu-") or name.startswith("browser-use/"):
return name
if "@" in name:
# RAGFlow model aliases may include provider suffix, e.g. qwen3.5-flash@Tongyi-Qianwen.
# browser-use OpenAI-compatible adapters need the pure model name.
name = name.split("@", 1)[0].strip()
return name
@staticmethod
def _safe_path_segment(value: Any) -> str:
token = str(value or "").strip()
if not token:
return "unknown"
token = re.sub(r"[^A-Za-z0-9._-]+", "_", token)
return token.strip("._-") or "unknown"
def _resolve_persistent_profile_dir(self) -> str:
root = os.path.join(tempfile.gettempdir(), "ragflow_browser_use_profiles")
tenant = self._safe_path_segment(self._canvas.get_tenant_id())
raw_canvas_id = getattr(self._canvas, "_id", "")
if not raw_canvas_id:
graph_text = json.dumps(
self._canvas.dsl.get("graph", {}),
sort_keys=True,
ensure_ascii=False,
)
raw_canvas_id = (
f"dsl_{hashlib.sha1(graph_text.encode('utf-8')).hexdigest()[:12]}"
)
canvas_id = self._safe_path_segment(raw_canvas_id)
node_id = self._safe_path_segment(self._id)
return os.path.join(root, tenant, canvas_id, node_id)
def _should_persist_session(self) -> bool:
return bool(self._param.persist_session)
def _infer_provider_name(self, cfg: dict[str, Any]) -> str:
provider = str(cfg.get("llm_factory") or "").strip()
if provider:
return provider
llm_id = str(self._param.llm_id or "")
if "@" in llm_id:
return llm_id.split("@", 1)[1].strip()
return ""
def _resolve_openai_compatible_base_url(self, cfg: dict[str, Any]) -> str:
explicit = str(cfg.get("base_url") or cfg.get("api_base") or "").strip()
if explicit:
return explicit
provider = self._infer_provider_name(cfg)
fallback = str(FACTORY_DEFAULT_BASE_URL.get(provider, "")).strip()
return fallback if fallback else ""
def _build_browser_llm(self):
from browser_use.llm import ChatBrowserUse, ChatOpenAI
chat_model_config = get_model_config_by_type_and_name(
self._canvas.get_tenant_id(),
TenantLLMService.llm_id2llm_type(self._param.llm_id),
self._param.llm_id,
)
cfg = self._as_model_config_dict(chat_model_config)
model_name = self._normalize_model_name(cfg.get("model_name") or cfg.get("model") or self._param.llm_id)
if not model_name:
raise ValueError(f"Invalid model config for Browser llm_id={self._param.llm_id}")
base_url = self._resolve_openai_compatible_base_url(cfg)
# ChatBrowserUse only supports bu-* models. For tenant models, use OpenAI-compatible adapter.
if model_name.startswith("bu-") or model_name.startswith("browser-use/"):
llm_kwargs = {
"model": model_name,
"api_key": cfg.get("api_key"),
"base_url": base_url,
"temperature": self._param.temperature,
"max_retries": self._param.max_retries,
}
llm_kwargs = {k: v for k, v in llm_kwargs.items() if v not in (None, "")}
return ChatBrowserUse(**llm_kwargs)
llm_kwargs = {
"model": model_name,
"api_key": cfg.get("api_key"),
"base_url": base_url,
"temperature": self._param.temperature,
"max_retries": self._param.max_retries,
}
llm_kwargs = {k: v for k, v in llm_kwargs.items() if v not in (None, "")}
return ChatOpenAI(**llm_kwargs)
async def _run_browser_use_async(
self,
task_text: str,
download_dir: str,
available_file_paths: list[str] | None = None,
profile_dir: str | None = None,
):
from browser_use import Agent as BrowserUseAgent, Browser as BrowserUseBrowser
llm = self._build_browser_llm()
# NOTE:
# _invoke() uses asyncio.run(), which creates a fresh event loop per task run.
# Reusing a Browser object created by a previous loop can deadlock/timestamp out
# in browser-use watchdog handlers on subsequent runs.
# We keep persistent user_data_dir for session continuity, but we do not keep
# browser instances alive across runs.
available_file_paths = available_file_paths or []
agent_kwargs: dict[str, Any] = {
"task": task_text,
"llm": llm,
"available_file_paths": available_file_paths,
}
browser_obj = None
previous_disable_extensions = os.environ.get("BROWSER_USE_DISABLE_EXTENSIONS")
previous_browser_binary_path = os.environ.get("BROWSER_USE_BROWSER_BINARY_PATH")
try:
enable_default_extensions = bool(self._param.enable_default_extensions)
if not enable_default_extensions:
os.environ["BROWSER_USE_DISABLE_EXTENSIONS"] = "1"
else:
os.environ.pop("BROWSER_USE_DISABLE_EXTENSIONS", None)
executable_path = self._resolve_browser_executable()
browser_kwargs = {
"headless": self._param.headless,
"downloads_path": download_dir,
# Docker often runs as root without user namespaces; disable sandbox by default.
"chromium_sandbox": bool(self._param.chromium_sandbox),
# Disable runtime extension download by default for intranet/offline environments.
# Enable only when explicitly required and extensions are pre-cached.
"enable_default_extensions": enable_default_extensions,
}
if executable_path:
browser_kwargs["executable_path"] = executable_path
# Keep browser-use watchdog fallback in sync with our resolved path.
os.environ["BROWSER_USE_BROWSER_BINARY_PATH"] = executable_path
else:
logging.warning(
"Browser no local browser executable found. "
"Set BROWSER_USE_EXECUTABLE_PATH or preinstall chromium in image to avoid runtime playwright install."
)
if profile_dir:
browser_kwargs["user_data_dir"] = profile_dir
# browser-use expects profile_directory to be a profile name
# such as "Default" / "Profile 1", not an absolute path.
browser_kwargs["profile_directory"] = "Default"
browser_obj = BrowserUseBrowser(**browser_kwargs)
agent_kwargs["browser"] = browser_obj
except (OSError, RuntimeError, TypeError, ValueError) as e:
logging.warning("Browser browser context customization skipped: %s", e)
agent = BrowserUseAgent(**agent_kwargs)
history = None
run_fn = getattr(agent, "run", None)
if run_fn is None:
raise RuntimeError("browser-use Agent does not provide run().")
run_kwargs = {"max_steps": self._param.max_steps}
try:
if inspect.iscoroutinefunction(run_fn):
history = await run_fn(**run_kwargs)
else:
history = await asyncio.to_thread(run_fn, **run_kwargs)
except Exception as e:
logging.error("Browser agent.run failed. error_chain=%s", self._error_chain(e))
logging.exception("Browser agent.run traceback")
raise
finally:
if browser_obj:
close_fn = getattr(browser_obj, "close", None)
if close_fn:
try:
if inspect.iscoroutinefunction(close_fn):
await close_fn()
else:
await asyncio.to_thread(close_fn)
except Exception as close_err:
logging.warning("Browser failed to close browser object cleanly: %s", close_err)
self._restore_env_var("BROWSER_USE_DISABLE_EXTENSIONS", previous_disable_extensions)
self._restore_env_var("BROWSER_USE_BROWSER_BINARY_PATH", previous_browser_binary_path)
return history
def _prepare_upload_files(self, upload_dir: str) -> list[dict[str, Any]]:
upload_refs = self._extract_ids(self._param.upload_sources)
prepared = []
for file_ref in upload_refs:
if self._is_http_url(file_ref):
prepared_url_file = self._prepare_upload_url_file(file_ref, upload_dir)
if prepared_url_file:
prepared.append(prepared_url_file)
continue
file_id = file_ref
exists, file = FileService.get_by_id(file_id)
if not exists:
logging.warning("Browser upload file_id not found: %s", file_id)
continue
try:
blob = settings.STORAGE_IMPL.get(file.parent_id, file.location)
if not blob:
logging.warning("Browser upload blob not found: %s", file_id)
continue
local_name = os.path.basename(file.location) if file.location else (file.name or f"{file_id}.bin")
local_path = os.path.join(upload_dir, local_name)
index = 1
while os.path.exists(local_path):
stem, ext = os.path.splitext(local_name)
local_path = os.path.join(upload_dir, f"{stem}_{index}{ext}")
index += 1
with open(local_path, "wb") as f:
f.write(blob)
except OSError as e:
logging.warning("Browser failed to prepare upload file. file_id=%s, error=%s", file_id, e)
continue
except Exception as e:
logging.warning("Browser failed to fetch upload blob. file_id=%s, error=%s", file_id, e)
continue
prepared.append(
{
"file_id": file.id,
"name": file.name,
"size": file.size,
"local_path": local_path,
}
)
return prepared
def _save_downloads(self, download_dir: str, parent_id: str) -> list[dict[str, Any]]:
downloaded_files: list[dict[str, Any]] = []
exists, folder = FileService.get_by_id(parent_id)
if not exists or folder.type != FileType.FOLDER.value:
raise ValueError(f"RAGFlow target folder does not exist or is not a folder: {parent_id}")
tenant_id = self._canvas.get_tenant_id()
storage_put = settings.STORAGE_IMPL.put
storage_rm = getattr(settings.STORAGE_IMPL, "rm", None)
insert_file = FileService.insert
for path in Path(download_dir).rglob("*"):
if not path.is_file():
continue
try:
if path.stat().st_size <= 0:
continue
blob = path.read_bytes()
except OSError as e:
logging.warning("Browser failed to read downloaded file. path=%s, error=%s", path, e)
continue
if not blob:
continue
display_name = ""
blob_stored = False
try:
display_name = duplicate_name(FileService.query, name=path.name, parent_id=parent_id)
storage_put(parent_id, display_name, blob)
blob_stored = True
file_data = {
"id": get_uuid(),
"parent_id": parent_id,
"tenant_id": tenant_id,
"created_by": tenant_id,
"type": filename_type(display_name),
"name": display_name,
"location": display_name,
"size": len(blob),
}
inserted = insert_file(file_data)
downloaded_files.append(
{
"file_id": inserted.id,
"name": inserted.name,
"size": inserted.size,
"parent_id": inserted.parent_id,
}
)
except Exception as e:
if blob_stored and callable(storage_rm):
try:
storage_rm(parent_id, display_name)
except Exception as rollback_err:
logging.warning(
"Browser rollback stored download failed. path=%s, parent_id=%s, display_name=%s, error=%s",
path,
parent_id,
display_name,
rollback_err,
)
logging.error(
"Browser failed to save download. path=%s, tenant_id=%s, parent_id=%s, display_name=%s, error=%s",
path,
tenant_id,
parent_id,
display_name,
e,
)
continue
return downloaded_files
@staticmethod
def _extract_history_text(history: Any) -> str:
if history is None:
return ""
def pick_final_result(value: Any) -> str:
if value is None:
return ""
if isinstance(value, str):
return value.strip()
if isinstance(value, (int, float, bool)):
return str(value)
return ""
# Only trust browser-use's explicit final_result API/property.
final_result_fn = getattr(history, "final_result", None)
if callable(final_result_fn):
try:
final_result_value = final_result_fn()
return pick_final_result(final_result_value)
except Exception:
return ""
return pick_final_result(final_result_fn)
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 20 * 60)))
def _invoke(self, **kwargs):
profile_dir = None
persist_session = self._should_persist_session()
try:
user_prompt = self._resolve_text(kwargs.get("prompts", self._param.prompts))
with tempfile.TemporaryDirectory(prefix="browser_use_upload_") as upload_dir, tempfile.TemporaryDirectory(
prefix="browser_use_download_"
) as download_dir:
uploaded_files = self._prepare_upload_files(upload_dir)
upload_lines = [
f"- file_id={item['file_id']}, name={item['name']}, local_path={item['local_path']}"
for item in uploaded_files
]
task_text = user_prompt
if upload_lines:
task_text += (
"\n\nYou can upload files from these local paths when operating web pages:\n"
+ "\n".join(upload_lines)
)
upload_local_paths = [item.get("local_path", "") for item in uploaded_files if item.get("local_path")]
if persist_session:
profile_dir = self._resolve_persistent_profile_dir()
os.makedirs(profile_dir, exist_ok=True)
else:
try:
profile_dir = tempfile.mkdtemp(prefix="browser_use_profile_")
except OSError:
profile_dir = None
history = asyncio.run(
self._run_browser_use_async(
task_text, download_dir, upload_local_paths, profile_dir
)
)
target_dir_id = FileService.get_root_folder(self._canvas.get_tenant_id())["id"]
downloaded_files = self._save_downloads(download_dir, target_dir_id)
self.set_output("content", self._extract_history_text(history))
self.set_output("downloaded_files", downloaded_files)
return self.output()
except Exception as e:
logging.exception("Browser invoke failed")
self.set_output("_ERROR", str(e))
return self.output()
finally:
if profile_dir and not persist_session:
shutil.rmtree(profile_dir, ignore_errors=True)
def thoughts(self) -> str:
return "Planning and executing browser actions..."

View File

@@ -21,6 +21,7 @@ from abc import ABC
from common.constants import LLMType
from api.db.services.llm_service import LLMBundle
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from agent.component.llm import LLMParam, LLM
from common.connection_utils import timeout
from rag.llm.chat_model import ERROR_PREFIX
@@ -122,7 +123,8 @@ class Categorize(LLM, ABC):
msg[-1]["content"] = query_value
self.set_input_value(query_key, msg[-1]["content"])
self._param.update_prompt()
chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.CHAT, self._param.llm_id)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), LLMType.CHAT, self._param.llm_id)
chat_mdl = LLMBundle(self._canvas.get_tenant_id(), chat_model_config)
user_prompt = """
---- Real Data ----

View File

@@ -73,7 +73,7 @@ class DataOperations(ComponentBase,ABC):
continue
if self._param.operations == "select_keys":
self._select_keys()
elif self._param.operations == "recursive_eval":
elif self._param.operations == "literal_eval":
self._literal_eval()
elif self._param.operations == "combine":
self._combine()
@@ -94,9 +94,9 @@ class DataOperations(ComponentBase,ABC):
def _recursive_eval(self, data):
if isinstance(data, dict):
return {k: self.recursive_eval(v) for k, v in data.items()}
return {k: self._recursive_eval(v) for k, v in data.items()}
if isinstance(data, list):
return [self.recursive_eval(item) for item in data]
return [self._recursive_eval(item) for item in data]
if isinstance(data, str):
try:
if (

File diff suppressed because it is too large Load Diff

View File

@@ -27,6 +27,7 @@ class UserFillUpParam(ComponentParamBase):
super().__init__()
self.enable_tips = True
self.tips = "Please fill up the form"
self.layout_recognize = ""
def check(self) -> bool:
return True
@@ -61,6 +62,7 @@ class UserFillUp(ComponentBase):
content = re.sub(r"\{%s\}"%k, ans, content)
self.set_output("tips", content)
layout_recognize = self._param.layout_recognize or None
for k, v in kwargs.get("inputs", {}).items():
if self.check_if_canceled("UserFillUp processing"):
return
@@ -71,7 +73,7 @@ class UserFillUp(ComponentBase):
file_value = v["value"]
# Support both single file (backward compatibility) and multiple files
files = file_value if isinstance(file_value, list) else [file_value]
v = FileService.get_files(files)
v = FileService.get_files(files, layout_recognize=layout_recognize)
else:
v = v.get("value")
self.set_output(k, v)

View File

@@ -19,6 +19,7 @@ import os
import re
import time
from abc import ABC
from functools import partial
import requests
@@ -29,7 +30,7 @@ from deepdoc.parser import HtmlParser
class InvokeParam(ComponentParamBase):
"""
Define the Crawler component parameters.
Define the Invoke component parameters.
"""
def __init__(self):
@@ -41,7 +42,7 @@ class InvokeParam(ComponentParamBase):
self.url = ""
self.timeout = 60
self.clean_html = False
self.datatype = "json" # New parameter to determine data posting type
self.datatype = "json"
def check(self):
self.check_valid_value(self.method.lower(), "Type of content from the crawler", ["get", "post", "put"])
@@ -53,92 +54,196 @@ class InvokeParam(ComponentParamBase):
class Invoke(ComponentBase, ABC):
component_name = "Invoke"
header_variable_ref_patt = r"\{([a-zA-Z_][a-zA-Z0-9_.@-]*)\}"
@staticmethod
def _coerce_json_arg_if_possible(key, value):
raw_value = value
if isinstance(value, str):
try:
value = json.loads(value)
logging.debug(
"Invoke JSON arg coercion succeeded. key=%s parsed_type=%s",
key,
type(value).__name__,
)
except json.JSONDecodeError as exc:
logging.info(
"Invoke JSON arg coercion skipped; value is not valid JSON. key=%s raw=%r error=%s",
key,
raw_value,
exc,
)
return raw_value
try:
json.dumps(value, allow_nan=False)
except (TypeError, ValueError) as exc:
logging.warning(
"Invoke JSON arg is not JSON-serializable. key=%s value_type=%s value=%r error=%s",
key,
type(value).__name__,
value,
exc,
)
raise ValueError(f"Invoke JSON argument '{key}' is not JSON-serializable.") from exc
return value
def get_input_form(self) -> dict[str, dict]:
res = {}
for item in self._param.variables or []:
if not isinstance(item, dict):
continue
ref = (item.get("ref") or "").strip()
if not ref or ref in res:
continue
elements = self.get_input_elements_from_text("{" + ref + "}")
element = elements.get(ref, {})
res[ref] = {
"type": "line",
"name": element.get("name") or item.get("key") or ref,
}
return res
def _resolve_variable_value(self, variable_name: str, kwargs: dict | None = None):
kwargs = kwargs or {}
value = kwargs.get(variable_name, self._canvas.get_variable_value(variable_name))
if isinstance(value, partial):
value = "".join(value())
self.set_input_value(variable_name, value)
return "" if value is None else value
def _render_template(self, content: str, pattern: str, kwargs: dict | None = None, *, flags: int = 0) -> str:
content = content or ""
if not content:
return content
def replace_variable(match_obj):
return str(self._resolve_variable_value(match_obj.group(1), kwargs))
return re.sub(pattern, replace_variable, content, flags=flags)
def _resolve_template_text(self, content: str, kwargs: dict | None = None) -> str:
return self._render_template(content, self.variable_ref_patt, kwargs, flags=re.DOTALL)
def _resolve_header_text(self, content: str, kwargs: dict | None = None) -> str:
# Headers support plain {token} placeholders, so they cannot reuse the canvas variable regex.
return self._render_template(content, self.header_variable_ref_patt, kwargs)
def _resolve_arg_value(self, para: dict, kwargs: dict) -> object:
ref = (para.get("ref") or "").strip()
if ref and (ref in kwargs or self._canvas.get_variable_value(ref) is not None):
return self._resolve_variable_value(ref, kwargs)
if para.get("value") is not None:
value = para["value"]
if isinstance(value, str):
return self._resolve_template_text(value, kwargs)
return value
if ref:
return self._resolve_variable_value(ref, kwargs)
return ""
def _is_json_mode(self) -> bool:
return self._param.datatype.lower() == "json"
def _build_request_args(self, kwargs: dict) -> dict:
args = {}
for para in self._param.variables:
key = para["key"]
value = self._resolve_arg_value(para, kwargs)
if self._is_json_mode():
# JSON mode accepts stringified JSON so complex payloads can be passed through variables.
value = self._coerce_json_arg_if_possible(key, value)
args[key] = value
if para.get("ref"):
self.set_input_value(para["ref"], value)
return args
def _build_url(self, kwargs: dict) -> str:
url = self._resolve_template_text(self._param.url.strip(), kwargs)
if not url.startswith(("http://", "https://")):
url = "http://" + url
return url
def _build_headers(self, kwargs: dict) -> dict:
if not self._param.headers:
return {}
headers = json.loads(self._param.headers)
if not isinstance(headers, dict):
raise ValueError("Invoke headers must be a JSON object.")
return {key: self._resolve_header_text(value, kwargs) if isinstance(value, str) else value for key, value in headers.items()}
def _build_proxies(self) -> dict | None:
if not re.sub(r"https?:?/?/?", "", self._param.proxy):
return None
return {"http": self._param.proxy, "https": self._param.proxy}
def _send_request(self, url: str, args: dict, headers: dict, proxies: dict | None):
method = self._param.method.lower()
request = getattr(requests, method)
request_kwargs = {
"url": url,
"headers": headers,
"proxies": proxies,
"timeout": self._param.timeout,
}
# GET sends query params; POST/PUT send either JSON or form data based on datatype.
if method == "get":
request_kwargs["params"] = args
return request(**request_kwargs)
body_key = "json" if self._is_json_mode() else "data"
request_kwargs[body_key] = args
return request(**request_kwargs)
def _format_response(self, response) -> str:
if not self._param.clean_html:
return response.text
# HtmlParser keeps the Invoke output text-focused when the endpoint returns HTML.
sections = HtmlParser()(None, response.content)
return "\n".join(sections)
@timeout(int(os.environ.get("COMPONENT_EXEC_TIMEOUT", 3)))
def _invoke(self, **kwargs):
if self.check_if_canceled("Invoke processing"):
return
args = {}
for para in self._param.variables:
if para.get("value"):
args[para["key"]] = para["value"]
else:
args[para["key"]] = self._canvas.get_variable_value(para["ref"])
args = self._build_request_args(kwargs)
url = self._build_url(kwargs)
headers = self._build_headers(kwargs)
proxies = self._build_proxies()
url = self._param.url.strip()
def replace_variable(match):
var_name = match.group(1)
try:
value = self._canvas.get_variable_value(var_name)
return str(value or "")
except Exception:
return ""
# {base_url} or {component_id@variable_name}
url = re.sub(r"\{([a-zA-Z_][a-zA-Z0-9_.@-]*)\}", replace_variable, url)
if url.find("http") != 0:
url = "http://" + url
method = self._param.method.lower()
headers = {}
if self._param.headers:
headers = json.loads(self._param.headers)
proxies = None
if re.sub(r"https?:?/?/?", "", self._param.proxy):
proxies = {"http": self._param.proxy, "https": self._param.proxy}
last_e = ""
last_error = None
for _ in range(self._param.max_retries + 1):
if self.check_if_canceled("Invoke processing"):
return
try:
if method == "get":
response = requests.get(url=url, params=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
sections = HtmlParser()(None, response.content)
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
if method == "put":
if self._param.datatype.lower() == "json":
response = requests.put(url=url, json=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
else:
response = requests.put(url=url, data=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
sections = HtmlParser()(None, response.content)
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
if method == "post":
if self._param.datatype.lower() == "json":
response = requests.post(url=url, json=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
else:
response = requests.post(url=url, data=args, headers=headers, proxies=proxies, timeout=self._param.timeout)
if self._param.clean_html:
self.set_output("result", "\n".join(sections))
else:
self.set_output("result", response.text)
return self.output("result")
response = self._send_request(url, args, headers, proxies)
result = self._format_response(response)
self.set_output("result", result)
return result
except Exception as e:
if self.check_if_canceled("Invoke processing"):
return
last_e = e
last_error = e
logging.exception(f"Http request error: {e}")
time.sleep(self._param.delay_after_error)
if last_e:
self.set_output("_ERROR", str(last_e))
return f"Http request error: {last_e}"
assert False, self.output()
if last_error:
self.set_output("_ERROR", str(last_error))
return f"Http request error: {last_error}"
def thoughts(self) -> str:
return "Waiting for the server respond..."

View File

@@ -54,7 +54,11 @@ class IterationItem(ComponentBase, ABC):
if self.check_if_canceled("IterationItem processing"):
return
self.set_output("item", arr[self._idx])
current_item = arr[self._idx]
self.set_output("item", current_item)
# Keep `result` as a compatibility alias because existing DSL examples
# and downstream references may still consume IterationItem via `@result`.
self.set_output("result", current_item)
self.set_output("index", self._idx)
self._idx += 1
@@ -69,7 +73,7 @@ class IterationItem(ComponentBase, ABC):
if p._id != pid:
continue
if p.component_name.lower() in ["categorize", "message", "switch", "userfillup", "interationitem"]:
if p.component_name.lower() in ["categorize", "message", "switch", "userfillup", "iterationitem"]:
continue
for k, o in p._param.outputs.items():

View File

@@ -10,8 +10,9 @@ class ListOperationsParam(ComponentParamBase):
def __init__(self):
super().__init__()
self.query = ""
self.operations = "topN"
self.n=0
self.operations = "nth"
self.n = 0
self.strict = False
self.sort_method = "asc"
self.filter = {
"operator": "=",
@@ -34,7 +35,11 @@ class ListOperationsParam(ComponentParamBase):
def check(self):
self.check_empty(self.query, "query")
self.check_valid_value(self.operations, "Support operations", ["topN","head","tail","filter","sort","drop_duplicates"])
self.check_valid_value(
self.operations,
"Support operations",
["nth", "head", "tail", "filter", "sort", "drop_duplicates"],
)
def get_input_form(self) -> dict[str, dict]:
return {}
@@ -51,8 +56,8 @@ class ListOperations(ComponentBase,ABC):
if not isinstance(self.inputs, list):
raise TypeError("The input of List Operations should be an array.")
self.set_input_value(inputs, self.inputs)
if self._param.operations == "topN":
self._topN()
if self._param.operations == "nth":
self._nth()
elif self._param.operations == "head":
self._head()
elif self._param.operations == "tail":
@@ -70,35 +75,74 @@ class ListOperations(ComponentBase,ABC):
return int(getattr(self._param, "n", 0))
except Exception:
return 0
def _is_strict(self):
strict = getattr(self._param, "strict", False)
if isinstance(strict, str):
return strict.strip().lower() in {"1", "true", "yes", "on"}
return bool(strict)
def _set_outputs(self, outputs):
self._param.outputs["result"]["value"] = outputs
self._param.outputs["first"]["value"] = outputs[0] if outputs else None
self._param.outputs["last"]["value"] = outputs[-1] if outputs else None
def _topN(self):
def _raise_strict_range_error(self, operation, n):
raise ValueError(
f"{operation} requires n to be within the valid range in strict mode, got {n}."
)
def _nth(self):
n = self._coerce_n()
if n < 1:
strict = self._is_strict()
if n == 0:
if strict:
self._raise_strict_range_error("nth", n)
outputs = []
elif n > 0:
if n <= len(self.inputs):
outputs = [self.inputs[n - 1]]
elif strict:
self._raise_strict_range_error("nth", n)
else:
outputs = []
else:
n = min(n, len(self.inputs))
outputs = self.inputs[:n]
if abs(n) <= len(self.inputs):
outputs = [self.inputs[n]]
elif strict:
self._raise_strict_range_error("nth", n)
else:
outputs = []
self._set_outputs(outputs)
def _head(self):
n = self._coerce_n()
if 1 <= n <= len(self.inputs):
outputs = [self.inputs[n - 1]]
strict = self._is_strict()
if strict:
if 1 <= n <= len(self.inputs):
outputs = self.inputs[:n]
else:
self._raise_strict_range_error("head", n)
else:
outputs = []
if n < 1:
outputs = []
else:
outputs = self.inputs[:n]
self._set_outputs(outputs)
def _tail(self):
n = self._coerce_n()
if 1 <= n <= len(self.inputs):
outputs = [self.inputs[-n]]
strict = self._is_strict()
if strict:
if 1 <= n <= len(self.inputs):
outputs = self.inputs[-n:]
else:
self._raise_strict_range_error("tail", n)
else:
outputs = []
if n < 1:
outputs = []
else:
outputs = self.inputs[-n:]
self._set_outputs(outputs)
def _filter(self):
@@ -107,7 +151,7 @@ class ListOperations(ComponentBase,ABC):
def _norm(self,v):
s = "" if v is None else str(v)
return s
def _eval(self, v, operator, value):
if operator == "=":
return v == value
@@ -163,6 +207,6 @@ class ListOperations(ComponentBase,ABC):
if isinstance(x, set):
return tuple(sorted(self._hashable(v) for v in x))
return x
def thoughts(self) -> str:
return "ListOperation in progress"

View File

@@ -25,6 +25,7 @@ from functools import partial
from common.constants import LLMType
from api.db.services.llm_service import LLMBundle
from api.db.services.tenant_llm_service import TenantLLMService
from api.db.joint_services.tenant_model_service import get_model_config_by_type_and_name
from agent.component.base import ComponentBase, ComponentParamBase
from common.connection_utils import timeout
from rag.prompts.generator import tool_call_summary, message_fit_in, citation_prompt, structured_output_prompt
@@ -84,10 +85,10 @@ class LLM(ComponentBase):
def __init__(self, canvas, component_id, param: ComponentParamBase):
super().__init__(canvas, component_id, param)
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id),
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
chat_model_config = get_model_config_by_type_and_name(self._canvas.get_tenant_id(), TenantLLMService.llm_id2llm_type(self._param.llm_id), self._param.llm_id)
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), chat_model_config,
max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error)
self.imgs = []
def get_input_form(self) -> dict[str, dict]:
@@ -125,23 +126,119 @@ class LLM(ComponentBase):
msg.append(p)
return msg, self.string_format(self._param.sys_prompt, args)
def _prepare_prompt_variables(self):
if self._param.visual_files_var:
self.imgs = self._canvas.get_variable_value(self._param.visual_files_var)
if not self.imgs:
self.imgs = []
self.imgs = [img for img in self.imgs if img[:len("data:image/")] == "data:image/"]
if self.imgs and TenantLLMService.llm_id2llm_type(self._param.llm_id) == LLMType.CHAT.value:
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.IMAGE2TEXT.value,
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
@staticmethod
def _extract_data_images(value) -> list[str]:
imgs = []
def walk(v):
if v is None:
return
if isinstance(v, str):
v = v.strip()
if v.startswith("data:image/"):
imgs.append(v)
return
if isinstance(v, (list, tuple, set)):
for item in v:
walk(item)
return
if isinstance(v, dict):
if "content" in v:
walk(v.get("content"))
else:
for item in v.values():
walk(item)
walk(value)
return imgs
@staticmethod
def _uniq_images(images: list[str]) -> list[str]:
seen = set()
uniq = []
for img in images:
if not isinstance(img, str):
continue
if not img.startswith("data:image/"):
continue
if img in seen:
continue
seen.add(img)
uniq.append(img)
return uniq
@classmethod
def _remove_data_images(cls, value):
if value is None:
return None
if isinstance(value, str):
return None if value.strip().startswith("data:image/") else value
if isinstance(value, list):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return cleaned
if isinstance(value, tuple):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return tuple(cleaned)
if isinstance(value, set):
cleaned = []
for item in value:
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned.append(v)
return cleaned
if isinstance(value, dict):
if value.get("type") in {"image_url", "input_image", "image"} and cls._extract_data_images(value):
return None
cleaned = {}
for k, item in value.items():
v = cls._remove_data_images(item)
if v is None:
continue
if isinstance(v, (list, tuple, set, dict)) and not v:
continue
cleaned[k] = v
return cleaned
return value
def _prepare_prompt_variables(self):
self.imgs = []
if self._param.visual_files_var:
visual_val = self._canvas.get_variable_value(self._param.visual_files_var)
self.imgs.extend(self._extract_data_images(visual_val))
args = {}
vars = self.get_input_elements() if not self._param.debug_inputs else self._param.debug_inputs
extracted_imgs = []
for k, o in vars.items():
args[k] = o["value"]
raw_value = o["value"]
extracted_imgs.extend(self._extract_data_images(raw_value))
args[k] = self._remove_data_images(raw_value)
if args[k] is None:
args[k] = ""
if not isinstance(args[k], str):
try:
args[k] = json.dumps(args[k], ensure_ascii=False)
@@ -149,6 +246,13 @@ class LLM(ComponentBase):
args[k] = str(args[k])
self.set_input_value(k, args[k])
self.imgs = self._uniq_images(self.imgs + extracted_imgs)
if self.imgs and TenantLLMService.llm_id2llm_type(self._param.llm_id) == LLMType.CHAT.value:
self.chat_mdl = LLMBundle(self._canvas.get_tenant_id(), LLMType.IMAGE2TEXT.value,
self._param.llm_id, max_retries=self._param.max_retries,
retry_interval=self._param.delay_after_error
)
msg, sys_prompt = self._sys_prompt_and_msg(self._canvas.get_history(self._param.message_history_window_size)[:-1], args)
user_defined_prompt, sys_prompt = self._extract_prompts(sys_prompt)
if self._param.cite and self._canvas.get_reference()["chunks"]:
@@ -241,6 +345,8 @@ class LLM(ComponentBase):
return re.sub(r"(<think>|</think>)", "", delta_ans)
stream_kwargs = {"images": self.imgs} if self.imgs else {}
extra_chat_kwargs = self._get_chat_template_kwargs()
stream_kwargs.update(extra_chat_kwargs)
async for ans in self.chat_mdl.async_chat_streamly(msg[0]["content"], msg[1:], self._param.gen_conf(), **stream_kwargs):
if self.check_if_canceled("LLM streaming"):
return
@@ -271,6 +377,7 @@ class LLM(ComponentBase):
return re.sub(r"```\n*$", "", ans, flags=re.DOTALL)
prompt, msg, _ = self._prepare_prompt_variables()
extra_chat_kwargs = self._get_chat_template_kwargs()
error: str = ""
output_structure = None
try:
@@ -289,7 +396,7 @@ class LLM(ComponentBase):
int(self.chat_mdl.max_length * 0.97),
)
error = ""
ans = await self._generate_async(msg_fit)
ans = await self._generate_async(msg_fit, **extra_chat_kwargs)
msg_fit.pop(0)
if ans.find("**ERROR**") >= 0:
logging.error(f"LLM response error: {ans}")
@@ -322,7 +429,7 @@ class LLM(ComponentBase):
[{"role": "system", "content": prompt}, *deepcopy(msg)], int(self.chat_mdl.max_length * 0.97)
)
error = ""
ans = await self._generate_async(msg_fit)
ans = await self._generate_async(msg_fit, **extra_chat_kwargs)
msg_fit.pop(0)
if ans.find("**ERROR**") >= 0:
logging.error(f"LLM response error: {ans}")
@@ -341,6 +448,24 @@ class LLM(ComponentBase):
def _invoke(self, **kwargs):
return asyncio.run(self._invoke_async(**kwargs))
def _get_chat_template_kwargs(self) -> dict[str, Any]:
chat_template_kwargs = self._canvas.globals.get("sys.chat_template_kwargs")
if chat_template_kwargs is None:
return {}
# The API should pass this as a JSON object, but accept a JSON string for compatibility.
if isinstance(chat_template_kwargs, str):
try:
chat_template_kwargs = json_repair.loads(chat_template_kwargs)
except Exception:
logging.warning("Ignore invalid sys.chat_template_kwargs: expected JSON object or JSON string object.")
return {}
if not isinstance(chat_template_kwargs, dict):
logging.warning("Ignore invalid sys.chat_template_kwargs type: %s", type(chat_template_kwargs).__name__)
return {}
return {"chat_template_kwargs": chat_template_kwargs}
async def add_memory(self, user:str, assist:str, func_name: str, params: dict, results: str, user_defined_prompt:dict={}):
summ = await tool_call_summary(self.chat_mdl, func_name, params, results, user_defined_prompt)
logging.info(f"[MEMORY]: {summ}")

View File

@@ -56,7 +56,7 @@ class Loop(ComponentBase, ABC):
for item in self._param.loop_variables:
if any([not item.get("variable"), not item.get("input_mode"), not item.get("value"),not item.get("type")]):
assert "Loop Variable is not complete."
raise ValueError("Loop Variable is not complete.")
if item["input_mode"]=="variable":
self.set_output(item["variable"],self._canvas.get_variable_value(item["value"]))
elif item["input_mode"]=="constant":

View File

@@ -64,6 +64,16 @@ class LoopItem(ComponentBase, ABC):
elif operator == "not empty":
return var != ""
elif isinstance(var, bool):
if operator == "is":
return var is value
elif operator == "is not":
return var is not value
elif operator == "empty":
return var is None
elif operator == "not empty":
return var is not None
elif isinstance(var, (int, float)):
if operator == "=":
return var == value
@@ -82,16 +92,6 @@ class LoopItem(ComponentBase, ABC):
elif operator == "not empty":
return var is not None
elif isinstance(var, bool):
if operator == "is":
return var is value
elif operator == "is not":
return var is not value
elif operator == "empty":
return var is None
elif operator == "not empty":
return var is not None
elif isinstance(var, dict):
if operator == "empty":
return len(var) == 0

View File

@@ -14,8 +14,11 @@
# limitations under the License.
#
import asyncio
import nest_asyncio
nest_asyncio.apply()
try:
import nest_asyncio
nest_asyncio.apply()
except Exception:
pass
import inspect
import json
import os
@@ -27,7 +30,9 @@ from functools import partial
from typing import Any
from agent.component.base import ComponentBase, ComponentParamBase
from jinja2 import Template as Jinja2Template
from jinja2.sandbox import SandboxedEnvironment
_jinja2_sandbox = SandboxedEnvironment()
from common.connection_utils import timeout
from common.misc_utils import get_uuid
@@ -49,6 +54,9 @@ class MessageParam(ComponentParamBase):
self.outputs = {
"content": {
"type": "str"
},
"downloads": {
"type": "list"
}
}
@@ -61,15 +69,99 @@ class MessageParam(ComponentParamBase):
class Message(ComponentBase):
component_name = "Message"
@staticmethod
def _is_download_info(value: Any) -> bool:
return isinstance(value, dict) and all(
key in value for key in ("doc_id", "filename", "mime_type")
)
@staticmethod
def _download_info_includes_content(value: Any) -> bool:
return isinstance(value, dict) and bool(value.get("include_download_info_in_content"))
@staticmethod
def _normalize_download_info(value: Any) -> Any:
if isinstance(value, list):
return [Message._normalize_download_info(item) for item in value]
if not isinstance(value, dict):
return value
normalized = value.copy()
normalized.pop("include_download_info_in_content", None)
return normalized
def _extract_downloads(self, value: Any) -> list[dict[str, Any]]:
if isinstance(value, str):
try:
value = json.loads(value)
except Exception:
return []
if self._is_download_info(value):
return [value]
if isinstance(value, list) and all(self._is_download_info(item) for item in value):
return value
return []
def _stringify_message_value(
self,
value: Any,
delimiter: str = None,
downloads: list[dict[str, Any]] | None = None,
fallback_to_str: bool = False,
) -> str:
extracted_downloads = self._extract_downloads(value)
if extracted_downloads:
if downloads is not None:
downloads.extend(self._normalize_download_info(item) for item in extracted_downloads)
if any(self._download_info_includes_content(item) for item in extracted_downloads):
if isinstance(value, str):
try:
value = json.loads(value)
except Exception:
return value
try:
return json.dumps(self._normalize_download_info(value), ensure_ascii=False)
except Exception:
if fallback_to_str:
return str(value)
return ""
return ""
if value is None:
return ""
if isinstance(value, list) and delimiter:
return delimiter.join([str(vv) for vv in value])
if isinstance(value, str):
return value
try:
return json.dumps(value, ensure_ascii=False)
except Exception:
if fallback_to_str:
return str(value)
return ""
def get_input_elements(self) -> dict[str, Any]:
return self.get_input_elements_from_text("".join(self._param.content))
def get_kwargs(self, script:str, kwargs:dict = {}, delimiter:str=None) -> tuple[str, dict[str, str | list | Any]]:
def get_kwargs(
self,
script: str,
kwargs: dict = {},
delimiter: str = None,
downloads: list[dict[str, Any]] | None = None,
) -> tuple[str, dict[str, str | list | Any]]:
for k,v in self.get_input_elements_from_text(script).items():
if k in kwargs:
continue
v = v["value"]
if not v:
if v is None:
v = ""
ans = ""
if isinstance(v, partial):
@@ -79,15 +171,8 @@ class Message(ComponentBase):
else:
for t in iter_obj:
ans += t
elif isinstance(v, list) and delimiter:
ans = delimiter.join([str(vv) for vv in v])
elif not isinstance(v, str):
try:
ans = json.dumps(v, ensure_ascii=False)
except Exception:
pass
else:
ans = v
ans = self._stringify_message_value(v, delimiter, downloads)
if not ans:
ans = ""
kwargs[k] = ans
@@ -110,6 +195,7 @@ class Message(ComponentBase):
s = 0
all_content = ""
cache = {}
downloads = []
for r in re.finditer(self.variable_ref_patt, rand_cnt, flags=re.DOTALL):
if self.check_if_canceled("Message streaming"):
return
@@ -149,11 +235,9 @@ class Message(ComponentBase):
continue
elif inspect.isawaitable(v):
v = await v
elif not isinstance(v, str):
try:
v = json.dumps(v, ensure_ascii=False)
except Exception:
v = str(v)
v = self._stringify_message_value(
v, downloads=downloads, fallback_to_str=True
)
yield v
self.set_input_value(exp, v)
all_content += v
@@ -166,6 +250,7 @@ class Message(ComponentBase):
all_content += rand_cnt[s: ]
yield rand_cnt[s: ]
self.set_output("downloads", downloads)
self.set_output("content", all_content)
self._convert_content(all_content)
await self._save_to_memory(all_content)
@@ -186,12 +271,14 @@ class Message(ComponentBase):
self.set_output("content", partial(self._stream, rand_cnt))
return
rand_cnt, kwargs = self.get_kwargs(rand_cnt, kwargs)
template = Jinja2Template(rand_cnt)
downloads = []
rand_cnt, kwargs = self.get_kwargs(rand_cnt, kwargs, downloads=downloads)
template = _jinja2_sandbox.from_string(rand_cnt)
try:
content = template.render(kwargs)
except Exception:
pass
except Exception as e:
logging.warning(f"Jinja2 template rendering failed: {e}")
content = rand_cnt # fallback to unrendered content
if self.check_if_canceled("Message processing"):
return
@@ -199,6 +286,7 @@ class Message(ComponentBase):
for n, v in kwargs.items():
content = re.sub(n, v, content)
self.set_output("downloads", downloads)
self.set_output("content", content)
self._convert_content(content)
self._save_to_memory(content)
@@ -224,6 +312,38 @@ class Message(ComponentBase):
rows = []
headers = None
def _coerce_excel_cell_type(cell: str):
# Convert markdown cell text to native numeric types when safe,so Excel writes numeric cells instead of text.
if not isinstance(cell, str):
return cell
value = cell.strip()
if value == "":
return ""
# Keep values like "00123" as text to avoid losing leading zeros.
if re.match(r"^[+-]?0\d+$", value):
return cell
# Support thousand separators like 1,234 or 1,234.56
numeric_candidate = value
if re.match(r"^[+-]?\d{1,3}(,\d{3})+(\.\d+)?$", value):
numeric_candidate = value.replace(",", "")
if re.match(r"^[+-]?\d+$", numeric_candidate):
try:
return int(numeric_candidate)
except ValueError:
return cell
if re.match(r"^[+-]?(\d+\.\d+|\d+\.|\.\d+)([eE][+-]?\d+)?$", numeric_candidate) or re.match(r"^[+-]?\d+[eE][+-]?\d+$", numeric_candidate):
try:
return float(numeric_candidate)
except ValueError:
return cell
return cell
for line in table_lines:
# Split by | and clean up
@@ -234,6 +354,7 @@ class Message(ComponentBase):
if headers is None:
headers = cells
else:
cells = [_coerce_excel_cell_type(c) for c in cells]
rows.append(cells)
if headers and rows:
@@ -430,8 +551,15 @@ class Message(ComponentBase):
if not hasattr(self._param, "memory_ids") or not self._param.memory_ids:
return True, "No memory selected."
user_id = self._param.user_id if hasattr(self._param, "user_id") else ""
if user_id:
import re
# is variable
if re.match(r"^{.*}$", user_id):
user_id = self._canvas.get_variable_value(user_id)
message_dict = {
"user_id": self._canvas._tenant_id,
"user_id": user_id,
"agent_id": self._canvas._id,
"session_id": self._canvas.task_id,
"user_input": self._canvas.get_sys_query(),

View File

@@ -18,7 +18,9 @@ import re
from abc import ABC
from typing import Any
from jinja2 import Template as Jinja2Template
from jinja2.sandbox import SandboxedEnvironment
_jinja2_sandbox = SandboxedEnvironment()
from agent.component.base import ComponentParamBase
from common.connection_utils import timeout
from .message import Message
@@ -96,14 +98,14 @@ class StringTransform(Message, ABC):
script, kwargs = self.get_kwargs(script, kwargs, self._param.delimiters[0])
if self._is_jinjia2(script):
template = Jinja2Template(script)
template = _jinja2_sandbox.from_string(script)
try:
script = template.render(kwargs)
except Exception:
pass
for k,v in kwargs.items():
if not v:
if v is None:
v = ""
script = re.sub(k, lambda match: v, script)

View File

@@ -134,7 +134,7 @@ class Switch(ComponentBase, ABC):
except Exception:
return True if input <= value else False
raise ValueError('Not supported operator' + operator)
raise ValueError(f'Not supported operator: {operator}')
def thoughts(self) -> str:
return "Im weighing a few options and will pick the next step shortly."

View File

@@ -48,7 +48,7 @@ class VariableAssigner(ComponentBase,ABC):
else:
for item in self._param.variables:
if any([not item.get("variable"), not item.get("operator"), not item.get("parameter")]):
assert "Variable is not complete."
raise ValueError("Variable is not complete.")
variable=item["variable"]
operator=item["operator"]
parameter=item["parameter"]
@@ -92,12 +92,12 @@ class VariableAssigner(ComponentBase,ABC):
return ""
elif isinstance(variable,dict):
return {}
elif isinstance(variable,bool):
return False
elif isinstance(variable,int):
return 0
elif isinstance(variable,float):
return 0.0
elif isinstance(variable,bool):
return False
else:
return None
@@ -141,20 +141,18 @@ class VariableAssigner(ComponentBase,ABC):
return variable + parameter
def _remove_first(self,variable):
if len(variable)==0:
return variable
if not isinstance(variable,list):
return "ERROR:VARIABLE_NOT_LIST"
else:
return variable[1:]
if len(variable)==0:
return variable
return variable[1:]
def _remove_last(self,variable):
if len(variable)==0:
return variable
if not isinstance(variable,list):
return "ERROR:VARIABLE_NOT_LIST"
else:
return variable[:-1]
if len(variable)==0:
return variable
return variable[:-1]
def is_number(self, value):
if isinstance(value, bool):

178
agent/dsl_migration.py Normal file
View File

@@ -0,0 +1,178 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import copy
import re
# Keep all legacy chunker renames in one place so the migration rule stays readable.
COMPONENT_RENAMES = {
"Splitter": "TokenChunker",
"HierarchicalMerger": "TitleChunker",
"PDFGenerator": "DocGenerator",
}
NODE_TYPE_RENAMES = {
"splitterNode": "chunkerNode",
}
VARIABLE_REF_PATTERN = re.compile(r"(\{+\s*)([A-Za-z0-9:_-]+)(@[A-Za-z0-9_.-]+)(\s*\}+)")
def normalize_chunker_dsl(dsl: dict) -> dict:
"""
Rewrite legacy chunker component names and ids into the current DSL schema.
This is intentionally a pure migration step:
- it does not change business params
- it only rewrites structural identifiers used by the canvas/runtime
- custom human-authored names are preserved unless they are still the exact
built-in legacy operator name
"""
if not isinstance(dsl, dict):
return dsl
normalized = copy.deepcopy(dsl)
components = normalized.get("components")
if not isinstance(components, dict):
return normalized
component_id_map: dict[str, str] = {}
for component_id in components.keys():
new_component_id = component_id
for old_name, new_name in COMPONENT_RENAMES.items():
prefix = f"{old_name}:"
if component_id.startswith(prefix):
new_component_id = f"{new_name}:{component_id[len(prefix):]}"
break
component_id_map[component_id] = new_component_id
def rewrite_variable_refs(text: str) -> str:
if text in component_id_map:
return component_id_map[text]
def repl(match: re.Match[str]) -> str:
component_id = match.group(2)
return (
match.group(1)
+ component_id_map.get(component_id, component_id)
+ match.group(3)
+ match.group(4)
)
return VARIABLE_REF_PATTERN.sub(repl, text)
def rewrite_value(value):
if isinstance(value, str):
return rewrite_variable_refs(value)
if isinstance(value, list):
return [rewrite_value(item) for item in value]
if isinstance(value, dict):
return {key: rewrite_value(item) for key, item in value.items()}
return value
rewritten_components = {}
for old_component_id, component in components.items():
new_component_id = component_id_map[old_component_id]
new_component = rewrite_value(component)
if isinstance(new_component, dict):
obj = new_component.get("obj")
if isinstance(obj, dict):
component_name = obj.get("component_name")
obj["component_name"] = COMPONENT_RENAMES.get(component_name, component_name)
if isinstance(new_component.get("downstream"), list):
new_component["downstream"] = [
component_id_map.get(component_id, component_id)
for component_id in new_component["downstream"]
]
if isinstance(new_component.get("upstream"), list):
new_component["upstream"] = [
component_id_map.get(component_id, component_id)
for component_id in new_component["upstream"]
]
parent_id = new_component.get("parent_id")
if isinstance(parent_id, str):
new_component["parent_id"] = component_id_map.get(parent_id, parent_id)
rewritten_components[new_component_id] = new_component
normalized["components"] = rewritten_components
if isinstance(normalized.get("path"), list):
normalized["path"] = [
component_id_map.get(component_id, component_id)
for component_id in normalized["path"]
]
graph = normalized.get("graph")
if isinstance(graph, dict):
nodes = graph.get("nodes")
if isinstance(nodes, list):
for node in nodes:
if not isinstance(node, dict):
continue
node_id = node.get("id")
if isinstance(node_id, str):
node["id"] = component_id_map.get(node_id, node_id)
parent_id = node.get("parentId")
if isinstance(parent_id, str):
node["parentId"] = component_id_map.get(parent_id, parent_id)
node_type = node.get("type")
if isinstance(node_type, str):
node["type"] = NODE_TYPE_RENAMES.get(node_type, node_type)
data = node.get("data")
if not isinstance(data, dict):
continue
label = data.get("label")
if isinstance(label, str):
data["label"] = COMPONENT_RENAMES.get(label, label)
name = data.get("name")
if isinstance(name, str) and name in COMPONENT_RENAMES:
data["name"] = COMPONENT_RENAMES[name]
if "form" in data:
data["form"] = rewrite_value(data["form"])
edges = graph.get("edges")
if isinstance(edges, list):
replacements = sorted(component_id_map.items(), key=lambda item: len(item[0]), reverse=True)
for edge in edges:
if not isinstance(edge, dict):
continue
for key in ("source", "target"):
value = edge.get(key)
if isinstance(value, str):
edge[key] = component_id_map.get(value, value)
edge_id = edge.get("id")
if isinstance(edge_id, str):
for old_component_id, new_component_id in replacements:
edge_id = edge_id.replace(old_component_id, new_component_id)
edge["id"] = edge_id
for key in ("history", "messages", "reference"):
if key in normalized:
normalized[key] = rewrite_value(normalized[key])
return normalized

99
agent/plugin/README_tr.md Normal file
View File

@@ -0,0 +1,99 @@
[English](./README.md) | [简体中文](./README_zh.md) | Türkçe
# Eklentiler
Bu klasör, RAGFlow'un eklenti mekanizmasını içerir.
RAGFlow, `embedded_plugins` alt klasöründen eklentileri özyinelemeli olarak yükleyecektir.
## Desteklenen eklenti türleri
Şu anda desteklenen tek eklenti türü `llm_tools`'dur.
- `llm_tools`: LLM'nin çağırması için bir araç.
## Eklenti nasıl eklenir
Bir LLM araç eklentisi eklemek basittir: bir eklenti dosyası oluşturun, içine `LLMToolPlugin` sınıfından türetilmiş bir sınıf koyun, ardından `get_metadata` ve `invoke` metodlarını uygulayın.
- `get_metadata` metodu: Bu metod, aracın açıklamasını içeren bir `LLMToolMetadata` nesnesi döndürür.
ıklama, LLM'ye çağrı için ve RAGFlow web ön yüzüne görüntüleme amacıyla sağlanacaktır.
- `invoke` metodu: Bu metod, LLM tarafından üretilen parametreleri kabul eder ve aracın yürütme sonucunu içeren bir `str` döndürür.
Bu aracın tüm yürütme mantığı bu metoda konulmalıdır.
RAGFlow'u başlattığınızda, günlükte eklentinizin yüklendiğini göreceksiniz:
```
2025-05-15 19:29:08,959 INFO 34670 Recursively importing plugins from path `/some-path/ragflow/agent/plugin/embedded_plugins`
2025-05-15 19:29:08,960 INFO 34670 Loaded llm_tools plugin BadCalculatorPlugin version 1.0.0
```
Veya eklentinizi düzeltmeniz gereken hatalar da içerebilir.
### Örnek
Yanlış cevaplar veren bir hesap makinesi aracı ekleyerek eklenti ekleme sürecini göstereceğiz.
Önce, `embedded_plugins/llm_tools` klasörü altında `bad_calculator.py` adında bir eklenti dosyası oluşturun.
Ardından, `LLMToolPlugin` temel sınıfından türetilmiş bir `BadCalculatorPlugin` sınıfı oluşturuyoruz:
```python
class BadCalculatorPlugin(LLMToolPlugin):
_version_ = "1.0.0"
```
`_version_` alanı zorunludur ve eklentinin sürüm numarasını belirtir.
Hesap makinemizin girdileri olarak `a` ve `b` olmak üzere iki sayısı vardır, bu yüzden `BadCalculatorPlugin` sınıfımıza aşağıdaki `invoke` metodunu ekliyoruz:
```python
def invoke(self, a: int, b: int) -> str:
return str(a + b + 100)
```
`invoke` metodu LLM tarafından çağrılacaktır. Birçok parametreye sahip olabilir, ancak dönüş tipi `str` olmalıdır.
Son olarak, LLM'ye `bad_calculator` aracımızı nasıl kullanacağını anlatmak için bir `get_metadata` metodu eklememiz gerekiyor:
```python
@classmethod
def get_metadata(cls) -> LLMToolMetadata:
return {
# Bu aracın adı, LLM'ye sağlanır
"name": "bad_calculator",
# Bu aracın görüntüleme adı, RAGFlow ön yüzüne sağlanır
"displayName": "$t:bad_calculator.name",
# Bu aracın kullanım açıklaması, LLM'ye sağlanır
"description": "A tool to calculate the sum of two numbers (will give wrong answer)",
# Bu aracın açıklaması, RAGFlow ön yüzüne sağlanır
"displayDescription": "$t:bad_calculator.description",
# Bu aracın parametreleri
"parameters": {
# Birinci parametre - a
"a": {
# Parametre tipi, seçenekler: number, string veya LLM'nin tanıyabileceği herhangi bir tip
"type": "number",
# Bu parametrenin açıklaması, LLM'ye sağlanır
"description": "The first number",
# Bu parametrenin açıklaması, RAGFlow ön yüzüne sağlanır
"displayDescription": "$t:bad_calculator.params.a",
# Bu parametrenin zorunlu olup olmadığı
"required": True
},
# İkinci parametre - b
"b": {
"type": "number",
"description": "The second number",
"displayDescription": "$t:bad_calculator.params.b",
"required": True
}
}
```
`get_metadata` metodu bir `classmethod`'dur. Bu aracın açıklamasını LLM'ye sağlayacaktır.
`display` ile başlayan alanlar özel bir gösterim kullanabilir: `$t:xxx`, bu gösterim RAGFlow ön yüzündeki uluslararasılaştırma (i18n) mekanizmasını kullanarak `llmTools` kategorisinden metin alır. Bu gösterimi kullanmazsanız, ön yüz buraya yazdığınız metni doğrudan gösterecektir.
Artık aracımız hazırdır. `Yanıt Üret` bileşeninde seçip deneyebilirsiniz.

View File

@@ -189,7 +189,19 @@ Currently, the following languages are officially supported:
### 🐍 Python
To add Python dependencies, simply edit the following file:
Pre-installed packages: `requests`, `numpy`, `pandas`, `matplotlib`.
> `matplotlib` uses the `Agg` (non-interactive) backend by default in the sandbox (`MPLBACKEND=Agg`). No display server is available, so always save figures to files (e.g. `fig.savefig("artifacts/chart.png")`) rather than calling `plt.show()`.
>
> Tip: if Chinese text renders as missing boxes/squares in `matplotlib`, install Debian package `fonts-noto-cjk` in your custom image. We do not preinstall it by default to keep the base image smaller. The sandbox base image ships a `matplotlibrc` that already lists common CJK fonts in the `font.sans-serif` fallback chain, so no code-level font configuration is needed — just install the font package and rebuild the image.
>
> Example:
>
> ```dockerfile
> RUN apt-get update && apt-get install -y --no-install-recommends fonts-noto-cjk && rm -rf /var/lib/apt/lists/*
> ```
To add more dependencies, edit:
```bash
sandbox_base_image/python/requirements.txt
@@ -199,6 +211,8 @@ Add any additional packages you need, one per line (just like a normal pip requi
### 🟨 Node.js
Pre-installed packages: `axios`.
To add Node.js dependencies:
1. Navigate to the Node.js base image directory:

View File

@@ -27,7 +27,7 @@ from typing import Dict, Any, Optional
from api.db.services.system_settings_service import SystemSettingsService
from agent.sandbox.providers import ProviderManager
from agent.sandbox.providers.base import ExecutionResult
from agent.sandbox.providers.base import ExecutionResult, SandboxProviderConfigError
logger = logging.getLogger(__name__)
@@ -48,7 +48,6 @@ def get_provider_manager() -> ProviderManager:
if _provider_manager is not None:
return _provider_manager
# Initialize provider manager with system settings
_provider_manager = ProviderManager()
_load_provider_from_settings()
@@ -59,8 +58,8 @@ def _load_provider_from_settings() -> None:
"""
Load sandbox provider from system settings and configure the provider manager.
This function reads the system settings to determine which provider is active
and initializes it with the appropriate configuration.
This function resolves the active provider type, then loads configuration
from system settings.
"""
global _provider_manager
@@ -68,38 +67,24 @@ def _load_provider_from_settings() -> None:
return
try:
# Get active provider type
provider_type_settings = SystemSettingsService.get_by_name("sandbox.provider_type")
if not provider_type_settings:
raise RuntimeError(
"Sandbox provider type not configured. Please set 'sandbox.provider_type' in system settings."
)
provider_type = provider_type_settings[0].value
# Get provider configuration
provider_config_settings = SystemSettingsService.get_by_name(f"sandbox.{provider_type}")
if not provider_config_settings:
logger.warning(f"No configuration found for provider: {provider_type}")
config = {}
else:
try:
config = json.loads(provider_config_settings[0].value)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse sandbox config for {provider_type}: {e}")
config = {}
provider_type = _resolve_provider_type()
config = _load_provider_config(provider_type)
# Import and instantiate the provider
from agent.sandbox.providers import (
SelfManagedProvider,
AliyunCodeInterpreterProvider,
E2BProvider,
LocalProvider,
SSHProvider,
)
provider_classes = {
"self_managed": SelfManagedProvider,
"aliyun_codeinterpreter": AliyunCodeInterpreterProvider,
"e2b": E2BProvider,
"local": LocalProvider,
"ssh": SSHProvider,
}
if provider_type not in provider_classes:
@@ -111,17 +96,44 @@ def _load_provider_from_settings() -> None:
# Initialize the provider
if not provider.initialize(config):
logger.error(f"Failed to initialize sandbox provider: {provider_type}. Config keys: {list(config.keys())}")
message = f"Failed to initialize sandbox provider: {provider_type}. Config keys: {list(config.keys())}"
if provider_type in {"local", "ssh"}:
raise SandboxProviderConfigError(message)
logger.error(message)
return
# Set the active provider
_provider_manager.set_provider(provider_type, provider)
logger.info(f"Sandbox provider '{provider_type}' initialized successfully")
except SandboxProviderConfigError:
raise
except Exception as e:
logger.error(f"Failed to load sandbox provider from settings: {e}")
import traceback
traceback.print_exc()
def _load_provider_config_from_settings(provider_type: str) -> Dict[str, Any]:
provider_config_settings = SystemSettingsService.get_by_name(f"sandbox.{provider_type}")
if not provider_config_settings:
logger.warning(f"No configuration found for provider: {provider_type}")
return {}
try:
return json.loads(provider_config_settings[0].value)
except json.JSONDecodeError as e:
logger.error(f"Failed to parse sandbox config for {provider_type}: {e}")
return {}
def _resolve_provider_type() -> str:
provider_type_settings = SystemSettingsService.get_by_name("sandbox.provider_type")
if not provider_type_settings:
return "self_managed"
return provider_type_settings[0].value
def _load_provider_config(provider_type: str) -> Dict[str, Any]:
return _load_provider_config_from_settings(provider_type)
def reload_provider() -> None:
@@ -166,6 +178,14 @@ def execute_code(
)
provider = provider_manager.get_provider()
provider_name = provider_manager.get_provider_name() or getattr(provider, "__class__", type(provider)).__name__
logger.info(
"CodeExec using sandbox provider '%s' (language=%s, timeout=%ss)",
provider_name,
language,
timeout,
)
# Create a sandbox instance
instance = provider.create_instance(template=language)

View File

@@ -7,7 +7,7 @@ services:
runtime: runc
privileged: true
ports:
- "${EXECUTOR_PORT:-9385}:9385"
- "${SANDBOX_EXECUTOR_MANAGER_PORT:-9385}:9385"
volumes:
- /var/run/docker.sock:/var/run/docker.sock
networks:

View File

@@ -1,6 +1,10 @@
FROM python:3.11-slim-bookworm
RUN grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.tuna.tsinghua.edu.cn|g' && \
ARG NEED_MIRROR=1
RUN if [ "$NEED_MIRROR" = 1 ]; then \
grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.tuna.tsinghua.edu.cn|g'; \
fi; \
apt-get update && \
apt-get install -y curl gcc && \
rm -rf /var/lib/apt/lists/*
@@ -27,11 +31,11 @@ RUN set -eux; \
ln -sf /usr/local/bin/docker /usr/bin/docker
COPY --from=ghcr.io/astral-sh/uv:0.7.5 /uv /uvx /bin/
ENV UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
WORKDIR /app
COPY . .
RUN uv pip install --system -r requirements.txt
RUN if [ "$NEED_MIRROR" = 1 ]; then export UV_INDEX_URL="https://pypi.tuna.tsinghua.edu.cn/simple"; else export UV_INDEX_URL="https://pypi.org/simple"; fi && \
uv pip install --system -r requirements.txt
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "9385"]

View File

@@ -19,6 +19,7 @@ from api.handlers import healthz_handler, run_code_handler
router = APIRouter()
router.get("/")(healthz_handler)
router.get("/healthz")(healthz_handler)
router.post("/run")(run_code_handler)

View File

@@ -14,13 +14,26 @@
# limitations under the License.
#
import base64
from typing import Optional
from typing import Any, Optional
from pydantic import BaseModel, Field, field_validator
from models.enums import ResourceLimitType, ResultStatus, RuntimeErrorType, SupportLanguage, UnauthorizedAccessType
class ArtifactItem(BaseModel):
name: str
mime_type: str
size: int
content_b64: str
class ExecutionStructuredResult(BaseModel):
present: bool
value: Any = None
type: str = "json"
class CodeExecutionResult(BaseModel):
status: ResultStatus
stdout: str
@@ -37,6 +50,12 @@ class CodeExecutionResult(BaseModel):
unauthorized_access_type: Optional[UnauthorizedAccessType] = None
runtime_error_type: Optional[RuntimeErrorType] = None
# File artifacts produced by code execution (images, PDFs, CSVs, etc.)
artifacts: list[ArtifactItem] = []
# Structured return value produced by main()
result: Optional[ExecutionStructuredResult] = None
class CodeExecutionRequest(BaseModel):
code_b64: str = Field(..., description="Base64 encoded code string")

View File

@@ -19,17 +19,181 @@ import json
import os
import time
import uuid
from core.config import TIMEOUT
from core.container import allocate_container_blocking, release_container
from core.logger import logger
from models.enums import ResourceLimitType, ResultStatus, RuntimeErrorType, SupportLanguage, UnauthorizedAccessType
from models.schemas import CodeExecutionRequest, CodeExecutionResult
from models.schemas import ArtifactItem, CodeExecutionRequest, CodeExecutionResult, ExecutionStructuredResult
from utils.common import async_run_command
RESULT_MARKER_PREFIX = "__RAGFLOW_RESULT__:"
def _extract_result_envelope(stdout: str) -> tuple[str, ExecutionStructuredResult | None]:
if not stdout:
return "", None
cleaned_lines: list[str] = []
envelope: ExecutionStructuredResult | None = None
for line in str(stdout).splitlines():
if line.startswith(RESULT_MARKER_PREFIX):
payload_b64 = line[len(RESULT_MARKER_PREFIX) :].strip()
if not payload_b64:
continue
try:
payload = base64.b64decode(payload_b64).decode("utf-8")
envelope = ExecutionStructuredResult.model_validate_json(payload)
except Exception as exc:
logger.warning(f"Failed to decode structured result marker: {exc}")
cleaned_lines.append(line)
continue
cleaned_lines.append(line)
cleaned_stdout = "\n".join(cleaned_lines)
if stdout.endswith("\n") and cleaned_stdout and not cleaned_stdout.endswith("\n"):
cleaned_stdout += "\n"
return cleaned_stdout, envelope
def _build_execution_bundle(req: CodeExecutionRequest, workdir: str) -> dict[str, str | bytes]:
arguments = req.arguments or {}
args_source = json.dumps(arguments, ensure_ascii=False)
args_name = "args.json"
code_bytes = base64.b64decode(req.code_b64)
if req.language == SupportLanguage.PYTHON:
code_name = "main.py"
runner_name = "runner.py"
runner_source = f"""import base64
import json
import os
import sys
os.makedirs(os.path.join(os.getcwd(), "artifacts"), exist_ok=True)
sys.path.insert(0, os.path.dirname(__file__))
from main import main
RESULT_MARKER_PREFIX = {RESULT_MARKER_PREFIX!r}
def emit_result(value):
payload = json.dumps(
{{
"present": True,
"value": value,
"type": "json",
}},
ensure_ascii=False,
separators=(",", ":"),
)
print(RESULT_MARKER_PREFIX + base64.b64encode(payload.encode("utf-8")).decode("ascii"))
if __name__ == "__main__":
with open(os.path.join(os.path.dirname(__file__), "args.json"), encoding="utf-8") as f:
args = json.load(f)
result = main(**args)
emit_result(result)
"""
elif req.language == SupportLanguage.NODEJS:
code_name = "main.js"
runner_name = "runner.js"
runner_source = """
const fs = require('fs');
const path = require('path');
const args = JSON.parse(fs.readFileSync(path.join(__dirname, 'args.json'), 'utf8'));
const mainPath = path.join(__dirname, 'main.js');
const RESULT_MARKER_PREFIX = '__RESULT_MARKER_PREFIX__';
function isPromise(value) {
return Boolean(value && typeof value.then === 'function');
}
function emitResult(value) {
if (typeof value === 'undefined') {
console.error('Error: main() must return a value. Use null for an empty result.');
process.exit(1);
}
const payload = JSON.stringify({ present: true, value, type: 'json' });
if (typeof payload === 'undefined') {
console.error('Error: main() returned a non-JSON-serializable value.');
process.exit(1);
}
console.log(RESULT_MARKER_PREFIX + Buffer.from(payload, 'utf8').toString('base64'));
}
if (fs.existsSync(mainPath)) {
const mod = require(mainPath);
const main = typeof mod === 'function' ? mod : mod.main;
if (typeof main !== 'function') {
console.error('Error: main is not a function');
process.exit(1);
}
if (typeof args === 'object' && args !== null) {
try {
const result = Promise.resolve(main(args));
if (isPromise(result)) {
result.then(output => {
emitResult(output);
}).catch(err => {
console.error('Error in async main function:', err);
process.exit(1);
});
} else {
emitResult(result);
}
} catch (err) {
console.error('Error when executing main:', err);
process.exit(1);
}
} else {
console.error('Error: args is not a valid object:', args);
process.exit(1);
}
} else {
console.error('main.js not found in the current directory');
process.exit(1);
}
"""
runner_source = runner_source.replace("__RESULT_MARKER_PREFIX__", RESULT_MARKER_PREFIX)
else:
assert False, "Will never reach here"
return {
"code_name": code_name,
"code_bytes": code_bytes,
"runner_name": runner_name,
"runner_source": runner_source,
"args_name": args_name,
"args_source": args_source,
}
def _build_container_run_args(language: SupportLanguage, task_id: str, container: str, runner_name: str) -> list[str]:
run_args = [
"docker",
"exec",
"--workdir",
f"/workspace/{task_id}",
container,
"timeout",
str(TIMEOUT),
language,
]
if language == SupportLanguage.PYTHON:
run_args.extend(["-I", "-B"])
run_args.append(runner_name)
return run_args
async def execute_code(req: CodeExecutionRequest):
"""Fully asynchronous execution logic"""
language = req.language
container = await allocate_container_blocking(language)
if not container:
@@ -46,93 +210,31 @@ async def execute_code(req: CodeExecutionRequest):
os.makedirs(workdir, mode=0o700, exist_ok=True)
try:
if language == SupportLanguage.PYTHON:
code_name = "main.py"
# code
code_path = os.path.join(workdir, code_name)
with open(code_path, "wb") as f:
f.write(base64.b64decode(req.code_b64))
# runner
runner_name = "runner.py"
runner_path = os.path.join(workdir, runner_name)
with open(runner_path, "w") as f:
f.write("""import json
import os
import sys
sys.path.insert(0, os.path.dirname(__file__))
from main import main
if __name__ == "__main__":
args = json.loads(sys.argv[1])
result = main(**args)
if result is not None:
print(result)
""")
bundle = _build_execution_bundle(req, workdir)
code_name = str(bundle["code_name"])
runner_name = str(bundle["runner_name"])
elif language == SupportLanguage.NODEJS:
code_name = "main.js"
code_path = os.path.join(workdir, "main.js")
with open(code_path, "wb") as f:
f.write(base64.b64decode(req.code_b64))
code_path = os.path.join(workdir, code_name)
with open(code_path, "wb") as f:
f.write(bundle["code_bytes"])
runner_name = "runner.js"
runner_path = os.path.join(workdir, "runner.js")
with open(runner_path, "w") as f:
f.write("""
const fs = require('fs');
const path = require('path');
runner_path = os.path.join(workdir, runner_name)
with open(runner_path, "w", encoding="utf-8") as f:
f.write(str(bundle["runner_source"]))
const args = JSON.parse(process.argv[2]);
const mainPath = path.join(__dirname, 'main.js');
args_path = os.path.join(workdir, str(bundle["args_name"]))
with open(args_path, "w", encoding="utf-8") as f:
f.write(str(bundle["args_source"]))
function isPromise(value) {
return Boolean(value && typeof value.then === 'function');
}
if (fs.existsSync(mainPath)) {
const mod = require(mainPath);
const main = typeof mod === 'function' ? mod : mod.main;
if (typeof main !== 'function') {
console.error('Error: main is not a function');
process.exit(1);
}
if (typeof args === 'object' && args !== null) {
try {
const result = main(args);
if (isPromise(result)) {
result.then(output => {
if (output !== null) {
console.log(output);
}
}).catch(err => {
console.error('Error in async main function:', err);
});
} else {
if (result !== null) {
console.log(result);
}
}
} catch (err) {
console.error('Error when executing main:', err);
}
} else {
console.error('Error: args is not a valid object:', args);
}
} else {
console.error('main.js not found in the current directory');
}
""")
# dirs
returncode, _, stderr = await async_run_command("docker", "exec", container, "mkdir", "-p", f"/workspace/{task_id}", timeout=5)
if returncode != 0:
raise RuntimeError(f"Directory creation failed: {stderr}")
# archive
tar_proc = await asyncio.create_subprocess_exec("tar", "czf", "-", "-C", workdir, code_name, runner_name, stdout=asyncio.subprocess.PIPE)
tar_proc = await asyncio.create_subprocess_exec(
"tar", "czf", "-", "-C", workdir, code_name, runner_name, str(bundle["args_name"]), stdout=asyncio.subprocess.PIPE
)
tar_stdout, _ = await tar_proc.communicate()
# unarchive
docker_proc = await asyncio.create_subprocess_exec(
"docker", "exec", "-i", container, "tar", "xzf", "-", "-C", f"/workspace/{task_id}", stdin=asyncio.subprocess.PIPE, stderr=asyncio.subprocess.PIPE
)
@@ -141,29 +243,11 @@ if (fs.existsSync(mainPath)) {
if docker_proc.returncode != 0:
raise RuntimeError(stderr.decode())
# exec
start_time = time.time()
try:
logger.info(f"Passed in args: {req.arguments}")
args_json = json.dumps(req.arguments or {})
run_args = [
"docker",
"exec",
"--workdir",
f"/workspace/{task_id}",
container,
"timeout",
str(TIMEOUT),
language,
]
# flags
if language == SupportLanguage.PYTHON:
run_args.extend(["-I", "-B"])
elif language == SupportLanguage.NODEJS:
run_args.extend([])
else:
assert False, "Will never reach here"
run_args.extend([runner_name, args_json])
arguments = req.arguments or {}
logger.info("Passed in args keys=%s size_bytes=%s", list(arguments.keys()), len(json.dumps(arguments, ensure_ascii=False).encode("utf-8")))
run_args = _build_container_run_args(language=language, task_id=task_id, container=container, runner_name=runner_name)
returncode, stdout, stderr = await async_run_command(
*run_args,
@@ -177,15 +261,18 @@ if (fs.existsSync(mainPath)) {
logger.info(f"{returncode=}")
logger.info(f"{stdout=}")
logger.info(f"{stderr=}")
logger.info(f"{args_json=}")
if returncode == 0:
clean_stdout, structured_result = _extract_result_envelope(stdout)
artifacts = await _collect_artifacts(container, task_id, workdir)
return CodeExecutionResult(
status=ResultStatus.SUCCESS,
stdout=str(stdout),
stdout=clean_stdout,
stderr=stderr,
exit_code=0,
time_used_ms=time_used_ms,
artifacts=artifacts,
result=structured_result,
)
elif returncode == 124:
return CodeExecutionResult(
@@ -223,12 +310,89 @@ if (fs.existsSync(mainPath)) {
return CodeExecutionResult(status=ResultStatus.PROGRAM_RUNNER_ERROR, stdout="", stderr=str(e), exit_code=-3, detail="internal_error")
finally:
# cleanup
cleanup_tasks = [async_run_command("docker", "exec", container, "rm", "-rf", f"/workspace/{task_id}"), async_run_command("rm", "-rf", workdir)]
await asyncio.gather(*cleanup_tasks, return_exceptions=True)
await release_container(container, language)
ALLOWED_ARTIFACT_EXTENSIONS = {
".png": "image/png",
".jpg": "image/jpeg",
".jpeg": "image/jpeg",
".svg": "image/svg+xml",
".pdf": "application/pdf",
".csv": "text/csv",
".json": "application/json",
".html": "text/html",
}
MAX_ARTIFACT_COUNT = 10
MAX_ARTIFACT_SIZE = 10 * 1024 * 1024 # 10MB per file
async def _collect_artifacts(container: str, task_id: str, host_workdir: str) -> list[ArtifactItem]:
artifacts_path = f"/workspace/{task_id}/artifacts"
# List files in the artifacts directory inside the container
returncode, stdout, _ = await async_run_command(
"docker", "exec", container, "find", artifacts_path,
"-maxdepth", "1", "-type", "f", timeout=5,
)
if returncode != 0 or not stdout.strip():
return []
raw_names = [line.split("/")[-1] for line in stdout.strip().splitlines() if line.strip()]
# Sanitize: reject names with path traversal or control characters
filenames = [n for n in raw_names if n and "/" not in n and "\\" not in n and ".." not in n and not n.startswith(".")]
if not filenames:
return []
items: list[ArtifactItem] = []
for fname in filenames[:MAX_ARTIFACT_COUNT]:
ext = os.path.splitext(fname)[1].lower()
mime_type = ALLOWED_ARTIFACT_EXTENSIONS.get(ext)
if not mime_type:
logger.warning(f"Skipping artifact with disallowed extension: {fname}")
continue
file_path = f"{artifacts_path}/{fname}"
# Check file size inside the container
returncode, size_str, _ = await async_run_command(
"docker", "exec", container, "stat", "-c", "%s", file_path, timeout=5,
)
if returncode != 0:
logger.warning(f"Failed to stat artifact {fname}")
continue
file_size = int(size_str.strip())
if file_size > MAX_ARTIFACT_SIZE:
logger.warning(f"Artifact {fname} too large ({file_size} bytes), skipping")
continue
if file_size == 0:
continue
# Read file content via docker exec (docker cp doesn't work with gVisor tmpfs)
returncode, content_b64, stderr = await async_run_command(
"docker", "exec", container, "base64", file_path, timeout=30,
)
if returncode != 0:
logger.warning(f"Failed to read artifact {fname}: {stderr}")
continue
content_b64 = content_b64.replace("\n", "").strip()
items.append(ArtifactItem(
name=fname,
mime_type=mime_type,
size=file_size,
content_b64=content_b64,
))
logger.info(f"Collected artifact: {fname} ({file_size} bytes, {mime_type})")
return items
def analyze_error_result(stderr: str, exit_code: int) -> CodeExecutionResult:
"""Analyze the error result and classify it"""
if "Permission denied" in stderr:

View File

@@ -14,6 +14,7 @@
# limitations under the License.
#
import ast
import re
from typing import List, Tuple
from core.logger import logger
@@ -25,7 +26,7 @@ class SecurePythonAnalyzer(ast.NodeVisitor):
An AST-based analyzer for detecting unsafe Python code patterns.
"""
DANGEROUS_IMPORTS = {"os", "subprocess", "sys", "shutil", "socket", "ctypes", "pickle", "threading", "multiprocessing", "asyncio", "http.client", "ftplib", "telnetlib"}
DANGEROUS_IMPORTS = {"os", "subprocess", "sys", "shutil", "socket", "ctypes", "pickle", "threading", "multiprocessing", "asyncio", "http.client", "ftplib", "telnetlib", "builtins"}
DANGEROUS_CALLS = {
"eval",
@@ -76,6 +77,16 @@ class SecurePythonAnalyzer(ast.NodeVisitor):
"""Check for dangerous function calls."""
if isinstance(node.func, ast.Name) and node.func.id in self.DANGEROUS_CALLS:
self.unsafe_items.append((f"Call: {node.func.id}", node.lineno))
elif isinstance(node.func, ast.Attribute) and node.func.attr in self.DANGEROUS_CALLS:
# Surface the attribute-style match in the analyzer log so that
# incident response can grep for it just like the other unsafe-item
# findings; the bare append is invisible to operators.
logger.warning(
"[SafeCheck] Attribute-style dangerous call detected: %s (line %s)",
node.func.attr,
node.lineno,
)
self.unsafe_items.append((f"Call: {node.func.attr}", node.lineno))
self.generic_visit(node)
def visit_Attribute(self, node: ast.Attribute):
@@ -151,6 +162,26 @@ class SecurePythonAnalyzer(ast.NodeVisitor):
self.generic_visit(node)
class SecureJavaScriptAnalyzer:
DANGEROUS_PATTERNS = [
(re.compile(r"""require\s*\(\s*['"`]child_process['"`]\s*\)"""), "Require: child_process"),
(re.compile(r"""require\s*\(\s*['"`]fs['"`]\s*\)"""), "Require: fs"),
(re.compile(r"""require\s*\(\s*['"`]worker_threads['"`]\s*\)"""), "Require: worker_threads"),
(re.compile(r"""\beval\s*\("""), "Call: eval"),
(re.compile(r"""\bFunction\s*\("""), "Call: Function"),
(re.compile(r"""\bprocess\s*\.\s*binding\s*\("""), "Call: process.binding"),
]
@classmethod
def analyze(cls, code: str) -> List[Tuple[str, int]]:
issues: List[Tuple[str, int]] = []
for pattern, description in cls.DANGEROUS_PATTERNS:
for match in pattern.finditer(code):
lineno = code.count("\n", 0, match.start()) + 1
issues.append((description, lineno))
return issues
def analyze_code_security(code: str, language: SupportLanguage) -> Tuple[bool, List[Tuple[str, int]]]:
"""
Analyze the provided code string and return whether it's safe and why.
@@ -168,6 +199,9 @@ def analyze_code_security(code: str, language: SupportLanguage) -> Tuple[bool, L
except Exception as e:
logger.error(f"[SafeCheck] Python parsing failed: {str(e)}")
return False, [(f"Parsing Error: {str(e)}", -1)]
else:
logger.warning(f"[SafeCheck] Unsupported language for security analysis: {language} — defaulting to SAFE (manual review recommended)")
return True, [(f"Unsupported language for security analysis: {language} — defaulted to SAFE, manual review recommended", -1)]
if language == SupportLanguage.NODEJS:
issues = SecureJavaScriptAnalyzer.analyze(code)
return len(issues) == 0, issues
logger.warning(f"[SafeCheck] Unsupported language for security analysis: {language}")
return False, [(f"Unsupported language for security analysis: {language}", -1)]

View File

@@ -24,20 +24,27 @@ This package contains:
- aliyun_codeinterpreter.py: Aliyun Code Interpreter provider implementation
Official Documentation: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
- e2b.py: E2B provider implementation
- local.py: Local process provider implementation
- ssh.py: Remote SSH provider implementation
"""
from .base import SandboxProvider, SandboxInstance, ExecutionResult
from .base import SandboxProvider, SandboxInstance, ExecutionResult, SandboxProviderConfigError
from .manager import ProviderManager
from .self_managed import SelfManagedProvider
from .aliyun_codeinterpreter import AliyunCodeInterpreterProvider
from .e2b import E2BProvider
from .local import LocalProvider
from .ssh import SSHProvider
__all__ = [
"SandboxProvider",
"SandboxInstance",
"ExecutionResult",
"SandboxProviderConfigError",
"ProviderManager",
"SelfManagedProvider",
"AliyunCodeInterpreterProvider",
"E2BProvider",
"LocalProvider",
"SSHProvider",
]

View File

@@ -30,6 +30,7 @@ https://api.aliyun.com/api/AgentRun/2025-09-10/CreateSandbox?lang=PYTHON
import logging
import os
import time
import json
from typing import Dict, Any, List, Optional
from datetime import datetime, timezone
@@ -37,6 +38,7 @@ from agentrun.sandbox import TemplateType, CodeLanguage, Template, TemplateInput
from agentrun.utils.config import Config
from agentrun.utils.exception import ServerError
from agent.sandbox.result_protocol import build_javascript_wrapper, build_python_wrapper, extract_structured_result
from .base import SandboxProvider, SandboxInstance, ExecutionResult
logger = logging.getLogger(__name__)
@@ -51,9 +53,9 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
"""
def __init__(self):
self.access_key_id: Optional[str] = None
self.access_key_secret: Optional[str] = None
self.account_id: Optional[str] = None
self.access_key_id: Optional[str] = ""
self.access_key_secret: Optional[str] = ""
self.account_id: Optional[str] = ""
self.region: str = "cn-hangzhou"
self.template_name: str = ""
self.timeout: int = 30
@@ -68,7 +70,7 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
config: Configuration dictionary with keys:
- access_key_id: Aliyun AccessKey ID
- access_key_secret: Aliyun AccessKey Secret
- account_id: Aliyun primary account ID (主账号ID)
- account_id: Aliyun primary account ID
- region: Region (default: "cn-hangzhou")
- template_name: Optional sandbox template name
- timeout: Request timeout in seconds (default: 30, max 30)
@@ -97,7 +99,7 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
return False
if not self.account_id:
logger.error("Aliyun Code Interpreter: Missing account_id (主账号ID)")
logger.error("Aliyun Code Interpreter: Missing account_id (primary account ID)")
return False
# Create SDK configuration
@@ -146,8 +148,6 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
try:
# Get or create template
from agentrun.sandbox import Sandbox
if self.template_name:
# Use existing template
template_name = self.template_name
@@ -226,48 +226,17 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
# Connect to existing sandbox instance
sandbox = Sandbox.connect(sandbox_id=instance_id, config=self._config)
# Convert language string to CodeLanguage enum
code_language = CodeLanguage.PYTHON if normalized_lang == "python" else CodeLanguage.JAVASCRIPT
# agentrun-sdk 0.0.26 only exposes CodeLanguage.PYTHON; keep JS as string fallback.
code_language = CodeLanguage.PYTHON if normalized_lang == "python" else "javascript"
# Wrap code to call main() function
# Matches self_managed provider behavior: call main(**arguments)
if normalized_lang == "python":
# Build arguments string for main() call
if arguments:
import json as json_module
args_json = json_module.dumps(arguments)
wrapped_code = f'''{code}
if __name__ == "__main__":
import json
result = main(**{args_json})
print(json.dumps(result) if isinstance(result, dict) else result)
'''
else:
wrapped_code = f'''{code}
if __name__ == "__main__":
import json
result = main()
print(json.dumps(result) if isinstance(result, dict) else result)
'''
else: # javascript
if arguments:
import json as json_module
args_json = json_module.dumps(arguments)
wrapped_code = f'''{code}
// Call main and output result
const result = main({args_json});
console.log(typeof result === 'object' ? JSON.stringify(result) : String(result));
'''
else:
wrapped_code = f'''{code}
// Call main and output result
const result = main();
console.log(typeof result === 'object' ? JSON.stringify(result) : String(result));
'''
args_json = json.dumps(arguments or {})
wrapped_code = (
build_python_wrapper(code, args_json)
if normalized_lang == "python"
else build_javascript_wrapper(code, args_json)
)
logger.debug(f"Aliyun Code Interpreter: Wrapped code (first 200 chars): {wrapped_code[:200]}")
start_time = time.time()
@@ -314,6 +283,7 @@ console.log(typeof result === 'object' ? JSON.stringify(result) : String(result)
stdout = "\n".join(stdout_parts)
stderr = "\n".join(stderr_parts)
stdout, structured_result = extract_structured_result(stdout)
logger.info(f"Aliyun Code Interpreter: stdout length={len(stdout)}, stderr length={len(stderr)}, exit_code={exit_code}")
if stdout:
@@ -331,6 +301,9 @@ console.log(typeof result === 'object' ? JSON.stringify(result) : String(result)
"language": normalized_lang,
"context_id": result.get("contextId") if isinstance(result, dict) else None,
"timeout": timeout,
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
},
)
@@ -429,7 +402,7 @@ console.log(typeof result === 'object' ? JSON.stringify(result) : String(result)
"required": True,
"label": "Account ID",
"placeholder": "1234567890...",
"description": "Aliyun primary account ID (主账号ID), required for API calls",
"description": "Aliyun primary account ID, required for API calls",
},
"region": {
"type": "string",

View File

@@ -26,6 +26,10 @@ from dataclasses import dataclass
from typing import Dict, Any, Optional, List
class SandboxProviderConfigError(Exception):
"""Raised when the selected provider is explicitly configured but unusable."""
@dataclass
class SandboxInstance:
"""Represents a sandbox execution instance"""
@@ -209,4 +213,4 @@ class SandboxProvider(ABC):
>>> return True, None
"""
# Default implementation: no custom validation
return True, None
return True, None

View File

@@ -0,0 +1,352 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import base64
import json
import mimetypes
import os
import shutil
import signal
import subprocess
import time
import uuid
from pathlib import Path
from typing import Any, Dict, List, Optional
from agent.sandbox.result_protocol import build_javascript_wrapper, build_python_wrapper, extract_structured_result
from .base import ExecutionResult, SandboxInstance, SandboxProvider, SandboxProviderConfigError
ALLOWED_ARTIFACT_EXTENSIONS = {
".csv",
".html",
".jpeg",
".jpg",
".json",
".pdf",
".png",
".svg",
}
LOCAL_PYTHON_THREAD_ENV_VARS = (
"OPENBLAS_NUM_THREADS",
"OMP_NUM_THREADS",
"MKL_NUM_THREADS",
"NUMEXPR_NUM_THREADS",
"BLIS_NUM_THREADS",
"VECLIB_MAXIMUM_THREADS",
)
class LocalProvider(SandboxProvider):
"""
Execute code as a local child process.
This provider is intentionally gated by SANDBOX_LOCAL_ENABLED because it is
not a sandbox boundary. Use a low-privilege runtime account.
"""
def __init__(self):
self.python_bin = "python3"
self.node_bin = "node"
self.work_dir = Path("/tmp/ragflow-codeexec")
self.timeout = 30
self.max_memory_mb = 512
self.max_output_bytes = 1024 * 1024
self.max_artifacts = 20
self.max_artifact_bytes = 10 * 1024 * 1024
self._initialized = False
self._instances: dict[str, Path] = {}
def initialize(self, config: Dict[str, Any]) -> bool:
self.python_bin = str(config.get("python_bin", "python3"))
self.node_bin = str(config.get("node_bin", "node"))
self.work_dir = Path(str(config.get("work_dir", "/tmp/ragflow-codeexec"))).resolve()
self.timeout = int(config.get("timeout", 30))
self.max_memory_mb = int(config.get("max_memory_mb", 512))
self.max_output_bytes = int(config.get("max_output_bytes", 1024 * 1024))
self.max_artifacts = int(config.get("max_artifacts", 20))
self.max_artifact_bytes = int(config.get("max_artifact_bytes", 10 * 1024 * 1024))
self._validate_limits()
self.work_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
self._initialized = True
return True
def create_instance(self, template: str = "python") -> SandboxInstance:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
language = self._normalize_language(template)
instance_id = str(uuid.uuid4())
instance_dir = self.work_dir / instance_id
instance_dir.mkdir(mode=0o700)
(instance_dir / "artifacts").mkdir(mode=0o700)
self._instances[instance_id] = instance_dir
return SandboxInstance(
instance_id=instance_id,
provider="local",
status="running",
metadata={"language": language, "work_dir": str(instance_dir)},
)
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10,
arguments: Optional[Dict[str, Any]] = None,
) -> ExecutionResult:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
normalized_lang = self._normalize_language(language)
instance_dir = self._instances[instance_id]
args_json = json.dumps(arguments or {}, ensure_ascii=False)
command, script_path = self._prepare_script(instance_dir, normalized_lang, code, args_json)
requested_timeout = self.timeout if timeout is None else int(timeout)
if requested_timeout <= 0:
raise RuntimeError(f"Execution timeout must be greater than 0 seconds, got {requested_timeout}.")
exec_timeout = min(requested_timeout, self.timeout)
start_time = time.time()
process = subprocess.Popen(
command,
cwd=instance_dir,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
encoding="utf-8",
errors="replace",
env=self._build_child_env(instance_dir),
preexec_fn=self._limit_child_process if os.name == "posix" else None,
start_new_session=os.name == "posix",
)
try:
stdout, stderr = process.communicate(timeout=exec_timeout)
except subprocess.TimeoutExpired:
if os.name == "posix":
os.killpg(process.pid, signal.SIGKILL)
else:
process.kill()
process.communicate()
raise TimeoutError(f"Execution timed out after {exec_timeout} seconds")
execution_time = time.time() - start_time
self._validate_output_size(stdout, stderr)
stdout, structured_result = extract_structured_result(stdout)
return ExecutionResult(
stdout=stdout,
stderr=stderr,
exit_code=process.returncode,
execution_time=execution_time,
metadata={
"instance_id": instance_id,
"language": normalized_lang,
"script_path": str(script_path),
"status": "ok" if process.returncode == 0 else "error",
"timeout": exec_timeout,
"artifacts": self._collect_artifacts(instance_dir / "artifacts"),
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
},
)
def destroy_instance(self, instance_id: str) -> bool:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
instance_dir = self._instances.pop(instance_id)
shutil.rmtree(instance_dir)
return True
def health_check(self) -> bool:
return self._initialized and self.work_dir.exists() and os.access(self.work_dir, os.W_OK)
def get_supported_languages(self) -> List[str]:
return ["python", "javascript", "nodejs"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
return {
"python_bin": {
"type": "string",
"required": False,
"default": "python3",
"label": "Python Binary",
"description": "Python executable used for local code execution.",
},
"node_bin": {
"type": "string",
"required": False,
"default": "node",
"label": "Node.js Binary",
"description": "Node.js executable used for local JavaScript execution.",
},
"work_dir": {
"type": "string",
"required": False,
"default": "/tmp/ragflow-codeexec",
"label": "Working Directory",
"description": "Directory used to store temporary scripts and artifacts on the current host.",
},
"timeout": {
"type": "integer",
"required": False,
"default": 30,
"label": "Timeout (seconds)",
"description": "Maximum execution time for each local run. Unit: seconds.",
"min": 1,
"max": 600,
},
"max_memory_mb": {
"type": "integer",
"required": False,
"default": 512,
"label": "Max Memory (MB)",
"description": "Address-space memory limit for the local child process. Unit: MB.",
"min": 1,
"max": 65536,
},
"max_output_bytes": {
"type": "integer",
"required": False,
"default": 1048576,
"label": "Max Output (bytes)",
"description": "Maximum combined stdout and stderr size. Unit: bytes.",
"min": 1024,
"max": 10485760,
},
"max_artifacts": {
"type": "integer",
"required": False,
"default": 20,
"label": "Max Artifacts",
"description": "Maximum number of files collected from the artifacts directory.",
"min": 0,
"max": 100,
},
"max_artifact_bytes": {
"type": "integer",
"required": False,
"default": 10485760,
"label": "Max Artifact Size (bytes)",
"description": "Maximum size of a single artifact file. Unit: bytes.",
"min": 1024,
"max": 104857600,
},
}
def _validate_limits(self) -> None:
if self.timeout <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_TIMEOUT must be greater than 0.")
if self.max_memory_mb <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_MEMORY_MB must be greater than 0.")
if self.max_output_bytes <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_OUTPUT_BYTES must be greater than 0.")
if self.max_artifacts < 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_ARTIFACTS must be greater than or equal to 0.")
if self.max_artifact_bytes <= 0:
raise SandboxProviderConfigError("SANDBOX_LOCAL_MAX_ARTIFACT_BYTES must be greater than 0.")
def _prepare_script(self, instance_dir: Path, language: str, code: str, args_json: str) -> tuple[list[str], Path]:
if language == "python":
script_path = instance_dir / "main.py"
script_path.write_text(build_python_wrapper(code, args_json), encoding="utf-8")
return [self.python_bin, str(script_path)], script_path
if language in {"javascript", "nodejs"}:
script_path = instance_dir / "main.js"
script_path.write_text(build_javascript_wrapper(code, args_json), encoding="utf-8")
return [self.node_bin, str(script_path)], script_path
raise RuntimeError(f"Unsupported language for local provider: {language}")
def _build_child_env(self, instance_dir: Path) -> dict[str, str]:
env = {
"HOME": str(instance_dir),
"MPLBACKEND": "Agg",
"PATH": os.environ.get("PATH", ""),
"PYTHONUNBUFFERED": "1",
"TMPDIR": str(instance_dir),
}
for name in LOCAL_PYTHON_THREAD_ENV_VARS:
value = os.environ.get(name)
if value is not None:
env[name] = value
return env
def _limit_child_process(self) -> None:
import resource
self._set_resource_limit(resource.RLIMIT_CPU, self.timeout + 1)
self._set_resource_limit(resource.RLIMIT_AS, self.max_memory_mb * 1024 * 1024)
self._set_resource_limit(resource.RLIMIT_FSIZE, self.max_artifact_bytes)
self._set_resource_limit(resource.RLIMIT_NOFILE, 64)
@staticmethod
def _set_resource_limit(kind: int, value: int) -> None:
import resource
_, hard = resource.getrlimit(kind)
limit = value if hard == resource.RLIM_INFINITY else min(value, hard)
resource.setrlimit(kind, (limit, limit))
def _validate_output_size(self, stdout: str, stderr: str) -> None:
output_size = len((stdout or "").encode("utf-8")) + len((stderr or "").encode("utf-8"))
if output_size > self.max_output_bytes:
raise RuntimeError(f"Local execution output exceeded {self.max_output_bytes} bytes.")
def _collect_artifacts(self, artifacts_dir: Path) -> list[dict[str, Any]]:
artifacts: list[dict[str, Any]] = []
for path in sorted(artifacts_dir.rglob("*")):
if path.is_symlink():
raise RuntimeError(f"Artifact symlinks are not allowed: {path.name}")
if path.is_dir():
continue
if not path.is_file():
raise RuntimeError(f"Unsupported artifact entry: {path.name}")
if len(artifacts) >= self.max_artifacts:
raise RuntimeError(f"Local execution produced more than {self.max_artifacts} artifacts.")
size = path.stat().st_size
if size > self.max_artifact_bytes:
raise RuntimeError(f"Artifact exceeds {self.max_artifact_bytes} bytes: {path.name}")
ext = path.suffix.lower()
if ext not in ALLOWED_ARTIFACT_EXTENSIONS:
raise RuntimeError(f"Unsupported artifact type: {path.name}")
artifacts.append(
{
"name": path.relative_to(artifacts_dir).as_posix(),
"content_b64": base64.b64encode(path.read_bytes()).decode("ascii"),
"mime_type": mimetypes.guess_type(path.name)[0] or "application/octet-stream",
"size": size,
}
)
return artifacts
@staticmethod
def _normalize_language(language: str) -> str:
lang_lower = (language or "python").lower()
if lang_lower in {"python", "python3"}:
return "python"
if lang_lower in {"javascript", "nodejs"}:
return "nodejs"
return lang_lower

View File

@@ -22,6 +22,7 @@ a pool of Docker containers with gVisor for secure code execution.
"""
import base64
import os
import time
import uuid
from typing import Dict, Any, List, Optional
@@ -40,10 +41,10 @@ class SelfManagedProvider(SandboxProvider):
"""
def __init__(self):
self.endpoint: str = "http://localhost:9385"
self.endpoint: str = "http://sandbox-executor-manager:9385"
self.timeout: int = 30
self.max_retries: int = 3
self.pool_size: int = 10
self.pool_size: int = 3
self._initialized: bool = False
def initialize(self, config: Dict[str, Any]) -> bool:
@@ -52,7 +53,7 @@ class SelfManagedProvider(SandboxProvider):
Args:
config: Configuration dictionary with keys:
- endpoint: HTTP endpoint (default: "http://localhost:9385")
- endpoint: HTTP endpoint (default: "http://sandbox-executor-manager:9385")
- timeout: Request timeout in seconds (default: 30)
- max_retries: Maximum retry attempts (default: 3)
- pool_size: Container pool size for info (default: 10)
@@ -60,30 +61,13 @@ class SelfManagedProvider(SandboxProvider):
Returns:
True if initialization successful, False otherwise
"""
self.endpoint = config.get("endpoint", "http://localhost:9385")
self.endpoint = config.get("endpoint", "http://sandbox-executor-manager:9385")
self.timeout = config.get("timeout", 30)
self.max_retries = config.get("max_retries", 3)
self.pool_size = config.get("pool_size", 10)
self.pool_size = config.get("executor_manager_pool_size", config.get("pool_size", 3))
# Validate endpoint is accessible
if not self.health_check():
# Try to fall back to SANDBOX_HOST from settings if we are using localhost
if "localhost" in self.endpoint or "127.0.0.1" in self.endpoint:
try:
from api import settings
if settings.SANDBOX_HOST and settings.SANDBOX_HOST not in self.endpoint:
original_endpoint = self.endpoint
self.endpoint = f"http://{settings.SANDBOX_HOST}:9385"
if self.health_check():
import logging
logging.warning(f"Sandbox self_managed: Connected using settings.SANDBOX_HOST fallback: {self.endpoint} (original: {original_endpoint})")
self._initialized = True
return True
else:
self.endpoint = original_endpoint # Restore if fallback also fails
except ImportError:
pass
return False
self._initialized = True
@@ -187,6 +171,7 @@ class SelfManagedProvider(SandboxProvider):
)
result = response.json()
structured_result = result.get("result") or {}
return ExecutionResult(
stdout=result.get("stdout", ""),
@@ -199,6 +184,10 @@ class SelfManagedProvider(SandboxProvider):
"memory_used_kb": result.get("memory_used_kb"),
"detail": result.get("detail"),
"instance_id": instance_id,
"artifacts": result.get("artifacts", []),
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
}
)
@@ -265,9 +254,11 @@ class SelfManagedProvider(SandboxProvider):
"type": "string",
"required": True,
"label": "Executor Manager Endpoint",
"placeholder": "http://localhost:9385",
"default": "http://localhost:9385",
"description": "HTTP endpoint of the executor_manager service"
"placeholder": "http://sandbox-executor-manager:9385",
"default": "http://sandbox-executor-manager:9385",
"description": "HTTP endpoint used by RAGFlow to call sandbox-executor-manager.",
"scope": "runtime",
"readonly": False,
},
"timeout": {
"type": "integer",
@@ -276,26 +267,86 @@ class SelfManagedProvider(SandboxProvider):
"default": 30,
"min": 5,
"max": 300,
"description": "HTTP request timeout for code execution"
"description": "Maximum request time for a single code execution call. Unit: seconds.",
"scope": "runtime",
"readonly": False,
},
"max_retries": {
"type": "integer",
"executor_manager_image": {
"type": "string",
"required": False,
"label": "Max Retries",
"default": 3,
"min": 0,
"max": 10,
"description": "Maximum number of retry attempts for failed requests"
"label": "Executor Manager Image",
"default": os.getenv("SANDBOX_EXECUTOR_MANAGER_IMAGE", "infiniflow/sandbox-executor-manager:latest"),
"description": "Docker image used by sandbox-executor-manager.",
"scope": "deployment",
"readonly": True,
},
"pool_size": {
"executor_manager_pool_size": {
"type": "integer",
"required": False,
"label": "Container Pool Size",
"default": 10,
"default": int(os.getenv("SANDBOX_EXECUTOR_MANAGER_POOL_SIZE", "3")),
"min": 1,
"max": 100,
"description": "Size of the container pool (configured in executor_manager)"
}
"description": "Container pool size used by sandbox-executor-manager.",
"scope": "deployment",
"readonly": True,
},
"base_python_image": {
"type": "string",
"required": False,
"label": "Base Python Image",
"default": os.getenv("SANDBOX_BASE_PYTHON_IMAGE", "infiniflow/sandbox-base-python:latest"),
"description": "Python runtime image used by executor-managed containers.",
"scope": "deployment",
"readonly": True,
},
"base_nodejs_image": {
"type": "string",
"required": False,
"label": "Base Node.js Image",
"default": os.getenv("SANDBOX_BASE_NODEJS_IMAGE", "infiniflow/sandbox-base-nodejs:latest"),
"description": "Node.js runtime image used by executor-managed containers.",
"scope": "deployment",
"readonly": True,
},
"executor_manager_port": {
"type": "integer",
"required": False,
"label": "Executor Manager Port",
"default": int(os.getenv("SANDBOX_EXECUTOR_MANAGER_PORT", "9385")),
"min": 1,
"max": 65535,
"description": "Host port exposed by sandbox-executor-manager.",
"scope": "deployment",
"readonly": True,
},
"enable_seccomp": {
"type": "boolean",
"required": False,
"label": "Enable Seccomp",
"default": os.getenv("SANDBOX_ENABLE_SECCOMP", "false").lower() == "true",
"description": "Whether sandbox-executor-manager starts containers with seccomp enabled.",
"scope": "deployment",
"readonly": True,
},
"max_memory": {
"type": "string",
"required": False,
"label": "Max Memory",
"default": os.getenv("SANDBOX_MAX_MEMORY", "256m"),
"description": "Memory limit applied to each sandbox container. Common format: 256m or 1g.",
"scope": "deployment",
"readonly": True,
},
"sandbox_timeout": {
"type": "string",
"required": False,
"label": "Sandbox Timeout",
"default": os.getenv("SANDBOX_TIMEOUT", "10s"),
"description": "Executor-manager container timeout for each sandbox run. Common format: 10s or 1m.",
"scope": "deployment",
"readonly": True,
},
}
def _normalize_language(self, language: str) -> str:
@@ -342,7 +393,7 @@ class SelfManagedProvider(SandboxProvider):
return False, f"Invalid endpoint format: {endpoint}. Must start with http:// or https://"
# Validate pool_size is positive
pool_size = config.get("pool_size", 10)
pool_size = config.get("executor_manager_pool_size", config.get("pool_size", 3))
if isinstance(pool_size, int) and pool_size <= 0:
return False, "Pool size must be greater than 0"

View File

@@ -0,0 +1,664 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
from __future__ import annotations
import base64
import io
import json
import mimetypes
import os
import posixpath
import shlex
import stat
import time
import uuid
from typing import TYPE_CHECKING, Any, Dict, List, Optional
from agent.sandbox.result_protocol import (
build_javascript_wrapper,
build_python_wrapper,
extract_structured_result,
)
from .base import (
ExecutionResult,
SandboxInstance,
SandboxProvider,
SandboxProviderConfigError,
)
if TYPE_CHECKING:
import paramiko
ALLOWED_ARTIFACT_EXTENSIONS = {
".csv",
".html",
".jpeg",
".jpg",
".json",
".pdf",
".png",
".svg",
}
class SSHProvider(SandboxProvider):
"""Execute code on a remote host through SSH."""
def __init__(self):
self.host = ""
self.port = 22
self.username = ""
self.password = ""
self.private_key = ""
self.passphrase = ""
self.python_bin = "python3"
self.node_bin = "node"
self.work_dir = "/tmp"
self.timeout = 30
self.max_output_bytes = 1024 * 1024
self.max_artifacts = 20
self.max_artifact_bytes = 10 * 1024 * 1024
self._initialized = False
self._instances: dict[str, dict[str, Any]] = {}
def initialize(self, config: Dict[str, Any]) -> bool:
self.host = str(config.get("host", "")).strip()
self.port = int(config.get("port", 22) or 22)
self.username = str(config.get("username", "")).strip()
self.password = str(config.get("password", "") or "")
self.private_key = str(config.get("private_key", "") or "")
self.passphrase = str(config.get("passphrase", "") or "")
self.python_bin = str(config.get("python_bin", "python3") or "python3").strip() or "python3"
self.node_bin = str(config.get("node_bin", "node") or "node").strip() or "node"
self.work_dir = str(config.get("work_dir", "/tmp") or "/tmp").strip() or "/tmp"
self.timeout = int(config.get("timeout", 30) or 30)
self.max_output_bytes = int(config.get("max_output_bytes", 1024 * 1024) or 1024 * 1024)
self.max_artifacts = int(config.get("max_artifacts", 20) or 20)
self.max_artifact_bytes = int(config.get("max_artifact_bytes", 10 * 1024 * 1024) or 10 * 1024 * 1024)
is_valid, error_message = self.validate_config(
{
"host": self.host,
"port": self.port,
"username": self.username,
"password": self.password,
"private_key": self.private_key,
"passphrase": self.passphrase,
"python_bin": self.python_bin,
"node_bin": self.node_bin,
"work_dir": self.work_dir,
"timeout": self.timeout,
"max_output_bytes": self.max_output_bytes,
"max_artifacts": self.max_artifacts,
"max_artifact_bytes": self.max_artifact_bytes,
}
)
if not is_valid:
raise SandboxProviderConfigError(error_message or "Invalid SSH provider configuration.")
self._assert_connectivity()
self._initialized = True
return True
def create_instance(self, template: str = "python") -> SandboxInstance:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
language = self._normalize_language(template)
client = self._create_ssh_client()
sftp = client.open_sftp()
try:
remote_work_dir = self._create_remote_workspace(client)
stdout, stderr, exit_code = self._run_remote_command(
client,
f"mkdir -p {shlex.quote(posixpath.join(remote_work_dir, 'artifacts'))}",
timeout=min(self.timeout, 10),
)
if exit_code != 0:
raise RuntimeError(
f"Failed to create remote artifacts directory: {stderr or stdout or 'unknown error'}"
)
except Exception:
sftp.close()
client.close()
raise
instance_id = str(uuid.uuid4())
self._instances[instance_id] = {
"client": client,
"sftp": sftp,
"remote_work_dir": remote_work_dir,
"language": language,
}
return SandboxInstance(
instance_id=instance_id,
provider="ssh",
status="running",
metadata={"language": language, "remote_work_dir": remote_work_dir},
)
def execute_code(
self,
instance_id: str,
code: str,
language: str,
timeout: int = 10,
arguments: Optional[Dict[str, Any]] = None,
) -> ExecutionResult:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
if instance_id not in self._instances:
raise RuntimeError(f"Unknown SSH sandbox instance: {instance_id}")
normalized_lang = self._normalize_language(language)
instance = self._instances[instance_id]
client: paramiko.SSHClient = instance["client"]
sftp: paramiko.SFTPClient = instance["sftp"]
remote_work_dir: str = instance["remote_work_dir"]
args_json = json.dumps(arguments or {}, ensure_ascii=False)
remote_script_path, command = self._upload_script(
sftp=sftp,
remote_work_dir=remote_work_dir,
language=normalized_lang,
code=code,
args_json=args_json,
)
requested_timeout = self.timeout if timeout is None else int(timeout)
if requested_timeout <= 0:
raise RuntimeError(f"Execution timeout must be greater than 0 seconds, got {requested_timeout}.")
exec_timeout = min(requested_timeout, self.timeout)
start_time = time.time()
stdout, stderr, exit_code = self._run_remote_command(client, command, timeout=exec_timeout)
execution_time = time.time() - start_time
self._validate_output_size(stdout, stderr)
stdout, structured_result = extract_structured_result(stdout)
return ExecutionResult(
stdout=stdout,
stderr=stderr,
exit_code=exit_code,
execution_time=execution_time,
metadata={
"instance_id": instance_id,
"language": normalized_lang,
"script_path": remote_script_path,
"remote_work_dir": remote_work_dir,
"status": "ok" if exit_code == 0 else "error",
"timeout": exec_timeout,
"command": command,
"artifacts": self._collect_artifacts(
sftp, posixpath.join(remote_work_dir, "artifacts")
),
"result_present": structured_result.get("present", False),
"result_value": structured_result.get("value"),
"result_type": structured_result.get("type"),
},
)
def destroy_instance(self, instance_id: str) -> bool:
if not self._initialized:
raise RuntimeError("Provider not initialized. Call initialize() first.")
if instance_id not in self._instances:
return True
instance = self._instances.pop(instance_id)
client: paramiko.SSHClient = instance["client"]
sftp: paramiko.SFTPClient = instance["sftp"]
remote_work_dir: str = instance["remote_work_dir"]
cleanup_error: Optional[Exception] = None
try:
stdout, stderr, exit_code = self._run_remote_command(
client,
f"rm -rf {shlex.quote(remote_work_dir)}",
timeout=min(self.timeout, 10),
)
if exit_code != 0:
raise RuntimeError(stderr or stdout or "unknown error")
except Exception as exc:
cleanup_error = exc
finally:
try:
sftp.close()
finally:
client.close()
if cleanup_error is not None:
raise RuntimeError(f"Failed to clean remote workspace {remote_work_dir}: {cleanup_error}")
return True
def health_check(self) -> bool:
try:
self._assert_connectivity()
return True
except Exception:
return False
def _assert_connectivity(self) -> None:
try:
client = self._create_ssh_client()
try:
_, stderr, exit_code = self._run_remote_command(
client,
"true",
timeout=min(self.timeout, 10),
)
if exit_code != 0:
raise SandboxProviderConfigError(
f"SSH connectivity check failed on {self.username}@{self.host}:{self.port}: "
f"{stderr or 'remote command returned non-zero exit status'}"
)
finally:
client.close()
except SandboxProviderConfigError:
raise
except Exception as exc:
raise SandboxProviderConfigError(
f"Failed to connect to SSH host {self.username}@{self.host}:{self.port}: {exc}"
) from exc
def get_supported_languages(self) -> List[str]:
return ["python", "javascript", "nodejs"]
@staticmethod
def get_config_schema() -> Dict[str, Dict]:
return {
"host": {
"type": "string",
"required": True,
"label": "SSH Host",
"placeholder": "192.168.1.10",
"description": "Remote host that will execute generated code.",
},
"port": {
"type": "integer",
"required": True,
"label": "SSH Port",
"default": 22,
"min": 1,
"max": 65535,
"description": "SSH port on the remote host.",
},
"username": {
"type": "string",
"required": True,
"label": "SSH Username",
"placeholder": "ragflow",
"description": "Username used to connect to the remote host.",
},
"password": {
"type": "string",
"required": False,
"label": "SSH Password",
"secret": True,
"placeholder": "Optional when using a private key",
"description": "Password-based SSH authentication.",
},
"private_key": {
"type": "string",
"required": False,
"label": "SSH Private Key",
"secret": True,
"multiline": True,
"placeholder": "Paste PEM content or enter a local file path",
"description": "Private key PEM content or a readable private key path on the RAGFlow host.",
},
"passphrase": {
"type": "string",
"required": False,
"label": "Private Key Passphrase",
"secret": True,
"placeholder": "Optional",
"description": "Passphrase for the private key if it is encrypted.",
},
"python_bin": {
"type": "string",
"required": False,
"default": "python3",
"label": "Python Binary",
"description": "Python executable used for remote code execution.",
},
"node_bin": {
"type": "string",
"required": False,
"default": "node",
"label": "Node.js Binary",
"description": "Node.js executable used for remote JavaScript execution.",
},
"work_dir": {
"type": "string",
"required": False,
"label": "Remote Workspace Root",
"default": "/tmp",
"placeholder": "/tmp",
"description": "Writable remote directory used to create a temporary workspace.",
},
"timeout": {
"type": "integer",
"required": False,
"label": "Timeout (seconds)",
"default": 30,
"min": 1,
"max": 600,
"description": "Maximum SSH execution time for a single run.",
},
"max_output_bytes": {
"type": "integer",
"required": False,
"label": "Max Output Bytes",
"default": 1048576,
"min": 1024,
"max": 10485760,
"description": "Maximum combined stdout and stderr size.",
},
"max_artifacts": {
"type": "integer",
"required": False,
"label": "Max Artifacts",
"default": 20,
"min": 0,
"max": 100,
"description": "Maximum number of files collected from the remote artifacts directory.",
},
"max_artifact_bytes": {
"type": "integer",
"required": False,
"label": "Max Artifact Bytes",
"default": 10485760,
"min": 1024,
"max": 104857600,
"description": "Maximum size of a single artifact file in bytes.",
},
}
def validate_config(self, config: Dict[str, Any]) -> tuple[bool, Optional[str]]:
host = str(config.get("host", "") or "").strip()
username = str(config.get("username", "") or "").strip()
password = str(config.get("password", "") or "")
private_key = str(config.get("private_key", "") or "")
python_bin = str(config.get("python_bin", "python3") or "python3").strip()
node_bin = str(config.get("node_bin", "node") or "node").strip()
if not host:
return False, "SSH host is required"
if not username:
return False, "SSH username is required"
if not password and not private_key:
return False, "Either password or private_key must be provided"
if not python_bin:
return False, "Python binary is required"
if not node_bin:
return False, "Node.js binary is required"
try:
port = int(config.get("port", 22) or 22)
except (TypeError, ValueError):
return False, "SSH port must be an integer"
if port <= 0 or port > 65535:
return False, "SSH port must be between 1 and 65535"
for key in ("timeout", "max_output_bytes", "max_artifacts", "max_artifact_bytes"):
try:
value = int(config.get(key, 0) or 0)
except (TypeError, ValueError):
return False, f"{key} must be an integer"
if key == "max_artifacts":
if value < 0:
return False, "max_artifacts must be greater than or equal to 0"
elif value <= 0:
return False, f"{key} must be greater than 0"
return True, None
def _create_ssh_client(self) -> paramiko.SSHClient:
paramiko = _get_paramiko_module()
client = paramiko.SSHClient()
client.set_missing_host_key_policy(paramiko.AutoAddPolicy())
connect_kwargs: dict[str, Any] = {
"hostname": self.host,
"port": self.port,
"username": self.username,
"timeout": self.timeout,
"banner_timeout": self.timeout,
"auth_timeout": self.timeout,
"look_for_keys": False,
"allow_agent": False,
}
if self.private_key:
connect_kwargs["pkey"] = self._load_private_key()
if self.password:
connect_kwargs["password"] = self.password
client.connect(**connect_kwargs)
return client
def _load_private_key(self) -> paramiko.PKey:
paramiko = _get_paramiko_module()
loaders = (
paramiko.RSAKey,
paramiko.Ed25519Key,
paramiko.ECDSAKey,
paramiko.DSSKey,
)
errors: list[str] = []
private_key_value = self.private_key.strip()
passphrase = self.passphrase or None
if os.path.exists(private_key_value):
for key_cls in loaders:
try:
return key_cls.from_private_key_file(private_key_value, password=passphrase)
except Exception as exc:
errors.append(str(exc))
else:
for key_cls in loaders:
try:
return key_cls.from_private_key(io.StringIO(private_key_value), password=passphrase)
except Exception as exc:
errors.append(str(exc))
raise SandboxProviderConfigError(
"Failed to load SSH private key. " + "; ".join(error for error in errors if error)
)
def _create_remote_workspace(self, client: paramiko.SSHClient) -> str:
base_dir = self.work_dir.rstrip("/") or "/tmp"
template = posixpath.join(base_dir, "ragflow-codeexec.XXXXXX")
stdout, stderr, exit_code = self._run_remote_command(
client,
f"mkdir -p {shlex.quote(base_dir)} && mktemp -d {shlex.quote(template)}",
timeout=min(self.timeout, 10),
)
if exit_code != 0:
raise RuntimeError(
f"Failed to create remote workspace on {self.host}: {stderr or stdout or 'unknown error'}"
)
remote_work_dir = stdout.strip().splitlines()[-1] if stdout.strip() else ""
if not remote_work_dir:
raise RuntimeError("Remote workspace creation did not return a path.")
return remote_work_dir
def _upload_script(
self,
sftp: paramiko.SFTPClient,
remote_work_dir: str,
language: str,
code: str,
args_json: str,
) -> tuple[str, str]:
if language == "python":
script_name = "main.py"
script_content = build_python_wrapper(code, args_json)
elif language in {"javascript", "nodejs"}:
script_name = "main.js"
script_content = build_javascript_wrapper(code, args_json)
else:
raise RuntimeError(f"Unsupported language for SSH provider: {language}")
remote_script_path = posixpath.join(remote_work_dir, script_name)
with sftp.file(remote_script_path, "w") as remote_file:
remote_file.write(script_content)
command = self._build_execution_command(remote_work_dir, remote_script_path, language)
return remote_script_path, command
def _build_execution_command(self, remote_work_dir: str, remote_script_path: str, language: str) -> str:
normalized_lang = self._normalize_language(language)
if normalized_lang == "python":
executable = self.python_bin
elif normalized_lang == "nodejs":
executable = self.node_bin
else:
raise RuntimeError(f"Unsupported language for SSH provider: {language}")
return (
f"cd {shlex.quote(remote_work_dir)} && "
f"{shlex.quote(executable)} {shlex.quote(remote_script_path)}"
)
def _run_remote_command(
self,
client: paramiko.SSHClient,
command: str,
timeout: int,
) -> tuple[str, str, int]:
stdin, stdout_stream, stderr_stream = client.exec_command(command, timeout=timeout)
stdin.close()
channel = stdout_stream.channel
stdout_chunks: list[bytes] = []
stderr_chunks: list[bytes] = []
deadline = time.time() + timeout
while True:
while channel.recv_ready():
stdout_chunks.append(channel.recv(65536))
while channel.recv_stderr_ready():
stderr_chunks.append(channel.recv_stderr(65536))
if channel.exit_status_ready():
break
if time.time() > deadline:
channel.close()
raise TimeoutError(f"Execution timed out after {timeout} seconds")
time.sleep(0.1)
while channel.recv_ready():
stdout_chunks.append(channel.recv(65536))
while channel.recv_stderr_ready():
stderr_chunks.append(channel.recv_stderr(65536))
exit_code = channel.recv_exit_status()
stdout = b"".join(stdout_chunks).decode("utf-8", errors="replace")
stderr = b"".join(stderr_chunks).decode("utf-8", errors="replace")
return stdout, stderr, exit_code
def _validate_output_size(self, stdout: str, stderr: str) -> None:
output_size = len((stdout or "").encode("utf-8")) + len((stderr or "").encode("utf-8"))
if output_size > self.max_output_bytes:
raise RuntimeError(f"SSH execution output exceeded {self.max_output_bytes} bytes.")
def _collect_artifacts(
self,
sftp: paramiko.SFTPClient,
artifacts_dir: str,
) -> list[dict[str, Any]]:
artifacts: list[dict[str, Any]] = []
self._collect_artifacts_recursive(sftp, artifacts_dir, "", artifacts)
return artifacts
def _collect_artifacts_recursive(
self,
sftp: paramiko.SFTPClient,
current_dir: str,
relative_dir: str,
artifacts: list[dict[str, Any]],
) -> None:
try:
entries = sftp.listdir_attr(current_dir)
except FileNotFoundError:
return
for entry in sorted(entries, key=lambda item: item.filename):
name = entry.filename
remote_path = posixpath.join(current_dir, name)
relative_path = posixpath.join(relative_dir, name) if relative_dir else name
mode = entry.st_mode
if mode is None:
mode = sftp.lstat(remote_path).st_mode
if mode is None:
raise RuntimeError(f"Unable to determine artifact entry type: {relative_path}")
if stat.S_ISLNK(mode):
raise RuntimeError(f"Artifact symlinks are not allowed: {relative_path}")
if stat.S_ISDIR(mode):
self._collect_artifacts_recursive(sftp, remote_path, relative_path, artifacts)
continue
if not stat.S_ISREG(mode):
raise RuntimeError(f"Unsupported artifact entry: {relative_path}")
if len(artifacts) >= self.max_artifacts:
raise RuntimeError(f"SSH execution produced more than {self.max_artifacts} artifacts.")
size = int(entry.st_size or 0)
if size > self.max_artifact_bytes:
raise RuntimeError(f"Artifact exceeds {self.max_artifact_bytes} bytes: {relative_path}")
ext = os.path.splitext(name)[1].lower()
if ext not in ALLOWED_ARTIFACT_EXTENSIONS:
raise RuntimeError(f"Unsupported artifact type: {relative_path}")
with sftp.file(remote_path, "rb") as artifact_file:
content = artifact_file.read()
artifacts.append(
{
"name": relative_path,
"content_b64": base64.b64encode(content).decode("ascii"),
"mime_type": mimetypes.guess_type(name)[0] or "application/octet-stream",
"size": size,
}
)
@staticmethod
def _normalize_language(language: str) -> str:
lang_lower = (language or "python").lower()
if lang_lower in {"python", "python3"}:
return "python"
if lang_lower in {"javascript", "nodejs"}:
return "nodejs"
return lang_lower
def _get_paramiko_module():
try:
import paramiko
except ImportError as exc:
raise SandboxProviderConfigError(
"paramiko is required for the SSH sandbox provider. Install the project dependencies to enable it."
) from exc
return paramiko

View File

@@ -8,7 +8,7 @@ dependencies = [
"fastapi>=0.115.12",
"httpx>=0.28.1",
"pydantic>=2.11.4",
"requests>=2.32.3",
"requests>=2.32.4",
"slowapi>=0.1.9",
"uvicorn>=0.34.2",
]

View File

@@ -0,0 +1,85 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import base64
import json
from typing import Any
RESULT_MARKER_PREFIX = "__RAGFLOW_RESULT__:"
def build_python_wrapper(code: str, args_json: str) -> str:
return f'''{code}
if __name__ == "__main__":
import base64
import json
result = main(**{args_json})
payload = json.dumps({{"present": True, "value": result, "type": "json"}}, ensure_ascii=False, separators=(",", ":"))
print("{RESULT_MARKER_PREFIX}" + base64.b64encode(payload.encode("utf-8")).decode("ascii"))
'''
def build_javascript_wrapper(code: str, args_json: str) -> str:
return f'''{code}
const __ragflowArgs = {args_json};
(async () => {{
const __ragflowMain = typeof main !== 'undefined' ? main : module.exports && module.exports.main;
if (typeof __ragflowMain !== 'function') {{
throw new Error('main() must be defined or exported.');
}}
const output = await Promise.resolve(__ragflowMain(__ragflowArgs));
if (typeof output === 'undefined') {{
throw new Error('main() must return a value. Use null for an empty result.');
}}
const payload = JSON.stringify({{ present: true, value: output, type: 'json' }});
if (typeof payload === 'undefined') {{
throw new Error('main() returned a non-JSON-serializable value.');
}}
console.log('{RESULT_MARKER_PREFIX}' + Buffer.from(payload, 'utf8').toString('base64'));
}})();
'''
def extract_structured_result(stdout: str) -> tuple[str, dict[str, Any]]:
if not stdout:
return "", {}
cleaned_lines: list[str] = []
structured_result: dict[str, Any] = {}
for line in str(stdout).splitlines():
if line.startswith(RESULT_MARKER_PREFIX):
payload_b64 = line[len(RESULT_MARKER_PREFIX) :].strip()
if not payload_b64:
cleaned_lines.append(line)
continue
try:
payload = base64.b64decode(payload_b64, validate=True).decode("utf-8")
structured_result = json.loads(payload)
except Exception:
cleaned_lines.append(line)
continue
cleaned_lines.append(line)
cleaned_stdout = "\n".join(cleaned_lines)
if stdout.endswith("\n") and cleaned_stdout and not cleaned_stdout.endswith("\n"):
cleaned_stdout += "\n"
return cleaned_stdout, structured_result

View File

@@ -1,6 +1,12 @@
FROM node:24.13-bookworm-slim
RUN npm config set registry https://registry.npmmirror.com
ARG NEED_MIRROR=1
RUN if [ "$NEED_MIRROR" = 1 ]; then \
npm config set registry https://registry.npmmirror.com; \
else \
npm config set registry https://registry.npmjs.org; \
fi
# RUN grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.ustc.edu.cn|g' && \
# apt-get update && \

View File

@@ -19,13 +19,13 @@
"license": "MIT"
},
"node_modules/axios": {
"version": "1.12.0",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.12.0.tgz",
"integrity": "sha512-oXTDccv8PcfjZmPGlWsPSwtOJCZ/b6W5jAMCNcfwJbCzDckwG0jrYJFaWH1yvivfCXjVzV/SPDEhMB3Q+DSurg==",
"version": "1.13.6",
"resolved": "https://registry.npmjs.org/axios/-/axios-1.13.6.tgz",
"integrity": "sha512-ChTCHMouEe2kn713WHbQGcuYrr6fXTBiu460OTwWrWob16g1bXn4vtz07Ope7ewMozJAnEquLk5lWQWtBig9DQ==",
"license": "MIT",
"dependencies": {
"follow-redirects": "^1.15.6",
"form-data": "^4.0.4",
"follow-redirects": "^1.15.11",
"form-data": "^4.0.5",
"proxy-from-env": "^1.1.0"
}
},
@@ -123,9 +123,9 @@
}
},
"node_modules/follow-redirects": {
"version": "1.15.9",
"resolved": "https://registry.npmmirror.com/follow-redirects/-/follow-redirects-1.15.9.tgz",
"integrity": "sha512-gew4GsXizNgdoRyqmyfMHyAmXsZDk6mHkSxZFCzW9gwlbtOW44CDtYavM+y+72qD/Vq2l550kMF52DT8fOLJqQ==",
"version": "1.15.11",
"resolved": "https://registry.npmjs.org/follow-redirects/-/follow-redirects-1.15.11.tgz",
"integrity": "sha512-deG2P0JfjrTxl50XGCDyfI97ZGVCxIpfKYmfyrQ54n5FO/0gfIES8C/Psl6kWVDolizcaaxZJnTS0QSMxvnsBQ==",
"funding": [
{
"type": "individual",
@@ -143,9 +143,9 @@
}
},
"node_modules/form-data": {
"version": "4.0.4",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.4.tgz",
"integrity": "sha512-KrGhL9Q4zjj0kiUt5OO4Mr/A/jlI2jDYs5eHBpYHPcBEVSiipAvn2Ko2HnPe20rmcuuvMHNdZFp+4IlGTMF0Ow==",
"version": "4.0.5",
"resolved": "https://registry.npmjs.org/form-data/-/form-data-4.0.5.tgz",
"integrity": "sha512-8RipRLol37bNs2bhoV67fiTEvdTrbMUYcFTiy3+wuuOnUog2QBHCZWXDRijWQfAkhBj2Uf5UnVaiWwA5vdd82w==",
"license": "MIT",
"dependencies": {
"asynckit": "^0.4.0",

View File

@@ -1,15 +1,27 @@
FROM python:3.11-slim-bookworm
ARG NEED_MIRROR=1
COPY --from=ghcr.io/astral-sh/uv:0.7.5 /uv /uvx /bin/
ENV UV_INDEX_URL=https://pypi.tuna.tsinghua.edu.cn/simple
ENV MPLBACKEND=Agg
ENV MPLCONFIGDIR=/tmp/matplotlib
ENV MATPLOTLIBRC=/usr/local/etc/matplotlibrc
COPY requirements.txt .
COPY matplotlibrc /usr/local/etc/matplotlibrc
RUN grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.tuna.tsinghua.edu.cn|g' && \
RUN if [ "$NEED_MIRROR" = 1 ]; then \
grep -rl 'deb.debian.org' /etc/apt/ | xargs sed -i 's|http[s]*://deb.debian.org|https://mirrors.tuna.tsinghua.edu.cn|g'; \
export UV_INDEX_URL="https://pypi.tuna.tsinghua.edu.cn/simple"; \
else \
export UV_INDEX_URL="https://pypi.org/simple"; \
fi; \
apt-get update && \
apt-get install -y curl gcc && \
uv pip install --system -r requirements.txt
apt-get install -y --no-install-recommends curl gcc && \
mkdir -p /tmp/matplotlib && \
uv pip install --system -r requirements.txt && \
rm -rf /var/lib/apt/lists/*
WORKDIR /workspace
CMD ["sleep", "infinity"]
CMD ["sleep", "infinity"]

View File

@@ -0,0 +1,11 @@
## RAGFlow sandbox matplotlib defaults
## Only overrides are listed; all other settings use matplotlib built-in defaults.
# Prefer CJK-capable fonts so Chinese / Japanese / Korean text renders correctly.
# matplotlib silently skips fonts that are not installed, falling back to the
# next entry in the list, so this is safe even without any CJK font package.
font.family: sans-serif
font.sans-serif: Noto Sans CJK SC, Noto Sans CJK TC, Noto Sans CJK JP, Noto Sans CJK KR, Source Han Sans SC, Source Han Sans CN, WenQuanYi Zen Hei, Microsoft YaHei, SimHei, PingFang SC, Heiti SC, STHeiti, Arial Unicode MS, DejaVu Sans, Bitstream Vera Sans, Computer Modern Sans Serif, Lucida Grande, Verdana, Geneva, Lucid, Arial, Helvetica, Avant Garde, sans-serif
# Use ASCII hyphen-minus for the minus sign so it renders correctly with any font.
axes.unicode_minus: False

View File

@@ -1,3 +1,4 @@
numpy
pandas
matplotlib
requests

View File

@@ -654,7 +654,7 @@ class AliyunCodeInterpreterProvider(SandboxProvider):
"type": "string",
"required": True,
"label": "Account ID",
"description": "Aliyun primary account ID (主账号ID), required for API calls"
"description": "Aliyun primary account ID, required for API calls"
},
"region": {
"type": "string",
@@ -1739,8 +1739,9 @@ def execute_code(
1. **Self-managed provider** ([self_managed.py:164](agent/sandbox/providers/self_managed.py:164)):
- Passes arguments via HTTP API: `"arguments": arguments or {}`
- executor_manager receives and passes to code via command line
- Runner script: `args = json.loads(sys.argv[1])` then `result = main(**args)`
- executor_manager writes `args.json` into the per-task workspace
- Runner script loads arguments from `args.json`
- Python runner calls `main(**args)` and JavaScript runner calls `main(args)`
2. **Aliyun Code Interpreter** ([aliyun_codeinterpreter.py:260-275](agent/sandbox/providers/aliyun_codeinterpreter.py:260-275)):
- Wraps user code to call `main(**arguments)` or `main()` if no arguments

View File

@@ -1,53 +1,53 @@
# Aliyun Code Interpreter Provider - 使用官方 SDK
# Aliyun Code Interpreter Provider - Using the Official SDK
## 重要变更
## Important Changes
### 官方资源
### Official Resources
- **Code Interpreter API**: https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter
- **官方 SDK**: https://github.com/Serverless-Devs/agentrun-sdk-python
- **SDK 文档**: https://docs.agent.run
- **Official SDK**: https://github.com/Serverless-Devs/agentrun-sdk-python
- **SDK Documentation**: https://docs.agent.run
## 使用官方 SDK 的优势
## Advantages of Using the Official SDK
从手动 HTTP 请求迁移到官方 SDK (`agentrun-sdk`) 有以下优势:
Migrating from manual HTTP requests to the official SDK (`agentrun-sdk`) offers the following benefits:
### 1. **自动签名认证**
- SDK 自动处理 Aliyun API 签名(无需手动实现 `Authorization` 头)
- 支持多种认证方式:AccessKeySTS Token
- 自动读取环境变量
### 1. **Automatic Signature Authentication**
- The SDK automatically handles Aliyun API signing (no need to manually implement `Authorization` headers)
- Supports multiple authentication methods: AccessKey, STS Token
- Automatically reads environment variables
### 2. **简化的 API**
### 2. **Simplified API**
```python
# 旧实现(手动 HTTP 请求)
# Old implementation (manual HTTP requests)
response = requests.post(
f"{DATA_ENDPOINT}/sandboxes/{sandbox_id}/execute",
headers={"X-Acs-Parent-Id": account_id},
json={"code": code, "language": "python"}
)
# 新实现(使用 SDK
# New implementation (using SDK)
sandbox = CodeInterpreterSandbox(template_name="python-sandbox", config=config)
result = sandbox.context.execute(code="print('hello')")
```
### 3. **更好的错误处理**
- 结构化的异常类型 (`ServerError`)
- 自动重试机制
- 详细的错误信息
### 3. **Better Error Handling**
- Structured exception types (`ServerError`)
- Automatic retry mechanism
- Detailed error messages
## 主要变更
## Key Changes
### 1. 文件重命名
### 1. File Renames
| 旧文件名 | 新文件名 | 说明 |
| Old Filename | New Filename | Description |
|---------|---------|------|
| `aliyun_opensandbox.py` | `aliyun_codeinterpreter.py` | 提供商实现 |
| `test_aliyun_provider.py` | `test_aliyun_codeinterpreter.py` | 单元测试 |
| `test_aliyun_integration.py` | `test_aliyun_codeinterpreter_integration.py` | 集成测试 |
| `aliyun_opensandbox.py` | `aliyun_codeinterpreter.py` | Provider implementation |
| `test_aliyun_provider.py` | `test_aliyun_codeinterpreter.py` | Unit tests |
| `test_aliyun_integration.py` | `test_aliyun_codeinterpreter_integration.py` | Integration tests |
### 2. 配置字段变更
### 2. Configuration Field Changes
#### 旧配置(OpenSandbox
#### Old Configuration (OpenSandbox)
```json
{
"access_key_id": "LTAI5t...",
@@ -57,59 +57,59 @@ result = sandbox.context.execute(code="print('hello')")
}
```
#### 新配置(Code Interpreter
#### New Configuration (Code Interpreter)
```json
{
"access_key_id": "LTAI5t...",
"access_key_secret": "...",
"account_id": "1234567890...", // 新增阿里云主账号ID必需
"account_id": "1234567890...", // New: Aliyun primary account ID (required)
"region": "cn-hangzhou",
"template_name": "python-sandbox", // 新增:沙箱模板名称
"timeout": 30 // 最大 30 秒(硬限制)
"template_name": "python-sandbox", // New: sandbox template name
"timeout": 30 // Max 30 seconds (hard limit)
}
```
### 3. 关键差异
### 3. Key Differences
| 特性 | OpenSandbox | Code Interpreter |
| Feature | OpenSandbox | Code Interpreter |
|------|-------------|-----------------|
| **API 端点** | `opensandbox.{region}.aliyuncs.com` | `agentrun.{region}.aliyuncs.com` (控制面) |
| **API 版本** | `2024-01-01` | `2025-09-10` |
| **认证** | 需要 AccessKey | 需要 AccessKey + 主账号ID |
| **请求头** | 标准签名 | 需要 `X-Acs-Parent-Id` |
| **超时限制** | 可配置 | **最大 30 **(硬限制) |
| **上下文** | 不支持 | 支持上下文(Jupyter kernel |
| **API Endpoint** | `opensandbox.{region}.aliyuncs.com` | `agentrun.{region}.aliyuncs.com` (control plane) |
| **API Version** | `2024-01-01` | `2025-09-10` |
| **Authentication** | AccessKey required | AccessKey + primary account ID required |
| **Request Headers** | Standard signature | Requires `X-Acs-Parent-Id` header |
| **Timeout Limit** | Configurable | **Max 30 seconds** (hard limit) |
| **Context** | Not supported | Supports context (Jupyter kernel) |
### 4. API 调用方式变更
### 4. API Call Changes
#### 旧实现(假设的 OpenSandbox
#### Old Implementation (assumed OpenSandbox)
```python
# 单一端点
# Single endpoint
API_ENDPOINT = "https://opensandbox.cn-hangzhou.aliyuncs.com"
# 简单的请求/响应
# Simple request/response
response = requests.post(
f"{API_ENDPOINT}/execute",
json={"code": "print('hello')", "language": "python"}
)
```
#### 新实现(Code Interpreter
#### New Implementation (Code Interpreter)
```python
# 控制面 API - 管理沙箱生命周期
# Control plane API - manage sandbox lifecycle
CONTROL_ENDPOINT = "https://agentrun.cn-hangzhou.aliyuncs.com/2025-09-10"
# 数据面 API - 执行代码
# Data plane API - execute code
DATA_ENDPOINT = "https://{account_id}.agentrun-data.cn-hangzhou.aliyuncs.com"
# 创建沙箱(控制面)
# Create sandbox (control plane)
response = requests.post(
f"{CONTROL_ENDPOINT}/sandboxes",
headers={"X-Acs-Parent-Id": account_id},
json={"templateName": "python-sandbox"}
)
# 执行代码(数据面)
# Execute code (data plane)
response = requests.post(
f"{DATA_ENDPOINT}/sandboxes/{sandbox_id}/execute",
headers={"X-Acs-Parent-Id": account_id},
@@ -117,13 +117,13 @@ response = requests.post(
)
```
### 5. 迁移步骤
### 5. Migration Steps
#### 步骤 1: 更新配置
#### Step 1: Update Configuration
如果您之前使用的是 `aliyun_opensandbox`
If you were previously using `aliyun_opensandbox`:
**旧配置**:
**Old configuration**:
```json
{
"name": "sandbox.provider_type",
@@ -131,7 +131,7 @@ response = requests.post(
}
```
**新配置**:
**New configuration**:
```json
{
"name": "sandbox.provider_type",
@@ -139,123 +139,123 @@ response = requests.post(
}
```
#### 步骤 2: 添加必需的 account_id
#### Step 2: Add the Required account_id
在 Aliyun 控制台右上角点击头像,获取主账号 ID
1. 登录 [阿里云控制台](https://ram.console.aliyun.com/manage/ak)
2. 点击右上角头像
3. 复制主账号 ID16 位数字)
Get your primary account ID from the Aliyun console:
1. Log in to the [Aliyun Console](https://ram.console.aliyun.com/manage/ak)
2. Click on your avatar in the top-right corner
3. Copy the primary account ID (16-digit number)
#### 步骤 3: 更新环境变量
#### Step 3: Update Environment Variables
```bash
# 新增必需的环境变量
# New required environment variable
export ALIYUN_ACCOUNT_ID="1234567890123456"
# 其他环境变量保持不变
# Other environment variables remain unchanged
export ALIYUN_ACCESS_KEY_ID="LTAI5t..."
export ALIYUN_ACCESS_KEY_SECRET="..."
export ALIYUN_REGION="cn-hangzhou"
```
#### 步骤 4: 运行测试
#### Step 4: Run Tests
```bash
# 单元测试(不需要真实凭据)
# Unit tests (no real credentials required)
pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v
# 集成测试(需要真实凭据)
# Integration tests (real credentials required)
pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v -m integration
```
## 文件变更清单
## File Change Checklist
### ✅ 已完成
### ✅ Completed
- [x] 创建 `aliyun_codeinterpreter.py` - 新的提供商实现
- [x] 更新 `sandbox_spec.md` - 规范文档
- [x] 更新 `admin/services.py` - 服务管理器
- [x] 更新 `providers/__init__.py` - 包导出
- [x] 创建 `test_aliyun_codeinterpreter.py` - 单元测试
- [x] 创建 `test_aliyun_codeinterpreter_integration.py` - 集成测试
- [x] Created `aliyun_codeinterpreter.py` - new provider implementation
- [x] Updated `sandbox_spec.md` - specification document
- [x] Updated `admin/services.py` - service manager
- [x] Updated `providers/__init__.py` - package exports
- [x] Created `test_aliyun_codeinterpreter.py` - unit tests
- [x] Created `test_aliyun_codeinterpreter_integration.py` - integration tests
### 📝 可选清理
### 📝 Optional Cleanup
如果您想删除旧的 OpenSandbox 实现:
If you want to remove the old OpenSandbox implementation:
```bash
# 删除旧文件(可选)
# Remove old files (optional)
rm agent/sandbox/providers/aliyun_opensandbox.py
rm agent/sandbox/tests/test_aliyun_provider.py
rm agent/sandbox/tests/test_aliyun_integration.py
```
**注意**: 保留旧文件不会影响新功能,只是代码冗余。
**Note**: Keeping the old files does not affect the new functionality; it just results in redundant code.
## API 参考
## API Reference
### 控制面 API沙箱管理
### Control Plane API (Sandbox Management)
| 端点 | 方法 | 说明 |
| Endpoint | Method | Description |
|------|------|------|
| `/sandboxes` | POST | 创建沙箱实例 |
| `/sandboxes/{id}/stop` | POST | 停止实例 |
| `/sandboxes/{id}` | DELETE | 删除实例 |
| `/templates` | GET | 列出模板 |
| `/sandboxes` | POST | Create a sandbox instance |
| `/sandboxes/{id}/stop` | POST | Stop an instance |
| `/sandboxes/{id}` | DELETE | Delete an instance |
| `/templates` | GET | List templates |
### 数据面 API代码执行
### Data Plane API (Code Execution)
| 端点 | 方法 | 说明 |
| Endpoint | Method | Description |
|------|------|------|
| `/sandboxes/{id}/execute` | POST | 执行代码(简化版) |
| `/sandboxes/{id}/contexts` | POST | 创建上下文 |
| `/sandboxes/{id}/contexts/{ctx_id}/execute` | POST | 在上下文中执行 |
| `/sandboxes/{id}/health` | GET | 健康检查 |
| `/sandboxes/{id}/files` | GET/POST | 文件读写 |
| `/sandboxes/{id}/processes/cmd` | POST | 执行 Shell 命令 |
| `/sandboxes/{id}/execute` | POST | Execute code (simplified) |
| `/sandboxes/{id}/contexts` | POST | Create a context |
| `/sandboxes/{id}/contexts/{ctx_id}/execute` | POST | Execute within a context |
| `/sandboxes/{id}/health` | GET | Health check |
| `/sandboxes/{id}/files` | GET/POST | File read/write |
| `/sandboxes/{id}/processes/cmd` | POST | Execute shell command |
## 常见问题
## FAQ
### Q: 为什么要添加 account_id
### Q: Why is account_id required?
**A**: Code Interpreter API 需要在请求头中提供 `X-Acs-Parent-Id`阿里云主账号ID进行身份验证。这是 Aliyun Code Interpreter API 的必需参数。
**A**: The Code Interpreter API requires the `X-Acs-Parent-Id` (Aliyun primary account ID) header for authentication. This is a required parameter for the Aliyun Code Interpreter API.
### Q: 30 秒超时限制可以绕过吗?
### Q: Can the 30-second timeout limit be bypassed?
**A**: 不可以。这是 Aliyun Code Interpreter 的**硬限制**,无法通过配置或请求参数绕过。如果代码执行时间超过 30 秒,请考虑:
1. 优化代码逻辑
2. 分批处理数据
3. 使用上下文保持状态
**A**: No. This is a **hard limit** of Aliyun Code Interpreter and cannot be bypassed through configuration or request parameters. If your code execution exceeds 30 seconds, consider:
1. Optimizing the code logic
2. Processing data in batches
3. Using contexts to maintain state
### Q: 旧的 OpenSandbox 配置还能用吗?
### Q: Can the old OpenSandbox configuration still be used?
**A**: 不能。OpenSandbox Code Interpreter 是两个不同的服务API 不兼容。必须迁移到新的配置格式。
**A**: No. OpenSandbox and Code Interpreter are two different services with incompatible APIs. You must migrate to the new configuration format.
### Q: 如何获取阿里云主账号 ID
### Q: How do I get the Aliyun primary account ID?
**A**:
1. 登录阿里云控制台
2. 点击右上角的头像
3. 在弹出的信息中可以看到"主账号ID"
1. Log in to the Aliyun console
2. Click on your avatar in the top-right corner
3. The primary account ID will be displayed in the popup
### Q: 迁移后会影响现有功能吗?
### Q: Will the migration affect existing functionality?
**A**:
- **自我管理提供商(self_managed**: 不受影响
- **E2B 提供商**: 不受影响
- **Aliyun 提供商**: 需要更新配置并重新测试
- **Self-managed provider (self_managed)**: Not affected
- **E2B provider**: Not affected
- **Aliyun provider**: Configuration update and re-testing required
## 相关文档
## Related Documentation
- [官方文档](https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter)
- [sandbox 规范](../docs/develop/sandbox_spec.md)
- [测试指南](./README.md)
- [快速开始](./QUICKSTART.md)
- [Official Documentation](https://help.aliyun.com/zh/functioncompute/fc/sandbox-sandbox-code-interepreter)
- [Sandbox Specification](../docs/develop/sandbox_spec.md)
- [Testing Guide](./README.md)
- [Quick Start](./QUICKSTART.md)
## 技术支持
## Support
如有问题,请:
1. 查看官方文档
2. 检查配置是否正确
3. 查看测试输出中的错误信息
4. 联系 RAGFlow 团队
If you have any issues:
1. Review the official documentation
2. Verify the configuration is correct
3. Check the error messages in the test output
4. Contact the RAGFlow team

View File

@@ -1,45 +1,45 @@
# Aliyun OpenSandbox Provider - 快速测试指南
# Aliyun OpenSandbox Provider - Quick Test Guide
## 测试说明
## Test Overview
### 1. 单元测试(不需要真实凭据)
### 1. Unit Tests (No Credentials Required)
单元测试使用 mock**不需要**真实的 Aliyun 凭据,可以随时运行。
Unit tests use mocks and do **not** require real Aliyun credentials; they can be run at any time.
```bash
# 运行 Aliyun 提供商的单元测试
# Run unit tests for the Aliyun provider
pytest agent/sandbox/tests/test_aliyun_provider.py -v
# 预期输出:
# Expected output:
# test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_provider_initialization PASSED
# test_aliyun_provider.py::TestAliyunOpenSandboxProvider::test_initialize_success PASSED
# ...
# ========================= 48 passed in 2.34s ==========================
```
### 2. 集成测试(需要真实凭据)
### 2. Integration Tests (Real Credentials Required)
集成测试会调用真实的 Aliyun API需要配置凭据。
Integration tests call the real Aliyun API and require credentials to be configured.
#### 步骤 1: 配置环境变量
#### Step 1: Configure Environment Variables
```bash
export ALIYUN_ACCESS_KEY_ID="LTAI5t..." # 替换为真实的 Access Key ID
export ALIYUN_ACCESS_KEY_SECRET="..." # 替换为真实的 Access Key Secret
export ALIYUN_REGION="cn-hangzhou" # 可选,默认为 cn-hangzhou
export ALIYUN_ACCESS_KEY_ID="LTAI5t..." # Replace with your real Access Key ID
export ALIYUN_ACCESS_KEY_SECRET="..." # Replace with your real Access Key Secret
export ALIYUN_REGION="cn-hangzhou" # Optional, defaults to cn-hangzhou
```
#### 步骤 2: 运行集成测试
#### Step 2: Run Integration Tests
```bash
# 运行所有集成测试
# Run all integration tests
pytest agent/sandbox/tests/test_aliyun_integration.py -v -m integration
# 运行特定测试
# Run a specific test
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check -v
```
#### 步骤 3: 预期输出
#### Step 3: Expected Output
```
test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_initialize_provider PASSED
@@ -49,130 +49,130 @@ test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_pytho
========================== 10 passed in 15.67s ==========================
```
### 3. 测试场景
### 3. Test Scenarios
#### 基础功能测试
#### Basic Functionality Tests
```bash
# 健康检查
# Health check
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_health_check -v
# 创建实例
# Create instance
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_create_python_instance -v
# 执行代码
# Execute code
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code -v
# 销毁实例
# Destroy instance
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_destroy_instance -v
```
#### 错误处理测试
#### Error Handling Tests
```bash
# 代码执行错误
# Code execution error
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code_with_error -v
# 超时处理
# Timeout handling
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunOpenSandboxIntegration::test_execute_python_code_timeout -v
```
#### 真实场景测试
#### Real-World Scenario Tests
```bash
# 数据处理工作流
# Data processing workflow
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_data_processing_workflow -v
# 字符串操作
# String manipulation
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_string_manipulation -v
# 多次执行
# Multiple executions
pytest agent/sandbox/tests/test_aliyun_integration.py::TestAliyunRealWorldScenarios::test_multiple_executions_same_instance -v
```
## 常见问题
## FAQ
### Q: 没有凭据怎么办?
### Q: What if I don't have credentials?
**A:** 运行单元测试即可,不需要真实凭据:
**A:** Just run the unit tests — no real credentials needed:
```bash
pytest agent/sandbox/tests/test_aliyun_provider.py -v
```
### Q: 如何跳过集成测试?
### Q: How do I skip integration tests?
**A:** 使用 pytest 标记跳过:
**A:** Use pytest markers to skip them:
```bash
# 只运行单元测试,跳过集成测试
# Run only unit tests, skip integration tests
pytest agent/sandbox/tests/ -v -m "not integration"
```
### Q: 集成测试失败怎么办?
### Q: What should I do if integration tests fail?
**A:** 检查以下几点:
**A:** Check the following:
1. **凭据是否正确**
1. **Are the credentials correct?**
```bash
echo $ALIYUN_ACCESS_KEY_ID
echo $ALIYUN_ACCESS_KEY_SECRET
```
2. **网络连接是否正常**
2. **Is the network connection working?**
```bash
curl -I https://opensandbox.cn-hangzhou.aliyuncs.com
```
3. **是否有 OpenSandbox 服务权限**
- 登录阿里云控制台
- 检查是否已开通 OpenSandbox 服务
- 检查 AccessKey 权限
3. **Do you have OpenSandbox service permissions?**
- Log in to the Aliyun console
- Check if the OpenSandbox service is enabled
- Verify AccessKey permissions
4. **查看详细错误信息**
4. **View detailed error messages:**
```bash
pytest agent/sandbox/tests/test_aliyun_integration.py -v -s
```
### Q: 测试超时怎么办?
### Q: What should I do if tests time out?
**A:** 增加超时时间或检查网络:
**A:** Increase the timeout or check network connectivity:
```bash
# 使用更长的超时
# Use a longer timeout
pytest agent/sandbox/tests/test_aliyun_integration.py -v --timeout=60
```
## 测试命令速查表
## Quick Reference: Test Commands
| 命令 | 说明 | 需要凭据 |
| Command | Description | Credentials Required |
|------|------|---------|
| `pytest agent/sandbox/tests/test_aliyun_provider.py -v` | 单元测试 | ❌ |
| `pytest agent/sandbox/tests/test_aliyun_integration.py -v` | 集成测试 | ✅ |
| `pytest agent/sandbox/tests/ -v -m "not integration"` | 仅单元测试 | ❌ |
| `pytest agent/sandbox/tests/ -v -m integration` | 仅集成测试 | ✅ |
| `pytest agent/sandbox/tests/ -v` | 所有测试 | 部分需要 |
| `pytest agent/sandbox/tests/test_aliyun_provider.py -v` | Unit tests | ❌ |
| `pytest agent/sandbox/tests/test_aliyun_integration.py -v` | Integration tests | ✅ |
| `pytest agent/sandbox/tests/ -v -m "not integration"` | Unit tests only | ❌ |
| `pytest agent/sandbox/tests/ -v -m integration` | Integration tests only | ✅ |
| `pytest agent/sandbox/tests/ -v` | All tests | Partially required |
## 获取 Aliyun 凭据
## Getting Aliyun Credentials
1. 访问 [阿里云控制台](https://ram.console.aliyun.com/manage/ak)
2. 创建 AccessKey
3. 保存 AccessKey ID AccessKey Secret
4. 设置环境变量
1. Visit the [Aliyun Console](https://ram.console.aliyun.com/manage/ak)
2. Create an AccessKey
3. Save your AccessKey ID and AccessKey Secret
4. Set the environment variables
⚠️ **安全提示:**
- 不要在代码中硬编码凭据
- 使用环境变量或配置文件
- 定期轮换 AccessKey
- 限制 AccessKey 权限
⚠️ **Security Tips:**
- Do not hardcode credentials in your code
- Use environment variables or configuration files
- Rotate AccessKeys regularly
- Restrict AccessKey permissions
## 下一步
## Next Steps
1.**运行单元测试** - 验证代码逻辑
2. 🔧 **配置凭据** - 设置环境变量
3. 🚀 **运行集成测试** - 测试真实 API
4. 📊 **查看结果** - 确保所有测试通过
5. 🎯 **集成到系统** - 使用 admin API 配置提供商
1.**Run unit tests** - Verify code logic
2. 🔧 **Configure credentials** - Set environment variables
3. 🚀 **Run integration tests** - Test the real API
4. 📊 **Review results** - Ensure all tests pass
5. 🎯 **Integrate into your system** - Configure the provider via the admin API
## 需要帮助?
## Need Help?
- 查看 [完整文档](README.md)
- 检查 [sandbox 规范](../../../../../docs/develop/sandbox_spec.md)
- 联系 RAGFlow 团队
- See the [full documentation](README.md)
- Check the [sandbox specification](../../../../../docs/develop/sandbox_spec.md)
- Contact the RAGFlow team

View File

@@ -101,13 +101,15 @@ class TestAliyunCodeInterpreterProvider:
assert provider.region == "cn-hangzhou"
assert provider.template_name == ""
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
def test_create_instance_python(self, mock_sandbox_class):
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Template")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_create_instance_python(self, mock_sandbox_class, mock_template):
"""Test creating a Python instance."""
# Mock successful instance creation
mock_sandbox = MagicMock()
mock_sandbox.sandbox_id = "01JCED8Z9Y6XQVK8M2NRST5WXY"
mock_sandbox_class.return_value = mock_sandbox
mock_sandbox_class.create.return_value = mock_sandbox
mock_template.get_by_name.return_value = MagicMock()
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
@@ -119,12 +121,14 @@ class TestAliyunCodeInterpreterProvider:
assert instance.status == "READY"
assert instance.metadata["language"] == "python"
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
def test_create_instance_javascript(self, mock_sandbox_class):
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Template")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_create_instance_javascript(self, mock_sandbox_class, mock_template):
"""Test creating a JavaScript instance."""
mock_sandbox = MagicMock()
mock_sandbox.sandbox_id = "01JCED8Z9Y6XQVK8M2NRST5WXY"
mock_sandbox_class.return_value = mock_sandbox
mock_sandbox_class.create.return_value = mock_sandbox
mock_template.get_by_name.return_value = MagicMock()
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
@@ -141,7 +145,7 @@ class TestAliyunCodeInterpreterProvider:
with pytest.raises(RuntimeError, match="Provider not initialized"):
provider.create_instance("python")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_execute_code_success(self, mock_sandbox_class):
"""Test successful code execution."""
# Mock sandbox instance
@@ -150,7 +154,7 @@ class TestAliyunCodeInterpreterProvider:
"results": [{"type": "stdout", "text": "Hello, World!"}, {"type": "result", "text": "None"}, {"type": "endOfExecution", "status": "ok"}],
"contextId": "kernel-12345-67890",
}
mock_sandbox_class.return_value = mock_sandbox
mock_sandbox_class.connect.return_value = mock_sandbox
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
@@ -163,14 +167,14 @@ class TestAliyunCodeInterpreterProvider:
assert result.exit_code == 0
assert result.execution_time > 0
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_execute_code_timeout(self, mock_sandbox_class):
"""Test code execution timeout."""
from agentrun.utils.exception import ServerError
mock_sandbox = MagicMock()
mock_sandbox.context.execute.side_effect = ServerError(408, "Request timeout")
mock_sandbox_class.return_value = mock_sandbox
mock_sandbox_class.connect.return_value = mock_sandbox
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
@@ -179,14 +183,14 @@ class TestAliyunCodeInterpreterProvider:
with pytest.raises(TimeoutError, match="Execution timed out"):
provider.execute_code(instance_id="01JCED8Z9Y6XQVK8M2NRST5WXY", code="while True: pass", language="python", timeout=5)
@patch("agent.sandbox.providers.aliyun_codeinterpreter.CodeInterpreterSandbox")
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_execute_code_with_error(self, mock_sandbox_class):
"""Test code execution with error."""
mock_sandbox = MagicMock()
mock_sandbox.context.execute.return_value = {
"results": [{"type": "stderr", "text": "Traceback..."}, {"type": "error", "text": "NameError: name 'x' is not defined"}, {"type": "endOfExecution", "status": "error"}]
}
mock_sandbox_class.return_value = mock_sandbox
mock_sandbox_class.connect.return_value = mock_sandbox
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
@@ -197,6 +201,34 @@ class TestAliyunCodeInterpreterProvider:
assert result.exit_code != 0
assert len(result.stderr) > 0
@patch("agent.sandbox.providers.aliyun_codeinterpreter.Sandbox")
def test_execute_code_uses_structured_result_marker_for_async_javascript(self, mock_sandbox_class):
"""Test JavaScript wrapper uses the structured result marker and awaits async main."""
mock_sandbox = MagicMock()
mock_sandbox.context.execute.return_value = {
"results": [{"type": "stdout", "text": "__RAGFLOW_RESULT__:eyJwcmVzZW50Ijp0cnVlLCJ2YWx1ZSI6eyJhIjoiYiJ9LCJ0eXBlIjoianNvbiJ9"}],
"contextId": "kernel-12345-67890",
}
mock_sandbox_class.connect.return_value = mock_sandbox
provider = AliyunCodeInterpreterProvider()
provider._initialized = True
provider._config = MagicMock()
result = provider.execute_code(
instance_id="01JCED8Z9Y6XQVK8M2NRST5WXY",
code="async function main(args) { return { a: 'b' }; }",
language="javascript",
timeout=10,
)
wrapped_code = mock_sandbox.context.execute.call_args.kwargs["code"]
assert "__RAGFLOW_RESULT__:" in wrapped_code
assert "await Promise.resolve(main(" in wrapped_code
assert result.metadata["result_present"] is True
assert result.metadata["result_value"] == {"a": "b"}
assert result.metadata["result_type"] == "json"
def test_get_supported_languages(self):
"""Test getting supported languages."""
provider = AliyunCodeInterpreterProvider()

View File

@@ -22,7 +22,7 @@ To run these tests, set the following environment variables:
export AGENTRUN_ACCESS_KEY_ID="LTAI5t..."
export AGENTRUN_ACCESS_KEY_SECRET="..."
export AGENTRUN_ACCOUNT_ID="1234567890..." # Aliyun primary account ID (主账号ID)
export AGENTRUN_ACCOUNT_ID="1234567890..." # Aliyun primary account ID
export AGENTRUN_REGION="cn-hangzhou" # Note: AGENTRUN_REGION (SDK will read this)
Then run:

View File

@@ -254,6 +254,41 @@ class TestSelfManagedProvider:
assert result.metadata["status"] == "success"
assert result.metadata["instance_id"] == "test-123"
@patch('requests.post')
def test_execute_code_maps_structured_result_into_metadata(self, mock_post):
"""Test successful code execution with structured result envelope."""
mock_response = Mock()
mock_response.status_code = 200
mock_response.json.return_value = {
"status": "success",
"stdout": "debug line\n",
"stderr": "",
"exit_code": 0,
"time_used_ms": 100.0,
"memory_used_kb": 1024.0,
"result": {
"present": True,
"value": {"items": ["a", "b"]},
"type": "json",
},
}
mock_post.return_value = mock_response
provider = SelfManagedProvider()
provider._initialized = True
result = provider.execute_code(
instance_id="test-123",
code="def main(): return {'items': ['a', 'b']}",
language="python",
timeout=10
)
assert result.stdout == "debug line\n"
assert result.metadata["result_present"] is True
assert result.metadata["result_value"] == {"items": ["a", "b"]}
assert result.metadata["result_type"] == "json"
@patch('requests.post')
def test_execute_code_timeout(self, mock_post):
"""Test code execution timeout."""

View File

@@ -0,0 +1,109 @@
#
# Copyright 2026 The InfiniFlow Authors. All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
import sys
from pathlib import Path
EXECUTOR_MANAGER_ROOT = Path(__file__).resolve().parents[1] / "executor_manager"
if str(EXECUTOR_MANAGER_ROOT) not in sys.path:
sys.path.insert(0, str(EXECUTOR_MANAGER_ROOT))
from models.enums import SupportLanguage # noqa: E402
from services.security import analyze_code_security # noqa: E402
def test_javascript_child_process_is_rejected():
is_safe, issues = analyze_code_security(
"const cp = require('child_process'); async function main() { return 'ok'; }",
SupportLanguage.NODEJS,
)
assert is_safe is False
assert any("child_process" in issue for issue, _ in issues)
def test_javascript_eval_is_rejected():
is_safe, issues = analyze_code_security(
"async function main() { return eval('1+1'); }",
SupportLanguage.NODEJS,
)
assert is_safe is False
assert any("eval" in issue.lower() for issue, _ in issues)
def test_javascript_child_process_template_literal_is_rejected():
"""Template literal backticks bypass single/double-quote regex patterns."""
is_safe, issues = analyze_code_security(
"const cp = require(`child_process`); async function main() { return 'ok'; }",
SupportLanguage.NODEJS,
)
assert is_safe is False
assert any("child_process" in issue for issue, _ in issues)
def test_javascript_fs_template_literal_is_rejected():
is_safe, issues = analyze_code_security(
"const fs = require(`fs`); async function main() { return fs.readFileSync('/etc/passwd', 'utf8'); }",
SupportLanguage.NODEJS,
)
assert is_safe is False
assert any("fs" in issue for issue, _ in issues)
def test_python_builtins_import_is_rejected():
"""builtins module gives access to eval/exec and must be blocked."""
is_safe, issues = analyze_code_security(
"import builtins\ndef main():\n builtins.eval('1+1')",
SupportLanguage.PYTHON,
)
assert is_safe is False
# Pin the specific reason: rejection must come from the new ``builtins``
# entry in ``DANGEROUS_IMPORTS``, not from some unrelated parse error.
assert any("builtins" in issue for issue, _ in issues), (
f"expected an issue mentioning 'builtins', got {issues!r}"
)
def test_python_attribute_eval_call_is_rejected():
"""Attribute-style dangerous calls (builtins.eval) must be caught."""
is_safe, issues = analyze_code_security(
"import builtins\ndef main():\n builtins.exec('import os')",
SupportLanguage.PYTHON,
)
assert is_safe is False
# Pin the specific reason: rejection must come from the new
# ``ast.Attribute`` branch in ``visit_Call`` flagging the ``exec`` call,
# not from the ``import builtins`` line above. We assert ``exec`` is in at
# least one finding so the test fails if visit_Call's attribute branch is
# ever reverted.
assert any("exec" in issue for issue, _ in issues), (
f"expected an issue mentioning 'exec', got {issues!r}"
)
def test_javascript_safe_code_still_passes():
is_safe, issues = analyze_code_security(
"async function main(args) { return { answer: args.value ?? null }; }",
SupportLanguage.NODEJS,
)
assert is_safe is True
assert issues == []

16
agent/sandbox/uv.lock generated
View File

@@ -1,6 +1,6 @@
version = 1
revision = 3
requires-python = ">=3.10"
requires-python = ">=3.12, <3.15"
[[package]]
name = "annotated-doc"
@@ -161,7 +161,7 @@ requires-dist = [
{ name = "fastapi", specifier = ">=0.115.12" },
{ name = "httpx", specifier = ">=0.28.1" },
{ name = "pydantic", specifier = ">=2.11.4" },
{ name = "requests", specifier = ">=2.32.3" },
{ name = "requests", specifier = ">=2.32.4" },
{ name = "slowapi", specifier = ">=0.1.9" },
{ name = "uvicorn", specifier = ">=0.34.2" },
]
@@ -313,7 +313,7 @@ wheels = [
[[package]]
name = "requests"
version = "2.32.3"
version = "2.32.5"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
dependencies = [
{ name = "certifi" },
@@ -321,9 +321,9 @@ dependencies = [
{ name = "idna" },
{ name = "urllib3" },
]
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/63/70/2bf7780ad2d390a8d301ad0b550f1581eadbd9a20f896afe06353c2a2913/requests-2.32.3.tar.gz", hash = "sha256:55365417734eb18255590a9ff9eb97e9e1da868d4ccd6402399eaf68af20a760", size = 131218, upload-time = "2024-05-29T15:37:49.536Z" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c9/74/b3ff8e6c8446842c3f5c837e9c3dfcfe2018ea6ecef224c710c85ef728f4/requests-2.32.5.tar.gz", hash = "sha256:dbba0bac56e100853db0ea71b82b4dfd5fe2bf6d3754a8893c3af500cec7d7cf", size = 134517, upload-time = "2025-08-18T20:46:02.573Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/f9/9b/335f9764261e915ed497fcdeb11df5dfd6f7bf257d4a6a2a686d80da4d54/requests-2.32.3-py3-none-any.whl", hash = "sha256:70761cfe03c773ceb22aa2f671b4757976145175cdfca038c02654d061d6dcc6", size = 64928, upload-time = "2024-05-29T15:37:47.027Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/1e/db/4254e3eabe8020b458f1a747140d32277ec7a271daf1d235b70dc0b4e6e3/requests-2.32.5-py3-none-any.whl", hash = "sha256:2462f94637a34fd532264295e186976db0f5d453d1cdd31473c85a6a161affb6", size = 64738, upload-time = "2025-08-18T20:46:00.542Z" },
]
[[package]]
@@ -383,11 +383,11 @@ wheels = [
[[package]]
name = "urllib3"
version = "2.6.3"
version = "2.7.0"
source = { registry = "https://pypi.tuna.tsinghua.edu.cn/simple" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/c7/24/5f1b3bdffd70275f6661c76461e25f024d5a38a46f04aaca912426a2b1d3/urllib3-2.6.3.tar.gz", hash = "sha256:1b62b6884944a57dbe321509ab94fd4d3b307075e0c2eae991ac71ee15ad38ed", size = 435556, upload-time = "2026-01-07T16:24:43.925Z" }
sdist = { url = "https://pypi.tuna.tsinghua.edu.cn/packages/53/0c/06f8b233b8fd13b9e5ee11424ef85419ba0d8ba0b3138bf360be2ff56953/urllib3-2.7.0.tar.gz", hash = "sha256:231e0ec3b63ceb14667c67be60f2f2c40a518cb38b03af60abc813da26505f4c", size = 433602, upload-time = "2026-05-07T16:13:18.596Z" }
wheels = [
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/39/08/aaaad47bc4e9dc8c725e68f9d04865dbcb2052843ff09c97b08904852d84/urllib3-2.6.3-py3-none-any.whl", hash = "sha256:bf272323e553dfb2e87d9bfd225ca7b0f467b919d7bbd355436d3fd37cb0acd4", size = 131584, upload-time = "2026-01-07T16:24:42.685Z" },
{ url = "https://pypi.tuna.tsinghua.edu.cn/packages/7f/3e/5db95bcf282c52709639744ca2a8b149baccf648e39c8cc87553df9eae0c/urllib3-2.7.0-py3-none-any.whl", hash = "sha256:9fb4c81ebbb1ce9531cce37674bbc6f1360472bc18ca9a553ede278ef7276897", size = 131087, upload-time = "2026-05-07T16:13:17.151Z" },
]
[[package]]

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

View File

@@ -94,7 +94,7 @@
"type": "integer"
},
"item": {
"type": "unkown"
"type": "unknown"
}
}
}
@@ -252,7 +252,7 @@
"type": "integer"
},
"item": {
"type": "unkown"
"type": "unknown"
}
}
},

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

File diff suppressed because one or more lines are too long

Some files were not shown because too many files have changed in this diff Show More