Commit Graph

601 Commits

Author SHA1 Message Date
KeJun
cb78ce0a7b feat: support rss datasource (#13721)
### What problem does this PR solve?

Supporting public RSS/Atom feed URLs as data sources for RagFlow.

link https://github.com/infiniflow/ragflow/issues/12313

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-27 22:58:44 +08:00
Sank
992a15146d Add translate autoMetadata (#13807)
### What problem does this PR solve?

add translate autoMetadata in Russia

### Type of change

- [x] Other
2026-03-27 09:31:20 +08:00
Baki Burak Öğün
1319a25416 feat: complete Turkish localization (#13749)
## Summary
Complete and improve the existing Turkish (tr.ts) localization to fully
match the English (en.ts) reference file.

## Changes
- **Translate 6 English model tips** in the setting section
(chatModelTip, embeddingModelTip, img2txtModelTip, sequence2txtModelTip,
rerankModelTip, ttsModelTip) to Turkish
- **Expand all 13 truncated parser HTML descriptions** (book, laws,
manual, naive, paper, presentation, qa, resume, table, picture, one,
knowledgeGraph, tag) to match the full en.ts structure
- **Expand shortened tooltips** across knowledgeDetails,
knowledgeConfiguration, chat, and setting sections (~40+ tooltips
expanded)
- **Add missing translation details** for data source connectors
(SeaFile, Jira, Gmail, Moodle, Dropbox, Google Drive, etc.)

## Impact
- 182 insertions, 71 deletions in web/src/locales/tr.ts
- No structural changes, only translation content improvements
- All application terminology maintained consistently

Co-authored-by: bakiburakogun <bakiburakogun@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-24 18:58:58 +08:00
writinwaters
b5e0b37d69 Refact: Renamed 'Agent flow' to 'Workflow' (#13705)
### What problem does this PR solve?

'Agent flow' rebranded.

### Type of change

- [x] Refactoring
2026-03-19 20:17:25 +08:00
Mustafa YILDIZ
e4d8cdaff3 feat: add Turkish language support (#13670)
### What problem does this PR solve?
RAGFlow had no Turkish language support. This PR adds Turkish (tr)
locale translations to the UI.

### Type of change
- [x] New Feature (non-breaking change which adds functionality)

### What problem does this PR solve?

Co-authored-by: Mustafa YILDIZ <mustafa.yildiz@cilek.com>
2026-03-18 21:09:32 +08:00
balibabu
6cae364ac2 Feat: Export Agent Logs. (#13658)
### What problem does this PR solve?
Feat: Export Agent Logs.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-17 18:51:26 +08:00
balibabu
fa48ffe5de Feat: Translate embedded dialog text. (#13623)
### What problem does this PR solve?

Feat: Translate embedded dialog text.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-16 15:43:13 +08:00
Yingfeng
b686a60713 Switch from demo.ragflow.io to cloud.ragflow.io (#13624)
### What problem does this PR solve?

Switch from demo.ragflow.io to cloud.ragflow.io

### Type of change

- [x] Documentation Update
2026-03-16 14:44:39 +08:00
balibabu
cb49cd30c4 Feat: Add the user_id field to the agent log table and the embedded page. (#13596)
### What problem does this PR solve?

Feat: Add the `user_id` field to the agent log table and the embedded
page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-13 19:06:18 +08:00
balibabu
717f1f1362 Feat: Modify the style of the release confirmation box. (#13542)
### What problem does this PR solve?

Feat: Modify the style of the release confirmation box.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: balibabu <assassin_cike@163.com>
Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
2026-03-13 16:31:17 +08:00
Liu An
667f9c1c3a fix: remove duplicate "arabic" key in French translations (#13529)
### What problem does this PR solve?

Removed duplicate key that caused build warning during Vite build.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-13 10:57:19 +08:00
Jimmy Ben Klieve
1a4dee4313 refactor(ui): unify top level pages structure, use standard language codes and time zones (#13573)
### What problem does this PR solve?

- Unify top level pages structure
- Standardize locale language codes (BCP 47) and time zones (IANA tz)


> **Note:** 
> Newly created user info brings non-standard default values `timezone:
"UTC+8\tAsia/Shanghai"` and `language: "English"`.


### Type of change

- [x] Refactoring
2026-03-12 21:01:09 +08:00
Jimmy Ben Klieve
31a8184f63 refactor(ui): update ui for user settings, etc. (#13532)
### What problem does this PR solve?

Update UI styles:
- **User settings**
- Component styles: 
   - `ui/button.tsx`
   - `ui/checkbox.tsx`
   - `avatar-upload.tsx`
   - `file-uploader.tsx`
   - `icon-font.tsx`

### Type of change

- [x] Refactoring
2026-03-12 13:33:36 +08:00
Attili-sys
6afd13ff29 Feat/arabic language support (#13516)
### What problem does this PR solve?

This PR implements comprehensive Arabic language support for the RAGFlow
application. The changes include:
- Complete Arabic translation of all UI text elements in the web
interface
- RTL (right-to-left) layout support for Arabic content
- Localization updates for all supported languages (ar, bg, de, en, es,
fr, id, it, ja, pt-br, ru, vi, zh-traditional, zh)
- UI component adjustments to properly display Arabic text and support
RTL layout

The implementation ensures that Arabic-speaking users can fully interact
with the application in their native language with proper text rendering
and layout direction.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

<img width="2866" height="1617" alt="image"
src="https://github.com/user-attachments/assets/f2751b34-1b65-4867-b81d-a1068c17b9b7"
/>

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-11 15:06:07 +08:00
Jimmy Ben Klieve
507ba4ea20 refactor(ui): update knowledge graph, chunk, metadata, agent log styles (#13518)
### What problem does this PR solve?

Update UI styles:
- **Dataset** > **Knowledge graph** tooltip
- **Dataset** > **Files** > **Manage metadata** modal
- **Dataset** > **Files** > **Modify Chunking Method** > **Auto
metadata** > **Manage generation settings** modal
- **Agent** > **Canvas (Ingestion pipeline)** > **Dataflow result**

### Type of change

- [x] Refactoring
2026-03-11 11:27:20 +08:00
balibabu
d36e3c97d1 Feat: Add a user_id field to the message and retrieval operators. (#13508)
### What problem does this PR solve?

Feat: Add a user_id field to the message and retrieval operators.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-10 22:18:27 +08:00
balibabu
81461b4505 Fix: The number of deleted session prompts is displayed incorrectly. #13499 (#13500)
### What problem does this PR solve?

Fix: The number of deleted session prompts is displayed incorrectly.
#13499
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 16:01:31 +08:00
balibabu
aaf900cf16 Feat: Display release status in agent version history. (#13479)
### What problem does this PR solve?
Feat: Display release status in agent version history.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-03-10 14:25:27 +08:00
Jin Hai
610c1b507d Add more API of admin server of go (#13403)
### What problem does this PR solve?

Add APIs to admin server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 10:44:53 +08:00
Heyang Wang
c217b8f3d8 Feat: add DingTalk AI Table connector and integration for data synch… (#13413)
### What problem does this PR solve?

Add DingTalk AI Table connector and integration for data synchronization

Issue #13400

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: wangheyang <wangheyang@corp.netease.com>
2026-03-06 21:13:23 +08:00
Jimmy Ben Klieve
094eae3cf5 refactor(ui): adjust dataset page styles (#13452)
### What problem does this PR solve?

- Adjust UI styles in **Dataset** pages.
- Adjust several shared components styles
- Modify files and directory structure in `src/layouts`

### Type of change

- [x] Refactoring
2026-03-06 21:13:14 +08:00
chanx
ae4645e01b Fix: Add folder upload #9743 (#13448)
### What problem does this PR solve?

Fix: Add folder upload  #9743

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-06 20:17:29 +08:00
balibabu
82a616589b Feat: Add PublishConfirmDialog (#13447)
### What problem does this PR solve?

Feat: Add PublishConfirmDialog

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 20:17:21 +08:00
balibabu
6849d35bf5 Feat: Optimize the style of the chat page. (#13429)
### What problem does this PR solve?

Feat: Optimize the style of the chat page.
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 11:42:25 +08:00
chanx
8b534c895e Fix: UI Placeholder and Hint Optimization (#13416)
### What problem does this PR solve?

Fix: UI Placeholder and Hint Optimization

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-05 18:13:19 +08:00
Idriss Sbaaoui
b0ace2c5d0 feat: enable Arabic in production UI and add complete Arabic documentation (#13315)
### What problem does this PR solve?

This PR adds end-to-end Arabic support in production. It also adds a
full Arabic README

### Type of change

 - [x] New Feature (non-breaking change which adds functionality)
 - [x] Documentation Update
2026-03-02 19:10:11 +08:00
Magicbook1108
5fc3bd38b0 Feat: Support siliconflow.com (#13308)
### What problem does this PR solve?

Feat: Support siliconflow.com

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-02 15:37:42 +08:00
Attili-sys
21bc1ab7ec Feature rtl support (#13118)
### What problem does this PR solve?

This PR adds comprehensive **Right-to-Left (RTL) language support**,
primarily targeting Arabic and other RTL scripts (Hebrew, Persian, Urdu,
etc.).

Previously, RTL content had multiple rendering issues:

- Incorrect sentence splitting for Arabic punctuation in citation logic
- Misaligned text in chat messages and markdown components  
- Improper positioning of blockquotes and “think” sections  
- Incorrect table alignment  
- Citation placement ambiguity in RTL prompts  
- UI layout inconsistencies when mixing LTR and RTL text  

This PR introduces backend and frontend improvements to properly detect,
render, and style RTL content while preserving existing LTR behavior.

#### Backend
- Updated sentence boundary regex in `rag/nlp/search.py` to include
Arabic punctuation:
  - `،` (comma)
  - `؛` (semicolon)
  - `؟` (question mark)
  - `۔` (Arabic full stop)
- Ensures citation insertion works correctly in RTL sentences.
- Updated citation prompt instructions to clarify citation placement
rules for RTL languages.

#### Frontend
- Introduced a new utility: `text-direction.ts`
  - Detects text direction based on Unicode ranges.
  - Supports Arabic, Hebrew, Syriac, Thaana, and related scripts.
  - Provides `getDirAttribute()` for automatic `dir` assignment.

- Applied dynamic `dir` attributes across:
  - Markdown rendering
  - Chat messages
  - Search results
  - Tables
  - Hover cards and reference popovers

- Added proper RTL styling in LESS:
  - Text alignment adjustments
  - Blockquote border flipping
  - Section indentation correction
  - Table direction switching
  - Use of `<bdi>` for figure labels to prevent bidirectional conflicts

#### DevOps / Environment
- Added Windows backend launch script with retry handling.
- Updated dependency metadata.
- Adjusted development-only React debugging behavior.

---

### Type of change

- [x] Bug Fix (non-breaking change which fixes RTL rendering and
citation issues)
- [x] New Feature (non-breaking change which adds RTL detection and
dynamic direction handling)

---------

Co-authored-by: 6ba3i <isbaaoui09@gmail.com>
Co-authored-by: Ahmad Intisar <ahmadintisar@Ahmads-MacBook-M4-Pro.local>
Co-authored-by: Ahmad Intisar <168020872+ahmadintisar@users.noreply.github.com>
Co-authored-by: Liu An <asiro@qq.com>
2026-03-02 13:03:44 +08:00
chanx
0cdddea59a feat: pipeline add preprocess (#13302)
### What problem does this PR solve?

feat: pipeline add preprocess

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 11:50:48 +08:00
Jimmy Ben Klieve
7e0dd906f2 refactor: update admin ui (#13280)
### What problem does this PR solve?

Update for Admin UI:
- Update file picker input in **Registration whitelist** > **Import from
Excel** modal
- Modify DOM structure of **Sandbox Settings** and move several
hardcoded texts into translation files

### Type of change

- [x] Refactoring
2026-02-28 19:21:51 +08:00
Yesid Cano Castro
d1afcc9e71 feat(seafile): add library and directory sync scope support (#13153)
### What problem does this PR solve?

The SeaFile connector currently synchronises the entire account — every
library
visible to the authenticated user. This is impractical for users who
only need
a subset of their data indexed, especially on large SeaFile instances
with many
shared libraries.

This PR introduces granular sync scope support, allowing users to choose
between
syncing their entire account, a single library, or a specific directory
within a
library. It also adds support for SeaFile library-scoped API tokens
(`/api/v2.1/via-repo-token/` endpoints), enabling tighter access control
without
exposing account-level credentials.


### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Test

```
from seafile_connector import SeaFileConnector
import logging
import os

logging.basicConfig(level=logging.DEBUG)

URL = os.environ.get("SEAFILE_URL", "https://seafile.example.com")
TOKEN = os.environ.get("SEAFILE_TOKEN", "")
REPO_ID = os.environ.get("SEAFILE_REPO_ID", "")
SYNC_PATH = os.environ.get("SEAFILE_SYNC_PATH", "/Documents")
REPO_TOKEN = os.environ.get("SEAFILE_REPO_TOKEN", "")

def _test_scope(scope, repo_id=None, sync_path=None):
    print(f"\n{'='*50}")
    print(f"Testing scope: {scope}")
    print(f"{'='*50}")

    creds = {"seafile_token": TOKEN} if TOKEN else {}
    if REPO_TOKEN and scope in ("library", "directory"):
        creds["repo_token"] = REPO_TOKEN

    connector = SeaFileConnector(
        seafile_url=URL,
        batch_size=5,
        sync_scope=scope,
        include_shared = False,
        repo_id=repo_id,
        sync_path=sync_path,
    )
    connector.load_credentials(creds)
    connector.validate_connector_settings()

    count = 0
    for batch in connector.load_from_state():
        for doc in batch:
            count += 1
            print(f"  [{count}] {doc.semantic_identifier} "
                  f"({doc.size_bytes} bytes, {doc.extension})")

    print(f"\n-> {scope} scope: {count} document(s) found.\n")

# 1. Account scope
if TOKEN:
    _test_scope("account")
else:
    print("\nSkipping account scope (set SEAFILE_TOKEN)")

# 2. Library scope
if REPO_ID and (TOKEN or REPO_TOKEN):
    _test_scope("library", repo_id=REPO_ID)
else:
    print("\nSkipping library scope (set SEAFILE_REPO_ID + token)")

# 3. Directory scope
if REPO_ID and SYNC_PATH and (TOKEN or REPO_TOKEN):
    _test_scope("directory", repo_id=REPO_ID, sync_path=SYNC_PATH)
else:
    print("\nSkipping directory scope (set SEAFILE_REPO_ID + SEAFILE_SYNC_PATH + token)")
```
2026-02-28 10:24:28 +08:00
Jimmy Ben Klieve
c0823e8d6d refactor: update chat ui (#13269)
### What problem does this PR solve?

Update **Chat** UI:
- Align to the design.
- Update `<AudioButton>` visualizer logic.
- Fix keyboard navigation issue.

### Type of change

- [x] Refactoring
2026-02-27 22:26:19 +08:00
Jin Hai
a1549c0fdc Fix UI (#13239)
### What problem does this PR solve?

This pull request makes a minor update to the English locale strings for
the Table of Contents toggle buttons, changing the labels from "Show
TOC"/"Hide TOC" to "Show content"/"Hide content" for improved clarity.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-02-26 19:21:08 +08:00
Magicbook1108
1aa49a11f0 Feat: support AWS SES smtp (#13195)
### What problem does this PR solve?

Support AWS SES smtp #13179

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-26 13:49:53 +08:00
writinwaters
f9e0eb38ec Refact: Updated ingestion pipeline UI. (#13216)
### What problem does this PR solve?

Updated ingestion pipeline-specific UI tips.

### Type of change

- [x] Refactoring
2026-02-25 19:29:04 +08:00
PentaFDevs
8ad47bf242 feat: add 'Open in new tab' button for agents (#13044)
- Add new button in agent management dropdown to open agent in new tab
- Implement token-based authentication for shared agent access
- Add translations for 9 languages (en, zh, zh-tw, de, fr, it, ru,
pt-br, vi)
- Keep existing 'Embed into webpage' functionality intact

### What problem does this PR solve?

This allows users to open agents in a separate tab to work in background
while continuing to use other parts of the application.

<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/ca1719c8-2f00-4570-a730-1321fa0bfd57"
/>
<img width="254" height="222" alt="image"
src="https://github.com/user-attachments/assets/b3dd6d9f-b7e7-46b0-83e7-f0ea86e7b156"
/>
<img width="1920" height="1080" alt="image"
src="https://github.com/user-attachments/assets/e94e99f9-9039-43f7-b2d9-862b9448630c"
/>

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-25 13:39:02 +08:00
chanx
7db2fb200c Fix: Metadata mult-selected display error (#13189)
### What problem does this PR solve?

Fix: Metadata mult-selected display error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-24 13:10:32 +08:00
Trifon
ce71d87867 Add Bulgarian language support (#13147)
### What problem does this PR solve?

RAGFlow supports 12 UI languages but does not include Bulgarian. This PR
adds Bulgarian (`bg` / `Български`) as the 13th supported language,
covering the full UI translation (2001 keys across all 26 sections) and
OCR/PDF parser language mapping.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

### Changes

- **`web/src/constants/common.ts`** — Registered Bulgarian in all 5
language data structures (`LanguageList`, `LanguageMap`,
`LanguageAbbreviation` enum, `LanguageAbbreviationMap`,
`LanguageTranslationMap`)
- **`web/src/locales/config.ts`** — Added lazy-loading dynamic import
for the `bg` locale
- **`web/src/locales/bg.ts`** *(new)* — Full Bulgarian translation file
with all 26 sections, matching the English source (`en.ts`). All
interpolation placeholders, HTML tags, and technical terms are preserved
as-is
- **`deepdoc/parser/mineru_parser.py`** — Mapped `'Bulgarian'` to
`'cyrillic'` in `LANGUAGE_TO_MINERU_MAP` for OCR/PDF parser support

### How it works

The language selector automatically picks up the new entry. When a user
selects "Български", the translation bundle is lazy-loaded on demand.
The preference is persisted to the database and localStorage across
sessions.
2026-02-14 16:51:29 +08:00
chanx
f2a1d59c36 Refactor: Remove ant design component (#13143)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Refactoring
2026-02-13 18:40:41 +08:00
chanx
b922a5cbdf Fix: replace session page icons and fix nested list search functionality in filters (#13127)
### What problem does this PR solve?

Fix: replace session page icons and fix nested list search functionality
in filters

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-12 19:48:35 +08:00
writinwaters
4341d81e29 Refact: Updated UI tips. (#13093)
### What problem does this PR solve?

Updated UI tips.

### Type of change


- [x] Refactoring
2026-02-10 16:25:56 +08:00
balibabu
141157f529 Feat: Translation page index. (#13083)
### What problem does this PR solve?

Feat: Translation page index.

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-10 10:37:38 +08:00
balibabu
db37804f10 Feat: Add Explore page (#13043)
### What problem does this PR solve?

Feat: Add Explore page

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-02-09 19:53:51 +08:00
balibabu
bbd8ba64a1 Feat: Control interface documentation directory display and hiding (#13008)
### What problem does this PR solve?

Feat: Control interface documentation directory display and hiding

### Type of change


- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2026-02-05 16:59:20 +08:00
chanx
89fdb1d498 Feat: Add model verify (#13005)
### What problem does this PR solve?

Feat: Add model verify

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-02-05 15:53:20 +08:00
writinwaters
0ab02854d9 Refact: Updated UI tips (#12976)
### What problem does this PR solve?

Updated UI tips.

### Type of change

- [x] Refactoring
2026-02-04 09:48:59 +08:00
Yesid Cano Castro
deeae8dba4 feat(connector): add Seafile as data source (#12945)
### What problem does this PR solve?
This PR adds **Seafile** as a new data source connector for RAGFlow.

[Seafile](https://www.seafile.com/) is an open-source, self-hosted file
sync and share platform widely used by enterprises, universities, and
organizations that require data sovereignty and privacy. Users who store
documents in Seafile currently have no way to index and search their
content through RAGFlow.

This connector enables RAGFlow users to:
- Connect to self-hosted Seafile servers via API token
- Index documents from personal and shared libraries
- Support incremental polling for updated files
- Seamlessly integrate Seafile-stored documents into their RAG pipelines


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
### Changes included

- `SeaFileConnector` implementing `LoadConnector` and `PollConnector`
interfaces
- Support for API token
- Recursive file traversal across libraries
- Time-based filtering for incremental updates
- Seafile logo (sourced from Simple Icons, CC0)
- Connector configuration and registration

### Testing

- Tested against self-hosted Seafile Community Edition
- Verified authentication (token)
- Verified document ingestion from personal and shared libraries
- Verified incremental polling with time filters
2026-02-03 13:42:05 +08:00
Jimmy Ben Klieve
fafaaa26c3 feat: memory status (#12959)
### What problem does this PR solve?

Add memory status indicator and detail message dialog 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-02-03 11:16:18 +08:00
writinwaters
d99f6a611a Refact: Updated UI tips. (#12898)
### What problem does this PR solve?

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [x] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):
2026-01-29 17:56:07 +08:00
Zhichang Yu
fd11aca8e5 feat: Implement pluggable multi-provider sandbox architecture (#12820)
## Summary

Implement a flexible sandbox provider system supporting both
self-managed (Docker) and SaaS (Aliyun Code Interpreter) backends for
secure code execution in agent workflows.

**Key Changes:**
-  Aliyun Code Interpreter provider using official
`agentrun-sdk>=0.0.16`
-  Self-managed provider with gVisor (runsc) security
-  Arguments parameter support for dynamic code execution
-  Database-only configuration (removed fallback logic)
-  Configuration scripts for quick setup

Issue #12479

## Features

### 🔌 Provider Abstraction Layer

**1. Self-Managed Provider** (`agent/sandbox/providers/self_managed.py`)
- Wraps existing executor_manager HTTP API
- gVisor (runsc) for secure container isolation
- Configurable pool size, timeout, retry logic
- Languages: Python, Node.js, JavaScript
- ⚠️ **Requires**: gVisor installation, Docker, base images

**2. Aliyun Code Interpreter**
(`agent/sandbox/providers/aliyun_codeinterpreter.py`)
- SaaS integration using official agentrun-sdk
- Serverless microVM execution with auto-authentication
- Hard timeout: 30 seconds max
- Credentials: `AGENTRUN_ACCESS_KEY_ID`, `AGENTRUN_ACCESS_KEY_SECRET`,
`AGENTRUN_ACCOUNT_ID`, `AGENTRUN_REGION`
- Automatically wraps code to call `main()` function

**3. E2B Provider** (`agent/sandbox/providers/e2b.py`)
- Placeholder for future integration

### ⚙️ Configuration System

- `conf/system_settings.json`: Default provider =
`aliyun_codeinterpreter`
- `agent/sandbox/client.py`: Enforces database-only configuration
- Admin UI: `/admin/sandbox-settings`
- Configuration validation via `validate_config()` method
- Health checks for all providers

### 🎯 Key Capabilities

**Arguments Parameter Support:**
All providers support passing arguments to `main()` function:
```python
# User code
def main(name: str, count: int) -> dict:
    return {"message": f"Hello {name}!" * count}

# Executed with: arguments={"name": "World", "count": 3}
# Result: {"message": "Hello World!Hello World!Hello World!"}
```

**Self-Describing Providers:**
Each provider implements `get_config_schema()` returning form
configuration for Admin UI

**Error Handling:**
Structured `ExecutionResult` with stdout, stderr, exit_code,
execution_time

## Configuration Scripts

Two scripts for quick Aliyun sandbox setup:

**Shell Script (requires jq):**
```bash
source scripts/configure_aliyun_sandbox.sh
```

**Python Script (interactive):**
```bash
python3 scripts/configure_aliyun_sandbox.py
```

## Testing

```bash
# Unit tests
uv run pytest agent/sandbox/tests/test_providers.py -v

# Aliyun provider tests
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter.py -v

# Integration tests (requires credentials)
uv run pytest agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py -v

# Quick SDK validation
python3 agent/sandbox/tests/verify_sdk.py
```

**Test Coverage:**
- 30 unit tests for provider abstraction
- Provider-specific tests for Aliyun
- Integration tests with real API
- Security tests for executor_manager

## Documentation

- `docs/develop/sandbox_spec.md` - Complete architecture specification
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration from legacy
sandbox
- `agent/sandbox/tests/QUICKSTART.md` - Quick start guide
- `agent/sandbox/tests/README.md` - Testing documentation

## Breaking Changes

⚠️ **Migration Required:**

1. **Directory Move**: `sandbox/` → `agent/sandbox/`
   - Update imports: `from sandbox.` → `from agent.sandbox.`

2. **Mandatory Configuration**: 
   - SystemSettings must have `sandbox.provider_type` configured
   - Removed fallback default values
- Configuration must exist in database (from
`conf/system_settings.json`)

3. **Aliyun Credentials**:
   - Requires `AGENTRUN_*` environment variables (not `ALIYUN_*`)
   - `AGENTRUN_ACCOUNT_ID` is now required (Aliyun primary account ID)

4. **Self-Managed Provider**:
   - gVisor (runsc) must be installed for security
   - Install: `go install gvisor.dev/gvisor/runsc@latest`

## Database Schema Changes

```python
# SystemSettings.value: CharField → TextField
api/db/db_models.py: Changed for unlimited config length

# SystemSettingsService.get_by_name(): Fixed query precision
api/db/services/system_settings_service.py: startswith → exact match
```

## Files Changed

### Backend (Python)
- `agent/sandbox/providers/base.py` - SandboxProvider ABC interface
- `agent/sandbox/providers/manager.py` - ProviderManager
- `agent/sandbox/providers/self_managed.py` - Self-managed provider
- `agent/sandbox/providers/aliyun_codeinterpreter.py` - Aliyun provider
- `agent/sandbox/providers/e2b.py` - E2B provider (placeholder)
- `agent/sandbox/client.py` - Unified client (enforces DB-only config)
- `agent/tools/code_exec.py` - Updated to use provider system
- `admin/server/services.py` - SandboxMgr with registry & validation
- `admin/server/routes.py` - 5 sandbox API endpoints
- `conf/system_settings.json` - Default: aliyun_codeinterpreter
- `api/db/db_models.py` - TextField for SystemSettings.value
- `api/db/services/system_settings_service.py` - Exact match query

### Frontend (TypeScript/React)
- `web/src/pages/admin/sandbox-settings.tsx` - Settings UI
- `web/src/services/admin-service.ts` - Sandbox service functions
- `web/src/services/admin.service.d.ts` - Type definitions
- `web/src/utils/api.ts` - Sandbox API endpoints

### Documentation
- `docs/develop/sandbox_spec.md` - Architecture spec
- `agent/sandbox/tests/MIGRATION_GUIDE.md` - Migration guide
- `agent/sandbox/tests/QUICKSTART.md` - Quick start
- `agent/sandbox/tests/README.md` - Testing guide

### Configuration Scripts
- `scripts/configure_aliyun_sandbox.sh` - Shell script (jq)
- `scripts/configure_aliyun_sandbox.py` - Python script

### Tests
- `agent/sandbox/tests/test_providers.py` - 30 unit tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter.py` - Provider tests
- `agent/sandbox/tests/test_aliyun_codeinterpreter_integration.py` -
Integration tests
- `agent/sandbox/tests/verify_sdk.py` - SDK validation

## Architecture

```
Admin UI → Admin API → SandboxMgr → ProviderManager → [SelfManaged|Aliyun|E2B]
                                      ↓
                                  SystemSettings
```

## Usage

### 1. Configure Provider

**Via Admin UI:**
1. Navigate to `/admin/sandbox-settings`
2. Select provider (Aliyun Code Interpreter / Self-Managed)
3. Fill in configuration
4. Click "Test Connection" to verify
5. Click "Save" to apply

**Via Configuration Scripts:**
```bash
# Aliyun provider
export AGENTRUN_ACCESS_KEY_ID="xxx"
export AGENTRUN_ACCESS_KEY_SECRET="yyy"
export AGENTRUN_ACCOUNT_ID="zzz"
export AGENTRUN_REGION="cn-shanghai"
source scripts/configure_aliyun_sandbox.sh
```

### 2. Restart Service

```bash
cd docker
docker compose restart ragflow-server
```

### 3. Execute Code in Agent

```python
from agent.sandbox.client import execute_code

result = execute_code(
    code='def main(name: str) -> dict: return {"message": f"Hello {name}!"}',
    language="python",
    timeout=30,
    arguments={"name": "World"}
)

print(result.stdout)  # {"message": "Hello World!"}
```

## Troubleshooting

### "Container pool is busy" (Self-Managed)
- **Cause**: Pool exhausted (default: 1 container in `.env`)
- **Fix**: Increase `SANDBOX_EXECUTOR_MANAGER_POOL_SIZE` to 5+

### "Sandbox provider type not configured"
- **Cause**: Database missing configuration
- **Fix**: Run config script or set via Admin UI

### "gVisor not found"
- **Cause**: runsc not installed
- **Fix**: `go install gvisor.dev/gvisor/runsc@latest && sudo cp
~/go/bin/runsc /usr/local/bin/`

### Aliyun authentication errors
- **Cause**: Wrong environment variable names
- **Fix**: Use `AGENTRUN_*` prefix (not `ALIYUN_*`)

## Checklist

- [x] All tests passing (30 unit tests + integration tests)
- [x] Documentation updated (spec, migration guide, quickstart)
- [x] Type definitions added (TypeScript)
- [x] Admin UI implemented
- [x] Configuration validation
- [x] Health checks implemented
- [x] Error handling with structured results
- [x] Breaking changes documented
- [x] Configuration scripts created
- [x] gVisor requirements documented

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-28 13:28:21 +08:00