Commit Graph

2081 Commits

Author SHA1 Message Date
Wang Qi
899f76af6b Fix add OpenRouter base_url, UI need to select at least one model to verify (#15894)
Fix add OpenRouter base_url, UI need to select at least one model to verify
2026-06-10 14:59:27 +08:00
chanx
6822307436 fix: rename ark_api_key to api_key for volcengine provider config (#15896)
### What problem does this PR solve?

fix: rename ark_api_key to api_key for volcengine provider config

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-10 14:56:38 +08:00
chanx
c23809a4bd Fix: Fix some model provider-related UI issues (#15884)
### What problem does this PR solve?

Fix: Fix some model provider-related UI issues

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-10 14:05:57 +08:00
Wang Qi
9aa81e7cad Fix paddle ocr / minerU cannot add (#15858)
Fix paddle ocr / minerU cannot add
2026-06-10 13:04:13 +08:00
balibabu
10bbe6b5d4 Fix: The variables in the Visual Input File of the agent operator are not displayed. (#15856)
### What problem does this PR solve?

Fix: The variables in the Visual Input File of the agent operator are
not displayed.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-09 19:41:22 +08:00
chanx
84482762d5 feat: support custom editing for model list (#15855)
### What problem does this PR solve?

feat: support custom editing for model list

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-06-09 19:24:43 +08:00
Wang Qi
2773208159 Fix: MinerU cannot be added (#15841)
Fix: MinerU cannot be added
2026-06-09 19:06:51 +08:00
balibabu
287a4cfd2b Fix: An error message appears when accessing the agent's launch page: "pagesize exceeds maximum value". (#15835)
### What problem does this PR solve?
Fix: An error message appears when accessing the agent's launch page:
"pagesize exceeds maximum value".

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: balibabu <assassin_cike@163.com>
2026-06-09 16:56:47 +08:00
Jin Hai
d02eb6b596 Go: refactor CLI (#15728)
### What problem does this PR solve?

```
RAGFlow(user)> add api server 'ccc' host '127.0.0.1:9980';
SUCCESS
RAGFlow(user)> list api server;
+------------+---------------+-----------------+---------+-------------+---------------+
| api_server | api_server_ip | api_server_port | auth    | user_name   | user_password |
+------------+---------------+-----------------+---------+-------------+---------------+
| ccc        | 127.0.0.1     | 9980            | no auth |             |               |
| default    | 127.0.0.1     | 9384            | login   | aaa@aaa.com | ***           |
+------------+---------------+-----------------+---------+-------------+---------------+
RAGFlow(user)> delete api server 'ccc';
SUCCESS
RAGFlow(user)> list api server;
+------------+---------------+-----------------+---------+
| api_server | api_server_ip | api_server_port | auth    |
+------------+---------------+-----------------+---------+
| default    | 127.0.0.1     | 9384            | no auth |
+------------+---------------+-----------------+---------+

RAGFlow(user)> show admin server;
+--------------+-------+
| field        | value |
+--------------+-------+
| admin_server | N/A   |
+--------------+-------+
RAGFlow(user)> add admin server host '127.0.0.1:9880';
SUCCESS
RAGFlow(user)> show admin server;
+-------------------+-----------+
| field             | value     |
+-------------------+-----------+
| admin_server_ip   | 127.0.0.1 |
| admin_server_port | 9880      |
| auth              | no auth   |
+-------------------+-----------+
RAGFlow(user)> delete admin server;
SUCCESS
RAGFlow(user)> show admin server;
+--------------+-------+
| field        | value |
+--------------+-------+
| admin_server | N/A   |
+--------------+-------+

RAGFlow(user)> show current
+-----------------+-------------+
| field           | value       |
+-----------------+-------------+
| api_server_port | 9384        |
| user_name       | aaa@aaa.com |
| user_password   | ***         |
| mode            | api         |
| verbose         | false       |
| api_server      | default     |
| api_server_ip   | 127.0.0.1   |
| auth            | login       |
| output          | table       |
| interactive     | true        |
+-----------------+-------------+
```
### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-09 15:22:50 +08:00
chanx
298a23f74c fix: resolve issue where some models do not use modelInfo parameter (#15830)
### What problem does this PR solve?

fix: resolve issue where some models do not use modelInfo parameter

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-09 13:18:01 +08:00
Wang Qi
9c0cc77e35 Fix empty response set not take effect (#15824)
Fix empty response set not take effect
2026-06-09 13:06:58 +08:00
balibabu
d025e18176 Fix: Add a waiting status to the messages on the chat page. (#15773)
### What problem does this PR solve?

Fix: Add a waiting status to the messages on the chat page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-08 19:17:00 +08:00
chanx
7dd4030986 fix: Resolve error when checking pipeline parsing result (#15778)
### What problem does this PR solve?

fix: Resolve error when checking pipeline parsing result

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-08 19:16:21 +08:00
gaulin-ai
8abe627e69 i18n(it): complete Italian translation (49% → 100%) (#15729)
## Summary

Brings the Italian locale (`web/src/locales/it.ts`) from approximately
**49% coverage** (986 out of 2008 keys) to **100% coverage** (2008/2008
keys), fully aligned with `en.ts` in structure and key count.

### What was missing

Previously untranslated sections include:
- `skills`, `skillSearch` — agent skills UI
- `memories`, `memory` — memory management
- `datasetOverview` — dataset statistics
- `llmTools` — LLM tool configuration
- `explore` — explore/template page
- `dataflowParser` — ingestion pipeline parser settings
- `flow` (complete) — agent canvas / workflow editor
- `setting` connectors section — data source connectors (Google Drive,
Gmail, Box, RDBMS, etc.)
- Various `header`, `common`, `knowledgeBase`, `chat`, `fileManager`
additions

### Translation conventions

- Technical terms kept in English: RAG, LLM, API, token, chunk,
embedding, prompt, dataset, agent, canvas, knowledge graph, RAPTOR,
webhook, and all model/provider names (Bedrock, Tavily, SearXNG, etc.)
- `{{placeholder}}` variables preserved unchanged
- Informal *tu* form used consistently, matching the existing style
- All previously correct translations preserved
2026-06-08 18:06:47 +08:00
Worldwide
86b320e746 feat(web): show provider count on each model-type filter tag (#15444)
Fixes #15413 

### What problem does this PR solve?

In **Settings → Model providers**, the *Available models* panel lets you
filter
providers by model type (All, LLM, Embedding, Rerank, TTS, ASR, VLM, …),
but the
filter tags gave no hint of how many providers fall under each type.
Users had to
click a tag to find out, and an empty category looked identical to a
populated one.

This PR adds a count to each filter tag in `AvailableModels`:

- The **All** tag shows the total number of providers currently listed.
- Each model-type tag shows how many providers offer that model type.
- Counts respect the active search term, so the badge always matches the
number of
  cards shown once that tag is selected.
- Each provider is counted once per model type (deduplicated via a
`Set`), so a
  provider that lists the same type more than once isn't double-counted.

Counts are rendered with `tabular-nums` for stable width and dimmed via
`opacity-60`
so they read as secondary to the label. No API changes; the existing
filter logic is
untouched — this is purely an additive UI affordance.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-08 17:31:22 +08:00
chanx
2bd8900638 Fix: Model provider bugs (#15770)
### What problem does this PR solve?

Fix: Model provider bugs

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-08 17:04:05 +08:00
chanx
144abbe2eb feat: Unify the 'Add Model Provider' modal (#15768)
### What problem does this PR solve?

feat:Unify the 'Add Model Provider' modal

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-06-08 16:46:52 +08:00
balibabu
9c32b73cf7 Fix: The embedded website floating component on the agent page does not display citations. (#15767)
### What problem does this PR solve?

Fix: The embedded website floating component on the agent page does not
display citations.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-08 16:09:36 +08:00
balibabu
c935f305e2 Fix: The time zone is not displayed on the personal profile page. (#15759)
### What problem does this PR solve?

Fix: The time zone is not displayed on the personal profile page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-08 14:33:52 +08:00
balibabu
9c14e3f377 Fix: When adding a chat in the main interface, a warning will automatically pop up (#15685)
### What problem does this PR solve?

Fix: When adding a chat in the main interface, a warning will
automatically pop up (even if embedding and LLM model have already been
configured).
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-05 19:09:22 +08:00
web-dev0521
1d7e45115b feat(connectors): add Salesforce CRM data source connector (#15462)
### What problem does this PR solve?

Closes #15461.

RAGFlow had no way to ingest Salesforce CRM data, so support / sales
teams couldn't ground responses on live Accounts, Contacts,
Opportunities, Cases, or Knowledge articles. This adds a first-class
Salesforce data source connector that authenticates against a Connected
App via OAuth 2.0 client-credentials, queries selected SObjects via
SOQL, and turns each record into an indexable document with incremental
sync.

**Highlights**
- `common/data_source/salesforce_connector.py`: new
`SalesforceConnector` (`CheckpointedConnectorWithPermSync` +
`SlimConnectorWithPermSync`).
- OAuth 2.0 client-credentials flow; canonical `instance_url` from the
token response so multi-pod orgs route correctly.
- Per-object `SystemModstamp` cursor stored in
`SalesforceCheckpoint.cursors` — a failure mid-object doesn't rewind
sibling objects, and re-syncs only fetch changed rows.
- Deterministic record-to-text formatter (sorted keys) so SOQL field
reordering on the server doesn't mark every row "changed" on each poll.
- `_get_json` raises on non-2xx so 429 / 5xx never silently advance the
checkpoint past missing data.
- `Knowledge__kav` is in the default object set but is skipped silently
when the org doesn't have Salesforce Knowledge enabled (404 on
describe).
- Slim-doc IDs are scoped as `<Object>/<Id>` so prune deletes can't
collide across object types.
- `common/constants.py`, `common/data_source/config.py`,
`common/data_source/__init__.py`: register `salesforce` in `FileSource`
/ `DocumentSource` and export `SalesforceConnector`.
- `rag/svr/sync_data_source.py`: new `Salesforce(SyncBase)` class routed
through `load_from_checkpoint` (poll_source would re-walk every object
each run) and added to `func_factory`.
- Frontend:
- `web/src/pages/user-setting/data-source/constant/index.tsx`: new
`DataSourceKey.SALESFORCE`, form fields (instance URL, client ID/secret,
objects, api_version, batch size), `syncDeletedFiles` capability,
default form values, and tile entry with the new icon.
  - `web/src/locales/{en,zh}.ts`: description + per-field tooltips.
- `web/src/assets/svg/data-source/salesforce.svg`: 48x48 brand-style
icon to match the other Microsoft / cloud tiles.

**Verification**
- `npm run build` (vite + esbuild) passes (1m 26s).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-05 13:24:36 +08:00
web-dev0521
98f2a2e60b feat(connectors): add Azure Blob Storage data source connector (#15466)
### What problem does this PR solve?

Closes #15465.

RAGFlow supports S3, Google Cloud Storage, R2, and OCI as data sources
but not Azure Blob Storage, leaving Azure users without a way to index
container objects into a knowledge base. This adds a first-class Azure
Blob Storage data-source connector — distinct from RAGFlow's existing
Azure storage *backends* (`rag/utils/azure_sas_conn.py`,
`rag/utils/azure_spn_conn.py`) which store RAGFlow's own files.

**Highlights**
- `common/data_source/azure_blob_connector.py`: new `AzureBlobConnector`
(`CheckpointedConnectorWithPermSync` + `SlimConnectorWithPermSync`).
- Uses the existing `azure-storage-blob` dependency (already in
`pyproject.toml`).
  - Three auth modes, tried in order of precedence:
1. **Account key** — `account_name` + `account_key` + `container_name`.
    2. **Connection string** — `connection_string` + `container_name`.
3. **SAS token** — `container_url` + `sas_token` (same shape as
`RAGFlowAzureSasBlob`).
- ETag fingerprint stored per blob in `AzureBlobCheckpoint.etags` —
unchanged blobs (same ETag as last run) are skipped without a download.
Only new/modified blobs are fetched.
  - Optional `prefix` scopes indexing to a virtual folder.
- `validate_connector_settings()` probes `get_container_properties()`
and maps `AuthenticationFailed / 403 / ContainerNotFound` to typed
connector exceptions.
  - Slim-doc IDs are blob names so prune reconciles correctly.
- `common/constants.py`, `common/data_source/config.py`,
`common/data_source/__init__.py`: register `azure_blob` in `FileSource`
/ `DocumentSource` and export `AzureBlobConnector`.
- `rag/svr/sync_data_source.py`: new `AzureBlob(SyncBase)` class routed
through `load_from_checkpoint` (ETag fingerprint owns change-detection)
and added to `func_factory`.
- Frontend:
- `web/src/pages/user-setting/data-source/constant/index.tsx`: new
`DataSourceKey.AZURE_BLOB`, auth-mode selector (account key / connection
string / SAS token), all credential fields, prefix + batch-size,
`syncDeletedFiles` capability, default form values, tile entry with
icon.
- `web/src/locales/{en,zh}.ts`: description + per-field tooltips for all
9 new keys.
- `web/src/assets/svg/data-source/azure-blob.svg`: Azure-branded
stacked-cylinders icon.

**Verification**
- `npm run build` (vite + esbuild) passes (37 s).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-04 21:06:01 +08:00
Jack
6143205b37 feat: implement GET /api/v1/agents/<agent_id>/versions/<version_id> API (#15640)
## Summary

Implement the `GET /api/v1/agents/<agent_id>/versions/<version_id>`
endpoint in Go, returning full version details including DSL.

Depends on #15629 which introduced the version list endpoint and
`UserCanvasVersionDAO` infrastructure.

### Changes

- **Modified**: `internal/handler/agent.go` — Added `GetAgentVersion`
handler with auth check and ownership verification
- **Modified**: `internal/router/router.go` — Registered `GET
/:agent_id/versions/:version_id` route
- **New/Modified tests**: Service and handler tests for the version
detail endpoint

### Testing

```
=== RUN   TestGetVersion_Success       --- PASS
=== RUN   TestGetVersion_WrongCanvas   --- PASS
=== RUN   TestGetVersion_NotFound      --- PASS
=== RUN   TestGetAgentVersionHandler_Success      --- PASS
=== RUN   TestGetAgentVersionHandler_VersionNotFound --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 19:13:26 +08:00
buua436
04dc3bb19c fix: pass search id to searchbots ask (#15646)
### What problem does this PR solve?
This change ensures `/searchbots/ask` receives `search_id` from the
frontend, so the backend can load the matching search configuration when
the shared search flow invokes the endpoint.

### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-04 17:41:56 +08:00
Jack
23aae19898 feat: implement POST /api/v1/agents/<agent_id>/upload API (#15633)
## Summary

Implement the `POST /api/v1/agents/<agent_id>/upload` endpoint in Go,
allowing file uploads associated with agent canvases.

### Changes

- **Modified**: `internal/service/agent.go` — Added `CheckCanvasAccess`
method (owner + team-level permission semantics)
- **Modified**: `internal/handler/agent.go` — Added `UploadAgentFile`
handler with auth check, multipart file parsing, and delegation to
`FileService`. Added `fileUploader` interface for testability.
- **Modified**: `internal/router/router.go` — Registered `POST
/:agent_id/upload` route
- **Modified**: `cmd/server_main.go` — Wired `fileService` into
`AgentHandler`
- **New**: `internal/service/agent_test.go` — 4 service-level tests for
`CheckCanvasAccess` (owner, team member, private denial, not found)
- **New**: `internal/handler/agent_upload_test.go` — 3 handler-level
tests (success with fake file service, cross-user denial, empty file
rejection)

### Testing

All 7 tests pass with zero mocking of the DB layer (in-memory SQLite):

```
=== RUN   TestCheckCanvasAccess_Owner               --- PASS
=== RUN   TestCheckCanvasAccess_NotOwner            --- PASS
=== RUN   TestCheckCanvasAccess_PrivateCanvas_Denied --- PASS
=== RUN   TestCheckCanvasAccess_NotFound            --- PASS
=== RUN   TestUploadAgentFileHandler_Success        --- PASS
=== RUN   TestUploadAgentFileHandler_NoPermission   --- PASS
=== RUN   TestUploadAgentFileHandler_NoFiles        --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:21:47 +08:00
Jack
02d163a177 feat: implement GET /api/v1/agents/<agent_id>/versions API (#15629)
## Summary

Implement the `GET /api/v1/agents/<agent_id>/versions` endpoint in Go,
listing all version snapshots for an agent canvas in descending update
time order.

### Changes

- **New**: `internal/dao/user_canvas_version.go` —
`UserCanvasVersionDAO` with `ListByCanvasID` (ordered by update_time
DESC) and `GetByID`
- **Modified**: `internal/service/agent.go` — Added `CheckCanvasAccess`,
`ListVersions`, `GetVersion` methods
- **Modified**: `internal/handler/agent.go` — Added `ListAgentVersions`
handler with auth check
- **Modified**: `internal/router/router.go` — Registered `GET
/:agent_id/versions` route
- **New**: `internal/service/agent_test.go` — 5 service-level tests
(SQLite in-memory DB, zero mock)
- **Modified**: `internal/handler/agent_test.go` — 3 handler-level tests
(real DB, pre-authenticated context)

### Testing

All 8 tests pass with zero mocking (in-memory SQLite replaces MySQL):

```
=== RUN   TestListVersions_Success         --- PASS
=== RUN   TestListVersions_Empty           --- PASS
=== RUN   TestCheckCanvasAccess_Owner      --- PASS
=== RUN   TestCheckCanvasAccess_NotOwner   --- PASS
=== RUN   TestCheckCanvasAccess_NotFound   --- PASS
=== RUN   TestListAgentVersionsHandler_Success      --- PASS
=== RUN   TestListAgentVersionsHandler_NoPermission --- PASS
=== RUN   TestListAgentVersionsHandler_CanvasNotFound --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:36:26 +08:00
chanx
a678ed7b1f Fix: Switching pagesize on a chunk page did not reset the current page. (#15401)
### What problem does this PR solve?

Fix: Switching pagesize on a chunk page did not reset the current page.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-03 15:57:57 +08:00
chanx
c41855da81 Fix: Model provider add verify and fixed form in modal not resetting issue (#15520)
### What problem does this PR solve?

Fix: Model provider add verify and fixed form in modal not resetting
issue

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-03 11:59:57 +08:00
Lynn
36357a6afd Fix: model provider (#15517)
### What problem does this PR solve?

Fix:
- Handle siliconflow and siliconflow_intl api_key

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-02 19:04:20 +08:00
Dexterity
2819d0ea24 fix(go-models): use per call context timeouts so long streaming responses are not truncated (#15380)
### What problem does this PR solve?

Closes #15379 

Around 29 Go model providers in `internal/entity/models/` share an
`http.Client` configured with `Timeout: 120 * time.Second`, and reuse
that same client for `ChatStreamlyWithSender`. Go's
`http.Client.Timeout` is a hard ceiling on the whole request that also
covers reading the response body, so it behaves as a wall clock on
streaming. Any streamed chat response that lasts longer than 120 seconds
gets cut off in the middle with a timeout error. Long generations,
reasoning model outputs, and slow or overloaded upstreams are the common
victims.

The providers that already behave correctly (`groq`, `mistral`,
`voyage`, `anthropic`) set no client `Timeout` and instead wrap each
request in a `context.WithTimeout`. This change converges the affected
providers onto that same pattern.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-06-02 15:27:26 +08:00
Julian
33ef724b5f Add Bulk action for linking Multiple Files to Datasets (#14960)
### What problem does this PR solve?

Feature: #14961 


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-06-02 12:23:33 +08:00
Jack
67a3ed7558 Fix auto metadata type issue (#15338)
### What problem does this PR solve?

Fix auto metadata type issue
https://github.com/infiniflow/ragflow/issues/15323

Type information is missing at frontend - backend correctly store the
type information for the auto metadata type.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-02 10:23:04 +08:00
balibabu
f194e8b4c4 Fix: The newly added model did not appear in the drop-down menu. (#15476)
### What problem does this PR solve?

Fix: The newly added model did not appear in the drop-down menu.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-01 17:56:41 +08:00
balibabu
82202fa469 Fix: Unable to create dataset (#15472)
### What problem does this PR solve?

Fix: Unable to create dataset

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-01 15:30:52 +08:00
balibabu
e13431cdc0 Fix: If the filename is too long, it overflows the confirmation box for deleting the file. (#15287)
### What problem does this PR solve?

Fix: If the filename is too long, it overflows the confirmation box for
deleting the file.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-01 10:22:56 +08:00
web-dev0521
cd18cfab79 feat(connector): implement Outlook data source connector (issue #15332) (#15333)
### What problem does this PR solve?

Closes #15332.

RAGFlow can index Gmail and generic IMAP mailboxes but had no native
connector for Outlook / Microsoft 365 mail. Organisations on Microsoft
365 had no way to bring mailbox content into a knowledge base through
Microsoft Graph.

This PR adds a net-new Outlook data source that:

- Authenticates against Microsoft Graph with the same MSAL
client-credentials flow already used by the SharePoint and Teams
  connectors (no new auth primitives).
- Pages over `/users/{id}/mailFolders/{folder}/messages/delta` per
mailbox and persists `@odata.deltaLink` values in
`OutlookCheckpoint.delta_links`, so incremental syncs only fetch changed
messages.
- Supports two scoping modes:
- **Tenant-wide** (default): enumerates every user in the tenant via
`/users` and syncs each mailbox. Requires `User.Read.All`.
- **Targeted**: when `user_ids` is provided (comma-separated UPNs or
object IDs), only those mailboxes are synced. `User.Read.All` is not
needed in this mode.
- Lets the caller pick the mail folder (`inbox`, `sentitems`, `archive`,
...). Defaults to `inbox`.
- Maps each message to a `Document` shaped after the Gmail connector:
one `TextSection` carrying `From/To/Cc/Subject` headers + body, with
HTML bodies stripped to text inline (no extra dependency).
- Surfaces typed errors on the validation probe:
401 → `ConnectorMissingCredentialError`, 403 →
`InsufficientPermissionsError` (with `Mail.Read` / `User.Read.All`
hint), 404 on a configured mailbox → `ConnectorValidationError`, 5xx →
`UnexpectedValidationError`.
- Skips messages flagged `@removed` by the delta semantics and messages
whose `receivedDateTime` is older than `poll_range_start`.

#### Files

| File | Change |
|------|--------|
| `common/data_source/outlook_connector.py` | **New** —
`OutlookConnector` (`CheckpointedConnectorWithPermSync` +
`SlimConnectorWithPermSync`) + `OutlookCheckpoint` + tiny `_strip_html`
helper. |
| `common/data_source/config.py` | `DocumentSource.OUTLOOK = "outlook"`.
|
| `common/constants.py` | `FileSource.OUTLOOK = "outlook"`. |
| `common/data_source/__init__.py` | Export `OutlookConnector`. |
| `rag/svr/sync_data_source.py` | `Outlook(SyncBase)` with `batch_size`
normalisation, CSV/list parsing of `user_ids`; registered in
`func_factory`. |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`DataSourceKey.OUTLOOK`, visibility map (`syncDeletedFiles: true`), info
entry, form fields (tenant_id, client_id, client_secret, folder,
user_ids, batch_size), default values. |
| `web/src/locales/en.ts`, `web/src/locales/zh.ts` |
`outlookDescription` + 5 tooltip keys (EN + ZH). |
| `test/unit_test/data_source/test_outlook_connector_unit.py` | **New**
— 19 unit tests (`p1`/`p2`/`p3`) covering auth, validation (tenant-wide
vs specific user vs error paths), checkpoint helpers, user enumeration
pagination, message filtering, HTML body stripping. |

#### Required Azure AD permissions

- `Mail.Read` (Application, admin-granted) — always.
- `User.Read.All` (Application, admin-granted) — only when `user_ids` is
left blank so the connector can enumerate mailboxes.

#### Out of scope

- **Attachment indexing.** The current connector emits message body +
headers; binary attachments are flagged via `metadata.has_attachments`
but not pulled. Adding attachment hydration is straightforward but
scoped out per the issue's "decide whether attachments are indexed in
the first version" note.
- **Delegated (per-user) OAuth.** The connector uses app-only
credentials, consistent with the SharePoint / Teams precedent in this
codebase.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-29 21:52:29 +08:00
web-dev0521
bda2117a25 feat(connector): implement OneDrive data source connector (issue #15330) (#15331)
### What problem does this PR solve?

Closes #15330.

RAGFlow had no connector for OneDrive / OneDrive for Business. Users who
store working documents in OneDrive could not index them into a
knowledge base without manually downloading and re-uploading files.

This PR adds a net-new OneDrive data source that:

- Authenticates against Microsoft Graph with the same MSAL
client-credentials flow already used by the SharePoint and Teams
connectors (no new auth primitives).
- Enumerates every drive visible to the service principal and pages
through `/drives/{id}/root/delta`, persisting `@odata.deltaLink` values
per drive so subsequent syncs only fetch changed items.
- Optionally narrows ingestion to a sub-folder (`folder_path`) without
needing a separate code path.
- Surfaces typed errors on the validation probe (`GET /drives?$top=1`):
401 → `ConnectorMissingCredentialError`, 403 →
`InsufficientPermissionsError` (with a `Files.Read.All` hint), 5xx →
`UnexpectedValidationError`.
- Filters folders, soft-deleted items, and unsupported extensions (`.pdf
.docx .doc .xlsx .xls .pptx .ppt .txt .md .csv`).

#### Files

| File | Change |
|------|--------|
| `common/data_source/onedrive_connector.py` | **New** —
`OneDriveConnector` + `OneDriveCheckpoint`. |
| `common/data_source/config.py` | `DocumentSource.ONEDRIVE =
"onedrive"`. |
| `common/constants.py` | `FileSource.ONEDRIVE = "onedrive"`. |
| `common/data_source/__init__.py` | Export `OneDriveConnector`. |
| `rag/svr/sync_data_source.py` | `OneDrive(SyncBase)` with `batch_size`
normalisation; registered in `func_factory`. |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`DataSourceKey.ONEDRIVE`, visibility map (`syncDeletedFiles: true`),
info entry, form fields (tenant_id, client_id, client_secret,
folder_path, batch_size), default values. |
| `web/src/locales/en.ts`, `web/src/locales/zh.ts` |
`onedriveDescription` + 4 tooltip keys (EN + ZH). |
| `test/unit_test/data_source/test_onedrive_connector_unit.py` | **New**
— 13 unit tests (`p1`/`p2`) covering auth, validation, checkpoint
helpers, and document filtering. |

#### Required Azure AD permission

`Files.Read.All` (Application, admin-granted).

#### Out of scope

- Interactive end-user OAuth (delegated permissions) — the connector
uses app-only credentials, consistent with the SharePoint / Teams
precedent.
- Binary download of file contents — the sync layer emits `Document`s
carrying `webUrl` + metadata; bytes are hydrated downstream by the parse
pipeline.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-29 19:26:06 +08:00
Lynn
dc4b82523b Feat: tenant llm provider (#14595)
### What problem does this PR solve?

Python implementation of the Go-based model_provider API suite.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: bill <yibie_jingnian@163.com>
2026-05-29 17:39:41 +08:00
web-dev0521
98bc9ca6ac feat: implement Microsoft Teams data source connector (#15193)
### What problem does this PR solve?

Closes #15191.

RAGFlow shipped a Microsoft Teams connector stub
(`common/data_source/teams_connector.py`) whose document-loading methods
all returned `[]`, `Teams._generate()` was a `pass`, and Teams was
commented out of the data-source settings UI. As a result there was no
way to index Teams channel conversations into a knowledge base.

This PR implements the connector end to end on top of Microsoft Graph
(Office365-REST-Python-Client). It shares the MSAL client-credentials
auth shape with the SharePoint connector.

**Backend**

- `common/data_source/teams_connector.py`
- `load_credentials()` now builds the Graph client using an MSAL
client-credentials **token callback** — the form `GraphClient` actually
expects. (The previous stub passed a raw access-token string to
`GraphClient(...)`, which is not how that client is driven.) Token
acquisition is lazy, so credential loading performs no network call.
  - `validate_connector_settings()` lists teams via Graph.
- `load_from_checkpoint()` is now a generator that pages teams →
channels → messages, flattens each top-level post together with its
replies into one blob-based `Document` (`extension` `.txt`/`.html`,
`blob`, `size_bytes`, `doc_updated_at`). Incremental syncs are bounded
by message `lastModifiedDateTime` (falling back to `createdDateTime`).
Per-message errors surface as `ConnectorFailure` instead of aborting the
run.
- `retrieve_all_slim_docs_perm_sync()` yields id-only `SlimDocument`
batches and the checkpoint helpers return proper `TeamsCheckpoint`s.
- ACL → `ExternalAccess` mapping is intentionally left best-effort
(`load_from_checkpoint_with_perm_sync` delegates to the standard load)
because the sync pipeline does not currently persist `ExternalAccess`.
- `rag/svr/sync_data_source.py`
- Implemented `Teams._generate()` using the existing
`CheckpointOutputWrapper` pattern (same shape as Confluence/Jira/Google
Drive), supporting full reindex and incremental polling from
`poll_range_start`.
- `TeamsConnector` is already exported from
`common/data_source/__init__.py`.

**Frontend (`web/`)**

- Enabled the `TEAMS` data-source enum and added its form fields
(`tenant_id`, `client_id`, `client_secret`), default values, display
metadata, and a Teams icon.
- Added `teamsDescription` / `teamsTenantIdTip` to `en.ts` and `zh.ts`.

**Tests**

- `test/unit_test/data_source/test_teams_connector_unit.py`: mock-based
unit tests covering credential loading (incomplete creds raise, happy
path sets the Graph client, fetch-without-creds raises), post/reply
flattening (incl. the HTML vs text extension), incremental
`lastModifiedDateTime` filtering, and slim-doc listing. All 6 pass;
`ruff check` is clean.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-28 17:10:38 +08:00
web-dev0521
5de021ebb4 feat: implement Slack data source connector (#15188)
### What problem does this PR solve?

Closes #15187.

RAGFlow shipped a Slack connector
(`common/data_source/slack_connector.py`) but it was never usable:
`Slack._generate()` in the sync worker was a `pass` stub, the
connector's document-generating code was incompatible with the current
data model,
and Slack was commented out of the data-source settings UI. As a result,
teams had no way to index Slack channels/threads into a knowledge base.

This PR completes the connector end to end.

**Backend**

- `common/data_source/slack_connector.py`
- Rewrote `thread_to_doc` to produce a blob-based `Document`
(`extension`/`blob`/`size_bytes`). The previous implementation built the
doc with a `sections=[...]` argument and omitted the now-required
`blob`/`extension`/ `size_bytes` fields, so it raised a validation error
against the current `Document` model. Thread messages are now cleaned
and flattened into a single UTF-8 text blob.
- Added `load_from_state()` / `poll_source(start, end)` generators. The
connector's checkpoint interface is a no-op stub, so both full and
incremental syncs run through a single channel-iterating generator built
on the existing module helpers (`get_channels`, `filter_channels`,
`get_channel_messages`, `_process_message`), with per-channel thread
de-duplication.
- `rag/svr/sync_data_source.py`
- Implemented `Slack._generate()`. Credentials are loaded via
`StaticCredentialsProvider` (the connector requires `slack_bot_token`
and does not support `load_credentials`). Supports full reindex and
incremental polling from `poll_range_start`, plus the optional channel
filter. Modeled on the Confluence/Dropbox wrappers.
- `SlackConnector` was already exported from
`common/data_source/__init__.py`.

**Frontend (`web/`)**

- Enabled the `SLACK` data-source enum and added its form fields (Slack
bot token + optional channel filter), default values, display metadata,
and a Slack icon.
- Added `slackDescription` / `slackBotTokenTip` / `slackChannelsTip`
strings to `en.ts` and `zh.ts`.

**Tests**

- `test/unit_test/data_source/test_slack_connector_unit.py`: unit tests
covering credential loading (`load_credentials` raises,
`set_credentials_provider` initializes clients, missing credentials
raises) and document generation (standalone message + flattened thread,
blob/extension/size_bytes/metadata, and the incremental poll time
window). All 5 pass; `ruff check` is clean.

Required Slack scopes: `channels:read`, `channels:history`,
`users:read`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-28 15:46:07 +08:00
chanx
7e83643536 Fix: Clustering method echo error (#15322)
### What problem does this PR solve?

Fix: Clustering method echo error

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-28 14:32:31 +08:00
web-dev0521
c4c4e228e3 feat: implement SharePoint data source connector (#15190)
### What problem does this PR solve?

Closes #15189.

RAGFlow shipped a SharePoint connector stub
(`common/data_source/sharepoint_connector.py`) whose document-loading
methods all returned `[]`, `SharePoint._generate()` was a `pass`, and
SharePoint was commented out of the data-source settings UI. As a result
there was no way to index files stored in SharePoint document libraries.

This PR implements the connector end to end on top of Microsoft Graph
(Office365-REST-Python-Client).

**Backend**

- `common/data_source/sharepoint_connector.py`
- `load_credentials()` now builds the Graph client using an MSAL
client-credentials **token callback** — the form `GraphClient` actually
expects. (The previous stub passed a raw access-token string to
`GraphClient(...)`, which is not how that client is driven.) Token
acquisition is lazy, so credential loading does no network call.
- `validate_connector_settings()` resolves the configured site via
Graph.
- `load_from_checkpoint()` is now a generator that enumerates every
document library under the site, walks folders depth-first, downloads
each file, and yields blob-based `Document` objects (`extension` /
`blob` / `size_bytes` / `doc_updated_at`). Incremental syncs are bounded
by file `lastModifiedDateTime`. Per-file errors are surfaced as
`ConnectorFailure` rather than aborting the run.
- `retrieve_all_slim_docs_perm_sync()` yields id-only `SlimDocument`
batches (no downloads) and the checkpoint helpers return proper
checkpoints.
- ACL → `ExternalAccess` mapping is intentionally left best-effort
(`load_from_checkpoint_with_perm_sync` delegates to the standard load)
because the sync pipeline does not currently persist `ExternalAccess`;
this can be extended once that plumbing exists.
- `rag/svr/sync_data_source.py`
- Implemented `SharePoint._generate()` using the existing
`CheckpointOutputWrapper` pattern (same shape as Confluence/Jira/Google
Drive), supporting full reindex and incremental polling from
`poll_range_start`.
- `SharePointConnector` is already exported from
`common/data_source/__init__.py`.

**Frontend (`web/`)**

- Enabled the `SHAREPOINT` data-source enum and added its form fields
`site_url`, `tenant_id`, `client_id`, `client_secret`), default values,
display metadata, and a SharePoint icon.
- Added `sharepointDescription` / `sharepointSiteUrlTip` to `en.ts` and
`zh.ts`.

**Tests**

- `test/unit_test/data_source/test_sharepoint_connector_unit.py`:
mock-based unit tests covering credential loading (incomplete creds
raise, happy path sets the Graph client, fetch-without-creds raises),
drive traversal + file download, incremental `lastModifiedDateTime`
filtering, and slim-doc listing. All 6 pass; `ruff check` is clean.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-05-28 13:26:08 +08:00
balibabu
2c099bbb95 Fix: Uploading TSV format documents to the knowledge base did not generate any error messages. (#15284)
### What problem does this PR solve?

Fix: Uploading TSV format documents to the knowledge base did not
generate any error messages.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-27 14:42:53 +08:00
balibabu
187dc8a1e6 Fix: The Creativity parameter of chat was not saved. (#15243)
### What problem does this PR solve?

Fix: The Creativity parameter of chat was not saved.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-27 11:02:30 +08:00
Wang Qi
303221c1f4 Fix: show tag list for chunk (#15251) 2026-05-26 20:24:22 +08:00
chanx
bce11527c3 Fix: Fixed metadata issue (#15226)
### What problem does this PR solve?

Fix: Fixed metadata issue

- The dataset's built-in metadata is now active, but it appears to be
disabled in the individual file configuration.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-26 13:16:15 +08:00
balibabu
ed179ce684 Fix: The prompt variable for the agent operator disappears after input. (#15218)
### What problem does this PR solve?

Fix: The prompt variable for the agent operator disappears after input.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 20:36:51 +08:00
balibabu
c7c75c0a87 Feat: Enable agent messages to display base64 images (#15212)
### What problem does this PR solve?

Feat: Enable agent messages to display base64 images

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2026-05-25 19:02:03 +08:00
balibabu
0f92353bd9 Fix: Replace the red highlight at the top of the PDF document with yellow. (#15203)
### What problem does this PR solve?

Fix: Replace the red highlight at the top of the PDF document with
yellow.
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 17:21:36 +08:00
Wang Qi
4776bfa8a2 Fix: Correct the API path (#15204)
Follow on PR #15146 to reslove the backwad compatability issue.

1. /agents/<attachment_id>/download ->
/agents/attachments/<attachment_id>/download

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-25 17:11:24 +08:00