feat: implement Slack data source connector (#15188)

### What problem does this PR solve?

Closes #15187.

RAGFlow shipped a Slack connector
(`common/data_source/slack_connector.py`) but it was never usable:
`Slack._generate()` in the sync worker was a `pass` stub, the
connector's document-generating code was incompatible with the current
data model,
and Slack was commented out of the data-source settings UI. As a result,
teams had no way to index Slack channels/threads into a knowledge base.

This PR completes the connector end to end.

**Backend**

- `common/data_source/slack_connector.py`
- Rewrote `thread_to_doc` to produce a blob-based `Document`
(`extension`/`blob`/`size_bytes`). The previous implementation built the
doc with a `sections=[...]` argument and omitted the now-required
`blob`/`extension`/ `size_bytes` fields, so it raised a validation error
against the current `Document` model. Thread messages are now cleaned
and flattened into a single UTF-8 text blob.
- Added `load_from_state()` / `poll_source(start, end)` generators. The
connector's checkpoint interface is a no-op stub, so both full and
incremental syncs run through a single channel-iterating generator built
on the existing module helpers (`get_channels`, `filter_channels`,
`get_channel_messages`, `_process_message`), with per-channel thread
de-duplication.
- `rag/svr/sync_data_source.py`
- Implemented `Slack._generate()`. Credentials are loaded via
`StaticCredentialsProvider` (the connector requires `slack_bot_token`
and does not support `load_credentials`). Supports full reindex and
incremental polling from `poll_range_start`, plus the optional channel
filter. Modeled on the Confluence/Dropbox wrappers.
- `SlackConnector` was already exported from
`common/data_source/__init__.py`.

**Frontend (`web/`)**

- Enabled the `SLACK` data-source enum and added its form fields (Slack
bot token + optional channel filter), default values, display metadata,
and a Slack icon.
- Added `slackDescription` / `slackBotTokenTip` / `slackChannelsTip`
strings to `en.ts` and `zh.ts`.

**Tests**

- `test/unit_test/data_source/test_slack_connector_unit.py`: unit tests
covering credential loading (`load_credentials` raises,
`set_credentials_provider` initializes clients, missing credentials
raises) and document generation (standalone message + flattened thread,
blob/extension/size_bytes/metadata, and the incremental poll time
window). All 5 pass; `ruff check` is clean.

Required Slack scopes: `channels:read`, `channels:history`,
`users:read`.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
web-dev0521
2026-05-28 01:46:07 -06:00
committed by GitHub
parent 7e83643536
commit 5de021ebb4
8 changed files with 428 additions and 19 deletions

View File

@@ -0,0 +1,6 @@
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 122.8 122.8" width="122.8" height="122.8">
<path d="M25.8 77.6c0 7.1-5.8 12.9-12.9 12.9S0 84.7 0 77.6s5.8-12.9 12.9-12.9h12.9v12.9zm6.5 0c0-7.1 5.8-12.9 12.9-12.9s12.9 5.8 12.9 12.9v32.3c0 7.1-5.8 12.9-12.9 12.9s-12.9-5.8-12.9-12.9V77.6z" fill="#E01E5A"/>
<path d="M45.2 25.8c-7.1 0-12.9-5.8-12.9-12.9S38.1 0 45.2 0s12.9 5.8 12.9 12.9v12.9H45.2zm0 6.5c7.1 0 12.9 5.8 12.9 12.9s-5.8 12.9-12.9 12.9H12.9C5.8 58.1 0 52.3 0 45.2s5.8-12.9 12.9-12.9h32.3z" fill="#36C5F0"/>
<path d="M97 45.2c0-7.1 5.8-12.9 12.9-12.9s12.9 5.8 12.9 12.9-5.8 12.9-12.9 12.9H97V45.2zm-6.5 0c0 7.1-5.8 12.9-12.9 12.9s-12.9-5.8-12.9-12.9V12.9C64.7 5.8 70.5 0 77.6 0s12.9 5.8 12.9 12.9v32.3z" fill="#2EB67D"/>
<path d="M77.6 97c7.1 0 12.9 5.8 12.9 12.9s-5.8 12.9-12.9 12.9-12.9-5.8-12.9-12.9V97h12.9zm0-6.5c-7.1 0-12.9-5.8-12.9-12.9s5.8-12.9 12.9-12.9h32.3c7.1 0 12.9 5.8 12.9 12.9s-5.8 12.9-12.9 12.9H77.6z" fill="#ECB22E"/>
</svg>

After

Width:  |  Height:  |  Size: 965 B

View File

@@ -1238,6 +1238,12 @@ Example: Virtual Hosted Style`,
'Upload the OAuth JSON generated from Google Console. If it only contains client credentials, run the browser-based verification once to mint long-lived refresh tokens.',
dropboxDescription:
'Connect your Dropbox to sync files and folders from a chosen account.',
slackDescription:
'Connect your Slack workspace to sync channel messages and threads.',
slackBotTokenTip:
'Slack bot user OAuth token (starts with xoxb-). The app needs the channels:read, channels:history, and users:read scopes.',
slackChannelsTip:
'Optional: channel names to sync (e.g., general). Leave empty to sync all accessible channels.',
sharepointDescription:
'Connect a SharePoint site via Microsoft Graph to sync its document libraries.',
sharepointSiteUrlTip:

View File

@@ -1099,6 +1099,11 @@ NER使用 spaCy NER 和基于规则的关键词提取来抽取实体和关系
gmailTokenTip:
'请上传由 Google Console 生成的 OAuth JSON。如果仅包含 client credentials请通过浏览器授权一次以获取长期有效的刷新 Token。',
dropboxDescription: '连接 Dropbox同步指定账号下的文件与文件夹。',
slackDescription: '连接你的 Slack 工作区,同步频道消息与讨论串。',
slackBotTokenTip:
'Slack 机器人用户 OAuth Token以 xoxb- 开头)。应用需具备 channels:read、channels:history 和 users:read 权限。',
slackChannelsTip:
'可选:需要同步的频道名称(例如 general。留空则同步所有可访问的频道。',
sharepointDescription: '通过 Microsoft Graph 连接 SharePoint 站点,同步其文档库。',
sharepointSiteUrlTip:
'要索引的 SharePoint 站点完整 URL例如 https://contoso.sharepoint.com/sites/MySite。需要具备 Sites.Read.All 与 Files.Read.All 应用权限(管理员同意)的 Azure AD 应用。',

View File

@@ -43,9 +43,9 @@ export enum DataSourceKey {
POSTGRESQL = 'postgresql',
REST_API = 'rest_api',
RSS = 'rss',
SLACK = 'slack',
SHAREPOINT = 'sharepoint',
// SLACK = 'slack',
// TEAMS = 'teams',
}
@@ -130,6 +130,12 @@ export const DataSourceFeatureVisibilityMap: Partial<
[DataSourceKey.MOODLE]: {
syncDeletedFiles: true,
},
[DataSourceKey.SLACK]: {
syncDeletedFiles: true,
},
[DataSourceKey.SHAREPOINT]: {
syncDeletedFiles: true,
},
[DataSourceKey.MYSQL]: {
syncDeletedFiles: true,
},
@@ -213,6 +219,11 @@ export const generateDataSourceInfo = (t: TFunction) => {
description: t(`setting.${DataSourceKey.MOODLE}Description`),
icon: <SvgIcon name={'data-source/moodle'} width={38} />,
},
[DataSourceKey.SLACK]: {
name: 'Slack',
description: t(`setting.${DataSourceKey.SLACK}Description`),
icon: <SvgIcon name={'data-source/slack'} width={38} />,
},
[DataSourceKey.SHAREPOINT]: {
name: 'SharePoint',
description: t(`setting.${DataSourceKey.SHAREPOINT}Description`),
@@ -659,6 +670,22 @@ export const DataSourceFormFields = {
required: true,
},
],
[DataSourceKey.SLACK]: [
{
label: 'Slack Bot Token',
name: 'config.credentials.slack_bot_token',
type: FormFieldType.Password,
required: true,
tooltip: t('setting.slackBotTokenTip'),
},
{
label: 'Channels',
name: 'config.channels',
type: FormFieldType.Tag,
required: false,
tooltip: t('setting.slackChannelsTip'),
},
],
[DataSourceKey.SHAREPOINT]: [
{
label: 'Site URL',
@@ -1542,6 +1569,16 @@ export const DataSourceFormDefaultValues = {
},
},
},
[DataSourceKey.SLACK]: {
name: '',
source: DataSourceKey.SLACK,
config: {
channels: [],
credentials: {
slack_bot_token: '',
},
},
},
[DataSourceKey.SHAREPOINT]: {
name: '',
source: DataSourceKey.SHAREPOINT,