feat(connector): implement Outlook data source connector (issue #15332) (#15333)

### What problem does this PR solve?

Closes #15332.

RAGFlow can index Gmail and generic IMAP mailboxes but had no native
connector for Outlook / Microsoft 365 mail. Organisations on Microsoft
365 had no way to bring mailbox content into a knowledge base through
Microsoft Graph.

This PR adds a net-new Outlook data source that:

- Authenticates against Microsoft Graph with the same MSAL
client-credentials flow already used by the SharePoint and Teams
  connectors (no new auth primitives).
- Pages over `/users/{id}/mailFolders/{folder}/messages/delta` per
mailbox and persists `@odata.deltaLink` values in
`OutlookCheckpoint.delta_links`, so incremental syncs only fetch changed
messages.
- Supports two scoping modes:
- **Tenant-wide** (default): enumerates every user in the tenant via
`/users` and syncs each mailbox. Requires `User.Read.All`.
- **Targeted**: when `user_ids` is provided (comma-separated UPNs or
object IDs), only those mailboxes are synced. `User.Read.All` is not
needed in this mode.
- Lets the caller pick the mail folder (`inbox`, `sentitems`, `archive`,
...). Defaults to `inbox`.
- Maps each message to a `Document` shaped after the Gmail connector:
one `TextSection` carrying `From/To/Cc/Subject` headers + body, with
HTML bodies stripped to text inline (no extra dependency).
- Surfaces typed errors on the validation probe:
401 → `ConnectorMissingCredentialError`, 403 →
`InsufficientPermissionsError` (with `Mail.Read` / `User.Read.All`
hint), 404 on a configured mailbox → `ConnectorValidationError`, 5xx →
`UnexpectedValidationError`.
- Skips messages flagged `@removed` by the delta semantics and messages
whose `receivedDateTime` is older than `poll_range_start`.

#### Files

| File | Change |
|------|--------|
| `common/data_source/outlook_connector.py` | **New** —
`OutlookConnector` (`CheckpointedConnectorWithPermSync` +
`SlimConnectorWithPermSync`) + `OutlookCheckpoint` + tiny `_strip_html`
helper. |
| `common/data_source/config.py` | `DocumentSource.OUTLOOK = "outlook"`.
|
| `common/constants.py` | `FileSource.OUTLOOK = "outlook"`. |
| `common/data_source/__init__.py` | Export `OutlookConnector`. |
| `rag/svr/sync_data_source.py` | `Outlook(SyncBase)` with `batch_size`
normalisation, CSV/list parsing of `user_ids`; registered in
`func_factory`. |
| `web/src/pages/user-setting/data-source/constant/index.tsx` |
`DataSourceKey.OUTLOOK`, visibility map (`syncDeletedFiles: true`), info
entry, form fields (tenant_id, client_id, client_secret, folder,
user_ids, batch_size), default values. |
| `web/src/locales/en.ts`, `web/src/locales/zh.ts` |
`outlookDescription` + 5 tooltip keys (EN + ZH). |
| `test/unit_test/data_source/test_outlook_connector_unit.py` | **New**
— 19 unit tests (`p1`/`p2`/`p3`) covering auth, validation (tenant-wide
vs specific user vs error paths), checkpoint helpers, user enumeration
pagination, message filtering, HTML body stripping. |

#### Required Azure AD permissions

- `Mail.Read` (Application, admin-granted) — always.
- `User.Read.All` (Application, admin-granted) — only when `user_ids` is
left blank so the connector can enumerate mailboxes.

#### Out of scope

- **Attachment indexing.** The current connector emits message body +
headers; binary attachments are flagged via `metadata.has_attachments`
but not pulled. Adding attachment hydration is straightforward but
scoped out per the issue's "decide whether attachments are indexed in
the first version" note.
- **Delegated (per-user) OAuth.** The connector uses app-only
credentials, consistent with the SharePoint / Teams precedent in this
codebase.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
web-dev0521
2026-05-29 07:52:29 -06:00
committed by GitHub
parent 11af34a895
commit cd18cfab79
9 changed files with 1185 additions and 3 deletions

View File

@@ -1395,6 +1395,18 @@ Example: Virtual Hosted Style`,
'Client secret value generated in the Azure AD app registration.',
onedriveFolderPathTip:
'Optional sub-folder path to limit indexing (e.g. /Documents/Reports). Leave blank to index the entire drive.',
outlookDescription:
'Connect Outlook / Microsoft 365 mailboxes and index messages via Microsoft Graph delta queries.',
outlookTenantIdTip:
'Azure Active Directory tenant ID (Directory ID) of the Microsoft 365 organisation.',
outlookClientIdTip:
'Application (client) ID of the Azure AD app registration with Mail.Read permission.',
outlookClientSecretTip:
'Client secret value generated in the Azure AD app registration.',
outlookFolderTip:
'Mail folder to sync (e.g. inbox, sentitems, archive). Defaults to inbox.',
outlookUserIdsTip:
'Comma-separated UPNs or object IDs of mailboxes to sync. Leave blank to sync every mailbox in the tenant (requires User.Read.All).',
restApiQueryParamsTip:
'Key=value pairs (one per line) sent as URL query parameters. Use this instead of embedding params in the URL.',
restApiHeadersTip:

View File

@@ -1099,12 +1099,26 @@ NER使用 spaCy NER 和基于规则的关键词提取来抽取实体和关系
gmailTokenTip:
'请上传由 Google Console 生成的 OAuth JSON。如果仅包含 client credentials请通过浏览器授权一次以获取长期有效的刷新 Token。',
dropboxDescription: '连接 Dropbox同步指定账号下的文件与文件夹。',
onedriveDescription: '连接 OneDrive 或 OneDrive for Business通过 Microsoft Graph delta 查询索引文件和文件夹。',
onedriveDescription:
'连接 OneDrive 或 OneDrive for Business通过 Microsoft Graph delta 查询索引文件和文件夹。',
onedriveTenantIdTip: 'Microsoft 365 组织的 Azure Active Directory 租户 ID目录 ID。',
onedriveClientIdTip: '拥有 Files.Read.All 权限的 Azure AD 应用注册的应用程序客户端ID。',
onedriveClientSecretTip: '在 Azure AD 应用注册中生成的客户端密钥值。',
onedriveFolderPathTip: '可选的子文件夹路径,用于限制索引范围(例如 /Documents/Reports。留空则索引整个云盘。',
teamsDescription: '通过 Microsoft Graph 连接 Microsoft Teams同步频道帖子与回复。',
onedriveFolderPathTip:
'可选的子文件夹路径,用于限制索引范围(例如 /Documents/Reports。留空则索引整个云盘。',
outlookDescription:
'连接 Outlook / Microsoft 365 邮箱,通过 Microsoft Graph delta 查询索引邮件。',
outlookTenantIdTip:
'Microsoft 365 组织的 Azure Active Directory 租户 ID目录 ID。',
outlookClientIdTip:
'拥有 Mail.Read 权限的 Azure AD 应用注册的应用程序客户端ID。',
outlookClientSecretTip: '在 Azure AD 应用注册中生成的客户端密钥值。',
outlookFolderTip:
'要同步的邮件文件夹(例如 inbox、sentitems、archive默认为 inbox。',
outlookUserIdsTip:
'要同步的邮箱 UPN 或对象 ID 列表(逗号分隔)。留空则同步租户内的所有邮箱(需要 User.Read.All 权限)。',
teamsDescription:
'通过 Microsoft Graph 连接 Microsoft Teams同步频道帖子与回复。',
teamsTenantIdTip:
'Azure AD 租户 ID。需要具备 Team.ReadBasic.All 与 ChannelMessage.Read.All 应用权限(管理员同意)的应用。',
slackDescription: '连接你的 Slack 工作区,同步频道消息与讨论串。',

View File

@@ -44,6 +44,7 @@ export enum DataSourceKey {
REST_API = 'rest_api',
RSS = 'rss',
ONEDRIVE = 'onedrive',
OUTLOOK = 'outlook',
TEAMS = 'teams',
SLACK = 'slack',
SHAREPOINT = 'sharepoint',
@@ -133,6 +134,9 @@ export const DataSourceFeatureVisibilityMap: Partial<
[DataSourceKey.ONEDRIVE]: {
syncDeletedFiles: true,
},
[DataSourceKey.OUTLOOK]: {
syncDeletedFiles: true,
},
[DataSourceKey.TEAMS]: {
syncDeletedFiles: true,
},
@@ -326,6 +330,11 @@ export const generateDataSourceInfo = (t: TFunction) => {
description: t(`setting.${DataSourceKey.ONEDRIVE}Description`),
icon: <SvgIcon name={'data-source/onedrive'} width={38} />,
},
[DataSourceKey.OUTLOOK]: {
name: 'Outlook',
description: t(`setting.${DataSourceKey.OUTLOOK}Description`),
icon: <Mail className="text-text-primary" size={22} />,
},
};
};
@@ -455,6 +464,57 @@ export const DataSourceFormFields = {
},
},
],
[DataSourceKey.OUTLOOK]: [
{
label: 'Tenant ID',
name: 'config.credentials.tenant_id',
type: FormFieldType.Text,
required: true,
placeholder: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
tooltip: t('setting.outlookTenantIdTip'),
},
{
label: 'Client ID',
name: 'config.credentials.client_id',
type: FormFieldType.Text,
required: true,
placeholder: 'xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx',
tooltip: t('setting.outlookClientIdTip'),
},
{
label: 'Client Secret',
name: 'config.credentials.client_secret',
type: FormFieldType.Password,
required: true,
tooltip: t('setting.outlookClientSecretTip'),
},
{
label: 'Mail Folder',
name: 'config.folder',
type: FormFieldType.Text,
required: false,
placeholder: 'inbox',
tooltip: t('setting.outlookFolderTip'),
},
{
label: 'Mailbox User IDs (optional)',
name: 'config.user_ids',
type: FormFieldType.Text,
required: false,
placeholder: 'support@example.com, sales@example.com',
tooltip: t('setting.outlookUserIdsTip'),
},
{
label: 'Batch Size',
name: 'config.batch_size',
type: FormFieldType.Number,
required: false,
validation: {
min: 1,
message: 'Batch Size must be at least 1',
},
},
],
[DataSourceKey.RSS]: [
{
label: 'Feed URL',
@@ -1908,6 +1968,20 @@ export const DataSourceFormDefaultValues = {
},
},
},
[DataSourceKey.OUTLOOK]: {
name: '',
source: DataSourceKey.OUTLOOK,
config: {
folder: 'inbox',
user_ids: '',
batch_size: 2,
credentials: {
tenant_id: '',
client_id: '',
client_secret: '',
},
},
},
[DataSourceKey.REST_API]: {
name: '',
source: DataSourceKey.REST_API,