mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-07-17 05:07:23 +08:00
feat(connectors): add Salesforce CRM data source connector (#15462)
### What problem does this PR solve? Closes #15461. RAGFlow had no way to ingest Salesforce CRM data, so support / sales teams couldn't ground responses on live Accounts, Contacts, Opportunities, Cases, or Knowledge articles. This adds a first-class Salesforce data source connector that authenticates against a Connected App via OAuth 2.0 client-credentials, queries selected SObjects via SOQL, and turns each record into an indexable document with incremental sync. **Highlights** - `common/data_source/salesforce_connector.py`: new `SalesforceConnector` (`CheckpointedConnectorWithPermSync` + `SlimConnectorWithPermSync`). - OAuth 2.0 client-credentials flow; canonical `instance_url` from the token response so multi-pod orgs route correctly. - Per-object `SystemModstamp` cursor stored in `SalesforceCheckpoint.cursors` — a failure mid-object doesn't rewind sibling objects, and re-syncs only fetch changed rows. - Deterministic record-to-text formatter (sorted keys) so SOQL field reordering on the server doesn't mark every row "changed" on each poll. - `_get_json` raises on non-2xx so 429 / 5xx never silently advance the checkpoint past missing data. - `Knowledge__kav` is in the default object set but is skipped silently when the org doesn't have Salesforce Knowledge enabled (404 on describe). - Slim-doc IDs are scoped as `<Object>/<Id>` so prune deletes can't collide across object types. - `common/constants.py`, `common/data_source/config.py`, `common/data_source/__init__.py`: register `salesforce` in `FileSource` / `DocumentSource` and export `SalesforceConnector`. - `rag/svr/sync_data_source.py`: new `Salesforce(SyncBase)` class routed through `load_from_checkpoint` (poll_source would re-walk every object each run) and added to `func_factory`. - Frontend: - `web/src/pages/user-setting/data-source/constant/index.tsx`: new `DataSourceKey.SALESFORCE`, form fields (instance URL, client ID/secret, objects, api_version, batch size), `syncDeletedFiles` capability, default form values, and tile entry with the new icon. - `web/src/locales/{en,zh}.ts`: description + per-field tooltips. - `web/src/assets/svg/data-source/salesforce.svg`: 48x48 brand-style icon to match the other Microsoft / cloud tiles. **Verification** - `npm run build` (vite + esbuild) passes (1m 26s). ### Type of change - [x] New Feature (non-breaking change which adds functionality)
This commit is contained in:
4
web/src/assets/svg/data-source/salesforce.svg
Normal file
4
web/src/assets/svg/data-source/salesforce.svg
Normal file
@@ -0,0 +1,4 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" width="48" height="48">
|
||||
<path d="M20.5 13.5a7 7 0 0 1 11.7 1.6 6 6 0 0 1 2.4-.5 6.5 6.5 0 0 1 5.7 9.6 6 6 0 0 1-1.7 10.7 6 6 0 0 1-8 3.6 6.5 6.5 0 0 1-11 .7 6 6 0 0 1-9-5.4 6 6 0 0 1 .6-2.6A6.5 6.5 0 0 1 14.6 19a7 7 0 0 1 5.9-5.5z" fill="#00A1E0"/>
|
||||
<path d="M19 22.5h2.5v8H19zM22.5 22.5h2.4l3 5.3v-5.3h2.4v8h-2.4l-3-5.3v5.3h-2.4zM31 22.5h5v2h-3v1.2h2.8v1.7H33V29h3v1.5h-5z" fill="#FFFFFF"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 462 B |
@@ -1407,6 +1407,18 @@ Example: Virtual Hosted Style`,
|
||||
'Mail folder to sync (e.g. inbox, sentitems, archive). Defaults to inbox.',
|
||||
outlookUserIdsTip:
|
||||
'Comma-separated UPNs or object IDs of mailboxes to sync. Leave blank to sync every mailbox in the tenant (requires User.Read.All).',
|
||||
salesforceDescription:
|
||||
'Connect a Salesforce org and index CRM records (Accounts, Contacts, Opportunities, Cases, Knowledge articles) via SOQL with incremental sync.',
|
||||
salesforceInstanceUrlTip:
|
||||
'Salesforce org URL, e.g. https://your-domain.my.salesforce.com (no trailing slash).',
|
||||
salesforceClientIdTip:
|
||||
'Consumer Key of a Connected App with Client Credentials Flow enabled and the api scope.',
|
||||
salesforceClientSecretTip:
|
||||
'Consumer Secret of the Connected App used for client-credentials authentication.',
|
||||
salesforceObjectsTip:
|
||||
'Comma-separated SObject API names to index. Defaults to Account, Contact, Opportunity, Case, Knowledge__kav.',
|
||||
salesforceApiVersionTip:
|
||||
'Salesforce REST API version (e.g. v59.0). Use the version your org supports.',
|
||||
azure_blobDescription:
|
||||
'Index blobs from an Azure Blob Storage container into a knowledge base. Supports account-key, connection-string, and SAS-token auth. Unchanged blobs are skipped via ETag fingerprinting.',
|
||||
azureBlobAuthModeTip:
|
||||
|
||||
@@ -1119,6 +1119,18 @@ NER:使用 spaCy NER 和基于规则的关键词提取来抽取实体和关系
|
||||
'要同步的邮件文件夹(例如 inbox、sentitems、archive),默认为 inbox。',
|
||||
outlookUserIdsTip:
|
||||
'要同步的邮箱 UPN 或对象 ID 列表(逗号分隔)。留空则同步租户内的所有邮箱(需要 User.Read.All 权限)。',
|
||||
salesforceDescription:
|
||||
'连接 Salesforce 组织,通过 SOQL 增量同步并索引 CRM 记录(客户、联系人、商机、个案、知识文章)。',
|
||||
salesforceInstanceUrlTip:
|
||||
'Salesforce 组织地址,例如 https://your-domain.my.salesforce.com(不要包含末尾斜杠)。',
|
||||
salesforceClientIdTip:
|
||||
'已启用 Client Credentials Flow 且包含 api 权限的 Connected App 的 Consumer Key。',
|
||||
salesforceClientSecretTip:
|
||||
'用于客户端凭证身份验证的 Connected App 的 Consumer Secret。',
|
||||
salesforceObjectsTip:
|
||||
'要索引的 SObject API 名称列表(逗号分隔)。默认值:Account, Contact, Opportunity, Case, Knowledge__kav。',
|
||||
salesforceApiVersionTip:
|
||||
'Salesforce REST API 版本(例如 v59.0),请使用您组织所支持的版本。',
|
||||
azure_blobDescription:
|
||||
'将 Azure Blob 存储容器中的文件索引到知识库。支持账户密钥、连接字符串和 SAS 令牌三种认证方式,通过 ETag 指纹跳过未变更的文件。',
|
||||
azureBlobAuthModeTip:
|
||||
|
||||
@@ -45,6 +45,7 @@ export enum DataSourceKey {
|
||||
RSS = 'rss',
|
||||
ONEDRIVE = 'onedrive',
|
||||
OUTLOOK = 'outlook',
|
||||
SALESFORCE = 'salesforce',
|
||||
AZURE_BLOB = 'azure_blob',
|
||||
TEAMS = 'teams',
|
||||
SLACK = 'slack',
|
||||
@@ -138,6 +139,9 @@ export const DataSourceFeatureVisibilityMap: Partial<
|
||||
[DataSourceKey.OUTLOOK]: {
|
||||
syncDeletedFiles: true,
|
||||
},
|
||||
[DataSourceKey.SALESFORCE]: {
|
||||
syncDeletedFiles: true,
|
||||
},
|
||||
[DataSourceKey.AZURE_BLOB]: {
|
||||
syncDeletedFiles: true,
|
||||
},
|
||||
@@ -339,6 +343,11 @@ export const generateDataSourceInfo = (t: TFunction) => {
|
||||
description: t(`setting.${DataSourceKey.OUTLOOK}Description`),
|
||||
icon: <Mail className="text-text-primary" size={22} />,
|
||||
},
|
||||
[DataSourceKey.SALESFORCE]: {
|
||||
name: 'Salesforce',
|
||||
description: t(`setting.${DataSourceKey.SALESFORCE}Description`),
|
||||
icon: <SvgIcon name={'data-source/salesforce'} width={38} />,
|
||||
},
|
||||
[DataSourceKey.AZURE_BLOB]: {
|
||||
name: 'Azure Blob Storage',
|
||||
description: t(`setting.${DataSourceKey.AZURE_BLOB}Description`),
|
||||
@@ -524,6 +533,65 @@ export const DataSourceFormFields = {
|
||||
},
|
||||
},
|
||||
],
|
||||
[DataSourceKey.SALESFORCE]: [
|
||||
{
|
||||
label: 'Instance URL',
|
||||
name: 'config.credentials.instance_url',
|
||||
type: FormFieldType.Text,
|
||||
required: true,
|
||||
placeholder: 'https://your-domain.my.salesforce.com',
|
||||
tooltip: t('setting.salesforceInstanceUrlTip'),
|
||||
validation: {
|
||||
pattern: /^https:\/\/[a-zA-Z0-9.-]+\.salesforce\.com$/,
|
||||
message:
|
||||
'Must be a valid Salesforce domain (https://...salesforce.com)',
|
||||
},
|
||||
},
|
||||
{
|
||||
label: 'Client ID',
|
||||
name: 'config.credentials.client_id',
|
||||
type: FormFieldType.Text,
|
||||
required: true,
|
||||
tooltip: t('setting.salesforceClientIdTip'),
|
||||
},
|
||||
{
|
||||
label: 'Client Secret',
|
||||
name: 'config.credentials.client_secret',
|
||||
type: FormFieldType.Password,
|
||||
required: true,
|
||||
tooltip: t('setting.salesforceClientSecretTip'),
|
||||
},
|
||||
{
|
||||
label: 'Objects',
|
||||
name: 'config.objects',
|
||||
type: FormFieldType.Text,
|
||||
required: false,
|
||||
placeholder: 'Account,Contact,Opportunity,Case,Knowledge__kav',
|
||||
tooltip: t('setting.salesforceObjectsTip'),
|
||||
},
|
||||
{
|
||||
label: 'API Version',
|
||||
name: 'config.api_version',
|
||||
type: FormFieldType.Text,
|
||||
required: false,
|
||||
placeholder: 'v59.0',
|
||||
tooltip: t('setting.salesforceApiVersionTip'),
|
||||
validation: {
|
||||
pattern: /^v\d+\.\d+$/,
|
||||
message: 'API version must match format like v59.0',
|
||||
},
|
||||
},
|
||||
{
|
||||
label: 'Batch Size',
|
||||
name: 'config.batch_size',
|
||||
type: FormFieldType.Number,
|
||||
required: false,
|
||||
validation: {
|
||||
min: 1,
|
||||
message: 'Batch Size must be at least 1',
|
||||
},
|
||||
},
|
||||
],
|
||||
[DataSourceKey.AZURE_BLOB]: [
|
||||
{
|
||||
label: 'Auth Mode',
|
||||
@@ -2109,6 +2177,20 @@ export const DataSourceFormDefaultValues = {
|
||||
},
|
||||
},
|
||||
},
|
||||
[DataSourceKey.SALESFORCE]: {
|
||||
name: '',
|
||||
source: DataSourceKey.SALESFORCE,
|
||||
config: {
|
||||
objects: '',
|
||||
api_version: 'v59.0',
|
||||
batch_size: 2,
|
||||
credentials: {
|
||||
instance_url: '',
|
||||
client_id: '',
|
||||
client_secret: '',
|
||||
},
|
||||
},
|
||||
},
|
||||
[DataSourceKey.AZURE_BLOB]: {
|
||||
name: '',
|
||||
source: DataSourceKey.AZURE_BLOB,
|
||||
|
||||
Reference in New Issue
Block a user