mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-07-16 12:47:19 +08:00
feat(raptor): add Psi tree builder with original-space ranking and safe migration (#14679)
### What problem does this PR solve? Closes #14674. This PR improves RAPTOR configuration and tree construction while preserving the existing RAPTOR behavior as the default. RAPTOR currently builds summary layers with the original UMAP + GMM clustering path. This PR keeps that default path, and adds: - A hidden backend tree-builder option: - `tree_builder="raptor"`: default, existing RAPTOR behavior. - `tree_builder="psi"`: rank-aware Psi-style tree builder using original embedding-space cosine ranking. - A user-facing clustering method option for the default RAPTOR builder: - `clustering_method="gmm"`: existing default. - `clustering_method="ahc"`: agglomerative hierarchical clustering path. - A RAPTOR UI setting for `Clustering method` and `Max cluster`. ### What changed #### Backend - Added `tree_builder` support for RAPTOR/Psi. - Added `clustering_method` support for GMM/AHC. - Kept existing RAPTOR + GMM as the default. - Added Psi tree building from original-space cosine similarity. - Added bucketed Psi building controls for large inputs: - `raptor.ext.psi_exact_max_leaves` - `raptor.ext.psi_bucket_size` - Added method-aware RAPTOR summary metadata using existing `extra.raptor_method`. - Avoided adding a dedicated DB schema field for experimental method tracking. - Added cleanup/migration logic to avoid mixing stale RAPTOR summary trees. - Added defensive checks for Psi tree construction and summary failures. #### Frontend/UI - Added `Clustering method` in RAPTOR settings with `GMM` and `AHC`. - Added/kept `Max cluster` in RAPTOR settings. - Enlarged max cluster UI limit to `1024`, matching backend validation. - Kept AHC editable even when a RAPTOR task has already finished. - Fixed the UI save payload so `clustering_method` and `tree_builder` are serialized through `parser_config.raptor.ext`, avoiding backend validation errors for extra top-level RAPTOR fields. Example saved RAPTOR config: ```json { "raptor": { "max_cluster": 317, "ext": { "clustering_method": "ahc", "tree_builder": "raptor" } } } Co-authored-by: CaptainTimon <CaptainTimon@users.noreply.github.com>
This commit is contained in:
@@ -861,6 +861,11 @@ The above is the content you need to summarize.`,
|
||||
thresholdTip:
|
||||
'In RAPTOR, chunks are clustered by their semantic similarity. The Threshold parameter sets the minimum similarity required for chunks to be grouped together. A higher Threshold means fewer chunks in each cluster, while a lower one means more.',
|
||||
thresholdMessage: 'Threshold is required',
|
||||
clusteringMethod: 'Clustering method',
|
||||
clusteringMethodTip:
|
||||
'Select the RAPTOR clustering method. AHC can use a larger max cluster value, but may require more memory on large inputs.',
|
||||
clusteringMethodGmm: 'GMM',
|
||||
clusteringMethodAhc: 'AHC',
|
||||
maxCluster: 'Max cluster',
|
||||
maxClusterTip: 'The maximum number of clusters to create.',
|
||||
maxClusterMessage: 'Max cluster is required',
|
||||
|
||||
@@ -772,6 +772,11 @@ export default {
|
||||
maxTokenMessage: '最大token数是必填项',
|
||||
threshold: '阈值',
|
||||
thresholdMessage: '阈值是必填项',
|
||||
clusteringMethod: '聚类方法',
|
||||
clusteringMethodTip:
|
||||
'选择 RAPTOR 聚类方法。AHC 可以使用更大的最大聚类数,但在大规模输入时可能占用更多内存。',
|
||||
clusteringMethodGmm: 'GMM',
|
||||
clusteringMethodAhc: 'AHC',
|
||||
maxCluster: '最大聚类数',
|
||||
maxClusterMessage: '最大聚类数是必填项',
|
||||
randomSeed: '随机种子',
|
||||
|
||||
Reference in New Issue
Block a user