Commit Graph

35 Commits

Author SHA1 Message Date
Jack
c81081f8ef Refactor: Doc change parser (#14327)
### What problem does this PR solve?

Before migration
Web API: POST /v1/document/change_parser
HTTP API: PATCH /api/v1/datasets/<dataset_id>/documents

After consolidation, Restful API
PATCH /api/v1/datasets/<dataset_id>/documents

### Type of change

- [x] Refactoring
2026-04-27 23:42:57 +08:00
euvre
4dcc42e0e1 feat(api): add unified index API and dataset management endpoints (#14222)
### What problem does this PR solve?

## Summary

Refactor the dataset API layer into a clean service/REST separation
pattern, add a unified `/index` API for graph/raptor/mindmap operations,
and introduce several new dataset management endpoints with full test
coverage.

## Changes

### Service Layer (`dataset_api_service.py`)

- Added `trace_index(dataset_id, tenant_id, index_type)` — unified trace
function for all index types
- Added `run_index`, `delete_index` service functions
- Added `get_dataset`, `get_ingestion_summary`, `list_ingestion_logs`,
`get_ingestion_log`
- Added `run_embedding`, `list_tags`, `aggregate_tags`, `delete_tags`,
`rename_tag`
- Added `get_flattened_metadata`, `get_auto_metadata`,
`update_auto_metadata`

### REST API Layer (`dataset_api.py`)

**New unified routes:**

| Method | Route | Description |
|--------|-------|-------------|
| POST | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Run index
task |
| GET | `/datasets/<id>/index?type=graph\|raptor\|mindmap` | Trace index
task |
| DELETE | `/datasets/<id>/<index_type>` | Delete index |
| GET | `/datasets/<id>` | Get dataset details |
| GET | `/datasets/<id>/ingestions/summary` | Ingestion summary |
| GET | `/datasets/<id>/ingestions` | List ingestion logs |
| GET | `/datasets/<id>/ingestions/<log_id>` | Get single ingestion log
|
| POST | `/datasets/<id>/embedding` | Run embedding |
| GET | `/datasets/<id>/tags` | List tags |
| GET | `/datasets/tags/aggregation` | Aggregate tags across datasets |
| DELETE | `/datasets/<id>/tags` | Delete tags |
| PUT | `/datasets/<id>/tags` | Rename tag |
| GET | `/datasets/metadata/flattened` | Get flattened metadata |
| GET/PUT | `/datasets/<id>/metadata/config` | New metadata config path
|

**Removed routes (replaced by unified `/index`):**

- `POST /datasets/<id>/mindmap`
- `GET /datasets/<id>/mindmap`

**Preserved legacy routes (backward compatibility):**

- `/run_graphrag`, `/trace_graphrag`, `/run_raptor`, `/trace_raptor`
- `/auto_metadata` GET/PUT

### Test Suite

- Updated `common.py` helpers: added `trace_index`, removed
`run_mindmap`/`trace_mindmap`
- Added 7 new test files with 39 test cases total:

| Test File | Cases |
|-----------|-------|
| `test_get_dataset.py` | 4 |
| `test_ingestion_summary.py` | 2 |
| `test_ingestion_logs.py` | 5 |
| `test_index_api.py` | 14 |
| `test_embedding.py` | 2 |
| `test_tags.py` | 8 |
| `test_flattened_metadata.py` | 4 |

- Deleted `test_mindmap_tasks.py` (covered by unified index tests)

## Design Decisions

1. **Unified `/index?type=...`** — single endpoint replaces 3 separate
route pairs for graph/raptor/mindmap
2. **Backward compatibility** — old routes (`/run_graphrag`,
`/run_raptor`, `/auto_metadata`) preserved alongside new paths
3. **`_VALID_INDEX_TYPES = {"graph", "raptor", "mindmap"}`** — input
validation via constant set
4. **`_INDEX_TYPE_TO_TASK_ID_FIELD`** — maps index type to KB model task
ID field for clean dispatch

## Files Changed

- `api/apps/restful_apis/dataset_api.py`
- `api/apps/services/dataset_api_service.py`
- `sdk/python/ragflow_sdk/modules/dataset.py`
- `test/testcases/test_http_api/common.py`
- `test/testcases/test_http_api/test_dataset_management/` (7 new files)
### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: noob <yixiao121314@outlook.com>
2026-04-27 09:38:01 +08:00
Liu An
6e33d8722f Revert "Fix: forwarding highlight param" (#14249)
Reverts infiniflow/ragflow#14112
2026-04-21 15:23:18 +08:00
Lynn
c3387cd5b8 Fix: parent child config (#14199)
### What problem does this PR solve?

Correctly set and display parent-child config in parser_config, and
allow to pass `tenant_id` in PATCH `/api/v1/chats`.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 23:02:42 +08:00
Daniil Sivak
22c6648348 Fix: forwarding highlight param (#14112)
Closes #9078

### What problem does this PR solve?

The `retrieval_test` endpoint in `chunk_app.py` never forwarded the
`highlight` request parameter to `retriever.retrieval()`, so the search
engine never produced highlight snippets. Additionally, the frontend
always rendered `content_with_weight` instead of preferring the
`highlight` field, and the CSS rule color `var(--accent-primary)` didn't
work because the variable stores an RGB triplet `(45,212,191)` requiring
the `rgb()` wrapper.

### Before

- Search page: displayed raw content_with_weight as a wall of plain
white text with no term highlighting, including markdown headings
rendered as literal text
- Retrieval testing page: showed `content_with_weight` in a plain `<p>`
tag, no `<em>` tags rendered, no highlight coloring
- Children chunks: when child chunks were consolidated into a parent via
`retrieval_by_children`, any highlight data from children was discarded
- TOC chunks: chunks fetched via `retrieval_by_toc` had no `highlight`
field, appearing as plain text while other chunks had highlights

**Retrieval testing**:
<img width="1449" height="1178"
alt="before-retrieval-no-highlight-cropped"
src="https://github.com/user-attachments/assets/5c6f5a5e-6c11-461a-bdb4-049d7dfb7a33"
/>

**Search**:
<img width="1378" height="711" alt="before-search-no-highlight-cropped"
src="https://github.com/user-attachments/assets/be7b5152-72ef-40da-a8fd-921e997ae7d3"
/>

### After

- Search page: displays the highlight field with search terms rendered
in teal/cyan color (`rgb(var(--accent-primary))`)
- Retrieval testing page: sends highlight: true in the request, uses
`HighLightMarkdown` component to render `<em>` tags with proper coloring
- Children chunks: highlights from child chunks are joined and preserved
on the parent
- TOC chunks: when other chunks have highlights, TOC-fetched chunks use
`content_with_weight` as a highlight fallback

**Retrieval testing**:
<img width="1410" height="1015" alt="05-retrieval-testing-results"
src="https://github.com/user-attachments/assets/f0cff8cf-0962-4320-b559-cd5037f622d2"
/>

**Search**:
<img width="1294" height="455" alt="03-search-highlight-results"
src="https://github.com/user-attachments/assets/a90e0e3e-3837-46be-8ddd-2412ff7cbc19"
/>

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 20:59:20 +08:00
Lynn
f194a09cd6 Fix: dataset update parent child (#14167)
### What problem does this PR solve?

Correctly set parent child config in parser_config.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-17 10:41:50 +08:00
balibabu
1bc4868abe Fix: The file count in the file header did not change after uploading or deleting files. (#14034)
### What problem does this PR solve?
Fix: The file count in the file header did not change after uploading or
deleting files.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-04-14 18:07:32 +08:00
chanx
6ffa566ec3 Refactor: Standardize naming convention to camelCase (#14079)
### What problem does this PR solve?

Refactor: Standardize naming convention to camelCase

### Type of change

- [x] Refactoring
2026-04-13 21:07:07 +08:00
balibabu
462be53b76 Fix: When creating a dataset, if no chunk_method is selected, there is no indication that this is a required field. (#14039)
### What problem does this PR solve?
Fix: When creating a dataset, if no `chunk_method` is selected, there is
no indication that this is a required field.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-10 19:05:14 +08:00
balibabu
0507463f4e Fix: The retrieval_test interface is continuously requested when the user enters a question. #13719 (#13720)
### What problem does this PR solve?

Fix: The retrieval_test interface is continuously requested when the
user enters a question. #13719

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-20 15:46:41 +08:00
Lynn
4bb1acaa5b Refactor: dataset / kb API to RESTFul style (#13690)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 14:41:36 +08:00
Jin Hai
986dcf1cc8 Revert "Refactor: dataset / kb API to RESTFul style" (#13646)
Reverts infiniflow/ragflow#13619
2026-03-17 12:09:48 +08:00
Lynn
1db5409d82 Refactor: dataset / kb API to RESTFul style (#13619)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-16 22:51:34 +08:00
Jin Hai
a2d72202cf Revert "Refactor dataset / kb API to RESTFul style" (#13614)
Reverts infiniflow/ragflow#13263
2026-03-16 10:44:38 +08:00
Lynn
7c32e206be Refactor dataset / kb API to RESTFul style (#13263)
### What problem does this PR solve?

1. Split dataset api to gateway and service, and modify web UI to use
restful http api.
2. Old KB releated APIs are commented.

### Type of change

- [x] Refactoring
2026-03-13 20:02:35 +08:00
chanx
8bce212284 Fix: error in retrieval testing page (#13225)
### What problem does this PR solve?

Fix: error in retrieval testing page

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-02-26 12:39:09 +08:00
chanx
a8a060676a Refactor: UmiJs -> Vite (#12410)
### What problem does this PR solve?

Refactor: UmiJs -> Vite+React

### Type of change

- [x] Refactoring

---------

Co-authored-by: Liu An <asiro@qq.com>
2026-01-04 19:14:20 +08:00
Yongteng Lei
0f0fb53256 Refa: refactor metadata filter (#11907)
### What problem does this PR solve?

Refactor metadata filter.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

---------

Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2025-12-12 17:12:38 +08:00
balibabu
ba6e2af5fd Feat: Delete useless request hooks. #10427 (#11659)
### What problem does this PR solve?

Feat: Delete useless request hooks. #10427

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-12-02 17:24:29 +08:00
balibabu
2078d88c28 Feat: Modify the translation file of the workflow #9869 (#10430)
### What problem does this PR solve?
Feat: Modify the translation file of the workflow #9869


### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-10-09 13:55:36 +08:00
Kevin Hu
cbf04ee470 Feat: Use data pipeline to visualize the parsing configuration of the knowledge base (#10423)
### What problem does this PR solve?

#9869

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: dependabot[bot] <support@github.com>
Signed-off-by: jinhai <haijin.chn@gmail.com>
Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: chanx <1243304602@qq.com>
Co-authored-by: balibabu <cike8899@users.noreply.github.com>
Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: 纷繁下的无奈 <zhileihuang@126.com>
Co-authored-by: huangzl <huangzl@shinemo.com>
Co-authored-by: writinwaters <93570324+writinwaters@users.noreply.github.com>
Co-authored-by: Wilmer <33392318@qq.com>
Co-authored-by: Adrian Weidig <adrianweidig@gmx.net>
Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Yongteng Lei <yongtengrey@outlook.com>
Co-authored-by: Liu An <asiro@qq.com>
Co-authored-by: buua436 <66937541+buua436@users.noreply.github.com>
Co-authored-by: BadwomanCraZY <511528396@qq.com>
Co-authored-by: cucusenok <31804608+cucusenok@users.noreply.github.com>
Co-authored-by: Russell Valentine <russ@coldstonelabs.org>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Billy Bao <newyorkupperbay@gmail.com>
Co-authored-by: Zhedong Cen <cenzhedong2@126.com>
Co-authored-by: TensorNull <129579691+TensorNull@users.noreply.github.com>
Co-authored-by: TensorNull <tensor.null@gmail.com>
Co-authored-by: TeslaZY <TeslaZY@outlook.com>
Co-authored-by: Ajay <160579663+aybanda@users.noreply.github.com>
Co-authored-by: AB <aj@Ajays-MacBook-Air.local>
Co-authored-by: 天海蒼灆 <huangaoqin@tecpie.com>
Co-authored-by: He Wang <wanghechn@qq.com>
Co-authored-by: Atsushi Hatakeyama <atu729@icloud.com>
Co-authored-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Mohamed Mathari <155896313+melmathari@users.noreply.github.com>
Co-authored-by: Mohamed Mathari <nocodeventure@Mac-mini-van-Mohamed.fritz.box>
Co-authored-by: Stephen Hu <stephenhu@seismic.com>
Co-authored-by: Shaun Zhang <zhangwfjh@users.noreply.github.com>
Co-authored-by: zhimeng123 <60221886+zhimeng123@users.noreply.github.com>
Co-authored-by: mxc <mxc@example.com>
Co-authored-by: Dominik Novotný <50611433+SgtMarmite@users.noreply.github.com>
Co-authored-by: EVGENY M <168018528+rjohny55@users.noreply.github.com>
Co-authored-by: mcoder6425 <mcoder64@gmail.com>
Co-authored-by: lemsn <lemsn@msn.com>
Co-authored-by: lemsn <lemsn@126.com>
Co-authored-by: Adrian Gora <47756404+adagora@users.noreply.github.com>
Co-authored-by: Womsxd <45663319+Womsxd@users.noreply.github.com>
Co-authored-by: FatMii <39074672+FatMii@users.noreply.github.com>
2025-10-09 12:36:19 +08:00
balibabu
fc56217eb3 Fix: The enterprise version of the knowledge graph cannot be displayed. #10333 (#10334)
### What problem does this PR solve?
Fix: The enterprise version of the knowledge graph cannot be displayed.
#10333
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-09-28 12:18:58 +08:00
chanx
cf18231713 Fix: Optimized the test results page layout and internationalization #3221 (#9974)
### What problem does this PR solve?

Fix: Optimized the test results page layout and internationalization

- Added an empty data component for when test results are empty
- Optimized internationalization support for the paging component
- Updated the layout and style of the test results page
- Added a tooltip for when test results are empty

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-09-08 12:49:12 +08:00
balibabu
5fe8cf6018 Feat: Use AvatarUpload to replace the avatar settings on the dataset and search pages #3221 (#9785)
### What problem does this PR solve?

Feat: Use AvatarUpload to replace the avatar settings on the dataset and
search pages #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-28 14:45:20 +08:00
balibabu
76118000c1 Feat: Allow chat to use meta data #3221 (#9393)
### What problem does this PR solve?

Feat:  Allow chat to use meta data #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-08-12 10:15:10 +08:00
chanx
dbc267758e Fix: Generate avatar; Add knowledge graph; Modify the style of the MultiSelect component (#8952)
### What problem does this PR solve?

Fix: Generate avatar; Add knowledge graph; Modify the style of the
multi-select component
[#3221](https://github.com/infiniflow/ragflow/issues/3221)

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-21 19:11:27 +08:00
chanx
ed8d7291ff Fix: Remove antd from dataset-page (#8830)
### What problem does this PR solve?

remove antd from dataset-page
[#3221](https://github.com/infiniflow/ragflow/issues/3221)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-15 16:12:50 +08:00
BlueYu-0221
fa3e90c72e Refactor: Datasets UI #3221 (#8349)
### What problem does this PR solve?

Refactor Datasets UI #3221.
### Type of change

- [X] New Feature (non-breaking change which adds functionality)
2025-06-19 16:40:30 +08:00
balibabu
8e87436725 Feat: Modify the Python language template code of the code operator #4977 (#7714)
### What problem does this PR solve?

Feat: Modify the Python language template code of the code operator
#4977
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-19 19:34:43 +08:00
balibabu
bfaa469b9a Feat: Rendering recall test page #3221 (#7689)
### What problem does this PR solve?

Feat: Rendering recall test page #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-05-16 18:56:48 +08:00
balibabu
bdebd1b2e3 Feat: Filter document by running status and file type. #3221 (#7340)
### What problem does this PR solve?
Feat: Filter document by running status and file type. #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-27 14:39:05 +08:00
balibabu
02cc867c06 Feat: Display the document configuration dialog with shadcn #3221 (#7302)
### What problem does this PR solve?

Feat: Display the document configuration dialog with shadcn #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-25 11:15:44 +08:00
balibabu
bcac195a0c Put the knowledge base list related hooks into use-knowledge-request.ts #3221 (#7197)
### What problem does this PR solve?

Put the knowledge base list related hooks into use-knowledge-request.ts
#3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-22 15:01:35 +08:00
balibabu
42e236f464 Feat: Rendering a search test list with real data #3221 (#7138)
### What problem does this PR solve?

Feat: Rendering a search test list with real data #3221
### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-18 16:29:41 +08:00
balibabu
86f76df586 Feat: Retrieval test #3221 (#7121)
### What problem does this PR solve?

Feat: Retrieval test #3221

### Type of change


- [x] New Feature (non-breaking change which adds functionality)
2025-04-17 19:03:55 +08:00