Commit Graph

69 Commits

Author SHA1 Message Date
Wang Qi
f45ce00347 Not allow to sort by id (#14526)
### What problem does this PR solve?

id as "text", not a "keyword", order by it will cause error.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-30 14:52:43 +08:00
Lynn
e22cf333ed Fix: allow search id or _id (#14356)
### What problem does this PR solve?

Allow search id or _id when using es as doc_engine.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-04-24 21:38:19 +08:00
Zhichang Yu
b7744e053e fix: support dense_vector from ES fields response (ES 9.x compatibility) (#13972)
fix: support dense_vector from ES fields response (ES 9.x compatibility)

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Configuration Chore (non-breaking change which updates
configuration)


## Summary by CodeRabbit

* **Bug Fixes**
* More accurate handling and unwrapping of dense-vector fields so
returned values have correct shapes.
* Field selection reliably limits returned data and falls back to
alternate result locations when needed.
* Use of consistent result IDs and tolerant handling when score values
are missing.

* **Chores / Configuration**
* Increased build memory and adjusted build-time flags for the frontend
build.
* Simplified runtime model/GPU checks and removed an automated runtime
GPU-install attempt.

* **Build Fixes**
* `web/vite.config.ts`: make `build.minify` and `build.sourcemap`
respect `VITE_MINIFY` and `VITE_BUILD_SOURCEMAP` env vars from
Dockerfile instead of hardcoding `terser` and `true`.

* **Environment**
* Allow stack version override and default the runtime image tag to
"latest".

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Correct unwrapping of dense-vector fields and reliable field selection
with fallback locations.
* Consistent use of hit-level IDs and tolerant handling when score
values are missing.

* **Chores / Configuration**
* Increased frontend build memory and added build-time minify/sourcemap
flags; build minification and sourcemap now configurable.
* Removed runtime GPU detection for model initialization; force CPU
initialization.

* **Environment**
* Allow stack version override and default runtime image tag to
"latest".

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-09 17:44:13 +08:00
MkDev11
cfee2bc9db feat: Auto-adjust chunk recall weights based on user feedback (#12689)
### What problem does this PR solve?

Implements automatic adjustment of knowledge base chunk recall weights
based on user feedback (upvotes/downvotes). When users upvote or
downvote a response, the system locates the corresponding knowledge
snippets and adjusts their recall weight to improve future retrieval
quality.

**Closes #12670**

**How it works:**
1. User upvotes/downvotes a response via `POST /thumbup`
2. System extracts chunk IDs from the conversation reference
3. For each referenced chunk:
   - Reads current `pagerank_fea` value from document store
   - Increments (+1) for upvote or decrements (-1) for downvote
   - Clamps weight to [0, 100] range
   - Updates chunk in ES/Infinity/OceanBase
4. Future retrievals score these chunks higher/lower based on
accumulated feedback

**Files changed:**
- `api/db/services/chunk_feedback_service.py` - New service for updating
chunk pagerank weights
- `api/apps/conversation_app.py` - Integrated feedback service into
thumbup endpoint
- `test/testcases/test_web_api/test_chunk_feedback/` - Unit tests

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Chat message feedback now updates per-chunk relevance weights
(feature-flag gated), with configurable weighting and atomic updates
across storage backends.

* **Bug Fixes**
* Stricter validation for message feedback inputs and more robust
handling of feedback transitions.

* **Tests**
* Expanded test coverage for chunk-feedback behavior, weighting
strategies, storage backends, and thumb-flip scenarios.

* **Chores**
  * CI workflow extended to run the new chunk-feedback web API tests.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: mkdev11 <YOUR_GITHUB_ID+MkDev11@users.noreply.github.com>
Co-authored-by: mkdev11 <MkDev11@users.noreply.github.com>
2026-04-08 09:52:18 +08:00
qinling0210
1be07a0a34 Fix "Result window is too large" during meta data search (#13521)
### What problem does this PR solve?

Fix
https://github.com/infiniflow/ragflow/issues/13210#issuecomment-3982878498

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-12 18:59:56 +08:00
Magicbook1108
daec36e935 Fix: add soft limit for graph rag size (#13252)
### What problem does this PR solve?

Fix: add soft limit for graph rag size #13258 Q2

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-02 14:02:36 +08:00
qinling0210
9a5208976c Put document metadata in ES/Infinity (#12826)
### What problem does this PR solve?

Put document metadata in ES/Infinity.

Index name of meta data: ragflow_doc_meta_{tenant_id}

### Type of change

- [x] Refactoring
2026-01-28 13:29:34 +08:00
Vedant Madane
ac936005e6 fix: ensure deleted chunks are not returned in retrieval (#12520) (#12546)
## Summary
Fixes #12520 - Deleted chunks should not appear in retrieval/reference
results.

## Changes

### Core Fix
- **api/apps/chunk_app.py**: Include \doc_id\ in delete condition to
properly scope the delete operation

### Improved Error Handling
- **api/db/services/document_service.py**: Better separation of concerns
with individual try-catch blocks and proper logging for each cleanup
operation

### Doc Store Updates
- **rag/utils/es_conn.py**: Updated delete query construction to support
compound conditions
- **rag/utils/opensearch_conn.py**: Same updates for OpenSearch
compatibility

### Tests
- **test/testcases/.../test_retrieval_chunks.py**: Added
\TestDeletedChunksNotRetrievable\ class with regression tests
- **test/unit/test_delete_query_construction.py**: Unit tests for delete
query construction

## Testing
- Added regression tests that verify deleted chunks are not returned by
retrieval API
- Tests cover single chunk deletion and batch deletion scenarios
2026-01-15 14:45:55 +08:00
Jin Hai
01f0ced1e6 Fix IDE warnings (#12281)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-12-29 12:01:18 +08:00
Lynn
6e9691a419 Feat: message manage (#12196)
### What problem does this PR solve?

Manage message and use in agent.

Issue #4213 

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-12-25 21:18:13 +08:00
Kevin Hu
88a28212b3 Fix: Table parse method issue. (#11627)
### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-12-01 12:42:35 +08:00
Jin Hai
296476ab89 Refactor function name (#11210)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-12 19:00:15 +08:00
Jin Hai
f98b24c9bf Move api.settings to common.settings (#11036)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-06 09:36:38 +08:00
Jin Hai
02d10f8eda Move var from rag.settings to common.globals (#11022)
### What problem does this PR solve?

As title.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-05 15:48:50 +08:00
Stephen Hu
c20f5675c6 Fix: elasticsearch connection hardcoded (#10975)
### What problem does this PR solve?

https://github.com/infiniflow/ragflow/issues/10930

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-11-04 10:59:35 +08:00
Jin Hai
78631a3fd3 Move some functions out of 'api/utils/common.py' (#10948)
### What problem does this PR solve?

as title.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-03 12:34:47 +08:00
Jin Hai
44f2d6f5da Move 'get_project_base_directory' to common directory (#10940)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-02 21:05:28 +08:00
Jin Hai
6447b737ab Move singleton to common directory (#10935)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-11-02 12:24:08 +08:00
Jin Hai
766d900a41 Refactor: rename rmSpace to remove_redundant_spaces (#10796)
### What problem does this PR solve?

- rename rmSpace to remove_redundant_spaces
- move clean_markdown_block to common module
- add unit tests for remove_redundant_spaces and clean_markdown_block

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-10-28 09:46:32 +08:00
Lynn
fb950079ef Feat/service manage (#10381)
### What problem does this PR solve?

- Admin service support SHOW SERVICE <id>.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

issue: #10241
2025-09-30 16:23:09 +08:00
Kevin Hu
1e6bda735a Fix: add ES re-connect once request timeout. (#8678)
### What problem does this PR solve?

#8669

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-07-07 09:22:25 +08:00
symvation
32f8b3ad77 Fix: the output log is incorrect (#8577)
### What problem does this PR solve?

Fix: the output log is incorrect

### Type of change

- [ ] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: liang <xiaofeng.liang@landstech.com.cn>
2025-07-01 10:49:43 +08:00
changqingla
4784aa5b0b fix: List Chunks API fails to return the correct document status. (#8347)
### What problem does this PR solve?

The existing
/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks endpoint
fails to accurately return a document's chunk status. Even when a chunk
is explicitly marked as unavailable, the API still returns true.

![img_v3_02nc_3458a1b7-609e-4f20-8cb7-2156a489848g](https://github.com/user-attachments/assets/ab3b8f69-1284-49c1-8af3-bdfae3416583)

![img_v3_02nc_82f1d96e-7596-4def-ba75-5a2bd10d56cg](https://github.com/user-attachments/assets/a8a4162b-b50d-4dfc-af72-e1d7812a0a93)

Co-authored-by: zhoudeyong <zhoudeyong@idr.ai>
2025-06-19 11:12:53 +08:00
liwenju0
a73fbc61ff Fix: Handle the case of deleting empty blocks. Update the relevant message (#6643)
…gic to return the correct deletion message. Add handling for empty
arrays to ensure no errors occur during the deletion operation. Update
the test cases to verify the new logic.

### What problem does this PR solve?

fix this bug:https://github.com/infiniflow/ragflow/issues/6607

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: wenju.li <wenju.li@deepctr.cn>
2025-04-02 19:20:17 +08:00
Kevin Hu
dd81c30976 Fix: tag_feas deletion error. (#6257)
### What problem does this PR solve?

#6218

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-19 11:25:11 +08:00
Kevin Hu
1333d3c02a Fix: float transfer exception. (#6197)
### What problem does this PR solve?

#6177

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-18 11:13:44 +08:00
Zhichang Yu
4fffee6695 Regards kb_id at ElasticSearch insert, update, delete. (#6105)
### What problem does this PR solve?

Regards kb_id at ElasticSearch insert, update, delete. Close #6066

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-03-14 17:34:02 +08:00
Kevin Hu
3aa5c2a699 Ignore exception of empty index. (#5030)
### What problem does this PR solve?

### Type of change


- [x] Refactoring
2025-02-17 15:59:55 +08:00
Kevin Hu
0d3ed37b48 Make the update script shorter. (#4854)
### What problem does this PR solve?


### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-02-10 18:18:49 +08:00
Kevin Hu
c354239b79 Make infinity adapt to condition exist. (#4657)
### What problem does this PR solve?

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2025-01-26 18:45:36 +08:00
Kevin Hu
55f2b7c4d5 Code format. (#4611)
### What problem does this PR solve?


### Type of change

- [x] Refactoring
2025-01-23 18:43:32 +08:00
Kevin Hu
dd0ebbea35 Light GraphRAG (#4585)
### What problem does this PR solve?

#4543

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-22 19:43:14 +08:00
Jin Hai
3894de895b Update comments (#4569)
### What problem does this PR solve?

Add license statement.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-01-21 20:52:28 +08:00
Kevin Hu
c5da3cdd97 Tagging (#4426)
### What problem does this PR solve?

#4367

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2025-01-09 17:07:21 +08:00
Jin Hai
d64df4de9c Update error message (#4417)
### What problem does this PR solve?

1. Update error message
2. Remove space characters

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2025-01-08 20:18:27 +08:00
Kevin Hu
31d67c850e Fetch chunk by batches. (#4177)
### What problem does this PR solve?

#4173

### Type of change

- [x] Performance Improvement
2024-12-23 12:12:15 +08:00
Zhichang Yu
03f00c9e6f Rename page_num_list, top_list, position_list (#3940)
### What problem does this PR solve?

Rename page_num_list, top_list, position_list to page_num_int, top_int,
position_int

### Type of change

- [x] Refactoring
2024-12-10 16:32:58 +08:00
Zhichang Yu
0d68a6cd1b Fix errors detected by Ruff (#3918)
### What problem does this PR solve?

Fix errors detected by Ruff

### Type of change

- [x] Refactoring
2024-12-08 14:21:12 +08:00
Kevin Hu
74b28ef1b0 Add pagerank to KB. (#3809)
### What problem does this PR solve?

#3794

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-12-03 14:30:35 +08:00
Zhichang Yu
c5f13629af Set Log level by env (#3798)
### What problem does this PR solve?

Set Log level by env

### Type of change

- [x] Refactoring
2024-12-02 17:24:39 +08:00
Zhichang Yu
bc701d7b4c Edit chunk shall update instead of insert it (#3709)
### What problem does this PR solve?

Edit chunk shall update instead of insert it. Close #3679 

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-28 13:00:38 +08:00
Zhichang Yu
90ffcb4ddb Fix graphrag + infinity bugs (#3681)
### What problem does this PR solve?

Fix graphrag + infinity bugs

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-27 12:45:43 +08:00
liuhua
7f48acb3fd Fix enable/disable bug (#3662)
### What problem does this PR solve?

Fix enable/disable bug   #3628

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Co-authored-by: liuhua <10215101452@stu.ecun.edu.cn>
Co-authored-by: Kevin Hu <kevinhu.sh@gmail.com>
2024-11-27 09:37:11 +08:00
Zhichang Yu
7669fc8f52 Fix es get NotFoundError (#3659)
### What problem does this PR solve?

Fix es get NotFoundError

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-26 18:07:07 +08:00
Kevin Hu
9f3141804f Fix chunk enable/disable issue (#3579)
### What problem does this PR solve?

#3576

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-22 12:25:42 +08:00
Zhichang Yu
9541d7e7bc Added TRACE_MALLOC_DELTA and TRACE_MALLOC_FULL (#3555)
### What problem does this PR solve?

Added TRACE_MALLOC_DELTA and TRACE_MALLOC_FULL to debug task_executor.py
heap. Relates to #3518

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-22 12:00:25 +08:00
Kevin Hu
ee50f78d99 Add component 'Template' (#3562)
### What problem does this PR solve?

#3560

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2024-11-21 18:26:22 +08:00
Kevin Hu
0ac6dc8f8c Cut down the attempt times of ES (#3550)
### What problem does this PR solve?

#3541
### Type of change


- [x] Refactoring
- [x] Performance Improvement
2024-11-21 11:37:45 +08:00
Zhichang Yu
cad341e794 Added kb_id filter to knn. Fix #3458 (#3513)
### What problem does this PR solve?

Added kb_id filter to knn. Fix #3458

- [x] Bug Fix (non-breaking change which fixes an issue)
2024-11-20 20:53:30 +08:00
Jin Hai
883fafde72 Fix elasticsearch status display (#3487)
### What problem does this PR solve?

_Briefly describe what this PR aims to solve. Include background context
that will help reviewers understand the purpose of the PR._

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: jinhai <haijin.chn@gmail.com>
2024-11-19 14:40:58 +08:00