Yingfeng
b5bea72e4b
Add git-like file commit API ( #15978 )
...
### What problem does this PR solve?
| # | Method | Endpoint | Description | Git Equivalent |
|---|--------|----------|-------------|----------------|
| 1 | `POST` | `/api/v1/{prefix}/{folder_id}/commits` | Create a
snapshot commit with file changes (add/modify/delete/rename) | `git add`
+ `git commit` |
| 2 | `GET` | `/api/v1/{prefix}/{folder_id}/commits` | List commit
history (paginated) | `git log` |
| 3 | `GET` | `/api/v1/{prefix}/{folder_id}/commits/{commit_id}` | Get
commit detail with file changes | `git show` |
| 4 | `GET` | `/api/v1/{prefix}/{folder_id}/commits/{commit_id}/files` |
List file changes in a commit | `git show --name-status` |
| 5 | `GET` |
`/api/v1/{prefix}/{folder_id}/commits/diff?from=...&to=...` | Compare
two commits and return differences | `git diff` |
| 6 | `GET` | `/api/v1/{prefix}/{folder_id}/changes` | Get uncommitted
changes (add/modify/delete) | `git status` |
| 7 | `GET` | `/api/v1/{prefix}/{folder_id}/commits/{commit_id}/tree` |
Get the folder tree snapshot at commit time | `git ls-tree` |
| 8 | `GET` |
`/api/v1/{prefix}/{folder_id}/commits/{commit_id}/files/{file_id}/content`
| Get a file's content as it existed in a specific commit | `git show
HEAD:file` |
| 9 | `GET` | `/api/v1/{prefix}/{file_id}/versions` | Get version
history for a specific file across all commits | `git log -- file` |
Where `{prefix}/{id}` can be:
- `folders/{folder_id}` — direct folder access
- `workspaces/{workspace_id}` — alias of `folders/{folder_id}`
- `datasets/{dataset_id}` — resolves to the dataset's folder
- `memories/{memory_id}` — resolves to the memory's folder
- `skills/{skill_id}` — resolves to the skill's folder
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Documentation Update
2026-06-15 11:19:56 +08:00
Jin Hai
e96bc37d06
Go: use NATS as the message queue ( #15327 )
...
### What problem does this PR solve?
```
RAGFlow(admin)> mq publish 'msg2';
SUCCESS
RAGFlow(admin)> mq publish 'msg3';
SUCCESS
RAGFlow(admin)> mq list;
+---------+---------------+
| message | subject |
+---------+---------------+
| msg1 | tasks.RAGFLOW |
| msg2 | tasks.RAGFLOW |
| msg3 | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq pull 2;
+---------+---------------+
| message | subject |
+---------+---------------+
| msg1 | tasks.RAGFLOW |
| msg2 | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq pull noack;
+---------+---------------+
| message | subject |
+---------+---------------+
| abc | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq show
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+
| ack_pending_count | consumer_count | memory | message_count | pending_count | redelivered_count | waiting_count |
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+
| 2 | 1 | 0 | 2 | 0 | 1 | 0 |
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+
RAGFlow(admin)> list ingestors;
+--------------+-------------------------------------------+--------+
| host | name | status |
+--------------+-------------------------------------------+--------+
| 192.168.1.38 | ingestor-8f0e4bd5650a4ac58b0151969fbf6935 | alive |
+--------------+-------------------------------------------+--------+
RAGFlow(admin)> list ingestion tasks;
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+
| document_id | id | status | step | user | user_id |
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+
| ffe64fae423411f1a2d938a74640adcc | 90d3d0f6528941c1ac8eb0360effccc4 | COMPLETED | 5 | aaa@aaa.com | 2ba4881420fa11f19e9c38a74640adcc |
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+
RAGFlow(admin)> remove ingestion tasks '90d3d0f6528941c1ac8eb0360effccc4';
+---------+----------------------------------+
| delete | task_id |
+---------+----------------------------------+
| success | 90d3d0f6528941c1ac8eb0360effccc4 |
+---------+----------------------------------+
RAGFlow(admin)> stop ingestion tasks 'e89e20d9a25848a1b79bd9345ddbfe1d';
+----------+----------------------------------+
| status | task_id |
+----------+----------------------------------+
| STOPPING | e89e20d9a25848a1b79bd9345ddbfe1d |
+----------+----------------------------------+
# Publish a message
RAGFlow(admin)> mq publish 'cdd';
SUCCESS
# List current tasks in the message queue
RAGFlow(admin)> mq list
+----------------------------------+---------------+
| message | subject |
+----------------------------------+---------------+
| 7ce392a3c1624cd2be4b5276e8825059 | tasks.RAGFLOW |
+----------------------------------+---------------+
# Consume a task from the message queue
RAGFlow(admin)> mq pull
+------+-----+----------------+
| ack | id | type |
+------+-----+----------------+
| true | cdd | ingestion_test |
+------+-----+----------------+
# User mode
# List ingestion tasks, followed by dataset id
RAGFlow(user)> list ingestion tasks from '0abe79f9423311f1ad8d38a74640adcc';
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date | create_time | dataset_id | document_id | id | schema | status | update_date | update_time | user_id |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-05-30T20:21:06+08:00 | 1780143666289 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | 8d758cd14a8b4ba8ab505003fb52017d | | COMPLETED | 2026-05-30T20:21:26+08:00 | 1780143686431 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
RAGFlow(user)> list ingestion tasks;
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date | create_time | dataset_id | document_id | id | schema | status | update_date | update_time | user_id |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-06-02T19:02:31+08:00 | 1780398151417 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | e89e20d9a25848a1b79bd9345ddbfe1d | | COMPLETED | 2026-06-02T19:02:52+08:00 | 1780398172208 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
# Create an ingestion task
# First argument is document id, second argument is dataset id
RAGFlow(user)> start ingestion 'ffe64fae423411f1a2d938a74640adcc' from '0abe79f9423311f1ad8d38a74640adcc';
+----------------------------------+-------------------------------------------+
| document_id | result |
+----------------------------------+-------------------------------------------+
| ffe64fae423411f1a2d938a74640adcc | task_id: 8d758cd14a8b4ba8ab505003fb52017d |
+----------------------------------+-------------------------------------------+
# Pause an ingestion task, first argument is ingestion id
RAGFlow(user)> stop ingestion '8d758cd14a8b4ba8ab505003fb52017d';
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date | create_time | dataset_id | document_id | id | schema | status | update_date | update_time | user_id |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-05-30T20:21:06+08:00 | 1780143666289 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | 8d758cd14a8b4ba8ab505003fb52017d | | COMPLETED | 2026-05-30T20:21:26+08:00 | 1780143686431 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
# Delete an ingestion task
RAGFlow(api/default)> remove ingestion tasks 'f366450a27d54677aec1c7090add30f0';
+---------+----------------------------------+
| remove | task_id |
+---------+----------------------------------+
| success | f366450a27d54677aec1c7090add30f0 |
+---------+----------------------------------+
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-06-12 14:56:44 +08:00
Jin Hai
719ce15c95
Go CLI: update list supported models ( #15845 )
...
### What problem does this PR solve?
Now list supported models will show more info.
```
RAGFlow(api/default)> list supported models from 'gitee' 'test';
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
| dimension | max_tokens | model_types | name | thinking |
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
| | | | Wan2.7 | |
| | | | HappyHorse-1.0 | |
| | | | Qwen3.6-27B@Qwen | |
| | | | Qwen3.6-35B-A3B@Qwen | |
| | 1048576 | [chat] | DeepSeek-V4-Flash@deepseek-ai | map[clear_thinking:true default_value:true] |
| | 1048576 | [chat] | DeepSeek-V4-Pro@deepseek-ai | map[clear_thinking:true default_value:true] |
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-06-09 19:01:00 +08:00
Jack
67c3e73d70
feat: migrate DELETE /api/v1/datasets/:dataset_id/documents to Go ( #15577 )
...
## Summary
Migrate the batch document deletion endpoint from Python to Go. Two
modes supported: explicit `ids` list and `delete_all`.
## Changes
| File | Change |
|------|--------|
| `internal/dao/file2document.go` | Add `GetByDocumentID`,
`DeleteByDocumentID` |
| `internal/dao/file2document_test.go` | 5 new tests |
| `internal/dao/kb_test.go` | 2 new tests (`DecreaseDocumentNum`) |
| `internal/service/document.go` | Add `deleteDocumentFull` +
`DeleteDocuments`, refactor `DeleteDocument` |
| `internal/service/document_test.go` | 10 new tests |
| `internal/handler/document.go` | Add `documentServiceIface` +
`DeleteDocuments` handler |
| `internal/handler/document_test.go` | 7 new tests |
| `internal/router/router.go` | Register `DELETE /:dataset_id/documents`
|
| `cmd/server_main.go` | Support `RAGFLOW_DICT_PATH` env var |
| `internal/binding/rag_analyzer.go` | Use `-lpcre2-8` dynamic linking |
| `internal/dao/database.go` | Skip Error 1091/1138 during migration |
| `internal/service/llm.go` | Fix vet warning |
## Per-document cleanup
- Delete tasks from DB
- Hard-delete document + decrement KB counters
- Delete chunks from document engine (nil-guarded)
- Delete metadata from document engine (nil-guarded)
- Remove file2document mapping + file record + storage blob
## Test Results
**24 unit tests all passing** (7 DAO + 10 service + 7 handler) using
SQLite :memory: + gin.TestMode.
See [test report](docs/test_report_delete_documents.md) for manual
integration test results.
🤖 Generated with [Claude Code](https://claude.com/claude-code )
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com >
---------
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com >
2026-06-03 20:55:53 +08:00
Jin Hai
2061edd308
Remove unused codes ( #15579 )
...
### What problem does this PR solve?
Remove unused code.
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-06-03 17:35:36 +08:00
Jin Hai
aa57b5bd8b
Go: move logger to common module ( #14545 )
...
### What problem does this PR solve?
As title
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-05-06 10:41:58 +08:00
Yingfeng
4ee0702aed
Feat: add skills space to context engine ( #13908 )
...
### What problem does this PR solve?
issue #13714
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:36:03 +08:00
Jin Hai
6c29128de1
Refactor model provider and command ( #13887 )
...
### What problem does this PR solve?
Introduce 5 new tables, including model groups and provider instance.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-04-02 20:20:35 +08:00
Jin Hai
e20cf39735
Refactor Go server model provider reading and access ( #13831 )
...
### What problem does this PR solve?
1. Refactor model provider json file format
2. Use memory data structure to replace database
3. Add CLI command to access
```
RAGFlow(user)> list pool models from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| features | max_tokens | model_types | name |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
| map[] | 256000 | [llm] | grok-4 |
| map[] | 131072 | [llm] | grok-3 |
| map[] | 131072 | [llm] | grok-3-fast |
| map[] | 131072 | [llm] | grok-3-mini |
| map[] | 131072 | [llm] | grok-3-mini-mini-fast |
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768 | [vlm] | grok-2-vision |
+-------------------------------------------------------------------------------------+------------+-------------+-----------------------+
RAGFlow(user)> show pool model 'grok-2-vision' from 'xai';
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| features | max_tokens | model_types | name |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
| map[multimodal:map[enabled:true input_modalities:[image] output_modalities:[text]]] | 32768 | [vlm] | grok-2-vision |
+-------------------------------------------------------------------------------------+------------+-------------+---------------+
RAGFlow(user)> list pool providers;
+--------+------------------------------------------------------------+---------------------------+
| name | tags | url |
+--------+------------------------------------------------------------+---------------------------+
| OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | https://api.openai.com/v1 |
| xAI | LLM | https://api.x.ai/v1 |
+--------+------------------------------------------------------------+---------------------------+
RAGFlow(user)> show pool provider 'openai';
+---------------------------+--------+------------------------------------------------------------+--------------+
| base_url | name | tags | total_models |
+---------------------------+--------+------------------------------------------------------------+--------------+
| https://api.openai.com/v1 | OpenAI | LLM,TEXT EMBEDDING,TTS,TEXT RE-RANK,SPEECH2TEXT,MODERATION | 27 |
+---------------------------+--------+------------------------------------------------------------+--------------+
```
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-30 12:00:49 +08:00
Jin Hai
f32a832f92
Add rename model directory to entity to avoid name misunderstanding ( #13829 )
...
### What problem does this PR solve?
Model-> entity
### Type of change
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-27 19:25:18 +08:00
Jin Hai
74866371ef
Fix compatiblity issue ( #13667 )
...
### What problem does this PR solve?
1. Change go admin server port from 9385 to 9383 to avoid conflicts
2. Start go server after python servers are started completely, in
entrypoint.sh
3. Fix some database migration issue
4. Add more API routes in web to compliant with EE.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-18 11:51:03 +08:00
Jin Hai
5c955a31cc
Update go server ( #13589 )
...
### What problem does this PR solve?
1. Add more CLI command
2. Add some license hooks
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-13 14:41:02 +08:00
Jin Hai
2028e895fd
Add license and time record DAO ( #13522 )
...
### What problem does this PR solve?
1. Change go server default port to 9382
2. Compatible with EE data model.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-11 14:02:24 +08:00
Jin Hai
2133fd76a8
Add auth middleware ( #13506 )
...
### What problem does this PR solve?
Use auth middle-ware to check authorization.
### Type of change
- [x] Refactoring
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-11 11:23:13 +08:00
chanx
02108772d8
refactor: Moves the LLM factory initialization logic to the dao package. ( #13476 )
...
### What problem does this PR solve?
refactor: Moves the LLM factory initialization logic to the `dao`
package.
Removes the `init_data` package and integrates the LLM factory
initialization functionality into the `dao` package.
Adds a `utility` package to provide general utility functions.
Updates `server_main.go` to use the new initialization path.
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
Co-authored-by: Jin Hai <haijin.chn@gmail.com >
2026-03-10 10:35:55 +08:00
Jin Hai
01a100bb29
Fix data models ( #13444 )
...
### What problem does this PR solve?
Since database model is updated in python version, go server also need
to update
### Type of change
- [x] Bug Fix (non-breaking change which fixes an issue)
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
2026-03-06 20:05:10 +08:00
chanx
81fd1811b8
Feat：Using Go to implement user registration logic ( #13431 )
...
### What problem does this PR solve?
Feat：Using Go to implement user registration logic
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
2026-03-06 16:42:49 +08:00
Jin Hai
70e9743ef1
RAGFlow go API server ( #13240 )
...
# RAGFlow Go Implementation Plan 🚀
This repository tracks the progress of porting RAGFlow to Go. We'll
implement core features and provide performance comparisons between
Python and Go versions.
## Implementation Checklist
- [x] User Management APIs
- [x] Dataset Management Operations
- [x] Retrieval Test
- [x] Chat Management Operations
- [x] Infinity Go SDK
---------
Signed-off-by: Jin Hai <haijin.chn@gmail.com >
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com >
2026-03-04 19:17:16 +08:00
