Commit Graph

37 Commits

Author SHA1 Message Date
Jack
23aae19898 feat: implement POST /api/v1/agents/<agent_id>/upload API (#15633)
## Summary

Implement the `POST /api/v1/agents/<agent_id>/upload` endpoint in Go,
allowing file uploads associated with agent canvases.

### Changes

- **Modified**: `internal/service/agent.go` — Added `CheckCanvasAccess`
method (owner + team-level permission semantics)
- **Modified**: `internal/handler/agent.go` — Added `UploadAgentFile`
handler with auth check, multipart file parsing, and delegation to
`FileService`. Added `fileUploader` interface for testability.
- **Modified**: `internal/router/router.go` — Registered `POST
/:agent_id/upload` route
- **Modified**: `cmd/server_main.go` — Wired `fileService` into
`AgentHandler`
- **New**: `internal/service/agent_test.go` — 4 service-level tests for
`CheckCanvasAccess` (owner, team member, private denial, not found)
- **New**: `internal/handler/agent_upload_test.go` — 3 handler-level
tests (success with fake file service, cross-user denial, empty file
rejection)

### Testing

All 7 tests pass with zero mocking of the DB layer (in-memory SQLite):

```
=== RUN   TestCheckCanvasAccess_Owner               --- PASS
=== RUN   TestCheckCanvasAccess_NotOwner            --- PASS
=== RUN   TestCheckCanvasAccess_PrivateCanvas_Denied --- PASS
=== RUN   TestCheckCanvasAccess_NotFound            --- PASS
=== RUN   TestUploadAgentFileHandler_Success        --- PASS
=== RUN   TestUploadAgentFileHandler_NoPermission   --- PASS
=== RUN   TestUploadAgentFileHandler_NoFiles        --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:21:47 +08:00
Jack
67c3e73d70 feat: migrate DELETE /api/v1/datasets/:dataset_id/documents to Go (#15577)
## Summary

Migrate the batch document deletion endpoint from Python to Go. Two
modes supported: explicit `ids` list and `delete_all`.

## Changes

| File | Change |
|------|--------|
| `internal/dao/file2document.go` | Add `GetByDocumentID`,
`DeleteByDocumentID` |
| `internal/dao/file2document_test.go` | 5 new tests |
| `internal/dao/kb_test.go` | 2 new tests (`DecreaseDocumentNum`) |
| `internal/service/document.go` | Add `deleteDocumentFull` +
`DeleteDocuments`, refactor `DeleteDocument` |
| `internal/service/document_test.go` | 10 new tests |
| `internal/handler/document.go` | Add `documentServiceIface` +
`DeleteDocuments` handler |
| `internal/handler/document_test.go` | 7 new tests |
| `internal/router/router.go` | Register `DELETE /:dataset_id/documents`
|
| `cmd/server_main.go` | Support `RAGFLOW_DICT_PATH` env var |
| `internal/binding/rag_analyzer.go` | Use `-lpcre2-8` dynamic linking |
| `internal/dao/database.go` | Skip Error 1091/1138 during migration |
| `internal/service/llm.go` | Fix vet warning |

## Per-document cleanup

- Delete tasks from DB
- Hard-delete document + decrement KB counters
- Delete chunks from document engine (nil-guarded)
- Delete metadata from document engine (nil-guarded)
- Remove file2document mapping + file record + storage blob

## Test Results

**24 unit tests all passing** (7 DAO + 10 service + 7 handler) using
SQLite :memory: + gin.TestMode.

See [test report](docs/test_report_delete_documents.md) for manual
integration test results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 20:55:53 +08:00
Jin Hai
2061edd308 Remove unused codes (#15579)
### What problem does this PR solve?

Remove unused code.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-03 17:35:36 +08:00
web-dev0521
0a7662cf3e feat(go-api): implement GET /api/v1/agents list endpoint (issue #15328) (#15329)
## Summary

Closes: #15328 
- Implements `GET /api/v1/agents` — the agent/canvas listing endpoint
needed to complete the Home dashboard tile in `web/src/pages/home/`.
- Mirrors Python `api/apps/restful_apis/agent_api.py::list_agents`
exactly: tenant-join auth, optional `owner_ids` guard, keyword filter,
pagination, ordering, and `canvas_category` filter (default:
`agent_canvas`).
- **Scope:** read-only list only. Full agent CRUD and canvas runtime are
explicitly out of scope (separate slice of #15240).
2026-05-28 19:40:54 +08:00
Alexander Laurent
32d5bf9791 feat: add Go MCP server create API (#15260)
## What
Implementation for POST /api/v1/mcp/servers
#15240
2026-05-28 16:43:21 +08:00
qinling0210
af85aa9c7b Implement Elasticsearch functions in GO (#15160)
### What problem does this PR solve?

Implement Elasticsearch functions in GO (except for Search)

### Type of change

- [x] Refactoring
2026-05-25 19:15:07 +08:00
Jin Hai
f8c626bbc8 Go: add ingestion server (#15094)
### What problem does this PR solve?

1. Go ingestion server will connected with admin server with gRPC stream
2. Go ingestion server will be responsible for ingestion tasks
```

RAGFlow(admin)> list ingestors;
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+
| address         | cpu_usage | id                               | last_heartbeat            | name     | process_id | rss_usage    | status | task_count | vms_usage     |
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+
| 127.0.0.1:58564 | 0         | bdd1870eea2646e0aacb8a2cd3307aa2 | 2026-05-24T18:16:17+08:00 | ingestor | 680152     | 212.72265625 | active | 0          | 2589.12109375 |
+-----------------+-----------+----------------------------------+---------------------------+----------+------------+--------------+--------+------------+---------------+

RAGFlow(admin)> start ingestion 'abc';
+----------------------------------+
| task_id                          |
+----------------------------------+
| e714777639ca4760ab427b5f211e81ad |
+----------------------------------+

RAGFlow(admin)> stop ingestion 'f7bd39d0a724457eb5fdce6d81699776';
+----------------------------------+
| task_id                          |
+----------------------------------+
| f7bd39d0a724457eb5fdce6d81699776 |
+----------------------------------+

RAGFlow(admin)> list tasks;
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+
| ETA | assign_to                        | error | from | id                               | last_update               | start_time | status     |
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | eae6431da72a40e796cff3a03008091b | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 6cccdd174bd049ecb05a774bbb47593f | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | ef360d777e57485799adb96b30f2b4b8 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | bcc5c5448cb64de48b6b6171c36fb790 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | bfc25384c43a443294fe2da979a38ac2 | 2026-05-24T19:46:03+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 84960537b85d413b8990a9efd5952d67 | 2026-05-24T19:46:04+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 3d223c1b51e24b36861a3bfb2f1d58d4 | 2026-05-24T19:46:03+08:00 |            | CANCELED   |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | e433b0e356b846c89c301621a3c54494 | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 7c93a3880f074ebd8eca14e6b51bb7ef | 2026-05-24T19:46:03+08:00 |            | COMPLETED  |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | df2e4ef51aaf4390bff9a23f2692486e | 2026-05-24T19:46:04+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 7377c53010194ef7a83aa206698d66ff | 2026-05-24T19:46:05+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | df64d1a1f9d348e3a2f174c4d7d69e73 | 2026-05-24T19:46:05+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | b59834512e2847e1bdf13ace04b8a456 | 2026-05-24T19:46:06+08:00 |            | DISPATCHED |
| 0   | 17937da188b84f23a5c10bb87588944b |       | CLI  | 0064bb0ab69344028d1ecfda053826f4 | 2026-05-24T19:46:03+08:00 |            | QUEUED     |
+-----+----------------------------------+-------+------+----------------------------------+---------------------------+------------+------------+


```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-25 14:00:08 +08:00
qinling0210
dbef3e361f Update chunk/metadata cli (#15055)
### What problem does this PR solve?

Update chunk/metadata cli

### Type of change

- [ ] Refactoring
2026-05-20 20:32:06 +08:00
Jin Hai
335dd5a263 Go: add cli command, list dataset documents (#14948)
### What problem does this PR solve?
```
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
| created_at          | id                               | meta_fields | name            | size    | status | type |
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
| 2026-05-08 19:35:08 | f6aa38bb4ad111f1ba6338a74640adcc | map[]       |  abc.pdf        | 3387987 | 1      | pdf  |
+---------------------+----------------------------------+-------------+-----------------+---------+--------+------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-15 14:00:45 +08:00
Jin Hai
aa57b5bd8b Go: move logger to common module (#14545)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-05-06 10:41:58 +08:00
Yingfeng
4ee0702aed Feat: add skills space to context engine (#13908)
### What problem does this PR solve?

issue #13714

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-04-30 12:36:03 +08:00
Jin Hai
819257f257 Go: add volcengine (#14409)
### What problem does this PR solve?

1. Refactor server_main
2. Add volcengine

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-28 12:12:58 +08:00
Jin Hai
5fe6f7c9ac Go CLI: Add list configs and set log level command (#13983)
### What problem does this PR solve?

1. list configs
2. set log level debug/info/warn/error/fatal/panic

```

RAGFlow(user)> list configs;
+--------------------+-----------------------+
| key                | value                 |
+--------------------+-----------------------+
| redis_host         | localhost:6379        |
| doc_engine         | elasticsearch         |
| elasticsearch_host | http://localhost:1200 |
| log_level          | info                  |
| database           | mysql                 |
| database_host      | localhost:3306        |
| admin              | 0.0.0.0:9383          |
| storage_engine     | minio                 |
| minio_host         | localhost:9000        |
+--------------------+-----------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

## Release Notes

* **New Features**
* Added `LIST CONFIGS` command to view system configuration details
(Redis, database, log level, storage engine, and host settings).
* Added `SET LOG LEVEL` command to adjust logging verbosity at runtime.

* **Improvements**
* Enhanced log level configuration defaults and runtime state
management.
* Reorganized token management and system endpoints under `/system/`
routes for better API organization.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-08 19:32:53 +08:00
Jin Hai
5673245134 Refactor context command (#13948)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-07 11:30:09 +08:00
Jin Hai
6c29128de1 Refactor model provider and command (#13887)
### What problem does this PR solve?

Introduce 5 new tables, including model groups and provider instance.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-04-02 20:20:35 +08:00
Jin Hai
efd6ecc3e5 New provider and models API and CLI (#13865)
### What problem does this PR solve?

As title.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-31 18:42:12 +08:00
chanx
8a9bbf3d6d Feat: add memory function by go (#13754)
### What problem does this PR solve?

Feat: Add Memory function by go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-27 09:49:50 +08:00
Jin Hai
b308cd3a02 Update go cli (#13717)
### What problem does this PR solve?

Go cli

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-24 20:08:36 +08:00
qinling0210
7b86f577be Implement metadata search in Infinity in GO (#13706)
### What problem does this PR solve?

Add cli

LIST DOCUMENTS OF DATASET quoted_string ";"
LIST METADATA OF DATASETS quoted_string ("," quoted_string)* ";"
LIST METADATA SUMMARY OF DATASET quoted_string (DOCUMENTS quoted_string
("," quoted_string)*)? ";"

### Type of change

- [x] Refactoring
2026-03-21 18:10:00 +08:00
Jin Hai
9ce766192f Init storage engine (#13707)
### What problem does this PR solve?

1. Init Minio / S3 / OSS
2. Fix minio / s3 / oss config

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-20 13:15:41 +08:00
Magicbook1108
cfe6ea6f56 Feat: CREATE / DELETE / LIST dataset api in Go (#13695)
### What problem does this PR solve?

Feat: CREATE / DELETE / LIST dataset api in Go

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Lynn <lynn_inf@hotmail.com>
Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-03-19 20:48:32 +08:00
Jin Hai
7ebe1d2722 Fix docker building (#13681)
### What problem does this PR solve?

1. Refactor go server log
2. Update docker building, since nginx config should be set according to
the deployment.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-19 10:25:35 +08:00
Jin Hai
5c955a31cc Update go server (#13589)
### What problem does this PR solve?

1. Add more CLI command
2. Add some license hooks

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-13 14:41:02 +08:00
Jin Hai
3fbf8bc3d4 Expose go version server and admin server port out of docker in CI (#13572)
### What problem does this PR solve?

- Print Go version log when start server
- Expose the server port in CI docker container

### Type of change

- [x] Other (please describe): For CI

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:39:57 +08:00
Jin Hai
d688b72dff Go: Add admin server status checking (#13571)
### What problem does this PR solve?

RAGFlow server isn't available when admin server isn't connected.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 20:02:50 +08:00
Jin Hai
e78938c72c Update go admin server default port to 9383 (#13559)
### What problem does this PR solve?

As title

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 13:41:08 +08:00
Jin Hai
2fb1360d9d Add command line parameter and fix error message (#13526)
### What problem does this PR solve?

`./server_main -p 9380`

`./server_main -h`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-12 09:50:57 +08:00
Jin Hai
2133fd76a8 Add auth middleware (#13506)
### What problem does this PR solve?

Use auth middle-ware to check authorization.

### Type of change

- [x] Refactoring

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-11 11:23:13 +08:00
chanx
f14b53c764 feat(admin): Implemented default administrator initialization and login functionality. (#13504)
### What problem does this PR solve?

feat(admin): Implemented default administrator initialization and login
functionality.

Added support for default administrator configuration, including super
user nickname, email, and password.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-03-10 17:30:21 +08:00
chanx
02108772d8 refactor: Moves the LLM factory initialization logic to the dao package. (#13476)
### What problem does this PR solve?

refactor: Moves the LLM factory initialization logic to the `dao`
package.

Removes the `init_data` package and integrates the LLM factory
initialization functionality into the `dao` package.
Adds a `utility` package to provide general utility functions.
Updates `server_main.go` to use the new initialization path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 10:35:55 +08:00
Jin Hai
4fe706876c Service list and minio status (#13480)
### What problem does this PR solve?

1. Resolve standard user can access admin service
2. Get RAGFlow service status
3. Fix minio status fetching

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-10 09:56:43 +08:00
Jin Hai
52bcd98d29 Add scheduled tasks (#13470)
### What problem does this PR solve?

1. RAGFlow server will send heartbeat periodically.
2. This PR will including:
- Scheduled task
- API server message sending
- Admin server API to receive the message.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 17:48:29 +08:00
chanx
25ace613b0 feat: Added LLM factory initialization functionality and knowledge base related API interfaces (#13472)
### What problem does this PR solve?

feat: Added LLM factory initialization functionality and knowledge base
related API interfaces

refactor(dao): Refactored the TenantLLMDAO query method
feat(handler): Implemented knowledge base related API endpoints
feat(service): Added LLM API key setting functionality
feat(model): Extended the knowledge base model definition
feat(config): Added default user LLM configuration

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-03-09 15:52:14 +08:00
Jin Hai
610c1b507d Add more API of admin server of go (#13403)
### What problem does this PR solve?

Add APIs to admin server.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-09 10:44:53 +08:00
Jin Hai
01a100bb29 Fix data models (#13444)
### What problem does this PR solve?

Since database model is updated in python version, go server also need
to update

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-06 20:05:10 +08:00
Jin Hai
3e3b665b89 RAGFlow admin server go version (#13394)
### What problem does this PR solve?

1. init go admin server
2. refactor api server router
3. add benchmark CI to 450s time limit
4. remove docker builder container after building

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-03-05 15:18:40 +08:00
Jin Hai
70e9743ef1 RAGFlow go API server (#13240)
# RAGFlow Go Implementation Plan 🚀

This repository tracks the progress of porting RAGFlow to Go. We'll
implement core features and provide performance comparisons between
Python and Go versions.

## Implementation Checklist

- [x] User Management APIs
- [x] Dataset Management Operations
- [x] Retrieval Test
- [x] Chat Management Operations
- [x] Infinity Go SDK

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
Co-authored-by: Yingfeng Zhang <yingfeng.zhang@gmail.com>
2026-03-04 19:17:16 +08:00