Commit Graph

137 Commits

Author SHA1 Message Date
Yingfeng
5a7d7771a3 Decouple skill space from Python API (#15971)
### What problem does this PR solve?

Make skill space independent of Python filesystem API

### Type of change

- [x] Refactoring
2026-06-12 18:18:55 +08:00
bitloi
22a058f56c fix(go): redact internal handler errors (#15746)
### What problem does this PR solve?

Refs #15743

Some Go API handlers return raw `err.Error()` strings in
`CodeServerError` responses. Those errors can include internal backend
details such as database, storage, search engine, or host information.

This PR adds a small shared `jsonInternalError` helper for handler-level
internal failures. The helper logs the raw error server-side with
request method/path context, then returns the existing generic
`common.CodeServerError.Message()` to API clients.

This first slice migrates the existing `jsonError(c,
common.CodeServerError, err.Error())` production call sites in agent,
dataset graph, file, and system handlers. It intentionally does not
close the full issue because direct `c.JSON` error responses in other
handlers remain for follow-up PRs.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [ ] New Feature (non-breaking change which adds functionality)
- [ ] Documentation Update
- [ ] Refactoring
- [ ] Performance Improvement
- [ ] Other (please describe):

### Tests

- `/root/go/bin/go test ./internal/handler -count=1`

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-06-12 16:09:10 +08:00
Jin Hai
e96bc37d06 Go: use NATS as the message queue (#15327)
### What problem does this PR solve?

```
RAGFlow(admin)> mq publish 'msg2';
SUCCESS
RAGFlow(admin)> mq publish 'msg3';
SUCCESS
RAGFlow(admin)> mq list;
+---------+---------------+
| message | subject       |
+---------+---------------+
| msg1    | tasks.RAGFLOW |
| msg2    | tasks.RAGFLOW |
| msg3    | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq pull 2;
+---------+---------------+
| message | subject       |
+---------+---------------+
| msg1    | tasks.RAGFLOW |
| msg2    | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq pull noack;
+---------+---------------+
| message | subject       |
+---------+---------------+
| abc     | tasks.RAGFLOW |
+---------+---------------+
RAGFlow(admin)> mq show
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+
| ack_pending_count | consumer_count | memory | message_count | pending_count | redelivered_count | waiting_count |
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+
| 2                 | 1              | 0      | 2             | 0             | 1                 | 0             |
+-------------------+----------------+--------+---------------+---------------+-------------------+---------------+

RAGFlow(admin)> list ingestors;
+--------------+-------------------------------------------+--------+
| host         | name                                      | status |
+--------------+-------------------------------------------+--------+
| 192.168.1.38 | ingestor-8f0e4bd5650a4ac58b0151969fbf6935 | alive  |
+--------------+-------------------------------------------+--------+

RAGFlow(admin)> list ingestion tasks;
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+
| document_id                      | id                               | status    | step | user        | user_id                          |
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+
| ffe64fae423411f1a2d938a74640adcc | 90d3d0f6528941c1ac8eb0360effccc4 | COMPLETED | 5    | aaa@aaa.com | 2ba4881420fa11f19e9c38a74640adcc |
+----------------------------------+----------------------------------+-----------+------+-------------+----------------------------------+

RAGFlow(admin)> remove ingestion tasks '90d3d0f6528941c1ac8eb0360effccc4';
+---------+----------------------------------+
| delete  | task_id                          |
+---------+----------------------------------+
| success | 90d3d0f6528941c1ac8eb0360effccc4 |
+---------+----------------------------------+

RAGFlow(admin)> stop ingestion tasks 'e89e20d9a25848a1b79bd9345ddbfe1d';
+----------+----------------------------------+
| status   | task_id                          |
+----------+----------------------------------+
| STOPPING | e89e20d9a25848a1b79bd9345ddbfe1d |
+----------+----------------------------------+

# Publish a message
RAGFlow(admin)> mq publish 'cdd';
SUCCESS

# List current tasks in the message queue
RAGFlow(admin)> mq list
+----------------------------------+---------------+
| message                          | subject       |
+----------------------------------+---------------+
| 7ce392a3c1624cd2be4b5276e8825059 | tasks.RAGFLOW |
+----------------------------------+---------------+

# Consume a task from the message queue
RAGFlow(admin)> mq pull
+------+-----+----------------+
| ack  | id  | type           |
+------+-----+----------------+
| true | cdd | ingestion_test |
+------+-----+----------------+

# User mode
# List ingestion tasks, followed by dataset id
RAGFlow(user)> list ingestion tasks from '0abe79f9423311f1ad8d38a74640adcc';
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date               | create_time   | dataset_id                       | document_id                      | id                               | schema | status    | update_date               | update_time   | user_id                          |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-05-30T20:21:06+08:00 | 1780143666289 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | 8d758cd14a8b4ba8ab505003fb52017d |        | COMPLETED | 2026-05-30T20:21:26+08:00 | 1780143686431 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+

RAGFlow(user)> list ingestion tasks;
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date               | create_time   | dataset_id                       | document_id                      | id                               | schema | status    | update_date               | update_time   | user_id                          |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-06-02T19:02:31+08:00 | 1780398151417 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | e89e20d9a25848a1b79bd9345ddbfe1d |        | COMPLETED | 2026-06-02T19:02:52+08:00 | 1780398172208 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+

# Create an ingestion task
# First argument is document id, second argument is dataset id
RAGFlow(user)> start ingestion 'ffe64fae423411f1a2d938a74640adcc' from '0abe79f9423311f1ad8d38a74640adcc';
+----------------------------------+-------------------------------------------+
| document_id                      | result                                    |
+----------------------------------+-------------------------------------------+
| ffe64fae423411f1a2d938a74640adcc | task_id: 8d758cd14a8b4ba8ab505003fb52017d |
+----------------------------------+-------------------------------------------+

# Pause an ingestion task, first argument is ingestion id
RAGFlow(user)> stop ingestion '8d758cd14a8b4ba8ab505003fb52017d';
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| create_date               | create_time   | dataset_id                       | document_id                      | id                               | schema | status    | update_date               | update_time   | user_id                          |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+
| 2026-05-30T20:21:06+08:00 | 1780143666289 | 0abe79f9423311f1ad8d38a74640adcc | ffe64fae423411f1a2d938a74640adcc | 8d758cd14a8b4ba8ab505003fb52017d |        | COMPLETED | 2026-05-30T20:21:26+08:00 | 1780143686431 | 2ba4881420fa11f19e9c38a74640adcc |
+---------------------------+---------------+----------------------------------+----------------------------------+----------------------------------+--------+-----------+---------------------------+---------------+----------------------------------+

# Delete an ingestion task
RAGFlow(api/default)> remove ingestion tasks 'f366450a27d54677aec1c7090add30f0';
+---------+----------------------------------+
| remove  | task_id                          |
+---------+----------------------------------+
| success | f366450a27d54677aec1c7090add30f0 |
+---------+----------------------------------+

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-12 14:56:44 +08:00
Rene Arredondo
bf59eb77cc feat(go-api): port forgot-password flow to Go (#15282) (#15290)
## Summary

Implements **chunk 1** of #15282 — the four `/api/v1/auth/password/...`
endpoints from the login-page Go port. **Chunk 2 (OAuth/OIDC) is
deferred** to its own subtask, matching the issue author's own
confidence-low recommendation ("multi-provider, stateful redirect flow
with external dependencies; recommend its own subtask").

New endpoints, all registered under `apiNoAuth` (forgot-password users
are unauthenticated by definition):

| Method | Path | Status |
|--------|------|--------|
| `POST` | `/api/v1/auth/password/forgot/captcha` | new |
| `POST` | `/api/v1/auth/password/forgot/otp` | new |
| `POST` | `/api/v1/auth/password/forgot/otp/verify` | new |
| `POST` | `/api/v1/auth/password/reset` | new |

## Wire compatibility with the Python backend

The two backends share state through Redis, so the Go port had to use
identical keys, encodings, and constants. Either backend can now
validate a code the other minted.

- **Redis keys**: `captcha:<email>`, `otp:<email>`,
`otp_attempts:<email>`, `otp_last_sent:<email>`, `otp_lock:<email>`,
`otp:verified:<email>` — same as `api/utils/web_utils.py`.
- **Stored OTP value**: `"<hex_hash>:<hex_salt>"` — same as Python.
- **Hash**: HMAC-SHA256 with a `crypto/rand` 16-byte salt — same as
`hash_code()`.
- **Constants**: `OTP_LENGTH=4`, `OTP_TTL=5min`, `ATTEMPT_LIMIT=5`,
`ATTEMPT_LOCK_SECONDS=30min`, `RESEND_COOLDOWN_SECONDS=60s` — all match
`api/utils/web_utils.py`.
- **Email body**: matches `RESET_CODE_EMAIL_TMPL` byte-for-byte.

## Files

### New

| File | Purpose |
|---|---|
| `internal/utility/otp.go` | OTP/captcha constants, Redis key builders
(`CaptchaRedisKey`, `OTPRedisKeys`, `OTPVerifiedRedisKey`),
`HashOTPCode`, `GenerateOTPCode` / `GenerateCaptchaCode` /
`GenerateOTPSalt` via `crypto/rand`, and `EncodeOTPStorageValue` /
`DecodeOTPStorageValue` matching Python's storage shape. |
| `internal/utility/smtp.go` | Minimal stdlib `net/smtp` sender.
`SendResetCodeEmail(to, otp, ttlMin)` builds an RFC 5322 plain-text
message and dispatches via implicit TLS / STARTTLS / plain — same
selectors as Python `aiosmtplib`. Returns `SMTPNotConfiguredError` if
the config block is empty. |

### Modified

| File | Change |
|---|---|
| `internal/server/config.go` | New `SMTPConfig` struct + `Config.SMTP`
field. Field names mirror the `smtp:` keys in `common/settings.py`
(`mail_server`, `mail_port`, `mail_use_ssl`, `mail_use_tls`,
`mail_username`, `mail_password`, `mail_from_name`, `mail_from_address`,
`mail_frontend_url`) so a single `conf/service_conf.yaml` powers both
backends. |
| `internal/service/user.go` | Four methods — `ForgotIssueCaptcha`,
`ForgotSendOTP`, `ForgotVerifyOTP`, `ForgotResetPassword`. Reuses the
existing `decryptPassword`, `HashPassword`, `userDAO.Update`, and
`utility.GenerateToken` so the reset+auto-login path is identical to
`LoginByEmail`. |
| `internal/handler/user.go` | Four handlers in the same `c.JSON` shape
as `LoginByEmail`. The reset handler rotates the access token and emits
an `Authorization` header for auto-login (matches Python
`construct_response(auth=user.get_id())`). |
| `internal/router/router.go` | Routes registered under `apiNoAuth`,
with an explanatory comment on why they sit outside the auth middleware.
|

## Known divergence — captcha rendering

The Python endpoint returns a rendered `image/JPEG` from the
`python-captcha` library. The Go side has **no image-captcha dependency
vendored** in `go.mod`, and hand-rolling a raster generator was out of
scope for this PR.

This commit returns JSON `{captcha: "<text>"}` instead. Implications:

- **Backend gate is identical** — the OTP step still verifies the
user-submitted captcha string against the Redis value, so the security
model is unchanged.
- **Frontend impact**: the password-reset page rendering needs a small
tweak (text display instead of `<img>`) until a Go captcha library is
wired in.
- The handler comments call this out explicitly so the next PR knows
what to swap.

Possible follow-ups (any one closes the gap):
1. Add `github.com/mojocn/base64Captcha` or `github.com/dchest/captcha`
to `go.mod` and replace the JSON response with an `image/JPEG`.
2. Hand-roll a 5x7 bitmap font + `image/png` writer using only the
stdlib.
3. Render a server-side SVG (cheap, but trivially OCR-able — only useful
as a UI shim).

## Test plan

- [ ] **Captcha**: `POST
/api/v1/auth/password/forgot/captcha?email=<existing>` returns `{code:
0, data: {captcha: "ABCD"}}`. Redis shows `captcha:<email>` with that
value and ~60s TTL. Unknown email returns `code: CodeDataError`.
- [ ] **OTP send**: `POST /api/v1/auth/password/forgot/otp` with the
right captcha mints an OTP, stores `<hash>:<salt>` under `otp:<email>`
for 5 min, sends an email, returns success. With a wrong captcha returns
`CodeAuthenticationError`. Hitting it again within 60s returns "you
still have to wait …" with `CodeNotEffective`.
- [ ] **OTP verify**: correct OTP → `code: 0`, OTP keys cleared,
`otp:verified:<email>` = `"1"`. Wrong OTP → `code:
CodeAuthenticationError`, attempt counter bumped; after 5 wrong tries
`otp_lock:<email>` is set and further attempts hit `CodeNotEffective`.
- [ ] **Reset**: with the verified flag set, supply a new password
(RSA-encrypted+base64, same as `LoginByEmail`). Returns `code: 0`,
`Authorization` header set, verified flag deleted. Without the verified
flag returns `CodeAuthenticationError`.
- [ ] **Wire-compat smoke**: mint an OTP from the Python backend, verify
it via the Go endpoint, and vice versa. Should both succeed.
- [ ] **SMTP misconfigured**: drop `smtp.mail_server` from
`conf/service_conf.yaml`. The OTP-send endpoint should now return
"failed to send email" without panicking; check the log for the
`SMTPNotConfiguredError` warning.
- [ ] **End-to-end FE**: hit the password-reset flow from
`web/src/pages/login-next/`. Confirm the text-captcha shim works after
the FE tweak.
- [ ] `go build ./...` and `go vet ./...` — I could not run these in the
sandbox; please confirm a clean build before merging.
- [ ] `uv run pytest` to confirm no Python regressions (shared Redis
schema).

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-10 21:27:56 +08:00
Jonathan Chang
dfcf226ba3 feat: Implement API of ragflow server in Go (#15256)
## Summary
- Implemented the Go API endpoint for Memory message forgetting:
  - `DELETE /api/v1/messages/{memory_id}:{message_id}`
- Added route registration for the Memory message DELETE endpoint only.
- Added request path validation for `memory_id:message_id`.
- Added service logic to mark a message as forgotten by setting
`forget_at`.
- Preserved Python-compatible response behavior:
  - Success returns `code: 0`, `message: true`, `data: null`.
- Added focused unit tests for message path parsing and invalid message
ID handling.
- Fixed Linux cgo linker config to use the installed shared PCRE2
library so Go tests/builds can run in this environment.
## Related Issue
Closes: #15240 
## Change Type
- [x] Feature
- [x] Test
- [x] Build / CI compatibility

## Implemented API
- `DELETE /api/v1/messages/{memory_id}:{message_id}`
## Real Behavior Proof
Validated with targeted Go tests:
```bash
/tmp/go1.25.0/bin/go test ./internal/handler ./internal/router
```
Result:
```text
ok  	ragflow/internal/handler
?   	ragflow/internal/router	[no test files]
```
Validated server entrypoint build:
```bash
/tmp/go1.25.0/bin/go build -o /tmp/ragflow-server-main ./cmd/server_main.go
```

Result:
```text
build succeeded
```
Validated patch formatting:
```bash
git diff --check
```

Result:

```text
no whitespace errors
```
## Checklist
- [x] Implemented only `DELETE
/api/v1/messages/{memory_id}:{message_id}`.
- [x] Did not implement unrelated Memory message APIs.
- [x] Added route registration.
- [x] Added handler validation.
- [x] Added service-level memory access check.
- [x] Added tests.
- [x] Ran targeted Go tests.
- [x] Ran server build validation.
- [x] Ran `git diff --check`.
2026-06-10 21:27:35 +08:00
Hunnyboy1217
16d5b4fa02 feat[Go]: implement POST /api/v1/files/link-to-datasets (#15674)
### What problem does this PR solve?

Closes #15673 — ports the Python `file2document_api.py` `convert()`
endpoint to Go.

| Method | Path | Handler |
|--------|------|---------|
| POST | `/api/v1/files/link-to-datasets` | `FileHandler.LinkToDatasets`
|

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---

#### Implementation notes

**Files changed:**

```
internal/service/file2document.go  – new service (File2DocumentService)
internal/dao/file2document.go      – added Create method
internal/handler/file.go           – FileHandler gains file2DocumentService;
                                     LinkToDatasets HTTP handler
internal/router/router.go          – route registered
```

**Functional parity table:**

| Concern | Go behaviour |
|---------|-------------|
| Required fields | `file_ids` and `kb_ids` both required; missing
either → `CodeDataError` mirroring Python `@validate_request` |
| File existence | `fileDAO.GetByIDs(fileIDs)` builds a set; any missing
ID → `"File not found!"` |
| KB existence | `kbDAO.GetByID(kbID)` per KB; missing → `"Can't find
this dataset!"` |
| Folder expansion | `getAllInnermostFileIDs` recursively calls
`fileDAO.ListByParentID` — mirrors
`FileService.get_all_innermost_file_ids` |
| File permissions | `checkFileTeamPermission`: `file.TenantID ==
userID` OR user in tenant's team — mirrors `check_file_team_permission`
|
| KB permissions | `checkKBTeamPermission`: `kb.TenantID == userID` OR
user in tenant's team — mirrors `check_kb_team_permission` |
| Fire-and-forget | `go convertFiles(...)` goroutine after all
validation passes — mirrors `loop.run_in_executor(None, _convert_files,
…)` |
| Conversion | `convertFiles`: for each file → delete existing mappings
+ hard-delete old documents → create new `Document` in each target KB →
create `File2Document` mapping — mirrors Python `_convert_files` |
| `getParser` | Extension-based lookup with fallback to `kb.ParserID` —
mirrors `FileService.get_parser` |
| Immediate return | `true` returned to caller as soon as goroutine is
scheduled |

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-06-10 16:46:55 +08:00
Hz_
3796835c4d feat(go-api): migrate agent file download handler to Go with strict P… (#15769)
## What does this PR do?

This PR migrates the Agent Temporary File Download endpoint (`GET
/api/v1/agents/download`) from the Python backend to the Go backend,
optimizing the data retrieval flow and maintaining strict functional
parity. It also fixes a persistent parsing error in the Sandbox code
execution node.

## Checklist
- [x] Code logic matches Python implementation
- [x] All local unit tests passed
- [x] No breaking changes to existing router interfaces

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-06-10 16:09:36 +08:00
oktofeesh
bbc1f2ecec feat(go-api): add RAG retrieval to chat completions (#15739)
## Summary
- Add knowledge-base retrieval support to Go chat completions.

## What changed
- Routes KB-backed chat sessions through the Go retrieval service
instead of falling back to solo chat.
- Resolves embedding and rerank models, validates accessible knowledge
bases, and preserves tenant-aware retrieval.
- Rejects mixed embedding models across selected knowledge bases before
retrieval to avoid incompatible vector dimensions.
- Threads the HTTP request context into streaming retrieval so cancelled
requests can stop downstream retrieval work.
- Applies metadata filters and message-level `doc_ids` before retrieval.
- Expands parent/child chunks before building references and prompt
context.
- Injects retrieved knowledge through a copied dialog prompt config so
the caller's original dialog is not mutated.
- Honors configured empty responses when no chunks are found.
- Names the metadata no-match sentinel and reuses it across
retrieval/handler paths.
- Adds a defensive content cast while appending streamed answers.
- Adds focused unit coverage for retrieval, metadata filtering,
authorization, multimodal messages, references, empty-response behavior,
prompt immutability, and mixed embedding models.

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-10 11:07:45 +08:00
Haruko386
d56aeb2f5d feat[Go]: api datasets/<dataset_id>/documents/<document_id>/metadata/… (#15846)
### What problem does this PR solve?

As title

```
/api/v1/datasets/<dataset_id>/documents/<document_id>/metadata/config PUT
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-06-10 09:57:11 +08:00
Haruko386
a396b1ace2 feat[Go]: implement /api/v1/agents/<agent_id> and test_db_connection (#15771)
### What problem does this PR solve?

Add two API in go
```
/api/v1/agents/test_db_connection POST

/api/v1/agents/<agent_id>/sessions DELETE
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Yingfeng <yingfeng.zhang@gmail.com>
2026-06-10 09:54:07 +08:00
Jack
87b8062df4 feat: implement POST /api/v1/searchbots/ask — streaming RAG with citations and think-tag processing (#15825)
Implements POST /api/v1/searchbots/ask in Go with streaming SSE,
citations, and think-tag processing. 23 files, 90+ unit tests.

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-09 22:48:50 +08:00
JPette1783
acae932938 fix(go): guard four nil-pointer dereferences causing runtime panics (#15815)
### What problem does this PR solve?

Fixes four Go paths that dereference a pointer with no prior nil check,
each
causing a **runtime panic**. Closes #15814.

| # | File | Bug | Fix |
|---|------|-----|-----|
| 1 | `internal/entity/models/deepseek.go` | streaming path runs `switch
*chatModelConfig.Effort` inside `if *Thinking`; panics when
`Thinking=true` and `Effort==nil` | nil-check with default `"high"`,
matching the non-streaming path in the same file |
| 2 | `internal/entity/models/volcengine.go` | identical oversight:
`switch *modelConfig.Effort` with no guard | nil-check with default
`"medium"`, matching its non-streaming path |
| 3 | `internal/handler/auth.go` | `AuthMiddleware` does `if
*user.IsSuperuser`; panics on every authenticated request when the DB
column is `NULL` | guard with `user.IsSuperuser != nil &&`, matching
every other call site (`admin/handler.go`, `admin/service.go`,
`user.go`) |
| 4 | `internal/service/heartbeat_sender.go` |
`responseBody["code"].(float64)` panics on any non-200 response lacking
a numeric `code`; the upstream `recover()` calls `Fatal()` →
`os.Exit(1)`, taking down the whole server | comma-ok assertion (`code,
ok := ...`); return an error instead of panicking |

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-09 19:29:25 +08:00
Hz_
d4fe3bb148 feat(go-api): Add GET dataset metadata summary API (#15843)
## What

Adds the RESTful dataset metadata summary endpoint:

`GET /api/v1/datasets/{dataset_id}/metadata/summary`

The endpoint supports optional document filtering through:

`?doc_ids=doc_id_1,doc_id_2`
2026-06-09 19:27:47 +08:00
Jin Hai
719ce15c95 Go CLI: update list supported models (#15845)
### What problem does this PR solve?

Now list supported models will show more info.

```
RAGFlow(api/default)> list supported models from 'gitee' 'test';
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
| dimension | max_tokens | model_types | name                                                     | thinking                                    |
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
|           |            |             | Wan2.7                                                   |                                             |
|           |            |             | HappyHorse-1.0                                           |                                             |
|           |            |             | Qwen3.6-27B@Qwen                                         |                                             |
|           |            |             | Qwen3.6-35B-A3B@Qwen                                     |                                             |
|           | 1048576    | [chat]      | DeepSeek-V4-Flash@deepseek-ai                            | map[clear_thinking:true default_value:true] |
|           | 1048576    | [chat]      | DeepSeek-V4-Pro@deepseek-ai                              | map[clear_thinking:true default_value:true] |
+-----------+------------+-------------+----------------------------------------------------------+---------------------------------------------+
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-09 19:01:00 +08:00
Hz_
d1c436b804 feat(api): implement GET /api/v1/agents/prompts endpoint in Go (#15748)
### Description
This PR ports the `GET /api/v1/agents/prompts` endpoint from the Python
backend to the Go backend.

### Changes Made
- **Handler**: Added `GetPrompts` method to `internal/handler/agent.go`.
- **Router**: Registered the `agents.GET("/prompts")` route in
`internal/router/router.go`.
- **Logic**: Leveraged the existing `service.LoadPrompt` utility to read
`analyze_task_system`, `analyze_task_user`, `next_step`, `reflect`, and
`citation_prompt` templates directly from the `rag/prompts` directory.
- **Unit Test**: Added `TestGetPrompts_Success` to
`internal/handler/agent_test.go` to mock the HTTP context and validate
the JSON response structure.

### Motivation
This is part of the ongoing effort to port the Agent API surface to Go.
Since this specific endpoint only serves static prompt templates and
does not require the complex DAG/Canvas execution engine, it can be
seamlessly and safely handled by the Go backend.

### Testing
- [x] Added automated unit test `TestGetPrompts_Success` (verified
passing).
- [x] Tested locally via `curl` against the Go server (port 9380) and
Python server (port 9384).
- [x] Verified that the Go JSON response structure and loaded prompt
text are logically 100% identical to the Python implementation.
2026-06-09 17:03:42 +08:00
Jin Hai
55abf4f565 Go: new CLI command, list all models and show model (#15786)
### What problem does this PR solve?

```
RAGFlow(user)> list models;
+---------------------------+------------+-------------+--------------------+---------------------------------------------+
| alias                     | max_tokens | model_types | name               | thinking                                    |
+---------------------------+------------+-------------+--------------------+---------------------------------------------+
|                           | 1048576    | [chat]      | deepseek-v4-flash  | map[clear_thinking:true default_value:true] |
|                           | 1048576    | [chat]      | deepseek-v4-pro    | map[clear_thinking:true default_value:true] |
|                           | 1024000    | [chat]      | minimax-m3         | map[clear_thinking:true default_value:true] |
|                           | 64000      | [vision]    | glm-4.5v           | map[clear_thinking:true default_value:true] |
| [baai/bge-m3]             | 8192       | [embedding] | bge-m3             |                                             |
| [baai/bge-reranker-v2-m3] | 1024       | [rerank]    | bge-reranker-v2-m3 |                                             |
|                           |            | [tts]       | step-audio-tts-3b  |                                             |
| [qwen/qwen3-asr-1.7b]     |            | [asr]       | qwen3-asr-1.7b     |                                             |
| [paddleocr-vl-1.5]        |            | [ocr]       | paddleocr-vl-0.9b  |                                             |
+---------------------------+------------+-------------+--------------------+---------------------------------------------+
RAGFlow(user)> show model 'minimax-m3';
+--------------+---------------------------------------------+
| field        | value                                       |
+--------------+---------------------------------------------+
| name         | minimax-m3                                  |
| max_tokens   | 1024000                                     |
| model_types  | [chat]                                      |
| thinking     | map[clear_thinking:true default_value:true] |
| class        |                                             |
| alias        |                                             |
| ModelTypeMap |                                             |
+--------------+---------------------------------------------+
RAGFlow(user)> show model 'baai/bge-m3';
+--------------+---------------+
| field        | value         |
+--------------+---------------+
| model_types  | [embedding]   |
| thinking     |               |
| class        |               |
| alias        | [baai/bge-m3] |
| ModelTypeMap |               |
| name         | bge-m3        |
| max_tokens   | 8192          |
+--------------+---------------+
```

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-08 21:38:15 +08:00
Jack
338fdb65fb feat(ci): enable go test in CI pipeline (#15750)
## What problem does this PR solve?

Go test files are never compiled in CI — only production binaries via
`go build`. This allowed a missing `"sort"` import in
`metadata_filter_test.go` to be merged without detection.

## Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)

## Changes

- Add `go test -count=1 ./internal/...` step after Go build in CI
workflow
- Fix missing `"sort"` import in `metadata_filter_test.go` (pre-existing
compile error)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 20:06:57 +08:00
Haruko386
67ce0c896d feat[Go]: implement /api/v1/agents/<agent_id>/sessions (#15705)
### What problem does this PR solve?

As Title
Codes were tested by Postman

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-08 16:26:27 +08:00
Jack
8f4809d1b5 feat: implement POST /api/v1/searchbots/retrieval_test (#15710)
## What problem does this PR solve?

Implements `POST /api/v1/searchbots/retrieval_test` in the Go API
server, aligning with the Python `bot_api.py` counterpart. Also applies
security hardening and consistency fixes discovered during CTO-level
code review:

- **Missing endpoint**: `retrieval_test` was not available in Go,
requiring Python fallback
- **Security**: Both `chunkHandler` and `searchBotHandler` leaked
`err.Error()` to API consumers
- **Python alignment**: Default values, empty question handling, and
`top_k <= 0` validation differed from Python behavior
- **Test gaps**: `chunkHandler.RetrievalTest` had zero unit tests;
several edge cases uncovered

## Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring

## Summary

### New Endpoint
- `POST /api/v1/searchbots/retrieval_test` — retrieval test with full
field support (page, size, top_k, use_kg, cross_languages, keyword,
similarity_threshold, vector_similarity_weight)

### New Type
- `common.StringSlice` — JSON type that accepts both `"kb1"` and
`["kb1", "kb2"]`, matching Python API flexibility

### Security
- Both `searchBotHandler` and `chunkHandler` now use `common.Warn()` +
generic error messages instead of leaking `err.Error()` to API consumers
- All error responses include consistent `"data": nil` shape
- `chunkHandler.RetrievalTest` uses interface-based DI (`chunkService`)
to enable testability

### Python Alignment
- Handler-level defaults align with Python `bot_api.py` (page=1,
size=30, top_k=1024, similarity_threshold=0.0,
vector_similarity_weight=0.3)
- `top_k <= 0` validation matching Python behavior
- Empty/whitespace question returns 200 + empty result (matches
`chunk_api.py`)
- `chunkHandler` `Datasets` field uses `common.StringSlice` for
string-or-array flexibility

### Refactoring
- `ChunkServiceIface` → `ChunkRetriever`, `chunkSvcIface` →
`chunkService` (Go-conventional naming)
- Extracted `applyRetrievalDefaults`, `toRetrievalServiceRequest` from
handler body
- Regex moved to package-level var in `parseRelatedQuestions`
- `service.RetrievalTestRequest.Datasets` type changed to
`common.StringSlice`
- `chunkHandler` now uses consumer-side interface for DI

### Tests
- 37 unit tests across both handlers: auth, validation, defaults,
StringSlice edge cases, empty/whitespace KbID, service errors, JSON
format, `top_k <= 0`, field mapping verification

## Files Changed

| File | Change |
|------|--------|
| `cmd/server_main.go` | Wire new handler + chunkService +
difyRetrievalHandler |
| `internal/common/json_types.go` | New StringSlice type |
| `internal/common/json_types_test.go` | StringSlice tests |
| `internal/handler/chunk.go` | Interface-based DI, security, Python
alignment, defaults |
| `internal/handler/chunk_test.go` | New — 9 comprehensive tests |
| `internal/handler/searchbot.go` | New endpoint + refactoring + `top_k
<= 0` validation |
| `internal/handler/searchbot_test.go` | 18 tests covering all edge
cases |
| `internal/router/router.go` | Register new route +
difyRetrievalHandler |
| `internal/service/chunk.go` | Datasets type → StringSlice, Question
binding relaxed |

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 16:16:56 +08:00
dripsmvcp
3d7adf2193 feat[Go]: implement GET /plugin/tools (issue #15240) (#15570)
## Summary

Port the Python `GET /v1/plugin/tools` endpoint to the Go API server.
Listed in the Go-API port checklist of #15240.

Returns the metadata of every embedded LLM tool plugin in the same JSON
shape the Python endpoint emits (camelCase keys preserved), so existing
frontends bind to the Go server without changes.
2026-06-08 11:53:19 +08:00
qinling0210
c960dc2a4c Refine handling of POST /api/v1/datasets/search in GO (#15583)
### What problem does this PR solve?

Refine handling of POST /api/v1/datasets/search in GO

### Type of change

- [x] Refactoring
2026-06-08 11:49:37 +08:00
Hz_
074c331cdf fix(go-api): sync document handler interface and enforce preview acce… (#15688)
### Description

This PR syncs the `documentServiceIface` interface and introduces
handler methods for document preview, artifact fetching, and downloading
in the Go API. It also ensures that strict dataset alignment and access
checks are enforced when retrieving or downloading documents.

Furthermore, this PR introduces comprehensive unit tests for both the
newly added Handler and Service methods to ensure robustness and prevent
future regressions.

### Key Changes
* **Router & Handler Integration**: 
  * Added and wired new API endpoints in `internal/router/router.go`.
* Synchronized the `documentServiceIface` with `GetDocumentArtifact`,
`GetDocumentPreview`, and `DownloadDocument`.
* Implemented handlers for these endpoints in
`internal/handler/document.go`.
* **Access & Validation Enforcement**: 
* Refactored `internal/service/document.go` to strictly check if a
document belongs to the requested dataset before allowing downloads or
previews.
* Added robust artifact file sanitization (`sanitizeArtifactFilename`)
and attachment handling (`shouldForceArtifactAttachment`).
* **Comprehensive Unit Testing**:
* **Handler Layer (`internal/handler/document_test.go`)**: Added mock
service implementations and Gin router tests covering success,
not-found, and internal error states for all 3 new endpoints.
* **Service Layer (`internal/service/document_test.go`)**: Added
extensive business logic tests including dataset mismatch checks,
non-existent document checks, and artifact file validation.
2026-06-08 11:37:06 +08:00
Jack
5a04ac0864 feat: Dify-compatible retrieval API endpoint (#15704)
## Summary

Dify-compatible retrieval API for external knowledge base integration.

## Changes

- **New handler**: DifyRetrievalHandler with POST/GET
/api/v1/dify/retrieval
- **Health check**: GET /api/v1/dify/retrieval/health
- **Full pipeline**: KB validation -> permission check -> embedding ->
metadata filter -> chunk retrieval -> child chunk aggregation ->
optional KG search -> response assembly
- **12 tests** covering all paths (success, errors, metadata filter, KG
mode)
- **Testability**: Handler dependencies defined as interfaces
(KBServiceIface, ModelServiceIface, etc.)

## Files

| File | Type |
|------|------|
| internal/handler/dify_retrieval_handler.go | New — handler +
interfaces |
| internal/handler/dify_retrieval_handler_test.go | New — 12 tests |
| internal/router/router.go | Modified — route registration |
| cmd/server_main.go | Modified — handler wiring |
| internal/service/kg/pipeline.go | Modified — SetChatModel/SetEmbModel
|
| internal/service/kg/retrieval.go | New — helper functions |
| internal/service/kg/scoring.go | Moved from service package |
| internal/service/kg/search.go | New — KG search functions |
| internal/service/kg/types.go | New — type definitions |

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 21:16:25 +08:00
web-dev0521
b8db200757 feat(go-api): implement MCP server management endpoints (#15281)
## Summary

Ports the MCP (Model Context Protocol) server management endpoints that
power `web/src/pages/user-setting/mcp/` from Python
(`api/apps/restful_apis/mcp_api.py`) to Go. There were no MCP routes in
the Go server before this change.

Closes #15275 (subtask of #15240).

## Endpoints implemented (base path `/api/v1`)

| Method | Path | Description |
|--------|------|-------------|
| GET | `/mcp/servers` | List tenant servers (keyword / order /
pagination) |
| POST | `/mcp/servers` | Create a server |
| GET | `/mcp/servers/{mcp_id}` | Get one (`?mode=download` exports
config) |
| PUT | `/mcp/servers/{mcp_id}` | Update a server |
| DELETE | `/mcp/servers/{mcp_id}` | Delete a server |
| POST | `/mcp/import` | Bulk import from JSON config |
| POST | `/mcp/servers/{mcp_id}/test` | Connect + list tools (see notes)
|

## Implementation

Follows the existing `handler → service → dao` layering (per PR #14790):

- **entity** (`internal/entity/mcp.go`): added `MCPServerType` constants
and `IsValidMCPServerType` over the existing `MCPServer` model.
- **dao** (`internal/dao/mcp.go`): new `MCPServerDAO` with tenant-scoped
CRUD, a keyword filter, and a **whitelisted order-column map** (guards
against SQL injection via the caller-supplied `orderby`).
- **service** (`internal/service/mcp.go`): new `MCPService` —
list/get/export/create/update/delete/import/test — mirroring
`MCPServerService` and the `mcp_api` request validation, with sentinel
errors for clean code mapping.
- **handler** (`internal/handler/mcp.go`): new `MCPHandler` with the
seven handlers and Python-compatible response codes.
- **router / server_main**: registered the `/mcp` group and wired the
handler.

## Deviations from Python (documented in code)

1. **Bulk import is at `POST /mcp/import`, not `/mcp/servers/import`.**
gin (v1.9.1) cannot register a static segment and a path param at the
same tree node, so `/mcp/servers/import` would collide with
`/mcp/servers/:mcp_id` and panic at startup. The frontend should call
`/mcp/import`.
2. **No live tool discovery on create/update/import.** The Python path
runs `get_mcp_tools` over SSE / streamable-HTTP and stores
`variables.tools`. The Go server has no MCP client yet, so these persist
`variables`/`headers` but leave `variables.tools` unpopulated.
3. **`/test` returns a data error (`ErrMCPTestUnsupported`)** until a Go
MCP client lands. Per the issue, the live-connection path is scoped as a
follow-up; the handler still validates `url` + `server_type`.

## Testing

- Added `internal/service/mcp_test.go` covering `IsValidMCPServerType`
and the `TestServer` validation/short-circuit paths (no DB required).
- No Go toolchain was available in the dev environment, so `go build
./...` / `go vet ./...` verification is left to CI.

## Follow-ups

- Go MCP client (SSE / streamable-HTTP) to enable live tool discovery
and the real `/test` behavior.
- Reconcile the `/mcp/import` vs `/mcp/servers/import` path with the
frontend.

---------
2026-06-05 13:25:09 +08:00
Haruko386
4b2af1347c feat[Go]: implement Agent/Workflow PUT /api/v1/agents/<canvas_id>/tags (#15641)
feat[Go]: implement Agent/Workflow PUT /api/v1/agents/<canvas_id>/tags (#15641)
2026-06-05 13:22:23 +08:00
dripsmvcp
431f52a5d4 feat[Go]: implement GET /agents/templates (issue #15240) (#15573)
## Summary

Port the canvas-template catalogue endpoint to the Go API server. Listed
in the Go-API port checklist of #15240.

Mirrors `list_agent_template` in `api/apps/restful_apis/agent_api.py`:
returns every row from the `canvas_template` table so that the UI can
render the template gallery on the New-Agent screen.

## What

- `internal/dao/canvas_template.go` — new `CanvasTemplateDAO.GetAll()`
ordered by `create_time desc` (newest templates first).
- `internal/service/agent.go` — wire the new DAO into `AgentService` and
expose `ListTemplates() ([]*entity.CanvasTemplate, error)`.
- `internal/handler/agent.go` — new `AgentHandler.ListTemplates` HTTP
handler (auth-gated, mirrors Python `@login_required`).
- `internal/router/router.go` — `agents.GET("/templates",
r.agentHandler.ListTemplates)` registered alongside the existing `GET
/agents`.
- `internal/handler/agent_test.go` — three new tests covering: success
path, empty-list → JSON array (not `null`), and the auth gate.

## Notes

- `CanvasTemplate` entity, GORM tags, and DB migration already exist in
`internal/entity/canvas.go` and `internal/dao/database.go` — no schema
change required.
- The handler coerces a `nil` slice to `[]*entity.CanvasTemplate{}` so
the JSON payload is always an array (the frontend does `data.map(...)`
on it).

## Test plan

- [x] `go vet ./internal/handler ./internal/service ./internal/dao
./internal/router` clean
- [x] Three unit tests added; existing `TestListAgents_Success`
untouched
- [ ] CI runs `go test ./internal/handler` with cgo binding linked

## Related

- Tracker: #15240
2026-06-05 10:13:30 +08:00
Jack
e627f5d8c5 feat: implement POST /api/v1/searchbots/related_questions API (#15639)
## Summary

Implement the `POST /api/v1/searchbots/related_questions` endpoint in
Go, generating related search questions via LLM.

### Changes

- **New**: `internal/handler/related_questions.go` — Handler with
injectable LLM interface, prompt constant, and response parsing
- **New**: `internal/handler/related_questions_test.go` — 9 tests (4
handler + 5 parse)
- **Modified**: `internal/router/router.go` — Added route +
`RelatedQuestionsHandler` to struct
- **Modified**: `cmd/server_main.go` — Wired handler with
`SearchService` and `ModelProviderService`

### Testing

All 9 tests pass:

```
=== RUN   TestRelatedQuestionsHandler_Success        --- PASS
=== RUN   TestRelatedQuestionsHandler_EmptyResponse  --- PASS
=== RUN   TestRelatedQuestionsHandler_LLMFailure     --- PASS
=== RUN   TestRelatedQuestionsHandler_MissingQuestion --- PASS
=== RUN   TestParseRelatedQuestions_Standard         --- PASS
=== RUN   TestParseRelatedQuestions_Empty            --- PASS
=== RUN   TestParseRelatedQuestions_NoNumberedLines  --- PASS
=== RUN   TestParseRelatedQuestions_MixedContent     --- PASS
=== RUN   TestParseRelatedQuestions_MultiDigit       --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 19:13:58 +08:00
Jack
6143205b37 feat: implement GET /api/v1/agents/<agent_id>/versions/<version_id> API (#15640)
## Summary

Implement the `GET /api/v1/agents/<agent_id>/versions/<version_id>`
endpoint in Go, returning full version details including DSL.

Depends on #15629 which introduced the version list endpoint and
`UserCanvasVersionDAO` infrastructure.

### Changes

- **Modified**: `internal/handler/agent.go` — Added `GetAgentVersion`
handler with auth check and ownership verification
- **Modified**: `internal/router/router.go` — Registered `GET
/:agent_id/versions/:version_id` route
- **New/Modified tests**: Service and handler tests for the version
detail endpoint

### Testing

```
=== RUN   TestGetVersion_Success       --- PASS
=== RUN   TestGetVersion_WrongCanvas   --- PASS
=== RUN   TestGetVersion_NotFound      --- PASS
=== RUN   TestGetAgentVersionHandler_Success      --- PASS
=== RUN   TestGetAgentVersionHandler_VersionNotFound --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 19:13:26 +08:00
Jack
23aae19898 feat: implement POST /api/v1/agents/<agent_id>/upload API (#15633)
## Summary

Implement the `POST /api/v1/agents/<agent_id>/upload` endpoint in Go,
allowing file uploads associated with agent canvases.

### Changes

- **Modified**: `internal/service/agent.go` — Added `CheckCanvasAccess`
method (owner + team-level permission semantics)
- **Modified**: `internal/handler/agent.go` — Added `UploadAgentFile`
handler with auth check, multipart file parsing, and delegation to
`FileService`. Added `fileUploader` interface for testability.
- **Modified**: `internal/router/router.go` — Registered `POST
/:agent_id/upload` route
- **Modified**: `cmd/server_main.go` — Wired `fileService` into
`AgentHandler`
- **New**: `internal/service/agent_test.go` — 4 service-level tests for
`CheckCanvasAccess` (owner, team member, private denial, not found)
- **New**: `internal/handler/agent_upload_test.go` — 3 handler-level
tests (success with fake file service, cross-user denial, empty file
rejection)

### Testing

All 7 tests pass with zero mocking of the DB layer (in-memory SQLite):

```
=== RUN   TestCheckCanvasAccess_Owner               --- PASS
=== RUN   TestCheckCanvasAccess_NotOwner            --- PASS
=== RUN   TestCheckCanvasAccess_PrivateCanvas_Denied --- PASS
=== RUN   TestCheckCanvasAccess_NotFound            --- PASS
=== RUN   TestUploadAgentFileHandler_Success        --- PASS
=== RUN   TestUploadAgentFileHandler_NoPermission   --- PASS
=== RUN   TestUploadAgentFileHandler_NoFiles        --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 17:21:47 +08:00
Jack
02d163a177 feat: implement GET /api/v1/agents/<agent_id>/versions API (#15629)
## Summary

Implement the `GET /api/v1/agents/<agent_id>/versions` endpoint in Go,
listing all version snapshots for an agent canvas in descending update
time order.

### Changes

- **New**: `internal/dao/user_canvas_version.go` —
`UserCanvasVersionDAO` with `ListByCanvasID` (ordered by update_time
DESC) and `GetByID`
- **Modified**: `internal/service/agent.go` — Added `CheckCanvasAccess`,
`ListVersions`, `GetVersion` methods
- **Modified**: `internal/handler/agent.go` — Added `ListAgentVersions`
handler with auth check
- **Modified**: `internal/router/router.go` — Registered `GET
/:agent_id/versions` route
- **New**: `internal/service/agent_test.go` — 5 service-level tests
(SQLite in-memory DB, zero mock)
- **Modified**: `internal/handler/agent_test.go` — 3 handler-level tests
(real DB, pre-authenticated context)

### Testing

All 8 tests pass with zero mocking (in-memory SQLite replaces MySQL):

```
=== RUN   TestListVersions_Success         --- PASS
=== RUN   TestListVersions_Empty           --- PASS
=== RUN   TestCheckCanvasAccess_Owner      --- PASS
=== RUN   TestCheckCanvasAccess_NotOwner   --- PASS
=== RUN   TestCheckCanvasAccess_NotFound   --- PASS
=== RUN   TestListAgentVersionsHandler_Success      --- PASS
=== RUN   TestListAgentVersionsHandler_NoPermission --- PASS
=== RUN   TestListAgentVersionsHandler_CanvasNotFound --- PASS
```

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-04 15:36:26 +08:00
Jack
c6eee09ed3 feat: migrate POST /api/v1/datasets/<dataset_id>/documents/stop to Go (#15597)
## Summary

Migrate the stop parse documents endpoint from Python to Go.

### Python endpoint
`POST /api/v1/datasets/<dataset_id>/documents/stop` —
`api/apps/restful_apis/document_api.py:1542-1641`

### Changes
| File | Change |
|------|--------|
| `internal/dao/task.go` | Add `GetByDocID` method |
| `internal/dao/task_test.go` | 3 DAO tests (new file) |
| `internal/service/document.go` | Add `StopParseDocuments` + refactor
shared helpers |
| `internal/service/document_test.go` | 8 service tests |
| `internal/handler/document.go` | Add handler + request struct +
interface |
| `internal/handler/document_test.go` | 5 handler tests |
| `internal/router/router.go` | Add `POST /:dataset_id/documents/stop`
route |

### How it works
1. Validates all document IDs belong to the dataset
2. For each document in RUNNING/CANCEL state (or with unfinished tasks):
- Sets Redis cancel signal `{task_id}-cancel` for each associated task
   - Updates `document.run` to CANCEL ("2")
3. Returns `{"success_count": N, "errors": [...]}`

### Test strategy
- **DAO/Service**: SQLite in-memory DB, zero mocks. Redis is nil-safe by
design.
- **Handler**: `fakeDocumentService` implementing `documentServiceIface`
interface.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
2026-06-04 14:16:13 +08:00
Jack
67c3e73d70 feat: migrate DELETE /api/v1/datasets/:dataset_id/documents to Go (#15577)
## Summary

Migrate the batch document deletion endpoint from Python to Go. Two
modes supported: explicit `ids` list and `delete_all`.

## Changes

| File | Change |
|------|--------|
| `internal/dao/file2document.go` | Add `GetByDocumentID`,
`DeleteByDocumentID` |
| `internal/dao/file2document_test.go` | 5 new tests |
| `internal/dao/kb_test.go` | 2 new tests (`DecreaseDocumentNum`) |
| `internal/service/document.go` | Add `deleteDocumentFull` +
`DeleteDocuments`, refactor `DeleteDocument` |
| `internal/service/document_test.go` | 10 new tests |
| `internal/handler/document.go` | Add `documentServiceIface` +
`DeleteDocuments` handler |
| `internal/handler/document_test.go` | 7 new tests |
| `internal/router/router.go` | Register `DELETE /:dataset_id/documents`
|
| `cmd/server_main.go` | Support `RAGFLOW_DICT_PATH` env var |
| `internal/binding/rag_analyzer.go` | Use `-lpcre2-8` dynamic linking |
| `internal/dao/database.go` | Skip Error 1091/1138 during migration |
| `internal/service/llm.go` | Fix vet warning |

## Per-document cleanup

- Delete tasks from DB
- Hard-delete document + decrement KB counters
- Delete chunks from document engine (nil-guarded)
- Delete metadata from document engine (nil-guarded)
- Remove file2document mapping + file record + storage blob

## Test Results

**24 unit tests all passing** (7 DAO + 10 service + 7 handler) using
SQLite :memory: + gin.TestMode.

See [test report](docs/test_report_delete_documents.md) for manual
integration test results.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-03 20:55:53 +08:00
Haruko386
df55880b44 feat[Go] implement /connectors/google/oauth (#15584)
### What problem does this PR solve?

The following API is available in go

> /api/v1/connectors/google/oauth/web/start POST
> /api/v1/connectors/gmail/oauth/web/callback GET
> /api/v1/connectors/google-drive/oauth/web/callback GET
> /api/v1/connectors/google/oauth/web/result POST


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-03 20:08:55 +08:00
Jin Hai
2061edd308 Remove unused codes (#15579)
### What problem does this PR solve?

Remove unused code.

### Type of change

- [x] Refactoring

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-03 17:35:36 +08:00
Haruko386
473d06d1ad feat[Go]: implement add multi_models (#15563) 2026-06-03 15:26:46 +08:00
ちー
5f8926410d feat[Go]: implement /api/v1/connectors/<connector_id> PATCH (#15512)
### What problem does this PR solve?

As title, all test are passed

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-02 19:34:07 +08:00
Haruko386
9f969feb89 feat[Go] implement check connection by using apikey and region (#15475)
### What problem does this PR solve?

**Verified from PostMan**


GET http://127.0.0.1:9384/api/v1/providers/gitee/connection
```json
body: 

{
    "api_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "region": "default"

}

resp: 
{
    "code": 0,
    "message": "success"
}
```

GET http://127.0.0.1:9384/api/v1/providers/gitee/connection
```json
body: 

{
    "api_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "region": "deprecated"

}

resp: 
{
    "code": 0,
    "message": "success"
}
```

GET http://127.0.0.1:9384/api/v1/providers/gitee/connection
```json
body: 

{
    "api_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXX",
    "region": "china"

}

resp: 
{
    "code": 0,
    "message": "success"
}

```

GET http://127.0.0.1:9384/api/v1/providers/lmstudio/connection
```json
body: 

{
    "api_key": "",
    "region": "test"

}

resp: 
{
    "code": 0,
    "message": "success"
}
```


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-02 19:32:41 +08:00
Alexander Laurent
a98889cd76 feat: add Go MCP server update API (#15261)
## What

#15240
implementation for PUT /api/v1/mcp/servers/:mcp_id

## Changes

- Adds the Go implementation for `PUT /api/v1/mcp/servers/:mcp_id`.
- Wires MCP service and handler into the Go server/router for the update
route.
- Preserves Python-style behavior for ownership checks, partial update
fields, MCP type/name/URL validation, `headers`/`variables`
normalization, and tool metadata scrubbing.
2026-06-02 15:58:44 +08:00
Haruko386
0e9eeb7b88 feat[Go] implement /api/v1/datasets/<dataset_id>/metadata/config (#15493)
### What problem does this PR solve?

implement /api/v1/datasets/<dataset_id>/metadata/config

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-02 13:24:28 +08:00
web-dev0521
1696d4ead6 feat(go-api): implement password-reset flow (issue #15282) (#15293)
## Summary

Ports the Python password-reset flow to Go, adding 4 unauthenticated
endpoints under `/api/v1/auth/password/`:

- `POST /auth/password/forgot/captcha` — generates and returns a PNG
captcha image; stores the plaintext code in Redis (60 s TTL)
- `POST /auth/password/forgot/otp` — verifies captcha, enforces resend
cooldown (60 s), generates HMAC-SHA256-hashed OTP (300 s TTL), sends
plain-text email via SMTP
- `POST /auth/password/forgot/otp/verify` — verifies OTP with attempt
counting (lock after 5 failures for 30 min), sets a
`otp:verified:{email}` flag (300 s TTL) on success
- `POST /auth/password/reset` — checks verified flag, decrypts +
validates passwords, updates user record, auto-logs in (issues JWT,
returns user profile)

Closes #15282
2026-06-02 09:38:02 +08:00
Alexander Laurent
1748723971 feat: add Go MCP server list API (#15253)
## What
#15240 
Implements `GET /api/v1/mcp/servers` in the Go API server.

## Changes

- Added MCP server DAO list query with tenant scoping.
- Added MCP service response wrapper.
- Added MCP handler for list request parsing and response formatting.
- Wired `GET /api/v1/mcp/servers` under authenticated `/api/v1` routes.
- Initialized MCP service and handler in the Go server startup.
- update_time and update_date now both map to update_date
- create_time and create_date now both map to create_date
- default ordering now returns create_date
## API Behavior

Matches the Python endpoint behavior:

- Requires authenticated user.
- Lists MCP servers for the current user tenant.
- Supports `keywords`.
- Supports `mcp_id` and repeated/comma-separated `mcp_ids`.
- Supports `page`, `page_size`, `orderby`, and `desc`.
- Returns:

```json
{
  "code": 0,
  "message": "success",
  "data": {
    "mcp_servers": [],
    "total": 0
  }
}
```
2026-06-02 09:37:05 +08:00
sxxtony
12579dbc3d Go: implement dataset ingestion log APIs (#15421)
### What problem does this PR solve?

Part of the Python → Go API server rewrite tracked in #15240 (Dataset
ingestion section). This PR implements the three dataset ingestion
endpoints in the Go API server, mirroring the existing Python
`dataset_api_service` behaviour:

- `GET /api/v1/datasets/<dataset_id>/ingestions/summary`
- `GET /api/v1/datasets/<dataset_id>/ingestions`
- `GET /api/v1/datasets/<dataset_id>/ingestions/<log_id>`

### Type of change

- [x] Refactoring
- [x] New Feature (non-breaking change which adds functionality)

Co-authored-by: sxxtony <sxxtony@users.noreply.github.com>
2026-06-01 11:23:44 +08:00
Haruko386
2d7044b57e feat[Go] implement api/v1/thumbnails API (#15416)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality
2026-06-01 11:22:08 +08:00
Haruko386
d766e49128 feat[Go]: implement /system/stats and refactor /system/config/log (#15407)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-29 19:32:21 +08:00
Alexander Laurent
faa9c5469e feat: add Go MCP server delete API (#15262)
## What

#15240
Implementation for DELETE /api/v1/mcp/servers/:mcp_id
2026-05-29 19:29:55 +08:00
Hz_
09e91a8e61 Fix user registration initialization in Go API (#15349)
### What problem does this PR solve?

This PR fixes several behavior gaps in the Go implementation of the user
registration API.

### Type of change

- Make `nickname` required for user registration.
- Align registration error messages and response data with expected API
behavior.
- Handle password decryption errors for registration more consistently.
- Generate UUID v1-style IDs for new users, access tokens, tenants,
user-tenant records, and root files.
- Initialize default user fields during registration, including:
  - language
  - color schema
  - timezone
  - last login time
- Create user, tenant, user-tenant relation, tenant LLM records, and
root folder in a single DB transaction.
- Initialize default tenant LLM records from configured default models.
- Avoid partial registration data when one creation step fails.
- Use locale-based default language fallback for user profile responses.
2026-05-29 19:29:23 +08:00
bitloi
ea3a5dba11 fix: validate custom model inputs (#15200)
### What problem does this PR solve?

Closes #15199.

The add-custom-model endpoint is routed through
`/api/v1/providers/:provider_name/instances/:instance_name/models`, but
the handler previously trusted `provider_name` and `instance_name` from
the JSON body instead of the path target. A request could therefore hit
one provider/instance URL while operating on a different body
provider/instance.

The same handler only rejected `model_types` when the slice was nil. An
empty array passed validation and reached
`ModelProviderService.AddCustomModel`, where `request.ModelTypes[0]`
could panic.

This PR makes the path provider/instance authoritative, rejects
mismatched body values, rejects missing or empty `model_types`, and adds
a service-level guard so direct service callers cannot hit the same
panic path.

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
2026-05-29 10:15:01 +08:00
web-dev0521
550bdf215c feat(go-api): implement tenant member management (issue #15294) (#15295)
## Summary

Ports the Python `tenant_api` team/member management endpoints to Go,
adding 4 endpoints under `/api/v1/tenants/:tenant_id/`:

- `GET /tenants/:tenant_id/users` — list non-owner members with user
details (owner only)
- `POST /tenants/:tenant_id/users` — invite a user by email; creates
invite-role join record (owner only)
- `DELETE /tenants/:tenant_id/users` — remove a member by `user_id`;
owner can remove anyone, members can remove themselves
- `PATCH /tenants/:tenant_id` — accept a pending invitation,
transitioning role `invite → normal`

Closes #15294
2026-05-29 10:13:09 +08:00
Haruko386
834236a3ec feat[Go]: implement /api/v1/system/status GET (#15348)
### What problem does this PR solve?

As title

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-05-29 10:12:12 +08:00
nickmopen
e023c165b6 Fix(kb): enforce tenant authorization on UpdateMetadataSetting (#15268) (#15270)
## Summary

Closes #15268.

The `UpdateMetadataSetting` handler at `internal/handler/kb.go:126`
retrieved the authenticated user via `GetUser(c)` but discarded the user
object (`_, errorCode, errorMessage := GetUser(c)`), then forwarded the
caller-supplied `kb_id` straight to the service layer with no ownership
check. Any authenticated user could mutate the `parser_config` /
metadata of any knowledge base in the system by guessing or harvesting a
`kb_id` — a classic IDOR (CWE-284, OWASP A01).

This is the only handler in `internal/handler/kb.go` missing the check;
every sibling (`ListTags`, `ListTagsFromKbs`, `RenameTag`,
`KnowledgeGraph`, `DeleteKnowledgeGraph`, `GetMeta`, `GetBasicInfo`)
already calls `h.kbService.Accessible(kbID, user.ID)`. The same
defensive check on the document preview endpoint was added in PR #14625
— this PR closes the matching gap on the KB metadata endpoint.

---------

Co-authored-by: Jin Hai <haijin.chn@gmail.com>
2026-05-29 10:08:55 +08:00