156 Commits

Author SHA1 Message Date
Renzo
7d422ba67d feat(go): implement chatbots/<dialog_id>/info and searchbots/detail (#15420)
### What problem does this PR solve?

Part of #15240 (rewriting the RAGFlow API server in Go).

Implements the two public bot endpoints from
`api/apps/restful_apis/bot_api.py`:

- **`GET /api/v1/chatbots/<dialog_id>/info`** (`chatbots_inputs`) —
returns `{title, avatar, prologue, has_tavily_key}` for a dialog the
authenticated tenant owns (tenant match + `status == VALID`), otherwise
`"Authentication error: no access to this chatbot!"`.
- **`GET /api/v1/searchbots/detail`** (`detail_share_embedded`) —
returns search-app detail for a `search_id` the tenant can access.
Permission is checked across the tenant's joined tenants; denial returns
`"Has no permission for this operation."` (operating error, `data:
false`) and a missing app returns `"Can't find this Search App!"`.

Both endpoints authenticate with an SDK **beta token** (`Authorization:
Bearer <beta>`) rather than a session — the token is resolved to a
tenant via `APIToken.query(beta=token)`, backed by a new
`APITokenDAO.GetByBeta`. Because they perform their own token-based
auth, the routes are registered on the unauthenticated route group
(mirroring the Python blueprint, which has no `@login_required`).

Both live in a new `internal/handler/bot.go` + `internal/service/bot.go`
since they share the same source module. Handler unit tests cover the
auth, success, and error-mapping paths.

### Type of change

- [x] New Feature (non-breaking change which adds functionality)

---------

Co-authored-by: Claude Code <claude@anthropic.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Ling Qin <qinling0210@163.com>
2026-07-02 18:46:00 +08:00
Haruko386
9c8d8c7b83 fix: unable to load pic in chunk result (#16485)
### Summary

As title:
2026-07-02 16:05:49 +08:00
Hz_
0de69e5bba feat(go-api) sessions message update (#16517)
### Summary
```
/api/v1/chats/<chat_id>/sessions/<session_id>/messages/<msg_id> DELETE
/api/v1/chats/<chat_id>/sessions/<session_id>/messages/<msg_id>/feedback PUT
```
Migrates the chat session message delete and feedback APIs to the Go
server, matching the Python behavior for authorization, session
ownership checks, message/reference updates, and feedback validation.
2026-07-02 10:33:27 +08:00
qinling0210
133b1e15fd Port agent PRs to GO (#16529)
### Summary

Port the following PRs to GO in this PR

https://github.com/infiniflow/ragflow/pull/14210
https://github.com/infiniflow/ragflow/pull/14641
https://github.com/infiniflow/ragflow/pull/15220
https://github.com/infiniflow/ragflow/pull/15228
https://github.com/infiniflow/ragflow/pull/15384
https://github.com/infiniflow/ragflow/pull/15754
https://github.com/infiniflow/ragflow/pull/16413
https://github.com/infiniflow/ragflow/pull/16483
https://github.com/infiniflow/ragflow/pull/16419
https://github.com/infiniflow/ragflow/pull/16361   
https://github.com/infiniflow/ragflow/pull/16050
2026-07-02 09:45:01 +08:00
qinling0210
7862f69f39 Implement chat completions in go (#16491)
### Summary

POST   /api/v1/chat/completions
2026-07-01 15:52:52 +08:00
Wang Qi
8f24b30652 [Go] Add API /api/v1/chat/recommendation and consolidate with /api/v1/searchbots/related_questions (#16500) 2026-07-01 13:17:16 +08:00
Hz_
3633d08495 feat(go-api): Migrate Box web OAuth connector APIs to Go (#16480)
This PR migrates the Box web OAuth flow from Python to Go for:

  - POST /api/v1/connectors/box/oauth/web/start
  - GET /api/v1/connectors/box/oauth/web/callback
  - POST /api/v1/connectors/box/oauth/web/result
2026-06-30 18:10:36 +08:00
Haruko386
b3af9fc068 fix: remove dup-prefix in bot_routes (#16492) 2026-06-30 17:02:58 +08:00
Wang Qi
3bb976b383 [Go] Add /api/v1/searchbots/mindmap and /api/v1/chat/mindmap (#16443) 2026-06-30 16:35:33 +08:00
Zhichang Yu
4c54cefd29 Port 14 upstream agent security / correctness fixes to Go canvas (#16455)
Mirrors 14 merged upstream PRs into the Go agent port.

PRs ported:
  - #15609 ExeSQL SSRF guard + DNS pin
  - #15436 HTTP timeout on external API tools
  - #16363 be_output restore + DeepL error path
  - #15644 switch no longer matches empty condition
  - #15374 session_id bind to path agent_id (DAO idor guard)
  - #16169 sandbox artifact ownership gate
  - #15457 tenant ownership on agentbots
  - #15145 rerun agent document access check
- #15446 thinking switch (component portion; provider policy lives in
internal/llm)
  - #15426 Invoke URL/proxy SSRF + DNS pin + no-redirects
  - #15238 agentbot thinking-logs beta endpoint
  - #14589 UserFillUp SSE event propagation
  - #14890 anonymous webhook opt-in
- #15068 PipelineChunker new component (text/file_ref/parser_id
dispatch; file-format extraction is a follow-up)

40 files, +2355 / -58 lines. 33 new tests, all targeted package suites
pass (1721 + 4 skipped); 1 pre-existing flaky test unrelated.
2026-06-30 16:28:48 +08:00
Jin Hai
1087a25f22 Revert "feat(go-api): Add Go chat session message delete and feedback APIs" (#16465)
Reverts infiniflow/ragflow#16442
2026-06-29 21:37:11 +08:00
Haruko386
1c0cdd84ce feat[Go]: implement searches/<search_id>/completions POST (#16440)
### Summary

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
- [x] Bug Fix (non-breaking change which fixes an issue)
2026-06-29 20:07:12 +08:00
Wang Qi
48b77022f4 [Go] Fix beta auth for /documents/images/:image_id and /documents/:id/preview and /thumbnails (#16453) 2026-06-29 19:08:49 +08:00
Hz_
a553886989 feat(go-api): Add Go chat session message delete and feedback APIs (#16442)
### Summary

```
/api/v1/chats/<chat_id>/sessions/<session_id>/messages/<msg_id> DELETE
/api/v1/chats/<chat_id>/sessions/<session_id>/messages/<msg_id>/feedback PUT
```

Migrates the chat session message delete and feedback APIs to the Go
server, matching the Python behavior for authorization, session
ownership checks, message/reference updates, and feedback validation.

### Testing

  - `/usr/local/go/bin/go test ./internal/service ./internal/handler`
- Verified through the frontend page for deleting chat messages and
updating message feedback
2026-06-29 19:05:50 +08:00
Wang Qi
ec5cd6b1c0 [Go] Fix searchbot BETA auth (#16450) 2026-06-29 16:44:21 +08:00
Zhichang Yu
dfe2dc346d feat[Go]: port agent attachment download, chatbot + agentbot completion/info endpoints from Python (#16405)
## Summary

Ports five Python agent APIs to Go under the v1 Gin router:

- `GET  /api/v1/agents/attachments/<attachment_id>/download`
- `POST /api/v1/chatbots/<dialog_id>/completions`  (SSE)
- `GET  /api/v1/chatbots/<dialog_id>/info`
- `POST /api/v1/agentbots/<agent_id>/completions` (SSE)
- `GET  /api/v1/agentbots/<agent_id>/inputs`

Mirrors the existing Python wire shape (`{code, message,
data:{answer,reference,...}}` per Python `canvas_service.completion`) so
the iframe SDK and existing JS widgets keep working.

## Behavioural parity with Python

| # | Concern | How it's met |
|---|---------|--------------|
| R0 | Bot routes must not require regular user session | Routes mount
on `apiNoAuth` (router.go:198-202), with `BetaAuthMiddleware` only |
| R3 | Two SSE formats in Go drift | F2: `AgentChatCompletions` and
`AgentbotCompletion` share `service.WriteChatbotRunEvent` |
| R7 | `GetBySessionID` returns `(nil, nil)` on miss | Defensive
nil-check before `session.UserID != tenantID` |
| R8 | Begin component name vs ID | `FindBeginComponentID` resolves name
→ ID first, then `ExtractComponentInputForm(dsl, beginID)` |
| R9 | Defensive PromptConfig parsing | `stringFromMap` helper used for
`prologue` and `tavily_api_key` |
| R10 | `BetaAuthMiddleware` Bearer-prefix pre-filter | Removed —
`GetUserByToken` is called unconditionally, falls back to
`GetUserByBetaAPIToken` |
| F8 | Multi-turn chatbot history | `ChatbotCompletion` reads prior
turns from `session.Message`, appends user turn, calls LLM, persists new
pair via new `API4ConversationDAO.Update` |
| F9 | UUID gate stricter than plan | Removed — only `filepath.Base` +
CR/LF/quote header sanitization remains |
| H2 | Defence-in-depth IDOR | `AgentbotCompletion` calls `loadCanvas`
before delegating to `RunAgent` |
| M2 | SSE error leakage | `WriteChatbotFrame` emits generic `"an
internal error occurred"`; real error logged via `common.Error` |

## Verification

```bash
$ go vet ./...                                     # clean (only pre-existing issues)
$ go build ./...                                   # success
$ go test ./internal/handler/ ./internal/service/ ./internal/agent/dsl/ ./internal/common/ ./internal/dao/
ok  ragflow/internal/handler     0.617s
ok  ragflow/internal/service     1.729s
ok  ragflow/internal/agent/dsl   0.008s
ok  ragflow/internal/common      0.087s
ok  ragflow/internal/dao         0.083s
```

1199 tests pass across 5 packages.

## Known follow-ups (out of scope for this PR)

- **F1**: token-level streaming in `ChatbotCompletion` (currently emits
one frame per turn)
- **F3**: per-route `auth_types` attribute in Go (currently applied via
route group middleware)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-29 09:45:16 +08:00
Zhichang Yu
477f2fcebd feat[Go]: port agent webhook trigger, agent file upload/download, component input-form + debug endpoints from Python (#16403)
port agent webhook trigger, agent file upload/download, component
input-form + debug endpoints from Python
- [x] New Feature (non-breaking change which adds functionality)
2026-06-29 09:45:16 +08:00
Haruko386
a57a841a11 feat[Go]: implement Create-Chat/Session, Delete-Session (#16386)
### What problem does this PR solve?

As title:
implement:
```go
chats.POST("", r.chatHandler.Create)
chats.POST("/:chat_id/sessions", r.chatSessionHandler.CreateSession)
chats.DELETE("/:chat_id/sessions", r.chatSessionHandler.DeleteSessions)
```

bug fixed:

f80d4c7843/internal/handler/chat.go (L84)
↓
```go
result, err := h.chatService.ListChats(userID, "1", keywords, page, pageSize, orderby, desc)
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] New Feature (non-breaking change which adds functionality)
- [x] Refactoring
2026-06-26 19:23:45 +08:00
Hz_
e3063da390 feat(go-api): add chat update endpoints (#16378)
## Summary

- Added Go API route `PUT /api/v1/chats/:chat_id` to align with Python
`PUT /api/v1/chats/<chat_id>` chat update behavior.
- Added Go API route `PATCH /api/v1/chats/:chat_id` to align with Python
`PATCH /api/v1/chats/<chat_id>` partial chat update behavior.
- Added matching handler and service logic for owner checks, tenant
validation, persisted-field filtering, read-only field filtering,
`dataset_ids` to `kb_ids` conversion, and PATCH shallow merge semantics
for `prompt_config` and `llm_setting`.
2026-06-26 19:22:57 +08:00
Haruko386
a1f1dd5007 feat[Go]: implement Add messages for Go (#16375)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-26 19:21:52 +08:00
Jin Hai
2667995b25 Go CLI: Fix show model and list models (#16380)
### What problem does this PR solve?

```
RAGFlow(api/default)> show model 'WiseDiag-Z1 Think';

RAGFlow(api/default)> list models;

RAGFlow(admin)> show model 'WiseDiag-Z1 Think';

RAGFlow(admin)> list models;
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-26 15:36:01 +08:00
Jin Hai
8bc27d8df1 Go CLI: fix show variable (#16370)
### What problem does this PR solve?

```
RAGFlow(api/default)> show var 'mail.port';
+-----------+-----------+--------------+-------+
| data_type | name      | setting_type | value |
+-----------+-----------+--------------+-------+
| integer   | mail.port | config       | 30    |
+-----------+-----------+--------------+-------+
```

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-26 13:51:56 +08:00
Jin Hai
dbefadd86a Go CLI: refactor (#16355) 2026-06-25 20:36:50 +08:00
Hz_
e290a0d23e feat(go-api): Langfuse API key migration behavior (#16356)
## Summary

- Align Langfuse API key set/get/delete behavior with the Python
implementation.
- Improve DAO handling for Langfuse credential save/delete flows.
- Add tests for Langfuse service error handling and API key lifecycle
behavior.
2026-06-25 19:25:55 +08:00
Haruko386
43b96223b4 feat[go]: add router for connectors/<connector_id> PATCH (#16358)
### What problem does this PR solve?

As title

/api/v1/connectors/<connector_id> PATCH was implemented in #15512

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)
- [x] Refactoring
2026-06-25 19:07:52 +08:00
Haruko386
74597b8683 feat[Go]: implemet api: Search/Get/Update-Messages (#16307)
### What problem does this PR solve?

As title:
implement:
```
/api/v1/messages/search GET
/api/v1/messages GET
/api/v1/messages/<memory_id>:<message_id>/content GET
/api/v1/memories/<memory_id>/config GET
/api/v1/messages/<memory_id>:<message_id> PUT
```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-25 19:07:34 +08:00
Hz_
54fb5b0fa7 feat(go-api): add Go support for POST /api/v1/datasets/{dataset_id}/documents/{document_id}/chunks (#16256)
## Summary
Add the Go implementation of `POST
/api/v1/datasets/{dataset_id}/documents/{document_id}/chunks`.

This wires the full create-chunk path in Go:
- router and handler registration
- request/response structs
- chunk creation service logic
- embedding generation
- chunk insert into doc engine
- chunk/token counter increment
- `tag_feas` validation
- `image_base64` decoding and chunk image storage/merge
- unit tests for handler and service

## Testing
Unit tests:
- `/usr/local/go/bin/go test ./internal/handler`
- `/usr/local/go/bin/go test ./internal/service/chunk`
- `/usr/local/go/bin/go test ./internal/service`
- `/usr/local/go/bin/go test ./...`

All passed locally.

Manual curl checks:
- basic text chunk: Go passed
- chunk with `important_keywords` / `questions` / `tag_kwd` /
`tag_feas`: Go passed
- blank content validation: Go matched expected `code=102`
- invalid `image_base64` validation: Go matched expected `code=102`
- image upload and repeated image upload / merge path: Go passed twice
2026-06-25 14:15:29 +08:00
Hz_
a6cc3023c5 feat(go-api): implement dataset document upload API (#16295)
## Summary
Migrated the dataset document upload API (`POST
/api/v1/datasets/:dataset_id/documents`) from Python to the Go backend.
It supports local file uploads (`type=local`), web page ingestion
(`type=web`), and empty document creation (`type=empty`).

## Changes
- **Router**: Registered `POST /api/v1/datasets/:dataset_id/documents`
route.
- **Handler**: Implemented `UploadDocuments` handler and its routing
functions (`uploadLocalDocuments`, `uploadWebDocument`,
`uploadEmptyDocument`).
- **Service**: Implemented `UploadLocalDocuments`, `UploadWebDocument`,
and `UploadEmptyDocument` in `DocumentService`.
- **Refactoring**: Moved permission checking logic to a shared helper
for reuse in file and document services.
- **Tests**: Added comprehensive unit tests for the new handler and
service upload paths.

## Verification
Ran and passed the test suite for service and handler packages:
- `go test ./internal/service`
- `go test ./internal/handler`
2026-06-25 13:36:49 +08:00
Hz_
ced51114f4 feat(go-api): add dataset search endpoint (#16304)
### What problem does this PR solve?


- added the new dataset search route and handler
- reused the existing shared SearchDatasets service by adapting
single-dataset requests into dataset_ids=[dataset_id]
- aligned handler error responses with Python behavior for argument/data
errors
- aligned key service error messages such as invalid search_id and mixed
embedding models
- added focused handler and service tests for request mapping and error
behavior

### Tests:

`/usr/local/go/bin/go test ./internal/service -run
'TestSearchDatasetRequestToSearchDatasetsRequest|TestDatasetServiceSearchDatasets'`
`/usr/local/go/bin/go test ./internal/handler -run
'TestDatasetsHandlerSearchDataset'`
2026-06-25 13:32:22 +08:00
Haruko386
dd46ece3bc feat[go]: datasets/<dataset_id>/chunks DELETE (#16185)
### What problem does this PR solve?

As title:

`documents.POST("/ingest", r.documentHandler.Ingest)`:

---

<img width="3750" height="2039" alt="image"
src="https://github.com/user-attachments/assets/533c1c3d-af3e-47e6-9f51-a278539b7066"
/>

`datasets.DELETE("/:dataset_id/chunks", r.chunkHandler.StopParsing)`

---

<img width="3621" height="2040" alt="image"
src="https://github.com/user-attachments/assets/022adcdb-1e47-4883-9611-1a695c34007d"
/>


### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-24 19:43:18 +08:00
Haruko386
c2665d4ab1 implement: <dataset_id>/embedding/check POST (#16266) 2026-06-24 19:09:43 +08:00
Hz_
9a91564194 feat(go-api): align chat session get/update with python behavior (#16239)
## Summary

Align `/chats/:chat_id/sessions/:session_id` GET and PATCH with Python
behavior.
2026-06-24 17:34:01 +08:00
Hz_
dc8ff63f1d feat(go-api): add dataset tags endpoints (#16231)
## Summary

- add `GET /api/v1/datasets/:dataset_id/tags`
- add `PUT /api/v1/datasets/:dataset_id/tags`
- implement dataset tag listing and rename flow
- align rename tag validation and response shape with the Python API
- add handler and service tests for dataset tags

## Routes

- `GET /api/v1/datasets/:dataset_id/tags`
- `PUT /api/v1/datasets/:dataset_id/tags`

## Test

- Run specific tests for dataset tags:
```
go test -v ./internal/service ./internal/handler -run 'TestDatasetServiceListTags|TestDatasetServiceRenameTag|TestDatasetsHandlerListTags|TestDatasetsHandlerRenameTag'
```
- Run all tests for service and handler to verify no regressions:
```
go test ./internal/service ./internal/handler
```
- use curl cmd to test
2026-06-24 17:05:58 +08:00
Jin Hai
9624f70b22 Go CLI: refactor (#16299)
```
RAGFlow(api/default)> list dataset 'e93ab2c04ad111f1b17438a74640adcc' documents;
Total: 1

RAGFlow(api/default)> list datasets;


RAGFlow(api/default)> list chats;
Total: 2

RAGFlow(api/default)> list agents;
Total: 1

RAGFlow(api/default)> list searches;
Total: 1

RAGFlow(api/default)> list keys;
+----------------------------------+---------------+----------------------------------+-----------------------------------------------------+---------------+
| beta                             | create_time   | tenant_id                        | token                                               | update_time   |
+----------------------------------+---------------+----------------------------------+-----------------------------------------------------+---------------+
| GKsLEdSUkl76gJz1k_4fJpSQRIlWsiki | 1782285917523 | 2ba4881420fa11f19e9c38a74640adcc | ragflow-JgnarFSCUiV99oOvvMDei7ZzZg1cVlqGd1AMHrHeKE4 | 1782285917523 |
+----------------------------------+---------------+----------------------------------+-----------------------------------------------------+---------------+
RAGFlow(api/default)> create key;
SUCCESS

RAGFlow(api/default)> drop key 'ragflow-aA4R7AuUD158yh2LDh7IDBiqwOKFDKeTwUSQSLVdPdM';
SUCCESS
```

---------

Signed-off-by: Jin Hai <haijin.chn@gmail.com>
2026-06-24 16:50:40 +08:00
Hz_
e35860ad74 feat(go-api): Align document metadata batch APIs and upload_info with Python (#16269)
## Summary

  Align the Go implementations of these APIs with the Python behavior:

  - `POST /api/v1/datasets/:dataset_id/metadata/update`
  - `PATCH /api/v1/datasets/:dataset_id/documents/metadatas`
  - `POST /api/v1/documents/upload`

  ## What changed

  - Added the Go routes and handlers for the 3 APIs.
  - Aligned batch document metadata updates with Python semantics:
    - support `match` in update items
    - support list append / replace behavior
    - support deleting specific list values
    - remove metadata entirely when it becomes empty
- create metadata for documents that previously had none when updates
apply
    - count `updated` only when a document actually changes
- Aligned `documents/upload` file uploads with Python-style
`upload_info` behavior:
    - store upload-info blobs in the per-user downloads bucket
- return lightweight upload descriptors instead of normal
file-management responses
  - Improved URL upload behavior:
    - SSRF-guarded fetch with redirect validation
    - redirect limit aligned to Python behavior
    - normalize filename and MIME type
    - add `.pdf` when the fetched content is PDF
- normalize HTML content into readable text instead of storing raw HTML
shells

  ## Validation

  ### Unit tests

  Passed:

  - `go test ./internal/service`
  - `go test ./internal/handler`

  Also verified targeted cases for:

  - batch metadata update semantics
  - upload_info URL handling
  - upload_info download bucket behavior

  ### curl checks

Verified the new Go endpoints with `curl` and compared the response
shape and behavior with Python for:

  - `POST /api/v1/datasets/{dataset_id}/metadata/update`
  - `PATCH /api/v1/datasets/{dataset_id}/documents/metadatas`
  - `POST /api/v1/documents/upload`

  The Go responses were checked against Python for:
  - argument validation
  - success response shape
  - metadata update results
  - upload_info result structure
  - file vs URL input handling
2026-06-24 14:52:47 +08:00
Haruko386
97718ec779 feat[Go] implement datasets/<dataset_id>/<index_type> DELETE (#16257)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-24 14:47:55 +08:00
Hz_
368db6fa58 feat(go-api): migrate datasets tags aggregation API to Go (#16181)
### Description

Migrates the datasets tags aggregation API `GET
/api/v1/datasets/tags/aggregation` from Python to Go.

### Changes
- Registered the `GET /api/v1/datasets/tags/aggregation` route.
- Implemented `AggregateTags` in datasets `handler` and `service`.
- Added handler and service `unit tests`.

### Test Verification
- Verified by comparing results between Python (9380) and Go (9384)
services.
- Tested scenarios: single dataset, multiple datasets, empty parameters,
and unauthorized/invalid IDs.
- All tests and Go `unit tests` passed.
2026-06-24 14:42:10 +08:00
Haruko386
5046626c17 feat[Go]: implement /datasets/<dataset_id>/documents/batch-update-status (#16258)
### What problem does this PR solve?

accident close #16072

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-23 19:19:08 +08:00
Haruko386
6cbd069ea3 feat[Go]: implement <document_id>/chunks/<chunk_id> PATCH (#16232)
### What problem does this PR solve?

Implement: 
1. `/api/v1/datasets/<dataset_id>/documents/<document_id>/chunks GET`
2.
`/api/v1/datasets/<dataset_id>/documents/<document_id>/chunks/<chunk_id>
PATCH`
3. `/api/v1/datasets/<dataset_id>/documents/<document_id>/chunks PATCH`

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-23 18:50:36 +08:00
Jin Hai
dec2ce4a60 Go CLI: admin model framework (#16252) 2026-06-23 16:57:05 +08:00
Zhichang Yu
2362210caf refactor(log): unify Go logging to zap with rotation, strip per-package levels (#16261)
Refactor the Go agent port's logging so every log line — gin access,
agent canvas events, harness warnings, fatal boot errors — flows through
a single common.Logger (zap) backed by a rotated file, with structured
fields, level filtering, and configurable rotation.

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-23 16:21:46 +08:00
Hz_
9eb7cee473 feat(go-api): migrate searchbot share detail endpoint to go (#16124)
## Summary

- add public Go route for `/api/v1/searchbots/detail`
- implement beta-token auth flow for shared search access
- add tenant-based access check for shared search apps
- add joined search detail query for the share response
- align Go response shape with the current Python runtime behavior
- add DAO / service / handler tests for the new endpoint
2026-06-22 18:17:37 +08:00
Hz_
2856cde2d1 feat(go-api): Implement BulkDeleteChats Go API and fix ListChats (#16157)
### Description
- **Bulk Delete Chats**: Implemented Go endpoint `DELETE /api/v1/chats`
supporting bulk delete by `ids`, `delete_all` flag, and
backward-compatible `chat_id` body payload (with tenant-ownership
security checks).
- **Bug Fix**: Fixed a parameter swap in Go `ListChats` handler to
properly exclude soft-deleted chats.
2026-06-22 18:16:52 +08:00
Hz_
4e0db3053d feat(go-api): complete chat channel API migration with tests (#16139)
close #16132

## Summary

This PR completes the Go-side merge and cleanup for chat channel APIs,
including handler/service wiring, route registration, and test coverage.

Implemented and aligned 5 chat channel APIs:

```
- POST `/api/v1/chat-channels`
- GET `/api/v1/chat-channels`
- GET `/api/v1/chat-channels/:channel_id`
- PATCH `/api/v1/chat-channels/:channel_id`
- DELETE `/api/v1/chat-channels/:channel_id`
```


Co-authored-by: Haruko386 <tryeverypossible@163.com>
2026-06-22 18:16:15 +08:00
Haruko386
b777e50291 feat[Go]: implement api /api/v1/datasets/<dataset_id>/chunks POST (#16067)
### What problem does this PR solve?

As title

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-22 18:14:01 +08:00
Zhichang Yu
3f805a64f1 feat(agent): align Go agent behavior with Python (except retrieval component) (#16225)
## Summary

Aligns the **Go agent runtime/canvas/components/tools** behavior with
the **Python `agent/` implementation** so the same stored canvas DSL
produces the same execution result on either side. Every component,
tool, and runtime primitive in `internal/agent/` is now driven by the
same semantics as its Python counterpart — variable resolution, template
substitution, control flow, error reporting, retry/cancel, and stream
event shapes.

The **retrieval component is the one explicit exception** in this PR. It
is being reworked in a separate change and is excluded from this
alignment pass; the wrapper slot (`universe_a_wrappers.go →
newRetrievalComponent`) is preserved.

## Scope of alignment

### Components (all aligned with `agent/component/`)
`Begin` · `Message` · `LLM` (incl. ChatTemplateKwargs,
MessageHistoryWindowSize, VisualFiles, Cite, OutputStructure,
JSONOutput, TopP, MaxRetries, DelayAfterError, credentials) · `Agent`
(react + tool artifact capture + `Reset()` interface-assert) · `Switch`
(12/12 operators, Python-equivalent semantics) · `Categorize` · `Invoke`
· `Iteration` · `Loop` (macro-expansion through `workflowx.AddLoopNode`)
· `UserFillUp` (Python-equivalent interrupt/resume via eino
`compose.Interrupt`/`ResumeWithData`) · `FillUp` · `DataOperations` ·
`ListOperations` · `StringTransform` · `VariableAggregator` ·
`VariableAssigner` · `Browser` (full stagehand runtime parity) ·
`DocsGenerator` · `ExcelProcessor`.

### Tools (all aligned with `agent/tools/`)
`Retrieval` (wrapper slot only — logic out of scope) · `MCPToolAdapter`
(streamable-HTTP) · `CodeExec` (sandbox bridge with
`code_exec_contract.go` matching Python contract) · `AkShare` · `ArXiv`
· `Crawler` · `DeepL` · `DuckDuckGo` · `Email` · `ExeSQL` · `GitHub` ·
`Google` · `GoogleScholar` · `Jin10` · `PubMed` · `QWeather` · `SearXNG`
· `Tavily` · `Tushare` · `Wencai` · `Wikipedia` · `YahooFinance` —
uniform `eino tool.InvokableTool` interface, SSRF protection, shared
HTTP client.

### Canvas execution engine (`internal/agent/canvas/`)
Aligned with Python's `agent/canvas.py`:
- **Scheduler** (`scheduler.go`): state pre/post handlers, node lambdas,
per-component timeout resolver (4-level: per-class env → per-class table
→ uniform env → 600s fallback), `legacyNoOpNames`.
- **Loop subgraph** (`loop_subgraph.go`): Python-equivalent
`AddLoopNode` macro expansion + condition translation.
- **Multibranch** (`multibranch.go`): `Switch` / `Categorize` routing
via `compose.NewGraphMultiBranch` — same branch selection semantics as
Python.
- **Parallel subgraph** (`parallel_subgraph.go`): matches Python's
parallel fan-out contract.
- **Interrupt/Resume** (`interrupt_resume.go`): `UserFillUpNodeBody` /
`IsInterruptError` / `ExtractInterruptContexts` — replaces the
deprecated Python sentinel chain with eino's native interrupt API,
preserving the same external behavior.
- **Checkpoint** (`checkpoint_store.go`): `RedisCheckPointStore`
Get/Set/Delete, with business metadata (status / canvas_id /
parent_run_id) on a parallel Redis Hash.
- **RunTracker** (`run_tracker.go`): Start / MarkSucceeded / MarkFailed
/ MarkCancelled / AttachCheckpoint — same lifecycle as the Python run
record.
- **Cancel** (`cancel.go`): Redis pub/sub watch.
- **Stream** (`stream.go`): SSE channel with `messages` / `waiting` /
`errors` / `done` events, same shape as Python's `agent.canvas.RunEvent`
payload.

### DSL bridge (`internal/agent/dsl/`)
- `normalize.go`: v1↔v2 collapsed into a single wire format — Python and
Go consume the same stored JSON.
- `reset.go`: per-run state reset matches Python's `Canvas.reset()`
semantics.
- Testdata mirrors Python's `agent_msg.json` / `all.json` / etc.

### Runtime (`internal/agent/runtime/`)
- `CanvasState` / `NewCanvasState` / `GetVar` / `SetVar` / `ReadVars`:
same `{{cpn_id@param}}` resolution model.
- `ResolveTemplate` (regex fast path + gonja fallback) — Python
Jinja-style semantics.
- `selector.go`, `metrics.go`, `component.go`: shared runtime contracts.

## Out of scope (intentionally)

- **`Retrieval` component logic** — wrapped only; full parity lands in a
follow-up PR.
- **Frontend** — only minor dsl-bridge / canvas UX fixes ride along.
- **CLI / admin / model registry** — orthogonal to agent behavior.

## How alignment is verified

`internal/service/agent_run_e2e_test.go` exercises the **full production
chain** against real Python-shaped DSL fixtures:
```
loadCanvasForUser → versionDAO.GetLatest → decodeCanvasFromDSL →
canvas.Compile → cc.Workflow.Invoke → answer extraction
```
using in-memory SQLite + miniredis (no Docker). Covers:
- `TestRunAgent_RealCanvas_BeginMessage` — happy path, `{{sys.query}}`
resolution
- `TestRunAgent_RealCanvas_WaitForUserResume` — two-run resume cycle
(Python-equivalent)
- `TestRunAgent_RealCanvas_CompileFails` — unknown component name →
sanitized error (Python-equivalent)
- `TestRunAgent_RealCanvas_InvokeFails` — unresolvable template ref
(Python-equivalent)
- `TestRunAgent_RunTracker_AttachCheckpoint_CallSequence` —
Start→AttachCheckpoint→MarkSucceeded lifecycle

`internal/handler/agent_test.go` — SSE streaming parity (`Content-Type:
text/event-stream`, `data: {…}\n\n`, trailing `data: [DONE]\n\n`,
OpenAI-compatible non-stream `choices`).

`internal/agent/canvas/fixture_compile_test.go` + per-component tests
pin the Python-equivalent outputs.

```
go test -count=1 -v -run 'TestRunAgent_RealCanvas|TestRunAgent_RunTracker' ./internal/service/
```

## Design reference

`docs/develop/agent-go-port-design.md` (1329 lines, last cross-checked
2026-06-17) — module layout, per-component / per-tool inventory,
corner-case catalogue, and the actionable backlog (Section 14, including
the retrieval alignment follow-up).

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-06-22 11:58:29 +08:00
qinling0210
563d855780 Implement OpenAI chat completions in GO (#16177)
### What problem does this PR solve?

Implement OpenAI chat completions in GO

POST /api/v1/openai/<chat_id>/chat/completions

OpenAI chat cli: internal/development.md

### Type of change

- [x] Refactoring
2026-06-18 18:07:27 +08:00
Haruko386
217c2a94c2 feat[Go]: implement datasets/<dataset_id>/index P/G (#16153)
### What problem does this PR solve?

```
POST: http://localhost:9384/api/v1/datasets/433b390c630411f1a13eab5f89540b2a/index?type=graph

Output: {
    "code": 0,
    "data": {
        "task_id": "ff5a3546bafa49d794a9a050d99c4a52"
    },
    "message": "success"
}
```

---

```
GET: http://localhost:9384/api/v1/datasets/433b390c630411f1a13eab5f89540b2a/index?type=graph

Output: {
    "code": 0,
    "data": {
        "id": "ff5a3546bafa49d794a9a050d99c4a52",
        "doc_id": "graph_raptor_x",
        "from_page": 100000000,
        "to_page": 100000000,
        "task_type": "graphrag",
        "priority": 0,
        "begin_at": "2026-06-17T18:07:45+08:00",
        "process_duration": 4.108135,
        "progress": -1,
        "progress_msg": "18:07:45 created task graphrag\n18:07:47 Task has been received.\n18:07:49 [ERROR][Exception]: Model config not found: Qwen/Qwen3-235B-A22B@test@SILICONFLOW",
        "retry_count": 1,
        "digest": "f16fd067d5c92cec",
        "create_time": 1781690865552,
        "create_date": "2026-06-17T18:07:45+08:00",
        "update_time": 1781690869108,
        "update_date": "2026-06-17T18:07:49+08:00"
    },
    "message": "success"
}

```

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-18 17:57:24 +08:00
Haruko386
5f6ebc97c6 feat[go]: implement /api/v1/datasets/<dataset_id> PUT (#16122)
### What problem does this PR solve?

As pic shows

### Type of change

- [x] New Feature (non-breaking change which adds functionality)
2026-06-18 17:57:07 +08:00
Hz_
69dbc44983 feat(go-api): migrate MCP server detail and download API to Go (#16113)
### What problem does this PR solve?

- Migrated MCP server detail and export (download) API from Python to
Go.
- Registered route: `GET /api/v1/mcp/servers/:mcp_id` (supporting
`?mode=download` query parameter).
2026-06-18 11:09:22 +08:00