### Summary
As title
- [x] fix tool & component `Retrieval KB`
- [x] fix agent cannot use `Retrieval KB` in agent chat
#### Main cause
Model provider do not set up:
```Go
if chatModelConfig.Tools != nil {
reqBody["tools"] = chatModelConfig.Tools
}
```
#### Working Now
<img width="3774" height="2128" alt="image"
src="https://github.com/user-attachments/assets/400a349d-0211-43e5-a7ec-7a014acf77a6"
/>
```
____________________________________
< This PR takes me all day to do it. >
------------------------------------
\
\ (\__/)
(•ㅅ•)
/ づ
```
## Summary
- return the input form collected by
`DataOperationsComponent.GetInputForm`
- add a regression test covering system variables, component outputs,
and duplicate references
Ports two Python fixes to Go: the variable_ref_patt underscore/colon fix
(#16792) and the TokenChunker upstream-chunks fix (#16825). Keeps Go
behavior aligned with upstream Python.
## Summary
- Align Go WenCai and SearXNG behavior, schemas, and node parameters
with Python.
- Add the `WenCai` and `SearXNG` Canvas components and register their
tool factories.
- Match Python's current WenCai behavior by returning an empty report
while its upstream request is disabled.
- Add SearXNG request validation, SSRF-safe DNS pinning, raw result
preservation, and reference rendering.
- Support context cancellation, error envelopes, and lock-safe retrieval
references.
## Tests
Passed:
- `bash build.sh --test ./internal/agent/tool/...`
- `bash build.sh --test ./internal/agent/component/...`
- `bash build.sh --test ./internal/agent/runtime/...`
- `bash build.sh --test ./internal/agent/...`
- `cd web && npm run type-check`
<img width="1900" height="1102" alt="image"
src="https://github.com/user-attachments/assets/ec77d217-d9fd-455a-96ec-9aabf6841109"
/>
<img width="1900" height="1102" alt="image"
src="https://github.com/user-attachments/assets/52ac129f-cb65-453d-ae48-cc518803ac23"
/>
## Summary
- Add the GitHub Canvas component with tool registration and reference
propagation.
- Align the Invoke component with the Python contract for node config,
input form, response output, and timing fields.
- GitHub search and HTTP Invoke now work correctly in the Go Canvas
runtime.
## Tests
- `bash build.sh --test ./internal/agent/tool/...`
- `bash build.sh --test ./internal/agent/component/...`
Note: the untracked go_ragflow_cli file is not part of the PR changes.
<img width="1813" height="1102" alt="image"
src="https://github.com/user-attachments/assets/f69cef32-59a0-4287-a06b-6843d85198cf"
/>
<img width="1813" height="1102" alt="image"
src="https://github.com/user-attachments/assets/b37dfc31-bc9b-4937-a38e-d2184bb157fe"
/>
## Summary
- register the Go `ArXiv` canvas component and add its input form
- align the Go ArXiv request/schema with Python by keeping only `query`
in runtime args and moving `top_n`/`sort_by` to node params
- keep ArXiv results consistent for canvas output and tool response
handling
## Test
- `bash build.sh --test ./internal/agent/tool
./internal/agent/component`
<img width="1817" height="972" alt="image"
src="https://github.com/user-attachments/assets/7f726dfa-a996-4561-b481-cb0b44bec81c"
/>
## Summary
- Register Wikipedia component + tool alias
`wikipedia`/`wikipedia_search`
- Use `generator=search` to get title/summary/url in one request (was
N+1)
- Node params `top_n`/`language` with validation
- Return `formalized_content` for downstream
- tests pass
<img width="1817" height="972" alt="image"
src="https://github.com/user-attachments/assets/f6d79599-6d1f-4ea6-84f7-ac06d0de13b0"
/>
Two refactors on the Go port (agent-go-port):
- Remove the dead per-tenant canvas runtime selector (write-only Redis
scaffolding with no runtime callers) and its dependent metrics/admin
code.
- Move the tokenizer embedding-model id from the shared ingestion
globals into a Tokenizer-scoped setup, and wire the production embedder
resolver in the ingestion task package.
32 files changed, 861 insertions, 1228 deletions.
## Summary
- align the Go Google Scholar component with the Python-side config
pattern
- merge node-level params with runtime inputs so canvas defaults are
preserved and per-run inputs can override them
- add tests covering node param fallback and runtime override behavior
## Verification
- `bash build.sh --test ./internal/agent/component/... -run
TestGoogleScholar`
<img width="1873" height="1165" alt="image"
src="https://github.com/user-attachments/assets/67198c6f-6a0e-43bf-a500-8e88d82b8751"
/>
feat(ingestion): mirror Go pipeline progress into the document table;
harden resume guards
- pipeline: bind the owning document via WithDocumentID; after each
TrackProgress event aggregate ingestion_task_log progress and mirror
progress/run/progress_msg back into the document table, so GET
/api/v1/datasets/{dataset_id}/documents reflects live Go pipeline
progress without a bespoke endpoint.
- canvas: extend the S3 resume guard to reject legacy no-op nodes (e.g.
ExitLoop) so component_total equals the count of progress-reporting
components and the aggregate percent can reach 100%.
- runtime/canvas: route progress through TrackProgress; add interrupt
test coverage (r3_interrupt_test.go).
- dao/entity: add IngestionTask.DocumentID column and AggregateProgress
support used by the mirror; IngestionTaskLog keeps a Checkpoint column
alongside the progress fields.
feat(deepdoc): cache DocAnalyzer inference results in Redis (1h TTL)
- Redis-backed DocAnalyzerCache decorator over inference.Client; cache
key = "ddoc:cache:<method>:" + sha256 of the JPEG-encoded image bytes
(deterministic).
- TTL = 1h; hits skip the inner HTTP call and return cached JSON; inner
errors are not cached.
refactor(deepdoc): align figure cropping with Python cropout + bounded
page caches
- CropSectionByDLA mirrors Python cropout: best-overlap DLA
figure/equation region, fallback to section bbox per page, vertical
concat on gray background.
- sliding-window page-image cache bounds peak memory to the recent
window instead of the whole PDF.
- rename DLADebug -> DLARegions across parser/chunker/tests.
refactor(parser): drop lib_type selector; align NewXxxParser with
NewPDFParser
- remove config["lib_type"] lookup and the libType param/field/switch
from all nine constructors; surface the CGO-required error at
ParseWithResult time instead of construction time; drop resolveLibType,
its test, and the four lib_type constants.
feat(utility): add a reusable workerpool for bounded concurrent
execution
- internal/utility/workerpool.go (+ tests).
refactor: translate Chinese prose comments to English in non-harness Go
files.
chore: upgrade github.com/cloudwego/eino from v0.9.9 to v0.9.12.
## Summary
- register the Go `DuckDuckGo` canvas component and restore its dynamic
input form metadata
- align the Go component input/output surface with the current canvas
usage for `query`, `channel`, and `top_n`
- fix DuckDuckGo news search in Go by fetching the required `vqd` token
before calling `news.js`, and add targeted regression tests
## Testing
Passed:
- `bash build.sh --test ./internal/agent/tool/... -run 'DuckDuckGo'`
- `bash build.sh --test ./internal/agent/component/... -run
'DuckDuckGo|TestVerifyRegistration_P1'`
- `bash build.sh --test ./internal/agent/component/... -run
'DuckDuckGo'`
Not run:
- frontend tests
- frontend build
- full Go test suite
<img width="1776" height="1092" alt="image"
src="https://github.com/user-attachments/assets/9f3f8e4b-f6b4-4915-b96c-3c5b8c7b8b30"
/>
### Summary
- Implemented googleComponent wrapper to bridge the canvas component
contract with Eino's SerpApi-backed GoogleTool.
- Added parameter alias mapping (query to q, max_results to num) and
content formatting logic to match Python search result representation.
- Registered the "Google" component and the "google" tool factory in the
Go agent runtime to support web search nodes.
<img width="1776" height="1092" alt="image"
src="https://github.com/user-attachments/assets/e295ab88-e48c-4fe2-bcb7-47ca5b977c9b"
/>
### Summary
- TavilySearch now stores api_key from component params and injects it
into tool calls when runtime inputs omit it.
- TavilyExtract and BGPT now follow the same stored api_key behavior.
- Canvas decoding now recovers api_key from graph.nodes[].data.form when
components[].obj.params.api_key is empty, matching frontend payload
behavior without changing frontend data.
- Added regression tests for graph form key recovery and stored key
injection / caller key precedence.
Tests: build.sh --test ./internal/agent/component ./internal/service —
all pass.
<img width="1476" height="850" alt="image"
src="https://github.com/user-attachments/assets/0be31587-c1ba-4f3e-b43a-4fe0fca5a44c"
/>
<img width="1476" height="850" alt="image"
src="https://github.com/user-attachments/assets/e3edd92c-c62e-4db4-abe2-772bdf4fe1b2"
/>
## Summary
Adds the missing input form metadata for the Go BGPT canvas component.
## Root Cause
The standalone BGPT component was registered in Go, but it did not
implement GetInputForm(). During component trial run, the backend asks
the component for its input_form. Since BGPT had none, the API returned:
component has no input_form: BGPT:<node_id>
Python BGPT already exposes the query input form, so the Go component
needed the same contract.
## Change
Added GetInputForm() to the Go BGPT component with a single query line
input.
Added test coverage to ensure BGPT exposes the input form.
## Validation
Backend:
bash build.sh --test -run TestBGPT ./internal/agent/component
<img width="1369" height="1184" alt="image"
src="https://github.com/user-attachments/assets/f99e4a81-2359-42e5-80bb-dcc4e6a63fea"
/>
<img width="1736" height="1152" alt="image"
src="https://github.com/user-attachments/assets/c11240a5-2c42-4d08-88e3-c6dfbf49eedb"
/>
## Summary
- Add Go dynamic input form support for ExeSQL and Browser components.
- Align their input form metadata with the Python implementation.
- Add regression tests for `/components/:component_id/input-form`.
## Summary
Debugging YahooFinance component in agent canvas returns "unknown
component" and "no input_form".
YahooFinance was only registered as an eino tool, not as a runtime
component. The component factory only searches the runtime registry.
- `universe_a_wrappers.go`: add `yahooFinanceComponent` wrapper
delegating to `agenttool.YahooFinanceTool` with `GetInputForm()`
- `fixture_stubs.go`: register `"YahooFinance"` component
## TEST
`go build` and `go test ./internal/agent/component/...` all pass.
## Summary
- derive Go Agent debug input forms from prompt variable references
instead of Agent meta fields
- seed `sys.*` debug params into `CanvasState.Sys` so single-component
debug resolves prompt variables like Python
- restore Agent test-run parity for form rendering and debug execution
## Tests
- `go test ./internal/agent/component -run
'TestAgent_(GetInputForm_UsesPromptReferences|GetInputForm_DeduplicatesPromptReferences|Meta_DefaultsToEmpty|Reset_NoTools)$'`
- `go test ./internal/handler -run
'Test(DebugComponent_SeedsSysInputsIntoCanvasState|DebugComponent_HappyPath_Begin|GetComponentInputForm_HappyPath)$'`
AFTER:
<img width="669" height="456" alt="image"
src="https://github.com/user-attachments/assets/4fd86559-aafc-4027-91ae-6e666137ee1b"
/>
### Summary
The Go backend Agent component was not returning artifacts produced by
the CodeExec tool. While the Python agent collects the "`_ARTIFACTS`"
envelope from tool responses and appends artifact markdown to the final
content, the Go agent only returned the assistant text, so generated
images were missing from the chat output.
### Changes
- Wire `react.WithMessageFuture()` in `runEinoReActAgent` and store the
resulting `MessageFuture` in the invocation context.
- After the ReAct loop finishes, drain the future and extract
``_ARTIFACTS`` entries from every tool response message.
- Support reading the tool payload from both `msg.Content` and
`msg.UserInputMultiContent` to match eino's tool contract.
- De-duplicate artifacts by URL and render images as `!` and other files
as download links.
- Add `agent_artifact_test.go` with a regression test that simulates a
CodeExec-style tool response carrying an image artifact and verifies it
is collected and formatted.
### Verification
- `go test ./internal/agent/component/... -run
TestAgent_ReActAgent_CollectsArtifactsFromCodeExecTool` passes.
- `go test ./internal/agent/component/... -count=1` compiles; the only
failure is an unrelated DNS-pinning timeout test
(`TestInvoke_ProxyDNSPin`).
- `gofmt` clean for modified files.
### Related
Fixes the behavior shown in the screenshot where the Go agent ignored
the CodeExec-generated PNG artifact.
### Summary
Adopt sync.WaitGroup.Go (Go 1.25) to simplify tracked goroutine
spawning. This replaces the error-prone trio of wg.Add(1), go func(),
and defer wg.Done() with a single, self-contained call.
More info: https://github.com/golang/go/issues/63796
Signed-off-by: grandpig <grandpig@outlook.com>
Replace fragile wall-clock timeout assertions with semantic checks for
deadline errors, retry suppression, and event ordering. Keep only
lower-bound timing checks where they prove backoff behavior. This
reduces CPU-load flakes without weakening regression coverage.
