### Summary
The Go backend never seeded the `canvas_template` table, so the "Create
agent from template" page was blank when the frontend proxies to the Go
API (`API_PROXY_SCHEME=go`). This PR adds `SeedCanvasTemplates()` in
`internal/dao`, invoked from `InitDB()` after migrations, which loads
`agent/templates/*.json` and mirrors Python's `add_graph_templates()`
behavior.
Changes:
- Add `internal/dao/canvas_template_seed.go` to parse and upsert
built-in templates.
- Call `SeedCanvasTemplates()` in `InitDB()`.
- Add `CanvasTypes` (`JSONSlice`) to `entity.CanvasTemplate` so the
frontend can filter/group by category.
- Skip seeding gracefully when the templates directory is absent.
This fixes the blank template catalogue in Go mode.
## Summary
Adds the missing input form metadata for the Go BGPT canvas component.
## Root Cause
The standalone BGPT component was registered in Go, but it did not
implement GetInputForm(). During component trial run, the backend asks
the component for its input_form. Since BGPT had none, the API returned:
component has no input_form: BGPT:<node_id>
Python BGPT already exposes the query input form, so the Go component
needed the same contract.
## Change
Added GetInputForm() to the Go BGPT component with a single query line
input.
Added test coverage to ensure BGPT exposes the input form.
## Validation
Backend:
bash build.sh --test -run TestBGPT ./internal/agent/component
<img width="1369" height="1184" alt="image"
src="https://github.com/user-attachments/assets/f99e4a81-2359-42e5-80bb-dcc4e6a63fea"
/>
<img width="1736" height="1152" alt="image"
src="https://github.com/user-attachments/assets/c11240a5-2c42-4d08-88e3-c6dfbf49eedb"
/>
### Motivation
This PR evolves the harness from a pure execution runtime into an
**observable, replayable agent evaluation platform**. The current
`harness/graph` checkpoint mechanism is insufficient for true
event-sourced introspection—we need append-only event logs capturing
every tool call, state transition, memory write, and approval decision,
enabling deterministic replay, fork/diff, postmortem analysis, and
time-travel debugging.
### Key Design Goals
1. **Event-Sourced Execution Model**
Replace coarse checkpoints with granular, append-only event logs. Every
operation becomes a durable event: tool invocation, state mutation,
memory update, human approval. This unlocks deterministic replay,
branching execution histories, and regression datasets derived directly
from production failures.
2. **First-Class Replay & Evaluation Loop**
Replay is not an afterthought—it is a core primitive. A single live run
seeds an offline corpus that supports: repeated playback, model
substitution, tool result mocking, and strategy comparison. The harness
graduates from "executor" to "continuous evaluation platform" where
failed production traces convert directly into offline regression
suites.
3. **Operational Observability**
Beyond raw traces, expose metrics that prove stability over time:
- Tool success / failure rates
- Approval latency distributions
- Retry frequencies
- Checkpoint restore reliability
- Memory retrieval quality
- Cost per completed task
- Fork replay pass rates
The underlying thesis: the bottleneck for most agent systems is not
execution capability, but the inability to **demonstrate continuous,
measurable improvement**.
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Ports remaining Go parser wiring and PDF backends, adds tenant-aware VLM
dispatch, aligns post-processing with Python, and adds end-to-end
pipeline coverage with a generated six-page PDF.
### Summary
Keep `data` as the uploaded document array when dataset document upload
partially succeeds.
This matches the Python API behavior and allows parse-on-creation to run
for successfully uploaded files when other files in the same folder are
unsupported.
## Summary
- Add Go dynamic input form support for ExeSQL and Browser components.
- Align their input form metadata with the Python implementation.
- Add regression tests for `/components/:component_id/input-form`.
## Summary
Debugging YahooFinance component in agent canvas returns "unknown
component" and "no input_form".
YahooFinance was only registered as an eino tool, not as a runtime
component. The component factory only searches the runtime registry.
- `universe_a_wrappers.go`: add `yahooFinanceComponent` wrapper
delegating to `agenttool.YahooFinanceTool` with `GetInputForm()`
- `fixture_stubs.go`: register `"YahooFinance"` component
## TEST
`go build` and `go test ./internal/agent/component/...` all pass.
### What problem does this PR solve?
The Go chunk pipeline's `PostprocessOperator` `filter` stage
(`internal/ingestion/chunk/postprocess.go`) only filtered by length
(`min_length`/`max_length`). It could not drop empty/whitespace-only
chunks or duplicate chunks — both standard RAG post-processing steps
(blank chunks shouldn't be indexed; identical chunks waste embedding
compute and add redundant retrieval results).
This adds two optional, default-off booleans to the `filter` config:
- `drop_empty` — drop chunks whose content is empty or whitespace-only.
- `drop_duplicates` — drop chunks whose exact content already appeared
(order-preserving; the first occurrence is kept).
They compose with the existing length bounds and are reflected in
`String()` for plan explainability. Also adds the first unit tests for
the postprocess filter (length bounds, drop_empty, drop_duplicates,
combined, exact-content matching, and config parsing).
Validation: `gofmt` clean, `go vet ./internal/ingestion/chunk/` clean,
`go build` ok, `go test ./internal/ingestion/chunk/` — all tests pass.
Closes#16048
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: Ling Qin <qinling0210@163.com>
### What problem does this PR solve?
The Go ingestion chunk pipeline's `SplitOperator`
(`internal/ingestion/chunk/split.go`) supported only `sentence`, `char`,
and `paragraph` strategies, but not **fixed-size (length) chunking with
overlap** — the canonical RAG strategy for bounding chunk length while
preserving cross-boundary context.
This adds a `length` strategy alongside the existing ones, configurable
via DSL `params`:
- `chunk_size` — target window size in **runes** (rune-aware:
multi-byte/CJK text is windowed by character, never split mid-rune).
- `overlap` — runes carried from the end of each window into the next.
The window advances by `chunk_size - overlap`. `chunk_size` falls back
to a default (256) when unset/non-positive, and `overlap` is clamped to
`[0, chunk_size-1]` so the window always advances and the operation
terminates. Implementation follows the existing
`splitByChar`/`splitByParagraph` pattern and reuses `DetectLanguage` for
chunk metadata.
It also adds `split_test.go` — the first unit tests for the `chunk`
package — covering basic windowing, overlap, overlap
clamping/termination, rune-awareness (CJK), default sizing, no-overlap
reconstruction, empty input, and DSL param parsing.
Validation: `gofmt` clean, `go vet ./internal/ingestion/chunk/` clean,
`go build` ok, `go test ./internal/ingestion/chunk/` — all tests pass.
Closes#16046
### Type of change
- [x] New Feature (non-breaking change which adds functionality)
Co-authored-by: bittoby <218712309+bittoby@users.noreply.github.com>
## Summary
- derive Go Agent debug input forms from prompt variable references
instead of Agent meta fields
- seed `sys.*` debug params into `CanvasState.Sys` so single-component
debug resolves prompt variables like Python
- restore Agent test-run parity for form rendering and debug execution
## Tests
- `go test ./internal/agent/component -run
'TestAgent_(GetInputForm_UsesPromptReferences|GetInputForm_DeduplicatesPromptReferences|Meta_DefaultsToEmpty|Reset_NoTools)$'`
- `go test ./internal/handler -run
'Test(DebugComponent_SeedsSysInputsIntoCanvasState|DebugComponent_HappyPath_Begin|GetComponentInputForm_HappyPath)$'`
AFTER:
<img width="669" height="456" alt="image"
src="https://github.com/user-attachments/assets/4fd86559-aafc-4027-91ae-6e666137ee1b"
/>
## Summary
Adds ownership/access checks before updating or deleting documents,
setting document metadata, and reading file contents from storage. Also
adds tests for authorized and unauthorized access paths.
## Summary
Fix Elasticsearch-backed skill search by mapping skill search fields to
their indexed token fields.
`name`, `tags`, `description`, and `content` are stored for display but
are not searchable in the skill ES mapping. Search queries now target
`name_tks`, `tags_tks`, `description_tks`, and `content_tks`.
## Testing
- Ran Go unit tests:
```bash
/usr/local/go/bin/go test -count=1 ./internal/engine/elasticsearch
```
- Frontend verification:
1. Open /files/skills.
2. Enter a skill space.
3. Reindex the skill space if existing skills were created before this
fix.
4. Search by skill name or description keyword.
5. Confirm matching skills are returned.
### Summary
This PR fixes two issues that prevented the Agent component's
single-component debug/test run from working under the Go backend:
1. **Dynamic input_form generation**: Some components (e.g. `Agent`) do
not store a static `input_form` in the DSL. The Go handler now falls
back to the runtime component's `GetInputForm()` method, matching
Python's `Canvas.get_component_input_form` behavior. This resolves the
frontend 102 error: `component has no input_form`.
2. **Tenant ID injection for debug**: Single-component debug runs use a
fresh `CanvasState` that previously lacked `tenant_id`.
`AgentComponent.Invoke` resolves LLM credentials via the tenant tables,
so the debug run failed with `api key is required`. The handler now
seeds `state.Sys["tenant_id"]` with the authenticated user's ID,
mirroring Python's `@add_tenant_id_to_kwargs` decorator.
### Changes
- `internal/handler/agent_component.go`:
- Added `componentInputForm` helper that first reads the static
`input_form` and, if missing, instantiates the component and calls
`GetInputForm()`.
- In `DebugComponent`, set `debugState.Sys["tenant_id"] = user.ID`
before invoking the component.
### Summary
The Go backend Agent component was not returning artifacts produced by
the CodeExec tool. While the Python agent collects the "`_ARTIFACTS`"
envelope from tool responses and appends artifact markdown to the final
content, the Go agent only returned the assistant text, so generated
images were missing from the chat output.
### Changes
- Wire `react.WithMessageFuture()` in `runEinoReActAgent` and store the
resulting `MessageFuture` in the invocation context.
- After the ReAct loop finishes, drain the future and extract
``_ARTIFACTS`` entries from every tool response message.
- Support reading the tool payload from both `msg.Content` and
`msg.UserInputMultiContent` to match eino's tool contract.
- De-duplicate artifacts by URL and render images as `!` and other files
as download links.
- Add `agent_artifact_test.go` with a regression test that simulates a
CodeExec-style tool response carrying an image artifact and verifies it
is collected and formatted.
### Verification
- `go test ./internal/agent/component/... -run
TestAgent_ReActAgent_CollectsArtifactsFromCodeExecTool` passes.
- `go test ./internal/agent/component/... -count=1` compiles; the only
failure is an unrelated DNS-pinning timeout test
(`TestInvoke_ProxyDNSPin`).
- `gofmt` clean for modified files.
### Related
Fixes the behavior shown in the screenshot where the Go agent ignored
the CodeExec-generated PNG artifact.
### Summary
1. Move common functions to format.go
2. modify show name spaces to _
3. move _order _columns column sort group;
4. add dao empty enterprise file
## Summary
Use `DocumentService.RemoveDocumentKeepFile` when deleting files that
are linked to documents.
## Change
- inject `DocumentService` into `FileService`
- replace direct document deletion in `deleteSingleFile`
- remove the obsolete file-local engine deletion helper
## Result
Deleting a file now cleans up linked documents through the same service
path used elsewhere, keeping KB counters and document engine cleanup
consistent.
