Commit Graph

5 Commits

Author SHA1 Message Date
Jack
1aa4e3c1f3 refactor(chunker): converge delimiter_mode to {delimiter, one}, drop token_size (#17979)
Converge `TokenChunker.delimiter_mode` from three values (`token_size`,
`delimiter`, `one`) to two (`delimiter`, `one`). The unified `delimiter`
mode now carries the old `token_size` semantics: when no active
(backtick) delimiter is present, text/JSON chunks are merged up to
`chunk_token_size`; when a backtick delimiter is present, the text is
split by it and not merged. `one` continues to be handled by the
separate `OneChunker`.
2026-08-07 16:11:42 +08:00
Zhichang Yu
2e37997ab9 Go knowledge compiler with scheduler-driven dataset compilation (#17913)
Ports dataset knowledge compilation (wiki/graph/tree/mindmap) to the Go
scheduler with a status contract, aligns wiki storage/retrieval with
Python, sizes prompts by content_length, and resolves embedding batch
size from provider capability.
2026-08-06 15:54:00 +08:00
Jin Hai
cf13082a1a Revert "feat: Go knowledge compiler with scheduler-driven dataset compilation" (#17897)
Reverts infiniflow/ragflow#17881
2026-08-05 21:50:28 +08:00
Zhichang Yu
14b943a04a feat: Go knowledge compiler with scheduler-driven dataset compilation (#17881)
Ports the dataset knowledge compilation (wiki/graph/tree/mindmap) to the
Go scheduler with a status contract, aligns wiki storage/retrieval with
Python, and sizes prompts by content_length.
2026-08-05 20:00:42 +08:00
Zhichang Yu
29287ef74b refactor(knowledge_compiler): derive variant from template kind via single template id (#17630)
## Summary

Refactor the Go `KnowledgeCompilerComponent` so its parameter is a
**single string template id** instead of a DSL-level `variant` (or
plural group id list). The `variant` is no longer in the DSL — it is now
**derived at runtime from the resolved compilation template's `kind`
field**.

This aligns the Go ingestion port with the frontend Compiler operator,
which emits a singular `compilation_template_group_id` and does not
write `variant` into the generated `compiler.json`.
2026-07-31 18:01:55 +08:00