Jack 2fcc34904b fix(chunker): keep oversize text/markdown unit whole (OVER_CAP alignment) (#17854) ...

The Go `TokenChunker` text/markdown path (`mergeByTokenSize`)
unconditionally
called `splitOversizedUnit` on any unit that exceeded
`chunk_token_size`,
emitting Go-only sub-chunks. Python's `naive_merge`
(`_merge_paragraph_groups`,
`rag/nlp/__init__.py`) never atom-splits an oversize unit under either
`OVER_CAP` or `UNDER_CAP`: a paragraph larger than the budget becomes
its own
standalone chunk and the model layer truncates it later.

This aligns the text/markdown path with the **structured JSON path**
(`invokeJSONPayload` → `mergeByTokenSizeFromJSON(...,
subSplitOversize=false)`,
#17739). It completes the OVER_CAP alignment started in #17835.

2026-08-05 18:53:59 +08:00

..

compilation/extractor

refactor: use the built-in max/min to simplify the code (#17059)

2026-07-24 15:33:02 +08:00

component

fix(chunker): keep oversize text/markdown unit whole (OVER_CAP alignment) (#17854)

2026-08-05 18:53:59 +08:00

knowledge_compile

refactor(knowledge_compile): global compile pool, token-budget batching, and DocEngine-only deletion (#17679)

2026-08-02 17:06:29 +08:00

pipeline

Go: add context for DB access (#17861)

2026-08-05 16:22:36 +08:00

registry

Go: refactor to avoid conflicts (#17191)

2026-07-21 18:47:02 +08:00

service

[go] Merged memory extractor with ingestion task handler (#17730)

2026-08-03 17:08:14 +08:00

task

Go: add context for DB access (#17861)

2026-08-05 16:22:36 +08:00

testutil

Go: fix list services (#17664)

2026-08-01 14:52:37 +08:00

wire

Align Go ingestion boundaries with Python (#16647)

2026-07-05 20:43:52 +08:00