mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-14 20:54:30 +08:00
Go's `TokenChunker` previously ignored **bare (non-backtick) delimiters** such as `::`, `.`, or `;`. `compileDelimPattern` compiled them with `keepBare=false`, so a bare delimiter produced a `nil` pattern and the payload was routed to the single-section merge that never split on it — diverging from Python's `naive_merge`, which splits on bare delimiters and then merges by token size.