Zhichang Yu 64533e5b5e Refactor splitByTokens and wire wiki incremental compile (#18083) ...

Port dataset-level wiki incremental compile and refactor splitByTokens
token budgeting. Includes replace-only wiki merge, KNN dedup routing,
and template/config wiring.

2026-08-11 13:46:55 +08:00

..

chunker

fix(chunker): unify TokenChunker merge and strip coord tags in Python JSON path (#18002)

2026-08-07 21:55:07 +08:00

globals

Port PR14140 and PR16881 to GO (#17102)

2026-07-21 17:46:23 +08:00

knowledge_compiler

Refactor splitByTokens and wire wiki incremental compile (#18083)

2026-08-11 13:46:55 +08:00

schema

feat(knowledge_compile): materialize wiki page graph (wiki_entity/wiki_relation) (#17976)

2026-08-07 17:47:59 +08:00

dispatch_model.go

Go: introduce content_length and max_output (#17807)

2026-08-04 19:08:31 +08:00

document_storage.go

Go: add context to storage (#17690)

2026-08-03 15:08:21 +08:00

docx_vision_dispatch_test.go

Go: add context, part12 (#17435)

2026-07-27 19:30:41 +08:00

docx_vision_dispatch.go

Go: add context, part12 (#17435)

2026-07-27 19:30:41 +08:00

extractor_tag_test.go

fix(ingestion): improve Extractor/LLM robustness and Python->Go parity (#17470)

2026-07-28 19:22:18 +08:00

extractor_tag.go

Go: add context to storage (#17690)

2026-08-03 15:08:21 +08:00

extractor_test.go

fix(ingestion): avoid duplicate chunk text injection in Extractor prompts (#18034)

2026-08-11 11:53:34 +08:00

extractor.go

fix(ingestion): avoid duplicate chunk text injection in Extractor prompts (#18034)

2026-08-11 11:53:34 +08:00

file_test.go

Go: add context to storage (#17690)

2026-08-03 15:08:21 +08:00

file.go

Go: fix warnings (#17738)

2026-08-03 21:30:01 +08:00

idempotency.go

Align Go ingestion boundaries with Python (#16647)

2026-07-05 20:43:52 +08:00

image_uploader.go

Go: add context to storage (#17690)

2026-08-03 15:08:21 +08:00

markdown_vision_dispatch.go

Go: add context, part12 (#17435)

2026-07-27 19:30:41 +08:00

media_dispatch_test.go

fix: Go ingestion migration batch 5 (Parser 1.1/1.7/2.11, Chunker 1.7/1.8/2.6/2.7, Tokenizer 6x fixes) (#17419)

2026-07-28 11:12:52 +08:00

media_dispatch.go

Go: fix env variables (#18032)

2026-08-10 14:37:05 +08:00

parser_check_test.go

Go: fix warnings (#17738)

2026-08-03 21:30:01 +08:00

parser_dispatch_pdf_vision_cgo_test.go

Go: add context, part12 (#17435)

2026-07-27 19:30:41 +08:00

parser_dispatch_test.go

Go: fix SSRF (#17641)

2026-07-31 19:15:38 +08:00

parser_dispatch.go

Go: fix warnings (#17738)

2026-08-03 21:30:01 +08:00

parser_test.go

Go: add context to storage (#17690)

2026-08-03 15:08:21 +08:00

parser.go

Go: fix warnings (#17738)

2026-08-03 21:30:01 +08:00

pdf_vision_dispatch_cgo.go

Align Go parser backends and PDF pipeline with Python (#16676)

2026-07-06 19:50:54 +08:00

pdf_vision_dispatch_nocgo.go

Align Go parser backends and PDF pipeline with Python (#16676)

2026-07-06 19:50:54 +08:00

pdf_vision_dispatch_test.go

Fix(go): align ingestion pipeline with Python (parser/media dispatch + PDF coordinate chain + Chunker) (#17349)

2026-07-24 21:06:38 +08:00

pdf_vision_dispatch.go

fix: Go ingestion migration batch 5 (Parser 1.1/1.7/2.11, Chunker 1.7/1.8/2.6/2.7, Tokenizer 6x fixes) (#17419)

2026-07-28 11:12:52 +08:00

slice3_integration_test.go

Go: add context, part5 (#17392)

2026-07-27 10:20:16 +08:00

slice3_test.go

Refactor: refactor dataflow_service.go and guard nats message re-delivery (#16826)

2026-07-13 11:08:04 +08:00

ssrf_test_helper_test.go

Go: fix SSRF (#17641)

2026-07-31 19:15:38 +08:00

tokenizer_nonpersist_test.go

Feat(ingestion): add canvas pipeline debug (dry-run) mode with View result log (#17538)

2026-07-31 13:10:08 +08:00

tokenizer_test.go

Go knowledge compiler with scheduler-driven dataset compilation (#17913)

2026-08-06 15:54:00 +08:00

tokenizer_unit_test.go

Fix(tokenizer): pin alignment guards (#18011)

2026-08-11 10:09:07 +08:00

tokenizer.go

Fix(tokenizer): pin alignment guards (#18011)

2026-08-11 10:09:07 +08:00