Jack 0403d19b5a fix: honor parser params and image VLM system_prompt in Go ingestion (#17334) ...

## Summary
Fix the Go ingestion pipeline so that several parser setup switches and
the
image VLM prompt are actually honored end-to-end (previously the DSL
fields
existed but the Go code never read them).

- **DOCX** (`docx_parser.go`, `docx_postprocess.go`): read `remove_toc`
and
  `remove_header_footer`; apply to both JSON and markdown output paths
  (outline-based TOC removal with a text-heuristic fallback, plus
  header/footer section filtering).
- **HTML** (`html_parser.go`, `html_postprocess.go`, `text_toc.go`):
read
`remove_header_footer` (pre-parse strip of `<header>`/`<footer>` and
ARIA
`banner`/`contentinfo`) and `remove_toc` (post-parse
`remove_contents_table`
  heuristic).
- **Markdown** (`markdown_parser.go`): read `flatten_media_to_text` and
force
  media blocks to text when enabled.
- **Image VLM** (`media_dispatch.go`): read `system_prompt` instead of
`prompt`
  so the user-configured image VLM prompt is no longer silently dropped
  (`prompt` remains the video family key).

All flags are wired through `ConfigureFromSetup`, which the dispatch
layer
already invokes for every family, so the behavior is live rather than
dead code.

## Test plan
- New unit tests: `docx_postprocess_test.go`, `html_parser_test.go`,
`text_toc_test.go`, `markdown_parser_test.go`, `media_dispatch_test.go`.
- `bash build.sh --test ./internal/parser/parser/...
./internal/ingestion/component/...`

## Notes
- The `File` component is excluded from this migration scope.
- Relates to the Python→Go parity diff (Parser 1.8–1.11, 1.15).

2026-07-24 14:42:26 +08:00

..

chunker

fix(chunker): glue delimiter to preceding segment instead of emitting it as a standalone chunk (#17168)

2026-07-23 10:13:46 +08:00

globals

Port PR14140 and PR16881 to GO (#17102)

2026-07-21 17:46:23 +08:00

schema

Implement Tagger in Extractor component in GO (#17128)

2026-07-21 17:52:06 +08:00

dispatch_model.go

Go: fix db alignment (#16964)

2026-07-16 16:00:56 +08:00

document_storage.go

feat(agent): Go ingestion pipeline progress mirroring and DeepDOC parser hardening (#16795)

2026-07-10 10:36:10 +08:00

docx_vision_dispatch.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00

extractor_tag_test.go

Implement Tagger in Extractor component in GO (#17128)

2026-07-21 17:52:06 +08:00

extractor_tag.go

fix(ingestion): align laws DSL with Python — heading fallback, colon-title, short-line filter, remove_toc, and image extension mapping (#17200)

2026-07-22 19:14:32 +08:00

extractor_test.go

fix(ingestion): align laws DSL with Python — heading fallback, colon-title, short-line filter, remove_toc, and image extension mapping (#17200)

2026-07-22 19:14:32 +08:00

extractor.go

fix(ingestion): align laws DSL with Python — heading fallback, colon-title, short-line filter, remove_toc, and image extension mapping (#17200)

2026-07-22 19:14:32 +08:00

file_test.go

Implement builtin chunk method as ingestion pipeline in GO (#16822)

2026-07-13 13:51:40 +08:00

file.go

Implement builtin chunk method as ingestion pipeline in GO (#16822)

2026-07-13 13:51:40 +08:00

idempotency.go

Align Go ingestion boundaries with Python (#16647)

2026-07-05 20:43:52 +08:00

image_uploader.go

Feat(ingestion): align image to MinIO upload, unify chunk-id computation and add PPT parsing support (#17111)

2026-07-20 19:33:51 +08:00

markdown_vision_dispatch.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00

media_dispatch_test.go

fix: honor parser params and image VLM system_prompt in Go ingestion (#17334)

2026-07-24 14:42:26 +08:00

media_dispatch.go

fix: honor parser params and image VLM system_prompt in Go ingestion (#17334)

2026-07-24 14:42:26 +08:00

parser_check_test.go

feat: parser component param check and family mapping fixes (#17312)

2026-07-23 22:06:22 +08:00

parser_dispatch_pdf_vision_cgo_test.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00

parser_dispatch_test.go

feat: parser component param check and family mapping fixes (#17312)

2026-07-23 22:06:22 +08:00

parser_dispatch.go

feat: parser component param check and family mapping fixes (#17312)

2026-07-23 22:06:22 +08:00

parser_test.go

Support pipeline DSL modification through dataset configuration (backend) (#16991)

2026-07-17 14:40:09 +08:00

parser.go

feat: parser component param check and family mapping fixes (#17312)

2026-07-23 22:06:22 +08:00

pdf_vision_dispatch_cgo.go

Align Go parser backends and PDF pipeline with Python (#16676)

2026-07-06 19:50:54 +08:00

pdf_vision_dispatch_nocgo.go

Align Go parser backends and PDF pipeline with Python (#16676)

2026-07-06 19:50:54 +08:00

pdf_vision_dispatch.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00

slice3_integration_test.go

Refactor: refactor dataflow_service.go and guard nats message re-delivery (#16826)

2026-07-13 11:08:04 +08:00

slice3_test.go

Refactor: refactor dataflow_service.go and guard nats message re-delivery (#16826)

2026-07-13 11:08:04 +08:00

tokenizer_test.go

feat: parser pages range and parse type validation for dataset/document (#17293)

2026-07-23 19:57:27 +08:00

tokenizer_unit_test.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00

tokenizer.go

Go: add context to lots of interface (#17253)

2026-07-22 22:30:57 +08:00