Commit Graph

4 Commits

Author SHA1 Message Date
zcxGGmu
afc12b2bff fix: preserve mineru cross-page table positions (#17017)
## Summary
- Read MinerU `_middle.json` alongside the content-list output.
- Preserve each matched table block as a separate PDF position tag so
cross-page tables expose all highlighted ranges.
- Keep enrichment limited to table outputs and require the first-page
bbox anchor before adding matching continuation blocks.

## Tests
-
`/Users/zq/.config/superpowers/worktrees/ragflow/issue-16992-pdf-table-coordinates/.venv/bin/python
-m pytest --confcutdir=test/unit_test/deepdoc/parser
test/unit_test/deepdoc/parser/test_mineru_parser.py -q`
-
`/Users/zq/.config/superpowers/worktrees/ragflow/issue-16992-pdf-table-coordinates/.venv/bin/ruff
check deepdoc/parser/mineru_parser.py
test/unit_test/deepdoc/parser/test_mineru_parser.py`
-
`/Users/zq/.config/superpowers/worktrees/ragflow/issue-16992-pdf-table-coordinates/.venv/bin/python
-m py_compile deepdoc/parser/mineru_parser.py
test/unit_test/deepdoc/parser/test_mineru_parser.py`
- `git diff --check`

Closes #16883

---------

Co-authored-by: zq <zhouquan1511@163.com>
2026-07-18 20:48:52 +08:00
Harsh Kashyap
81361c2108 fix(mineru): thread page_from/page_to into MinerU API request (#16957) 2026-07-16 12:23:03 +08:00
monsterDavid
d398d617ca fix(mineru): skip page chrome blocks to prevent duplicate chunks (#15387)
## Summary
- Skip MinerU `header`, `footer`, and `page_number` blocks when
converting `content_list.json` into sections.
- Ignore unsupported block types explicitly so future MinerU output
types cannot re-emit the previous text block.

Fixes duplicate text in General/naive chunks when parsing PDFs via
MinerU (reported with repeated page headers and body text in slices).

Closes #15335

## Test plan
- [x] `pytest test/unit_test/deepdoc/parser/test_mineru_parser.py -v`
(4/4 passed)
2026-06-01 20:15:04 +08:00
Jonathan Chang
9d1006e4ec fix: The output of the parser in the ingestion pipeline contains HTML tags (#14920)
## Summary
This change fixes ingestion quality issues where MinerU parser output
may contain HTML fragments (for example, table-related tags like `<tr>`,
`<td>`, `<br>`), which were previously passed directly into
chunking/tokenization and degraded chunk quality.

The fix adds a sanitization step in the MinerU parser path so parsed
sections are normalized to clean text before chunking.

## Change Type (select all)
- [x] Bug fix
- [x] Ingestion pipeline improvement
- [x] Parser/chunking quality fix

## Related Issue
- https://github.com/infiniflow/ragflow/issues/14831
2026-05-25 16:06:36 +08:00