mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-07-17 13:17:20 +08:00
5c96fa51f0
Fixes #16917. ## Problem `deepdoc/parser/docling_parser.py::_parse_pdf_remote` decides whether the response is chunked based on which payload was sent, not on what came back. Docling Serve silently drops unknown fields such as `do_chunking` (Pydantic `extra="ignore"`) and returns a standard `{"document": ..., "status": ...}` conversion response. The code then: 1. sets `is_chunked_response = True` from the request shape, 2. logs `Successfully used native chunking on: <endpoint>`, 3. extracts 0 chunks from `response_json.get("results", [])`, 4. logs `Native chunks received: 0`, 5. falls through to the existing `md_content` fallback. The `md_content` fallback path is fine. The misleading log lines are the problem: operators see "Successfully used native chunking" immediately followed by "Native chunks received: 0" and "No chunk built", which looks like an internal regression rather than a server contract gap. ## Fix Decide chunked-vs-standard from the **response shape**, not the request: ```python response_is_chunk = self._looks_like_chunk_response(response_json) is_chunked_response = chunk_flag and response_is_chunk ``` `_looks_like_chunk_response` returns True iff the response is a non-empty list or a dict with a non-empty `results` or `chunks` list. A standard conversion response (`{"document": ..., "status": ...}`) does not match, so a server that ignored the chunking flag is correctly classified as standard even when the request payload asked for chunking. When chunking was requested but the server returned a standard response, log a single WARNING ("Server ignored chunking request on <endpoint>; treating response as standard conversion.") instead of the INFO success line. The misleading "Prioritizes native chunking endpoints" docstring is replaced with what the code actually does. ## Tests `test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (6 tests, all passing): - `test_remote_chunked_200_standard_payload_falls_back` (existing — still passes; the `md_content` path is unchanged) - `test_chunk_shape_helper_recognises_chunk_payloads` - `test_chunk_shape_helper_rejects_standard_payloads` - `test_remote_chunked_request_with_results_list_is_treated_as_chunked` - `test_remote_top_level_list_response_is_treated_as_chunked` - `test_remote_chunked_request_with_ignored_flag_does_not_log_success` ``` $ uv run pytest test/unit_test/deepdoc/parser/test_docling_parser_remote.py -v ============================== 6 passed in 0.26s ============================== ``` ## Files changed - `deepdoc/parser/docling_parser.py` (+35 / -5) - `test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (+89 / -4) ## Backward compatibility - All four payload/endpoint combinations continue to be tried in the same order. - The bundled-docling happy path (`parse_pdf`, not `_parse_pdf_remote`) is untouched. - A server that returns a real chunked response to a chunked request still goes down the chunked branch. A server that returns a standard response to a chunked request now goes down the standard branch with `is_chunked_response=False` instead of misleadingly logging success. ## Follow-up (out of scope) Calling the real Docling-Serve native chunk endpoints (`/v1/chunk/hybrid/source`, `/v1/chunk/hierarchical/source`) with `HybridChunkerOptions` is a larger feature change and warrants its own PR after this lands. Co-authored-by: Harsh23Kashyap <harsh@example.com>