mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-07-17 13:17:20 +08:00
5c96fa51f0
Fixes #16917. ## Problem `deepdoc/parser/docling_parser.py::_parse_pdf_remote` decides whether the response is chunked based on which payload was sent, not on what came back. Docling Serve silently drops unknown fields such as `do_chunking` (Pydantic `extra="ignore"`) and returns a standard `{"document": ..., "status": ...}` conversion response. The code then: 1. sets `is_chunked_response = True` from the request shape, 2. logs `Successfully used native chunking on: <endpoint>`, 3. extracts 0 chunks from `response_json.get("results", [])`, 4. logs `Native chunks received: 0`, 5. falls through to the existing `md_content` fallback. The `md_content` fallback path is fine. The misleading log lines are the problem: operators see "Successfully used native chunking" immediately followed by "Native chunks received: 0" and "No chunk built", which looks like an internal regression rather than a server contract gap. ## Fix Decide chunked-vs-standard from the **response shape**, not the request: ```python response_is_chunk = self._looks_like_chunk_response(response_json) is_chunked_response = chunk_flag and response_is_chunk ``` `_looks_like_chunk_response` returns True iff the response is a non-empty list or a dict with a non-empty `results` or `chunks` list. A standard conversion response (`{"document": ..., "status": ...}`) does not match, so a server that ignored the chunking flag is correctly classified as standard even when the request payload asked for chunking. When chunking was requested but the server returned a standard response, log a single WARNING ("Server ignored chunking request on <endpoint>; treating response as standard conversion.") instead of the INFO success line. The misleading "Prioritizes native chunking endpoints" docstring is replaced with what the code actually does. ## Tests `test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (6 tests, all passing): - `test_remote_chunked_200_standard_payload_falls_back` (existing — still passes; the `md_content` path is unchanged) - `test_chunk_shape_helper_recognises_chunk_payloads` - `test_chunk_shape_helper_rejects_standard_payloads` - `test_remote_chunked_request_with_results_list_is_treated_as_chunked` - `test_remote_top_level_list_response_is_treated_as_chunked` - `test_remote_chunked_request_with_ignored_flag_does_not_log_success` ``` $ uv run pytest test/unit_test/deepdoc/parser/test_docling_parser_remote.py -v ============================== 6 passed in 0.26s ============================== ``` ## Files changed - `deepdoc/parser/docling_parser.py` (+35 / -5) - `test/unit_test/deepdoc/parser/test_docling_parser_remote.py` (+89 / -4) ## Backward compatibility - All four payload/endpoint combinations continue to be tried in the same order. - The bundled-docling happy path (`parse_pdf`, not `_parse_pdf_remote`) is untouched. - A server that returns a real chunked response to a chunked request still goes down the chunked branch. A server that returns a standard response to a chunked request now goes down the standard branch with `is_chunked_response=False` instead of misleadingly logging success. ## Follow-up (out of scope) Calling the real Docling-Serve native chunk endpoints (`/v1/chunk/hybrid/source`, `/v1/chunk/hierarchical/source`) with `HybridChunkerOptions` is a larger feature change and warrants its own PR after this lands. Co-authored-by: Harsh23Kashyap <harsh@example.com>
(1). Deploy RAGFlow services and images
https://ragflow.io/docs/build_docker_image
(2). Configure the required environment for testing
Install Python dependencies (including test dependencies):
uv sync --python 3.13 --only-group test --no-default-groups --frozen
Activate the environment:
source .venv/bin/activate
Install SDK:
uv pip install sdk/python
Modify the .env file: Add the following code:
COMPOSE_PROFILES=${COMPOSE_PROFILES},tei-cpu
TEI_MODEL=BAAI/bge-small-en-v1.5
RAGFLOW_IMAGE=infiniflow/ragflow:v0.26.4 #Replace with the image you are using
Start the container（wait two minutes）:
docker compose -f docker/docker-compose.yml up -d
(3). Test Elasticsearch
a) Run sdk tests against Elasticsearch:
export HTTP_API_TEST_LEVEL=p2
export HOST_ADDRESS=http://127.0.0.1:9380 # Ensure that this port is the API port mapped to your localhost
pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_sdk_api
b) Run http api tests against Elasticsearch:
pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api
(4). Test Infinity
Modify the .env file:
DOC_ENGINE=${DOC_ENGINE:-infinity}
Start the container:
docker compose -f docker/docker-compose.yml down -v
docker compose -f docker/docker-compose.yml up -d
a) Run sdk tests against Infinity:
DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_sdk_api
b) Run http api tests against Infinity:
DOC_ENGINE=infinity pytest -s --tb=short --level=${HTTP_API_TEST_LEVEL} test/testcases/test_http_api