mirror of
https://github.com/infiniflow/ragflow.git
synced 2026-08-25 17:42:24 +08:00
fix(agent/tools): GoogleScholar empty json output and ignored top_n (#16419)
### What problem does this PR solve? Closes #16418. `scholarly.search_pubs(...)` returns a **lazy generator**, but `agent/tools/googlescholar.py` treated it as a re-iterable, bounded list: ```python scholar_client = scholarly.search_pubs(kwargs["query"], ...) # lazy generator self._retrieve_chunks(scholar_client, ...) # (1) iterates -> exhausts it self.set_output("json", list(scholar_client)) # (2) already empty -> [] ``` 1. **`json` output was always empty.** `_retrieve_chunks` iterates `scholar_client`, exhausting the generator; `list(scholar_client)` then returns `[]`. 2. **`top_n` was never applied.** Unlike `ArXiv` (`max_results=self._param.top_n`), the unbounded generator was passed straight to `_retrieve_chunks`, which has no internal limit — so the tool kept paginating well past Top N (until an error, rate-limit/block, or `COMPONENT_EXEC_TIMEOUT`). ### Type of change - [x] Bug Fix (non-breaking change which fixes an issue) ### Changes - Materialize at most `top_n` results once with `itertools.islice`, and reuse that list for both `_retrieve_chunks` and the `json` output. - Add regression tests (`test/unit_test/agent/component/test_googlescholar.py`, stubbing `scholarly.search_pubs`) covering the `top_n` bound, the non-empty `json` output, and the empty-query short-circuit. Verified: against `main` the new tests fail with `assert 30 == 5` (top_n ignored) and `assert 0 == 5` (empty json); with this fix all pass. Backend-only. --------- Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
This commit is contained in:
@@ -13,6 +13,7 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
import itertools
|
||||
import logging
|
||||
import os
|
||||
import time
|
||||
@@ -70,6 +71,9 @@ class GoogleScholar(ToolBase, ABC):
|
||||
|
||||
if not kwargs.get("query"):
|
||||
self.set_output("formalized_content", "")
|
||||
# Reset json too, otherwise a reused instance keeps stale results
|
||||
# from a previous successful call.
|
||||
self.set_output("json", [])
|
||||
return ""
|
||||
|
||||
last_e = ""
|
||||
@@ -84,12 +88,18 @@ class GoogleScholar(ToolBase, ABC):
|
||||
if self.check_if_canceled("GoogleScholar processing"):
|
||||
return
|
||||
|
||||
self._retrieve_chunks(scholar_client,
|
||||
# search_pubs returns a lazy generator: materialize at most top_n
|
||||
# results once so the bound is respected and the same list feeds
|
||||
# both _retrieve_chunks and the json output (iterating it twice
|
||||
# would otherwise leave json empty).
|
||||
results = list(itertools.islice(scholar_client, self._param.top_n))
|
||||
|
||||
self._retrieve_chunks(results,
|
||||
get_title=lambda r: r['bib']['title'],
|
||||
get_url=lambda r: r["pub_url"],
|
||||
get_content=lambda r: "\n author: " + ",".join(r['bib']['author']) + '\n Abstract: ' + r['bib'].get('abstract', 'no abstract')
|
||||
)
|
||||
self.set_output("json", list(scholar_client))
|
||||
self.set_output("json", results)
|
||||
return self.output("formalized_content")
|
||||
except Exception as e:
|
||||
if self.check_if_canceled("GoogleScholar processing"):
|
||||
|
||||
Reference in New Issue
Block a user