fix(agent/tools): GoogleScholar empty json output and ignored top_n (#16419)

### What problem does this PR solve?

Closes #16418.

`scholarly.search_pubs(...)` returns a **lazy generator**, but
`agent/tools/googlescholar.py` treated it as a re-iterable, bounded
list:

```python
scholar_client = scholarly.search_pubs(kwargs["query"], ...)   # lazy generator
self._retrieve_chunks(scholar_client, ...)                     # (1) iterates -> exhausts it
self.set_output("json", list(scholar_client))                  # (2) already empty -> []
```

1. **`json` output was always empty.** `_retrieve_chunks` iterates
`scholar_client`, exhausting the generator; `list(scholar_client)` then
returns `[]`.
2. **`top_n` was never applied.** Unlike `ArXiv`
(`max_results=self._param.top_n`), the unbounded generator was passed
straight to `_retrieve_chunks`, which has no internal limit — so the
tool kept paginating well past Top N (until an error, rate-limit/block,
or `COMPONENT_EXEC_TIMEOUT`).

### Type of change

- [x] Bug Fix (non-breaking change which fixes an issue)

### Changes

- Materialize at most `top_n` results once with `itertools.islice`, and
reuse that list for both `_retrieve_chunks` and the `json` output.
- Add regression tests
(`test/unit_test/agent/component/test_googlescholar.py`, stubbing
`scholarly.search_pubs`) covering the `top_n` bound, the non-empty
`json` output, and the empty-query short-circuit.

Verified: against `main` the new tests fail with `assert 30 == 5` (top_n
ignored) and `assert 0 == 5` (empty json); with this fix all pass.
Backend-only.

---------

Co-authored-by: Zhichang Yu <yuzhichang@gmail.com>
This commit is contained in:
Muhammad Furqan
2026-07-01 07:47:39 +05:00
committed by GitHub
parent 6648fe4151
commit 828c5789f6
2 changed files with 113 additions and 2 deletions

View File

@@ -13,6 +13,7 @@
# See the License for the specific language governing permissions and
# limitations under the License.
#
import itertools
import logging
import os
import time
@@ -70,6 +71,9 @@ class GoogleScholar(ToolBase, ABC):
if not kwargs.get("query"):
self.set_output("formalized_content", "")
# Reset json too, otherwise a reused instance keeps stale results
# from a previous successful call.
self.set_output("json", [])
return ""
last_e = ""
@@ -84,12 +88,18 @@ class GoogleScholar(ToolBase, ABC):
if self.check_if_canceled("GoogleScholar processing"):
return
self._retrieve_chunks(scholar_client,
# search_pubs returns a lazy generator: materialize at most top_n
# results once so the bound is respected and the same list feeds
# both _retrieve_chunks and the json output (iterating it twice
# would otherwise leave json empty).
results = list(itertools.islice(scholar_client, self._param.top_n))
self._retrieve_chunks(results,
get_title=lambda r: r['bib']['title'],
get_url=lambda r: r["pub_url"],
get_content=lambda r: "\n author: " + ",".join(r['bib']['author']) + '\n Abstract: ' + r['bib'].get('abstract', 'no abstract')
)
self.set_output("json", list(scholar_client))
self.set_output("json", results)
return self.output("formalized_content")
except Exception as e:
if self.check_if_canceled("GoogleScholar processing"):