mirror of
https://github.com/countbot-ai/CountBot.git
synced 2026-09-14 20:46:47 +08:00
RIC
799569e5a2
With chunk-level retrieval the top-k list can be monopolised by several high-scoring chunks of the same document: on the 52-doc eval corpus, collecting 10 distinct documents required scanning 25.3 chunks on average, starving multi-doc questions of sources. search()/search_chunks() now oversample (top_k * 5), keep at most max_per_doc chunks per document first, then backfill by score if the result list is short - so small knowledge bases keep full top_k while large ones gain source diversity. Measured on the 60-question eval (production top-6 injection): - cross-doc source coverage: 0.500 -> 0.556 - paraphrased-question hit: 0.600 -> 0.700 - overall positive hit: 0.800 -> 0.829 - single-doc / needle / cross hit: no regression max_per_doc<=0 restores the previous behaviour exactly. Tests: 4 new (front-cap respected, backfill keeps result count, cap disabled falls back to legacy ranking, top_k and ordering preserved). Full suite: 126 passed. Refs: #107