Files
RIC 235c6c0e78 feat(rag): corrective routing for chunk-level ask - grade, rewrite, refuse
Chunk-level ask used to inject the top-6 chunks into the generation
prompt unconditionally. On the 60-question eval, all 10 negative
questions (answer not in the corpus) were still answered, and the
top-1 BM25 scores of negatives overlap 7/10 with positives, so no
score threshold can gate this reliably.

The ask path now routes through a lightweight LLM grader (single call,
~200-400 tokens) before generation:
- all relevant -> generate as before
- partially relevant -> filter to the graded-relevant chunks, generate
- none relevant -> rewrite the query once with LLM, re-search and
  re-grade; still none -> refuse explicitly instead of hallucinating

Every LLM step degrades safely: no provider, grading failure or
unparseable output falls back to the previous plain-generation
behaviour, so nothing breaks when the flag is on but no provider is
configured.

Only active under COUNTBOT_RAG_CHUNKS=1. No new dependencies.

Tests: 12 new (three-way routing, refusal after one rewrite, retry
succeeds, grading failure/unparseable/no-provider fallbacks, grader
output parsing contract). Full suite: 138 passed.

Refs: #107
2026-09-03 20:46:11 +08:00
..