 ruv
|
57a4a1b735
|
feat(intelligence): cross-encoder reranker (ADR-080, 3.10.20)
Opt-in cross-encoder rerank pass over the hybrid+BM25 top-K. Uses
Xenova/ms-marco-MiniLM-L-6-v2 (int8, ~30MB) via direct AutoTokenizer +
AutoModelForSequenceClassification (xenova v2 pipeline can't do pair
encoding reliably). Lazy-loaded, one-shot load policy, graceful
degradation when model unavailable.
Architecture: hybrid top-K*6 → cross-encoder scores all pairs →
combined = 0.5*normalise(hybrid) + 0.5*normalise(ce) → top-K. MMR
bypassed when reranking is on (cross-encoder's calibrated score is
more precise than MMR's diversity heuristic at this stage).
Linear combination is essential — ablation showed:
- Hybrid alone: 8/10 top-1, 8/10 top-3, MRR 0.800
- CE alone: 6/10 top-1, 10/10 top-3, MRR 0.733
- Combined 0.5/0.5: 9/10 top-1, 10/10 top-3, MRR 0.933 ← default
Default is OFF (latency 25x hybrid: 1.0s vs 39ms at N=385).
Callers opt in via {rerank: true}. Weights tunable via
{hybridWeight, ceWeight}; grid-search shows 0.3-0.5 hybrid all hit
the same plateau.
Cumulative since cosine baseline (3.10.17 → 3.10.20):
| Metric | Cosine | 3.10.18 | 3.10.19 | 3.10.20 |
|------------------|-------:|--------:|--------:|--------:|
| Top-1 hit rate | 0% | 50% | 80% | 90% |
| Top-3 hit rate | 0% | 70% | 80% | 100% |
| MRR@3 | 0.000 | 0.583 | 0.800 | 0.933 |
| Top-1 diversity | 100% | 80% | 100% | 100% |
| Latency (rerank) | 29ms | 41ms | 39ms | 984ms |
5 new tests cover the degradation contract (no network needed).
Co-Authored-By: RuFlo <ruv@ruv.net>
|
2026-05-30 14:20:34 -04:00 |
|