ruv d3223dc3d9 feat(intelligence): joint rerank re-grid + conditional defaults (ADR-083, 3.10.23) ...

ADR-082 left the rerank path at hw=0.5/cw=0.5 because the original rerank
grid was tested against OLD α=0.6/sw=3.0 baselines. Joint re-grid (28 configs:
hw/cw × α × sw) reveals the rerank path wants DIFFERENT hybrid sub-params
than non-rerank.

Finding: when the cross-encoder is doing semantic understanding downstream,
the hybrid stage can be MORE keyword-focused (higher subjectWeight). When
hybrid is the final stage, lower subjectWeight gives the body tokens room
to contribute relevance.

Implementation: subjectWeight default is now CONDITIONAL on useRerank
(3.0 when reranking, 2.0 otherwise). Explicit param still overrides.

New rerank defaults:
  - subjectWeight: 2.0 → 3.0 (when rerank=true)
  - hybridWeight:  0.5 → 0.7
  - ceWeight:      0.5 → 0.3

Measured (rerank path, labelled, this checkout):

| Metric            | 3.10.22 | 3.10.23 |     Δ |
|-------------------|--------:|--------:|------:|
| Label top-1       |    90%  |    90%  |   tie |
| Label top-3       |    90%  |   100%  | +10pp |
| Label MRR@3       |  0.925  |  0.950  | +0.03 |
| Label precision@3 |  0.700  |  0.700  |   tie |
| Label nDCG@3      |  0.900  |  0.963  | +0.06 |
| Label nDCG@5      |  0.904  |  0.944  | +0.04 |

Both retrieval paths now hit nDCG@3 = 0.963 on this corpus — the choice
between them is purely cost vs richness:
  - Hybrid  (39 ms/query) — leaner top-3 (P3=0.533), use for hot paths
  - Rerank (1000 ms/query) — denser top-3 (P3=0.700), use for richness-first

Cumulative since cosine baseline (3.10.17 → 3.10.23): nDCG@3 0.000 → 0.963
on BOTH paths.

Pending for next iteration: cross-repo generalisation test (pretrain on
agentdb/agentic-flow, run a similar labelled bench, see if nDCG@3 holds).

Co-Authored-By: RuFlo <ruv@ruv.net>

2026-05-30 14:59:18 -04:00

9.9 KiB

Raw Permalink Blame History

View Raw