mirror of
https://github.com/vectorize-io/hindsight.git
synced 2026-09-14 19:31:49 +08:00
Sanderhoff-alt
b045794817
Cuts the within-batch semantic link pass to float32 and one reused buffer.
The batch was widened to float64, but PackedEmbedding is array("f") and pgvector's
vector column stores float32, so the extra 32 bits were padding nothing downstream
could read. Dropping to float32 halves the working set and puts BLAS on SGEMM;
normalising in place, deriving validity from the row norms instead of an (n, dim)
isfinite mask, and reusing one similarity buffer across blocks remove three further
copies. Peak transient falls 74-86% (at 5,000 facts, 235 MB -> 48 MB). argpartition
replaces a full sort that existed only to discard all but top_k, and the self-link
mask and score unboxing move out of the Python loop: 1.7-2.2x on a realistic
clustered batch, up to 4.8x when nearly every pair clears the threshold.
Norms are accumulated in float64 via einsum, since a float32 sum of 1536 squares
overflows above ~1e19 and flushes to zero below ~1e-22. The batch is copied with
np.array rather than aliased with asarray, as it is now normalised in place.
Verified against the float64 implementation across 120 randomised batches plus
NaN/inf/zero embeddings, degenerate magnitudes and all-ties: identical link pairs,
scores within 1e-6.