mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-09-14 14:01:28 +08:00
ruv
cff24dbff6
Horizon: guidance-sota-2026-05 (M4). The real SOTA win.
## What changed
`ShardRetriever` now maintains a packed Uint32Array of 1-bit sign
signatures alongside the existing Float32 embedding matrix. For
dim=384 that's 12 Uint32 words = 48 bytes per shard (vs 1,536 bytes
for the Float32 embedding — a 32x memory reduction for the index).
`scoreShards` runs the cosine via Hamming-distance over those
signatures when (a) the index is current, (b) shard count ≥ 100 (to
amortise the query-signature build), and (c) dims match. The
sign-random-projection theorem (Charikar 2002) gives:
cos(θ) ≈ cos(π · hamming(sig_q, sig_s) / dim)
which is accurate enough for the retriever's sort + intent-boost +
risk-boost downstream.
## Microbench (bench-quantization.mjs)
cosine.dot (float32, 384-d) 3,006,455 ops/s 332.62 ns/pair
hamming.popcount (uint32, 12 words) 32,862,982 ops/s 30.43 ns/pair
────────────
Hamming speedup vs dot: 10.93x
## End-to-end (bench-retriever-scale.mjs)
N baseline unfilt M4 unfilt speedup
10 63,910 70,777 1.11x
100 12,135 26,372 2.17x
500 2,470 6,468 2.62x
1000 1,303 3,522 🚀 2.70x
N baseline filt M4 filt speedup
10 166,274 198,451 1.19x
100 46,419 85,311 1.84x
500 12,974 27,081 2.09x
1000 6,662 16,073 2.41x
All 1,331 existing tests pass. Build clean.
## Why this works where Phase 1 + M3 didn't
Phase 1 / M3 attacked the dot product's *constants* (eliminate sqrt,
hoist regex, pack memory). Those are all within V8's optimisation
ceiling — the JIT already does most of it. M4 attacks the
*algorithm*: instead of 384 multiplies per pair, we do 12 XOR +
popcount. The instruction count itself drops ~30x, and even with JS's
no-native-popcnt penalty we measure 11x per-pair.
End-to-end speedup is bounded by Amdahl on the non-cosine work
(sorting, intent/risk boosts, result construction), so 2.7x is the
expected ceiling for this fixture. At larger N or with a heavier
embedding model, the cosine fraction grows and the speedup approaches
the full 11x.
## Memory
48 bytes/shard signature vs 1,536 bytes/embedding — a 32x reduction.
At N=10,000 shards: 480KB signatures vs 15MB embeddings. Real for
cold-start RAM on hooks-running daemons.
Co-Authored-By: RuFlo <ruflo-bot@users.noreply.github.com>
Guidance Performance Benchmarks
Phase 1 benchmarks for the @claude-flow/guidance SOTA optimization
horizon (guidance-sota-2026-05).
Scripts
| Script | Measures |
|---|---|
bench-phase-1.mjs |
Micro-benchmarks for the 3 hot paths identified by the researcher (analyzer extractMetrics, compiler parseRule, retriever cosine) |
bench-retriever-scale.mjs |
End-to-end retriever.retrieve() latency at N ∈ {10, 100, 500, 1000} shards — the production-facing scaling curve |
Running
cd v3/@claude-flow/guidance && npm run build
node v3/@claude-flow/guidance/scripts/bench-phase-1.mjs --tag=baseline
node v3/@claude-flow/guidance/scripts/bench-retriever-scale.mjs --tag=baseline
Output lands in docs/benchmarks/guidance-*-<tag>.json.
Findings (Phase 1, 2026-05-22)
| Bench | Baseline | Phase 1 | Δ |
|---|---|---|---|
| analyzer.analyze (150-line CLAUDE.md) | 2,896 ops/s | 2,860 ops/s | within noise |
| compiler.compile (150-line CLAUDE.md) | 3,752 ops/s | 3,704 ops/s | within noise |
| retriever.retrieve (N=500) | 2,457 ops/s | 2,724 ops/s | +10.9% |
| retriever.retrieve (N=1000) | 1,317 ops/s | 1,425 ops/s | +8.2% |
The micro-optimizations to extractMetrics (single-pass loop) and
parseRule (text.matchAll instead of new RegExp(...) per call) are
within run-to-run noise on V8 — the JIT already optimizes these patterns
heavily. The retriever changes (unit-vector dot-only cosine + same
single-pass philosophy) deliver a real 8-11% lift at scale.
The real opportunity is M3: replace scoreShards's O(n) linear scan
(retriever.ts:268) with an HNSW ANN query. Baseline shows latency goes
from 14µs at N=10 to 760µs at N=1000 — pure O(n) cost. An ANN index
will deliver O(log n), which at N=1000 means roughly 100x algorithmic
improvement on the dominant bottleneck.