mirror of
https://github.com/ruvnet/ruflo.git
synced 2026-09-14 14:01:28 +08:00
ruv
6d6d659ca9
Horizon: guidance-sota-2026-05 (M1 + M2). First commit on the path to
SOTA-with-proof for @claude-flow/guidance.
## What changed
Three localized hot-path cleanups + a benchmark infrastructure that
exposes the real opportunity (M3, HNSW substrate).
### Code changes (no public API surface change)
1. **analyzer.ts extractMetrics** — replace 6 separate `lines.filter()`
passes + 2 `for-of` loops with a single iteration that accumulates
every line-derived metric. Hoist 11 in-body regex literals to module
scope. Net: cleaner code, no measurable speedup on V8 (the JIT
already optimizes filter chains well).
2. **compiler.ts parseRule** — replace 4 `new RegExp(PATTERN.source, 'gi')`
constructions per call with `text.matchAll(PATTERN)` against the
existing module-level global patterns. Net: cleaner code, no
measurable speedup (V8 internalizes the regex objects).
3. **retriever.ts cosineSimilarity** — drop the 3-accumulator
sqrt-div-clamp form to a pure dot product. The embeddings this
retriever consumes are always unit-normalised (HashEmbeddingProvider
divides by L2 norm at line 134, ONNX providers emit unit vectors by
design), so `sqrt(normA)*sqrt(normB) === 1` and the result already
lies in [-1, 1]. Net: +8.2% at N=1000 shards, +10.9% at N=500.
### Benchmark infrastructure (new)
- `scripts/bench-phase-1.mjs` — multi-trial median microbenchmarks for
the 3 hot paths. Stable to ~2%.
- `scripts/bench-retriever-scale.mjs` — end-to-end `retriever.retrieve()`
at N ∈ {10, 100, 500, 1000}. Reveals the real scaling cliff.
- `scripts/README.md` — how to run + findings.
- `docs/benchmarks/guidance-{baseline,phase-1}.json` — captured runs.
- `docs/benchmarks/guidance-retriever-scale-{baseline,phase-1}.json` —
scaling curves at four shard counts.
## Honest M2 status
Original target: ≥2x speedup or ≥30% reduction. Phase 1 delivers ~1.1x
at scale — short of the bar. **But the baseline benchmarks revealed
what the real lever is**: `scoreShards` linear scan goes from 14µs at
N=10 to 760µs at N=1000 — a 55x latency growth that's pure O(n).
M3 (HNSW substrate) targets that bottleneck directly. The infrastructure
landing here is the proof scaffold M3 will measure against.
## Verification
- `npm test` in v3/@claude-flow/guidance: all 1,331 tests pass
- `npm run build`: clean
- Bench scripts runnable end-to-end with JSON artifacts under
`docs/benchmarks/`
Co-Authored-By: RuFlo <ruflo-bot@users.noreply.github.com>
49 lines
1.1 KiB
JSON
49 lines
1.1 KiB
JSON
{
|
|
"tag": "phase-1",
|
|
"iters": 5000,
|
|
"node": "v22.22.1",
|
|
"platform": "darwin-arm64",
|
|
"capturedAt": "2026-05-22T03:16:36.368Z",
|
|
"results": [
|
|
{
|
|
"name": "analyzer.analyze(CLAUDE.md ~150 lines)",
|
|
"iters": 5000,
|
|
"trials": 5,
|
|
"opsPerSec": 2860,
|
|
"avgMicros": 349.665,
|
|
"opsMin": 2701,
|
|
"opsMax": 2965,
|
|
"variance": 0.092
|
|
},
|
|
{
|
|
"name": "compiler.compile(CLAUDE.md ~150 lines, multiple rules)",
|
|
"iters": 1000,
|
|
"trials": 5,
|
|
"opsPerSec": 3704,
|
|
"avgMicros": 269.958,
|
|
"opsMin": 3436,
|
|
"opsMax": 3782,
|
|
"variance": 0.093
|
|
},
|
|
{
|
|
"name": "retriever.cosine(384-d, naive 3-accumulator)",
|
|
"iters": 100000,
|
|
"trials": 5,
|
|
"opsPerSec": 2256460,
|
|
"avgMicros": 0.443,
|
|
"opsMin": 2247109,
|
|
"opsMax": 2268667,
|
|
"variance": 0.01
|
|
},
|
|
{
|
|
"name": "retriever.cosine(384-d, unit-norm dot-only)",
|
|
"iters": 100000,
|
|
"trials": 5,
|
|
"opsPerSec": 2784972,
|
|
"avgMicros": 0.359,
|
|
"opsMin": 2777302,
|
|
"opsMax": 2804364,
|
|
"variance": 0.01
|
|
}
|
|
]
|
|
} |