4 Commits

Author	SHA1	Message	Date
ruv	cff24dbff6	perf(guidance): M4 quantization — 1-bit RaBitQ-style signatures, 2.7x at N=1000 ... Horizon: guidance-sota-2026-05 (M4). The real SOTA win. ## What changed `ShardRetriever` now maintains a packed Uint32Array of 1-bit sign signatures alongside the existing Float32 embedding matrix. For dim=384 that's 12 Uint32 words = 48 bytes per shard (vs 1,536 bytes for the Float32 embedding — a 32x memory reduction for the index). `scoreShards` runs the cosine via Hamming-distance over those signatures when (a) the index is current, (b) shard count ≥ 100 (to amortise the query-signature build), and (c) dims match. The sign-random-projection theorem (Charikar 2002) gives: cos(θ) ≈ cos(π · hamming(sig_q, sig_s) / dim) which is accurate enough for the retriever's sort + intent-boost + risk-boost downstream. ## Microbench (bench-quantization.mjs) cosine.dot (float32, 384-d) 3,006,455 ops/s 332.62 ns/pair hamming.popcount (uint32, 12 words) 32,862,982 ops/s 30.43 ns/pair ──────────── Hamming speedup vs dot: 10.93x ## End-to-end (bench-retriever-scale.mjs) N baseline unfilt M4 unfilt speedup 10 63,910 70,777 1.11x 100 12,135 26,372 2.17x 500 2,470 6,468 2.62x 1000 1,303 3,522 🚀 2.70x N baseline filt M4 filt speedup 10 166,274 198,451 1.19x 100 46,419 85,311 1.84x 500 12,974 27,081 2.09x 1000 6,662 16,073 2.41x All 1,331 existing tests pass. Build clean. ## Why this works where Phase 1 + M3 didn't Phase 1 / M3 attacked the dot product's *constants* (eliminate sqrt, hoist regex, pack memory). Those are all within V8's optimisation ceiling — the JIT already does most of it. M4 attacks the *algorithm*: instead of 384 multiplies per pair, we do 12 XOR + popcount. The instruction count itself drops ~30x, and even with JS's no-native-popcnt penalty we measure 11x per-pair. End-to-end speedup is bounded by Amdahl on the non-cosine work (sorting, intent/risk boosts, result construction), so 2.7x is the expected ceiling for this fixture. At larger N or with a heavier embedding model, the cosine fraction grows and the speedup approaches the full 11x. ## Memory 48 bytes/shard signature vs 1,536 bytes/embedding — a 32x reduction. At N=10,000 shards: 480KB signatures vs 15MB embeddings. Real for cold-start RAM on hooks-running daemons. Co-Authored-By: RuFlo <ruflo-bot@users.noreply.github.com>	2026-05-21 23:51:48 -04:00
ruv	12ecb39cbc	perf(guidance): M3 substrate — packed embedding matrix + filtered-query bench coverage ... Horizon: guidance-sota-2026-05 (M3). Honest report of what works and what didn't. ## What changed retriever.ts gains a contiguous Float32Array pack of all shard embeddings, populated once at indexShards() time. scoreShards then walks the pack inline (sequential reads, V8-friendly inner loop) instead of dereferencing per-shard heap Float32Array allocations. bench-retriever-scale.mjs gains a filtered-query column so we can measure the effect of risk-filter / repo-scope path. The original code already does filter-then-continue (skipping cosine on filtered shards), which is worth ~5x at N=1000 — the bench scaffold now proves that. ## Honest findings After three iterations of "optimization": N | unfilt baseline | unfilt M3v2 | filt baseline | filt M3v2 --|-----------------|-------------|---------------|---------- 10| 63,910 | 73,187 | 166,274 | 192,414 100| 12,135 | 12,234 | 46,419 | 47,742 500| 2,470 | 2,468 | 12,974 | 11,807 1000| 1,303 | 1,333 | 6,662 | 6,646 All speedups vs baseline are within ±5% (noise band). The package was already well-tuned: - cosineSimilarity was the dominant cost at scale (~400µs of 760µs at N=1000) - The existing filter-then-continue ordering already short-circuits cosine on 80% of shards in the typical riskFilter case - V8's JIT optimizes the original filter-chain patterns nearly as well as the manually-unrolled forms ## What this proves The benchmark infrastructure (multi-trial median, four scale points, both filtered and unfiltered query paths) IS the SOTA scaffold. It establishes: 1. Reproducible per-N measurement at ±2% variance 2. The filter optimization is real (5.1x at N=1000) 3. The cosine dominates at scale — further gains require RaBitQ-style 1-bit-per-dim quantization (M4) or algorithmic change (true HNSW with logarithmic graph traversal, requires sidecar dep) This commit lands the substrate for M4 to measure against. M3 closure is "infrastructure + filter-baseline proven", not "package made faster" — V8 already did most of the work that's easily expressible in JS. ## Verification - All 1,331 tests pass (`npm test` in v3/@claude-flow/guidance) - Build clean - 5-trial median benchmarks reproducible at ±2% Co-Authored-By: RuFlo <ruflo-bot@users.noreply.github.com>	2026-05-21 23:27:23 -04:00
ruv	6d6d659ca9	perf(guidance): Phase 1 — hot-path cleanup + scaling benchmark infrastructure ... Horizon: guidance-sota-2026-05 (M1 + M2). First commit on the path to SOTA-with-proof for @claude-flow/guidance. ## What changed Three localized hot-path cleanups + a benchmark infrastructure that exposes the real opportunity (M3, HNSW substrate). ### Code changes (no public API surface change) 1. **analyzer.ts extractMetrics** — replace 6 separate `lines.filter()` passes + 2 `for-of` loops with a single iteration that accumulates every line-derived metric. Hoist 11 in-body regex literals to module scope. Net: cleaner code, no measurable speedup on V8 (the JIT already optimizes filter chains well). 2. **compiler.ts parseRule** — replace 4 `new RegExp(PATTERN.source, 'gi')` constructions per call with `text.matchAll(PATTERN)` against the existing module-level global patterns. Net: cleaner code, no measurable speedup (V8 internalizes the regex objects). 3. **retriever.ts cosineSimilarity** — drop the 3-accumulator sqrt-div-clamp form to a pure dot product. The embeddings this retriever consumes are always unit-normalised (HashEmbeddingProvider divides by L2 norm at line 134, ONNX providers emit unit vectors by design), so `sqrt(normA)*sqrt(normB) === 1` and the result already lies in [-1, 1]. Net: +8.2% at N=1000 shards, +10.9% at N=500. ### Benchmark infrastructure (new) - `scripts/bench-phase-1.mjs` — multi-trial median microbenchmarks for the 3 hot paths. Stable to ~2%. - `scripts/bench-retriever-scale.mjs` — end-to-end `retriever.retrieve()` at N ∈ {10, 100, 500, 1000}. Reveals the real scaling cliff. - `scripts/README.md` — how to run + findings. - `docs/benchmarks/guidance-{baseline,phase-1}.json` — captured runs. - `docs/benchmarks/guidance-retriever-scale-{baseline,phase-1}.json` — scaling curves at four shard counts. ## Honest M2 status Original target: ≥2x speedup or ≥30% reduction. Phase 1 delivers ~1.1x at scale — short of the bar. **But the baseline benchmarks revealed what the real lever is**: `scoreShards` linear scan goes from 14µs at N=10 to 760µs at N=1000 — a 55x latency growth that's pure O(n). M3 (HNSW substrate) targets that bottleneck directly. The infrastructure landing here is the proof scaffold M3 will measure against. ## Verification - `npm test` in v3/@claude-flow/guidance: all 1,331 tests pass - `npm run build`: clean - Bench scripts runnable end-to-end with JSON artifacts under `docs/benchmarks/` Co-Authored-By: RuFlo <ruflo-bot@users.noreply.github.com>	2026-05-21 23:21:29 -04:00
Claude	e06f273a1b	feat: add CLAUDE.md generators, analyzer, and auto-optimizer ... Generators (src/generators.ts): - generateClaudeMd() - structured CLAUDE.md from project profile with language-specific invariants (TS, JS, Python, Rust, Go, Java) and framework rules (React, Next.js, Express, Django, etc.) - generateClaudeLocalMd() - CLAUDE.local.md for personal overrides with machine notes, editor prefs, and experimental rules - generateSkillMd() - Claude Code skills with YAML frontmatter - generateAgentMd() - Agent definitions with YAML frontmatter - generateAgentIndex() - Agent index.yaml grouped by category - scaffold() - Full .claude/ directory structure generation Analyzer (src/analyzer.ts): - 6-dimension scoring (0-100 each, weighted composite): Structure (20%), Coverage (20%), Enforceability (25%), Compilability (15%), Clarity (10%), Completeness (10%) - benchmark() - before/after comparison with per-dimension deltas - autoOptimize() - iteratively applies high-priority suggestions with concrete patches, re-scores to verify improvement - formatReport() / formatBenchmark() - human-readable output Analysis of existing CLAUDE.md files: - Root CLAUDE.md: 73/100 (C) -> 77/100 after optimization - v3/CLAUDE.md: 64/100 (D) -> 78/100 after optimization - Key findings: low rule density, oversized constitution section, missing Architecture section Tests: 141 new tests (68 generators + 73 analyzer), 1229 total passing https://claude.ai/code/session_01HLi9WmfvFgkDLJxLKGAUTf	2026-02-01 20:48:36 +00:00