Files
Dragan Spiridonov eebb76731e feat: implement RuVector Phase 5 Milestones 3-4 — Scale, Optimization & Advanced Learning (ADR-087)
Milestone 3 — Scale & Optimization (R7-R10):
- R7: Meta-learning enhancements (DecayingBeta, PlateauDetector with CUSUM,
  ParetoFront, CuriosityBonus) wired into DomainTransferEngine
- R8: PageRank pattern importance solver with citation graph pipeline
  (PatternCitationGraph bootstraps from 129 existing patterns, blended
  scoring in pattern-promotion.ts)
- R9: Spectral graph sparsifier (degree-based leverage heuristic) wired
  into mincut-wrapper.ts as optional preprocessing
- R10: Reservoir replay buffer with coherence-gated admission and CUSUM
  drift detection, wired into ExperienceReplay (write + read paths)

Milestone 4 — Advanced Learning (R11-R12):
- R11: E-prop online learning (12 bytes/synapse, eligibility traces,
  feedback alignment) — RL algorithm #10, extends BaseRLAlgorithm
- R12: Granger causality for test failure prediction (VAR + F-test with
  full statistical math) wired into defect-intelligence coordinator with
  persistent history accumulation

Infrastructure:
- 8 feature flags (all default true per opt-out policy), convenience
  functions, env var support, CLI descriptions
- RLAlgorithmType union: 9 → 10 members (added 'eprop')
- ADR-087 status: Proposed → Active (M1-4 complete)
- 234 new tests across 8 test files, all passing
- Performance benchmarks for PageRank, sparsification, reservoir admission
- 0 regressions in existing test suite

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-03-29 17:41:06 +00:00

57 KiB
Raw Permalink Blame History

RuVector Improvements Plan: 14 New Capabilities

Date: 2026-03-29 Version: AQE v3.8.11 Branch: march-fixes-and-improvements Status: ACTIVE (Milestones 1-4 complete, all feature flags enabled, Milestone 5 backlog) Companion: ruvector-integration-plan.md (Issue #355, 26 remaining items) Scope: 14 NEW capabilities NOT in Issue #355, plus EWC++ activation


Table of Contents

  1. Executive Summary
  2. Architecture Overview
  3. Integration Order and Dependencies
  4. Milestone 1: Pattern Intelligence
  5. Milestone 2: Graph Learning
  6. Milestone 3: Scale and Optimization
  7. Milestone 4: Advanced Learning
  8. Milestone 5: Backlog
  9. Risk Assessment
  10. Testing Strategy
  11. Deliverables Checklist

Executive Summary

What

14 new RuVector capabilities identified from analysis of 113 crates in /tmp/RuVector, plus completing the EWC++ Fisher regularization cycle in the existing @ruvector/sona integration (persistence is wired but production call sites are missing). These are distinct from the 26 items tracked in Issue #355.

Why

AQE's 150K+ pattern memory, 9 RL algorithms, and SONA three-loop engine are production-ready but operate without: compositional pattern fingerprinting (HDC), statistical drift detection (CUSUM), event-sourced pattern history (delta), zero-label graph learning (GraphMAE), associative memory (Hopfield), or sublinear solvers for 500K+ scale. Adding these capabilities addresses three gaps:

  1. Speed at scale: O(1) HDC fingerprints and O(log n) solvers prevent degradation as pattern count grows beyond 150K toward 500K+.
  2. Self-correction: CUSUM drift detection and delta event sourcing give the system the ability to detect when its own models are drifting and roll back to known-good states.
  3. Learning depth: GraphMAE, Hopfield, and e-prop add unsupervised/associative/online learning modes that complement the existing supervised RL suite.

Expected Outcomes

Metric Current After Milestone 2 After Milestone 4
Pattern fingerprint time N/A (no fingerprints) <1us per 10K-bit HDC bind <1us
Drift detection None CUSUM with 4-gate model CUSUM + coherence gated
Pattern history/rollback None Delta event sourcing Delta + CRDT consensus
Graph learning Supervised GNN only + GraphMAE zero-label + cold-tier, + Granger
Associative recall HNSW approximate + Hopfield exact recall + Hopfield
Scale ceiling ~150K patterns ~150K patterns 500K+ via O(log n) solver
EWC++ Fisher computation Persistence wired, three-loop never invoked Three-loop invoked by domain coordinators Active + monitored
Online learning algorithms 9 RL (all backprop) 9 RL 9 RL + e-prop (no backprop)

Architecture Overview

Where These Fit in AQE

src/integrations/ruvector/          <-- 43 files, primary integration layer
  |
  |-- [EXISTING] sona-wrapper.ts      -- SONA + three-loop + EWC++ (dormant)
  |-- [EXISTING] gnn-wrapper.ts       -- @ruvector/gnn NAPI wrapper
  |-- [EXISTING] coherence-gate.ts    -- Heuristic coherence scoring
  |-- [EXISTING] domain-transfer.ts   -- Thompson Sampling cross-domain
  |-- [EXISTING] feature-flags.ts     -- Runtime feature toggles
  |
  |-- [NEW R1]  hdc-fingerprint.ts    -- HDC binary hypervector fingerprints
  |-- [NEW R3]  delta-tracker.ts      -- Delta event sourcing for patterns
  |-- [NEW R5]  hopfield-memory.ts    -- Modern Hopfield associative memory
  |-- [NEW R8]  solver-adapter.ts     -- Sublinear PageRank solver
  |-- [NEW R9]  spectral-sparsifier.ts -- Graph sparsification adapter
  |-- [NEW R11] eprop-learner.ts      -- E-prop online learning
  |-- [NEW R12] temporal-causality.ts -- Granger causality for test history

src/learning/                        <-- Learning engine, pattern lifecycle
  |-- [MODIFY R10] experience-replay  -- Add reservoir replay with coherence
  |-- [MODIFY R7]  pattern-promotion  -- Meta-learning enhancements

src/integrations/rl-suite/           <-- RL algorithm suite
  |-- [MODIFY R11] algorithms/        -- Add e-prop as new algorithm

Binding Types

Improvement Source Crate Binding New npm Package?
R1: HDC ruvector-nervous-system::hdc WASM (no_std) Yes, or inline WASM
R2: CUSUM neural-trader-coherence TypeScript only No
R3: Delta ruvector-delta-wasm WASM Yes
R4: GraphMAE ruvector-gnn::graphmae Existing @ruvector/gnn No
R5: Hopfield ruvector-nervous-system::hopfield WASM Yes, or inline WASM
R6: Cold-Tier GNN ruvector-gnn::cold_tier Existing @ruvector/gnn No
R7: Meta-Learning ruvector-domain-expansion::meta_learning TypeScript No
R8: Solver ruvector-solver-node NAPI Yes
R9: Sparsifier ruvector-sparsifier WASM or TypeScript TBD
R10: Reservoir Replay neural-trader-replay TypeScript No
R11: E-prop ruvector-nervous-system::plasticity::eprop WASM Yes, or inline WASM
R12: Granger ruvector-graph-transformer::temporal WASM or NAPI TBD
R13: Cognitive Routing ruvector-nervous-system::routing WASM Yes
R14: Hyperbolic HNSW ruvector-hyperbolic-hnsw NAPI Yes
EWC++ @ruvector/sona (already installed) NAPI (existing) No

Integration Order and Dependencies

                    ┌─────────────┐
                    │  EWC++ Act. │ (no deps, existing code)
                    └──────┬──────┘
                           │
  ┌────────┐  ┌────────┐  │  ┌────────┐
  │ R1:HDC │  │ R2:CUS │  │  │ R3:Del │   Milestone 1 (all independent)
  └────┬───┘  └────┬───┘  │  └────┬───┘
       │           │      │       │
       └─────┬─────┘      │       │
             │             │       │
  ┌────────┐ │  ┌────────┐│  ┌────────┐
  │R4:GMAE │ │  │R5:Hopf ││  │R6:Cold │   Milestone 2
  └────┬───┘ │  └────┬───┘│  └────┬───┘
       │     │       │    │       │
       └──┬──┘       │    │       │
          │          │    │       │
  ┌───────┴┐  ┌──────┴┐  │  ┌────┴───┐
  │R7:Meta │  │R8:Solv │  │  │R9:Spar │   Milestone 3
  └────────┘  └───────┘   │  └────────┘
                    ┌──────┴──────┐
                    │ R10:Replay  │
                    └─────────────┘
  ┌────────┐  ┌────────┐
  │R11:Epr │  │R12:Grng│                   Milestone 4
  └────────┘  └────────┘

  ┌────────┐  ┌────────┐
  │R13:Cog │  │R14:Hyp │                   Milestone 5 (backlog)
  └────────┘  └────────┘

Hard Dependencies

  • R7 (Meta-Learning) depends on R2 (CUSUM) for plateau detection input
  • R8 (Solver) depends on R1 (HDC) for fingerprint-accelerated PageRank
  • R10 (Reservoir Replay) depends on R2 (CUSUM) for coherence-gated admission
  • R6 (Cold-Tier GNN) depends on R4 (GraphMAE) sharing the same GNN wrapper

Soft Dependencies (beneficial but not blocking)

  • R5 (Hopfield) benefits from R1 (HDC) for encoding, but can use raw float vectors
  • R12 (Granger) benefits from R3 (Delta) for historical event sequences
  • R9 (Sparsifier) benefits from R4 (GraphMAE) for pre-sparsification embeddings

Milestone 1: Pattern Intelligence (R1, R2, R3, EWC++)

Goal: Add compositional fingerprinting, drift detection, event-sourced history, and activate EWC++ Fisher computation. All four items are independent.

Timeline: 2-3 weeks


EWC++ Completion

Property Value
Complexity M (upgraded from S — more work than originally scoped)
New Files None
Modified Files sona-persistence.ts, domain coordinator files (src/domains/*/coordinator.ts)
Dependencies (npm) None (uses existing @ruvector/sona 0.1.5)

Description: The SONA three-loop engine (sona-three-loop.ts) has a complete TypeScript EWCPlusPlus class (Fisher diagonal, online blending, loss computation, task boundary detection) at line 318. Fisher persistence is already wired: sona-persistence.ts:273 calls setFisherPersistence() and line 287 calls restoreFisher() during initialization (when useSONAThreeLoop flag is enabled, which defaults to true). Fisher matrices persist to and restore from SQLite correctly.

However, the three-loop methods that actually drive EWC++ are never called in production:

  • backgroundConsolidate() — defined (line 744) but no domain coordinator calls it
  • instantAdapt() — defined (line 643) but never called
  • recordOutcome() — defined (line 706) but never called

Fisher matrices persist/restore on startup but never update because consolidation is never triggered. EWC++ is inert.

Changes:

  1. In domain coordinators (e.g., quality-assessment/coordinator.ts): call threeLoopEngine.recordOutcome(result) after pattern evaluation completes
  2. In domain coordinators: call threeLoopEngine.backgroundConsolidate() during dream cycle or after N outcomes (configurable threshold)
  3. In domain coordinators: call threeLoopEngine.instantAdapt(pattern) for real-time pattern adaptation
  4. Add round-trip persistence tests (persist → restart → verify Fisher trace matches)
  5. Add integration test: train domain A, consolidate, train domain B, verify EWC loss increases

Success Criteria:

  • backgroundConsolidate() is called by at least one domain coordinator during dream cycles
  • recordOutcome() is called after pattern evaluations in at least one domain
  • Fisher matrix updates after consolidation (verified by comparing getEWCMetrics() before and after)
  • Fisher matrix round-trip: persist → engine restart → restore → metrics match pre-restart values
  • EWC regularization loss is non-zero when patterns from multiple domains are loaded
  • No regression in existing 15,493 tests

Verification:

  • Unit test: mock domain coordinator, call recordOutcome 10 times, trigger backgroundConsolidate, verify Fisher diagonal is non-zero
  • Integration test: start engine, train on domain A, consolidate (Fisher persisted), train on domain B, verify EWC loss increases
  • Integration test: restart engine, verify Fisher trace matches pre-restart value
  • Grep verification: confirm backgroundConsolidate and recordOutcome have production call sites outside of test files

R1: HDC Pattern Fingerprinting

Property Value
Complexity M
New Files src/integrations/ruvector/hdc-fingerprint.ts, tests/unit/integrations/ruvector/hdc-fingerprint.test.ts
Modified Files feature-flags.ts, index.ts
Dependencies (npm) @ruvector/hdc-wasm (to be published from ruvector-nervous-system::hdc) or inline WASM blob

Description: 10K-bit binary hypervectors enable O(1) compositional pattern binding. Two patterns can be compared via Hamming distance in nanoseconds. The no_std Rust implementation compiles to a small WASM binary (~15KB). This replaces cosine-similarity fingerprinting for Tier 1 Agent Booster pattern matching.

Implementation:

  1. Create hdc-fingerprint.ts with:
    • HdcFingerprinter class wrapping WASM bind(), unbind(), similarity() ops
    • fingerprint(pattern: QEPattern): Uint8Array (10K-bit = 1250 bytes)
    • compositionalBind(a: Uint8Array, b: Uint8Array): Uint8Array
    • hammingDistance(a: Uint8Array, b: Uint8Array): number
    • Graceful fallback to TypeScript bit-array implementation if WASM unavailable
  2. Add feature flag useHDCFingerprinting in feature-flags.ts
  3. Export from index.ts

Success Criteria:

  • fingerprint() produces deterministic 10K-bit output for same input
  • hammingDistance() returns 0 for identical patterns, ~5000 for random pairs
  • compositionalBind() is associative: bind(A, bind(B, C)) = bind(bind(A, B), C)
  • Benchmark: fingerprint + compare < 1us for single pair (WASM path)
  • Benchmark: fingerprint + compare < 100us for single pair (TypeScript fallback)
  • Feature flag defaults to true (opt-out model — verified features ship enabled; users disable if needed)

Verification:

  • Unit test: determinism, associativity, distance properties
  • Benchmark: timing comparison WASM vs TypeScript fallback
  • Integration test: wire into pattern-store search as optional fast-path filter

R2: CUSUM Drift Detection

Property Value
Complexity M
New Files src/integrations/ruvector/cusum-detector.ts, tests/unit/integrations/ruvector/cusum-detector.test.ts
Modified Files coherence-gate-energy.ts (post-extraction), tests/unit/integrations/ruvector/coherence-gate.test.ts
Dependencies (npm) None (TypeScript implementation from neural-trader-coherence algorithm)
Prerequisite Coherence-gate extraction (see below)

Prerequisite: Coherence-Gate File Extraction

coherence-gate.ts is currently 930 lines — 86% over the project's 500-line limit. It must be split before adding CUSUM. Proposed extraction:

New File Lines Contents
coherence-gate-core.ts ~250 CoherenceGate class, public API, types, factory functions
coherence-gate-energy.ts ~300 Reflex/retrieval tiers, contradiction detection, Laplacian deviation
coherence-gate-vector.ts ~120 FNV-1a hashing, feature vectors, cosine similarity (pure math)
coherence-gate-cohomology.ts ~80 WASM CohomologyEngine lazy loader
coherence-gate.ts ~20 Barrel re-export for backward compatibility

This is a mechanical extraction with no logic changes. Two consumers import from coherence-gate.ts (coherence-validator.ts and the test file) — the barrel re-export preserves all existing imports.

Description: Add CUSUM (Cumulative Sum) change-point detection as a new standalone module. The 4-gate model (retrieve/write/learn/act) gains a statistical drift detector that fires when the cumulative deviation from expected coherence exceeds a threshold. CUSUM lives in its own file (cusum-detector.ts) and is integrated into coherence-gate-energy.ts post-extraction.

Implementation:

  1. First: Execute coherence-gate extraction (prerequisite above)
  2. Create cusum-detector.ts (~80 lines):
    • CusumDetector class
    • update(value: number): { driftDetected: boolean; cumulativeSum: number }
    • Configurable threshold, slack parameter, reset behavior
    • Two-sided detection (both positive and negative drift)
  3. Integrate into coherence-gate-energy.ts validateAction() method for the 4 gate types
  4. Emit drift-detected event via existing EventBus when CUSUM triggers

Success Criteria:

  • CUSUM fires within 10 samples of a mean shift of 2 sigma
  • No false positives on stationary coherence sequences (100K samples, <0.1% false alarm rate)
  • 4-gate model (retrieve/write/learn/act) each have independent CUSUM state
  • Drift events appear in observability logs with gate type and cumulative sum value
  • Backward compatible: existing coherence checks unaffected when CUSUM is not triggered

Verification:

  • Unit test: inject known mean shift, verify detection within expected sample count
  • Unit test: stationary sequence produces no alarms
  • Integration test: feed real coherence scores from pattern promotion, verify no spurious alarms

R3: Delta Event Sourcing

Property Value
Complexity L
New Files src/integrations/ruvector/delta-tracker.ts, tests/unit/integrations/ruvector/delta-tracker.test.ts
Modified Files feature-flags.ts, index.ts
Dependencies (npm) @ruvector/delta-wasm (to be published from ruvector-delta-core) or TypeScript-only initial implementation

Description: Track pattern version history as a sequence of delta events. Enables rollback to any previous pattern state, incremental sync between agents, and eventually CRDT-based consensus across fleet members. Initial implementation is TypeScript-only using the existing SQLite persistence layer.

Migration: Existing 150K+ patterns have no version history. On first activation (feature flag enabled), each existing pattern receives a "v0" genesis snapshot delta. History tracking begins from that point forward. No retroactive history synthesis.

Implementation:

  1. Create delta-tracker.ts with:
    • DeltaTracker class storing deltas in SQLite (pattern_deltas table)
    • recordDelta(patternId: string, before: object, after: object): DeltaEvent
    • rollback(patternId: string, toVersion: number): object
    • getHistory(patternId: string): DeltaEvent[]
    • incrementalSync(since: number): DeltaEvent[] (for agent-to-agent sync)
    • JSON diff using existing fast-json-patch dependency
  2. Add feature flag useDeltaEventSourcing in feature-flags.ts
  3. Wire into pattern-store.ts update path (after pattern modification)

Success Criteria:

  • Every pattern update creates a delta event with before/after diff
  • Rollback to version N restores exact pattern state (verified by deep equality)
  • incrementalSync(since) returns only deltas after the given timestamp
  • Delta storage overhead < 20% of pattern storage (measured on 1K pattern updates)
  • Feature flag defaults to true (opt-out model — verified features ship enabled; users disable if needed)
  • SQLite schema migration adds pattern_deltas table without touching existing tables

Verification:

  • Unit test: create, update 5 times, rollback to version 2, verify state
  • Unit test: incremental sync returns correct subset
  • Integration test: two simulated agents sync via delta exchange
  • Benchmark: 1K sequential updates complete in < 500ms

Milestone 2: Graph Learning (R4, R5, R6)

Goal: Add zero-label graph embeddings, associative pattern recall, and cold-tier GNN training for larger-than-RAM graphs.

Timeline: 2-3 weeks (after Milestone 1)


R4: GraphMAE Self-Supervised Learning

Property Value
Complexity M
New Files None (extends existing file)
Modified Files gnn-wrapper.ts, tests/unit/integrations/ruvector/gnn-wrapper.test.ts
Dependencies (npm) None (uses existing @ruvector/gnn 0.1.19 if graphmae is exposed)

Description: GraphMAE (Graph Masked Autoencoder) produces embeddings from code dependency graphs without labeled training data. The ruvector-gnn::graphmae module (439 lines Rust) masks random nodes, reconstructs them, and learns structural embeddings. If @ruvector/gnn 0.1.19 does not expose graphmae, implement the masking + reconstruction in TypeScript as a preprocessing step that feeds into the existing GNN forward pass.

Implementation:

  1. Add to gnn-wrapper.ts:
    • GraphMAEEncoder class (~120 lines)
    • maskAndEncode(graph: QEGraph, maskRatio: number): Float32Array[]
    • reconstructionLoss(original: Float32Array[], reconstructed: Float32Array[]): number
    • Random node masking (default 50%) with learnable mask token
  2. Wire into code-intelligence domain for dependency graph embedding

Success Criteria:

  • Produces 128-dim embeddings for nodes in a 1K-node graph
  • Reconstruction loss decreases over 10 training epochs (measured, not just asserted)
  • Embeddings cluster similar code modules (measured by intra-cluster distance < inter-cluster distance)
  • Falls back gracefully when GNN native module unavailable

Verification:

  • Unit test: mask ratio 0.5 masks ~50% of nodes (within 5% tolerance)
  • Unit test: reconstruction loss monotonically decreasing over epochs
  • Integration test: embed a real code dependency graph (AQE's own src/ imports) and verify clustering

R5: Modern Hopfield Networks

Property Value
Complexity M
New Files src/integrations/ruvector/hopfield-memory.ts, tests/unit/integrations/ruvector/hopfield-memory.test.ts
Modified Files feature-flags.ts, index.ts
Dependencies (npm) @ruvector/hopfield-wasm (to be published) or TypeScript implementation

Description: Modern Hopfield networks provide exponential-capacity associative memory with exact pattern recall (unlike HNSW which is approximate). For AQE, this means storing canonical "golden" patterns that must be retrieved exactly (e.g., known-good test templates, critical defect signatures).

Implementation:

  1. Create hopfield-memory.ts:
    • HopfieldMemory class with configurable capacity and dimension
    • store(pattern: Float32Array, metadata: object): void
    • recall(query: Float32Array): { pattern: Float32Array; metadata: object; energy: number }
    • batchRecall(queries: Float32Array[]): RecallResult[]
    • Exponential interaction function (modern Hopfield, not classical)
    • TypeScript implementation first, WASM upgrade path later
  2. Add feature flag useHopfieldMemory
  3. Wire into pattern-store as exact-recall tier (HNSW for approximate, Hopfield for exact)

Success Criteria:

  • Exact recall: store 1000 patterns, query each, 100% exact match
  • Capacity: store 10K patterns in 128-dim without retrieval degradation
  • Energy function: recall() returns energy score indicating confidence
  • Benchmark: single recall < 1ms for 1K stored patterns
  • Feature flag defaults to false

Verification:

  • Unit test: store N patterns, recall each, verify bit-exact match
  • Unit test: query with noisy version of stored pattern, verify correct recall
  • Benchmark: recall latency vs pattern count (100, 1K, 10K)

R6: Cold-Tier GNN Training

Property Value
Complexity M
New Files None
Modified Files gnn-wrapper.ts, tests/unit/integrations/ruvector/gnn-wrapper.test.ts
Dependencies (npm) None (uses existing @ruvector/gnn 0.1.19 if cold_tier is exposed)

Description: Block-aligned I/O with hotset caching enables GNN training on graphs that exceed available RAM. For AQE's pattern graph at 150K+ nodes, this prevents OOM during full-graph training by streaming cold nodes from disk.

Implementation:

  1. Add to gnn-wrapper.ts:
    • ColdTierTrainer class (~100 lines)
    • trainWithColdTier(graph: QEGraph, hotsetSize: number): TrainingResult
    • Block-aligned node loading (4KB blocks matching OS page size)
    • LRU hotset cache with configurable size (default: 10K nodes)
    • Falls back to full in-memory training when graph fits in hotsetSize
  2. Wire into existing GNN training paths as transparent optimization

Success Criteria:

  • Training produces equivalent loss to full in-memory training (within 5% tolerance)
  • Peak memory capped at hotsetSize * nodeSize (not full graph size)
  • Benchmark: 50K node graph trains to convergence with 10K hotset
  • Graceful degradation: if disk I/O is slow, falls back to in-memory

Verification:

  • Unit test: small graph (100 nodes) produces identical results in both modes
  • Benchmark: memory usage measurement during training of 50K node graph
  • Integration test: train on AQE pattern graph, verify embeddings are usable

Milestone 3: Scale and Optimization (R7, R8, R9, R10)

Goal: Meta-learning enhancements, sublinear solvers, graph sparsification, and reservoir replay. These are the items that unlock 500K+ pattern scale.

Timeline: 3-4 weeks (after Milestone 2)


R7: Meta-Learning Enhancements

Property Value
Complexity M
New Files None
Modified Files domain-transfer.ts, tests/unit/integrations/ruvector/domain-transfer.test.ts
Dependencies (npm) None (TypeScript implementation)
Depends On R2 (CUSUM) for plateau detection

Description: Enhance the existing cross-domain transfer engine with four mechanisms from ruvector-domain-expansion::meta_learning: DecayingBeta (exploration decay over time), PlateauDetector (detect when learning stalls), ParetoFront (multi-objective optimization), CuriosityBonus (reward novel patterns).

Implementation:

  1. Add to domain-transfer.ts (currently 324 lines, room for ~175 more):
    • DecayingBeta: modify Thompson Sampler alpha/beta to decay with time
    • PlateauDetector: use R2's CUSUM on learning rate to detect stalls
    • ParetoFront: track Pareto-optimal transfer candidates (multi-objective)
    • CuriosityBonus: add intrinsic reward for novel source-target pairs
  2. Integrate plateau detection with CUSUM from coherence gate

Success Criteria:

  • DecayingBeta reduces exploration rate by 50% after 100 successful transfers
  • PlateauDetector fires when transfer success rate is flat for 20+ attempts
  • ParetoFront correctly identifies non-dominated candidates (verified on synthetic data)
  • CuriosityBonus increases sampling probability of untried domain pairs
  • No regression in existing domain-transfer tests

Verification:

  • Unit test: each mechanism independently with synthetic data
  • Integration test: run full transfer cycle, verify exploration decays and plateaus are detected

R8: Sublinear Solver

Property Value
Complexity L
New Files src/integrations/ruvector/solver-adapter.ts, tests/unit/integrations/ruvector/solver-adapter.test.ts
Modified Files src/learning/pattern-promotion.ts, feature-flags.ts, index.ts
Dependencies (npm) @ruvector/solver-node (NAPI bindings from ruvector-solver-node)
Depends On R1 (HDC) for fingerprint-accelerated node identification

Description: O(log n) PageRank solver for graph-based pattern importance scoring. Currently pattern-promotion.ts (qe-patterns.ts:322) uses a simple O(1) weighted formula (confidence * 0.3 + usageScore * 0.2 + successRate * 0.5) with no inter-pattern relationship awareness. No pattern citation graph exists today. This improvement has two parts: (1) build a pattern citation/dependency graph, and (2) run sublinear PageRank over it. The ruvector-solver-node crate provides NAPI bindings for sublinear PageRank.

Implementation:

  1. Design and build pattern citation graph in pattern-promotion.ts:
    • Track which patterns co-occur in the same quality assessments
    • Track which patterns were derived from or supersede other patterns
    • Store graph edges in SQLite (pattern_citations table)
    • Bootstrap from existing usage data and co-occurrence analysis
  2. Create solver-adapter.ts:
    • SublinearSolver class wrapping @ruvector/solver-node
    • computeImportance(graph: PatternGraph): Map<string, number>
    • rankPatterns(patterns: QEPattern[]): RankedPattern[]
    • TypeScript fallback using power iteration (O(n) per iteration)
  3. Add feature flag useSublinearSolver
  4. Integrate graph-based scores with existing weighted formula (blend, not replace)

Success Criteria:

  • Pattern citation graph is populated from existing usage data (bootstrap)
  • PageRank scores on citation graph correlate with existing quality scores (Spearman rho > 0.5)
  • Benchmark: O(log n) confirmed via timing at 1K, 10K, 100K, 500K scales
  • Benchmark: 500K patterns scored in < 100ms (native), < 5s (TypeScript fallback)
  • Feature flag defaults to false
  • Optional dependency: works without @ruvector/solver-node installed
  • Graceful fallback: when graph is empty/sparse, fall back to existing weighted formula

Verification:

  • Unit test: compare solver output to known PageRank on small synthetic graph
  • Unit test: verify graph bootstrap populates edges from usage data
  • Benchmark: scaling curve (log n vs n) at multiple sizes
  • Integration test: feed real pattern graph, verify promotion ordering is reasonable

R9: Spectral Graph Sparsification

Property Value
Complexity M
New Files src/integrations/ruvector/spectral-sparsifier.ts, tests/unit/integrations/ruvector/spectral-sparsifier.test.ts
Modified Files feature-flags.ts, index.ts
Dependencies (npm) TBD (@ruvector/sparsifier-wasm or TypeScript implementation)

Description: Compress graphs while preserving Laplacian spectral properties. The sparsified graph has far fewer edges but the same eigenvalue structure, meaning coherence checks and min-cut analysis remain valid. Critical for scaling graph operations beyond 100K edges.

Implementation:

  1. Create spectral-sparsifier.ts:
    • SpectralSparsifier class
    • sparsify(graph: QEGraph, epsilon: number): QEGraph (epsilon controls approximation quality)
    • validateSpectral(original: QEGraph, sparsified: QEGraph): SpectralValidation
    • Effective resistance sampling (Spielman-Srivastava algorithm)
  2. Add feature flag useSpectralSparsification
  3. Wire into coherence checks and mincut-wrapper as preprocessing step

Success Criteria:

  • Sparsified graph has < 30% of original edges (at epsilon=0.3)
  • Laplacian eigenvalues within (1+epsilon) of original (top 10 eigenvalues)
  • Min-cut value preserved within 10% of original
  • Feature flag defaults to false

Verification:

  • Unit test: sparsify known graph, verify eigenvalue bounds
  • Unit test: min-cut on sparsified graph within tolerance
  • Benchmark: sparsification time at 10K, 50K, 100K edges

R10: Reservoir Replay with Coherence Gating

Property Value
Complexity M
New Files None
Modified Files src/integrations/agentic-flow/reasoning-bank/experience-replay.ts
Dependencies (npm) None (TypeScript implementation)
Depends On R2 (CUSUM) for coherence-gated admission

Description: Replace uniform random replay with reservoir sampling that uses the coherence gate to filter which experiences are admitted to the replay buffer. Segment experiences by coherence quality, prioritize replaying high-coherence examples during consolidation.

Implementation:

  1. Enhance existing experience-replay.ts:
    • Add ReservoirReplayBuffer class with coherence-gated admission
    • admit(experience: Experience, coherenceScore: number): boolean
    • sample(batchSize: number, minCoherence: number): Experience[]
    • Reservoir sampling (Algorithm R) for fixed-size buffer
    • Segment classification: high/medium/low coherence tiers
  2. Wire coherence gate from R2 into admission logic

Success Criteria:

  • Buffer maintains fixed size (configurable, default 10K)
  • High-coherence experiences are 3x more likely to be replayed than low-coherence
  • Reservoir sampling is statistically uniform within each coherence tier
  • Benchmark: admission decision < 0.1ms per experience
  • No regression in existing experience replay tests

Verification:

  • Unit test: fill buffer beyond capacity, verify reservoir sampling properties
  • Unit test: verify coherence-based admission rates
  • Integration test: run learning cycle with replay, verify quality improvement vs uniform replay

Milestone 4: Advanced Learning (R11, R12)

Goal: Add online learning without backpropagation and temporal causal discovery for test execution history.

Timeline: 2-3 weeks (after Milestone 3)


R11: E-prop Online Learning

Property Value
Complexity L
New Files src/integrations/ruvector/eprop-learner.ts, src/integrations/rl-suite/algorithms/eprop.ts, tests/unit/integrations/ruvector/eprop-learner.test.ts
Modified Files src/integrations/rl-suite/index.ts, feature-flags.ts
Dependencies (npm) @ruvector/eprop-wasm (to be published) or TypeScript implementation

Description: E-prop (eligibility propagation) provides online learning with 12 bytes per synapse, no backpropagation-through-time required. This is ideal for real-time pattern adaptation where the full backprop pipeline (used by the existing 9 RL algorithms: a2c, actor-critic, ddpg, decision-transformer, dqn, policy-gradient, ppo, q-learning, sarsa) is too expensive.

Implementation:

  1. Create eprop-learner.ts (WASM wrapper):
    • EpropNetwork class with configurable layer sizes
    • forward(input: Float32Array): Float32Array
    • updateOnline(reward: number): void (no explicit backward pass)
    • Eligibility trace tracking (12 bytes/synapse)
  2. Create algorithms/eprop.ts implementing RLAlgorithm interface (from src/integrations/rl-suite/interfaces.ts:133)
  3. Register in RL suite as algorithm #10
  4. Add feature flag useEpropOnlineLearning

Success Criteria:

  • Learns XOR problem within 100 episodes (sanity check)
  • Memory usage: 12 bytes/synapse verified (1M synapses = 12MB)
  • Online update time < 0.1ms per step (WASM), < 1ms (TypeScript fallback)
  • Integrates with existing RL suite via RLAlgorithm interface
  • Feature flag defaults to false

Verification:

  • Unit test: learn simple classification task
  • Unit test: verify memory footprint matches 12 bytes/synapse
  • Benchmark: compare online update latency to existing RL algorithms
  • Integration test: use as routing policy for agent task assignment

R12: Granger Causality (Temporal Graph Transformer)

Property Value
Complexity L
New Files src/integrations/ruvector/temporal-causality.ts, tests/unit/integrations/ruvector/temporal-causality.test.ts
Modified Files feature-flags.ts, index.ts
Dependencies (npm) TBD (@ruvector/temporal-wasm or TypeScript implementation)

Description: Discover causal chains in test execution history. When test A fails, does test B always fail 2 minutes later? Granger causality analysis on temporal test data reveals these hidden dependencies, improving defect prediction and test scheduling.

Implementation:

  1. Create temporal-causality.ts:
    • GrangerAnalyzer class
    • analyzeCausality(timeSeries: TestExecutionHistory[]): CausalLink[]
    • significanceTest(link: CausalLink, alpha: number): boolean
    • VAR model estimation (vector autoregression)
    • F-test for Granger causality significance
  2. Add feature flag useGrangerCausality
  3. Wire into defect prediction domain

Success Criteria:

  • Correctly identifies causal link in synthetic data (A causes B with lag 2)
  • F-test p-value < 0.05 for true causal links, > 0.05 for spurious
  • Handles up to 1000 time series of length 500 within 10 seconds
  • Feature flag defaults to false

Verification:

  • Unit test: synthetic causal data with known ground truth
  • Unit test: independent time series produce no significant links
  • Integration test: analyze AQE's own test execution history for causal patterns

Milestone 5: Backlog (R13, R14)

Goal: Fleet communication optimization and hierarchical embedding. Lower priority, implement when bandwidth constraints or hierarchy-aware search become bottlenecks.

Timeline: TBD (after Milestones 1-4 complete)


R13: Cognitive Routing

Property Value
Complexity L
New Files src/integrations/ruvector/cognitive-routing.ts
Dependencies (npm) @ruvector/cognitive-routing-wasm

Description: Predictive coding (90-99% bandwidth reduction), oscillatory routing, and global workspace theory for fleet communication. Only relevant when fleet has 10+ concurrent agents with heavy inter-agent communication.

Success Criteria:

  • Bandwidth reduction > 80% on synthetic fleet traffic
  • Message latency increase < 10% compared to uncompressed
  • Oscillatory routing correctly multiplexes 5+ concurrent message streams

R14: Hyperbolic HNSW

Property Value
Complexity L
New Files src/integrations/ruvector/hyperbolic-hnsw.ts
Dependencies (npm) @ruvector/hyperbolic-hnsw

Description: Poincare ball embeddings for hierarchical data. Code module trees and test suite hierarchies have natural tree structure that Euclidean HNSW distorts. Hyperbolic HNSW preserves hierarchical distances.

Success Criteria:

  • Nearest-neighbor recall > 95% for hierarchical data (vs ~85% Euclidean HNSW)
  • Correctly embeds AQE's module tree such that parent-child distance < sibling distance
  • Search latency within 2x of Euclidean HNSW at same recall level

Risk Assessment

Risk Impact Probability Mitigation
WASM packages not published to npm Blocks R1, R3, R5, R11 Medium All WASM items have TypeScript fallback implementations specified. Build WASM from source as fallback.
@ruvector/gnn 0.1.19 does not expose graphmae/cold_tier Blocks native path for R4, R6 Medium TypeScript implementations specified for both. Request NAPI exposure in next GNN release.
@ruvector/solver-node NAPI bindings not published Blocks native path for R8 Medium TypeScript power iteration fallback. O(n) is acceptable up to 100K.
Coherence gate file (930 lines) already exceeds 500-line limit Code quality Certain Mandatory prerequisite: extract into 4 modules before R2 work begins. Not conditional.
EWC++ backgroundConsolidate() causes training instability when first invoked Learning regression Medium Lambda is already tuned (1000.0). Add kill switch via feature flag. Monitor EWC loss and auto-disable if loss exceeds 10x initial. 48-hour monitoring window.
No pattern citation graph exists for R8 Solver has nothing to traverse Certain Design graph schema and bootstrap from usage data as prerequisite. Document in R8 implementation.
SQLite write contention from delta event writes (R3) DB lock under 8 concurrent agents Medium Use WAL mode (already configured); batch delta writes; test with concurrent writers.
Delta event sourcing storage bloat Disk usage Medium Implement delta compaction (merge consecutive small deltas). Set max history depth (default 100 versions).
Sublinear solver NAPI binary unavailable on ARM64 Platform support Medium TypeScript fallback is specified. Test on both x64 and ARM64 in CI.
Memory pressure from Hopfield networks at 10K patterns OOM risk Low Modern Hopfield uses softmax attention, not full pattern storage. Memory is O(stored * dim), not O(stored^2). Cap at configurable limit.

Testing Strategy

Unit Tests (per improvement)

Each new file gets a corresponding test file in tests/unit/integrations/ruvector/. Each modified file gets new test cases added to its existing test file.

Improvement Test File Min Test Count
EWC++ Extend sona-persistence.test.ts + new round-trip tests 7
R1: HDC hdc-fingerprint.test.ts 8
R2: CUSUM cusum-detector.test.ts + extend coherence-gate.test.ts 6
R3: Delta delta-tracker.test.ts 10
R4: GraphMAE Extend gnn-wrapper.test.ts 6
R5: Hopfield hopfield-memory.test.ts 8
R6: Cold-Tier Extend gnn-wrapper.test.ts 5
R7: Meta-Learning Extend domain-transfer.test.ts 8
R8: Solver solver-adapter.test.ts + graph bootstrap tests 10
R9: Sparsifier spectral-sparsifier.test.ts 6
R10: Replay Extend experience-replay tests 6
R11: E-prop eprop-learner.test.ts 8
R12: Granger temporal-causality.test.ts 6
R13: Cognitive cognitive-routing.test.ts 5
R14: Hyperbolic hyperbolic-hnsw.test.ts 5

Integration Tests

  • Prerequisites gate: Coherence-gate extraction complete (all existing tests pass); performance baselines measured and stored in tests/performance/baselines.json; pattern citation graph schema designed.
  • Milestone 1 gate: EWC++ backgroundConsolidate() called from production code paths, Fisher round-trip verified; HDC fingerprints match pattern-store search results; CUSUM drift events fire on real coherence data; Delta rollback restores exact state; existing 150K patterns get v0 genesis snapshots on R3 activation.
  • Milestone 2 gate: GraphMAE embeddings are usable by downstream consumers; Hopfield exact recall verified end-to-end; Cold-tier training does not OOM on 50K graph.
  • Milestone 3 gate: Pattern citation graph bootstrapped from usage data; solver PageRank correlates with existing quality scores (rho > 0.5); sparsified graph yields valid min-cut; reservoir replay improves learning quality.
  • Milestone 4 gate: E-prop integrates as RL algorithm #10 via RLAlgorithm interface; Granger correctly identifies known causal test dependencies.

Benchmarks

Every improvement with performance claims gets a benchmark in tests/performance/:

Benchmark Target Method
HDC fingerprint + compare < 1us (WASM) vitest bench with 10K iterations
CUSUM update < 0.01ms per sample vitest bench with 100K samples
Delta record + rollback < 500ms for 1K operations vitest bench sequential ops
Hopfield recall < 1ms at 1K patterns vitest bench
Solver PageRank O(log n) scaling curve Measure at 1K/10K/100K/500K
E-prop online update < 0.1ms (WASM) vitest bench

Regression Safety

  • Run npm test (15,493 tests) after every milestone
  • Run npm run build to verify TypeScript compilation
  • Feature flags default to false for all new capabilities
  • No existing test should be modified unless explicitly required

Deliverables Checklist

Milestone 1: Pattern Intelligence

  • EWC++ Completion

    • sona-persistence.ts calls setFisherPersistence() on init (already done at line 273)
    • sona-persistence.ts calls restoreFisher() on startup (already done at line 287)
    • Domain coordinators call backgroundConsolidate() during dream cycles
    • Domain coordinators call recordOutcome() after pattern evaluations
    • Fisher matrix round-trip verified (persist → restart → restore → metrics match)
    • Unit tests: 7 new tests (including round-trip and production call site verification)
    • No test regression
  • R1: HDC Pattern Fingerprinting

    • src/integrations/ruvector/hdc-fingerprint.ts created
    • TypeScript fallback implementation working
    • WASM integration (when package available)
    • Feature flag useHDCFingerprinting added
    • Unit tests: 8 new tests
    • Benchmark: < 1us WASM, < 100us TypeScript
  • R2: CUSUM Drift Detection

    • Prerequisite: coherence-gate.ts extracted into 4 modules (core, energy, vector, cohomology)
    • CusumDetector class in src/integrations/ruvector/cusum-detector.ts
    • Integrated into coherence-gate-energy.ts
    • 4-gate CUSUM state (retrieve/write/learn/act)
    • Drift event emission via EventBus
    • Unit tests: 6 new tests
    • False alarm rate < 0.1% on stationary data
  • R3: Delta Event Sourcing

    • src/integrations/ruvector/delta-tracker.ts created
    • SQLite pattern_deltas table migration
    • Rollback to any version works
    • Incremental sync method works
    • Feature flag useDeltaEventSourcing added
    • Unit tests: 10 new tests

Milestone 2: Graph Learning

  • R4: GraphMAE Self-Supervised Learning

    • GraphMAEEncoder class in graphmae-encoder.ts (separate file; gnn-wrapper.ts already 861 lines)
    • Mask-and-reconstruct pipeline with decoder head (encode → decode → SCE loss vs originals)
    • Reconstruction loss decreasing over epochs (SPSA optimizer, lossHistory exposed)
    • SCE loss matches paper: (1 - cos^γ) / γ with configurable gamma
    • 1K-node graph scale test passing
    • Consumer wired: coordinator-gnn.ts:generateGraphMAEEmbeddings()
    • Unit tests: 21 tests (masking, encoding, decoding, convergence, clustering, scale, flags)
  • R5: Modern Hopfield Networks

    • src/integrations/ruvector/hopfield-memory.ts created (331 lines)
    • L2-normalized patterns on store/recall for consistent attention weights
    • Exact recall verified at 1K patterns (cosine > 0.999) and 10K patterns (cosine > 0.95)
    • Feature flag useHopfieldMemory added (default false)
    • Consumer wired: pattern-store.ts store() + search() exact recall path
    • Dimension-aware singleton prevents mismatch bugs
    • Zero-magnitude patterns rejected
    • Unit tests: 28 tests (exact recall, capacity, energy, noisy, batch, benchmark, flags)
    • Benchmark: < 2ms recall at 1K patterns (CI-safe threshold)
  • R6: Cold-Tier GNN Training

    • ColdTierTrainer class in cold-tier-trainer.ts (separate file; gnn-wrapper.ts already 861 lines)
    • FileBackedGraph for real disk-backed larger-than-RAM graphs
    • Memory-capped training verified (peakMemoryNodes ≤ hotsetSize)
    • Equivalent loss to in-memory training (within 15% tolerance)
    • Consumer wired: coordinator-gnn.ts:trainWithColdTier()
    • Unbiased Fisher-Yates shuffle (rejection sampling)
    • Unit tests: 25 tests (in-memory, cold-tier, convergence, cache, FileBackedGraph, flags)
  • Shared Infrastructure

    • Xorshift128 PRNG extracted to src/shared/utils/xorshift128.ts (eliminates 3 duplicates)
    • Feature flag activation criteria documented for all 3 flags
    • Barrel exports in index.ts for all types, classes, and factory functions

Milestone 3: Scale and Optimization

  • R7: Meta-Learning Enhancements

    • DecayingBeta in domain-transfer.ts
    • PlateauDetector using CUSUM from R2 (CusumDetector on 'learn' gate)
    • ParetoFront for multi-objective transfer
    • CuriosityBonus for novel domain pairs
    • Feature flag useMetaLearningEnhancements gates both config and system flag
    • Unit tests: 46 tests (meta-learning.test.ts)
  • R8: PageRank Solver (renamed from Sublinear Solver — TS fallback is O(n*m), native is O(log n))

    • Pattern citation graph schema: PatternCitationGraph class with PATTERN_CITATIONS_SCHEMA
    • Co-occurrence recording: recordCoOccurrence(), recordDerivation(), buildGraph()
    • Graph populated during promotePattern() — records co-occurrence with same-domain long-term patterns
    • src/integrations/ruvector/solver-adapter.ts created (PageRankSolver class)
    • TypeScript power iteration fallback with dangling node handling
    • Integrated into pattern-promotion.ts via computeBlendedImportance()
    • Feature flag useSublinearSolver added
    • Unit tests: 34 tests (solver-adapter.test.ts) + 7 integration tests (milestone3-integration.test.ts)
    • Benchmark: O(log n) confirmed (requires native @ruvector/solver-node — TS fallback is O(n*m))
    • Spearman rho > 0.5 correlation with quality scores (needs real pattern data)
  • R9: Spectral Graph Sparsification (degree-based leverage heuristic, not true effective resistance)

    • src/integrations/ruvector/spectral-sparsifier.ts created
    • Eigenvalue computation via power iteration with deflation
    • Wired into mincut-wrapper.ts via maybeSparsify() (> 100 edges, behind flag)
    • Feature flag useSpectralSparsification added
    • Unit tests: 36 tests (spectral-sparsifier.test.ts) including connectivity and compression validation
    • Eigenvalue bounds formally verified (degree heuristic does not guarantee (1+eps) bounds)
  • R10: Reservoir Replay with Coherence Gating

    • ReservoirReplayBuffer in reservoir-replay.ts (standalone, generic)
    • Coherence-gated admission with CUSUM drift-aware threshold tightening
    • Wired into experience-replay.ts: admits on storeExperience(), samples in getGuidance()
    • Feature flag useReservoirReplay added
    • Unit tests: 32 tests (reservoir-replay.test.ts) + 3 integration tests (milestone3-integration.test.ts)
  • Shared Infrastructure

    • 4 feature flags with convenience functions, env var support
    • All types/classes exported via index.ts
    • CLI flag descriptions in ruvector-commands.ts
    • Integration test suite: milestone3-integration.test.ts (13 tests)

Milestone 4: Advanced Learning

  • R11: E-prop Online Learning

    • src/integrations/ruvector/eprop-learner.ts created (444 lines) — EpropNetwork with 12 bytes/synapse
    • src/integrations/rl-suite/algorithms/eprop.ts created (373 lines) — extends BaseRLAlgorithm
    • Registered as RL algorithm #10 (type 'eprop', category 'online-learning')
    • Exported from rl-suite/algorithms/index.ts and rl-suite/index.ts
    • Feature flag useEpropOnlineLearning added (default false)
    • Unit tests: 30 tests (eprop-learner.test.ts) — XOR convergence, memory budget, traces, benchmark
    • Benchmark: forward+update < 1ms for 100x50 network (TypeScript fallback)
  • R12: Granger Causality

    • src/integrations/ruvector/temporal-causality.ts created (342 lines) — GrangerAnalyzer with VAR + F-test
    • Full statistical math: lnGamma (Lanczos), regularized incomplete beta, F-distribution CDF
    • OLS regression via Gaussian elimination with partial pivoting
    • Wired into defect-intelligence/coordinator.ts — enriches predictDefects() behind flag
    • Feature flag useGrangerCausality added (default false)
    • Unit tests: 33 tests (temporal-causality.test.ts) — synthetic causal data, independence, bidirectional, scale
  • Shared Infrastructure

    • 2 feature flags with convenience functions, env var support
    • All types/classes exported via index.ts
    • CLI flag descriptions in ruvector-commands.ts

Milestone 5: Backlog

  • R13: Cognitive Routing

    • src/integrations/ruvector/cognitive-routing.ts created
    • Bandwidth reduction > 80%
    • Unit tests: 5 new tests
  • R14: Hyperbolic HNSW

    • src/integrations/ruvector/hyperbolic-hnsw.ts created
    • Hierarchical distance preservation verified
    • Unit tests: 5 new tests

Prerequisites (Before Milestone 1)

  • Coherence-gate extraction: Split 930-line coherence-gate.ts into 4 modules
    • coherence-gate-core.ts (~250 lines) — CoherenceGate class, types, factories
    • coherence-gate-energy.ts (~300 lines) — reflex/retrieval tiers, contradiction detection
    • coherence-gate-vector.ts (~120 lines) — FNV-1a, feature vectors, cosine similarity
    • coherence-gate-cohomology.ts (~80 lines) — WASM CohomologyEngine loader
    • coherence-gate.ts converted to barrel re-export (~20 lines)
    • All existing tests pass unchanged (import via barrel)
  • Performance baselines measured (see Performance Baselines section)
  • Pattern citation graph schema designed (prerequisite for R8)

Cross-Cutting

  • All feature flags documented in feature-flags.ts
  • All new exports added to index.ts
  • All new interfaces added to interfaces.ts
  • npm test passes (15,493+ tests)
  • npm run build succeeds
  • No file exceeds 500 lines
  • No secrets or credentials committed

Performance Baselines

Baseline measurements must be captured before implementation begins so improvement claims are verifiable. Use the existing benchmark harness in src/benchmarks/performance-benchmarks.ts and tests/performance/.

Measurement Points

Operation File:Line Current Algorithm Expected Complexity Instrumentation Status
Pattern cosine similarity shared/utils/vector-math.ts:5 Dot product + L2 norms O(dim) per pair Not instrumented — add timing
Coherence reflex tier coherence-gate.ts:428 Heuristic threshold checks O(1) Tracked (perf.now at line 277)
Coherence retrieval tier coherence-gate.ts:463 Sheaf Laplacian + contradictions O(n^2) contradiction Tier-level only — need per-component
Contradiction detection coherence-gate.ts:544 Pairwise string matching O(n^2) Not instrumented
Pattern importance qe-patterns.ts:322 Weighted formula O(1) per pattern Not instrumented (trivial)
MinCut (weighted degree) mincut-calculator.ts:43 Weighted degree heuristic O(V) Tracked (Date.now at line 44)
Find weak vertices mincut-calculator.ts:97 Variance + scoring O(V^2) worst Not instrumented
HNSW search hnsw-index.ts:77 Progressive backend O(log n) theoretical Benchmarked in tests/performance/

Existing Benchmarks to Run

# HNSW scale benchmarks (insert, search, compression, filter)
RUN_SCALE_BENCH=1 npx vitest bench tests/performance/ruvector-scale-benchmarks.bench.ts

# Migration benchmarks (throughput, compression, rollback, memory)
RUN_SCALE_BENCH=1 npx vitest bench tests/performance/ruvector-migration-benchmark.bench.ts

New Benchmarks to Add

Benchmark Baseline For Target File
Cosine similarity at 1K/10K/100K pairs R1 (HDC replaces cosine) tests/performance/pattern-similarity.bench.ts
Coherence per-component breakdown R2 (CUSUM augments) tests/performance/coherence-components.bench.ts
Pattern importance at 1K/10K/100K patterns R8 (solver replaces formula) tests/performance/pattern-importance.bench.ts
Graph ops at 10K/50K/100K edges R9 (sparsification) tests/performance/graph-operations.bench.ts

Note: HNSW performance numbers in code comments (0.1ms at 1K, 0.13ms at 10K, 0.17ms at 100K) in hnsw-index.ts are estimated, not measured. Run the existing scale benchmarks to get real numbers.


Migration Path

Existing 150K+ Patterns

Capability Migration Strategy
R1: HDC Fingerprinting Lazy fingerprinting on next access, or batch during dream cycle. No schema migration needed — fingerprints stored alongside existing vectors.
R3: Delta Event Sourcing On activation, each existing pattern gets a "v0" genesis snapshot delta. History begins from that point. No retroactive history synthesis.
R8: Pattern Citation Graph Bootstrap from existing usage data: co-occurrence in assessments, pattern derivation metadata. Graph starts sparse and densifies over time.
EWC++: Fisher Matrices Fisher starts from zero on first backgroundConsolidate(). No migration — just begins accumulating.

Schema Additions

Table Added By Migration
pattern_deltas R3 New table, no existing table changes
pattern_citations R8 New table, no existing table changes

All schema additions are additive — no existing tables are modified or dropped.


CI/CD Pipeline Changes

New Requirements

  1. Optional native dependencies: @ruvector/solver-node, @ruvector/hdc-wasm, @ruvector/hopfield-wasm, @ruvector/eprop-wasm must be declared as optionalDependencies in package.json. Dynamic imports with try/catch ensure CI without Rust toolchains still passes.

  2. Platform matrix: NAPI binaries need x64 and ARM64 builds. Add to GitHub Actions:

    strategy:
      matrix:
        os: [ubuntu-latest]
        arch: [x64, arm64]
    
  3. Benchmark regression detection: Add tests/performance/ to CI with threshold-based alerts:

    • Run benchmarks on PR branches
    • Compare against baseline (stored in tests/performance/baselines.json)
    • Alert if any benchmark regresses > 20%
  4. Test time budget: ~106 new tests across 14 improvements. Estimate ~30-60s additional CI time. Ensure CI timeout accommodates this.

  5. WASM binary caching: WASM blobs (R1, R3, R5, R11) should be cached in CI to avoid recompilation on every run.


Relationship to Issue #355

These 14 capabilities are distinct from the 26 remaining items tracked in Issue #355. Key clarifications:

  • No conflicts: None of the 14 items contradict or block #355 items
  • Shared infrastructure: R2 (CUSUM) and R3 (Delta) improve infrastructure that #355 items also use (coherence gate, persistence layer)
  • Parallel work: Milestones 1-2 can proceed in parallel with #355 work
  • Ordering recommendation: Complete coherence-gate extraction (prerequisite) before any #355 items that touch coherence-gate.ts