proffesor-for-testing__agentic-qe/docs/agentics/M0.5.3-Quick-Start-Guide.md

HybridRouter RuVector Integration - Quick Start Guide

5-Minute Setup

Step 1: Start RuVector Service

# Start RuVector Docker container
docker-compose -f docker/docker-compose.ruvector.yml up -d

# Verify it's running
curl http://localhost:8080/health
# Expected: {"status":"healthy","gnnStatus":"active","loraStatus":"active"}

Step 2: Initialize HybridRouter

import { HybridRouter } from './src/providers/HybridRouter';

const router = new HybridRouter({
  ruvector: {
    enabled: true,
    cacheThreshold: 0.85  // 85% confidence for cache hits
  }
});

await router.initialize();

Step 3: Make Requests

// First request - will be a cache miss
const response1 = await router.complete({
  model: 'default',
  messages: [
    { role: 'user', content: 'How do I test async functions in Jest?' }
  ]
});
console.log('Response:', response1.content[0].text);
// Latency: ~2000ms (LLM call)

// Similar request - should be a cache hit!
const response2 = await router.complete({
  model: 'default',
  messages: [
    { role: 'user', content: 'How to test async code with Jest?' }
  ]
});
console.log('Response:', response2.content[0].text);
// Latency: ~5ms (cache hit!)

Step 4: Monitor Performance

// Check cache hit rate
const hitRate = router.getCacheHitRate();
console.log(`Cache hit rate: ${(hitRate * 100).toFixed(1)}%`);

// Get detailed metrics
const metrics = await router.getRuVectorMetrics();
console.log('Patterns stored:', metrics?.patternCount);
console.log('LoRA updates:', metrics?.loraUpdates);

// Get cost savings
const report = router.getCostSavingsReport();
console.log(`Saved: $${report.savings.toFixed(2)} (${report.savingsPercentage.toFixed(1)}%)`);

Common Use Cases

Use Case 1: Cost Optimization

Goal: Minimize cloud API costs by maximizing cache hits.

const router = new HybridRouter({
  ruvector: {
    enabled: true,
    cacheThreshold: 0.80,  // Lower threshold = more cache hits
    learningEnabled: true
  },
  defaultStrategy: RoutingStrategy.COST_OPTIMIZED
});

Expected Results:

• 70-80% cache hit rate after warmup
• 70-80% cost reduction
• Sub-5ms latency for cache hits

Use Case 2: Privacy-First

Goal: Never send sensitive data to cloud APIs.

const router = new HybridRouter({
  ruvector: {
    enabled: true,
    cacheThreshold: 0.85
  },
  defaultStrategy: RoutingStrategy.PRIVACY_FIRST,
  privacyKeywords: [
    'password', 'secret', 'token', 'api_key',
    'credential', 'private', 'confidential'
  ]
});

Behavior:

• Privacy-sensitive queries skip cache
• Routed directly to local LLM
• Never touches cloud API

Use Case 3: Quality-First

Goal: Prioritize accuracy over cost/speed.

const router = new HybridRouter({
  ruvector: {
    enabled: true,
    cacheThreshold: 0.90,  // Higher threshold = fewer, but more accurate cache hits
    skipCacheForComplexTasks: true
  },
  defaultStrategy: RoutingStrategy.QUALITY_OPTIMIZED
});

Behavior:

• Only cache hits with very high confidence (90%+)
• Complex tasks always go to cloud LLM
• Prioritizes accuracy over cost

Use Case 4: Latency Optimization

Goal: Minimize response time for common queries.

const router = new HybridRouter({
  ruvector: {
    enabled: true,
    cacheThreshold: 0.85,
    learningEnabled: true
  },
  defaultStrategy: RoutingStrategy.LATENCY_OPTIMIZED
});

Expected Results:

• Sub-5ms for cache hits
• Learns common patterns quickly
• Routes to fastest available provider for misses

Configuration Reference

RuVectorCacheConfig

Option	Type	Default	Description
enabled	boolean	required	Enable RuVector cache layer
baseUrl	string	'http://localhost:8080'	RuVector service URL
cacheThreshold	number	0.85	Minimum confidence for cache hits (0-1)
learningEnabled	boolean	true	Auto-learn from LLM responses
loraRank	number	8	LoRA adapter rank (4-16 recommended)
ewcEnabled	boolean	true	Prevent catastrophic forgetting
skipCacheForComplexTasks	boolean	false	Skip cache for complex/very complex tasks

Threshold Tuning Guide

Threshold	Cache Hit Rate	Accuracy	Use Case
0.70	Very High (80-90%)	Lower	Cost optimization
0.80	High (60-80%)	Good	Balanced
0.85	Moderate (40-60%)	Very Good	Recommended
0.90	Low (20-40%)	Excellent	Quality-first
0.95	Very Low (5-20%)	Perfect	Critical accuracy

Monitoring & Debugging

Enable Debug Logging

const router = new HybridRouter({
  debug: true,
  ruvector: {
    enabled: true,
    debug: true
  }
});

Check Health Status

const health = await router.healthCheck();

console.log('Overall healthy:', health.healthy);
console.log('RuVector status:', health.metadata?.providers?.find(p => p.provider === 'ruvector-cache'));
console.log('Cache hit rate:', health.metadata?.cacheHitRate);

Force Learning Consolidation

// Manually trigger LoRA parameter consolidation
const result = await router.forceRuVectorLearn();

if (result.success) {
  console.log(`✅ Updated ${result.updatedParameters} parameters in ${result.duration}ms`);
} else {
  console.error('❌ Learning failed:', result.error);
}

View Routing Statistics

const stats = router.getRoutingStats();

console.log('Total decisions:', stats.totalDecisions);
console.log('Local routes:', stats.localDecisions);
console.log('Cloud routes:', stats.cloudDecisions);
console.log('Cache hits:', stats.cacheHits);
console.log('Cache misses:', stats.cacheMisses);
console.log('Cache hit rate:', `${(stats.cacheHitRate * 100).toFixed(1)}%`);
console.log('Avg local latency:', `${stats.averageLocalLatency}ms`);
console.log('Avg cloud latency:', `${stats.averageCloudLatency}ms`);
console.log('Success rate:', `${stats.successRate.toFixed(1)}%`);

Troubleshooting

Problem: No Cache Hits

Symptoms:

• getCacheHitRate() returns 0
• All requests showing as cache misses

Solutions:

1. Check RuVector service is running:

   curl http://localhost:8080/health
   

2. Verify threshold isn't too high:

   // Try lowering threshold
   ruvector: {
     cacheThreshold: 0.75  // Was 0.85
   }
   

3. Check if learning is disabled:

   ruvector: {
     learningEnabled: true  // Must be true to build cache
   }
   

4. Wait for warmup period:

   • Cache needs 10-20 queries to build patterns
   • Similar queries required for hits

Problem: RuVector Initialization Failed

Symptoms:

• Warning: "Failed to initialize RuVector cache layer"
• Router falls back to LLM-only mode

Solutions:

1. Check Docker container:

   docker ps | grep ruvector
   docker logs <container-id>
   

2. Verify port is accessible:

   nc -zv localhost 8080
   

3. Check firewall settings:

   sudo ufw status
   

4. Fallback gracefully:

   // Router continues working even if RuVector fails
   // Check status:
   const metrics = await router.getRuVectorMetrics();
   if (!metrics) {
     console.warn('RuVector not available, using LLM-only mode');
   }
   

Problem: Cache Hit But Wrong Response

Symptoms:

• Getting cached responses that don't match query
• Low accuracy despite cache hits

Solutions:

1. Increase confidence threshold:

   ruvector: {
     cacheThreshold: 0.90  // Was 0.85
   }
   

2. Skip cache for complex queries:

   ruvector: {
     skipCacheForComplexTasks: true
   }
   

3. Check embedding quality:

   // Make sure local provider is generating good embeddings
   const embeddingTest = await router.embed({
     text: 'test query',
     model: 'default'
   });
   console.log('Embedding dimension:', embeddingTest.embedding.length);
   // Should be 768
   

Problem: High Latency on Cache Hits

Symptoms:

• Cache hits taking > 50ms
• Expected sub-5ms performance

Solutions:

1. Check RuVector Docker resources:

   docker stats
   # Look for CPU/memory constraints
   

2. Reduce vector count (if too large):

   # Check metrics
   curl http://localhost:8080/v1/metrics
   

3. Optimize Docker configuration:

   # docker-compose.ruvector.yml
   services:
     ruvector:
       deploy:
         resources:
           limits:
             cpus: '2.0'
             memory: 4G
   

Best Practices

1. Warmup Period

// Build cache with common queries
const commonQueries = [
  'How to test async functions?',
  'How to mock API calls?',
  'How to handle errors in tests?',
  'How to test React components?',
  'How to use jest.fn()?'
];

for (const query of commonQueries) {
  await router.complete({
    model: 'default',
    messages: [{ role: 'user', content: query }]
  });
}

console.log('Cache warmed up with', commonQueries.length, 'patterns');

2. Privacy-Sensitive Data Handling

// Automatically detect and route private data
const router = new HybridRouter({
  privacyKeywords: [
    'password', 'secret', 'token', 'api_key',
    'credential', 'ssn', 'credit_card'
  ],
  ruvector: {
    enabled: true
  }
});

// This will skip cache and go to local LLM
await router.complete({
  messages: [{
    role: 'user',
    content: 'My password is secret123, is it secure?'
  }]
});
// Route: Cache skipped → Local LLM (privacy-first)

3. Cost Monitoring

// Set up periodic cost reporting
setInterval(async () => {
  const report = router.getCostSavingsReport();
  const metrics = await router.getRuVectorMetrics();

  console.log('=== Cost Report ===');
  console.log(`Total requests: ${report.totalRequests}`);
  console.log(`Cache hits: ${report.cacheHits} (${metrics?.cacheHitRate}%)`);
  console.log(`Actual cost: $${report.totalCost.toFixed(2)}`);
  console.log(`Cloud cost: $${report.estimatedCloudCost.toFixed(2)}`);
  console.log(`Savings: $${report.savings.toFixed(2)} (${report.savingsPercentage.toFixed(1)}%)`);
  console.log('==================');
}, 3600000); // Every hour

4. Graceful Degradation

// Always handle RuVector being unavailable
const router = new HybridRouter({
  ruvector: {
    enabled: true
  }
});

try {
  await router.initialize();

  const metrics = await router.getRuVectorMetrics();
  if (metrics?.healthy) {
    console.log('✅ RuVector cache active');
  } else {
    console.warn('⚠️ RuVector unhealthy, using LLM-only mode');
  }
} catch (error) {
  console.error('❌ Initialization failed:', error);
  // Router still works with LLM providers
}

Performance Benchmarks

Typical Performance (After Warmup)

Metric                     Value
────────────────────────────────────────
Cache Hit Latency          2-5ms
Cache Miss → Local         1000-3000ms
Cache Miss → Cloud         800-2000ms
Cache Hit Rate (day 1)     10-20%
Cache Hit Rate (day 7)     40-60%
Cache Hit Rate (day 30)    60-80%
Learning Latency           Async (non-blocking)
LoRA Update Time           50-200ms
Memory Usage               256-512MB

Cost Analysis Example

Scenario: 1000 requests/day, $0.01/request cloud cost

Timeframe	Hit Rate	Daily Cost	Monthly Cost	Annual Savings
Day 1	15%	$8.50	$255	-
Week 1	45%	$5.50	$165	$1,080/year
Month 1	65%	$3.50	$105	$1,800/year
Month 3	75%	$2.50	$75	$2,700/year

Next Steps

1. Production Deployment

   • Set up monitoring and alerting
   • Configure backup/restore for vector store
   • Implement rate limiting and quotas

2. Optimization

   • Tune threshold based on accuracy metrics
   • Analyze common query patterns
   • Adjust LoRA rank for your use case

3. Integration

   • Connect to your application
   • Set up logging and analytics
   • Implement A/B testing for threshold tuning

---

Quick Start Status: Ready Documentation: Complete Support: See M0.5.3-HybridRouter-RuVector-Integration-Complete.md