proffesor-for-testing__agentic-qe/docs/planning/ruvllm-quick-reference.md

ruvllm Integration - Quick Reference Card

Version: 1.0.0 | Status: Planning | Target: v2.2.0

---

🎯 Project Goals (1-Minute Overview)

Goal	Description	Impact
G1: LLM Abstraction	Decouple agents from specific providers	Pluggable backends
G2: Local Inference	Enable ruvllm for offline capability	80% requests local
G3: Hybrid Routing	Smart local/cloud selection	Cost + performance
G4: Privacy Mode	Guarantee no external calls	GDPR/HIPAA compliance
G5: CI/CD Optimization	3x faster pipelines	Developer experience

---

📅 Timeline (8 Weeks)

Week 1-2: Foundation → LLM abstraction, BaseAgent refactor
Week 3-4: Core       → ruvllm provider, hybrid routing
Week 5-6: Optimize   → Performance, privacy mode
Week 7:   CI/CD      → Docker, GitHub Actions
Week 8:   QA         → Validation, release prep

---

🔑 Key Deliverables

Phase 1 (Weeks 1-2)

• ✅ ILLMProvider interface
• ✅ All 18 agents use abstraction
• ✅ Can switch providers via config

Phase 2 (Weeks 3-4)

• ✅ RuvllmProvider implementation
• ✅ Qwen2.5-Coder-7B model support
• ✅ Hybrid routing: 80% local, 20% cloud

Phase 3 (Weeks 5-6)

• ✅ Model warm pool (<5s cold start)
• ✅ Batch inference (20 tests/min)
• ✅ Privacy mode (strict/balanced/off)

Phase 4 (Week 7)

• ✅ Docker image (<4GB)
• ✅ CI pipeline 3x faster

Phase 5 (Week 8)

• ✅ A/B testing (local vs cloud)
• ✅ Release v2.2.0

---

💰 Cost Impact

Break-Even Point

• Low usage (<1M tokens/day): Cloud cheaper
• Medium usage (1-5M tokens/day): Hybrid competitive
• High usage (>5M tokens/day): 63% savings ($2,902 → $1,080/month)

ROI Calculator

Monthly savings = (Cloud cost - Hybrid cost)
Cloud cost = tokens/month × mix × price
Hybrid cost = $500 (infra) + 20% × Cloud cost

Example (2.7B tokens/month):
Cloud: $2,902
Hybrid: $1,080
Savings: $1,822 (63%)

---

🛠️ Technical Architecture

Agents (18)
    ↓
ILLMProvider interface
    ↓
HybridModelRouter
    ├─→ RuvllmProvider (local, 80%)
    └─→ AnthropicProvider (cloud, 20%)

---

📊 Success Metrics

Metric	Current	Target
Inference latency	500ms	<2s
Throughput	10/min	20+/min
Cold start	N/A	<5s
CI speed	120s	<40s
Cost (high-vol)	$2,902	$1,080
Test coverage	100%	90%+
Opt-in rate	N/A	30%

---

🔒 Privacy Mode

Configuration

privacy:
  mode: strict         # strict | balanced | off
  allowedProviders: [ruvllm]
  auditLog: true
  blockExternalCalls: true

Modes

• Strict: Local-only, no external calls
• Balanced: Prefer local, fallback to cloud
• Off: Cloud-first (current behavior)

---

🚀 Quick Start (Post-Launch)

Install

npm install -g agentic-qe@2.2.0

Configure Local Inference

# Download model (one-time)
aqe model download Qwen2.5-Coder-7B-Instruct-Q5_K_M

# Enable hybrid mode
aqe config set inference.mode hybrid

# Generate tests with local inference
aqe generate --provider ruvllm --files src/**/*.ts

Enable Privacy Mode

aqe config set privacy.mode strict
aqe generate --files src/  # Guaranteed local-only

---

🐛 Troubleshooting

Common Issues

Issue: Model download fails

# Solution: Specify custom cache dir
export RUVLLM_CACHE=/path/to/cache
aqe model download <model-name>

Issue: GPU not detected

# Solution: Verify CUDA/Metal
nvidia-smi  # Linux
system_profiler SPDisplaysDataType  # macOS

# Force CPU mode
aqe config set ruvllm.gpuLayers 0

Issue: Local inference too slow

# Solution: Use smaller model
aqe model download Phi-3-Mini-4K-Instruct  # 2.3GB vs 5.6GB
aqe config set ruvllm.model Phi-3-Mini-4K-Instruct

Issue: Out of memory (OOM)

# Solution: Reduce context length
aqe config set ruvllm.contextLength 4096  # Default: 8192

---

📚 Documentation Links

User Guides

• Getting Started with Local Inference
• Privacy Mode Guide
• CI/CD Integration

Technical Docs

• LLM Provider Architecture
• Hybrid Routing Design
• Migration Guide

Planning Docs

• Executive Summary
• Full GOAP Plan
• Visual Roadmap

---

🧪 Testing Commands

Unit Tests

npm run test:unit -- src/core/llm

Integration Tests

npm run test:integration -- tests/integration/llm

Benchmark

npm run test:benchmark -- benchmarks/local-vs-cloud.ts

A/B Test

aqe test compare --local ruvllm --cloud anthropic --files src/

---

🔥 CI/CD Example

GitHub Actions

name: Tests with Local Inference

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Pull ruvllm image
        run: docker pull ghcr.io/agentic-qe/ruvllm:2.2.0

      - name: Generate tests
        run: |
          aqe generate \
            --provider ruvllm \
            --model Qwen2.5-Coder-7B-Instruct-Q5_K_M \
            --files src/**/*.ts

      - name: Run tests
        run: npm test

---

🎛️ Configuration Reference

Environment Variables

# Model configuration
export RUVLLM_MODEL=Qwen2.5-Coder-7B-Instruct-Q5_K_M
export RUVLLM_CACHE=/path/to/models
export RUVLLM_GPU_LAYERS=-1  # -1 = auto-detect

# Privacy mode
export AQE_PRIVACY_MODE=strict
export AQE_AUDIT_LOG=true

# Performance tuning
export RUVLLM_CONTEXT_LENGTH=8192
export RUVLLM_BATCH_SIZE=4
export RUVLLM_THREADS=8

Config File (.aqe/config.yml)

inference:
  mode: hybrid  # local | cloud | hybrid
  preferLocal: true
  fallbackToCloud: true

ruvllm:
  model: Qwen2.5-Coder-7B-Instruct-Q5_K_M
  gpuLayers: -1  # Auto-detect
  contextLength: 8192
  temperature: 0.7
  maxTokens: 2048

privacy:
  mode: strict  # strict | balanced | off
  allowedProviders: [ruvllm]
  auditLog: true
  blockExternalCalls: true

routing:
  rules:
    - name: privacy-code
      condition: privacy === 'strict'
      action: route-to-local
      priority: 100

    - name: high-complexity
      condition: complexity > 0.8
      action: route-to-cloud
      priority: 80

    - name: cost-optimization
      condition: taskType === 'test-generation'
      action: prefer-local
      priority: 50

---

🚦 Status Codes

CLI Exit Codes

• 0: Success
• 1: General error
• 2: Configuration error
• 3: Model not found
• 4: Inference error (fallback to cloud if enabled)
• 5: Privacy violation (strict mode)

---

📞 Support

Issues: https://github.com/proffesor-for-testing/agentic-qe/issues Discussions: https://github.com/proffesor-for-testing/agentic-qe/discussions Slack: #aqe-ruvllm-integration

---

🏆 Key Features (Marketing)

For Developers

• ✅ Offline Development: Work on flights, remote locations
• ✅ Faster Feedback: 3x faster CI pipelines
• ✅ No API Keys: Simplified local development

For Enterprises

• ✅ Compliance: GDPR, HIPAA, SOC2 via privacy mode
• ✅ Cost Savings: 63% reduction for high-volume users
• ✅ Air-Gapped: Deploy in isolated environments

For Open Source

• ✅ Sustainability: No ongoing cloud costs
• ✅ Community: Enable broader participation
• ✅ Privacy: Code stays on contributor machines

---

🔮 Future Enhancements (v2.3.0+)

1. Custom Model Fine-Tuning

   • Train on your codebase for better accuracy
   • Enterprise tier feature

2. Distributed Inference

   • Cluster of GPU servers for scale
   • Kubernetes operator

3. Model Marketplace

   • Community-trained models
   • Domain-specific models (React, Python, etc.)

4. Edge Deployment

   • Run on developer laptops
   • Mobile device support (iOS/Android)

5. Federated Learning

   • Learn from distributed deployments
   • Privacy-preserving model updates

---

📋 Checklist for New Users

Getting Started

• Install agentic-qe@2.2.0
• Download model: aqe model download Qwen2.5-Coder-7B-Instruct-Q5_K_M
• Configure hybrid mode: aqe config set inference.mode hybrid
• Test generation: aqe generate --provider ruvllm --files src/
• Validate: npm test

For Enterprise (Privacy Mode)

• Review privacy policy requirements (GDPR, HIPAA, etc.)
• Enable strict mode: aqe config set privacy.mode strict
• Configure audit logging: aqe config set privacy.auditLog true
• Test with sample code
• Generate compliance report: aqe privacy report
• Review with legal/compliance team

For CI/CD

• Build Docker image: docker build -t aqe-local:latest .
• Update GitHub Actions workflow
• Test in CI: Run a pilot pipeline
• Benchmark: Compare speed (baseline vs local)
• Rollout: Enable for all pipelines

---

🎉 Celebration Milestones

• Week 2: LLM abstraction complete 🎊
• Week 4: Local inference working 🚀
• Week 6: Performance targets met ⚡
• Week 8: Release v2.2.0 shipped 🎉
• Month 3: 30% adoption reached 🏆

---

Print this card and keep it handy!

Last Updated: 2025-12-04 Version: 1.0.0