proffesor-for-testing__agentic-qe/v2/docs/MULTI-MODEL-ROUTER-GUIDE.md

Multi-Model Router Setup & Usage Guide

Overview

The agentic-flow Multi-Model Router provides intelligent model selection with 70-85% cost savings while maintaining quality. It's fully integrated in your DevPod workspace with all environment variables configured globally.

✅ Current Setup Status

Your workspace has agentic-flow v1.9.4 installed globally with:

✅ agentic-flow CLI: /usr/local/share/nvm/versions/node/v22.19.0/bin/agentic-flow
✅ Version: v1.9.4
✅ Environment Variables: Configured in ~/.bashrc
   - AGENTIC_FLOW_TRANSPORT=quic
   - QUIC_PORT=4433
   - AGENTIC_FLOW_OPTIMIZE=true
   - AGENTIC_FLOW_PRIORITY=quality
   - ANTHROPIC_API_KEY (set)
   - OPENROUTER_API_KEY (set)
   - OPENAI_API_KEY (set)

🚀 Quick Start

1. Basic Model Optimization

# Auto-select best model for task (balanced priority)
agentic-flow --agent coder --task "Create REST API" --optimize

# Optimize for cost (cheapest models)
agentic-flow --agent coder --task "Simple function" --optimize --priority cost

# Optimize for quality (flagship models)
agentic-flow --agent reviewer --task "Security audit" --optimize --priority quality

# Optimize for speed (fastest models)
agentic-flow --agent researcher --task "Quick research" --optimize --priority speed

# Optimize for privacy (local ONNX models)
agentic-flow --agent coder --task "Offline task" --optimize --priority privacy

2. Budget Constraints

# Set maximum cost per task ($0.001 = 1/10th of a cent)
agentic-flow --agent coder --task "Build feature" --optimize --max-cost 0.001

# Cost-optimized with budget cap
agentic-flow --agent qe-test-generator \
  --task "Generate tests" \
  --optimize \
  --priority cost \
  --max-cost 0.002

3. QE Agent Integration

# Test generation with cost optimization
agentic-flow --agent qe-test-generator \
  --task "Generate unit tests for UserService" \
  --optimize \
  --priority cost

# Coverage analysis with balanced optimization
agentic-flow --agent qe-coverage-analyzer \
  --task "Analyze test coverage gaps" \
  --optimize \
  --priority balanced

# Security scanning with quality optimization
agentic-flow --agent qe-security-scanner \
  --task "Security audit of authentication" \
  --optimize \
  --priority quality

# Performance testing with speed optimization
agentic-flow --agent qe-performance-tester \
  --task "Load test API endpoints" \
  --optimize \
  --priority speed

🎯 Optimization Priorities

Priority	Use Case	Models	Cost Savings
quality	Security audits, architecture reviews	Claude Sonnet 4.5, GPT-4o	0% (baseline)
balanced	General development, code reviews	DeepSeek R1, Gemini 2.5 Flash	60-70%
cost	Simple tasks, test generation	DeepSeek Chat V3, Llama 3.1 8B	85-98%
speed	Quick research, rapid iterations	Gemini 2.5 Flash	40-60%
privacy	Offline work, sensitive data	ONNX Phi-4 (local)	100% (no API)

📊 Model Capabilities

Flagship Models (Quality Priority)

Claude Sonnet 4.5
├─ Quality: 95/100
├─ Speed: 85/100
├─ Cost: 20/100
└─ Best for: Security, architecture, complex algorithms

GPT-4o
├─ Quality: 92/100
├─ Speed: 80/100
├─ Cost: 25/100
└─ Best for: Multi-modal tasks, long-form content

Mid-Tier Models (Balanced Priority)

DeepSeek R1
├─ Quality: 85/100
├─ Speed: 70/100
├─ Cost: 90/100
└─ Best for: Code generation, reasoning tasks

Gemini 2.5 Flash
├─ Quality: 80/100
├─ Speed: 95/100
├─ Cost: 85/100
└─ Best for: Fast iterations, research

Budget Models (Cost Priority)

DeepSeek Chat V3
├─ Quality: 80/100
├─ Speed: 75/100
├─ Cost: 98/100
└─ Best for: Simple tasks, test generation

Llama 3.1 8B
├─ Quality: 75/100
├─ Speed: 85/100
├─ Cost: 95/100
└─ Best for: Quick tasks, documentation

🔧 MCP Tool Integration

Using via Claude Code

// MCP tool for model optimization
mcp__agentic-flow__agentic_flow_optimize_model({
  agent: "qe-test-generator",
  task: "Generate unit tests for authentication",
  priority: "cost",
  max_cost: 0.001
})

// Execute agent with optimization
mcp__agentic-flow__agentic_flow_agent({
  agent: "coder",
  task: "Create REST API",
  provider: "openrouter", // or "anthropic", "gemini", "onnx"
  model: "deepseek/deepseek-r1:free", // optional, auto-selected if not specified
  temperature: 0.7,
  maxTokens: 4096,
  outputFormat: "markdown"
})

Available MCP Tools (7 total)

agentic-flow mcp list

Output:
• agentic_flow_agent           - Execute agent with specific task
• agentic_flow_optimize_model  - Get model recommendations
• agentic_flow_list_agents     - List all 150+ agents
• agentic_flow_create_agent    - Create custom agent
• agentic_flow_list_all_agents - List package + local agents
• agentic_flow_agent_info      - Get agent details
• agentic_flow_check_conflicts - Check agent conflicts

💰 Cost Savings Examples

Example 1: Test Generation

Before Optimization (always Claude Sonnet 4.5):

• 100 test generation tasks
• Cost: $0.015/task × 100 = $1.50

After Optimization (cost priority):

• 100 test generation tasks → DeepSeek Chat V3
• Cost: $0.0008/task × 100 = $0.08
• Savings: $1.42 (94.7%)

Example 2: Mixed Workload

Before Optimization:

• 50 simple tasks (Claude): $0.75
• 30 complex tasks (Claude): $0.45
• 20 security audits (Claude): $0.30
• Total: $1.50

After Optimization:

• 50 simple → DeepSeek Chat ($0.04)
• 30 complex → DeepSeek R1 ($0.24)
• 20 security → Claude Sonnet 4.5 ($0.30)
• Total: $0.58
• Savings: $0.92 (61%)

🔄 Provider Fallback (v1.9.4)

Automatic failover between providers with circuit breaker:

# Automatic fallback chain
Primary: OpenRouter (DeepSeek R1)
   ↓ (if fails)
Secondary: Gemini (Gemini 2.5 Flash)
   ↓ (if fails)
Tertiary: Claude (Sonnet 4.5)
   ↓ (if fails)
Final: ONNX (local Phi-4)

Features

• ✅ Circuit Breaker: Auto-recovery after timeout
• ✅ Health Monitoring: Success rate, latency tracking
• ✅ Cost Optimization: 70% savings with smart routing
• ✅ Crash Recovery: Checkpointing for long tasks
• ✅ Budget Controls: Hard limits on spending

Configuration

# Provider fallback is automatic
# Configure via environment variables or config files

export AGENTIC_FLOW_FALLBACK_ENABLED=true
export AGENTIC_FLOW_PRIMARY_PROVIDER=openrouter
export AGENTIC_FLOW_FALLBACK_CHAIN=gemini,anthropic,onnx

🚀 QUIC Transport (Ultra-Low Latency)

Your workspace has QUIC transport enabled (50-70% faster than TCP):

# Environment variable already set
echo $AGENTIC_FLOW_TRANSPORT
# Output: quic

echo $QUIC_PORT
# Output: 4433

# Start QUIC server manually (if needed)
agentic-flow quic --port 4433

# Test QUIC with WASM bindings
npm run test:quic:wasm

QUIC Benefits

• ✅ 0-RTT: Instant reconnection without handshake
• ✅ Stream Multiplexing: 100+ concurrent messages
• ✅ Built-in TLS 1.3: Encrypted by default
• ✅ Connection Migration: Survives network changes
• ✅ Reduced Latency: Perfect for agent coordination

📈 Monitoring & Debugging

View Optimization Output

# Enable verbose mode to see optimization details
agentic-flow --agent coder \
  --task "Test task" \
  --optimize \
  --priority balanced \
  --verbose

# Output includes:
# ════════════════════════════════════════════════════════════
# 🎯 Optimized Model Selection
# ════════════════════════════════════════════════════════════
# Model: DeepSeek R1
# Provider: openrouter
# Tier: mid-tier
#
# Scores:
#   Quality:  85/100
#   Speed:    70/100
#   Cost:     90/100
#   Overall:  80/100
#
# Cost: $0.16/1M input, $0.64/1M output
#
# Reasoning:
#   Balanced selection (overall: 80/100). Optimized for coder
#   agent tasks. Estimated cost: $0.0008 per task. Tier: mid-tier
# ════════════════════════════════════════════════════════════

Check Provider Status

# Check provider configuration
agentic-flow --help | grep -A 5 "Provider Selection"

# List available models
agentic-flow --help | grep -A 10 "OPENROUTER MODELS"

# Check MCP server status
agentic-flow mcp status

Cost Tracking

# View cost estimates in optimization output
agentic-flow --agent coder --task "Test" --optimize --verbose | grep -i cost

# Example output:
# Cost: $0.16/1M input, $0.64/1M output
# Estimated cost: $0.0008 per task

🎓 Best Practices

1. Choose Right Priority for Task

✅ GOOD:
agentic-flow --agent qe-security-scanner \
  --task "Security audit" \
  --optimize --priority quality  # Security needs quality!

❌ BAD:
agentic-flow --agent qe-security-scanner \
  --task "Security audit" \
  --optimize --priority cost  # Security is not the place to cut costs!

2. Use Budget Caps for Experiments

✅ GOOD:
agentic-flow --agent coder \
  --task "Experimental feature" \
  --optimize --max-cost 0.005  # Safe budget limit

❌ BAD:
agentic-flow --agent coder \
  --task "Experimental feature"  # No cost limit, could use expensive model

3. Leverage Agent-Specific Optimization

# The router knows qe-test-generator can use cheaper models
agentic-flow --agent qe-test-generator \
  --task "Generate tests" \
  --optimize  # Auto-selects cost-effective model

# The router knows qe-security-scanner needs quality
agentic-flow --agent qe-security-scanner \
  --task "Security scan" \
  --optimize  # Auto-selects high-quality model

🔧 Advanced Configuration

Environment Variables

# Already set in ~/.bashrc
export AGENTIC_FLOW_OPTIMIZE=true          # Enable optimization by default
export AGENTIC_FLOW_PRIORITY=quality       # Default priority
export AGENTIC_FLOW_TRANSPORT=quic         # Use QUIC transport
export QUIC_PORT=4433                      # QUIC server port

# Optional additional settings
export AGENTIC_FLOW_MAX_COST=0.01         # Global max cost per task
export AGENTIC_FLOW_FALLBACK_ENABLED=true # Enable provider fallback
export AGENTIC_FLOW_VERBOSE=true          # Always show optimization details

Config File (Alternative to Env Vars)

Create ~/.agentic-flow/config.json:

{
  "optimization": {
    "enabled": true,
    "defaultPriority": "balanced",
    "maxCostPerTask": 0.01
  },
  "providers": {
    "primary": "openrouter",
    "fallbackChain": ["gemini", "anthropic", "onnx"],
    "circuitBreakerTimeout": 30000
  },
  "transport": {
    "type": "quic",
    "port": 4433
  }
}

🧪 Testing the Setup

Run the comprehensive integration tests:

# Run multi-model router tests
npm run test:integration -- tests/integration/multi-model-router.test.ts

# Or run all integration tests
npm run test:integration

📚 Documentation References

• Main Docs: docs/agentic-flow/benchmarks/MODEL_CAPABILITIES.md
• Provider Fallback: docs/PROVIDER-FALLBACK-GUIDE.md
• QUIC Transport: docs/agentic-flow/QUIC-TRANSPORT.md
• MCP Tools: Run agentic-flow mcp list

🆘 Troubleshooting

Issue: Optimization not working

# Check if environment variables are set
env | grep AGENTIC_FLOW

# Verify API keys are configured
env | grep -E "ANTHROPIC_API_KEY|OPENROUTER_API_KEY"

# Test with verbose mode
agentic-flow --agent coder --task "Test" --optimize --verbose

Issue: Provider fallback not triggering

# Enable fallback explicitly
export AGENTIC_FLOW_FALLBACK_ENABLED=true

# Check provider status
agentic-flow mcp status

# Test with verbose mode to see provider selection
agentic-flow --agent coder --task "Test" --optimize --verbose 2>&1 | grep "Provider"

Issue: QUIC transport errors

# Verify QUIC is configured
echo $AGENTIC_FLOW_TRANSPORT  # Should output: quic
echo $QUIC_PORT               # Should output: 4433

# Test QUIC server
agentic-flow quic --port 4433

# Fallback to TCP if needed
export AGENTIC_FLOW_TRANSPORT=tcp

🎯 Summary

Your DevPod workspace is fully configured for Multi-Model Router:

✅ agentic-flow v1.9.4 installed globally ✅ Environment variables configured in ~/.bashrc ✅ QUIC transport enabled (50-70% faster) ✅ API keys configured (Anthropic, OpenRouter, OpenAI) ✅ Optimization enabled by default ✅ Provider fallback available ✅ MCP tools ready for Claude Code integration

Start using it now:

# Simple test
agentic-flow --agent coder --task "Hello world" --optimize

# QE agent with cost optimization
agentic-flow --agent qe-test-generator \
  --task "Generate tests" \
  --optimize --priority cost

# Security audit with quality optimization
agentic-flow --agent qe-security-scanner \
  --task "Security scan" \
  --optimize --priority quality

Expected savings: 70-85% cost reduction with maintained quality!