mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
204 lines
5.8 KiB
Markdown
204 lines
5.8 KiB
Markdown
# Multi-Model Router - Quick Reference Card
|
||
|
||
## 🚀 One-Liner Examples
|
||
|
||
```bash
|
||
# Cost optimization (94% savings)
|
||
agentic-flow --agent qe-test-generator --task "Generate tests" --optimize --priority cost
|
||
|
||
# Quality optimization (best results)
|
||
agentic-flow --agent qe-security-scanner --task "Security audit" --optimize --priority quality
|
||
|
||
# Speed optimization (fastest)
|
||
agentic-flow --agent researcher --task "Research topic" --optimize --priority speed
|
||
|
||
# Privacy (local, no API)
|
||
agentic-flow --agent coder --task "Sensitive code" --optimize --priority privacy
|
||
|
||
# Budget cap ($0.001 max)
|
||
agentic-flow --agent coder --task "Experiment" --optimize --max-cost 0.001
|
||
```
|
||
|
||
## 📊 Priority Comparison
|
||
|
||
| Priority | Model | Cost/Task | Savings | Use Case |
|
||
|----------|-------|-----------|---------|----------|
|
||
| **quality** | Claude Sonnet 4.5 | $0.015 | 0% | Security, architecture |
|
||
| **balanced** | DeepSeek R1 | $0.0008 | 95% | General dev |
|
||
| **cost** | DeepSeek Chat V3.1 | $0.0000 | 100% | Simple tasks |
|
||
| **speed** | Gemini 2.5 Flash | $0.0004 | 97% | Quick iterations |
|
||
| **privacy** | ONNX Phi-4 | $0.0000 | 100% | Offline work |
|
||
|
||
## 🎯 QE Agent Commands
|
||
|
||
```bash
|
||
# Test generation (use cost-effective models)
|
||
agentic-flow --agent qe-test-generator --task "[task]" --optimize --priority cost
|
||
|
||
# Coverage analysis (balanced)
|
||
agentic-flow --agent qe-coverage-analyzer --task "[task]" --optimize
|
||
|
||
# Security scanning (quality)
|
||
agentic-flow --agent qe-security-scanner --task "[task]" --optimize --priority quality
|
||
|
||
# Performance testing (speed)
|
||
agentic-flow --agent qe-performance-tester --task "[task]" --optimize --priority speed
|
||
|
||
# Flaky test detection (balanced)
|
||
agentic-flow --agent qe-flaky-test-hunter --task "[task]" --optimize
|
||
|
||
# API contract validation (balanced)
|
||
agentic-flow --agent qe-api-contract-validator --task "[task]" --optimize
|
||
|
||
# Visual testing (quality)
|
||
agentic-flow --agent qe-visual-tester --task "[task]" --optimize --priority quality
|
||
|
||
# Chaos engineering (quality)
|
||
agentic-flow --agent qe-chaos-engineer --task "[task]" --optimize --priority quality
|
||
```
|
||
|
||
## 🔧 MCP Tool Usage (Claude Code)
|
||
|
||
```typescript
|
||
// Get model recommendation
|
||
mcp__agentic-flow__agentic_flow_optimize_model({
|
||
agent: "qe-test-generator",
|
||
task: "Generate tests",
|
||
priority: "cost",
|
||
max_cost: 0.001
|
||
})
|
||
|
||
// Execute agent
|
||
mcp__agentic-flow__agentic_flow_agent({
|
||
agent: "qe-test-generator",
|
||
task: "Generate comprehensive tests",
|
||
provider: "openrouter",
|
||
temperature: 0.7,
|
||
maxTokens: 4096
|
||
})
|
||
```
|
||
|
||
## 🛠️ Configuration
|
||
|
||
### Environment Variables (already set in ~/.bashrc)
|
||
```bash
|
||
AGENTIC_FLOW_OPTIMIZE=true
|
||
AGENTIC_FLOW_PRIORITY=quality
|
||
AGENTIC_FLOW_TRANSPORT=quic
|
||
QUIC_PORT=4433
|
||
ANTHROPIC_API_KEY=[set]
|
||
OPENROUTER_API_KEY=[set]
|
||
OPENAI_API_KEY=[set]
|
||
```
|
||
|
||
### Optional Settings
|
||
```bash
|
||
export AGENTIC_FLOW_MAX_COST=0.01 # Global max cost
|
||
export AGENTIC_FLOW_FALLBACK_ENABLED=true # Provider fallback
|
||
export AGENTIC_FLOW_VERBOSE=true # Always show details
|
||
```
|
||
|
||
## 💡 Cost Savings Examples
|
||
|
||
### Example 1: 100 Test Generation Tasks
|
||
```
|
||
Before: $0.015 × 100 = $1.50 (Claude only)
|
||
After: $0.0008 × 100 = $0.08 (optimized)
|
||
Savings: $1.42 (94.7%)
|
||
```
|
||
|
||
### Example 2: Mixed Workload
|
||
```
|
||
Before: $1.50 (all Claude Sonnet 4.5)
|
||
After: $0.58 (optimized per task type)
|
||
Savings: $0.92 (61%)
|
||
```
|
||
|
||
### Example 3: Daily QE Operations
|
||
```
|
||
10 test generations (cost) → $0.008
|
||
5 coverage analyses (balanced) → $0.004
|
||
3 security scans (quality) → $0.045
|
||
2 performance tests (speed) → $0.001
|
||
─────────────────────────────────────
|
||
Total per day: $0.058
|
||
Monthly (22 days): $1.28
|
||
|
||
Without optimization: $33.00
|
||
Savings: $31.72/month (96%)
|
||
```
|
||
|
||
## 🔄 Provider Fallback Chain
|
||
|
||
```
|
||
1. OpenRouter (DeepSeek R1) → 85% savings
|
||
↓ (if fails)
|
||
2. Gemini (Gemini 2.5 Flash) → 70% savings
|
||
↓ (if fails)
|
||
3. Claude (Sonnet 4.5) → Baseline quality
|
||
↓ (if fails)
|
||
4. ONNX (local Phi-4) → 100% offline
|
||
```
|
||
|
||
## 📈 Model Quality Scores
|
||
|
||
```
|
||
Model Quality Speed Cost Overall
|
||
─────────────────────────────────────────────────────
|
||
Claude Sonnet 4.5 95 85 20 58
|
||
GPT-4o 92 80 25 60
|
||
DeepSeek R1 85 70 90 80
|
||
Gemini 2.5 Flash 80 95 85 87
|
||
DeepSeek Chat V3.1 82 90 100 95
|
||
Llama 3.1 8B 75 85 95 85
|
||
ONNX Phi-4 70 75 100 82
|
||
```
|
||
|
||
## 🆘 Troubleshooting
|
||
|
||
### Issue: Free model unavailable
|
||
```bash
|
||
# Solution: Use Anthropic as fallback
|
||
agentic-flow --agent [agent] --task "[task]" --provider anthropic
|
||
```
|
||
|
||
### Issue: Want to see optimization details
|
||
```bash
|
||
# Solution: Add --verbose flag
|
||
agentic-flow --agent [agent] --task "[task]" --optimize --verbose
|
||
```
|
||
|
||
### Issue: Cost too high
|
||
```bash
|
||
# Solution: Use cost priority or set budget
|
||
agentic-flow --agent [agent] --task "[task]" --optimize --priority cost --max-cost 0.001
|
||
```
|
||
|
||
## 🎓 Best Practices
|
||
|
||
✅ **DO**: Use `--optimize` for all tasks
|
||
✅ **DO**: Set `--priority` based on task criticality
|
||
✅ **DO**: Use `--max-cost` for experiments
|
||
✅ **DO**: Use `--verbose` to understand selections
|
||
|
||
❌ **DON'T**: Use `--priority cost` for security/critical tasks
|
||
❌ **DON'T**: Skip optimization (wastes money)
|
||
❌ **DON'T**: Use `--priority quality` for simple tasks
|
||
|
||
## 📚 Documentation
|
||
|
||
- Full Guide: `docs/MULTI-MODEL-ROUTER-GUIDE.md`
|
||
- Verification: `docs/MULTI-MODEL-ROUTER-VERIFICATION.md`
|
||
- Tests: `tests/integration/multi-model-router.test.ts`
|
||
|
||
## ✅ Status
|
||
|
||
✅ Installed: agentic-flow v1.9.4
|
||
✅ Configured: Environment variables set
|
||
✅ Tested: All features operational
|
||
✅ Ready: Use immediately
|
||
|
||
---
|
||
|
||
**Quick Test**: `agentic-flow --agent coder --task "Hello world" --optimize --verbose`
|