proffesor-for-testing__agentic-qe/docs/guides/improvement-loop.md

Continuous Improvement Loop - Configuration Guide

Overview

The Continuous Improvement Loop is a Phase 2 (Milestone 2.2) feature that implements automated learning and optimization for the Agentic QE Fleet. It runs periodic cycles that analyze performance, detect patterns, run A/B tests, and apply proven strategies.

Architecture

┌─────────────────────────────────────────────────────────┐
│           ImprovementWorker (Background)                │
│  - Schedules periodic cycles                            │
│  - Handles retries and error recovery                   │
│  - Monitors execution status                            │
└───────────────────┬─────────────────────────────────────┘
                    │
                    ▼
┌─────────────────────────────────────────────────────────┐
│              ImprovementLoop (Core)                     │
│  ┌────────────────────────────────────────────────┐    │
│  │ 1. Analyze Performance (PerformanceTracker)    │    │
│  │ 2. Identify Failure Patterns (LearningEngine)  │    │
│  │ 3. Discover Optimizations                      │    │
│  │ 4. Run A/B Tests                                │    │
│  │ 5. Apply Best Strategies (opt-in)              │    │
│  └────────────────────────────────────────────────┘    │
└───────────┬──────────────────────┬──────────────────────┘
            │                      │
            ▼                      ▼
┌──────────────────────┐  ┌──────────────────────┐
│  PerformanceTracker  │  │   LearningEngine     │
│  - Metrics tracking  │  │  - Q-learning        │
│  - Improvement rate  │  │  - Pattern detection │
│  - Trend analysis    │  │  - Strategy learning │
└──────────────────────┘  └──────────────────────┘

Key Features

1. Performance Analysis

• Tracks metrics over time (success rate, execution time, error rate)
• Calculates improvement rates vs baseline
• Target: 20% improvement over 30 days

2. Failure Pattern Detection

• Automatically identifies recurring failure patterns
• Frequency threshold: 5+ occurrences
• Confidence threshold: 0.7+
• Suggests mitigations for common failures

3. A/B Testing Framework

• Compare multiple strategies side-by-side
• Automatic winner determination
• Weighted scoring: 70% success rate, 30% execution time
• Support for multiple concurrent tests

4. Auto-Apply Best Strategies ⚠️

• OPT-IN feature (disabled by default for safety)
• Only applies strategies with:
  • Confidence >0.9
  • Success rate >0.8
  • Maximum 3 strategies per cycle
• Requires explicit configuration

5. Background Worker

• Runs cycles at configurable intervals (default: 1 hour)
• Automatic retry logic (3 attempts)
• Status monitoring and statistics

Configuration

Basic Setup

import {
  ImprovementLoop,
  ImprovementWorker,
  loadImprovementConfig
} from '@agentic-qe/learning';

// Initialize components
const config = loadImprovementConfig();
const improvementLoop = new ImprovementLoop(
  agentId,
  memoryStore,
  learningEngine,
  performanceTracker
);
await improvementLoop.initialize();

// Start background worker
const worker = new ImprovementWorker(improvementLoop, {
  intervalMs: config.cycleIntervalMs,
  enabled: config.worker.enabled
});
await worker.start();

Configuration Profiles

1. Default (Production) - Conservative & Safe

{
  enabled: true,
  cycleIntervalMs: 3600000, // 1 hour
  autoApplyEnabled: false, // ⚠️ DISABLED by default
  autoApplyMinConfidence: 0.9,
  autoApplyMinSuccessRate: 0.8,
  failurePatterns: {
    minFrequency: 5,
    minConfidence: 0.7
  },
  abTesting: {
    defaultSampleSize: 100
  }
}

2. Aggressive - For Mature Systems

{
  cycleIntervalMs: 1800000, // 30 minutes
  autoApplyEnabled: true, // ⚠️ Use with caution
  autoApplyMinConfidence: 0.95,
  autoApplyMinSuccessRate: 0.9,
  failurePatterns: {
    minFrequency: 3,
    minConfidence: 0.6
  },
  abTesting: {
    defaultSampleSize: 50 // faster tests
  }
}

3. Development - Fast Iterations

{
  cycleIntervalMs: 300000, // 5 minutes
  autoApplyEnabled: false, // Still disabled for safety
  abTesting: {
    defaultSampleSize: 10 // small samples
  },
  worker: {
    maxRetries: 2,
    retryDelayMs: 10000
  }
}

Enabling Auto-Apply (Opt-In)

Step 1: Update Configuration

// config/improvement-loop.config.ts
export const CUSTOM_CONFIG = {
  ...DEFAULT_IMPROVEMENT_CONFIG,
  autoApplyEnabled: true,
  autoApplyMinConfidence: 0.9, // High threshold
  autoApplyMinSuccessRate: 0.8
};

Step 2: Enable at Runtime

// Enable for specific agent
await improvementLoop.setAutoApply(true);

// Or via configuration
const worker = new ImprovementWorker(improvementLoop, {
  ...config.worker
});

Step 3: Monitor and Validate

// Check status
const status = worker.getStatus();
console.log('Cycles completed:', status.cyclesCompleted);
console.log('Success rate:', worker.getStatistics().successRate);

// Review applied strategies
const strategies = improvementLoop.getStrategies();
strategies.forEach(s => {
  console.log(`${s.name}: ${s.usageCount} uses, ${s.successRate} success`);
});

A/B Testing

Creating Tests

// Create A/B test
const testId = await improvementLoop.createABTest(
  'Parallelization Strategy Test',
  [
    { name: 'high-parallel', config: { parallelization: 0.9 } },
    { name: 'low-parallel', config: { parallelization: 0.3 } }
  ],
  100 // sample size
);

// Record results as tests execute
await improvementLoop.recordTestResult(
  testId,
  'high-parallel',
  true, // success
  1200 // execution time (ms)
);

// Test automatically completes when sample size is reached
// Winner is determined by weighted score: 70% success + 30% speed

Monitoring Tests

// Get active tests
const activeTests = improvementLoop.getActiveTests();
activeTests.forEach(test => {
  console.log(`Test: ${test.name}`);
  console.log(`Status: ${test.status}`);
  console.log(`Progress: ${getTotalSamples(test)}/${test.sampleSize}`);

  test.results.forEach(result => {
    console.log(`  ${result.strategy}: ${result.successRate} success, ${result.averageTime}ms avg`);
  });
});

Failure Pattern Analysis

Automatic Detection

// Patterns detected automatically during improvement cycles
const patterns = learningEngine.getFailurePatterns();

patterns.forEach(pattern => {
  console.log(`Pattern: ${pattern.pattern}`);
  console.log(`Frequency: ${pattern.frequency}`);
  console.log(`Confidence: ${pattern.confidence}`);
  console.log(`Mitigation: ${pattern.mitigation}`);
});

Pattern Types and Mitigations

Pattern Type	Auto-Suggested Mitigation
timeout	Increase timeout threshold or implement progress checkpointing
memory	Implement memory pooling and garbage collection optimization
validation	Add input validation and sanitization before processing
network	Implement retry logic with exponential backoff
parsing	Add robust error handling for malformed input
permission	Implement proper permission checking before operations

Monitoring and Metrics

Worker Status

const status = worker.getStatus();
console.log('Running:', status.isRunning);
console.log('Last cycle:', status.lastCycleAt);
console.log('Next cycle:', status.nextCycleAt);
console.log('Cycles completed:', status.cyclesCompleted);
console.log('Cycles failed:', status.cyclesFailed);

Performance Statistics

const stats = worker.getStatistics();
console.log('Total cycles:', stats.cyclesCompleted + stats.cyclesFailed);
console.log('Success rate:', (stats.successRate * 100).toFixed(1), '%');
console.log('Uptime:', stats.uptime, 'ms');

Cycle Results

// Run manual cycle and inspect results
const result = await improvementLoop.runImprovementCycle();

console.log('Improvement rate:', result.improvement.improvementRate.toFixed(2), '%');
console.log('Failure patterns analyzed:', result.failurePatternsAnalyzed);
console.log('Opportunities found:', result.opportunitiesFound);
console.log('Active tests:', result.activeTests);
console.log('Strategies applied:', result.strategiesApplied);

Safety Considerations

1. Auto-Apply is Opt-In

• Disabled by default for safety
• Requires explicit configuration
• High confidence thresholds (0.9+)
• Limited to 3 strategies per cycle

2. Failure Recovery

• Worker has retry logic (3 attempts)
• Failed cycles don't crash the system
• Errors are logged for investigation

3. Gradual Rollout

1. Start with monitoring only (auto-apply OFF)
2. Review patterns and metrics for 30 days
3. Enable auto-apply for low-risk strategies
4. Gradually increase confidence based on results

4. Testing in Development

# Use development config for testing
NODE_ENV=development npm start

# Monitor logs
tail -f logs/improvement-loop.log

# Run manual cycle for immediate testing
await worker.runNow();

Integration with Existing Systems

With PerformanceTracker

// Performance tracker feeds metrics to improvement loop
await performanceTracker.recordSnapshot({
  metrics: {
    tasksCompleted: 10,
    successRate: 0.85,
    averageExecutionTime: 2000,
    errorRate: 0.15,
    userSatisfaction: 0.8,
    resourceEfficiency: 0.75
  },
  trends: []
});

// Improvement loop uses these metrics
const result = await improvementLoop.runImprovementCycle();
console.log('Improvement:', result.improvement.improvementRate, '%');

With LearningEngine

// Learning engine trains patterns
await learningEngine.learnFromExecution(
  task,
  result,
  feedback
);

// Improvement loop leverages learned patterns
const opportunities = await improvementLoop.discoverOptimizations();

Troubleshooting

Issue: Cycles Not Running

// Check worker status
const status = worker.getStatus();
if (!status.isRunning) {
  console.log('Worker is not running');
  await worker.start();
}

// Check configuration
if (!config.enabled) {
  console.log('Improvement loop disabled in config');
}

Issue: No Strategies Applied

// Check auto-apply setting
const autoApply = await improvementLoop.isAutoApplyEnabled();
console.log('Auto-apply enabled:', autoApply);

// Check pattern confidence
const patterns = learningEngine.getPatterns();
patterns.forEach(p => {
  console.log(`${p.pattern}: confidence=${p.confidence}, success=${p.successRate}`);
});
// Need confidence >0.9 AND success >0.8

Issue: High Failure Rate

// Review failure patterns
const failures = learningEngine.getFailurePatterns();
failures.forEach(f => {
  console.log(`${f.pattern}: ${f.frequency} occurrences`);
  console.log(`Mitigation: ${f.mitigation}`);
});

// Adjust thresholds if needed
config.failurePatterns.minFrequency = 3; // Lower threshold

Best Practices

1. Start Conservative: Use default config with auto-apply OFF
2. Monitor First: Run for 30+ days before enabling auto-apply
3. Review Patterns: Examine learned patterns and failure analysis
4. Test Thoroughly: Use A/B tests to validate strategies
5. Gradual Rollout: Enable auto-apply for one agent at a time
6. Track Metrics: Monitor improvement rates and success rates
7. Document Changes: Log all configuration changes
8. Regular Reviews: Review applied strategies weekly

Success Criteria (Phase 2 Goals)

• ✅ Improvement loop operational
• ✅ A/B tests running and completing
• ✅ Failure patterns detected (frequency >5, confidence >0.7)
• ✅ Mitigations suggested automatically
• ✅ Auto-apply available (opt-in, confidence >0.9)
• ✅ Target: 20% performance improvement over 30 days

Support

For issues or questions:

• Review logs in logs/improvement-loop.log
• Check configuration in config/improvement-loop.config.ts
• Run diagnostics: npm run test:improvement-loop
• File issues with detailed logs and configuration