proffesor-for-testing__agentic-qe/docs/implementation/specs/SPEC-044-A-domain-rl-mapping.md

SPEC-044-A: Domain RL Algorithm Mapping

Field	Value
Specification ID	SPEC-044-A
Parent ADR	ADR-044
Version	1.0
Status	Implemented
Last Updated	2026-01-13
Author	Claude Code

---

Overview

This specification maps each of the 6 V3 QE domains to their RL algorithms and @ruvector wrappers, with implementation patterns and key methods.

---

Domain-Algorithm Mapping

Domain	RL Algorithm	@ruvector Wrapper	Key Benefit
Requirements-Validation	PPO	QESONA	Optimized BDD scenarios (3-5 optimal)
Code-Intelligence	-	QEGNNEmbeddingIndex + QESONA	150x faster code similarity search
Security-Compliance	DQN	QEFlashAttention	Smart test prioritization + 2.49x faster clustering
Contract-Testing	SARSA	QESONA	Optimal contract validation order
Visual-Accessibility	A2C	QEFlashAttention	Prioritized visual tests + fast image comparison
Chaos-Resilience	PolicyGradient	QESONA	Smart chaos experiment selection

---

Integration Pattern Template

// 1. Import RL algorithms and wrappers
import { [ALGORITHM] } from '../../integrations/rl-suite/algorithms/[algorithm].js';
import { [WRAPPER] } from '../../integrations/ruvector/wrappers.js';

// 2. Add to config
export interface CoordinatorConfig {
  // ... existing config
  enable[ALGORITHM]: boolean;
  enable[WRAPPER]: boolean;
}

// 3. Initialize in coordinator
private [algorithm]?: [ALGORITHM];
private [wrapper]?: [WRAPPER];

private async initializeRLIntegrations(): Promise<void> {
  if (this.config.enable[ALGORITHM]) {
    this.[algorithm] = new [ALGORITHM](config);
    await this.[algorithm].initialize();
  }
  if (this.config.enable[WRAPPER]) {
    this.[wrapper] = create[WRAPPER](config);
  }
}

// 4. Use in domain methods
private async useRLFeature(): Promise<void> {
  if (this.[algorithm] && this.rlInitialized) {
    const prediction = await this.[algorithm].predict(state);
    // Use prediction to drive behavior
  }
}

// 5. Train with feedback
private async trainWithFeedback(): Promise<void> {
  if (this.[algorithm] && this.rlInitialized) {
    const experience: RLExperience = { state, action, reward, nextState, done };
    await this.[algorithm].train([experience]);
  }
}

---

Domain Details

1. Requirements-Validation (PPO + QESONA)

Key Methods:

• optimizeScenarioGeneration(): Uses PPO to determine optimal scenario count
• optimizeScenarioOrdering(): Prioritizes scenarios for maximum coverage
• trainPPOWithScenarioFeedback(): Trains model with generation outcomes
• adaptRequirementPattern(): Uses SONA to find similar past requirements
• storeRequirementPattern(): Stores successful patterns for future use

RL State Features:

• Requirement length, priority, type, dependencies, complexity
• Optimizes scenario count (3-5 optimal range)

2. Code-Intelligence (QEGNNEmbeddingIndex + QESONA)

Key Methods:

• indexCodeEmbeddings(): Generates and indexes code embeddings
• generateCodeEmbedding(): Creates 384-dim embeddings from code features
• searchCodeWithGNN(): Fast similarity search using HNSW
• enhanceImpactAnalysisWithGNN(): Finds semantically similar files
• adaptSearchPattern(): Uses SONA for search strategy adaptation

Performance: 150x-12,500x faster similarity search via HNSW

3. Security-Compliance (DQN + QEFlashAttention)

Key Methods:

• prioritizeSecurityTests(): Uses DQN to rank security tests
• trainDQNWithAuditFeedback(): Trains with audit results
• enhanceVulnerabilityAnalysis(): Groups similar vulnerabilities
• createVulnerabilityEmbedding(): Creates embeddings from vulnerability features

Performance: 2.49x-7.47x faster vulnerability clustering

4. Contract-Testing (SARSA + QESONA)

Key Methods:

• prioritizeContracts(): Uses SARSA to rank contracts for validation
• Integration in verifyAllConsumers() workflow

RL Features: Complexity, change frequency, failure history, dependency count

5. Visual-Accessibility (A2C + QEFlashAttention)

Key Methods:

• prioritizeVisualTests(): Uses A2C to determine optimal test order
• Integration in runVisualTests() workflow

Features: Historical failure rate, urgency, available resources

6. Chaos-Resilience (PolicyGradient + QESONA)

Key Methods:

• selectChaosStrategy(): Uses PolicyGradient to select optimal experiments
• runStrategicChaosSuite(): New workflow that uses RL-selected experiments

Features: System state, blast radius tolerance, recent incidents

---

Integration Test Coverage

Domain	Tests	Coverage
Contract-Testing (SARSA)	6	Interface compliance, workflow integration
Visual-Accessibility (A2C)	6	Interface compliance, workflow integration
Chaos-Resilience (PolicyGradient)	6	Interface compliance, workflow integration
Cross-Domain Verification	1	ADR-044 compliance verification
Total	19	All passing

---

Changelog

Version	Date	Author	Changes
1.0	2026-01-13	Claude Code	Initial specification

---

References

• Parent ADR
• Integration Tests: /v3/tests/integration/domain-rl-integration.test.ts