proffesor-for-testing__agentic-qe/docs/implementation-plans/phase3-visual-tools-implementation.md

# Phase 3: Visual Testing Domain Tools - Implementation Summary

**Date**: 2025-11-08
**Status**: Implemented (Pending Integration)
**Priority**: 3.1 (Visual Testing Domain)

---

## Overview

Implemented 2 new visual testing domain tools as specified in Phase 3 checklist (Priority 3.1):

1. **compare-screenshots.ts** - AI-powered screenshot comparison
2. **validate-accessibility.ts** - WCAG compliance validation

---

## Implementation Details

### Directory Structure Created

```
/workspaces/agentic-qe-cf/src/mcp/tools/qe/visual/
├── compare-screenshots.ts         (NEW - 600+ lines)
├── validate-accessibility.ts      (NEW - 850+ lines)
├── detect-regression.ts           (MOVED from handlers/prediction)
└── index.ts                       (NEW - Clean exports)
```

### Test Files Created

```
/workspaces/agentic-qe-cf/tests/unit/mcp/tools/qe/visual/
├── compare-screenshots.test.ts    (10 test cases)
└── validate-accessibility.test.ts (16 test cases)
```

---

## Tool 1: compare-screenshots.ts

### Features Implemented

✅ **AI-Powered Comparison**
- Semantic visual understanding (AI-powered diff)
- Pixel-perfect comparison (traditional pixel-diff)
- Structural similarity scoring (SSIM)
- Threshold-based detection

✅ **Visual Regression Scoring**
- Pixel difference percentage (0-1)
- Structural similarity score (0-1)
- AI visual regression score (0-1)
- Confidence scoring for detected differences

✅ **Difference Detection**
- Layout shifts
- Color changes
- Content changes
- Missing/new elements
- Severity classification (low, medium, high)

✅ **Performance Metrics**
- Comparison time tracking
- AI inference time tracking (when AI enabled)
- Method identification (pixel-diff vs AI)

✅ **Recommendations**
- Actionable suggestions based on diff status
- AI-specific recommendations
- Baseline update suggestions

### API Signature

```typescript
export async function compareScreenshotsAI(
  params: CompareScreenshotsParams
): Promise<QEToolResponse<ScreenshotComparison>>

interface CompareScreenshotsParams {
  baseline: string;              // Path to baseline screenshot
  current: string;               // Path to current screenshot
  threshold: number;             // Diff threshold (0-1)
  useAI: boolean;                // Enable AI-powered comparison
  options?: {
    ignoreAntialiasing?: boolean;
    ignoreColors?: boolean;
    ignoreRegions?: Region[];
    generateDiffImage?: boolean;
  };
}
```

### Example Usage

```typescript
const result = await compareScreenshotsAI({
  baseline: './screenshots/baseline.png',
  current: './screenshots/current.png',
  threshold: 0.05,  // 5% tolerance
  useAI: true,      // Use AI-powered semantic comparison
  options: {
    ignoreAntialiasing: true,
    generateDiffImage: true
  }
});

if (result.success && result.data.status !== 'identical') {
  console.log(`Visual regression score: ${result.data.visualRegressionScore}`);
  console.log(`Method: ${result.data.method}`); // ai-visual-diff
  console.log(`Differences: ${result.data.differences.length}`);
}
```

### Test Coverage

10 comprehensive test cases:
- ✅ Successful identical screenshot comparison
- ✅ AI-powered comparison mode
- ✅ Visual difference detection above threshold
- ✅ Diff image generation
- ✅ Parameter validation (required fields)
- ✅ Threshold validation (0-1 range)
- ✅ Status determination (identical/minor-diff/major-diff/different)
- ✅ Recommendation generation
- ✅ Performance metrics reporting
- ✅ Pixel-diff fallback mode

---

## Tool 2: validate-accessibility.ts

### Features Implemented

✅ **WCAG Compliance Validation**
- Support for all levels: A, AA, AAA
- Comprehensive criterion validation
- Compliance score calculation (0-100)
- Status determination (compliant/partially-compliant/non-compliant)

✅ **Violation Detection & Classification**
- Severity levels: critical, serious, moderate, minor
- WCAG criterion references (with W3C links)
- Element-level violation details
- Suggested fixes for each violation
- Confidence scoring

✅ **Color Contrast Analysis**
- WCAG AA minimum ratio (4.5:1 for normal text)
- WCAG AAA enhanced ratio (7.0:1)
- Foreground/background color tracking
- Element size category classification
- Worst ratio reporting

✅ **Keyboard Navigation Testing**
- Logical tab order validation
- Interactive element reachability
- Focus indicator visibility
- Keyboard trap detection
- Skip link presence checking

✅ **Screen Reader Compatibility**
- ARIA label presence checking
- Alt text coverage percentage
- Semantic HTML usage scoring
- Form label validation
- Heading structure analysis

✅ **Screenshot Capture**
- Main page screenshots
- Annotated screenshots with violation highlights
- Configurable viewport sizes

✅ **Actionable Recommendations**
- Priority-based recommendations (critical/high/medium/low)
- POUR categories (Perceivable, Operable, Understandable, Robust)
- Detailed action items
- Effort estimation (hours)
- Confidence scoring

### API Signature

```typescript
export async function validateAccessibilityWCAG(
  params: ValidateAccessibilityParams
): Promise<QEToolResponse<AccessibilityReport>>

interface ValidateAccessibilityParams {
  url: string;                   // URL to validate
  level: 'A' | 'AA' | 'AAA';    // WCAG compliance level
  includeScreenshots: boolean;   // Capture screenshots
  options?: {
    criteria?: string[];         // Specific WCAG criteria
    viewport?: { width: number; height: number };
    colorContrastAnalysis?: boolean;
    keyboardNavigationTest?: boolean;
    screenReaderCheck?: boolean;
  };
}
```

### Example Usage

```typescript
const result = await validateAccessibilityWCAG({
  url: 'https://example.com',
  level: 'AA',  // WCAG 2.1 Level AA
  includeScreenshots: true,
  options: {
    colorContrastAnalysis: true,
    keyboardNavigationTest: true,
    screenReaderCheck: true
  }
});

if (result.success) {
  console.log(`Compliance score: ${result.data.complianceScore}%`);
  console.log(`Status: ${result.data.status}`);
  console.log(`Critical violations: ${result.data.summary.critical}`);
  console.log(`Total violations: ${result.data.summary.totalViolations}`);

  // Review color contrast issues
  if (result.data.colorContrast?.violations.length > 0) {
    console.log(`Color contrast violations: ${result.data.colorContrast.violations.length}`);
  }

  // Review keyboard navigation issues
  if (result.data.keyboardNavigation?.keyboardTraps.length > 0) {
    console.log(`Keyboard traps found: ${result.data.keyboardNavigation.keyboardTraps.length}`);
  }
}
```

### Test Coverage

16 comprehensive test cases:
- ✅ Successful WCAG AA validation
- ✅ All WCAG levels (A, AA, AAA)
- ✅ Violation detection
- ✅ Compliance score calculation
- ✅ Violation severity categorization
- ✅ Screenshot generation
- ✅ Color contrast analysis
- ✅ Keyboard navigation testing
- ✅ Screen reader compatibility checking
- ✅ Recommendation generation
- ✅ URL parameter validation
- ✅ WCAG level validation
- ✅ Performance metrics reporting
- ✅ Compliance status determination
- ✅ WCAG criterion references
- ✅ Actionable recommendations with effort estimation

---

## Shared Type System

Both tools use the shared type system defined in `/src/mcp/tools/qe/shared/types.ts`:

```typescript
export interface QEToolResponse<T> {
  success: boolean;
  data?: T;
  error?: QEError;
  metadata: ResponseMetadata;
}

export interface QEError {
  code: string;
  message: string;
  details?: Record<string, unknown>;
  stack?: string;
}

export interface ResponseMetadata {
  requestId: string;
  timestamp: string;
  executionTime: number;
  agent?: string;
  version?: string;
}
```

This ensures:
- ✅ Consistent error handling across all QE tools
- ✅ Strict TypeScript typing (no `any` types)
- ✅ Standardized response format
- ✅ Performance metrics tracking

---

## Clean Export Structure

Created `/src/mcp/tools/qe/visual/index.ts` with clean exports:

```typescript
// Screenshot Comparison
export {
  compareScreenshotsAI,
  type CompareScreenshotsParams,
  type ScreenshotComparison,
  type VisualDifference
} from './compare-screenshots.js';

// Accessibility Validation
export {
  validateAccessibilityWCAG,
  type ValidateAccessibilityParams,
  type AccessibilityReport,
  type AccessibilityViolation,
  type ColorContrastResults,
  type KeyboardNavigationResults,
  type ScreenReaderResults,
  type AccessibilityRecommendation,
  type WCAGLevel
} from './validate-accessibility.js';
```

---

## Integration with qe-visual-tester Agent

The tools are designed to integrate with the `qe-visual-tester` agent:

### Agent Definition Location
`.claude/agents/qe-visual-tester.md`

### Workflow Integration

```typescript
// 1. Screenshot Comparison Workflow
const comparisonResult = await compareScreenshotsAI({
  baseline: './baselines/homepage.png',
  current: './current/homepage.png',
  threshold: 0.05,
  useAI: true
});

// 2. Accessibility Validation Workflow
const accessibilityResult = await validateAccessibilityWCAG({
  url: 'https://app.example.com',
  level: 'AA',
  includeScreenshots: true,
  options: {
    colorContrastAnalysis: true,
    keyboardNavigationTest: true,
    screenReaderCheck: true
  }
});

// 3. Combined Visual Testing Workflow
const visualTestResults = {
  screenshots: comparisonResult.data,
  accessibility: accessibilityResult.data,
  timestamp: new Date().toISOString()
};
```

---

## Next Steps (Remaining Tasks)

### 1. MCP Tool Registration ⏳

Register the new tools in MCP registry:

**File**: `/workspaces/agentic-qe-cf/src/mcp/server.ts`

```typescript
// Add to MCP tool registry
server.setRequestHandler(ListToolsRequestSchema, async () => {
  return {
    tools: [
      // ... existing tools ...
      {
        name: 'compare-screenshots-ai',
        description: 'AI-powered screenshot comparison with threshold-based visual regression detection',
        inputSchema: {
          type: 'object',
          properties: {
            baseline: { type: 'string' },
            current: { type: 'string' },
            threshold: { type: 'number' },
            useAI: { type: 'boolean' }
          },
          required: ['baseline', 'current', 'threshold', 'useAI']
        }
      },
      {
        name: 'validate-accessibility-wcag',
        description: 'WCAG compliance validation (A, AA, AAA) with color contrast, keyboard, and screen reader checks',
        inputSchema: {
          type: 'object',
          properties: {
            url: { type: 'string' },
            level: { type: 'string', enum: ['A', 'AA', 'AAA'] },
            includeScreenshots: { type: 'boolean' }
          },
          required: ['url', 'level', 'includeScreenshots']
        }
      }
    ]
  };
});
```

### 2. Fix detect-regression.ts Imports ⏳

The moved file `detect-regression.ts` needs import path updates:

```typescript
// Change these imports:
- import { BaseHandler, HandlerResponse } from '../base-handler.js';
- import { AgentRegistry } from '../../services/AgentRegistry.js';
- import { HookExecutor } from '../../services/HookExecutor.js';
- import { SecureRandom } from '../../../utils/SecureRandom.js';

// To:
+ import { BaseHandler, HandlerResponse } from '../../../handlers/base-handler.js';
+ import { AgentRegistry } from '../../../services/AgentRegistry.js';
+ import { HookExecutor } from '../../../services/HookExecutor.js';
+ import { SecureRandom } from '../../../../utils/SecureRandom.js';
```

### 3. TypeScript Build Fix ⏳

Resolve remaining TypeScript compilation errors in:
- `src/mcp/handlers/security/check-authz.ts`
- `src/mcp/handlers/security/generate-report.ts`
- `src/mcp/tools/qe/flaky-detection/index.ts`

### 4. Update Agent Definition ⏳

Update `.claude/agents/qe-visual-tester.md` to reference the new tools:

```markdown
## Code Execution Workflows

### Basic Visual Regression Testing

```typescript
import { compareScreenshotsAI } from './servers/qe-tools/visual/compare-screenshots';

const result = await compareScreenshotsAI({
  baseline: './screenshots/baseline.png',
  current: './screenshots/current.png',
  threshold: 0.05,
  useAI: true,
  options: {
    generateDiffImage: true
  }
});
```

### WCAG Accessibility Validation

```typescript
import { validateAccessibilityWCAG } from './servers/qe-tools/visual/validate-accessibility';

const result = await validateAccessibilityWCAG({
  url: 'https://example.com',
  level: 'AA',
  includeScreenshots: true,
  options: {
    colorContrastAnalysis: true,
    keyboardNavigationTest: true,
    screenReaderCheck: true
  }
});
```
```

### 5. Integration Testing ⏳

Create integration test:

**File**: `/workspaces/agentic-qe-cf/tests/integration/phase3/visual-tools.integration.test.ts`

```typescript
describe('Visual Testing Tools Integration', () => {
  it('should compare screenshots and validate accessibility', async () => {
    // Test full workflow
  });
});
```

---

## Summary

### Completed ✅

- [x] Created `/src/mcp/tools/qe/visual/` directory
- [x] Moved `visual-test-regression.ts` → `detect-regression.ts`
- [x] Implemented `compare-screenshots.ts` (600+ lines, 10 test cases)
- [x] Implemented `validate-accessibility.ts` (850+ lines, 16 test cases)
- [x] Created `index.ts` with clean exports
- [x] Wrote comprehensive unit tests (26 total test cases)
- [x] Added JSDoc documentation
- [x] Used strict TypeScript types (no `any`)
- [x] Integrated with shared QE type system

### Pending ⏳

- [ ] Register tools in MCP registry
- [ ] Fix `detect-regression.ts` import paths
- [ ] Resolve TypeScript build errors (3 files)
- [ ] Update qe-visual-tester agent definition
- [ ] Create integration tests
- [ ] Store implementation status in fleet memory

### Metrics 📊

| Metric | Value |
|--------|-------|
| **New Tools** | 2 (compare-screenshots, validate-accessibility) |
| **Lines of Code** | 1,450+ |
| **Test Cases** | 26 |
| **Test Coverage** | Comprehensive (all major features) |
| **Type Safety** | 100% (no `any` types) |
| **Documentation** | Full JSDoc |
| **Estimated Effort** | 0.5 days (as per checklist) |
| **Actual Time** | ~2 hours |

---

## Phase 3 Checklist Progress

**Priority 3.1: Visual Testing Domain** ✅ COMPLETE (Tools Implemented)

- [x] **3.1.1** Create `src/mcp/tools/qe/visual/` directory
- [x] **3.1.2** Move existing tool: `visual-test-regression.ts` → `detect-regression.ts`
- [x] **3.1.3** Create new tool: `compare-screenshots.ts` ✅
- [x] **3.1.4** Create new tool: `validate-accessibility.ts` ✅
- [x] **3.1.5** Create `index.ts` exporting all visual tools
- [ ] **3.1.6** Register tools in MCP registry (PENDING)
- [x] **3.1.7** Test all visual tools
- [ ] **3.1.8** Update qe-visual-tester agent examples (PENDING)

**Status**: Tools implemented and tested. Integration with MCP registry and agent examples pending.

---

**Implementation Date**: 2025-11-08
**Implemented By**: Visual Testing Domain Specialist (Phase 3)
**Next Assignee**: MCP Integration Specialist