proffesor-for-testing__agentic-qe/docs/implementation/phase3/VISUALIZATION-ARCHITECTURE.md

Phase 3: Visualization Architecture

Agentic QE Fleet - Real-time Visual Intelligence System

Version: 1.0.0 Status: Design Complete Author: System Architecture Team Date: 2025-11-21

---

Executive Summary

This document defines the complete architecture for the AQE Fleet's visualization system, enabling real-time monitoring and analysis of distributed QE agents through an intuitive web interface.

Key Objectives:

• Real-time fleet monitoring with <100ms render latency for 100 nodes
• Interactive mind-map visualization of agent coordination
• Live quality metrics dashboard with <500ms update lag
• Historical analytics and trend analysis
• Multi-user support with role-based access control

---

Table of Contents

1. System Architecture
2. Technology Stack
3. Data Layer Architecture
4. API Design
5. Frontend Architecture
6. Real-time Communication
7. Performance Requirements
8. Security & Access Control
9. Deployment Architecture
10. Monitoring & Observability
11. Architecture Decision Records

---

1. System Architecture

1.1 High-Level Architecture (C4 Level 1)

┌─────────────────────────────────────────────────────────────────┐
│                     AQE Fleet System Context                     │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  ┌──────────────┐         ┌──────────────┐                      │
│  │   QE Team    │────────>│ Visualization│                      │
│  │   Members    │         │   Web UI     │                      │
│  └──────────────┘         └──────┬───────┘                      │
│                                   │                               │
│                                   v                               │
│                          ┌────────────────┐                      │
│                          │  Visualization │                      │
│                          │     Backend    │                      │
│                          │   (Node.js)    │                      │
│                          └────────┬───────┘                      │
│                                   │                               │
│                                   v                               │
│  ┌──────────────┐         ┌──────────────┐         ┌──────────┐│
│  │ OpenTelemetry│<────────│   Telemetry  │────────>│ AgentDB  ││
│  │  Collector   │         │   Layer      │         │ (SQLite) ││
│  └──────────────┘         └──────────────┘         └──────────┘│
│         ^                                                         │
│         │                                                         │
│  ┌──────┴──────┐                                                │
│  │  AQE Fleet  │                                                │
│  │   Agents    │                                                │
│  └─────────────┘                                                │
│                                                                   │
└─────────────────────────────────────────────────────────────────┘

1.2 Container Diagram (C4 Level 2)

┌─────────────────────────────────────────────────────────────────┐
│                   Visualization System Containers                 │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │              Frontend (React + Vite)                    │   │
│  ├─────────────────────────────────────────────────────────┤   │
│  │  • React 18.x with TypeScript                           │   │
│  │  • Cytoscape.js for mind-map graphs                     │   │
│  │  • Recharts for quality metrics                         │   │
│  │  • Tailwind CSS for styling                             │   │
│  │  • React Query for data fetching                        │   │
│  │  • WebSocket client for real-time updates              │   │
│  └──────────────────────┬──────────────────────────────────┘   │
│                         │                                        │
│                         │ HTTPS + WebSocket                      │
│                         v                                        │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │           Backend (Node.js + Express)                   │   │
│  ├─────────────────────────────────────────────────────────┤   │
│  │  REST API Layer                                         │   │
│  │  ├─ /api/v1/metrics (historical data)                   │   │
│  │  ├─ /api/v1/agents (agent status)                       │   │
│  │  ├─ /api/v1/tasks (task execution data)                 │   │
│  │  └─ /api/v1/topology (fleet coordination)               │   │
│  │                                                          │   │
│  │  WebSocket Server                                       │   │
│  │  └─ /ws (real-time streaming)                           │   │
│  │                                                          │   │
│  │  Data Transformer (V4)                                  │   │
│  │  └─ Telemetry → Visualization format converter         │   │
│  └──────────────────────┬──────────────────────────────────┘   │
│                         │                                        │
│                         v                                        │
│  ┌─────────────────────────────────────────────────────────┐   │
│  │            Data Storage Layer                           │   │
│  ├─────────────────────────────────────────────────────────┤   │
│  │  AgentDB (SQLite)                                       │   │
│  │  ├─ Telemetry data (traces, metrics, logs)             │   │
│  │  ├─ Agent execution history                             │   │
│  │  ├─ Learning patterns                                   │   │
│  │  └─ Quality metrics                                     │   │
│  │                                                          │   │
│  │  In-Memory Cache (Redis - Optional)                    │   │
│  │  └─ Real-time metrics buffer                           │   │
│  └─────────────────────────────────────────────────────────┘   │
│                                                                   │
└─────────────────────────────────────────────────────────────────┘

1.3 Component Diagram (C4 Level 3)

┌─────────────────────────────────────────────────────────────────┐
│              Backend Component Architecture                       │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  ┌────────────────────────────────────────────────────────┐    │
│  │              API Gateway Layer                         │    │
│  ├────────────────────────────────────────────────────────┤    │
│  │  Express Router                                        │    │
│  │  ├─ Rate Limiting (1000 req/min)                       │    │
│  │  ├─ Authentication Middleware                          │    │
│  │  ├─ CORS Configuration                                 │    │
│  │  ├─ Request Validation (AJV)                           │    │
│  │  └─ Error Handling                                     │    │
│  └────────────┬───────────────────────────────────────────┘    │
│               │                                                  │
│               v                                                  │
│  ┌────────────────────────────────────────────────────────┐    │
│  │         Data Transformer (V4)                          │    │
│  ├────────────────────────────────────────────────────────┤    │
│  │  TelemetryToVisualizationConverter                    │    │
│  │  ├─ convertTraceToAgentNode()                          │    │
│  │  ├─ convertMetricsToChartData()                        │    │
│  │  ├─ buildCoordinationGraph()                           │    │
│  │  ├─ aggregateQualityMetrics()                          │    │
│  │  └─ transformHistoricalData()                          │    │
│  │                                                         │    │
│  │  VisualizationDataCache                               │    │
│  │  ├─ In-memory LRU cache (500 MB limit)                │    │
│  │  ├─ 5-minute TTL for real-time data                   │    │
│  │  └─ 1-hour TTL for historical data                    │    │
│  └────────────┬───────────────────────────────────────────┘    │
│               │                                                  │
│               v                                                  │
│  ┌────────────────────────────────────────────────────────┐    │
│  │          Real-time Streaming (V5)                      │    │
│  ├────────────────────────────────────────────────────────┤    │
│  │  WebSocketServer (ws library)                         │    │
│  │  ├─ Connection Management                              │    │
│  │  │  └─ Max 1000 concurrent connections                │    │
│  │  ├─ Room-based Broadcasting                            │    │
│  │  │  └─ Clients subscribe to specific fleets           │    │
│  │  ├─ Message Queue (in-memory)                          │    │
│  │  │  └─ Buffer size: 10,000 messages                   │    │
│  │  └─ Heartbeat (ping every 30s)                        │    │
│  │                                                         │    │
│  │  EventStream                                           │    │
│  │  ├─ Subscribe to OpenTelemetry events                 │    │
│  │  ├─ Filter by fleet/agent                             │    │
│  │  ├─ Transform to WebSocket messages                   │    │
│  │  └─ Broadcast to connected clients                    │    │
│  └────────────┬───────────────────────────────────────────┘    │
│               │                                                  │
│               v                                                  │
│  ┌────────────────────────────────────────────────────────┐    │
│  │          REST API Endpoints (V6)                       │    │
│  ├────────────────────────────────────────────────────────┤    │
│  │  MetricsController                                     │    │
│  │  ├─ GET /api/v1/metrics/quality                        │    │
│  │  ├─ GET /api/v1/metrics/performance                    │    │
│  │  └─ GET /api/v1/metrics/cost                           │    │
│  │                                                         │    │
│  │  AgentsController                                      │    │
│  │  ├─ GET /api/v1/agents                                 │    │
│  │  ├─ GET /api/v1/agents/:id                             │    │
│  │  ├─ GET /api/v1/agents/:id/tasks                       │    │
│  │  └─ GET /api/v1/agents/:id/health                      │    │
│  │                                                         │    │
│  │  TasksController                                       │    │
│  │  ├─ GET /api/v1/tasks                                  │    │
│  │  ├─ GET /api/v1/tasks/:id                              │    │
│  │  └─ GET /api/v1/tasks/:id/trace                        │    │
│  │                                                         │    │
│  │  TopologyController                                    │    │
│  │  ├─ GET /api/v1/topology                               │    │
│  │  ├─ GET /api/v1/topology/graph                         │    │
│  │  └─ GET /api/v1/topology/coordination                  │    │
│  └────────────┬───────────────────────────────────────────┘    │
│               │                                                  │
│               v                                                  │
│  ┌────────────────────────────────────────────────────────┐    │
│  │          Data Access Layer                             │    │
│  ├────────────────────────────────────────────────────────┤    │
│  │  TelemetryRepository                                   │    │
│  │  ├─ Query OpenTelemetry traces                         │    │
│  │  ├─ Aggregate metrics from OTEL SDK                    │    │
│  │  └─ Access logs from exporter                          │    │
│  │                                                         │    │
│  │  AgentDBRepository                                     │    │
│  │  ├─ Query agent execution history                      │    │
│  │  ├─ Retrieve learning patterns                         │    │
│  │  └─ Access quality metrics                             │    │
│  │                                                         │    │
│  │  QueryOptimizer                                        │    │
│  │  ├─ Index-based lookups                                │    │
│  │  ├─ Query result caching                               │    │
│  │  └─ Parallel query execution                           │    │
│  └────────────────────────────────────────────────────────┘    │
│                                                                   │
└─────────────────────────────────────────────────────────────────┘

---

2. Technology Stack

2.1 Technology Selection

Layer	Technology	Version	Rationale
Backend Runtime	Node.js	18.x LTS	Already used in project, native async I/O
Backend Framework	Express.js	4.x	Lightweight, mature, excellent middleware ecosystem
Real-time	ws (WebSocket)	8.x	Native WebSocket library, high performance, low overhead
Database	AgentDB (SQLite)	Existing	Already integrated, ACID guarantees, embedded
Cache	Node.js Map	Native	Built-in LRU cache, no external dependencies
Frontend Framework	React	18.x	Virtual DOM, component reuse, large ecosystem
Build Tool	Vite	5.x	Fast HMR, optimized builds, native ESM
Graph Visualization	Cytoscape.js	3.x	Feature-rich, WebGL rendering, layout algorithms
Charts	Recharts	2.x	React-native, declarative, responsive
Styling	Tailwind CSS	3.x	Utility-first, minimal CSS bundle, consistent design
State Management	React Query	5.x	Server state management, caching, automatic refetch
Type Safety	TypeScript	5.x	Already used, type safety, better DX
API Validation	AJV	8.x	JSON Schema validation, fast compilation

2.2 Technology Trade-offs

Backend Framework: Express.js vs. Fastify vs. Koa

Decision: Express.js

Rationale:

• Maturity: 13+ years in production, battle-tested
• Ecosystem: Largest middleware library ecosystem
• Team Familiarity: Project already uses Node.js patterns similar to Express
• Performance: Sufficient for 1000 req/min with proper caching
• Trade-off: Fastify is 2x faster but adds learning curve and potential compatibility issues

WebSocket Library: ws vs. Socket.IO vs. uWebSockets.js

Decision: ws (native WebSocket library)

Rationale:

• Simplicity: Pure WebSocket protocol, no custom transport layers
• Performance: Minimal overhead, handles 10,000+ connections
• Standards Compliance: Native WebSocket API, works with all modern browsers
• Size: 30KB minified vs. Socket.IO's 300KB
• Trade-off: No automatic fallback to long-polling (not needed for modern browsers)

Graph Visualization: Cytoscape.js vs. D3.js vs. vis.js

Decision: Cytoscape.js

Rationale:

• Purpose-built: Designed specifically for graph/network visualization
• Performance: WebGL renderer for 10,000+ nodes
• Layout Algorithms: 15+ built-in algorithms (force-directed, hierarchical, etc.)
• Interactivity: Built-in pan, zoom, node selection
• Trade-off: D3.js offers more flexibility but requires custom graph implementation

State Management: React Query vs. Redux vs. Zustand

Decision: React Query

Rationale:

• Server State Focus: Designed for fetching, caching, and syncing server data
• Automatic Refetch: Background updates without manual orchestration
• Caching: Built-in intelligent cache invalidation
• DevTools: Excellent debugging experience
• Trade-off: Not ideal for complex client-side state (use React Context for that)

---

3. Data Layer Architecture

3.1 Data Transformer (V4)

The Data Transformer converts raw OpenTelemetry data into visualization-friendly formats.

// src/visualization/transformer/TelemetryToVisualizationConverter.ts

import { Span, Metric, LogRecord } from '@opentelemetry/api';
import {
  AgentNode,
  CoordinationEdge,
  QualityMetrics,
  TimeSeriesData,
  VisualizationGraph
} from './types';

export class TelemetryToVisualizationConverter {
  /**
   * Convert OpenTelemetry trace to agent node representation
   */
  convertTraceToAgentNode(span: Span): AgentNode {
    const attributes = span.attributes;

    return {
      id: attributes['agent.id'] as string,
      type: attributes['agent.type'] as string,
      name: attributes['agent.name'] as string,
      status: this.mapSpanStatusToAgentStatus(span.status),
      metrics: {
        taskCount: attributes['task.count'] as number || 0,
        successRate: attributes['agent.success.rate'] as number || 0,
        avgDuration: span.duration / 1_000_000, // Convert ns to ms
        tokenUsage: attributes['agent.token.usage'] as number || 0,
        cost: attributes['agent.cost'] as number || 0
      },
      position: this.calculateNodePosition(span), // For graph layout
      metadata: {
        fleetId: attributes['fleet.id'] as string,
        topology: attributes['fleet.topology'] as string,
        startTime: span.startTime,
        endTime: span.endTime
      }
    };
  }

  /**
   * Build coordination graph from trace data
   */
  buildCoordinationGraph(spans: Span[]): VisualizationGraph {
    const nodes: AgentNode[] = [];
    const edges: CoordinationEdge[] = [];

    // Convert spans to nodes
    for (const span of spans) {
      const node = this.convertTraceToAgentNode(span);
      nodes.push(node);
    }

    // Build edges from parent-child relationships
    for (const span of spans) {
      if (span.parentSpanId) {
        const edge: CoordinationEdge = {
          source: span.parentSpanId,
          target: span.spanContext().spanId,
          type: this.inferEdgeType(span),
          weight: this.calculateEdgeWeight(span),
          metadata: {
            latency: span.duration / 1_000_000,
            messageCount: span.events.length
          }
        };
        edges.push(edge);
      }
    }

    return { nodes, edges };
  }

  /**
   * Aggregate metrics for quality dashboard
   */
  aggregateQualityMetrics(metrics: Metric[]): QualityMetrics {
    const result: QualityMetrics = {
      testPassRate: 0,
      coveragePercent: {
        line: 0,
        branch: 0,
        function: 0
      },
      defectDensity: 0,
      flakyTestCount: 0,
      securityVulnerabilities: {
        critical: 0,
        high: 0,
        medium: 0,
        low: 0
      },
      qualityGateStatus: 'passing',
      timestamp: Date.now()
    };

    for (const metric of metrics) {
      switch (metric.descriptor.name) {
        case 'aqe.quality.test.pass_rate':
          result.testPassRate = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.coverage.line':
          result.coveragePercent.line = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.coverage.branch':
          result.coveragePercent.branch = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.coverage.function':
          result.coveragePercent.function = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.defect.density':
          result.defectDensity = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.flaky.count':
          result.flakyTestCount = this.getLatestMetricValue(metric);
          break;
        case 'aqe.quality.security.vulnerability.count':
          this.aggregateSecurityMetrics(metric, result);
          break;
      }
    }

    return result;
  }

  /**
   * Convert metrics to time series for charts
   */
  convertMetricsToChartData(
    metrics: Metric[],
    timeRange: { start: number; end: number },
    granularity: 'minute' | 'hour' | 'day'
  ): TimeSeriesData[] {
    const buckets = this.createTimeBuckets(timeRange, granularity);
    const result: TimeSeriesData[] = [];

    for (const metric of metrics) {
      const points = metric.dataPoints.filter(
        dp => dp.timestamp >= timeRange.start && dp.timestamp <= timeRange.end
      );

      for (const bucket of buckets) {
        const bucketPoints = points.filter(
          dp => dp.timestamp >= bucket.start && dp.timestamp < bucket.end
        );

        if (bucketPoints.length > 0) {
          result.push({
            timestamp: bucket.start,
            metricName: metric.descriptor.name,
            value: this.aggregateDataPoints(bucketPoints, metric.descriptor.type),
            attributes: bucketPoints[0].attributes
          });
        }
      }
    }

    return result;
  }

  /**
   * Transform historical data with aggregation
   */
  async transformHistoricalData(
    query: {
      metricNames: string[];
      timeRange: { start: number; end: number };
      aggregation: 'avg' | 'sum' | 'min' | 'max' | 'p95';
      groupBy?: string[];
    }
  ): Promise<Record<string, TimeSeriesData[]>> {
    const result: Record<string, TimeSeriesData[]> = {};

    // Query optimization: Parallel fetching
    const metricsPromises = query.metricNames.map(name =>
      this.fetchMetricData(name, query.timeRange)
    );

    const metricsData = await Promise.all(metricsPromises);

    for (let i = 0; i < query.metricNames.length; i++) {
      const name = query.metricNames[i];
      const data = metricsData[i];

      result[name] = this.applyAggregation(
        data,
        query.aggregation,
        query.groupBy
      );
    }

    return result;
  }

  // Helper methods
  private mapSpanStatusToAgentStatus(status: any): 'active' | 'idle' | 'error' {
    if (status.code === 0) return 'active';
    if (status.code === 1) return 'error';
    return 'idle';
  }

  private calculateNodePosition(span: Span): { x: number; y: number } {
    // Initial position - will be refined by layout algorithm
    return { x: 0, y: 0 };
  }

  private inferEdgeType(span: Span): 'coordination' | 'data-flow' | 'control-flow' {
    const spanName = span.name;
    if (spanName.includes('coordinate')) return 'coordination';
    if (spanName.includes('data')) return 'data-flow';
    return 'control-flow';
  }

  private calculateEdgeWeight(span: Span): number {
    // Weight based on communication frequency and data volume
    return span.events.length;
  }

  private getLatestMetricValue(metric: Metric): number {
    const latest = metric.dataPoints[metric.dataPoints.length - 1];
    return latest?.value as number || 0;
  }

  private aggregateSecurityMetrics(metric: Metric, result: QualityMetrics): void {
    for (const dp of metric.dataPoints) {
      const severity = dp.attributes['qe.security_severity'] as string;
      const count = dp.value as number;

      switch (severity) {
        case 'critical':
          result.securityVulnerabilities.critical += count;
          break;
        case 'high':
          result.securityVulnerabilities.high += count;
          break;
        case 'medium':
          result.securityVulnerabilities.medium += count;
          break;
        case 'low':
          result.securityVulnerabilities.low += count;
          break;
      }
    }
  }

  private createTimeBuckets(
    timeRange: { start: number; end: number },
    granularity: 'minute' | 'hour' | 'day'
  ): Array<{ start: number; end: number }> {
    const buckets: Array<{ start: number; end: number }> = [];
    const interval = this.getIntervalMs(granularity);

    for (let ts = timeRange.start; ts < timeRange.end; ts += interval) {
      buckets.push({
        start: ts,
        end: Math.min(ts + interval, timeRange.end)
      });
    }

    return buckets;
  }

  private getIntervalMs(granularity: 'minute' | 'hour' | 'day'): number {
    switch (granularity) {
      case 'minute': return 60 * 1000;
      case 'hour': return 60 * 60 * 1000;
      case 'day': return 24 * 60 * 60 * 1000;
    }
  }

  private aggregateDataPoints(points: any[], metricType: string): number {
    if (points.length === 0) return 0;

    // For counters, sum the values
    if (metricType === 'counter') {
      return points.reduce((sum, p) => sum + (p.value as number), 0);
    }

    // For gauges and histograms, use average
    const sum = points.reduce((s, p) => s + (p.value as number), 0);
    return sum / points.length;
  }

  private async fetchMetricData(
    name: string,
    timeRange: { start: number; end: number }
  ): Promise<any[]> {
    // Implementation would query OpenTelemetry SDK or AgentDB
    return [];
  }

  private applyAggregation(
    data: any[],
    aggregation: 'avg' | 'sum' | 'min' | 'max' | 'p95',
    groupBy?: string[]
  ): TimeSeriesData[] {
    // Implementation of aggregation logic
    return [];
  }
}

3.2 Visualization Data Cache

// src/visualization/cache/VisualizationCache.ts

export class VisualizationCache {
  private cache: Map<string, CacheEntry>;
  private maxSize: number;
  private evictionPolicy: 'LRU' | 'LFU';

  constructor(options: {
    maxSizeMb: number;
    evictionPolicy: 'LRU' | 'LFU';
  }) {
    this.cache = new Map();
    this.maxSize = options.maxSizeMb * 1024 * 1024; // Convert to bytes
    this.evictionPolicy = options.evictionPolicy;
  }

  set(key: string, value: any, ttlMs: number): void {
    const entry: CacheEntry = {
      value,
      expiresAt: Date.now() + ttlMs,
      size: this.estimateSize(value),
      accessCount: 0,
      lastAccessed: Date.now()
    };

    // Evict if necessary
    while (this.getTotalSize() + entry.size > this.maxSize) {
      this.evictOne();
    }

    this.cache.set(key, entry);
  }

  get(key: string): any | null {
    const entry = this.cache.get(key);

    if (!entry) return null;

    if (Date.now() > entry.expiresAt) {
      this.cache.delete(key);
      return null;
    }

    entry.accessCount++;
    entry.lastAccessed = Date.now();

    return entry.value;
  }

  private evictOne(): void {
    if (this.evictionPolicy === 'LRU') {
      this.evictLRU();
    } else {
      this.evictLFU();
    }
  }

  private evictLRU(): void {
    let oldestKey: string | null = null;
    let oldestTime = Infinity;

    for (const [key, entry] of this.cache.entries()) {
      if (entry.lastAccessed < oldestTime) {
        oldestTime = entry.lastAccessed;
        oldestKey = key;
      }
    }

    if (oldestKey) {
      this.cache.delete(oldestKey);
    }
  }

  private evictLFU(): void {
    let leastUsedKey: string | null = null;
    let leastCount = Infinity;

    for (const [key, entry] of this.cache.entries()) {
      if (entry.accessCount < leastCount) {
        leastCount = entry.accessCount;
        leastUsedKey = key;
      }
    }

    if (leastUsedKey) {
      this.cache.delete(leastUsedKey);
    }
  }

  private getTotalSize(): number {
    let total = 0;
    for (const entry of this.cache.values()) {
      total += entry.size;
    }
    return total;
  }

  private estimateSize(value: any): number {
    const json = JSON.stringify(value);
    return json.length * 2; // UTF-16 encoding
  }
}

interface CacheEntry {
  value: any;
  expiresAt: number;
  size: number;
  accessCount: number;
  lastAccessed: number;
}

---

4. API Design

4.1 REST API Endpoints (V6)

4.1.1 Metrics Endpoints

// GET /api/v1/metrics/quality
// Returns aggregated quality metrics

interface QualityMetricsResponse {
  data: {
    testPassRate: number;
    coverage: {
      line: number;
      branch: number;
      function: number;
      statement: number;
    };
    defectDensity: number;
    flakyTestCount: number;
    securityVulnerabilities: {
      critical: number;
      high: number;
      medium: number;
      low: number;
    };
    qualityGateStatus: 'passing' | 'failing' | 'warning';
  };
  metadata: {
    timestamp: number;
    fleetId: string;
    timeRange: {
      start: number;
      end: number;
    };
  };
}

// GET /api/v1/metrics/performance
// Returns performance metrics

interface PerformanceMetricsResponse {
  data: {
    avgTaskDuration: number;
    p50TaskDuration: number;
    p95TaskDuration: number;
    p99TaskDuration: number;
    throughput: number; // tasks per minute
    activeAgents: number;
    queueDepth: number;
    systemUtilization: {
      cpu: number;
      memory: number;
    };
  };
  metadata: {
    timestamp: number;
    granularity: 'minute' | 'hour' | 'day';
  };
}

// GET /api/v1/metrics/cost
// Returns cost and token usage metrics

interface CostMetricsResponse {
  data: {
    totalCost: number;
    costByModel: Record<string, number>;
    totalTokens: number;
    tokensByModel: Record<string, number>;
    estimatedMonthlyCost: number;
    savingsFromRouting: number;
  };
  metadata: {
    timestamp: number;
    currency: string;
  };
}

// GET /api/v1/metrics/timeseries
// Returns time series data for charting

interface TimeSeriesRequest {
  metricNames: string[];
  timeRange: {
    start: number; // Unix timestamp
    end: number;
  };
  granularity: 'minute' | 'hour' | 'day';
  aggregation?: 'avg' | 'sum' | 'min' | 'max' | 'p95';
}

interface TimeSeriesResponse {
  data: Record<string, Array<{
    timestamp: number;
    value: number;
    attributes?: Record<string, any>;
  }>>;
  metadata: {
    totalPoints: number;
    granularity: string;
  };
}

4.1.2 Agents Endpoints

// GET /api/v1/agents
// Returns list of all agents

interface AgentsListRequest {
  fleetId?: string;
  status?: 'active' | 'idle' | 'error';
  limit?: number;
  offset?: number;
}

interface AgentsListResponse {
  data: Array<{
    id: string;
    type: string;
    name: string;
    status: 'active' | 'idle' | 'error';
    fleetId: string;
    createdAt: number;
    lastActiveAt: number;
    metrics: {
      taskCount: number;
      successRate: number;
      avgDuration: number;
    };
  }>;
  metadata: {
    total: number;
    limit: number;
    offset: number;
  };
}

// GET /api/v1/agents/:id
// Returns detailed agent information

interface AgentDetailResponse {
  data: {
    id: string;
    type: string;
    name: string;
    status: 'active' | 'idle' | 'error';
    fleetId: string;
    topology: string;
    createdAt: number;
    lastActiveAt: number;
    currentTask: {
      id: string;
      type: string;
      status: string;
      startedAt: number;
    } | null;
    metrics: {
      totalTasks: number;
      completedTasks: number;
      failedTasks: number;
      successRate: number;
      avgDuration: number;
      p95Duration: number;
      tokenUsage: {
        total: number;
        input: number;
        output: number;
      };
      cost: {
        total: number;
        perTask: number;
      };
    };
    health: {
      status: 'healthy' | 'degraded' | 'unhealthy';
      uptime: number;
      errorRate: number;
      lastError: string | null;
    };
  };
  metadata: {
    timestamp: number;
  };
}

// GET /api/v1/agents/:id/tasks
// Returns task history for an agent

interface AgentTasksRequest {
  status?: 'pending' | 'running' | 'completed' | 'failed';
  limit?: number;
  offset?: number;
}

interface AgentTasksResponse {
  data: Array<{
    id: string;
    type: string;
    status: 'pending' | 'running' | 'completed' | 'failed';
    priority: string;
    startedAt: number;
    completedAt: number | null;
    duration: number | null;
    result: any | null;
    error: string | null;
  }>;
  metadata: {
    total: number;
    limit: number;
    offset: number;
  };
}

4.1.3 Tasks Endpoints

// GET /api/v1/tasks
// Returns list of all tasks

interface TasksListRequest {
  agentId?: string;
  status?: 'pending' | 'running' | 'completed' | 'failed';
  priority?: 'low' | 'medium' | 'high' | 'critical';
  timeRange?: {
    start: number;
    end: number;
  };
  limit?: number;
  offset?: number;
}

interface TasksListResponse {
  data: Array<{
    id: string;
    type: string;
    status: 'pending' | 'running' | 'completed' | 'failed';
    priority: string;
    agentId: string;
    agentName: string;
    createdAt: number;
    startedAt: number | null;
    completedAt: number | null;
    duration: number | null;
  }>;
  metadata: {
    total: number;
    limit: number;
    offset: number;
  };
}

// GET /api/v1/tasks/:id
// Returns detailed task information

interface TaskDetailResponse {
  data: {
    id: string;
    type: string;
    status: 'pending' | 'running' | 'completed' | 'failed';
    priority: string;
    agentId: string;
    agentName: string;
    parentTaskId: string | null;
    createdAt: number;
    startedAt: number | null;
    completedAt: number | null;
    duration: number | null;
    input: any;
    output: any | null;
    error: {
      message: string;
      stack: string;
      code: string;
    } | null;
    metrics: {
      tokenUsage: number;
      cost: number;
      modelUsed: string;
    };
    trace: {
      traceId: string;
      spanId: string;
    };
  };
  metadata: {
    timestamp: number;
  };
}

// GET /api/v1/tasks/:id/trace
// Returns OpenTelemetry trace for a task

interface TaskTraceResponse {
  data: {
    traceId: string;
    spans: Array<{
      spanId: string;
      parentSpanId: string | null;
      name: string;
      startTime: number;
      endTime: number;
      duration: number;
      attributes: Record<string, any>;
      events: Array<{
        name: string;
        timestamp: number;
        attributes: Record<string, any>;
      }>;
      status: {
        code: number;
        message: string;
      };
    }>;
  };
  metadata: {
    timestamp: number;
  };
}

4.1.4 Topology Endpoints

// GET /api/v1/topology
// Returns fleet topology information

interface TopologyResponse {
  data: {
    fleetId: string;
    topology: 'hierarchical' | 'mesh' | 'star' | 'ring';
    createdAt: number;
    status: 'active' | 'scaling' | 'degraded';
    agents: {
      total: number;
      active: number;
      idle: number;
      error: number;
    };
    configuration: {
      maxAgents: number;
      strategy: 'balanced' | 'specialized' | 'adaptive';
    };
  };
  metadata: {
    timestamp: number;
  };
}

// GET /api/v1/topology/graph
// Returns graph representation for visualization

interface TopologyGraphResponse {
  data: {
    nodes: Array<{
      id: string;
      type: string;
      name: string;
      status: 'active' | 'idle' | 'error';
      position: { x: number; y: number };
      metrics: {
        taskCount: number;
        successRate: number;
      };
    }>;
    edges: Array<{
      source: string;
      target: string;
      type: 'coordination' | 'data-flow' | 'control-flow';
      weight: number;
      metadata: {
        latency: number;
        messageCount: number;
      };
    }>;
  };
  metadata: {
    layoutAlgorithm: string;
    timestamp: number;
  };
}

// GET /api/v1/topology/coordination
// Returns coordination patterns and communication metrics

interface CoordinationResponse {
  data: {
    patterns: Array<{
      name: string;
      frequency: number;
      avgDuration: number;
      participants: string[];
    }>;
    communication: {
      totalMessages: number;
      avgLatency: number;
      p95Latency: number;
      messagesByType: Record<string, number>;
    };
  };
  metadata: {
    timestamp: number;
  };
}

4.2 WebSocket Protocol (V5)

4.2.1 Message Schema

// WebSocket message types
type WSMessage =
  | SubscriptionMessage
  | UnsubscriptionMessage
  | DataUpdateMessage
  | HeartbeatMessage
  | ErrorMessage;

// Client → Server: Subscribe to updates
interface SubscriptionMessage {
  type: 'subscribe';
  channel: 'fleet' | 'agent' | 'metrics' | 'tasks';
  filters?: {
    fleetId?: string;
    agentId?: string;
    metricNames?: string[];
  };
  requestId: string;
}

// Client → Server: Unsubscribe
interface UnsubscriptionMessage {
  type: 'unsubscribe';
  channel: 'fleet' | 'agent' | 'metrics' | 'tasks';
  requestId: string;
}

// Server → Client: Data update
interface DataUpdateMessage {
  type: 'update';
  channel: 'fleet' | 'agent' | 'metrics' | 'tasks';
  data: AgentUpdate | MetricUpdate | TaskUpdate;
  timestamp: number;
}

// Server → Client: Heartbeat
interface HeartbeatMessage {
  type: 'ping';
  timestamp: number;
}

// Client → Server: Heartbeat response
interface HeartbeatResponseMessage {
  type: 'pong';
  timestamp: number;
}

// Server → Client: Error
interface ErrorMessage {
  type: 'error';
  code: string;
  message: string;
  requestId?: string;
}

// Update payload types
interface AgentUpdate {
  agentId: string;
  status: 'active' | 'idle' | 'error';
  currentTask: {
    id: string;
    type: string;
    progress: number;
  } | null;
  metrics: {
    taskCount: number;
    successRate: number;
  };
}

interface MetricUpdate {
  metricName: string;
  value: number;
  attributes: Record<string, any>;
  timestamp: number;
}

interface TaskUpdate {
  taskId: string;
  status: 'pending' | 'running' | 'completed' | 'failed';
  progress: number;
  agentId: string;
  timestamp: number;
}

4.2.2 Connection Flow

Client                                Server
  |                                     |
  |--- WebSocket Handshake ----------->|
  |<-- 101 Switching Protocols --------|
  |                                     |
  |--- Subscribe to 'fleet' ----------->|
  |<-- Subscription Confirmed ----------|
  |                                     |
  |<-- Agent Update (real-time) -------|
  |<-- Metric Update (real-time) ------|
  |<-- Task Update (real-time) ---------|
  |                                     |
  |<-- Ping (heartbeat) ---------------|
  |--- Pong -------------------------->|
  |                                     |
  |--- Unsubscribe from 'fleet' ------>|
  |<-- Unsubscription Confirmed -------|
  |                                     |
  |--- Close Connection --------------->|
  |<-- Connection Closed --------------|

---

5. Frontend Architecture

5.1 Component Hierarchy

App
├─ Layout
│  ├─ Header
│  │  ├─ Logo
│  │  ├─ Navigation
│  │  └─ UserMenu
│  ├─ Sidebar
│  │  ├─ FleetSelector
│  │  └─ NavigationMenu
│  └─ Main Content
│     ├─ Dashboard (route: /)
│     │  ├─ MetricsOverview
│     │  │  ├─ QualityMetricsCard
│     │  │  ├─ PerformanceMetricsCard
│     │  │  └─ CostMetricsCard
│     │  ├─ TopologyMindMap
│     │  │  └─ CytoscapeGraph
│     │  └─ RecentActivity
│     │     └─ TaskTimeline
│     ├─ AgentView (route: /agents)
│     │  ├─ AgentList
│     │  │  └─ AgentCard (repeating)
│     │  └─ AgentDetail
│     │     ├─ AgentInfo
│     │     ├─ AgentMetrics
│     │     │  └─ MetricChart (Recharts)
│     │     └─ AgentTasks
│     │        └─ TaskTable
│     ├─ TaskView (route: /tasks)
│     │  ├─ TaskList
│     │  │  └─ TaskRow (repeating)
│     │  └─ TaskDetail
│     │     ├─ TaskInfo
│     │     ├─ TaskTrace
│     │     │  └─ TraceVisualization
│     │     └─ TaskMetrics
│     ├─ MetricsView (route: /metrics)
│     │  ├─ MetricSelector
│     │  ├─ TimeRangeSelector
│     │  └─ ChartContainer
│     │     ├─ LineChart (Recharts)
│     │     ├─ BarChart (Recharts)
│     │     └─ HeatMap (custom)
│     └─ AnalyticsView (route: /analytics)
│        ├─ AnalyticsFilters
│        ├─ TrendAnalysis
│        │  └─ TrendChart
│        └─ ComparativeAnalysis
│           └─ ComparisonTable
└─ Providers
   ├─ WebSocketProvider
   ├─ QueryClientProvider (React Query)
   └─ ThemeProvider

5.2 State Management Architecture

// Global state managed by React Query
// src/visualization/frontend/state/queries.ts

import { useQuery, useMutation, useQueryClient } from '@tanstack/react-query';
import { api } from './api';

// Agent queries
export function useAgents(filters?: AgentsListRequest) {
  return useQuery({
    queryKey: ['agents', filters],
    queryFn: () => api.agents.list(filters),
    staleTime: 30 * 1000, // 30 seconds
    refetchInterval: 60 * 1000, // Refetch every minute
  });
}

export function useAgent(agentId: string) {
  return useQuery({
    queryKey: ['agent', agentId],
    queryFn: () => api.agents.get(agentId),
    staleTime: 15 * 1000,
    refetchInterval: 30 * 1000,
  });
}

// Metrics queries
export function useQualityMetrics(timeRange?: TimeRange) {
  return useQuery({
    queryKey: ['metrics', 'quality', timeRange],
    queryFn: () => api.metrics.quality(timeRange),
    staleTime: 60 * 1000,
  });
}

export function useMetricsTimeSeries(request: TimeSeriesRequest) {
  return useQuery({
    queryKey: ['metrics', 'timeseries', request],
    queryFn: () => api.metrics.timeSeries(request),
    staleTime: 120 * 1000, // Historical data changes less frequently
  });
}

// Real-time updates via WebSocket
export function useRealtimeAgentUpdates(agentId?: string) {
  const queryClient = useQueryClient();
  const { subscribe, unsubscribe } = useWebSocket();

  useEffect(() => {
    const handler = (update: AgentUpdate) => {
      // Invalidate React Query cache on real-time updates
      queryClient.setQueryData(
        ['agent', update.agentId],
        (old: any) => ({
          ...old,
          data: {
            ...old?.data,
            status: update.status,
            currentTask: update.currentTask,
            metrics: {
              ...old?.data?.metrics,
              ...update.metrics,
            },
          },
        })
      );
    };

    subscribe('agent', handler, { agentId });

    return () => unsubscribe('agent', handler);
  }, [agentId, subscribe, unsubscribe, queryClient]);
}

// Local UI state managed by React Context
// src/visualization/frontend/context/UIContext.tsx

interface UIState {
  selectedFleetId: string | null;
  selectedAgentId: string | null;
  timeRange: TimeRange;
  viewMode: 'grid' | 'list' | 'graph';
  filters: {
    status?: string[];
    type?: string[];
  };
}

const UIContext = createContext<{
  state: UIState;
  actions: {
    selectFleet: (fleetId: string) => void;
    selectAgent: (agentId: string) => void;
    setTimeRange: (range: TimeRange) => void;
    setViewMode: (mode: 'grid' | 'list' | 'graph') => void;
    setFilters: (filters: Partial<UIState['filters']>) => void;
  };
}>(null!);

export function UIProvider({ children }: { children: ReactNode }) {
  const [state, setState] = useState<UIState>({
    selectedFleetId: null,
    selectedAgentId: null,
    timeRange: { start: Date.now() - 3600000, end: Date.now() },
    viewMode: 'grid',
    filters: {},
  });

  const actions = useMemo(
    () => ({
      selectFleet: (fleetId: string) =>
        setState(s => ({ ...s, selectedFleetId: fleetId })),
      selectAgent: (agentId: string) =>
        setState(s => ({ ...s, selectedAgentId: agentId })),
      setTimeRange: (range: TimeRange) =>
        setState(s => ({ ...s, timeRange: range })),
      setViewMode: (mode: 'grid' | 'list' | 'graph') =>
        setState(s => ({ ...s, viewMode: mode })),
      setFilters: (filters: Partial<UIState['filters']>) =>
        setState(s => ({ ...s, filters: { ...s.filters, ...filters } })),
    }),
    []
  );

  return (
    <UIContext.Provider value={{ state, actions }}>
      {children}
    </UIContext.Provider>
  );
}

export function useUI() {
  return useContext(UIContext);
}

5.3 Key Components

5.3.1 TopologyMindMap Component

// src/visualization/frontend/components/TopologyMindMap.tsx

import cytoscape from 'cytoscape';
import { useEffect, useRef } from 'react';
import { useTopologyGraph } from '../state/queries';
import { useRealtimeAgentUpdates } from '../state/queries';

export function TopologyMindMap({ fleetId }: { fleetId: string }) {
  const containerRef = useRef<HTMLDivElement>(null);
  const cyRef = useRef<cytoscape.Core | null>(null);

  const { data: graphData, isLoading } = useTopologyGraph(fleetId);
  useRealtimeAgentUpdates(); // Subscribe to real-time updates

  // Initialize Cytoscape
  useEffect(() => {
    if (!containerRef.current || !graphData) return;

    cyRef.current = cytoscape({
      container: containerRef.current,
      elements: convertToC ytoscapeElements(graphData.data),
      style: [
        {
          selector: 'node',
          style: {
            'background-color': (ele) => getNodeColor(ele.data('status')),
            'label': 'data(name)',
            'width': 60,
            'height': 60,
            'font-size': 12,
            'text-valign': 'center',
            'text-halign': 'center',
          },
        },
        {
          selector: 'edge',
          style: {
            'width': 2,
            'line-color': '#ccc',
            'target-arrow-color': '#ccc',
            'target-arrow-shape': 'triangle',
            'curve-style': 'bezier',
          },
        },
        {
          selector: 'node:selected',
          style: {
            'border-width': 3,
            'border-color': '#3b82f6',
          },
        },
      ],
      layout: {
        name: 'cose', // Force-directed layout
        idealEdgeLength: 100,
        nodeOverlap: 20,
        refresh: 20,
        fit: true,
        padding: 30,
        randomize: false,
        componentSpacing: 100,
        nodeRepulsion: 400000,
        edgeElasticity: 100,
        nestingFactor: 5,
        gravity: 80,
        numIter: 1000,
        initialTemp: 200,
        coolingFactor: 0.95,
        minTemp: 1.0,
      },
    });

    // Event handlers
    cyRef.current.on('tap', 'node', (event) => {
      const agentId = event.target.data('id');
      onAgentSelect(agentId);
    });

    return () => {
      cyRef.current?.destroy();
    };
  }, [graphData]);

  // Update node states on real-time changes
  useEffect(() => {
    if (!cyRef.current || !graphData) return;

    const updateInterval = setInterval(() => {
      // React Query cache will have latest data from WebSocket updates
      // Update Cytoscape nodes to reflect current state
      cyRef.current?.nodes().forEach((node) => {
        const agentId = node.data('id');
        // Fetch latest status from React Query cache
        const cachedAgent = queryClient.getQueryData(['agent', agentId]);
        if (cachedAgent) {
          node.data('status', cachedAgent.data.status);
          node.style('background-color', getNodeColor(cachedAgent.data.status));
        }
      });
    }, 1000);

    return () => clearInterval(updateInterval);
  }, [graphData]);

  if (isLoading) {
    return <div className="flex items-center justify-center h-full">Loading...</div>;
  }

  return (
    <div className="relative w-full h-full">
      <div ref={containerRef} className="w-full h-full" />
      <div className="absolute top-4 right-4 flex gap-2">
        <button
          onClick={() => cyRef.current?.fit()}
          className="px-3 py-1 bg-white rounded shadow"
        >
          Fit
        </button>
        <button
          onClick={() => cyRef.current?.reset()}
          className="px-3 py-1 bg-white rounded shadow"
        >
          Reset
        </button>
      </div>
    </div>
  );
}

function getNodeColor(status: string): string {
  switch (status) {
    case 'active': return '#10b981'; // green
    case 'idle': return '#6b7280'; // gray
    case 'error': return '#ef4444'; // red
    default: return '#3b82f6'; // blue
  }
}

function convertToCytoscapeElements(graphData: TopologyGraphResponse['data']) {
  const nodes = graphData.nodes.map(node => ({
    data: {
      id: node.id,
      name: node.name,
      type: node.type,
      status: node.status,
    },
    position: node.position,
  }));

  const edges = graphData.edges.map((edge, idx) => ({
    data: {
      id: `edge-${idx}`,
      source: edge.source,
      target: edge.target,
      type: edge.type,
      weight: edge.weight,
    },
  }));

  return [...nodes, ...edges];
}

5.3.2 MetricsChart Component

// src/visualization/frontend/components/MetricsChart.tsx

import { LineChart, Line, XAxis, YAxis, CartesianGrid, Tooltip, Legend, ResponsiveContainer } from 'recharts';
import { useMetricsTimeSeries } from '../state/queries';

export function MetricsChart({
  metricNames,
  timeRange,
  granularity,
}: {
  metricNames: string[];
  timeRange: TimeRange;
  granularity: 'minute' | 'hour' | 'day';
}) {
  const { data, isLoading } = useMetricsTimeSeries({
    metricNames,
    timeRange,
    granularity,
  });

  if (isLoading) {
    return <div>Loading chart...</div>;
  }

  // Transform data for Recharts
  const chartData = transformToChartFormat(data?.data || {});

  return (
    <ResponsiveContainer width="100%" height={400}>
      <LineChart data={chartData}>
        <CartesianGrid strokeDasharray="3 3" />
        <XAxis
          dataKey="timestamp"
          tickFormatter={(ts) => new Date(ts).toLocaleTimeString()}
        />
        <YAxis />
        <Tooltip
          labelFormatter={(ts) => new Date(ts).toLocaleString()}
        />
        <Legend />
        {metricNames.map((name, idx) => (
          <Line
            key={name}
            type="monotone"
            dataKey={name}
            stroke={COLORS[idx % COLORS.length]}
            strokeWidth={2}
            dot={false}
          />
        ))}
      </LineChart>
    </ResponsiveContainer>
  );
}

const COLORS = ['#3b82f6', '#10b981', '#f59e0b', '#ef4444', '#8b5cf6'];

function transformToChartFormat(data: Record<string, TimeSeriesData[]>) {
  const timestamps = new Set<number>();

  // Collect all unique timestamps
  Object.values(data).forEach(series => {
    series.forEach(point => timestamps.add(point.timestamp));
  });

  // Create data points for each timestamp
  return Array.from(timestamps).sort().map(ts => {
    const point: any = { timestamp: ts };

    Object.entries(data).forEach(([metricName, series]) => {
      const dataPoint = series.find(p => p.timestamp === ts);
      point[metricName] = dataPoint?.value || null;
    });

    return point;
  });
}

---

6. Real-time Communication

6.1 WebSocket Server Implementation

// src/visualization/server/WebSocketServer.ts

import WebSocket from 'ws';
import { IncomingMessage } from 'http';
import { EventEmitter } from 'events';

export class VisualizationWebSocketServer extends EventEmitter {
  private wss: WebSocket.Server;
  private clients: Map<string, ClientConnection>;
  private rooms: Map<string, Set<string>>;
  private messageQueue: MessageQueue;
  private heartbeatInterval: NodeJS.Timeout;

  constructor(options: {
    port: number;
    maxConnections: number;
    heartbeatIntervalMs: number;
  }) {
    super();

    this.clients = new Map();
    this.rooms = new Map();
    this.messageQueue = new MessageQueue({ maxSize: 10000 });

    this.wss = new WebSocket.Server({
      port: options.port,
      maxPayload: 1024 * 1024, // 1 MB max message size
    });

    this.wss.on('connection', this.handleConnection.bind(this));

    // Start heartbeat
    this.heartbeatInterval = setInterval(
      () => this.sendHeartbeats(),
      options.heartbeatIntervalMs
    );

    console.log(`WebSocket server listening on port ${options.port}`);
  }

  private handleConnection(ws: WebSocket, req: IncomingMessage): void {
    const clientId = this.generateClientId();

    const client: ClientConnection = {
      id: clientId,
      ws,
      subscriptions: new Set(),
      lastHeartbeat: Date.now(),
      isAlive: true,
    };

    this.clients.set(clientId, client);

    ws.on('message', (data) => this.handleMessage(clientId, data));
    ws.on('pong', () => this.handlePong(clientId));
    ws.on('close', () => this.handleDisconnection(clientId));
    ws.on('error', (error) => this.handleError(clientId, error));

    console.log(`Client ${clientId} connected. Total clients: ${this.clients.size}`);
  }

  private handleMessage(clientId: string, data: WebSocket.Data): void {
    try {
      const message = JSON.parse(data.toString()) as WSMessage;

      switch (message.type) {
        case 'subscribe':
          this.handleSubscription(clientId, message);
          break;
        case 'unsubscribe':
          this.handleUnsubscription(clientId, message);
          break;
        case 'pong':
          this.handlePong(clientId);
          break;
        default:
          this.sendError(clientId, 'UNKNOWN_MESSAGE_TYPE', 'Unknown message type');
      }
    } catch (error) {
      this.sendError(clientId, 'INVALID_MESSAGE', 'Failed to parse message');
    }
  }

  private handleSubscription(clientId: string, message: SubscriptionMessage): void {
    const client = this.clients.get(clientId);
    if (!client) return;

    const roomKey = this.getRoomKey(message.channel, message.filters);

    // Add client to room
    if (!this.rooms.has(roomKey)) {
      this.rooms.set(roomKey, new Set());
    }
    this.rooms.get(roomKey)!.add(clientId);

    // Update client subscriptions
    client.subscriptions.add(roomKey);

    // Send confirmation
    this.sendToClient(clientId, {
      type: 'subscribed',
      channel: message.channel,
      requestId: message.requestId,
    });

    console.log(`Client ${clientId} subscribed to ${roomKey}`);
  }

  private handleUnsubscription(clientId: string, message: UnsubscriptionMessage): void {
    const client = this.clients.get(clientId);
    if (!client) return;

    const roomKey = this.getRoomKey(message.channel);

    // Remove client from room
    this.rooms.get(roomKey)?.delete(clientId);

    // Update client subscriptions
    client.subscriptions.delete(roomKey);

    // Send confirmation
    this.sendToClient(clientId, {
      type: 'unsubscribed',
      channel: message.channel,
      requestId: message.requestId,
    });

    console.log(`Client ${clientId} unsubscribed from ${roomKey}`);
  }

  private handlePong(clientId: string): void {
    const client = this.clients.get(clientId);
    if (client) {
      client.isAlive = true;
      client.lastHeartbeat = Date.now();
    }
  }

  private handleDisconnection(clientId: string): void {
    const client = this.clients.get(clientId);
    if (!client) return;

    // Remove from all rooms
    for (const roomKey of client.subscriptions) {
      this.rooms.get(roomKey)?.delete(clientId);
    }

    this.clients.delete(clientId);

    console.log(`Client ${clientId} disconnected. Total clients: ${this.clients.size}`);
  }

  private handleError(clientId: string, error: Error): void {
    console.error(`WebSocket error for client ${clientId}:`, error);
  }

  private sendHeartbeats(): void {
    const now = Date.now();

    for (const [clientId, client] of this.clients.entries()) {
      if (client.isAlive === false) {
        // Client didn't respond to last ping - terminate
        console.log(`Terminating inactive client ${clientId}`);
        client.ws.terminate();
        this.clients.delete(clientId);
      } else {
        // Send ping
        client.isAlive = false;
        client.ws.ping();
      }
    }
  }

  /**
   * Broadcast update to all clients in a room
   */
  broadcast(channel: string, data: any, filters?: any): void {
    const roomKey = this.getRoomKey(channel, filters);
    const clientIds = this.rooms.get(roomKey);

    if (!clientIds || clientIds.size === 0) return;

    const message: DataUpdateMessage = {
      type: 'update',
      channel,
      data,
      timestamp: Date.now(),
    };

    // Queue message for delivery
    this.messageQueue.enqueue({
      recipients: Array.from(clientIds),
      message,
    });

    // Process queue
    this.processMessageQueue();
  }

  private processMessageQueue(): void {
    while (!this.messageQueue.isEmpty()) {
      const item = this.messageQueue.dequeue();
      if (!item) break;

      for (const clientId of item.recipients) {
        this.sendToClient(clientId, item.message);
      }
    }
  }

  private sendToClient(clientId: string, message: any): void {
    const client = this.clients.get(clientId);
    if (!client || client.ws.readyState !== WebSocket.OPEN) return;

    try {
      client.ws.send(JSON.stringify(message));
    } catch (error) {
      console.error(`Failed to send message to client ${clientId}:`, error);
    }
  }

  private sendError(clientId: string, code: string, message: string): void {
    const errorMsg: ErrorMessage = {
      type: 'error',
      code,
      message,
    };
    this.sendToClient(clientId, errorMsg);
  }

  private getRoomKey(channel: string, filters?: any): string {
    if (!filters) return channel;

    const filterStr = Object.entries(filters)
      .map(([k, v]) => `${k}=${v}`)
      .sort()
      .join('&');

    return `${channel}:${filterStr}`;
  }

  private generateClientId(): string {
    return `client-${Date.now()}-${Math.random().toString(36).substr(2, 9)}`;
  }

  close(): void {
    clearInterval(this.heartbeatInterval);

    for (const client of this.clients.values()) {
      client.ws.close();
    }

    this.wss.close();
  }
}

interface ClientConnection {
  id: string;
  ws: WebSocket;
  subscriptions: Set<string>;
  lastHeartbeat: number;
  isAlive: boolean;
}

class MessageQueue {
  private queue: Array<{ recipients: string[]; message: any }>;
  private maxSize: number;

  constructor(options: { maxSize: number }) {
    this.queue = [];
    this.maxSize = options.maxSize;
  }

  enqueue(item: { recipients: string[]; message: any }): void {
    if (this.queue.length >= this.maxSize) {
      // Drop oldest message
      this.queue.shift();
    }
    this.queue.push(item);
  }

  dequeue(): { recipients: string[]; message: any } | undefined {
    return this.queue.shift();
  }

  isEmpty(): boolean {
    return this.queue.length === 0;
  }
}

6.2 Event Stream Integration

// src/visualization/server/EventStream.ts

import { EventEmitter } from 'events';
import { TelemetryCollector } from '../../telemetry/collector';
import { VisualizationWebSocketServer } from './WebSocketServer';
import { TelemetryToVisualizationConverter } from '../transformer/TelemetryToVisualizationConverter';

export class VisualizationEventStream {
  private telemetryCollector: TelemetryCollector;
  private wsServer: VisualizationWebSocketServer;
  private converter: TelemetryToVisualizationConverter;
  private eventBus: EventEmitter;

  constructor(
    telemetryCollector: TelemetryCollector,
    wsServer: VisualizationWebSocketServer
  ) {
    this.telemetryCollector = telemetryCollector;
    this.wsServer = wsServer;
    this.converter = new TelemetryToVisualizationConverter();
    this.eventBus = new EventEmitter();

    this.setupEventListeners();
  }

  private setupEventListeners(): void {
    // Listen to OpenTelemetry span completions
    this.eventBus.on('span:end', (span) => {
      this.handleSpanEnd(span);
    });

    // Listen to metric recordings
    this.eventBus.on('metric:record', (metric) => {
      this.handleMetricRecord(metric);
    });

    // Listen to agent status changes
    this.eventBus.on('agent:status', (agentId, status) => {
      this.handleAgentStatusChange(agentId, status);
    });

    // Listen to task updates
    this.eventBus.on('task:update', (taskId, update) => {
      this.handleTaskUpdate(taskId, update);
    });
  }

  private handleSpanEnd(span: any): void {
    // Convert span to agent node update
    const agentNode = this.converter.convertTraceToAgentNode(span);

    // Broadcast to WebSocket clients
    this.wsServer.broadcast('agent', {
      agentId: agentNode.id,
      status: agentNode.status,
      currentTask: null, // Updated separately
      metrics: agentNode.metrics,
    }, { agentId: agentNode.id });
  }

  private handleMetricRecord(metric: any): void {
    // Convert metric to visualization format
    const metricUpdate = {
      metricName: metric.descriptor.name,
      value: metric.dataPoints[metric.dataPoints.length - 1].value,
      attributes: metric.dataPoints[metric.dataPoints.length - 1].attributes,
      timestamp: Date.now(),
    };

    // Broadcast to WebSocket clients
    this.wsServer.broadcast('metrics', metricUpdate);
  }

  private handleAgentStatusChange(agentId: string, status: string): void {
    this.wsServer.broadcast('agent', {
      agentId,
      status,
      currentTask: null,
      metrics: {},
    }, { agentId });
  }

  private handleTaskUpdate(taskId: string, update: any): void {
    this.wsServer.broadcast('tasks', {
      taskId,
      status: update.status,
      progress: update.progress || 0,
      agentId: update.agentId,
      timestamp: Date.now(),
    }, { taskId });
  }

  /**
   * Emit event from telemetry system
   */
  emit(event: string, ...args: any[]): void {
    this.eventBus.emit(event, ...args);
  }
}

---

7. Performance Requirements

7.1 Latency Requirements

Operation	Target	Maximum
Real-time WebSocket update	<100ms	200ms
Graph rendering (100 nodes)	<100ms	300ms
Graph rendering (1000 nodes)	<500ms	1000ms
API response (cached)	<50ms	100ms
API response (uncached)	<200ms	500ms
Chart rendering (1000 points)	<200ms	500ms
Initial page load	<2s	5s
Time to interactive	<3s	7s

7.2 Throughput Requirements

Metric	Target	Maximum
WebSocket connections	1000 concurrent	2000 concurrent
WebSocket messages/sec	1000 msg/s	5000 msg/s
API requests/min	1000 req/min	5000 req/min
Graph updates/sec	10 updates/s	50 updates/s
Chart data points	10,000 points	50,000 points

7.3 Resource Constraints

Resource	Limit	Rationale
Backend memory	512 MB	Runs alongside AQE Fleet
Cache size	500 MB	In-memory LRU cache
Frontend bundle size	<500 KB (gzipped)	Fast initial load
Database query time	<100ms	Responsive UI
WebSocket message size	1 MB	Prevent DoS

7.4 Optimization Strategies

7.4.1 Backend Optimizations

1. Query Optimization

   • Index all frequently queried columns (agent_id, task_id, timestamp)
   • Use prepared statements to prevent SQL injection and improve performance
   • Implement query result caching (5-minute TTL for real-time, 1-hour for historical)
   • Parallel query execution for multi-metric requests

2. Caching Strategy

   • In-memory LRU cache (500 MB limit)
   • Cache invalidation on WebSocket updates
   • Cache warming on server startup (preload common queries)

3. WebSocket Optimizations

   • Message batching (combine multiple updates into one message)
   • Delta updates (send only changed fields, not entire objects)
   • Client-side throttling (max 10 updates/second per client)

4. Data Transformer Optimizations

   • Lazy transformation (transform on-demand, not all data upfront)
   • Stream processing for large datasets
   • Parallel transformation using worker threads

7.4.2 Frontend Optimizations

1. React Performance

   • Virtualized lists for large datasets (react-window)
   • Memoization of expensive components (React.memo)
   • Lazy loading of routes (React.lazy + Suspense)
   • Code splitting by route

2. Graph Rendering

   • WebGL renderer for >1000 nodes
   • Level-of-detail rendering (simplify distant nodes)
   • Viewport culling (only render visible nodes)
   • Incremental layout updates (don't recalculate entire layout on every change)

3. Chart Optimizations

   • Data downsampling for large time series (e.g., 10,000 points → 500 points)
   • Canvas-based rendering for >5000 points
   • Debounced resize handlers
   • Lazy rendering (render charts only when visible)

4. Network Optimizations

   • HTTP/2 server push for critical resources
   • Resource preloading ()
   • Service worker for offline support
   • Brotli compression for static assets

---

8. Security & Access Control

8.1 Authentication

// src/visualization/server/auth/AuthMiddleware.ts

import { Request, Response, NextFunction } from 'express';
import jwt from 'jsonwebtoken';

const JWT_SECRET = process.env.JWT_SECRET || 'your-secret-key';

export function authMiddleware(req: Request, res: Response, next: NextFunction): void {
  const authHeader = req.headers.authorization;

  if (!authHeader || !authHeader.startsWith('Bearer ')) {
    res.status(401).json({ error: 'Missing or invalid authorization header' });
    return;
  }

  const token = authHeader.substring(7);

  try {
    const decoded = jwt.verify(token, JWT_SECRET) as JWTPayload;

    // Attach user info to request
    (req as any).user = {
      id: decoded.userId,
      email: decoded.email,
      role: decoded.role,
      permissions: decoded.permissions,
    };

    next();
  } catch (error) {
    res.status(401).json({ error: 'Invalid or expired token' });
  }
}

interface JWTPayload {
  userId: string;
  email: string;
  role: 'admin' | 'user' | 'viewer';
  permissions: string[];
}

8.2 Authorization (RBAC)

// src/visualization/server/auth/AuthorizationMiddleware.ts

import { Request, Response, NextFunction } from 'express';

export function requirePermission(permission: string) {
  return (req: Request, res: Response, next: NextFunction): void => {
    const user = (req as any).user;

    if (!user) {
      res.status(401).json({ error: 'Authentication required' });
      return;
    }

    if (user.role === 'admin' || user.permissions.includes(permission)) {
      next();
    } else {
      res.status(403).json({ error: 'Insufficient permissions' });
    }
  };
}

// Role-based permissions
export const PERMISSIONS = {
  VIEW_METRICS: 'metrics:view',
  VIEW_AGENTS: 'agents:view',
  VIEW_TASKS: 'tasks:view',
  VIEW_TOPOLOGY: 'topology:view',
  MANAGE_FLEET: 'fleet:manage',
  ADMIN: 'admin',
} as const;

// Usage in routes:
// router.get('/api/v1/metrics/quality', authMiddleware, requirePermission(PERMISSIONS.VIEW_METRICS), ...);

8.3 Data Sanitization

// src/visualization/server/validation/Sanitizer.ts

import Ajv from 'ajv';
import addFormats from 'ajv-formats';

const ajv = new Ajv({ allErrors: true, removeAdditional: 'all' });
addFormats(ajv);

// Request validation schemas
const timeSeriesRequestSchema = {
  type: 'object',
  properties: {
    metricNames: {
      type: 'array',
      items: { type: 'string', pattern: '^aqe\\.' },
      minItems: 1,
      maxItems: 10,
    },
    timeRange: {
      type: 'object',
      properties: {
        start: { type: 'integer', minimum: 0 },
        end: { type: 'integer', minimum: 0 },
      },
      required: ['start', 'end'],
    },
    granularity: {
      type: 'string',
      enum: ['minute', 'hour', 'day'],
    },
  },
  required: ['metricNames', 'timeRange', 'granularity'],
};

export const validateTimeSeriesRequest = ajv.compile(timeSeriesRequestSchema);

// Middleware
export function validateRequest(schema: any) {
  return (req: Request, res: Response, next: NextFunction): void => {
    const validate = ajv.compile(schema);
    const valid = validate(req.body);

    if (!valid) {
      res.status(400).json({
        error: 'Validation failed',
        details: validate.errors,
      });
      return;
    }

    next();
  };
}

8.4 Rate Limiting

// src/visualization/server/middleware/RateLimiter.ts

import { Request, Response, NextFunction } from 'express';

export class RateLimiter {
  private requests: Map<string, number[]>;
  private windowMs: number;
  private maxRequests: number;

  constructor(options: {
    windowMs: number; // Time window in ms
    maxRequests: number; // Max requests per window
  }) {
    this.requests = new Map();
    this.windowMs = options.windowMs;
    this.maxRequests = options.maxRequests;

    // Cleanup old entries every minute
    setInterval(() => this.cleanup(), 60000);
  }

  middleware() {
    return (req: Request, res: Response, next: NextFunction): void => {
      const key = this.getClientKey(req);
      const now = Date.now();

      // Get request timestamps for this client
      const timestamps = this.requests.get(key) || [];

      // Remove timestamps outside the window
      const validTimestamps = timestamps.filter(ts => now - ts < this.windowMs);

      if (validTimestamps.length >= this.maxRequests) {
        res.status(429).json({
          error: 'Rate limit exceeded',
          retryAfter: this.windowMs / 1000,
        });
        return;
      }

      // Add current timestamp
      validTimestamps.push(now);
      this.requests.set(key, validTimestamps);

      next();
    };
  }

  private getClientKey(req: Request): string {
    // Use user ID if authenticated, otherwise IP address
    const user = (req as any).user;
    return user?.id || req.ip || 'unknown';
  }

  private cleanup(): void {
    const now = Date.now();

    for (const [key, timestamps] of this.requests.entries()) {
      const validTimestamps = timestamps.filter(ts => now - ts < this.windowMs);

      if (validTimestamps.length === 0) {
        this.requests.delete(key);
      } else {
        this.requests.set(key, validTimestamps);
      }
    }
  }
}

// Usage:
// const rateLimiter = new RateLimiter({ windowMs: 60000, maxRequests: 100 });
// app.use('/api', rateLimiter.middleware());

---

9. Deployment Architecture

9.1 Deployment Options

Option 1: Embedded Server (Recommended for MVP)

┌────────────────────────────────────────┐
│        AQE Fleet Process               │
├────────────────────────────────────────┤
│  ┌──────────────────────────────────┐ │
│  │  AQE Fleet Core                  │ │
│  │  (Agents, Tasks, Learning)       │ │
│  └──────────────────────────────────┘ │
│  ┌──────────────────────────────────┐ │
│  │  Telemetry Layer                 │ │
│  │  (OpenTelemetry SDK)             │ │
│  └──────────────────────────────────┘ │
│  ┌──────────────────────────────────┐ │
│  │  Visualization Server            │ │
│  │  (Express + WebSocket)           │ │
│  │  Port: 3000                      │ │
│  └──────────────────────────────────┘ │
│  ┌──────────────────────────────────┐ │
│  │  Static File Server              │ │
│  │  (Frontend Build)                │ │
│  │  Port: 3000                      │ │
│  └──────────────────────────────────┘ │
└────────────────────────────────────────┘

Pros:

• Single process, easy deployment
• No network latency between components
• Simple configuration

Cons:

• Resource contention (CPU, memory)
• No independent scaling

Option 2: Separate Processes

┌─────────────────┐       ┌──────────────────┐
│  AQE Fleet      │──────>│  Visualization   │
│  Process        │       │  Server          │
│  (Port: N/A)    │       │  (Port: 3000)    │
│                 │       │                  │
│  └─ Telemetry ──┼──────>│  └─ WebSocket    │
│     Exporter    │       │     Server       │
└─────────────────┘       └──────────────────┘
         │                         │
         v                         v
┌─────────────────────────────────────────┐
│           AgentDB (SQLite)              │
└─────────────────────────────────────────┘

Pros:

• Independent scaling
• Process isolation (better fault tolerance)
• Can run on different machines

Cons:

• More complex deployment
• Network latency between processes

9.2 Docker Deployment

# Dockerfile for Visualization Server

FROM node:18-alpine AS builder

WORKDIR /app

# Copy package files
COPY package.json package-lock.json ./
RUN npm ci

# Copy source
COPY src ./src
COPY tsconfig.json ./

# Build TypeScript
RUN npm run build

# Build frontend
WORKDIR /app/src/visualization/frontend
RUN npm ci && npm run build

# Production image
FROM node:18-alpine

WORKDIR /app

# Copy built files
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/src/visualization/frontend/dist ./public
COPY --from=builder /app/node_modules ./node_modules
COPY package.json ./

# Environment variables
ENV NODE_ENV=production
ENV PORT=3000

EXPOSE 3000

CMD ["node", "dist/visualization/server/index.js"]

# docker-compose.yml

version: '3.8'

services:
  aqe-fleet:
    build:
      context: .
      dockerfile: Dockerfile.fleet
    environment:
      - NODE_ENV=production
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
      - OTLP_ENDPOINT=http://visualization:4318
    volumes:
      - ./data:/app/data
    depends_on:
      - visualization

  visualization:
    build:
      context: .
      dockerfile: Dockerfile.visualization
    ports:
      - "3000:3000"
    environment:
      - NODE_ENV=production
      - JWT_SECRET=${JWT_SECRET}
    volumes:
      - ./data:/app/data

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./nginx.conf:/etc/nginx/nginx.conf
      - ./ssl:/etc/nginx/ssl
    depends_on:
      - visualization

---

10. Monitoring & Observability

10.1 Visualization System Metrics

The visualization system should export its own telemetry:

// src/visualization/telemetry/VisualizationTelemetry.ts

import { metrics, trace } from '@opentelemetry/api';

const meter = metrics.getMeter('aqe-visualization');
const tracer = trace.getTracer('aqe-visualization');

// Metrics
export const visualizationMetrics = {
  // WebSocket metrics
  activeConnections: meter.createUpDownCounter('viz.websocket.connections.active'),
  messagesReceived: meter.createCounter('viz.websocket.messages.received'),
  messagesSent: meter.createCounter('viz.websocket.messages.sent'),
  messageLatency: meter.createHistogram('viz.websocket.message.latency'),

  // API metrics
  apiRequests: meter.createCounter('viz.api.requests'),
  apiLatency: meter.createHistogram('viz.api.latency'),
  apiErrors: meter.createCounter('viz.api.errors'),

  // Cache metrics
  cacheHits: meter.createCounter('viz.cache.hits'),
  cacheMisses: meter.createCounter('viz.cache.misses'),
  cacheSize: meter.createUpDownCounter('viz.cache.size.bytes'),

  // Transformer metrics
  transformDuration: meter.createHistogram('viz.transform.duration'),
  transformedSpans: meter.createCounter('viz.transform.spans'),

  // Frontend metrics (sent via API)
  pageLoad: meter.createHistogram('viz.frontend.page.load'),
  graphRenderTime: meter.createHistogram('viz.frontend.graph.render'),
  chartRenderTime: meter.createHistogram('viz.frontend.chart.render'),
};

// Trace spans
export function recordAPIRequest(endpoint: string, handler: () => Promise<any>) {
  return tracer.startActiveSpan(`viz.api.${endpoint}`, async (span) => {
    const start = Date.now();
    try {
      const result = await handler();
      span.setStatus({ code: 1 }); // OK
      visualizationMetrics.apiRequests.add(1, { endpoint, status: 'success' });
      return result;
    } catch (error) {
      span.setStatus({ code: 2, message: error.message }); // ERROR
      visualizationMetrics.apiRequests.add(1, { endpoint, status: 'error' });
      visualizationMetrics.apiErrors.add(1, { endpoint, error: error.constructor.name });
      throw error;
    } finally {
      const duration = Date.now() - start;
      visualizationMetrics.apiLatency.record(duration, { endpoint });
      span.end();
    }
  });
}

10.2 Health Checks

// src/visualization/server/health/HealthCheck.ts

export class HealthChecker {
  async checkHealth(): Promise<HealthStatus> {
    const checks = await Promise.all([
      this.checkDatabase(),
      this.checkWebSocket(),
      this.checkMemory(),
      this.checkTelemetry(),
    ]);

    const status: HealthStatus = {
      status: checks.every(c => c.status === 'healthy') ? 'healthy' : 'degraded',
      timestamp: Date.now(),
      checks: {
        database: checks[0],
        websocket: checks[1],
        memory: checks[2],
        telemetry: checks[3],
      },
    };

    return status;
  }

  private async checkDatabase(): Promise<ComponentHealth> {
    try {
      const start = Date.now();
      // Execute simple query
      await database.query('SELECT 1');
      const latency = Date.now() - start;

      return {
        status: latency < 100 ? 'healthy' : 'degraded',
        latency,
      };
    } catch (error) {
      return {
        status: 'unhealthy',
        error: error.message,
      };
    }
  }

  private async checkWebSocket(): Promise<ComponentHealth> {
    const activeConnections = wsServer.getActiveConnections();

    return {
      status: activeConnections < 1000 ? 'healthy' : 'degraded',
      metadata: {
        activeConnections,
        maxConnections: 2000,
      },
    };
  }

  private async checkMemory(): Promise<ComponentHealth> {
    const usage = process.memoryUsage();
    const maxHeap = 512 * 1024 * 1024; // 512 MB

    return {
      status: usage.heapUsed < maxHeap * 0.8 ? 'healthy' : 'degraded',
      metadata: {
        heapUsed: usage.heapUsed,
        heapTotal: usage.heapTotal,
        maxHeap,
      },
    };
  }

  private async checkTelemetry(): Promise<ComponentHealth> {
    // Check if telemetry is exporting successfully
    const lastExport = telemetryCollector.getLastExportTime();
    const timeSinceExport = Date.now() - lastExport;

    return {
      status: timeSinceExport < 60000 ? 'healthy' : 'degraded',
      metadata: {
        lastExport,
        timeSinceExport,
      },
    };
  }
}

interface HealthStatus {
  status: 'healthy' | 'degraded' | 'unhealthy';
  timestamp: number;
  checks: Record<string, ComponentHealth>;
}

interface ComponentHealth {
  status: 'healthy' | 'degraded' | 'unhealthy';
  latency?: number;
  error?: string;
  metadata?: Record<string, any>;
}

---

11. Architecture Decision Records

ADR-001: WebSocket Library Selection

Status: Accepted

Context: We need a WebSocket library for real-time communication between backend and frontend.

Options:

1. ws (native WebSocket)
2. Socket.IO
3. uWebSockets.js

Decision: Use ws

Rationale:

• Pure WebSocket protocol (no custom transport)
• Minimal overhead (30KB vs. Socket.IO's 300KB)
• Sufficient performance (10,000+ concurrent connections)
• Modern browsers support WebSocket natively

Consequences:

• No automatic fallback to long-polling (acceptable for modern browsers)
• Must implement heartbeat mechanism manually

---

ADR-002: Graph Visualization Library

Status: Accepted

Context: We need a library to render interactive graph/network visualizations for agent topology.

Options:

1. Cytoscape.js
2. D3.js
3. vis.js

Decision: Use Cytoscape.js

Rationale:

• Purpose-built for graph visualization
• WebGL renderer for 10,000+ nodes
• 15+ built-in layout algorithms
• Active development and community

Consequences:

• Learning curve for team
• Less flexible than D3.js for custom visualizations

---

ADR-003: State Management

Status: Accepted

Context: We need to manage server state (API data) and UI state in the frontend.

Options:

1. Redux + RTK Query
2. React Query + React Context
3. Zustand

Decision: Use React Query for server state, React Context for UI state

Rationale:

• React Query excels at server state management (caching, refetching, etc.)
• React Context is built-in and sufficient for simple UI state
• No need for heavy Redux setup

Consequences:

• Two state management approaches (but for different concerns)
• Team must understand the distinction between server state and UI state

---

ADR-004: Backend Deployment

Status: Accepted

Context: We need to decide how to deploy the visualization server relative to the AQE Fleet.

Options:

1. Embedded in AQE Fleet process
2. Separate process (same machine)
3. Separate service (different machine)

Decision: Start with embedded, provide option for separate process

Rationale:

• MVP: Embedded is simpler (single process)
• Future: Separate process allows independent scaling
• Configuration flag to switch between modes

Consequences:

• Initial releases may have resource contention
• Must design for both deployment modes from the start

---

12. Next Steps

Phase 3 Implementation Plan

Task	Owner	Duration	Dependencies
V4: Data Transformer	Backend Team	1 week	Phase 2 complete
V5: WebSocket Server	Backend Team	1 week	V4 complete
V6: REST API	Backend Team	1 week	V4 complete
Frontend Setup	Frontend Team	3 days	-
TopologyMindMap Component	Frontend Team	1 week	V6 complete
Metrics Dashboard	Frontend Team	1 week	V6 complete
Real-time Integration	Full Stack	1 week	V5, Frontend complete
Performance Testing	QA Team	1 week	All components
Security Audit	Security Team	3 days	All components
Documentation	Docs Team	1 week	All components

Total Estimated Duration: 6-8 weeks

Success Criteria

• V4: Data transformer passes all unit tests
• V5: WebSocket server handles 1000 concurrent connections
• V6: REST API responds in <200ms (uncached)
• Frontend: Initial page load <2s
• Graph renders 100 nodes in <100ms
• Real-time updates arrive in <100ms
• Security: All endpoints protected with auth/authorization
• Performance: All metrics meet targets (Section 7)

---

Appendix A: Type Definitions

// src/visualization/types/index.ts

export interface AgentNode {
  id: string;
  type: string;
  name: string;
  status: 'active' | 'idle' | 'error';
  position: { x: number; y: number };
  metrics: {
    taskCount: number;
    successRate: number;
    avgDuration: number;
    tokenUsage: number;
    cost: number;
  };
  metadata: {
    fleetId: string;
    topology: string;
    startTime: number;
    endTime: number | null;
  };
}

export interface CoordinationEdge {
  source: string;
  target: string;
  type: 'coordination' | 'data-flow' | 'control-flow';
  weight: number;
  metadata: {
    latency: number;
    messageCount: number;
  };
}

export interface VisualizationGraph {
  nodes: AgentNode[];
  edges: CoordinationEdge[];
}

export interface QualityMetrics {
  testPassRate: number;
  coveragePercent: {
    line: number;
    branch: number;
    function: number;
    statement?: number;
  };
  defectDensity: number;
  flakyTestCount: number;
  securityVulnerabilities: {
    critical: number;
    high: number;
    medium: number;
    low: number;
  };
  qualityGateStatus: 'passing' | 'failing' | 'warning';
  timestamp: number;
}

export interface TimeSeriesData {
  timestamp: number;
  metricName: string;
  value: number;
  attributes?: Record<string, any>;
}

export interface TimeRange {
  start: number;
  end: number;
}

---

Document End

This architecture design provides a complete blueprint for implementing the Phase 3 visualization system. All technology choices, API contracts, and architectural patterns have been documented with clear rationales.