proffesor-for-testing__agentic-qe/docs/planning/knowledge-graph-implementation-roadmap.md

# Knowledge Graph Implementation Roadmap
## Phased Rollout Strategy

**Document Version:** 1.0.0
**Target Release:** Q1 2026
**Complexity:** High-Impact, Medium-Risk

---

## Visual Architecture

```
┌────────────────────────────────────────────────────────────────────┐
│                     PROJECT KNOWLEDGE GRAPH                         │
│                    PostgreSQL + pgvector Backend                    │
└────────────────────────────────────────────────────────────────────┘
                                  │
                ┌─────────────────┴─────────────────┐
                │                                   │
┌───────────────▼────────────────┐  ┌──────────────▼──────────────┐
│   CODE ENTITIES (Vectors)      │  │   RELATIONSHIPS (Edges)     │
├────────────────────────────────┤  ├─────────────────────────────┤
│ • Functions (768-dim vectors)  │  │ • IMPORTS (module → module) │
│ • Classes (semantic embeddings)│  │ • TESTS (test → function)   │
│ • Modules (aggregated context) │  │ • DOCUMENTS (doc → code)    │
│ • Variables (usage patterns)   │  │ • IMPLEMENTS (code → iface) │
│ • Interfaces (contract vectors)│  │ • CALLS (func → func)       │
└────────────────────────────────┘  └─────────────────────────────┘
                │                                   │
                └─────────────────┬─────────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │    QUERY INTERFACES       │
                    ├───────────────────────────┤
                    │ • Semantic Search (HNSW)  │
                    │ • Graph Traversal (CTE)   │
                    │ • Impact Analysis (BFS)   │
                    │ • Context Building (LLM)  │
                    └───────────────────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │   INTEGRATION LAYER       │
                    ├───────────────────────────┤
                    │ • BaseAgent Mixin         │
                    │ • CLI Commands (aqe kg)   │
                    │ • File Watcher (chokidar) │
                    │ • MCP Tools (91 tools)    │
                    └───────────────────────────┘
```

---

## Phase 0: Preparation (Week 0)

### Pre-Flight Checklist

**Goal**: Validate infrastructure readiness

- [ ] **RuVector Health Check**
  ```bash
  aqe ruvector status
  # Expected: Connected, Healthy, Extension Loaded
  ```

- [ ] **PostgreSQL Performance Baseline**
  ```sql
  -- Current pattern search latency
  SELECT AVG(ruvector_cosine_distance(embedding, $1)) FROM qe_patterns;
  -- Target: <1ms
  ```

- [ ] **Storage Capacity Planning**
  ```bash
  # Estimate entities: src files * avg 10 entities/file
  find src -name "*.ts" | wc -l
  # Example: 500 files * 10 = 5000 entities * 3.6KB = ~18MB
  ```

- [ ] **TypeScript Compiler API Test**
  ```typescript
  // Verify AST parsing works
  import * as ts from 'typescript';
  const program = ts.createProgram(['test.ts'], {});
  const sourceFile = program.getSourceFile('test.ts');
  console.log(sourceFile?.statements.length); // Should print statement count
  ```

**Deliverable**: Go/No-Go decision document

---

## Phase 1: Foundation (Week 1-2)

### Milestone 1.1: Database Schema Extension

**Owner**: Database/Backend Engineer
**Effort**: 2 days
**Dependencies**: PostgreSQL admin access

**Tasks**:
1. Create migration script: `migrations/001_knowledge_graph_schema.sql`
2. Add `code_entities` table with pgvector column
3. Add `entity_relationships` table with indexes
4. Test schema with sample data
5. Document schema in `docs/database/knowledge-graph-schema.md`

**Validation**:
```sql
-- Verify tables exist
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
  AND table_name IN ('code_entities', 'entity_relationships');

-- Verify indexes
SELECT indexname FROM pg_indexes
WHERE tablename = 'code_entities';
-- Expected: idx_entities_type, idx_entities_file, idx_entities_embedding
```

**Rollback Plan**: Drop tables if needed (data loss acceptable in dev)

---

### Milestone 1.2: AST Parser Service

**Owner**: Core Developer
**Effort**: 3 days
**Dependencies**: TypeScript compiler API knowledge

**Tasks**:
1. Create `src/code-intelligence/ASTParser.ts`
2. Implement function extraction with JSDoc
3. Implement class extraction with members
4. Implement interface extraction
5. Implement import/export extraction
6. Write unit tests (>80% coverage)

**Validation**:
```bash
# Test on sample file
npx tsx src/code-intelligence/cli-test-parser.ts src/agents/BaseAgent.ts

# Expected output:
# Found 15 entities:
#   - class BaseAgent (line 50-200)
#   - function execute (line 75-100)
#   - interface AgentConfig (line 10-25)
```

**Risk Mitigation**:
- Use proven TypeScript compiler API patterns
- Handle edge cases (arrow functions, default exports)
- Graceful degradation for unparseable files

---

### Milestone 1.3: Embedding Generator

**Owner**: ML/Vector Engineer
**Effort**: 2 days
**Dependencies**: Embedding API access

**Tasks**:
1. Create `src/code-intelligence/EmbeddingGenerator.ts`
2. Integrate with sentence-transformers or OpenAI embeddings
3. Implement semantic text builder (code + docs + signature)
4. Add caching layer (Map or Redis)
5. Benchmark embedding quality

**Validation**:
```typescript
// Test semantic similarity
const func1 = "function calculateTax(income: number): number { return income * 0.2; }";
const func2 = "function computeTax(salary: number): number { return salary * 0.2; }";

const emb1 = await embedder.generateEmbedding({ content: func1, ... });
const emb2 = await embedder.generateEmbedding({ content: func2, ... });

const similarity = cosineSimilarity(emb1, emb2);
console.log(similarity); // Expected: >0.9 (very similar semantically)
```

**Embedding Options**:
| Provider | Dimensions | Cost/1M | Latency | Quality |
|----------|-----------|---------|---------|---------|
| OpenAI `text-embedding-3-small` | 768 | $0.02 | ~50ms | ⭐⭐⭐⭐ |
| Sentence-Transformers (local) | 768 | Free | ~10ms | ⭐⭐⭐ |
| Anthropic (if available) | 768 | TBD | TBD | ⭐⭐⭐⭐⭐ |

**Recommendation**: Start with sentence-transformers (local, free), migrate to OpenAI if quality issues.

---

## Phase 2: Relationship Extraction (Week 3-4)

### Milestone 2.1: Dependency Graph Builder

**Owner**: Graph Engineer
**Effort**: 3 days
**Dependencies**: AST Parser complete

**Tasks**:
1. Create `src/code-intelligence/DependencyGraphBuilder.ts`
2. Extract `import` statements → `IMPORTS` relationships
3. Extract `export` statements → `EXPORTS` relationships
4. Resolve module paths (handle aliases, node_modules)
5. Handle circular dependencies gracefully

**Validation**:
```typescript
// Test on real project
const deps = await depBuilder.buildDependencyGraph(allEntities);

// Verify UserService imports Database
const userServiceDeps = deps.filter(r =>
  r.sourceId.includes('UserService') && r.type === 'IMPORTS'
);

console.log(userServiceDeps);
// Expected: [{sourceId: 'UserService', targetId: 'Database', type: 'IMPORTS'}]
```

**Edge Cases**:
- Dynamic imports (`import(...)`)
- Re-exports (`export * from './module'`)
- Type-only imports (`import type { ... }`)

---

### Milestone 2.2: Test Coverage Mapper

**Owner**: QE/Testing Engineer
**Effort**: 4 days
**Dependencies**: Dependency Graph Builder

**Tasks**:
1. Create `src/code-intelligence/TestCoverageMapper.ts`
2. Implement name-based matching (80% accuracy)
3. Implement import-based matching (90% accuracy)
4. Implement semantic similarity matching (70% accuracy, high recall)
5. Combine strategies with weighted confidence scores
6. Validate against known test-to-code mappings

**Validation**:
```typescript
// Test on agentic-qe-cf project
const testMappings = await testMapper.mapTestsToCode(testEntities, codeEntities);

// Find tests for RuVectorPostgresAdapter
const adapterTests = testMappings.filter(r =>
  r.targetId.includes('RuVectorPostgresAdapter') && r.type === 'TESTS'
);

console.log(adapterTests);
// Expected: 3 test files found with confidence >0.8
```

**Benchmark Dataset**:
- Use existing agentic-qe-cf tests (543 files)
- Manual validation on 50 random files
- Precision/Recall targets: P>90%, R>85%

---

### Milestone 2.3: Documentation Linker

**Owner**: Documentation Engineer
**Effort**: 2 days
**Dependencies**: AST Parser

**Tasks**:
1. Create `src/code-intelligence/DocumentationLinker.ts`
2. Extract JSDoc comments → link to functions/classes
3. Parse Markdown files → extract code references
4. Link README mentions to code entities
5. Detect stale documentation (code changed, doc didn't)

**Validation**:
```typescript
// Test JSDoc linking
const jsdocLinks = await docLinker.linkDocumentation(docEntities, codeEntities);

// Find docs for BaseAgent
const agentDocs = jsdocLinks.filter(r =>
  r.targetId.includes('BaseAgent') && r.type === 'DOCUMENTS'
);

console.log(agentDocs);
// Expected: JSDoc comment above BaseAgent class linked
```

---

## Phase 3: Indexing Pipeline (Week 5-6)

### Milestone 3.1: Full Project Indexer

**Owner**: Performance Engineer
**Effort**: 4 days
**Dependencies**: All extractors complete

**Tasks**:
1. Create `src/code-intelligence/IncrementalIndexer.ts`
2. Implement parallel file processing (10 workers)
3. Implement batch embedding generation (100/batch)
4. Optimize database inserts (use `COPY` for bulk)
5. Add progress bar (show files/sec, ETA)
6. Handle errors gracefully (skip unparseable files)

**Validation**:
```bash
# Index agentic-qe-cf project
time aqe kg index

# Target performance:
# - 500 TypeScript files
# - ~5000 entities
# - <30 seconds total
# - ~167 entities/sec (meets <5s per 100 files target)
```

**Optimization Techniques**:
- Worker threads for AST parsing
- Batch PostgreSQL inserts (1000 rows at a time)
- Connection pooling (10 concurrent connections)
- Embedding cache (avoid re-embedding unchanged code)

---

### Milestone 3.2: File Watcher Integration

**Owner**: DevOps Engineer
**Effort**: 2 days
**Dependencies**: Indexer working

**Tasks**:
1. Integrate `chokidar` for file watching
2. Implement incremental reindexing (single file)
3. Debounce rapid changes (e.g., during mass refactor)
4. Handle file deletions (cascade delete entities)
5. Handle file renames (update file_path references)

**Validation**:
```bash
# Start watcher
aqe kg index --watch &

# In another terminal, modify file
echo "// New comment" >> src/agents/BaseAgent.ts

# Expected: Watcher detects change, reindexes in <1s
# Output: "✅ Reindexed BaseAgent.ts in 847ms"
```

**Debouncing Strategy**:
- Wait 500ms after last change before reindexing
- Batch multiple file changes into single reindex
- Skip `.git` directory, `node_modules`, `dist`

---

### Milestone 3.3: Query API Implementation

**Owner**: API Engineer
**Effort**: 3 days
**Dependencies**: Database populated

**Tasks**:
1. Create `src/code-intelligence/KnowledgeGraphQueryAPI.ts`
2. Implement semantic search (vector similarity)
3. Implement graph traversal (1-hop, n-hop)
4. Implement impact analysis (recursive CTE)
5. Implement context builder for agents
6. Optimize SQL queries (use EXPLAIN ANALYZE)

**Validation**:
```typescript
// Test semantic search
const results = await api.findSimilarCode("database connection pooling");

console.log(results[0]);
// Expected: RuVectorPostgresAdapter.pool (high similarity)

// Test graph traversal
const deps = await api.findDependencies('UserService');

console.log(deps.length);
// Expected: >5 dependencies (Database, Logger, etc.)

// Test impact analysis
const impact = await api.analyzeImpact('BaseAgent', 2);

console.log(impact.directDependents.length);
// Expected: >20 agents that extend BaseAgent
```

**Performance Targets**:
- Semantic search: <10ms
- Graph traversal (1-hop): <5ms
- Impact analysis (3-hop): <50ms

---

## Phase 4: Integration (Week 7-8)

### Milestone 4.1: Agent Integration

**Owner**: Agent Framework Lead
**Effort**: 3 days
**Dependencies**: Query API ready

**Tasks**:
1. Create `src/agents/KnowledgeGraphContextBuilder.ts`
2. Add mixin to `BaseAgent` for knowledge graph access
3. Implement intelligent context retrieval
4. Add fallback for when knowledge graph disabled
5. Measure LLM token reduction
6. Update agent documentation

**Validation**:
```typescript
// Enable knowledge graph
process.env.AQE_KNOWLEDGE_GRAPH_ENABLED = 'true';

// Execute agent with KG
const agent = new TestGeneratorAgent();
const result = await agent.execute('Generate tests for UserService.createUser');

// Check context size
console.log(result.contextTokens);
// Expected: ~2K tokens (vs ~10K without KG)
// LLM cost savings: 80% ✅
```

**Integration Points**:
| Agent | KG Usage | Expected Benefit |
|-------|----------|------------------|
| TestGeneratorAgent | Find similar tests | Reuse test structure |
| CoverageAnalyzerAgent | Find uncovered code | Semantic gap detection |
| SecurityScannerAgent | Find vulnerable patterns | Cross-file vulnerability tracking |
| FlakyTestHunterAgent | Find code-test links | Correlate code changes with flakiness |

---

### Milestone 4.2: CLI Commands

**Owner**: CLI Engineer
**Effort**: 2 days
**Dependencies**: All APIs ready

**Tasks**:
1. Add `aqe kg` command group
2. Implement `aqe kg index` (with --watch, --force)
3. Implement `aqe kg query <natural-language>`
4. Implement `aqe kg stats` (entity counts, relationships)
5. Implement `aqe kg impact <file-path>`
6. Write CLI documentation

**Validation**:
```bash
# Test all commands
aqe kg index
aqe kg stats
aqe kg query "find database connection handling"
aqe kg impact src/providers/RuVectorClient.ts

# Expected: All commands succeed with useful output
```

**CLI UX Requirements**:
- Progress indicators for long operations
- Colorized output (success=green, warning=yellow, error=red)
- JSON output mode (`--json`) for scripting
- Helpful error messages with troubleshooting steps

---

### Milestone 4.3: Documentation & Examples

**Owner**: Technical Writer
**Effort**: 2 days
**Dependencies**: All features complete

**Tasks**:
1. Write `docs/guides/knowledge-graph-user-guide.md`
2. Create examples: `examples/knowledge-graph-queries.ts`
3. Update `README.md` with knowledge graph section
4. Create video tutorial (5 min demo)
5. Add troubleshooting guide

**Documentation Outline**:
```markdown
# Knowledge Graph User Guide

## Quick Start
- Installation
- Initial indexing
- Basic queries

## Concepts
- Code entities
- Relationships
- Semantic search

## Advanced Usage
- Impact analysis
- Agent integration
- Custom queries

## Troubleshooting
- Indexing errors
- Query performance
- Storage optimization
```

---

## Post-Launch (Week 9+)

### Continuous Improvement

**Monitoring**:
- Track LLM cost savings (target: 70-81%)
- Monitor query latency (target: <10ms p50)
- Measure agent satisfaction (qualitative feedback)
- Track adoption rate (% of agents using KG)

**Optimization Opportunities**:
1. **Embedding Model Fine-Tuning**
   - Collect user feedback on search quality
   - Fine-tune sentence-transformers on code corpus
   - Target: +10% search accuracy

2. **Relationship Quality Improvement**
   - Machine learning for test-to-code mapping
   - Graph embedding for improved link prediction
   - Target: 95%+ precision/recall

3. **Storage Optimization**
   - Quantize embeddings to 256-dim
   - Use halfvec (float16) for 50% reduction
   - Prune stale entities (not accessed in 6 months)

4. **Advanced Features**
   - Code clone detection (high similarity search)
   - API migration assistance (find all usages)
   - Dead code detection (no incoming relationships)
   - Architectural drift analysis (expected vs actual deps)

---

## Risk Register

| Risk ID | Description | Probability | Impact | Mitigation | Owner |
|---------|-------------|-------------|--------|------------|-------|
| R1 | Initial indexing exceeds 5s per 100 files | Medium | Medium | Optimize parallelization, use faster embedding API | Performance Engineer |
| R2 | Test-to-code mapping accuracy <80% | Medium | High | Implement ML-based approach, collect training data | QE Engineer |
| R3 | Storage costs exceed budget (>1GB per 10K files) | Low | Medium | Use quantization, prune old entities | Database Engineer |
| R4 | Agent adoption <50% after 3 months | Medium | High | Improve documentation, add success stories, training | Product Manager |
| R5 | Query latency >10ms degrades UX | Low | High | Optimize SQL queries, add query caching | API Engineer |
| R6 | File watcher misses changes in edge cases | Low | Low | Add health check, fallback to polling | DevOps Engineer |

---

## Success Metrics Dashboard

### Week 1-2 (Foundation)
- [ ] Database schema created
- [ ] AST parser extracts >90% of entities
- [ ] Embedding generation <100ms per entity

### Week 3-4 (Relationships)
- [ ] Dependency graph 100% accurate
- [ ] Test coverage mapper >80% accuracy
- [ ] Documentation links created

### Week 5-6 (Indexing)
- [ ] Full project indexed in <30s
- [ ] File watcher updates in <1s
- [ ] Query API latency <10ms

### Week 7-8 (Integration)
- [ ] Agent context size reduced by 80%
- [ ] CLI commands functional
- [ ] Documentation complete

### Post-Launch (Week 9+)
- [ ] 70-81% LLM cost reduction achieved
- [ ] >50% agent adoption
- [ ] <1% error rate

---

## Deployment Strategy

### Development (Week 1-6)
```bash
# Local development with Docker
docker run -d ruvnet/ruvector:latest
export AQE_KNOWLEDGE_GRAPH_ENABLED=true
export AQE_KNOWLEDGE_GRAPH_DEV=true  # Enable debug logging

# Run indexer
aqe kg index --watch

# Test queries
aqe kg query "database connection pooling"
```

### Staging (Week 7)
```bash
# Deploy to staging environment
kubectl apply -f k8s/knowledge-graph-staging.yaml

# Index production-like codebase
aqe kg index --environment staging

# Run acceptance tests
npm run test:kg:acceptance
```

### Production (Week 8)
```bash
# Gradual rollout
# Phase 1: 10% of users
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=10

# Phase 2: 50% after 3 days
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=50

# Phase 3: 100% after 1 week
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=100
```

---

## Rollback Plan

### If Critical Issues Arise

**Immediate Rollback** (within 1 hour):
```bash
# Disable knowledge graph globally
export AQE_KNOWLEDGE_GRAPH_ENABLED=false

# Agents automatically fallback to traditional context building
# No data loss, immediate recovery
```

**Partial Rollback** (specific agents):
```typescript
// In agent code
class ProblematicAgent extends BaseAgent {
  constructor() {
    super();
    this.knowledgeGraphEnabled = false; // Override for this agent only
  }
}
```

**Data Preservation**:
- Knowledge graph tables remain intact
- Can re-enable at any time
- No migration needed for rollback

---

## Team Responsibilities

| Role | Responsibilities | Time Commitment |
|------|-----------------|-----------------|
| **Project Lead** | Overall coordination, stakeholder communication | 25% (10h/week) |
| **Database Engineer** | Schema design, query optimization | 75% (30h/week, Week 1-3) |
| **Core Developer** | AST parsing, embedding generation | 100% (40h/week, Week 1-4) |
| **Graph Engineer** | Relationship extraction, graph algorithms | 75% (30h/week, Week 3-5) |
| **Performance Engineer** | Indexing optimization, benchmarking | 75% (30h/week, Week 5-6) |
| **Agent Framework Lead** | Agent integration, context building | 100% (40h/week, Week 7-8) |
| **QE Engineer** | Test coverage mapping, validation | 50% (20h/week, Week 3-4) |
| **Technical Writer** | Documentation, examples | 50% (20h/week, Week 7-8) |

**Total Effort**: ~8 person-weeks

---

## Budget Estimate

| Category | Cost | Notes |
|----------|------|-------|
| **Development** | $40,000 | 2 engineers @ $100/h * 200h |
| **Infrastructure** | $500 | PostgreSQL hosting (3 months) |
| **Embedding API** | $100 | OpenAI embeddings (initial indexing) |
| **Testing/QA** | $5,000 | QA engineer time |
| **Documentation** | $3,000 | Technical writer |
| **Contingency (20%)** | $9,720 | Risk buffer |
| **Total** | **$58,320** | 8-week project |

**ROI Analysis**:
- LLM cost savings: $1,000/month (70% reduction)
- Payback period: ~58 months
- Long-term value: Code intelligence platform foundation

---

## Conclusion

This roadmap provides a detailed, week-by-week plan for implementing the project knowledge graph. The phased approach ensures:

1. **Incremental Value**: Each phase delivers usable functionality
2. **Risk Mitigation**: Early validation catches issues before full rollout
3. **Team Coordination**: Clear responsibilities and dependencies
4. **Measurable Progress**: Success metrics at each milestone
5. **Flexibility**: Can pause/adjust based on learnings

**Recommended Next Steps**:
1. ✅ Review roadmap with stakeholders
2. ✅ Allocate team resources
3. ✅ Create sprint boards (Jira/GitHub Projects)
4. ✅ Kick off Week 0 preparation
5. ✅ Schedule weekly syncs

Let's build the future of code intelligence! 🚀

---

**Document Metadata**:
- **Version**: 1.0.0
- **Last Updated**: 2025-12-21
- **Next Review**: After Phase 1 completion
- **Approvers**: Project Lead, CTO, Head of Engineering