proffesor-for-testing__agentic-qe/docs/planning/knowledge-graph-implementation-roadmap.md

Knowledge Graph Implementation Roadmap

Phased Rollout Strategy

Document Version: 1.0.0 Target Release: Q1 2026 Complexity: High-Impact, Medium-Risk

---

Visual Architecture

┌────────────────────────────────────────────────────────────────────┐
│                     PROJECT KNOWLEDGE GRAPH                         │
│                    PostgreSQL + pgvector Backend                    │
└────────────────────────────────────────────────────────────────────┘
                                  │
                ┌─────────────────┴─────────────────┐
                │                                   │
┌───────────────▼────────────────┐  ┌──────────────▼──────────────┐
│   CODE ENTITIES (Vectors)      │  │   RELATIONSHIPS (Edges)     │
├────────────────────────────────┤  ├─────────────────────────────┤
│ • Functions (768-dim vectors)  │  │ • IMPORTS (module → module) │
│ • Classes (semantic embeddings)│  │ • TESTS (test → function)   │
│ • Modules (aggregated context) │  │ • DOCUMENTS (doc → code)    │
│ • Variables (usage patterns)   │  │ • IMPLEMENTS (code → iface) │
│ • Interfaces (contract vectors)│  │ • CALLS (func → func)       │
└────────────────────────────────┘  └─────────────────────────────┘
                │                                   │
                └─────────────────┬─────────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │    QUERY INTERFACES       │
                    ├───────────────────────────┤
                    │ • Semantic Search (HNSW)  │
                    │ • Graph Traversal (CTE)   │
                    │ • Impact Analysis (BFS)   │
                    │ • Context Building (LLM)  │
                    └───────────────────────────┘
                                  │
                    ┌─────────────▼─────────────┐
                    │   INTEGRATION LAYER       │
                    ├───────────────────────────┤
                    │ • BaseAgent Mixin         │
                    │ • CLI Commands (aqe kg)   │
                    │ • File Watcher (chokidar) │
                    │ • MCP Tools (91 tools)    │
                    └───────────────────────────┘

---

Phase 0: Preparation (Week 0)

Pre-Flight Checklist

Goal: Validate infrastructure readiness

• RuVector Health Check

  aqe ruvector status
  # Expected: Connected, Healthy, Extension Loaded
  

• PostgreSQL Performance Baseline

  -- Current pattern search latency
  SELECT AVG(ruvector_cosine_distance(embedding, $1)) FROM qe_patterns;
  -- Target: <1ms
  

• Storage Capacity Planning

  # Estimate entities: src files * avg 10 entities/file
  find src -name "*.ts" | wc -l
  # Example: 500 files * 10 = 5000 entities * 3.6KB = ~18MB
  

• TypeScript Compiler API Test

  // Verify AST parsing works
  import * as ts from 'typescript';
  const program = ts.createProgram(['test.ts'], {});
  const sourceFile = program.getSourceFile('test.ts');
  console.log(sourceFile?.statements.length); // Should print statement count
  

Deliverable: Go/No-Go decision document

---

Phase 1: Foundation (Week 1-2)

Milestone 1.1: Database Schema Extension

Owner: Database/Backend Engineer Effort: 2 days Dependencies: PostgreSQL admin access

Tasks:

1. Create migration script: migrations/001_knowledge_graph_schema.sql
2. Add code_entities table with pgvector column
3. Add entity_relationships table with indexes
4. Test schema with sample data
5. Document schema in docs/database/knowledge-graph-schema.md

Validation:

-- Verify tables exist
SELECT table_name FROM information_schema.tables
WHERE table_schema = 'public'
  AND table_name IN ('code_entities', 'entity_relationships');

-- Verify indexes
SELECT indexname FROM pg_indexes
WHERE tablename = 'code_entities';
-- Expected: idx_entities_type, idx_entities_file, idx_entities_embedding

Rollback Plan: Drop tables if needed (data loss acceptable in dev)

---

Milestone 1.2: AST Parser Service

Owner: Core Developer Effort: 3 days Dependencies: TypeScript compiler API knowledge

Tasks:

1. Create src/code-intelligence/ASTParser.ts
2. Implement function extraction with JSDoc
3. Implement class extraction with members
4. Implement interface extraction
5. Implement import/export extraction
6. Write unit tests (>80% coverage)

Validation:

# Test on sample file
npx tsx src/code-intelligence/cli-test-parser.ts src/agents/BaseAgent.ts

# Expected output:
# Found 15 entities:
#   - class BaseAgent (line 50-200)
#   - function execute (line 75-100)
#   - interface AgentConfig (line 10-25)

Risk Mitigation:

• Use proven TypeScript compiler API patterns
• Handle edge cases (arrow functions, default exports)
• Graceful degradation for unparseable files

---

Milestone 1.3: Embedding Generator

Owner: ML/Vector Engineer Effort: 2 days Dependencies: Embedding API access

Tasks:

1. Create src/code-intelligence/EmbeddingGenerator.ts
2. Integrate with sentence-transformers or OpenAI embeddings
3. Implement semantic text builder (code + docs + signature)
4. Add caching layer (Map or Redis)
5. Benchmark embedding quality

Validation:

// Test semantic similarity
const func1 = "function calculateTax(income: number): number { return income * 0.2; }";
const func2 = "function computeTax(salary: number): number { return salary * 0.2; }";

const emb1 = await embedder.generateEmbedding({ content: func1, ... });
const emb2 = await embedder.generateEmbedding({ content: func2, ... });

const similarity = cosineSimilarity(emb1, emb2);
console.log(similarity); // Expected: >0.9 (very similar semantically)

Embedding Options:

Provider	Dimensions	Cost/1M	Latency	Quality
OpenAI text-embedding-3-small	768	$0.02	~50ms	⭐⭐⭐⭐
Sentence-Transformers (local)	768	Free	~10ms	⭐⭐⭐
Anthropic (if available)	768	TBD	TBD	⭐⭐⭐⭐⭐

Recommendation: Start with sentence-transformers (local, free), migrate to OpenAI if quality issues.

---

Phase 2: Relationship Extraction (Week 3-4)

Milestone 2.1: Dependency Graph Builder

Owner: Graph Engineer Effort: 3 days Dependencies: AST Parser complete

Tasks:

1. Create src/code-intelligence/DependencyGraphBuilder.ts
2. Extract import statements → IMPORTS relationships
3. Extract export statements → EXPORTS relationships
4. Resolve module paths (handle aliases, node_modules)
5. Handle circular dependencies gracefully

Validation:

// Test on real project
const deps = await depBuilder.buildDependencyGraph(allEntities);

// Verify UserService imports Database
const userServiceDeps = deps.filter(r =>
  r.sourceId.includes('UserService') && r.type === 'IMPORTS'
);

console.log(userServiceDeps);
// Expected: [{sourceId: 'UserService', targetId: 'Database', type: 'IMPORTS'}]

Edge Cases:

• Dynamic imports (import(...))
• Re-exports (export * from './module')
• Type-only imports (import type { ... })

---

Milestone 2.2: Test Coverage Mapper

Owner: QE/Testing Engineer Effort: 4 days Dependencies: Dependency Graph Builder

Tasks:

1. Create src/code-intelligence/TestCoverageMapper.ts
2. Implement name-based matching (80% accuracy)
3. Implement import-based matching (90% accuracy)
4. Implement semantic similarity matching (70% accuracy, high recall)
5. Combine strategies with weighted confidence scores
6. Validate against known test-to-code mappings

Validation:

// Test on agentic-qe-cf project
const testMappings = await testMapper.mapTestsToCode(testEntities, codeEntities);

// Find tests for RuVectorPostgresAdapter
const adapterTests = testMappings.filter(r =>
  r.targetId.includes('RuVectorPostgresAdapter') && r.type === 'TESTS'
);

console.log(adapterTests);
// Expected: 3 test files found with confidence >0.8

Benchmark Dataset:

• Use existing agentic-qe-cf tests (543 files)
• Manual validation on 50 random files
• Precision/Recall targets: P>90%, R>85%

---

Milestone 2.3: Documentation Linker

Owner: Documentation Engineer Effort: 2 days Dependencies: AST Parser

Tasks:

1. Create src/code-intelligence/DocumentationLinker.ts
2. Extract JSDoc comments → link to functions/classes
3. Parse Markdown files → extract code references
4. Link README mentions to code entities
5. Detect stale documentation (code changed, doc didn't)

Validation:

// Test JSDoc linking
const jsdocLinks = await docLinker.linkDocumentation(docEntities, codeEntities);

// Find docs for BaseAgent
const agentDocs = jsdocLinks.filter(r =>
  r.targetId.includes('BaseAgent') && r.type === 'DOCUMENTS'
);

console.log(agentDocs);
// Expected: JSDoc comment above BaseAgent class linked

---

Phase 3: Indexing Pipeline (Week 5-6)

Milestone 3.1: Full Project Indexer

Owner: Performance Engineer Effort: 4 days Dependencies: All extractors complete

Tasks:

1. Create src/code-intelligence/IncrementalIndexer.ts
2. Implement parallel file processing (10 workers)
3. Implement batch embedding generation (100/batch)
4. Optimize database inserts (use COPY for bulk)
5. Add progress bar (show files/sec, ETA)
6. Handle errors gracefully (skip unparseable files)

Validation:

# Index agentic-qe-cf project
time aqe kg index

# Target performance:
# - 500 TypeScript files
# - ~5000 entities
# - <30 seconds total
# - ~167 entities/sec (meets <5s per 100 files target)

Optimization Techniques:

• Worker threads for AST parsing
• Batch PostgreSQL inserts (1000 rows at a time)
• Connection pooling (10 concurrent connections)
• Embedding cache (avoid re-embedding unchanged code)

---

Milestone 3.2: File Watcher Integration

Owner: DevOps Engineer Effort: 2 days Dependencies: Indexer working

Tasks:

1. Integrate chokidar for file watching
2. Implement incremental reindexing (single file)
3. Debounce rapid changes (e.g., during mass refactor)
4. Handle file deletions (cascade delete entities)
5. Handle file renames (update file_path references)

Validation:

# Start watcher
aqe kg index --watch &

# In another terminal, modify file
echo "// New comment" >> src/agents/BaseAgent.ts

# Expected: Watcher detects change, reindexes in <1s
# Output: "✅ Reindexed BaseAgent.ts in 847ms"

Debouncing Strategy:

• Wait 500ms after last change before reindexing
• Batch multiple file changes into single reindex
• Skip .git directory, node_modules, dist

---

Milestone 3.3: Query API Implementation

Owner: API Engineer Effort: 3 days Dependencies: Database populated

Tasks:

1. Create src/code-intelligence/KnowledgeGraphQueryAPI.ts
2. Implement semantic search (vector similarity)
3. Implement graph traversal (1-hop, n-hop)
4. Implement impact analysis (recursive CTE)
5. Implement context builder for agents
6. Optimize SQL queries (use EXPLAIN ANALYZE)

Validation:

// Test semantic search
const results = await api.findSimilarCode("database connection pooling");

console.log(results[0]);
// Expected: RuVectorPostgresAdapter.pool (high similarity)

// Test graph traversal
const deps = await api.findDependencies('UserService');

console.log(deps.length);
// Expected: >5 dependencies (Database, Logger, etc.)

// Test impact analysis
const impact = await api.analyzeImpact('BaseAgent', 2);

console.log(impact.directDependents.length);
// Expected: >20 agents that extend BaseAgent

Performance Targets:

• Semantic search: <10ms
• Graph traversal (1-hop): <5ms
• Impact analysis (3-hop): <50ms

---

Phase 4: Integration (Week 7-8)

Milestone 4.1: Agent Integration

Owner: Agent Framework Lead Effort: 3 days Dependencies: Query API ready

Tasks:

1. Create src/agents/KnowledgeGraphContextBuilder.ts
2. Add mixin to BaseAgent for knowledge graph access
3. Implement intelligent context retrieval
4. Add fallback for when knowledge graph disabled
5. Measure LLM token reduction
6. Update agent documentation

Validation:

// Enable knowledge graph
process.env.AQE_KNOWLEDGE_GRAPH_ENABLED = 'true';

// Execute agent with KG
const agent = new TestGeneratorAgent();
const result = await agent.execute('Generate tests for UserService.createUser');

// Check context size
console.log(result.contextTokens);
// Expected: ~2K tokens (vs ~10K without KG)
// LLM cost savings: 80% ✅

Integration Points:

Agent	KG Usage	Expected Benefit
TestGeneratorAgent	Find similar tests	Reuse test structure
CoverageAnalyzerAgent	Find uncovered code	Semantic gap detection
SecurityScannerAgent	Find vulnerable patterns	Cross-file vulnerability tracking
FlakyTestHunterAgent	Find code-test links	Correlate code changes with flakiness

---

Milestone 4.2: CLI Commands

Owner: CLI Engineer Effort: 2 days Dependencies: All APIs ready

Tasks:

1. Add aqe kg command group
2. Implement aqe kg index (with --watch, --force)
3. Implement aqe kg query <natural-language>
4. Implement aqe kg stats (entity counts, relationships)
5. Implement aqe kg impact <file-path>
6. Write CLI documentation

Validation:

# Test all commands
aqe kg index
aqe kg stats
aqe kg query "find database connection handling"
aqe kg impact src/providers/RuVectorClient.ts

# Expected: All commands succeed with useful output

CLI UX Requirements:

• Progress indicators for long operations
• Colorized output (success=green, warning=yellow, error=red)
• JSON output mode (--json) for scripting
• Helpful error messages with troubleshooting steps

---

Milestone 4.3: Documentation & Examples

Owner: Technical Writer Effort: 2 days Dependencies: All features complete

Tasks:

1. Write docs/guides/knowledge-graph-user-guide.md
2. Create examples: examples/knowledge-graph-queries.ts
3. Update README.md with knowledge graph section
4. Create video tutorial (5 min demo)
5. Add troubleshooting guide

Documentation Outline:

# Knowledge Graph User Guide

## Quick Start
- Installation
- Initial indexing
- Basic queries

## Concepts
- Code entities
- Relationships
- Semantic search

## Advanced Usage
- Impact analysis
- Agent integration
- Custom queries

## Troubleshooting
- Indexing errors
- Query performance
- Storage optimization

---

Post-Launch (Week 9+)

Continuous Improvement

Monitoring:

• Track LLM cost savings (target: 70-81%)
• Monitor query latency (target: <10ms p50)
• Measure agent satisfaction (qualitative feedback)
• Track adoption rate (% of agents using KG)

Optimization Opportunities:

1. Embedding Model Fine-Tuning

   • Collect user feedback on search quality
   • Fine-tune sentence-transformers on code corpus
   • Target: +10% search accuracy

2. Relationship Quality Improvement

   • Machine learning for test-to-code mapping
   • Graph embedding for improved link prediction
   • Target: 95%+ precision/recall

3. Storage Optimization

   • Quantize embeddings to 256-dim
   • Use halfvec (float16) for 50% reduction
   • Prune stale entities (not accessed in 6 months)

4. Advanced Features

   • Code clone detection (high similarity search)
   • API migration assistance (find all usages)
   • Dead code detection (no incoming relationships)
   • Architectural drift analysis (expected vs actual deps)

---

Risk Register

Risk ID	Description	Probability	Impact	Mitigation	Owner
R1	Initial indexing exceeds 5s per 100 files	Medium	Medium	Optimize parallelization, use faster embedding API	Performance Engineer
R2	Test-to-code mapping accuracy <80%	Medium	High	Implement ML-based approach, collect training data	QE Engineer
R3	Storage costs exceed budget (>1GB per 10K files)	Low	Medium	Use quantization, prune old entities	Database Engineer
R4	Agent adoption <50% after 3 months	Medium	High	Improve documentation, add success stories, training	Product Manager
R5	Query latency >10ms degrades UX	Low	High	Optimize SQL queries, add query caching	API Engineer
R6	File watcher misses changes in edge cases	Low	Low	Add health check, fallback to polling	DevOps Engineer

---

Success Metrics Dashboard

Week 1-2 (Foundation)

• Database schema created
• AST parser extracts >90% of entities
• Embedding generation <100ms per entity

Week 3-4 (Relationships)

• Dependency graph 100% accurate
• Test coverage mapper >80% accuracy
• Documentation links created

Week 5-6 (Indexing)

• Full project indexed in <30s
• File watcher updates in <1s
• Query API latency <10ms

Week 7-8 (Integration)

• Agent context size reduced by 80%
• CLI commands functional
• Documentation complete

Post-Launch (Week 9+)

• 70-81% LLM cost reduction achieved
• >50% agent adoption
• <1% error rate

---

Deployment Strategy

Development (Week 1-6)

# Local development with Docker
docker run -d ruvnet/ruvector:latest
export AQE_KNOWLEDGE_GRAPH_ENABLED=true
export AQE_KNOWLEDGE_GRAPH_DEV=true  # Enable debug logging

# Run indexer
aqe kg index --watch

# Test queries
aqe kg query "database connection pooling"

Staging (Week 7)

# Deploy to staging environment
kubectl apply -f k8s/knowledge-graph-staging.yaml

# Index production-like codebase
aqe kg index --environment staging

# Run acceptance tests
npm run test:kg:acceptance

Production (Week 8)

# Gradual rollout
# Phase 1: 10% of users
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=10

# Phase 2: 50% after 3 days
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=50

# Phase 3: 100% after 1 week
export AQE_KNOWLEDGE_GRAPH_ROLLOUT_PERCENT=100

---

Rollback Plan

If Critical Issues Arise

Immediate Rollback (within 1 hour):

# Disable knowledge graph globally
export AQE_KNOWLEDGE_GRAPH_ENABLED=false

# Agents automatically fallback to traditional context building
# No data loss, immediate recovery

Partial Rollback (specific agents):

// In agent code
class ProblematicAgent extends BaseAgent {
  constructor() {
    super();
    this.knowledgeGraphEnabled = false; // Override for this agent only
  }
}

Data Preservation:

• Knowledge graph tables remain intact
• Can re-enable at any time
• No migration needed for rollback

---

Team Responsibilities

Role	Responsibilities	Time Commitment
Project Lead	Overall coordination, stakeholder communication	25% (10h/week)
Database Engineer	Schema design, query optimization	75% (30h/week, Week 1-3)
Core Developer	AST parsing, embedding generation	100% (40h/week, Week 1-4)
Graph Engineer	Relationship extraction, graph algorithms	75% (30h/week, Week 3-5)
Performance Engineer	Indexing optimization, benchmarking	75% (30h/week, Week 5-6)
Agent Framework Lead	Agent integration, context building	100% (40h/week, Week 7-8)
QE Engineer	Test coverage mapping, validation	50% (20h/week, Week 3-4)
Technical Writer	Documentation, examples	50% (20h/week, Week 7-8)

Total Effort: ~8 person-weeks

---

Budget Estimate

Category	Cost	Notes
Development	$40,000	2 engineers @ $100/h * 200h
Infrastructure	$500	PostgreSQL hosting (3 months)
Embedding API	$100	OpenAI embeddings (initial indexing)
Testing/QA	$5,000	QA engineer time
Documentation	$3,000	Technical writer
Contingency (20%)	$9,720	Risk buffer
Total	$58,320	8-week project

ROI Analysis:

• LLM cost savings: $1,000/month (70% reduction)
• Payback period: ~58 months
• Long-term value: Code intelligence platform foundation

---

Conclusion

This roadmap provides a detailed, week-by-week plan for implementing the project knowledge graph. The phased approach ensures:

1. Incremental Value: Each phase delivers usable functionality
2. Risk Mitigation: Early validation catches issues before full rollout
3. Team Coordination: Clear responsibilities and dependencies
4. Measurable Progress: Success metrics at each milestone
5. Flexibility: Can pause/adjust based on learnings

Recommended Next Steps:

1. ✅ Review roadmap with stakeholders
2. ✅ Allocate team resources
3. ✅ Create sprint boards (Jira/GitHub Projects)
4. ✅ Kick off Week 0 preparation
5. ✅ Schedule weekly syncs

Let's build the future of code intelligence! 🚀

---

Document Metadata:

• Version: 1.0.0
• Last Updated: 2025-12-21
• Next Review: After Phase 1 completion
• Approvers: Project Lead, CTO, Head of Engineering