mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
1e3fbb5ea8
* SwarmMemoryManager Refactoring Plan * fix: add missing directories to npm package (v1.9.3) Added templates/, .claude/helpers/, and docs/reference/ to package.json files array. These were missing from the npm distribution, causing `aqe init` to fail with "Template file not found" errors. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * checkpoint improvements * fix: critical memory/learning system fixes for v1.9.4 ## Critical Fixes - Vector embeddings now stored correctly (was storing NULL) - SQL parameter style fixed for agentdb's SqlJsDatabase wrapper - HNSW index schema mismatch resolved with pattern_id column - Hooks now save to correct database (.agentic-qe/agentdb.db) ## New Features - Added 5 SwarmMemoryManager methods for learning data retrieval: - getBestAction() - Q-learning best action selection - getRecentLearningExperiences() - Recent experience retrieval - getLearningExperiencesByTaskType() - Task-filtered experiences - getHighRewardExperiences() - Successful experience extraction - getLearningStats() - Aggregate learning statistics - Added comprehensive integration test for memory/learning loop - Added RuVector pattern store integration (experimental) ## Technical Details - Vector embeddings: 384 dimensions × 4 bytes = 1,536 bytes per pattern - All 12 integration tests pass - HNSW index: 150x faster vector search enabled 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: move ARM64-only ruvector packages to optionalDependencies Fixes CI failures caused by platform-specific dependencies: - @ruvector/node-linux-arm64-gnu (ARM64 only) - ruvector-core-linux-arm64-gnu (ARM64 only) These packages caused npm ci to fail on x64 CI runners with: "EBADPLATFORM - wanted linux/arm64, current linux/x64" Solution: Move ARM64 packages to optionalDependencies and add x64 variants for cross-platform compatibility. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * chore: regenerate package-lock.json with x64 ruvector packages Updates lockfile to include optional x64 platform packages for CI compatibility: - @ruvector/node-linux-x64-gnu@0.1.17 - ruvector-core-linux-x64-gnu@0.1.15 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: replace fake QUIC with proper implementation and clean up skills manifest QUIC Transport: - Remove fake HTTP/2-based QUICTransport.ts that pretended to be QUIC - Add proper QUIC implementation via Rust/WASM (@agentic-flow/quic-wasm) - Add WebSocket fallback transport for when WASM unavailable - Add dedicated type declarations for optional WASM module - Features: 0-RTT connection, stream multiplexing, TLS 1.3 built-in Skills Manifest: - Rewrite skills-manifest.json with only QE skills (41 skills) - Remove Claude Flow platform skills (agentdb-*, flow-nexus-*, github-*, etc.) - Organize into proper categories: qe-core, testing-methodologies, etc. Bug Fixes: - Fix UnifiedMemoryCoordinator.ts TypeScript errors - Use Logger.getInstance() instead of private constructor - Use swarmMemory.query() instead of non-existent list() method - Use correct close() method signatures for backends 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: add QX Partner agent, compact subagents, fix CLAUDE.md dynamic counts - Add QX Partner agent for Quality Experience analysis combining QA + UX perspectives - Compact 9 subagent definitions to ~80 lines each (88% reduction) - Fix CLAUDE.md template to dynamically count agents (including subagents) and skills - Move CLAUDE.md creation after parallel phases for accurate counts - Remove 'Production-Ready' claim from README and CHANGELOG - Clean up backup files and move docs to proper directory Initialization now correctly reports: - 31 QE Agents (20 main + 11 subagents) - 53 QE Skills - 8 Slash Commands 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * fix: resolve CI failures and security alert #36 - Replace MD5 with SHA-256 for ETag calculation (security alert #36) - Fix CRLF line endings in shell scripts causing parse errors - Remove 'local' keyword from for-loop in test-integration-batched.sh - Use dynamic process.cwd() instead of hardcoded path in jest.config.js - Enhance quality-gate.ts to parse test logs when results file missing 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: Complete Agentic QE Fleet v2.1 implementation (closes #101) ## Implemented Features ### Error Recovery Enhancement (Task 1.3) - CircuitBreaker class with state management (CLOSED/OPEN/HALF_OPEN) - RetryStrategy with exponential/linear/constant backoff - RecoveryOrchestrator with 6 recovery strategies - Transport fallback mechanisms ### Dynamic Skill Loader (Task 2.2) - Metadata-only initial loading (98% token reduction) - On-demand full skill retrieval - LRU caching with configurable TTL (30min default) - Preload suggestions for critical workflows ### Swarm Learning Optimizer (Task 4.1) - Topology recommendation engine (hierarchical/mesh/ring/star) - Agent allocation with capability matching and load balancing - Bottleneck detection (agent/memory/coordination/IO) - Real-time performance adjustments - Added getLatestRecommendation() method ### Workflow Orchestrator (Task 4.2) - AdaptiveScheduler with dynamic strategy selection - Priority-based task queue with age boosting - Task dependency resolution - Work stealing and backpressure handling - Added getQueueDepth() and getStatus() methods ### Unified Memory Coordinator (Task 4.3) - Consolidated memory managers (SQLite, AgentDB, Vector, JSON) - Automatic failover chain with health monitoring - Batch operations and namespace isolation - Periodic sync between backends ### SwarmCoordinator Improvements - Real CPU utilization tracking via process.cpuUsage() - Queue depth from WorkflowOrchestrator - Optimizer recommendations in feedback data - Calculated average complexity from metrics ## Additional Changes - Migration documentation and test restructuring plan - GitHub issue templates and workflows - Test suite improvements and cleanup 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * feat: optimize 38 QE skills with agent-focused format (#102) - Add enhanced YAML frontmatter (tokenEstimate, agents, tags, etc.) - Add <default_to_action> blocks for immediate actionable guidance - Add Quick Reference Cards with tables for at-a-glance info - Add Agent Integration examples using Task() pattern - Add Agent Coordination Hints (memory namespace + fleet coordination) - Condense content optimized for agent consumption - Reduce total token footprint by ~65% across skills Skills updated: - Core QE: agentic-quality-engineering, holistic-testing-pact - Testing Types: api-testing, chaos-engineering, compatibility, compliance, contract, database, exploratory, localization, mobile, mutation, performance, regression, security, visual-testing - Methodologies: tdd-london-chicago, xp-practices, context-driven, risk-based, shift-left, shift-right - Analysis: code-review-quality, quality-metrics, test-design-techniques - Strategy: test-automation-strategy, test-data-management, test-environment-management, test-reporting-analytics - CI/CD: cicd-pipeline-qe-orchestrator, accessibility-testing - Communication: brutal-honesty-review, bug-reporting-excellence, consultancy-practices, sherlock-review, six-thinking-hats, technical-writing - Refactoring: refactoring-patterns Also includes: package.json updates, docs/optimization/, docs/templates/, scripts/validate-skill-optimization.ts Skipped: testability-scoring (pending PR from @fndlalit) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> * release: v2.1.0 - Comprehensive QX Analysis & Skills Optimization ## Summary - Comprehensive QX Analysis with 23+ heuristics (PR #104 by @fndlalit) - 38 QE Skills optimized with agent-focused format (PR #102) - Testability Scoring skill v2.1 with optimized format - QX Partner Agent v2.1 with enhanced capabilities - Fix MCP SDK compatibility for @modelcontextprotocol/sdk bump (PR #105) ## Changes - .claude/agents/qx-partner.md: Updated to v2.1 with domain-specific detection - .claude/skills/testability-scoring/SKILL.md: Optimized format (59% token reduction) - src/mcp/server.ts: Removed deprecated 'description' field for MCP SDK compatibility - package.json: Bumped version to 2.1.0 - README.md: Updated version and release notes - CHANGELOG.md: Added v2.1.0 release notes 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
342 lines
10 KiB
TypeScript
342 lines
10 KiB
TypeScript
/**
|
|
* Tests for NeuralCapableMixin
|
|
*
|
|
* Validates neural pattern matching, prediction, caching, and error handling.
|
|
*/
|
|
|
|
import {
|
|
DefaultNeuralMatcher,
|
|
createNeuralMatcher,
|
|
safeNeuralPredict,
|
|
mergeWithNeuralPrediction,
|
|
getNeuralMetrics,
|
|
NeuralInput,
|
|
NeuralConfig,
|
|
DEFAULT_NEURAL_CONFIG
|
|
} from '@agents/mixins/NeuralCapableMixin';
|
|
|
|
describe('NeuralCapableMixin', () => {
|
|
describe('DefaultNeuralMatcher', () => {
|
|
let matcher: DefaultNeuralMatcher;
|
|
|
|
beforeEach(() => {
|
|
matcher = new DefaultNeuralMatcher({ enabled: true });
|
|
});
|
|
|
|
test('should initialize with default config', () => {
|
|
const status = matcher.getStatus();
|
|
expect(status.available).toBe(true);
|
|
expect(status.predictions).toBe(0);
|
|
});
|
|
|
|
test('should make flakiness predictions', async () => {
|
|
const input: NeuralInput = {
|
|
type: 'flakiness',
|
|
data: {
|
|
testName: 'test_example',
|
|
results: [
|
|
{ testName: 'test_example', duration: 100, passed: true, timestamp: new Date() },
|
|
{ testName: 'test_example', duration: 120, passed: false, timestamp: new Date() },
|
|
{ testName: 'test_example', duration: 110, passed: true, timestamp: new Date() },
|
|
{ testName: 'test_example', duration: 105, passed: true, timestamp: new Date() },
|
|
{ testName: 'test_example', duration: 115, passed: false, timestamp: new Date() }
|
|
]
|
|
}
|
|
};
|
|
|
|
const prediction = await matcher.predict(input);
|
|
|
|
expect(prediction).toBeDefined();
|
|
expect(prediction).not.toBeNull();
|
|
expect(prediction.result).toBeDefined();
|
|
expect(prediction.confidence).toBeGreaterThan(0);
|
|
expect(prediction.confidence).toBeLessThanOrEqual(1);
|
|
expect(typeof prediction.result.isFlaky).toBe('boolean');
|
|
expect(prediction.result.confidence).toBeGreaterThan(0);
|
|
expect(prediction.timestamp).toBeInstanceOf(Date);
|
|
expect(prediction.reasoning).toBeInstanceOf(Array);
|
|
expect(prediction.reasoning.length).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('should make test generation predictions', async () => {
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: {
|
|
codeSignature: { functionName: 'calculateSum' },
|
|
framework: 'jest'
|
|
}
|
|
};
|
|
|
|
const prediction = await matcher.predict(input);
|
|
|
|
expect(prediction).toBeDefined();
|
|
expect(prediction.result).toHaveProperty('suggestedTests');
|
|
expect(prediction.result.suggestedTests).toBeInstanceOf(Array);
|
|
expect(prediction.confidence).toBeGreaterThan(0.7);
|
|
});
|
|
|
|
test('should make coverage gap predictions', async () => {
|
|
const input: NeuralInput = {
|
|
type: 'coverage-gap',
|
|
data: {
|
|
currentCoverage: 0.65,
|
|
codebase: { files: [] }
|
|
}
|
|
};
|
|
|
|
const prediction = await matcher.predict(input);
|
|
|
|
expect(prediction).toBeDefined();
|
|
expect(prediction.result).toHaveProperty('gaps');
|
|
expect(prediction.result.gaps).toBeInstanceOf(Array);
|
|
});
|
|
|
|
test('should make risk score predictions', async () => {
|
|
const input: NeuralInput = {
|
|
type: 'risk-score',
|
|
data: {
|
|
changes: [],
|
|
historicalData: {}
|
|
}
|
|
};
|
|
|
|
const prediction = await matcher.predict(input);
|
|
|
|
expect(prediction).toBeDefined();
|
|
expect(prediction.result).toHaveProperty('riskScore');
|
|
expect(prediction.result.riskLevel).toMatch(/LOW|MEDIUM|HIGH|CRITICAL/);
|
|
});
|
|
|
|
test('should cache predictions', async () => {
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: { codeSignature: {}, framework: 'jest' }
|
|
};
|
|
|
|
// First prediction
|
|
const pred1 = await matcher.predict(input);
|
|
const status1 = matcher.getStatus();
|
|
|
|
// Second prediction (should be cached)
|
|
const pred2 = await matcher.predict(input);
|
|
const status2 = matcher.getStatus();
|
|
|
|
expect(pred1.confidence).toBe(pred2.confidence);
|
|
expect(status2.cacheHitRate).toBeGreaterThan(status1.cacheHitRate);
|
|
});
|
|
|
|
test('should respect cache TTL', async () => {
|
|
const shortCacheMatcher = new DefaultNeuralMatcher({
|
|
enabled: true,
|
|
cacheTTL: 100 // 100ms
|
|
});
|
|
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: { codeSignature: {}, framework: 'jest' }
|
|
};
|
|
|
|
// First prediction
|
|
await shortCacheMatcher.predict(input);
|
|
|
|
// Wait for cache to expire
|
|
await new Promise(resolve => setTimeout(resolve, 150));
|
|
|
|
// Should generate new prediction
|
|
const pred2 = await shortCacheMatcher.predict(input);
|
|
expect(pred2).toBeDefined();
|
|
});
|
|
|
|
test('should handle fallback on errors', async () => {
|
|
const matcher = new DefaultNeuralMatcher({
|
|
enabled: true,
|
|
fallbackEnabled: true
|
|
});
|
|
|
|
const input: NeuralInput = {
|
|
type: 'test-generation' as any,
|
|
data: null // Invalid data to trigger error
|
|
};
|
|
|
|
const prediction = await matcher.predict(input);
|
|
|
|
expect(prediction).toBeDefined();
|
|
expect(prediction.confidence).toBe(0.5); // Fallback confidence
|
|
expect(prediction.reasoning).toContain('Neural prediction unavailable');
|
|
});
|
|
});
|
|
|
|
describe('createNeuralMatcher', () => {
|
|
test('should create matcher when enabled', () => {
|
|
const matcher = createNeuralMatcher({ enabled: true });
|
|
expect(matcher).not.toBeNull();
|
|
expect(matcher?.isAvailable()).toBe(true);
|
|
});
|
|
|
|
test('should return null when disabled', () => {
|
|
const matcher = createNeuralMatcher({ enabled: false });
|
|
expect(matcher).toBeNull();
|
|
});
|
|
|
|
test('should apply custom configuration', () => {
|
|
const config: Partial<NeuralConfig> = {
|
|
enabled: true,
|
|
confidence: 0.85,
|
|
cacheEnabled: false
|
|
};
|
|
|
|
const matcher = createNeuralMatcher(config);
|
|
expect(matcher).not.toBeNull();
|
|
});
|
|
});
|
|
|
|
describe('safeNeuralPredict', () => {
|
|
test('should return null for null matcher', async () => {
|
|
const result = await safeNeuralPredict(null, {} as NeuralInput);
|
|
expect(result).toBeNull();
|
|
});
|
|
|
|
test('should return null for unavailable matcher', async () => {
|
|
const matcher = createNeuralMatcher({ enabled: false });
|
|
const result = await safeNeuralPredict(matcher, {} as NeuralInput);
|
|
expect(result).toBeNull();
|
|
});
|
|
|
|
test('should return prediction for valid matcher', async () => {
|
|
const matcher = createNeuralMatcher({ enabled: true });
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: { codeSignature: {}, framework: 'jest' }
|
|
};
|
|
|
|
const result = await safeNeuralPredict(matcher, input);
|
|
expect(result).not.toBeNull();
|
|
expect(result?.confidence).toBeGreaterThan(0);
|
|
});
|
|
|
|
test('should handle prediction errors gracefully', async () => {
|
|
const matcher = new DefaultNeuralMatcher({ enabled: true, fallbackEnabled: false });
|
|
|
|
// Mock predict to throw error
|
|
jest.spyOn(matcher, 'predict').mockRejectedValue(new Error('Test error'));
|
|
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: {}
|
|
};
|
|
|
|
const result = await safeNeuralPredict(matcher, input);
|
|
expect(result).toBeNull(); // Should return null on error
|
|
});
|
|
});
|
|
|
|
describe('mergeWithNeuralPrediction', () => {
|
|
test('should return traditional result when no neural prediction', () => {
|
|
const traditional = { value: 42 };
|
|
const merged = mergeWithNeuralPrediction(traditional, null);
|
|
|
|
expect(merged).toEqual({ value: 42 });
|
|
});
|
|
|
|
test('should include neural prediction in result', () => {
|
|
const traditional = { value: 42 };
|
|
const neural = {
|
|
result: { suggestedValue: 50 },
|
|
confidence: 0.8,
|
|
timestamp: new Date()
|
|
};
|
|
|
|
const merged = mergeWithNeuralPrediction(traditional, neural);
|
|
|
|
expect(merged.value).toBe(42);
|
|
expect(merged.neural).toBeDefined();
|
|
expect(merged.neural?.confidence).toBe(0.8);
|
|
});
|
|
|
|
test('should prefer neural with high confidence in neural-first strategy', () => {
|
|
const traditional = { value: 42 };
|
|
const neural = {
|
|
result: { value: 50 },
|
|
confidence: 0.9,
|
|
timestamp: new Date()
|
|
};
|
|
|
|
const merged = mergeWithNeuralPrediction(traditional, neural, 'neural-first');
|
|
|
|
expect(merged.value).toBe(50); // Neural value used
|
|
expect(merged.neural).toBeDefined();
|
|
});
|
|
});
|
|
|
|
describe('getNeuralMetrics', () => {
|
|
test('should return disabled metrics for null matcher', () => {
|
|
const metrics = getNeuralMetrics(null);
|
|
|
|
expect(metrics.enabled).toBe(false);
|
|
expect(metrics.predictions).toBe(0);
|
|
expect(metrics.avgConfidence).toBe(0);
|
|
expect(metrics.cacheHitRate).toBe(0);
|
|
});
|
|
|
|
test('should return metrics for active matcher', async () => {
|
|
const matcher = createNeuralMatcher({ enabled: true });
|
|
|
|
// Make some predictions
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: { codeSignature: {}, framework: 'jest' }
|
|
};
|
|
|
|
await matcher?.predict(input);
|
|
await matcher?.predict(input); // Cached
|
|
|
|
const metrics = getNeuralMetrics(matcher);
|
|
|
|
expect(metrics.enabled).toBe(true);
|
|
expect(metrics.predictions).toBeGreaterThan(0);
|
|
expect(metrics.cacheHitRate).toBeGreaterThan(0);
|
|
});
|
|
});
|
|
|
|
describe('Integration Tests', () => {
|
|
test('should work in complete workflow', async () => {
|
|
// Create matcher
|
|
const matcher = createNeuralMatcher({
|
|
enabled: true,
|
|
confidence: 0.75,
|
|
cacheEnabled: true
|
|
});
|
|
|
|
expect(matcher).not.toBeNull();
|
|
|
|
// Make prediction
|
|
const input: NeuralInput = {
|
|
type: 'test-generation',
|
|
data: {
|
|
codeSignature: { functionName: 'processData' },
|
|
framework: 'jest',
|
|
complexity: { cyclomaticComplexity: 5 }
|
|
}
|
|
};
|
|
|
|
const prediction = await safeNeuralPredict(matcher, input);
|
|
|
|
expect(prediction).not.toBeNull();
|
|
expect(prediction?.confidence).toBeGreaterThan(0.7);
|
|
|
|
// Merge with traditional analysis
|
|
const traditional = { testsGenerated: 10 };
|
|
const merged = mergeWithNeuralPrediction(traditional, prediction, 'weighted');
|
|
|
|
expect(merged.testsGenerated).toBe(10);
|
|
expect(merged.neural).toBeDefined();
|
|
|
|
// Get metrics
|
|
const metrics = getNeuralMetrics(matcher);
|
|
|
|
expect(metrics.enabled).toBe(true);
|
|
expect(metrics.predictions).toBeGreaterThan(0);
|
|
});
|
|
});
|
|
});
|