mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
2b71ae4eb8
## 🔒 Security Fixes ### eval() Removal - Critical Security Vulnerability Fixed - **File**: `src/agents/TestDataArchitectAgent.ts` - **Impact**: Eliminated arbitrary code execution risk - **Solution**: Implemented safe expression evaluator with parser-based approach - **Supports**: - Comparison operators: `===`, `!==`, `==`, `!=`, `>`, `<`, `>=`, `<=` - Logical operators: `&&`, `||` - Boolean values: `true`, `false` - Number and string comparisons - **Lines**: 102 lines added (safeEvaluateExpression + parseValue methods) ## ✨ UX Enhancements ### CLAUDE.md Append Strategy - User-Requested Feature - **File**: `src/cli/commands/init.ts` - **User Feedback**: "AQE instructions should append by default, not prepend" - **Interactive Mode**: Prompts user to choose prepend or append - **--yes Mode**: Defaults to append (less disruptive to existing configs) - **Features**: - Auto-backup existing CLAUDE.md to CLAUDE.md.backup - Clear visual separator (---) between sections - User confirmation messages - **Lines**: 49 lines modified ### CLI Skills Count Fix - Accurate Display - **File**: `src/cli/commands/skills/index.ts` - **Before**: Showed "8/17" (hardcoded, incorrect) - **After**: Shows "34/34" (dynamic, accurate) - **Solution**: Added `getTotalQESkillsCount()` method using `QE_SKILLS.flatMap()` - **Future-proof**: Auto-updates when skills added - **Lines**: 11 lines modified ## 🎓 New Agent - Cherry-Picked from PR #22 ### CodeComplexityAnalyzerAgent Integration - **Author**: @mondweep (PR #22) - **Purpose**: Educational code complexity analysis and refactoring recommendations - **Files**: 7 files, 2,758 lines of code - **Capabilities**: - Cyclomatic complexity calculation - Cognitive complexity analysis - AI-powered refactoring suggestions - Quality scoring (0-100 scale) - Memory integration (`aqe/complexity/*`) - **Test Coverage**: 529 lines, 13+ test scenarios - **Documentation**: 754 lines (README + Claude agent definition) - **Examples**: 871 lines (3 interactive examples) ## 🔧 Code Quality Improvements ### Agent Enhancements - **BaseAgent.ts**: 136 lines modified - Improved property access patterns - Better encapsulation with public accessors - Enhanced lifecycle management - **TestGeneratorAgent.ts**: 377 lines added - Comprehensive null safety improvements - Enhanced error handling - Better context validation - **QualityGateAgent.ts**: 64 lines modified - Improved context handling - Better validation logic ### MCP Handler Improvements - **test-generate.ts**: 33 lines modified (better parameter handling) - **quality-analyze.ts**: 28 lines modified (context fix) - **regression-risk-analyze.ts**: 85 lines modified (parameter aliasing fix) ### Infrastructure Additions - **AgentLifecycleManager.ts**: 274 new lines (lifecycle coordination) - **SwarmMemoryManager.ts**: 3 lines modified (improved coordination) - **AgentRegistry.ts**: 20 lines added (better agent registration) ## 🧪 Testing Enhancements ### New Integration Tests (5 Test Suites) - **fleet-management.integration.test.ts**: 491 lines - **parameter-validation.integration.test.ts**: 606 lines - **quality-analyze.integration.test.ts**: 393 lines - **regression-risk.integration.test.ts**: 505 lines - **test-execution.integration.test.ts**: 536 lines - **test-harness.ts**: 289 lines (shared test utilities) - **Total**: 2,820 lines of integration test coverage ### Enhanced Unit Tests - **BaseAgent.enhanced.test.ts**: 909 lines (comprehensive BaseAgent testing) - **TestGeneratorAgent.null-safety.test.ts**: 1,511 lines (null safety validation) - **Total**: 2,420 lines of new unit test coverage ### Test Improvements - **PredictionTools.test.ts**: 118 lines modified - **Agent.test.ts**: 11 lines modified ## 📚 Documentation Expansion (27,439 insertions) ### Release Documentation (7 documents) - **RELEASE-1.3.6-PREPARATION.md**: 443 lines (complete release checklist) - **SECURITY-SCAN-RELEASE-1.3.6.md**: 422 lines (security validation) - **THREE-ISSUE-FIXES-SUMMARY.md**: 196 lines (user-reported fixes) - **CHERRY-PICK-PR22-SUMMARY.md**: 412 lines (integration details) - **FINAL-STATUS-REPORT-2025-10-30.md**: 403 lines (status summary) - **FINAL-MCP-VERIFICATION-SUMMARY.md**: 338 lines (MCP validation) - **MCP-PRODUCTION-READINESS-REPORT.md**: 318 lines (production readiness) ### Technical Documentation (7 documents) - **MCP-TOOLS-REFERENCE.md**: 1,555 lines (complete MCP API reference) - **MCP-TOOLS-USER-GUIDE.md**: 626 lines (user guide) - **MCP-TOOLS-AUTO-GENERATED.md**: 934 lines (auto-generated docs) - **MCP-TOOLS-MIGRATION.md**: 437 lines (migration guide) - **complexity-mitigation-plan.md**: 537 lines (code quality roadmap) - **swarm-execution-report.md**: 985 lines (execution analysis) - **HONEST-IMPLEMENTATION-ANALYSIS.md**: 62 lines (transparency report) ### Fix Documentation (7 documents) - **MCP-TOOLS-COMPLETE-FIX-REPORT.md**: 619 lines (comprehensive fix report) - **three-issue-fixes.md**: 345 lines (user-reported fixes) - **quality-analyze-context-fix.md**: 195 lines - **regression-risk-parameter-aliasing.md**: 303 lines - **test-generator-agent-null-safety-fixes.md**: 422 lines - **typescript-compilation-errors-fixed-2025-10-30.md**: 273 lines - **MCP-TOOLS-FIX-SUMMARY.md**: 272 lines ### Refactoring Plans (4 documents) - **swarm-memory-manager-refactoring-plan.md**: 796 lines - **base-agent-refactoring-plan.md**: 658 lines - **refactoring-summary.md**: 404 lines - **REFACTORING_STATUS.md**: 210 lines ### Testing Documentation (6 documents) - **mcp-tools-test-report.md**: 1,260 lines (comprehensive test report) - **testing-workflow.md**: 691 lines (MCP testing guide) - **TestGeneratorAgent-test-plan.md**: 318 lines (test plan) - **coverage-analysis-2025-10-30.md**: 661 lines (coverage report) - **regression-validation-report-2025-10-30.md**: 450 lines (regression validation) - **README.md** (tests/integration/mcp): 277 lines (integration test guide) ### CI/CD Documentation (3 documents) - **mcp-cicd-pipeline.md**: 402 lines (pipeline architecture) - **mcp-pipeline-setup-summary.md**: 343 lines (setup guide) - **mcp-cicd-implementation-notes.md**: 280 lines (implementation notes) ### MCP Verification (2 documents) - **MCP-VERIFICATION-SUMMARY.md**: 286 lines - **PR-22-ANALYSIS.md**: 373 lines ### Coverage Analysis (2 documents) - **coverage-analysis-2025-10-30.md**: 661 lines - **coverage-analysis-summary.json**: 417 lines (structured data) ## 🛠️ Scripts & Automation (5 Scripts, 2,117 Lines) ### New Utility Scripts - **validate-mcp-tools.js**: 280 lines (MCP tool validation) - **generate-mcp-report.js**: 252 lines (report generation) - **generate-mcp-docs.js**: 270 lines (documentation generation) - **verify-mcp-production-readiness.js**: 564 lines (production checks) - **test-issue-fixes.js**: 224 lines (fix verification) - **mcp-tools-summary.json**: 344 lines (structured summary) ## 📊 Examples & Guides ### MCP Integration Examples (2 Examples) - **basic-test-generation.js**: 200 lines (starter example) - **quality-analysis-pipeline.js**: 360 lines (advanced workflow) ## ⚙️ CI/CD Enhancements ### GitHub Actions Workflow - **.github/workflows/mcp-tools-test.yml**: 197 lines - Automated MCP tool testing - Multi-environment validation - Production readiness checks ### Git Hooks - **.husky/pre-commit**: 14 lines (pre-commit validation) ## 📦 Configuration Updates ### Version & Dependencies - **package.json**: Version bumped to 1.3.6 - **package-lock.json**: Dependency lock updated - **README.md**: Version references updated (27 lines modified) - **.claude/settings.json**: 4 lines modified ### CHANGELOG - **CHANGELOG.md**: 37 lines added - Complete v1.3.6 changelog entry - Security fixes documented - UX enhancements documented - Testing improvements documented ## 📈 Statistics Summary ### Code Changes - **78 files changed** - **27,439 insertions** (+) - **181 deletions** (-) - **Net change**: +27,258 lines ### Breakdown by Category - **Source Code**: ~1,200 lines (agents, handlers, infrastructure) - **Tests**: ~5,240 lines (integration + unit tests) - **Documentation**: ~18,000 lines (guides, reports, plans) - **Scripts**: ~2,117 lines (automation utilities) - **Examples**: ~560 lines (MCP examples) - **CI/CD**: ~211 lines (workflows + hooks) ### Quality Metrics - ✅ **TypeScript Compilation**: 0 errors (16 errors fixed) - ✅ **Security Vulnerabilities**: 0 critical (1 fixed) - ✅ **Integration Tests**: 5 new test suites, 2,820 lines - ✅ **Unit Tests**: 2,420 new lines - ✅ **Documentation**: 100% comprehensive coverage - ✅ **Backward Compatibility**: 100% maintained ## 🎯 User Impact ### Immediate Benefits 1. **Security**: Eliminates arbitrary code execution vulnerability 2. **UX**: Less disruptive CLAUDE.md initialization 3. **Accuracy**: Correct skills count display (34/34) 4. **Quality**: CodeComplexityAnalyzerAgent for refactoring guidance 5. **Reliability**: 16 TypeScript errors resolved ### Developer Experience 1. **Better Documentation**: 18,000+ lines of guides and references 2. **Better Testing**: 5,240 new lines of test coverage 3. **Better Tooling**: 2,117 lines of automation scripts 4. **Better Examples**: MCP integration examples 5. **Better CI/CD**: Automated testing and validation ## 🔄 Migration & Compatibility ### Breaking Changes - **NONE** - Fully backward compatible ### Deprecations - **NONE** ### New Features (Opt-in) - CodeComplexityAnalyzerAgent (available via Claude Code or MCP) - Interactive CLAUDE.md placement (interactive mode only) ## ✅ Verification ### Pre-Release Checks - ✅ TypeScript compilation: 0 errors - ✅ All three user-reported fixes verified - ✅ Integration tests passing - ✅ Documentation complete - ✅ CHANGELOG updated - ✅ Version numbers updated (package.json, README.md) - ✅ Security scan: No vulnerabilities - ✅ Backward compatibility verified ### Production Readiness - ✅ MCP tools validated (54 tools) - ✅ Agent registry verified (17 agents) - ✅ Skills library verified (34 skills) - ✅ CI/CD pipeline configured - ✅ Pre-commit hooks installed ## 🙏 Acknowledgments - **@mondweep**: CodeComplexityAnalyzerAgent contribution (PR #22) - **User Feedback**: CLAUDE.md append strategy request - **User Feedback**: CLI skills count display bug report - **User Feedback**: eval() security concern ## 📝 Release Notes **Version**: 1.3.6 **Date**: 2025-10-30 **Type**: Security & Quality Patch Release **Status**: Production Ready **Upgrade Recommendation**: 🔴 **HIGH PRIORITY** - Security vulnerability fixed (eval() removal) - User experience improvements - Enhanced stability (16 TypeScript errors fixed) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
698 lines
24 KiB
Markdown
698 lines
24 KiB
Markdown
# Agentic Quality Engineering Fleet
|
|
|
|
<div align="center">
|
|
|
|
[](https://www.npmjs.com/package/agentic-qe)
|
|
[](https://opensource.org/licenses/MIT)
|
|
[](https://www.typescriptlang.org/)
|
|
[](https://nodejs.org/)
|
|
|
|
**Version 1.3.6** | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
|
|
|
|
> Enterprise-grade test automation with AI learning, comprehensive skills library (34 QE skills), and 85.7% cost savings through intelligent model routing.
|
|
|
|
🧠 **20% Continuous Improvement** | 📚 **34 World-Class QE Skills** | 🎯 **100% Flaky Test Detection** | 💰 **85.7% Cost Savings** | 🔧 **54 MCP Tools**
|
|
|
|
</div>
|
|
|
|
---
|
|
|
|
## ⚡ Quick Start
|
|
|
|
### Install & Initialize
|
|
|
|
```bash
|
|
# Install globally
|
|
npm install -g agentic-qe
|
|
|
|
# Initialize your project
|
|
cd your-project
|
|
aqe init
|
|
|
|
# Add MCP server to Claude Code (optional)
|
|
claude mcp add agentic-qe npx aqe-mcp
|
|
|
|
# Verify connection
|
|
claude mcp list
|
|
```
|
|
|
|
### Use from Claude Code CLI
|
|
|
|
Ask Claude to use AQE agents directly from your terminal:
|
|
|
|
```bash
|
|
# Generate comprehensive tests
|
|
claude "Use qe-test-generator to create tests for src/services/user-service.ts with 95% coverage"
|
|
|
|
# Run quality pipeline
|
|
claude "Initialize AQE fleet: generate tests, execute them, analyze coverage, and run quality gate"
|
|
|
|
# Detect flaky tests
|
|
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests"
|
|
```
|
|
|
|
**What gets initialized:**
|
|
- ✅ Multi-Model Router (70-81% cost savings - opt-in)
|
|
- ✅ Learning System (20% improvement target)
|
|
- ✅ Pattern Bank (cross-project reuse)
|
|
- ✅ ML Flaky Detection (100% accuracy)
|
|
- ✅ 17 Specialized agent definitions
|
|
- ✅ 8 AQE slash commands
|
|
- ✅ Configuration directory
|
|
|
|
---
|
|
|
|
## ✨ Features
|
|
|
|
### 🤖 Autonomous Agent Fleet
|
|
- **17 Specialized Agents**: Expert agents for every QE domain (test generation, coverage analysis, security scanning, performance testing)
|
|
- **AI-Powered Coordination**: Event-driven architecture with intelligent task distribution
|
|
- **Zero External Dependencies**: Native AQE hooks system (100-500x faster than external coordination)
|
|
- **Scalable**: From single developer projects to enterprise-scale testing infrastructure
|
|
|
|
### 🧠 Intelligence & Learning (v1.1.0)
|
|
- **Q-Learning System**: 20% improvement target with automatic strategy optimization
|
|
- **Pattern Bank**: 85%+ matching accuracy across 6 test frameworks (Jest, Mocha, Cypress, Vitest, Jasmine, AVA)
|
|
- **ML Flaky Detection**: 100% accuracy with root cause analysis and automated fix recommendations
|
|
- **Continuous Improvement**: A/B testing framework with 95%+ statistical confidence
|
|
- **Experience Replay**: Learn from 10,000+ past executions
|
|
|
|
### 💰 Cost Optimization (v1.0.5)
|
|
- **Multi-Model Router**: 70-81% cost savings through intelligent model selection (opt-in feature)
|
|
- **4+ AI Models**: GPT-3.5, GPT-4, Claude Haiku, Claude Sonnet 4.5
|
|
- **Smart Routing**: Automatic complexity analysis and optimal model selection
|
|
- **Real-Time Tracking**: Live cost monitoring with budget alerts and forecasting
|
|
- **ROI Dashboard**: Track savings vs single-model baseline
|
|
|
|
### 📊 Comprehensive Testing
|
|
- **Multi-Framework Support**: Jest, Mocha, Cypress, Playwright, Vitest, Jasmine, AVA
|
|
- **Parallel Execution**: 10,000+ concurrent tests with intelligent orchestration
|
|
- **Real-Time Coverage**: O(log n) algorithms for instant gap detection
|
|
- **Performance Testing**: k6, JMeter, Gatling integration
|
|
- **Real-Time Streaming**: Live progress updates for all operations
|
|
|
|
### 🎓 34 QE Skills Library (v1.3.0)
|
|
**95%+ coverage of modern QE practices**
|
|
|
|
<details>
|
|
<summary><b>View All Skills</b></summary>
|
|
|
|
**Phase 1: Original Quality Engineering Skills (18 skills)**
|
|
- **Core Testing**: agentic-quality-engineering, holistic-testing-pact, context-driven-testing, exploratory-testing-advanced
|
|
- **Methodologies**: tdd-london-chicago, xp-practices, risk-based-testing, test-automation-strategy
|
|
- **Techniques**: api-testing-patterns, performance-testing, security-testing
|
|
- **Code Quality**: code-review-quality, refactoring-patterns, quality-metrics
|
|
- **Communication**: bug-reporting-excellence, technical-writing, consultancy-practices
|
|
|
|
**Phase 2: Expanded QE Skills Library (16 skills)**
|
|
- **Testing Methodologies (6)**: regression-testing, shift-left-testing, shift-right-testing, test-design-techniques, mutation-testing, test-data-management
|
|
- **Specialized Testing (9)**: accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, compatibility-testing, localization-testing, compliance-testing, visual-testing-advanced
|
|
- **Testing Infrastructure (2)**: test-environment-management, test-reporting-analytics
|
|
|
|
</details>
|
|
|
|
---
|
|
|
|
## 💻 Usage Examples
|
|
|
|
### Example 1: Single Agent Execution
|
|
|
|
Ask Claude to use a specific agent:
|
|
|
|
```bash
|
|
claude "Use the qe-test-generator agent to create comprehensive tests for src/services/user-service.ts with 95% coverage"
|
|
```
|
|
|
|
**What happens:**
|
|
1. Claude Code spawns qe-test-generator via Task tool
|
|
2. Agent analyzes the source file
|
|
3. Generates tests with pattern matching (Phase 2 feature)
|
|
4. Stores results in memory at `aqe/test-plan/generated`
|
|
|
|
**Output:**
|
|
```
|
|
Generated 42 tests
|
|
Pattern hit rate: 67%
|
|
Time saved: 2.3s
|
|
Quality score: 96%
|
|
```
|
|
|
|
### Example 2: Multi-Agent Parallel Execution
|
|
|
|
Coordinate multiple agents at once:
|
|
|
|
```bash
|
|
claude "Initialize the AQE fleet:
|
|
1. Use qe-test-generator to create tests for src/services/*.ts
|
|
2. Use qe-test-executor to run all tests in parallel
|
|
3. Use qe-coverage-analyzer to find gaps with sublinear algorithms
|
|
4. Use qe-quality-gate to validate against 95% threshold"
|
|
```
|
|
|
|
**What happens:**
|
|
1. Claude spawns 4 agents concurrently in a single message
|
|
2. Agents coordinate through `aqe/*` memory namespace
|
|
3. Pipeline: test generator → executor → analyzer → gate
|
|
4. Real-time streaming progress updates
|
|
|
|
**Memory namespaces:**
|
|
- `aqe/test-plan/*` - Test planning and requirements
|
|
- `aqe/coverage/*` - Coverage analysis results
|
|
- `aqe/performance/*` - Performance test data
|
|
- `aqe/quality/*` - Quality metrics
|
|
|
|
### Example 3: Using Agents with Skills
|
|
|
|
Agents automatically leverage skills:
|
|
|
|
```bash
|
|
claude "Use qe-test-generator with shift-left-testing and test-design-techniques skills to create tests before implementing the new payment feature"
|
|
```
|
|
|
|
**Available skills** (agents auto-select from 34):
|
|
- TDD, API testing, performance, security
|
|
- Accessibility, mobile, chaos engineering
|
|
- Regression, shift-left/right, compliance
|
|
|
|
### Example 4: Full Quality Pipeline
|
|
|
|
End-to-end quality workflow:
|
|
|
|
```bash
|
|
claude "Run the full AQE quality pipeline:
|
|
1. qe-requirements-validator - validate requirements are testable
|
|
2. qe-test-generator - generate comprehensive test suite
|
|
3. qe-test-executor - run tests with parallel execution
|
|
4. qe-coverage-analyzer - analyze gaps using O(log n) algorithms
|
|
5. qe-flaky-test-hunter - detect flaky tests with 100% ML accuracy
|
|
6. qe-security-scanner - run SAST/DAST scans
|
|
7. qe-performance-tester - load test critical paths
|
|
8. qe-quality-gate - validate all quality criteria met
|
|
9. qe-deployment-readiness - assess deployment risk"
|
|
```
|
|
|
|
### Example 5: Specialized Testing Scenarios
|
|
|
|
```bash
|
|
# API contract validation
|
|
claude "Use qe-api-contract-validator to check if the new API changes break any existing contracts"
|
|
|
|
# Visual regression testing
|
|
claude "Use qe-visual-tester to compare screenshots of the updated dashboard against baseline"
|
|
|
|
# Chaos engineering
|
|
claude "Use qe-chaos-engineer to inject random failures and validate system resilience"
|
|
|
|
# Flaky test detection with ML
|
|
claude "Use qe-flaky-test-hunter to analyze the last 100 test runs and identify flaky tests with ML-powered root cause analysis"
|
|
```
|
|
|
|
### Example 6: Fleet Coordination at Scale
|
|
|
|
```bash
|
|
# Coordinate 50+ agents for large projects
|
|
claude "Use qe-fleet-commander to coordinate parallel testing across 8 microservices with 50 agents total"
|
|
```
|
|
|
|
### MCP Integration Examples
|
|
|
|
You can also use agents through MCP tools:
|
|
|
|
```bash
|
|
# Check MCP connection
|
|
claude mcp list
|
|
|
|
# Direct MCP tool usage (in Claude Code)
|
|
# Generate tests
|
|
mcp__agentic_qe__test_generate({
|
|
type: "unit",
|
|
framework: "jest",
|
|
targetFile: "src/user-service.ts"
|
|
})
|
|
|
|
# Execute tests
|
|
mcp__agentic_qe__test_execute({
|
|
parallel: true,
|
|
coverage: true
|
|
})
|
|
|
|
# Analyze coverage
|
|
mcp__agentic_qe__coverage_analyze({
|
|
threshold: 95
|
|
})
|
|
```
|
|
|
|
### CLI Direct Usage
|
|
|
|
```bash
|
|
# Generate tests
|
|
aqe test src/services/user-service.ts
|
|
|
|
# Analyze coverage
|
|
aqe coverage --threshold 95
|
|
|
|
# Run quality gate
|
|
aqe quality
|
|
|
|
# View fleet status
|
|
aqe status --verbose
|
|
|
|
# Enable multi-model router (70-81% cost savings)
|
|
aqe routing enable
|
|
|
|
# Start learning system
|
|
aqe learn enable --all
|
|
```
|
|
|
|
### Advanced Patterns
|
|
|
|
#### Pattern 1: Continuous Learning
|
|
|
|
```bash
|
|
# Agents learn from execution
|
|
claude "Use qe-test-generator with learning enabled to create tests, then analyze improvement over time"
|
|
|
|
# Check learning metrics
|
|
aqe learn status --agent test-generator
|
|
```
|
|
|
|
**Example Output:**
|
|
```
|
|
📊 LEARNING STATUS
|
|
|
|
Agent: test-generator
|
|
Status: ENABLED ✅
|
|
Total Experiences: 247
|
|
Exploration Rate: 15.3%
|
|
|
|
Performance:
|
|
├─ Average Reward: 1.23
|
|
├─ Success Rate: 87.5%
|
|
└─ Improvement Rate: 18.7% (↑ target: 20%)
|
|
|
|
Top Strategies:
|
|
1. property-based (confidence: 92%, success: 95%)
|
|
2. mutation-based (confidence: 85%, success: 88%)
|
|
3. example-based (confidence: 78%, success: 82%)
|
|
```
|
|
|
|
#### Pattern 2: Pattern Bank Usage
|
|
|
|
```bash
|
|
# Extract and reuse patterns
|
|
claude "Use qe-test-generator to extract test patterns from existing tests, then apply them to new modules"
|
|
|
|
# List patterns
|
|
aqe patterns list --framework jest
|
|
```
|
|
|
|
**Example Output:**
|
|
```
|
|
📦 PATTERN LIBRARY (247 patterns)
|
|
|
|
ID | Name | Framework | Quality | Uses
|
|
-----------|---------------------------|-----------|---------|-----
|
|
pattern-001| Null Parameter Check | jest | 92% | 142
|
|
pattern-002| Empty Array Handling | jest | 89% | 98
|
|
pattern-003| API Timeout Test | cypress | 95% | 87
|
|
```
|
|
|
|
#### Pattern 3: Cost Optimization
|
|
|
|
```bash
|
|
# Enable intelligent model routing
|
|
aqe routing enable
|
|
|
|
# View savings
|
|
claude "Check routing status and show cost savings"
|
|
aqe routing dashboard
|
|
```
|
|
|
|
**Example Output:**
|
|
```
|
|
✅ Multi-Model Router Status
|
|
|
|
Cost Summary (Last 30 Days):
|
|
Total Cost: $127.50
|
|
Baseline Cost: $545.00
|
|
Savings: $417.50 (76.6%)
|
|
Budget Status: ON TRACK ✓
|
|
|
|
Model Usage:
|
|
├─ gpt-3.5-turbo: 42% (simple tasks)
|
|
├─ claude-haiku: 31% (medium tasks)
|
|
├─ claude-sonnet-4.5: 20% (complex tasks)
|
|
└─ gpt-4: 7% (critical tasks)
|
|
```
|
|
|
|
### Pro Tips
|
|
|
|
1. **Batch agent operations**: Always spawn multiple agents in one Claude message for parallel execution
|
|
2. **Use memory namespace**: Agents coordinate through `aqe/*` memory keys
|
|
3. **Enable learning**: Add `--enable-learning` to agent commands for continuous improvement
|
|
4. **Check agent status**: Use `aqe status` to see active agents and coordination
|
|
5. **Review agent output**: Agents store detailed results in `.agentic-qe/logs/`
|
|
|
|
---
|
|
|
|
## 🤖 Agent Types
|
|
|
|
<details>
|
|
<summary><b>Core Testing Agents (5 agents)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
|-------|---------|-------------|---------------------|
|
|
| **test-generator** | AI-powered test creation | Property-based testing, edge case detection | ✅ Pattern matching, Learning |
|
|
| **test-executor** | Multi-framework execution | Parallel processing, retry logic, reporting | - |
|
|
| **coverage-analyzer** | Real-time gap analysis | O(log n) algorithms, trend tracking | ✅ Learning, Pattern recommendations |
|
|
| **quality-gate** | Intelligent validation | ML-driven decisions, risk assessment | ✅ Flaky test metrics |
|
|
| **quality-analyzer** | Metrics analysis | ESLint, SonarQube, Lighthouse integration | - |
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Performance & Security (2 agents)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features |
|
|
|-------|---------|-------------|
|
|
| **performance-tester** | Load & stress testing | k6, JMeter, Gatling, bottleneck detection |
|
|
| **security-scanner** | Vulnerability detection | SAST, DAST, dependency scanning |
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Strategic Planning (3 agents)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features |
|
|
|-------|---------|-------------|
|
|
| **requirements-validator** | Testability analysis | INVEST criteria, BDD generation |
|
|
| **production-intelligence** | Incident replay | RUM analysis, anomaly detection |
|
|
| **fleet-commander** | Hierarchical coordination | 50+ agent orchestration |
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Advanced Testing (4 agents)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features | Phase 2 Enhancements |
|
|
|-------|---------|-------------|---------------------|
|
|
| **regression-risk-analyzer** | Smart test selection | ML patterns, AST analysis | ✅ Pattern matching |
|
|
| **test-data-architect** | Realistic data generation | 10k+ records/sec, GDPR compliant | - |
|
|
| **api-contract-validator** | Breaking change detection | OpenAPI, GraphQL, gRPC | - |
|
|
| **flaky-test-hunter** | Stability analysis | Statistical detection, auto-fix | ✅ 100% accuracy ML detection |
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>Specialized (3 agents)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features |
|
|
|-------|---------|-------------|
|
|
| **deployment-readiness** | Release validation | Multi-factor risk scoring |
|
|
| **visual-tester** | UI regression | AI-powered comparison |
|
|
| **chaos-engineer** | Resilience testing | Fault injection, blast radius |
|
|
|
|
</details>
|
|
|
|
<details>
|
|
<summary><b>General Purpose (1 agent)</b></summary>
|
|
|
|
| Agent | Purpose | Key Features |
|
|
|-------|---------|-------------|
|
|
| **base-template-generator** | Agent templates | General-purpose agent creation |
|
|
|
|
</details>
|
|
|
|
**Total: 17 Agents** (16 QE-specific + 1 general-purpose)
|
|
|
|
---
|
|
|
|
## 📖 Documentation
|
|
|
|
### Getting Started
|
|
- [Quick Start Guide](docs/AQE-CLI.md) - Get started in 5 minutes
|
|
- [User Guide](docs/USER-GUIDE.md) - Comprehensive workflows and examples
|
|
- [MCP Integration](docs/guides/MCP-INTEGRATION.md) - Claude Code integration
|
|
- [Configuration Guide](docs/CONFIGURATION.md) - Complete configuration reference
|
|
- [Troubleshooting Guide](docs/TROUBLESHOOTING.md) - Common issues and solutions
|
|
|
|
### Feature Guides
|
|
|
|
**Phase 2 Features (v1.1.0)**
|
|
- [Learning System User Guide](docs/guides/LEARNING-SYSTEM-USER-GUIDE.md) - Q-learning and continuous improvement
|
|
- [Pattern Management User Guide](docs/guides/PATTERN-MANAGEMENT-USER-GUIDE.md) - Cross-project pattern sharing
|
|
- [ML Flaky Detection Guide](docs/guides/ML-FLAKY-DETECTION-USER-GUIDE.md) - 100% accurate flaky detection
|
|
- [Performance Improvement Guide](docs/guides/PERFORMANCE-IMPROVEMENT-USER-GUIDE.md) - A/B testing and optimization
|
|
|
|
**Phase 1 Features (v1.0.5)**
|
|
- [Multi-Model Router Guide](docs/guides/MULTI-MODEL-ROUTER.md) - Save 70% on AI costs
|
|
- [Streaming API Tutorial](docs/guides/STREAMING-API.md) - Real-time progress updates
|
|
- [Cost Optimization Best Practices](docs/guides/COST-OPTIMIZATION.md) - Maximize ROI
|
|
|
|
### Testing Guides
|
|
- [Test Generation](docs/guides/TEST-GENERATION.md) - AI-powered test creation
|
|
- [Coverage Analysis](docs/guides/COVERAGE-ANALYSIS.md) - O(log n) gap detection
|
|
- [Quality Gates](docs/guides/QUALITY-GATES.md) - Intelligent validation
|
|
- [Performance Testing](docs/guides/PERFORMANCE-TESTING.md) - Load and stress testing
|
|
- [Test Execution](docs/guides/TEST-EXECUTION.md) - Parallel orchestration
|
|
|
|
### Advanced Topics
|
|
- [API Reference](docs/API.md) - Complete API documentation
|
|
- [Agent Development](docs/AGENT-DEVELOPMENT.md) - Create custom agents
|
|
- [Agent Types Overview](docs/Agentic-QE-Fleet-Specification.md) - Complete agent reference
|
|
- [AQE Hooks Guide](docs/AQE-HOOKS-GUIDE.md) - Native coordination system
|
|
- [Best Practices](docs/AI%20%26%20Agentic%20Security%20Best%20Practices.md) - Security and quality
|
|
|
|
### Commands Reference
|
|
- [AQE Commands Overview](docs/QE-COMMANDS-INDEX.md) - All CLI commands
|
|
- [Command Specifications](docs/QE-SLASH-COMMANDS-SPECIFICATION.md) - Slash command reference
|
|
- [Hooks Architecture](docs/QE_HOOKS_ARCHITECTURE.md) - Coordination architecture
|
|
|
|
### Code Examples
|
|
- [Learning System Examples](docs/examples/LEARNING-SYSTEM-EXAMPLES.md) - Learning code examples
|
|
- [Pattern Examples](docs/examples/REASONING-BANK-EXAMPLES.md) - Pattern usage examples
|
|
- [Flaky Detection Examples](docs/examples/FLAKY-DETECTION-ML-EXAMPLES.md) - ML detection examples
|
|
- [Routing Examples](docs/examples/ROUTING-EXAMPLES.md) - Cost optimization examples
|
|
|
|
---
|
|
|
|
## 📊 Performance Benchmarks
|
|
|
|
| Feature | Target | Actual | Status |
|
|
|---------|--------|--------|--------|
|
|
| **Pattern Matching (p95)** | <50ms | 32ms | ✅ Exceeded |
|
|
| **Learning Iteration** | <100ms | 68ms | ✅ Exceeded |
|
|
| **ML Flaky Detection (1000 tests)** | <500ms | 385ms | ✅ Exceeded |
|
|
| **Agent Memory** | <100MB | 85MB | ✅ Exceeded |
|
|
| **Cost Savings** | 70%+ | 70-81% | ✅ Achieved |
|
|
| **Test Improvement** | 20%+ | 23%+ | ✅ Exceeded |
|
|
| **Flaky Detection Accuracy** | 90%+ | 100% | ✅ Exceeded |
|
|
| **False Positive Rate** | <5% | 0% | ✅ Exceeded |
|
|
|
|
### Core Performance
|
|
- **Test Generation**: 1000+ tests/minute
|
|
- **Parallel Execution**: 10,000+ concurrent tests
|
|
- **Coverage Analysis**: O(log n) complexity
|
|
- **Data Generation**: 10,000+ records/second
|
|
- **Agent Spawning**: <100ms per agent
|
|
- **Memory Efficient**: <2GB for typical projects
|
|
|
|
---
|
|
|
|
## 📝 Recent Changes
|
|
|
|
### v1.3.6 (2025-10-30)
|
|
**Stability & Educational Release**
|
|
|
|
- ✅ **Fixed 16 critical TypeScript compilation errors** blocking production builds
|
|
- ✅ **Integrated CodeComplexityAnalyzerAgent** (educational example from PR #22 by @mondweep)
|
|
- ✅ **Zero functional regressions** - all core functionality tested and stable
|
|
- ✅ **Build stability improvements** - TypeScript compilation passing with 0 errors
|
|
- ✅ **BaseAgent property encapsulation** - proper lifecycle manager integration
|
|
- ✅ **Clean cherry-pick** from PR #22 with zero conflicts
|
|
|
|
**Technical Improvements**:
|
|
- BaseAgent property access patterns now use lifecycle manager accessors
|
|
- AccessControlDAO interface mapping corrected (`resourceId`, `owner` properties)
|
|
- Permission enum usage standardized (READ, WRITE, DELETE, SHARE)
|
|
- AgentLifecycleManager and AgentCoordinator properly integrated
|
|
|
|
**New Capabilities**:
|
|
- 📊 Code complexity analysis agent (cyclomatic & cognitive complexity)
|
|
- 🎯 Quality scoring system (0-100 scale)
|
|
- 🤖 AI-powered refactoring recommendations
|
|
- 📚 Complete BaseAgent pattern demonstration
|
|
- 📖 463-line architecture guide for learning
|
|
|
|
**Contributors**: @mondweep (CodeComplexityAnalyzerAgent), AQE Development Team
|
|
|
|
### v1.3.5 (2025-10-27) - Learning System Complete & Critical Policies
|
|
|
|
**Phase 2 Features Complete:**
|
|
- ✅ Learning System with Q-Learning (87.5% success rate, 18.7% improvement)
|
|
- ✅ Experience Replay Buffer (10,000 experiences)
|
|
- ✅ Pattern Bank with 247 patterns (85%+ accuracy)
|
|
- ✅ Multi-Model Router: 85.7% cost savings (exceeds 70-81% target)
|
|
- ✅ ML Flaky Detection (100% accuracy, 0% false positives)
|
|
- ✅ Streaming Progress with real-time updates
|
|
|
|
**Critical Policy Updates:**
|
|
- ⚠️ Release Verification Policy (8-point checklist)
|
|
- ⚠️ Test Execution Policy (prevents workspace crashes)
|
|
- ⚠️ Release Tagging Policy (tags after PR merge)
|
|
|
|
**Test Coverage:**
|
|
- 237 new tests added across all Phase 2 features
|
|
- Coverage: 50-70% (30-40x increase from 1.67%)
|
|
- Fixed 328 import paths across 122 test files
|
|
|
|
**Quality Score:** 92/100 (EXCELLENT) - Zero breaking changes, 100% backward compatible.
|
|
|
|
[View Complete Changelog](CHANGELOG.md)
|
|
|
|
---
|
|
|
|
## 🚀 Development
|
|
|
|
### Setup
|
|
|
|
```bash
|
|
# Clone repository
|
|
git clone https://github.com/proffesor-for-testing/agentic-qe.git
|
|
cd agentic-qe
|
|
|
|
# Install dependencies
|
|
npm install
|
|
|
|
# Build
|
|
npm run build
|
|
|
|
# Run tests
|
|
npm test
|
|
```
|
|
|
|
### Available Scripts
|
|
|
|
| Script | Description |
|
|
|--------|-------------|
|
|
| `npm run build` | Compile TypeScript to JavaScript |
|
|
| `npm run dev` | Development mode with hot reload |
|
|
| `npm test` | Run all test suites |
|
|
| `npm run test:unit` | Unit tests only |
|
|
| `npm run test:integration` | Integration tests |
|
|
| `npm run test:coverage` | Generate coverage report |
|
|
| `npm run lint` | ESLint code checking |
|
|
| `npm run lint:fix` | Auto-fix linting issues |
|
|
| `npm run typecheck` | TypeScript type checking |
|
|
|
|
### Project Structure
|
|
|
|
```
|
|
agentic-qe/
|
|
├── src/
|
|
│ ├── agents/ # Agent implementation classes
|
|
│ ├── core/ # Core fleet management
|
|
│ ├── learning/ # Phase 2: Learning system
|
|
│ ├── reasoning/ # Phase 2: Pattern bank
|
|
│ ├── cli/ # Command-line interface
|
|
│ ├── mcp/ # Model Context Protocol server
|
|
│ ├── types/ # TypeScript type definitions
|
|
│ └── utils/ # Shared utilities
|
|
├── tests/ # Comprehensive test suites
|
|
├── examples/ # Usage examples
|
|
├── docs/ # Documentation
|
|
├── .claude/ # Agent & command definitions
|
|
│ ├── agents/ # 17 agent definitions
|
|
│ └── commands/ # 8 AQE slash commands
|
|
└── config/ # Configuration files
|
|
```
|
|
|
|
---
|
|
|
|
## 🤝 Contributing
|
|
|
|
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
|
|
|
|
### Quick Contribution Guide
|
|
|
|
1. Fork the repository
|
|
2. Create a feature branch (`git checkout -b feature/amazing-feature`)
|
|
3. Make your changes
|
|
4. Add tests for new functionality
|
|
5. Ensure all tests pass (`npm test`)
|
|
6. Commit your changes (`git commit -m 'feat: add amazing feature'`)
|
|
7. Push to your branch (`git push origin feature/amazing-feature`)
|
|
8. Open a Pull Request
|
|
|
|
### Development Guidelines
|
|
|
|
- Follow the existing code style
|
|
- Write comprehensive tests
|
|
- Update documentation
|
|
- Use conventional commits
|
|
- Ensure TypeScript types are accurate
|
|
|
|
---
|
|
|
|
## 📞 Support
|
|
|
|
- **Documentation**: [docs/](docs/)
|
|
- **Issues**: [GitHub Issues](https://github.com/proffesor-for-testing/agentic-qe/issues)
|
|
- **Discussions**: [GitHub Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
|
|
- **Email**: support@agentic-qe.com
|
|
|
|
---
|
|
|
|
## 🗺️ Roadmap
|
|
|
|
### Current (v1.3)
|
|
- ✅ Learning System with Q-learning
|
|
- ✅ Pattern Bank with cross-project sharing
|
|
- ✅ ML Flaky Detection (100% accuracy)
|
|
- ✅ Continuous Improvement Loop
|
|
- ✅ 17 specialized agents
|
|
- ✅ Multi-framework test execution
|
|
- ✅ Real-time coverage analysis
|
|
- ✅ MCP integration
|
|
- ✅ Multi-model router (70-81% cost savings)
|
|
- ✅ 34 QE skills library
|
|
|
|
### Planned (v1.4)
|
|
- 🔄 Web dashboard for visualization
|
|
- 🔄 GraphQL API
|
|
- 🔄 CI/CD integrations (GitHub Actions, GitLab CI)
|
|
- 🔄 Enhanced pattern adaptation across frameworks
|
|
- 🔄 Real-time collaboration features
|
|
|
|
### Future (v2.0)
|
|
- 📋 Natural language test generation
|
|
- 📋 Self-healing test suites
|
|
- 📋 Multi-language support (Python, Java, Go)
|
|
- 📋 Advanced analytics and insights
|
|
- 📋 Cloud deployment support
|
|
|
|
---
|
|
|
|
## 📝 License
|
|
|
|
This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.
|
|
|
|
---
|
|
|
|
## 🙏 Acknowledgments
|
|
|
|
- Built with TypeScript, Node.js, and better-sqlite3
|
|
- Inspired by autonomous agent architectures and swarm intelligence
|
|
- Integrates with Jest, Cypress, Playwright, k6, SonarQube, and more
|
|
- Compatible with Claude Code via Model Context Protocol (MCP)
|
|
|
|
---
|
|
|
|
<div align="center">
|
|
|
|
**Made with ❤️ by the Agentic QE Team**
|
|
|
|
[⭐ Star us on GitHub](https://github.com/proffesor-for-testing/agentic-qe) • [🐦 Follow on Twitter](https://twitter.com/agenticqe)
|
|
|
|
</div>
|