mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
d7455132bf
- Update agent count from 18 to 19 (added qx-partner)
- Update skill count references to 41 (was inconsistent 37/38/39)
- Add Self-Learning System section to README explaining RL algorithms
- Add Why AQE? section with Problem-Solution table
- Remove outdated What's New and Roadmap sections from README
- Fix flaky detection accuracy claim from 100% to 90%+
- Remove unverified SWE-Bench and token reduction claims
- Add qx-partner to agents list in skills and manifest
- Update all user-facing docs with correct counts
- Add docs/PERFORMANCE.md for verified benchmarks
Files updated:
- README.md, CLAUDE.md, FUNDING.md
- docs/reference/{agents,skills,usage}.md
- .agentic-qe/docs/{agents,skills,usage}.md
- .claude/skills/agentic-quality-engineering/SKILL.md
- .claude/skills/cicd-pipeline-qe-orchestrator/README.md
- .claude/skills/skills-manifest.json
- src/cli/init/{agents,database-init}.ts
- src/telemetry/instrumentation/{agent,index}.ts
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
573 lines
13 KiB
Markdown
573 lines
13 KiB
Markdown
# Agentic QE Fleet - Usage Guide
|
|
|
|
Complete guide for using the Agentic QE Fleet in your projects.
|
|
|
|
## Quick Start
|
|
|
|
### Installation
|
|
|
|
```bash
|
|
npm install agentic-qe-cf --save-dev
|
|
```
|
|
|
|
### Initialization
|
|
|
|
```bash
|
|
# Initialize AQE in your project
|
|
npx aqe init
|
|
|
|
# This creates:
|
|
# - .claude/agents/ (19 QE agents)
|
|
# - .claude/skills/ (41 QE skills)
|
|
# - .claude/commands/ (8 slash commands)
|
|
# - .agentic-qe/config/ (fleet configuration)
|
|
# - .agentic-qe/db/ (learning databases)
|
|
# - CLAUDE.md (project configuration)
|
|
```
|
|
|
|
---
|
|
|
|
## Using Agents via Claude Code Task Tool
|
|
|
|
**Recommended Method:** Use Claude Code's Task tool for spawning agents.
|
|
|
|
### Single Agent Execution
|
|
|
|
```javascript
|
|
Task("Generate tests", "Create comprehensive test suite for UserService", "qe-test-generator")
|
|
```
|
|
|
|
### Parallel Agent Execution
|
|
|
|
```javascript
|
|
// Execute multiple agents concurrently
|
|
Task("Test Generation", "Generate unit tests", "qe-test-generator")
|
|
Task("Coverage Analysis", "Analyze current coverage", "qe-coverage-analyzer")
|
|
Task("Security Scan", "Run security checks", "qe-security-scanner")
|
|
Task("Performance Test", "Load test critical paths", "qe-performance-tester")
|
|
```
|
|
|
|
### Agent Coordination Example
|
|
|
|
```javascript
|
|
// Step 1: Test generator stores results
|
|
Task("Generate tests", "Create tests and store in memory at aqe/test-plan/generated", "qe-test-generator")
|
|
|
|
// Step 2: Test executor reads from memory
|
|
Task("Execute tests", "Read test plan from aqe/test-plan/generated and execute", "qe-test-executor")
|
|
|
|
// Step 3: Coverage analyzer processes results
|
|
Task("Analyze coverage", "Check coverage from aqe/coverage/results", "qe-coverage-analyzer")
|
|
```
|
|
|
|
---
|
|
|
|
## Using MCP Tools
|
|
|
|
### Check MCP Connection
|
|
|
|
```bash
|
|
claude mcp list
|
|
# Should show: agentic-qe: npm run mcp:start - ✓ Connected
|
|
```
|
|
|
|
### Tool Discovery System (New in v2.1.0)
|
|
|
|
The MCP server uses **lazy loading** to reduce initial context by 87%.
|
|
|
|
#### Discover Available Domains
|
|
|
|
```javascript
|
|
// List all available tool domains
|
|
mcp__agentic_qe__tools_discover()
|
|
|
|
// Returns:
|
|
// {
|
|
// domains: [
|
|
// "security", "coverage", "test-generation", "test-execution",
|
|
// "quality-gates", "fleet-management", "memory", "learning",
|
|
// "patterns", "reporting"
|
|
// ]
|
|
// }
|
|
```
|
|
|
|
#### Load Domain-Specific Tools
|
|
|
|
```javascript
|
|
// Load security tools (4 tools)
|
|
mcp__agentic_qe__tools_load_domain({ domain: "security" })
|
|
|
|
// Load coverage tools (3 tools)
|
|
mcp__agentic_qe__tools_load_domain({ domain: "coverage" })
|
|
|
|
// Load test generation tools (2 tools)
|
|
mcp__agentic_qe__tools_load_domain({ domain: "test-generation" })
|
|
```
|
|
|
|
#### Automatic Loading
|
|
|
|
Tools are **auto-loaded based on keywords** in agent instructions:
|
|
|
|
```javascript
|
|
// Keyword "security" triggers auto-load of security domain
|
|
Task("Security scan", "Run security checks for vulnerabilities", "qe-security-scanner")
|
|
|
|
// Keyword "coverage" triggers auto-load of coverage domain
|
|
Task("Coverage analysis", "Find coverage gaps in UserService", "qe-coverage-analyzer")
|
|
|
|
// Keyword "generate" triggers auto-load of test-generation domain
|
|
Task("Generate tests", "Create unit tests for API endpoints", "qe-test-generator")
|
|
```
|
|
|
|
**Recognized Keywords:**
|
|
- **security**: security, vulnerability, sast, dast, owasp
|
|
- **coverage**: coverage, gaps, untested, branches
|
|
- **test-generation**: generate, create tests, test suite
|
|
- **test-execution**: execute, run tests, parallel
|
|
- **quality-gates**: quality, gate, threshold, deployment
|
|
- **fleet-management**: fleet, agents, orchestrate, swarm
|
|
- **memory**: memory, store, retrieve, namespace
|
|
- **learning**: learn, patterns, q-learning, improve
|
|
- **patterns**: patterns, extract, search, templates
|
|
- **reporting**: report, metrics, dashboard, export
|
|
|
|
### Using MCP Tools in Claude Code
|
|
|
|
```javascript
|
|
// Test generation
|
|
mcp__agentic_qe__test_generate({ type: "unit", framework: "jest" })
|
|
|
|
// Test execution
|
|
mcp__agentic_qe__test_execute({ parallel: true, coverage: true })
|
|
|
|
// Quality analysis
|
|
mcp__agentic_qe__quality_analyze({ scope: "full" })
|
|
```
|
|
|
|
---
|
|
|
|
## Using CLI Commands
|
|
|
|
### Quick Commands
|
|
|
|
```bash
|
|
# Generate tests
|
|
aqe test <module-name>
|
|
|
|
# Analyze coverage
|
|
aqe coverage
|
|
|
|
# Run quality gate
|
|
aqe quality
|
|
|
|
# Check fleet status
|
|
aqe status
|
|
```
|
|
|
|
### Learning System Commands
|
|
|
|
```bash
|
|
# Check learning status
|
|
aqe learn status --agent test-gen
|
|
|
|
# View learned patterns
|
|
aqe learn history --agent test-gen --limit 50
|
|
|
|
# Export learning data
|
|
aqe learn export --agent test-gen --output learning.json
|
|
```
|
|
|
|
### Pattern Management
|
|
|
|
```bash
|
|
# List test patterns
|
|
aqe patterns list --framework jest
|
|
|
|
# Search patterns
|
|
aqe patterns search "api validation"
|
|
|
|
# Extract patterns from tests
|
|
aqe patterns extract ./tests --framework jest
|
|
```
|
|
|
|
### Improvement Loop
|
|
|
|
```bash
|
|
# Start continuous improvement
|
|
aqe improve start
|
|
|
|
# Check improvement status
|
|
aqe improve status
|
|
|
|
# Run single improvement cycle
|
|
aqe improve cycle
|
|
```
|
|
|
|
---
|
|
|
|
## Using Slash Commands
|
|
|
|
### Available Commands
|
|
|
|
- `/aqe-execute` - Execute test suites with parallel orchestration
|
|
- `/aqe-generate` - Generate comprehensive test suites
|
|
- `/aqe-coverage` - Analyze test coverage
|
|
- `/aqe-quality` - Run quality gate validation
|
|
- `/aqe-optimize` - Optimize test suites using sublinear algorithms
|
|
- `/aqe-report` - Generate quality engineering reports
|
|
- `/aqe-analyze` - Analyze coverage gaps
|
|
- `/aqe-fleet-status` - Display fleet health and metrics
|
|
|
|
### Example Usage
|
|
|
|
```bash
|
|
# In Claude Code chat
|
|
/aqe-generate
|
|
/aqe-execute
|
|
/aqe-coverage
|
|
```
|
|
|
|
---
|
|
|
|
## Using Skills
|
|
|
|
### Via CLI
|
|
|
|
```bash
|
|
# List all available skills
|
|
aqe skills list
|
|
|
|
# Search for specific skills
|
|
aqe skills search "testing"
|
|
|
|
# Show skill details
|
|
aqe skills show agentic-quality-engineering
|
|
|
|
# Show skill statistics
|
|
aqe skills stats
|
|
```
|
|
|
|
### Via Skill Tool in Claude Code
|
|
|
|
```javascript
|
|
// Execute a skill
|
|
Skill("agentic-quality-engineering")
|
|
Skill("tdd-london-chicago")
|
|
Skill("api-testing-patterns")
|
|
```
|
|
|
|
---
|
|
|
|
## Agent Coordination
|
|
|
|
All agents coordinate through **AQE hooks** (zero external dependencies, 100-500x faster than shell hooks).
|
|
|
|
### Automatic Lifecycle Hooks
|
|
|
|
Agents extend `BaseAgent` and override lifecycle methods:
|
|
|
|
```typescript
|
|
protected async onPreTask(data: { assignment: TaskAssignment }): Promise<void> {
|
|
// Load context before task execution
|
|
const context = await this.memoryStore.retrieve('aqe/context', {
|
|
partition: 'coordination'
|
|
});
|
|
|
|
this.logger.info('Pre-task hook complete');
|
|
}
|
|
|
|
protected async onPostTask(data: { assignment: TaskAssignment; result: any }): Promise<void> {
|
|
// Store results after task completion
|
|
await this.memoryStore.store('aqe/' + this.agentId.type + '/results', data.result, {
|
|
partition: 'agent_results',
|
|
ttl: 86400 // 24 hours
|
|
});
|
|
|
|
// Emit completion event
|
|
this.eventBus.emit('task:completed', {
|
|
agentId: this.agentId,
|
|
result: data.result
|
|
});
|
|
|
|
this.logger.info('Post-task hook complete');
|
|
}
|
|
|
|
protected async onTaskError(data: { assignment: TaskAssignment; error: Error }): Promise<void> {
|
|
// Handle task errors
|
|
await this.memoryStore.store('aqe/errors/' + data.assignment.id, {
|
|
error: data.error.message,
|
|
stack: data.error.stack,
|
|
timestamp: Date.now()
|
|
}, {
|
|
partition: 'errors',
|
|
ttl: 604800 // 7 days
|
|
});
|
|
|
|
this.logger.error('Task failed', { error: data.error });
|
|
}
|
|
```
|
|
|
|
### Performance Comparison
|
|
|
|
| Feature | AQE Hooks | External Hooks |
|
|
|---------|-----------|----------------|
|
|
| **Speed** | <1ms | 100-500ms |
|
|
| **Dependencies** | Zero | External package |
|
|
| **Type Safety** | Full TypeScript | Shell strings |
|
|
| **Integration** | Direct API | Shell commands |
|
|
| **Performance** | 100-500x faster | Baseline |
|
|
|
|
---
|
|
|
|
## Memory Namespace
|
|
|
|
Agents share state through the **`aqe/*` memory namespace**:
|
|
|
|
- `aqe/test-plan/*` - Test planning and requirements
|
|
- `aqe/coverage/*` - Coverage analysis and gaps
|
|
- `aqe/quality/*` - Quality metrics and gates
|
|
- `aqe/performance/*` - Performance test results
|
|
- `aqe/security/*` - Security scan findings
|
|
- `aqe/swarm/coordination` - Cross-agent coordination
|
|
|
|
---
|
|
|
|
## Fleet Configuration
|
|
|
|
Configuration is stored in `.agentic-qe/config/fleet.json`:
|
|
|
|
```json
|
|
{
|
|
"topology": "hierarchical",
|
|
"maxAgents": 10,
|
|
"testingFocus": ["unit", "integration"],
|
|
"environments": ["development"],
|
|
"frameworks": ["jest"]
|
|
}
|
|
```
|
|
|
|
### Topology Options
|
|
|
|
- **hierarchical** - Queen-led coordination with specialized workers
|
|
- **mesh** - Peer-to-peer with distributed decision making
|
|
- **ring** - Circular coordination pattern
|
|
- **star** - Centralized hub-and-spoke
|
|
|
|
---
|
|
|
|
## Multi-Model Router
|
|
|
|
**Status:** Disabled by default (opt-in)
|
|
|
|
Provides **70-81% cost savings** by intelligently selecting AI models based on task complexity.
|
|
|
|
### Enabling Routing
|
|
|
|
**Option 1: Via Configuration**
|
|
```json
|
|
// .agentic-qe/config/routing.json
|
|
{
|
|
"multiModelRouter": {
|
|
"enabled": true
|
|
}
|
|
}
|
|
```
|
|
|
|
**Option 2: Via Environment Variable**
|
|
```bash
|
|
export AQE_ROUTING_ENABLED=true
|
|
```
|
|
|
|
### Model Selection Rules
|
|
|
|
| Task Complexity | Model | Est. Cost | Use Case |
|
|
|----------------|-------|-----------|----------|
|
|
| **Simple** | GPT-3.5 | $0.0004 | Unit tests, basic validation |
|
|
| **Moderate** | GPT-3.5 | $0.0008 | Integration tests, mocks |
|
|
| **Complex** | GPT-4 | $0.0048 | Property-based, edge cases |
|
|
| **Critical** | Claude Sonnet 4.5 | $0.0065 | Security, architecture review |
|
|
|
|
### Monitoring Costs
|
|
|
|
```bash
|
|
# View cost dashboard
|
|
aqe routing dashboard
|
|
|
|
# Export cost report
|
|
aqe routing report --format json
|
|
|
|
# Check savings
|
|
aqe routing stats
|
|
```
|
|
|
|
---
|
|
|
|
## Streaming Progress
|
|
|
|
**Status:** Enabled by default
|
|
|
|
Real-time progress updates for long-running operations using AsyncGenerator pattern.
|
|
|
|
### Example Usage
|
|
|
|
```javascript
|
|
// Using streaming MCP tool
|
|
const handler = new TestExecuteStreamHandler();
|
|
|
|
for await (const event of handler.execute(params)) {
|
|
if (event.type === 'progress') {
|
|
console.log(`Progress: ${event.percent}% - ${event.message}`);
|
|
} else if (event.type === 'result') {
|
|
console.log('Completed:', event.data);
|
|
}
|
|
}
|
|
```
|
|
|
|
### Supported Operations
|
|
|
|
- ✅ Test execution (test-by-test progress)
|
|
- ✅ Coverage analysis (incremental gap detection)
|
|
- ⚠️ Test generation (coming soon)
|
|
- ⚠️ Security scanning (coming soon)
|
|
|
|
---
|
|
|
|
## Q-Learning Integration
|
|
|
|
All agents automatically learn from task execution through Q-learning.
|
|
|
|
### Observability
|
|
|
|
```bash
|
|
# Check learning status
|
|
aqe learn status --agent test-gen
|
|
|
|
# View learned patterns
|
|
aqe learn history --agent test-gen --limit 50
|
|
|
|
# Export learning data
|
|
aqe learn export --agent test-gen --output learning.json
|
|
```
|
|
|
|
### Pattern Management
|
|
|
|
```bash
|
|
# List test patterns
|
|
aqe patterns list --framework jest
|
|
|
|
# Search patterns
|
|
aqe patterns search "api validation"
|
|
|
|
# Extract patterns from tests
|
|
aqe patterns extract ./tests --framework jest
|
|
```
|
|
|
|
### Improvement Loop
|
|
|
|
```bash
|
|
# Start continuous improvement
|
|
aqe improve start
|
|
|
|
# Check improvement status
|
|
aqe improve status
|
|
|
|
# Run single improvement cycle
|
|
aqe improve cycle
|
|
```
|
|
|
|
---
|
|
|
|
## Best Practices
|
|
|
|
1. **Use Task Tool**: Claude Code's Task tool is the primary way to spawn agents
|
|
2. **Batch Operations**: Always spawn multiple related agents in a single message
|
|
3. **Memory Keys**: Use the `aqe/*` namespace for agent coordination
|
|
4. **AQE Hooks**: Agents automatically use native AQE hooks for coordination (100-500x faster)
|
|
5. **Parallel Execution**: Leverage concurrent agent execution for speed
|
|
|
|
---
|
|
|
|
## Troubleshooting
|
|
|
|
### Check MCP Connection
|
|
|
|
```bash
|
|
claude mcp list
|
|
# Should show: agentic-qe: npm run mcp:start - ✓ Connected
|
|
```
|
|
|
|
### View Agent Definitions
|
|
|
|
```bash
|
|
ls -la .claude/agents/
|
|
# Should show 18 agent markdown files
|
|
```
|
|
|
|
### Check Fleet Status
|
|
|
|
```bash
|
|
aqe status --verbose
|
|
```
|
|
|
|
### View Logs
|
|
|
|
```bash
|
|
tail -f .agentic-qe/logs/fleet.log
|
|
```
|
|
|
|
### Verify Databases
|
|
|
|
```bash
|
|
# Check database files exist
|
|
ls -la .agentic-qe/db/
|
|
|
|
# Query database
|
|
node -e "const db = require('better-sqlite3')('.agentic-qe/db/memory.db'); console.log('Tables:', db.prepare('SELECT name FROM sqlite_master WHERE type=\"table\"').all()); db.close();"
|
|
```
|
|
|
|
---
|
|
|
|
## Common Workflows
|
|
|
|
### Full Test Generation and Execution
|
|
|
|
```javascript
|
|
// 1. Generate tests
|
|
Task("Generate tests", "Create comprehensive test suite for UserService", "qe-test-generator")
|
|
|
|
// 2. Execute tests
|
|
Task("Execute tests", "Run generated tests with coverage", "qe-test-executor")
|
|
|
|
// 3. Analyze coverage
|
|
Task("Analyze coverage", "Find coverage gaps", "qe-coverage-analyzer")
|
|
|
|
// 4. Run quality gate
|
|
Task("Quality check", "Validate against quality thresholds", "qe-quality-gate")
|
|
```
|
|
|
|
### Security and Performance Validation
|
|
|
|
```javascript
|
|
// Run in parallel
|
|
Task("Security scan", "SAST/DAST security checks", "qe-security-scanner")
|
|
Task("Performance test", "Load test with 1000 users", "qe-performance-tester")
|
|
Task("API contract check", "Validate API contracts", "qe-api-contract-validator")
|
|
```
|
|
|
|
### Flaky Test Detection and Stabilization
|
|
|
|
```javascript
|
|
Task("Hunt flaky tests", "Detect and stabilize flaky tests with ML", "qe-flaky-test-hunter")
|
|
```
|
|
|
|
---
|
|
|
|
**Related Documentation:**
|
|
- [Agent Reference](agents.md) - All 19 QE agents
|
|
- [Skills Reference](skills.md) - All 41 QE skills
|
|
|
|
**Related Policies:**
|
|
- [Release Verification Policy](../policies/release-verification.md)
|
|
- [Test Execution Policy](../policies/test-execution.md)
|
|
- [Git Operations Policy](../policies/git-operations.md)
|