Files
Profa d7455132bf docs: update agent/skill counts and improve README
- Update agent count from 18 to 19 (added qx-partner)
- Update skill count references to 41 (was inconsistent 37/38/39)
- Add Self-Learning System section to README explaining RL algorithms
- Add Why AQE? section with Problem-Solution table
- Remove outdated What's New and Roadmap sections from README
- Fix flaky detection accuracy claim from 100% to 90%+
- Remove unverified SWE-Bench and token reduction claims
- Add qx-partner to agents list in skills and manifest
- Update all user-facing docs with correct counts
- Add docs/PERFORMANCE.md for verified benchmarks

Files updated:
- README.md, CLAUDE.md, FUNDING.md
- docs/reference/{agents,skills,usage}.md
- .agentic-qe/docs/{agents,skills,usage}.md
- .claude/skills/agentic-quality-engineering/SKILL.md
- .claude/skills/cicd-pipeline-qe-orchestrator/README.md
- .claude/skills/skills-manifest.json
- src/cli/init/{agents,database-init}.ts
- src/telemetry/instrumentation/{agent,index}.ts

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-06 21:12:40 +00:00

13 KiB

Agentic QE Fleet - Usage Guide

Complete guide for using the Agentic QE Fleet in your projects.

Quick Start

Installation

npm install agentic-qe-cf --save-dev

Initialization

# Initialize AQE in your project
npx aqe init

# This creates:
# - .claude/agents/       (19 QE agents)
# - .claude/skills/       (41 QE skills)
# - .claude/commands/     (8 slash commands)
# - .agentic-qe/config/   (fleet configuration)
# - .agentic-qe/db/       (learning databases)
# - CLAUDE.md             (project configuration)

Using Agents via Claude Code Task Tool

Recommended Method: Use Claude Code's Task tool for spawning agents.

Single Agent Execution

Task("Generate tests", "Create comprehensive test suite for UserService", "qe-test-generator")

Parallel Agent Execution

// Execute multiple agents concurrently
Task("Test Generation", "Generate unit tests", "qe-test-generator")
Task("Coverage Analysis", "Analyze current coverage", "qe-coverage-analyzer")
Task("Security Scan", "Run security checks", "qe-security-scanner")
Task("Performance Test", "Load test critical paths", "qe-performance-tester")

Agent Coordination Example

// Step 1: Test generator stores results
Task("Generate tests", "Create tests and store in memory at aqe/test-plan/generated", "qe-test-generator")

// Step 2: Test executor reads from memory
Task("Execute tests", "Read test plan from aqe/test-plan/generated and execute", "qe-test-executor")

// Step 3: Coverage analyzer processes results
Task("Analyze coverage", "Check coverage from aqe/coverage/results", "qe-coverage-analyzer")

Using MCP Tools

Check MCP Connection

claude mcp list
# Should show: agentic-qe: npm run mcp:start - ✓ Connected

Tool Discovery System (New in v2.1.0)

The MCP server uses lazy loading to reduce initial context by 87%.

Discover Available Domains

// List all available tool domains
mcp__agentic_qe__tools_discover()

// Returns:
// {
//   domains: [
//     "security", "coverage", "test-generation", "test-execution",
//     "quality-gates", "fleet-management", "memory", "learning",
//     "patterns", "reporting"
//   ]
// }

Load Domain-Specific Tools

// Load security tools (4 tools)
mcp__agentic_qe__tools_load_domain({ domain: "security" })

// Load coverage tools (3 tools)
mcp__agentic_qe__tools_load_domain({ domain: "coverage" })

// Load test generation tools (2 tools)
mcp__agentic_qe__tools_load_domain({ domain: "test-generation" })

Automatic Loading

Tools are auto-loaded based on keywords in agent instructions:

// Keyword "security" triggers auto-load of security domain
Task("Security scan", "Run security checks for vulnerabilities", "qe-security-scanner")

// Keyword "coverage" triggers auto-load of coverage domain
Task("Coverage analysis", "Find coverage gaps in UserService", "qe-coverage-analyzer")

// Keyword "generate" triggers auto-load of test-generation domain
Task("Generate tests", "Create unit tests for API endpoints", "qe-test-generator")

Recognized Keywords:

  • security: security, vulnerability, sast, dast, owasp
  • coverage: coverage, gaps, untested, branches
  • test-generation: generate, create tests, test suite
  • test-execution: execute, run tests, parallel
  • quality-gates: quality, gate, threshold, deployment
  • fleet-management: fleet, agents, orchestrate, swarm
  • memory: memory, store, retrieve, namespace
  • learning: learn, patterns, q-learning, improve
  • patterns: patterns, extract, search, templates
  • reporting: report, metrics, dashboard, export

Using MCP Tools in Claude Code

// Test generation
mcp__agentic_qe__test_generate({ type: "unit", framework: "jest" })

// Test execution
mcp__agentic_qe__test_execute({ parallel: true, coverage: true })

// Quality analysis
mcp__agentic_qe__quality_analyze({ scope: "full" })

Using CLI Commands

Quick Commands

# Generate tests
aqe test <module-name>

# Analyze coverage
aqe coverage

# Run quality gate
aqe quality

# Check fleet status
aqe status

Learning System Commands

# Check learning status
aqe learn status --agent test-gen

# View learned patterns
aqe learn history --agent test-gen --limit 50

# Export learning data
aqe learn export --agent test-gen --output learning.json

Pattern Management

# List test patterns
aqe patterns list --framework jest

# Search patterns
aqe patterns search "api validation"

# Extract patterns from tests
aqe patterns extract ./tests --framework jest

Improvement Loop

# Start continuous improvement
aqe improve start

# Check improvement status
aqe improve status

# Run single improvement cycle
aqe improve cycle

Using Slash Commands

Available Commands

  • /aqe-execute - Execute test suites with parallel orchestration
  • /aqe-generate - Generate comprehensive test suites
  • /aqe-coverage - Analyze test coverage
  • /aqe-quality - Run quality gate validation
  • /aqe-optimize - Optimize test suites using sublinear algorithms
  • /aqe-report - Generate quality engineering reports
  • /aqe-analyze - Analyze coverage gaps
  • /aqe-fleet-status - Display fleet health and metrics

Example Usage

# In Claude Code chat
/aqe-generate
/aqe-execute
/aqe-coverage

Using Skills

Via CLI

# List all available skills
aqe skills list

# Search for specific skills
aqe skills search "testing"

# Show skill details
aqe skills show agentic-quality-engineering

# Show skill statistics
aqe skills stats

Via Skill Tool in Claude Code

// Execute a skill
Skill("agentic-quality-engineering")
Skill("tdd-london-chicago")
Skill("api-testing-patterns")

Agent Coordination

All agents coordinate through AQE hooks (zero external dependencies, 100-500x faster than shell hooks).

Automatic Lifecycle Hooks

Agents extend BaseAgent and override lifecycle methods:

protected async onPreTask(data: { assignment: TaskAssignment }): Promise<void> {
  // Load context before task execution
  const context = await this.memoryStore.retrieve('aqe/context', {
    partition: 'coordination'
  });

  this.logger.info('Pre-task hook complete');
}

protected async onPostTask(data: { assignment: TaskAssignment; result: any }): Promise<void> {
  // Store results after task completion
  await this.memoryStore.store('aqe/' + this.agentId.type + '/results', data.result, {
    partition: 'agent_results',
    ttl: 86400 // 24 hours
  });

  // Emit completion event
  this.eventBus.emit('task:completed', {
    agentId: this.agentId,
    result: data.result
  });

  this.logger.info('Post-task hook complete');
}

protected async onTaskError(data: { assignment: TaskAssignment; error: Error }): Promise<void> {
  // Handle task errors
  await this.memoryStore.store('aqe/errors/' + data.assignment.id, {
    error: data.error.message,
    stack: data.error.stack,
    timestamp: Date.now()
  }, {
    partition: 'errors',
    ttl: 604800 // 7 days
  });

  this.logger.error('Task failed', { error: data.error });
}

Performance Comparison

Feature AQE Hooks External Hooks
Speed <1ms 100-500ms
Dependencies Zero External package
Type Safety Full TypeScript Shell strings
Integration Direct API Shell commands
Performance 100-500x faster Baseline

Memory Namespace

Agents share state through the aqe/* memory namespace:

  • aqe/test-plan/* - Test planning and requirements
  • aqe/coverage/* - Coverage analysis and gaps
  • aqe/quality/* - Quality metrics and gates
  • aqe/performance/* - Performance test results
  • aqe/security/* - Security scan findings
  • aqe/swarm/coordination - Cross-agent coordination

Fleet Configuration

Configuration is stored in .agentic-qe/config/fleet.json:

{
  "topology": "hierarchical",
  "maxAgents": 10,
  "testingFocus": ["unit", "integration"],
  "environments": ["development"],
  "frameworks": ["jest"]
}

Topology Options

  • hierarchical - Queen-led coordination with specialized workers
  • mesh - Peer-to-peer with distributed decision making
  • ring - Circular coordination pattern
  • star - Centralized hub-and-spoke

Multi-Model Router

Status: Disabled by default (opt-in)

Provides 70-81% cost savings by intelligently selecting AI models based on task complexity.

Enabling Routing

Option 1: Via Configuration

// .agentic-qe/config/routing.json
{
  "multiModelRouter": {
    "enabled": true
  }
}

Option 2: Via Environment Variable

export AQE_ROUTING_ENABLED=true

Model Selection Rules

Task Complexity Model Est. Cost Use Case
Simple GPT-3.5 $0.0004 Unit tests, basic validation
Moderate GPT-3.5 $0.0008 Integration tests, mocks
Complex GPT-4 $0.0048 Property-based, edge cases
Critical Claude Sonnet 4.5 $0.0065 Security, architecture review

Monitoring Costs

# View cost dashboard
aqe routing dashboard

# Export cost report
aqe routing report --format json

# Check savings
aqe routing stats

Streaming Progress

Status: Enabled by default

Real-time progress updates for long-running operations using AsyncGenerator pattern.

Example Usage

// Using streaming MCP tool
const handler = new TestExecuteStreamHandler();

for await (const event of handler.execute(params)) {
  if (event.type === 'progress') {
    console.log(`Progress: ${event.percent}% - ${event.message}`);
  } else if (event.type === 'result') {
    console.log('Completed:', event.data);
  }
}

Supported Operations

  • ✅ Test execution (test-by-test progress)
  • ✅ Coverage analysis (incremental gap detection)
  • ⚠️ Test generation (coming soon)
  • ⚠️ Security scanning (coming soon)

Q-Learning Integration

All agents automatically learn from task execution through Q-learning.

Observability

# Check learning status
aqe learn status --agent test-gen

# View learned patterns
aqe learn history --agent test-gen --limit 50

# Export learning data
aqe learn export --agent test-gen --output learning.json

Pattern Management

# List test patterns
aqe patterns list --framework jest

# Search patterns
aqe patterns search "api validation"

# Extract patterns from tests
aqe patterns extract ./tests --framework jest

Improvement Loop

# Start continuous improvement
aqe improve start

# Check improvement status
aqe improve status

# Run single improvement cycle
aqe improve cycle

Best Practices

  1. Use Task Tool: Claude Code's Task tool is the primary way to spawn agents
  2. Batch Operations: Always spawn multiple related agents in a single message
  3. Memory Keys: Use the aqe/* namespace for agent coordination
  4. AQE Hooks: Agents automatically use native AQE hooks for coordination (100-500x faster)
  5. Parallel Execution: Leverage concurrent agent execution for speed

Troubleshooting

Check MCP Connection

claude mcp list
# Should show: agentic-qe: npm run mcp:start - ✓ Connected

View Agent Definitions

ls -la .claude/agents/
# Should show 18 agent markdown files

Check Fleet Status

aqe status --verbose

View Logs

tail -f .agentic-qe/logs/fleet.log

Verify Databases

# Check database files exist
ls -la .agentic-qe/db/

# Query database
node -e "const db = require('better-sqlite3')('.agentic-qe/db/memory.db'); console.log('Tables:', db.prepare('SELECT name FROM sqlite_master WHERE type=\"table\"').all()); db.close();"

Common Workflows

Full Test Generation and Execution

// 1. Generate tests
Task("Generate tests", "Create comprehensive test suite for UserService", "qe-test-generator")

// 2. Execute tests
Task("Execute tests", "Run generated tests with coverage", "qe-test-executor")

// 3. Analyze coverage
Task("Analyze coverage", "Find coverage gaps", "qe-coverage-analyzer")

// 4. Run quality gate
Task("Quality check", "Validate against quality thresholds", "qe-quality-gate")

Security and Performance Validation

// Run in parallel
Task("Security scan", "SAST/DAST security checks", "qe-security-scanner")
Task("Performance test", "Load test with 1000 users", "qe-performance-tester")
Task("API contract check", "Validate API contracts", "qe-api-contract-validator")

Flaky Test Detection and Stabilization

Task("Hunt flaky tests", "Detect and stabilize flaky tests with ML", "qe-flaky-test-hunter")

Related Documentation:

Related Policies: