skills/proffesor-for-testing__agentic-qe
1
0
Fork 0
mirror of https://github.com/proffesor-for-testing/agentic-qe.git synced 2026-09-19 08:45:47 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
context-meetup
proffesor-for-testing__agen…/README.md
T
Dragan Spiridonov 5d187de2a1 fix(install,rvf,hypergraph): unblock Windows install, fix RVF FsyncFailed, synthesize covers edges 
Bundles fixes for issue #439 (Windows install) and Jordi RUFLO patches
P020 (RVF init) + P220 (hypergraph blindness).

Windows install (#439):
- Move hnswlib-node from dependencies to optionalDependencies so
  installs no longer fail on Windows boxes without VS C++ Build Tools;
  HnswAdapter already routes around a missing native binary at runtime.
- Replace HNSWIndex.ts top-level static `import hnswlib from
  'hnswlib-node'` with a lazy require so package load survives the
  optional dep being absent.
- README + preinstall.cjs: document toolchain requirement and disclose
  that the JS fallback degrades to O(N) brute-force when @ruvector/gnn
  is also unavailable (Windows default — no win32 prebuilds ship).
- ADR-090 amendment reconciling the move with ADR-081's optional-dep +
  JS-fallback intent.

RVF init (Jordi P020):
- Root cause: shared-rvf-adapter and pattern-store factory always called
  RvfDatabase.create(), which throws `0x0303 FsyncFailed` when the .rvf
  file already exists from a prior init phase. Reproduced and confirmed
  against both 0.1.7 and 0.1.8 native binaries — not a version
  regression.
- Fix: race-tolerant open-or-create ladder (try open → fall back to
  create → retry open on create-race) with dim() verification after
  open. Mismatched-dim files are closed and the caller degrades rather
  than corrupting silently.

Hypergraph (Jordi P220):
- Drop the original flushEdgesToHypergraph (wrote `call`/`extends`
  edges that no production query reads — devil's-advocate review caught
  it as dead infrastructure).
- Add HypergraphEngine.synthesizeTestCoverage that re-tags test files
  type='test' and writes `covers` edges from each test file to
  functions in the source files it imports. This is the shape
  findUntestedFunctions / findImpactedTests actually filter on.
- E2E verified: untested correctly excludes covered functions;
  impacted-tests returns the right test file.

Verification:
- `npm run build` clean
- 276/276 tests pass across HNSW, RVF, KG, coordinator, hypergraph
  suites (5 new synthesizeTestCoverage tests; replaced 4 obsolete
  flush tests)
- E2E on fresh project: aqe init clean, hooks stats shows
  nativeAvailable: true, code index logs `Test coverage synthesized`,
  hg untested + hg impacted return useful results

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-08 12:22:58 +00:00

476 lines
21 KiB
Markdown
Raw Permalink Blame History

# Agentic Quality Engineering Fleet
<div align="center">
[![npm version](https://img.shields.io/npm/v/agentic-qe.svg)](https://www.npmjs.com/package/agentic-qe)
[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
[![TypeScript](https://img.shields.io/badge/TypeScript-5.0+-blue.svg)](https://www.typescriptlang.org/)
[![Monthly Downloads](https://img.shields.io/npm/dm/agentic-qe)](https://www.npmjs.com/package/agentic-qe)
[![Total Downloads](https://img.shields.io/npm/dt/agentic-qe?label=total%20downloads)](https://www.npmjs.com/package/agentic-qe)
[Release Notes](docs/releases/README.md) | [Changelog](CHANGELOG.md) | [Issues](https://github.com/proffesor-for-testing/agentic-qe/issues) | [Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
**AI-powered quality engineering agents that generate tests, find coverage gaps, detect flaky tests, and learn your codebase patterns — across 11 coding agent platforms.**
</div>
---
## What AQE Does For You
- **Generates comprehensive tests automatically** — unit, integration, property-based, and BDD scenarios for your codebase with framework-specific output (Jest, Vitest, Playwright, Cypress, pytest, JUnit, Go, Rust, Swift, Flutter, and more)
- **Finds coverage gaps and prioritizes what to test** — risk-weighted analysis identifies the most impactful untested code paths
- **Detects and fixes flaky tests** — ML-powered detection with root cause analysis and stabilization recommendations
- **Learns your codebase patterns over time** — remembered patterns are reused across sessions and projects, improving with every interaction
- **Coordinates 60 specialized QE agents** — from test generation to security scanning to chaos engineering, orchestrated by a central coordinator
- **Reduces AI costs with intelligent routing** — automatically routes tasks to the right model tier (fast/cheap for simple tasks, powerful for complex ones)
- **Works with your existing tools** — integrates with 11 coding agent platforms and your existing CI/CD pipeline
---
## Quick Start
```bash
# Install
npm install -g agentic-qe
# Initialize your project (auto-detects tech stack, configures MCP)
cd your-project && aqe init --auto
# That's it — MCP tools are available immediately in Claude Code
# For other clients: aqe-mcp
```
After init, your coding agent can use AQE tools directly. For example in Claude Code:
```
"Generate tests for src/services/UserService.ts with 90% coverage target"
"Find coverage gaps in src/ and prioritize by risk"
"Run security scan on the authentication module"
"Analyze why tests in auth/ are flaky and suggest fixes"
```
### Windows install
`agentic-qe` runs on Windows, but several of its performance-oriented native
dependencies (`hnswlib-node` for HNSW search, `@ruvector/gnn` for graph
neural networks, `@ruvector/rvf-node` for the RVF pattern store) ship as
optional native modules. `hnswlib-node` in particular has no prebuilt
binaries and compiles from source via `node-gyp`. `npm install` does not
fail when these modules can't build — they're declared optional and AQE
falls back to JavaScript paths at runtime.
**Important caveat about the fallback.** The pure-JavaScript HNSW fallback
(`ProgressiveHnswBackend`) is correct but degrades to O(N) brute-force search
when neither `hnswlib-node` nor `@ruvector/gnn` is available. That's fine for
small projects but **unsuitable for large indexes** (tens of thousands of
vectors and up). If you plan to run AQE against a sizeable codebase on
Windows, install the native build toolchain so `hnswlib-node` compiles:
- **Python 3** (added to `PATH`), and
- **Visual Studio 2022 Build Tools** with the *Desktop development with C++*
workload — works with any node-gyp version, **or**
- **Visual Studio 2026** with the same workload **plus** an upgraded npm
(`npm install -g npm@latest` — VS 2026 detection requires npm ≥ 11.6.3,
shipped via node-gyp ≥ 12.1.0). Node 22 LTS still ships npm 10.x by
default, which cannot detect VS 2026; either upgrade npm globally or use
VS 2022 Build Tools instead.
If the native dep fails to build you'll see a `node-gyp` warning during
install (e.g. `gyp ERR! find VS`); the install itself completes. To verify
which backend is active:
```bash
aqe health
# Look for "HNSW backend: native" (hnswlib-node) or "HNSW backend: js"
# (ProgressiveHnswBackend — see caveat above).
```
Linux and macOS users: no extra setup required. The native binary compiles
out of the box.
---
## Claude Code Plugin (Alternative Install)
If you only need a slim, scoped fleet inside Claude Code — without the full `aqe init` setup — install the **`agentic-qe-fleet`** plugin. It bundles 11 specialized QE agents, 9 slash commands, 9 skills, and auto-registers the MCP server.
### Install from a local checkout
```bash
git clone https://github.com/proffesor-for-testing/agentic-qe.git
claude --plugin-dir ./agentic-qe/plugins/agentic-qe-fleet
```
### Install from the marketplace
In any Claude Code session:
```
/plugin marketplace add proffesor-for-testing/agentic-qe
/plugin install agentic-qe-fleet
```
### What you get
| Asset | Count | Notes |
|---|---|---|
| **Agents** (Task tool) | 11 | Model-routed: 6 on Opus (heavy reasoning), 5 on Sonnet (focused execution) |
| **Slash commands** | 9 | `/aqe-analyze`, `/aqe-execute`, `/aqe-generate`, `/aqe-optimize`, `/aqe-chaos`, `/aqe-fleet-status`, `/aqe-report`, `/aqe-benchmark`, `/aqe-costs` |
| **Skills** | 9 | All trust-tier 2 or 3 (validated/verified). Tier-1 untested skills excluded per policy. |
| **MCP server** | 1 | Auto-registers via `npx -y agentic-qe@latest mcp` — no separate `claude mcp add` |
**Bundled agents:** `qe-test-architect`, `qe-coverage-specialist`, `qe-flaky-hunter`, `qe-chaos-engineer`, `qe-fleet-commander`, `qe-quality-gate`, `qe-security-scanner`, `qe-performance-tester`, `qe-regression-analyzer`, `qe-tdd-specialist`, `qe-requirements-validator`.
**Bundled skills:** `qe-test-generation`, `qe-coverage-analysis`, `qe-test-execution`, `qe-chaos-resilience`, `qe-quality-assessment`, `chaos-engineering-resilience`, `mutation-testing`, `risk-based-testing`, `tdd-london-chicago`.
### Use it
After loading the plugin, the slash commands and agents are available immediately:
```
/aqe-fleet-status # health and metrics
/aqe-generate src/services/Auth.ts
/aqe-analyze src/ # coverage gap analysis
```
Or invoke an agent through the Task tool:
```
"Use qe-test-architect to generate tests for src/services/PaymentService.ts"
"Use qe-flaky-hunter to find and stabilize flaky tests in tests/integration/"
"Use qe-chaos-engineer to inject network partitions into the order workflow"
```
### Plugin vs `aqe init` — which to use?
| | **Plugin** | **`aqe init`** |
|---|---|---|
| Setup | One slash command | Full project setup |
| Scope | 11 agents, 9 skills | 60 agents, 85 skills |
| Persistent learning DB | No (uses MCP server's) | Yes (`.agentic-qe/memory.db`) |
| Cross-platform support | Claude Code only | 11 platforms (Cursor, Copilot, Cline, etc.) |
| Use when | Quick start, single Claude Code project | Production team setup, multi-platform, full fleet |
You can run both — the plugin's MCP server uses the same `agentic-qe` package, so installing both gives you the full fleet via `aqe init` and the slash-command shortcuts via the plugin.
---
## Platform Support
AQE works with **11 coding agent platforms** through a single MCP server:
| Platform | Setup |
|----------|-------|
| **Claude Code** | `aqe init --auto` (built-in) |
| **GitHub Copilot** | `aqe init --auto --with-copilot` |
| **Cursor** | `aqe init --auto --with-cursor` |
| **Cline** | `aqe init --auto --with-cline` |
| **OpenCode** | `aqe init --auto --with-opencode` |
| **AWS Kiro** | `aqe init --auto --with-kiro` |
| **Kilo Code** | `aqe init --auto --with-kilocode` |
| **Roo Code** | `aqe init --auto --with-roocode` |
| **OpenAI Codex CLI** | `aqe init --auto --with-codex` |
| **Windsurf** | `aqe init --auto --with-windsurf` |
| **Continue.dev** | `aqe init --auto --with-continuedev` |
```bash
# Set up all platforms at once
aqe init --auto --with-all-platforms
# Or add a platform later
aqe platform setup cursor
aqe platform list # show install status
aqe platform verify cursor # validate config
```
For detailed per-platform instructions, see [Platform Setup Guide](docs/platform-setup-guide.md).
---
## Usage Examples
### Generate Tests
```bash
claude "Use qe-test-architect to create tests for PaymentService with 95% coverage target"
```
Output:
```
Generated 48 tests across 4 files
- unit/PaymentService.test.ts (32 unit tests)
- property/PaymentValidation.property.test.ts (8 property tests)
- integration/PaymentFlow.integration.test.ts (8 integration tests)
Coverage: 96.2%
Pattern reuse: 78% from learned patterns
```
### Full Quality Pipeline
```bash
claude "Use qe-queen-coordinator to run full quality assessment:
1. Generate tests for src/services/*.ts
2. Analyze coverage gaps with risk scoring
3. Run security scan
4. Validate quality gate at 90% threshold
5. Provide deployment recommendation"
```
The Queen Coordinator spawns domain-specific agents, runs them in parallel, and synthesizes a final recommendation.
### TDD Workflow
```bash
claude "Use qe-tdd-specialist to implement UserAuthentication with full RED-GREEN-REFACTOR cycle"
```
Coordinates 5 subagents: write failing tests → implement minimal code → refactor → code review → security review.
### Security Audit
```bash
claude "Coordinate security audit:
- SAST/DAST scanning with qe-security-scanner
- Dependency vulnerability scanning with qe-dependency-mapper
- API security with qe-contract-validator
- Chaos resilience testing with qe-chaos-engineer"
```
---
## 60 QE Agents
The fleet is organized into **13 domains**, coordinated by the **qe-queen-coordinator**:
| Domain | Agents | What They Do |
|--------|--------|-------------|
| **Test Generation** | test-architect, tdd-specialist, mutation-tester, property-tester | Generate tests, TDD workflows, validate test effectiveness |
| **Test Execution** | parallel-executor, retry-handler, integration-tester | Run tests in parallel, handle retries, integration testing |
| **Coverage Analysis** | coverage-specialist, gap-detector | Find untested code, prioritize by risk |
| **Quality Assessment** | quality-gate, risk-assessor, deployment-advisor, devils-advocate | Go/no-go decisions, risk scoring, adversarial review |
| **Defect Intelligence** | defect-predictor, root-cause-analyzer, flaky-hunter, regression-analyzer | Predict bugs, find root causes, fix flaky tests |
| **Requirements** | requirements-validator, bdd-generator | Validate testability, generate BDD scenarios |
| **Code Intelligence** | code-intelligence, kg-builder, dependency-mapper, impact-analyzer | Knowledge graphs, semantic search, change impact |
| **Security** | security-scanner, security-auditor, pentest-validator | SAST/DAST, compliance audits, exploit validation |
| **Contracts** | contract-validator, graphql-tester | API contracts, GraphQL schema testing |
| **Visual & A11y** | visual-tester, accessibility-auditor, responsive-tester | Visual regression, WCAG compliance, viewport testing |
| **Chaos & Performance** | chaos-engineer, load-tester, performance-tester | Fault injection, load testing, performance validation |
| **Learning** | learning-coordinator, pattern-learner, transfer-specialist, metrics-optimizer | Cross-project learning, pattern discovery |
| **Enterprise** | soap-tester, sap-rfc-tester, sap-idoc-tester, sod-analyzer, odata-contract-tester, middleware-validator, message-broker-tester | SAP, SOAP, ESB, OData, JMS/AMQP/Kafka |
Plus **7 TDD subagents** (red, green, refactor, code/integration/performance/security reviewers) and the **fleet-commander** for large-scale orchestration.
---
## 75 QE Skills
Agents automatically apply relevant skills from the skill library. Skills are rated by **trust tier**:
| Tier | Count | Meaning |
|------|-------|---------|
| **Tier 3 — Verified** | 49 | Full evaluation test suite, production-ready |
| **Tier 2 — Validated** | 7 | Has executable validator |
| **Tier 1 — Structured** | 5 | Has JSON output schema |
| **Tier 0 — Advisory** | 5 | Guidance only |
<details>
<summary><b>View all 75 skills</b></summary>
**Core Testing (12):** agentic-quality-engineering, holistic-testing-pact, context-driven-testing, tdd-london-chicago, xp-practices, risk-based-testing, test-automation-strategy, refactoring-patterns, shift-left-testing, shift-right-testing, regression-testing, verification-quality
**Specialized Testing (13):** accessibility-testing, mobile-testing, database-testing, contract-testing, chaos-engineering-resilience, visual-testing-advanced, security-visual-testing, compliance-testing, compatibility-testing, localization-testing, mutation-testing, performance-testing, security-testing
**Browser Automation (1):** qe-browser (Vibium engine — assert, batch, visual-diff, prompt-injection scanning, semantic intents; see [ADR-091](docs/implementation/adrs/ADR-091-qe-browser-skill-vibium-engine.md))
**Domain Skills (11):** qe-test-generation, qe-test-execution, qe-coverage-analysis, qe-quality-assessment, qe-defect-intelligence, qe-requirements-validation, qe-code-intelligence, qe-visual-accessibility, qe-chaos-resilience, qe-learning-optimization, qe-iterative-loop
**Strategic (8):** six-thinking-hats, brutal-honesty-review, sherlock-review, cicd-pipeline-qe-orchestrator, bug-reporting-excellence, consultancy-practices, quality-metrics, pair-programming
**Testing Techniques (9):** exploratory-testing-advanced, test-design-techniques, test-data-management, test-environment-management, test-reporting-analytics, testability-scoring, technical-writing, code-review-quality, api-testing-patterns
**On-Demand Hooks (5):** strict-tdd, no-skip, coverage-guard, freeze-tests, security-watch
**Runbooks & Analysis (5):** test-failure-investigator, coverage-drop-investigator, e2e-flow-verifier, test-metrics-dashboard, skill-stats
**n8n Workflow Testing (5):** n8n-workflow-testing-fundamentals, n8n-expression-testing, n8n-security-testing, n8n-trigger-testing-strategies, n8n-integration-testing-patterns
**QCSD Swarms (5):** qcsd-ideation-swarm, qcsd-refinement-swarm, qcsd-development-swarm, qcsd-cicd-swarm, qcsd-production-swarm
**Accessibility (2):** a11y-ally, accessibility-testing
**Enterprise Integration (5):** enterprise-integration-testing, middleware-testing-patterns, observability-testing-patterns, wms-testing-patterns, pentest-validation
**Validation (1):** validation-pipeline
</details>
---
## How It Works
### Agent Coordination
The **Queen Coordinator** orchestrates agents across all 13 domains. When you ask for a quality assessment, the Queen decomposes the task, spawns the right agents, coordinates their work in parallel, and synthesizes results. Agents communicate through shared memory namespaces and use consensus protocols for critical quality decisions.
### Pattern Learning
AQE learns from every interaction. Successful test patterns, coverage strategies, and defect indicators are stored and indexed for fast retrieval. When generating tests for a new service, AQE searches for similar patterns from past sessions — even across different projects. Patterns improve over time through experience replay and dream cycles (background consolidation).
```bash
aqe learning stats # view learning statistics
aqe learning dream # trigger pattern consolidation
aqe brain export # export learned patterns for sharing
```
### Intelligent Model Routing
**TinyDancer** routes tasks to the right model tier to minimize cost without sacrificing quality:
| Task Complexity | Model | Examples |
|----------------|-------|---------|
| Simple (0-20) | Haiku | Type additions, simple refactors |
| Moderate (20-70) | Sonnet | Bug fixes, test generation |
| Critical (70+) | Opus | Architecture, security, complex reasoning |
### Quality Gates
Anti-sycophancy scoring catches hollow tests. Tautological assertions (`expect(true).toBe(true)`) are rejected. Edge cases from historical patterns are injected into test generation. See [Loki-mode features](docs/loki-mode-features.md).
---
## CLI Reference
```bash
aqe init [--auto] # Initialize project
aqe agent list # List available agents
aqe fleet status # Fleet health and coordination
aqe learning stats # Learning statistics
aqe learning dream # Trigger dream cycle
aqe brain export/import # Portable intelligence
aqe platform list/setup/verify # Manage coding agent platforms
aqe health # System health check
# Code intelligence
aqe code index src/ # Index codebase into knowledge graph
aqe code index src/ --incremental # Incremental index (changed files only)
aqe code index . --git-since HEAD~5 # Index files changed in last 5 commits
aqe code search "authentication" # Semantic code search
aqe code impact src/ # Change impact analysis
aqe code deps src/ # Dependency mapping
aqe code complexity src/ # Complexity metrics and hotspots
```
---
## LLM Providers
| Provider | Type | Cost | Best For |
|----------|------|------|----------|
| **Ollama** | Local | Free | Privacy, offline |
| **OpenRouter** | Cloud | Varies | 300+ models |
| **Groq** | Cloud | Free | High-speed |
| **Claude API** | Cloud | Paid | Highest quality |
| **Google AI** | Cloud | Free | Gemini models |
```bash
export GROQ_API_KEY="gsk_..." # or any supported provider
aqe init --auto
```
---
## Documentation
| Guide | Description |
|-------|-------------|
| [Platform Setup](docs/platform-setup-guide.md) | Per-platform configuration instructions |
| [Skill Validation](docs/guides/skill-validation.md) | Trust tiers and evaluation system |
| [Learning System](docs/guides/reasoningbank-learning-system.md) | ReasoningBank pattern learning |
| [Code Intelligence](docs/guides/fleet-code-intelligence-integration.md) | Knowledge graph and semantic search |
| [Loki-mode Features](docs/loki-mode-features.md) | Anti-sycophancy and quality gates |
| [Release Notes](docs/releases/README.md) | Version history and changelogs |
| [Architecture Glossary](docs/v3-technical-architecture-glossary.md) | Technical terms and concepts |
---
## Development
```bash
git clone https://github.com/proffesor-for-testing/agentic-qe.git
cd agentic-qe
npm install
npm run build
npm test -- --run
```
| Script | Description |
|--------|-------------|
| `npm run build` | Compile TypeScript + CLI + MCP bundles |
| `npm test -- --run` | Run all tests |
| `npm run cli` | Run CLI in dev mode |
| `npm run mcp` | Start MCP server |
---
## Contributing
We welcome contributions! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for details.
---
## Support
- **Documentation**: [docs/](docs/)
- **Issues**: [GitHub Issues](https://github.com/proffesor-for-testing/agentic-qe/issues)
- **Discussions**: [GitHub Discussions](https://github.com/proffesor-for-testing/agentic-qe/discussions)
---
## License
MIT — see [LICENSE](LICENSE).
---
## Contributors
<!-- ALL-CONTRIBUTORS-LIST:START -->
| <img src="https://github.com/proffesor-for-testing.png" width="60" style="border-radius:50%"/><br/>**[@proffesor-for-testing](https://github.com/proffesor-for-testing)**<br/>Project Lead | <img src="https://github.com/fndlalit.png" width="60" style="border-radius:50%"/><br/>**[@fndlalit](https://github.com/fndlalit)**<br/>QX Partner, Testability | <img src="https://github.com/shaal.png" width="60" style="border-radius:50%"/><br/>**[@shaal](https://github.com/shaal)**<br/>Core Development | <img src="https://github.com/mondweep.png" width="60" style="border-radius:50%"/><br/>**[@mondweep](https://github.com/mondweep)**<br/>Architecture |
|:---:|:---:|:---:|:---:|
<!-- ALL-CONTRIBUTORS-LIST:END -->
[View all contributors](CONTRIBUTORS.md) | [Become a contributor](CONTRIBUTING.md)
---
## Support the Project
If you find AQE valuable, consider supporting its development:
| | Monthly | Annual (Save $10) |
|---|:---:|:---:|
| **Price** | $5/month | $50/year |
| **Subscribe** | [**Monthly**](https://www.paypal.com/webapps/billing/plans/subscribe?plan_id=P-88G03706DU8150205NEYZZAY) | [**Annual**](https://www.paypal.com/webapps/billing/plans/subscribe?plan_id=P-39189175UE6623540NEYZ2CI) |
[View sponsorship details](FUNDING.md)
---
## Acknowledgments
- **[Claude Flow](https://github.com/ruvnet/claude-flow)** by [@ruvnet](https://github.com/ruvnet) — Multi-agent orchestration and MCP integration
- **[Agentic Flow](https://github.com/ruvnet/agentic-flow)** by [@ruvnet](https://github.com/ruvnet) — Agent patterns and learning systems
- Built with TypeScript, Node.js, and better-sqlite3
- Compatible with Jest, Cypress, Playwright, Vitest, Mocha, pytest, JUnit, and more
---
<div align="center">
**Made with care by the Agentic QE Team**
[Star us on GitHub](https://github.com/proffesor-for-testing/agentic-qe) | [Sponsor](FUNDING.md) | [Contributors](CONTRIBUTORS.md)
</div>
Reference in New Issue View Git Blame Copy Permalink