mirror of
https://github.com/proffesor-for-testing/agentic-qe.git
synced 2026-09-19 08:45:47 +08:00
Dragan Spiridonov
99511738c1
* fix(v3.4.1): bundle missing dependencies and test timeouts - Fixed issue #219: MCP bundle failing due to missing packages - Changed build scripts from --packages=external to selective externalization - Pure JS deps (fast-json-patch, jose, uuid) now bundled inline - Native modules properly externalized (better-sqlite3, hnswlib-node, etc.) - CommonJS modules with dynamic requires externalized (typescript, fast-glob, yaml, commander, cli-progress, ora) - Bundle size reduced from ~15MB to ~5MB - Added timeouts to handleQualityAssess tests to prevent CI failures - Updated version to 3.4.1 Closes #219 Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * feat(v3.4.2): add skill validation system with trust tiers (ADR-056) Implement 4-layer skill validation system for deterministic outputs: - Tier 3 (Verified): 46 skills with full eval test suites - Tier 2 (Validated): 7 skills with executable validators - Tier 1 (Structured): 5 skills with JSON output schemas - Tier 0 (Advisory): 5 skills with SKILL.md guidance only Key additions: - JSON Schema validation templates for skill outputs - Executable validator scripts for correctness verification - Evaluation test framework with multi-model support - CLI commands: aqe skill report/summary/compare, aqe eval run/status - ReasoningBank integration for validation pattern learning - GitHub Actions workflow for CI skill validation - Trust tier frontmatter in all 63 skill SKILL.md files - Comprehensive documentation and user guides Infrastructure: - .claude/skills/.validation/ - Templates and frameworks - .claude/skills/*/schemas/ - Output schemas per skill - .claude/skills/*/scripts/ - Validator scripts per skill - .claude/skills/*/evals/ - Test cases per skill Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(ci): trust tier-based skill validation - Tier 3 (Verified): Blocks PR on validation failure - Tier 2 (Validated): Warns only, doesn't block - Tier 1 (Structured): Warns only, doesn't block - Tier 0 (Advisory): Skipped entirely Reads trust_tier from SKILL.md frontmatter to determine validation requirements. Only 46 Tier 3 skills block merge. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(validators): remove local keyword from self-test blocks Shell script fix: 'local' can only be used inside functions, not in if/else blocks. Both accessibility-testing and compatibility-testing validators had 'local' declarations in their self-test sections that caused errors when run outside of a function context. - Remove local test_file from accessibility-testing self-test - Remove local test_file, browser_count, tier1_pass from compatibility-testing self-test - Sync fixes to v3/assets/skills/ Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> * fix(validators): add --self-test support to n8n skill validators The 5 n8n skill validators were failing CI because they didn't handle the --self-test flag properly. Added self-test handling that: - Checks for required tools (jq) - Verifies schema file exists and is valid JSON - Returns exit 0 on success Fixed validators: - n8n-expression-testing - n8n-integration-testing-patterns - n8n-security-testing - n8n-trigger-testing-strategies - n8n-workflow-testing-fundamentals Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
2.6 KiB
2.6 KiB
Implementation Plans
This directory contains GOAP-style implementation plans for major AQE features.
Active Plans
Skill Validation: Trust But Verify
Status: Planning Priority: High Estimated Effort: 214 hours across 5 phases
Implements a 4-layer validation system for AQE skills:
Level 3: Evaluation Suite (measure behavior)
Level 2: Executable Validator (verify output)
Level 1: JSON Schema (structure output)
Level 0: SKILL.md Instructions (guide intent)
Key Deliverables:
- Trust tier frontmatter extension for SKILL.md
- JSON output schemas for all P0 skills
- Executable validators with graceful degradation
- YAML-based evaluation test suites
- CI/CD workflow for multi-model validation
- ReasoningBank integration for learning
Related Resources:
- Schemas:
/docs/schemas/ - Templates:
/docs/templates/ - Exemplars:
.claude/skills/testability-scoring/,.claude/skills/brutal-honesty-review/
Plan Structure
Each plan follows the SPARC-GOAP methodology:
- Specification: Define goal state and success criteria
- Pseudocode: Plan action sequences and state transitions
- Architecture: Structure solution components
- Refinement: TDD implementation with iteration
- Completion: Integration, validation, deployment
Milestone Template
Milestone X.Y: Name
Description: What this milestone achieves
Complexity: Low/Medium/High
Estimated Hours: N
Agent Assignment: qe-agent-name
Dependencies: [milestone IDs]
Parallel Opportunity: Yes/No
Deliverables:
- path/to/deliverable
Success Criteria:
- [ ] Measurable outcome
Memory Namespaces
Plans define memory namespaces for pattern learning:
aqe/{feature}/
├── patterns/ - Learned patterns
├── outcomes/ - Execution outcomes
├── metrics/ - Performance metrics
└── trends/ - Historical trends
Creating New Plans
- Copy the structure from an existing plan
- Define clear goal state vs current state
- Break into SPARC phases with milestones
- Assign agents and estimate complexity
- Map dependencies and parallel opportunities
- Define success criteria for each milestone
- Add to this README