proffesor-for-testing__agentic-qe/docs/optimization

Skill Optimization Framework for Claude 4.5

Overview

This framework enables systematic optimization of all 41 Agentic QE skills for Claude 4.5, targeting 40-50% token reduction while improving agent coordination and maintaining quality.

Quick Start

Validate a Single Skill

npm run validate-skill .claude/skills/performance-testing/SKILL.md

Validate All Skills

npm run validate-all-skills

Generate Optimization Report

npm run optimize:report

Framework Components

1. Claude 4.5 Patterns Guide

Location: claude-4-5-patterns.md

Comprehensive guide covering:

• Action-first directive patterns (<default_to_action>)
• Token compression techniques (60-75% reduction strategies)
• Structured output templates
• Quick reference card format
• Agent coordination patterns
• Implementation status tracking

Key Techniques:

• Remove verbose explanations → 75% token savings
• Use tables for dense information → 60% savings
• Consolidate redundant content → 60% savings
• Add action-first directives → Immediate agent guidance

2. Optimized Skill Template

Location: ../templates/optimized-skill-template.md

Standardized template with:

• YAML frontmatter (metadata, agents, dependencies)
• <default_to_action> block (100-200 tokens)
• Quick reference card section (200-400 tokens)
• Agent coordination hints with memory schemas
• Compressed core content
• Structured examples

Token Budget:

• YAML frontmatter: ~80 tokens
• <default_to_action>: 100-200 tokens
• Quick reference card: 200-400 tokens
• Core concepts: 300-500 tokens
• Implementation patterns: 200-400 tokens
• Agent coordination: 150-250 tokens
• Examples: 150-300 tokens
• Total target: 1,200-1,500 tokens

3. Validation Script

Location: ../../scripts/validate-skill-optimization.ts

Automated validation checking:

• ✅ YAML frontmatter completeness
• ✅ Required sections (<default_to_action>, quick reference, agent coordination)
• ✅ Token estimates (target: ≤1,500 tokens)
• ✅ Agent associations match manifest
• ✅ Implementation status accuracy
• ✅ Cross-references to dependencies

Features:

• Single skill or full fleet validation
• Optimization metrics (before/after comparison)
• Error reporting with severity levels
• Warning generation for improvement opportunities
• Detailed reports with actionable insights

Optimization Workflow

Phase 1: Baseline Analysis

# Validate current state of all skills
npm run validate-all-skills

# Review baseline metrics
# Total Skills: 41
# Baseline: 41 (needs optimization)
# Estimated baseline tokens: ~90,000

Phase 2: Apply Optimization Patterns

For each skill:

1. Read existing skill

   cat .claude/skills/performance-testing/SKILL.md
   

2. Apply optimization patterns

   • Add YAML frontmatter with metadata
   • Insert <default_to_action> block at top
   • Create quick reference card section
   • Compress verbose sections (paragraphs → bullets, tables)
   • Add agent coordination hints
   • Update implementation_status to optimized

3. Validate optimized skill

   npm run validate-skill .claude/skills/performance-testing/SKILL.md
   

4. Test with actual agent

   Task("Performance test", "Run load test on API", "qe-performance-tester")
   

Phase 3: Validation & Testing

# Full validation report
npm run optimize:report

# Expected output:
# ✅ Optimized: 1/41
# Token Savings: 1,300 tokens saved
# Average Reduction: 50%
# Target (45%): ✅ TARGET MET

Phase 4: Quality Gates

Before marking as production:

• All validation errors resolved
• Token reduction ≥40% from baseline
• Quick reference card present
• Agent coordination section complete
• Tested with at least one agent
• Cross-references updated
• Implementation status accurate

Validation Output Examples

Single Skill Validation

$ npm run validate-skill .claude/skills/performance-testing/SKILL.md

✅ PASSED performance-testing
  Path: /workspaces/agentic-qe-cf/.claude/skills/performance-testing/SKILL.md
  Status: optimized
  Tokens: 1,300 (Target: ≤1,500)
  Reduction: 50.0% from baseline (2,600 tokens)

  Sections:
    YAML Frontmatter: ✅
    <default_to_action>: ✅
    Quick Reference Card: ✅
    Agent Coordination: ✅
    Core Content: ✅
    Examples: ✅
    Related Skills: ✅

Full Fleet Report

$ npm run validate-all-skills

================================================================================
SKILL OPTIMIZATION REPORT
================================================================================

Total Skills: 41
  ✅ Optimized: 5
  📝 Baseline: 35
  ⚠️  Draft: 1

Optimization Progress: 12.2%

Token Savings: 6,500 tokens saved
Average Reduction: 48.3%
Target (45%): ✅ TARGET MET

Top Optimizations:
  1. performance-testing: 50.0% reduction
  2. testability-scoring: 48.5% reduction
  3. api-testing-patterns: 47.2% reduction
  4. security-testing: 46.8% reduction
  5. tdd-london-chicago: 45.3% reduction

Skills Needing Optimization (35):
  • agentic-quality-engineering (4,200 tokens)
  • cicd-pipeline-qe-orchestrator (3,200 tokens)
  • six-thinking-hats (2,300 tokens)
  • exploratory-testing-advanced (2,200 tokens)
  • chaos-engineering-resilience (2,200 tokens)
  ... and 30 more

================================================================================

Error Types & Fixes

Critical Errors (Block Production)

Missing Frontmatter:

---
name: skill-name
category: specialized-testing
priority: high
tokenEstimate: 1300
agents: [qe-agent-1]
implementation_status: optimized
---

Missing <default_to_action>:

<default_to_action>
When [context]:
1. ACTION: Step with outcome
2. ACTION: Step with validation
3. ACTION: Step with completion
</default_to_action>

Missing Quick Reference:

## Quick Reference Card

### When to Use
- Trigger 1
- Trigger 2

### Key Actions
1. Action with outcome
2. Validation with criteria

High Severity Warnings

Token Limit Exceeded:

• Current: 2,600 tokens
• Target: ≤1,500 tokens
• Fix: Apply compression techniques from patterns guide

Agent Mismatch:

• Manifest agents: [qe-agent-1, qe-agent-2]
• Skill agents: [qe-agent-1]
• Fix: Add missing agents to frontmatter

Medium Severity Warnings

Inaccurate Token Estimate:

• Estimated: 1,300 tokens
• Declared: 2,000 tokens
• Fix: Update tokenEstimate in frontmatter

Optimization Metrics Dashboard

Current State (Example)

fleet_optimization_status:
  total_skills: 41
  optimized: 5
  baseline: 35
  draft: 1

  progress: 12.2%

  tokens:
    baseline_total: 90,000
    current_total: 83,500
    saved: 6,500
    avg_reduction: 48.3%

  target:
    reduction_goal: 45%
    status: "ON TRACK" # ✅

  priorities:
    critical: 2 skills
    high: 12 skills
    medium: 23 skills
    low: 4 skills

Target Metrics

completion_goals:
  optimized_skills: 41/41 (100%)
  token_reduction: 40-50%
  total_token_savings: 40,500 tokens
  estimated_final: 49,500 tokens

  quality_gates:
    - All skills have quick reference cards
    - All skills have agent coordination sections
    - Zero critical validation errors
    - 100% agent association accuracy
    - All skills tested with real agents

Priority-Based Optimization Order

1. Critical Skills (2 skills) - Do First

1. agentic-quality-engineering (4,200 tokens) - Foundation for all agents
2. holistic-testing-pact (1,500 tokens) - Core PACT principles

2. High Priority (12 skills) - Do Second

3. tdd-london-chicago (2,800 tokens)
4. api-testing-patterns (2,500 tokens)
5. cicd-pipeline-qe-orchestrator (3,200 tokens)
6. performance-testing (2,600 tokens) ✅ DONE
7. security-testing (2,400 tokens)
8. test-design-techniques (2,400 tokens)
9. shift-left-testing (1,700 tokens)
10. shift-right-testing (1,900 tokens)
11. test-automation-strategy (2,200 tokens)
12. context-driven-testing (1,800 tokens)
13. exploratory-testing-advanced (2,200 tokens)
14. risk-based-testing (1,900 tokens)

3. Medium Priority (23 skills) - Do Third

All specialized testing, analysis-review, and infrastructure skills.

4. Low Priority (4 skills) - Do Last

38. consultancy-practices (1,600 tokens)
39. refactoring-patterns (1,900 tokens)
40. technical-writing (1,700 tokens)
41. pair-programming (2,100 tokens)

Integration with Agents

Memory Coordination

Optimized skills use standardized memory namespaces:

// Store test results
memory.store('aqe/performance/load-test/2025-12-02', {
  testType: 'load',
  target: 'https://api.example.com',
  results: { p95: 180, throughput: 5420 }
});

// Retrieve for coordination
const perfData = await memory.retrieve('aqe/performance/load-test/2025-12-02');

Blackboard Coordination

// Post completion event
blackboard.post('performance/test-completed', {
  priority: 'high',
  payload: { status: 'success', artifacts: ['report.html'] }
});

// Subscribe to quality gate decisions
blackboard.subscribe('quality/gate-evaluation', callback);

Fleet Orchestration

const fleet = await FleetManager.coordinate({
  strategy: 'performance-testing',
  agents: ['qe-performance-tester', 'qe-quality-analyzer'],
  memoryNamespace: 'aqe/performance',
  coordination: { shareResults: true }
});

Best Practices

✅ Do This

1. Validate before and after optimization
2. Test with actual agents to ensure functionality
3. Keep examples minimal but runnable
4. Use tables for comparison data
5. Add clear action directives at the top
6. Document all agent associations
7. Cross-reference related skills
8. Track optimization metrics

❌ Avoid This

1. Over-compression that loses clarity
2. Removing critical context or examples
3. Skipping validation before marking as done
4. Ignoring agent feedback during testing
5. Incomplete frontmatter metadata
6. Missing quick reference cards
7. Vague action directives
8. Breaking cross-skill dependencies

Troubleshooting

Issue: Validation Fails with Token Count Error

ERROR: Token estimate 2,000 differs from actual 1,300

Solution:

# Update frontmatter tokenEstimate to match actual
vim .claude/skills/my-skill/SKILL.md
# Change: tokenEstimate: 2000
# To:     tokenEstimate: 1300

Issue: Missing Section Error

ERROR: Quick Reference Card section is missing

Solution:

# Add section from template
cat docs/templates/optimized-skill-template.md
# Copy "Quick Reference Card" section into skill

Issue: Agent Mismatch

ERROR: Missing agents from manifest: qe-agent-2

Solution:

# Update frontmatter agents array
agents: [qe-agent-1, qe-agent-2]

Monitoring Progress

Daily Progress Tracking

# Generate daily report
npm run optimize:report > reports/optimization-progress-$(date +%Y-%m-%d).txt

# Track changes
git diff HEAD~1 -- .claude/skills/*/SKILL.md

Weekly Milestones

# Week 1: Critical skills (2 skills)
# Week 2: High priority (12 skills)
# Week 3: Medium priority (15 skills)
# Week 4: Medium priority (8 skills)
# Week 5: Low priority (4 skills)

Success Criteria

• 41/41 skills optimized
• Average token reduction ≥45%
• Zero critical validation errors
• All skills tested with agents
• Complete agent coordination documentation
• Full cross-reference integrity

Resources

Documentation

• Patterns Guide: claude-4-5-patterns.md
• Template: optimized-skill-template.md
• Skills Manifest: skills-manifest.json

Tools

• Validation Script: validate-skill-optimization.ts
• NPM Commands:
  • npm run validate-skill [path]
  • npm run validate-all-skills
  • npm run optimize:report

Reference Implementations

• Optimized Example: .claude/skills/performance-testing/SKILL.md (50% reduction)
• Baseline Example: .claude/skills/testability-scoring/SKILL.md (pre-optimization)

Contributing

Adding New Optimization Patterns

1. Document pattern in claude-4-5-patterns.md
2. Add example to template
3. Update validation script checks
4. Test with multiple skills
5. Measure token reduction impact

Reporting Issues

• Validation Errors: Open issue with skill name and error output
• Pattern Suggestions: Share token reduction techniques that worked
• Template Improvements: Suggest better section structures

---

Framework Version: 1.0.0 Last Updated: 2025-12-02 Maintained By: Agentic QE Fleet Architecture Team Target: 40-50% token reduction across 41 skills Status: Ready for full fleet optimization