Evolve the QE framework from PACT (Proactive, Autonomous, Collaborative,
Targeted) to PACTS by adding a fifth principle: Structured — governance,
observability, and explainability of agent behavior. We measure confidence,
not trust (trust is a human feeling; confidence can be mathematically
explained). Especially important in regulated industries. Inspired by DORA's
research on AI-assisted delivery and platform-engineering quality measurement.
Anti-pattern: bolting agentic AI onto chaos.
Scope:
- Source skills (.claude/skills): holistic-testing-pact + agentic-quality-
engineering SKILL.md, agentic-quality-engineering schema (added optional
`structured` dimension, kept slug keys), README, skills-manifest.json
- Regenerated derived artifacts: assets/skills (via prepare-assets equivalent
copy) and the two affected .opencode yaml (via generate-opencode-skills)
- Hand-maintained trees updated consistently: .kiro skills, docs (catalog,
glossary, intro), plugin/marketplace manifests, qe-learning-coordinator
Stability:
- Kept skill id/dir `holistic-testing-pact` (and `.kiro` `qe-holistic-testing-pact`)
unchanged — referenced across other repos. Only titles/descriptions/content
changed to PACTS.
Attribution: PACT originated with Reuven Cohen (Agentics Foundation); adapted
to QE by Dragan Spiridonov. PACTS adds Structured.
Co-Authored-By: claude-flow & Agentic QE fleet
Follow-up to d95f0d7b / 2f4d119c — the earlier passes updated .claude/skills
and assets/skills but missed several other eval surfaces that still pinned
retired IDs (claude-3.5-sonnet, claude-3-haiku):
- `.github/workflows/test-qe-browser.yml` — the `aqe eval run --model` flag
(inert in command-eval mode, but a real dead-ID leftover) -> claude-sonnet-4-6.
- `plugins/agentic-qe-fleet/skills/` — the tracked plugin-marketplace skill copy
(8 eval suites + a sample-output fixture); chaos skills get the opus-4-8 ceiling.
- `docs/schemas/skill-eval.schema.json` — models `enum` was missing claude-opus-4-8
and still allowed retired/deprecated IDs; trimmed to current+active, default
flipped to claude-sonnet-4-6. Plus description examples in the sibling schemas
and the modelUsed values in docs/examples/*.json fixtures.
- `docs/templates/skill-frontmatter.example.yaml` validation_models list.
- `src/cli/commands/eval.ts` — the `aqe eval` `--model` help example and the
`run-all --models` default (`claude-3.5-sonnet` -> `claude-sonnet-4-6`).
Verified: typecheck clean; all edited JSON/YAML parse; zero retired IDs remain in
any eval suite, schema, template, workflow, or eval-CLI default. Intentionally
left untouched: the consensus-provider subsystem (pricing tables / type-unions /
defaults — separate routing layer, bucket-2 data) and historical release notes /
QE audit reports (which document the old IDs by design).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Ports ruflo's 'smoke.sh as a plugin contract' discipline. AQE already ships
the marketplace.json + a lean auto-discovery plugin (plugins/agentic-qe-fleet:
11 agents, 9 commands, 9 skills, lean plugin.json), but had no machine-checkable
contract — so silent drift could ship a broken bundle.
plugins/agentic-qe-fleet/scripts/smoke.sh (npm run plugin:smoke) asserts:
- plugin.json valid JSON, name == agentic-qe-fleet, has a version;
- plugin.json does NOT declare skills/commands/agents arrays (Claude Code
auto-discovers them — declaring them is an error, per ruflo);
- marketplace.json indexes the plugin at ./plugins/agentic-qe-fleet;
- exact counts (11 agents / 9 commands / 9 skills), each with name+description
frontmatter / SKILL.md (bump the constants deliberately on change);
- README present; top-level plugin.json registers the agentic-qe MCP server.
Exit 0 = contract holds, 1 = any violation.
Verified: passes 10/10 checks (exit 0), and correctly FAILS (exit 1) when a
skill is removed ('skills: 8 (expected 9)').
Scope note: AQE ships ONE bundle today. Splitting the 93 QE skills into multiple
lean subset bundles (aqe-security, aqe-browser, ...) is a larger restructure of
shipped artifacts — deferred as a follow-up (needs a distribution decision); the
skills-manifest.json already has the category data to drive it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Dragan Spiridonov