Commit Graph

3 Commits

Author SHA1 Message Date
Claude 671831dacd feat: add A/B benchmark harness with 20 tasks, gate simulation, and composite scoring
Implements the Measurement Plan: Config A (no control plane) vs Config B
(Phase 1 guidance) across 20 representative tasks from Claude Flow repo
history spanning 7 task classes (bug-fix, feature, refactor, security,
deployment, test, performance).

KPIs: success rate, tool calls, token spend, violations, human
interventions. Composite score formula:
score = success_rate - 0.1*cost - 0.2*violations - 0.1*interventions

Category shift detection: B beats A by ≥0.2 on composite across ≥3 task
classes. Gate simulation detects destructive commands, hardcoded secrets,
force push, unsafe types, skipped hooks, missing tests, policy violations.

1,328 tests passing across 26 files (38 new AB benchmark tests).

https://claude.ai/code/session_01HLi9WmfvFgkDLJxLKGAUTf
2026-02-01 22:22:17 +00:00
Claude 2810c8bdc4 docs: update README and API reference with generators, analyzer, and validation suite
Add documentation for all features built in recent sessions:
- Generators module (generateClaudeMd, scaffold, skills, agents)
- Analyzer module (6-dimension scoring, autoOptimize, optimizeForSize)
- Headless benchmarking (claude -p integration with proof chain)
- Empirical validation (validateEffect with Pearson r, Spearman rho,
  Cohen's d, content-aware executors, multi-trial support)
- Update test count: 1,290 tests across 26 files
- Update module count: 31 modules across 9 layers
- Add generators and analyzer to API quick reference import map
- Add full API reference tables for both modules

https://claude.ai/code/session_01HLi9WmfvFgkDLJxLKGAUTf
2026-02-01 21:54:54 +00:00
Claude 176f6ad235 docs: add comprehensive user documentation with guides, tutorials, and diagrams
13 documentation files covering all 20+ modules:

Guides:
- Getting Started: installation, setup, core loop
- Architecture Overview: 7-layer design with mermaid diagrams
- Multi-Agent Security: threats, collusion, quorum, trust
- Capabilities and Trust: permission algebra, trust tiers, authority
- Knowledge Management: uncertainty, temporal, truth anchors
- WASM Kernel: building, using, benchmarking

Tutorials:
- Enforcement Gates: wire gates into hooks
- ContinueGate: prevent runaway agent loops
- Proof & Audit Trail: tamper-evident chains
- CI/CD & Headless Testing: automated compliance

Reference:
- API Quick Reference: all exports, methods, types

Diagrams:
- 10 mermaid diagrams: system overview, enforcement pipeline,
  decision trees, trust flow, capability delegation, memory
  security layers, knowledge stack, proof chain, meta-governance
  lifecycle, compile-retrieve-gate sequence

https://claude.ai/code/session_01HLi9WmfvFgkDLJxLKGAUTf
2026-02-01 19:55:55 +00:00