Dragan Spiridonov 9cb44d3252 feat(learning): self-learning verification harness (#510 item 1) ...

Ports ruflo's benchmark-self-learning.mjs discipline (the 'reports success
but persists nothing' detector) to AQE. Proves the learning stores actually
persist what they claim, that stat aggregators agree with the underlying
index, that nothing writes accidentally, and that data survives restart.

Sections (each isolated in a temp dir, each gating):
  A store persistence  — N creates => totalPatterns delta == N
  B outcome learning   — N outcomes => learningOutcomes delta == N, successRate matches fed ratio
  C negative control   — read-only ops => 0 delta (no accidental writes)
  D restart survival   — dispose + reopen on same temp path => counts persist
  E consistency        — RvfPatternStore.getStats == adapter.totalVectors == bank.totalPatterns
  F audit-real         — copies real stores read-only, checks banner numbers round-trip

Data protection (CLAUDE.md): synthetic mode uses fresh temp dirs; audit-real
copies stores and opens only copies. Verified real .rvf stores byte-identical
(mtime+size+sha256) before/after a run.

Audit-real finding on the live store: banner numbers corroborated
(72.5% success, 39.7% confidence, 2131 routings) BUT RVF vector count (338)
!= SQLite qe_patterns rows (345), a ~7-pattern index/table drift the banner
hides — exactly the divergence this harness exists to surface (reported as a
diagnostic note, non-gating). Synthetic sections A-E pass; exit 1 on failure.

npm run benchmark:self-learning

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

2026-06-02 08:29:42 +00:00

28 KiB

Raw Permalink Blame History

View Raw