rUv cd8a9c0988 feat: memory distillation self-learning loop (ADR-174) + page-agent browser intent (ADR-175) (#2570) ...

* feat(memory): distillation service — mine memory_entries into intelligence substrate (ADR-174 M0+M1)

Root cause the goal-planner surfaced: the daemon's consolidate worker
(worker-daemon.ts:1443) is a STUB writing hardcoded zeros — 6000+ commits
recorded to memory_entries but never distilled. reasoning_patterns /
episodes / pattern_embeddings / causal_edges were all empty (hence 🧠 0%).

M0 (safety) + M1 (service): runDistillation() mines memory_entries into
episodes → reasoning_patterns (+pattern_embeddings) → causal_edges.
- $0: reuses structured-distill.ts extractor + the embeddings already on
  each row; no LLM, no re-embedding.
- Incremental: distill_state rowid cursor per namespace; never rescans.
- Non-destructive: only inserts into the empty target tables; never
  touches memory_entries. Per-batch transactions.
- Safe on the just-recovered DB: quick_check gate skips corrupt DBs;
  better-sqlite3 optional (silent no-op).
- Provenance (ADR-171): feedback outcomes = oracle:test-exec (promotable);
  everything else = proxy:structural (written, NEVER promoted).

Measured on a copy of the real 7,900-entry DB: 7,899 → 4,260 patterns
(182 promoted, 4,078 proxy), 4,259 embeddings, 4,258 edges; source
unchanged; dry-run wrote nothing; 2nd run processed 0. 6 tests green.

ADR-174 records the decision, root cause, param surface, self-opt plan,
alternatives, rollback. Follow-ups on branch: M2 CLI, M3 daemon (replace
the stub), M4 self-optimization, M5 platform default.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(memory): distill CLI, daemon auto-distill, self-optimization (ADR-174 M2+M3+M4)

M2 — CLI: 'memory distill run|status|config' consuming the frozen M1
runDistillation. Full param surface: --dry-run, --namespace, --batch-size,
--dedup-distance, --max-entries, --since, --budget-usd, --judge (fable
gated on budget per ADR-172), --aggressive/--conservative presets,
--config, --db. status reports substrate counts + cursor + provenance
breakdown; degrades gracefully. 12 tests.

M3 — daemon: replaced the runConsolidateWorker STUB (the root cause — it
wrote hardcoded zeros) with a real runDistillation call every 30-min tick,
bounded to 1000 entries/tick (cursor drains a backlog across ticks, stays
well under the 16-min worker timeout). Real DistillReport written to the
metrics file. Opt-out via --no-distill / RUFLO_DAEMON_NO_DISTILL and the
existing 'daemon enable -w consolidate --disable'. Never throws (corrupt/
skipped surfaced, not fatal). 7 tests. The loop is now self-sustaining.

M4 — self-optimization ('ruflo tuning ruflo'): tuneDistillation grid-
searches {batchSize, dedupDistance, promoteThreshold} on ISOLATED copies,
scored by MRR@10/recall@10 on a rowid-based held-out split (train/held-out
disjointness asserted; held-out scored once). Source checksum unchanged.
Honest measured result on the real 7.9k-entry corpus: winner batchSize=200
dedupDistance=0.2 promoteThreshold=execution+corroborated, held-out MRR@10
0.753 vs 0.749 baseline — roughly on par (the substrate/training value is
the payoff, not raw retrieval uplift; measured-not-marketing). 11 tests.

Combined M1+M2+M3+M4: 30 tests green.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(browser): page-agent natural-language intent layer — browser_act (ADR-175)

Adds an in-page NL intent tool on top of the selector-based browser tools:
browser_act injects page-agent (Alibaba, MIT) into the agent-browser-driven
page and runs 'execute(task)' — one natural-language call instead of manual
selector orchestration.

Verified against the real page-agent@1.11.0 (ESM-only, split @page-agent/*
packages) — three load-bearing deviations handled:
- import('page-agent') throws 'window is not defined' in Node even when
  installed → availability probed via require.resolve (path only, no eval),
  like browser-session-tools.ts.
- the injectable IIFE auto-constructs a second PageAgent pointed at Alibaba's
  PUBLIC test endpoint (data leak on every call) → stripped before injection
  (stripDemoAutoInit).
- execute() returns { success, data, history } → mapped to { result, steps }.

Safety + routing:
- Optional dependency (~1.11.0) + v3 pnpm-lock regenerated same change;
  browser_act returns { degraded:true } (never throws) when absent.
- LLM routed through this CLI's Tier-3 ladder (OPENROUTER/OLLAMA, OpenAI-
  compatible); bare ANTHROPIC_API_KEY degrades (wrong API shape).
- KEY SAFETY: page-agent runs in page context, so the real key never enters
  the injected string — startLocalLLMProxy holds it server-side on a loopback
  http server and injects Authorization itself; page sees 127.0.0.1 + a
  placeholder. Tests assert no key substring in the injection.
- Trajectory → memory namespace 'browser' (feeds ADR-174 distillation).
- AIDefence gates returned content (optional; passes through if absent).

ADR-175 records the decision, the outer-harness/in-page split, constraints,
the 6 follow-on use cases, alternatives, and rollback. 16 tool tests +
145 mcp-client regression tests green; full build clean.

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix(browser): fail-closed demo-endpoint firewall for page-agent injection

The demo auto-connect (page content → Alibaba's public sandbox at
page-ag-testing-*.fcapp.run) was only neutralized by a best-effort text
strip that fell OPEN — stripDemoAutoInit returned the source unchanged if
the DEMO_MODEL marker moved, silently re-enabling the leak on an upstream
bundle change.

Verified via the real tarball: the demo endpoint lives ONLY in
dist/iife/page-agent.demo.js (6 refs); the library core ESM entry is clean
(0 refs). So no fork is needed — the leak is a property of the demo bundle,
not the library.

Fix: findDemoLeak() firewall + DemoLeakError. buildPageAgentInjection now
scans the stripped bundle for known demo-endpoint signatures
(page-ag-testing, .fcapp.run, DEMO_MODEL) and THROWS if any survive; the
browser_act handler catches it and degrades rather than injecting. This is
version-independent and fail-CLOSED — a moved/renamed marker can no longer
leak, because the guarantee is 'endpoint absent', not 'marker found'.

+4 tests (signature detection, clean-bundle pass, moved-marker refusal,
clean-bundle injection). 20/20 green. ADR-175 updated with the fork-vs-fix
rationale.

Co-Authored-By: RuFlo <ruv@ruv.net>

* fix(browser): add ADR-112 discoverability guidance to browser_act description

CI failed 4 checks (tool-description audit + 3 smoke contracts that run it):
browser_act was the 1 tool of 361 without a 'Use when …' guidance signal.
The audit regex captures only the FIRST quoted segment of a multi-line
description, so the guidance phrase must lead. Reworded to open with
'Use when …' + 'Prefer … over …' + 'pair with …' + 'Falls back …'.
Audit now 361/361, exit 0.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(memory): M5 — promote M4 winning distill config as daemon default (ADR-174)

The daemon's consolidate worker now runs the M4-tuned config
(dedupDistance=0.2, chosen by the self-optimization grid-search on the real
corpus). Honest note in code + ADR: held-out MRR@10 0.753 vs 0.749 baseline
— on-par, the substrate/training value is the payoff. Overridable per-run
via 'memory distill'. Completes ADR-174 M0-M5.

Co-Authored-By: RuFlo <ruv@ruv.net>

* docs+fix(memory): address ADR-174 review — honest default, edge contract, invariants (ADR-174)

Review fixes:
- STATUS: Proposed → Accepted (M0-M5 implemented+tested, pending #2570 merge).
- TUNED DEFAULT honestly justified: the M4 retrieval delta (0.753 vs 0.749)
  is a TIE, so dedupDistance=0.2 is sold as COMPRESSION, not retrieval lift —
  measured 2,723 vs 4,350 patterns (37% fewer) at 11.6ms/batch, 0 violations.
  Added an explicit promotion rule (retrieval-neutral within 0.002 AND >=15%
  compression AND <250ms/batch AND 0 proxy violations).
- PARAM TABLE / DEFAULT consistency: dedup default is 0.2 everywhere (service,
  CLI, daemon). Presets now bracket it: --aggressive 0.3 / --conservative 0.1.
- CAUSAL_EDGES OVERCLAIM: edges now carry an explicit contract
  (edge_type=cooccurrence, provenance_tier=proxy:structural, confidence=0.3,
  promoted=false) with the rule 'may rank retrieval, must NOT justify
  autonomous action'. Also fixed a latent bug: edges linked epId-1 (assumed
  consecutive rowids) → now the actual previous episode id.
- EMBEDDING 1:1 invariant enforced: a cluster with no parseable vector is
  skipped, so every pattern has exactly one embedding (was off-by-one on a
  malformed source vector).
- ADR adds operational invariants + audit SQL, business-value framing, and an
  acceptance test.

+2 invariant tests (embedding coverage, edge contract). 32 distill tests green.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(hooks): capture edit/task FAILURES, not just successes (ADR-174)

The DB analysis found 898 feedback records — 100% success, 0 failures —
because the post-edit/post-task hooks recorded a hardcoded success:true and
never inspected the tool outcome. With no negative examples the oracle tier
can't teach good-vs-bad, which caps what distillation can learn.

Fix: read Claude Code's PostToolUse tool_response and record the REAL outcome.
- isToolFailure(hookInput): conservative detector (positive error signal only)
  — tool_response.is_error / error / success:false, non-zero Bash exit code,
  or an error string. Exported + unit-tested (5 cases).
- Generated hook (helpers-generator.ts) + the shipped .claude/helpers/
  hook-handler.cjs both inline it: post-edit → recordEdit(file, !toolFailed),
  post-task → feedback(!toolFailed) (feedback() already boosts on success /
  decays on failure — it was just never called with false).
- intelligence.cjs recordEdit(file, success) persists success to
  pending-insights so consolidation/distillation gets the negative example.

The distillation JUDGE (memory-distillation.ts judgeFeedback) already maps
success:false → reward 0 / not-promoted, so the substrate is failure-ready;
this closes the upstream gap that never produced failures.

End-to-end: a failed-edit PostToolUse payload now logs '[LEARN] Edit FAILURE
recorded' through the actual shipped hook; a success stays '[OK] Edit recorded'.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(init): version-stamped auto-refresh of critical helpers — hook fixes reach all users, no re-init (ADR-174)

The propagation gap: Claude Code hooks run the PROJECT-LOCAL
.claude/helpers/*.cjs copies, not the npm package — so npx ruflo@latest
never updated them, and users didn't know to re-run init. Any hook fix
(e.g. the ADR-174 failure-capture change) reached only people who happened
to re-init.

Fix: stamp the critical helpers (hook-handler.cjs, intelligence.cjs,
auto-memory-hook.mjs) with the installed CLI version; on the next CLI
command, if a project's stamp is older than the installed version, silently
re-copy them. New lightweight src/init/helper-refresh.ts (fs/path/module
only at load; lazily imports the heavy generators only on the rare
source-unresolvable fallback). Wired AWAITED into index.ts startup so a fast
command can't exit before the copy lands — fast path is a stamp read +
string compare (sub-ms); the copy runs at most once per version bump. No-op
outside a ruflo project (never scaffolds files in an unrelated dir). Skipped
for init/update. executeUpgrade also stamps so an explicit upgrade is current.

Verified end-to-end: a stale-stamped project with an OLD (no-failure-capture)
hook silently upgrades to the toolFailed logic on a plain 'ruflo --help', and
a current project is a no-op (a marker file survives). +4 tests.

Co-Authored-By: RuFlo <ruv@ruv.net>

* feat(security): Ed25519 signed provenance gate for the helper auto-refresh (ADR-174)

The auto-refresh copies auto-EXECUTING hook code from the installed package
into projects. npm verifies the tarball at install, not the files on disk
after — a sibling package's postinstall (or disk tamper) could overwrite the
helpers and the refresh would faithfully propagate the malicious code.

Fix (fail-closed): every helper is verified against a ruflo-signed manifest
before install.
- helper-signing.ts: baked RUFLO_HELPERS_PUBKEY + verifyHelpersManifest()
  (native Ed25519, same primitive as rvfa-signing.ts). Fail-closed — any
  bad signature / tampered hash / wrong algorithm → null → refuse.
- scripts/sign-helpers.mjs: publish-time signer; private key from
  $RUFLO_HELPERS_SIGNING_KEY, NEVER committed (kept out of the repo tree).
  Not shipped (scripts/ is narrowed to postinstall.cjs in files).
- helper-refresh.writeCriticalHelpers: verifies the signed manifest + every
  source helper's SHA-256 BEFORE copying anything; a mismatch is refused and
  the project's existing helpers are left intact. The generator fallback
  needs no manifest (it's the CLI's own compiled code = trust root).
- index.ts warns loudly on a blocked refresh (potential tampering signal).

Threat model: closes post-install/on-disk tampering of the helper files; does
not defend a wholesale-compromised CLI (which could swap the baked key too —
but then it owns the binary you run). A CI test ('hashes match shipped files')
fails if a helper changes without re-signing, so a stale manifest can't ship.

Verified end-to-end: a tampered shipped helper → refresh REFUSED (blocked,
project helper untouched); restored → refresh succeeds. +6 signing tests.

Co-Authored-By: RuFlo <ruv@ruv.net>

* chore(security): source helper-signing key from GCP Secret Manager

sign-helpers.mjs now resolves the private key from GCP Secret Manager
(RUFLO_HELPERS_SIGNING_SECRET, via gcloud secrets versions access) as the
preferred source for CI/publish, with the local-PEM fallback
(RUFLO_HELPERS_SIGNING_KEY) for air-gapped signing. Key never committed.
ADR-174 security note updated.

Co-Authored-By: RuFlo <ruv@ruv.net>

2026-07-04 17:10:45 -04:00

4.5 KiB

Raw Permalink Blame History

View Raw