Reuven 0fd61e3e5b feat(cost-tracker): ADR-0002 — verified Agent Booster integration (v0.4.0) ...

Wires `npm agent-booster` (Tier 1, WASM, $0/edit, sub-millisecond) into
the cost-tracker plugin and proves the speedup against frontier LLMs
running on the deployed GCP endpoints — Gemini 2.0 Flash, Claude
Sonnet 4.6, and Claude Opus 4.7.

What lands:

- ADR-0002 (Accepted): 6 decisions covering booster routing, optimizer
  bridge, model-outcome feedback loop, loop-worker consumption,
  tier-aware reporting, and the version+keywords bump.
- 4 new skills: cost-booster-route (router-side classifier),
  cost-booster-edit (direct apply via agent-booster.apply()),
  cost-benchmark (corpus harness), cost-compact-context (token-optimizer
  bridge with graceful fallback).
- Updates to 2 existing skills: cost-optimize closes the
  hooks_model-outcome loop; cost-report now reads measured booster data
  from runs/latest.json and aggregates by tier.
- New `cost benchmark [--llm] [--anthropic]` subcommand.
- 16-case verification corpus (12 Tier 1 + 4 adversarial), bench.mjs
  harness with split metrics (winRate / escalationRate / overallCorrect)
  and gcloud-secret-aware LLM/Anthropic baselines.
- Smoke contract grew 10 → 27 checks, ~85 ms wall-time.

Verified live (16 cases, 4 endpoints, 2026-05-05):

  Endpoint            Tier 1   Adversarial   Avg latency   Cost/edit
  Booster (WASM)      12/12    escalates 4/4 0.50 ms       $0
  Gemini 2.0 Flash    12/12    3/4           762.13 ms     $0.000027
  Claude Sonnet 4.6   12/12    3/4           1158.06 ms    $0.000982
  Claude Opus 4.7     12/12    3/4           1517.94 ms    $0.006049

Measured speedups (booster vs LLM): 1524× / 2316× / 3036× — exceed
upstream's 350× claim against every endpoint. All three LLMs
(including Opus 4.7) failed on the same adversarial case; booster
correctly refused all four with min confidence 0.000.

Proof: https://github.com/ruvnet/ruflo/issues/1743
       https://gist.github.com/ruvnet/15fae7a5495026f025fb3baf721c20ea

Co-Authored-By: RuFlo <ruv@ruv.net>

2026-05-04 23:35:52 -04:00

19 KiB

Executable File

Raw Permalink Blame History

View Raw