The codemod benchmark now writes its run JSON (tagged summary.benchmark=
"codemod-tier1") into the cost-tracker plugin's runs dir — the exact path
cost-trend reads — instead of a stopgap repo-docs location. cost-trend is now
benchmark-aware: BENCH_NAME=codemod-tier1 shows the codemod series, while the
default trend keeps showing only legacy booster runs (untagged / "booster"),
so the two corpora never conflate.
Co-Authored-By: RuFlo <ruv@ruv.net>
Wires `npm agent-booster` (Tier 1, WASM, $0/edit, sub-millisecond) into
the cost-tracker plugin and proves the speedup against frontier LLMs
running on the deployed GCP endpoints — Gemini 2.0 Flash, Claude
Sonnet 4.6, and Claude Opus 4.7.
What lands:
- ADR-0002 (Accepted): 6 decisions covering booster routing, optimizer
bridge, model-outcome feedback loop, loop-worker consumption,
tier-aware reporting, and the version+keywords bump.
- 4 new skills: cost-booster-route (router-side classifier),
cost-booster-edit (direct apply via agent-booster.apply()),
cost-benchmark (corpus harness), cost-compact-context (token-optimizer
bridge with graceful fallback).
- Updates to 2 existing skills: cost-optimize closes the
hooks_model-outcome loop; cost-report now reads measured booster data
from runs/latest.json and aggregates by tier.
- New `cost benchmark [--llm] [--anthropic]` subcommand.
- 16-case verification corpus (12 Tier 1 + 4 adversarial), bench.mjs
harness with split metrics (winRate / escalationRate / overallCorrect)
and gcloud-secret-aware LLM/Anthropic baselines.
- Smoke contract grew 10 → 27 checks, ~85 ms wall-time.
Verified live (16 cases, 4 endpoints, 2026-05-05):
Endpoint Tier 1 Adversarial Avg latency Cost/edit
Booster (WASM) 12/12 escalates 4/4 0.50 ms $0
Gemini 2.0 Flash 12/12 3/4 762.13 ms $0.000027
Claude Sonnet 4.6 12/12 3/4 1158.06 ms $0.000982
Claude Opus 4.7 12/12 3/4 1517.94 ms $0.006049
Measured speedups (booster vs LLM): 1524× / 2316× / 3036× — exceed
upstream's 350× claim against every endpoint. All three LLMs
(including Opus 4.7) failed on the same adversarial case; booster
correctly refused all four with min confidence 0.000.
Proof: https://github.com/ruvnet/ruflo/issues/1743https://gist.github.com/ruvnet/15fae7a5495026f025fb3baf721c20ea
Co-Authored-By: RuFlo <ruv@ruv.net>
ruv