The codemod benchmark now writes its run JSON (tagged summary.benchmark=
"codemod-tier1") into the cost-tracker plugin's runs dir — the exact path
cost-trend reads — instead of a stopgap repo-docs location. cost-trend is now
benchmark-aware: BENCH_NAME=codemod-tier1 shows the codemod series, while the
default trend keeps showing only legacy booster runs (untagged / "booster"),
so the two corpora never conflate.
Co-Authored-By: RuFlo <ruv@ruv.net>
The smoke gate is binary (winRate ≥ 0.80 → pass). A win rate slowly
creeping from 100% to 85% would still "pass" smoke but represents real
degradation. cost-trend turns the gate into a curve.
What lands:
- scripts/trend.mjs — reads every docs/benchmarks/runs/*.json (excluding
latest.json), reports first→last deltas on winRate, avg/p99 latency,
escalationRate, speedupVsLlm; per-run series with Sonnet 4.6 + Opus 4.7
latency columns; regression flags (`⚠ Regression`) when win rate drops
or avg latency rises ≥ 1.5×
- skills/cost-trend/SKILL.md
- `cost trend` subcommand
- 2 new smoke checks (now 39/39): trend.mjs has metrics + regression
flag, ruflo-cost.md documents the subcommand
Live verified on the 6 persisted runs from this session:
- Win rate stable at 100% across all 6
- Avg latency drifted 0.58ms → 0.50ms (slight improvement)
- Sonnet 4.6 latency varied 1072→1158ms (real LLM noise visible)
- Escalation rate appeared in last 2 runs (corpus v2 added adversarial
cases — backward-compat preserved)
Optional env: TREND_FORMAT=json, TREND_LIMIT=N.
Version bump: 0.8.0 → 0.9.0. Keywords: + trend-analysis.
Co-Authored-By: RuFlo <ruv@ruv.net>
ruv