2 Commits

Author	SHA1	Message	Date
Patrick szymkowiak	d13c185aac	fix(benchmark): address review feedback from @FlorianBruniaux ... - Strict exit codes: cargo/python/go tests expect exact exit codes instead of "any" (catches real regressions) - Binary size limit documented: 8MB for ARM Linux VM vs 5MB x86 stripped - --phase NaN guard: error message instead of silent no-op - Verdict: 0 failures = READY, any failure = NOT READY (no more "minor issues" budget) - rtk err exit code bug tracked in #846 Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>	2026-04-12 10:36:37 +02:00
Patrick szymkowiak	d22759b8c5	feat(benchmark): add multipass VM integration test suite ... Bun/TypeScript orchestrator that creates an Ubuntu 24.04 VM via multipass, installs all dev tools (Rust, Go, Node, Python, .NET, Terraform, etc.), builds RTK, and runs 103 tests across 11 phases: - Cargo quality (fmt, clippy, test) - 47 Rust built-in commands (git, ls, grep, cargo, pytest, go, tsc...) - 21 TOML filter commands (df, ps, shellcheck, hadolint, helm...) - Hook rewrite engine (17 rewrite assertions) - Exit code preservation - Token savings verification (avg 81%) - Pipe compatibility - Edge cases (unicode, ANSI, empty output) - Performance (memory < 20MB) - Concurrency (10 parallel executions) Usage: bun run scripts/benchmark/run.ts # Full suite (~3 min) bun run scripts/benchmark/run.ts --quick # Skip perf/concurrency bun run scripts/benchmark/run.ts --phase 3 # Single phase bun run scripts/benchmark/cleanup.ts # Delete VM bun run scripts/benchmark/rebuild.ts # Fast rebuild Prerequisites: brew install multipass, bun Signed-off-by: Patrick szymkowiak <patrick.szymkowiak@innovtech.eu>	2026-04-12 10:36:37 +02:00