Commit Graph

1 Commits

Author SHA1 Message Date
Colby McHenry 3bce3718cf docs(benchmarks): A/B with/without codegraph across every language (S/M/L)
37-cell matrix (every flow-relevant language × small/medium/large indexed
repos): a headless agent answers one canonical flow question per repo, with the
codegraph MCP vs without any MCP. Fresh re-index per cell so the with-arm
reflects current resolvers.

Result: 75% fewer file reads with codegraph (40 vs 158 across cells), ~70%
fewer greps, never more reads in any cell. Biggest wins on medium/large
backends (excalidraw 0R vs 9R, spring-halo 0R vs 9R+8 Bash, jellyfin 4R vs 13R+
21 Bash + a spawned sub-agent); tie zone on tiny repos where the flow fits in
1-2 files.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-23 14:59:55 -05:00