mirror of
https://github.com/rohitg00/agentmemory.git
synced 2026-09-14 20:16:33 +08:00
Rohit Ghumare
d1517465ca
Adds a reproducible, dependency-free load harness so we can answer "what's p99 at 100k memories under concurrency 100?" with a number instead of a shrug. The harness seeds N synthetic memories against a local agentmemory daemon (defaults to http://localhost:3111, optional autostart via AGENTMEMORY_BENCH_AUTOSTART=1), then drives a matrix of (N, concurrency, endpoint) cells with hand-rolled Promise.allSettled batches. Per-request latency is collected via performance.now() and summarized as nearest-rank p50 / p90 / p99 plus min / max / errors and wall-clock throughput. Results are written to benchmark/results/load-100k-<short-git-sha>.json with a schema_version field so future format changes don't silently break consumers. Defaults match issue #346: N in {1000, 10000, 100000} x C in {1, 10, 100} x three endpoints (POST /agentmemory/remember, POST /agentmemory/smart-search, GET /agentmemory/memories?latest=true). Each cell issues BENCH_OPS=200 requests by default — enough samples for stable p99 without dragging a 100k-seed run past tens of minutes. Content is generated by a small noun/verb/concept vocabulary fed by a mulberry32(BENCH_SEED) PRNG so re-running the harness against the same daemon build yields the same seed corpus. Reproducibility, not realism, is the point — latency variance comes from the daemon, not JSON payload jitter. Files: - benchmark/load-100k.ts: main harness - benchmark/lib/percentiles.ts: zero-dep pXX helper, nearest-rank - benchmark/README.md: how to run, what gets measured, where results land, and why p99 is the number you want for capacity planning - benchmark/results/load-100k-96c0ed0.json: example result from a small-N (N=1000, C=10) verification run against a fresh daemon - package.json: wires `npm run bench:load` - CHANGELOG.md: Unreleased entry + a Performance section placeholder describing where per-release numbers should land going forward Verified locally at BENCH_N=1000 BENCH_C=10 BENCH_OPS=200 — three cells, zero errors, JSON written. Full 100k matrix is intentionally deferred to CI/release time. Closes #346.