mirror of
https://github.com/rohitg00/agentmemory.git
synced 2026-09-14 20:16:33 +08:00
v0.8.7
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Rohit Ghumare
|
ae8f061cd6 |
feat: demo command, benchmark comparison, OpenClaw integration (#113)
* feat: add demo command, benchmark comparison, OpenClaw integration, token cost in viewer
Product & ecosystem improvements surfaced from the last30days research
and competitive analysis of the AI agent memory space.
## New
### agentmemory demo command
Seeds 3 realistic sessions (JWT auth, N+1 query fix, rate limiting)
and runs smart-search queries against them. Shows token-aware semantic
search finding "N+1 query fix" when you search "database performance
optimization" — proof that keyword matching can't do that.
npx @agentmemory/agentmemory demo
Surfaces the "show don't tell" moment in 30 seconds without requiring
the user to integrate anything.
### benchmark/COMPARISON.md
Head-to-head comparison vs mem0 (53K stars, 68.5% LoCoMo),
Letta/MemGPT (22K stars, 83.2% LoCoMo), Khoj (34K stars),
claude-mem (46K stars), and Hippo. Feature matrix across 18
dimensions. Token efficiency table. Honest caveats about LongMemEval
vs LoCoMo apples-vs-oranges. Lets users compare without trusting our
README.
### integrations/openclaw/
OpenClaw gateway plugin following the same pattern as the Hermes
integration. README, plugin.yaml with hooks config, and plugin.mjs
implementing the 4 lifecycle hooks (on_session_start, on_pre_llm_call,
on_post_tool_use, on_session_end). Gets agentmemory into the OpenClaw
ecosystem for distribution.
## Improved
### Viewer token savings card
Now shows tokens saved AND dollar cost saved (using $0.30/1K as a
reasonable mid-tier model rate) instead of just raw numbers. Makes
the ROI immediately tangible.
### README additions
- "Try it in 30 seconds" section featuring the new demo command
- Competitor comparison link in benchmarks section
- OpenClaw integration link in supported agents table
* refactor: simplify demo command and OpenClaw plugin
- Extract postJson helper in cli.ts runDemo (4x repeated fetch boilerplate → single call site)
- Break runDemo into buildDemoSessions, seedDemoSession, runDemoSearch for single-responsibility
- Replace three-branch if/else command dispatch with a commands map (single error handler)
- Extract typed DemoSession/DemoObservation/SearchResult interfaces
- Simplify OpenClaw plugin.mjs: collapse headers() + fetchJson() into a single postJson method
Each hook is now ~5 lines instead of ~10
No behavior change. 654/654 tests passing, CLI help and error paths verified.
* fix: address CodeRabbit review on growth changes
## plugin.mjs
- Honor the enabled config flag: constructor now sets this.enabled and
all four hooks (onSessionStart, onPreLlmCall, onPostToolUse,
onSessionEnd) return early when disabled
- Fail-fast on non-OK HTTP: previously !res.ok returned null regardless
of fallbackOnError. Now throws with status/statusText/body when
fallbackOnError is false
- Update header JSDoc to use the real hook names instead of the
fictional prefetch/capture/consolidate
## openclaw README.md
- Replace stale hook names (prefetch/capture/consolidate) with the
actual onSessionStart/onPreLlmCall/onPostToolUse/onSessionEnd names
matching plugin.mjs
- Add 'text' language identifier to the Project profile code fence to
satisfy markdownlint MD040
## cli.ts
- Replace ad-hoc Date.now()-based session IDs with generateId("demo")
from src/state/schema.ts, matching the rest of the codebase
(snapshot.ts, migrate.ts, consolidation-pipeline.ts)
- Add postJsonStrict helper that throws on non-OK with status + body
- seedDemoSession now uses postJsonStrict for session/start and
session/end (critical lifecycle calls that should fail fast)
- The observation POST in the loop now inlines the fetch to surface
HTTP errors via p.log.warn with status/body, still only incrementing
the stored counter on success
## viewer/index.html
- Fix token-to-dollar conversion that was 100x underreporting:
tokensSaved / 1000 * 0.3 returns DOLLARS, not cents. Now computes
dollars first, then cents = Math.round(dollars * 100)
- 100K tokens saved now correctly displays $30.00, not 30ct
All 654 tests still passing.
* fix: address 2 more CodeRabbit findings on plugin.mjs
- Use nullish coalescing (??) for config defaults so valid falsy
values like min_confidence: 0 or timeout_ms: 0 aren't overwritten
by defaults. || would clobber them.
- Fix /agentmemory/context payload to match the server contract at
src/triggers/api.ts:115. The handler expects { sessionId, project,
budget? } but the plugin was sending { sessionId, query, tokenBudget,
minConfidence }. Three issues:
1. Missing required 'project' field
2. Wrong field name: 'tokenBudget' should be 'budget'
3. 'query' and 'minConfidence' are not part of this endpoint's
contract (probably belong on /smart-search instead)
|
||
|
Rohit Ghumare
|
57b0eeebfa |
docs: update README with LongMemEval results and fix inconsistencies
- Lead with 95.2% R@5 on LongMemEval-S (ICLR 2025) instead of old 64% internal number - Add LongMemEval benchmark table to README benchmarks section - Fix Quick Start: reorder to start→connect→verify, add MCP configs for Cursor/Windsurf/Claude Desktop/REST API (not just Claude Code plugin) - Fix version in health example (0.7.4 → 0.7.7) - Fix test count (627 → 646) - Fix source file count (113 → 118) - Fix LOC count (~20,000 → ~21,800) - Fix function count (89 → 123 mem:: functions) - Fix MCP tools table header (38 → 43) - Fix MCP tools reference in chooser table (41 → 43) - Cross-reference LongMemEval from QUALITY.md |
||
|
Rohit Ghumare
|
565c2388a2 |
bench: add LongMemEval-S benchmark (95.2% R@5 hybrid, 86.2% BM25) (#91)
Run agentmemory on LongMemEval-S (ICLR 2025, 500 questions) using the same dataset and metric (recall_any@5) as MemPalace for direct comparison. Results: - BM25+Vector hybrid: 95.2% R@5, 98.6% R@10, 87.9% NDCG@10 - BM25-only: 86.2% R@5, 94.6% R@10, 73.0% NDCG@10 - MemPalace raw vector: 96.6% R@5 (1.4pp gap, same embedding model) Clearly labels these as retrieval recall, not end-to-end QA accuracy. |
||
|
Rohit Ghumare
|
857f71e3c6 |
feat: v0.6.0 advanced retrieval with real-world benchmarks (#76)
* feat: add 9 orchestration modules for v0.5.0 Add actions, frontier, leases, routines, signals, checkpoints, flow-compress, mesh, and branch-aware modules with full MCP tools, REST endpoints, and 170 new tests. Includes SSRF protection, race-condition-safe keyed mutex locking, SHA-256 fingerprinting, and fixes for 29 CodeRabbit review findings. - 9 new source files (src/functions/*) - 8 new test files (170 tests, total 386) - 10 new MCP tools, 23 new REST endpoints - 8 new KV scopes, new types for orchestration - Version bump to 0.5.0, README and viewer updated * feat: add sentinels, sketches, crystallize, diagnostics, facets modules 5 new modules inspired by beads patterns but with original naming and iii-engine real-time streaming (no polling): - sentinels: event-driven condition watchers (webhook, timer, threshold, pattern, approval) that auto-unblock gated actions via SSE - sketches: ephemeral action graphs with auto-expiry, promote or discard - crystallize: LLM-powered compaction of completed action chains into compact crystal digests with key outcomes and lessons - diagnostics: self-diagnosis across 8 categories (actions, leases, sentinels, sketches, signals, sessions, memories, mesh) with auto-heal - facets: multi-dimensional tagging (dimension:value) with AND/OR queries - 5 new source files, 5 new test files (132 tests, total 518) - 9 new MCP tools (total 37), 21 new REST endpoints (total 93) - 4 new KV scopes (total 33), new types for all modules - README stats and function table updated * fix: address code review findings across v0.5.0 modules - actions: validate edges before persisting, set blocked status for requires deps - leases: use mem:action lock key, reject blocked actions, check expiry on release - checkpoints: validate linkedActionIds exist, check requires edges in unblock - mesh: add SSRF validation on peer registration - routines: remove invalid "failed" action status check - export-import: add v0.5.0 scope export/import (actions, sentinels, sketches, etc) - mcp/server: validate CSV inputs are strings before splitting - schema: replace runtime require with static import - README: fix stale tool/endpoint counts (28→37, 72→93) * fix: second round code review — mesh locks, routines DAG, MCP input validation, README counts - mesh.ts: add withKeyedLock on action writes in receive path, add IPv6 private ranges to SSRF check - routines.ts: validate DAG (duplicate orders, unknown deps), set dep actions to blocked, refresh stepStatus in routine-status - checkpoints.ts: runtime type enum validation, set linked pending actions to blocked - leases.ts: validate ttlMs is finite positive number - export-import.ts: add skip strategy checks for all v0.5.0 import blocks - mcp/server.ts: typeof guards on tags, config JSON.parse, actionIds, linkedActionIds, categories - README.md: Tools 18→37, Functions 33→50, stats line updated - test: update checkpoint test for new blocked-on-create behavior * fix: third round review — lease renew/release safety, mesh locking, export replace cleanup, MCP input guards - leases.ts: renew extends from max(now, existing expiry) instead of now; release verifies action ownership before mutation - mesh.ts: withKeyedLock on memory writes in mesh-receive; applySyncData validates id/updatedAt and locks both memory and action writes - routines.ts: stepStatus maps "blocked" to "pending" explicitly; progress includes blocked/cancelled counts; routine-freeze wrapped in withKeyedLock - export-import.ts: remove unused RoutineRun import; replace strategy clears all orchestration namespaces; skip strategy for graphNodes/graphEdges/semantic/procedural - mcp/server.ts: sentinel_trigger JSON.parse with typeof+try/catch; facet_query typeof guards on matchAll/matchAny; remove redundant requires cast; .filter(Boolean) on concepts/files/tags/requires CSV splits - README.md: clarify API table is a representative subset * fix: fourth round review — redirect SSRF, blocked-on-create, missing action handling, boolean normalization - mesh.ts: add redirect:"error" to both outbound fetch calls to prevent SSRF via redirect - routines.ts: create actions with status "blocked" directly when hasDeps (eliminates two-pass race); handle missing actions in routine-status as cancelled; progress.total uses run.actionIds.length - mcp/server.ts: sentinel config accepts object values directly; normalize unreadOnly/dryRun for both JSON booleans and string values - README.md: consistent bundle size (365KB) across both occurrences * feat: v0.6.0 advanced retrieval — triple-stream search, stemming, real benchmarks Search improvements: - Porter stemmer for word normalization (authentication ↔ authenticating) - 40+ coding-domain synonym groups (db ↔ database, k8s ↔ kubernetes) - Binary-search prefix matching replaces O(n) full scan - Session diversification (max 3 results per session) - Co-occurrence graph edges between all concept pairs New retrieval modules: - Sliding window inference pipeline (context enrichment at ingestion) - Adaptive query expansion (LLM-generated reformulations) - Triple-stream search (BM25 + Vector + Graph with RRF fusion) - Append-only temporal knowledge graph (versioned edges, point-in-time queries) - Graph-augmented retrieval (entity search + neighborhood expansion) - Ebbinghaus retention scoring (decay + tiered hot/warm/cold/evictable) Real-world benchmarks (240 observations, 20 labeled queries): - Quality eval: 64.1% recall@10 with Xenova embeddings (vs 55.8% grep) - Scale eval: 92-100% token savings vs built-in memory at 240-50K observations - Cross-session: 12/12 queries found vs 10/12 for 200-line MEMORY.md cap - Token measurement uses actual search results (fixed fake constant bug) - Removed old microbenchmarks (bench.ts, run-bench.ts, COMPARISON.md) |