Run agentmemory on LongMemEval-S (ICLR 2025, 500 questions) using the
same dataset and metric (recall_any@5) as MemPalace for direct comparison.
Results:
- BM25+Vector hybrid: 95.2% R@5, 98.6% R@10, 87.9% NDCG@10
- BM25-only: 86.2% R@5, 94.6% R@10, 73.0% NDCG@10
- MemPalace raw vector: 96.6% R@5 (1.4pp gap, same embedding model)
Clearly labels these as retrieval recall, not end-to-end QA accuracy.
Rohit Ghumare