Long-term conversational memory on LoCoMo
79.44Overall Acc (LoCoMo)RAG (Dense)
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| RAG (Dense)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 79.44 | — | — | — | — | — | 61.8 | 17.06 | 0.075 | |
| MEMGUARD (2 hop)Base LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=15402026.05 | 79.1 | 58.33 | 77.57 | 84.54 | 75.89 | 76.46 | — | — | — | |
| RAG (BM25)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 77.69 | — | — | — | — | — | 61 | 17.19 | 0.08 | |
| MEMGUARDBase LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=16052026.05 | 77.29 | 59.38 | 78.01 | 83.12 | 70.92 | 75.11 | — | — | — | |
| RAG (Dense)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 76.1 | — | — | — | — | — | 58.4 | 13 | 0.065 | |
| MemOSBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=15892026.05 | 75.8 | 67.49 | 75.18 | 55.9 | 81.09 | — | — | — | — | |
| MEMGUARD (2 hop)Base LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=12502026.05 | 75.78 | 51.04 | 62.93 | 76.58 | 73.76 | 90.13 | — | — | — | |
| MEMGUARDBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=12972026.05 | 75.53 | 48.96 | 64.8 | 75.98 | 73.4 | 89.46 | — | — | — | |
| PISATokens/Q=2.1k2025.10 | 74.8 | 70.49 | 79.29 | 68.48 | 79.52 | 76.24 | — | — | — | |
| RAG (BM25)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 74.58 | — | — | — | — | — | 57.1 | 7.72 | 0.029 | |
| FullContextTokens/Q=21.09k2025.10 | 74.02 | 74.08 | 91.6 | 64.35 | 86.08 | 54.01 | — | — | — | |
| HAGEBackbone=gpt-4o-mini2026.05 | 73.9 | 54.7 | 66.7 | 49.7 | 79.7 | 83.9 | — | — | — | |
| MemobaseBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=21022026.05 | 72.01 | 64.65 | 81.2 | 53.12 | 73.12 | — | — | — | — | |
| LangMemTokens/Q=0.76k2025.10 | 71.97 | 69.75 | 79.23 | 61.2 | 78.44 | 71.25 | — | — | — | |
| MAGMABackbone=gpt-4o-mini2026.05 | 70 | 52.8 | 65 | 51.7 | 77.6 | 74.2 | — | — | — | |
| Mem0-GraphBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=36162026.05 | 68.44 | 47.19 | 58.13 | 75.71 | 65.71 | — | — | — | — | |
| Mem0LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 68.22 | — | — | — | — | — | 50.8 | 6.29 | 0.024 | |
| RAG (BM25)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 66.86 | — | — | — | — | — | 54.2 | — | — | |
| Mem0LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=with GOAL-MEM2026.05 | 65.95 | — | — | — | — | — | 50.1 | 2.53 | 0.004 | |
| GraphRAGLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 65.82 | — | — | — | — | — | 51 | 10.31 | 0.054 | |
| RAPTORLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 65.3 | — | — | — | — | — | 51.9 | 14.2 | 0.078 | |
| A-MEMLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 65.05 | — | — | — | — | — | 52.3 | 23.28 | 0.145 | |
| Mem0Base LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=11722026.05 | 64.57 | 58.75 | 52.34 | 45.83 | 73.33 | — | — | — | — | |
| GraphRAGLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 64.5 | — | — | — | — | — | 50.9 | 6.59 | 0.033 | |
| MIRIXBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini2026.05 | 64.33 | 54.26 | 68.54 | 46.88 | 68.22 | — | — | — | — | |
| MemoryOSTokens/Q=7.44k2025.10 | 64.33 | 62.41 | 65.57 | 73.79 | 77.64 | 42.26 | — | — | — | |
| Mem0Tokens/Q=6.34k2025.10 | 64.21 | 62.46 | 73.78 | 60.87 | 78.04 | 45.91 | — | — | — | |
| Mem0LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 63.42 | — | — | — | — | — | 50.5 | — | — | |
| Mem0-GraphLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 63.36 | — | — | — | — | — | 46.8 | 3.83 | 0.018 | |
| RAPTORLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 63.23 | — | — | — | — | — | 52.1 | 7.33 | 0.042 | |
| RAG (Dense)LLM Backbone=GPT-5.4-mini, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 63.1 | — | — | — | — | — | 51.9 | — | — | |
| Mem0-GraphLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 62.91 | — | — | — | — | — | 46.3 | 0.97 | 0.006 | |
| RAG (Dense)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 62.39 | — | — | — | — | — | 54.3 | — | — | |
| Mem0LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 61.93 | — | — | — | — | — | 48.4 | — | — | |
| Mem0-GraphLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 61.93 | — | — | — | — | — | 46.9 | — | — | |
| AMemTokens/Q=3.11k2025.10 | 61.53 | 52.5 | 76.03 | 52.83 | 63.44 | 62.87 | — | — | — | |
| RAG (BM25)LLM Backbone=Gemma-4-26B, Memory Category=No Structure, Augmentation Strategy=Backbone Alone2026.05 | 60.51 | — | — | — | — | — | 53 | — | — | |
| MemTreeLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 60.51 | — | — | — | — | — | 46.8 | 15.89 | 0.081 | |
| A-MemBase LLM=GPT-4.1-mini, Judge LLM=GPT-4.1, # Avg. Token=72442026.05 | 59.62 | 45.83 | 44.24 | 64.8 | 58.87 | 64.35 | — | — | — | |
| Mem0-GraphLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 59.53 | — | — | — | — | — | 45 | — | — | |
| ZepBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=27012026.05 | 59.22 | 52.12 | 54.82 | 33.33 | 66.23 | — | — | — | — | |
| A-MEMLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 59.08 | — | — | — | — | — | 45.7 | 14.66 | 0.071 | |
| NemoriBackbone=gpt-4o-mini2026.05 | 59 | 56.9 | 64.9 | 48.5 | 76.4 | 32.5 | — | — | — | |
| A-MEMBackbone=gpt-4o-mini2026.05 | 58 | 49.5 | 47.4 | 38.5 | 65.3 | 61.6 | — | — | — | |
| GraphRAGLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 57.91 | — | — | — | — | — | 47.6 | — | — | |
| A-MemBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=72442026.05 | 56.34 | 38.54 | 23.68 | 62.78 | 60.64 | 68.83 | — | — | — | |
| RAPTORLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 55.9 | — | — | — | — | — | 47.8 | — | — | |
| GraphRAGLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 55.51 | — | — | — | — | — | 45.6 | — | — | |
| SupermemoryBase LLM=GPT-4o-mini, Judge LLM=GPT-4o-mini, # Avg. Token=5002026.05 | 55.34 | 51.12 | 31.77 | 42.67 | 67.3 | — | — | — | — | |
| MemoryOSBackbone=gpt-4o-mini2026.05 | 55.3 | 55.2 | 42.2 | 50.4 | 67.4 | 42.8 | — | — | — | |
| HAGEBackbone=Qwen2.5-3B2026.05 | 54.8 | 31.5 | 45.7 | 33.5 | 65.7 | 60.3 | — | — | — | |
| MemTreeLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=with GOAL-MEM2026.05 | 54.6 | — | — | — | — | — | 41.4 | 12.19 | 0.082 | |
| RAPTORLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 51.1 | — | — | — | — | — | 44.1 | — | — | |
| MemSkillBackbone=gpt-4o-mini2026.05 | 50.1 | 48 | 45.3 | 49.8 | 61.4 | 31.7 | — | — | — | |
| MAGMABackbone=Qwen2.5-3B2026.05 | 49.9 | 30.1 | 40.2 | 33.4 | 57.6 | 58.9 | — | — | — | |
| Full ContextBackbone=gpt-4o-mini2026.05 | 48.1 | 46.8 | 56.2 | 48.6 | 63 | 20.5 | — | — | — | |
| MemTreeLLM Backbone=GPT-5.4-mini, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 44.62 | — | — | — | — | — | 38.8 | — | — | |
| A-MEMLLM Backbone=Gemma-4-26B, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 44.42 | — | — | — | — | — | 38.6 | — | — | |
| MemTreeLLM Backbone=Gemma-4-26B, Memory Category=Tree-Based, Augmentation Strategy=Backbone Alone2026.05 | 42.41 | — | — | — | — | — | 33.2 | — | — | |
| A-MEMLLM Backbone=GPT-5.4-mini, Memory Category=Graph-Based, Augmentation Strategy=Backbone Alone2026.05 | 41.76 | — | — | — | — | — | 37.8 | — | — | |
| NemoriBackbone=Qwen2.5-3B2026.05 | 41.2 | 31.7 | 45 | 37.9 | 64.1 | 3.6 | — | — | — | |
| A-MEMBackbone=Qwen2.5-3B2026.05 | 41 | 25.8 | 20.3 | 21.9 | 41.6 | 68.4 | — | — | — | |
| MemoryOSBackbone=Qwen2.5-3B2026.05 | 28 | 28.5 | 21.2 | 19.4 | 34.1 | 22.9 | — | — | — | |
| Full ContextBackbone=Qwen2.5-3B2026.05 | 21.5 | 22.9 | 9.5 | 33.5 | 22.7 | 24.4 | — | — | — | |
| MemSkillBackbone=Qwen2.5-3B2026.05 | 17.9 | 14.9 | 7.9 | 15.8 | 18.7 | 26.6 | — | — | — |