Long-context Memory Retrieval on LoCoMo
97.1Single-hopMnemis
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| MnemisLLM=GPT-4.1-mini, k=302026.02 | 97.1 | 92.9 | 90.7 | 79.2 | 93.9 | — | — | |
| MnemisLLM=GPT-4.1-mini2026.02 | 96.2 | 91.8 | 90.3 | 82.3 | 93.3 | — | — | |
| EverMemOSLLM=GPT-4.1-mini2026.02 | 96.1 | 91.1 | 89.7 | 70.8 | 92.3 | — | — | |
| EverMemOSEvaluation Protocol=EverMemOS protocol (gpt-4.1-mini answer+judge, 7-step CoT), Tokens=2,2982026.03 | 96.1 | 91.1 | 89.7 | 70.8 | 92.3 | — | — | |
| MnemisLLM=GPT-4o-mini2026.02 | 95.7 | 89.7 | 77.6 | 79.2 | 89.8 | — | — | |
| Full-contextEvaluation Protocol=EverMemOS protocol (gpt-4.1-mini answer+judge, 7-step CoT), Tokens=20,2812026.03 | 94.9 | 90.4 | 88 | 71.9 | 91.2 | — | — | |
| MemMachine v0.2Evaluation LLM=gpt-4o-mini2026.04 | 94.65 | 87.59 | 73.52 | 70.83 | 87.47 | — | — | |
| SmartSearch (indexed)Evaluation Protocol=MemOS protocol (gpt-4o-mini answer+judge, binary J-score), Tokens=3,1412026.03 | 91.8 | 86.6 | 79.2 | 95.4 | 91.9 | — | — | |
| MemoraEvaluation Protocol=MemOS protocol (gpt-4o-mini answer+judge, binary J-score), Tokens=~8,5002026.03 | 91.8 | 78.7 | 86.6 | 59.4 | 86.3 | — | — | |
| SmartSearch (indexed)Evaluation Protocol=EverMemOS protocol (gpt-4.1-mini answer+judge, 7-step CoT), Tokens=3,1412026.03 | 91.1 | 91.3 | 80.2 | 96.7 | 93.5 | — | — | |
| SmartSearch (index-free)Evaluation Protocol=MemOS protocol (gpt-4o-mini answer+judge, binary J-score), Tokens=3,1222026.03 | 91.1 | 86 | 75 | 94.6 | 91 | — | — | |
| ZepEvaluation Protocol=EverMemOS protocol (gpt-4.1-mini answer+judge, 7-step CoT), Tokens=1,4112026.03 | 90.8 | 81.9 | 77.3 | 75 | 85.2 | — | — | |
| EMem-GLLM=GPT-4.1-mini2026.02 | 90.5 | 79.6 | 80.8 | 71.7 | 85.3 | — | — | |
| Backboard2025.12 | 89.36 | 75 | 91.9 | 91.2 | 90 | — | — | |
| Full-ContextBase Model=gpt-4.1-mini, Avg Tokens=21.4k2026.02 | 88.53 | 77.7 | 92.7 | 71.88 | 87.52 | — | — | |
| Full ContextLLM=GPT-4.1-mini2026.02 | 86.9 | 77.2 | 74.2 | 56.6 | 80.6 | — | — | |
| HindsightBackbone=Gemini-32025.12 | 86.17 | 70.83 | 83.8 | 95.12 | 89.61 | — | — | |
| MemOSEvaluation Protocol=EverMemOS protocol (gpt-4.1-mini answer+judge, 7-step CoT), Tokens=2,4982026.03 | 85.4 | 79.4 | 75.1 | 64.6 | 80.8 | — | — | |
| HyMemBase Model=gpt-4.1-mini, Avg Tokens=1.5k2026.02 | 85.15 | 88.16 | 92.98 | 77.08 | 89.55 | — | — | |
| MirixBase Model=gpt-4.1-mini, Avg Tokens=-2026.02 | 85.11 | 83.7 | 88.39 | 65.62 | 85.38 | — | — | |
| NemoriBase Model=gpt-4.1-mini, Avg Tokens=4.4k2026.02 | 84.9 | 75.1 | 77.6 | 51 | 79.4 | — | — | |
| NemoriLLM=GPT-4.1-mini2026.02 | 84.9 | 75.1 | 77.6 | 51 | 79.5 | — | — | |
| LangMemLLM=GPT-4.1-mini2026.02 | 84.5 | 71 | 50.8 | 59 | 73.4 | — | — | |
| Full ContextLLM=GPT-4o-mini2026.02 | 83 | 66.8 | 56.2 | 48.6 | 72.3 | — | — | |
| EMem-GLLM=GPT-4o-mini2026.02 | 82.3 | 74.7 | 76 | 57.3 | 78 | — | — | |
| NemoriBase Model=gpt-4o-mini, Avg Tokens=4.7k2026.02 | 82.1 | 65.3 | 71 | 44.8 | 74.4 | — | — | |
| NemoriLLM=GPT-4o-mini2026.02 | 82.1 | 65.3 | 71 | 44.8 | 74.4 | — | — | |
| BMAMTokens=–2026.01 | 82 | 70.42 | 62.31 | 79.55 | 78.45 | — | — | |
| Full-ContextBase Model=gpt-4o-mini, Avg Tokens=21.4k2026.02 | 80.14 | 46.89 | 90.49 | 58.33 | 77.51 | — | — | |
| ZepBase Model=gpt-4.1-mini, Avg Tokens=1.4k2026.02 | 79.43 | 69.16 | 83.33 | 73.96 | 79.09 | — | — | |
| MemOSLLM=GPT-4o-mini2026.02 | 78.4 | 64.3 | 73.2 | 55.2 | 73.3 | — | — | |
| HindsightBackbone=OSS-120B2025.12 | 76.79 | 62.5 | 79.44 | 93.68 | 85.67 | — | — | |
| LightMemBase Model=gpt-4o-mini, Avg Tokens=0.8k2026.02 | 76.61 | 67.02 | 76.32 | 45.83 | 72.99 | — | — | |
| RAGLLM=GPT-4.1-mini2026.02 | 76.5 | 64.9 | 76.6 | 67.7 | 73.8 | — | — | |
| MemUBase Model=gpt-4.1-mini, Avg Tokens=4.0k2026.02 | 74.91 | 72.34 | 54.17 | 43.61 | 66.67 | — | — | |
| LangMemBase Model=gpt-4.1-mini, Avg Tokens=-2026.02 | 74.47 | 61.06 | 86.92 | 67.71 | 78.05 | — | — | |
| Zep2025.12 | 74.11 | 66.04 | 79.79 | 67.71 | 75.14 | — | — | |
| HindsightBackbone=OSS-20B2025.12 | 74.11 | 64.58 | 76.32 | 90.96 | 83.18 | — | — | |
| ZepBase Model=gpt-4o-mini, Avg Tokens=1.4k2026.02 | 74.11 | 66.04 | 79.76 | 67.71 | 75.14 | — | — | |
| Zep2026.04 | 74.11 | 66.04 | 79.79 | 67.71 | 75.14 | — | — | |
| RAGLLM=GPT-4o-mini2026.02 | 73.5 | 59.9 | 62.9 | 63.5 | 68.2 | — | — | |
| Mem0Tokens=11722026.01 | 73.33 | 58.75 | 52.54 | 45.83 | 64.57 | — | — | |
| MemobaseTokens=21022026.01 | 73.12 | 64.65 | 81.2 | 53.12 | 72.01 | — | — | |
| Naive RAG(k=5)Base Model=gpt-4.1-mini, Avg Tokens=4.1k2026.02 | 72.69 | 74.14 | 81.21 | 68.75 | 77.4 | — | — | |
| Full-contextEvaluation Protocol=MemOS protocol (gpt-4o-mini answer+judge, binary J-score), Tokens=26,7922026.03 | 72.3 | 55.1 | 63.5 | 88.6 | 77.1 | — | — | |
| Mem0LLM=GPT-4.1-mini2026.02 | 71.4 | 68.2 | 56.9 | 47.9 | 66.3 | — | — | |
| Memobaseversion=v0.0.372025.12 | 70.92 | 46.88 | 85.05 | 77.17 | 75.78 | — | — | |
| Naive RAG(k=3)Base Model=gpt-4.1-mini, Avg Tokens=2.5k2026.02 | 70.92 | 69.47 | 75.74 | 65.62 | 72.97 | — | — | |
| Memobase (v0.0.37)2026.04 | 70.92 | 46.88 | 85.05 | 77.17 | 75.78 | — | — | |
| MIRIXTokens=–2026.01 | 68.32 | 54.26 | 68.54 | 46.88 | 64.33 | — | — | |
| Mem0LLM=GPT-4o-mini2026.02 | 68.1 | 60.3 | 50.4 | 40.6 | 61.3 | — | — | |
| MemUTokens=5072026.01 | 67.8 | 51.12 | 31.7 | 52.67 | 56.38 | — | — | |
| Mem02025.12 | 67.13 | 51.15 | 55.51 | 72.93 | 66.88 | — | — | |
| Mem0Base Model=gpt-4o-mini, Avg Tokens=1.7k2026.02 | 67.13 | 51.15 | 55.51 | 72.93 | 66.88 | — | — | |
| Mem02026.04 | 67.13 | 51.15 | 55.51 | 72.93 | 66.88 | — | — | |
| ZepLLM=GPT-4.1-mini2026.02 | 66.9 | 53.7 | 60.2 | 43.8 | 61.6 | — | — | |
| SupermemoryTokens=6172026.01 | 66.54 | 63.12 | 27.17 | 50.01 | 56.55 | — | — | |
| PREMemLLM=GPT-4.1-mini2026.02 | 66.2 | 61 | 74.8 | 46.9 | 65.8 | — | — | |
| Mem0-Graph2025.12 | 65.71 | 47.19 | 58.13 | 75.71 | 68.44 | — | — | |
| ZepTokens=20712026.01 | 65.23 | 52.12 | 54.82 | 33.33 | 59.22 | — | — | |
| MemOS-1031Tokens=1582, Note=Re-run using official scripts2026.01 | 64.54 | 57.29 | 71.34 | 79.9 | 73.9 | — | — | |
| OpenAI2025.12 | 63.79 | 42.92 | 21.71 | 62.29 | 52.9 | — | — | |
| OpenAI2026.04 | 63.79 | 42.92 | 21.71 | 62.29 | 52.9 | — | — | |
| ZepLLM=GPT-4o-mini2026.02 | 63.2 | 50.5 | 58.9 | 39.6 | 58.5 | — | — | |
| Mem0Base Model=gpt-4.1-mini, Avg Tokens=1.0k2026.02 | 62.41 | 57.32 | 66.47 | 44.79 | 62.47 | — | — | |
| LangMem2025.12 | 62.23 | 47.92 | 23.43 | 71.12 | 58.1 | — | — | |
| LangMemBase Model=gpt-4o-mini, Avg Tokens=-2026.02 | 62.23 | 47.92 | 23.43 | 71.12 | 49.91 | — | — | |
| LangMem2026.04 | 62.23 | 47.92 | 23.43 | 71.12 | 58.1 | — | — | |
| LangMemLLM=GPT-4o-mini2026.02 | 61.4 | 52.4 | 24.9 | 47.6 | 51.3 | — | — | |
| Context2LoRAMemory Category=Parametric Memory2026.06 | 60.11 | 37.95 | 34.99 | 16.75 | 48.11 | — | — | |
| δ-Mem (MSW)Writing Granularity=Multi-State2026.06 | 58.59 | 42.57 | 39.31 | 18.12 | 49.12 | — | — | |
| δ-Mem (SSW)Writing Granularity=Sequence-State2026.06 | 56.88 | 41 | 36.48 | 14.08 | 47.05 | — | — | |
| δ-Mem (TSW)Writing Granularity=Token-State2026.06 | 55.36 | 42.14 | 37.2 | 13.35 | 46.53 | — | — | |
| MemOS + MemGateBase Model=GPT-4o-mini2026.06 | 51.6 | 45.1 | 42.5 | 18.6 | 46.3 | — | — | |
| MemOS + MemGateBase Model=Qwen-3-8B2026.06 | 51.3 | 47.8 | 17.8 | 24.4 | 42.9 | — | — | |
| MemOSBase Model=Qwen-3-8B2026.06 | 50.7 | 46.1 | 14.6 | 18.4 | 41.7 | — | — | |
| MemOSBase Model=GPT-4o-mini2026.06 | 50.4 | 44.2 | 38.9 | 17.3 | 45.4 | — | — | |
| Mem0 + MemGateBase Model=GPT-4o-mini2026.06 | 49.8 | 43.2 | 40.4 | 15.1 | 44.5 | — | — | |
| LLMLingua-2Memory Category=Textual Memory2026.06 | 49.19 | 39.07 | 30.13 | 10.98 | 40.98 | — | — | |
| Mem0Base Model=GPT-4o-mini2026.06 | 48.6 | 42.1 | 35.3 | 15.1 | 42.9 | — | — | |
| Mem0Base Model=Qwen-3-8B2026.06 | 48.5 | 44.5 | 12.5 | 16.7 | 40 | — | — | |
| MemGenMemory Category=Parametric Memory2026.06 | 48.13 | 32.93 | 33.3 | 12.67 | 40.05 | — | — | |
| Qwen3-4B-InstructModel Type=Base Model2026.06 | 48.05 | 38.39 | 32.89 | 10.77 | 40.79 | — | — | |
| MemoryBankMemory Category=Textual Memory2026.06 | 47.31 | 37.88 | 21.76 | 13.35 | 38.14 | — | — | |
| Mem0 + MemGateBase Model=Qwen-3-8B2026.06 | 46.6 | 46.2 | 15.2 | 23.9 | 40.2 | — | — | |
| BM25 RAGMemory Category=Textual Memory2026.06 | 45.47 | 38.12 | 20.34 | 9.99 | 36.68 | — | — | |
| OpenClaw + MemGateBase Model=Qwen-3-8B2026.06 | 45 | 38.7 | 19.5 | 24.1 | 39.4 | — | — | |
| A-Mem + MemGateBase Model=Qwen-3-8B2026.06 | 44.9 | 38.6 | 17.3 | 21.7 | 38.6 | — | — | |
| OpenClaw + MemGateBase Model=GPT-4o-mini2026.06 | 44.9 | 39.4 | 38.5 | 16.9 | 40.8 | — | — | |
| OpenClawBase Model=Qwen-3-8B2026.06 | 44.6 | 38.1 | 17.5 | 21.9 | 38.5 | — | — | |
| A-MemBase Model=Qwen-3-8B2026.06 | 43.7 | 37.4 | 15.9 | 18.6 | 37.3 | — | — | |
| A-Mem + MemGateBase Model=GPT-4o-mini2026.06 | 42.6 | 37.1 | 36.3 | 19.8 | 39.1 | — | — | |
| OpenClawBase Model=GPT-4o-mini2026.06 | 42.5 | 36.9 | 35.1 | 22.8 | 38.9 | — | — | |
| A-MemBase Model=GPT-4o-mini2026.06 | 41.3 | 35.6 | 33.4 | 20.1 | 37.8 | — | — | |
| A-MemBase Model=gpt-4o-mini, Avg Tokens=2.7k2026.02 | 40.22 | 19.01 | 50.12 | 55.1 | 49.01 | — | — | |
| MLP MemoryMemory Category=Outside-channel Memory2026.06 | 30.75 | 32.87 | 16.72 | 8.81 | 26.85 | — | — | |
| A-MEM graphStorage Budget (%)=Base2026.06 | — | — | — | — | — | 40.13 | 17.12 | |
| A-MEM graphStorage Budget (%)=702026.06 | — | — | — | — | — | 40.14 | 16.9 | |
| A-MEM graphStorage Budget (%)=602026.06 | — | — | — | — | — | 39.77 | 17.15 | |
| A-MEM graphStorage Budget (%)=502026.06 | — | — | — | — | — | 39.02 | 16.28 |