Memory-augmented Question Answering on LoCoMo (Accuracy Breakdown)
92.6Accuracy (Temporal)Human
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| Human2026.04 | 92.6 | 95.1 | 85.8 | 75.4 | 89.4 | — | — | — | |
| Full-ContextLLM Model=GPT-4.1-mini2026.05 | 92.21 | 88.7 | 77.3 | 72.92 | — | 86.36 | — | — | |
| Synthius-Mem2026.04 | 89.32 | 96.73 | 94.34 | 77.33 | 99.55 | — | — | — | |
| MirixLLM Model=GPT-4.1-mini2026.05 | 88.39 | 85.11 | 83.7 | 65.62 | — | 85.38 | 10.9 | 25.76 | |
| VikingMemLLM Model=GPT-4.1-mini2026.05 | 88.16 | 93.46 | 85.89 | 79.79 | — | 90.12 | 0.2 | 0.39 | |
| Full-ContextLLM Model=GPT-4o-mini2026.05 | 83.8 | 78 | 74.82 | 59.38 | — | 77.47 | — | — | |
| ZepLLM Model=GPT-4.1-mini2026.05 | 83.33 | 79.43 | 69.16 | 73.96 | — | 79.09 | 0.42 | 0.63 | |
| VikingMemLLM Model=GPT-4o-mini2026.05 | 82.24 | 94.89 | 81.91 | 78.12 | — | 88.83 | 0.2 | 0.39 | |
| MirixLLM Model=GPT-4o-mini2026.05 | 80.86 | 79.08 | 76.01 | 66.67 | — | 78.66 | 10.9 | 25.76 | |
| ZepLLM Model=GPT-4o-mini2026.05 | 79.79 | 74.11 | 66.04 | 67.71 | — | 75.14 | 0.42 | 0.63 | |
| TiMem2026.04 | 77.63 | 81.43 | 62.2 | 52.08 | — | — | — | — | |
| MemMachine2026.04 | 73.52 | 94.65 | 87.59 | 70.83 | — | — | — | — | |
| ENGRAM2026.04 | 72.68 | 79.9 | 79.79 | 72.92 | — | — | — | — | |
| Mem0LLM Model=GPT-4.1-mini2026.05 | 66.47 | 62.41 | 57.32 | 44.79 | — | 62.47 | 0.15 | 0.2 | |
| RAGLLM Model=GPT-4.1-mini2026.05 | 64.17 | 69.56 | 56.74 | 57.29 | — | 65.32 | 0.27 | 0.91 | |
| RAGLLM Model=GPT-4o-mini2026.05 | 59.5 | 65.16 | 50.35 | 48.96 | — | 60.26 | 0.22 | 0.43 | |
| Mem0-Graph2026.04 | 58.13 | — | — | — | — | — | — | — | |
| Mem0-graphLLM Model=GPT-4o-mini2026.05 | 58.13 | 75.71 | 65.71 | 47.19 | — | 68.44 | 0.48 | 0.66 | |
| Mem02026.04 | 55.51 | 67.13 | 51.15 | 72.93 | — | — | — | — | |
| Mem0LLM Model=GPT-4o-mini2026.05 | 55.51 | 72.93 | 67.13 | 51.15 | — | 66.88 | 0.15 | 0.2 | |
| ClaudeLLM Model=GPT-4.1-mini2026.05 | 55.45 | 72.29 | 68.44 | 56.25 | — | 67.08 | 10.81 | 18.62 | |
| Mem0-graphLLM Model=GPT-4.1-mini2026.05 | 54.52 | 74.44 | 68.44 | 51.04 | — | 67.73 | 0.48 | 0.66 | |
| ClaudeLLM Model=GPT-4o-mini2026.05 | 47.35 | 63.5 | 55.67 | 50 | — | 57.86 | 18.9 | 25.42 | |
| OpenclawLLM Model=GPT-4.1-mini2026.05 | 21.81 | 26.52 | 18.09 | 26.04 | — | 23.96 | 13.73 | 19.26 | |
| OpenclawLLM Model=GPT-4o-mini2026.05 | 13.4 | 17.6 | 13.48 | 30.21 | — | 16.75 | 17.19 | 26.26 | |
| A-Mem2026.06 | — | — | — | — | — | 61.6 | — | — | |
| Infinite Context2026.06 | — | — | — | — | — | 33.4 | — | — | |
| Mem-R12026.06 | — | — | — | — | — | 62.7 | — | — | |
| MemoryGAS2026.06 | — | — | — | — | — | 49.1 | — | — | |
| SaliMoryBackbone=Qwen3.5-9B2026.06 | — | — | — | — | — | 72.9 | — | — | |
| Zero-shot AgenticBackbone=Gemini-3-Flash2026.06 | — | — | — | — | — | 70.9 | — | — |