Long-context Memory Retrieval and Reasoning on LOCOMO (Evaluation)
48.01Single-Hop Retrieval F1Ours
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| OursModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 48.01 | 38.08 | 43.97 | 24.27 | 43.87 | |
| OursModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 45.11 | 30.01 | 32.23 | 18.92 | 38.03 | |
| MemOSModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 44.3 | 36.67 | 49.59 | 24.88 | 42.79 | |
| MemoryOSModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 43.56 | 36.32 | 37.36 | 22.57 | 39.63 | |
| ZepModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 42.46 | 33.11 | 34.58 | 15.87 | 37.45 | |
| A-MEMModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 42.24 | 27.87 | 28.01 | 23.27 | 30.35 | |
| ZepModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 40.42 | 30.97 | 22.64 | 21.79 | 33.82 | |
| MemOSModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 39.55 | 34.04 | 37.55 | 22.55 | 37.05 | |
| MemTreeModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 38.73 | 33.01 | 29.81 | 23.02 | 34.85 | |
| MemTreeModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 33.36 | 26.4 | 25.62 | 20.67 | 29.68 | |
| MemoryOSModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 33.14 | 28.52 | 17.65 | 22.05 | 28.34 | |
| A-MEMModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 33.11 | 25.38 | 14.69 | 14.69 | 26.71 | |
| Mem0Model Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 32.83 | 25.35 | 40.76 | 21.04 | 32.38 | |
| Mem0^gModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 29.58 | 30.22 | 43.89 | 20.37 | 32.07 | |
| MemGPTModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 28.78 | 24.21 | 20.28 | 16.42 | 25.4 | |
| Mem0^gModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 27.9 | 21.14 | 35.7 | 18.67 | 27.71 | |
| Mem0Model Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 25.92 | 19.43 | 37.49 | 16.93 | 26.58 | |
| MemoryBankModel Size=72B, Backbone LLM=Qwen2.5-Instruct2026.04 | 24.54 | 23.28 | 13.21 | 19.12 | 21.61 | |
| MemGPTModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 21.31 | 18.92 | 17.37 | 11.14 | 19.42 | |
| MemoryBankModel Size=7B, Backbone LLM=Qwen2.5-Instruct2026.04 | 15.34 | 16.63 | 11.57 | 16.09 | 14.84 |