Long-context memory evaluation on LoCoMo full
61.56Single-Hop F1Full-Context
Evaluation Results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| Full-ContextBase Model=GPT-4o, Training Protocol=training-free2026.05 | 61.56 | 54.19 | 9.09 | 5.78 | 16.47 | 14.8 | 28 | 18.47 | 29.49 | 22.14 | |
| SimpleMemBase Model=Qwen3-8B, Training Protocol=training-free2026.05 | 46.62 | 40.69 | 42.85 | 36.49 | 15.35 | 13.9 | 28.97 | 24.93 | 34.25 | 29.54 | |
| PPRO (w/o GRPO)Base Model=GPT-4o, Training Protocol=training-free2026.05 | 45.67 | 40.73 | 59.68 | 48.68 | 27.06 | 24.25 | 47 | 41.17 | 48.16 | 41.6 | |
| SimpleMemBase Model=GPT-4o, Training Protocol=training-free2026.05 | 45.41 | 39.25 | 56.71 | 20.57 | 18.23 | 16.34 | 35.89 | 32.83 | 40.87 | 30.42 | |
| A-MemBase Model=GPT-4o, Training Protocol=training-free2026.05 | 44.43 | 38.97 | 39.41 | 31.23 | 17.1 | 15.84 | 32.86 | 23.76 | 35.36 | 27.61 | |
| MemGPTBase Model=GPT-4o, Training Protocol=training-free2026.05 | 40.16 | 36.55 | 17.29 | 13.18 | 12.24 | 11.87 | 30.36 | 22.83 | 28.3 | 22.65 | |
| Mem0Base Model=GPT-4o, Training Protocol=training-free2026.05 | 39.12 | 35.43 | 52.38 | 44.15 | 17.73 | 15.92 | 35.13 | 27.56 | 38.37 | 31.73 | |
| PPRO (w/o GRPO)Base Model=Qwen3-8B, Training Protocol=training-free2026.05 | 38.44 | 29.32 | 53.38 | 43.97 | 24.45 | 19.2 | 43.35 | 37.89 | 43.36 | 36.42 | |
| LightMemBase Model=GPT-4o, Training Protocol=training-free2026.05 | 33.76 | 28.02 | 36.53 | 29.12 | 13.38 | 11.54 | 28.15 | 21.83 | 30 | 23.84 | |
| Mem0Base Model=Qwen3-8B, Training Protocol=training-free2026.05 | 33.05 | 27.24 | 32.48 | 26.13 | 15.23 | 12.54 | 22.42 | 16.83 | 26.02 | 20.41 | |
| PPRO (w/o GRPO)Base Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 30.27 | 23.04 | 49.39 | 37.08 | 16.97 | 13.95 | 37.01 | 32.08 | 37.11 | 30.34 | |
| LightMemBase Model=Qwen3-8B, Training Protocol=training-free2026.05 | 29.48 | 23.83 | 26.78 | 21.52 | 14.12 | 11.24 | 18.53 | 14.23 | 21.98 | 17.32 | |
| A-MemBase Model=Qwen3-8B, Training Protocol=training-free2026.05 | 26.8 | 21.5 | 22.5 | 18.2 | 13.2 | 10.5 | 20.5 | 13.8 | 21.62 | 15.92 | |
| SimpleMemBase Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 20.9 | 18.01 | 21.47 | 19.5 | 12.52 | 10.19 | 17.03 | 11.87 | 18.38 | 14.48 | |
| LightMemBase Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 18.28 | 15.24 | 6.92 | 4.56 | 8.06 | 7.23 | 16.43 | 11.54 | 14.27 | 10.49 | |
| A-MemBase Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 17.23 | 13.12 | 27.59 | 25.07 | 7.12 | 7.28 | 12.57 | 9.01 | 16.21 | 13 | |
| Mem0Base Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 16.47 | 12.43 | 8.52 | 6.23 | 10.24 | 8.82 | 16.89 | 11.56 | 14.65 | 10.44 | |
| Full-ContextBase Model=Qwen3-8B, Training Protocol=training-free2026.05 | 14.5 | 11.2 | 6.8 | 5.5 | 10.1 | 8.8 | 13.5 | 9.2 | 12.07 | 8.77 | |
| MemGPTBase Model=Qwen3-8B, Training Protocol=training-free2026.05 | 11.5 | 9.1 | 5.5 | 4.2 | 12.5 | 10.8 | 14.2 | 9.8 | 11.79 | 8.57 | |
| MemGPTBase Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 7.26 | 5.52 | 2.94 | 2.95 | 7.04 | 7.1 | 5.07 | 4.31 | 5.15 | 4.42 | |
| Full-ContextBase Model=Qwen2.5-3B, Training Protocol=training-free2026.05 | 7.03 | 5.69 | 3.11 | 2.71 | 4.55 | 5.97 | 4.61 | 4.29 | 4.74 | 4.32 |