Memory-Augmented Question Answering on LoCoMo Temporal
20.77Token F1Mem0
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| Mem0Model=GPT-4o mini2026.05 | 20.77 | 15.51 | 42.71 | |
| SAGEModel=GPT-4o mini2026.05 | 19.87 | 14.73 | 35.42 | |
| Mem0Model=Qwen2.5 1.5B2026.05 | 12.34 | 10.77 | 68.75 | |
| SAGEModel=Qwen2.5 7B2026.05 | 12.3 | 10.83 | 76.04 | |
| SAGEModel=Qwen2.5 3B2026.05 | 12.05 | 10.07 | 67.71 | |
| Mem0Model=Qwen2.5 7B2026.05 | 11.98 | 9.14 | 77.08 | |
| Mem0gModel=Qwen2.5 7B2026.05 | 11.87 | 8.7 | 73.96 | |
| Mem0gModel=Qwen2.5 3B2026.05 | 11.75 | 9.43 | 71.88 | |
| SAGEModel=Qwen2.5 1.5B2026.05 | 11.25 | 10.18 | 67.71 | |
| Mem0Model=Qwen2.5 3B2026.05 | 10.58 | 9.38 | 64.58 | |
| SAGEModel=DeepSeek-R1 1.5B2026.05 | 9.8 | 7.36 | 87.5 | |
| SAGEModel=DeepSeek-R1 7B2026.05 | 9.64 | 7.22 | 87.5 | |
| Mem0gModel=Qwen2.5 1.5B2026.05 | 9.58 | 9.49 | 60.42 | |
| Mem0gModel=DeepSeek-R1 7B2026.05 | 9.53 | 6.94 | 96.88 | |
| Mem0Model=DeepSeek-R1 7B2026.05 | 9.21 | 7.27 | 92.71 | |
| Mem0gModel=DeepSeek-R1 1.5B2026.05 | 8.49 | 6.01 | 89.58 | |
| Mem0Model=DeepSeek-R1 1.5B2026.05 | 7.58 | 4.83 | 92.71 | |
| SAGEModel=Llama-3.2 1B2026.05 | 5.57 | 2.57 | 86.46 | |
| SAGEModel=Llama-3.2 3B2026.05 | 5.22 | 2.34 | 75 | |
| Mem0gModel=Llama-3.2 3B2026.05 | 4.97 | 2.33 | 81.25 | |
| Mem0Model=Llama-3.2 3B2026.05 | 4.49 | 2.03 | 72.92 | |
| Mem0Model=Llama-3.2 1B2026.05 | 4.48 | 2 | 55.21 | |
| Mem0gModel=Llama-3.2 1B2026.05 | 4.34 | 1.93 | 65.62 |