Memory-Augmented Question Answering on LoCoMo Open-domain
44.79Mean Token F1SAGE
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| SAGEModel=GPT-4o mini2026.05 | 44.79 | 36.65 | 63.26 | |
| Mem0Model=GPT-4o mini2026.05 | 44.02 | 36.71 | 62.9 | |
| SAGEModel=Qwen2.5 7B2026.05 | 36.3 | 31.22 | 92.75 | |
| SAGEModel=Qwen2.5 3B2026.05 | 35.69 | 30.12 | 90.61 | |
| Mem0Model=Qwen2.5 3B2026.05 | 33.25 | 28.76 | 90.84 | |
| Mem0Model=Qwen2.5 7B2026.05 | 32.95 | 28.65 | 90.84 | |
| Mem0gModel=Qwen2.5 3B2026.05 | 31.98 | 26.94 | 88.35 | |
| Mem0gModel=Qwen2.5 7B2026.05 | 31.86 | 27.76 | 87.75 | |
| SAGEModel=DeepSeek-R1 7B2026.05 | 14.76 | 8.76 | 94.77 | |
| Mem0Model=DeepSeek-R1 7B2026.05 | 14.56 | 9.04 | 94.17 | |
| Mem0gModel=DeepSeek-R1 7B2026.05 | 14.42 | 8.76 | 94.05 | |
| SAGEModel=DeepSeek-R1 1.5B2026.05 | 10.67 | 5.77 | 94.05 | |
| Mem0Model=DeepSeek-R1 1.5B2026.05 | 8.21 | 4.21 | 93.1 | |
| SAGEModel=Qwen2.5 1.5B2026.05 | 8.04 | 6.81 | 78.6 | |
| Mem0gModel=DeepSeek-R1 1.5B2026.05 | 8 | 4.4 | 91.44 | |
| Mem0gModel=Qwen2.5 1.5B2026.05 | 7.74 | 6.12 | 78.12 | |
| Mem0Model=Qwen2.5 1.5B2026.05 | 7.28 | 5.72 | 79.19 | |
| SAGEModel=Llama-3.2 1B2026.05 | 6.52 | 2.49 | 90.01 | |
| SAGEModel=Llama-3.2 3B2026.05 | 6.1 | 1.94 | 75.86 | |
| Mem0Model=Llama-3.2 3B2026.05 | 5.88 | 1.91 | 72.18 | |
| Mem0gModel=Llama-3.2 3B2026.05 | 5.56 | 1.8 | 72.65 | |
| Mem0Model=Llama-3.2 1B2026.05 | 3.15 | 1.1 | 59.22 | |
| Mem0gModel=Llama-3.2 1B2026.05 | 3.05 | 1.07 | 60.88 |