Long-term Question Answering on LoCoMo
44.24Multi-Hop F1AriadneMem
Evaluation results
| Method | Links | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AriadneMemModel=GPT-4.1-mini2026.02 | 44.24 | 37.04 | 64.28 | 52.33 | 22.56 | 18.86 | 54.11 | 48.65 | 46.3 | 39.22 | 916 | |
| SimpleMemModel=GPT-4.1-mini2026.02 | 43.46 | 38.82 | 58.62 | 50.1 | 19.76 | 18.04 | 51.12 | 43.53 | 43.24 | 37.62 | 531 | |
| AriadneMemModel=Qwen3-Plus2026.02 | 42.17 | 36.06 | 63.67 | 49.84 | 24.46 | 22.92 | 53.8 | 48.71 | 46.03 | 39.38 | 460 | |
| AriadneMemModel=GPT-4o2026.02 | 41.34 | 34.49 | 57.94 | 46.34 | 25.02 | 22.93 | 45.97 | 40.11 | 42.57 | 35.97 | 497 | |
| SimpleMemModel=GPT-4o2026.02 | 35.89 | 32.83 | 56.71 | 20.57 | 18.23 | 16.34 | 45.41 | 39.25 | 39.06 | 27.25 | 550 | |
| Mem0Model=GPT-4o2026.02 | 35.13 | 27.56 | 52.38 | 44.15 | 17.73 | 15.92 | 39.12 | 35.43 | 36.09 | 30.77 | 985 | |
| SimpleMemModel=Qwen3-Plus2026.02 | 33.74 | 29.04 | 50.87 | 43.31 | 18.41 | 16.24 | 46.94 | 38.16 | 37.49 | 31.69 | 583 | |
| A-MemModel=GPT-4o2026.02 | 32.86 | 23.76 | 39.41 | 31.23 | 17.1 | 15.84 | 44.43 | 38.97 | 33.45 | 27.45 | 1,216 | |
| Mem0Model=Qwen3-Plus2026.02 | 32.42 | 21.24 | 47.53 | 39.82 | 17.18 | 14.53 | 46.25 | 37.52 | 35.85 | 28.28 | 1,020 | |
| MemGPTModel=GPT-4o2026.02 | 30.36 | 22.83 | 17.29 | 13.18 | 12.24 | 11.87 | 40.16 | 36.35 | 25.01 | 21.06 | 16,987 | |
| Mem0Model=GPT-4.1-mini2026.02 | 30.14 | 27.62 | 48.91 | 44.82 | 16.43 | 14.94 | 41.3 | 36.17 | 34.2 | 30.89 | 973 | |
| LightMemModel=Qwen3-Plus2026.02 | 28.95 | 24.13 | 42.58 | 38.52 | 16.54 | 13.23 | 40.78 | 36.52 | 32.21 | 28.1 | 606 | |
| LightMemModel=GPT-4o2026.02 | 28.15 | 21.83 | 36.53 | 29.12 | 13.38 | 11.54 | 33.76 | 28.02 | 27.96 | 22.63 | 645 | |
| LoCoMoModel=GPT-4o2026.02 | 28 | 18.47 | 9.09 | 5.78 | 16.47 | 14.8 | 61.56 | 54.19 | 28.78 | 23.31 | 16,910 | |
| A-MemModel=Qwen3-Plus2026.02 | 26.5 | 19.8 | 46.1 | 35.1 | 11.9 | 11.5 | 43.8 | 36.5 | 32.08 | 25.73 | 1,427 | |
| MemGPTModel=Qwen3-Plus2026.02 | 25.8 | 17.5 | 24.1 | 18.5 | 9.5 | 7.8 | 40.2 | 42.1 | 24.9 | 21.48 | 16,958 | |
| A-MemModel=GPT-4.1-mini2026.02 | 25.06 | 17.32 | 51.01 | 44.75 | 13.22 | 14.75 | 41.02 | 36.99 | 32.58 | 28.45 | 2,520 | |
| LoCoMoModel=GPT-4.1-mini2026.02 | 25.02 | 21.62 | 12.04 | 10.63 | 19.05 | 17.07 | 18.68 | 15.87 | 18.7 | 16.3 | 16,910 | |
| LightMemModel=GPT-4.1-mini2026.02 | 24.96 | 21.66 | 20.55 | 18.39 | 19.21 | 17.68 | 33.79 | 29.66 | 24.63 | 21.85 | 612 | |
| LoCoMoModel=Qwen3-Plus2026.02 | 24.15 | 18.94 | 16.57 | 13.28 | 11.81 | 10.58 | 38.58 | 28.16 | 22.78 | 17.74 | 16,910 | |
| MemGPTModel=GPT-4.1-mini2026.02 | 17.72 | 16.02 | 19.44 | 16.54 | 11.29 | 10.18 | 25.59 | 24.25 | 18.51 | 16.75 | 16,977 | |
| ReadAgentModel=GPT-4o2026.02 | 14.61 | 9.95 | 4.16 | 3.19 | 8.84 | 8.37 | 12.46 | 10.29 | 10.02 | 7.95 | 805 | |
| ReadAgentModel=Qwen3-Plus2026.02 | 9.52 | 6.83 | 11.22 | 8.15 | 5.41 | 5.23 | 9.85 | 7.96 | 9 | 7.04 | 742 | |
| MemoryBankModel=GPT-4o2026.02 | 6.49 | 4.69 | 2.47 | 2.43 | 6.43 | 5.3 | 8.28 | 7.1 | 5.92 | 4.88 | 569 | |
| ReadAgentModel=GPT-4.1-mini2026.02 | 6.48 | 5.6 | 5.31 | 4.23 | 7.66 | 6.62 | 9.18 | 7.91 | 7.16 | 6.09 | 643 | |
| MemoryBankModel=Qwen3-Plus2026.02 | 5.25 | 4.94 | 1.77 | 6.26 | 5.88 | 6 | 6.9 | 5.57 | 4.95 | 5.69 | 302 | |
| MemoryBankModel=GPT-4.1-mini2026.02 | 5 | 4.68 | 5.94 | 4.78 | 5.16 | 4.52 | 5.72 | 4.86 | 5.46 | 4.71 | 432 |