Long-term Conversational Memory Question Answering on LoCoMo (test)
60.2Single Hop F1H-MEM
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| H-MEMBackbone=GPT-4o-mini2026.05 | 60.2 | 95.01 | 47.5 | 89.01 | 61 | 80.06 | 28.4 | 63.54 | 56.06 | 88.83 | |
| H-MEMBackbone=GPT-4.1-mini2026.05 | 57.67 | 96.08 | 46.81 | 89.01 | 64 | 89.72 | 34.9 | 72.92 | 55.58 | 92.01 | |
| H-MEMBackbone=GPT-4.1-mini, k=302026.05 | 56.8 | 96.31 | 46.79 | 93.62 | 64.41 | 90.03 | 30.62 | 69.79 | 54.92 | 92.86 | |
| EverMemOSBackbone=GPT-4.1-mini2026.05 | 53.79 | 94.29 | 45.22 | 90.43 | 63.26 | 88.16 | 33.66 | 69.79 | 52.94 | 90.78 | |
| MemOSBackbone=GPT-4.1-mini2026.05 | 52.16 | 87.63 | 41.36 | 79.79 | 61.58 | 77.88 | 30.65 | 68.75 | 50.8 | 82.99 | |
| ZepBackbone=GPT-4.1-mini2026.05 | 51.76 | 88.47 | 38.22 | 74.11 | 57.78 | 75.39 | 28.17 | 67.71 | 49.06 | 81.81 | |
| Mem0Backbone=GPT-4.1-mini2026.05 | 49.08 | 72.53 | 38.77 | 60.28 | 49.22 | 57.63 | 29.78 | 57.29 | 46.02 | 66.23 | |
| EverMemOSBackbone=GPT-4o-mini2026.05 | 48.72 | 89.06 | 39.03 | 85.11 | 58.79 | 82.87 | 28.11 | 64.58 | 47.76 | 85.52 | |
| MemTreeBackbone=GPT-4.1-mini2026.05 | 48.35 | 78.36 | 40.25 | 65.25 | 50.9 | 67.29 | 30.1 | 58.33 | 46.26 | 72.4 | |
| MemoryOSBackbone=GPT-4.1-mini2026.05 | 47.2 | 68.49 | 39.8 | 62.06 | 50.3 | 62.62 | 29.4 | 51.04 | 45.38 | 65 | |
| MemOSBackbone=GPT-4o-mini2026.05 | 46.95 | 82.52 | 36.95 | 74.47 | 55.24 | 72.9 | 26.79 | 62.5 | 45.59 | 77.79 | |
| ZepBackbone=GPT-4o-mini2026.05 | 46.81 | 83.23 | 33.57 | 69.15 | 55.93 | 70.09 | 24.26 | 61.46 | 44.88 | 76.56 | |
| Mem0Backbone=GPT-4o-mini2026.05 | 45.01 | 67.3 | 34.64 | 54.96 | 44.36 | 53.27 | 24.89 | 52.08 | 41.72 | 61.17 | |
| MemTreeBackbone=GPT-4o-mini2026.05 | 44.9 | 72.77 | 35.66 | 60.64 | 45.11 | 62.3 | 25.43 | 52.08 | 42.04 | 67.08 | |
| MemoryOSBackbone=GPT-4o-mini2026.05 | 42.95 | 63.85 | 35.18 | 56.74 | 44.41 | 57.63 | 24.63 | 45.83 | 40.69 | 60.13 |