Long-term Memory Question Answering on LongMemEvalS
78.7SSU F1H-MEM
Evaluation results
| Method | Links | |||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| H-MEM2026.05 | 78.7 | 98.57 | 46.7 | 83.46 | 12.08 | 93.33 | 55.35 | 84.96 | 66.6 | 91.03 | 82.35 | 96.43 | 58.5 | 89.2 | — | — | — | — | — | — | — | |
| EverMemOS2026.05 | 78.03 | 97.14 | 43.27 | 73.68 | 13.52 | 93.33 | 43.88 | 77.44 | 58.61 | 89.74 | 79.46 | 83.93 | 52.96 | 82.8 | — | — | — | — | — | — | — | |
| MemTree2026.05 | 69.84 | 87.14 | 34.18 | 63.91 | 10.96 | 63.33 | 36.92 | 69.17 | 48.63 | 69.23 | 68.74 | 75 | 44.63 | 70.6 | — | — | — | — | — | — | — | |
| MemOS2026.05 | 69.68 | 95.71 | 33.21 | 70.68 | 12.74 | 96.67 | 34.02 | 76.69 | 45.86 | 79.49 | 58.35 | 67.86 | 42.09 | 78.4 | — | — | — | — | — | — | — | |
| MemoryOS2026.05 | 67.88 | 88.57 | 30.42 | 61.65 | 11.21 | 76.67 | 33.06 | 66.17 | 44.83 | 69.23 | 60.74 | 76.79 | 40.86 | 70.4 | — | — | — | — | — | — | — | |
| Zep2026.05 | 64.11 | 88.57 | 24.83 | 45.11 | 10.26 | 50 | 30.72 | 51.88 | 47.09 | 74.36 | 62.35 | 75 | 38.7 | 61.2 | — | — | — | — | — | — | — | |
| Mem02026.05 | 61.73 | 82.86 | 29.86 | 63.16 | 11.58 | 90 | 33.47 | 72.18 | 46.21 | 66.67 | 40.36 | 26.79 | 37.91 | 66.4 | — | — | — | — | — | — | — | |
| Full-contextLLM judge=gpt-4.1-mini2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 27.06 | 30.08 | 41.03 | 47.14 | 32.14 | 60 | 39.57 | |
| LightMemLLM judge=gpt-4.1-mini2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 85.71 | 47.37 | 92.3 | 88.57 | 21.43 | 76.67 | 68.67 | |
| Mem0LLM judge=gpt-4.1-mini2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 40.6 | 50.37 | 69.23 | 87.14 | 48.21 | 63.33 | 59.81 | |
| RaMemLLM judge=gpt-4.1-mini2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 86.47 | 65.41 | 83.33 | 87.14 | 78.57 | 80 | 80.15 | |
| SimpleMemLLM judge=gpt-4.1-mini2026.06 | — | — | — | — | — | — | — | — | — | — | — | — | — | — | 83.46 | 60.92 | 79.48 | 85.71 | 75 | 76.67 | 76.87 |