Memory Retrieval on LongMemEval n=500 (test)
80.43MAPMemReranker-4B
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| MemReranker-4BParameters=4B2026.05 | 80.43 | 83.15 | 78 | 80.69 | 83.54 | 83.69 | 82.03 | 87.34 | 89.6 | 70.39 | |
| MemReranker-0.6BParameters=0.6B2026.05 | 75.38 | 78.76 | 70.2 | 75.62 | 79.87 | 80.23 | 78.72 | 85.08 | 89.69 | 63.75 | |
| Gemini-3-Flash2026.05 | 72.59 | 75.75 | 66.6 | 72.22 | 77.33 | 78.08 | 75.56 | 82.8 | 89.77 | 60.1 | |
| BGE-v2-m32026.05 | 70.69 | 75.1 | 66 | 70.51 | 75.73 | 76.96 | 74.12 | 80.95 | 88.74 | 56.34 | |
| Qwen3-Reranker-8BParameters=8B2026.05 | 69.66 | 73.4 | 63 | 68.51 | 75.51 | 76.17 | 72.1 | 81.08 | 89.31 | 54.9 | |
| Qwen3-Reranker-0.6BParameters=0.6B2026.05 | 69.55 | 73.67 | 62.6 | 69 | 75.2 | 76.16 | 73.13 | 81.35 | 89 | 55.67 | |
| Qwen3-Reranker-4BParameters=4B2026.05 | 65.01 | 69.37 | 56.4 | 64.03 | 71.85 | 72.9 | 68.72 | 79 | 89.34 | 49.81 | |
| GPT-4o-miniNote=Evaluated 481 out of 500 queries due to API failures2026.05 | 56.84 | 63.85 | 53.22 | 55.51 | 62.65 | 66.76 | 57.81 | 64.63 | 83.49 | 44.44 |