End-to-End Performance on LongMemEval
59.9Top-5 RecallQRhead+ReAttn
Evaluation results
| Method | Links | ||
|---|---|---|---|
| QRhead+ReAttnBase LLM=Llama-3.1-8B-Instruct2026.02 | 59.9 | 60.5 | |
| QRheadBase LLM=Llama-3.1-8B-Instruct2026.02 | 59.8 | 60.2 | |
| ICR+ReAttnBase LLM=Llama-3.1-8B-Instruct2026.02 | 59.4 | 56.3 | |
| ICRBase LLM=Llama-3.1-8B-Instruct2026.02 | 59.3 | 56.1 | |
| ContrieverBase LLM=Llama-3.1-8B-Instruct2026.02 | 52.6 | 55.4 | |
| StellaBase LLM=Llama-3.1-8B-Instruct2026.02 | 50.9 | 58.4 | |
| BM25Base LLM=Llama-3.1-8B-Instruct2026.02 | 48.8 | 50.9 | |
| ContrieverBase LLM=Llama-3.2-3B-Instruct2026.02 | 48.6 | 46.5 | |
| QRhead+ReAttnBase LLM=Llama-3.2-3B-Instruct2026.02 | 47.6 | 48 | |
| QRheadBase LLM=Llama-3.2-3B-Instruct2026.02 | 47.4 | 47.7 | |
| ICR+ReAttnBase LLM=Llama-3.2-3B-Instruct2026.02 | 46.6 | 45.5 | |
| ICRBase LLM=Llama-3.2-3B-Instruct2026.02 | 46.5 | 45.1 | |
| Full contextBase LLM=Llama-3.1-8B-Instruct2026.02 | 46.5 | — | |
| BM25Base LLM=Llama-3.2-3B-Instruct2026.02 | 46.1 | 44.9 | |
| StellaBase LLM=Llama-3.2-3B-Instruct2026.02 | 44.9 | 47.7 | |
| Full contextBase LLM=Llama-3.2-3B-Instruct2026.02 | 28.1 | — | |
| RankGPTBubbleBase LLM=Llama-3.1-8B-Instruct2026.02 | 28.1 | 27 | |
| RankGPTw/oBase LLM=Llama-3.1-8B-Instruct2026.02 | 26.7 | 24.2 | |
| RankGPTBubbleBase LLM=Llama-3.2-3B-Instruct2026.02 | 24 | 24.4 | |
| RankGPTw/oBase LLM=Llama-3.2-3B-Instruct2026.02 | 23.5 | 23.3 |