Long-context Question Answering on 2WikiMultiHopQA (Out-Of-Distribution)
63.9AccuracyReMemR1
Evaluation Results
| Method | Links | |
|---|---|---|
| ReMemR1Scale=7B, Number of Context Documents=502025.09 | 63.9 | |
| ReMemR1Scale=7B, Number of Context Documents=1002025.09 | 63.1 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=502025.09 | 62.5 | |
| MemAgentScale=7B, Number of Context Documents=502025.09 | 61.7 | |
| Qwen2.5Scale=7B, Number of Context Documents=2002025.09 | 61.1 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=1002025.09 | 59.4 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=2002025.09 | 57.8 | |
| MemAgentScale=7B, Number of Context Documents=1002025.09 | 57.8 | |
| ReMemR1Scale=7B, Number of Context Documents=2002025.09 | 55.6 | |
| ReMemR1Scale=7B, Number of Context Documents=8002025.09 | 54.7 | |
| ReMemR1Scale=7B, Number of Context Documents=4002025.09 | 54.5 | |
| Qwen2.5Scale=7B, Number of Context Documents=502025.09 | 53.9 | |
| ReMemR1Scale=3B, Number of Context Documents=502025.09 | 53.5 | |
| MemAgentScale=7B, Number of Context Documents=2002025.09 | 50.8 | |
| MemAgentScale=7B, Number of Context Documents=8002025.09 | 50.7 | |
| ReMemR1Scale=3B, Number of Context Documents=1002025.09 | 50.4 | |
| ReMemR1Scale=7B, Number of Context Documents=64002025.09 | 50.3 | |
| Qwen2.5Scale=7B, Number of Context Documents=1002025.09 | 49.2 | |
| ReMemR1Scale=7B, Number of Context Documents=32002025.09 | 48.9 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=4002025.09 | 47.7 | |
| MemAgentScale=7B, Number of Context Documents=4002025.09 | 47.6 | |
| MemAgentScale=7B, Number of Context Documents=32002025.09 | 46.9 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=8002025.09 | 46.1 | |
| ReMemR1Scale=7B, Number of Context Documents=16002025.09 | 45.4 | |
| MemAgentScale=3B, Number of Context Documents=1002025.09 | 45.3 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=16002025.09 | 45.3 | |
| MemAgentScale=7B, Number of Context Documents=64002025.09 | 44.7 | |
| MemAgentScale=7B, Number of Context Documents=16002025.09 | 44.5 | |
| ReMemR1Scale=3B, Number of Context Documents=2002025.09 | 42.5 | |
| ReMemR1Scale=3B, Number of Context Documents=4002025.09 | 41.7 | |
| MemAgentScale=3B, Number of Context Documents=502025.09 | 41.4 | |
| MemAgentScale=3B, Number of Context Documents=2002025.09 | 40.2 | |
| Qwen2.5Scale=3B, Number of Context Documents=502025.09 | 39.8 | |
| MemAgentScale=3B, Number of Context Documents=4002025.09 | 39.4 | |
| Qwen2.5Scale=3B, Number of Context Documents=1002025.09 | 39.1 | |
| Qwen2.5Scale=3B, Number of Context Documents=2002025.09 | 39 | |
| ReMemR1Scale=3B, Number of Context Documents=64002025.09 | 37.8 | |
| ReMemR1Scale=3B, Number of Context Documents=8002025.09 | 37 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=502025.09 | 36.7 | |
| MemAgentScale=3B, Number of Context Documents=8002025.09 | 36.3 | |
| ReMemR1Scale=3B, Number of Context Documents=16002025.09 | 36.2 | |
| ReMemR1Scale=3B, Number of Context Documents=32002025.09 | 35.4 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=1002025.09 | 29.7 | |
| MemAgentScale=3B, Number of Context Documents=16002025.09 | 28.9 | |
| MemAgentScale=3B, Number of Context Documents=32002025.09 | 26.7 | |
| MemAgentScale=3B, Number of Context Documents=64002025.09 | 25.9 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=2002025.09 | 25.8 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=32002025.09 | 25.8 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=16002025.09 | 2.3 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=32002025.09 | 2.3 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=8002025.09 | 0.8 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=64002025.09 | 0.8 | |
| R1-Distill-QwenScale=7B, Number of Context Documents=4002025.09 | 0 | |
| Qwen2.5-1MScale=7B, Number of Context Documents=64002025.09 | 0 |