Multi-hop Question Answering on 2WikiMultiHopQA Out-Of-Distribution (OOD)
74.2AccuracyQwenLong-L1-32B
Evaluation results
| Method | Links | |
|---|---|---|
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=502025.09 | 74.2 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=502025.09 | 71.9 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=1002025.09 | 69.5 | |
| Qwen3-4BScale=<7B, Number of Context Documents=502025.09 | 67.2 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=502025.09 | 67.2 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=2002025.09 | 65.6 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=502025.09 | 63.9 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=1002025.09 | 63.1 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=502025.09 | 62.5 | |
| Qwen3-4BScale=<7B, Number of Context Documents=1002025.09 | 60.9 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=1002025.09 | 60.9 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=1002025.09 | 59.4 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=502025.09 | 58.6 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=4002025.09 | 58.6 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=1002025.09 | 57.8 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=2002025.09 | 57.8 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=2002025.09 | 57 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=1002025.09 | 56.3 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=2002025.09 | 56.3 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=2002025.09 | 55.6 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=8002025.09 | 54.7 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=4002025.09 | 54.5 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=502025.09 | 53.5 | |
| Qwen3-4BScale=<7B, Number of Context Documents=2002025.09 | 53.1 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=2002025.09 | 52.3 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=4002025.09 | 51.6 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=1002025.09 | 50.4 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=64002025.09 | 50.3 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=8002025.09 | 49.2 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=4002025.09 | 49.2 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=32002025.09 | 48.9 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=4002025.09 | 47.7 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=8002025.09 | 47.7 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=8002025.09 | 46.1 | |
| ReMemR1Scale=>=7B, backbone=Qwen2.5-7B, Number of Context Documents=16002025.09 | 45.4 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=16002025.09 | 45.3 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=16002025.09 | 45.3 | |
| Qwen3-4BScale=<7B, Number of Context Documents=4002025.09 | 43 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=2002025.09 | 42.5 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=4002025.09 | 42.2 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=4002025.09 | 41.7 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=8002025.09 | 38.3 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=64002025.09 | 37.8 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=8002025.09 | 37 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=502025.09 | 36.7 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=16002025.09 | 36.2 | |
| ReMemR1Scale=<7B, backbone=Qwen2.5-3B, Number of Context Documents=32002025.09 | 35.4 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=32002025.09 | 34.4 | |
| Qwen3-4BScale=<7B, Number of Context Documents=8002025.09 | 32 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=64002025.09 | 32 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=64002025.09 | 31.3 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=64002025.09 | 29.9 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=1002025.09 | 29.7 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=16002025.09 | 29.7 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=8002025.09 | 28.1 | |
| R1-Distill-Qwen-14BScale=>=7B, Number of Context Documents=32002025.09 | 28.1 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=16002025.09 | 28.1 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=32002025.09 | 26.6 | |
| Qwen3-4BScale=<7B, Number of Context Documents=64002025.09 | 25.8 | |
| Qwen3-8BScale=>=7B, Number of Context Documents=16002025.09 | 25.8 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=2002025.09 | 25.8 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=32002025.09 | 25.8 | |
| Qwen3-4BScale=<7B, Number of Context Documents=16002025.09 | 25 | |
| QwenLong-L1-32BScale=>=7B, Number of Context Documents=32002025.09 | 24.6 | |
| Qwen3-4BScale=<7B, Number of Context Documents=32002025.09 | 21.1 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=16002025.09 | 2.3 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=32002025.09 | 2.3 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=8002025.09 | 0.8 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=64002025.09 | 0.8 | |
| R1-Distill-Qwen-7BScale=>=7B, Number of Context Documents=4002025.09 | 0 | |
| Qwen2.5-1M-7BScale=>=7B, Number of Context Documents=64002025.09 | 0 | |
| Qwen2.5-1M-14BScale=>=7B, Number of Context Documents=64002025.09 | 0 |