Long-context Question Answering on HotpotQA In-Distribution
85.2AccuracyQwenLong-L1-32B
Evaluation results
| Method | Links | |
|---|---|---|
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=1002025.09 | 85.2 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=1002025.09 | 83.6 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=502025.09 | 83.6 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=1002025.09 | 82.8 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=502025.09 | 82.3 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=8002025.09 | 82 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=502025.09 | 81.3 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=2002025.09 | 81.1 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=64002025.09 | 80.8 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=32002025.09 | 80 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=502025.09 | 79.7 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=16002025.09 | 79.7 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=1002025.09 | 78.9 | |
| ReMemR1Scale=⩾7B, Backbone=Qwen2.5-7B, Number of Context Documents=4002025.09 | 78.9 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=502025.09 | 78.1 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=1002025.09 | 76.6 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=2002025.09 | 76.6 | |
| Qwen3-4BScale=<7B, Number of Context Documents=1002025.09 | 75.8 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=502025.09 | 75.8 | |
| Qwen3-4BScale=<7B, Number of Context Documents=502025.09 | 75 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=8002025.09 | 74.2 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=2002025.09 | 74.2 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=4002025.09 | 74 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=4002025.09 | 73.4 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=4002025.09 | 73.4 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=2002025.09 | 71.9 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=1002025.09 | 71.9 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=1002025.09 | 71.7 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=502025.09 | 70.9 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=4002025.09 | 70.3 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=8002025.09 | 70.3 | |
| Qwen3-4BScale=<7B, Number of Context Documents=2002025.09 | 69.5 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=8002025.09 | 69.5 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=2002025.09 | 68 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=4002025.09 | 67.2 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=64002025.09 | 66.1 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=8002025.09 | 65.4 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=32002025.09 | 65.4 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=16002025.09 | 65 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=2002025.09 | 64.1 | |
| ReMemR1Scale=<7B, Backbone=Qwen2.5-3B, Number of Context Documents=2002025.09 | 63.8 | |
| Qwen3-4BScale=<7B, Number of Context Documents=4002025.09 | 63.3 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=16002025.09 | 60.9 | |
| Qwen3-4BScale=<7B, Number of Context Documents=8002025.09 | 60.2 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=4002025.09 | 57.8 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=8002025.09 | 57.8 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=16002025.09 | 54.7 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=16002025.09 | 45.3 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=32002025.09 | 42.2 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=502025.09 | 40.6 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=8002025.09 | 40.6 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=32002025.09 | 38.9 | |
| QwenLong-L1-32BScale=⩾7B, Number of Context Documents=64002025.09 | 38.3 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=16002025.09 | 33.6 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=16002025.09 | 33.6 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=64002025.09 | 31.3 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=1002025.09 | 25.8 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=32002025.09 | 23.4 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=32002025.09 | 22.7 | |
| Qwen3-4BScale=<7B, Number of Context Documents=16002025.09 | 21.9 | |
| R1-Distill-Qwen-14BScale=⩾7B, Number of Context Documents=32002025.09 | 20.3 | |
| Qwen3-8BScale=⩾7B, Number of Context Documents=64002025.09 | 19.5 | |
| Qwen3-4BScale=<7B, Number of Context Documents=32002025.09 | 18.8 | |
| Qwen3-4BScale=<7B, Number of Context Documents=64002025.09 | 18.8 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=2002025.09 | 10.2 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=64002025.09 | 3.1 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=16002025.09 | 2.3 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=8002025.09 | 1.6 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=32002025.09 | 1.5 | |
| R1-Distill-Qwen-7BScale=⩾7B, Number of Context Documents=4002025.09 | 0.8 | |
| Qwen2.5-1M-7BScale=⩾7B, Number of Context Documents=64002025.09 | 0 | |
| Qwen2.5-1M-14BScale=⩾7B, Number of Context Documents=64002025.09 | 0 |