Multi-hop Reasoning on RULER QA
95.4Accuracy (32K Context)Qwen2.5-14B-1M-LongRLVR
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Qwen2.5-14B-1M-LongRLVRBackbone=Qwen2.5-14B-1M, Protocol=LongRLVR2026.03 | 95.4 | 87.8 | 83.5 | 88.9 | |
| Qwen3-14B (Thinking)Backbone=Qwen3-14B, Thinking Mode=true2026.03 | 91.2 | 89 | 82.6 | 87.6 | |
| Qwen2.5-14B-1MBackbone=Qwen2.5-14B-1M2026.03 | 90.6 | 70.6 | 64.4 | 75.2 | |
| QwenLong-L1-32BBackbone=QwenLong-L1-32B2026.03 | 89 | 77 | 72.4 | 79.47 | |
| Qwen2.5-14B-1M-SFTBackbone=Qwen2.5-14B-1M, Protocol=SFT2026.03 | 88 | 66.5 | 62.2 | 72.23 | |
| Qwen3-8B (Thinking)Backbone=Qwen3-8B, Thinking Mode=true2026.03 | 86.5 | 84 | 81.8 | 84.1 | |
| Qwen2.5-14B-1M-RLVRBackbone=Qwen2.5-14B-1M, Protocol=RLVR2026.03 | 86.3 | 69 | 64.2 | 73.17 | |
| LLaMA-3.1-8B-LongRLVRBackbone=LLaMA-3.1-8B, Protocol=LongRLVR2026.03 | 85.5 | 76.5 | 79 | 80.33 | |
| Qwen2.5-7B-1M-LongRLVRBackbone=Qwen2.5-7B-1M, Protocol=LongRLVR2026.03 | 82.5 | 76.5 | 77 | 78.67 | |
| Qwen2.5-7B-1M-RLVRBackbone=Qwen2.5-7B-1M, Protocol=RLVR2026.03 | 74.4 | 68.5 | 57.8 | 66.9 | |
| Qwen2.5-7B-1M-SFTBackbone=Qwen2.5-7B-1M, Protocol=SFT2026.03 | 72.4 | 64.2 | 56.8 | 64.47 | |
| LLaMA-3.1-8B-RLVRBackbone=LLaMA-3.1-8B, Protocol=RLVR2026.03 | 72 | 68.8 | 62.6 | 67.8 | |
| Qwen2.5-7B-1MBackbone=Qwen2.5-7B-1M2026.03 | 70.5 | 66 | 58.5 | 65 | |
| LLaMA-3.1-70BBackbone=LLaMA-3.1-70B2026.03 | 70.4 | 64.2 | 47.6 | 60.73 | |
| LLaMA-3.1-8B-SFTBackbone=LLaMA-3.1-8B, Protocol=SFT2026.03 | 68.4 | 65.3 | 60.4 | 64.7 | |
| Qwen2.5-72B-YaRNBackbone=Qwen2.5-72B, Extension=YaRN2026.03 | 66.9 | 54.5 | 47.2 | 56.2 | |
| LLaMA-3.1-8BBackbone=LLaMA-3.1-8B2026.03 | 65.8 | 63.7 | 58.8 | 62.77 |