Long-context Reasoning on Long-context Benchmarks 16K (DocMath, Frames, LB-MQA, V2) (test)
64.1DocMathQwen3-30B-A3B-Thinking + SPELL
Evaluation Results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Qwen3-30B-A3B-Thinking + SPELLModel Category=Reasoning Models, Input Length=16K2025.09 | 64.1 | 66.5 | 78 | 42.3 | 62.7 | |
| Qwen3-30B-A3B-Instruct + SPELLModel Category=Instruct Models, Input Length=16K2025.09 | 63 | 63.1 | 75.1 | 41.5 | 60.7 | |
| Qwen3-30B-A3B-ThinkingModel Category=Reasoning Models, Input Length=16K2025.09 | 62.9 | 64.5 | 75.7 | 39.7 | 60.7 | |
| Qwen3-30B-A3B-Thinking + RLVRModel Category=Reasoning Models, Input Length=16K2025.09 | 62.7 | 64.7 | 77 | 38.5 | 60.7 | |
| Qwen3-30B-A3B-Instruct + RLVRModel Category=Instruct Models, Input Length=16K2025.09 | 62.5 | 59.9 | 71.8 | 39.8 | 58.5 | |
| Qwen2.5-32B-Instruct + SPELLModel Category=Instruct Models, Input Length=16K2025.09 | 62.3 | 61.2 | 74.4 | 40.1 | 59.5 | |
| Qwen3-30B-A3B-InstructModel Category=Instruct Models, Input Length=16K2025.09 | 62.3 | 55.3 | 70.5 | 36.9 | 56.3 | |
| Qwen3-4B-Thinking + SPELLModel Category=Reasoning Models, Input Length=16K2025.09 | 61.9 | 56.6 | 71.6 | 36.8 | 56.7 | |
| Qwen2.5-32B + SPELLModel Category=Base Models, Input Length=16K2025.09 | 61.8 | 50.2 | 62.1 | 34.2 | 52.1 | |
| R1-Distill-Qwen-14B + SPELLModel Category=Reasoning Models, Input Length=16K2025.09 | 61.6 | 62.3 | 76.2 | 39 | 59.8 | |
| Qwen3-4B-Thinking + RLVRModel Category=Reasoning Models, Input Length=16K2025.09 | 60.5 | 56.6 | 71.1 | 33.8 | 55.5 | |
| Qwen2.5-32B-InstructModel Category=Instruct Models, Input Length=16K2025.09 | 60 | 49.9 | 61.4 | 36 | 51.8 | |
| Qwen2.5-32B-Instruct + RLVRModel Category=Instruct Models, Input Length=16K2025.09 | 59.9 | 60.5 | 70.4 | 36.3 | 56.8 | |
| Qwen2.5-14B-Instruct + SPELLModel Category=Instruct Models, Input Length=16K2025.09 | 59.6 | 62.1 | 72.8 | 36.8 | 57.8 | |
| R1-Distill-Qwen-14B + RLVRModel Category=Reasoning Models, Input Length=16K2025.09 | 59.6 | 61.7 | 74.6 | 37.2 | 58.3 | |
| Qwen3-4B-ThinkingModel Category=Reasoning Models, Input Length=16K2025.09 | 58.6 | 56.7 | 69.9 | 32.9 | 54.5 | |
| Qwen2.5-32B + RLVRModel Category=Base Models, Input Length=16K2025.09 | 58.3 | 50 | 59.5 | 32.8 | 50.2 | |
| R1-Distill-Qwen-14BModel Category=Reasoning Models, Input Length=16K2025.09 | 57.7 | 59.2 | 72.4 | 36.2 | 56.4 | |
| Qwen2.5-14B + SPELLModel Category=Base Models, Input Length=16K2025.09 | 57.6 | 52.6 | 63 | 33.5 | 51.7 | |
| Qwen2.5-14B-InstructModel Category=Instruct Models, Input Length=16K2025.09 | 56.3 | 51.6 | 63 | 32.2 | 50.8 | |
| Qwen2.5-14B-Instruct + RLVRModel Category=Instruct Models, Input Length=16K2025.09 | 56.1 | 59.6 | 71 | 36.4 | 55.8 | |
| Qwen2.5-14B + RLVRModel Category=Base Models, Input Length=16K2025.09 | 52.2 | 51 | 63.3 | 32.9 | 49.9 | |
| R1-Distill-Llama-8B + SPELLModel Category=Reasoning Models, Input Length=16K2025.09 | 48.9 | 53.4 | 68.4 | 30.2 | 50.2 | |
| Qwen2.5-32BModel Category=Base Models, Input Length=16K2025.09 | 46.8 | 42.6 | 49 | 33.7 | 43 | |
| Qwen2.5-7B-Instruct + SPELLModel Category=Instruct Models, Input Length=16K2025.09 | 45.8 | 46.7 | 63.1 | 33.2 | 47.2 | |
| Qwen2.5-7B-Instruct + RLVRModel Category=Instruct Models, Input Length=16K2025.09 | 45 | 48.7 | 59.6 | 30.1 | 45.9 | |
| R1-Distill-Llama-8B + RLVRModel Category=Reasoning Models, Input Length=16K2025.09 | 43.4 | 51.4 | 67.8 | 30 | 48.2 | |
| R1-Distill-Llama-8BModel Category=Reasoning Models, Input Length=16K2025.09 | 42 | 50.3 | 66.8 | 27.9 | 46.8 | |
| Qwen2.5-7B + RLVRModel Category=Base Models, Input Length=16K2025.09 | 41.8 | 41 | 50 | 30.2 | 40.8 | |
| Qwen2.5-7B + SPELLModel Category=Base Models, Input Length=16K2025.09 | 40 | 39.2 | 50.9 | 32.3 | 40.6 | |
| Llama3.1-8B-Instruct + SPELLModel Category=Instruct Models, Input Length=16K2025.09 | 39.2 | 48.9 | 61.6 | 28.4 | 44.5 | |
| Qwen2.5-7B-InstructModel Category=Instruct Models, Input Length=16K2025.09 | 38.4 | 40.3 | 45.1 | 29 | 38.2 | |
| Qwen2.5-14BModel Category=Base Models, Input Length=16K2025.09 | 38 | 37.2 | 41.9 | 32.1 | 37.3 | |
| Llama3.1-8B-Instruct + RLVRModel Category=Instruct Models, Input Length=16K2025.09 | 37.9 | 45 | 58.8 | 27.5 | 42.3 | |
| Llama3.1-8B-InstructModel Category=Instruct Models, Input Length=16K2025.09 | 33.2 | 45.6 | 52.5 | 29.1 | 40.1 | |
| Qwen2.5-7BModel Category=Base Models, Input Length=16K2025.09 | 10.9 | 27.9 | 36.7 | 31.2 | 26.7 |