Reasoning on Overall Combined Benchmarks
88.7AccuracyVanilla
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| VanillaBackbone=Qwen3-14B2026.04 | 88.7 | 5,062 | — | |
| DTSRBackbone=Qwen3-14B2026.04 | 88.7 | 3,246 | — | |
| TALEBackbone=Qwen3-14B2026.04 | 87.7 | 4,872 | — | |
| PUMAModel=Qwen3-30B-A3B-Thinking2026.05 | 82.5 | — | 28.2 | |
| Full CoTModel=Qwen3-30B-A3B-Thinking2026.05 | 81.7 | — | 0 | |
| DEERModel=Qwen3-30B-A3B-Thinking2026.05 | 81.3 | — | 22.4 | |
| DynasorModel=Qwen3-30B-A3B-Thinking2026.05 | 79.7 | — | 3 | |
| CoDBackbone=Qwen3-14B2026.04 | 71.1 | 2,077 | — | |
| No-ThinkModel=Qwen3-30B-A3B-Thinking2026.05 | 68.8 | — | 45.3 | |
| PUMAModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 64.7 | — | 20.1 | |
| Full CoTModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 64.3 | — | 0 | |
| DEERModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 61.2 | — | -30.7 | |
| PUMAModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 60.2 | — | 35.6 | |
| Plan&BudgetModel=Qwen3-30B-A3B-Thinking2026.05 | 59.6 | — | 53.9 | |
| DynasorModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 58.3 | — | -4.7 | |
| Full CoTModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 58 | — | 0 | |
| DEERModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 57.9 | — | 21.5 | |
| CCoTModel=Qwen3-30B-A3B-Thinking2026.05 | 54.3 | — | 55.5 | |
| Plan&BudgetModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 52.1 | — | 18.6 | |
| CoDModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 51.7 | — | 31.7 | |
| CCoTModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 50.4 | — | 58.6 | |
| CoDModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 49.4 | — | 56.6 | |
| CCoTModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 48.8 | — | 34.1 | |
| Plan&BudgetModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 47.8 | — | 47.9 | |
| DynasorModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 45.9 | — | 36.6 | |
| CoDModel=Qwen3-30B-A3B-Thinking2026.05 | 45.3 | — | 40.4 | |
| No-ThinkModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 38.7 | — | 79.9 | |
| Ans. Conv.Model=DeepSeek-R1-Distill-Qwen-7B2026.05 | 35.1 | — | 81.9 | |
| No-ThinkModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 32.8 | — | -80.5 | |
| Ans. Conv.Model=Qwen3-30B-A3B-Thinking2026.05 | 23.2 | — | 90.9 | |
| Ans. Conv.Model=Llama-3.1-Nemotron-Nano-8B2026.05 | 21.9 | — | 83.7 |