Mathematical Reasoning on OlymBench
75AccuracyFull CoT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Full CoTModel=Qwen3-30B-A3B-Thinking2026.05 | 75 | — | 0 | — | — | |
| DEERModel=Qwen3-30B-A3B-Thinking2026.05 | 73.5 | — | 23.4 | — | — | |
| DynasorModel=Qwen3-30B-A3B-Thinking2026.05 | 72.4 | — | -0.6 | — | — | |
| PUMAModel=Qwen3-30B-A3B-Thinking2026.05 | 72.3 | — | 31.7 | — | — | |
| No-ThinkModel=Qwen3-30B-A3B-Thinking2026.05 | 68.4 | — | 56.6 | — | — | |
| CEEH-MEExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 66.9 | — | — | 4,383 | -1.37 | |
| CEEH-EAExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 66.3 | — | — | 5,253 | -0.66 | |
| R1-Distill-Qwen2.5-7BReproduced=true2026.02 | 65.5 | — | — | 7,413 | — | |
| Length-PenaltyExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B, Reproduced=true2026.02 | 64.8 | — | — | 6,411 | 0.39 | |
| Full CoTModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 63.1 | — | 0 | — | — | |
| PUMAModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 62.2 | — | 27 | — | — | |
| DynasorModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 61.9 | — | -3 | — | — | |
| Plan&BudgetModel=Qwen3-30B-A3B-Thinking2026.05 | 61.3 | — | 60.2 | — | — | |
| DEERModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 61.2 | — | 12.4 | — | — | |
| LC-R1Execution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 58.7 | — | — | 4,041 | 7 | |
| Full CoTModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 57.6 | — | 0 | — | — | |
| DEERModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 57.5 | — | 20.2 | — | — | |
| ThinkSwitcherExecution Strategy=Offline, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 57 | — | — | 5,147 | 7.17 | |
| CCoTModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 56.9 | — | 34.6 | — | — | |
| AutoThinkExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-7B2026.02 | 56.4 | — | — | 5,498 | 7.06 | |
| PUMAModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 55.6 | — | 43.2 | — | — | |
| Plan&BudgetModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 53.9 | — | 22.7 | — | — | |
| CCoTModel=Qwen3-30B-A3B-Thinking2026.05 | 53 | — | 58.8 | — | — | |
| CoDModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 51.3 | — | 32 | — | — | |
| Plan&BudgetModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 49.3 | — | 50.3 | — | — | |
| CCoTModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 49.2 | — | 57.7 | — | — | |
| DynasorModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 49 | — | 34.1 | — | — | |
| DSBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 46.8 | 9,684 | — | — | — | |
| PEARBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 45.3 | 7,810 | — | — | — | |
| CoDModel=Qwen3-30B-A3B-Thinking2026.05 | 45.3 | — | 50.3 | — | — | |
| GRPO-AccBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 45.2 | 9,162 | — | — | — | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 45 | 11,570 | — | — | — | |
| AutoThinkExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 44.8 | — | — | 5,559 | -4.9 | |
| InfoDensityBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 44.7 | 9,215 | — | — | — | |
| CEEH-EAExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 44.7 | — | — | 4,693 | -5.32 | |
| GRPO-LPBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.03 | 44.3 | 10,218 | — | — | — | |
| CEEH-MEExecution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 44.3 | — | — | 5,469 | -4.21 | |
| CoDModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 43 | — | 55.8 | — | — | |
| No-ThinkModel=DeepSeek-R1-Distill-Qwen-7B2026.05 | 42.8 | — | 79.1 | — | — | |
| LC-R1Execution Strategy=Online, Base Model=R1-Distill-Qwen2.5-1.5B2026.02 | 42.7 | — | — | 3,780 | -2.2 | |
| R1-Distill-Qwen2.5-1.5BReproduced=true2026.02 | 41.5 | — | — | 8,957 | — | |
| Qwen2.5-7B-InsExecution Strategy=Prompting2026.02 | 39.2 | — | — | 827 | 37.85 | |
| Qwen2.5-7B-Math-InsExecution Strategy=Prompting2026.02 | 38.9 | — | — | 1,027 | 37.69 | |
| GRPO-AccBackbone=Qwen3-0.6B2026.03 | 38.7 | 9,409 | — | — | — | |
| PEARBackbone=Qwen3-0.6B2026.03 | 37.3 | 7,846 | — | — | — | |
| OriginalBackbone=Qwen3-0.6B2026.03 | 36.4 | 9,950 | — | — | — | |
| InfoDensityBackbone=Qwen3-0.6B2026.03 | 36.4 | 7,205 | — | — | — | |
| DSBackbone=Qwen3-0.6B2026.03 | 34.4 | 8,085 | — | — | — | |
| GRPO-LPBackbone=Qwen3-0.6B2026.03 | 34.1 | 8,014 | — | — | — | |
| Ans. Conv.Model=DeepSeek-R1-Distill-Qwen-7B2026.05 | 32.6 | — | 82.8 | — | — | |
| No-ThinkModel=Llama-3.1-Nemotron-Nano-8B2026.05 | 31.9 | — | -74 | — | — | |
| Qwen2.5-7B-MathExecution Strategy=Prompting2026.02 | 31.5 | — | — | 1,037 | 48.14 | |
| Ans. Conv.Model=Llama-3.1-Nemotron-Nano-8B2026.05 | 23.6 | — | 85.1 | — | — | |
| Ans. Conv.Model=Qwen3-30B-A3B-Thinking2026.05 | 23.1 | — | 91.7 | — | — |