Mathematical Reasoning on AIME 2024 (Accuracy and Token Counts vs Base)
77.9AccuracyTHINKBRAKE
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| THINKBRAKEBase Model=Qwen3-14B2025.10 | 77.9 | — | — | — | -7.4 | |
| Qwen3-4B-ThinkingStrategy=Baseline2025.10 | 77.1 | — | — | — | — | |
| THINKBRAKEBase Model=Qwen3-32B2025.10 | 77.1 | — | — | — | -6.2 | |
| Qwen3-32BStrategy=Baseline2025.10 | 75.8 | — | — | — | — | |
| THINKBRAKEBase Model=Phi-4-Reasoning2025.10 | 73.3 | — | — | — | -6.4 | |
| Qwen3-14BStrategy=Baseline2025.10 | 71.7 | — | — | — | — | |
| BaselineBackbone=QwQ-32B, Temperature=0.02025.12 | 70 | — | 9,590.47 | — | — | |
| NoWaitBase Model=Qwen3-14B2025.10 | 70 | — | — | — | -19.8 | |
| Qwen3-4BStrategy=Baseline2025.10 | 68.8 | — | — | — | — | |
| Phi-4-ReasoningStrategy=Baseline2025.10 | 67.5 | — | — | — | — | |
| THINKBRAKEBase Model=Qwen3-4B-Thinking2025.10 | 67.2 | — | — | — | -33.5 | |
| NoWaitBase Model=Qwen3-4B-Thinking2025.10 | 66.7 | — | — | — | -30.5 | |
| NoWaitBase Model=Qwen3-32B2025.10 | 66.7 | — | — | — | -12 | |
| DEERBackbone=QwQ-32B, Temperature=0.62025.12 | 66.67 | 11.12 | 8,767.23 | -13.3 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.97, Temperature=0.02025.12 | 66.67 | -4.76 | 8,440.9 | -12 | — | |
| THINKBRAKEBase Model=Qwen3-4B2025.10 | 64.6 | — | — | — | -12.7 | |
| LYNXBackbone=QwQ-32B, Conf=0.95, Temperature=0.02025.12 | 63.33 | -9.53 | 7,725.73 | -19.4 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.90, Temperature=0.02025.12 | 63.33 | -9.53 | 7,308.63 | -23.8 | — | |
| DEERBase Model=Qwen3-4B-Thinking2025.10 | 63.3 | — | — | — | -34.3 | |
| Dynasor-CoTBase Model=Qwen3-4B-Thinking2025.10 | 63.3 | — | — | — | -28.8 | |
| NoWaitBase Model=Phi-4-Reasoning2025.10 | 63.3 | — | — | — | -9.4 | |
| BaselineBackbone=QwQ-32B, Temperature=0.62025.12 | 60 | — | 10,115.93 | — | — | |
| LYNXBackbone=QwQ-32B, Conf=0.97, Temperature=0.62025.12 | 60 | 0 | 7,343.97 | -27.4 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.95, Temperature=0.62025.12 | 60 | 0 | 6,743.93 | -33.3 | — | |
| DEERBackbone=QwQ-32B, Temperature=0.02025.12 | 60 | -14.29 | 9,261.73 | -3.4 | — | |
| NoWaitBase Model=Qwen3-4B2025.10 | 56.7 | — | — | — | -16.6 | |
| ToNBackbone=QwQ-32B, Temperature=0.02025.12 | 53.33 | -23.81 | 9,612.26 | 0.2 | — | |
| ToNBackbone=QwQ-32B, Temperature=0.62025.12 | 50 | -16.67 | 8,111.83 | -19.8 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.9, Temperature=0.62025.12 | 50 | -16.67 | 5,940.8 | -41.3 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.80, Temperature=0.02025.12 | 50 | -28.57 | 6,396.4 | -33.3 | — | |
| DeepSeek-R1-7BStrategy=Baseline2025.10 | 49.5 | — | — | — | — | |
| LYNXBackbone=QwQ-32B, Conf=0.8, Temperature=0.62025.12 | 46.67 | -22.22 | 5,262.7 | -48 | — | |
| LYNXBackbone=QwQ-32B, Conf=0.70, Temperature=0.02025.12 | 46.67 | -33.33 | 5,691.37 | -40.7 | — | |
| THINKBRAKEBase Model=DeepSeek-R1-7B2025.10 | 45.2 | — | — | — | -25 | |
| NoWaitBase Model=DeepSeek-R1-7B2025.10 | 40 | — | — | — | -32.4 | |
| THINKBRAKE-pBase Model=Qwen3-4B-Thinking2025.10 | 35 | — | — | — | -72.6 | |
| LYNXBackbone=QwQ-32B, Conf=0.7, Temperature=0.62025.12 | 33.33 | -44.45 | 4,248.9 | -58 | — | |
| DEERBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.62025.12 | 30 | 79.96 | 8,303.06 | -31.9 | — | |
| BaselineBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.02025.12 | 23.33 | — | 13,427.43 | — | — | |
| ToNBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.02025.12 | 23.33 | 0 | 11,928.16 | -11.2 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.97, Temperature=0.02025.12 | 23.33 | 0 | 13,322.6 | -0.8 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.95, Temperature=0.02025.12 | 23.33 | 0 | 13,004.03 | -3.2 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.80, Temperature=0.02025.12 | 23.33 | 0 | 8,501.83 | -36.7 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.70, Temperature=0.02025.12 | 23.33 | 0 | 5,981.83 | -55.5 | — | |
| ToNBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.62025.12 | 20 | 19.98 | 10,408.63 | -14.6 | — | |
| DEERBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.02025.12 | 20 | -14.27 | 9,154.13 | -31.8 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.90, Temperature=0.02025.12 | 20 | -14.27 | 11,635.8 | -13.3 | — | |
| BaselineBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Temperature=0.62025.12 | 16.67 | — | 12,189 | — | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.97, Temperature=0.62025.12 | 16.67 | 0 | 10,817.13 | -11.3 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.95, Temperature=0.62025.12 | 16.67 | 0 | 9,963 | -18.3 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.9, Temperature=0.62025.12 | 13.33 | -20.04 | 7,814.43 | -35.9 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.8, Temperature=0.62025.12 | 13.33 | -20.04 | 5,503.27 | -54.9 | — | |
| LYNXBackbone=DeepSeek-R1-Distill-Qwen-1.5B, Conf=0.7, Temperature=0.62025.12 | 10 | -40.01 | 3,900.03 | -68 | — | |
| ThinkLessBase Model=Qwen3-4B-Thinking2025.10 | 3.3 | — | — | — | -100 |