Mathematical Problem Solving on AIME 24
15.58Average TimeCoT
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| CoTModel=Qwen3-1.7B2025.07 | 15.58 | — | — | — | — | 9.91 | — | |
| CoTModel=Qwen3-4B2025.07 | 22.6 | — | — | — | — | 15.63 | — | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 23.24 | — | — | — | — | 15.46 | — | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 27.19 | — | — | — | — | 16.94 | 1.48 | |
| Guided SearchModel=Qwen3-1.7B, Variant=MUR2025.07 | 27.25 | — | — | — | — | 16.8 | -35.81 | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=MUR2025.07 | 30.16 | — | — | — | — | 17.75 | 2.29 | |
| Guided SearchModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 31.67 | — | — | — | — | 22.41 | -30.2 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=MUR2025.07 | 35.21 | — | — | — | — | 21.81 | -21.57 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=Avg uncertainty2025.07 | 36.33 | — | — | — | — | 23.55 | -19.83 | |
| LLM As a CriticModel=Qwen3-1.7B, Variant=SMART2025.07 | 48.2 | — | — | — | — | 26.66 | 11.2 | |
| LLM As a CriticModel=Qwen3-4B, Variant=SMART2025.07 | 49.47 | — | — | — | — | 37.26 | -21.96 | |
| Guided SearchModel=Qwen3-4B, Variant=MUR2025.07 | 53.93 | — | — | — | — | 34.11 | -60.24 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=SMART2025.07 | 62.73 | — | — | — | — | 34.45 | -8.93 | |
| Guided SearchModel=Qwen3-1.7B, Variant=SMART2025.07 | 62.77 | — | — | — | — | 37.37 | -15.24 | |
| LLM As a CriticModel=Qwen3-4B, Variant=MUR2025.07 | 63.43 | — | — | — | — | 38.41 | -20.81 | |
| LLM As a CriticModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 64.27 | — | — | — | — | 47.86 | -11.36 | |
| ϕ-DecodingModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 65.28 | — | — | — | — | 43.38 | — | |
| CoTModel=Qwen3-8B2025.07 | 68.28 | — | — | — | — | 45.38 | — | |
| Guided SearchModel=Qwen3-1.7B, Variant=Per-Step Scale2025.07 | 92.47 | — | — | — | — | 52.61 | — | |
| ϕ-DecodingModel=Qwen3-4B, Variant=MUR2025.07 | 94.87 | — | — | — | — | 58.06 | -36.95 | |
| LLM As a CriticModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 104.63 | — | — | — | — | 59.22 | — | |
| ϕ-DecodingModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 109.3 | — | — | — | — | 59.94 | -35.07 | |
| Guided SearchModel=Qwen3-4B, Variant=Avg uncertainty2025.07 | 114.83 | — | — | — | — | 62.78 | -31.57 | |
| LLM As a CriticModel=Qwen3-8B, Variant=MUR2025.07 | 120.03 | — | — | — | — | 72.22 | -6.65 | |
| LLM As a CriticModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 123.18 | — | — | — | — | 78.87 | — | |
| LLM As a CriticModel=Qwen3-8B, Variant=SMART2025.07 | 125.73 | — | — | — | — | 77.09 | -1.78 | |
| ϕ-DecodingModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 128.6 | — | — | — | — | 95.01 | — | |
| LLM As a CriticModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 129.33 | — | — | — | — | 78.8 | -0.07 | |
| Guided SearchModel=Qwen3-4B, Variant=SMART2025.07 | 134.07 | — | — | — | — | 96.38 | 2.03 | |
| Guided SearchModel=Qwen3-4B, Variant=Per-Step Scale2025.07 | 134.4 | — | — | — | — | 94.35 | — | |
| ϕ-DecodingModel=Qwen3-4B, Variant=SMART2025.07 | 149.07 | — | — | — | — | 91.93 | -3.08 | |
| Guided SearchModel=Qwen3-8B, Variant=MUR2025.07 | 173.93 | — | — | — | — | 113.52 | -93.68 | |
| Guided SearchModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 188.9 | — | — | — | — | 114.81 | -92.39 | |
| Guided SearchModel=Qwen3-8B, Variant=SMART2025.07 | 214.95 | — | — | — | — | 137.16 | -70.04 | |
| ϕ-DecodingModel=Qwen3-8B, Variant=MUR2025.07 | 286.79 | — | — | — | — | 160.16 | -79.94 | |
| ϕ-DecodingModel=Qwen3-8B, Variant=Avg uncertainty2025.07 | 288.96 | — | — | — | — | 163.97 | -76.13 | |
| ϕ-DecodingModel=Qwen3-8B, Variant=SMART2025.07 | 351.19 | — | — | — | — | 207.32 | -32.78 | |
| Guided SearchModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 351.58 | — | — | — | — | 207.2 | — | |
| ϕ-DecodingModel=Qwen3-8B, Variant=Per-Step Scale2025.07 | 397.23 | — | — | — | — | 240.1 | — | |
| DVIER2025.09 | — | 76.7 | 80 | 83.3 | 86.7 | — | — | |
| Entropy-RL2025.09 | — | 70 | 73.3 | 76.7 | 76.7 | — | — | |
| GPRO w/ Clip-HigherConditioning=Clip-Higher2025.09 | — | 70 | 73.3 | 76.7 | 76.7 | — | — | |
| Pass@k Training2025.09 | — | 70 | 76.7 | 80 | 80 | — | — | |
| Qwen2.5-Math-7BParameters=7B2025.09 | — | 63.3 | 70 | 76.7 | 80 | — | — |