Mathematical Problem Solving on AIME 2024 (Top-1 Accuracy w/u/iw)
76.67Top-1 AccuracySelf-MoA
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Self-MoACategory=Other Methods2025.07 | 76.67 | — | — | — | |
| GPT-o3-miniAccess=Close-source2025.07 | 73.33 | — | — | — | |
| SMCSCategory=Ours2025.07 | 73.33 | — | — | — | |
| GLM-Z1-32B-0414Access=Open-source, Parameters=32B2025.07 | 66.7 | — | — | — | |
| Open-source Upper BoundCategory=Baselines2025.07 | 66.7 | — | — | — | |
| DeepSeek-R1-Distill-Llama-70BAccess=Open-source, Parameters=70B2025.07 | 60 | — | — | — | |
| Self Consistency (Best on Validation)Category=Other Methods2025.07 | 60 | — | — | — | |
| DeepSeek-R1-Distill-Qwen-32BAccess=Open-source, Parameters=32B2025.07 | 56.7 | — | — | — | |
| Majority VotingCategory=Other Methods2025.07 | 56.67 | — | — | — | |
| MoACategory=Other Methods2025.07 | 53.33 | — | — | — | |
| Qwen3-32BAccess=Open-source, Parameters=32B2025.07 | 53.3 | — | — | — | |
| GPT-4.1Access=Close-source2025.07 | 50 | — | — | — | |
| Symbolic-MoE*Category=Other Methods2025.07 | 50 | — | — | — | |
| QwQ-32BAccess=Open-source, Parameters=32B2025.07 | 46.7 | — | — | — | |
| Simple RouterCategory=Other Methods2025.07 | 46.7 | — | — | — | |
| EXAONE-Deep-32BAccess=Open-source, Parameters=32B2025.07 | 33.3 | — | — | — | |
| Gemma-3-27b-itAccess=Open-source, Parameters=27B2025.07 | 30 | — | — | — | |
| Llama-3.3-70B-InstructAccess=Open-source, Parameters=70B2025.07 | 30 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-1.7B, Budget weighting=proportional weighting (w)2025.10 | 29.13 | — | — | — | |
| ePFBackbone=Qwen3-1.7B, Budget weighting=proportional weighting (w)2025.10 | 28.83 | — | — | — | |
| Claude-3.7-SonnetAccess=Close-source2025.07 | 26.7 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-1.7B, Budget weighting=uniform (u)2025.10 | 25.6 | — | — | — | |
| ePFBackbone=Qwen3-1.7B, Budget weighting=uniform (u)2025.10 | 25 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-1.7B, Budget weighting=inverse weighting (iw)2025.10 | 23.13 | — | — | — | |
| ePFBackbone=Qwen3-1.7B, Budget weighting=inverse weighting (iw)2025.10 | 21.96 | — | — | — | |
| Best-of-NBackbone=Qwen3-1.7B, Budget weighting=proportional weighting (w)2025.10 | 20.51 | — | — | — | |
| PFBackbone=Qwen3-1.7B, Budget weighting=proportional weighting (w)2025.10 | 20.45 | — | — | — | |
| PFBackbone=Qwen3-1.7B, Budget weighting=inverse weighting (iw)2025.10 | 20.45 | — | — | — | |
| PFBackbone=Qwen3-1.7B, Budget weighting=uniform (u)2025.10 | 20.4 | — | — | — | |
| Qwen2.5-32b-InstructAccess=Open-source, Parameters=32B2025.07 | 20 | — | — | — | |
| Best-of-NBackbone=Qwen3-1.7B, Budget weighting=uniform (u)2025.10 | 18.8 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-0.6B, Budget weighting=proportional weighting (w)2025.10 | 17.66 | — | — | — | |
| Best-of-NBackbone=Qwen3-1.7B, Budget weighting=inverse weighting (iw)2025.10 | 17.12 | — | — | — | |
| Claude-3.5-SonnetAccess=Close-source2025.07 | 16.7 | — | — | — | |
| Qwen-2.5-72B-InstructAccess=Open-source, Parameters=72B2025.07 | 16.7 | — | — | — | |
| Qwen2.5-Coder-32B-InstructAccess=Open-source, Parameters=32B2025.07 | 16.7 | — | — | — | |
| Llama-3.3-Nemotron-Super-49B-v1Access=Open-source, Parameters=49B2025.07 | 16.7 | — | — | — | |
| HuatuoGPT-o1-72BAccess=Open-source, Parameters=72B2025.07 | 16.7 | — | — | — | |
| ePFBackbone=Qwen3-0.6B, Budget weighting=proportional weighting (w)2025.10 | 14.59 | — | — | — | |
| PFBackbone=Qwen3-0.6B, Budget weighting=proportional weighting (w)2025.10 | 14.46 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-0.6B, Budget weighting=uniform (u)2025.10 | 13.5 | — | — | — | |
| Base SamplingBackbone=Qwen3-1.7B, Budget weighting=uniform (u)2025.10 | 13.4 | — | — | — | |
| PFBackbone=Qwen3-0.6B, Budget weighting=uniform (u)2025.10 | 12.2 | — | — | — | |
| ePFBackbone=Qwen3-0.6B, Budget weighting=uniform (u)2025.10 | 11.4 | — | — | — | |
| GPT-4oAccess=Close-source2025.07 | 10 | — | — | — | |
| TeleChat2-35B-32KAccess=Open-source, Parameters=35B2025.07 | 10 | — | — | — | |
| ePF w/ LaMBackbone=Qwen3-0.6B, Budget weighting=inverse weighting (iw)2025.10 | 9.67 | — | — | — | |
| PFBackbone=Qwen3-0.6B, Budget weighting=inverse weighting (iw)2025.10 | 9.38 | — | — | — | |
| ePFBackbone=Qwen3-0.6B, Budget weighting=inverse weighting (iw)2025.10 | 8.79 | — | — | — | |
| Best-of-NBackbone=Qwen3-0.6B, Budget weighting=proportional weighting (w)2025.10 | 7.9 | — | — | — | |
| Best-of-NBackbone=Qwen3-0.6B, Budget weighting=uniform (u)2025.10 | 6 | — | — | — | |
| Best-of-NBackbone=Qwen3-0.6B, Budget weighting=inverse weighting (iw)2025.10 | 4.32 | — | — | — | |
| Base SamplingBackbone=Qwen3-0.6B, Budget weighting=uniform (u)2025.10 | 3.4 | — | — | — | |
| InternLM2.5-20B-ChatAccess=Open-source, Parameters=20B2025.07 | 3.3 | — | — | — | |
| Base SamplingBackbone=Qwen2.5-1.5B-In2025.10 | — | — | 3.33 | — | |
| Base SamplingBackbone=Qwen2.5-7B-In2025.10 | — | — | 10 | — | |
| Beam-SearchBackbone=Qwen2.5-1.5B-In2025.10 | — | 10.29 | 8.4 | 5.55 | |
| Beam-SearchBackbone=Qwen2.5-7B-In2025.10 | — | 17.93 | 14 | 11.26 | |
| Best-of-NBackbone=Qwen2.5-1.5B-In2025.10 | — | 9.9 | 7.2 | 4 | |
| Best-of-NBackbone=Qwen2.5-7B-In2025.10 | — | 23.09 | 20.2 | 17.48 | |
| ePFBackbone=Qwen2.5-1.5B-In2025.10 | — | 17.06 | 11.2 | 5.55 | |
| ePFBackbone=Qwen2.5-7B-In2025.10 | — | 26.23 | 21 | 16.06 | |
| PFBackbone=Qwen2.5-1.5B-In2025.10 | — | 11.16 | 9 | 6.99 | |
| PFBackbone=Qwen2.5-7B-In2025.10 | — | 26.06 | 21.6 | 18.13 |