Mathematical Reasoning on AIME 2024 (Pass@1 Accuracy)
83.33Pass@1 AccuracySeLaR
Evaluation Results
| Method | Links | |
|---|---|---|
| SeLaRBackbone=Qwen3-8B2026.04 | 83.33 | |
| SeLaRBackbone=Qwen3-32B2026.04 | 83.33 | |
| SwiRBackbone=Qwen3-32B2026.04 | 82.92 | |
| CoTBackbone=Qwen3-32B, Decoding Strategy=Sampling2026.04 | 80.42 | |
| CoTBackbone=Qwen3-32B, Decoding Strategy=Greedy2026.04 | 80 | |
| Mem2EvolveBackbone=GPT-5-Chat2026.04 | 76.7 | |
| DeepSeek-R1-Distill-Qwen-32BModel=DeepSeek-R1-Distill-Qwen-32B, #Bits=W16A162026.03 | 76.67 | |
| SliderQuantModel=DeepSeek-R1-Distill-Qwen-32B, #Bits=W4A162026.03 | 76.67 | |
| CoTBackbone=Qwen3-8B, Decoding Strategy=Sampling2026.04 | 76.67 | |
| Qwen3-32BStudent Model=-, Teacher Model=Qwen3-32B, Method=Base2026.05 | 75.42 | |
| OmniOPDStudent Model=Qwen3-4B, Teacher Model=Qwen3-30B-A3B-Inst., Method=OmniOPD2026.05 | 75 | |
| Gemini-2.5-flashStudent Model=-, Teacher Model=Gemini-2.5-flash, Method=Base2026.05 | 74.79 | |
| Soft ThinkingBackbone=Qwen3-32B2026.04 | 74.58 | |
| DeepSeek-R1-Distill-Qwen-14BModel=DeepSeek-R1-Distill-Qwen-14B, #Bits=W16A162026.03 | 73.33 | |
| OmniOPDStudent Model=Qwen3-4B, Teacher Model=Gemini-2.5-flash, Method=OmniOPD2026.05 | 73.13 | |
| OmniOPDStudent Model=Qwen3-4B, Teacher Model=Claude-4.5-haiku, Method=OmniOPD2026.05 | 72.08 | |
| SFTStudent Model=Qwen3-4B, Teacher Model=Gemini-2.5-flash, Method=SFT2026.05 | 71.25 | |
| SliderQuantModel=DeepSeek-R1-Distill-Qwen-14B, #Bits=W4A162026.03 | 70 | |
| CoTBackbone=Qwen3-8B, Decoding Strategy=Greedy2026.04 | 70 | |
| Soft ThinkingBackbone=Qwen3-8B2026.04 | 70 | |
| AlitaBackbone=GPT-5-Chat2026.04 | 70 | |
| OmniOPDStudent Model=Qwen3-4B, Teacher Model=Qwen3-32B, Method=OmniOPD2026.05 | 68.67 | |
| Qwen3-4B (GRPO)Student Model=Qwen3-4B, Teacher Model=-, Method=GRPO2026.05 | 68.62 | |
| GPT-5-Chat (CoT)Backbone=GPT-5-Chat2026.04 | 66.67 | |
| GPT-5-Chat (ReAct)Backbone=GPT-5-Chat2026.04 | 66.67 | |
| EvoAgentBackbone=GPT-5-Chat2026.04 | 66.67 | |
| AFLOWBackbone=GPT-5-Chat2026.04 | 66.67 | |
| DSPyBackbone=GPT-5-Chat2026.04 | 66.67 | |
| OPDStudent Model=Qwen3-4B, Teacher Model=Qwen3-32B, Method=OPD2026.05 | 61.45 | |
| Qwen3-30B-A3B-Inst.Student Model=-, Teacher Model=Qwen3-30B-A3B-Inst., Method=Base2026.05 | 61.04 | |
| SFTStudent Model=Qwen3-4B, Teacher Model=Qwen3-32B, Method=SFT2026.05 | 60.83 | |
| SFTStudent Model=Qwen3-4B, Teacher Model=Claude-4.5-haiku, Method=SFT2026.05 | 60.83 | |
| SwiRBackbone=Qwen3-8B2026.04 | 60 | |
| GPT-5-Chat (Direct)Backbone=GPT-5-Chat2026.04 | 60 | |
| AgentVerseBackbone=GPT-5-Chat2026.04 | 60 | |
| OmniQuantModel=DeepSeek-R1-Distill-Qwen-32B, #Bits=W4A162026.03 | 56.66 | |
| Meta-ReasonerBase Model=DeepSeek-R1-Distill-Qwen-7B2025.02 | 55.5 | |
| Claude-4.5-haikuStudent Model=-, Teacher Model=Claude-4.5-haiku, Method=Base2026.05 | 55.2 | |
| SFPOModel=DS-distilled-Qwen-7B2025.10 | 54.17 | |
| HEALModels=Qwen2.5-14B-Instruct2026.03 | 53.63 | |
| SeLaRBackbone=Qwen3-1.7B2026.04 | 53.33 | |
| rStar-MathBase Model=Qwen2.5-Math-7B2025.02 | 53.3 | |
| Qwen3-4BStudent Model=Qwen3-4B, Teacher Model=-, Method=Base2026.05 | 50.83 | |
| GRPOModel=DS-distilled-Qwen-7B2025.10 | 50 | |
| OmniQuantModel=DeepSeek-R1-Distill-Qwen-14B, #Bits=W4A162026.03 | 50 | |
| TTVSBackbone=Qwen3-8B2026.04 | 50 | |
| OPDStudent Model=Qwen3-1.7B, Teacher Model=Qwen3-32B, Method=OPD2026.05 | 48.95 | |
| Qwen3-1.7BStudent Model=Qwen3-1.7B, Teacher Model=-, Method=Base2026.05 | 48.54 | |
| OPDStudent Model=Qwen3-4B, Teacher Model=Qwen3-30B-A3B-Inst., Method=OPD2026.05 | 48.54 | |
| Qwen3-1.7B (GRPO)Student Model=Qwen3-1.7B, Teacher Model=-, Method=GRPO2026.05 | 47.5 | |
| OmniOPDStudent Model=Qwen3-1.7B, Teacher Model=Qwen3-32B, Method=OmniOPD2026.05 | 47.08 | |
| TTRLBackbone=Qwen3-8B2026.04 | 46.7 | |
| CoTBackbone=Qwen3-1.7B, Decoding Strategy=Sampling2026.04 | 46.67 | |
| SwiRBackbone=Qwen3-1.7B2026.04 | 46.67 | |
| DyLANBackbone=GPT-5-Chat2026.04 | 46.67 | |
| SwarmAgenticBackbone=GPT-5-Chat2026.04 | 46.67 | |
| rStar-MathBase Model=Phi3-mini-3.8B2025.02 | 43.3 | |
| MCTS-RAPBase Model=DeepSeek-R1-Distill-Qwen-7B2025.02 | 40.2 | |
| CoTBackbone=Qwen3-1.7B, Decoding Strategy=Greedy2026.04 | 40 | |
| AutoAgentsBackbone=GPT-5-Chat2026.04 | 40 | |
| Full Fine-TuningRank (r)=N/A, n=642026.06 | 37.03 | |
| TTVSBackbone=Qwen3-4B2026.04 | 36.7 | |
| Soft ThinkingBackbone=Qwen3-1.7B2026.04 | 36.67 | |
| NFPOBase Model=Qwen3-8B-Base, Algorithm=NFPO2026.05 | 36.4 | |
| Curriculum SFTModels=Qwen2.5-14B-Instruct2026.03 | 36.27 | |
| SFTStudent Model=Qwen3-1.7B, Teacher Model=Qwen3-32B, Method=SFT2026.05 | 35.42 | |
| SFPOModel=Qwen2.5-Math-7B2025.10 | 35 | |
| DPPOBase Model=Qwen3-8B-Base, Algorithm=DPPO2026.05 | 34.7 | |
| SFTStudent Model=Qwen3-4B, Teacher Model=Qwen3-30B-A3B-Inst., Method=SFT2026.05 | 34.58 | |
| LoRA-αRank (r)=256, n=642026.06 | 34.21 | |
| GRPOModel=Qwen2.5-Math-7B2025.10 | 34.2 | |
| LoRA (10× LR)Rank (r)=256, n=642026.06 | 33.69 | |
| Meta-ReasonerBase Model=GPT-4o2025.02 | 33.3 | |
| SFTModels=Qwen2.5-14B-Instruct2026.03 | 33.23 | |
| SFPOModel=DS-distilled-Qwen-1.5B2025.10 | 32.5 | |
| HEALModels=Qwen3-4B-Base2026.03 | 30.1 | |
| GRPOModel=DS-distilled-Qwen-1.5B2025.10 | 30 | |
| LoRA-αRank (r)=64, n=642026.06 | 28.69 | |
| BaseModel=DS-distilled-Qwen-7B2025.10 | 28.33 | |
| LoRA (10× LR)Rank (r)=64, n=642026.06 | 28.29 | |
| GRPOBase Model=Qwen3-8B-Base, Algorithm=GRPO2026.05 | 27.4 | |
| Qwen3-8BTest-time method=Base2026.04 | 26.9 | |
| Rule-basedBackbone=Qwen3-8B-Base, Reward Method=Rule-based, Training Data Domain=General-domain2026.03 | 26.7 | |
| General-verifierBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 26.7 | |
| SimpleRL-Zero-7BLabeled Data Size=8.9K2026.04 | 26.7 | |
| TTRLBackbone=Qwen3-4B2026.04 | 26.7 | |
| Meta-ReasonerBase Model=GPT-4o-mini2025.02 | 26.7 | |
| LoRARank (r)=256, n=642026.06 | 26.51 | |
| Rule+CERBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 26.5 | |
| Rule-basedBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 26.3 | |
| CERBackbone=Qwen3-8B-Base, Reward Method=CER, Training Data Domain=General-domain2026.03 | 25.8 | |
| Exact-MatchBackbone=Qwen3-8B-Base, Reward Method=Exact-Match, Training Data Domain=General-domain2026.03 | 25.4 | |
| General-verifierBackbone=Qwen3-8B-Base, Reward Method=General-verifier, Training Data Domain=General-domain2026.03 | 25 | |
| DDRLBackbone=Qwen2.5-Math-1.5B2026.04 | 25 | |
| CERBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 24.8 | |
| CERBackbone=Qwen3-8B-Base, Training Dataset=mathematical dataset2026.03 | 23.8 | |
| Rule+CERBackbone=Qwen3-8B-Base, Reward Method=Rule+CER, Training Data Domain=General-domain2026.03 | 23.5 | |
| MCTS-RAPBase Model=GPT-4o2025.02 | 23.4 | |
| Rule+CERBackbone=Qwen3-4B-Base, Training Dataset=mathematical dataset2026.03 | 23.3 | |
| TTVSBackbone=Qwen3-1.7B-Instruct2026.04 | 23.3 |