Mathematical Reasoning on In-Distribution AIME24 MATH-500 OlympiadBench Minerva
14.6AIME24 ScoreHINT
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| HINTBackbone=Qwen2.5-7B2025.10 | 14.6 | 80.4 | 42.2 | 34.4 | 42.9 | |
| GRPO + Meta-HintsBackbone=Qwen2.5-7B2025.10 | 14.4 | 79.6 | 40.2 | 34 | 42.1 | |
| BREADBackbone=Qwen2.5-7B2025.10 | 14 | 77.4 | 38 | 31 | 39.9 | |
| GRPOBackbone=Qwen2.5-7B2025.10 | 13.9 | 76.8 | 38 | 31 | 39.9 | |
| LUFFYBackbone=Qwen2.5-7B2025.10 | 13.4 | 77 | 38.6 | 34.2 | 40.8 | |
| SFTBackbone=Qwen2.5-7B2025.10 | 11.2 | 72.8 | 36.2 | 28.8 | 37.3 | |
| VanillaBackbone=Qwen2.5-7B2025.10 | 9.8 | 50.2 | 34 | 19.5 | 28.4 | |
| HINTBackbone=Qwen2.5-3B2025.10 | 7.4 | 68.8 | 32.8 | 26 | 33.8 | |
| GRPO + Meta-HintsBackbone=Qwen2.5-3B2025.10 | 6.8 | 66.4 | 30.4 | 25.2 | 32.2 | |
| BREADBackbone=Qwen2.5-3B2025.10 | 6.3 | 62 | 29 | 24.4 | 30.4 | |
| GRPOBackbone=Qwen2.5-3B2025.10 | 6 | 60.4 | 26 | 23.6 | 29 | |
| LUFFYBackbone=Qwen2.5-3B2025.10 | 5.8 | 62.2 | 29.6 | 22.2 | 30 | |
| SFTBackbone=Qwen2.5-3B2025.10 | 5.3 | 54.8 | 20.6 | 19.6 | 25.1 | |
| VanillaBackbone=Qwen2.5-3B2025.10 | 2.9 | 39.8 | 12 | 9.8 | 16.1 | |
| HINTBackbone=LLaMa3.1-8B2025.10 | 1 | 28 | 7 | 15.2 | 12.8 | |
| BREADBackbone=LLaMa3.1-8B2025.10 | 0.7 | 23 | 7 | 16.6 | 11.8 | |
| LUFFYBackbone=LLaMa3.1-8B2025.10 | 0.5 | 25.2 | 7.4 | 14.4 | 11.9 | |
| GRPOBackbone=LLaMa3.1-8B2025.10 | 0.5 | 23.2 | 6.3 | 12.2 | 10.6 | |
| GRPO + Meta-HintsBackbone=LLaMa3.1-8B2025.10 | 0.5 | 26.8 | 6.6 | 14.4 | 12.1 | |
| SFTBackbone=LLaMa3.1-8B2025.10 | 0.2 | 14.4 | 4.4 | 8.4 | 6.9 | |
| VanillaBackbone=LLaMa3.1-8B2025.10 | 0 | 9.4 | 2.1 | 3.2 | 3.7 |