Mathematical Reasoning on MMLU STEM
77.8AccuracyFP16
Evaluation results
| Method | Links | ||
|---|---|---|---|
| FP16Model=7B2025.08 | 77.8 | — | |
| GPT-42024.05 | 77.1 | — | |
| MAmmoTH2Parameters=8x7B, Variant=Plus2024.05 | 71.8 | — | |
| Qwen-1.5Parameters=110B2024.05 | 71.5 | — | |
| Qwen-1.5Parameters=72B2024.05 | 68.8 | — | |
| FP8Model=7B2025.08 | 68.7 | — | |
| JiuZhang3.0Parameters=8x7B2024.05 | 66.9 | — | |
| MicroMixModel=7B2025.08 | 66.5 | — | |
| MAmmoTH2Parameters=8B, Variant=Plus2024.05 | 65.7 | — | |
| MAmmoTH2Parameters=7B, Variant=Plus2024.05 | 64 | — | |
| ChatGPT2024.05 | 63.5 | — | |
| MixtralParameters=8x7B2024.05 | 62.3 | — | |
| Intern-MathParameters=20B2024.05 | 62.3 | — | |
| JiuZhang3.0Parameters=8B2024.05 | 60.4 | — | |
| DeepSeekMathParameters=7B, Variant=Instruct2024.05 | 57.9 | — | |
| GemmaParameters=7B2024.05 | 57.7 | — | |
| DeepSeekMathParameters=7B2024.05 | 56.1 | — | |
| Rho-1-MathParameters=7B2024.05 | 54.9 | — | |
| LLAMA-3Parameters=8B2024.05 | 54.4 | — | |
| LlemmaParameters=34B2024.05 | 54.3 | — | |
| JiuZhang3.0Parameters=7B2024.05 | 53.6 | — | |
| MistralParameters=7B2024.05 | 49.5 | — | |
| LlemmaParameters=7B2024.05 | 45.8 | — | |
| BaselineBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=5-shot2025.10 | — | 49.8 | |
| BaselineBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 57.4 | |
| BaselineBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=5-shot2025.10 | — | 22.3 | |
| BaselineBase Model=Qwen2.5-Math-7B, Evaluation Protocol=5-shot2025.10 | — | 69.8 | |
| BaselineBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 69.3 | |
| LightRBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=5-shot2025.10 | — | 54.9 | |
| LightRBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 55.2 | |
| LightRBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=5-shot2025.10 | — | 26.2 | |
| LightRBase Model=Qwen2.5-Math-7B, Evaluation Protocol=5-shot2025.10 | — | 70.5 | |
| LightRBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 67.8 | |
| SFTBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=5-shot2025.10 | — | 47.7 | |
| SFTBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 56.2 | |
| SFTBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=5-shot2025.10 | — | 26.2 | |
| SFTBase Model=Qwen2.5-Math-7B, Evaluation Protocol=5-shot2025.10 | — | 68.5 | |
| SFTBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=5-shot2025.10 | — | 68.2 |