Mathematical Reasoning on MATH in-domain (test)
80.8AccuracyGEAR
Evaluation Results
| Method | Links | |
|---|---|---|
| GEARBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 80.8 | |
| MT-GRPOBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 76.8 | |
| ARPOBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 76.3 | |
| GRPOBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 75.5 | |
| OPSDBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 74.1 | |
| GEARBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 73.8 | |
| GEARBackbone=Qwen3-4B, Evaluation Context=Multi-Domain Generalization2026.05 | 72.4 | |
| OPSD+RLBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 70.1 | |
| OPSDBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 69.8 | |
| GPT-4 Code2023.10 | 69.7 | |
| ARPOBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 69.1 | |
| MT-GRPOBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 68 | |
| GRPOBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 67.6 | |
| OPSD+RLBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 66.8 | |
| GRPOBackbone=Qwen3-4B, Evaluation Context=Multi-Domain Generalization2026.05 | 64.3 | |
| BaseBackbone=Qwen3-8B, Evaluation Context=Standard2026.05 | 57.2 | |
| BaseBackbone=Qwen3-4B, Evaluation Context=Standard2026.05 | 56.5 | |
| MathCoder-CLBase=CodeLlama, Size=34B2023.10 | 45.2 | |
| MathCoder-LBase=Llama-2, Size=70B2023.10 | 45.1 | |
| GPT-42023.10 | 42.5 | |
| MathCoder-CLBase=CodeLlama, Size=13B2023.10 | 35.9 | |
| PaLM-22023.10 | 34.3 | |
| ChatGPT-3.52023.10 | 34.1 | |
| MathCoder-CLBase=CodeLlama, Size=7B2023.10 | 30.2 | |
| MathCoder-LBase=Llama-2, Size=13B2023.10 | 29.9 | |
| MathCoder-LBase=Llama-2, Size=7B2023.10 | 23.3 | |
| WizardMathBase=Llama-2, Size=70B2023.10 | 22.7 | |
| WizardMathBase=Llama-2, Size=13B2023.10 | 14 | |
| GalacticaSize=30B2023.10 | 12.7 | |
| WizardMathBase=Llama-2, Size=7B2023.10 | 10.7 | |
| Llama-1 RFTBase=Llama-1, Size=34B2023.10 | 7.4 | |
| Llama-1 RFTBase=Llama-1, Size=7B2023.10 | 5.2 | |
| Llama-1 RFTBase=Llama-1, Size=13B2023.10 | 5.1 | |
| GalacticaSize=6.7B2023.10 | 2.2 |