Mathematical Reasoning on SVAMP out-of-domain (test)
97AccuracyGPT-4
Evaluation Results
| Method | Links | |
|---|---|---|
| GPT-42023.10 | 97 | |
| CausalRMEvaluation Type=Downstream RLHF2026.01 | 93.9 | |
| GoalRMEvaluation Type=Downstream RLHF2026.01 | 92.9 | |
| Standard RMEvaluation Type=Downstream RLHF2026.01 | 92.1 | |
| SFTEvaluation Type=Downstream RLHF2026.01 | 88.6 | |
| MathCoder-LBase=Llama-2, Size=70B2023.10 | 84.9 | |
| MathCoder-CLBase=CodeLlama, Size=34B2023.10 | 82.5 | |
| MathCoder-CLBase=CodeLlama, Size=13B2023.10 | 78 | |
| MathCoder-LBase=Llama-2, Size=13B2023.10 | 76.9 | |
| SIKeD (All)Model=Gemma 7B2024.10 | 76.4 | |
| SIKeD (Adaptive)Model=Qwen 1.5B2024.10 | 75.4 | |
| PoTModel=Gemma 7B2024.10 | 75 | |
| PoTModel=Qwen 1.5B2024.10 | 74.3 | |
| SIKeD (All)Model=Qwen 1.5B2024.10 | 73.5 | |
| CombinedModel=Qwen 1.5B2024.10 | 73.2 | |
| SIKeD (Adaptive)Model=Gemma 7B2024.10 | 72.9 | |
| WizardMathBase=Llama-2, Size=70B2023.10 | 71.8 | |
| MathCoder-LBase=Llama-2, Size=7B2023.10 | 71.5 | |
| MathCoder-CLBase=CodeLlama, Size=7B2023.10 | 70.7 | |
| CombinedModel=Gemma 7B2024.10 | 69.4 | |
| CoTModel=Gemma 7B2024.10 | 66.8 | |
| CoTModel=Qwen 1.5B2024.10 | 66.3 | |
| L2MModel=Gemma 7B2024.10 | 64.8 | |
| L2MModel=Qwen 1.5B2024.10 | 62.3 | |
| SIKeD (Adaptive)Model=Gemma 2B2024.10 | 58.6 | |
| PoTModel=Gemma 2B2024.10 | 56.8 | |
| SIKeD (All)Model=Gemma 2B2024.10 | 56.7 | |
| CombinedModel=Gemma 2B2024.10 | 56.2 | |
| Llama-1 RFTBase=Llama-1, Size=34B2023.10 | 55.4 | |
| InfoRMEvaluation Type=Downstream RLHF2026.01 | 53.9 | |
| SIKeD (All)Model=Qwen 0.5B2024.10 | 52.1 | |
| WizardMathBase=Llama-2, Size=13B2023.10 | 51.9 | |
| SIKeD (Adaptive)Model=Qwen 0.5B2024.10 | 51.7 | |
| PoTModel=Qwen 0.5B2024.10 | 51.4 | |
| CombinedModel=Qwen 0.5B2024.10 | 51.4 | |
| Llama-1 RFTBase=Llama-1, Size=13B2023.10 | 46.5 | |
| L2MModel=Qwen 0.5B2024.10 | 44.6 | |
| CoTModel=Qwen 0.5B2024.10 | 43 | |
| CoTModel=Gemma 2B2024.10 | 41.9 | |
| GalacticaSize=30B2023.10 | 41.6 | |
| SIKeD (All)Model=SmolLM 1.7B2024.10 | 39.5 | |
| SIKeD (Adaptive)Model=SmolLM 1.7B2024.10 | 37.8 | |
| WizardMathBase=Llama-2, Size=7B2023.10 | 36.1 | |
| CombinedModel=SmolLM 1.7B2024.10 | 35.9 | |
| PoTModel=SmolLM 1.7B2024.10 | 34.5 | |
| L2MModel=Gemma 2B2024.10 | 31.6 | |
| GalacticaSize=6.7B2023.10 | 25.6 | |
| CoTModel=SmolLM 1.7B2024.10 | 22.6 | |
| Llama-1 RFTBase=Llama-1, Size=7B2023.10 | 21.1 | |
| L2MModel=SmolLM 1.7B2024.10 | 17.9 |