Mathematical Reasoning on OlympiadBench Math (test)
35.9AccuracyTATA
Evaluation Results
| Method | Links | |
|---|---|---|
| TATAModel=Qwen2.5Math-7B2025.02 | 35.9 | |
| hybridModel=Qwen2.5-14B2025.02 | 35.3 | |
| TATAModel=Qwen2.5-14B2025.02 | 35.3 | |
| hybridModel=Qwen2.5Math-7B2025.02 | 34.4 | |
| ensembleModel=Qwen2.5-14B2025.02 | 32.3 | |
| hybridModel=Qwen2.5-7B2025.02 | 31.7 | |
| TATAModel=Qwen2.5-7B2025.02 | 31.1 | |
| ensembleModel=Qwen2.5Math-7B2025.02 | 30.8 | |
| ensembleModel=Qwen2.5-7B2025.02 | 30.2 | |
| ensembleModel=Qwen2.5Math-1.5B2025.02 | 27.4 | |
| TATAModel=Qwen2.5Math-1.5B2025.02 | 26.8 | |
| hybridModel=Qwen2.5Math-1.5B2025.02 | 26.2 | |
| TATAModel=Qwen2.5-3B2025.02 | 24.9 | |
| GPT-SelectModel=Qwen2.5Math-7B2025.02 | 24.6 | |
| hybridModel=Qwen2.5-3B2025.02 | 24.4 | |
| GPT-SelectModel=Qwen2.5-7B2025.02 | 23.3 | |
| ensembleModel=Qwen2.5-3B2025.02 | 23.1 | |
| GPT-SelectModel=Qwen2.5-14B2025.02 | 23 | |
| DART-Math-DSMath-7B (Prop2Diff)number of samples=0.59M, Base Model=DeepSeekMath-7B, Sampling Strategy=Prop2Diff2024.06 | 21.7 | |
| TATAModel=LLaMA-3-8B2025.02 | 21.5 | |
| DART-Math-DSMath-7B (Uniform)number of samples=0.59M, Base Model=DeepSeekMath-7B, Sampling Strategy=Uniform2024.06 | 21.3 | |
| hybridModel=LLaMA-3-8B2025.02 | 21.3 | |
| GPT-SelectModel=Qwen2.5Math-1.5B2025.02 | 20.6 | |
| DART-Math-Llama3-70B (Prop2Diff)number of samples=0.59M, Base Model=Llama3-70B, Sampling Strategy=Prop2Diff2024.06 | 20 | |
| hybridModel=Qwen2.5-1.5B2025.02 | 19.7 | |
| Llama3-70B-VRTnumber of samples=0.59M, Base Model=Llama3-70B2024.06 | 19.3 | |
| DART-Math-Llama3-70B (Uniform)number of samples=0.59M, Base Model=Llama3-70B, Sampling Strategy=Uniform2024.06 | 19.1 | |
| DeepSeekMath-7B-VRTnumber of samples=0.59M, Base Model=DeepSeekMath-7B2024.06 | 19.1 | |
| ensembleModel=Qwen2.5-1.5B2025.02 | 18.8 | |
| TATAModel=Qwen2.5-1.5B2025.02 | 18.8 | |
| GPT-SelectModel=Qwen2.5-3B2025.02 | 17.3 | |
| Llama2-70B-Xwin-Math-V1.1+number of samples=1.4M, Base Model=Llama2-70B2024.06 | 16.3 | |
| Llama3-70B-MMIQCnumber of samples=2.3M, Base Model=Llama3-70B2024.06 | 15.3 | |
| ensembleModel=LLaMA-3-8B2025.02 | 15.3 | |
| GPT-SelectModel=Qwen2.5-1.5B2025.02 | 14.8 | |
| DART-Math-Mistral-7B (Prop2Diff)number of samples=0.59M, Base Model=Mistral-7B, Sampling Strategy=Prop2Diff2024.06 | 14.7 | |
| DART-Math-Llama3-8B (Prop2Diff)number of samples=0.59M, Base Model=Llama3-8B, Sampling Strategy=Prop2Diff2024.06 | 14.5 | |
| DeepSeekMath-7B-Instructnumber of samples=0.78M, Base Model=DeepSeekMath-7B2024.06 | 14.2 | |
| GPT-SelectModel=LLaMA-3-8B2025.02 | 13.9 | |
| DART-Math-Llama3-8B (Uniform)number of samples=0.59M, Base Model=Llama3-8B, Sampling Strategy=Uniform2024.06 | 13.6 | |
| DART-Math-Mistral-7B (Uniform)number of samples=0.59M, Base Model=Mistral-7B, Sampling Strategy=Uniform2024.06 | 13.2 | |
| DeepSeekMath-7B-MMIQCnumber of samples=2.3M, Base Model=DeepSeekMath-7B2024.06 | 13 | |
| Llama3-70B-MetaMathnumber of samples=0.40M, Base Model=Llama3-70B2024.06 | 11.6 | |
| Llama3-70B-ICLBase Model=Llama3-70B2024.06 | 10.8 | |
| Llama2-7B-Xwin-Math-V1.1+number of samples=1.4M, Base Model=Llama2-7B2024.06 | 10.5 | |
| Llama3-8B-MMIQCnumber of samples=2.3M, Base Model=Llama3-8B2024.06 | 9.6 | |
| Mistral-7B-MMIQCnumber of samples=2.3M, Base Model=Mistral-7B2024.06 | 9.4 | |
| DeepSeekMath-7B-ICLBase Model=DeepSeekMath-7B2024.06 | 9.3 | |
| Llama3-8B-VRTnumber of samples=0.59M, Base Model=Llama3-8B2024.06 | 9.3 | |
| Mistral-7B-VRTnumber of samples=0.59M, Base Model=Mistral-7B2024.06 | 8.7 | |
| TATAModel=Qwen2.5-0.5B2025.02 | 8.6 | |
| hybridModel=Qwen2.5-0.5B2025.02 | 7.9 | |
| Mistral-7B-WizardMath-V1.1 (RL)Base Model=Mistral-7B2024.06 | 7.7 | |
| ensembleModel=Qwen2.5-0.5B2025.02 | 7.7 | |
| GPT-SelectModel=Qwen2.5-0.5B2025.02 | 7.1 | |
| Mistral-7B-MetaMathnumber of samples=0.40M, Base Model=Mistral-7B2024.06 | 5.9 | |
| Llama3-8B-MetaMathnumber of samples=0.40M, Base Model=Llama3-8B2024.06 | 5.5 | |
| Llama3-8B-ICLBase Model=Llama3-8B2024.06 | 4.2 | |
| Mistral-7B-ICLBase Model=Mistral-7B2024.06 | 3.7 |