Mathematical Reasoning on TheoremQA (test)
48.4AccuracyGPT-4-Turbo (24-04-09)
Evaluation results
| Method | Links | |
|---|---|---|
| GPT-4-Turbo (24-04-09)Sampling=N/A2024.06 | 48.4 | |
| DART-Math-DSMath-7B (Uniform)number of samples=0.59M, Base Model=DeepSeekMath-7B, Sampling Strategy=Uniform2024.06 | 32.5 | |
| DART-Math-DSMath-7B (Prop2Diff)number of samples=0.59M, Base Model=DeepSeekMath-7B, Sampling Strategy=Prop2Diff2024.06 | 32.2 | |
| Llama3-70B-VRTnumber of samples=0.59M, Base Model=Llama3-70B2024.06 | 28.6 | |
| DART-Math-Llama3-70B (Prop2Diff)number of samples=0.59M, Base Model=Llama3-70B, Sampling Strategy=Prop2Diff2024.06 | 28.2 | |
| DeepSeekMath-7B-Instructnumber of samples=0.78M, Base Model=DeepSeekMath-7B2024.06 | 28.1 | |
| DART-Math-Llama3-70B (Uniform)number of samples=0.59M, Base Model=Llama3-70B, Sampling Strategy=Uniform2024.06 | 27.4 | |
| DeepSeekMath-7B-VRTnumber of samples=0.59M, Base Model=DeepSeekMath-7B2024.06 | 27.2 | |
| Llama3-70B-ICLBase Model=Llama3-70B2024.06 | 27 | |
| Llama3-70B-MMIQCnumber of samples=2.3M, Base Model=Llama3-70B2024.06 | 23.5 | |
| DeepSeekMath-7B-ICLBase Model=DeepSeekMath-7B2024.06 | 23.5 | |
| DeepSeekMath-7B-MMIQCnumber of samples=2.3M, Base Model=DeepSeekMath-7B2024.06 | 23.4 | |
| Llama3-70B-MetaMathnumber of samples=0.40M, Base Model=Llama3-70B2024.06 | 21.9 | |
| Llama3-8B-ICLBase Model=Llama3-8B2024.06 | 19.8 | |
| DART-Math-Llama3-8B (Prop2Diff)number of samples=0.59M, Base Model=Llama3-8B, Sampling Strategy=Prop2Diff2024.06 | 19.4 | |
| DART-Math-Mistral-7B (Prop2Diff)number of samples=0.59M, Base Model=Mistral-7B, Sampling Strategy=Prop2Diff2024.06 | 17 | |
| Mistral-7B-WizardMath-V1.1 (RL)Base Model=Mistral-7B2024.06 | 16.6 | |
| DART-Math-Mistral-7B (Uniform)number of samples=0.59M, Base Model=Mistral-7B, Sampling Strategy=Uniform2024.06 | 16.4 | |
| Mistral-7B-MMIQCnumber of samples=2.3M, Base Model=Mistral-7B2024.06 | 16.2 | |
| Mistral-7B-VRTnumber of samples=0.59M, Base Model=Mistral-7B2024.06 | 16.2 | |
| Llama3-8B-MMIQCnumber of samples=2.3M, Base Model=Llama3-8B2024.06 | 16.2 | |
| DART-Math-Llama3-8B (Uniform)number of samples=0.59M, Base Model=Llama3-8B, Sampling Strategy=Uniform2024.06 | 15.4 | |
| Llama2-7B-Xwin-Math-V1.1+number of samples=1.4M, Base Model=Llama2-7B2024.06 | 15 | |
| Llama2-70B-Xwin-Math-V1.1+number of samples=1.4M, Base Model=Llama2-70B2024.06 | 14.9 | |
| Llama3-8B-VRTnumber of samples=0.59M, Base Model=Llama3-8B2024.06 | 14.9 | |
| Mistral-7B-ICLBase Model=Mistral-7B2024.06 | 14.2 | |
| Mistral-7B-MetaMathnumber of samples=0.40M, Base Model=Mistral-7B2024.06 | 14 | |
| Llama3-8B-MetaMathnumber of samples=0.40M, Base Model=Llama3-8B2024.06 | 13.8 |