Mathematical Reasoning on MMLU STEM (test)
82.8AccuracyQwen2.5-Math-72B
Evaluation Results
| Method | Links | |
|---|---|---|
| Qwen2.5-Math-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 82.8 | |
| Qwen2-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 79.9 | |
| Qwen2-Math-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 79.1 | |
| Llama-3.1-70Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 78.1 | |
| Qwen2.5-Math-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 67.8 | |
| Qwen2-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 67.6 | |
| Qwen2-Math-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 65.7 | |
| MinervaParameters=540B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 63.9 | |
| InternLM2-Math-Base-20Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 63 | |
| DeepSeek-Coder-V2-Lite-Baseshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 59.5 | |
| DeepSeekMath-Base-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 56.5 | |
| MinervaParameters=62B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 53.9 | |
| Llama-3.1-8Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 53.1 | |
| Qwen2.5-Math-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 51.3 | |
| Qwen2-Math-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 50.4 | |
| LLEMMAParameters=34B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 49 | |
| Qwen2-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.09 | 44.8 | |
| Code LlamaParameters=34B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 40.5 | |
| LLEMMAParameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 37.7 | |
| MinervaParameters=8B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 35.6 | |
| Llama 2Parameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 29.9 | |
| Code LlamaParameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.10 | 25.1 |