List your company
ResearchFormerly WizWand

Mathematical Reasoning on MMLU STEM (test)

82.8AccuracyQwen2.5-Math-72B
22.79238.37153.9569.529Oct 16, 2023Dec 11, 2023Feb 5, 2024Apr 2, 2024May 28, 2024Jul 23, 2024Sep 18, 2024
Updated 8mo ago

Evaluation results

MethodLinks
Qwen2.5-Math-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.0982.8
Qwen2-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.0979.9
Qwen2-Math-72Bshot=4-shot, prompting=few-shot chain-of-thought2024.0979.1
Llama-3.1-70Bshot=4-shot, prompting=few-shot chain-of-thought2024.0978.1
Qwen2.5-Math-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.0967.8
Qwen2-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.0967.6
Qwen2-Math-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.0965.7
MinervaParameters=540B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1063.9
InternLM2-Math-Base-20Bshot=4-shot, prompting=few-shot chain-of-thought2024.0963
DeepSeek-Coder-V2-Lite-Baseshot=4-shot, prompting=few-shot chain-of-thought2024.0959.5
DeepSeekMath-Base-7Bshot=4-shot, prompting=few-shot chain-of-thought2024.0956.5
MinervaParameters=62B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1053.9
Llama-3.1-8Bshot=4-shot, prompting=few-shot chain-of-thought2024.0953.1
Qwen2.5-Math-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.0951.3
Qwen2-Math-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.0950.4
LLEMMAParameters=34B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1049
Qwen2-1.5Bshot=4-shot, prompting=few-shot chain-of-thought2024.0944.8
Code LlamaParameters=34B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1040.5
LLEMMAParameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1037.7
MinervaParameters=8B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1035.6
Llama 2Parameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1029.9
Code LlamaParameters=7B, Decoding=Greedy, Prompting=Few-shot chain-of-thought2023.1025.1