Advanced Mathematical Problem Solving on MATH
52.3AccuracyQWEN2-SFT w/ SSO_DPO
Evaluation results
| Method | Links | |
|---|---|---|
| QWEN2-SFT w/ SSO_DPOBackbone=QWEN2-SFT2024.10 | 52.3 | |
| QWEN2-SFT w/ PBAA_DPOBackbone=QWEN2-SFT2024.10 | 50.8 | |
| QWEN2-SFT w/ ULTRAFEEDBACKBackbone=QWEN2-SFT2024.10 | 46.7 | |
| QWEN2-SFTBackbone=QWEN2-SFT2024.10 | 44.6 | |
| QWEN2-INSTRUCT w/ SSO_DPOBackbone=QWEN2-INSTRUCT2024.10 | 44.2 | |
| QWEN2-INSTRUCT w/ ULTRAFEEDBACKBackbone=QWEN2-INSTRUCT2024.10 | 42.9 | |
| GPT-4Prompt=CoT2023.10 | 42.5 | |
| QWEN2-INSTRUCT w/ PBAA_DPOBackbone=QWEN2-INSTRUCT2024.10 | 42.1 | |
| QWEN2-INSTRUCTBackbone=QWEN2-INSTRUCT2024.10 | 40 | |
| SEGOBase=CodeLLaMA, Prompt=PoT, Params=13B2023.10 | 40 | |
| Sigma-MoE-Tiny Base# Shots=4-shot, Architecture=MoE, # Activated Params=0.5B, # Total Params=20B2025.12 | 36.88 | |
| SEGOBase=CodeLLaMA, Prompt=PoT, Params=7B2023.10 | 36.8 | |
| PaLM-2Base=PaLM, Prompt=CoT, Params=540B2023.10 | 34.3 | |
| MinervaBase=PaLM, Prompt=CoT, Params=540B2023.10 | 33.6 | |
| Qwen3 0.6B Base# Shots=4-shot, Architecture=Dense, # Activated Params=0.6B, # Total Params=0.6B2025.12 | 32.44 | |
| LLAMA3-SFT w/ SSO_DPOBackbone=LLAMA3-SFT2024.10 | 30.8 | |
| LLAMA3-SFT w/ ULTRAFEEDBACKBackbone=LLAMA3-SFT2024.10 | 30.5 | |
| LLAMA3-SFT w/ PBAA_DPOBackbone=LLAMA3-SFT2024.10 | 30.4 | |
| LLAMA3-INSTRUCT w/ ULTRAFEEDBACKBackbone=LLAMA3-INSTRUCT2024.10 | 29.6 | |
| LLAMA3-INSTRUCT w/ SSO_DPOBackbone=LLAMA3-INSTRUCT2024.10 | 29.6 | |
| LLAMA3-SFTBackbone=LLAMA3-SFT2024.10 | 29.5 | |
| LLAMA3-INSTRUCT w/ PBAA_DPOBackbone=LLAMA3-INSTRUCT2024.10 | 28.9 | |
| LLAMA3-INSTRUCTBackbone=LLAMA3-INSTRUCT2024.10 | 28.5 | |
| Mixtral 8x7BActive Params=13B2024.01 | 28.4 | |
| Gemma-3 4B Base# Shots=4-shot, Architecture=Dense, # Activated Params=4B, # Total Params=4B2025.12 | 26.1 | |
| CodeLLaMABase=CodeLLaMA, Prompt=PoT, Params=13B2023.10 | 18.1 | |
| DeepSeek-V2 Lite# Shots=4-shot, Architecture=MoE, # Activated Params=2.4B, # Total Params=15.7B2025.12 | 17.1 | |
| CodeLLaMABase=CodeLLaMA, Prompt=PoT, Params=7B2023.10 | 14.2 | |
| WizardMATHBase=LLaMA2, Prompt=CoT, Params=13B2023.10 | 14 | |
| LLaMA 2 70BActive Params=70B2024.01 | 13.8 | |
| Mistral 7BModality=Pretrained2023.10 | 13.1 | |
| Mistral 7BActive Params=7B2024.01 | 12.7 | |
| WizardMATHBase=LLaMA2, Prompt=CoT, Params=7B2023.10 | 10.7 | |
| LLaMA 1 33BActive Params=33B2024.01 | 8.4 | |
| LLaMA 2 13BModality=Pretrained2023.10 | 6 | |
| LLaMA 2 13BActive Params=13B2024.01 | 6 | |
| Code-Llama 7BModality=Finetuned2023.10 | 5.2 | |
| LLAMA 2 7BModality=Pretrained2023.10 | 3.9 | |
| LLaMA2Base=LLaMA2, Prompt=CoT, Params=13B2023.10 | 3.9 | |
| LLaMA 2 7BActive Params=7B2024.01 | 3.9 | |
| LLaMA2Base=LLaMA2, Prompt=CoT, Params=7B2023.10 | 2.5 |