Mathematical Reasoning on MGSM
93.2AccuracyDeepSeek V3.2
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| DeepSeek V3.2SLM=DeepSeek V3.22026.05 | 93.2 | — | — | |
| standard cascadeModel=GPT-4o-mini2026.06 | 92.42 | — | — | |
| Qwen-3-32BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 91.7 | — | — | |
| Llama 3 405Bshot=0-shot CoT, prompting=Native prompts2024.07 | 91.6 | — | — | |
| Claude 3.5 Sonnetshot=0-shot CoT, prompting=Native prompts2024.07 | 91.6 | — | — | |
| Llama-3.3-70BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 91.6 | — | — | |
| context-aware cascadeModel=GPT-4o-mini2026.06 | 91.52 | — | — | |
| +PACESLM=Qwen3.5-9B2026.05 | 90.9 | — | 5.9 | |
| +PACESLM=Ministral-3-14B2026.05 | 90.8 | — | 17.9 | |
| GPT-4oshot=0-shot CoT, prompting=Native prompts2024.07 | 90.5 | — | — | |
| Qwen-3-30B-A3BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 90.5 | — | — | |
| +CESLM=Qwen3.5-9B2026.05 | 90.4 | — | 5.3 | |
| Qwen-3-14BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 90 | — | — | |
| Mistral-3.2-24BOpenness=Open-weights, Region=European, Tuning=Instruction-tuned2026.02 | 89.6 | — | — | |
| o3-miniversion=2025-01-312025.02 | 89.3 | — | — | |
| Mistral-Large2025.02 | 89.01 | — | — | |
| ACESLM=Qwen3.5-9B2026.05 | 88.9 | — | 3.6 | |
| Gemma-3-27BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 88.4 | — | — | |
| Qwen2.5-72B-InstructType=Instruct2025.02 | 88.16 | — | — | |
| +CESLM=Ministral-3-14B2026.05 | 88 | — | 14.3 | |
| GEPASLM=Qwen3.5-9B2026.05 | 87.8 | — | 2.3 | |
| OLMo-3.1-32BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 87.4 | — | — | |
| GPT-4o-mini2025.02 | 87.36 | — | — | |
| +PESLM=Qwen3.5-9B2026.05 | 87.2 | — | 1.6 | |
| Llama 3 70Bshot=0-shot CoT, prompting=Native prompts2024.07 | 86.9 | — | — | |
| Gödel AgentSLM=Qwen3.5-9B2026.05 | 86.4 | — | 0.7 | |
| end-to-endModel=GPT-4o-mini2026.06 | 86.36 | — | — | |
| MIPROv2SLM=Qwen3.5-9B2026.05 | 86.1 | — | 0.4 | |
| Gemma-3-12BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 86 | — | — | |
| GPT-4shot=0-shot CoT, prompting=Native prompts2024.07 | 85.9 | — | — | |
| VanillaSLM=Qwen3.5-9B2026.05 | 85.8 | — | — | |
| +PACESLM=Qwen3-4B2026.05 | 83.1 | — | 4.3 | |
| Gödel AgentSLM=Ministral-3-14B2026.05 | 82.5 | — | 7.1 | |
| RL+SAEModel=Qwen3-30B-A3B2026.05 | 82.4 | — | — | |
| Qwen2.5-14B-InstructType=Instruct2025.02 | 82.27 | — | — | |
| +CESLM=Qwen3-4B2026.05 | 82.1 | — | 3.3 | |
| +PESLM=Qwen3-4B2026.05 | 81.5 | — | 2.5 | |
| +PESLM=Ministral-3-14B2026.05 | 81.4 | — | 5.5 | |
| Gödel AgentSLM=Qwen3-4B2026.05 | 81.2 | — | 2.1 | |
| ACESLM=Qwen3-4B2026.05 | 80.8 | — | 1.6 | |
| GEPASLM=Qwen3-4B2026.05 | 80.2 | — | 0.1 | |
| VanillaSLM=Qwen3-4B2026.05 | 79.5 | — | — | |
| MIPROv2SLM=Qwen3-4B2026.05 | 79.5 | — | 0 | |
| ACESLM=Ministral-3-14B2026.05 | 78.5 | — | 1.9 | |
| Gemma2-9B2025.02 | 78.37 | — | — | |
| MIPROv2SLM=Ministral-3-14B2026.05 | 78.3 | — | 1.6 | |
| Vanilla RLModel=Qwen3-30B-A3B2026.05 | 77.64 | — | — | |
| GEPASLM=Ministral-3-14B2026.05 | 77.5 | — | 0.6 | |
| VanillaSLM=Ministral-3-14B2026.05 | 77 | — | — | |
| OLMo-3-7BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.02 | 76.6 | — | — | |
| Before RLModel=Qwen3-30B-A3B2026.05 | 76.56 | — | — | |
| EuroLLM-22B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 76.1 | — | — | |
| Flan-PaLM 2 (L)shot=8-shot CoT2023.05 | 75.9 | — | — | |
| Llama-3.1-8BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.02 | 73 | — | — | |
| Apertus-70BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 72.7 | — | — | |
| RL+SAEModel=Qwen3-8B2026.05 | 72.4 | — | — | |
| PaLM 2 (L)shot=8-shot CoT2023.05 | 72.2 | — | — | |
| EuroLLM-22B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 71.9 | — | — | |
| Mixtral 8×22Bshot=0-shot CoT, prompting=Native prompts2024.07 | 71.1 | — | — | |
| Vanilla RLModel=Qwen3-8B2026.05 | 70.96 | — | — | |
| Before RLModel=Qwen3-8B2026.05 | 70.12 | — | — | |
| Llama 3 8Bshot=0-shot CoT, prompting=Native prompts2024.07 | 68.9 | — | — | |
| EuroLLM-9B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 67.3 | — | — | |
| QuaSARBackbone=Llama-3-8B2025.02 | 66.9 | — | — | |
| SFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 65.71 | — | — | |
| SFT+PenaltyBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 64.13 | — | — | |
| SASFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 63.92 | — | — | |
| SFT+GRPOBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.07 | 61.87 | — | — | |
| CoTBackbone=Llama-3-8B2025.02 | 60.8 | — | — | |
| EuroLLM-9B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 60.5 | — | — | |
| Flan-U-PaLM-540Bshot=8-shot CoT2023.05 | 60.4 | — | — | |
| baselineBackbone=Llama-3-8B2025.02 | 59 | — | — | |
| Apertus-8BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.02 | 58.9 | — | — | |
| SASFTModel=Qwen3-8B-Base2025.07 | 58.45 | — | — | |
| SASFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 58.45 | — | — | |
| Gemma-3Parameters=12B, Training Stage=PT2026.01 | 58.44 | — | — | |
| SFTModel=Qwen3-8B-Base2025.07 | 58.03 | — | — | |
| SFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 58.03 | — | — | |
| SFT+PenaltyModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.07 | 57.25 | — | — | |
| SFT+PenaltyModel=Qwen3-8B-Base2025.07 | 56.29 | — | — | |
| SFT+PenaltyModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 56.29 | — | — | |
| SFT+GRPOModel=Qwen3-8B-Base2025.07 | 55.28 | — | — | |
| SFT+GRPOModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.05 | 55.28 | — | — | |
| SFTModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.07 | 55.09 | — | — | |
| SASFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 53.89 | — | — | |
| SFT+GRPOBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 53.44 | — | — | |
| Gemma 2 9Bshot=0-shot CoT, prompting=Native prompts2024.07 | 53.2 | — | — | |
| BYOL-nyaParameters=12B, Pre-training Type=CPT2026.01 | 53.2 | — | — | |
| RL+SAEModel=Qwen3-1.7B2026.05 | 52.36 | — | — | |
| BYOL-mriParameters=12B, Training Stage=CPT2026.01 | 52 | — | — | |
| SASFTModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.07 | 51.95 | — | — | |
| end-to-endModel=Llama-3.1-8B-Instruct2026.06 | 51.82 | — | — | |
| GPT-3.5 Turboshot=0-shot CoT, prompting=Native prompts2024.07 | 51.4 | — | — | |
| SASFTBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 50.88 | — | — | |
| SFT+GRPOModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.07 | 50.75 | — | — | |
| SFT+PenaltyBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.07 | 50.43 | — | — | |
| SASFTModel=Gemma-2-9B, Training Set=Chinese 210k2025.07 | 50.29 | — | — | |
| SFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 50.03 | — | — | |
| SFT+PenaltyModel=Qwen3-8B-Base, Dataset=Russian 110k2025.07 | 50.03 | — | — | |
| U-PaLM-540Bshot=8-shot CoT2023.05 | 49.9 | — | — |