List your company
ResearchFormerly WizWand

Mathematical Reasoning on MGSM

93.2AccuracyDeepSeek V3.2
48.16859.85971.5583.241May 17, 2023Nov 22, 2023May 29, 2024Dec 4, 2024Jun 11, 2025Dec 17, 2025Jun 25, 2026
Updated 4mo ago

Evaluation results

MethodLinks
DeepSeek V3.2SLM=DeepSeek V3.22026.0593.2——
standard cascadeModel=GPT-4o-mini2026.0692.42——
Qwen-3-32BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0291.7——
Llama 3 405Bshot=0-shot CoT, prompting=Native prompts2024.0791.6——
Claude 3.5 Sonnetshot=0-shot CoT, prompting=Native prompts2024.0791.6——
Llama-3.3-70BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0291.6——
context-aware cascadeModel=GPT-4o-mini2026.0691.52——
+PACESLM=Qwen3.5-9B2026.0590.9—5.9
+PACESLM=Ministral-3-14B2026.0590.8—17.9
GPT-4oshot=0-shot CoT, prompting=Native prompts2024.0790.5——
Qwen-3-30B-A3BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0290.5——
+CESLM=Qwen3.5-9B2026.0590.4—5.3
Qwen-3-14BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0290——
Mistral-3.2-24BOpenness=Open-weights, Region=European, Tuning=Instruction-tuned2026.0289.6——
o3-miniversion=2025-01-312025.0289.3——
Mistral-Large2025.0289.01——
ACESLM=Qwen3.5-9B2026.0588.9—3.6
Gemma-3-27BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0288.4——
Qwen2.5-72B-InstructType=Instruct2025.0288.16——
+CESLM=Ministral-3-14B2026.0588—14.3
GEPASLM=Qwen3.5-9B2026.0587.8—2.3
OLMo-3.1-32BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.0287.4——
GPT-4o-mini2025.0287.36——
+PESLM=Qwen3.5-9B2026.0587.2—1.6
Llama 3 70Bshot=0-shot CoT, prompting=Native prompts2024.0786.9——
Gödel AgentSLM=Qwen3.5-9B2026.0586.4—0.7
end-to-endModel=GPT-4o-mini2026.0686.36——
MIPROv2SLM=Qwen3.5-9B2026.0586.1—0.4
Gemma-3-12BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0286——
GPT-4shot=0-shot CoT, prompting=Native prompts2024.0785.9——
VanillaSLM=Qwen3.5-9B2026.0585.8——
+PACESLM=Qwen3-4B2026.0583.1—4.3
Gödel AgentSLM=Ministral-3-14B2026.0582.5—7.1
RL+SAEModel=Qwen3-30B-A3B2026.0582.4——
Qwen2.5-14B-InstructType=Instruct2025.0282.27——
+CESLM=Qwen3-4B2026.0582.1—3.3
+PESLM=Qwen3-4B2026.0581.5—2.5
+PESLM=Ministral-3-14B2026.0581.4—5.5
Gödel AgentSLM=Qwen3-4B2026.0581.2—2.1
ACESLM=Qwen3-4B2026.0580.8—1.6
GEPASLM=Qwen3-4B2026.0580.2—0.1
VanillaSLM=Qwen3-4B2026.0579.5——
MIPROv2SLM=Qwen3-4B2026.0579.5—0
ACESLM=Ministral-3-14B2026.0578.5—1.9
Gemma2-9B2025.0278.37——
MIPROv2SLM=Ministral-3-14B2026.0578.3—1.6
Vanilla RLModel=Qwen3-30B-A3B2026.0577.64——
GEPASLM=Ministral-3-14B2026.0577.5—0.6
VanillaSLM=Ministral-3-14B2026.0577——
OLMo-3-7BOpenness=Fully-open, Region=Non-European, Tuning=Instruction-tuned2026.0276.6——
Before RLModel=Qwen3-30B-A3B2026.0576.56——
EuroLLM-22B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0276.1——
Flan-PaLM 2 (L)shot=8-shot CoT2023.0575.9——
Llama-3.1-8BOpenness=Open-weights, Region=Non-European, Tuning=Instruction-tuned2026.0273——
Apertus-70BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0272.7——
RL+SAEModel=Qwen3-8B2026.0572.4——
PaLM 2 (L)shot=8-shot CoT2023.0572.2——
EuroLLM-22B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0271.9——
Mixtral 8×22Bshot=0-shot CoT, prompting=Native prompts2024.0771.1——
Vanilla RLModel=Qwen3-8B2026.0570.96——
Before RLModel=Qwen3-8B2026.0570.12——
Llama 3 8Bshot=0-shot CoT, prompting=Native prompts2024.0768.9——
EuroLLM-9B (new)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0267.3——
QuaSARBackbone=Llama-3-8B2025.0266.9——
SFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.0765.71——
SFT+PenaltyBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.0764.13——
SASFTBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.0763.92——
SFT+GRPOBackbone=Qwen3-8B-Base, Training dataset setting=Korean 210k2025.0761.87——
CoTBackbone=Llama-3-8B2025.0260.8——
EuroLLM-9B (old)Openness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0260.5——
Flan-U-PaLM-540Bshot=8-shot CoT2023.0560.4——
baselineBackbone=Llama-3-8B2025.0259——
Apertus-8BOpenness=Fully-open, Region=European, Tuning=Instruction-tuned2026.0258.9——
SASFTModel=Qwen3-8B-Base2025.0758.45——
SASFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.0558.45——
Gemma-3Parameters=12B, Training Stage=PT2026.0158.44——
SFTModel=Qwen3-8B-Base2025.0758.03——
SFTModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.0558.03——
SFT+PenaltyModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.0757.25——
SFT+PenaltyModel=Qwen3-8B-Base2025.0756.29——
SFT+PenaltyModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.0556.29——
SFT+GRPOModel=Qwen3-8B-Base2025.0755.28——
SFT+GRPOModel=Qwen3-8B-Base, Training Dataset=Chinese 110k2026.0555.28——
SFTModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.0755.09——
SASFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.0753.89——
SFT+GRPOBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.0753.44——
Gemma 2 9Bshot=0-shot CoT, prompting=Native prompts2024.0753.2——
BYOL-nyaParameters=12B, Pre-training Type=CPT2026.0153.2——
RL+SAEModel=Qwen3-1.7B2026.0552.36——
BYOL-mriParameters=12B, Training Stage=CPT2026.0152——
SASFTModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.0751.95——
end-to-endModel=Llama-3.1-8B-Instruct2026.0651.82——
GPT-3.5 Turboshot=0-shot CoT, prompting=Native prompts2024.0751.4——
SASFTBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.0750.88——
SFT+GRPOModel=Qwen3-8B-Base, Training Set=Chinese 210k2025.0750.75——
SFT+PenaltyBackbone=Gemma-2-9B, Training dataset setting=Korean 210k2025.0750.43——
SASFTModel=Gemma-2-9B, Training Set=Chinese 210k2025.0750.29——
SFTModel=Qwen3-8B-Base, Dataset=Russian 110k2025.0750.03——
SFT+PenaltyModel=Qwen3-8B-Base, Dataset=Russian 110k2025.0750.03——
U-PaLM-540Bshot=8-shot CoT2023.0549.9——