List your company
ResearchFormerly WizWand

Mathematical Reasoning on SVAMP (Pass@1 Accuracy)

94.2Pass@1 AccuracyBaseline
36.563251.526666.4981.4534Oct 9, 2025Nov 18, 2025Dec 28, 2025Feb 7, 2026Mar 19, 2026Apr 28, 2026Jun 8, 2026
Updated 5mo ago

Evaluation results

MethodLinks
BaselineBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1094.2
SFTBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1094.1
BaselineBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1093.9
SFTBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1093.5
GPT-4Zero-shot=true2026.0493.1
LightRBase Model=Qwen2.5-Math-7B-Instruct, Evaluation Protocol=Zero-shot2025.1093.1
LightRBase Model=Qwen2.5-Math-1.5B-Instruct, Evaluation Protocol=Zero-shot2025.1093
IRISBackbone=Qwen2.5, Zero-shot=true2026.0490.6
LightRBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1083.5
Qwen2.5 BaseZero-shot=true2026.0482.1
SFTBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1081.5
BaselineBase Model=DeepSeek-R1-Distill-Qwen-1.5B, Evaluation Protocol=Zero-shot2025.1079.9
LightRBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1077.2
SFTBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1076.2
LightRBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1076
DeepSeek-MathZero-shot=true2026.0473.2
IRISBackbone=WizMath, Zero-shot=true2026.0470.4
MetaMathZero-shot=true2026.0468.8
BaselineBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1068.8
BaselineBase Model=Qwen2.5-Math-7B, Evaluation Protocol=Zero-shot2025.1067.9
SFTBase Model=Qwen2.5-Math-1.5B, Evaluation Protocol=Zero-shot2025.1064.1
WizMath BaseZero-shot=true2026.0457.3
Coconut + dropout-GRPOseeds=32026.0644
Qwen2.5-1.5Bmode=zero-shot2026.0643
Coconut SFTbaseline=true, seeds=12026.0641.67
Coconut + GRPOrollout=deterministic, seeds=32026.0641.67
Coconut + REINFORCE + dropoutbaseline_type=EMA, seeds=32026.0638.78