List your company
ResearchFormerly WizWand

Mathematical Reasoning on AIME24 (pass@1)

47Pass@1MinPRO
-1.8810.8123.536.19Dec 16, 2025Dec 28, 2025Jan 10, 2026Jan 22, 2026Feb 4, 2026Feb 16, 2026Mar 1, 2026
Updated 8mo ago

Evaluation results

MethodLinks
MinPROModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.0147
M2POModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.0146.3
CISPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.0145.3
GSPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.0144.1
MinPROModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.0136.1
CISPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.0135.9
M2POModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.0133.9
GRPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.0126.2
GSPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.0125.9
GRPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.0123.1
s1.1-7BBackbone=Qwen2.5-7B, Pure SFT?=false2025.1219
Qwen BaseBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.1211.3
Full SFTBackbone=Qwen2.5-7B, Pure SFT?=false2025.1210
UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.0310
Qwen BaseBackbone=Qwen2.5-7B, Pure SFT?=true2025.128.7
Qwen InstructBackbone=Qwen2.5-7B, Pure SFT?=false2025.128.7
QLoRABackbone=Qwen2.5-7B, Pure SFT?=true2025.128.7
Qwen InstructBackbone=Qwen2.5-Math-1.5B, Pure SFT?=false2025.128.7
LadderBackbone=Qwen2.5-7B, Pure SFT?=true2025.128
LadderBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.128
Full SFTBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.127.9
QLoRABackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.126.7
BaseModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.013.5
Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.033.3
Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.033.3
IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.033.3
IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.033.3
IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.033.3
UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.033.3
BaseModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.012.8
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.030
Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.030