Mathematical Reasoning on AIME24 (pass@1)
47Pass@1MinPRO
Evaluation Results
| Method | Links | |
|---|---|---|
| MinPROModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 47 | |
| M2POModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 46.3 | |
| CISPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 45.3 | |
| GSPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 44.1 | |
| MinPROModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 36.1 | |
| CISPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 35.9 | |
| M2POModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 33.9 | |
| GRPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 26.2 | |
| GSPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 25.9 | |
| GRPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 23.1 | |
| s1.1-7BBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 19 | |
| Qwen BaseBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 11.3 | |
| Full SFTBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 10 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 10 | |
| Qwen BaseBackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 8.7 | |
| Qwen InstructBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 8.7 | |
| QLoRABackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 8.7 | |
| Qwen InstructBackbone=Qwen2.5-Math-1.5B, Pure SFT?=false2025.12 | 8.7 | |
| LadderBackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 8 | |
| LadderBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 8 | |
| Full SFTBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 7.9 | |
| QLoRABackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 6.7 | |
| BaseModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 3.5 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 3.3 | |
| BaseModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 2.8 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 0 |