Mathematical Reasoning on AIME 25 (pass@1)
80pass@1CompassMax-V3-Thinking
Evaluation Results
| Method | Links | |
|---|---|---|
| CompassMax-V3-Thinking2025.12 | 80 | |
| Chain of Mindset (CoM)Base Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 73.33 | |
| DeepSeek-R12025.12 | 70 | |
| ReActBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 63.33 | |
| w/o MergingModels=Math2026.02 | 61.25 | |
| Zero-shot CoTBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 60 | |
| MRPBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 60 | |
| w/o MergingModels=Code2026.02 | 58.44 | |
| SCF-RKLModels=Fuse2026.02 | 58.02 | |
| Direct I/OBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 56.67 | |
| Chain of CodeBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 56.67 | |
| SCEModels=Fuse2026.02 | 56.25 | |
| Task ArithmeticModels=Fuse2026.02 | 54.17 | |
| Dare Task ArithmeticModels=Fuse2026.02 | 54.06 | |
| Ties MergingModels=Fuse2026.02 | 52.81 | |
| Tree of ThoughtsBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 50 | |
| Dare Ties MergingModels=Fuse2026.02 | 49.27 | |
| Meta-ReasonerBase Model=Qwen3-VL-32B-Instruct, Temperature=0.7, Top-p=0.95, Max Tokens=327682026.02 | 36.67 | |
| MinPROModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 33.4 | |
| CoMBase Model=Gemini-2.0-Flash2026.02 | 33.33 | |
| Chain of Mindset (CoM)Base Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 33.33 | |
| CISPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 32.7 | |
| M2POModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 32 | |
| GSPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 31.8 | |
| MinPROModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 30.3 | |
| Chain of CodeBase Model=Gemini-2.0-Flash2026.02 | 30 | |
| Chain of CodeBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 30 | |
| Direct I/OBase Model=Gemini-2.0-Flash2026.02 | 26.67 | |
| MRPBase Model=Gemini-2.0-Flash2026.02 | 26.67 | |
| Meta-ReasonerBase Model=Gemini-2.0-Flash2026.02 | 26.67 | |
| Direct I/OBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 26.67 | |
| MRPBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 26.67 | |
| Meta-ReasonerBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 26.67 | |
| M2POModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 25.8 | |
| CISPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 25.1 | |
| Zero-shot CoTBase Model=Gemini-2.0-Flash2026.02 | 23.33 | |
| Tree of ThoughtsBase Model=Gemini-2.0-Flash2026.02 | 23.33 | |
| ReActBase Model=Gemini-2.0-Flash2026.02 | 23.33 | |
| Zero-shot CoTBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 23.33 | |
| Tree of ThoughtsBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 23.33 | |
| ReActBase Model=Gemini-2.0-Flash, Temperature=0.7, Top-p=0.95, Max Tokens=81922026.02 | 23.33 | |
| GRPOModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 23.1 | |
| s1.1-7BBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 21 | |
| GRPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 18.5 | |
| GSPOModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 18.3 | |
| CompassMax-V32025.12 | 13.33 | |
| SetPO+GRPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 9.7 | |
| SetPO+DAPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 8.9 | |
| Qwen InstructBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 8.7 | |
| GRPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 8.2 | |
| LadderBackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 8 | |
| LadderBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 8 | |
| Qwen BaseBackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 7.9 | |
| SetPO+GSPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 7.4 | |
| Qwen InstructBackbone=Qwen2.5-Math-1.5B, Pure SFT?=false2025.12 | 7.3 | |
| Full SFTBackbone=Qwen2.5-7B, Pure SFT?=false2025.12 | 7.1 | |
| QLoRABackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 6.7 | |
| DAPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 6.6 | |
| QLoRABackbone=Qwen2.5-7B, Pure SFT?=true2025.12 | 6 | |
| Qwen BaseBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 5.7 | |
| GSPOBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 5.6 | |
| BaseModel Scale=8B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 3.5 | |
| BaseModel Scale=14B-Base, Off-policyness=Large, Generations per prompt=1282026.01 | 2.9 | |
| Full SFTBackbone=Qwen2.5-Math-1.5B, Pure SFT?=true2025.12 | 2.1 | |
| Qwen2.5-Math-1.5BBackbone=Qwen2.5-Math-1.5B, Inference Temperature=0.5, Sampling Count (k)=642026.02 | 1.8 |