Mathematical Reasoning on MATH-500 (test)
99.2AccuracyCIKA
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| CIKAParameter Count=7B, Status=Frozen2026.05 | 99.2 | — | — | — | — | — | — | — | — | — | |
| Agent 1Controller Backbone=Qwen3-4B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.05 | 93.8 | — | — | — | — | — | — | — | — | — | |
| Agent 1Controller Backbone=Qwen3-8B-Base, Agent 1 Model=Qwen3-30B-A3B-Instruct2026.05 | 93.8 | — | — | — | — | — | — | — | — | — | |
| o1-miniParameter Count=>7B2026.05 | 93 | — | — | — | — | — | — | — | — | — | |
| Self-consistencySelection Regime=Regime-independent, Backbone=Qwen2.5-Math-7B-Instruct, n_cal=151, epsilon=0.15, K=20 bootstrap2026.05 | 91 | 13.8 | — | 91 | — | — | — | — | — | — | |
| COVCAL+fallbackSelection Regime=dev-then-cal, Backbone=Qwen2.5-Math-7B-Instruct, n_cal=151, epsilon=0.15, K=20 bootstrap2026.05 | 90.5 | 14.4 | — | 90.5 | — | — | — | — | — | — | |
| rStarFine-tuned=true2026.05 | 90 | — | — | — | — | — | — | — | — | — | |
| Raw Lean + fallbackSelection Regime=Regime-independent, Backbone=Qwen2.5-Math-7B-Instruct, n_cal=151, epsilon=0.15, K=20 bootstrap2026.05 | 89.1 | 15.9 | — | 89.1 | — | — | — | — | — | — | |
| GraphDPOground-truth anchoring=true2026.05 | 88.87 | — | — | — | — | — | — | — | — | — | |
| Iterative Critique-and-Routing ControllerController Backbone=Qwen3-4B-Base2026.05 | 87.8 | — | — | — | — | — | — | — | — | — | |
| Iterative Critique-and-Routing ControllerController Backbone=Qwen3-8B-Base2026.05 | 87.6 | — | — | — | — | — | — | — | — | — | |
| GraphDPOground-truth anchoring=false2026.05 | 87.4 | — | — | — | — | — | — | — | — | — | |
| Maj@8Decoding Strategy=Majority Voting @ 82026.05 | 87.1 | — | — | — | — | — | — | — | — | — | |
| Confidence-only abst.Selection Regime=Regime-independent, Backbone=Qwen2.5-Math-7B-Instruct, n_cal=151, epsilon=0.15, K=20 bootstrap2026.05 | 86.6 | 6.8 | — | 96.9 | — | — | — | — | — | — | |
| SWEPO2026.05 | 85.6 | — | — | — | — | — | — | — | — | — | |
| LiPO2026.05 | 85.6 | — | — | — | — | — | — | — | — | — | |
| GRPO2026.05 | 84.2 | — | — | — | — | — | — | — | — | — | |
| MPO2026.05 | 84.2 | — | — | — | — | — | — | — | — | — | |
| Greedy CoTDecoding Strategy=Greedy Chain-of-Thought2026.05 | 83.6 | — | — | — | — | — | — | — | — | — | |
| PRO2026.05 | 83.6 | — | — | — | — | — | — | — | — | — | |
| Router-R1Controller Backbone=Qwen3-4B-Base2026.05 | 80.8 | — | — | — | — | — | — | — | — | — | |
| Controller V2Controller Backbone=Qwen3-4B-Base2026.05 | 80.4 | — | — | — | — | — | — | — | — | — | |
| Controller V2Controller Backbone=Qwen3-8B-Base2026.05 | 79.2 | — | — | — | — | — | — | — | — | — | |
| RouterDCController Backbone=Qwen3-4B-Base2026.05 | 76.4 | — | — | — | — | — | — | — | — | — | |
| Agent 2Controller Backbone=Qwen3-4B-Base, Agent 2 Model=Qwen2.5-7B-Instruct2026.05 | 75 | — | — | — | — | — | — | — | — | — | |
| RoBERTa RouterController Backbone=Qwen3-4B-Base2026.05 | 75 | — | — | — | — | — | — | — | — | — | |
| Random RouterController Backbone=Qwen3-4B-Base2026.05 | 74.4 | — | — | — | — | — | — | — | — | — | |
| Controller V1Controller Backbone=Qwen3-4B-Base2026.05 | 73.8 | — | — | — | — | — | — | — | — | — | |
| Controller V1Controller Backbone=Qwen3-8B-Base2026.05 | 73.8 | — | — | — | — | — | — | — | — | — | |
| Agent 2Controller Backbone=Qwen3-8B-Base, Agent 2 Model=Ministral-3-8B-Instruct2026.05 | 72 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 71.4 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 71.4 | — | — | — | — | — | — | — | — | — | |
| SPIN2026.05 | 70.8 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 70.6 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 70.6 | — | — | — | — | — | — | — | — | — | |
| ImplicitPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 70.4 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 70.4 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 70.2 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 69.8 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 69.6 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 69.6 | — | — | — | — | — | — | — | — | — | |
| ImplicitPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 69.2 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 69.2 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 69.2 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 69 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@642026.06 | 68.4 | — | — | — | — | — | — | — | — | — | |
| ImplicitPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 68.2 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@322026.06 | 68 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 67.8 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 67.8 | — | — | — | — | — | — | — | — | — | |
| RouterDCController Backbone=Qwen3-8B-Base2026.05 | 67.4 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 67.4 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 67.2 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 66.4 | — | — | — | — | — | — | — | — | — | |
| RoBERTa RouterController Backbone=Qwen3-8B-Base2026.05 | 66.2 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Qwen-2.5-7B-Inst, Generator Pass@1=65.8, Sampling Count=@82026.06 | 66.2 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 66 | — | — | — | — | — | — | — | — | — | |
| Random RouterController Backbone=Qwen3-8B-Base2026.05 | 65.9 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 65.6 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 65.4 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 65.2 | — | — | — | — | — | — | — | — | — | |
| MAGSBase Model=Gemma-4-E4b-it2026.05 | 64.8 | — | — | — | — | 1.108 | — | — | — | — | |
| ITIBase Model=Gemma-4-E4b-it2026.05 | 64.6 | — | — | — | — | 1.121 | — | — | — | — | |
| Majority VoteGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 63.2 | — | — | — | — | — | — | — | — | — | |
| Router-R1Controller Backbone=Qwen3-8B-Base2026.05 | 62.8 | — | — | — | — | — | — | — | — | — | |
| ImplicitPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 62.8 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 62.2 | — | — | — | — | — | — | — | — | — | |
| UnsteeredBase Model=Gemma-4-E4b-it2026.05 | 61.4 | — | — | — | — | 1.107 | — | — | — | — | |
| ImplicitPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 61 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 60.6 | — | — | — | — | — | — | — | — | — | |
| CDBase Model=Gemma-4-E4b-it2026.05 | 60.4 | — | — | — | — | 1.13 | — | — | — | — | |
| PQMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 60.4 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 59.8 | — | — | — | — | — | — | — | — | — | |
| ImplicitPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 59.4 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 59.4 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 59 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 58.2 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@642026.06 | 57.4 | — | — | — | — | — | — | — | — | — | |
| ASBase Model=Gemma-4-E4b-it2026.05 | 56.6 | — | — | — | — | 1.279 | — | — | — | — | |
| Majority VoteGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 56.6 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@322026.06 | 55.4 | — | — | — | — | — | — | — | — | — | |
| SFT2026.05 | 54.8 | — | — | — | — | — | — | — | — | — | |
| Agent 3Controller Backbone=Qwen3-4B-Base, Agent 3 Model=Qwen2.5-1.5B-Instruct2026.05 | 54.2 | — | — | — | — | — | — | — | — | — | |
| OmegaPRMGenerator Model=Llama-3.1-8B-Inst, Generator Pass@1=44.6, Sampling Count=@82026.06 | 53.4 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@642026.06 | 35.4 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@642026.06 | 33.6 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@642026.06 | 33.2 | — | — | — | — | — | — | — | — | — | |
| OTS2026.05 | 32.8 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@322026.06 | 31.2 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@322026.06 | 31 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@322026.06 | 30 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@642026.06 | 28.8 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@322026.06 | 27 | — | — | — | — | — | — | — | — | — | |
| SCANGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@82026.06 | 26.4 | — | — | — | — | — | — | — | — | — | |
| Agent 3Controller Backbone=Qwen3-8B-Base, Agent 3 Model=Llama-3-2-1B-Instruct2026.05 | 26 | — | — | — | — | — | — | — | — | — | |
| LCAalpha=1, Generator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@82026.06 | 25 | — | — | — | — | — | — | — | — | — | |
| MathShepherdGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@82026.06 | 24.6 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@642026.06 | 22.8 | — | — | — | — | — | — | — | — | — | |
| PQMGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@82026.06 | 21.8 | — | — | — | — | — | — | — | — | — | |
| Majority VoteGenerator Model=Mistral-7B-Inst-v0.2, Generator Pass@1=9.6, Sampling Count=@322026.06 | 21 | — | — | — | — | — | — | — | — | — |