Math Reasoning on MATH500 (Pass@1)
87.2Pass@1 RateUMAD
Evaluation results
| Method | Links | |
|---|---|---|
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 87.2 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 86 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 85.1 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 84.6 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 82.6 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 81 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 79.8 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 78.2 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 77.8 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 77.8 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 76.8 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 76 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 75.2 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 73.6 | |
| LCO-KLDBackbone=Qwen-3-4B2026.03 | 73.2 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 72 | |
| LCO-LCHBackbone=Qwen-3-4B2026.03 | 69.4 | |
| ϕDPOBackbone=Qwen-3-4B2026.03 | 69.2 | |
| PPOBackbone=Qwen-3-4B2026.03 | 67.8 | |
| DAPOBackbone=Qwen-3-4B2026.03 | 67.8 | |
| GRPOBackbone=Qwen-3-4B2026.03 | 67.6 | |
| LCO-MSEBackbone=Qwen-3-4B2026.03 | 67.2 | |
| GSPOBackbone=Qwen-3-4B2026.03 | 66.4 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 65.8 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 65.8 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 65.6 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 65.4 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 65.2 | |
| REINFORCEBackbone=Qwen-3-4B2026.03 | 64.8 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 64.6 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 64.6 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 64.4 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 64.4 | |
| GRPO with ground-truth rewardModel=Qwen2.5-3B2026.05 | 64.4 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 64.2 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 63.8 | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 63.6 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 63.4 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 63.4 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 63 | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 62.6 | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 62.4 | |
| Multi-reward + KL-CovModel=Qwen2.5-3B2026.05 | 62.4 | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 62.2 | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 62.2 | |
| LCO-LCHBackbone=Qwen-2.5-3B2026.03 | 61.4 | |
| π*Backbone=Qwen-3-4B2026.03 | 61.35 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 61 | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 61 | |
| LCO-KLDBackbone=Qwen-2.5-3B2026.03 | 60.8 | |
| Multi-rewardModel=Qwen2.5-3B2026.05 | 60 | |
| LCO-MSEBackbone=Qwen-2.5-3B2026.03 | 59.2 | |
| SFTBackbone=Qwen-3-4B2026.03 | 58.8 | |
| ϕDPOBackbone=Qwen-2.5-3B2026.03 | 58.3 | |
| GRPOBackbone=Qwen-2.5-3B2026.03 | 57.79 | |
| DAPOBackbone=Qwen-2.5-3B2026.03 | 57.05 | |
| GSPOBackbone=Qwen-2.5-3B2026.03 | 56.8 | |
| PPOBackbone=Qwen-2.5-3B2026.03 | 55.4 | |
| GRPO with ground-truth rewardModel=Qwen2.5-1.5B2026.05 | 55.2 | |
| INTUITORModel=Qwen2.5-3B2026.05 | 55 | |
| REINFORCEBackbone=Qwen-2.5-3B2026.03 | 54.6 | |
| Multi-reward + KL-CovModel=Qwen2.5-1.5B2026.05 | 51.6 | |
| π*Backbone=Qwen-2.5-3B2026.03 | 51.3 | |
| Multi-rewardModel=Qwen2.5-1.5B2026.05 | 46 | |
| SFTBackbone=Qwen-2.5-3B2026.03 | 41.6 | |
| INTUITORModel=Qwen2.5-1.5B2026.05 | 22.6 |