Math Reasoning on AMC23 (Pass@1 to Pass@256)
87.5Pass@1UMAD
Evaluation results
| Method | Links | ||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 87.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 82.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 77.5 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen3-4B-Instruct-25072026.03 | 75 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 72.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 70.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 70.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 67.5 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen3-4B-Instruct-25072026.03 | 65 | — | — | — | — | — | — | — | — | — | |
| GRPOBase Model=Qwen2.5-Math-7B2025.10 | 60.8 | 72.7 | 81.3 | 86.8 | 89.8 | 92 | 94.2 | 95.9 | 97.5 | — | |
| THRBase Model=Qwen2.5-Math-7B, p=< 02025.10 | 60.2 | 72.2 | 80.7 | 85.9 | 89.5 | 92.8 | 95.9 | 98.3 | 100 | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 60 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 60 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 60 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 60 | — | — | — | — | — | — | — | — | — | |
| THRBase Model=Qwen2.5-Math-7B2025.10 | 58.1 | 71.3 | 80.7 | 87.1 | 90.9 | 93.5 | 95.9 | 98.3 | 100 | — | |
| THRBase Model=Qwen2.5-Math-7B, p=> 02025.10 | 57 | 70 | 79.8 | 86.8 | 91.2 | 94 | 96.1 | 97.3 | 97.5 | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen3-4B-Instruct-25072026.03 | 55 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 52.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 50 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 50 | — | — | — | — | — | — | — | — | — | |
| THRBase Model=Qwen2.5-Math-1.5B, p=< 02025.10 | 47.9 | 61 | 72.2 | 81.1 | 87.3 | 91.6 | 95.1 | 98 | 100 | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 47.5 | — | — | — | — | — | — | — | — | — | |
| GRPOBase Model=Qwen2.5-Math-1.5B2025.10 | 46.6 | 59.1 | 70 | 78.9 | 85.5 | 90.2 | 93.7 | 96 | 97.5 | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 45 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 45 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 45 | — | — | — | — | — | — | — | — | — | |
| THRBase Model=Qwen2.5-Math-1.5B2025.10 | 44.8 | 57.8 | 69.1 | 78.2 | 85.1 | 90.1 | 93.6 | 95.9 | 97.5 | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 42.5 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=5, Backbone=Qwen2.5-3B-Instruct2026.03 | 42.5 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 42.5 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 42.5 | — | — | — | — | — | — | — | — | — | |
| THRBase Model=Qwen2.5-Math-1.5B, p=> 02025.10 | 41.4 | 54.8 | 66.8 | 76.6 | 84.2 | 89.5 | 93.2 | 95.8 | 97.5 | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 40 | — | — | — | — | — | — | — | — | — | |
| UMADDebate Setting=Homogeneous MAD, Agent=A1, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 37.5 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 37.5 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 35 | — | — | — | — | — | — | — | — | — | |
| Zero-ShotDebate Setting=Homogeneous MAD, Agent=A0, Round (T)=2, Backbone=Qwen2.5-3B-Instruct2026.03 | 30 | — | — | — | — | — | — | — | — | — | |
| IPPODebate Setting=Heterogeneous MAD, Agent=A0, Round (T)=1, Backbone=Qwen2.5-3B-Instruct2026.03 | 27.5 | — | — | — | — | — | — | — | — | — | |
| BaseBase Model=Qwen2.5-Math-7B2025.10 | 25 | 40.6 | 58.2 | 72.9 | 82.8 | 88.7 | 92.6 | 96.2 | 100 | — | |
| BaseBase Model=Qwen2.5-Math-1.5B2025.10 | 15.3 | 26.7 | 42.1 | 58.6 | 72.3 | 81.9 | 88.8 | 94.3 | 97.5 | — | |
| THRBase Model=Qwen2.5-0.5B-Instruct2025.10 | 12 | 20.2 | 30.8 | 43 | 56.1 | 68.6 | 79.5 | 88 | 92.5 | — | |
| THRBase Model=Qwen2.5-0.5B-Instruct, p=< 02025.10 | 12 | 20.1 | 30.6 | 42.7 | 56.5 | 70.8 | 82.7 | 89.6 | 92.5 | — | |
| GRPOBase Model=Qwen2.5-0.5B-Instruct2025.10 | 11.4 | 18.7 | 28.3 | 39.7 | 52.3 | 64.5 | 74.9 | 81.8 | 85 | — | |
| THRBase Model=Qwen2.5-0.5B-Instruct, p=> 02025.10 | 11.1 | 18.8 | 29.2 | 41.9 | 56 | 69.3 | 80.1 | 87.5 | 92.5 | — | |
| BaseBase Model=Qwen2.5-0.5B-Instruct2025.10 | 4.1 | 7.8 | 14 | 23.4 | 36.1 | 50.6 | 64.4 | 75.4 | 82.5 | — | |
| BaseBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 22.8 | — | — | — | — | — | |
| BaseBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 53.4 | — | — | — | — | — | |
| EntroPICEntropy-preserving method=EntroPIC, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 92.5 | — | — | — | 55.3 | |
| GRPOBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 29.5 | — | — | — | — | — | |
| GRPOBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 55.5 | — | — | — | — | — | |
| GRPORL algorithm=GRPO, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 92.5 | — | — | — | 57.4 | |
| GRPO + Clip-CovRL algorithm=GRPO, Entropy-preserving method=Clip-Cov, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 90 | — | — | — | 57.1 | |
| GRPO + Clip-HigherRL algorithm=GRPO, Entropy-preserving method=Clip-Higher, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 95 | — | — | — | 55.5 | |
| GRPO + EntrocraftRL algorithm=GRPO, Entropy-preserving method=Entrocraft, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 95 | — | — | — | 65 | |
| GRPO + Entropy LossRL algorithm=GRPO, Entropy-preserving method=Entropy Loss, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 92.5 | — | — | — | 55 | |
| GSPORL algorithm=GSPO, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 92.5 | — | — | — | 57.7 | |
| GSPO + Clip-CovRL algorithm=GSPO, Entropy-preserving method=Clip-Cov, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 92.5 | — | — | — | 56.3 | |
| GSPO + Clip-HigherRL algorithm=GSPO, Entropy-preserving method=Clip-Higher, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 95 | — | — | — | 54.7 | |
| GSPO + EntrocraftRL algorithm=GSPO, Entropy-preserving method=Entrocraft, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 95 | — | — | — | 56.1 | |
| GSPO + Entropy LossRL algorithm=GSPO, Entropy-preserving method=Entropy Loss, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 97.5 | — | — | — | 55.9 | |
| HiLLBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 34.8 | — | — | — | — | — | |
| HiLLBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 63 | — | — | — | — | — | |
| HiLLw/o TWBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 32.5 | — | — | — | — | — | |
| HiLLw/o TWBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 60.5 | — | — | — | — | — | |
| LUFFYBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 18.6 | — | — | — | — | — | |
| LUFFYBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 55.2 | — | — | — | — | — | |
| SAGEBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 34.7 | — | — | — | — | — | |
| SAGEBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 60.3 | — | — | — | — | — | |
| Scaf-GRPOBackbone=Llama-3.2-3B-Instruct2026.04 | — | — | — | — | 28.8 | — | — | — | — | — | |
| Scaf-GRPOBackbone=Qwen2.5-7B-Instruct2026.04 | — | — | — | — | 58.8 | — | — | — | — | — | |
| W-ReinforceEntropy-preserving method=W-Reinforce, Temperature=0.6, Answers per question=322026.04 | — | — | — | — | — | 90 | — | — | — | 53.5 |