Mathematical Reasoning on AMC 23 (Pass@1 Accuracy, Average output length)
100Pass@1 AccuracyD-L
Evaluation results
| Method | Links | ||
|---|---|---|---|
| D-LBackbone=Qwen3-4B, Voting size budget=5122026.05 | 100 | 0.8 | |
| DDCBackbone=Qwen3-4B, Voting size budget=5122026.05 | 100 | 0.5 | |
| DDCBackbone=DeepSeek-R1-0528-Qwen3-8B, Voting size budget=5122026.05 | 100 | 0.8 | |
| DDCBackbone=Qwen3-8B, Voting size budget=5122026.05 | 97.5 | 1 | |
| SCBackbone=Qwen3-32B, Voting size budget=5122026.05 | 97.5 | 10.2 | |
| DDCBackbone=Qwen3-32B, Voting size budget=5122026.05 | 97.5 | 0.6 | |
| D-LBackbone=Qwen3-32B, Voting size budget=5122026.05 | 97.3 | 7.9 | |
| ACBackbone=Qwen3-32B, Voting size budget=5122026.05 | 97 | 3.4 | |
| D-HBackbone=Qwen3-32B, Voting size budget=5122026.05 | 96.8 | 9.3 | |
| D-LBackbone=Qwen3-8B, Voting size budget=5122026.05 | 96.7 | 12.5 | |
| SCBackbone=Qwen3-8B, Voting size budget=5122026.05 | 96.5 | 15.4 | |
| ACBackbone=Qwen3-8B, Voting size budget=5122026.05 | 96.5 | 6.1 | |
| D-HBackbone=Qwen3-4B, Voting size budget=5122026.05 | 96.2 | 10.4 | |
| SCBackbone=Qwen3-4B, Voting size budget=5122026.05 | 96 | 16.6 | |
| ACBackbone=Qwen3-4B, Voting size budget=5122026.05 | 96 | 4.5 | |
| D-HBackbone=Qwen3-8B, Voting size budget=5122026.05 | 96 | 18.3 | |
| DDCBackbone=Qwen3-1.7B, Voting size budget=5122026.05 | 95 | 1.5 | |
| SCBackbone=Qwen3-1.7B, Voting size budget=5122026.05 | 94.5 | 18.7 | |
| ACBackbone=Qwen3-1.7B, Voting size budget=5122026.05 | 94.3 | 10.2 | |
| D-HBackbone=DeepSeek-R1-0528-Qwen3-8B, Voting size budget=5122026.05 | 94 | 9.7 | |
| D-LBackbone=DeepSeek-R1-0528-Qwen3-8B, Voting size budget=5122026.05 | 93.9 | 11 | |
| SCBackbone=DeepSeek-R1-0528-Qwen3-8B, Voting size budget=5122026.05 | 93.5 | 14.1 | |
| ACBackbone=DeepSeek-R1-0528-Qwen3-8B, Voting size budget=5122026.05 | 93.5 | 4.7 | |
| D-LBackbone=Qwen3-1.7B, Voting size budget=5122026.05 | 92.5 | 9.7 | |
| D-HBackbone=Qwen3-1.7B, Voting size budget=5122026.05 | 91.1 | 15.7 | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 87.97 | 6,966 | |
| DASTBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 87.66 | 6,422 | |
| CoDBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 87.34 | 4,880 | |
| TrEffBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 87.03 | 4,420 | |
| LCPOBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 86.88 | 3,108 | |
| LaserBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 82.8 | 2,882.9 | |
| AutoThinkBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 81.9 | 4,616 | |
| TRiMSBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 81.9 | 1,088.3 | |
| VanillaBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 79.5 | 6,668.7 | |
| AdaptThinkBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 79.5 | 4,444.9 | |
| JETBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 79.5 | 3,979.2 | |
| L1-MaxBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 75.94 | 3,197 | |
| GRPOModel=Qwen3-8B-Base2026.04 | 75 | — | |
| GRPO + PIPOModel=Qwen3-8B-Base2026.04 | 75 | — | |
| DAPO + PIPOModel=Qwen3-8B-Base2026.04 | 75 | — | |
| LC-R1Base Model=DeepSeek-R1-Distill-Qwen, Model Scale=7B2026.03 | 74.2 | 3,127.9 | |
| FREIABackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 72.5 | — | |
| TRiMSBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 71.1 | 1,382.9 | |
| OriginalBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 70.62 | 10,162 | |
| L1-MaxBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 70.62 | 3,250 | |
| LCPOBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 70.47 | 4,026 | |
| GSPOModel=Qwen3-8B-Base2026.04 | 70 | — | |
| GSPO + PIPOModel=Qwen3-8B-Base2026.04 | 70 | — | |
| GRPO (Supervised)Backbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 70 | — | |
| TTRLBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 70 | — | |
| CoDBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 69.84 | 9,052 | |
| TrEffBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 69.84 | 5,166 | |
| L1-ExactBackbone=DeepSeek-R1-Distill-Qwen-7B2025.08 | 69.53 | 3,337 | |
| DAPOModel=Qwen3-8B-Base2026.04 | 68.5 | — | |
| GSPO + PIPOModel=Qwen3-4B-Base2026.04 | 67.5 | — | |
| DAPO + PIPOModel=Qwen3-4B-Base2026.04 | 67.5 | — | |
| L1-ExactBackbone=DeepSeek-R1-Distill-Qwen-1.5B2025.08 | 67.19 | 3,406 | |
| Eurus-2-7B-PRIMEBackbone=Qwen2.5-Math-7B, Supervision Type=With Process Supervision2026.04 | 65 | 4,368 | |
| EntropyBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 65 | — | |
| IntuitorBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 65 | — | |
| GRPO-VPSBackbone=Qwen2.5-Math-7B, Supervision Type=With Process Supervision2026.04 | 64.8 | 3,220 | |
| DrGRPOBackbone=Qwen2.5-Math-7B, Supervision Type=Outcome Supervision Only2026.04 | 64.4 | 3,496 | |
| GRPO w/ Skywork-7BBackbone=Qwen2.5-Math-7B, Supervision Type=With Process Supervision2026.04 | 63.8 | 3,990 | |
| AdaptThinkBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 63.3 | 3,415.2 | |
| GRPO + PIPOModel=Qwen3-4B-Base2026.04 | 62.5 | — | |
| DAPOModel=Qwen3-4B-Base2026.04 | 62.5 | — | |
| EEPOConfiguration=Standard2025.10 | 62.5 | 38.3 | |
| GRPOBackbone=Qwen2.5-Math-7B, Supervision Type=Outcome Supervision Only2026.04 | 62.5 | 3,552 | |
| BaseBackbone=DeepSeek-R1-Distill-Qwen-1.5B2026.04 | 62.5 | — | |
| S-GRPOBackbone=Qwen2.5-Math-7B, Supervision Type=With Process Supervision2026.04 | 61.9 | 3,235 | |
| VanillaBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 61.1 | 8,458.9 | |
| GSPOBackbone=Qwen2.5-Math-7B, Supervision Type=Outcome Supervision Only2026.04 | 60.6 | 3,725 | |
| LaserBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 60.2 | 4,714.9 | |
| GRPOModel=Qwen3-4B-Base2026.04 | 60 | — | |
| GSPOModel=Qwen3-4B-Base2026.04 | 60 | — | |
| GRPOConfiguration=Increased Ent.2025.10 | 60 | 35.9 | |
| AutoThinkBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 57.8 | 4,244.9 | |
| JETBase Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 57.8 | 3,563.4 | |
| GRPOConfiguration=More rollouts.2025.10 | 57.5 | 35.1 | |
| LC-R1Base Model=DeepSeek-R1-Distill-Qwen, Model Scale=1.5B2026.03 | 56.5 | 3,862.9 | |
| GRPOConfiguration=Higher Temp.2025.10 | 55 | 35.2 | |
| GRPOConfiguration=DAPO Clip High.2025.10 | 55 | 32.7 | |
| GRPO-VPSBackbone=Qwen2.5-Math-1.5B, Supervision Type=With Process Supervision2026.04 | 55 | 3,425 | |
| Qwen3-8B-BaseConfiguration=Base Model2025.10 | 52.5 | 29 | |
| GRPO (Supervised)Backbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 52.5 | — | |
| TTRLBackbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 52.5 | — | |
| FREIABackbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 52.5 | — | |
| GRPOConfiguration=Standard2025.10 | 50 | 34.7 | |
| GRPO w/ Skywork-1.5BBackbone=Qwen2.5-Math-1.5B, Supervision Type=With Process Supervision2026.04 | 50 | 3,527 | |
| BaseBackbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 47.5 | — | |
| EntropyBackbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 47.5 | — | |
| IntuitorBackbone=Qwen2.5-Math-1.5B-Instruct2026.04 | 47.5 | — | |
| GRPOBackbone=Qwen2.5-Math-1.5B, Supervision Type=Outcome Supervision Only2026.04 | 46.9 | 3,872 | |
| S-GRPOBackbone=Qwen2.5-Math-1.5B, Supervision Type=With Process Supervision2026.04 | 46.3 | 3,535 | |
| Base ModelModel=Qwen3-8B-Base2026.04 | 45 | — | |
| GSPOBackbone=Qwen2.5-Math-1.5B, Supervision Type=Outcome Supervision Only2026.04 | 45 | 4,044 | |
| DrGRPOBackbone=Qwen2.5-Math-1.5B, Supervision Type=Outcome Supervision Only2026.04 | 44.4 | 3,681 | |
| TTRLBackbone=Qwen2.5-3B-Instruct2026.04 | 40 | — | |
| Base ModelModel=Qwen3-4B-Base2026.04 | 37.5 | — | |
| GRPO (Supervised)Backbone=Qwen2.5-3B-Instruct2026.04 | 37.5 | — |