Mathematical Reasoning on MATH500, AIME24, Minerva-Math, AMC23
90.6MATH500 ScoreDVPO
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| DVPOBackbone=Qwen3-8B, Training Steps=5002025.12 | 90.6 | 56.67 | 31.99 | 86.67 | 66.48 | 39.63 | |
| Reinforce++Backbone=Qwen3-8B, Training Steps=5002025.12 | 89.4 | 50 | 30.51 | 83.33 | 63.31 | 37.72 | |
| Dr.GRPOBackbone=Qwen3-8B, Training Steps=5002025.12 | 88.6 | 48.33 | 27.21 | 80 | 61.04 | 36.25 | |
| BaseBackbone=Qwen3-8B, Training Steps=5002025.12 | 87.4 | 41.67 | 28.68 | 75.83 | 58.4 | 34.64 | |
| Robust BellmanBackbone=Qwen3-8B, Implementation=PPO based on robust Bellman operator, Training Steps=5002025.12 | 87.4 | 45 | 27.21 | 83.33 | 60.74 | 36.23 | |
| PPOBackbone=Qwen3-8B, Training Steps=5002025.12 | 87 | 51.67 | 29.41 | 85 | 63.27 | 37.34 | |
| GRPOBackbone=Qwen3-8B, Training Steps=5002025.12 | 86.8 | 45 | 26.84 | 82.5 | 60.29 | 35.77 | |
| λ-GRPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 85 | 40 | 22.06 | 70.83 | 54.47 | 32.33 | |
| DFPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 85 | 40 | 24.63 | 78.33 | 56.99 | 33.75 | |
| BaseBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 84.8 | 35 | 22.43 | 75 | 54.31 | 32.13 | |
| Reinforce++Backbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 84.6 | 38.33 | 23.16 | 75.83 | 55.48 | 32.75 | |
| Robust BellmanBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 84.6 | 31.67 | 22.79 | 75 | 53.52 | 31.72 | |
| Dr. GRPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 84 | 35 | 22.79 | 76.67 | 54.62 | 32.18 | |
| FlowRLBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 84 | 36.67 | 22.06 | 75.83 | 54.64 | 32.3 | |
| PPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 83.8 | 33.33 | 22.43 | 79.17 | 54.68 | 32.45 | |
| BAPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 82.4 | 30 | 19.85 | 64.17 | 49.11 | 29.12 | |
| GRPOBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 82.2 | 33.33 | 21.32 | 77.5 | 53.59 | 31.7 | |
| KTAEBackbone=Qwen3-1.7B, Supervision Type=Noisy supervision2026.02 | 82.2 | 26.67 | 23.9 | 64.17 | 49.24 | 29.41 |