Mathematical Reasoning on AIME 2025 (Accuracy, Average output tokens)
36.7AccuracySAPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| SAPOModel=Qwen3-8B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 36.7 | — | |
| VESPOModel=Qwen3-30B-A3B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 34.2 | — | |
| VESPOModel=Qwen3-8B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 33.5 | — | |
| GSPOModel=Qwen3-8B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 28.8 | — | |
| GRPOModel=Qwen3-30B-A3B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 28.2 | — | |
| GRPOModel=Qwen3-8B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 27.4 | — | |
| GSPOModel=Qwen3-30B-A3B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 24.6 | — | |
| SAPOModel=Qwen3-30B-A3B-Base, gbs/mbs=8, avg@k protocol=avg@322026.02 | 21.4 | — | |
| Kimi K2.52026.02 | 0.961 | 25,000 | |
| Gemini-3.0version=Pro2026.02 | 0.95 | 15,000 | |
| Kimi K2mode=Thinking2026.02 | 0.945 | 30,000 | |
| DeepSeek-V3.2mode=Thinking2026.02 | 0.931 | 16,000 | |
| Qwen3-8Bstrategy=Full COT2025.08 | 0.7692 | 19,902.43 | |
| Qwen3-8Bstrategy=step-entropy pruning, pruning_ratio=80%2025.08 | 0.76 | 11,716.63 | |
| SAPOModel=Llama-3.2-3B-Instruct, gbs/mbs=8, avg@k protocol=avg@322026.02 | 0.7 | — | |
| QwQ-32Bstrategy=Full COT2025.08 | 0.6667 | 23,711.6 | |
| QwQ-32Bstrategy=step-entropy pruning, pruning_ratio=80%2025.08 | 0.6552 | 13,434.17 | |
| VESPOModel=Llama-3.2-3B-Instruct, gbs/mbs=8, avg@k protocol=avg@322026.02 | 0.6 | — | |
| DeepSeek-R1-14Bstrategy=Full COT2025.08 | 0.5862 | 18,000.1 | |
| DeepSeek-R1-14Bstrategy=step-entropy pruning, pruning_ratio=80%2025.08 | 0.5172 | 10,842.07 | |
| GRPOModel=Llama-3.2-3B-Instruct, gbs/mbs=8, avg@k protocol=avg@322026.02 | 0.5 | — | |
| DeepSeek-R1-7Bstrategy=Full COT2025.08 | 0.3571 | 18,203.23 | |
| DeepSeek-R1-7Bstrategy=step-entropy pruning, pruning_ratio=80%2025.08 | 0.3571 | 11,471.17 | |
| GSPOModel=Llama-3.2-3B-Instruct, gbs/mbs=8, avg@k protocol=avg@322026.02 | 0.2 | — |