Mathematical Reasoning on GSM8K, MATH, AMC23, Olympiad, and Minerva (test)
94GSM8K AccuracyR-Few (1%)
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| R-Few (1%)Backbone=Qwen3-8B-Base2026.01 | 94 | 81.6 | 69.3 | 44 | 59.6 | 69.7 | |
| DARCBackbone=Qwen3-8B-Base2026.01 | 94 | 83 | 68.9 | 48.4 | 61.4 | 71.1 | |
| SPICEBackbone=Qwen3-8B-Base2026.01 | 93.8 | 81.4 | 60.9 | 48 | 55.2 | 67.9 | |
| General-ReasonerBackbone=Qwen3-8B-Base, Training Strategy=Supervised Reference2026.01 | 92.7 | 83.4 | 64.8 | 46.3 | 62.6 | 70 | |
| R-ZeroBackbone=Qwen3-8B-Base2026.01 | 92.4 | 80.6 | 62.8 | 43.4 | 58.8 | 67.6 | |
| DEPOTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=782 (1.0×)2026.02 | 92.3 | 63.9 | 63.5 | 28.7 | 25.5 | 54.8 | |
| R-Few (1%)Backbone=Qwen3-4B-Base2026.01 | 92.3 | 77.8 | 52.7 | 42.4 | 52.1 | 63.5 | |
| General-ReasonerBackbone=Qwen3-4B-Base, Training Strategy=Supervised Reference2026.01 | 92.2 | 80.6 | 60 | 47.7 | 57.7 | 67.6 | |
| GRPOTraining Dataset=OR1, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@322026.02 | 92 | 63.3 | 48.9 | 26.4 | 26.2 | 51.4 | |
| Absolute ZeroBackbone=Qwen3-8B-Base2026.01 | 92 | 76.6 | 62.5 | 47.8 | 52.9 | 66.4 | |
| GRPOTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=776 (1.0×)2026.02 | 91.9 | 64.1 | 63.4 | 27.9 | 25 | 54.5 | |
| SPICEBackbone=Qwen3-4B-Base2026.01 | 91.9 | 77.9 | 50.9 | 41.9 | 55.5 | 63.6 | |
| DARCBackbone=Qwen3-4B-Base2026.01 | 91.9 | 77.6 | 60.3 | 45.8 | 57.7 | 66.7 | |
| DEPOTraining Dataset=OR1, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@322026.02 | 91.8 | 64 | 51 | 27.6 | 26.6 | 52.2 | |
| Base ModelBackbone=Qwen3-8B-Base2026.01 | 90.9 | 74.4 | 61.5 | 40.4 | 49.3 | 63.3 | |
| DEPOTraining Dataset=NT, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@322026.02 | 90.8 | 63.2 | 53.2 | 25.6 | 25 | 51.6 | |
| R-ZeroBackbone=Qwen3-4B-Base2026.01 | 90.6 | 74.8 | 48.2 | 40.6 | 51.2 | 61.1 | |
| GRPOTraining Dataset=NT, Backbone=Qwen2.5-7B-Instruct, Evaluation Protocol=Avg@322026.02 | 90.1 | 62.7 | 48.9 | 25.3 | 23.8 | 50.1 | |
| Absolute ZeroBackbone=Qwen3-4B-Base2026.01 | 89.3 | 76.2 | 50 | 41.5 | 41.9 | 59.8 | |
| DARCBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 88 | 62.4 | 31.9 | 30.7 | 43 | 51.2 | |
| SPICEBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 87.3 | 58.4 | 35.2 | 25.6 | 40.8 | 49.5 | |
| Absolute ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 87 | 56.8 | 32.5 | 25.6 | 34.9 | 47.4 | |
| R-ZeroBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 85.2 | 58.4 | 32.5 | 22.6 | 33.1 | 46.4 | |
| DAPOTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=905 (1.7×)2026.02 | 78.5 | 50.1 | 39.3 | 17.8 | 13.1 | 39.8 | |
| DEPO + DAPO w/o Dynamic SamplingTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@322026.02 | 78.3 | 50.6 | 41.7 | 17.5 | 13.3 | 40.3 | |
| PolarisTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=584 (1.1×)2026.02 | 77.1 | 47.3 | 40.8 | 16.4 | 11.8 | 38.7 | |
| DEPOTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=530 (1.0×)2026.02 | 77 | 48.9 | 42.3 | 16.7 | 12.2 | 39.4 | |
| DEPO (w/o ranking loss)Training Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@322026.02 | 76.6 | 48 | 40.9 | 16.3 | 12.1 | 38.8 | |
| GRPOTraining Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@32, GPU Hours ↓=528 (1.0×)2026.02 | 75.6 | 48.1 | 38.4 | 15.8 | 11.4 | 37.9 | |
| DEPO (w/o distill loss)Training Dataset=DAPO-MATH-17K, Backbone=Qwen2.5-1.5B-Instruct, Evaluation Protocol=Avg@322026.02 | 75.2 | 48 | 39 | 15.9 | 12 | 38 | |
| Base ModelBackbone=Qwen3-4B-Base2026.01 | 72.6 | 68.2 | 47.5 | 34.8 | 42.3 | 53.1 | |
| Base ModelBackbone=OctoThinker-8B-Hybrid-Base2026.01 | 68.6 | 44.2 | 27.5 | 16.7 | 22.1 | 35.8 |