Mathematical Reasoning on Mathematical Reasoning Benchmarks (test)
53.72Average ScoreR-Zero
Evaluation results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| R-ZeroBackbone=Qwen3-8B-Base, Training Setting=our method2025.08 | 53.72 | 61.67 | 60.66 | 82 | 94.09 | 48.89 | 13.3 | 15.4 | |
| Absolute ZeroBackbone=Qwen3-8B-Base2025.08 | 52.68 | 57.89 | 57.9 | 76.6 | 92 | 47.8 | 18.2 | 18.4 | |
| R-Zero (snowflake challenger baseline)Backbone=Qwen3-8B-Base, Challenger=Untrained2025.08 | 52.1 | 60.7 | 57.72 | 81.6 | 92.56 | 46.44 | 11.6 | 14.1 | |
| R-ZeroBackbone=Qwen3-4B-Base, Training Setting=our method2025.08 | 49.93 | 57.27 | 52.94 | 79.6 | 92.12 | 44.59 | 9.6 | 13.4 | |
| Base Model (w/o training)Backbone=Qwen3-8B-Base2025.08 | 48.64 | 51.95 | 50 | 78 | 89.08 | 44.74 | 12.1 | 14.6 | |
| Absolute ZeroBackbone=Qwen3-4B-Base2025.08 | 46.42 | 52.45 | 41.96 | 76.2 | 89.34 | 42.56 | 10.2 | 12.2 | |
| R-Zero (snowflake challenger baseline)Backbone=Qwen3-4B-Base, Challenger=Untrained2025.08 | 45.01 | 45 | 45.22 | 72.8 | 87.87 | 41.19 | 10.2 | 12.8 | |
| Base Model (w/o training)Backbone=Qwen3-4B-Base2025.08 | 42.57 | 45.7 | 38.24 | 68.2 | 87.79 | 41.04 | 10.3 | 6.7 | |
| R-ZeroBackbone=OctoThinker-8B, Training Setting=our method2025.08 | 38.52 | 34.03 | 48.22 | 68.8 | 87.19 | 27.56 | 0.42 | 3.44 | |
| R-Zero (snowflake challenger baseline)Backbone=OctoThinker-8B, Challenger=Untrained2025.08 | 36.98 | 29.3 | 42.28 | 66.2 | 88.1 | 27.56 | 1.04 | 4.38 | |
| Absolute ZeroBackbone=OctoThinker-8B2025.08 | 36.6 | 32.5 | 44.9 | 62.8 | 87 | 25.6 | 3.3 | 0.1 | |
| Base Model (w/o training)Backbone=OctoThinker-8B2025.08 | 36.41 | 32.11 | 41.91 | 65.2 | 86.96 | 26.52 | 1.56 | 0.62 | |
| R-ZeroBackbone=OctoThinker-3B, Training Setting=our method2025.08 | 29.32 | 27.03 | 27.57 | 54.2 | 74.98 | 18.22 | 3.23 | 0 | |
| R-Zero (snowflake challenger baseline)Backbone=OctoThinker-3B, Challenger=Untrained2025.08 | 27.51 | 20.19 | 24.63 | 54.6 | 74.98 | 15.7 | 0.1 | 2.4 | |
| Absolute ZeroBackbone=OctoThinker-3B2025.08 | 27.23 | 22.5 | 22.7 | 53.2 | 75.8 | 13.2 | 0.5 | 2.7 | |
| Base Model (w/o training)Backbone=OctoThinker-3B2025.08 | 26.64 | 17.19 | 24.26 | 55 | 73.69 | 16.15 | 0.21 | 0 |