Mathematical Reasoning on DeepScaleR (test)
39.2Greedy SuccessAvg Probability
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| Avg ProbabilityBackbone=Qwen 3B, Group size (G)=322026.02 | 39.2 | 28.79 | -0.59 | -0.85 | 2.33 | 460 | — | |
| ProbabilityBackbone=Qwen 3B, Group size (G)=322026.02 | 38.68 | 36.83 | -1.68 | -2.73 | 15.28 | 481.15 | — | |
| Base RLBackbone=Qwen 3B, Group size (G)=322026.02 | 38.3 | 38.56 | -2.38 | -3.55 | 34.85 | 453.69 | — | |
| Avg LogprobBackbone=Qwen 3B, Group size (G)=322026.02 | 38.12 | 25.82 | -0.57 | -0.7 | 2.02 | 463.46 | — | |
| Log-probBackbone=Qwen 3B, Group size (G)=322026.02 | 37.92 | 25.79 | -0.56 | -0.59 | 1.8 | 532.79 | — | |
| Log-probBackbone=Llama 3B, Group size (G)=322026.02 | 32.4 | 17.97 | -0.79 | -0.83 | 2.3 | 437.11 | — | |
| Avg LogprobBackbone=Llama 3B, Group size (G)=322026.02 | 32.2 | 18.32 | -0.8 | -0.84 | 2.32 | 371.88 | — | |
| ProbabilityBackbone=Llama 3B, Group size (G)=322026.02 | 31.75 | 28.47 | -1.63 | -2.32 | 10.28 | 340.01 | — | |
| Avg ProbabilityBackbone=Llama 3B, Group size (G)=322026.02 | 30.75 | 21.71 | -0.85 | -1.31 | 3.71 | 335.89 | — | |
| Base RLBackbone=Llama 3B, Group size (G)=322026.02 | 28.55 | 28.54 | -2.01 | -2.77 | 16.89 | 294.52 | — | |
| Base modelBackbone=Qwen 3B, Group size (G)=322026.02 | 14.02 | 11 | — | -2.57 | 13.12 | 218.12 | — | |
| SFT (no CoT)Backbone=Qwen 3B, Group size (G)=322026.02 | 11.52 | 6.48 | -0.76 | -0.76 | 2.14 | 5 | — | |
| Base modelBackbone=Llama 3B, Group size (G)=322026.02 | 10.05 | 5.2 | — | -4.92 | 137.25 | 354.91 | — | |
| SFT (no CoT)Backbone=Llama 3B, Group size (G)=322026.02 | 9.88 | 5.14 | -1.01 | -1.01 | 2.74 | 5 | — | |
| AdaBackModel Size=1B2025.06 | — | — | — | — | — | — | 9 | |
| AdaBackModel Size=3B2025.06 | — | — | — | — | — | — | 10.6 | |
| Base+RLModel Size=1B2025.06 | — | — | — | — | — | — | 6.8 | |
| Base+RLModel Size=3B2025.06 | — | — | — | — | — | — | 6.6 | |
| SFT+AdaBackModel Size=1B2025.06 | — | — | — | — | — | — | 9.5 | |
| SFT+AdaBackModel Size=3B2025.06 | — | — | — | — | — | — | 12.5 | |
| SFT+RLModel Size=1B2025.06 | — | — | — | — | — | — | 7.1 | |
| SFT+RLModel Size=3B2025.06 | — | — | — | — | — | — | 9.1 |