Mathematical Reasoning on Minerva (Pass@1, #Tk)
40.14Pass@1GRPO
Evaluation results
| Method | Links | ||
|---|---|---|---|
| GRPOBackbone=Qwen3-8B2026.02 | 40.14 | 1,549 | |
| LACONICBackbone=Qwen3-8B, Target Tokens=10002026.02 | 38.46 | 783 | |
| L1-MaxBackbone=Qwen3-8B2026.02 | 37.59 | 759 | |
| L1-ExactBackbone=Qwen3-8B2026.02 | 37.5 | 800 | |
| FROSTBackbone=GPT-oss-20B2026.01 | 27.94 | 691.71 | |
| FROSTBackbone=Phi-4-Reasoning2026.01 | 27.16 | 401.19 | |
| DRPBackbone=GPT-oss-20B2026.01 | 27.15 | 1,885.15 | |
| DRPBackbone=Phi-4-Reasoning2026.01 | 27.01 | 1,289.5 | |
| TALEBackbone=GPT-oss-20B2026.01 | 27 | 3,262.47 | |
| TALEBackbone=Phi-4-Reasoning2026.01 | 26.27 | 2,093.17 | |
| ThinkLessBackbone=GPT-oss-20B2026.01 | 25.8 | 1,220 | |
| ThinkLessBackbone=Phi-4-Reasoning2026.01 | 25.75 | 1,708.7 | |
| BaseBackbone=GPT-oss-20B2026.01 | 25.74 | 1,586.95 | |
| SelfBudgeterBackbone=GPT-oss-20B2026.01 | 25.5 | 1,298 | |
| BaseBackbone=Phi-4-Reasoning2026.01 | 25 | 1,898.86 | |
| SelfBudgeterBackbone=Phi-4-Reasoning2026.01 | 23.57 | 2,618.23 | |
| FROSTBackbone=Magistral-Small-1.12026.01 | 15.51 | 109.23 | |
| TALEBackbone=Magistral-Small-1.12026.01 | 15.44 | 748.18 | |
| ThinkLessBackbone=Magistral-Small-1.12026.01 | 14.5 | 1,220 | |
| SelfBudgeterBackbone=Magistral-Small-1.12026.01 | 13 | 1,298 | |
| DRPBackbone=Magistral-Small-1.12026.01 | 11.2 | 1,604.22 | |
| BaseBackbone=Magistral-Small-1.12026.01 | 6.99 | 1,288.04 |