Mathematical Reasoning on Minerva (Acc, Avg Tok, SuperBPE, Deltas)
37.4AccuracyQWEN-3-30B-A3B Baseline
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| QWEN-3-30B-A3B BaselineBase Model=QWEN-3-30B-A3B, Training Condition=Baseline2026.04 | 37.4 | 3,595 | — | — | — | |
| Shorthand for Thought (SFT)Base Model=QWEN-3-30B-A3B, Training Condition=SFT2026.04 | 37 | 3,571 | 9.7 | -1.1 | -0.66 | |
| QwQ-32B BaselineBase Model=QwQ-32B, Training Condition=Baseline2026.04 | 34 | 5,324 | — | — | — | |
| Shorthand for Thought (SFT)Base Model=QwQ-32B, Training Condition=SFT2026.04 | 33.5 | 5,418 | 8.15 | -1.5 | -1.1 | |
| DeepSeek-R1-Llama-70B-Distill BaselineBase Model=DeepSeek-R1-Llama-70B-Distill, Training Condition=Baseline2026.04 | 31.6 | 3,239 | — | — | — | |
| Shorthand for Thought (SFT)Base Model=DeepSeek-R1-Llama-70B-Distill, Training Condition=SFT2026.04 | 31.4 | 3,126 | 9.54 | -0.6 | -3.5 |