Reasoning on MATH, SuperGPQA, BBEH, and MMLU-Pro
43.55Overall Average ScorePretrainZero
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| PretrainZeroModel Architecture=Qwen3-30B-A3B-MoE-Base, Training Strategy=PretrainZero2025.12 | 43.55 | 58.12 | 36.58 | 14.91 | 64.59 | |
| PretrainZeroModel Architecture=Qwen3-8B-Base, Training Strategy=PretrainZero2025.12 | 42.78 | 57.72 | 34.46 | 14.67 | 64.28 | |
| Random RLPTModel Architecture=Qwen3-8B-Base, Training Strategy=Random RLPT baseline2025.12 | 40.96 | 55.08 | 34.19 | 12.96 | 61.59 | |
| PretrainZeroRLPT / RLVR steps=2000 / 4002025.12 | 40.46 | 53.77 | 33.3 | 13.61 | 61.15 | |
| Random RLPTModel Architecture=Qwen3-30B-A3B-MoE-Base, Training Strategy=Random RLPT baseline2025.12 | 40.38 | 52.62 | 36.33 | 12.99 | 59.57 | |
| PretrainZeroModel Architecture=Qwen3-4B-Base, Training Strategy=PretrainZero2025.12 | 39.61 | 53.13 | 32.28 | 12.68 | 60.37 | |
| PretrainZeroRLPT / RLVR steps=1000 / 4002025.12 | 39.15 | 51.84 | 32.32 | 12.39 | 60.03 | |
| Base ModelModel Architecture=Qwen3-30B-A3B-MoE-Base, Training Strategy=Base Model2025.12 | 38.88 | 52.49 | 33.73 | 10.51 | 58.79 | |
| Random RLPTRLPT / RLVR steps=2000 / 4002025.12 | 38.43 | 51.49 | 30.77 | 12.83 | 58.62 | |
| Base Model (Qwen3-4B-Base)RLPT / RLVR steps=– / 4002025.12 | 37.9 | 50.96 | 30.26 | 11.59 | 58.8 | |
| Base ModelModel Architecture=Qwen3-8B-Base, Training Strategy=Base Model2025.12 | 37.07 | 47.48 | 31.12 | 10.49 | 59.19 | |
| Random RLPTModel Architecture=Qwen3-4B-Base, Training Strategy=Random RLPT baseline2025.12 | 35.41 | 47.87 | 29.1 | 9.45 | 55.21 | |
| Base ModelModel Architecture=Qwen3-4B-Base, Training Strategy=Base Model2025.12 | 32.36 | 42.53 | 26.32 | 8.67 | 51.94 | |
| Supervised FTModel Architecture=Qwen3-8B-Base, Training Strategy=Supervised Fine-Tuning2025.12 | 26.8 | 19.23 | 29.02 | 11.17 | 47.78 | |
| Supervised FTModel Architecture=Qwen3-4B-Base, Training Strategy=Supervised Fine-Tuning2025.12 | 24.27 | 15.55 | 26.38 | 12.28 | 42.88 | |
| PretrainZeroModel Architecture=SmolLM3-3B-Base, Training Strategy=PretrainZero2025.12 | 23.41 | 38.03 | 19.44 | 3.78 | 32.41 | |
| Random RLPTModel Architecture=SmolLM3-3B-Base, Training Strategy=Random RLPT baseline2025.12 | 20.25 | 35.95 | 14.48 | 7.85 | 22.74 | |
| Base ModelModel Architecture=SmolLM3-3B-Base, Training Strategy=Base Model2025.12 | 16.23 | 32.31 | 12.62 | 3.32 | 16.66 | |
| Continue PTModel Architecture=Qwen3-4B-Base, Training Strategy=Continue Pre-Training2025.12 | 15.89 | 24.65 | 9.67 | 0.04 | 29.21 | |
| Continue PTModel Architecture=Qwen3-8B-Base, Training Strategy=Continue Pre-Training2025.12 | 12.32 | 27.78 | 9.94 | 0.04 | 11.51 |