General Intelligence on AGI Eval
40.2AGI Eval ScoreWarmup-Stable-Only (WSO)
Evaluation results
| Method | Links | |
|---|---|---|
| Warmup-Stable-Only (WSO)Model=8B, αpre=1.0, αmid=1.02026.03 | 40.2 | |
| WSDModel=8B, αpre=1.0, αmid=0.02026.03 | 40 | |
| WSDModel=8B, αpre=0.1, αmid=0.02026.03 | 36.4 | |
| WSDModel=8B, αpre=0.1, αmid=1.02026.03 | 36.3 | |
| CosineModel=8B, αpre=0.1, αmid=1.02026.03 | 34.6 | |
| LinearModel=8B, αpre=0.1, αmid=0.02026.03 | 33.6 | |
| LinearModel=8B, αpre=0.1, αmid=1.02026.03 | 33 | |
| CosineModel=8B, αpre=0.1, αmid=0.02026.03 | 32.3 | |
| Warmup-Stable-Only (WSO)Model=1B, Pre-training Scheduler=Warmup-Stable-Only (WSO), alpha_pre=1.0, alpha_mid=1.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 24.1 | |
| CosineModel=1B, Pre-training Scheduler=Cosine, alpha_pre=0.1, alpha_mid=0.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 23.2 | |
| WSDModel=1B, αpre=0.1, αmid=1.02026.03 | 23.1 | |
| CosineModel=1B, Pre-training Scheduler=Cosine, alpha_pre=0.1, alpha_mid=1.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 23 | |
| LinearModel=1B, αpre=0.1, αmid=1.02026.03 | 22.8 | |
| LinearModel=1B, Pre-training Scheduler=Linear, alpha_pre=0.1, alpha_mid=0.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 22.6 | |
| LinearModel=1B, Pre-training Scheduler=Linear, alpha_pre=0.1, alpha_mid=1.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 22.6 | |
| CosineModel=1B, αpre=0.1, αmid=1.02026.03 | 22.6 | |
| WSDModel=1B, αpre=0.1, αmid=0.02026.03 | 22.4 | |
| CosineModel=1B, αpre=0.1, αmid=0.02026.03 | 22.1 | |
| LinearModel=1B, αpre=0.1, αmid=0.02026.03 | 22.1 | |
| Warmup-Stable-Only (WSO)Model=1B, αpre=1.0, αmid=1.02026.03 | 21.8 | |
| WSDModel=1B, αpre=1.0, αmid=0.02026.03 | 21.5 | |
| WSDModel=1B, Pre-training Scheduler=WSD, alpha_pre=0.1, alpha_mid=0.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 21.4 | |
| WSDModel=1B, Pre-training Scheduler=WSD, alpha_pre=1.0, alpha_mid=0.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 21.3 | |
| WSDModel=1B, Pre-training Scheduler=WSD, alpha_pre=0.1, alpha_mid=1.0, Pre-training tokens=2T, Mid-training tokens=500B, Training Stage=Supervised fine-tuned (SFT)2026.03 | 21.1 |