Zero-shot Language Understanding on Qwen3-0.6B Evaluation Suite (test)
68.64Accuracy (AE)AMO
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| AMOOptimizer=AMO, Step=6000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 68.64 | 31.97 | 86.8 | 26.04 | 6.23 | 39.05 | 25.4 | 69.7 | 33.78 | 54.96 | 20.31 | 17.6 | 40.04 | |
| AMOOptimizer=AMO, Step=8580, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 68.56 | 31.74 | 88.2 | 25 | 7.99 | 39.1 | 26 | 70.18 | 34.45 | 54.56 | 20.23 | 17.68 | 40.31 | |
| AMOOptimizer=AMO, Step=8000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 68.39 | 30.29 | 86.6 | 25.06 | 7.77 | 39.18 | 26.4 | 70.38 | 33.88 | 54.96 | 21.64 | 17.19 | 40.15 | |
| Muon-PEOptimizer=Muon-PE, Step=6000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 68.32 | 32.08 | 87 | 25 | 6.18 | 39 | 25 | 70.84 | 33 | 54.22 | 19.74 | 16.46 | 39.74 | |
| Muon-PEOptimizer=Muon-PE, Step=8000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 68.3 | 31 | 87 | 24.63 | 5.54 | 39.11 | 26 | 72.09 | 33.78 | 55 | 20.72 | 17.74 | 40.08 | |
| Muon-KJOptimizer=Muon-KJ, Step=6000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 67.47 | 31.4 | 87.1 | 26.51 | 6.99 | 39.14 | 24.4 | 69.59 | 32.73 | 54.06 | 20.15 | 17.86 | 39.78 | |
| Muon-KJOptimizer=Muon-KJ, Step=8000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 66.88 | 31.74 | 86.8 | 26.21 | 7.25 | 39.07 | 25 | 70.35 | 33.59 | 53.67 | 21.62 | 17.71 | 39.99 | |
| Muon-PEOptimizer=Muon-PE, Step=2000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 66.51 | 29.61 | 86 | 25 | 7.21 | 38.63 | 22 | 70.24 | 33.88 | 52.56 | 19.77 | 16.43 | 38.99 | |
| Muon-KJOptimizer=Muon-KJ, Step=8580, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 66.5 | 31.57 | 86.6 | 26.2 | 7.77 | 39.12 | 25 | 70.08 | 33.4 | 54.22 | 21.7 | 17.94 | 40.01 | |
| Muon-PEOptimizer=Muon-PE, Step=8580, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 66 | 32 | 88 | 25.6 | 7.99 | 39.1 | 25.6 | 70.5 | 34 | 53.43 | 21.29 | 18.3 | 40.15 | |
| AMOOptimizer=AMO, Step=4000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 65.53 | 31.23 | 86 | 24.55 | 5.72 | 37.74 | 23.4 | 70.02 | 33.68 | 53.99 | 21.05 | 17.72 | 39.22 | |
| Muon-KJOptimizer=Muon-KJ, Step=4000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 65.32 | 29.61 | 86.4 | 24.63 | 6.46 | 37.94 | 23.6 | 68.66 | 32.54 | 53.43 | 19.98 | 17.96 | 38.88 | |
| Muon-PEOptimizer=Muon-PE, Step=4000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 65.17 | 31.23 | 86 | 24.89 | 7.8 | 37.94 | 24 | 68 | 31.53 | 54.06 | 19.77 | 17.72 | 39.01 | |
| Muon-KJOptimizer=Muon-KJ, Step=2000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 64.94 | 28.41 | 87 | 24.2 | 6.59 | 37.05 | 23.2 | 68.5 | 33.11 | 55.01 | 20.72 | 17.6 | 38.86 | |
| AMOOptimizer=AMO, Step=2000, Backbone=Qwen3-0.6B, Pre-training tokens=60B + 36B2026.05 | 63.72 | 29.27 | 86.2 | 24.79 | 7.01 | 36.52 | 24.2 | 68.99 | 33.4 | 54.62 | 22.44 | 18.07 | 39.1 |