Language Modeling on GPT Pre-training (val)
19.98Validation PerplexityMUON+
Evaluation Results
| Method | Links | |
|---|---|---|
| MUON+Model Scale=GPT-Base2026.02 | 19.98 | |
| NorMuonModel Scale=GPT-Base2026.02 | 21.31 | |
| Turbo-MuonModel Scale=GPT-Base2026.02 | 21.91 | |
| AdaMuonModel Scale=GPT-Base2026.02 | 22.38 | |
| MUON+Model Scale=GPT-Small2026.02 | 27.64 | |
| NorMuonModel Scale=GPT-Small2026.02 | 28.44 | |
| AdaMuonModel Scale=GPT-Small2026.02 | 29.27 | |
| Turbo-MuonModel Scale=GPT-Small2026.02 | 29.69 |