Language Modeling on GPT-Base (val)
19.96Validation PerplexityMUON²
Evaluation Results
| Method | Links | |
|---|---|---|
| MUON²NS Steps=Ns = 52026.04 | 19.96 | |
| MUON²NS Steps=Ns = 32026.04 | 20.39 | |
| PolarExpressNS Steps=Ns = 52026.04 | 21.16 | |
| NorMuonNS Steps=Ns = 52026.04 | 21.27 | |
| MUONNS Steps=Ns = 52026.04 | 21.47 | |
| Turbo-MuonNS Steps=Ns = 52026.04 | 21.93 | |
| AdaMuonNS Steps=Ns = 52026.04 | 22.42 | |
| PolarExpressNS Steps=Ns = 32026.04 | 22.74 | |
| NorMuonNS Steps=Ns = 32026.04 | 23.33 | |
| Turbo-MuonNS Steps=Ns = 32026.04 | 23.46 | |
| MUONNS Steps=Ns = 32026.04 | 24.69 | |
| AdaMuonNS Steps=Ns = 32026.04 | 26.07 |