Language Modeling on GPT Small (val)
27.95Validation PerplexityMUON²
Evaluation Results
| Method | Links | |
|---|---|---|
| MUON²NS Steps=Ns = 52026.04 | 27.95 | |
| MUON²NS Steps=Ns = 32026.04 | 28.12 | |
| NorMuonNS Steps=Ns = 52026.04 | 28.4 | |
| AdaMuonNS Steps=Ns = 52026.04 | 29.3 | |
| PolarExpressNS Steps=Ns = 52026.04 | 29.42 | |
| MUONNS Steps=Ns = 52026.04 | 29.51 | |
| Turbo-MuonNS Steps=Ns = 52026.04 | 29.66 | |
| Turbo-MuonNS Steps=Ns = 32026.04 | 29.7 | |
| PolarExpressNS Steps=Ns = 32026.04 | 30.01 | |
| NorMuonNS Steps=Ns = 32026.04 | 30.35 | |
| AdaMuonNS Steps=Ns = 32026.04 | 31.2 | |
| MUONNS Steps=Ns = 32026.04 | 32.7 |