Language Modeling on C4 GPT2-355M (val)
21.55PPLNAdam
Evaluation Results
| Method | Links | |
|---|---|---|
| NAdamArchitecture=GPT2-355M, Training Tokens=2.2B2026.05 | 21.55 | |
| Adam+MoLSArchitecture=GPT2-355M, Training Tokens=2.2B2026.05 | 21.66 | |
| LAMBArchitecture=GPT2-355M, Training Tokens=2.2B2026.05 | 21.7 | |
| AdamArchitecture=GPT2-355M, Training Tokens=2.2B2026.05 | 22.19 | |
| MuonArchitecture=GPT2-355M, Training Tokens=2.2B2026.05 | 22.39 |