Language Modeling on C4 Qwen2.5-350M (val)
16.34PerplexityAdam+MoLS
Evaluation Results
| Method | Links | |
|---|---|---|
| Adam+MoLSArchitecture=Qwen2.5-350M, Training Tokens=6.4B2026.05 | 16.34 | |
| NAdamArchitecture=Qwen2.5-350M, Training Tokens=6.4B2026.05 | 16.48 | |
| LAMBArchitecture=Qwen2.5-350M, Training Tokens=6.4B2026.05 | 16.72 | |
| MuonArchitecture=Qwen2.5-350M, Training Tokens=6.4B2026.05 | 16.78 | |
| AdamArchitecture=Qwen2.5-350M, Training Tokens=6.4B2026.05 | 16.87 |