Language Modeling on MiniPile (val)
40.83Validation PerplexityLiMuon
Evaluation Results
| Method | Links | |
|---|---|---|
| LiMuonMemory (GB)=55.15, rank=full rank2025.09 | 40.83 | |
| LiMuonMemory (GB)=54.21, rank=162025.09 | 46.77 | |
| LiMuonMemory (GB)=52.55, rank=82025.09 | 54.87 | |
| MuonMemory (GB)=54.142025.09 | 67.6 | |
| Muon++Memory (GB)=54.302025.09 | 82.26 | |
| SUMOMemory (GB)=52.732025.09 | 88.73 | |
| LiMuonMemory (GB)=50.86, rank=42025.09 | 92.06 | |
| AdamWMemory (GB)=55.262025.09 | 113.25 | |
| LionMemory (GB)=54.082025.09 | 113.92 | |
| AdamMemory (GB)=55.242025.09 | 181.27 |