Language Modeling on WikiText-103 (Throughput and Speedup)
159,000Throughput (tokens/s)AdamW
Evaluation Results
| Method | Links | |||
|---|---|---|---|---|
| AdamWGPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 159,000 | — | 2.2 | |
| AdamWGPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 142,900 | — | 4.25 | |
| AdamWGPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.03 | 134,000 | — | 1.5 | |
| MUD1GPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.03 | 116,200 | 1.3 | — | |
| MUD1GPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 114,600 | 1.58 | — | |
| MuonGPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.03 | 89,400 | — | — | |
| MuonGPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 72,400 | — | — | |
| MUD1GPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 68,500 | 2.04 | — | |
| AdamWGPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 55,200 | — | 2.09 | |
| AdamWGPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 49,300 | — | 3.91 | |
| MUD1GPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 43,450 | 1.65 | — | |
| MuonGPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 33,600 | — | — | |
| AdamWGPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 30,000 | — | 1.32 | |
| MUD1GPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 29,100 | 2.31 | — | |
| AdamWGPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 28,100 | — | 1.89 | |
| MUD1GPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 27,800 | 1.22 | — | |
| MuonGPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 26,400 | — | — | |
| MUD1GPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 23,000 | 1.54 | — | |
| MuonGPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03 | 22,700 | — | — | |
| MuonGPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 14,900 | — | — | |
| MuonGPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03 | 12,600 | — | — |