List your company
ResearchFormerly WizWand

Language Modeling on WikiText-103 (Throughput and Speedup)

159,000Throughput (tokens/s)AdamW
6,74446,27285,800125,328Mar 18, 2026
Updated 7mo ago

Evaluation results

MethodLinks
AdamWGPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03159,000—2.2
AdamWGPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.03142,900—4.25
AdamWGPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.03134,000—1.5
MUD1GPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.03116,2001.3—
MUD1GPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.03114,6001.58—
MuonGPU=GH200, (B, T)=(12, 4096), Model Architecture=GPT-2 medium2026.0389,400——
MuonGPU=GH200, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0372,400——
MUD1GPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0368,5002.04—
AdamWGPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0355,200—2.09
AdamWGPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0349,300—3.91
MUD1GPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0343,4501.65—
MuonGPU=GH200, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0333,600——
AdamWGPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0330,000—1.32
MUD1GPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0329,1002.31—
AdamWGPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0328,100—1.89
MUD1GPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0327,8001.22—
MuonGPU=A100, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0326,400——
MUD1GPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0323,0001.54—
MuonGPU=MI250, (B, T)=(12, 2048), Model Architecture=GPT-2 medium2026.0322,700——
MuonGPU=MI250, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0314,900——
MuonGPU=A100, (B, T)=(8, 1024), Model Architecture=GPT-2 medium2026.0312,600——