Masked Language Modeling on omg prot50 (val)
78Wall-clock Time (17.5 Target PPL)MUD1
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| MUD1Hardware=AMD MI300X, Total training steps=5,000 steps, Model Architecture=ESM-2 150M2026.03 | 78 | 110 | 126 | 16.855 | 636 | 1.81 | |
| MUD2Hardware=AMD MI300X, Total training steps=5,000 steps, Model Architecture=ESM-2 150M2026.03 | 78.2 | 86.2 | 131.3 | 16.795 | 710 | 1.81 | |
| MuonHardware=AMD MI300X, Total training steps=5,000 steps, Model Architecture=ESM-2 150M2026.03 | 127.8 | 188.4 | 236.7 | 16.84 | 997 | 1.1 | |
| AdamWHardware=AMD MI300X, Total training steps=5,000 steps, Model Architecture=ESM-2 150M2026.03 | 141.2 | 245.1 | 392.8 | 17.176 | 466 | — |