Scaling Law Fitting on Enoki (train)
0.405Scaling Parameter bAdamW
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| AdamWOptimizer=AdamW2026.02 | 0.405 | 0.211 | 2.68 | 0.03 | 1 | |
| Dana-Star-MK4Optimizer=Dana-Star-MK4, kappa=0.852026.02 | 0.404 | 0.217 | 2.64 | 0.03 | 0.9999 | |
| ADanaOptimizer=ADana, kappa=0.852026.02 | 0.404 | 0.217 | 2.64 | 0.03 | 1 | |
| MuonOptimizer=Muon2026.02 | 0.404 | 0.217 | 2.65 | 0.029 | 0.9999 | |
| AdemamixOptimizer=Ademamix, Weight Decay=dec. WD2026.02 | 0.404 | 0.218 | 2.64 | 0.029 | 1 |