Learned Optimization on Optimizer Benchmark Suite (Cifar-10, IN32, IN64, LM, ViT) - OoD (Large) (test)
1.8Average Rank (1k steps)μLO_M
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| μLO_MWidth=2048, Steps=5000, Seeds=52024.05 | 1.8 | 1.8 | 1.8 | |
| μVeLO_MWidth=2048, Steps=5000, Seeds=52024.05 | 2.6 | 2.4 | 2.4 | |
| AdamWWidth=2048, Steps=5000, Seeds=52024.05 | 3 | 2.8 | 2.6 | |
| μAdamWidth=2048, Steps=5000, Seeds=52024.05 | 3.4 | 3 | 3.2 | |
| VeLO_MWidth=2048, Steps=5000, Seeds=52024.05 | 4.6 | 5.4 | 6 | |
| LO_MWidth=2048, Steps=5000, Seeds=52024.05 | 5.6 | 5.6 | 5 |