Learned Optimization on Optimizer Benchmark Suite (Cifar-10, IN32, IN64, LM, ViT) - OoD (XXL) (test)
1.8Average Rank (1k Steps)μVeLO_M
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| μVeLO_MWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 1.8 | 2.4 | 2 | |
| μLO_MWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 2 | 1.2 | 1.6 | |
| μAdamWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 2.2 | 2.4 | 2.6 | |
| AdamWWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 4.4 | 4 | 3.8 | |
| VeLO_MWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 5 | 5.8 | 5.8 | |
| LO_MWidth=XXL (largest size for each task), Steps=5000, Seeds=52024.05 | 5.6 | 5.2 | 5.2 |