Image Classification on CIFAR-100 (Weight Norm and AutoAtt Metrics)
13.41L2 Norm (||W||2)Adaptive Weight Decay (AWD)
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Adaptive Weight Decay (AWD)lambda_awd=0.022, Optimizer=SGD2022.09 | 13.41 | 61.39 | 27.15 | 1.51 | |
| Non-adaptive Weight Decaylambda_wd=0.0005, Optimizer=SGD2022.09 | 51.32 | 60.15 | 22.53 | 0.67 |