Depth-Learning Rate scaling analysis on CIFAR-10
-1.339Fitted Depth Exponent (alpha)CNN
Evaluation results
| Method | Links | |
|---|---|---|
| CNNOptimizer=SGD without momentum, Activation=ReLU, Normalization=None, Dropout=None2026.02 | -1.339 | |
| ResNetOptimizer=SGD without momentum, Activation=ReLU, Normalization=None, Dropout=None, Initialization=Scaled He fan-in2026.02 | -1.435 | |
| ViTOptimizer=SGD without momentum, Normalization=Post-LN, Patch Size=4, Embedding Dimension=384, Attention Heads=62026.02 | -1.441 | |
| Theory2026.02 | -1.5 |