Depth-Learning Rate scaling analysis on CIFAR-100
-1.355Fitted Depth Exponent (alpha)ResNet
Evaluation results
| Method | Links | |
|---|---|---|
| ResNetOptimizer=SGD without momentum, Activation=ReLU, Normalization=None, Dropout=None, Initialization=Scaled He fan-in2026.02 | -1.355 | |
| ViTOptimizer=SGD without momentum, Normalization=Post-LN, Patch Size=4, Embedding Dimension=384, Attention Heads=62026.02 | -1.371 | |
| CNNOptimizer=SGD without momentum, Activation=ReLU, Normalization=None, Dropout=None2026.02 | -1.392 | |
| Theory2026.02 | -1.5 |