Image Classification on CIFAR-100 (test) (Accuracy, Train Loss, and GELU Delta)
69.57Accuracy (%)GELU
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| GELUBackbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 69.57 | 0.36 | 0.0368 | — | |
| GELU (tanh)Backbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 69.2 | 0.27 | 0.0398 | 0.37 | |
| ReLUBackbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 68.58 | 0.35 | 0.0824 | 0.99 | |
| SiLU/SwishBackbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 68.2 | 0.15 | 0.0411 | 1.37 | |
| GEM (N = 1)Backbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 67.45 | 0.96 | 0.0739 | 2.12 | |
| GEM (N = 1)Backbone=ResNet-20, Epochs=200, Seeds=32026.04 | 67.26 | 0.28 | 0.2999 | 0.28 | |
| GELU (tanh)Backbone=ResNet-20, Epochs=200, Seeds=32026.04 | 67.11 | 0.21 | 0.2536 | 0.43 | |
| GEM (N = 2)Backbone=ResNet-20, Epochs=200, Seeds=32026.04 | 66.79 | 0.5 | 0.3819 | 0.75 | |
| GEM (N = 2)Backbone=ResNet-56, Parameters=861K params, Epochs=200, Seeds=32026.04 | 63.47 | 2.11 | 0.2925 | 6.1 |