Model Sparsification on MNIST LeNet-300-100 (test)
1.41Test ErrorVARDROP+LR
Evaluation results
| Method | Links | ||
|---|---|---|---|
| VARDROP+LRλ=1/60,000, Learning Rate=10^-32021.06 | 1.41 | 0.088 | |
| VARDROPλ=1/60,000, Learning Rate=10^-32021.06 | 1.54 | 0.595 | |
| η-TRICKλ=10^-3, Learning Rate=10^-32021.06 | 2.16 | 0.051 | |
| ADATIKHONOVλ=10^-3, Learning Rate=10^-42021.06 | 2.88 | 0.018 | |
| LOGSUMλ=10^-3, Learning Rate=10^-42021.06 | 2.93 | 0.019 | |
| ADAPROXλ=10^-3, Learning Rate=10^-42021.06 | 2.94 | 0.028 | |
| VARDROP+LR+ARλ=1/60,000, Learning Rate=10^-42021.06 | 3.21 | 0.024 |