Image Classification Certified Robustness on MNIST (test)
1.823Overall ACRSmoothMix
Evaluation results
| Method | Links | |||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| SmoothMixsigma (Smoothing factor)=1.00, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.823 | 93.3 | 90.9 | 87.5 | 83 | 77.5 | 70.6 | 62.7 | 53.4 | 44.9 | 37.1 | 29.3 | 22.4 | |
| SmoothMixsigma (Smoothing factor)=1.00, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.82 | 93.7 | 91.6 | 88.1 | 83.5 | 77.9 | 70.9 | 62.7 | 53.8 | 44.8 | 36.6 | 28.9 | 21.5 | |
| SmoothMixsigma (Smoothing factor)=1.00, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.816 | 94.7 | 92.4 | 89.2 | 84.6 | 79.4 | 72.5 | 64 | 54.5 | 44.8 | 36.2 | 27.4 | 18.7 | |
| SmoothMixsigma (Smoothing factor)=1.00, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.788 | 95.5 | 93.5 | 90.5 | 86.2 | 80.6 | 73.4 | 64.3 | 53.7 | 43.2 | 33.5 | 23.9 | 14.1 | |
| SmoothAdvsigma (Smoothing factor)=1.00, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.779 | 95.8 | 93.9 | 90.6 | 86.5 | 80.8 | 73.7 | 64.6 | 53.9 | 43.3 | 32.8 | 22.2 | 12.1 | |
| Consistencysigma (Smoothing factor)=1.00, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.74 | 95 | 93 | 89.7 | 85.4 | 79.7 | 72.7 | 63.6 | 53 | 41.7 | 30.8 | 20.3 | 10.7 | |
| SmoothMixsigma (Smoothing factor)=0.50, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.694 | 98.8 | 98.1 | 97.1 | 95.3 | 92.7 | 88.3 | 81.7 | 69.5 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.50, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.694 | 98.7 | 98 | 97 | 95.3 | 92.7 | 88.5 | 81.8 | 70 | 0 | 0 | 0 | 0 | |
| SmoothAdvsigma (Smoothing factor)=0.50, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.687 | 99 | 98.3 | 97.3 | 95.8 | 93.2 | 88.5 | 81.1 | 67.5 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.50, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.685 | 98.2 | 97.5 | 96.3 | 94.5 | 91.3 | 87.4 | 81 | 70.7 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.50, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.678 | 99 | 98.4 | 97.4 | 95.7 | 93 | 88.1 | 80 | 65.6 | 0 | 0 | 0 | 0 | |
| Consistencysigma (Smoothing factor)=0.50, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.657 | 99.2 | 98.6 | 97.6 | 95.9 | 93 | 87.8 | 78.5 | — | 0 | 0 | 0 | 0 | |
| Stability trainingsigma (Smoothing factor)=1.00, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.634 | 96.5 | 94.6 | 91.6 | 87.2 | 80.7 | 71.7 | 60.5 | 47 | 33.4 | 20.6 | 11.2 | 5.9 | |
| Gaussiansigma (Smoothing factor)=1.00, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.62 | 96.3 | 94.4 | 91.4 | 86.8 | 79.8 | 70.9 | 59.4 | 46.2 | 32.5 | 19.7 | 10.9 | 5.8 | |
| MACERsigma (Smoothing factor)=1.00, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.598 | 91.6 | 88.1 | 83.5 | 77.7 | 71.1 | 63.7 | 55.7 | 46.8 | 38.4 | 29.2 | 20 | 11.5 | |
| MACERsigma (Smoothing factor)=0.50, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.594 | 98.5 | 97.5 | 96.2 | 93.7 | 90 | 83.7 | 72.2 | 54 | 0 | 0 | 0 | 0 | |
| Stability trainingsigma (Smoothing factor)=0.50, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.57 | 99.2 | 98.5 | 97.1 | 94.8 | 90.7 | 83.2 | 69.2 | 45.4 | 0 | 0 | 0 | 0 | |
| Gaussiansigma (Smoothing factor)=0.50, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 1.553 | 99.2 | 98.3 | 96.8 | 94.3 | 89.7 | 81.9 | 67.3 | 43.6 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.25, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.933 | 99.4 | 99 | 98.2 | 96.9 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.25, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, One-step adversary=true, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.933 | 99.3 | 99 | 98.2 | 97 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| SmoothAdvsigma (Smoothing factor)=0.25, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.932 | 99.4 | 99 | 98.2 | 96.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.25, Backbone=LeNet, eta (SmoothMix hyperparameter)=5.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.932 | 99.4 | 99 | 98.2 | 96.7 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| SmoothMixsigma (Smoothing factor)=0.25, Backbone=LeNet, eta (SmoothMix hyperparameter)=1.0, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.931 | 99.5 | 98.9 | 98.2 | 96.4 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Consistencysigma (Smoothing factor)=0.25, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.928 | 99.5 | 98.9 | 98 | 96 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| MACERsigma (Smoothing factor)=0.25, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.92 | 99.3 | 98.7 | 97.5 | 94.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Stability trainingsigma (Smoothing factor)=0.25, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.915 | 99.3 | 98.6 | 97.1 | 93.8 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | |
| Gaussiansigma (Smoothing factor)=0.25, Backbone=LeNet, n (number of noise samples)=100,000, alpha (failure probability bound)=0.0012021.11 | 0.911 | 99.2 | 98.5 | 96.7 | 93.3 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |