Image Classification on MNIST (test) (Adversarial Robustness: Certified & L2 PGD)
99.5Test Accuracyl2
Evaluation results
| Method | Links | ||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| l2epsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.5 | — | 2.6 | — | — | — | — | — | 98.6 | 98 | 2.6 | — | — | — | |
| SEPnegModel Architecture=ResNet182024.03 | 99.4 | — | — | — | — | — | — | — | — | — | — | 9,356.306 | 8,248.627 | 93.95 | |
| NormalModel Architecture=ResNet182024.03 | 99.36 | — | — | — | — | — | — | — | — | — | — | 266.834 | 146.16 | 0 | |
| SandwichModel=2C2F, Cert. UB=42024.10 | 99.3 | 2.969 | 98.4 | 96.9 | 93.6 | 92.5 | 63.3 | 54 | — | — | — | — | — | — | |
| VanillaModel=2CP2F, Cert. UB=–2024.10 | 99.3 | 148 | 0 | 0 | 0 | 73.2 | 56.2 | 53.7 | — | — | — | — | — | — | |
| CURE-Maxepsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.3 | — | 97.6 | — | — | — | — | — | 97.3 | 96.8 | 96.8 | — | — | — | |
| NormalModel Architecture=ConvNet2024.03 | 99.29 | — | — | — | — | — | — | — | — | — | — | 261.183 | 204.825 | 0 | |
| MATModel Architecture=ResNet182024.03 | 99.28 | — | — | — | — | — | — | — | — | — | — | 916.017 | 778.003 | 94.6 | |
| l∞epsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.2 | — | 97 | — | — | — | — | — | 96.5 | 95 | 94.9 | — | — | — | |
| CURE-Jointepsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.2 | — | 97.6 | — | — | — | — | — | 97.9 | 97.5 | 97.2 | — | — | — | |
| CURE-Randomepsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.2 | — | 97.3 | — | — | — | — | — | 97.2 | 97.1 | 96.9 | — | — | — | |
| CURE-Scratchepsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.2 | — | 97.5 | — | — | — | — | — | 98 | 97.9 | 97.5 | — | — | — | |
| CURE-Finetuneepsilon_inf=0.1, epsilon_2=0.5, epsilon_1=1.02024.10 | 99.1 | — | 97 | — | — | — | — | — | 97.3 | 96.8 | 96.5 | — | — | — | |
| VanillaModel=2C2F, Cert. UB=–2024.10 | 99 | 221.7 | 0 | 0 | 0 | 69.5 | 61.9 | 59.6 | — | — | — | — | — | — | |
| MATModel Architecture=ConvNet2024.03 | 99 | — | — | — | — | — | — | — | — | — | — | 373.262 | 298.729 | 89.92 | |
| SEPnegModel Architecture=ConvNet2024.03 | 98.97 | — | — | — | — | — | — | — | — | — | — | 806.204 | 657.18 | 90.34 | |
| SandwichModel=2C2F, Cert. UB=22024.10 | 98.9 | 1.703 | 98.2 | 97 | 95.4 | 93.1 | 74 | 67.2 | — | — | — | — | — | — | |
| LipKernelModel=2C2F, Cert. UB=42024.10 | 98.9 | 3.11 | 97.5 | 95.3 | 91.3 | 88.6 | 49.6 | 39.7 | — | — | — | — | — | — | |
| OrthogonModel=2C2F, Cert. UB=42024.10 | 98.8 | 2.894 | 97.4 | 94.4 | 88.6 | 89.6 | 56 | 46 | — | — | — | — | — | — | |
| SEPposModel Architecture=ConvNet2024.03 | 98.8 | — | — | — | — | — | — | — | — | — | — | 93.033 | 61.545 | 89.4 | |
| SEPposModel Architecture=ResNet182024.03 | 98.4 | — | — | — | — | — | — | — | — | — | — | 92.371 | 59.278 | 91.63 | |
| LipKernelModel=2C2F, Cert. UB=22024.10 | 98.2 | 1.703 | 97.1 | 95.6 | 93.6 | 89.8 | 66.1 | 58.9 | — | — | — | — | — | — | |
| OrthogonModel=2C2F, Cert. UB=22024.10 | 97.7 | 1.744 | 96.3 | 94.4 | 91.8 | 89.1 | 66 | 58.2 | — | — | — | — | — | — | |
| SandwichModel=2C2F, Cert. UB=12024.10 | 97.3 | 0.914 | 96.3 | 95.2 | 93.8 | 90.5 | 76.5 | 72 | — | — | — | — | — | — | |
| LipKernelModel=2CP2F, Cert. UB=42024.10 | 97.1 | 2.455 | 93.7 | 87.2 | 75.7 | 80 | 36.8 | 29 | — | — | — | — | — | — | |
| LipKernelModel=2C2F, Cert. UB=12024.10 | 96.6 | 0.952 | 95.6 | 94.3 | 92.6 | 88.3 | 72.2 | 67.8 | — | — | — | — | — | — | |
| AOLModel=2CP2F, Cert. UB=42024.10 | 95.9 | 2.939 | 92.4 | 86.2 | 76.3 | 78.2 | 37 | 29.6 | — | — | — | — | — | — | |
| LipKernelModel=2CP2F, Cert. UB=22024.10 | 94.9 | 1.312 | 91.1 | 85.4 | 77.8 | 80.9 | 53.8 | 45.8 | — | — | — | — | — | — | |
| OrthogonModel=2C2F, Cert. UB=12024.10 | 94.6 | 0.96 | 92.9 | 91 | 88.3 | 83.7 | 65.2 | 60.4 | — | — | — | — | — | — | |
| AOLModel=2CP2F, Cert. UB=22024.10 | 93 | 1.718 | 89.9 | 85.9 | 80.4 | 75.8 | 46.6 | 38.1 | — | — | — | — | — | — | |
| LipKernelModel=2CP2F, Cert. UB=12024.10 | 91.7 | 0.759 | 88 | 83.1 | 77.3 | 77.3 | 57.2 | 52.2 | — | — | — | — | — | — | |
| AOLModel=2CP2F, Cert. UB=12024.10 | 88.7 | 0.926 | 85.5 | 81.7 | 77.2 | 70.6 | 49.2 | 44.6 | — | — | — | — | — | — |