Image Classification on CIFAR-100 (PGD and A3 Robustness Evaluation)
87.1Nominal AccuracyISyNet-N3
Evaluation results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| ISyNet-N32021.09 | 87.1 | — | — | — | — | — | — | — | — | |
| ResNet-50+2021.09 | 86.96 | — | — | — | — | — | — | — | — | |
| ISyNet-N22021.09 | 86.65 | — | — | — | — | — | — | — | — | |
| ISyNet-N1-S32021.09 | 86.23 | — | — | — | — | — | — | — | — | |
| ResNet-34+2021.09 | 86.13 | — | — | — | — | — | — | — | — | |
| ISyNet-N1-S22021.09 | 85.91 | — | — | — | — | — | — | — | — | |
| ISyNet-N1-S12021.09 | 85.73 | — | — | — | — | — | — | — | — | |
| ISyNet-N12021.09 | 85.27 | — | — | — | — | — | — | — | — | |
| ISyNet-N02021.09 | 84.31 | — | — | — | — | — | — | — | — | |
| ResNet-18+2021.09 | 83.9 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 77.35 | — | — | — | — | — | — | — | — | |
| DKDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 77.07 | — | — | — | — | — | — | — | — | |
| DKDTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 76.7 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 76.57 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 76.55 | — | — | — | — | — | — | — | — | |
| DKDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 76.45 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 76.4 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 76.34 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 76.05 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 75.65 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 75.42 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 75.39 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 75.11 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 74.87 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 74.83 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 74.82 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 74.69 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 74.45 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 74.37 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 74.3 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 74.18 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 74.11 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 74.1 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 74.07 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 73.89 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 73.81 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 73.73 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 73.61 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 73.59 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 73.54 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 73.4 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 73.38 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 73.32 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 73.21 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 73.01 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=resnet32x4, Student architecture=ShuffleNetV2, Teacher baseline accuracy=79.42, Student baseline accuracy=71.822024.02 | 72.73 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 72.28 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=WRN-40-2, Student architecture=ShuffleNetV1, Teacher baseline accuracy=75.61, Student baseline accuracy=70.52024.02 | 72.21 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 71.84 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=resnet32x4, Student architecture=ShuffleNetV1, Teacher baseline accuracy=79.42, Student baseline accuracy=70.52024.02 | 71.73 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 71.5 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 70.69 | — | — | — | — | — | — | — | — | |
| DKDTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 70.35 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=ResNet50, Student architecture=vgg8, Teacher baseline accuracy=79.34, Student baseline accuracy=70.362024.02 | 70.3 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 69.73 | — | — | — | — | — | — | — | — | |
| DKDTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 69.71 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 69.59 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 69.24 | — | — | — | — | — | — | — | — | |
| WTTMTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 69.16 | — | — | — | — | — | — | — | — | |
| CRDTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 69.11 | — | — | — | — | — | — | — | — | |
| TTMTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 68.98 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 68.66 | — | — | — | — | — | — | — | — | |
| DISTTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 68.5 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 67.57 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 67.37 | — | — | — | — | — | — | — | — | |
| KDTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 67.35 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 67.13 | — | — | — | — | — | — | — | — | |
| PKTTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 66.52 | — | — | — | — | — | — | — | — | |
| VIDTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 65.56 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 64.52 | — | — | — | — | — | — | — | — | |
| RKDTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 64.43 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 64.14 | — | — | — | — | — | — | — | — | |
| FitNetTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 63.16 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=vgg13, Student architecture=MobileNetV2, Teacher baseline accuracy=74.64, Student baseline accuracy=64.62024.02 | 59.4 | — | — | — | — | — | — | — | — | |
| ATTeacher architecture=ResNet50, Student architecture=MobileNetV2, Teacher baseline accuracy=79.34, Student baseline accuracy=64.62024.02 | 58.58 | — | — | — | — | — | — | — | — | |
| Full dataset2024.06 | 56.2 | — | — | — | — | — | — | — | — | |
| PPipc=502024.06 | 48.4 | — | — | — | — | — | — | — | — | |
| MTTipc=502024.06 | 47.7 | — | — | — | — | — | — | — | — | |
| MCTipc=502024.06 | 46.8 | — | — | — | — | — | — | — | — | |
| DMipc=502024.06 | 43.6 | — | — | — | — | — | — | — | — | |
| PPipc=102024.06 | 43.1 | — | — | — | — | — | — | — | — | |
| DSAipc=502024.06 | 42.8 | — | — | — | — | — | — | — | — | |
| MCTipc=102024.06 | 42.5 | — | — | — | — | — | — | — | — | |
| MTTipc=102024.06 | 40.1 | — | — | — | — | — | — | — | — | |
| CAFEipc=502024.06 | 37.9 | — | — | — | — | — | — | — | — | |
| ULATModel (Backbone)=WRN-70-16, Training with unlabeled datasets (†)=true, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 36.88 | 69.15 | 38.64 | 37.41 | 36.88 | 36.86 | 2.56 | 1.25 | 0.02 | |
| Fixing DataModel (Backbone)=WRN-70-16, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=false2022.03 | 34.64 | 63.56 | 35.95 | 34.98 | 34.64 | 34.55 | 2.38 | 1.16 | 0.04 | |
| SCAFFOLDNIID Partition Strategy=LDA2021.06 | 33.68 | — | — | — | — | — | — | — | — | |
| Randomipc=502024.06 | 33.4 | — | — | — | — | — | — | — | — | |
| FedNTDNIID Partition Strategy=LDA2021.06 | 32.37 | — | — | — | — | — | — | — | — | |
| DSAipc=102024.06 | 32.3 | — | — | — | — | — | — | — | — | |
| FedNovaNIID Partition Strategy=LDA2021.06 | 32.15 | — | — | — | — | — | — | — | — | |
| FedNTDNIID Partition Strategy=Sharding2021.06 | 31.69 | — | — | — | — | — | — | — | — | |
| SCAFFOLDNIID Partition Strategy=Sharding2021.06 | 30.82 | — | — | — | — | — | — | — | — | |
| FedAvgNIID Partition Strategy=LDA2021.06 | 30.69 | — | — | — | — | — | — | — | — | |
| MOONNIID Partition Strategy=LDA2021.06 | 30.64 | — | — | — | — | — | — | — | — | |
| FedProxNIID Partition Strategy=LDA2021.06 | 30.6 | — | — | — | — | — | — | — | — | |
| LBGATModel (Backbone)=WRN-34-20, Training with unlabeled datasets (†)=false, Perturbation budget (epsilon = 0.031) (‡)=true2022.03 | 30.2 | 62.55 | 31.65 | 30.49 | 30.2 | 30.12 | 2.16 | 1.05 | 0.08 | |
| DMipc=102024.06 | 29.7 | — | — | — | — | — | — | — | — | |
| FedCurvNIID Partition Strategy=LDA2021.06 | 29.16 | — | — | — | — | — | — | — | — |