Image Classification on CIFAR-100 (test) (Standard Accuracy)
91.1Top-1 AccEfficientNet-B5
Evaluation results
| Method | Links | ||
|---|---|---|---|
| EfficientNet-B5Params (M)=30, Resolution=384x384, Pre-training=ImageNet2021.02 | 91.1 | — | |
| TNT-BParams (M)=65.6, Resolution=384x384, Pre-training=ImageNet2021.02 | 91.1 | — | |
| DeiT-BParams (M)=86.4, Resolution=384x384, Pre-training=ImageNet2021.02 | 90.8 | — | |
| TNT-SParams (M)=23.8, Resolution=384x384, Pre-training=ImageNet2021.02 | 90.1 | — | |
| ViT-B/16Params (M)=86.4, Resolution=384x384, Pre-training=ImageNet2021.02 | 87.1 | — | |
| Teacher AccuracyModels (Teacher/Student)=vgg8/ResNet50, Teacher training method=SFTN2021.02 | 82.52 | — | |
| SWABackbone=WideResNet2023.06 | 82.4 | — | |
| SWAGBackbone=WideResNet2023.06 | 82.27 | — | |
| CIBERBackbone=WideResNet2023.06 | 81.25 | — | |
| SGLDBackbone=WideResNet2023.06 | 80.94 | — | |
| KFACBackbone=WideResNet2023.06 | 80.94 | — | |
| SGDBackbone=WideResNet2023.06 | 80.76 | — | |
| SWABackbone=PreResNet-1642023.06 | 80.19 | — | |
| SageMixBackbone=PreActResNet182022.10 | 80.16 | — | |
| Co-MixupBackbone=PreActResNet182022.10 | 80.13 | — | |
| SGLDBackbone=PreResNet-1642023.06 | 80.09 | — | |
| Teacher Accuracy BaselineTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 80.03 | — | |
| Teacher AccuracyModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 80.03 | — | |
| OHModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 79.98 | — | |
| NoisyDARTSSearch Space=S3, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 79.93 | — | |
| FitNetsTeacher Architecture=resnet32x4, Student Architecture=ShuffleV2, Teacher Training=SFTN2021.02 | 79.68 | 2.68 | |
| FitNetModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 79.68 | — | |
| SSKDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 79.65 | — | |
| Teacher AccuracyModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 79.58 | — | |
| OHTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 79.56 | 2.05 | |
| OHModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 79.56 | — | |
| NoisyDARTSSearch Space=S3, Search Space Strategy=OFS, Aggregation Protocol=Average of 3 runs2020.05 | 79.48 | — | |
| SDARTS-ADVSearch Space=S2, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 79.44 | — | |
| Puzzle MixBackbone=PreActResNet182022.10 | 79.38 | — | |
| Teacher Accuracy BaselineTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=Standard2021.02 | 79.25 | — | |
| Teacher AccuracyModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=Standard2021.02 | 79.25 | — | |
| CIBERBackbone=PreResNet-1642023.06 | 79.23 | — | |
| FitNetModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 79.21 | — | |
| OHTeacher Architecture=WRN40-2, Student Architecture=ShuffleV2, Teacher Training=SFTN2021.02 | 79.14 | 1.32 | |
| OHModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 79.14 | — | |
| SSKDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 79.1 | 1.05 | |
| SSKDModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 79.1 | — | |
| SaliencyMixBackbone=PreActResNet182022.10 | 79.06 | — | |
| SDARTS-ADVSearch Space=S3, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 78.92 | — | |
| SDARTS-RSSearch Space=S3, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 78.91 | — | |
| NoisyDARTSSearch Space=S4, Search Space Strategy=NFA, Aggregation Protocol=Best of 3 runs2020.05 | 78.84 | — | |
| NoisyDARTSSearch Space=S1, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 78.83 | — | |
| NoisyDARTSSearch Space=S2, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 78.82 | — | |
| CRDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.81 | — | |
| ATModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.79 | — | |
| VIDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.78 | — | |
| ABModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.77 | — | |
| SDARTS-ADVSearch Space=S4, Search Space Strategy=NFA, Aggregation Protocol=Best of 3 runs2020.05 | 78.75 | — | |
| FitNetsTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.75 | 2.8 | |
| FitNetModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.75 | — | |
| CutMixBackbone=PreActResNet182022.10 | 78.71 | — | |
| Teacher AccuracyModels (Teacher/Student)=vgg8/ResNet50, Teacher training method=Standard2021.02 | 78.7 | — | |
| ABModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.68 | — | |
| SSKDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=Standard2021.02 | 78.66 | — | |
| ATTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.63 | 2.51 | |
| ATModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.63 | — | |
| SDARTS-RSSearch Space=S4, Search Space Strategy=NFA, Aggregation Protocol=Best of 3 runs2020.05 | 78.54 | — | |
| ABTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.53 | 2.58 | |
| ABModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.53 | — | |
| KFACBackbone=PreResNet-1642023.06 | 78.51 | — | |
| SGDBackbone=PreResNet-1642023.06 | 78.5 | — | |
| VIDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.49 | — | |
| CRDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.43 | — | |
| SPModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.38 | — | |
| NoisyDARTSSearch Space=S4, Search Space Strategy=NFA, Aggregation Protocol=Average of 3 runs2020.05 | 78.37 | — | |
| FTModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.37 | — | |
| SPTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.36 | 2.56 | |
| SPModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.36 | — | |
| ManifoldBackbone=PreActResNet182022.10 | 78.36 | — | |
| SSKDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.34 | — | |
| ATModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.29 | — | |
| PKTModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.28 | — | |
| PKTModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.28 | — | |
| Teacher AccuracyModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.21 | — | |
| CRDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.2 | 2.6 | |
| Teacher AccuracyModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=Standard2021.02 | 78.2 | — | |
| CRDModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.2 | — | |
| NoisyDARTSSearch Space=S2, Search Space Strategy=OFS, Aggregation Protocol=Average of 3 runs2020.05 | 78.15 | — | |
| SPModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.12 | — | |
| RKDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.11 | — | |
| OHModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=Standard2021.02 | 78.08 | — | |
| KDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV2, Teacher Training=SFTN2021.02 | 78.07 | 2.82 | |
| KDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 78.07 | — | |
| RDARTS-L2Search Space=S4, Search Space Strategy=NFA, Aggregation Protocol=Best of 3 runs2020.05 | 78.06 | — | |
| KDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=SFTN2021.02 | 78.06 | — | |
| SSKDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=Standard2021.02 | 78.05 | — | |
| SSKDModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=Standard2021.02 | 78.05 | — | |
| VIDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 78.03 | 2.87 | |
| VIDModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 78.03 | — | |
| KDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 77.93 | 3.67 | |
| KDModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 77.93 | — | |
| NoisyDARTSSearch Space=S1, Search Space Strategy=OFS, Aggregation Protocol=Average of 3 runs2020.05 | 77.89 | — | |
| FTTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 77.84 | 2.26 | |
| FTModels (Teacher/Student)=ShuffleV1/resnet32x4, Teacher training method=SFTN2021.02 | 77.84 | — | |
| OHModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=Standard2021.02 | 77.82 | — | |
| SSKDModels (Teacher/Student)=ShuffleV2/wrn40-2, Teacher training method=Standard2021.02 | 77.81 | — | |
| RKDModels (Teacher/Student)=ShuffleV2/resnet32x4, Teacher training method=SFTN2021.02 | 77.77 | — | |
| RDARTS-L2Search Space=S2, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 77.76 | — | |
| SDARTS-RSSearch Space=S2, Search Space Strategy=OFS, Aggregation Protocol=Best of 3 runs2020.05 | 77.72 | — | |
| RKDTeacher Architecture=resnet32x4, Student Architecture=ShuffleV1, Teacher Training=SFTN2021.02 | 77.72 | 2.88 |