Knowledge Distillation on CIFAR-100 (test)
80.86Top-1 AccuracyHSAKD
Evaluation results
| Method | Links | ||
|---|---|---|---|
| HSAKDTeacher=ResNet32x4, Student=ShuffleNetV2, Teacher Training Strategy=Joint2021.07 | 80.86 | — | |
| Teacher BaselineTeacher=WRN-40-2, Teacher Training Strategy=Joint2021.07 | 80.7 | — | |
| HSAKDTeacher=WRN-40-2, Student=ShuffleNetV1, Teacher Training Strategy=Joint2021.07 | 80.11 | — | |
| HSAKDTeacher=ResNet32x4, Student=ShuffleNetV2, Teacher Training Strategy=Fixed2021.07 | 79.93 | — | |
| HSAKDTeacher=ResNet50, Student=MobileNetV2, Teacher Training Strategy=Joint2021.07 | 79.43 | — | |
| HSAKDTeacher=VGG13, Student=MobileNetV2, Teacher Training Strategy=Joint2021.07 | 79.27 | — | |
| HSAKDTeacher=ResNet50, Student=MobileNetV2, Teacher Training Strategy=Fixed2021.07 | 78.79 | — | |
| HSAKDTeacher=WRN-40-2, Student=WRN-16-2, Teacher Training Strategy=Joint2021.07 | 78.67 | — | |
| HSAKDTeacher=WRN-40-2, Student=ShuffleNetV1, Teacher Training Strategy=Fixed2021.07 | 78.51 | — | |
| HSAKDTeacher=WRN-40-2, Student=WRN-40-1, Teacher Training Strategy=Joint2021.07 | 78.12 | — | |
| HSAKDTeacher=ResNet32x4, Student=ResNet8x4, Teacher Training Strategy=Joint2021.07 | 77.69 | — | |
| HSAKDTeacher=VGG13, Student=MobileNetV2, Teacher Training Strategy=Fixed2021.07 | 77.45 | — | |
| HSAKDTeacher=ResNet32x4, Student=ResNet8x4, Teacher Training Strategy=Fixed2021.07 | 77.26 | — | |
| HSAKDTeacher=WRN-40-2, Student=WRN-16-2, Teacher Training Strategy=Fixed2021.07 | 77.2 | — | |
| HSAKDTeacher=WRN-40-2, Student=WRN-40-1, Teacher Training Strategy=Fixed2021.07 | 77 | — | |
| Teacher BaselineTeacher=WRN-40-2, Teacher Training Strategy=Fixed2021.07 | 76.45 | — | |
| SSKDTeacher=WRN-40-2, Student=WRN-16-22021.07 | 76.16 | — | |
| CRDTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.81 | — | |
| ATTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.64 | — | |
| RKDTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.33 | — | |
| VIDTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.31 | — | |
| FitNetTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.3 | — | |
| CCTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.3 | — | |
| KDTeacher=WRN-40-2, Student=WRN-16-22021.07 | 75.23 | — | |
| SPTeacher=WRN-40-2, Student=WRN-16-22021.07 | 74.35 | — | |
| HSAKDTeacher=ResNet56, Student=ResNet20, Teacher Training Strategy=Joint2021.07 | 73.73 | — | |
| Student BaselineStudent=WRN-16-22021.07 | 73.57 | — | |
| HSAKDTeacher=ResNet56, Student=ResNet20, Teacher Training Strategy=Fixed2021.07 | 72.58 | — | |
| ABTeacher=WRN-40-2, Student=WRN-16-22021.07 | 71.26 | — | |
| CMISDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 55.2 | |
| FASTSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 74.34 | |
| PRE-DFKDSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 70.2 | |
| Single ImageSDATA=City, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 69.34 | |
| Single ImageSDATA=Animals, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 71.01 | |
| SpaceshipSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 73.4 | |
| TeacherBackbone=ResNet342024.11 | — | 78.05 | |
| Vanilla KDSDATA=Original, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 79.03 | |
| Vanilla KDSDATA=Tiny, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 76.24 | |
| Vanilla KDSDATA=Food, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 71.74 | |
| Vanilla KDSDATA=OpenGL, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | — | 73.86 |