Knowledge Distillation on CIFAR-100 (val)
77.78Top-1 AccuracyReviewKD
Evaluation results
| Method | Links | |
|---|---|---|
| ReviewKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Features2025.03 | 77.78 | |
| GKL-KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Logits2025.03 | 77.49 | |
| ReviewKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 77.45 | |
| GKL-KDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 77.42 | |
| IKL-KDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 77.19 | |
| IKL-KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Logits2025.03 | 77.16 | |
| ReviewKDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 77.14 | |
| DKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Logits2025.03 | 77.07 | |
| OFDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Features2025.03 | 76.82 | |
| GKL-KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 76.76 | |
| DKDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 76.7 | |
| IKL-KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 76.64 | |
| DKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 76.45 | |
| CRDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 76.05 | |
| OFDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 75.98 | |
| OFDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 75.85 | |
| CRDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Features2025.03 | 75.65 | |
| CRDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 75.11 | |
| KDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 74.83 | |
| KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Logits2025.03 | 74.45 | |
| KDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Logits2025.03 | 74.07 | |
| FitNetTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 73.73 | |
| FitNetTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 73.59 | |
| FitNetTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Features2025.03 | 73.54 | |
| RKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V2, Distillation Manner=Features2025.03 | 73.21 | |
| RKDTeacher architecture=ResNet32x4, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 72.28 | |
| RKDTeacher architecture=WRN-40-2, Student architecture=ShuffleNet-V1, Distillation Manner=Features2025.03 | 72.21 | |
| GKL-KDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 70.78 | |
| IKL-KDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 70.62 | |
| GKL-KDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 70.61 | |
| IKL-KDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 70.4 | |
| ReviewKDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 70.37 | |
| DKDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 70.35 | |
| ReviewKDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 69.89 | |
| CRDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 69.73 | |
| DKDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 69.71 | |
| OFDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 69.48 | |
| CRDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 69.11 | |
| OFDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 69.04 | |
| KDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 67.37 | |
| KDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Logits2025.03 | 67.35 | |
| RKDTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 64.52 | |
| RKDTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 64.43 | |
| FitNetTeacher architecture=VGG13, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 64.14 | |
| FitNetTeacher architecture=ResNet50, Student architecture=MobileNet-V2, Distillation Manner=Features2025.03 | 63.16 |