Image Classification on CIFAR-100 1.0 (val)
79.42Top-1 AccTeacher
Evaluation results
| Method | Links | ||
|---|---|---|---|
| TeacherTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x42024.03 | 79.42 | — | |
| MLKD + logit standardizationTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 78.28 | 1.2 | |
| SimKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 78.08 | — | |
| MLKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 77.08 | — | |
| DKD + logit standardizationTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 77.01 | 0.69 | |
| CAT-KDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 76.91 | — | |
| KD+CTKD + logit standardizationTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 76.67 | 3.28 | |
| KD + logit standardizationTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 76.62 | 3.29 | |
| DKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 76.32 | — | |
| ReviewKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 75.63 | — | |
| CRDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 75.51 | — | |
| OFDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 74.95 | — | |
| FitNetTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 73.5 | — | |
| ATTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 73.44 | — | |
| KD+CTKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 73.39 | — | |
| KDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Logit2024.03 | 73.33 | — | |
| StudentTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x42024.03 | 72.5 | — | |
| RKDTeacher Architecture=ResNet32x4, Student Architecture=ResNet8x4, Distillation Type=Feature2024.03 | 71.9 | — |