Knowledge Distillation on CIFAR-10 (test)
95.7AccuracyTeacher
Evaluation Results
| Method | Links | |
|---|---|---|
| TeacherBackbone=ResNet342024.11 | 95.7 | |
| Vanilla KDSDATA=Original, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 95.53 | |
| SpaceshipSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 95.19 | |
| Vanilla KDSDATA=Tiny, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 94.81 | |
| HerdingTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.07 | 94.46 | |
| RandomTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.07 | 94.17 | |
| FASTSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 94.05 | |
| Vanilla KDSDATA=OpenGL, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 93.79 | |
| Single ImageSDATA=Animals, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 93.69 | |
| Single ImageSDATA=City, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 93.47 | |
| Vanilla KDSDATA=Food, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 93.2 | |
| Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.07 | 87.9 | |
| PRE-DFKDSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 87.4 | |
| CMISDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.11 | 82.4 | |
| HerdingTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=12026.07 | 67.76 | |
| Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=12026.07 | 66.52 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=1282026.07 | 57.75 | |
| Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=1282026.07 | 49.96 | |
| RandomTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=1282026.07 | 47.06 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=642026.07 | 47.01 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=322026.07 | 41.68 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=162026.07 | 34.31 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=82026.07 | 30.16 | |
| HerdingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=22026.07 | 22.25 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=42026.07 | 22.02 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=22026.07 | 21.65 | |
| Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.07 | 18.05 | |
| HerdingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.07 | 17.17 | |
| K-centerTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.07 | 16.13 | |
| RandomTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.07 | 16.13 | |
| PCA-guided matchingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.07 | 15.92 | |
| HerdingTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=12026.07 | 15.79 | |
| Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=12026.07 | 14.68 |