List your company
ResearchFormerly WizWand

Knowledge Distillation on CIFAR-10 (test)

95.7AccuracyTeacher
11.439233.314655.1977.0654Nov 19, 2024Feb 26, 2025Jun 5, 2025Sep 12, 2025Dec 20, 2025Mar 29, 2026Jul 7, 2026
Updated 3mo ago

Evaluation results

MethodLinks
TeacherBackbone=ResNet342024.1195.7
Vanilla KDSDATA=Original, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1195.53
SpaceshipSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1195.19
Vanilla KDSDATA=Tiny, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1194.81
HerdingTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.0794.46
RandomTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.0794.17
FASTSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1194.05
Vanilla KDSDATA=OpenGL, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1193.79
Single ImageSDATA=Animals, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1193.69
Single ImageSDATA=City, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1193.47
Vanilla KDSDATA=Food, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1193.2
Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=1282026.0787.9
PRE-DFKDSDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1187.4
CMISDATA=Data-Free, Teacher Backbone=ResNet34, Student Backbone=ResNet182024.1182.4
HerdingTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=12026.0767.76
Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ViT-Small, k=12026.0766.52
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=1282026.0757.75
Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=1282026.0749.96
RandomTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=1282026.0747.06
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=642026.0747.01
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=322026.0741.68
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=162026.0734.31
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=82026.0730.16
HerdingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=22026.0722.25
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=42026.0722.02
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=22026.0721.65
Few-medoidsTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.0718.05
HerdingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.0717.17
K-centerTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.0716.13
RandomTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.0716.13
PCA-guided matchingTeacher architecture=ResNet-34, Student architecture=ResNet-18, k=12026.0715.92
HerdingTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=12026.0715.79
Few-medoidsTeacher architecture=ViT-B/16, Student architecture=ResNet-50, k=12026.0714.68