# Knowledge distillation using unlabeled mismatched images

Source: [Knowledge distillation using unlabeled mismatched images](<https://www.sota2.com/research/paper/68fa63321e209131ce9fd4ab>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-25T20:35:17.063Z

Authors: Mandar Kulkarni, Kalpesh Patil, Shirish Karande

Published: 2017-03-21T10:34:59.000Z

## Abstract

Current approaches for Knowledge Distillation (KD) either directly use training data or sample from the training data distribution. In this paper, we demonstrate effectiveness of 'mismatched' unlabeled stimulus to perform KD for image classification networks. For illustration, we consider scenarios where this is a complete absence of training data, or mismatched stimulus has to be used for augmenting a small amount of training data. We demonstrate that stimulus complexity is a key factor for distillation's good performance. Our examples include use of various datasets for stimulating MNIST and CIFAR teachers.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/1703.07131>)

[PDF](<https://arxiv.org/pdf/1703.07131.pdf>)