# Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions Source: [Beyond Dark Knowledge: Mixup-Based Distillation for Reliable Predictions]() Collection index: [llms.txt]() Data updated: 2026-06-12T05:00:14.585Z Authors: Jos\\'e Medina, Paul Honeine, Abdelaziz Bensrhair, Amnir Hadachi Published: 2026-06-10T14:59:26.000Z ## Abstract Knowledge Distillation (KD) and mixup have proven effective at inducing smoothness in class boundaries; KD captures inherent class relationships in probability distributions, and mixup enforces them through convex combinations of inputs. Their interaction, however, remains poorly understood, particularly when mixup is applied only during student training. In this setting, the teacher is queried on inputs drawn from a vicinal distribution it never saw during training, a controlled mismatch whose effect on knowledge transfer has not been characterised. We show that this mismatch causes the teacher's supervisory signal to be dominated by distributional confusion rather than inter-class structure. Despite it, the student does not merely imitate the teacher: it independently acquires greater linearity in the vicinal region, a structural property that the teacher lacks, and goes beyond dark-knowledge transfer. KD with mixup consistently improves student accuracy and reduces overconfidence by an order of magnitude relative to the baseline, across CIFAR and ImageNet with varying-capacity teachers. Crucially, calibration propagates from teacher to student independently of accuracy transfer, and temperature scaling governs a measurable accuracy-calibration trade-off that becomes more pronounced under vicinal training. These results reframe mixup distillation not as a degraded version of standard KD, but as a richer transfer channel that simultaneously shapes discriminative performance, uncertainty estimation, and representational geometry. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on CIFAR-100 (Accuracy and Calibration)](): Image Classification; CIFAR-100; Accuracy: 86.3. - [Image Classification on ImageNet (test) (Accuracy and Calibration)](): Image Classification; ImageNet (test); Top-1 Accuracy: 71.6. ## Original sources and implementation [Original paper]() [PDF]()