# PLD: A Choice-Theoretic List-Wise Knowledge Distillation Source: [PLD: A Choice-Theoretic List-Wise Knowledge Distillation]() Collection index: [llms.txt]() Data updated: 2026-05-13T05:00:05.563Z Authors: Ejafa Bassam, Dawei Zhu, Kaigui Bian Published: 2025-06-14T15:31:54.000Z ## Abstract Knowledge distillation is a model compression technique in which a compact "student" network is trained to replicate the predictive behavior of a larger "teacher" network. In logit-based knowledge distillation, it has become the de facto approach to augment cross-entropy with a distillation term. Typically, this term is either a KL divergence that matches marginal probabilities or a correlation-based loss that captures intra- and inter-class relationships. In every case, it acts as an additional term to cross-entropy. This term has its own weight, which must be carefully tuned. In this paper, we adopt a choice-theoretic perspective and recast knowledge distillation under the Plackett-Luce model by interpreting teacher logits as "worth" scores. We introduce "Plackett-Luce Distillation (PLD)", a weighted list-wise ranking loss. In PLD, the teacher model transfers knowledge of its full ranking of classes, weighting each ranked choice by its own confidence. PLD directly optimizes a single "teacher-optimal" ranking. The true label is placed first, followed by the remaining classes in descending teacher confidence. This process yields a convex and translation-invariant surrogate that subsumes weighted cross-entropy. Empirically, across CIFAR-100, ImageNet-1K, and MS-COCO, PLD achieves consistent gains across diverse architectures and distillation objectives, including divergence-based, correlation-based, and feature-based methods, in both homogeneous and heterogeneous teacher-student pairs. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet-1K (Top-1 Accuracy, Teacher Metrics)](): Image Classification; ImageNet-1K; Top-1 Accuracy: 77.3. - [Object Detection on MS-COCO 2017 (val)](): Object Detection; MS-COCO 2017 (val); mAP: not recorded. - [Image Classification on ImageNet-1K (KD/DIST/PLD Accuracy)](): Image Classification; ImageNet-1K; Top-1 Accuracy (Teacher): not recorded. ## Original sources and implementation [Original paper]() [PDF]()