# Unsupervised Representation Learning Meets Pseudo-Label Supervised Self-Distillation: A New Approach to Rare Disease Classification Source: [Unsupervised Representation Learning Meets Pseudo-Label Supervised Self-Distillation: A New Approach to Rare Disease Classification]() Collection index: [llms.txt]() Data updated: 2026-01-04T05:07:06.012Z Authors: Jinghan Sun, Dong Wei, Kai Ma, Liansheng Wang, Yefeng Zheng Published: 2021-10-09T12:56:09.000Z ## Abstract Rare diseases are characterized by low prevalence and are often chronically debilitating or life-threatening. Imaging-based classification of rare diseases is challenging due to the severe shortage in training examples. Few-shot learning (FSL) methods tackle this challenge by extracting generalizable prior knowledge from a large base dataset of common diseases and normal controls, and transferring the knowledge to rare diseases. Yet, most existing methods require the base dataset to be labeled and do not make full use of the precious examples of the rare diseases. To this end, we propose in this work a novel hybrid approach to rare disease classification, featuring two key novelties targeted at the above drawbacks. First, we adopt the unsupervised representation learning (URL) based on self-supervising contrastive loss, whereby to eliminate the overhead in labeling the base dataset. Second, we integrate the URL with pseudo-label supervised classification for effective self-distillation of the knowledge about the rare diseases, composing a hybrid approach taking advantages of both unsupervised and (pseudo-) supervised learning on the base dataset. Experimental results on classification of rare skin lesions show that our hybrid approach substantially outperforms existing FSL methods (including those using fully supervised base dataset) for rare disease classification via effective integration of the URL and pseudo-label driven self-distillation, thus establishing a new state of the art. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Skin Lesion Classification on ISIC 3-way 1-shot 2018 (test)](): Skin lesion classification; ISIC 3-way 1-shot 2018 (test); Accuracy: 64.15. - [Skin Lesion Classification on ISIC 3-way 3-shot 2018 (test)](): Skin lesion classification; ISIC 3-way 3-shot 2018 (test); Accuracy: 75.82. - [Skin Lesion Classification on ISIC 3-way 5-shot 2018 (test)](): Skin lesion classification; ISIC 3-way 5-shot 2018 (test); Accuracy: 81.16. ## Original sources and implementation [Original paper]() [PDF]() [Implementation]()