# Multi-Level Knowledge Distillation and Dynamic Self-Supervised Learning for Continual Learning Source: [Multi-Level Knowledge Distillation and Dynamic Self-Supervised Learning for Continual Learning]() Collection index: [llms.txt]() Data updated: 2026-04-02T05:00:04.464Z Authors: Taeheon Kim, San Kim, Minhyuk Seo, Dongjae Jeon, Wonje Jeung, Jonghyun Choi Published: 2025-08-18T07:50:20.000Z ## Abstract Class-incremental with repetition (CIR), where previously trained classes repeatedly introduced in future tasks, is a more realistic scenario than the traditional class incremental setup, which assumes that each task contains unseen classes. CIR assumes that we can easily access abundant unlabeled data from external sources, such as the Internet. Therefore, we propose two components that efficiently use the unlabeled data to ensure the high stability and the plasticity of models trained in CIR setup. First, we introduce multi-level knowledge distillation (MLKD) that distills knowledge from multiple previous models across multiple perspectives, including features and logits, so the model can maintain much various previous knowledge. Moreover, we implement dynamic self-supervised loss (SSL) to utilize the unlabeled data that accelerates the learning of new classes, while dynamic weighting of SSL keeps the focus of training to the primary task. Both of our proposed components significantly improve the performance in CIR setup, achieving 2nd place in the CVPR 5th CLVISION Challenge. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Image Classification on ImageNet-1K (130 classes, continual learning stream)](): Image Classification; ImageNet-1K 130 classes (continual learning stream); Final Accuracy: 42. ## Original sources and implementation [Original paper]() [PDF]()