# Representational Continuity for Unsupervised Continual Learning

Source: [Representational Continuity for Unsupervised Continual Learning](<https://www.sota2.com/research/paper/68fa6c2a1e209131cea3276b>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T00:05:48.516Z

Authors: Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, Sung Ju Hwang

Published: 2021-10-13T18:38:06.000Z

## Abstract

Continual learning (CL) aims to learn a sequence of tasks without forgetting the previously acquired knowledge. However, recent CL advances are restricted to supervised continual learning (SCL) scenarios. Consequently, they are not scalable to real-world applications where the data distribution is often biased and unannotated. In this work, we focus on unsupervised continual learning (UCL), where we learn the feature representations on an unlabelled sequence of tasks and show that reliance on annotated data is not necessary for continual learning. We conduct a systematic study analyzing the learned feature representations and show that unsupervised visual representations are surprisingly more robust to catastrophic forgetting, consistently achieve better performance, and generalize better to out-of-distribution tasks than SCL. Furthermore, we find that UCL achieves a smoother loss landscape through qualitative analysis of the learned representations and learns meaningful feature representations. Additionally, we propose Lifelong Unsupervised Mixup (LUMP), a simple yet effective technique that interpolates between the current task and previous tasks' instances to alleviate catastrophic forgetting for unsupervised representations.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Unsupervised Continual Clustering on CIFAR-10](<https://www.sota2.com/research/sota/unsupervised-continual-clustering-on-cifar-10.md>): Unsupervised Continual Clustering; CIFAR-10; Accuracy (ACC): 56.43.

- [Unsupervised Continual Clustering on Tiny-ImageNet](<https://www.sota2.com/research/sota/unsupervised-continual-clustering-on-tiny-imagenet.md>): Unsupervised Continual Clustering; Tiny-ImageNet; Clustering Accuracy (ACC): 10.53.

- [Unsupervised Continual Clustering on ImageNet 100](<https://www.sota2.com/research/sota/unsupervised-continual-clustering-on-imagenet-100.md>): Unsupervised Continual Clustering; ImageNet-100; Accuracy (ACC): 32.7.

- [Unsupervised Continual Clustering on CIFAR-100](<https://www.sota2.com/research/sota/unsupervised-continual-clustering-on-cifar-100.md>): Unsupervised Continual Clustering; CIFAR-100; Accuracy: 19.53.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2110.06976>)

[PDF](<https://arxiv.org/pdf/2110.06976.pdf>)