# Preventing Latent Rehearsal Decay in Online Continual SSL with SOLAR

Source: [Preventing Latent Rehearsal Decay in Online Continual SSL with SOLAR](<https://www.sota2.com/research/paper/69ddca167d0172140ec79a08>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-04-15T05:00:11.244Z

Authors: Giacomo Cignoni, Simone Magistri, Andrew D. Bagdanov, Antonio Carta

Published: 2026-04-12T11:11:06.000Z

## Abstract

This paper explores Online Continual Self-Supervised Learning (OCSSL), a scenario in which models learn from continuous streams of unlabeled, non-stationary data, where methods typically employ replay and fast convergence is a central desideratum. We find that OCSSL requires particular attention to the stability-plasticity trade-off: stable methods (e.g. replay with Reservoir sampling) are able to converge faster compared to plastic ones (e.g. FIFO buffer), but incur in performance drops under certain conditions. We explain this collapse phenomenon with the Latent Rehearsal Decay hypothesis, which attributes it to latent space degradation under excessive stability of replay. We introduce two metrics (Overlap and Deviation) that diagnose latent degradation and correlate with accuracy declines. Building on these insights, we propose SOLAR, which leverages efficient online proxies of Deviation to guide buffer management and incorporates an explicit Overlap loss, allowing SOLAR to adaptively managing plasticity. Experiments demonstrate that SOLAR achieves state-of-the-art performance on OCSSL vision benchmarks, with both high convergence speed and final performance.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Online Continual Self-Supervised Learning on CIFAR-100 streaming online 20 experiences](<https://www.sota2.com/research/sota/online-continual-self-supervised-learning-on-cifar-100-streaming-online-20-experiences.md>): Online Continual Self-Supervised Learning; CIFAR-100 streaming online 20 experiences; Final Accuracy: 49.5.

- [Online Continual Self-Supervised Learning on ImageNet100 streaming online 20 experiences](<https://www.sota2.com/research/sota/online-continual-self-supervised-learning-on-imagenet100-streaming-online-20-experiences.md>): Online Continual Self-Supervised Learning; ImageNet100 streaming online 20 experiences; Final Accuracy: 49.4.

- [Online Continual Self-Supervised Learning on CLEAR100 11 experiences (streaming online)](<https://www.sota2.com/research/sota/online-continual-self-supervised-learning-on-clear100-11-experiences-streaming-online.md>): Online Continual Self-Supervised Learning; CLEAR100 11 experiences (streaming online); Final Accuracy: 51.5.

- [Online Continual Self-Supervised Learning on iNaturalist](<https://www.sota2.com/research/sota/online-continual-self-supervised-learning-on-inaturalist.md>): Online Continual Self-Supervised Learning; iNaturalist; Final Accuracy: 44.11.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2604.10586>)

[PDF](<https://arxiv.org/pdf/2604.10586.pdf>)