# Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learning

Source: [Understanding Imbalanced Forgetting in Rehearsal-Based Class-Incremental Learning](<https://www.sota2.com/research/paper/6a06a8b8dbc836b87d82a204>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-16T05:00:12.406Z

Authors: Alberto Tamajo, Srinandan Dasmahapatra, Rahman Attar

Published: 2026-05-14T12:55:07.000Z

## Abstract

Neural networks suffer from catastrophic forgetting in class-incremental learning (CIL) settings. Rehearsal$\\unicode{x2013}$replaying a subset of past samples$\\unicode{x2013}$is a well-established mitigation strategy. However, recent results suggest that, despite balanced rehearsal allocation, some classes are forgotten substantially more than others. Despite its relevance, this imbalanced forgetting phenomenon remains underexplored. This work shows that imbalanced forgetting arises systematically and severely in rehearsal-based CIL and investigates it extensively. Specifically, we construct, from a principled analysis, three last-layer coefficients that capture different gradient-level sources of interference affecting each past class during an incremental step. We then demonstrate that, together, they reliably predict how past classes will rank in terms of forgetting at the end of that step. While predictive performance alone does not establish causality, these results support the interpretation of the coefficients as a plausible mechanistic account linking last-layer gradient-level interactions during training to class-level forgetting outcomes. Notably, one coefficient$\\unicode{x2013}$capturing self-induced interference$\\unicode{x2013}$emerges as the strongest predictor, with controlled experiments providing evidence consistent with this coefficient being influenced by the new-class interference coefficient. Overall, our findings provide valuable insights and suggest promising directions for mitigating imbalanced forgetting by reducing class-wise disparities in the identified sources of interference.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Ranking past classes by forgetting on CIFAR-100 (incremental)](<https://www.sota2.com/research/sota/ranking-past-classes-by-forgetting-on-cifar-100-incremental.md>): Ranking past classes by forgetting; CIFAR-100 (incremental); Spearman's Rho: 0.91.

- [Class-Incremental Learning on Tiny-ImageNet (incremental)](<https://www.sota2.com/research/sota/class-incremental-learning-on-tiny-imagenet-incremental.md>): Ranking past classes by forgetting; Tiny-ImageNet (incremental); Spearman's Rho: 0.77.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.14785>)

[PDF](<https://arxiv.org/pdf/2605.14785.pdf>)