# Continual Learning with Low Rank Adaptation Source: [Continual Learning with Low Rank Adaptation]() Collection index: [llms.txt]() Data updated: 2025-10-26T02:03:39.970Z Authors: Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, Giovanni Zappella Published: 2023-11-29T12:53:32.000Z ## Abstract Recent work using pretrained transformers has shown impressive performance when fine-tuned with data from the downstream problem of interest. However, they struggle to retain that performance when the data characteristics changes. In this paper, we focus on continual learning, where a pre-trained transformer is updated to perform well on new data, while retaining its performance on data it was previously trained on. Earlier works have tackled this primarily through methods inspired from prompt tuning. We question this choice, and investigate the applicability of Low Rank Adaptation (LoRA) to continual learning. On a range of domain-incremental learning benchmarks, our LoRA-based solution, CoLoR, yields state-of-the-art performance, while still being as parameter efficient as the prompt tuning based methods. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Action Recognition on NTU-RGB+D 120](): Action Recognition; NTU RGB+D 120; FAA: 67.8. - [Semantic Segmentation on Domains Campus](): Semantic segmentation; Domains Campus; FAA: 44.8. ## Original sources and implementation [Original paper]() [PDF]()