# Task Arithmetic with LoRA for Continual Learning Source: [Task Arithmetic with LoRA for Continual Learning]() Collection index: [llms.txt]() Data updated: 2025-10-26T01:58:41.298Z Authors: Rajas Chitale, Ankit Vaidya, Aditya Kane, Archana Ghotkar Published: 2023-11-04T15:12:24.000Z ## Abstract Continual learning refers to the problem where the training data is available in sequential chunks, termed "tasks". The majority of progress in continual learning has been stunted by the problem of catastrophic forgetting, which is caused by sequential training of the model on streams of data. Moreover, it becomes computationally expensive to sequentially train large models multiple times. To mitigate both of these problems at once, we propose a novel method to continually train transformer-based vision models using low-rank adaptation and task arithmetic. Our method completely bypasses the problem of catastrophic forgetting, as well as reducing the computational requirement for training models on each task. When aided with a small memory of 10 samples per class, our method achieves performance close to full-set finetuning. We present rigorous ablations to support the prowess of our method. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Action Recognition on NTU-RGB+D 120](): Action Recognition; NTU RGB+D 120; FAA: 66.9. - [Semantic Segmentation on Domains Campus](): Semantic segmentation; Domains Campus; FAA: 41. ## Original sources and implementation [Original paper]() [PDF]()