# Task Addition and Weight Disentanglement in Closed-Vocabulary Models

Source: [Task Addition and Weight Disentanglement in Closed-Vocabulary Models](<https://www.sota2.com/research/paper/691d4f51d6b0dd334292d930>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-11-20T05:03:41.519Z

Authors: Adam Hazimeh, Alessandro Favero, Pascal Frossard

Published: 2025-11-18T15:12:21.000Z

## Abstract

Task arithmetic has recently emerged as a promising method for editing pre-trained \\textit{open-vocabulary} models, offering a cost-effective alternative to standard multi-task fine-tuning. However, despite the abundance of \\textit{closed-vocabulary} models that are not pre-trained with language supervision, applying task arithmetic to these models remains unexplored. In this paper, we deploy and study task addition in closed-vocabulary image classification models. We consider different pre-training schemes and find that \\textit{weight disentanglement} -- the property enabling task arithmetic -- is a general consequence of pre-training, as it appears in different pre-trained closed-vocabulary models. In fact, we find that pre-trained closed-vocabulary vision transformers can also be edited with task arithmetic, achieving high task addition performance and enabling the efficient deployment of multi-task models. Finally, we demonstrate that simple linear probing is a competitive baseline to task addition. Overall, our findings expand the applicability of task arithmetic to a broader class of pre-trained models and open the way for more efficient use of pre-trained models in diverse settings.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.14569>)

[PDF](<https://arxiv.org/pdf/2511.14569.pdf>)
