# Model Unlearning Objectives Vary for Distinct Language Functions Source: [Model Unlearning Objectives Vary for Distinct Language Functions]() Collection index: [llms.txt]() Data updated: 2026-05-28T05:00:05.405Z Authors: Berk Atil, Vipul Gupta, Rebecca J. Passonneau Published: 2026-05-26T02:10:55.000Z ## Abstract Large language models (LLMs) learn undesirable properties during pretraining, including dangerous knowledge and toxic text generation. Just as post-training uses different objectives to shape different behaviors, we argue that unlearning methods should be designed for the language function at issue. To study this, we consider two mechanistically distinct unlearning goals, dangerous-knowledge unlearning and toxicity unlearning. For dangerous knowledge, we introduce a cosine-based, meta-learned variant of RMU. For toxicity, we propose a multi-layer objective based on layer-specific probe directions. Across four open-source 7-8B models, our methods achieve strong results, based on distinct training objectives for the two types of unlearning. Overall, our results suggest that unlearning should be studied as a family of problems, analogous to the multiple types of LLM post-training. ## Recorded benchmark results A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage. - [Dangerous Knowledge Unlearning on WMDP](): Dangerous Knowledge Unlearning; WMDP; S-unlearning Score: 43. - [Toxicity Unlearning on Toxicity](): Toxicity Unlearning; Toxicity; S-unlearning Score: 0.45. ## Original sources and implementation [Original paper]() [PDF]()