# Geometric-disentangelment Unlearning

Source: [Geometric-disentangelment Unlearning](<https://www.sota2.com/research/paper/6923e65dd6b0dd334292ed3a>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-02-04T05:00:15.039Z

Authors: Duo Zhou, Yuji Zhang, Tianxin Wei, Ruizhong Qiu, Ke Yang, Xiao Lin, Cheng Qian, Jingrui He, Hanghang Tong, Chengxiang Zhai, Heng Ji, Huan Zhang

Published: 2025-11-21T09:58:25.000Z

## Abstract

Large language models (LLMs) can internalize private or harmful content, motivating unlearning that removes a forget set while preserving retaining knowledge. However, forgetting updates often cause collateral degradation on retaining knowledge, creating a persistent trade-off. Existing LLM unlearning methods are often heuristic, and other theoretical approaches rely on offline feature constructions that do not capture update-time forget-retain interaction in LLMs. To address this limitation, we aim to develop an LLM unlearning method that reduces the forget-retain trade-off with theoretical guarantees. We take a first-principles view by formalizing "no side effects" as local retain invariance under small parameter updates, and prove an equivalence under optimizer-induced geometry: the retain loss is locally invariant if and only if the update direction is orthogonal to the subspace spanned by retain gradients. Based on the insight, we propose Geometric-disentanglement Unlearning (GU), a lightweight and theoretically grounded projection that can be plug-and-play to existing gradient-based unlearning methods to mitigate forget-retain side effects. Experiments on TOFU, MUSE, and WMDP-cyber show that GU strengthens forgetting while reducing retain drift. When added to SimNPO, it achieves up to 62\\% improved forgetting Extraction Strength (ES) and 31\\% higher retain ES. We open-sourced our code in https://github.com/Lemutisme/Geometric-Unlearning.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Unlearning on Tiny-ImageNet Random forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-tiny-imagenet-random-forgetting.md>): Machine Unlearning; Tiny-ImageNet Random forgetting; RA: 99.98.

- [Machine Unlearning on CIFAR-10 Random forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-random-forgetting.md>): Machine Unlearning; CIFAR-10 Random forgetting; RA: 98.6.

- [Machine Unlearning on Tiny-ImageNet (Class-wise forgetting)](<https://www.sota2.com/research/sota/machine-unlearning-on-tiny-imagenet-class-wise-forgetting.md>): Machine Unlearning; Tiny-ImageNet (Class-wise forgetting); Retention Accuracy: 99.54.

- [Random Forgetting Unlearning on CIFAR-100](<https://www.sota2.com/research/sota/random-forgetting-unlearning-on-cifar-100.md>): Random Forgetting Unlearning; CIFAR-100; Retention Accuracy: 96.48.

- [Machine Unlearning on CIFAR-10 Class-wise forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-class-wise-forgetting.md>): Machine Unlearning; CIFAR-10 Class-wise forgetting; Retention Accuracy (RA): 94.2.

- [Class-wise Forgetting Unlearning on CIFAR-100](<https://www.sota2.com/research/sota/class-wise-forgetting-unlearning-on-cifar-100.md>): Class-wise Forgetting Unlearning; CIFAR-100; Retention Accuracy (RA): 90.34.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2511.17100>)

[PDF](<https://arxiv.org/pdf/2511.17100.pdf>)