# Retain-Neutral Surrogates for Min-Max Unlearning

Source: [Retain-Neutral Surrogates for Min-Max Unlearning](<https://www.sota2.com/research/paper/69fd6e435497b3b670c895f9>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-05-09T05:00:04.796Z

Authors: Junhao Cai, Dohun Kim, Dowon Kim, Sung Il Choi, Chengjun Jin, Juhyun Park, Changhee Joo

Published: 2026-05-07T08:38:09.000Z

## Abstract

Machine unlearning seeks to remove the influence of designated training data while preserving performance on the remaining data. Approximate unlearning can be viewed as a local editing problem; in min-max unlearning, the key local object is the surrogate point at which the retain objective is evaluated. When forget and retain gradients are strongly aligned, an unconstrained forget-maximizing perturbation can move to a surrogate point that increases retain loss. We propose Retain-Orthogonal Surrogate Unlearning (ROSU), which constrains the inner surrogate construction by maximizing first-order forget gain subject to zero first-order retain change under a fixed perturbation budget. This yields a closed-form retain-orthogonal perturbation, a lightweight transported outer update, and amplification along the retain-neutral direction. Our analysis establishes (i) a curvature-controlled second-order bound on retain damage, (ii) a positive-alignment regime in which ROSU strictly reduces surrogate retain loss relative to standard min-max perturbations, and (iii) near-equivalence when the two gradients are nearly orthogonal. Across vision and language benchmarks (CIFAR-10/100, Tiny-ImageNet, TOFU, WMDP), the empirical pattern follows this geometry: ROSU gives its clearest gains in high-coupling regimes while remaining competitive elsewhere.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [LLM Unlearning on TOFU (Metrics: Agg., Mem., Priv., Util.)](<https://www.sota2.com/research/sota/llm-unlearning-on-tofu-metrics-agg-mem-priv-util.md>): LLM Unlearning; TOFU; Aggregated Score: 58.

- [Machine Unlearning on CIFAR-10 Class-wise forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-class-wise-forgetting.md>): Machine Unlearning; CIFAR-10 Class-wise forgetting; Retention Accuracy (RA): 99.96.

- [Machine Unlearning on CIFAR-10 Random forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-random-forgetting.md>): Machine Unlearning; CIFAR-10 Random forgetting; RA: 99.84.

- [Machine Unlearning on Tiny-ImageNet (Class-wise forgetting)](<https://www.sota2.com/research/sota/machine-unlearning-on-tiny-imagenet-class-wise-forgetting.md>): Machine Unlearning; Tiny-ImageNet (Class-wise forgetting); Retention Accuracy: 99.97.

- [Class-wise Forgetting Unlearning on CIFAR-100](<https://www.sota2.com/research/sota/class-wise-forgetting-unlearning-on-cifar-100.md>): Class-wise Forgetting Unlearning; CIFAR-100; Retention Accuracy (RA): 99.98.

- [Random Forgetting Unlearning on CIFAR-100](<https://www.sota2.com/research/sota/random-forgetting-unlearning-on-cifar-100.md>): Random Forgetting Unlearning; CIFAR-100; Retention Accuracy: 99.78.

- [Machine Unlearning on Tiny-ImageNet Random forgetting](<https://www.sota2.com/research/sota/machine-unlearning-on-tiny-imagenet-random-forgetting.md>): Machine Unlearning; Tiny-ImageNet Random forgetting; RA: 98.7.

- [LLM Unlearning on WMDP](<https://www.sota2.com/research/sota/llm-unlearning-on-wmdp.md>): LLM Unlearning; WMDP; Delta Score: 5.4.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2605.05871>)

[PDF](<https://arxiv.org/pdf/2605.05871.pdf>)