# Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack

Source: [Unlearning's Blind Spots: Over-Unlearning and Prototypical Relearning Attack](<https://www.sota2.com/research/paper/68fa79171e209131cea7c144>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2026-06-02T05:00:05.013Z

Authors: SeungBum Ha, Saerom Park, Sung Whan Yoon

Published: 2025-06-02T05:01:30.000Z

## Abstract

Machine unlearning (MU) aims to expunge a designated forget set from a trained model without costly retraining, yet the existing techniques overlook two critical blind spots: "over-unlearning" that deteriorates retained data near the forget set, and post-hoc "relearning" attacks that aim to resurrect the forgotten knowledge. Focusing on class-level unlearning, we first derive an over-unlearning metric, OU@epsilon, which quantifies collateral damage in regions proximal to the forget set, where over-unlearning mainly occurs. Next, we expose an unforeseen relearning threat on MU, i.e., the Prototypical Relearning Attack, which exploits the per-class prototype of the forget class with just a few samples, and easily restores the pre-unlearning performance. To counter both blind spots in class-level unlearning, we introduce Spotter, a plug-and-play objective that combines (i) a masked knowledge-distillation penalty on the nearby region of forget classes to suppress OU@epsilon, and (ii) an intra-class dispersion loss that scatters forget-class embeddings, neutralizing Prototypical Relearning Attacks. Spotter achieves state-of-the-art results across CIFAR, TinyImageNet, and CASIA-WebFace datasets, offering a practical remedy to unlearning's blind spots.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Unlearning on Tiny-ImageNet Forget 10 Class](<https://www.sota2.com/research/sota/machine-unlearning-on-tiny-imagenet-forget-10-class.md>): Machine Unlearning; Tiny-ImageNet Forget 10 Class; Accuracy (Forgetting): 0.56.

- [Class-level Machine Unlearning on ImageNet (test)](<https://www.sota2.com/research/sota/class-level-machine-unlearning-on-imagenet-test.md>): Class-level Machine Unlearning; ImageNet (test); Retained Accuracy (Test): 0.

- [Machine Unlearning on CIFAR-10 (Utility and Specific Privacy Metrics)](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-utility-and-specific-privacy-metrics.md>): Machine Unlearning; CIFAR-10; Accuracy (Forget): 0.5.

- [Machine Unlearning on CASIA-WebFace Forget 500 Class](<https://www.sota2.com/research/sota/machine-unlearning-on-casia-webface-forget-500-class.md>): Machine Unlearning; CASIA-WebFace Forget 500 Class; Accuracy (Forget): 0.74.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2506.01318>)

[PDF](<https://arxiv.org/pdf/2506.01318.pdf>)