# LetheViT: Selective Machine Unlearning for Vision Transformers via Attention-Guided Contrastive Learning

Source: [LetheViT: Selective Machine Unlearning for Vision Transformers via Attention-Guided Contrastive Learning](<https://www.sota2.com/research/paper/68fa7a671e209131cea82fca>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:20:02.433Z

Authors: Yujia Tong, Tian Zhang, Jingling Yuan, Yuze Wang, Chuang Hu

Published: 2025-08-03T03:37:31.000Z

## Abstract

Vision Transformers (ViTs) have revolutionized computer vision tasks with their exceptional performance. However, the introduction of privacy regulations such as GDPR and CCPA has brought new challenges to them. These laws grant users the right to withdraw their data, necessitating not only the deletion of data but also the complete removal of its influence from trained models. Machine unlearning emerges as a critical solution, with exact unlearning being computationally prohibitive and approximate methods offering a more practical approach. This work addresses the particularly challenging scenario of random data forgetting in ViTs, where the model must forget specific samples while retaining others, even within the same class. We first reveal the core characteristics of ViTs through selective masking experiments: when high-attention areas are masked, the model retains its recognition capability but significantly weakens its memorization ability. Based on the above insights, we propose LetheViT, a contrastive unlearning method tailored for ViTs. LetheViT uses masked image inputs to generate positive logits and original image inputs to generate negative logits, guiding the model to forget specific details while retaining the general cl category outlines. Experimental results demonstrate that LetheViT achieves state-of-the-art performance, effectively balancing privacy compliance with model efficacy.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Unlearning on CIFAR-10 Random Forget 10% (train)](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-random-forget-10-percent-train.md>): Machine Unlearning; CIFAR-10 Random Forget 10% (train); Retain Accuracy: 99.88.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2508.01569>)

[PDF](<https://arxiv.org/pdf/2508.01569.pdf>)