# NOVO: Unlearning-Compliant Vision Transformers

Source: [NOVO: Unlearning-Compliant Vision Transformers](<https://www.sota2.com/research/paper/68fa79dd1e209131cea7fd18>)

Collection index: [llms.txt](<https://www.sota2.com/research/llms.txt>)

Data updated: 2025-10-26T05:07:32.191Z

Authors: Soumya Roy, Soumya Banerjee, Vinay Verma, Soumik Dasgupta, Deepak Gupta, Piyush Rai

Published: 2025-07-04T04:12:34.000Z

## Abstract

Machine unlearning (MUL) refers to the problem of making a pre-trained model selectively forget some training instances or class(es) while retaining performance on the remaining dataset. Existing MUL research involves fine-tuning using a forget and/or retain set, making it expensive and/or impractical, and often causing performance degradation in the unlearned model. We introduce {\\pname}, an unlearning-aware vision transformer-based architecture that can directly perform unlearning for future unlearning requests without any fine-tuning over the requested set. The proposed model is trained by simulating unlearning during the training process itself. It involves randomly separating class(es)/sub-class(es) present in each mini-batch into two disjoint sets: a proxy forget-set and a retain-set, and the model is optimized so that it is unable to predict the forget-set. Forgetting is achieved by withdrawing keys, making unlearning on-the-fly and avoiding performance degradation. The model is trained jointly with learnable keys and original weights, ensuring withholding a key irreversibly erases information, validated by membership inference attack scores. Extensive experiments on various datasets, architectures, and resolutions confirm {\\pname}'s superiority over both fine-tuning-free and fine-tuning-based methods.

## Recorded benchmark results

A research result applies to its stated dataset, split, metric and evaluation configuration. Compare like-for-like settings, preserve the metric's unit and direction, and distinguish paper-reported results from independent reproduction. The best result recorded here is not a claim of exhaustive, current global coverage.

- [Machine Unlearning on CIFAR-10 Random Forget 10% (train)](<https://www.sota2.com/research/sota/machine-unlearning-on-cifar-10-random-forget-10-percent-train.md>): Machine Unlearning; CIFAR-10 Random Forget 10% (train); Retain Accuracy: 99.78.

## Original sources and implementation

[Original paper](<https://arxiv.org/abs/2507.03281>)

[PDF](<https://arxiv.org/pdf/2507.03281.pdf>)