Continual Learning and Private Unlearning

About

As intelligent agents become autonomous over longer periods of time, they may eventually become lifelong counterparts to specific people. If so, it may be common for a user to want the agent to master a task temporarily but later on to forget the task due to privacy concerns. However enabling an agent to \emph{forget privately} what the user specified without degrading the rest of the learned knowledge is a challenging problem. With the aim of addressing this challenge, this paper formalizes this continual learning and private unlearning (CLPU) problem. The paper further introduces a straightforward but exactly private solution, CLPU-DER++, as the first step towards solving the CLPU problem, along with a set of carefully designed benchmark problems to evaluate the effectiveness of the proposed solution. The code is available at https://github.com/Cranial-XIX/Continual-Learning-Private-Unlearning.

Bo Liu, Qiang Liu, Peter Stone · 2022

Related benchmarks

Task	Dataset	Result
Knowledge Unlearning	WMDP bio	Accuracy41.5	93
Machine Unlearning	TOFU (5%)	Forget Quality0.11	82
Machine Unlearning	TOFU (10%)	Forget Quality (FQ)0.009	60
Knowledge Retention	MMLU (full)	MMLU Accuracy43.6	60
Machine Unlearning	TOFU (1%)	Forget Quality (FQ)0.27	59
Machine Unlearning	TOFU Forget 1%	Aggregation Score50	54
Machine Unlearning	TOFU forget05 1.0	Model Utility (MU)62	53
Machine Unlearning	TOFU 1.0 (forget01)	Average Score57	53
Classification	MLLMU-Bench Forget Set	Accuracy60.83	51
Machine Unlearning	TOFU	Forget Quality (FQ)0.0268	43