Machine Unlearning on TOFU
1Forget Quality (FQ)Retrained LLM
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| Retrained LLMBase LLM=Llama2-7B2026.02 | 1 | 0.6267 | 0.408 | 0.9833 | |
| Retrained LLMBase LLM=Phi-1.5B2026.02 | 1 | 0.5233 | 0.4272 | 0.9269 | |
| Residual Knowledge risk scoreAbstention Mechanism=Risk Score, Unlearning Algorithm=RMU, Backbone=Llama2026.01 | 0.83 | 0.49 | — | — | |
| Predictive EntropyAbstention Mechanism=Predictive Entropy, Unlearning Algorithm=RMU, Backbone=Llama2026.01 | 0.77 | 0.48 | — | — | |
| No AbstentionAbstention Mechanism=None, Unlearning Algorithm=RMU, Backbone=Llama2026.01 | 0.72 | 0.51 | — | — | |
| MOXBase LLM=Llama2-7B, mode=momentum2026.02 | 0.068 | 0.6528 | 0.441 | 0.9802 | |
| MOXBase LLM=Llama2-7B, mode=targeted2026.02 | 0.0677 | 0.6412 | 0.4788 | 0.971 | |
| MOXBase LLM=Llama2-7B, alpha=42026.02 | 0.0625 | 0.6504 | 0.4697 | 0.9653 | |
| MOXBase LLM=Phi-1.5B, mode=momentum2026.02 | 0.0598 | 0.551 | 0.312 | 0.8988 | |
| MOXBase LLM=Phi-1.5B, alpha=42026.02 | 0.0582 | 0.5219 | 0.3138 | 0.881 | |
| POBase LLM=Llama2-7B2026.02 | 0.0541 | 0.6308 | 0.364 | 0.8811 | |
| LLMUBase LLM=Llama2-7B2026.02 | 0.0541 | 0.6337 | 0.448 | 0.8865 | |
| DPOBase LLM=Llama2-7B2026.02 | 0.0541 | 0.6359 | 0.586 | 0.8852 | |
| DPOBase LLM=Phi-1.5B2026.02 | 0.0521 | 0.5125 | 0.3437 | 0.7349 | |
| RMUBase LLM=Phi-1.5B2026.02 | 0.0387 | 0.5528 | 0.515 | 0.8488 | |
| MOXBase LLM=Phi-1.5B, alpha=22026.02 | 0.0364 | 0.5012 | 0.433 | 0.8928 | |
| MOXBase LLM=Phi-1.5B, alpha=82026.02 | 0.034 | 0.515 | 0.3436 | 0.8562 | |
| MOXBase LLM=Phi-1.5B, mode=targeted2026.02 | 0.0328 | 0.5012 | 0.3366 | 0.8858 | |
| MOXBase LLM=Llama2-7B, alpha=82026.02 | 0.0319 | 0.642 | 0.4658 | 0.9016 | |
| SimNPOBase LLM=Phi-1.5B2026.02 | 0.0315 | 0.5665 | 0.5135 | 0.8514 | |
| POBase LLM=Phi-1.5B2026.02 | 0.0286 | 0.5127 | 0.317 | 0.7468 | |
| LLMUBase LLM=Phi-1.5B2026.02 | 0.0286 | 0.511 | 0.3058 | 0.727 | |
| GADBase LLM=Llama2-7B2026.02 | 0.0268 | 0.632 | 0.4773 | 0.8912 | |
| MOXBase LLM=Llama2-7B, alpha=22026.02 | 0.0256 | 0.641 | 0.4555 | 0.9701 | |
| AltPOBase LLM=Phi-1.5B2026.02 | 0.0231 | 0.5312 | 0.52 | 0.8288 | |
| GADBase LLM=Phi-1.5B2026.02 | 0.0215 | 0.511 | 0.4996 | 0.8496 | |
| GABase LLM=Phi-1.5B2026.02 | 0.0213 | 0.5069 | 0.5114 | 0.8048 | |
| RMUBase LLM=Llama2-7B2026.02 | 0.0211 | 0.6378 | 0.4645 | 0.9032 | |
| MOXBase LLM=Llama2-7B, alpha=12026.02 | 0.0182 | 0.6358 | 0.4732 | 0.9788 | |
| MOXBase LLM=Phi-1.5B, alpha=12026.02 | 0.018 | 0.5026 | 0.4366 | 0.912 | |
| SimNPOBase LLM=Llama2-7B2026.02 | 0.0172 | 0.645 | 0.4601 | 0.9201 | |
| KLBase LLM=Llama2-7B2026.02 | 0.0168 | 0.63 | 0.5281 | 0.9398 | |
| MOXBase LLM=Phi-1.5B, alpha=0.52026.02 | 0.0163 | 0.5002 | 0.4512 | 0.92 | |
| TVBase LLM=Phi-1.5B2026.02 | 0.0156 | 0.5012 | 0.4366 | 0.881 | |
| MOXBase LLM=Llama2-7B, alpha=0.52026.02 | 0.0146 | 0.6305 | 0.4812 | 0.981 | |
| GABase LLM=Llama2-7B2026.02 | 0.0143 | 0.6333 | 0.4862 | 0.9008 | |
| AltPOBase LLM=Llama2-7B2026.02 | 0.012 | 0.6432 | 0.465 | 0.8953 | |
| KLBase LLM=Phi-1.5B2026.02 | 0.012 | 0.5047 | 0.5059 | 0.8109 | |
| TVBase LLM=Llama2-7B2026.02 | 0.0069 | 0.634 | 0.4512 | 0.981 | |
| NPOBase LLM=Llama2-7B2026.02 | 0.0068 | 0.6321 | 0.4632 | 0.895 | |
| NPOBase LLM=Phi-1.5B2026.02 | 0.003 | 0.5057 | 0.5196 | 0.8 | |
| Original LLMBase LLM=Phi-1.5B2026.02 | 0.0013 | 0.5184 | 0.9607 | 0.9199 | |
| Original LLMBase LLM=Llama2-7B2026.02 | 0 | 0.6346 | 0.9851 | 0.9833 |