Machine Unlearning on TOFU 10% forget
-99.87Privacy LeakageRef
Evaluation results
| Method | Links | |||||||
|---|---|---|---|---|---|---|---|---|
| RefBackbone=Llama 2-7B2026.04 | -99.87 | 24.703 | 0 | 0 | — | — | — | |
| RefBackbone=Llama 3.2-3B2026.04 | -99.75 | 27.159 | 0 | 0 | — | — | — | |
| RefBackbone=Llama 3.2-1B2026.04 | -99.43 | 20.78 | 0 | 0 | — | — | — | |
| SCNBackbone=Llama 2-7B2026.04 | -73.54 | 8.361 | 0.101 | 0.848 | — | — | — | |
| SCNBackbone=Llama 3.2-3B2026.04 | -57.97 | 7.224 | 4.959 | 35.827 | — | — | — | |
| TNPOBackbone=Llama 3.2-1B2026.04 | -49.09 | 4.301 | 4.508 | 19.39 | — | — | — | |
| GABackbone=Llama 3.2-1B2026.04 | -48.59 | 2.639 | 4.271 | 11.273 | — | — | — | |
| GABackbone=Llama 3.2-3B2026.04 | -40.36 | 4.738 | 4.431 | 20.99 | — | — | — | |
| SCNBackbone=Llama 3.2-1B2026.04 | -39.98 | 2.754 | 3.548 | 9.772 | — | — | — | |
| NPOBackbone=Llama 3.2-1B2026.04 | -39.15 | 1.029 | 6.642 | 6.835 | — | — | — | |
| NPOBackbone=Llama 3.2-3B2026.04 | -35.38 | 1.106 | 4.343 | 4.804 | — | — | — | |
| SCEBackbone=Llama 3.2-3B2026.04 | -32.03 | 7.224 | 5.343 | 38.6 | — | — | — | |
| TNPOBackbone=Llama 2-7B2026.04 | -29.35 | 2.527 | 3.202 | 8.092 | — | — | — | |
| GABackbone=Llama 2-7B2026.04 | -25.74 | 6.169 | 1.325 | 8.175 | — | — | — | |
| SatImpBackbone=Llama 3.2-1B2026.04 | -19.65 | 2.871 | 5.258 | 15.093 | — | — | — | |
| ETWBackbone=Llama 3.2-3B2026.04 | -18.96 | 0.33 | 3.652 | 1.204 | — | — | — | |
| NPOBackbone=Llama 2-7B2026.04 | -15.42 | 0.33 | 6.33 | 2.087 | — | — | — | |
| SCEBackbone=Llama 2-7B2026.04 | -12.16 | 5.672 | -2.133 | -12.099 | — | — | — | |
| SCEBackbone=Llama 3.2-1B2026.04 | -12 | 7.971 | 3.867 | 30.822 | — | — | — | |
| ETWBackbone=Llama 3.2-1B2026.04 | -9.56 | 0.492 | 3.471 | 1.707 | — | — | — | |
| WGABackbone=Llama 3.2-1B2026.04 | -3.14 | 2.309 | 5.365 | 12.388 | — | — | — | |
| RetrainBackbone=Llama 3.2-1B2026.04 | -0.53 | 0 | 1.1 | 0 | — | — | — | |
| RetrainBackbone=Llama 3.2-3B2026.04 | -0.39 | 0 | 1.984 | 0 | — | — | — | |
| RetrainBackbone=Llama 2-7B2026.04 | -0.36 | 0 | 2.329 | 0 | — | — | — | |
| ETWBackbone=Llama 2-7B2026.04 | 0.76 | 1.999 | 1.816 | 3.63 | — | — | — | |
| NPOBackbone=Llama-3.2-1B-Instruct2026.06 | 1.99 | — | — | — | 0.205 | 0.075 | 0.528 | |
| SatImpBackbone=Llama 2-7B2026.04 | 2.96 | 4.16 | 2.549 | 10.605 | — | — | — | |
| SatImpBackbone=Llama-3.1-8B-Instruct2026.06 | 5.66 | — | — | — | 0.578 | 0.046 | 0.596 | |
| TNPOBackbone=Llama 3.2-3B2026.04 | 7.34 | 5.51 | 5.556 | 30.613 | — | — | — | |
| VanillaBackbone=Llama-3.2-1B-Instruct2026.06 | 10.43 | — | — | — | 0.055 | 0.055 | 0.281 | |
| VanillaBackbone=Llama-3.1-8B-Instruct2026.06 | 11.48 | — | — | — | 0.062 | 0.056 | 0.275 | |
| VanillaBackbone=Llama-3.2-3B-Instruct2026.06 | 13.76 | — | — | — | 0.063 | 0.053 | 0.272 | |
| NPO w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 14.05 | — | — | — | 0.267 | 0.072 | 0.55 | |
| NPOBackbone=Llama-3.2-3B-Instruct2026.06 | 16.62 | — | — | — | 0.132 | 0.06 | 0.529 | |
| WGABackbone=Llama 3.2-3B2026.04 | 18.81 | 6.511 | 2.581 | 16.805 | — | — | — | |
| SatImpBackbone=Llama 3.2-3B2026.04 | 19.13 | 8.959 | 3.242 | 29.044 | — | — | — | |
| NPO w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 21.52 | — | — | — | 0.521 | 0.06 | 0.649 | |
| SatImp w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 23.62 | — | — | — | 0.736 | 0.045 | 0.596 | |
| UNDIAL w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 34.68 | — | — | — | 0.75 | 0.041 | 0.678 | |
| SatImp w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 36.91 | — | — | — | 0.598 | 0.048 | 0.641 | |
| SatImpBackbone=Llama-3.2-3B-Instruct2026.06 | 38.29 | — | — | — | 0.462 | 0.048 | 0.614 | |
| NPO w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 39.3 | — | — | — | 0.543 | 0.067 | 0.613 | |
| NPOBackbone=Llama-3.1-8B-Instruct2026.06 | 40.45 | — | — | — | 0.233 | 0.071 | 0.601 | |
| WGABackbone=Llama 2-7B2026.04 | 46.73 | 7.224 | 1.655 | 11.955 | — | — | — | |
| UNDIAL w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 49.35 | — | — | — | 0.625 | 0.044 | 0.597 | |
| UNDIAL w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 55.31 | — | — | — | 0.932 | 0.05 | 0.644 | |
| RMU w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 55.37 | — | — | — | 0.709 | 0.033 | 0.595 | |
| WGA w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 56.59 | — | — | — | 0.809 | 0.033 | 0.609 | |
| WGABackbone=Llama-3.1-8B-Instruct2026.06 | 57.99 | — | — | — | 0.537 | 0.033 | 0.593 | |
| SatImp w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 58.35 | — | — | — | 0.453 | 0.057 | 0.569 | |
| WGA w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 58.76 | — | — | — | 0.734 | 0.033 | 0.6 | |
| RMUBackbone=Llama-3.2-1B-Instruct2026.06 | 58.81 | — | — | — | 0.707 | 0.033 | 0.592 | |
| WGABackbone=Llama-3.2-3B-Instruct2026.06 | 58.91 | — | — | — | 0.633 | 0.036 | 0.648 | |
| WGA w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 59.11 | — | — | — | 0.744 | 0.035 | 0.652 | |
| RMUBackbone=Llama-3.1-8B-Instruct2026.06 | 59.52 | — | — | — | 0.986 | 0.033 | 0.652 | |
| RMU w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 60.45 | — | — | — | 0.979 | 0.033 | 0.626 | |
| WGABackbone=Llama-3.2-1B-Instruct2026.06 | 60.71 | — | — | — | 0.627 | 0.033 | 0.59 | |
| RMU w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 61.06 | — | — | — | 0.854 | 0.033 | 0.666 | |
| RMUBackbone=Llama-3.2-3B-Instruct2026.06 | 62.83 | — | — | — | 0.859 | 0.033 | 0.675 | |
| SatImpBackbone=Llama-3.2-1B-Instruct2026.06 | 72 | — | — | — | 0.358 | 0.059 | 0.552 | |
| UNDIALBackbone=Llama-3.2-3B-Instruct2026.06 | 89.69 | — | — | — | 0.357 | 0.041 | 0.656 | |
| UNDIALBackbone=Llama-3.1-8B-Instruct2026.06 | 92.15 | — | — | — | 0.631 | 0.051 | 0.689 | |
| SimNPO w. SAGEBackbone=Llama-3.2-1B-Instruct2026.06 | 92.48 | — | — | — | 0.647 | 0.148 | 0.593 | |
| UNDIALBackbone=Llama-3.2-1B-Instruct2026.06 | 93.5 | — | — | — | 0.268 | 0.044 | 0.563 | |
| SimNPO w. SAGEBackbone=Llama-3.2-3B-Instruct2026.06 | 93.99 | — | — | — | 0.625 | 0.206 | 0.645 | |
| SimNPO w. SAGEBackbone=Llama-3.1-8B-Instruct2026.06 | 94.3 | — | — | — | 0.68 | 0.223 | 0.604 | |
| SimNPOBackbone=Llama-3.2-1B-Instruct2026.06 | 95.12 | — | — | — | 0.637 | 0.155 | 0.586 | |
| SimNPOBackbone=Llama-3.2-3B-Instruct2026.06 | 96.76 | — | — | — | 0.583 | 0.212 | 0.646 | |
| SimNPOBackbone=Llama-3.1-8B-Instruct2026.06 | 97 | — | — | — | 0.568 | 0.23 | 0.602 | |
| Fully Fine-tunedBackbone=Llama-3.2-1B-Instruct2026.06 | 99.46 | — | — | — | 0.736 | 0.712 | 0.598 | |
| Fully Fine-tunedBackbone=Llama-3.2-3B-Instruct2026.06 | 99.72 | — | — | — | 0.885 | 0.888 | 0.665 | |
| Fully Fine-tunedBackbone=Llama-3.1-8B-Instruct2026.06 | 99.94 | — | — | — | 0.992 | 0.979 | 0.627 |