Unlearning on MUSE-News 1.0 (test)
-99.8114Privacy LeakF target
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| F targetalpha=5, Quantization Precision=16-bit2026.01 | -99.8114 | 64.4088 | — | — | — | — | — | — | 42.2135 | 53.9108 | — | — | |
| GA GDR L1alpha=20, gamma=10, Quantization Precision=4-bit2026.01 | -99.8114 | 53.1294 | — | — | — | — | — | — | 35.6968 | 48.3322 | — | — | |
| GA GDR L1alpha=20, gamma=20, Quantization Precision=4-bit2026.01 | -99.8114 | 52.5442 | — | — | — | — | — | — | 36.9799 | 48.2426 | — | — | |
| GA GDR L1alpha=20, gamma=50, Quantization Precision=4-bit2026.01 | -99.8114 | 51.5712 | — | — | — | — | — | — | 35.7732 | 48.9254 | — | — | |
| F targetalpha=5, Quantization Precision=4-bit2026.01 | -99.7904 | 54.4482 | — | — | — | — | — | — | 36.1779 | 47.6531 | — | — | |
| GA GDR L1alpha=5, gamma=8, Quantization Precision=4-bit2026.01 | -99.7904 | 49.6625 | — | — | — | — | — | — | 35.0309 | 50.1803 | — | — | |
| GA GDR L1alpha=5, gamma=1, Quantization Precision=4-bit2026.01 | -99.7695 | 50.7248 | — | — | — | — | — | — | 32.8572 | 47.3032 | — | — | |
| GA GDR L1alpha=5, gamma=5, Quantization Precision=4-bit2026.01 | -99.7695 | 50.7248 | — | — | — | — | — | — | 32.8572 | 47.3032 | — | — | |
| GA GDR L1alpha=5, gamma=9, Quantization Precision=4-bit2026.01 | -99.7695 | 47.8559 | — | — | — | — | — | — | 34.5838 | 48.0864 | — | — | |
| GA GDR L1alpha=20, gamma=50, Quantization Precision=16-bit2026.01 | -99.7695 | 56.7639 | — | — | — | — | — | — | 34.5025 | 45.17 | — | — | |
| PDUBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=false2026.01 | -99.7271 | 0.7775 | 0.0298 | 0.4128 | 0.2018 | 0.7439 | 0.3529 | 0.4788 | — | — | — | — | |
| PDUBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=true2026.01 | -99.7271 | 0.784 | 0.0279 | 0.3984 | 0.2161 | 0.7396 | 0.3748 | 0.4975 | — | — | — | — | |
| PDUBackbone=Llama-2-7B, CLReg=false2026.01 | -99.7271 | 0.7775 | 0.0298 | 0.4128 | 0.2018 | 0.7439 | 0.3529 | 0.4788 | — | — | — | — | |
| PDUBackbone=Llama-2-7B, CLReg=true2026.01 | -99.7271 | 0.784 | 0.0279 | 0.3984 | 0.2161 | 0.7396 | 0.3748 | 0.4975 | — | — | — | — | |
| GA GDR L1alpha=20, gamma=20, Quantization Precision=16-bit2026.01 | -99.7066 | 56.4008 | — | — | — | — | — | — | 33.6179 | 45.5953 | — | — | |
| GA GDR L1alpha=20, gamma=10, Quantization Precision=16-bit2026.01 | -99.6438 | 56.4339 | — | — | — | — | — | — | 34.4243 | 45.8381 | — | — | |
| GA GDR L1alpha=1, gamma=5, Quantization Precision=4-bit2026.01 | -99.6018 | 51.7035 | — | — | — | — | — | — | 30.0899 | 52.4894 | — | — | |
| UNDIALBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=false2026.01 | -99.4962 | 0.7508 | 0.0357 | 0.2844 | 0.2027 | 0.8402 | 0.3632 | 0.5072 | — | — | — | — | |
| UNDIALBackbone=Llama-2-7B, CLReg=false2026.01 | -99.4962 | 0.7508 | 0.0357 | 0.2844 | 0.2027 | 0.8402 | 0.3632 | 0.5072 | — | — | — | — | |
| UNDIALBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=true2026.01 | -99.4752 | 0.7553 | 0.0328 | 0.2729 | 0.2126 | 0.8304 | 0.3694 | 0.5113 | — | — | — | — | |
| UNDIALBackbone=Llama-2-7B, CLReg=true2026.01 | -99.4752 | 0.7553 | 0.0328 | 0.2729 | 0.2126 | 0.8304 | 0.3694 | 0.5113 | — | — | — | — | |
| NPOBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=false2026.01 | -92.6952 | 0.7869 | 0.0442 | 0.5015 | 0.2055 | 0.6238 | 0.454 | 0.5255 | — | — | — | — | |
| NPOBackbone=Llama-2-7B, CLReg=false2026.01 | -92.6952 | 0.7869 | 0.0442 | 0.5015 | 0.2055 | 0.6238 | 0.454 | 0.5255 | — | — | — | — | |
| NPOBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=true2026.01 | -92.5483 | 0.7872 | 0.045 | 0.4993 | 0.207 | 0.6259 | 0.4657 | 0.534 | — | — | — | — | |
| NPOBackbone=Llama-2-7B, CLReg=true2026.01 | -92.5483 | 0.7872 | 0.045 | 0.4993 | 0.207 | 0.6259 | 0.4657 | 0.534 | — | — | — | — | |
| SimNPOBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=false2026.01 | -90.5542 | 0.848 | 0.072 | 0.5449 | 0.2449 | 0.4459 | 0.4778 | 0.4613 | — | — | — | — | |
| SimNPOBackbone=Llama-2-7B, CLReg=false2026.01 | -90.5542 | 0.848 | 0.072 | 0.5449 | 0.2449 | 0.4459 | 0.4778 | 0.4613 | — | — | — | — | |
| GradDiffBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=false2026.01 | -87.8464 | 0.8381 | 0.0683 | 0.5309 | 0.2414 | 0.4864 | 0.4662 | 0.4761 | — | — | — | — | |
| GradDiffBackbone=Llama-2-7B, CLReg=false2026.01 | -87.8464 | 0.8381 | 0.0683 | 0.5309 | 0.2414 | 0.4864 | 0.4662 | 0.4761 | — | — | — | — | |
| GA GDR L1alpha=1, gamma=1, Quantization Precision=4-bit2026.01 | -81.4334 | 47.3608 | — | — | — | — | — | — | 24.1533 | 45.5512 | — | — | |
| GA GDRalpha=5, Quantization Precision=4-bit2026.01 | -79.0444 | 49.541 | — | — | — | — | — | — | 24.3584 | 49.0015 | — | — | |
| GA GDR L1alpha=5, gamma=10, Quantization Precision=4-bit2026.01 | -74.3713 | 47.6524 | — | — | — | — | — | — | 24.7937 | 48.171 | — | — | |
| GA GDR L1alpha=5, gamma=9, Quantization Precision=16-bit2026.01 | -1.7393 | 54.78 | — | — | — | — | — | — | 28.0958 | 43.5021 | — | — | |
| GA GDR L1alpha=5, gamma=8, Quantization Precision=16-bit2026.01 | 0.5658 | 54.6848 | — | — | — | — | — | — | 26.132 | 54.6849 | — | — | |
| NPORuntime (min)=69.1672025.09 | 15.486 | — | — | — | — | 0 | — | 1 | — | 0 | 0 | 5 | |
| GARuntime (min)=67.7222025.09 | 18.588 | — | — | — | — | 0 | — | 1 | — | 0 | 0 | 5 | |
| GAalpha=5, Quantization Precision=4-bit2026.01 | 30.197 | 0 | — | — | — | — | — | — | 0 | 0 | — | — | |
| GAalpha=5, Quantization Precision=16-bit2026.01 | 36.4837 | 0 | — | — | — | — | — | — | 0 | 0 | — | — | |
| PRISMRuntime (min)=184.9672025.09 | 82.564 | — | — | — | — | 0 | — | 0.621 | — | 43.553 | 45.505 | 1 | |
| SimNPOBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=true2026.01 | 88.6566 | 0.3899 | 0.0141 | 0.4414 | 0.0404 | 0.8773 | 0.424 | 0.5717 | — | — | — | — | |
| SimNPOBackbone=Llama-2-7B, CLReg=true2026.01 | 88.6566 | 0.3899 | 0.0141 | 0.4414 | 0.0404 | 0.8773 | 0.424 | 0.5717 | — | — | — | — | |
| Original2025.09 | 99.148 | — | — | — | — | 58.302 | — | — | — | 55.4 | 63.879 | — | |
| GradDiffBackbone=Llama-2-7B, Contrastive Learning Regularization (CLReg)=true2026.01 | 99.6851 | 0.3441 | 0.012 | 0.4267 | 0.0345 | 0.8981 | 0.41 | 0.563 | — | — | — | — | |
| GradDiffBackbone=Llama-2-7B, CLReg=true2026.01 | 99.6851 | 0.3441 | 0.012 | 0.4267 | 0.0345 | 0.8981 | 0.41 | 0.563 | — | — | — | — | |
| Task VectorRuntime (min)=196.2832025.09 | 99.811 | — | — | — | — | 56.258 | — | 0 | — | 54.634 | 63.657 | 5 | |
| DOORRuntime (min)=435.5002025.09 | 99.895 | — | — | — | — | 52.383 | — | 0.069 | — | 44.575 | 59.307 | 2 | |
| GA GDR L1alpha=1, gamma=1, Quantization Precision=16-bit2026.01 | 106.7477 | 20.5398 | — | — | — | — | — | — | 0 | 12.7084 | — | — | |
| GA GDR L1alpha=5, gamma=10, Quantization Precision=16-bit2026.01 | 107.3764 | 0 | — | — | — | — | — | — | 0 | 0 | — | — | |
| GA GDR L1alpha=1, gamma=5, Quantization Precision=16-bit2026.01 | 107.9422 | 22.1245 | — | — | — | — | — | — | 0 | 21.8765 | — | — | |
| NPOGDRRuntime (min)=131.4612025.09 | 109.493 | — | — | — | — | 0 | — | 0.599 | — | 40.055 | 48.14 | 3 | |
| GAGDRRuntime (min)=78.8382025.09 | 109.493 | — | — | — | — | 4.891 | — | 0.687 | — | 10.56 | 32.65 | 4 | |
| GA GDRalpha=5, Quantization Precision=16-bit2026.01 | 109.5557 | 24.345 | — | — | — | — | — | — | 0 | 0.6667 | — | — | |
| GA GDR L1alpha=5, gamma=1, Quantization Precision=16-bit2026.01 | 109.5557 | 34.6993 | — | — | — | — | — | — | 0 | 31.5268 | — | — | |
| GA GDR L1alpha=5, gamma=5, Quantization Precision=16-bit2026.01 | 109.5557 | 44.9129 | — | — | — | — | — | — | 1.9445 | 22.3284 | — | — | |
| SAM+NPORuntime (min)=175.4332025.09 | 109.556 | — | — | — | — | 0 | — | 0.609 | — | 41.581 | 46.906 | 5 |