Safety Evaluation on Harmful and Jailbreak datasets
16Harm-1 ScoreRetainOnlyFT
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| RetainOnlyFTBackbone=Llama-3.2-1B-Instruct2026.04 | 16 | 19.4 | 48.6 | 52.6 | |
| RetainOnlyFTBackbone=OLMo-2-1B-Instruct2026.04 | 11 | 18.9 | 39.2 | 52.6 | |
| RetainOnlyFTBackbone=Llama-3.2-3B-Instruct2026.04 | 9 | 18.9 | 60.3 | 58.9 | |
| Llama-3.2-1B-InstructBackbone=Llama-3.2-1B-Instruct2026.04 | 7 | 8.8 | 52.5 | 51.1 | |
| RetainOnlyFTBackbone=OLMo-2-7B-Instruct2026.04 | 6 | 13.4 | 45.9 | 45.6 | |
| SKUBackbone=Llama-3.2-1B-Instruct2026.04 | 5 | 6 | 11.9 | 10.9 | |
| DPOBackbone=Llama-3.2-1B-Instruct2026.04 | 2 | 2.8 | 6.7 | 5.3 | |
| Llama-3.2-3B-InstructBackbone=Llama-3.2-3B-Instruct2026.04 | 2 | 4.6 | 18.1 | 10.2 | |
| DPOBackbone=Llama-3.2-3B-Instruct2026.04 | 2 | 2.3 | 3.5 | 2.5 | |
| OLMo-2-1B-InstructBackbone=OLMo-2-1B-Instruct2026.04 | 1 | 0.9 | 14.2 | 11.1 | |
| EraserBackbone=OLMo-2-1B-Instruct, Lambda (λ)=0.952026.04 | 1 | 9.2 | 7.3 | 6.5 | |
| EUBackbone=OLMo-2-1B-Instruct, Lambda (λ)=0.952026.04 | 1 | 0 | 0 | 0 | |
| DPOBackbone=OLMo-2-7B-Instruct2026.04 | 1 | 0 | 28.6 | 25.8 | |
| UnlearningBackbone=Llama-3.2-1B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 21 | 22.9 | |
| EraserBackbone=Llama-3.2-1B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0.5 | 17.2 | 16 | |
| EUBackbone=Llama-3.2-1B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 0 | 0.3 | |
| UnlearningBackbone=Llama-3.2-3B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 25.5 | 23.3 | |
| EraserBackbone=Llama-3.2-3B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 0.7 | 0.3 | |
| SKUBackbone=Llama-3.2-3B-Instruct2026.04 | 0 | 1.3 | 7.6 | 5.7 | |
| EUBackbone=Llama-3.2-3B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 0.1 | 0.2 | |
| DPOBackbone=OLMo-2-1B-Instruct2026.04 | 0 | 0.1 | 20.8 | 20.1 | |
| UnlearningBackbone=OLMo-2-1B-Instruct, Lambda (λ)=0.952026.04 | 0 | 0 | 22.5 | 22.9 | |
| SKUBackbone=OLMo-2-1B-Instruct2026.04 | 0 | 0 | 6.2 | 5.7 | |
| OLMo-2-7B-InstructBackbone=OLMo-2-7B-Instruct2026.04 | 0 | 0.9 | 29.2 | 26.6 | |
| UnlearningBackbone=OLMo-2-7B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 42.1 | 41.9 | |
| EraserBackbone=OLMo-2-7B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 45.1 | 40.3 | |
| SKUBackbone=OLMo-2-7B-Instruct2026.04 | 0 | 0 | 7 | 6.5 | |
| EUBackbone=OLMo-2-7B-Instruct, Lambda (λ)=0.62026.04 | 0 | 0 | 0.3 | 0.2 |