Jailbreak Attempt Forgetting on JB-1 Jailbreak Harm-1
73.1ASR (%)RetainOnlyFT
Evaluation Results
| Method | Links | |
|---|---|---|
| RetainOnlyFTModel=Llama-3.2-1B-Instruct2026.04 | 73.1 | |
| RetainOnlyFTModel=OLMo-2-1B-Instruct2026.04 | 55.8 | |
| RetainOnlyFTModel=OLMo-2-7B-Instruct2026.04 | 54.5 | |
| Llama-3.2-1B-InstructModel=Llama-3.2-1B-Instruct2026.04 | 52.5 | |
| EraserModel=OLMo-2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 45.5 | |
| UnlearningModel=OLMo-2-7B-Instruct, Hyperparameter (λ)=0.62026.04 | 42 | |
| RetainOnlyFTModel=Llama-3.2-3B-Instruct2026.04 | 29.4 | |
| EraserModel=OLMo-2-7B-Instruct, Hyperparameter (λ)=0.62026.04 | 29.3 | |
| OLMo-2-7B-InstructModel=OLMo-2-7B-Instruct2026.04 | 29.2 | |
| DPOModel=OLMo-2-7B-Instruct2026.04 | 28.5 | |
| DPOModel=OLMo-2-1B-Instruct2026.04 | 19.6 | |
| Llama-3.2-3B-InstructModel=Llama-3.2-3B-Instruct2026.04 | 18.1 | |
| DPOModel=Llama-3.2-1B-Instruct2026.04 | 16.3 | |
| OLMo-2-1B-InstructModel=OLMo-2-1B-Instruct2026.04 | 14.2 | |
| UnlearningModel=Llama-3.2-3B-Instruct, Hyperparameter (λ)=0.62026.04 | 10.7 | |
| DPOModel=Llama-3.2-3B-Instruct2026.04 | 8.7 | |
| SKUModel=Llama-3.2-1B-Instruct2026.04 | 8.5 | |
| SKUModel=Llama-3.2-3B-Instruct2026.04 | 8.1 | |
| SKUModel=OLMo-2-1B-Instruct2026.04 | 6.1 | |
| SKUModel=OLMo-2-7B-Instruct2026.04 | 3.6 | |
| UnlearningModel=Llama-3.2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 0.6 | |
| EraserModel=Llama-3.2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 0.3 | |
| EUModel=Llama-3.2-3B-Instruct, Hyperparameter (λ)=0.62026.04 | 0.1 | |
| EUModel=Llama-3.2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 0 | |
| EraserModel=Llama-3.2-3B-Instruct, Hyperparameter (λ)=0.62026.04 | 0 | |
| UnlearningModel=OLMo-2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 0 | |
| EUModel=OLMo-2-1B-Instruct, Hyperparameter (λ)=0.62026.04 | 0 | |
| EUModel=OLMo-2-7B-Instruct, Hyperparameter (λ)=0.62026.04 | 0 |