Jailbreak Defense on Attack Success Rate (ASR) Benchmarks
2.5SRGoal Prioritization w/o thoughts
Evaluation Results
| Method | Links | |||||||||
|---|---|---|---|---|---|---|---|---|---|---|
| Goal Prioritization w/o thoughtsModel=Llama2-7B, Setting=w/ training2023.11 | 2.5 | 13.6 | 1.7 | 53.8 | 2 | 36.3 | 7.5 | 1.7 | 15.1 | |
| Goal PrioritizationModel=Llama2-7B, Setting=w/ training2023.11 | 5.6 | 5 | 4.2 | 9.4 | 1 | 8.8 | 1.3 | 0 | 4.6 | |
| Goal Prioritization w/o harmModel=Llama2-13B, Setting=w/ training2023.11 | 6.9 | 8.6 | 3.3 | 11.9 | 9.5 | 10 | 7.5 | 1.7 | 8 | |
| Goal PrioritizationModel=Llama2-13B, Setting=w/ training2023.11 | 7.5 | 6.4 | 2.5 | 13.1 | 6.5 | 6.3 | 3.8 | 0 | 6.6 | |
| Goal Prioritization w/o harmModel=Llama2-7B, Setting=w/ training2023.11 | 8.8 | 7.9 | 1.7 | 13.1 | 4.5 | 12.5 | 3.8 | 0 | 7 | |
| Aligned SFTModel=Llama2-7B, Setting=w/ training2023.11 | 10 | 5.7 | 0 | 42.5 | 27.5 | 43.8 | 6.3 | 0 | 18.7 | |
| Goal Prioritization w/o thoughtsModel=Llama2-13B, Setting=w/ training2023.11 | 10.6 | 10 | 0 | 48.8 | 0 | 32.5 | 11.3 | 0 | 14.4 | |
| Aligned SFTModel=Llama2-13B, Setting=w/ training2023.11 | 12.5 | 19.3 | 0 | 48.8 | 16.5 | 43.8 | 12.5 | 0 | 20.3 | |
| Vanilla SFTModel=Llama2-7B, Setting=w/ training2023.11 | 88.8 | 90 | 88.3 | 66.9 | 71.5 | 73.8 | 16.3 | 0 | 69.6 | |
| Vanilla SFTModel=Llama2-13B, Setting=w/ training2023.11 | 92.5 | 93.6 | 89.2 | 65.6 | 75.5 | 65 | 20 | 0 | 71 |