Jailbreaking Defense on Jailbreaking Evaluation Suite (test)
2.5No Attack ScoreLlama3-8B
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Llama3-8BDefense Method=None2026.02 | 2.5 | 17.5 | 0 | 18.5 | 3.1 | 9.8 | |
| Gemma2-2BDefense Method=None2026.02 | 2 | 59.5 | 89.5 | 31 | 39 | 54.8 | |
| Augmented RTBase Model=Gemma2-9B2026.02 | 2 | 23.5 | 64.5 | 31.5 | 33 | 38.1 | |
| Gemma2-9BDefense Method=None2026.02 | 1.5 | 40 | 81 | 32 | 47.1 | 50 | |
| Llama2-7BDefense Method=None2026.02 | 1 | 58 | 3 | 11 | 1.1 | 18.3 | |
| Augmented RTBase Model=Gemma2-2B2026.02 | 0.5 | 8 | 0 | 5.5 | 0 | 3.4 | |
| CATBase Model=Gemma2-2B2026.02 | 0.5 | 2 | 0 | 15.5 | 0.3 | 4.5 | |
| CATBase Model=Gemma2-9B2026.02 | 0.5 | 6.5 | 0.5 | 23 | 0.7 | 7.7 | |
| Augmented RTBase Model=Llama2-7B2026.02 | 0 | 5.5 | 0 | 9 | 0 | 3.6 | |
| CATBase Model=Llama2-7B2026.02 | 0 | 19.5 | 0 | 31 | 0.3 | 12.7 | |
| LATBase Model=Llama2-7B2026.02 | 0 | 1 | 0 | 6 | 0 | 1.8 | |
| DeepRefusalBase Model=Llama2-7B2026.02 | 0 | 5.5 | 0 | 5.5 | 0 | 2.8 | |
| Fail-Closed AlignmentBase Model=Llama2-7B2026.02 | 0 | 1.5 | 0 | 4 | 0.1 | 1.4 | |
| Augmented RTBase Model=Llama3-8B2026.02 | 0 | 8.5 | 0 | 4.5 | 0.1 | 3.3 | |
| CATBase Model=Llama3-8B2026.02 | 0 | 1 | 0 | 15.5 | 0 | 4.1 | |
| LATBase Model=Llama3-8B2026.02 | 0 | 4.5 | 0 | 5.5 | 0 | 2.5 | |
| DeepRefusalBase Model=Llama3-8B2026.02 | 0 | 2.5 | 0 | 5.5 | 0 | 2 | |
| Fail-Closed AlignmentBase Model=Llama3-8B2026.02 | 0 | 1 | 0 | 1.5 | 0 | 0.6 | |
| LATBase Model=Gemma2-2B2026.02 | 0 | 3.5 | 0.5 | 15 | 0.2 | 4.8 | |
| DeepRefusalBase Model=Gemma2-2B2026.02 | 0 | 1.5 | 0 | 13 | 0 | 3.6 | |
| Fail-Closed AlignmentBase Model=Gemma2-2B2026.02 | 0 | 3 | 1.5 | 2 | 0.2 | 1.7 | |
| LATBase Model=Gemma2-9B2026.02 | 0 | 1 | 0 | 17.5 | 0.3 | 4.7 | |
| DeepRefusalBase Model=Gemma2-9B2026.02 | 0 | 11 | 6 | 26 | 0.5 | 10.9 | |
| Fail-Closed AlignmentBase Model=Gemma2-9B2026.02 | 0 | 1 | 0.5 | 3 | 1.2 | 1.4 |