Jailbreak Defense on HarmBench and AdvBench (test)
91.2GCG ScoreNo Defense
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| No DefenseVictim Model=Vicuna-7B2026.01 | 91.2 | 85.3 | 79.5 | 94.8 | 41.2 | 95.6 | |
| No DefenseVictim Model=Qwen2.5-7B2026.01 | 85.2 | 80.4 | 75.1 | 92.4 | 38.6 | 94.5 | |
| No DefenseVictim Model=Mistral3-8B2026.01 | 84.3 | 75.7 | 73.5 | 89.2 | 36.4 | 90.5 | |
| Self-reminderVictim Model=Vicuna-7B2026.01 | 82.6 | 76.3 | 70.2 | 16.2 | 12.4 | 11.5 | |
| Self-reminderVictim Model=Qwen2.5-7B2026.01 | 79.2 | 66.8 | 45.3 | 12.4 | 8.5 | 9.6 | |
| No DefenseVictim Model=Llama3-8B2026.01 | 78.4 | 68.2 | 64.5 | 82.1 | 32.4 | 84.6 | |
| Self-reminderVictim Model=Mistral3-8B2026.01 | 78.3 | 64.7 | 42.6 | 15.4 | 10.2 | 9.8 | |
| Self-reminderVictim Model=Llama3-8B2026.01 | 72.5 | 59.4 | 38.2 | 10.5 | 7.2 | 8.4 | |
| R2D2Victim Model=Vicuna-7B2026.01 | 35.5 | 41.6 | 33.2 | 21.4 | 15.8 | 17.6 | |
| R2D2Victim Model=Qwen2.5-7B2026.01 | 32.4 | 38.6 | 35.2 | 18.4 | 12.6 | 15.2 | |
| MARTVictim Model=Vicuna-7B2026.01 | 29.3 | 16.6 | 19.4 | 9.2 | 12.2 | 7.1 | |
| R2D2Victim Model=Mistral3-8B2026.01 | 28.6 | 33.4 | 30.2 | 16.5 | 11.2 | 13.6 | |
| MARTVictim Model=Qwen2.5-7B2026.01 | 26.8 | 12.1 | 13.4 | 8.5 | 10.4 | 6.2 | |
| CATVictim Model=Qwen2.5-7B2026.01 | 26.2 | 31.4 | 28.6 | 14.8 | 9.5 | 11.4 | |
| MARTVictim Model=Mistral3-8B2026.01 | 25.3 | 11.4 | 12.3 | 8.8 | 10.2 | 6.5 | |
| CATVictim Model=Vicuna-7B2026.01 | 24.3 | 32.8 | 26.3 | 17.2 | 12.4 | 14.6 | |
| CATVictim Model=Mistral3-8B2026.01 | 22.5 | 28.1 | 24.7 | 13.4 | 9.6 | 11.8 | |
| MARTVictim Model=Llama3-8B2026.01 | 22.4 | 10.2 | 11.5 | 7.2 | 8.5 | 5.1 | |
| PPLVictim Model=Vicuna-7B2026.01 | 19.1 | 81.5 | 73.1 | 86.4 | 35.2 | 21.4 | |
| SmoothLLMVictim Model=Qwen2.5-7B2026.01 | 18.4 | 31.2 | 29.5 | 10.2 | 6.4 | 5.2 | |
| SmoothLLMVictim Model=Mistral3-8B2026.01 | 16.2 | 29.7 | 27.4 | 12.8 | 7.2 | 6.1 | |
| SmoothLLMVictim Model=Llama3-8B2026.01 | 15.2 | 28.5 | 25.4 | 8.5 | 5.4 | 4.1 | |
| PPLVictim Model=Qwen2.5-7B2026.01 | 14.5 | 68.4 | 58.2 | 85.6 | 32.4 | 18.2 | |
| R2D2Victim Model=Llama3-8B2026.01 | 14.4 | 12.6 | 10.2 | 9.8 | 7.5 | 8.4 | |
| SmoothLLMVictim Model=Vicuna-7B2026.01 | 14.3 | 31.2 | 28 | 13.5 | 8.1 | 6.8 | |
| SafeDecodingVictim Model=Qwen2.5-7B2026.01 | 13.4 | 16.5 | 18.9 | 5.1 | 3.2 | 1.8 | |
| CATVictim Model=Llama3-8B2026.01 | 13.1 | 13.7 | 11.2 | 8.6 | 6.8 | 7.9 | |
| SafeDecodingVictim Model=Mistral3-8B2026.01 | 12.4 | 15.2 | 17.7 | 5.6 | 3.3 | 2.1 | |
| SafeDecodingVictim Model=Vicuna-7B2026.01 | 12.2 | 12.6 | 10.4 | 6.8 | 4.1 | 2.6 | |
| CircuitBreakersVictim Model=Vicuna-7B2026.01 | 11.6 | 13.4 | 13.2 | 5.4 | 3.5 | 2.2 | |
| SafeDecodingVictim Model=Llama3-8B2026.01 | 11.2 | 13.4 | 15.2 | 4.1 | 2.8 | 2.4 | |
| PPLVictim Model=Mistral3-8B2026.01 | 11.2 | 67.7 | 56.7 | 81.2 | 30.5 | 14.8 | |
| CircuitBreakersVictim Model=Qwen2.5-7B2026.01 | 10.2 | 13.4 | 16.2 | 4.2 | 2.5 | 1.5 | |
| PPLVictim Model=Llama3-8B2026.01 | 10.2 | 58.4 | 49.2 | 75.8 | 28.5 | 12.5 | |
| CircuitBreakersVictim Model=Mistral3-8B2026.01 | 9.3 | 12.6 | 15.4 | 4.9 | 2.9 | 3.7 | |
| SSPVictim Model=Vicuna-7B2026.01 | 8.8 | 6.7 | 7.4 | 2.6 | 3.4 | 5.1 | |
| ACE-safetyVictim Model=Vicuna-7B2026.01 | 8.5 | 9.8 | 7.3 | 6.1 | 5.4 | 4.2 | |
| CircuitBreakersVictim Model=Llama3-8B2026.01 | 8.5 | 11.2 | 12.4 | 3.4 | 2.1 | 3.2 | |
| SSPVictim Model=Mistral3-8B2026.01 | 8.5 | 6.5 | 7.3 | 2.8 | 3.7 | 2.7 | |
| ACE-safetyVictim Model=Mistral3-8B2026.01 | 8.1 | 9.1 | 9.5 | 7.4 | 6.8 | 5.1 | |
| ACE-safetyVictim Model=Llama3-8B2026.01 | 4.5 | 3.8 | 4.2 | 5.2 | 4.5 | 3.2 | |
| ACE-safetyVictim Model=Qwen2.5-7B2026.01 | 2.5 | 3.1 | 2.9 | 5.2 | 4.1 | 3.1 | |
| SSPVictim Model=Qwen2.5-7B2026.01 | 1.7 | 2.4 | 1.4 | 1.3 | 2.1 | 3 | |
| SSPVictim Model=Llama3-8B2026.01 | 1.5 | 2.2 | 1.3 | 1.5 | 2.4 | 3.5 |