Jailbreak Defense on JailbreakBench and AdvBench
0.1ASRCertified Semantic Smoothing
Evaluation results
| Method | Links | |||||
|---|---|---|---|---|---|---|
| Certified Semantic SmoothingAttack Algorithm=GCG, Retention rate (k)=0.32026.02 | 0.1 | 97.1 | 22.5 | 68.2 | 2.15 | |
| Certified Semantic SmoothingAttack Algorithm=TAP, Retention rate (k)=0.32026.02 | 0.1 | 97.4 | 22.9 | 68.2 | 2.15 | |
| Certified Semantic SmoothingAttack Algorithm=TAP, Retention rate (k)=0.52026.02 | 0.3 | 94.8 | 18.9 | 86.5 | 2.05 | |
| Certified Semantic SmoothingAttack Algorithm=GCG, Retention rate (k)=0.52026.02 | 0.5 | 94.2 | 18.4 | 86.5 | 2.05 | |
| Certified Semantic SmoothingAttack Algorithm=AutoDAN, Retention rate (k)=0.52026.02 | 0.6 | 93.5 | 18.1 | 86.5 | 2.05 | |
| Certified Semantic SmoothingAttack Algorithm=TAP, Retention rate (k)=0.72026.02 | 0.8 | 90.1 | 15.2 | 94.1 | 1.95 | |
| Certified Semantic SmoothingAttack Algorithm=GCG, Retention rate (k)=0.72026.02 | 1.2 | 89.4 | 14.6 | 94.1 | 1.95 | |
| Certified Semantic SmoothingAttack Algorithm=AutoDAN, Retention rate (k)=0.72026.02 | 1.5 | 88.7 | 14.3 | 94.1 | 1.95 | |
| SmoothLLMAttack Algorithm=GCG2026.02 | 3.5 | 62.1 | 4.2 | 74.3 | 2.1 | |
| SmoothLLMAttack Algorithm=TAP2026.02 | 4.1 | 61.8 | 4.1 | 74.3 | 2.1 | |
| SmoothLLMAttack Algorithm=AutoDAN2026.02 | 8.2 | 58.4 | 3.8 | 74.3 | 2.1 | |
| Erase-CheckAttack Algorithm=GCG2026.02 | 12.4 | — | — | 88.5 | 4.2 | |
| Erase-CheckAttack Algorithm=TAP2026.02 | 14.1 | — | — | 88.5 | 4.2 | |
| Self-DenoiseAttack Algorithm=GCG2026.02 | 18.9 | 24.5 | 1.2 | 89.2 | 3.45 | |
| Self-DenoiseAttack Algorithm=TAP2026.02 | 22.1 | 23.8 | 1.1 | 89.2 | 3.45 | |
| PPL FilterAttack Algorithm=TAP2026.02 | 38.4 | — | — | 91.2 | 0.92 | |
| PPL FilterAttack Algorithm=AutoDAN2026.02 | 41.2 | — | — | 91.2 | 0.92 | |
| PPL FilterAttack Algorithm=GCG2026.02 | 45.3 | — | — | 91.2 | 0.92 | |
| VanillaAttack Algorithm=TAP (Black-box)2026.02 | 68.9 | 0 | 0 | 96.5 | 0.85 | |
| VanillaAttack Algorithm=AutoDAN2026.02 | 72.1 | 0 | 0 | 96.5 | 0.85 | |
| VanillaAttack Algorithm=GCG (White-box)2026.02 | 84.2 | 0 | 0 | 96.5 | 0.85 |