Jailbreak Defense on PAIR
0ASRSelf-Examination
Evaluation results
| Method | Links | ||
|---|---|---|---|
| Self-ExaminationModel=Llama22024.02 | 0 | 1.04 | |
| ICDModel=Llama22024.02 | 0 | 1.02 | |
| SAGEModel=Qwen2.52025.05 | 0 | 1 | |
| PreSafeBase Model=DS-R1-7B2026.03 | 0 | — | |
| PreSafeBase Model=DS-R1-14B2026.03 | 0 | — | |
| R2DBase Model=Qwen-8B2026.03 | 0 | — | |
| PreSafeBase Model=Qwen-8B2026.03 | 0 | — | |
| SafeDecodingModel=Vicuna2024.02 | 0.04 | 1.22 | |
| SafeDecodingModel=Llama22024.02 | 0.04 | 1.14 | |
| Self-ExaminationModel=Vicuna2024.02 | 0.12 | 1.6 | |
| ParaphraseModel=Llama22024.02 | 0.12 | 1.02 | |
| Self-ReminderModel=Llama22024.02 | 0.14 | 1.14 | |
| No DefenseModel=Llama22024.02 | 0.18 | 1.18 | |
| PPLModel=Llama22024.02 | 0.18 | 1.18 | |
| RetokenizationModel=Llama22024.02 | 0.2 | 1.16 | |
| ParaphraseModel=Vicuna2024.02 | 0.26 | 2.02 | |
| Self-ReminderModel=Vicuna2024.02 | 0.48 | 2.72 | |
| ICDModel=Vicuna2024.02 | 0.54 | 3.22 | |
| RetokenizationModel=Vicuna2024.02 | 0.76 | 3.76 | |
| No DefenseModel=Vicuna2024.02 | 0.88 | 4.66 | |
| PPLModel=Vicuna2024.02 | 0.88 | 4.66 | |
| R2DBase Model=DS-R1-14B2026.03 | 1.2 | — | |
| PreSafeBase Model=Qwen-34B2026.03 | 1.2 | — | |
| PreSafeBase Model=Skywork-7B2026.03 | 1.2 | — | |
| Self-ExaminationModel=Qwen2.52025.05 | 2 | 1.14 | |
| ICDModel=Llama3.12025.05 | 2 | 1.24 | |
| SAGEModel=Llama3.12025.05 | 2 | 1 | |
| PreSafeBase Model=DS-R1-8B2026.03 | 3.7 | — | |
| Erase-And-CheckBase Model=DS-R1-14B2026.03 | 3.7 | — | |
| Goal PrioritizationModel=Qwen2.52025.05 | 6 | 1.06 | |
| Erase-And-CheckBase Model=DS-R1-7B2026.03 | 6.2 | — | |
| Erase-And-CheckBase Model=DS-R1-8B2026.03 | 6.2 | — | |
| Erase-And-CheckBase Model=Qwen-34B2026.03 | 7.4 | — | |
| Erase-And-CheckBase Model=Qwen-8B2026.03 | 7.4 | — | |
| Self-ExaminationModel=Llama3.12025.05 | 8 | 1.24 | |
| R2DBase Model=DS-R1-7B2026.03 | 8.6 | — | |
| Erase-And-CheckBase Model=Skywork-7B2026.03 | 8.6 | — | |
| Goal PrioritizationModel=Llama3.12025.05 | 10 | 1.08 | |
| SAGEModel=Gemma22025.05 | 14 | 1.22 | |
| R2DBase Model=DS-R1-8B2026.03 | 14.8 | — | |
| Self-ReminderModel=Llama3.12025.05 | 18 | 1.24 | |
| Deliberative Alignment (single sampling)Student Model=Qwen 2.5 7B Instruct2026.04 | 18 | — | |
| latent similarity based BoN samplingStudent Model=Qwen 2.5 7B Instruct2026.04 | 18 | — | |
| Goal PrioritizationModel=Gemma22025.05 | 20 | 1.14 | |
| R2DBase Model=Skywork-7B2026.03 | 22.2 | — | |
| SafeChainBase Model=DS-R1-14B2026.03 | 23.5 | — | |
| SafeChainBase Model=Skywork-7B2026.03 | 25.9 | — | |
| Self-ExaminationModel=Gemma22025.05 | 28 | 1.68 | |
| ICDModel=Qwen2.52025.05 | 29 | 2.1 | |
| Self-ReminderModel=Qwen2.52025.05 | 30 | 1.96 | |
| IAModel=Qwen2.52025.05 | 30 | 1.86 | |
| IAModel=Gemma22025.05 | 34 | 1.8 | |
| SafeChainBase Model=Qwen-8B2026.03 | 35.8 | — | |
| No DefenseModel=Qwen2.52025.05 | 36 | 2.4 | |
| Synonym SubstitutionBase Model=DS-R1-8B2026.03 | 38.3 | — | |
| R2DBase Model=Qwen-34B2026.03 | 39.5 | — | |
| SafeChainBase Model=DS-R1-8B2026.03 | 40.7 | — | |
| SafeChainBase Model=Qwen-34B2026.03 | 43.2 | — | |
| NoneBase Model=DS-R1-8B2026.03 | 44.4 | — | |
| Synonym SubstitutionBase Model=DS-R1-14B2026.03 | 44.4 | — | |
| Remove Non-DictionaryBase Model=DS-R1-14B2026.03 | 45.7 | — | |
| Smooth LLMBase Model=DS-R1-14B2026.03 | 46.9 | — | |
| Perplexity FilterBase Model=DS-R1-14B2026.03 | 46.9 | — | |
| Remove Non-DictionaryBase Model=DS-R1-8B2026.03 | 48.1 | — | |
| Smooth LLMBase Model=DS-R1-7B2026.03 | 49.4 | — | |
| Synonym SubstitutionBase Model=DS-R1-7B2026.03 | 49.4 | — | |
| Synonym SubstitutionBase Model=Qwen-34B2026.03 | 49.4 | — | |
| Synonym SubstitutionBase Model=Qwen-8B2026.03 | 49.4 | — | |
| Synonym SubstitutionBase Model=Skywork-7B2026.03 | 49.4 | — | |
| Smooth LLMBase Model=DS-R1-8B2026.03 | 50.6 | — | |
| SafeChainBase Model=DS-R1-7B2026.03 | 53.1 | — | |
| NoneBase Model=DS-R1-14B2026.03 | 53.1 | — | |
| Perplexity FilterBase Model=DS-R1-8B2026.03 | 55.6 | — | |
| Self-ReminderModel=Gemma22025.05 | 56 | 2.46 | |
| IAModel=Llama3.12025.05 | 56 | 2.68 | |
| Smooth LLMBase Model=Skywork-7B2026.03 | 56.8 | — | |
| NoneBase Model=Qwen-34B2026.03 | 58 | — | |
| Remove Non-DictionaryBase Model=Qwen-8B2026.03 | 58 | — | |
| NoneBase Model=DS-R1-7B2026.03 | 59.3 | — | |
| Smooth LLMBase Model=Qwen-8B2026.03 | 59.3 | — | |
| Remove Non-DictionaryBase Model=Skywork-7B2026.03 | 59.3 | — | |
| Perplexity FilterBase Model=DS-R1-7B2026.03 | 60.5 | — | |
| Remove Non-DictionaryBase Model=DS-R1-7B2026.03 | 61.7 | — | |
| Perplexity FilterBase Model=Qwen-34B2026.03 | 61.7 | — | |
| NoneBase Model=Qwen-8B2026.03 | 61.7 | — | |
| Remove Non-DictionaryBase Model=Qwen-34B2026.03 | 63 | — | |
| Perplexity FilterBase Model=Skywork-7B2026.03 | 63 | — | |
| ICDModel=Gemma22025.05 | 64 | 3.16 | |
| Perplexity FilterBase Model=Qwen-8B2026.03 | 64.2 | — | |
| latent similarity based BoN samplingStudent Model=Llama 3.2 1B Instruct2026.04 | 66 | — | |
| Smooth LLMBase Model=Qwen-34B2026.03 | 67.9 | — | |
| Deliberative Alignment (single sampling)Student Model=Llama 3.2 1B Instruct2026.04 | 68 | — | |
| Instruction tuningStudent Model=Qwen 2.5 7B Instruct2026.04 | 68 | — | |
| NoneBase Model=Skywork-7B2026.03 | 69.1 | — | |
| Instruction tuningStudent Model=Llama 3.2 1B Instruct2026.04 | 88 | — | |
| No DefenseModel=Llama3.12025.05 | 90 | 4.5 | |
| No DefenseModel=Gemma22025.05 | 100 | 4.62 |