Jailbreak Attack on SafeBench (Specific Metrics)
100ADU Success RateCOMET
Evaluation Results
| Method | Links | ||||||||
|---|---|---|---|---|---|---|---|---|---|
| COMETVictim Model=Gemini-2.5-Flash, Defense Setting=Vanilla2026.02 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | 100 | |
| COMETVictim Model=Qwen3-VL-235B-A22B, Defense Setting=Vanilla2026.02 | 98 | 98 | 94 | 100 | 100 | 98 | 98 | 98 | |
| COMETVictim Model=Gemini-2.5-Flash, Defense Setting=Defended2026.02 | 96 | 98 | 100 | 96 | 100 | 96 | 100 | 98 | |
| COMETVictim Model=GPT-4.1, Defense Setting=Vanilla2026.02 | 96 | 98 | 88 | 100 | 98 | 92 | 100 | 96 | |
| COMETVictim Model=GPT-4.1, Defense Setting=Defended2026.02 | 90 | 92 | 86 | 100 | 96 | 88 | 98 | 93 | |
| COMETVictim Model=Qwen3-VL-235B-A22B, Defense Setting=Defended2026.02 | 90 | 92 | 88 | 96 | 94 | 86 | 98 | 92 | |
| COMETVictim Model=GPT-4.1-mini, Defense Setting=Vanilla2026.02 | 86 | 100 | 92 | 92 | 98 | 98 | 92 | 94 | |
| COMETVictim Model=GPT-4.1-mini, Defense Setting=Defended2026.02 | 84 | 96 | 84 | 88 | 90 | 92 | 88 | 89 | |
| CS-DJVictim Model=Gemini-2.5-Flash, Defense Setting=Vanilla2026.02 | 36 | 72 | 76 | 84 | 72 | 60 | 96 | 71 | |
| CS-DJVictim Model=Gemini-2.5-Flash, Defense Setting=Defended2026.02 | 36 | 76 | 72 | 80 | 68 | 60 | 92 | 69 | |
| CS-DJVictim Model=Qwen3-VL-235B-A22B, Defense Setting=Vanilla2026.02 | 20 | 22 | 18 | 8 | 24 | 38 | 24 | 22 | |
| CS-DJVictim Model=GPT-4.1, Defense Setting=Vanilla2026.02 | 6 | 72 | 36 | 64 | 6 | 72 | 24 | 40 | |
| CS-DJVictim Model=GPT-4.1, Defense Setting=Defended2026.02 | 4 | 68 | 32 | 60 | 4 | 60 | 20 | 35 | |
| CS-DJVictim Model=GPT-4.1-mini, Defense Setting=Vanilla2026.02 | 0 | 32 | 24 | 28 | 20 | 56 | 24 | 26 | |
| CS-DJVictim Model=GPT-4.1-mini, Defense Setting=Defended2026.02 | 0 | 2 | 0 | 2 | 4 | 2 | 0 | 1 | |
| CS-DJVictim Model=Qwen3-VL-235B-A22B, Defense Setting=Defended2026.02 | 0 | 16 | 0 | 0 | 24 | 32 | 18 | 13 |