Jailbreak Attack Success Rate Breakdown on JailbreakBench
97.98Safe2Harm ASR (Answer)Gemini 2.5 Flash
Evaluation results
| Method | Links | ||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 2.5 FlashThinking Mode=false2025.12 | 97.98 | — | 19.41 | — | 3 | — | 22 | — | 34.58 | — | 42 | — | |
| DeepSeekThinking Mode=false2025.12 | 97.98 | — | 98.31 | — | 14 | — | 35 | — | 72.08 | — | 81 | — | |
| Qwen3 8BModel Size=8B, Thinking Mode=false2025.12 | 90.91 | — | 30.8 | — | 2.5 | — | 37 | — | 71.67 | — | 77 | — | |
| Qwen3 8B ThinkingModel Size=8B, Thinking Mode=true2025.12 | 90.91 | 100 | 49.79 | 91.56 | 8 | 91 | 77 | 95 | 82.43 | 80 | 87 | 87 | |
| Qwen3 4B ThinkingModel Size=4B, Thinking Mode=true2025.12 | 86.87 | 100 | 54.01 | 92.41 | 11 | 93.5 | 82 | 96 | 82.92 | 79.58 | 91 | 91 | |
| Qwen3 1.7BModel Size=1.7B, Thinking Mode=false2025.12 | 80.81 | — | 61.6 | — | 26.63 | — | 87 | — | 81.25 | — | 71 | — | |
| Llama-3-8B-InstructBackbone=Llama-3, Parameters=8B, Thinking Mode=false2025.12 | 79.8 | — | 31.65 | — | 29.5 | — | 17 | — | 35 | — | 19 | — | |
| Qwen3 4BModel Size=4B, Thinking Mode=false2025.12 | 75.76 | — | 34.6 | — | 6.5 | — | 75 | — | 74.48 | — | 87 | — | |
| Qwen3 1.7B ThinkingModel Size=1.7B, Thinking Mode=true2025.12 | 72.73 | 69.7 | 74.68 | 92.4 | 43 | 94.5 | 88 | 97 | 84.45 | 78.15 | 69.39 | 86 | |
| GPT-5Thinking Mode=false2025.12 | 67 | — | 24.15 | — | 3 | — | 39 | — | 32.5 | — | 5 | — |