Jailbreak Discovery on Jailbreak Discovery Evaluation Set (test)
95.3JDRBOA
Evaluation results
| Method | Links | ||
|---|---|---|---|
| BOAModel=Vicuna 1.5 (7B), Strategy=BOA2025.06 | 95.3 | 4.3 | |
| Naive SamplingModel=Vicuna 1.5 (7B), Strategy=Sampling2025.06 | 91.4 | — | |
| Beam SearchModel=Vicuna 1.5 (7B), Strategy=BSearch2025.06 | 84.4 | — | |
| BOAModel=Gemma 3 (4B), Strategy=BOA2025.06 | 34.4 | 7.3 | |
| Beam SearchModel=Gemma 3 (4B), Strategy=BSearch2025.06 | 32.8 | — | |
| Naive SamplingModel=Gemma 3 (4B), Strategy=Sampling2025.06 | 32 | — | |
| BOAModel=Llama 3.1 (8B), Strategy=BOA2025.06 | 24.2 | 106.7 | |
| BOAModel=Qwen 3 (8B), Strategy=BOA2025.06 | 21.1 | 50 | |
| BOAModel=Qwen 2.5 (32B), Strategy=BOA2025.06 | 16.4 | 16.7 | |
| Beam SearchModel=Qwen 3 (8B), Strategy=BSearch2025.06 | 14.8 | — | |
| Beam SearchModel=Llama 3.1 (8B), Strategy=BSearch2025.06 | 14.1 | — | |
| Naive SamplingModel=Qwen 3 (8B), Strategy=Sampling2025.06 | 14.1 | — | |
| Naive SamplingModel=Qwen 2.5 (32B), Strategy=Sampling2025.06 | 14.1 | — | |
| BOAModel=Llama 3.1 (70B), Strategy=BOA2025.06 | 13.3 | 21.4 | |
| Naive SamplingModel=Llama 3.1 (8B), Strategy=Sampling2025.06 | 11.7 | — | |
| Beam SearchModel=Qwen 2.5 (32B), Strategy=BSearch2025.06 | 11.7 | — | |
| Naive SamplingModel=Llama 3.1 (70B), Strategy=Sampling2025.06 | 10.9 | — | |
| Beam SearchModel=Llama 3.1 (70B), Strategy=BSearch2025.06 | 9.4 | — | |
| BOAModel=Qwen 2.5 (72B), Strategy=BOA2025.06 | 7.8 | 66.5 | |
| BOAModel=Llama 2 (7B), Strategy=BOA2025.06 | 7 | 79.8 | |
| Naive SamplingModel=Qwen 2.5 (72B), Strategy=Sampling2025.06 | 4.7 | — | |
| Naive SamplingModel=Llama 2 (7B), Strategy=Sampling2025.06 | 3.9 | — | |
| Beam SearchModel=Llama 2 (7B), Strategy=BSearch2025.06 | 3.1 | — | |
| Beam SearchModel=Qwen 2.5 (72B), Strategy=BSearch2025.06 | 3.1 | — |