Jailbreak Attack Success Rate on Harmful prompts dataset
97Attack Success RateDaredevil-8B
Evaluation Results
| Method | Links | |
|---|---|---|
| Daredevil-8BDecomposition Strategy=Monitor2026.02 | 97 | |
| gpt-4o-miniDecomposition Strategy=Monitor2026.02 | 96 | |
| Daredevil-8BDecomposition Strategy=Cost+Monitor2026.02 | 92.5 | |
| Deepseek-v3.2Decomposition Strategy=Monitor2026.02 | 92.3 | |
| gpt-4o-miniDecomposition Strategy=Cost+Monitor2026.02 | 91 | |
| Daredevil-8BDecomposition Strategy=Cost2026.02 | 89.5 | |
| PAPBase LLM=Vicuna-33B, Guard Model=None, Threat Model=No Access2024.02 | 88 | |
| PRPBase LLM=Vicuna-33B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 88 | |
| gemini-3-flashDecomposition Strategy=Cost+Monitor2026.02 | 87.5 | |
| PAPBase LLM=Guanaco-13B, Guard Model=None, Threat Model=No Access2024.02 | 84 | |
| PRPBase LLM=Guanaco-13B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 84 | |
| Deepseek-v3.2Decomposition Strategy=Cost2026.02 | 84 | |
| Daredevil-8BDecomposition Strategy=Standard2026.02 | 83.5 | |
| Hermes3-70bDecomposition Strategy=Cost+Monitor2026.02 | 82.5 | |
| Deepseek-v3.2Decomposition Strategy=Cost+Monitor2026.02 | 81.2 | |
| PRPBase LLM=Llama2-70B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 80 | |
| PRPBase LLM=Vicuna-33B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 80 | |
| PRPBase LLM=Vicuna-33B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 80 | |
| PRPBase LLM=Llama2-70B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 78 | |
| PRPBase LLM=Guanaco-13B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 78 | |
| Hermes3-70bDecomposition Strategy=Monitor2026.02 | 77.8 | |
| PRPBase LLM=Guanaco-13B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 76 | |
| PRPBase LLM=Llama2-70B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 74 | |
| PAPBase LLM=Llama2-70B, Guard Model=None, Threat Model=No Access2024.02 | 66 | |
| PAPBase LLM=Vicuna-33B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 64 | |
| PAPBase LLM=Llama2-70B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 56 | |
| PAPBase LLM=Vicuna-33B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 56 | |
| PAPBase LLM=Llama2-70B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 50 | |
| PAPBase LLM=Guanaco-13B, Guard Model=Llama2-70B, Threat Model=No Access2024.02 | 50 | |
| gpt-4o-miniDecomposition Strategy=Cost2026.02 | 49 | |
| PAPBase LLM=Guanaco-13B, Guard Model=Gemini-Pro, Threat Model=No Access2024.02 | 46 | |
| Deepseek-v3.2Decomposition Strategy=Standard2026.02 | 44.4 | |
| gemini-3-flashDecomposition Strategy=Cost2026.02 | 37.5 | |
| Hermes3-70bDecomposition Strategy=Cost2026.02 | 28 | |
| Hermes3-70bDecomposition Strategy=Standard2026.02 | 26.5 | |
| gpt-4o-miniDecomposition Strategy=Standard2026.02 | 19 | |
| PAPBase LLM=Vicuna-33B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 12 | |
| claude-haiku-4.5Decomposition Strategy=Cost2026.02 | 7.1 | |
| PAPBase LLM=Guanaco-13B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 4 | |
| claude-haiku-4.5Decomposition Strategy=Cost+Monitor2026.02 | 3.7 | |
| gpt-oss-20bDecomposition Strategy=Cost+Monitor2026.02 | 3.5 | |
| claude-haiku-4.5Decomposition Strategy=Standard2026.02 | 1.1 | |
| gpt-oss-20bDecomposition Strategy=Standard2026.02 | 1 | |
| gpt-oss-20bDecomposition Strategy=Monitor2026.02 | 1 | |
| PAPBase LLM=Llama2-70B, Guard Model=GPT3.5, Threat Model=No Access2024.02 | 0 | |
| gpt-oss-20bDecomposition Strategy=Cost2026.02 | 0 | |
| gemini-3-flashDecomposition Strategy=Standard2026.02 | 0 | |
| gemini-3-flashDecomposition Strategy=Monitor2026.02 | 0 | |
| claude-haiku-4.5Decomposition Strategy=Monitor2026.02 | 0 |