Jailbreak Attack on Real-world LLM applications Poe
94.1ASRMetaBreak
Evaluation results
| Method | Links | ||
|---|---|---|---|
| MetaBreakModel=Llama-3.1-405B2025.10 | 94.1 | — | |
| GPTFuzzerModel=Gemma-2-27B, MetaBreak Integration=true2025.10 | 90.9 | 7 | |
| GPTFuzzerModel=Gemma-2-27B, MetaBreak Integration=false2025.10 | 83.9 | — | |
| PAPModel=Qwen-2.5-72B, MetaBreak Integration=true2025.10 | 82.7 | 19.7 | |
| PAPModel=Llama-3.1-405B, MetaBreak Integration=true2025.10 | 79.8 | 25.7 | |
| PAPModel=Phi-4, MetaBreak Integration=true2025.10 | 79.5 | 36.5 | |
| GPTFuzzerModel=Llama-3.1-405B, MetaBreak Integration=true2025.10 | 78 | 43.9 | |
| GPTFuzzerModel=Qwen-2.5-72B, MetaBreak Integration=true2025.10 | 74.8 | 15.5 | |
| PAPModel=Gemma-2-27B, MetaBreak Integration=true2025.10 | 66.4 | 27.1 | |
| PAPModel=Qwen-2.5-72B, MetaBreak Integration=false2025.10 | 63 | — | |
| GPTFuzzerModel=Qwen-2.5-72B, MetaBreak Integration=false2025.10 | 59.3 | — | |
| GPTFuzzerModel=Phi-4, MetaBreak Integration=true2025.10 | 57.3 | 35.9 | |
| PAPModel=Llama-3.1-405B, MetaBreak Integration=false2025.10 | 54.1 | — | |
| MetaBreakModel=Qwen-2.5-72B2025.10 | 50.7 | — | |
| MetaBreakModel=Gemma-2-27B2025.10 | 47.5 | — | |
| PAPModel=Phi-4, MetaBreak Integration=false2025.10 | 43 | — | |
| MetaBreakModel=Phi-42025.10 | 39.8 | — | |
| PAPModel=Gemma-2-27B, MetaBreak Integration=false2025.10 | 39.3 | — | |
| GPTFuzzerModel=Llama-3.1-405B, MetaBreak Integration=false2025.10 | 34.1 | — | |
| DIModel=Qwen-2.5-72B2025.10 | 21.4 | — | |
| DIModel=Phi-42025.10 | 21.4 | — | |
| GPTFuzzerModel=Phi-4, MetaBreak Integration=false2025.10 | 21.4 | — | |
| DIModel=Gemma-2-27B2025.10 | 14.3 | — | |
| DIModel=Llama-3.1-405B2025.10 | 13.6 | — |