Jailbreak Detection on Jailbreak data (70/30 stratified)
100AUCLLMScan
Evaluation results
| Method | Links | |||
|---|---|---|---|---|
| LLMScanArchitecture=Llama-3-8B (32L), Attack=GCG, Supervision=supervised MLP2026.04 | 100 | 0.14 | 0.16 | |
| LLMScanArchitecture=Llama-3-8B (32L), Attack=DAN, Supervision=supervised MLP2026.04 | 100 | 0.07 | 0.16 | |
| LLMScanArchitecture=Llama-3-8B (32L), Attack=Roleplay, Supervision=supervised MLP2026.04 | 100 | 0.1 | 0.16 | |
| LLMScanArchitecture=Qwen-2.5-7B (28L), Attack=DAN, Supervision=supervised MLP2026.04 | 100 | 0.04 | 0.09 | |
| LLMScanArchitecture=Qwen-2.5-7B (28L), Attack=Roleplay, Supervision=supervised MLP2026.04 | 100 | 0.03 | 0.09 | |
| LCFArchitecture=Gemma-2-9B (42L), Attack=DAN, Supervision=unsupervised2026.04 | 100 | — | — | |
| LLMScanArchitecture=Gemma-2-9B (42L), Attack=DAN, Supervision=supervised MLP2026.04 | 100 | 0 | 0.06 | |
| LLMScanArchitecture=Gemma-2-9B (42L), Attack=Roleplay, Supervision=supervised MLP2026.04 | 100 | 0.08 | 0.06 | |
| LLMScanArchitecture=Qwen-2.5-14B (48L), Attack=GCG, Supervision=supervised MLP2026.04 | 100 | 0.03 | 0.03 | |
| LLMScanArchitecture=Qwen-2.5-14B (48L), Attack=DAN, Supervision=supervised MLP2026.04 | 100 | 0.01 | 0.03 | |
| LLMScanArchitecture=Qwen-2.5-14B (48L), Attack=Roleplay, Supervision=supervised MLP2026.04 | 100 | 0.02 | 0.03 | |
| LLMScanArchitecture=Gemma-2-9B (42L), Attack=Goals_only, Supervision=supervised MLP2026.04 | 99.9 | 0.16 | 0.06 | |
| LLMScanArchitecture=Qwen-2.5-7B (28L), Attack=GCG, Supervision=supervised MLP2026.04 | 99.4 | 0.08 | 0.09 | |
| LLMScanArchitecture=Gemma-2-9B (42L), Attack=GCG, Supervision=supervised MLP2026.04 | 99.1 | 0.02 | 0.06 | |
| LLMScanArchitecture=Qwen-2.5-7B (28L), Attack=Goals_only, Supervision=supervised MLP2026.04 | 98.8 | 0.18 | 0.09 | |
| LCFArchitecture=Qwen-2.5-14B (48L), Attack=DAN, Supervision=unsupervised2026.04 | 98.8 | — | — | |
| LLMScanArchitecture=Llama-3-8B (32L), Attack=Goals_only, Supervision=supervised MLP2026.04 | 98.6 | 0.34 | 0.16 | |
| LLMScanArchitecture=Qwen-2.5-14B (48L), Attack=Goals_only, Supervision=supervised MLP2026.04 | 98.1 | 0.04 | 0.03 | |
| LCFArchitecture=Qwen-2.5-14B (48L), Attack=Roleplay, Supervision=unsupervised2026.04 | 97.9 | — | — | |
| LCFArchitecture=Gemma-2-9B (42L), Attack=GCG, Supervision=unsupervised2026.04 | 97.6 | — | — | |
| LCFArchitecture=Qwen-2.5-14B (48L), Attack=GCG, Supervision=unsupervised2026.04 | 97.2 | — | — | |
| LCFArchitecture=Qwen-2.5-7B (28L), Attack=Roleplay, Supervision=unsupervised2026.04 | 96.8 | — | — | |
| LCFArchitecture=Qwen-2.5-7B (28L), Attack=DAN, Supervision=unsupervised2026.04 | 95.5 | — | — | |
| LCFArchitecture=Qwen-2.5-14B (48L), Attack=Goals_only, Supervision=unsupervised2026.04 | 93.7 | — | — | |
| LCFArchitecture=Llama-3-8B (32L), Attack=DAN, Supervision=unsupervised2026.04 | 92.7 | — | — | |
| LCFArchitecture=Gemma-2-9B (42L), Attack=Roleplay, Supervision=unsupervised2026.04 | 92.3 | — | — | |
| LCFArchitecture=Qwen-2.5-7B (28L), Attack=GCG, Supervision=unsupervised2026.04 | 91.2 | — | — | |
| LCFArchitecture=Llama-3-8B (32L), Attack=Roleplay, Supervision=unsupervised2026.04 | 89.6 | — | — | |
| LCFArchitecture=Llama-3-8B (32L), Attack=GCG, Supervision=unsupervised2026.04 | 86.4 | — | — | |
| LCFArchitecture=Gemma-2-9B (42L), Attack=Goals_only, Supervision=unsupervised2026.04 | 84.2 | — | — | |
| LCFArchitecture=Qwen-2.5-7B (28L), Attack=Goals_only, Supervision=unsupervised2026.04 | 80.5 | — | — | |
| LCFArchitecture=Llama-3-8B (32L), Attack=Goals_only, Supervision=unsupervised2026.04 | 64.7 | — | — |