Jailbreak Detection on JailbreaksOverTime (test)
38.06Throughput (items/s)MLJailDe
Evaluation results
| Method | Links | ||||
|---|---|---|---|---|---|
| MLJailDeHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 38.06 | — | — | — | |
| DeBERTa-ftHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Fine-tuned2026.04 | 37.71 | — | — | — | |
| JBShieldHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 32.05 | — | — | — | |
| PromptGuardHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 6.629 | — | — | — | |
| GPT-4o-pHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Prompted2026.04 | 0.8 | — | — | — | |
| GPT-4.1-pHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Prompted2026.04 | 0.778 | — | — | — | |
| Qwen2.5-ftHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Fine-tuned2026.04 | 0.651 | — | — | — | |
| Llama-3.1-ftHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Fine-tuned2026.04 | 0.562 | — | — | — | |
| Llama-2-ftHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Fine-tuned2026.04 | 0.456 | — | — | — | |
| ModerationHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 0.296 | — | — | — | |
| Qwen3-ftHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Fine-tuned2026.04 | 0.255 | — | — | — | |
| SelfReminderHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 0.225 | — | — | — | |
| Claude-4.5-pHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Prompted2026.04 | 0.168 | — | — | — | |
| SelfDefendHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,5262026.04 | 0.158 | — | — | — | |
| GPT-5-pHardware=Two NVIDIA A40 48GB GPUs, Average tokens per item=1,526, Configuration=Prompted2026.04 | 0.07 | — | — | — | |
| Claude-4.5-pMode=Prompt-based2026.04 | — | 85.5 | 100 | 92.2 | |
| DeBERTa-ftMode=Fine-tuned2026.04 | — | 96.9 | 47.3 | 63.5 | |
| GPT-4.1-pMode=Prompt-based2026.04 | — | 88.7 | 92.4 | 90.5 | |
| GPT-4o-pMode=Prompt-based2026.04 | — | 80.2 | 94.5 | 86.8 | |
| GPT-5-pMode=Prompt-based2026.04 | — | 91.6 | 99.7 | 95.5 | |
| JBShield2026.04 | — | 45.2 | 57.6 | 50.7 | |
| Llama-2-ftMode=Fine-tuned2026.04 | — | 56.8 | 13.9 | 22.4 | |
| Llama-3.1-ftMode=Fine-tuned2026.04 | — | 81.5 | 40 | 53.7 | |
| MLJailDeContext=Table V result2026.04 | — | 99.7 | 97.3 | 98.5 | |
| Moderation2026.04 | — | 95 | 28.5 | 43.8 | |
| PromptGuard2026.04 | — | 95.1 | 52.7 | 67.8 | |
| Qwen2.5-ftMode=Fine-tuned2026.04 | — | 74.2 | 34.8 | 47.4 | |
| Qwen3-ftMode=Fine-tuned2026.04 | — | 72.1 | 36.1 | 48.1 | |
| SelfDefend2026.04 | — | 57.7 | 64.9 | 61.1 | |
| SelfReminder2026.04 | — | 97.3 | 75.8 | 85.2 |