Jailbreak Robustness on HarmBench
0HarmBench ASRDream-Instruct
Evaluation results
| Method | Links | ||||||
|---|---|---|---|---|---|---|---|
| Dream-InstructAttack=Zero-shot, Defense=Self-rem.2026.04 | 0 | 73 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=Zero-shot, Defense=Self-rem.2026.04 | 0 | 40.5 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defense=Self-rem.2026.04 | 0.8 | 67 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=Zero-shot, Defense=Self-rem.2026.04 | 0.8 | 5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=Zero-shot, Defense=DiffuGuard2026.04 | 1 | 40.5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=Zero-shot, Defense=PPL2026.04 | 1.2 | 40.5 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=Zero-shot, Defense=Self-rem.2026.04 | 1.5 | 6 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=Zero-shot, Defense=DiffuGuard2026.04 | 1.8 | 5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=Zero-shot, Defense=None2026.04 | 1.8 | 40.5 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=Zero-shot, Defense=None2026.04 | 2.2 | 5 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=Zero-shot, Defense=PPL2026.04 | 2.2 | 5 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defense=Self-rem.2026.04 | 2.2 | 69.2 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defense=None2026.04 | 2.8 | 73 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defense=PPL2026.04 | 2.8 | 73 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=Zero-shot, Defense=PPL2026.04 | 3.2 | 6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defense=DiffuGuard2026.04 | 3.5 | 73 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=Zero-shot, Defense=DiffuGuard2026.04 | 3.8 | 6 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=Zero-shot, Defense=None2026.04 | 4 | 6 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defense=None2026.04 | 4.5 | 67 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defense=None2026.04 | 4.5 | 69.2 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defense=PPL2026.04 | 4.5 | 69.2 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defense=DiffuGuard2026.04 | 4.5 | 69.2 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=PAD, Defense=Self-rem.2026.04 | 4.5 | 35.5 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defense=PPL2026.04 | 5 | 67 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defense=DiffuGuard2026.04 | 5.2 | 67 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defense=Self-rem.2026.04 | 7.5 | 39 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=DIJA, Defense=PPL2026.04 | 8.4 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defense=PPL2026.04 | 9.5 | 0.2 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=DIJA, Defense=PPL2026.04 | 10.2 | 0 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defense=PPL2026.04 | 11.5 | 0.2 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defense=None2026.04 | 12.8 | 39 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=PAD, Defense=Self-rem.2026.04 | 13.5 | 0.5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=PAD, Defense=None2026.04 | 14 | 35.5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=PAD, Defense=DiffuGuard2026.04 | 14.8 | 35.5 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=PAD, Defense=PPL2026.04 | 15.8 | 35.5 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defense=PPL2026.04 | 16 | 39 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defense=DiffuGuard2026.04 | 16.2 | 39 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defense=Self-rem.2026.04 | 21.2 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=PAD, Defense=Self-rem.2026.04 | 23 | 0.8 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defense=Self-rem.2026.04 | 28 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=PAD, Defense=None2026.04 | 30 | 0.8 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=PAD, Defense=DiffuGuard2026.04 | 30.5 | 0.5 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=PAD, Defense=None2026.04 | 31.5 | 0.5 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=PAD, Defense=PPL2026.04 | 32.2 | 0.5 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=PAD, Defense=DiffuGuard2026.04 | 32.8 | 0.8 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defense=PPL2026.04 | 38.2 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defense=DiffuGuard2026.04 | 38.8 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defense=PPL2026.04 | 39.1 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defense=None2026.04 | 40.5 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=PAD, Defense=PPL2026.04 | 40.5 | 0.8 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defense=DiffuGuard2026.04 | 40.8 | 0 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defense=None2026.04 | 42 | 0 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=DIJA, Defense=DiffuGuard2026.04 | 47.5 | 0 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defense=None2026.04 | 50.7 | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defense=Self-rem.2026.04 | 51 | 0.2 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defense=DiffuGuard2026.04 | 51.5 | 0 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=DIJA, Defense=Self-rem.2026.04 | 51.7 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=DIJA, Defense=None2026.04 | 52 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=DIJA, Defense=DiffuGuard2026.04 | 52 | 0 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=DIJA, Defense=DiffuGuard2026.04 | 52.5 | 0 | — | — | — | — | |
| LLaDA-Instruct (+SAD)Attack=DIJA, Defense=Self-rem.2026.04 | 53 | 0 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=DIJA, Defense=Self-rem.2026.04 | 53.2 | 0 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=DIJA, Defense=None2026.04 | 53.2 | 0 | — | — | — | — | |
| Dream-Instruct (+SAD)Attack=DIJA, Defense=PPL2026.04 | 53.2 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defense=None2026.04 | 53.5 | 0.2 | — | — | — | — | |
| LLaDA-1.5 (+SAD)Attack=DIJA, Defense=None2026.04 | 53.5 | 0 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defense=DiffuGuard2026.04 | 54 | 0.2 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defense=Self-rem.2026.04 | 54.2 | 0.2 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defense=Self-rem.2026.04 | 54.2 | 0 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defense=PPL2026.04 | 56 | 0 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defense=None2026.04 | 56.2 | 0 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defense=DiffuGuard2026.04 | 56.5 | 0.2 | — | — | — | — | |
| BaseModel=Llama-3.1-8B-In, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | 63.5 | 68 | 77 | 60 | |
| BaseModel=Qwen2.5-7B-In, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | 39.5 | 44 | 48.8 | 42 | |
| BaseModel=GPT-4o, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | 10.2 | 10.5 | 46 | 89.3 | |
| BaseModel=Gemini 2.5 Pro, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | 10.5 | 10.8 | 57 | 36.4 | |
| Goal PrioritizationModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 52 | 55 | 63 | 50 | |
| Goal PrioritizationModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 31.5 | 34.5 | 39 | 33 | |
| Goal PrioritizationModel=GPT-4o, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 9.3 | 9.4 | 43 | 86.5 | |
| Goal PrioritizationModel=Gemini 2.5 Pro, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 9.4 | 9.6 | 52.5 | 33 | |
| ICDModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 55 | 58.5 | 66 | 52 | |
| ICDModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 34 | 37 | 41.5 | 35.5 | |
| ICDModel=GPT-4o, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 9.7 | 9.8 | 44.5 | 88 | |
| ICDModel=Gemini 2.5 Pro, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 9.8 | 10 | 55 | 34.8 | |
| MTSAModel=Llama-3.1-8B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | 28 | 30.5 | 34.5 | 27 | |
| MTSAModel=Qwen2.5-7B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | 16.5 | 18 | 20 | 17 | |
| MUSEModel=Llama-3.1-8B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | 25 | 27 | 31 | 24.5 | |
| MUSEModel=Qwen2.5-7B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | 14.5 | 16 | 18.5 | 15.5 | |
| PPL-FilterModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 56.5 | 60 | 68 | 53.5 | |
| PPL-FilterModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 35.5 | 38.5 | 43 | 37 | |
| SafeEndModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 4.2 | 4.5 | 0 | 0 | |
| SafeEndModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 3.5 | 3.8 | 0 | 0 | |
| SafeEndModel=GPT-4o, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 4.5 | 4.7 | 0 | 0 | |
| SafeEndModel=Gemini 2.5 Pro, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | 4.2 | 4.5 | 0 | 0 |