Jailbreak Robustness on AdvBench
0Harmbench ASRLLaDA-Instruct
Evaluation results
| Method | Links | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| LLaDA-InstructAttack=Zero-shot, Defence=None, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=None, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 3.5 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=PPL, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 3.5 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=Self-rem., SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=Self-rem., SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 3.5 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=DiffuGuard, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-InstructAttack=Zero-shot, Defence=DiffuGuard, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 3.5 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=None, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.4 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=None, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 33.9 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.4 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=PPL, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 33.9 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=Self-rem., SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.4 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=Self-rem., SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 33.9 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=DiffuGuard, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.4 | — | — | — | — | |
| LLaDA-InstructAttack=DIJA, Defence=DiffuGuard, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 33.9 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=PPL, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=None, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=Self-rem., SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=Self-rem., SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 4 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=DiffuGuard, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.6 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=None, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.2 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=None, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 39.6 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.2 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=PPL, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 39.6 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=Self-rem., SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.2 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=Self-rem., SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 39.6 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=DiffuGuard, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 99.2 | — | — | — | — | |
| LLaDA-1.5Attack=DIJA, Defence=DiffuGuard, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 39.6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=None, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 90.6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=None, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 47.1 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=PPL, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 90.6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=PPL, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 47.1 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=Self-rem., SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 90.6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=Self-rem., SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 47.1 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=DiffuGuard, SAD=false2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 90.6 | — | — | — | — | |
| Dream-InstructAttack=Zero-shot, Defence=DiffuGuard, SAD=true2026.04 | 0 | — | — | — | — | — | — | — | — | — | — | 47.1 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=None, SAD=true2026.04 | 0.2 | — | — | — | — | — | — | — | — | — | — | 4 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=PPL, SAD=true2026.04 | 0.2 | — | — | — | — | — | — | — | — | — | — | 4 | — | — | — | — | |
| LLaDA-1.5Attack=Zero-shot, Defence=DiffuGuard, SAD=true2026.04 | 0.2 | — | — | — | — | — | — | — | — | — | — | 4 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=Self-rem., SAD=true2026.04 | 1.5 | — | — | — | — | — | — | — | — | — | — | 45.4 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=Self-rem., SAD=true2026.04 | 1.5 | — | — | — | — | — | — | — | — | — | — | 59 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=Self-rem., SAD=false2026.04 | 2.9 | — | — | — | — | — | — | — | — | — | — | 37.3 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=Self-rem., SAD=false2026.04 | 3.3 | — | — | — | — | — | — | — | — | — | — | 57.9 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=None, SAD=true2026.04 | 12.1 | — | — | — | — | — | — | — | — | — | — | 45.4 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=Self-rem., SAD=true2026.04 | 13.1 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=DiffuGuard, SAD=true2026.04 | 16.3 | — | — | — | — | — | — | — | — | — | — | 45.4 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=None, SAD=true2026.04 | 17.3 | — | — | — | — | — | — | — | — | — | — | 59 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=DiffuGuard, SAD=true2026.04 | 17.5 | — | — | — | — | — | — | — | — | — | — | 59 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=PPL, SAD=true2026.04 | 17.9 | — | — | — | — | — | — | — | — | — | — | 45.4 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=None, SAD=false2026.04 | 18.5 | — | — | — | — | — | — | — | — | — | — | 57.9 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=DiffuGuard, SAD=false2026.04 | 18.5 | — | — | — | — | — | — | — | — | — | — | 57.9 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=PPL, SAD=true2026.04 | 18.8 | — | — | — | — | — | — | — | — | — | — | 59 | — | — | — | — | |
| Dream-InstructAttack=PAD, Defence=PPL, SAD=false2026.04 | 19.6 | — | — | — | — | — | — | — | — | — | — | 57.9 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=Self-rem., SAD=false2026.04 | 20.6 | — | — | — | — | — | — | — | — | — | — | 0 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=Self-rem., SAD=true2026.04 | 28.8 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=None, SAD=false2026.04 | 31.3 | — | — | — | — | — | — | — | — | — | — | 37.3 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=PPL, SAD=false2026.04 | 32.1 | — | — | — | — | — | — | — | — | — | — | 37.3 | — | — | — | — | |
| Dream-InstructAttack=DIJA, Defence=DiffuGuard, SAD=false2026.04 | 32.1 | — | — | — | — | — | — | — | — | — | — | 37.3 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=Self-rem., SAD=false2026.04 | 38.3 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=PPL, SAD=true2026.04 | 52.7 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=None, SAD=true2026.04 | 53.8 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=DiffuGuard, SAD=true2026.04 | 54.8 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=DiffuGuard, SAD=true2026.04 | 63.5 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=None, SAD=true2026.04 | 63.7 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=DiffuGuard, SAD=false2026.04 | 68.3 | — | — | — | — | — | — | — | — | — | — | 0 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=PPL, SAD=false2026.04 | 68.8 | — | — | — | — | — | — | — | — | — | — | 0 | — | — | — | — | |
| LLaDA-1.5Attack=PAD, Defence=None, SAD=false2026.04 | 69 | — | — | — | — | — | — | — | — | — | — | 0 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=None, SAD=false2026.04 | 77.9 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| LLaDA-InstructAttack=PAD, Defence=DiffuGuard, SAD=false2026.04 | 81.2 | — | — | — | — | — | — | — | — | — | — | 0.2 | — | — | — | — | |
| BaseModel=Llama-3.1-8B-In, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 74.8 | 96 | 86.2 | 88 | |
| BaseModel=Qwen2.5-7B-In, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 43.5 | 50 | 46 | 47.5 | |
| BaseModel=GPT-4o, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 10.1 | 10.6 | 100 | 96.2 | |
| BaseModel=Gemini 2.5 Pro, Defense Type=Undefended, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 10.2 | 10.6 | 88.9 | 95.7 | |
| DPOBackbone=LLaDA2025.10 | — | 34 | 2 | 42 | 88 | 78 | 64 | 78 | 60 | 82 | — | — | — | — | — | — | |
| DPOBackbone=LLaDA 1.52025.10 | — | 30 | 2 | 34 | 76 | 68 | 68 | 94 | 76 | 80 | — | — | — | — | — | — | |
| DPOBackbone=MMaDA2025.10 | — | 60 | 40 | 54 | 90 | 80 | 57 | 74 | 66 | 86 | — | — | — | — | — | — | |
| FedDetoxSetting=Ours2026.04 | — | — | — | — | — | — | — | — | — | — | 14 | — | — | — | — | — | |
| FedDPOSetting=Poisoned2026.04 | — | — | — | — | — | — | — | — | — | — | 30.8 | — | — | — | — | — | |
| Goal PrioritizationModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 61 | 80 | 73 | 74.5 | |
| Goal PrioritizationModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 34 | 40 | 37.5 | 38.5 | |
| Goal PrioritizationModel=GPT-4o, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 9.2 | 9.4 | 97.5 | 93.8 | |
| Goal PrioritizationModel=Gemini 2.5 Pro, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 9.2 | 9.5 | 83 | 90.5 | |
| ICDModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 65 | 84 | 75 | 76.5 | |
| ICDModel=Qwen2.5-7B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 37 | 43 | 40 | 41 | |
| ICDModel=GPT-4o, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 9.6 | 9.9 | 99 | 95 | |
| ICDModel=Gemini 2.5 Pro, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 9.7 | 10 | 86.5 | 94 | |
| MOSABackbone=LLaDA2025.10 | — | 16 | 0 | 14 | 90 | 84 | 60 | 80 | 56 | 74 | — | — | — | — | — | — | |
| MOSABackbone=LLaDA 1.52025.10 | — | 16 | 0 | 18 | 82 | 82 | 68 | 72 | 48 | 70 | — | — | — | — | — | — | |
| MOSABackbone=MMaDA2025.10 | — | 56 | 8 | 48 | 80 | 68 | 70 | 80 | 57 | 76 | — | — | — | — | — | — | |
| MTSAModel=Llama-3.1-8B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 32 | 41 | 36 | 37 | |
| MTSAModel=Qwen2.5-7B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 18.5 | 21 | 19 | 20 | |
| MUSEModel=Llama-3.1-8B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 29.5 | 38 | 33.5 | 34 | |
| MUSEModel=Qwen2.5-7B-In, Defense Type=Training-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 16.5 | 19 | 17.5 | 18.5 | |
| OriginalBackbone=LLaDA2025.10 | — | 48 | 4 | 38 | 92 | 86 | 74 | 98 | 84 | 94 | — | — | — | — | — | — | |
| OriginalBackbone=LLaDA 1.52025.10 | — | 48 | 0 | 36 | 94 | 88 | 70 | 98 | 80 | 86 | — | — | — | — | — | — | |
| OriginalBackbone=MMaDA2025.10 | — | 90 | 54 | 78 | 98 | 94 | 90 | 82 | 64 | 90 | — | — | — | — | — | — | |
| PPL-FilterModel=Llama-3.1-8B-In, Defense Type=Inference-Time, Context Setting=32-shot2026.05 | — | — | — | — | — | — | — | — | — | — | — | — | 67 | 86 | 77 | 78.5 |